# Glossary

Expansions for the acronyms and short names used across these docs and the
codebase. Terms are referenceable from other pages with MyST `` {term}`NCCL` ``
syntax. Widely-known terms (GPU, HTTP, JSON, YAML, CLI, SSH, VM) are omitted.

## Algorithms and losses

```{glossary}
GRPO
  Group Relative Policy Optimization — replaces PPO's value network with a
  group-relative baseline: several rollouts per prompt, each advantage is its
  reward minus the group mean. See [GRPO and the loss](design-docs/grpo-and-loss.md).

GSPO
  Group Sequence Policy Optimization — a GRPO variant using sequence-level
  importance ratios instead of token-level (the `sequence_level_importance_ratios`
  knob on the shared loss).

CISPO
  Clipped Importance-Sampling-weight Policy Optimization (MiniMax-M1) — a clip
  variant that keeps every token in the gradient but freezes its importance
  weight at the clipped value via a stop-gradient, instead of dropping clipped
  tokens as the `min`/`max` surrogate does (the `use_cispo` knob on the shared
  loss). See [GRPO and the loss](design-docs/grpo-and-loss.md).

DAPO
  Decoupled Clip and Dynamic Sampling Policy Optimization — GRPO with an
  asymmetric "clip-higher" range and dynamic resampling of degenerate groups.

GDPO
  A multi-reward (multi-objective) advantage estimator — a GRPO generalization
  that computes a per-component leave-one-out baseline for each reward axis,
  sums them, then renormalizes. The name is not an expanded acronym in this
  codebase; read it as "multi-reward GRPO". See
  [GRPO and the loss](design-docs/grpo-and-loss.md).

OPD
  On-Policy Distillation — the student generates on-policy and a frozen teacher
  scores those rollouts via log-probs; `OPDAdvantageEstimator` forms the
  advantage `sg[log π_teacher − log π_student]` (selected by
  `adv_estimator='opd'`). MOPD is the multi-teacher form (routed per sample).
  See [Distillation](design-docs/distillation.md).

xtoken
  Cross-tokenizer distillation — distillation from a teacher whose tokenizer
  differs from the student's: a token aligner maps teacher to student tokens and
  the teacher's logits are projected onto the student vocabulary for a
  chunk-averaged KL. Generalizes to several teachers, each with its own
  projection (a null projection marks a same-tokenizer teacher), combined per
  `kd_loss_mode`. See [Distillation](design-docs/distillation.md).

RLOO
  REINFORCE Leave-One-Out — the leave-one-out baseline (each sample's baseline
  excludes itself), used by the GRPO estimator.

PPO
  Proximal Policy Optimization — the clipped-surrogate policy-gradient method
  that GRPO/DAPO/GSPO are expressed as variants of.

PG
  Policy Gradient. "Clipped-PG" is the single configurable clipped
  policy-gradient loss (`ClippedPGLossFn`) that expresses PPO, GRPO, RLOO, DAPO,
  GSPO, and CISPO.

DPO
  Direct Preference Optimization — trains directly on preference pairs, no
  reward model or sampling loop. See [Preference optimization](design-docs/dpo.md).

DPOP
  DPO-Positive — adds a positive-penalty margin term to the DPO loss
  (`extra_margin`).

R-DPO
  Regularized DPO — adds a response-length penalty (`pre_sigmoid_offset`).

cDPO
  conservative DPO — label-smoothed DPO that mixes in the flipped target.

SimPO
  Simple Preference Optimization — reference-free preference loss (drops the
  reference-model term).

ORPO
  Odds Ratio Preference Optimization — a reference-free preference method.

RLAIF
  Reinforcement Learning from AI Feedback — preference/critique signal from an
  LLM judge rather than human labels.

SFT
  Supervised Fine-Tuning — standard next-token training on demonstrations; also
  the auxiliary term blended into DPO to keep the policy on-distribution.

RM
  Reward Model / Reward Modeling — training a model to score responses.

KL
  Kullback–Leibler divergence — the reference-policy penalty that bounds how far
  the policy drifts (`k1`/`k2`/`k3` estimators).

NLL
  Negative Log-Likelihood.

IS
  Importance Sampling — the off-policy correction that reweights rollouts
  generated by a slightly older policy (mandatory under async GRPO).
```

## Models, parallelism, precision

```{glossary}
DTensor
  Distributed Tensor — PyTorch's sharded-tensor abstraction underpinning the
  default FSDP2 trainer backend.

FSDP2
  Fully Sharded Data Parallel, version 2 — PyTorch-native parameter/gradient/
  optimizer-state sharding.

TP
  Tensor Parallelism — splits individual layers (matmuls) across devices.

CP
  Context Parallelism — splits the sequence dimension across devices.

EP
  Expert Parallelism — splits Mixture-of-Experts experts across devices.

DP
  Data Parallelism — replicates the model and splits the batch.

MoE
  Mixture of Experts — a sparse layer that routes each token to a few of many
  expert MLPs. See [Mixture of Experts](design-docs/moe.md).

Router replay (R3)
  Reusing generation's recorded per-token expert selection in the trainer's MoE
  forward, so train-time routing matches generation and their log-probs line up
  (removing the router-nondeterminism component of the log-prob error). Gated by
  `policy.router_replay.enabled`. See [Mixture of Experts](design-docs/moe.md).

GEMM
  General Matrix Multiply. "Grouped-GEMM" batches the per-expert matmuls of an
  MoE layer into one call.

LoRA
  Low-Rank Adaptation — parameter-efficient fine-tuning via low-rank adapter
  matrices.

FP8
  8-bit floating point — the optional quantized format for vLLM inference
  serving (training stays bf16).

NVFP4
  NVIDIA 4-bit floating point — a ModelOpt quantized format optionally served
  during rollout (`real_quant`) through vLLM's FP4 kernel and re-applied across
  refits. See [Quantized generation](design-docs/quantized-generation.md).

bf16
  bfloat16 — 16-bit "brain" floating point; the default training precision.

KV cache
  Key–Value cache — the attention cache the inference engine reuses across
  decode steps.
```

## JAX backend

```{glossary}
NNX
  Flax NNX — the Flax neural-network module system the JAX trainer is built on
  (a name, not an acronym). See [the JAX trainer](design-docs/jax-trainer.md).

GSPMD
  General/Global Single-Program Multiple-Data — XLA's automatic-sharding
  parallelization that `jax.sharding` drives.

XLA
  Accelerated Linear Algebra — the compiler backend JAX lowers to.

HF
  Hugging Face — the source of model weights/configs; the trainer keeps
  HF-layout parity and an HF-weight name map.

J0–J11
  The JAX trainer re-platform build phases. J0–J9/J11 are CPU-validated;
  J10 (GPU bring-up) and J7b are hardware-gated. Tracked in the HV ledger
  (`hardware-deferred-validation.md`).
```

## Systems, networking, deployment

```{glossary}
NCCL
  NVIDIA Collective Communications Library — the GPU collective transport used
  for the trainer→inference weight sync. See [weight sync](architecture/weight-sync.md).

IPC
  Inter-Process Communication — a colocated weight-sync transport.

ZMQ
  ZeroMQ — the messaging library behind one of the weight-sync/data-plane transports.

RDMA
  Remote Direct Memory Access — NIC-to-NIC memory transfer used by high-speed
  cluster fabrics.

RoCE
  RDMA over Converged Ethernet — RDMA on an Ethernet fabric; auto-provisioned as
  an opt-in DRA resource on supported nodes.

CNI
  Container Network Interface — the Kubernetes networking plugin; NetworkPolicy
  is only enforced by a CNI that supports it.

PDB
  PodDisruptionBudget — a Kubernetes object that bounds voluntary pod eviction
  (node drain/upgrade). (Not the Python debugger.)

DRA
  Dynamic Resource Allocation — the Kubernetes mechanism for claiming
  specialized devices (here, NVLink ComputeDomain and RoCE NICs).

KAI
  The KAI Scheduler — gang scheduling + hierarchical fair-share queues on
  Kubernetes (a product name).

KVM
  Kernel-based Virtual Machine — a Linux hypervisor used as one of the
  computer-use agent's guest backends.

ABC
  Abstract Base Class — the Python interface classes the trainer backends
  implement (`models/policy/interfaces.py`).

HV
  Hardware-deferred Validation — the ledger of checks that are written and
  CPU-validated but unverifiable without GPU/cluster hardware
  (`hardware-deferred-validation.md`), each tagged `HV-<n>`.
```

## Benchmarks and environments

```{glossary}
SWE-bench
  Software Engineering benchmark — real GitHub issues scored by running the
  repository's held-out tests. See [SWE environments](environments/swe.md).

CUA
  Computer-Use Agent — the multimodal GUI-control agent (mouse/keyboard on a
  desktop). See [OSWorld / CUA](environments/osworld-cua.md).

HLE
  Humanity's Last Exam — a hard knowledge/reasoning benchmark.

GDPval
  A benchmark of 220 long-horizon, file-producing professional tasks (the name
  is the benchmark's own). See [GDPval](environments/gdpval.md).
```