Design docs

The “why” behind each subsystem — the reasoning, invariants, and trade-offs that the code alone doesn’t make obvious. Where the architecture docs describe how the system fits together at runtime, these describe why each piece is built the way it is.

• GRPO and the clipped-PG loss — advantage estimation and the single configurable loss behind PPO / GRPO / RLOO / DAPO / GSPO.

• Rewards and integrity — execution-grounded scoring and the anti-reward-hacking path.

• The sandbox task executor — the ubuntu-swe REST executor and its client.

• Structured tool-use protocol — native Hermes tool-calls, validation, and RL-safe constraining.

• Mixture-of-Experts — expert parallelism, grouped GEMM, aux-loss-free load balancing, and router replay.

• Quantized generation — block-wise FP8 and ModelOpt NVFP4 real-quant rollout, re-applied across refits.

• The JAX trainer backend — the Flax NNX re-platform below the policy interface.

• Preference optimization (DPO) — DPO and online DPO.

• Distillation — on-policy logit distillation, advantage distillation (OPD), and cross-tokenizer (xtoken) distillation.