# Verifier evals — HLE and math

Two single-turn verifier environments. The agent answers once, the answer is
extracted and graded, and the episode ends. Both are primarily **validation**
signals rather than training targets.

## HLE (Humanity's Last Exam)

`environments/hle_environment.py` — single-turn, no sandbox. The agent answers in
the fixed HLE format (Explanation / Answer / Confidence); the extracted answer is
graded against the gold answer by an LLM judge when one is configured, falling
back to a normalized exact-match otherwise (`rewards/hle_grader.py`).

Intended as a validation benchmark — judge cost is bounded to validation passes.
Training on the 2,500-question exam is not recommended. Config:
`examples/configs/grpo_hle.yaml`.

## Math

`environments/math_environment.py` — single-turn. The model's answer is verified
with `math_verify` (symbolic/numeric equivalence grading rather than string
match), so algebraically-equivalent answers in a different form still score. A
verification timeout is handled gracefully.

## Related

`environments/code_jaccard_environment.py` scores a diff by token-Jaccard
similarity to a reference (a cheap, judge-free proxy), and
`environments/reward_model_environment.py` scores with a learned reward model for
the RLAIF / reward-model paths. Both follow the same single-shot
`EnvironmentInterface` contract.