dockyard_rl.environments.code_jaccard_environment

Module Contents

Classes

CodeJaccardEnvConfig

CodeJaccardEnvironmentMetadata

CodeJaccardVerifyWorker

Worker that computes Jaccard similarity between predicted code and ground truth.

CodeJaccardEnvironment

Environment that scores code generation via Jaccard similarity.

API

class dockyard_rl.environments.code_jaccard_environment.CodeJaccardEnvConfig

Bases: typing.TypedDict

num_workers: int

None

stop_strings: NotRequired[list[str] | None]

None

class dockyard_rl.environments.code_jaccard_environment.CodeJaccardEnvironmentMetadata

Bases: typing.TypedDict

ground_truth: str

None

class dockyard_rl.environments.code_jaccard_environment.CodeJaccardVerifyWorker

Worker that computes Jaccard similarity between predicted code and ground truth.

verify(pred_responses: list[str], ground_truths: list[str]) → list[float]

Compute Jaccard similarity reward for each (prediction, ground_truth) pair.

Jaccard similarity is computed over the whitespace-tokenized word sets of the two strings. This gives a soft, differentiable-friendly reward signal that avoids the binary pass/fail cliff of test-execution-based rewards, and is useful during early training when the model’s code quality is too low to pass any tests.

Args: pred_responses: Predicted response strings from the LLM. ground_truths: Ground truth strings to compare against.

Returns: List of Jaccard similarity scores in [0, 1].

class dockyard_rl.environments.code_jaccard_environment.CodeJaccardEnvironment(cfg: dockyard_rl.environments.code_jaccard_environment.CodeJaccardEnvConfig)

Bases: dockyard_rl.environments.interfaces.EnvironmentInterface[dockyard_rl.environments.code_jaccard_environment.CodeJaccardEnvironmentMetadata]

Environment that scores code generation via Jaccard similarity.

This is a soft-reward alternative to binary test execution. Useful when:

  • Test runners are not yet set up for a new task domain.

  • Early-stage training where the model cannot yet pass any tests.

  • Tasks where partial credit (overlapping tokens) is a meaningful signal.

Rewards are continuous in [0, 1]. Episodes always terminate after one step.

Initialization

shutdown() → None
step(message_log_batch: list[dockyard_rl.data.interfaces.LLMMessageLogType], metadata: list[dockyard_rl.environments.code_jaccard_environment.CodeJaccardEnvironmentMetadata]) → dockyard_rl.environments.interfaces.EnvironmentReturn[dockyard_rl.environments.code_jaccard_environment.CodeJaccardEnvironmentMetadata]

Score a batch of model completions via Jaccard similarity.

Args: message_log_batch: Batch of conversation message logs. The last assistant message in each log is taken as the prediction. metadata: Per-sample metadata dicts; must contain a "ground_truth" key.

Returns: EnvironmentReturn with scalar rewards in [0, 1] and all episodes terminated.

global_post_process_and_metrics(batch: dockyard_rl.distributed.batched_data_dict.BatchedDataDict[Any]) → tuple[dockyard_rl.distributed.batched_data_dict.BatchedDataDict[Any], dict[str, float | int]]

Compute aggregate metrics over the completed rollout batch.

Args: batch: Batched rollout data. Expected keys: "rewards", "is_end", "generation_lengths", "prompt_lengths", "text".

Returns: Tuple of (unmodified batch, metrics dict).