`dockyard_rl.utils.device_capability`¶

Device compute-capability resolution and precision validation.

Before this, the entire tree contained exactly one capability check – torch.cuda.get_device_capability()[0] >= 8 at models/generation/vllm/vllm_worker.py, used only as a default for enable_prefix_caching. Nothing gated a requested precision on what the target hardware can actually run, so a config asking for precision: fp8 on an A100 failed somewhere inside vLLM rather than at startup with a usable message.

Thresholds here are derived from the shipped binaries, not from documentation. vllm._custom_ops.cutlass_scaled_mm_supports_fp8 and its block/group-gemm siblings take a capability integer, so the real cutoffs are checkable without a GPU. Measured against vLLM 0.27.1:

sm       device                fp8    block-fp8   group-gemm
/ 75  V100 / T4             no     no          no
/ 86  A100 / A10            no     no          no
     L40S / L4 (Ada)       YES    no          no
     H100 / H200           YES    YES         YES
100/103  B200 / GB300          YES    YES         YES
    RTX PRO 6000          YES    YES         no

Two consequences that documentation alone would have got wrong:

dockyard’s FP8 needs sm90, not sm89. Plain FP8 starts at Ada (sm89), but dockyard only ever emits block-scaled FP8 – FP8_BLOCK_QUANT_KWARGS pins weight_block_size=[128, 128] with no config override, and is_fp8_model asserts block scaling is present. Block FP8 starts at sm90. An L40S therefore passes a naive supports_fp8 check and still cannot run this path.
torch.cuda.is_bf16_supported() must not be used to gate bf16. It defaults to including_emulation=True: below sm80 it falls through to _check_bf16_tensor_supported, which only asks whether a bf16 tensor can be created. On a T4 it returns True while every bf16 op is emulated. Native bf16 is major >= 8, and that is what supports_native_bf16 reports.

No MXFP8 / NVFP4 predicate ships here yet, and the omission is deliberate. A supports_mxfp8_nvfp4 property gating on sm100 was written alongside the three above; measuring vLLM 0.27.1’s kernel ladder showed both halves of it to be wrong. _POSSIBLE_MXFP8_KERNELS[CUDA] is FlashInfer CuteDSL, FlashInfer CUTLASS, Marlin, emulation, Humming – and only the two FlashInfer kernels require sm100. MarlinMxfp8LinearKernel.is_supported defers to is_fp8_marlin_supported(), which is has_device_capability(75), and Marlin sits above emulation in that list, so MXFP8 linear reaches a real kernel from sm75 up. One integer cannot express that: sm100 is the native-tensor-core line, sm75 the Marlin-fallback line, and which one a deployment should refuse below depends on a performance claim no CPU measurement can settle. The predicate also had no consumer – dockyard has no MXFP8 precision path, and a capability question nothing asks is the shape the wiring gate exists to catch. It returns when C2 lands a path that reads it, with the threshold that path actually needs.

Degradation policy: refuse, never downgrade. If a config asks for a precision the device cannot run natively, startup fails with the device name, its capability, and what would be required. Silently substituting a different dtype would change training numerics without telling the operator – the same failure shape as every defect in Track B, where nothing errored and the run looked healthy.

Module Contents¶

Classes¶

DeviceCapability

What a single CUDA device can run natively.

Functions¶

`resolve_device_capability`	Read the current device’s capability, or None when there is no CUDA device.
`validate_precision`	Refuse a precision the device cannot run natively.
`fp8_e4m3_dtype`	The e4m3 FP8 encoding the current device actually implements.

Data¶

`logger`
`SM_NATIVE_BF16`
`SM_FP8`
`SM_BLOCK_FP8`
`SM_PREFIX_CACHING`

API¶

dockyard_rl.utils.device_capability.logger¶: ‘getLogger(…)’

dockyard_rl.utils.device_capability.SM_NATIVE_BF16¶: 80

dockyard_rl.utils.device_capability.SM_FP8¶: 89

dockyard_rl.utils.device_capability.SM_BLOCK_FP8¶: 90

dockyard_rl.utils.device_capability.SM_PREFIX_CACHING¶: 80

class dockyard_rl.utils.device_capability.DeviceCapability¶

What a single CUDA device can run natively.

Constructed directly in tests; resolve_device_capability builds it from the live device.

major: int¶: None

minor: int¶: None

name: str¶: ‘unknown device’

property sm: int¶: Capability as the integer vLLM’s predicates take (e.g. 90).

property supports_native_bf16: bool¶

Native bfloat16, i.e. Ampere or newer.

Deliberately not torch.cuda.is_bf16_supported(), which counts emulation and returns True on a T4.

property supports_fp8: bool¶: Plain FP8 scaled_mm (Ada sm89 and newer).

property supports_block_fp8: bool¶: Block-scaled FP8 (Hopper sm90 and newer) – the only FP8 dockyard emits.

property supports_prefix_caching: bool¶: Matches the pre-existing vllm_worker default this module replaces.

describe() → str¶

dockyard_rl.utils.device_capability.resolve_device_capability(device: Optional[int] = None) → Optional[dockyard_rl.utils.device_capability.DeviceCapability]¶

Read the current device’s capability, or None when there is no CUDA device.

None is a legitimate answer, not an error: the sandbox fleet is CPU-only and the unit suite runs without a GPU. Callers treat None as “cannot validate” and skip, rather than refusing to start.

Args: device: CUDA device index; defaults to the current device.

Returns: The device’s capability, or None if CUDA is unavailable.

dockyard_rl.utils.device_capability.validate_precision(precision: str, capability: Optional[dockyard_rl.utils.device_capability.DeviceCapability], *, context: str) → None¶

Refuse a precision the device cannot run natively.

Args: precision: The configured precision string. capability: The device’s capability, or None to skip (no CUDA). context: Where the precision came from, for the message (e.g. “policy.precision”).

Raises: ValueError: The device cannot run this precision natively.

dockyard_rl.utils.device_capability.fp8_e4m3_dtype() → torch.dtype¶

The e4m3 FP8 encoding the current device actually implements.

Two encodings exist and they are not interchangeable. NVIDIA and CDNA4 (gfx950) use OCP float8_e4m3fn; CDNA3 (gfx942 – MI300X/MI325X) uses float8_e4m3fnuz, which differs in how it encodes zero’s sign and NaN. Casting with the wrong one produces wrong numbers rather than an error, which is why this is resolved rather than hardcoded.

The answer is taken from the pinned vLLM’s own platform layer rather than reimplemented here: RocmPlatform.is_fp8_fnuz() matches "gfx94" as a prefix of the reported gcnArchName, and a second copy of that rule in this tree could only drift away from the kernels that consume the tensors.

Raises: RuntimeError: On a HIP build where vLLM cannot be imported. The encoding is genuinely ambiguous there and guessing it would corrupt weights silently; on a CUDA build it is not ambiguous, so vLLM’s absence is not an error.

dockyard_rl.utils.device_capability¶