dockyard_rl.models.generation.sglang.sglang_copied_utils

Standalone utility functions copied from the SGLang project.

This module contains utility functions that were originally part of the SGLang repository (https://github.com/sgl-project/sglang). They have been copied here to avoid requiring sglang as a runtime dependency for weight refitting functionality.

IMPORTANT: This module should NOT contain any imports from the sglang package. All functions are standalone and self-contained.

Each function includes a permalink to its original source in the SGLang repository. These functions were copied from sglang version 0.5.2.

Module Contents

Classes

MultiprocessingSerializer

Serialize/deserialize Python objects using ForkingPickler for IPC.

Functions

monkey_patch_torch_reductions

Monkey patch torch multiprocessing reductions to use GPU UUIDs.

API

class dockyard_rl.models.generation.sglang.sglang_copied_utils.MultiprocessingSerializer

Serialize/deserialize Python objects using ForkingPickler for IPC.

This class enables serialization of objects (including CUDA tensors with IPC handles) for transfer between processes via HTTP or other mechanisms.

Original source (sglang v0.5.2): https://github.com/sgl-project/sglang/blob/v0.5.2/python/sglang/srt/utils.py#L589-L623

static serialize(obj, output_str: bool = False)

Serialize a Python object using ForkingPickler.

Args: obj: The object to serialize. output_str (bool): If True, return a base64-encoded string instead of raw bytes.

Returns: bytes or str: The serialized object.

static deserialize(data)

Deserialize a previously serialized object.

Args: data (bytes or str): The serialized data, optionally base64-encoded.

Returns: The deserialized Python object.

dockyard_rl.models.generation.sglang.sglang_copied_utils.monkey_patch_torch_reductions()

Monkey patch torch multiprocessing reductions to use GPU UUIDs.

This patch modifies PyTorch’s CUDA tensor IPC mechanism to use GPU UUIDs instead of device indices. This enables proper weight transfer between processes that may have different CUDA_VISIBLE_DEVICES configurations.

The patch is idempotent - calling it multiple times is safe.

This is a workaround before PyTorch https://github.com/pytorch/pytorch/pull/149248 is merged and released.

Original source (sglang v0.5.2): https://github.com/sgl-project/sglang/blob/v0.5.2/python/sglang/srt/patch_torch.py#L20-L33