# {py:mod}`dockyard_rl.models.generation.sglang.sglang_generation`

```{py:module} dockyard_rl.models.generation.sglang.sglang_generation
```

```{autodoc2-docstring} dockyard_rl.models.generation.sglang.sglang_generation
:allowtitles:
```

## Module Contents

### Classes

````{list-table}
:class: autosummary longtable
:align: left

* - {py:obj}`SGLangGeneration <dockyard_rl.models.generation.sglang.sglang_generation.SGLangGeneration>`
  -
````

### Data

````{list-table}
:class: autosummary longtable
:align: left

* - {py:obj}`TOP_K_THRESHOLD <dockyard_rl.models.generation.sglang.sglang_generation.TOP_K_THRESHOLD>`
  - ```{autodoc2-docstring} dockyard_rl.models.generation.sglang.sglang_generation.TOP_K_THRESHOLD
    :summary:
    ```
* - {py:obj}`TOP_P_THRESHOLD <dockyard_rl.models.generation.sglang.sglang_generation.TOP_P_THRESHOLD>`
  - ```{autodoc2-docstring} dockyard_rl.models.generation.sglang.sglang_generation.TOP_P_THRESHOLD
    :summary:
    ```
* - {py:obj}`logger <dockyard_rl.models.generation.sglang.sglang_generation.logger>`
  - ```{autodoc2-docstring} dockyard_rl.models.generation.sglang.sglang_generation.logger
    :summary:
    ```
````

### API

````{py:data} TOP_K_THRESHOLD
:canonical: dockyard_rl.models.generation.sglang.sglang_generation.TOP_K_THRESHOLD
:value: >
   8000

```{autodoc2-docstring} dockyard_rl.models.generation.sglang.sglang_generation.TOP_K_THRESHOLD
```

````

````{py:data} TOP_P_THRESHOLD
:canonical: dockyard_rl.models.generation.sglang.sglang_generation.TOP_P_THRESHOLD
:value: >
   0.99

```{autodoc2-docstring} dockyard_rl.models.generation.sglang.sglang_generation.TOP_P_THRESHOLD
```

````

````{py:data} logger
:canonical: dockyard_rl.models.generation.sglang.sglang_generation.logger
:value: >
   'getLogger(...)'

```{autodoc2-docstring} dockyard_rl.models.generation.sglang.sglang_generation.logger
```

````

`````{py:class} SGLangGeneration(cluster: dockyard_rl.distributed.virtual_cluster.RayVirtualCluster, config: dockyard_rl.models.generation.sglang.config.SGLangConfig, name_prefix: str = 'sglang_policy', workers_per_node: typing.Optional[typing.Union[int, list[int]]] = None)
:canonical: dockyard_rl.models.generation.sglang.sglang_generation.SGLangGeneration

Bases: {py:obj}`dockyard_rl.models.generation.interfaces.GenerationInterface`

````{py:method} init_collective(ip: str, port: int, world_size: int, *, train_world_size: int) -> list[ray.ObjectRef]
:canonical: dockyard_rl.models.generation.sglang.sglang_generation.SGLangGeneration.init_collective

```{autodoc2-docstring} dockyard_rl.models.generation.sglang.sglang_generation.SGLangGeneration.init_collective
```

````

````{py:method} generate(data: dockyard_rl.distributed.batched_data_dict.BatchedDataDict[dockyard_rl.models.generation.interfaces.GenerationDatumSpec], greedy: bool = False) -> dockyard_rl.distributed.batched_data_dict.BatchedDataDict[dockyard_rl.models.generation.interfaces.GenerationOutputSpec]
:canonical: dockyard_rl.models.generation.sglang.sglang_generation.SGLangGeneration.generate

```{autodoc2-docstring} dockyard_rl.models.generation.sglang.sglang_generation.SGLangGeneration.generate
```

````

````{py:method} generate_async(data: dockyard_rl.distributed.batched_data_dict.BatchedDataDict[dockyard_rl.models.generation.interfaces.GenerationDatumSpec], greedy: bool = False) -> typing.AsyncGenerator[tuple[int, dockyard_rl.distributed.batched_data_dict.BatchedDataDict[dockyard_rl.models.generation.interfaces.GenerationOutputSpec]], None]
:canonical: dockyard_rl.models.generation.sglang.sglang_generation.SGLangGeneration.generate_async
:async:

```{autodoc2-docstring} dockyard_rl.models.generation.sglang.sglang_generation.SGLangGeneration.generate_async
```

````

````{py:method} prepare_refit_info(state_dict_info: dict[str, typing.Any]) -> None
:canonical: dockyard_rl.models.generation.sglang.sglang_generation.SGLangGeneration.prepare_refit_info

````

````{py:method} update_weights_via_ipc_zmq() -> list[ray.ObjectRef]
:canonical: dockyard_rl.models.generation.sglang.sglang_generation.SGLangGeneration.update_weights_via_ipc_zmq

````

````{py:method} update_weights_from_collective() -> list[ray.ObjectRef]
:canonical: dockyard_rl.models.generation.sglang.sglang_generation.SGLangGeneration.update_weights_from_collective

````

````{py:method} get_sglang_server_urls() -> list[str]
:canonical: dockyard_rl.models.generation.sglang.sglang_generation.SGLangGeneration.get_sglang_server_urls

```{autodoc2-docstring} dockyard_rl.models.generation.sglang.sglang_generation.SGLangGeneration.get_sglang_server_urls
```

````

````{py:method} get_sglang_url_to_gpu_uuids() -> dict[str, list[str]]
:canonical: dockyard_rl.models.generation.sglang.sglang_generation.SGLangGeneration.get_sglang_url_to_gpu_uuids

```{autodoc2-docstring} dockyard_rl.models.generation.sglang.sglang_generation.SGLangGeneration.get_sglang_url_to_gpu_uuids
```

````

````{py:method} prepare_for_generation(*args: typing.Any, **kwargs: typing.Any) -> bool
:canonical: dockyard_rl.models.generation.sglang.sglang_generation.SGLangGeneration.prepare_for_generation

```{autodoc2-docstring} dockyard_rl.models.generation.sglang.sglang_generation.SGLangGeneration.prepare_for_generation
```

````

````{py:method} finish_generation(*args: typing.Any, **kwargs: typing.Any) -> bool
:canonical: dockyard_rl.models.generation.sglang.sglang_generation.SGLangGeneration.finish_generation

```{autodoc2-docstring} dockyard_rl.models.generation.sglang.sglang_generation.SGLangGeneration.finish_generation
```

````

````{py:method} shutdown() -> bool
:canonical: dockyard_rl.models.generation.sglang.sglang_generation.SGLangGeneration.shutdown

```{autodoc2-docstring} dockyard_rl.models.generation.sglang.sglang_generation.SGLangGeneration.shutdown
```

````

````{py:method} invalidate_kv_cache() -> bool
:canonical: dockyard_rl.models.generation.sglang.sglang_generation.SGLangGeneration.invalidate_kv_cache

```{autodoc2-docstring} dockyard_rl.models.generation.sglang.sglang_generation.SGLangGeneration.invalidate_kv_cache
```

````

`````