TRTLLMConfigurationV2

class baseten.client.modelconfig.TRTLLMConfigurationV2(*, inference_stack='v2', build, runtime, version_overrides=VersionsOverrides(engine_builder_version=None, briton_version=None, bei_version=None, bei_bert_version=None, v2_llm_version=None), **extra_data)

Bases: BaseModel

Parameters:

• inference_stack (Literal['v2'])

• build (ModelTRTLLMBuildConfiguration)

• runtime (TRTLLMRuntimeConfigurationV2)

• version_overrides (VersionsOverrides)

• extra_data (Any)

model_config = {'extra': 'allow'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].