ModelTRTLLMPluginConfiguration

class baseten.client.modelconfig.ModelTRTLLMPluginConfiguration(*, paged_kv_cache=True, use_paged_context_fmha=True, use_fp8_context_fmha=False, **extra_data)

Bases: BaseModel

Parameters:

• paged_kv_cache (bool | None)

• use_paged_context_fmha (bool | None)

• use_fp8_context_fmha (bool | None)

• extra_data (Any)

model_config = {'extra': 'allow'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].