TRTLLMConfigurationV1

class baseten.client.modelconfig.TRTLLMConfigurationV1(*, build, inference_stack='v1', runtime=ModelTRTLLMRuntimeConfiguration(kv_cache_free_gpu_mem_fraction=0.9, kv_cache_host_memory_bytes=None, lora_cache_max_adapter_size=None, lora_cache_optimal_adapter_size=None, lora_cache_gpu_memory_fraction=None, lora_cache_host_memory_bytes=None, enable_chunked_context=True, batch_scheduler_policy=<ModelTRTLLMBatchSchedulerPolicy.guaranteed_no_evict: 'guaranteed_no_evict'>, request_default_max_tokens=None, served_model_name=None, total_token_limit=500000, webserver_default_route=None), version_overrides=VersionsOverrides(engine_builder_version=None, briton_version=None, bei_version=None, bei_bert_version=None, v2_llm_version=None), **extra_data)

Bases: BaseModel

Parameters:
model_config = {'extra': 'allow'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].