LLMBenchmarkMetrics

class baseten.client.managementapi.LLMBenchmarkMetrics(*, ttft_ms_p50=None, output_tokens_per_sec_per_user_p50=None, max_concurrent_users_at_50ms_tpot=None, requests_per_sec_p50=None, cost_per_1m_tokens_usd=None)

Bases: BaseModel

Parameters:

• ttft_ms_p50 (float | None)

• output_tokens_per_sec_per_user_p50 (float | None)

• max_concurrent_users_at_50ms_tpot (int | None)

• requests_per_sec_p50 (float | None)

• cost_per_1m_tokens_usd (float | None)

model_config = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].