Skip to main content

module cleanlab_tlm.utils.chat_completions

Real-time evaluation of responses from OpenAI Chat Completions API.

If you are using OpenAI’s Chat Completions API, this module allows you to incorporate TLM trust scoring without any change to your existing code. It works for any OpenAI LLM model, as well as the many other non-OpenAI LLMs that are also usable via Chat Completions API (Gemini, DeepSeek, Llama, etc).

class TLMChatCompletion​

Represents a Trustworthy Language Model (TLM) instance specifically designed for evaluating OpenAI Chat Completions responses.

This class provides a TLM wrapper that can be used to evaluate the quality and trustworthiness of responses from any OpenAI model by passing in the inputs to OpenAI’s Chat Completions API and the ChatCompletion response object.

Args:

  • quality_preset ({“base”, “low”, “medium”}, default = “medium”): an optional preset configuration to control the quality of TLM trustworthiness scores vs. latency/costs.

  • api_key (str, optional): Cleanlab TLM API key. If not provided, will attempt to read from CLEANLAB_API_KEY environment variable.

  • options (TLMOptions, optional): a typed dict of configurations you can optionally specify. See detailed documentation under TLMOptions.

  • timeout (float, optional): timeout (in seconds) to apply to each TLM evaluation.

method score​

score(response: 'ChatCompletion', **openai_kwargs: Any) → TLMScore

Score the trustworthiness of an OpenAI ChatCompletion response.

Args:

  • response (ChatCompletion): The OpenAI ChatCompletion response object to evaluate
  • **openai_kwargs (Any): The original kwargs passed to OpenAI’s create() method, must include ‘messages’

Returns:

  • TLMScore: A dict containing the trustworthiness score and optional logs