Skip to main content

module cleanlab_codex.types.validator

class ThresholdedEvalMetric​

Evaluation metric reporting a quality score and optional logs.

Attributes:

  • score (float, optional): score between 0-1 corresponding to the evaluation metric. A higher score indicates a higher rating for the specific evaluation criteria being measured.

  • log (dict, optional): additional logs and metadata, reported only if the log key was specified in TLMOptions.

is_bad: bool Whether the score is a certain threshold.

class ThresholdedTrustworthyRAGScore​

Object returned by Validator.detect containing evaluation scores from TrustworthyRAGScore along with a boolean flag, is_bad, indicating whether the score is below the threshold.

Example:

    {
        "trustworthiness": {
            "score": 0.92,
            "log": {"explanation": "Did not find a reason to doubt trustworthiness."},
            "is_bad": False
        },
        "response_helpfulness": {
            "score": 0.35,
            "is_bad": True
        },
        ...
    }