Optional additionalA container for additional, undeclared properties. This is a holder for any undeclared properties as specified with the 'additionalProperties' keyword in the OAS document.
Optional assessmentCriteria used to assess the pass/fail result of a custom evaluator.
Optional contextQuery used to extract additional context for the evaluation.
LLM inference parameters for a custom evaluator.
Optional lastName of the last library prompt template used.
Optional modifiedWhether the library prompt template was modified.
Optional outputJSON schema describing the expected output format of the LLM judge. Omit for a deterministic evaluator.
Optional parsingOutput parsing type for a custom LLM judge evaluator.
Optional promptList of messages forming the LLM judge prompt template. Omit for a deterministic evaluator.
Optional targetQuery used to extract the target value to evaluate.
Optional userUser-provided JavaScript function applied to post-process the JSON output of the LLM judge. For a deterministic evaluator it is the evaluator itself: it receives the evaluated span (span scope), {spans} (trace scope), or {session_id, traces} (session scope), and its return value sets the result. A boolean, number, string or object becomes a boolean, score, categorical or JSON metric; null skips the evaluation; and {value, assessment, reasoning, metadata} also sets pass or fail, reasoning, and metadata.
Generated using TypeDoc
LLM judge configuration for a custom evaluator.