Class LLMObservabilityApi

Create an annotation queue. The name and project_id fields are required. An optional annotation_schema can be provided to define the labels for the queue. Fields such as created_by, owned_by, created_at, modified_by, and modified_at are inferred by the backend.

Add one or more interactions (traces or sessions) to an annotation queue. At least one interaction must be provided.

Create a new LLM Observability dataset within the specified project.

Append one or more records to an LLM Observability dataset.

Create a new LLM Observability experiment.

Push spans and metrics for an LLM Observability experiment.

Create a new LLM Observability project. Returns the existing project if a name conflict occurs.

Delete an annotation queue by its ID.

Delete one or more interactions from an annotation queue.

Delete a custom LLM Observability evaluator configuration by its name.

Delete one or more records from an LLM Observability dataset.

Delete one or more LLM Observability datasets within the specified project.

Delete one or more LLM Observability experiments.

Delete one or more LLM Observability projects.

Retrieve all interactions (traces and sessions) and their annotations for a given annotation queue.

Retrieve the label schema for a given annotation queue.

Retrieve a custom LLM Observability evaluator configuration by its name.

List annotation queues. Optionally filter by project ID or queue IDs. These parameters are mutually exclusive. If neither is provided, all queues in the organization are returned.

List all records in an LLM Observability dataset, sorted by creation date, newest first.

List all LLM Observability datasets for a project, sorted by creation date, newest first.

List all LLM Observability experiments sorted by creation date, newest first.

List all LLM Observability projects sorted by creation date, newest first.

Partially update an annotation queue. The name, description, and annotation_schema fields can be updated.

Create or replace the label schema for a given annotation queue. The label schema defines the labels annotators can apply to interactions in the queue. Label names must be unique within the queue and match the pattern ^[a-zA-Z0-9_-]+$. Each label must have a valid type: score, categorical, boolean, or text.

Create or update a custom LLM Observability evaluator configuration by its name.

Partially update an existing LLM Observability dataset within the specified project.

Update one or more existing records in an LLM Observability dataset.

Partially update an existing LLM Observability experiment.

Partially update an existing LLM Observability project.