Class LLMObservabilityApi

Create a new annotation queue. Only name, project_id, and description are accepted. Fields such as created_by, owned_by, created_at, modified_by, and modified_at are inferred by the backend.

Add one or more interactions (traces) to an annotation queue. At least one interaction must be provided.

Create a new LLM Observability dataset within the specified project.

Append one or more records to an LLM Observability dataset.

Create a new LLM Observability experiment.

Push spans and metrics for an LLM Observability experiment.

Create a new LLM Observability project. Returns the existing project if a name conflict occurs.

Delete an annotation queue by its ID.

Delete one or more interactions from an annotation queue.

Delete a custom LLM Observability evaluator configuration by its name.

Delete one or more records from an LLM Observability dataset.

Delete one or more LLM Observability datasets within the specified project.

Delete one or more LLM Observability experiments.

Delete one or more LLM Observability projects.

Retrieve all interactions and their annotations for a given annotation queue.

Retrieve a custom LLM Observability evaluator configuration by its name.

List annotation queues. Optionally filter by project ID or queue IDs. These parameters are mutually exclusive. If neither is provided, all queues in the organization are returned.

List all records in an LLM Observability dataset, sorted by creation date, newest first.

List all LLM Observability datasets for a project, sorted by creation date, newest first.

List all LLM Observability experiments sorted by creation date, newest first.

List all LLM Observability projects sorted by creation date, newest first.

Partially update an annotation queue. Only name and description can be updated.

Create or update a custom LLM Observability evaluator configuration by its name.

Partially update an existing LLM Observability dataset within the specified project.

Update one or more existing records in an LLM Observability dataset.

Partially update an existing LLM Observability experiment.

Partially update an existing LLM Observability project.