Skip to main content
POST
/
api
/
repos
/
{namespace}
/
{repo_name}
/
evaluations
/
*resource_path
Create an evaluation
curl --request POST \
  --url 'https://hub.oxen.ai/api/repos/{namespace}/{repo_name}/evaluations/*resource_path' \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "input_type": "text",
  "model_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
  "output_type": "text",
  "prompt": "<string>",
  "target_column": "<string>",
  "auto_commit": false,
  "batch_size": 10,
  "commit_message": "<string>",
  "is_sample": false,
  "name": "<string>",
  "sample_count": 10,
  "target_branch": "<string>",
  "target_path": "<string>"
}
'
{
  "evaluation": {
    "cancelled_at": "<string>",
    "completed_at": "<string>",
    "completion_tokens_used": 123,
    "created_by": {
      "id": "<string>",
      "image": "<string>",
      "name": "<string>",
      "username": "<string>"
    },
    "credits_used": 123,
    "error_message": "<string>",
    "id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
    "input_type": "text",
    "inserted_at": "<string>",
    "is_sample": true,
    "model": {},
    "name": "<string>",
    "output_type": "text",
    "progress": {
      "processed": 123,
      "total": 123
    },
    "prompt": "<string>",
    "prompt_tokens_used": 123,
    "repository_id": "<string>",
    "resource": {
      "path": "<string>",
      "version": "<string>"
    },
    "sample_count": 123,
    "started_at": "<string>",
    "status": "queued",
    "target_branch": "<string>",
    "target_column": "<string>",
    "target_path": "<string>",
    "tokens_used": 123
  },
  "status": "<string>",
  "status_message": "<string>"
}

Authorizations

​
Authorization
string
header
required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Body

application/json

Create evaluation request

Request payload to create an evaluation.

The resource path (branch + file path) is specified in the URL after /evaluations/, for example: POST /api/repos/{namespace}/{repo}/evaluations/main/datasets/training.parquet.

​
input_type
enum<string>
required

Type of input data

Available options:
text,
image,
video
​
model_id
string<uuid>
required

ID of the model to run inference with

​
output_type
enum<string>
required

Type of output produced by the model

Available options:
text,
image,
video,
embeddings
​
prompt
string
required

Prompt template sent to the model. Use {column_name} placeholders to inject values from each row.

​
target_column
string
required

Column where the model output will be written

​
auto_commit
boolean
default:false

If true, automatically commit the results when the evaluation completes

​
batch_size
integer
default:10

Number of rows to process per inference batch

​
commit_message
string | null

Commit message used when auto_commit is true

​
is_sample
boolean
default:false

If true, only evaluate a subset of rows. If false, evaluate all rows.

​
name
string

Human-readable name for the evaluation

​
sample_count
integer
default:10

Number of rows to sample when is_sample is true

​
target_branch
string

Branch where evaluation results are committed

​
target_path
string

File path for the output data frame

Response

Evaluation created

Standard response wrapper for a single evaluation.

​
evaluation
Evaluation · object

Evaluation resource

​
status
string

High-level status string (for example, 'success').

​
status_message
string

Human-readable status message (for example, 'resource_found').