⚡️ Model Inference

​

Model API

Oxen.ai’s Model API allows you to start building on top of the latest and greatest models from OpenAI, Anthropic, Google, Meta, and more with a single API key.

The API supports fine-tuned open-source models. If a model is too slow, costly, inaccurate, or if you want full control, you can use our one-click interface to fine-tune and deploy a custom model using the same API.

curl -X POST https://hub.oxen.ai/api/chat/completions \
-H "Authorization: Bearer $OXEN_API_KEY" \
-H "Content-Type: application/json" \
-d '{
  "model": "openai:gpt-4o-mini",
  "messages": [{"role": "user", "content": "Hello, how are you?"}]
}'

The response format is compatible with OpenAI, so you can use any supported client to interact with the model. Simply prepend the model name with a provider like openai:gpt-4o-mini and use https://hub.oxen.ai/api as your base_url.

The currently supported list of providers is:

• openai
• anthropic
• fireworks
• google
• mistral
• deepseek
• oxenai (for fine-tuned models)

To see the list of generally available models, visit the models page.

​

Chat Interface

To try out models without writing code, you can quickly test out the boundaries of any model by chatting with it in the UI. This is a great way to kick the tires of a model you fine-tuned after deploying it.

Feel free to experiment with a model and see how it performs before you decide to use it in your application.

​

Run Inference on a Dataset

At the end of the day, AI is about automating workflows and making sure your model performs well given your data.

With Oxen.ai, you can kick off a model to run on a dataset while you sit back and relax. This is a great way to iterate on your system prompt, evaluate a model, or to augment your dataset with new data.

​

How it Works

You provide a prompt, choose a model, and run the evaluation on any dataset file in your repository. The system uses column values and inserts them into variables marked with {variable_name} to give the model context.

​

Upload a Dataset

Upload your dataset to Oxen.ai and click the “Actions” button and “Run Inference” to select the model you want to run inference on.

​

Select a Model and Prompt

In the prompt editor you can explore models from a variety of developers. Setup your prompt and decide how many rows to run the model on. Click the Run Sample button.

​

Run a Sample

This will run the model on the first N rows of data and show you the results as well as an estimated price. Make sure the output looks like what you expect. If not, you can edit the prompt and run again.

​

Run the Inference

Once you are satisfied with your prompt, you can pick a destination branch and write a commit message for once the job is finished.

Now you can grab some coffee, sit back, and watch the model run. Feel free to close the tab, the job will be running in the background and committed to specified file and branch.