oxen-server is the storage backend for Oxen. It is where the merkle tree, commit history, and other metadata is stored.
You can deploy your own oxen-server instance on your own infrastructure, or use the hosted version on OxenHub. If you want to kick the tires of Oxen in the privacy of your own infrastructure, we recommend you setup a local server.
oxen-server start -p 3000 -i 0.0.0.0
The hosted solution comes with a UI and the benefits of not having to setup infrastructure yourself. Sign up here to get started.
To setup a local Oxen Server instance, first install the oxen-server binary.
oxen-server is supported on Linux, macOS, and Docker. It is not supported on Windows. The oxen CLI and Python client are supported on Windows and can connect to a server hosted on a supported platform.
oxen-server does not authenticate requests itself. To require authentication, run it behind a reverse proxy or gateway that checks each request before forwarding it. If the gateway checks a bearer token, give each client its token with the config command. It keeps one token per host in ~/.config/oxen/auth_config.toml and sends it as Authorization: Bearer <TOKEN> on every request to that host.
oxen config --auth <HOST> <TOKEN>
oxen-server 0.59.0 removed the -a flag and the add-user command, and a server started with -a refuses to start. See the upgrade steps before upgrading a server that uses them.
The default directory that Oxen stores data is data, relative to the working directory the server was started from, which is convenient for trying things out but not what you want in production. To change it set the SYNC_DIR environment variable to an absolute path.
export SYNC_DIR=/var/oxen/dataoxen-server start
Output:
Running 🐂 server on 0.0.0.0:3000Syncing to directory: /var/oxen/data[2022-06-08T10:00:48Z INFO actix_server::builder] Starting 8 workers[2022-06-08T10:00:48Z INFO actix_server::server] Actix runtime found; starting in Actix runtime
If you want to change the default IP ADDRESS and PORT you can do so by passing them in with the -i and -p parameters.
The oxen server provides debug logs, which are off by default. You can turn these on with the RUST_LOG variable.
export RUST_LOG="debug"
You can also set RUST_LOG to info or warn for more restrictive debug logs. Be aware, turning on debug logs can significantly slow down some metadata-heavy operations like oxen commit
Note: The host and scheme are optional and default to hub.oxen.ai and https respectively. If you are running a local server, you can set the host to localhost:3000 and the scheme to http.You can either clone data from this remote repository, or push data to it.
When you create a remote repository, Oxen will create a directory for it on the server. The directory structure is $SYNC_DIR/<namespace>/<repo_name>/.oxen.
ls /var/oxen/data/my_namespace/repo_name/.oxen
Output:
config.tomlhistory/refs/tree/versions/
All of the metadata and versioned files for a repository are stored in the .oxen directory. This directory mirrors the .oxen directory in your local repository, so that logic can be reused between the client and server.
Oxen allows you to configure the storage backend of self-hosted oxen servers. By default, a repository’s version files are stored in the .oxen/versions/files folder, but this can be changed by using the --storage-backend and --storage-backend-path parameters
This is useful if you have a large amount of data and you want to store it on a virtual file system. If you set --storage-backend-path to a location on your VFS, files that you push to the remote will be stored there. Depending on your VFS, this can slow down some upload and download operations.
To upload data to the server, you can use the oxen CLI to initialize a local repository, add data to it, and push it to the server.
# Create a directory for the new datasetmkdir my-datasetcd my-dataset# Initialize a local repositoryoxen init# Add data to the repositoryecho "prompt,response" > data.csvoxen add data.csv# Commit the changesoxen commit -m "Initial commit"
If you look in your local repository, you will see the .oxen directory.
ls .oxen
Output:
config.tomlhistory/refs/tree/versions/
You can set the remote to the server by running the following command. This will update the config.toml file in your local repository.
