oxen-server is the storage backend for Oxen. It is where the merkle tree, commit history, and other metadata is stored.
You can deploy your own oxen-server instance on your own infrastructure, or use the hosted version on OxenHub. If you want to kick the tires of Oxen in the privacy of your own infrastructure, we recommend you setup a local server.
oxen-server start -p 3000 -i 0.0.0.0
The hosted solution comes with a UI and the benefits of not having to setup infrastructure yourself. Sign up here to get started.
To setup a local Oxen Server instance, first install the oxen-server binary.
oxen-server is supported on Linux, macOS, and Docker. It is not supported on Windows. The oxen CLI and Python client are supported on Windows and can connect to a server hosted on a supported platform.
User access token created:XXXXXXXXTo give user access have them run the command `oxen config --auth <HOST> <TOKEN>`
You may have different authentication tokens for different hosts. From the client side, you can setup an auth token per host with the config command. If you ever need to debug or edit the tokens manually, they are stored in the ~/.config/oxen/auth_config.toml file.
The default directory that Oxen stores data is /tmp/oxen_sync, which is not a good idea for production. To change it set the SYNC_DIR environment variable to a path.
export SYNC_DIR=/var/oxen/dataoxen-server start -a
Output:
Running 🐂 server on 0.0.0.0:3000Syncing to directory: /var/oxen/data[2022-06-08T10:00:48Z INFO actix_server::builder] Starting 8 workers[2022-06-08T10:00:48Z INFO actix_server::server] Actix runtime found; starting in Actix runtime
If you want to change the default IP ADDRESS and PORT you can do so by passing them in with the -i and -p parameters.
The oxen server provides debug logs, which are off by default. You can turn these on with the RUST_LOG variable.
export RUST_LOG="debug"
You can also set RUST_LOG to info or warn for more restrictive debug logs. Be aware, turning on debug logs can significantly slow down some metadata-heavy operations like oxen commit
Note: The host and scheme are optional and default to hub.oxen.ai and https respectively. If you are running a local server, you can set the host to localhost:3000 and the scheme to http.You can either clone data from this remote repository, or push data to it.
When you create a remote repository, Oxen will create a directory for it on the server. The directory structure is $SYNC_DIR/<namespace>/<repo_name>/.oxen.
ls /var/oxen/data/my_namespace/repo_name/.oxen
Output:
config.tomlhistory/refs/tree/versions/
All of the metadata and versioned files for a repository are stored in the .oxen directory. This directory mirrors the .oxen directory in your local repository, so that logic can be reused between the client and server.
Oxen allows you to configure the storage backend of self-hosted oxen servers. By default, a repository’s version files are stored in the .oxen/versions/files folder, but this can be changed by using the --storage-backend and --storage-backend-path parameters
This is useful if you have a large amount of data and you want to store it on a virtual file system. If you set --storage-backend-path to a location on your VFS, files that you push to the remote will be stored there. Depending on your VFS, this can slow down some upload and download operations.
To upload data to the server, you can use the oxen CLI to initialize a local repository, add data to it, and push it to the server.
# Create a directory for the new datasetmkdir my-datasetcd my-dataset# Initialize a local repositoryoxen init# Add data to the repositoryecho "prompt,response" > data.csvoxen add data.csv# Commit the changesoxen commit -m "Initial commit"
If you look in your local repository, you will see the .oxen directory.
ls .oxen
Output:
config.tomlhistory/refs/tree/versions/
You can set the remote to the server by running the following command. This will update the config.toml file in your local repository.
