​

oxen.oxen_fs

​

OxenFS Objects

class OxenFS(fsspec.AbstractFileSystem)

OxenFS is a filesystem interface for Oxen repositories that implements the fsspec protocol. This allows you to interact with Oxen repositories using familiar filesystem operations and integrate with other compatible libraries like Pandas.

​

Basic Usage

​

Creating a Filesystem Instance

import oxen

# For Oxen Hub repositories
fs = oxen.OxenFS("ox", "Flowers")

# For local oxen-server
fs = oxen.OxenFS("ox", "test-repo", host="localhost:3000", scheme="http")

​

Reading Files

with fs.open("data/train.csv") as f:
    content = f.read()

​

Writing Files

You must have write access to the repository to write files. See: https://docs.oxen.ai/getting-started/python#private-repositories

OxenFS will automatically commit the file to the repository when the context is exited (or the file is closed some other way). New directories are automatically created as needed.

# Write with custom commit message
with fs.open("data/test.txt", mode="wb", commit_message="Added test.txt") as f:
    f.write("Hello, world!")

# You can also set/update the commit message inside the context
with fs.open("data/test.txt", mode="wb") as f:
    f.commit_message = "Updated test.txt"
    f.write("Hello, world again!")

​

Writing file objects

If you’re integrating Oxen in a situation where you already have a file object, you can save it to your repo by using shutil.copyfileobj like this:

import shutil

file_object_from_somewhere = open("data.csv")

with fs.open("train/data.csv", mode="wb") as output_file:
    output_file.commit_message = "Copy from a file object"
    shutil.copyfileobj(file_object_from_somewhere, output_file)

​

Integration with Third Party Libraries (Pandas, etc.)

OxenFS works seamlessly with Pandas and other fsspec-compatible libraries using the URL format: oxen://namespace:repo@revision/path/to/file

​

Reading Data

These will work with Pandas {to,from}_{csv,parquet,json,etc.} functions.

import pandas as pd

# Read parquet directly from Oxen repository
df = pd.read_parquet("oxen://openai:gsm8k@main/gsm8k_test.parquet")

​

Writing Data

# Write DataFrame directly to Oxen repository
df.to_csv("oxen://ox:my-repo@main/data/test.csv", index=False)

​

Notes

• Only binary read (“rb”) and write (“wb”) modes are currently supported
  • But writing will automatically encode strings to bytes
• Does not yet support streaming files. All operations use temporary local files.

​

__init__

def __init__(namespace: str,
             repo: str,
             host: str = "hub.oxen.ai",
             revision: str = "main",
             scheme: str = "https",
             **kwargs)

Initialize the OxenFS instance.

Arguments:

• namespace - str The namespace of the repository.
• repo - str The name of the repository.
• host - str The host to connect to. Defaults to ‘hub.oxen.ai’
• revision - str The branch name or commit id to checkout. Defaults to ‘main’
• scheme - str The scheme to use for the remote url. Default: ‘https’

​

ls

def ls(path: str = "", detail: bool = False)

List the contents of a directory.

Arguments:

• path - str The path to list the contents of.
• detail - bool If True, return a list of dictionaries with detailed metadata. Otherwise, return a list of strings with the filenames.

​

OxenFSFileWriter Objects

class OxenFSFileWriter()

A file writer for the OxenFS backend.

This is normally called through OxenFS.open() or fsspec.open().

​

write

def write(data: str | bytes)

Write string or binary data to the file.

​

flush

def flush()

Flush the file to disk.

​

tell

def tell()

Return the current position of the file.

​

seek

def seek(offset: int, whence: int = os.SEEK_SET)

Seek to a specific position in the file.

​

commit

def commit(commit_message: Optional[str] = None)

Commit the file to the remote repo.

​

close

def close()

Close the file writer. This will commit the file to the remote repo.