Evaluate and run large AI models affordably with Bytez – achieve GPU performance at CPU pricing.

Two steps to run inference in minutes:

1. Get your API Key
2. Choose your path: run inference locally via Docker or use our API (javascript, python, REST API)

Join the Bytez Discord or go to Bytez.com, sign in, and visit user settings to get your key.

All Bytez models are available on Docker Hub or our About page 🤙

• Python
• Javascript
• Julia

Load and run a model after installing our python library (pip install Bytez).

import os
from bytez import Bytez
client = Bytez(api_key=os.environ.get("YOUR_API_KEY")

# Grab a model
model = client.model('openai-community/gpt2')

# Start a model
model.load()

# Run a model
output = model.run("Once upon a time there was a", model_params={"max_new_tokens":1,"min_new_tokens":1})
print(output)

See the API Documentation for all examples.

Load and run a model after installing our Typescript library (npm i bytez.js).

import Bytez from "bytez.js";
client = new Bytez("YOUR_API_KEY");

// Grab a model
model = client.model("openai-community/gpt2");

// Start a model
await model.load();
console.log(results);

// Run a model
output = await model.run("Once upon a time there was a");
console.log(output);

See API Documentation for all examples.

Load and run a model after installing our Bytez library (add Bytez).

Interactive Notebook! (Coming Soon)

using Bytez
client = Bytez.init("YOUR_API_KEY");

# Grab a model
# args => modelId, concurrency = 1, timeout = 300 secs
model = client.model("openai-community/gpt2")

# Start a model
model.load()

# Run a model
output = model.run("Roses are")
println(output)

Bytez has a REST API for loading, running, and requesting new models.

curl --location 'https://api.bytez.com/model/load' \
--header 'Authorization: Key API_KEY' \
--header 'Content-Type: application/json' \
--data '{
    "model": "openai-community/gpt2",
    "concurrency": 1
}'

curl --location 'https://api.bytez.com/model/run' \
--header 'Authorization: Key API_KEY' \
--header 'Content-Type: application/json' \
--data '{
    "model": "openai-community/gpt2",
    "prompt": "Once upon a time there was a",
    "params": {
        "min_length": 30,
        "max_length": 256
    },
    "stream": true
}'

curl --location 'https://api.bytez.com/model/job' \
--header 'Authorization: Key API_KEY' \
--header 'Content-Type: application/json' \
--data '{
    "model": "openai-community/gpt2"
}'

See the API Documentation for all endpoints.

We currently support 14K+ open source AI models across 30+ ML tasks.

To see the full list, run:

models = client.list_models()
print(models)

Here are some models that can be run - with their required RAM.

• About Us
• Cold Boot Times and Billing