How to run an OpenAI-compatible API on your server

Many apps and libraries are written for the OpenAI API. You can run a local API with the same structure, so your apps use your models without major changes.

Ollama exposes a compatible API

Ollama already offers an OpenAI-compatible endpoint:

curl http://localhost:11434/v1/chat/completions -d '{
  "model": "llama3.2",
  "messages": [{"role": "user", "content": "Hello"}]
}'

Configure the app

In your app, you only change the API base address and the key:

OPENAI_API_BASE=http://localhost:11434/v1
OPENAI_API_KEY=ollama

Code that used OpenAI now works with your local model.

Alternatives

For more features (images, embeddings, multiple models), projects like LocalAI offer a more complete OpenAI-compatible API, also running on your server.

Why it is worth it

• Fixed cost instead of per-token billing.
• Privacy: data does not leave the server.
• No externally imposed rate limits.

For large models and speed, see the GPU plans.