Calling the Ollama API from Your Own Applications

Ollama exposes an HTTP API on port 11434, and everything you build on top of it goes through that one service. The request shapes differ per endpoint: /api/generate is stateless, /api/chat carries the conversation, and the OpenAI-compatible routes under /v1 expect a different payload again. This tutorial covers the calls you need in application code: streaming, conversation state, JSON schema output, tool calling and model residency.

What is the Ollama API?

The Ollama API is a REST interface served by the ollama daemon on http://localhost:11434, which accepts JSON payloads on endpoints such as /api/generate, /api/chat and /api/embed and returns either NDJSON streams or single JSON objects.

The API has no authentication layer of its own. Anything that can reach the port can load models and run inference, which matters as soon as you move the daemon off your workstation.

Prerequisites

• Ollama 0.5.0 or newer (ollama --version); structured outputs and the /api/embed endpoint are not available in older builds
• At least one pulled model, for example ollama pull llama3.2
• curl, plus Python 3.10+ or Node.js 18+ for the client examples
• A host with enough VRAM for the model you intend to serve. For 7B models and larger, run the daemon on a GPU instance for LLM inference rather than on CPU

Check that the API is reachable

Before writing client code, confirm the daemon answers and knows the model you plan to call.

$ curl http://localhost:11434/api/version
$ curl -s http://localhost:11434/api/tags | jq '.models[].name'

The first command returns {"version":"0.5.x"}. The second lists model names such as llama3.2:latest. If /api/tags is empty, pull a model first, because the API does not download models implicitly on a chat request.

Which endpoint should you use: /api/generate or /api/chat?

Use /api/generate for single-shot completions such as summarization or classification, and use /api/chat whenever the model needs the previous turns, a system prompt or tool definitions.

/api/generate takes a flat prompt string and can return a context array for simple continuation. /api/chat takes a messages array with system, user, assistant and tool roles, and it is the only endpoint that supports the tools field. New integrations should default to /api/chat.

Send your first request

Start with a non-streaming request. A single JSON object is easier to inspect than an NDJSON stream.

$ curl http://localhost:11434/api/chat -d '{
  "model": "llama3.2",
  "messages": [{"role": "user", "content": "Explain what an inode is in two sentences."}],
  "stream": false,
  "options": {"temperature": 0.2, "num_ctx": 4096}
}'

The answer sits in message.content. The response also carries total_duration, load_duration, prompt_eval_count and eval_count, all in nanoseconds or tokens. Log eval_count divided by eval_duration to track tokens per second per host.

The options object holds the runtime parameters: temperature, top_p, seed for reproducible sampling, and num_ctx for the context window. Values you do not set fall back to the model's Modelfile defaults.

Stream tokens instead of waiting

With "stream": true, which is the default when you omit the field, Ollama returns newline-delimited JSON where each line is a complete JSON object carrying one content fragment.

graph TD
  A["Your application"] -->|"POST /api/chat"| B["Ollama daemon on :11434"]
  B --> C{"Model already resident?"}
  C -->|"No"| D["Load weights into VRAM"]
  C -->|"Yes"| E["Run inference"]
  D --> E
  E --> F{"stream: true?"}
  F -->|"Yes"| G["NDJSON: one JSON object per line"]
  F -->|"No"| H["Single JSON object"]
  G --> I["Final line with done: true and timings"]
  H --> I

Parse each line separately. Do not buffer the whole body and call json.loads once, because the payload is not a JSON array.

import json
import requests
PAYLOAD = {
    "model": "llama3.2",
    "messages": [{"role": "user", "content": "List three uses for /proc."}],
    "stream": True,
}
with requests.post("http://localhost:11434/api/chat", json=PAYLOAD, stream=True, timeout=120) as r:
    r.raise_for_status()
    for line in r.iter_lines():
        if not line:
            continue
        chunk = json.loads(line)
        if chunk.get("done"):
            print()
            break
        print(chunk["message"]["content"], end="", flush=True)

The same request from Node.js without streaming looks like this:

const res = await fetch("http://localhost:11434/api/chat", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    model: "llama3.2",
    messages: [{ role: "user", content: "Explain the TCP backlog." }],
    stream: false
  })
});
const data = await res.json();
console.log(data.message.content);

Set a generous client timeout. A cold model load on a large quantization can take 30 seconds or more before the first token arrives.

How do you keep conversation context?

The Ollama server keeps no conversation state, so your application must resend the complete messages array on every request, including the previous assistant replies you want the model to remember.

import requests
URL = "http://localhost:11434/api/chat"
history = [{"role": "system", "content": "You answer as a Linux administrator. Be terse."}]
def ask(question):
    history.append({"role": "user", "content": question})
    r = requests.post(URL, json={"model": "llama3.2", "messages": history, "stream": False}, timeout=120)
    r.raise_for_status()
    reply = r.json()["message"]
    history.append(reply)
    return reply["content"]
print(ask("Which command shows open sockets?"))
print(ask("Add the process name to that command."))

The history grows with every turn and is limited by num_ctx. Once the token count approaches the context window, drop the oldest user and assistant pairs while keeping the system message, or summarize them into a single message.

Force structured JSON output

Pass a JSON Schema in the format field to make the model emit machine-readable output. This works on /api/chat and /api/generate since Ollama 0.5.0. The older "format": "json" value still works, but it only guarantees valid JSON, not a specific shape.

$ curl http://localhost:11434/api/chat -d '{
  "model": "llama3.2",
  "messages": [{"role": "user", "content": "Parse: web01 is down since 14:05, severity high"}],
  "stream": false,
  "format": {
    "type": "object",
    "properties": {
      "host": {"type": "string"},
      "since": {"type": "string"},
      "severity": {"type": "string", "enum": ["low", "medium", "high"]}
    },
    "required": ["host", "since", "severity"]
  }
}'

message.content is now a JSON string matching the schema, so your client parses it with a second json.loads. Keep temperature low for extraction tasks and state the expected fields in the prompt as well; the schema constrains the grammar, not the semantics.

Call your own functions with tool calling

Declare functions in the tools array of /api/chat. The model does not execute anything. It returns a tool_calls entry, your code runs the function and sends the result back as a message with the role tool.

{
  "model": "llama3.1",
  "stream": false,
  "messages": [{"role": "user", "content": "What is the disk usage on web01?"}],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_disk_usage",
        "description": "Return disk usage in percent for a host",
        "parameters": {
          "type": "object",
          "properties": {"host": {"type": "string"}},
          "required": ["host"]
        }
      }
    }
  ]
}

The reply contains message.tool_calls[0].function.name and a parsed arguments object. Append the assistant message, then append {"role": "tool", "content": "71"} and call /api/chat again so the model can phrase the final answer. Tool calling requires a model trained for it, such as llama3.1, qwen2.5 or mistral-nemo. Models without tool support ignore the tools field and answer in prose.

Generate embeddings for retrieval

Use POST /api/embed with an embedding model to turn text into vectors. The input field accepts a single string or an array, which is the cheaper option for batch indexing.

$ curl http://localhost:11434/api/embed -d '{
  "model": "nomic-embed-text",
  "input": ["systemd unit files live in /etc/systemd/system", "journalctl reads the binary journal"]
}'

The response holds embeddings as an array of float arrays, one per input, with 768 dimensions for nomic-embed-text. Embedding models are small enough to run acceptably without a GPU, so an indexing worker and its vector database fit on a scalable Linux VM while the chat model stays on GPU hardware. Keep the generation and embedding workloads on separate daemons if both are under load, because a single Ollama instance serializes requests per loaded model by default.

Use the OpenAI-compatible endpoint

Ollama serves /v1/chat/completions, /v1/embeddings and /v1/models in the OpenAI request format, so existing SDK code works after changing the base URL.

from openai import OpenAI
client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
resp = client.chat.completions.create(
    model="llama3.2",
    messages=[{"role": "user", "content": "One sentence on epoll."}],
)
print(resp.choices[0].message.content)

The api_key value is required by the SDK but ignored by Ollama. Use this route for compatibility with existing tooling. Use the native /api/chat route when you need keep_alive, num_ctx or the raw timing fields, which the compatibility layer does not expose.

Why is the first request slow?

The first request after an idle period is slow because Ollama unloads model weights from memory after five minutes of inactivity and has to reload them, which shows up as a large load_duration in the response.

Control this with the keep_alive field per request or the OLLAMA_KEEP_ALIVE environment variable for the daemon. A value of -1 keeps the model resident indefinitely, 0 unloads it immediately after the request.

$ curl http://localhost:11434/api/chat -d '{"model":"llama3.2","messages":[],"keep_alive":-1}'
$ curl -s http://localhost:11434/api/ps | jq '.models[] | {name, size_vram, expires_at}'

An empty messages array loads the model without running inference, which is the standard warm-up call after a deployment. /api/ps then shows the resident models and their VRAM footprint.

Expose the API beyond localhost

The daemon binds to 127.0.0.1:11434 by default. To accept requests from other hosts, set OLLAMA_HOST in a systemd override.

$ sudo systemctl edit ollama.service

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"
Environment="OLLAMA_KEEP_ALIVE=30m"

$ sudo systemctl daemon-reload
$ sudo systemctl restart ollama

The port is now unauthenticated on every interface. Restrict it with a firewall rule to the application subnet and put a reverse proxy in front that terminates TLS and checks an API key. For browser clients, set OLLAMA_ORIGINS to the allowed origins, otherwise the preflight request fails.

Verification

Run one scripted request end to end and check the fields your application depends on.

$ curl -s http://localhost:11434/api/chat -d '{
  "model": "llama3.2",
  "messages": [{"role": "user", "content": "Reply with the single word: ok"}],
  "stream": false
}' | jq '{content: .message.content, done_reason, eval_count}'

Expected output resembles the following:

{
  "content": "ok",
  "done_reason": "stop",
  "eval_count": 3
}

A done_reason of length means the model hit the token limit, not the end of its answer. Raise num_predict in options in that case.

Troubleshooting

• curl: (7) Failed to connect to ... port 11434: the daemon is not running or still bound to localhost. Check with systemctl status ollama and ss -tlnp | grep 11434.
• {"error":"model 'llama3.2' not found, try pulling it first"}: the API does not download models on demand. Run ollama pull llama3.2 on the host serving the API, not on the client.
• Stream parsing crashes with Extra data: the body is NDJSON, not a JSON array. Parse line by line, as in the streaming example above.
• CORS errors in the browser: set OLLAMA_ORIGINS="https://<your-domain>" in the systemd override and restart the service.

Wrap-up

For application code, /api/chat with an explicit messages array is the endpoint to build on: it covers system prompts, tool calls and JSON schema output in one request shape. Resend the full history yourself, keep keep_alive aligned with your traffic pattern, and never expose port 11434 without a proxy that authenticates requests.

Read next

• Connecting Ollama to MCP Servers: Tools for Local Models
• Expose Ollama on the Network: Port, Bind Address and Hardening
• Install Ollama on Ubuntu 24.04 and Run It as a Service
• Use a Local Coding Model with Ollama in Your IDE
• Managing Ollama Models: Updates, Cleanup and Disk Space
• Ollama vs LM Studio vs llama.cpp: Which Tool for What?
• Run Ollama in a Docker Container
• Set Up Open WebUI as a Front End for Ollama
• What Is Ollama? How It Works, Models and Use Cases