Install Ollama on Ubuntu 24.04 and Run It as a Service

Ollama serves local language models through a single HTTP API on port 11434. The official install script already ships a systemd unit, but its defaults bind to localhost only and keep every model under /usr/share/ollama, which is rarely the right place on a server with a separate data volume. This tutorial covers the installation on Ubuntu 24.04, the service configuration through a drop-in file, and the verification steps.

What is Ollama?

Ollama is a self-hosted runtime that downloads and serves open-weight language models such as Llama 3.2, Mistral or Qwen behind an HTTP API on 127.0.0.1:11434, including an OpenAI-compatible endpoint under /v1.

Models are distributed as quantized GGUF blobs and addressed by name and tag, for example llama3.2:3b. The server loads a model into RAM or VRAM on the first request, keeps it resident for five minutes by default, and unloads it afterwards. That behaviour is configurable, and it is the main reason the service should run as a long-lived daemon rather than being started per session.

Prerequisites

• Ubuntu 24.04 LTS (Server or Desktop), architecture x86_64 or arm64
• A user account with sudo rights
• Outbound HTTPS access to ollama.com and registry.ollama.ai
• At least 8 GB RAM for 7B/8B models in 4-bit quantization, 16 GB for comfortable headroom
• Roughly 5 GB free disk space per model
• Optional: an NVIDIA GPU with a working driver, verified through nvidia-smi

CPU-only inference works, it is simply slower. A 3B model on four modern vCPUs produces a few tokens per second, which is enough for testing and for low-volume internal tools. If you want to size the machine to the model instead of the other way round, run the setup on a scalable Cloud-VPS where vCPU and RAM can be adjusted later.

Install Ollama on Ubuntu 24.04

Update the package index and install the two packages the installer needs:

$ sudo apt update
$ sudo apt install -y curl ca-certificates

Run the official install script:

$ curl -fsSL https://ollama.com/install.sh | sh

The script performs four actions:

1. Downloads the release tarball for your architecture and unpacks the binary to /usr/local/bin/ollama.
2. Creates the system user and group ollama with the home directory /usr/share/ollama.
3. Writes the unit file /etc/systemd/system/ollama.service.
4. Runs systemctl enable --now ollama.

If your policy forbids piping a remote script into a shell, install manually. Inspect the tarball first, then extract it:

$ curl -fsSL -o /tmp/ollama-linux-amd64.tgz https://ollama.com/download/ollama-linux-amd64.tgz
$ sudo tar -C /usr -xzf /tmp/ollama-linux-amd64.tgz
$ sudo useradd -r -s /bin/false -U -m -d /usr/share/ollama ollama

With the manual route you write the unit file yourself, using the same content shown in the next section. For AMD GPUs, add the ROCm payload from ollama-linux-amd64-rocm.tgz on top of the base tarball.

Confirm the binary is in place:

$ ollama --version
ollama version is 0.12.3

How does the Ollama systemd service work?

The Ollama systemd service runs /usr/local/bin/ollama serve as the unprivileged system user ollama, listens on 127.0.0.1:11434 by default, and loads model weights from /usr/share/ollama/.ollama/models on demand.

The CLI is a thin client. When you type ollama run, the command talks to the same HTTP API that any other client would use, so a running service and an interactive session share one loaded model instead of allocating memory twice.

graph TD
    A["ollama CLI"] --> C["HTTP API 127.0.0.1:11434"]
    B["Application / curl / OpenAI SDK"] --> C
    C --> D["systemd unit ollama.service"]
    D --> E["ollama serve process, user ollama"]
    E --> F["Model store /usr/share/ollama/.ollama/models"]
    E --> G["CPU or GPU inference"]

This is the unit the installer writes:

[Unit]
Description=Ollama Service
After=network-online.target
[Service]
ExecStart=/usr/local/bin/ollama serve
User=ollama
Group=ollama
Restart=always
RestartSec=3
Environment="PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
[Install]
WantedBy=multi-user.target

Do not edit this file directly. The next installer run overwrites it.

Configure the service with a drop-in

Use systemctl edit to create an override that survives upgrades:

$ sudo systemctl edit ollama.service

Add the following block between the marker comments. It moves the model store to /var/lib/ollama/models, keeps models loaded for 30 minutes and allows two concurrent requests per model:

[Service]
Environment="OLLAMA_MODELS=/var/lib/ollama/models"
Environment="OLLAMA_KEEP_ALIVE=30m"
Environment="OLLAMA_NUM_PARALLEL=2"
Environment="OLLAMA_MAX_LOADED_MODELS=1"

The most relevant variables:

Create the target directory with the correct ownership, then reload and restart:

$ sudo mkdir -p /var/lib/ollama/models
$ sudo chown -R ollama:ollama /var/lib/ollama
$ sudo systemctl daemon-reload
$ sudo systemctl restart ollama

Exposing the API to other hosts

Setting OLLAMA_HOST=0.0.0.0:11434 makes the API reachable from the network. Ollama has no authentication of its own, so anyone who can reach the port can run inference, pull models and consume disk space. Restrict access at the firewall and terminate TLS plus authentication in front of it:

$ sudo ufw allow from 10.0.0.0/24 to any port 11434 proto tcp

For anything reachable from the public internet, put a reverse proxy such as nginx or Caddy in front with HTTP basic auth or an API gateway, and leave Ollama itself bound to localhost.

Pull and run a model

Download a small model first to confirm the whole path works:

$ ollama pull llama3.2:3b

Start an interactive session:

$ ollama run llama3.2:3b
>>> Summarize what systemd drop-in files do in two sentences.

Leave the session with /bye. The model stays loaded for the duration of OLLAMA_KEEP_ALIVE, so the next request answers without a reload. List and remove models with:

$ ollama list
$ ollama rm llama3.2:3b

Model files are large and grow quickly across a fleet of machines. Track the size of /var/lib/ollama/models in your monitoring alongside the usual disk metrics, and set a threshold well below full, because a failed pull on a full filesystem leaves partial blobs behind.

Verify the installation

Check the service state and the effective environment:

$ systemctl is-active ollama
active
$ systemctl show ollama --property=Environment

Query the list of installed models over the API:

$ curl -s http://127.0.0.1:11434/api/tags | jq '.models[].name'
"llama3.2:3b"

Send a single non-streaming completion:

$ curl -s http://127.0.0.1:11434/api/generate -d '{"model":"llama3.2:3b","prompt":"Reply with the word OK.","stream":false}' | jq -r '.response'
OK

The OpenAI-compatible endpoint answers under /v1 and accepts any dummy API key:

$ curl -s http://127.0.0.1:11434/v1/chat/completions -H "Content-Type: application/json" -d '{"model":"llama3.2:3b","messages":[{"role":"user","content":"ping"}]}' | jq -r '.choices[0].message.content'

If all four commands return output, the service is working and reachable.

Troubleshooting

Service restarts in a loop

Read the journal first:

$ journalctl -u ollama -n 50 --no-pager

listen tcp 127.0.0.1:11434: bind: address already in use means a second instance is running, usually one started manually with ollama serve in a terminal. Find and stop it:

$ sudo ss -lntp | grep 11434

Permission denied on the model directory

After changing OLLAMA_MODELS, the new path must belong to the ollama user. A log line containing permission denied on a path under /var/lib/ollama points to a missing chown. Re-apply ownership and restart the unit.

GPU is not used

The startup log states which backend was selected. Search for it:

$ journalctl -u ollama | grep -i "inference compute"

If no GPU is listed, verify the driver with nvidia-smi. On a fresh Ubuntu 24.04 install, sudo ubuntu-drivers install pulls a matching NVIDIA driver. A reboot is required afterwards, and Ollama must be restarted so it re-detects the device.

Next steps

The installation is complete once systemctl is-active ollama reports active and /api/generate returns a response. Before putting the host to work, decide who may reach port 11434 and keep the API behind a firewall or an authenticating proxy. Then size the model to the hardware: a 3B model in 4-bit quantization fits comfortably in 8 GB RAM, while an 8B model at longer context lengths wants 16 GB or a GPU.

Read next

• Calling the Ollama API from Your Own Applications
• Connecting Ollama to MCP Servers: Tools for Local Models
• Expose Ollama on the Network: Port, Bind Address and Hardening
• Use a Local Coding Model with Ollama in Your IDE
• Managing Ollama Models: Updates, Cleanup and Disk Space
• Ollama vs LM Studio vs llama.cpp: Which Tool for What?
• Run Ollama in a Docker Container
• Set Up Open WebUI as a Front End for Ollama
• What Is Ollama? How It Works, Models and Use Cases