Ollama vs LM Studio vs llama.cpp: Which Tool for What?

Three tools dominate local LLM inference and are routinely presented as interchangeable. They are not: llama.cpp is the inference engine, Ollama is a daemon and model manager built on top of one, and LM Studio is a desktop application. This tutorial installs all three, runs a quantized GGUF model on each, and states which one belongs in which setup.

What is the difference between Ollama, LM Studio and llama.cpp?

llama.cpp is the C/C++ inference engine that executes GGUF model files, Ollama is a background service that wraps an inference engine with model pulling and lifecycle management, and LM Studio is a closed-source desktop application that bundles llama.cpp behind a chat GUI and a local OpenAI-compatible server.

The relationship matters as soon as you debug performance. LM Studio ships llama.cpp as its backend, plus MLX on Apple Silicon. Ollama started as a llama.cpp wrapper and still builds on the ggml tensor library, with its own engine handling newer multimodal model families. A token-throughput problem is therefore almost always a ggml problem, no matter which of the three you launched.

Prerequisites

• A Linux host with at least 16 GB RAM for 7B and 8B models at 4-bit quantization
• 20 GB free disk space for model files and build artifacts
• A user account with sudo rights
• Optional for GPU inference: an NVIDIA GPU with driver 550 or newer and CUDA Toolkit 12.x

CPU-only inference works for all three tools, but expect single-digit tokens per second on a 7B model. A scalable Linux cloud VM is enough to evaluate the tooling and the API surface before you commit to accelerator hardware.

Size the machine by the quantized weights plus KV cache:

Install and run Ollama

Install Ollama with the official script. It adds a systemd unit named ollama.service and a system user, and it detects an NVIDIA or AMD GPU during setup.

$ curl -fsSL https://ollama.com/install.sh | sh
$ systemctl status ollama --no-pager

Pull and run a model. Tags encode both parameter count and quantization, so llama3.1:8b resolves to the default Q4_K_M build:

$ ollama pull llama3.1:8b
$ ollama run llama3.1:8b "Summarize the CAP theorem in two sentences."
$ ollama ps

The PROCESSOR column in ollama ps reports the CPU/GPU split for the loaded model. A value of 100% GPU means the whole model is offloaded.

Expose and tune the API

Ollama binds to 127.0.0.1:11434 by default. Change the binding and the memory behaviour through a systemd drop-in rather than by editing the shipped unit:

$ sudo systemctl edit ollama.service

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"
Environment="OLLAMA_KEEP_ALIVE=30m"
Environment="OLLAMA_NUM_PARALLEL=2"
Environment="OLLAMA_CONTEXT_LENGTH=8192"

Apply the change with sudo systemctl daemon-reload && sudo systemctl restart ollama. Ollama has no authentication of its own, so anything beyond 127.0.0.1 belongs behind a reverse proxy with TLS and an access token.

The daemon serves both its native REST API and an OpenAI-compatible route:

$ curl http://localhost:11434/api/generate -d '{"model":"llama3.1:8b","prompt":"ping","stream":false}'
$ curl http://localhost:11434/v1/chat/completions -H "Content-Type: application/json" -d '{"model":"llama3.1:8b","messages":[{"role":"user","content":"ping"}]}'

Build and run llama.cpp

llama.cpp gives you the engine without the abstraction layer. Build it when you need specific compile flags, sampler parameters that no wrapper exposes, or upstream support for a model architecture released last week.

$ sudo apt update && sudo apt install -y build-essential cmake git libcurl4-openssl-dev
$ git clone https://github.com/ggml-org/llama.cpp.git /opt/llama.cpp
$ cmake -S /opt/llama.cpp -B /opt/llama.cpp/build -DGGML_CUDA=ON
$ cmake --build /opt/llama.cpp/build --config Release -j $(nproc)

Drop -DGGML_CUDA=ON for a CPU-only build. The flag was called LLAMA_CUBLAS before mid-2024, and the binaries were renamed from main/server to llama-cli/llama-server in the same period. Guides that still use the old names predate that change.

Compiling CUDA kernels for a specific architecture and running a 70B model at usable speed is where dedicated hardware pays off. A GPU instance for CUDA and LLM inference workloads removes the driver and toolkit setup from the equation, and you can pin -DCMAKE_CUDA_ARCHITECTURES to the exact compute capability of the card instead of building fat binaries.

Start the server. The -hf flag downloads a GGUF file straight from Hugging Face into the local cache:

$ /opt/llama.cpp/build/bin/llama-server -hf ggml-org/gemma-3-4b-it-GGUF -ngl 99 -c 8192 --host 127.0.0.1 --port 8080

The relevant flags:

• -m /path/to/model.gguf loads a local file instead of pulling from Hugging Face.
• -ngl 99 offloads all layers to the GPU. Lower the number to split layers between GPU and CPU when VRAM is short.
• -c 8192 sets the context window in tokens. Larger contexts increase KV cache memory linearly.
• --host/--port control the binding. llama-server also has no built-in authentication.

llama-server exposes /v1/chat/completions and a browser UI on the same port.

What is LM Studio best used for?

LM Studio is best used on a workstation where a graphical model browser, side-by-side prompt comparison and one-click GPU offload matter more than scriptable deployment, since it is a proprietary desktop application rather than a service you provision.

It is distributed as a .dmg, .exe or Linux AppImage for x86_64. After the first launch you can bootstrap its CLI and drive it without the window:

$ ~/.lmstudio/bin/lms bootstrap
$ lms ls
$ lms load qwen2.5-7b-instruct --gpu max
$ lms server start --port 1234

The strengths are concrete: the model catalogue flags whether a given quantization fits your available VRAM before you download it, the parameter panel exposes temperature, top-k and repeat penalty without a config file, and the runtime picker lets you switch between llama.cpp builds. The limits are equally concrete: no MIT-style source access, x86_64 only on Linux, and headless operation still requires the desktop application to be installed on the machine.

Which tool should you choose?

Map the decision to the deployment target rather than to feature lists.

graph TD
  A["Run an LLM locally"] --> B{"Desktop or headless server?"}
  B -->|"Desktop workstation"| C["LM Studio"]
  B -->|"Headless Linux server"| D{"Need custom build flags or new architectures?"}
  D -->|"No"| E["Ollama daemon"]
  D -->|"Yes"| F["llama.cpp llama-server"]
  C --> G["OpenAI API on port 1234"]
  E --> H["REST + OpenAI API on port 11434"]
  F --> I["OpenAI API on port 8080"]

Three rules of thumb hold up in practice:

1. Evaluating models interactively on a laptop: LM Studio. Downloading, swapping and comparing quantizations is faster in the GUI than in a shell.
2. Serving an internal API that other services call: Ollama. Model pulls, unit files, keep-alive and concurrent slots are handled for you.
3. Squeezing out throughput or running a model released this week: llama.cpp. You get every flag, and you can rebuild against the current master.

A mixed setup is normal. Prototype prompts in LM Studio, then deploy the same GGUF file behind llama-server or Ollama.

Verify the setup

Each tool answers on the OpenAI-compatible route, so one request shape verifies all of them. Adjust port and model name:

$ curl -s http://localhost:11434/v1/chat/completions -H "Content-Type: application/json" -d '{"model":"llama3.1:8b","messages":[{"role":"user","content":"Reply with the single word OK"}]}' | jq -r '.choices[0].message.content'

Expected output:

OK

For llama.cpp use port 8080, for LM Studio port 1234. If jq returns null, print the raw response: the body usually contains an explicit error such as model not found.

Troubleshooting

Inference runs on CPU although a GPU is present. Check ollama ps for the PROCESSOR column, or the llama-server startup log for a line reporting offloaded layers. Confirm the driver with nvidia-smi, then check journalctl -u ollama -n 50 for a message about insufficient VRAM. A model that does not fit gets split automatically, which collapses throughput.

The llama.cpp build fails with No CMAKE_CUDA_COMPILER could be found. The CUDA Toolkit is missing or nvcc is not on PATH. Install the toolkit and re-run CMake with -DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc. Delete the build directory before retrying, because CMake caches the failed detection.

The Ollama API is unreachable from another host. The daemon still listens on 127.0.0.1. Verify with ss -tlnp | grep 11434 and set OLLAMA_HOST=0.0.0.0:11434 through the systemd drop-in shown above. Restrict access at the firewall or reverse proxy in the same step.

Wrap-up

The three tools sit at different layers: llama.cpp executes the model, Ollama operates it, and LM Studio makes it clickable. Pick by deployment target, keep the GGUF files portable between them, and remember that neither ollama serve nor llama-server authenticates requests on its own. Next step: put a reverse proxy with TLS and token auth in front of whichever endpoint you expose.

Read next

• Calling the Ollama API from Your Own Applications
• Connecting Ollama to MCP Servers: Tools for Local Models
• Expose Ollama on the Network: Port, Bind Address and Hardening
• Install Ollama on Ubuntu 24.04 and Run It as a Service
• Use a Local Coding Model with Ollama in Your IDE
• Managing Ollama Models: Updates, Cleanup and Disk Space
• Run Ollama in a Docker Container
• Set Up Open WebUI as a Front End for Ollama
• What Is Ollama? How It Works, Models and Use Cases