Connecting Ollama to MCP Servers: Tools for Local Models

Ollama runs models locally and supports tool calling, but it does not speak the Model Context Protocol. To use MCP servers such as the filesystem, fetch or Git servers with a local model, you need a client in between that discovers the tools and translates them into Ollama's tool format. This tutorial builds that bridge in about 60 lines of Python and verifies it against a real MCP server.

What is MCP and why does Ollama need a bridge?

The Model Context Protocol (MCP) is an open JSON-RPC protocol in which a server exposes tools, resources and prompts over stdio or streamable HTTP, and a host application discovers them at runtime and calls them on behalf of a model. Ollama implements the model side of tool calling through /api/chat, but it has no MCP client built in, so nothing in Ollama starts an MCP server or reads its tool list.

That gap is what the bridge fills. It performs three jobs: it starts or connects to the MCP server, converts each MCP tool definition into the function schema Ollama expects, and runs the loop that feeds tool results back into the conversation until the model produces a final answer.

Prerequisites

• A Linux host with Ollama 0.3.0 or newer. Tool calling was added in 0.3.0; the examples here were checked against 0.12.
• Python 3.10 or newer, which is the minimum for the MCP Python SDK.
• Node.js 18 or newer, only for the example MCP server started through npx.
• At least 8 GB of free RAM for an 8B model in Q4 quantisation. A scalable Linux VM with dedicated CPU cores is enough for the walkthrough.

Check the base installation first:

$ ollama --version
$ curl -s http://127.0.0.1:11434/api/tags | head -n 5

If curl fails, start the daemon with systemctl start ollama or run ollama serve in a separate shell.

Which Ollama models support tool calling?

Only Ollama models whose Modelfile template declares the tools capability can emit structured tool calls; this includes the Llama 3.1 and 3.2 families, Qwen 2.5 and Qwen 3, Mistral Nemo and Command R. Everything else will describe a tool in prose instead of calling it.

Verify a model before wiring it in:

$ ollama pull qwen3:8b
$ ollama show qwen3:8b

The output contains a Capabilities block. It must list tools:

  Capabilities
    completion
    tools
    thinking

Smaller models accept tool schemas but pick the wrong tool or invent argument names as soon as a server exposes more than a handful of tools. Anything below 7B parameters is worth testing carefully before you rely on it.

Install the MCP SDK and a test server

Create a virtual environment and install the two client libraries:

$ python3 -m venv /opt/ollama-mcp/venv
$ /opt/ollama-mcp/venv/bin/pip install mcp ollama

Create a directory the MCP filesystem server is allowed to touch, and put a file in it:

$ sudo mkdir -p /srv/mcp-data
$ sudo chown $USER /srv/mcp-data
$ printf 'db_host=10.0.0.5\ndb_port=5432\n' > /srv/mcp-data/service.conf

Confirm the server itself starts. It stays in the foreground and waits for JSON-RPC on stdin:

$ npx -y @modelcontextprotocol/server-filesystem /srv/mcp-data
Secure MCP Filesystem Server running on stdio

Stop it with Ctrl+C. The bridge will start it as a subprocess from now on.

Write the bridge

The script below starts the MCP server over stdio, lists its tools, converts them and runs the chat loop. Save it as /opt/ollama-mcp/bridge.py.

import asyncio
import json
import sys
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client
from ollama import AsyncClient
MODEL = "qwen3:8b"
MAX_ROUNDS = 5
SERVER = StdioServerParameters(
    command="npx",
    args=["-y", "@modelcontextprotocol/server-filesystem", "/srv/mcp-data"],
)
def to_ollama_tools(mcp_tools):
    """Convert MCP tool definitions into Ollama function definitions."""
    return [
        {
            "type": "function",
            "function": {
                "name": tool.name,
                "description": (tool.description or "")[:1024],
                "parameters": tool.inputSchema,
            },
        }
        for tool in mcp_tools
    ]
async def main(prompt):
    async with stdio_client(SERVER) as (read, write):
        async with ClientSession(read, write) as session:
            await session.initialize()
            listed = await session.list_tools()
            tools = to_ollama_tools(listed.tools)
            print(f"[bridge] {len(tools)} MCP tools: {[t.name for t in listed.tools]}")
            client = AsyncClient(host="http://127.0.0.1:11434")
            messages = [{"role": "user", "content": prompt}]
            for _ in range(MAX_ROUNDS):
                reply = await client.chat(
                    model=MODEL,
                    messages=messages,
                    tools=tools,
                    options={"num_ctx": 16384, "temperature": 0},
                )
                messages.append(reply.message)
                if not reply.message.tool_calls:
                    print(reply.message.content)
                    return
                for call in reply.message.tool_calls:
                    name = call.function.name
                    args = call.function.arguments
                    print(f"[bridge] -> {name}({json.dumps(args)})")
                    result = await session.call_tool(name, args)
                    text = "\n".join(
                        block.text for block in result.content if block.type == "text"
                    )
                    messages.append({
                        "role": "tool",
                        "tool_name": name,
                        "content": text or "(empty result)",
                    })
            print("[bridge] round limit reached without a final answer")
if __name__ == "__main__":
    question = " ".join(sys.argv[1:]) or "Read service.conf and tell me the database port."
    asyncio.run(main(question))

Three details matter here:

• tool.inputSchema is already a JSON Schema object, so it can be passed to Ollama unchanged in most cases. Servers that use $ref or $defs need flattening, see Troubleshooting.
• tool_name in the tool result message is supported from Ollama 0.6 onward and helps the model match a result to its call. On older versions, drop the key and keep only role and content.
• num_ctx is raised because tool schemas plus results easily exceed the 4096-token default context, which silently truncates the conversation.

How does the tool-call loop work?

The tool-call loop is a request cycle in which the bridge sends the tool list with every chat request, executes any tool_calls the model returns against the MCP server, appends each result as a message with role tool, and repeats until the model answers without requesting a tool.

sequenceDiagram
    participant U as User
    participant B as Bridge
    participant M as MCP Server
    participant O as Ollama
    B->>M: initialize + tools/list
    M-->>B: Tool schemas as JSON Schema
    U->>B: Prompt
    B->>O: POST /api/chat with tools
    O-->>B: message.tool_calls
    B->>M: tools/call with arguments
    M-->>B: Result content
    B->>O: POST /api/chat with tool result
    O-->>B: Final answer
    B-->>U: Answer

The round limit is not optional. A model that misreads a tool result can request the same tool indefinitely, and without MAX_ROUNDS the bridge would keep paying for inference on every iteration.

Run and verify

Run the bridge with a prompt that cannot be answered without filesystem access:

$ /opt/ollama-mcp/venv/bin/python /opt/ollama-mcp/bridge.py \
    "Read service.conf in the allowed directory and report the database port."

Expected output, shortened:

[bridge] 12 MCP tools: ['read_file', 'read_multiple_files', 'write_file', ...]
[bridge] -> read_file({"path": "/srv/mcp-data/service.conf"})
The database port configured in service.conf is 5432.

Two things confirm the setup works: the [bridge] -> line proves the model emitted a structured tool call rather than prose, and the final answer contains the value 5432, which only exists inside the file. If the answer is correct but no tool line appears, the model guessed and the wiring is not actually being used.

Generation speed on CPU is the usual bottleneck once a server exposes 20 or more tools, because the schemas are re-sent with every request in the loop. If the loop feels slow, move inference to GPU instances for local LLM workloads and keep the bridge and MCP servers where the data is.

Connect a remote MCP server over HTTP

Stdio only works for servers on the same host. For a server on another machine, swap the transport and leave the rest of the script untouched:

from mcp.client.streamable_http import streamablehttp_client
async def main(prompt):
    url = "https://mcp.internal.example.com/mcp"
    headers = {"Authorization": "Bearer YOUR_TOKEN"}
    async with streamablehttp_client(url, headers=headers) as (read, write, _):
        async with ClientSession(read, write) as session:
            await session.initialize()
            ...

The streamable HTTP transport replaced the older HTTP+SSE transport in the March 2025 MCP specification. Servers that still expose only SSE need mcp.client.sse.sse_client instead. Remote MCP servers have no sandbox by default, so restrict them at the network layer and never expose a filesystem or shell server without authentication.

Troubleshooting

The model describes a tool instead of calling it. The model has no tools capability, or the tool list was not passed. Check ollama show <model> and confirm the tools= argument reaches client.chat.

The bridge hangs at session.initialize(). An MCP server that writes log output to stdout corrupts the JSON-RPC stream. Well-behaved servers log to stderr. Also verify the command exists on PATH: a missing npx produces a hang rather than a clear error in some SDK versions.

Tool calls fail with a schema validation error. Some MCP servers publish schemas using $defs and $ref, which many local models cannot fill correctly. Inline the referenced definitions in to_ollama_tools, or filter the tool list down to the tools you actually need.

Answers ignore the tool result. The context window is too small for the schemas plus results. Raise num_ctx to 16384 or higher, or reduce the number of exposed tools.

Wrap-up

With around 60 lines of Python, an Ollama model reaches every tool an MCP server exposes: filesystem, Git, HTTP fetch, databases, or your own server. The pattern stays the same regardless of the server. Discover tools, translate the schema, loop on tool_calls.

For production use, add per-tool allowlists, a timeout around session.call_tool and structured logging of every call and its arguments. Those three additions turn the prototype into something you can audit after the fact.

Read next

• Calling the Ollama API from Your Own Applications
• Expose Ollama on the Network: Port, Bind Address and Hardening
• Install Ollama on Ubuntu 24.04 and Run It as a Service
• Use a Local Coding Model with Ollama in Your IDE
• Managing Ollama Models: Updates, Cleanup and Disk Space
• Ollama vs LM Studio vs llama.cpp: Which Tool for What?
• Run Ollama in a Docker Container
• Set Up Open WebUI as a Front End for Ollama
• What Is Ollama? How It Works, Models and Use Cases