πŸ€– AI Tools
Β· 3 min read
Last updated on

Ollama Setup: A Local Model, API and Safe Operating Boundary


Ollama runs supported models behind a local API. A working setup needs three things to agree: the server you are reaching, the exact model installed on that server, and enough memory for the request. Start with those checks before adding an editor, web interface or retrieval pipeline.

This guide covers that first working boundary. The local inference hub maps the later decisions. These are documentation-based configuration instructions, not hardware benchmarks or a claim that we tested every operating system.

Install once, identify the running server

Use the installer for your operating system from Ollama’s download page. On Windows, follow the native Windows setup; on a container host, use the Docker deployment. Do not run a Linux installation script on macOS.

The desktop application or an existing service may already own the API port. Check it before starting another server:

ollama --version
curl --fail http://127.0.0.1:11434/api/version
curl --fail http://127.0.0.1:11434/api/tags

The CLI version and API version describe different processes. If they disagree, investigate client/server version mismatch. Connection refused means there is no reachable listener at that address; an empty models list means the server is reachable but has no downloaded models.

Select a model that actually exists

Copy an exact local model tag from the Ollama library, check its licence and supported capabilities, then download it. In the commands below, replace MODEL_NAME with that tag; it is not a model recommendation.

ollama pull MODEL_NAME
ollama list
ollama run MODEL_NAME

A model’s download size is not its total runtime memory requirement. Weights, context/KV cache, parallel work and other applications compete for memory. Begin with a smaller supported model and short task. Use the quantization decision guide before buying hardware or downloading another format.

Verify the API independently of your application

Replace the model placeholder with the same installed tag:

curl --fail-with-body http://127.0.0.1:11434/api/chat \
  -H 'Content-Type: application/json' \
  -d '{"model":"MODEL_NAME","messages":[{"role":"user","content":"Reply with one short sentence."}],"stream":false}'
ollama ps

A successful response proves basic generation, not tool-call reliability, retrieval quality or adequate throughput. ollama ps helps identify the loaded model and processor allocation. A client using an OpenAI-compatible endpoint must use the compatible API path and supported features, not assume all hosted API behavior is reproduced.

Keep the network boundary explicit

The local API does not require authentication. Keep it on loopback for a single-machine setup. Do not expose port 11434 to the internet to solve a client connection problem. Remote users need authenticated access and TLS, with administration endpoints restricted.

β€œLocal” also describes a deployment choice, not a complete privacy guarantee. Ollama supports cloud models; editors, web tools, telemetry, embeddings and agent tools can independently contact external services. Check each configured provider and its data path. Hardware, electricity and maintenance still cost money even when local generation has no hosted per-token bill.

Add one integration at a time

Your next jobGuideAcceptance check
Browser chat and accountsOpen WebUIUI reaches the intended server; access is restricted
IDE assistanceContinueChat works before autocomplete or agent tools
Repository editsAiderA disposable branch passes review and tests
Document answersLocal RAGRetrieved evidence supports the answer
Shared API capacityRuntime comparisonMeasure your actual concurrent workload

Diagnose the failing layer

Use connection refused for listener/network failures; model not found for a reachable server missing a tag; loading failed for backend startup errors. Once generation starts, separate timeouts, memory exhaustion and slow inference. They are not interchangeable reasons to delete a model cache.

Sources checked November 10, 2026: API introduction, FAQ and configuration, hardware support. Capabilities remain dependent on the installed runtime and selected model.