Local AI Runners: Which One to Pick

This article uses product names

The blog usually avoids naming vendors — but this post is explicitly a comparison of tools, and there the name is the information. Every status here is as of 13 September 2026, verified.

Who is alive, who has died

The bottom of that list is the most important part: GPT4All has not had a release in 19 months. An August 2025 issue, still unanswered, says in so many words that there is no support for newer models. If someone recommends GPT4All to you in 2026, that guide is old too.

Three name changes worth knowing about:

1. LM Studio has split. Today "LM Studio" is the desktop application; the headless background service is llmster; the command-line tool is lms; and there is an agent called Bionic. Anyone writing about "LM Studio" in 2026 has to say which one they mean.
2. llamafile is no longer at Mozilla but has been in the mozilla.ai organisation since October 2025. For existing users nothing changed.
3. Ollama is no longer local only. In July 2026 it raised 65 million dollars (88 million in total), it has a cloud service priced by GPU time, and by its own account it serves 8.9 million developers a month. In 2026 the name means two things, and only one of them is local.

Where they really differ

vLLM is the only one that is architecturally different — and the only one built for multi-user serving. The rest are the same engine in different wrapping.

The defaults that catch you out

This is the most useful part of the article. All of it comes from primary documentation.

Ollama:

The llama.cpp server: the context window defaults to 0, which means "loaded from the model".

Here is the sharpest contrast in the whole article:

OLLAMA TRUNCATES at 4096 tokens by default,
even when the model could handle 256k.
-> Your text disappears SILENTLY.

LLAMA.CPP tries to allocate the model's FULL
advertised context by default.
-> If there is no memory for it, it fails LOUDLY.

Two opposite defaults,
two opposite failure modes.

The silent one is the more insidious: the model does not error, it just suddenly fails to remember something from three steps ago. And people blame the model, not the setting.

LM Studio: the server listens on port 1234; a loaded model stays in memory for 60 minutes; and "Auto-Evict" is on by default, so at most one model is loaded at a time. For commercial use no licence has been required since July 2025.

text-generation-webui: according to the May 2026 release notes, browser access is restricted "to the local machine only by default from now on". Which means it was not, until then — anyone running an older version may have had an API reachable from any website.

Is there a measurable speed difference?

Yes, but not where people look for it.

For a single user, barely any. In one comparison, on the same hardware with the same model, vLLM reached 793 tokens per second and Ollama 41 — but only under parallel load. In single-user mode both sat around 130-180 tokens/s, effectively identical. The nineteen-fold gap comes from continuous batching, not from the engine: under parallel requests, a runner without batching queues them up, and its total throughput is divided among the users.

On a Mac there is a real gap, though, and it comes from an academic measurement. A 2026 study on a unified-memory machine, with four-bit models:

On a Mac, MLX-based runners are 20-90% faster — the gap is not even; it was largest on the smallest model. The explanation is native handling of unified memory and copy-free tensor operations.

And the backend library matters too. On another manufacturer's hardware, on the same machine with the same model, the two available backend libraries differed by 2.6× in prompt processing — and at an eight-thousand-token context the order reversed for generation.

So the question "which runner is faster" often really means: "which backend library was compiled into it".

The part that gets less airtime: security

There is an unspoken dimension to picking a runner.

• Every llama.cpp-based runner inherits the bugs of the model-format parser. Serious vulnerabilities were found in 2024, 2025 and 2026, including ones that can lead to code execution.
• Ollama's container image listens on every network interface by default — unlike the native install, which is only reachable from the machine itself. In a 2024 vulnerability that was exactly the aggravating factor, and more than a thousand exposed instances were found.
• A model file is code, not data. In early 2025 two malicious models were found on a popular sharing site, having got past the security scanner.
• The public question about Ollama's telemetry has been open without an answer since February 2024. The documentation says only that they do not see prompts — which is not the same as sending nothing.

That side of it is what the private AI post covers, in more depth.

Which one should you pick?

"IT SHOULD JUST WORK"
-> Ollama. One command and it runs.
   First thing to do: change the context window.

YOU WANT A GUI, AND YOU ARE ON A MAC
-> LM Studio. With MLX support, which on a Mac
   is measurably faster.

YOU WANT EVERY SWITCH
-> llama.cpp. The engine itself, directly.

YOU SERVE OTHERS (several users)
-> vLLM. A 19-fold difference under parallel
   load - this is the only one built for it.

LOCAL AND CLOUD IN ONE APP
-> Jan.

YOU ALREADY USE CONTAINERS
-> Docker Model Runner.

The start-up checklist

Six items, every one of them following from primary documentation:

1. SET THE CONTEXT WINDOW
   The default is either too small (4096) or too
   big ("from the model"). Neither is what you want.

2. KV CACHE TO 8 BITS IF THE CONTEXT IS LONG
   Half the memory, unmeasurable quality loss.

3. CHECK WHAT THE SERVER BINDS TO
   The native install binds to loopback, the
   container image to EVERY interface. That matters.

4. CHECK THE UNLOAD TIMER
   5 minutes or 60 minutes. This decides whether
   the first request will be slow.

5. UPDATE THE RUNNER
   The model-format parser had several serious
   vulnerabilities between 2024 and 2026.

6. ONLY DOWNLOAD MODELS FROM TRUSTED SOURCES
   A model file can contain executable code.

The first item takes five seconds, and it heads off the most common disappointment.

Common misconceptions

• "Runner X is faster than runner Y." In single-user mode most of them run the same engine, and in measurement they perform essentially identically. The difference shows up under parallel load and on a Mac.
• "GPT4All is a good starting point." It has not had a release since February 2025, and there is no support for newer models.
• "Ollama is local, so it is private." In 2026 the name means both a local runner and a cloud service. Only one of them is local.
• "The factory settings will be fine." One runner truncates at 4,096 tokens, the other would allocate the full context. Neither is what you want.
• "The size of the context window depends on the model." What the model is capable of, yes. What you actually use is decided by the runner's setting.
• "A downloaded model is just data." In a documented case it can contain executable code that runs on load.

What to take away

Picking a runner is not a speed decision, because most of the field runs the same engine. It is a decision about packaging, defaults and working style.

Three things that really do matter: on a Mac the MLX-based runner is measurably faster (20-90%); serving several users needs different software (a nineteen-fold difference under parallel load); and the default context window is wrong everywhere — either too small or too big.

If you do one single thing after installing: set the context window. Those five seconds are worth more than the choice between runners. Sizing is what the hardware for local AI post works through, and quantization is the subject of the quantization piece, with the measurements.