Self-Hosted Personal Agents and Their Risks

Why did this become a category of its own in 2026?

If it is not yet clear what makes a program an agent rather than a chat window, the what an agent actually does primer sums it up. Two years ago "local AI" meant chatting with a smaller model on your own machine. In 2026 a new phrase turned up in the model line-ups: models built "for always-on local agents" — tuned specifically for tool use, for long tasks and for recovering from errors, on consumer hardware.

The orders of magnitude you have to reckon with:

Download size is not the same as memory requirement. For one agent-oriented model the vendor spells it out separately: at least 36 GB of memory is needed on a normal desktop machine, even though its download is 20 GB (the 36–67 GB row in the table above, by contrast, is the download size of higher-precision variants). This is the most common source of disappointment.

The trap almost everyone walks into

This is the single most practical fact in the article, and it is worth reading for this alone.

The default context window of the popular local model runners is 4,096 tokens.

You download the model whose description mentions a 256 thousand token context — and it runs on four thousand and ninety-six, unless you change it by hand with an environment variable.

For chatting this is imperceptible. For agent work it is fatal: a single longer file or a few tool calls eat it up, and from then on the agent forgets what it did three steps ago. The symptom is not an error message but stupidity — and people blame the model, not the setting. What the context window means exactly is covered by the tokens and the context window piece, with examples.

How much weaker is a local model? — measured

Here you have to be honest, because the difference is not small.

From an independent, university function-calling measurement (as of April 2026):

And the most important detail: with the smaller models the multi-turn task simply collapses.

This is the most important measurement in the article. Single-call tool use — "check the weather" — is something small models can do to some extent. The multi-turn, state-carrying agent task is where the local model hits the floor. And this is not a ten or twenty percent difference, it is an order of magnitude.

One important counterpoint is needed here, though. In the same measurement, an open-weight, permissively licensed model practically caught up with the frontier closed model in the multi-turn category (68.0% against 68.4%), at a fraction of the cost.

But: open weights are not the same thing as runnable on a laptop. That model is far bigger than what a notebook can handle. What open weights are good for is the subject of the open weights post — though that is a different question from whether it fits on your machine.

(The measurement referred to here was last updated in April 2026, so it does not contain the newest models. The direction of the gap is probably durable, its exact size is not.)

What it reaches by default — and this is the point

The documentation of the most widely used open, locally running personal agent is unusually straight. Three sentences from it, worth reading literally:

• "The sandbox is off by default."
• "Tools run on the host machine in the main session, unless you configure the sandbox."
• And even switched on it is not perfect: "not a perfect security boundary, but it meaningfully restricts file system and process access."

What does that mean in practice? The capability list from the vendor's own description: email, calendar, web browsing, file operations, running shell commands, and persistent memory across conversations.

So on default settings, this is a program that runs commands on your machine, reads your mail, and remembers. Individually these are useful capabilities. Together, without a sandbox, reading text that arrives from the internet, this is the riskiest setup you can put into service at home.

The sentence everyone ought to read

The security policy lists what does not count as a reportable vulnerability. That section is more honest than any marketing material:

NOT COUNTED AS A SECURITY BUG

- prompt injection, if it does not bypass policy,
  authentication, approval, the sandbox or a tool
  boundary;

- a trusted operator using a deliberate local
  feature - local shell access, for instance, or
  running a script;

- a malicious extension, once a trusted operator
  has installed or enabled it.

And the framing sentence:
"Anyone who can operate an agent can make it do
 anything that agent is able to do."

Translated: if your personal agent has access to a shell, then prompt injection is not a vulnerability, it is how the system works. There will be no fix for it, because there is nothing to fix — you gave it the capability.

This is not an accusation against the developers; on the contrary, this is the correct and honest framing. But it means that you have to build the security yourself, not expect it from the software. The mechanics of the attack are covered by the prompt injection basics introduction, and narrowing permissions by the least-privilege agent post, step by step.

The documented risks

The vendor's own risk list, because it is more itemised than anything you could assemble from outside:

What is set up well by default: the main process only binds to the machine's own network address (so it cannot be reached from outside), and most messaging channels ask for a pairing code from an unknown sender. But the container install opens outwards by default — and that is exactly the step everyone takes on a "I am building my own agent" weekend, so they can reach it from their other machine.

That this is not theoretical: a 2024 vulnerability in one of the widespread local model runners allowed remote code execution, and at the time of disclosure more than a thousand exposed instances were reachable on the internet. As an aggravating circumstance the researchers highlighted separately that the container install runs with administrator rights and listens on every network interface. The developers fixed it within four hours — the exposed instances stayed there for weeks.

(This is a 2024 case. From 2025–2026 I could not confirm any publicly documented data loss caused by a local agent — which is not evidence that none happened.)

Is there a privacy benefit?

This is the most common reason somebody starts with a local agent. The answer: yes, but not where most people think.

Where it genuinely holds:

• The state, the memory and the credentials stay on your hardware.
• Good solutions send no telemetry by default — nothing beyond a daily version check, and that can be turned off too.
• There is no provider-side data retention, because there is no provider.

There is a strong argument for it too that rarely comes up: with managed, cloud agent services the provider itself writes out that, because of the stateful operation, that particular service is not suitable for zero data retention and for certain healthcare privacy agreements. If you are going local for privacy reasons, this is your best argument.

Where it does NOT hold — and this has to be said:

1. The screenshot leaves the machine. A court finding recorded literally, about a browser agent, that the program takes screenshots of the browser, sends them from the user's machine to the provider's servers, and gets the instruction back from there on how to navigate next. "Runs locally" does not mean the content stays local.

2. The local harness often calls a cloud model. The local runtime and the local model are two different things. The agent running on your machine does not stop the thinking from going to the cloud — and the data with it.

3. The extensions. The extension marketplace is a world of its own, and — as noted above — a malicious extension after installation does not count as a security bug. What you install is your responsibility.

4. The logs are on your machine — but they are there. If you switch on detailed logging, the prompts, the tool inputs and even the full requests land on disk. The same risk, only on your side, and often unencrypted. Where data is stored is what the EU data residency post goes into in more detail.

What to set up before you start it?

Ten points, every one of them following from primary documentation or from a documented incident — not from opinion:

1. SWITCH THE SANDBOX ON
   It is off by default. Not a perfect boundary,
   but a meaningful restriction.

2. CHANGE THE CONTEXT WINDOW
   The default is 4096 tokens. Too few for agents.

3. DO NOT PUT IT ON THE NETWORK UNAUTHENTICATED
   The container install opens outwards by default.
   Instances exposed this way: more than a thousand.

4. A LIST OF ALLOWED DOMAINS
   Restrict where it can go out to. This is the
   cheapest defence against exfiltration.

5. DO NOT HAND OVER YOUR PASSWORDS
   A separate account, browser profile, minimal rights.

6. HUMAN APPROVAL FOR CONSEQUENTIAL STEPS
   Moving money, accepting a contract, sending mail.

7. A DEDICATED VIRTUAL MACHINE OR CONTAINER
   Not on your main work machine, next to your files.

8. SCREEN CONTENT IS NOT AN INSTRUCTION
   What the agent reads on a page is data.
   It cannot grant authority.

9. AN ALLOWLIST FOR SENDERS
   If it can be driven by message, known senders only.
   Whoever can message it can approve in your place.

10. CHECK WHAT GOES INTO THE LOG AND THE .ENV
    File permissions, and never store your keys
    in the working directory.

The seventh point is the one that brings the most for the least work. If the agent runs on a separate virtual machine and only sees what you put in front of it, then the failure of the remaining nine points does limited damage as well.

Common misconceptions

• "It runs locally, so it is private." Screenshots, cloud model calls and extensions can all get the data out. Local running makes the storage local, not necessarily the processing.
• "A small model is enough for a simple agent." For a single call, yes. For a multi-turn, state-carrying task the measurement dropped below 6% at 12 billion parameters and under.
• "I downloaded the 256 thousand token model, so that is my context." The default is 4,096 tokens. You have to change it by hand.
• "An open-weight model is cheaper and almost as good." For some open models that is true — but those are typically not laptop-sized. Open weights and local runnability are two different things.
• "If there is a security hole in it, they will fix it." Prompt injection through local shell access is explicitly not treated as a security bug by the developers. There is nothing there to fix.
• "Only I can reach it, it is at home." Until you change the setting so you can reach it from your other machine. In the documented incident that was exactly the entry point.

What to take away

An agent running on your own machine gives you real freedom: no subscription, no content limits, the data stays on your disk. In exchange you have to build the security boundary yourself, because the software explicitly does not promise to do it for you.

Two numbers decide whether it is worth it: 27 percentage points of gap on multi-turn agent tasks against the frontier closed models, and 4,096 tokens, the default context window nobody tells you about. Because of the first, a local agent today is better for well-bounded tasks than as a general assistant; because of the second, many people never even experience that much of it.

And if you only set up one thing: put it on a separate virtual machine. That one step limits the consequences of most of the mistakes — and it needs nothing but half an hour of patience. What kind of lasting memories an agent collects about you in the meantime is what the agent memory post is about, with examples. What that boundary-building looks like in a real, home-grown system is shown decision by decision in building an agent of your own, trade-offs included.