Data, RAG & Self-Hosted AI tag

13 Articles

Data, RAG & Self-Hosted AI ×

Fine-Tuning: When Is It Finally Worth It?

A fine-tuned model with 350 million parameters hit a **77.6%** success rate on a tool-calling task — while a general model with 175 billion parameters managed **26%** on the same task. Five hundred times fewer parameters, three times the result. That is the real promise of fine-tuning. And meanwhile one large provider is **retiring its fine-tuning platform**: no new users are being accepted, and existing ones have until January 2027.

Advanced Developer Explainer Data, RAG & Self-Hosted AI

Sep 13, 2026

Krisztian Eyssen, Krisztian Eyssen

Private AI: Keeping Data Inside Your Company

One large provider's zero data retention agreement lists item by item which endpoints it covers — and which it does not. On the does-not list: **file upload** and the **vector store**: **exactly the two features you would use to build RAG on that platform.** And on the other side, the assumption that "local means safe": a 293-day survey found **175,108 publicly reachable local model servers** in 130 countries.

Advanced Business owner How-To Data, RAG & Self-Hosted AI

Sep 13, 2026

Krisztian Eyssen, Krisztian Eyssen

Local AI Runners: Which One to Pick

The most important thing to know: **Ollama, LM Studio, Jan, KoboldCpp and Docker Model Runner all use the same engine** — llama.cpp, and MLX on a Mac. They do not differ in speed but in the packaging and in the **defaults**. And the defaults are exactly where the trap is: Ollama **truncates at 4,096 tokens** out of the box, even when the model could handle 256k.

Beginner Developer Tool review Ollama Data, RAG & Self-Hosted AI

Sep 13, 2026

Krisztian Eyssen, Krisztian Eyssen

Quantization: How Big Models Fit on Small Machines

According to a unified 2026 measurement, four-bit quantization **cuts model size by 71%** and costs on average **0.43 percentage points** of accuracy. Essentially free. Except **not for everyone**: the same four-bit quantization also produced a **59%** collapse on long input, and the degradation is **up to five times worse in languages other than English** — measured across 26 languages, Hungarian among them.

Advanced Developer Explainer Data, RAG & Self-Hosted AI

Sep 13, 2026

Krisztian Eyssen, Krisztian Eyssen

Hardware for Local AI: VRAM, RAM and Apple Silicon

The rule of thumb everyone repeats says you need about half the parameter count in memory. `[Own calculation]` The four-bit weights of an eight-billion-parameter model really do eat 4.6 gigabytes — **but the same model's 128k-token context eats 18 gigabytes**, four times the model itself. Anyone who only looks at model size has worked out a fifth of the memory they actually need. And there is one measurement that turns the whole picture around: **a chip with half the GPU cores processes a prompt 83% faster than a more expensive one** — while generating a fifth slower.

Advanced Developer How-To Data, RAG & Self-Hosted AI

Sep 13, 2026

Krisztian Eyssen, Krisztian Eyssen

Why Run AI on Your Own Machine?

Of the four usual reasons, two hold up and two do not. `[Own calculation]` Against the cheapest API tier, a $3,500 machine breaks even at **2.6 billion tokens a year** — 133 days of continuous generation. And here is the twist that holds the whole article together: **the volume at which local running pays off is exactly the volume at which the local model is no longer good enough.** There is a fifth reason, though, that nobody states out loud, and measurably it is the strongest of the lot.

Beginner Business owner Explainer Data, RAG & Self-Hosted AI

Sep 13, 2026

Krisztian Eyssen, Krisztian Eyssen

Keeping the Knowledge Base Fresh

A measurement built on 96 thousand question-answer pairs and 219 thousand documents found that the mere presence of outdated content causes, according to the study, **at least a 20% drop in performance**, with some models doing worse than chance. But the most important finding is not the number: **models stay appropriately uncertain when they find nothing — and are confidently wrong when they find something stale.** An empty knowledge base is less dangerous than an old one.

Intermediate Business owner How-To Data, RAG & Self-Hosted AI

Sep 13, 2026

Krisztian Eyssen, Krisztian Eyssen

A Company Knowledge Chatbot: A Realistic Plan

An airline's chatbot gave the wrong rules for its bereavement fare. The company's defence was that it is not responsible for "information provided by its agents, including a chatbot" — and it floated the idea that **the chatbot is a separate legal entity**. The court's answer is one sentence worth printing out: **"It makes no difference whether the information comes from a static page or from a chatbot."** The damages came to $812. The precedent is not so cheap.

Intermediate Business owner Recipe Data, RAG & Self-Hosted AI

Sep 13, 2026

Krisztian Eyssen, Krisztian Eyssen

How Do You Know Your RAG System Is Right?

At a public evaluation campaign a judge model was measured against a human assessor. Their agreement, corrected for chance, came out at **0.29**. On the very same task, **two humans agreed at −0.03** — worse than chance. So this is not a story of "the machine is bad, the human is good". The task itself is ill-defined at the edge cases. This article is about what can still be measured reliably.

Advanced Business owner How-To Data, RAG & Self-Hosted AI

Sep 13, 2026

Krisztian Eyssen, Krisztian Eyssen

Hybrid Search: Why Vectors Alone Are Not Enough

In an academic measurement of 24 thousand questions, vector search scored **19.2%** on questions of the "who is [X]'s child?" type — the decades-old keyword algorithm scored **82.9%**. Sixty-three points apart. On a different set of questions the very same methods come out the other way round. **Neither one is better than the other: they have different failure modes.** Except that hybrid search is not what improves things most — re-ranking is.

Advanced Developer How-To Data, RAG & Self-Hosted AI

Sep 13, 2026

Krisztian Eyssen, Krisztian Eyssen

Vector Databases Without the Jargon

A widespread myth says the vector index eats a lot of memory. According to the vendor's own capacity-planning formula, **the graph is 5% of total memory** — 0.29 GB of index alongside 5.72 GB of vectors. It is the vectors that eat memory. And there is an even more important number: **ten thousand vectors are 75 megabytes.** That fits on a phone. So the question is not whether you need a separate vector database, but **when** — and the answer is usually "later than you think".

Intermediate Developer Explainer Data, RAG & Self-Hosted AI

Sep 13, 2026

Krisztian Eyssen, Krisztian Eyssen

Preparing and Chunking Your Documents

In one measurement, simply cutting a long document into chunks lifts retrieval from **28.2 to 45.7** — **17.5 points for free**. The same measurement also shows that **bigger chunks are monotonically worse**. The awkward part: the most widely used tools ship with defaults of 800 and 1,024 tokens, or 4,000 characters — and in an independent test the 800-token setting finished **near the bottom of the field**.

Advanced Business owner How-To Data, RAG & Self-Hosted AI

Sep 13, 2026

Tags

Audience

Everyone Office worker Marketer Creative Business owner Developer

Follow Us

Archives

Tags