The idea that drives the whole thing
Picture a map that has meanings on it instead of cities. Points close to each other mean similar things.
An embedding is the operation that gives a piece of text its coordinates on that map. The coordinate is not two numbers, as on a real map, but several hundred or a few thousand — but the principle is the same: texts with similar meanings end up close together.
From there it is simple. When you search for something, your search phrase is placed on the same map and you look at what is nearby. That is vector search.
Why is this better than keyword search?
Because it compares meanings rather than words.
For the question "how can I terminate the contract", a keyword search returns the paragraphs containing the word "terminate". Vector search also finds the one that begins "A declaration of intent to bring the agreement to an end shall…" — even though not a single word is shared.
In a company knowledge base that difference is not a nuance, it is the whole basis of the thing. Your documents were written in the language of the lawyer, the engineer and the salesperson; the questions are asked by customers, in their own.
What does it look like in practice?
Four steps, and you can get all four wrong.
1. Chunking. You break the documents into smaller pieces. A 40-page contract does not have a single meaning — the notice period and the warranty are two separate points on the map.
2. Embedding. You run an embedding model over every chunk and store the resulting row of numbers.
3. Storage. You put those rows into a database that can find the nearest points fast. This is the vector database.
4. Search. You compute an embedding for the question with the same model and retrieve the few nearest chunks.
The four traps
1. Chunking decides everything
This is the most common source of failure and the least visible. If a single idea gets cut in half across two chunks, neither chunk will be a good answer — and the search will rate both of them weak.
A practical rule: let chunk boundaries follow the structure of the document — a paragraph, a section heading, a list item — rather than a fixed character count. And let neighbouring chunks overlap, so a sentence on the boundary survives in both.
2. Search and storage need the same model
If you filled the database using one embedding model, you have to embed the question with that same model. Two different models draw two different maps, and the coordinates have nothing to do with each other.
There is an operational consequence: if you change embedding model, the whole database has to be rebuilt. Plan for that in advance, because it is not a small job.
3. Non-English content is not automatically fine
Not every embedding model is multilingual, and those that are are not equally strong in every language. Before you build anything, test it on text in your own language: take twenty question-and-paragraph pairs where you know the right answer, and see whether it finds them.
It can well turn out that an English-first model is weaker in your language than a smaller but explicitly multilingual one. This is not a matter of size.
4. Meaning alone is not enough
Vector search searches on meaning, and that makes it bad at exact identifiers: part numbers, order numbers, statutory references, dates. Good old keyword search is more accurate for those.
The answer is not to choose but to add: hybrid search — keyword and meaning-based search together, with the results merged. On most systems this single change improves things more than anything else.
Where does all this connect to AI?
Embeddings and vector search are not a generative technology — they write nothing, they only find. The language model enters when it composes an answer from the retrieved passages. That pairing is what we call RAG, and there is a separate article on it: Fine-tuning, RAG or a Better Prompt?
It is worth keeping the two apart in your head, because most bad answers are not the model's fault but the search's. Hand it the wrong paragraph and even the best model cannot produce a good answer from it.
A half-day test before you build anything
Take thirty real questions that your customers or colleagues have actually asked. For each one, write down which document and which paragraph holds the answer.
Then build a simple index and check: in how many cases is the correct paragraph among the top five?
If that number is below eighty per cent, do not go any further towards the language model — the search has to be fixed first. If it is above, the hard part is behind you.
What to take away
Embeddings turn meaning into numbers, and vector search searches within that. Eighty per cent of a RAG system's quality comes down to chunking and retrieval, not generation — which is exactly why those two steps are where the time should go.