Internal search with AI: how to stop losing your company's knowledge

There's a scene that repeats in every company past thirty people. Someone asks in chat, "what's the returns policy for wholesale customers?" Three people answer differently. A fourth sends a PDF. The PDF is from 2021 and there's a newer one nobody can find.

That isn't an organisational problem. It's a search problem, and it's now solvable.

Why traditional search isn't enough

The search box in your file system, your drive or your intranet looks for exact words. If the document says "refund" and you type "return", nothing surfaces. If the answer is on page 14 of a 60-page manual, search hands you the whole manual and leaves the work to you.

Above all: it doesn't answer. It gives you a list of files. The person asking wanted an answer.

How AI search actually works, minus the hype

The mechanism is retrieval-augmented generation (RAG), and it has three parts:

1. Index. Documents are split into fragments, and each fragment is turned into a numeric representation that captures its meaning rather than its words. That's why "refund" and "return" end up close together even though they share no letters.

2. Retrieve. When someone asks, the system finds the fragments closest to the meaning of the question. Not the ones sharing words — the ones about the same thing.

3. Answer. The model writes a response using only those fragments, and cites where each claim came from.

That last point is the difference between a usable tool and a generator of plausible answers. If it can't cite, it doesn't answer.

The decisions that determine whether it's useful

Permissions, from day one

The most expensive mistake and the easiest to make. If the index doesn't respect who can see what, you've just built a machine that tells anyone about salaries, contracts and the HR folder.

Permissions are applied at retrieval time, filtering to what that user is allowed to see — not afterwards on the finished answer. A system that generates the response and then tries to censor it has already lost: the model saw the data.

What gets indexed, and what doesn't

The temptation is to index everything. That's a mistake: noise degrades answers faster than coverage improves them.

Drafts, superseded versions of live documents, inconclusive email threads and anything nobody designated as a source of truth should stay out. A small curated corpus beats a huge dirty one every time.

Dates matter

A current policy and a repealed one look very similar in meaning. Without date and status metadata, the system will cheerfully retrieve the policy that was withdrawn in 2023.

Every fragment needs to know which document it came from, when it dates from, and whether it's still in force. And the answer has to show it: "per the operations manual, March 2026 revision."

The ability to say "I don't know"

A system that always produces an answer is a system you can't trust. When retrieval comes back thin, the correct response is "I couldn't find this in the documentation" and, where relevant, who to ask.

This is configured and tested. It doesn't come free: by default, models would rather invent something than admit a gap.

Where to start

Not by indexing the whole company. With one team and one set of current documents.

Customer support is usually the best first case: the questions repeat, the documentation exists in writing, and impact is easy to measure in response time.

Before starting, someone has to do an unglamorous piece of work: decide which documents are the truth and archive the rest. That work has to happen anyway, AI or no AI. The difference is that now it has an immediate payoff.

How long it takes and what to expect

Internal search scoped to one team goes into production in three to six weeks. The technical part isn't the slow one — gathering and curating the material is.

A reasonable expectation is that 60–70% of repeated internal questions resolve themselves, with a citation to the source. The rest still needs a person — which is fine, because that's usually exactly where judgement is required.

If your company's knowledge lives in three people's heads and one shared folder, let's talk: it's among the fastest-returning projects we do.