AI can search company knowledge reliably when it draws its answers from the company’s own documents and backs every statement with a file and page reference. What does not work is asking a general-purpose language model about internal matters, or pasting an entire archive into a chat window and hoping for the best.
This article sets out the main approaches, what each is good for, and how to tell a dependable system from a convincing demo.
Why isn’t a general AI assistant enough?
Language models such as ChatGPT or Claude were trained on publicly available text. Your expert reports, contracts, bids and minutes were not part of it. Ask the model about an internal project and it can do one of two things: admit it does not know, or produce a plausible answer that is wrong. The second is called a hallucination.
For company knowledge this is a fundamental problem. In an engineering consultancy, a law practice or a public authority, what matters is not whether an answer sounds right but whether it is written in a specific document. The assistant has to be connected to the archive, and it has to show where its statements come from.
How does search with source references work?
The established method is called retrieval-augmented generation, or RAG, a term introduced in a 2020 research paper by Lewis et al. The principle is simple: the language model does not answer from memory, but from passages that a search step has first retrieved from a document collection.
In practice it works in five steps:
- Load and make readable. Documents are read from the archive. Scans are converted to text with OCR.
- Split. Each document is cut into short sections, and each section keeps a record of its file and page.
- Index. Each section gets two search paths: a classic keyword index and an embedding, a numerical representation of its meaning. Combining both is called hybrid search. It finds exact terms such as case numbers and standards as well as passages that say the same thing in other words.
- Retrieve. For a given question, the system picks the most relevant sections.
- Answer with evidence. The language model writes the answer from those sections and cites the source for each statement.
The language model is not trained on your documents. It only sees the passages found for the question in hand, and a new document is searchable as soon as it has been indexed.
Which approaches work, and which don’t?
Most organisations try several routes before settling on one. The table summarises what the common approaches are suited to.
| Approach | Works well for | Limits |
|---|---|---|
| Classic full-text search (file share, DMS) | Exact terms, reference numbers, file names | Misses paraphrases; returns lists of hits rather than answers |
| Purely semantic search (embeddings only) | Passages with similar meaning, loosely phrased questions | Often misses exact identifiers such as standard numbers or project codes |
| Hybrid search with RAG and citations | Specific questions across a whole archive, with an answer you can verify | Needs careful preparation (OCR, splitting, metadata) and index maintenance |
| Loading everything into a long context window | One or a few documents | Limited volume; models demonstrably make less use of information in the middle of long inputs (Liu et al., 2023) |
| Fine-tuning a model on your documents | Style, terminology, formats | Not suited to reliable fact retrieval; no citations; every change needs retraining |
For the question “what do our files say about X?”, hybrid search with RAG and citations is the right fit in most cases. Fine-tuning and long context windows have their uses, but they do not solve this particular problem.
How can you tell an answer is reliable?
A reliable answer can be checked in seconds. That takes three things.
First, a precise reference. “See the project handbook” is not much help. A proper reference names the file and page, ideally with the quoted passage, and opens the original.
Second, checks on extracted values. When AI reads data out of documents, such as deadlines, amounts or client names, the value should not simply be accepted. In UNID AI Second Brain, an extracted value is stored only if it comes with a word-for-word quote and that quote is actually found on the stated page. If not, the value goes to a review list for a person to check.
Third, an honest “not found”. A good system says when the archive contains nothing on a question, instead of constructing an answer.
Where do the documents stay?
With any AI search, the question is which data goes where. The answer depends on the architecture and should be settled in writing before you start.
With UNID AI Second Brain, the archive, the search index and the extracted records stay on a controlled server. Loading, OCR and embedding run there. Only the passages needed for a specific answer are sent to the AI assistant. That is far less than a full upload, but it does mean those passages are processed by the language model provider. Personal data therefore needs a legal basis and a data processing agreement under Article 28 GDPR. Our security and privacy page has the details.
Second Brain connects to the assistant through the Model Context Protocol (MCP), so the archive appears as a connector inside the assistant your staff already use. Our article What is an MCP server? explains how that works.
What limits remain?
Even a well-built search has limits, and knowing them prevents disappointment.
- Poor originals: handwriting, faded copies and complex tables in scans are often misread by OCR. What cannot be read cannot be found.
- Conflicting versions: if the archive holds five drafts of a contract, search finds all five. Which one is valid has to come from metadata or filing structure.
- Access rights: anyone who cannot see a document on the file share must not see it in AI search either. The system has to enforce this.
- Vague questions: “What do we know about fire safety?” returns a lot. Specific questions get better answers.
- Professional judgement: the AI finds and summarises. A person still makes the professional call.
How should you get started?
A controlled start tells you more than a big roll-out. This sequence works well:
- Choose a well-defined part of the archive, for example closed projects from one department.
- Collect real questions that staff currently ask by email or in the corridor.
- Decide who will check the answers against the original documents.
- Check each answer on three points: is the statement correct, is the reference correct, is anything important missing?
- Only then decide on extending scope, permissions and operation.
The Second Brain product page describes how UNID AI Second Brain puts these steps into practice. Terms such as RAG, embedding and hybrid search are explained in our glossary. To discuss a pilot with your own archive, get in touch.