People Finder: when the question is a relationship, not a similarity

Large organisations have a knowledge location problem that has nothing to do with documentation. The information exists, mostly in people's heads and in the pattern of who has worked with whom, and the difficulty is finding the right person in a company of tens of thousands.

People Finder was built to answer that question directly: given a topic, who should I speak to.

The default answer to "build a search assistant" in 2024 was retrieval augmented generation over a document corpus. That was the pattern everyone reached for. The trouble was that it did not match the question being asked.

• Vector search answers "what text is similar to this query". The user was asking "who is connected to this topic", which is a different question with a different shape.
• Organisational data is already relational: reporting lines, project membership, skills, collaboration history. Flattening that into embedded chunks throws away the structure that answers the question.
• Wrong answers here are unusually costly. Sending someone to an irrelevant colleague wastes two people's time and erodes trust in the agent immediately.

I argued that the retrieval layer should be a knowledge graph built over people, skills, projects and interactions, with the language model used for interpreting the question and phrasing the answer rather than for the retrieval itself.

Put simply: let the graph find the candidates, and let the model explain why. The alternative, vector search across documents and profiles, would have been faster to build and easier to demo, and it would have been the wrong tool. It would have retrieved people whose profiles used similar words, not people who were actually connected to the work.

The second half of the decision was to define what "good" meant before building the agent at all. I insisted on an evaluation set of roughly a hundred realistic questions with known correct answers, so that accuracy was something we could measure rather than assert.

• Made the retrieval-architecture call early, and documented the reasoning so the team was not relitigating it mid-build.
• Defined the evaluation set of roughly 100 cases with expected answers, which turned "the agent feels good" into a number we could track.
• Worked with engineering to keep the graph schema tied to the questions users actually asked, rather than modelling the org chart in full.
• Drove the answer experience: the agent had to show its reasoning, because a recommendation to contact someone is only credible if the user can see why.

The agent reached above ninety percent answer accuracy against the hundred-case evaluation set.

The general lesson is that the retrieval architecture is a product decision, not an implementation detail. Choosing between a graph, a vector index and a plain relational query determines what the product can answer well, and that choice should be made by whoever owns the problem definition, before engineering starts optimising the wrong thing.