We talk a lot about Gen AI, but what about the knowledge powering it?
Every organization is on an AI journey, building use cases, exploring platforms, pushing towards real business value, and for good reason. The technology is ready and the potential is real.
But one thing that consistently flies under the radar in these implementations is knowledge quality. We see this firsthand. The demo looks great, AI sounds confident, stakeholders are excited. Then it goes live, cracks appear, outdated answers, contradictions, confident responses that are subtly wrong. The instinct is to blame the model, swap it out, tune the prompts, try a different vendor. But more often than not, the model isn’t the problem, the knowledge is.
Organizational knowledge is written for humans who bring context and experience. AI has its own form of reasoning, but it doesn’t share our assumptions, our organizational context, or our unwritten rules. It takes what it finds literally.
The core problems we run into:
- Content assumes the reader already knows things, AI doesn’t share that background
- Internal guidelines and customer-facing content often contradict each other, intentionally
- Documents are structured for readability, not machine comprehension
- Knowledge is spread across multiple sources with no single source of truth, and AI will use all of them
You don’t need to solve this before you start, but it needs to be on your roadmap early.
What we’re learning:
- Audience tagging: Same topic, different instructions for customers vs. internal agents. Without clear labeling the AI doesn’t know which applies
- Shared terminology: Domain terms, product names, internal system names, and abbreviations that feel obvious to us need to be explicitly defined so AI interprets them consistently
- Explicit context: Don’t assume prior knowledge. What a human infers from experience needs to be stated clearly for AI
- Contradiction resolution: Where internal and external content conflict, define which source takes priority in which context
- Metadata: Tagging content with topic, audience, validity date, and channel makes retrieval significantly smarter
- Use AI to restructure: Ironically, AI is great at rewriting human-written content into cleaner, machine-friendly formats
- Update process: Knowledge without a clear owner and review cycle becomes a liability; AI will keep using outdated content until someone tells it not to
Beyond structure, process and ownership matter too
Restructuring knowledge is the technical part. The harder part is organizational, defining who owns it, who keeps it current, and how changes are governed as products and policies evolve. Think of it less like a content project and more like a living system.
AI agents don’t just answer questions, they ask relevant follow-up questions, make decisions, and take actions on behalf of users. Knowledge quality affects all of it.
The AI solution is only as good as the knowledge it runs on
And structuring knowledge well is only part of the story, how you retrieve and connect that knowledge is equally important. Approaches like GraphRAG are changing what’s possible here, but that’s a topic for another article.
