Writing

An LLM wiki is not a self-maintaining knowledge base

The assumption I keep hearing is that connecting an LLM to a personal wiki will let the knowledge base largely maintain itself. The reasoning usually goes like this: once the model can read the documents, generate summaries, and link related pages, the old chores of organizing and updating should mostly disappear. I understand the appeal. A system that ingests scattered notes and turns them into something coherent without constant human effort sounds like the whole point of using an LLM in the first place.

Casey Newton's experiment points elsewhere. His LLM wiki does save real research time, but it still demands ongoing maintenance. Pages drift as new information comes in and old summaries no longer match the underlying sources. Timelines need checking because different articles can contradict each other or reflect outdated reporting. Suggested links require validation before they can be trusted as part of the knowledge graph. None of this means the tool is broken. It is the actual operating model, not a temporary onboarding phase.

The real shift is from searching documents to tending a living knowledge graph. A traditional search index can sit mostly untouched and still return relevant documents when asked. A wiki connected to an LLM is different because the model actively reshapes what it knows based on the connections it has already made. If those connections are wrong or stale, retrieval becomes less useful, sometimes quietly. Keeping the graph healthy means reviewing links, correcting mistaken associations, and deciding what should be merged or retired. That work is not overhead. It is the product.

If you are planning an enterprise version of this, do not budget for the model alone. The model will consume compute, but the larger cost is likely to be the human loops that keep retrieval trustworthy. Someone has to check whether a recommended link points to the right page, whether a generated summary still reflects the current source material, and whether a timeline drawn from multiple reports holds together. These tasks do not go away as the system matures. They become more important because the system will be used for decisions that depend on its accuracy.

The lesson is not that LLM wikis are a bad idea or that maintenance means the approach has failed. It is that the assumptions behind the project need to change. Treat curation as a core function, not a cleanup activity. A model can amplify what people already know and help them see connections faster than they could on their own, but it cannot replace the judgment required to keep those connections reliable. Curation is the product, and the people doing that curation are part of the system, not a temporary fix.

Originally covered at platformer.news ↗