AI Engineeringrag

RAG, LLM Wiki or Fat Skills: Choosing a Knowledge Layer for AI Agents

P&C

Prompt & Circuit Editorial

May 14, 2026 · 12 Min Read

RAG, LLM Wiki or Fat Skills: Choosing a Knowledge Layer for AI Agents

Why "RAG is dead" misses the point

In April 2026 Andrej Karpathy published "LLM Wiki", a GitHub gist that he describes as an "idea file": a prose description of a pattern, meant to be pasted into a coding agent, rather than a piece of code. Its argument is that instead of retrieving raw document chunks for every question, an LLM should incrementally build and maintain a persistent, interlinked markdown wiki. The gist has since collected more than 5,000 stars and 5,000 forks.

Around the same time Garry Tan of Y Combinator wrote "Thin Harness, Fat Skills", an essay that ships inside his open-source GBrain repository. GBrain is a memory layer for AI agents in which most of the behaviour lives in markdown "skills" that the agent reads and follows, some of them run on a schedule.

Both ideas were widely summarised as "RAG is dead". That is the wrong conclusion. Classic Retrieval-Augmented Generation (RAG), introduced by Lewis et al. at NeurIPS 2020, is a retrieval technique. The LLM Wiki is a way of compiling knowledge. Fat skills are a way of acting on it. They answer different questions, so the useful first question for any team is:

What is your agent's job?

  • Answering questions over a large, changing corpus?
  • Building expertise that should get better over time?
  • Taking action on what it knows without being asked?

Each answer points to a different pattern.

Pattern 1: RAG, the retriever that scales

How it works

  1. Split documents into chunks and embed each chunk into a vector store.
  2. When a query arrives, retrieve the top-k most similar chunks (often combined with keyword search and a reranker).
  3. Insert those chunks into the prompt.
  4. Generate an answer that can cite them.

LangChain's current RAG tutorial follows exactly this shape: load, split, embed and store at indexing time, then retrieve and generate at query time.

Where RAG is strong

  • Large corpora. Indexing is a mechanical, parallelisable job, so tens of thousands of documents are routine.
  • Freshness. Re-embed a changed document and the next query sees the new version.
  • Auditability. Retrieval logs show which chunks informed each answer, which helps in regulated settings.

Where it falls short

  • Chunking. A clause and the rationale that qualifies it can land in different chunks, and the retriever may return only one, producing an answer that is correct but incomplete.
  • Re-derivation. Every query starts from raw chunks. Synthesis done for yesterday's question is not reused today, so the same compute is spent again. This is the core of Karpathy's critique.
  • Passivity. A plain RAG system does not notice that a new document contradicts an older one unless someone asks.

Use RAG when the corpus is large and changes often, and the priority is a dependable question-answering assistant. It is a weaker fit when the agent should accumulate understanding or act on its own.

Pattern 2: LLM Wiki, the compiler that compounds

The idea

Karpathy's gist defines three layers:

  • Raw sources: immutable documents the LLM reads but never edits.
  • The wiki: LLM-written markdown pages (summaries, entity and concept pages, an overview) that the LLM owns and maintains.
  • The schema: a configuration file such as CLAUDE.md or AGENTS.md that tells the agent how the wiki is structured and which workflows to follow.

How compounding happens

  • Ingest. The LLM reads a new source, writes a summary and updates related pages, the index and a log. The gist notes that a single source might touch 10 to 15 wiki pages.
  • Query. The LLM reads an index.md catalogue first, then the relevant pages, and answers with citations. Good answers can be filed back into the wiki.
  • Lint. A periodic health check looks for contradictions, stale claims, orphan pages and missing cross-references.

Where it breaks

  • Scale. The gist says the index-file approach works at roughly 100 sources and hundreds of pages, and suggests adding a proper search tool (it mentions qmd, a local BM25 and vector search engine) beyond that. In other words, a large wiki needs retrieval again.
  • Ingest cost. Every new source means the model reads it plus the affected wiki pages and rewrites them, which costs far more per document than embedding a chunk.
  • Governance. Derived pages are new records. If source material is subject to retention or access rules, the summaries may be too, and a single shared markdown folder has no built-in access control.

Use an LLM Wiki for a curated set of high-value sources where deep, cumulative synthesis matters more than real-time coverage.

Pattern 3: Fat skills, the operator that acts

The idea

In "Thin Harness, Fat Skills", Tan argues that results depend less on the model than on what surrounds it. The harness should be thin (he suggests a CLI of about 200 lines) that runs the model, manages files and context, and enforces safety. Judgment lives in skills: reusable markdown procedures that take parameters like a method call. Deterministic work such as SQL or exact lookups goes to code. A resolver routes each task to the right skill, and in Claude Code that matching happens through each skill's description field. In Tan's words, "the description IS the resolver."

What a skill looks like

GBrain's enrich skill begins with this front matter (abridged):

name: enrich
version: 1.0.0
description: |
  Enrich brain pages with tiered enrichment protocol. Creates and updates
  person/company pages with compiled truth, timeline, and cross-links.
  Use when a new entity is mentioned or an existing page needs updating.
triggers:
  - "enrich"
  - "create person page"
  - "update company page"
tools:
  - get_page
  - put_page
  - search
  - query
  - add_link
  - add_timeline_entry
  - get_backlinks
mutating: true
writes_pages: true
writes_to:
  - people/
  - companies/

The body defines three enrichment tiers (key, notable, minor) and a seven-step protocol: identify entities, check whether a page exists, extract the signal, look up external data, save raw API responses for provenance, write the page, and cross-reference related pages.

Always-on and scheduled layers

GBrain includes an opt-in signal detector that captures durable ideas and entity mentions from conversations, and a cron layer that runs skills on a schedule. Tan's essay gives the rule of thumb: once a task has been done by hand and approved, codify it as a skill, and if it should run automatically, put it on a cron. As of October 2026 the README reports 66 cron jobs in Tan's own deployment and a brain of more than 155,000 pages. Storage defaults to PGLite (Postgres compiled to WebAssembly), with Postgres plus pgvector for larger or shared setups.

Where it gets hard

  • Engineering effort. The repository's skills directory holds dozens of skills plus a RESOLVER.md and filing rules. This is a maintained software project, not a prompt.
  • Personalisation. The skills encode one person's workflows and sources, so adopting them means rewriting them.
  • Breadth of audience. Skill-driven agents suit a few power users with well-defined routines better than a general knowledge assistant for a whole organisation.

Use fat skills when the agent needs to act, when you can fund ongoing engineering, and when a small number of owners can define the workflows.

A decision framework

What does the agent need to do?
 ├─ Answer questions over a large, fast-changing corpus?   → RAG
 ├─ Build deep, evolving expertise from a modest corpus?   → LLM Wiki
 └─ Act on knowledge autonomously (schedules, alerts)?     → Fat skills

In practice, the layers combine

The patterns are not exclusive, and the two "RAG is dead" examples already show it. The LLM Wiki needs a search engine once it grows. GBrain stores its pages in Postgres with vector search and runs skills over them. A realistic production design looks like:

  1. A retrieval layer that finds raw material at scale.
  2. A compiled layer that turns the most important material into maintained, cross-referenced pages.
  3. A skill layer that watches for changes and acts on them.

Coding agents show a small version of this: Claude Code loads CLAUDE.md instruction files and its own auto memory notes at the start of every session, searches the codebase on demand, and acts through tools.

Takeaway

The knowledge layer is often where an agent breaks first, and also where the biggest gains come from.

  • RAG: scale first, no accumulation.
  • LLM Wiki: depth first, knowledge that compounds, limited scale.
  • Fat skills: action first, at the cost of engineering effort.

Decide what the agent must do, start with the matching pattern, and add the others as the need becomes real rather than putting RAG everywhere, or nowhere, by default.

Sources

Tags:ragllm-wikiai-agentsknowledge-managementarchitecture
Share Article

Responses (0)

Sign in to join the conversation and share your thoughts.

P&C

Prompt & Circuit Editorial

Contributing Editor

Articles on AI infrastructure for cloud architects, written from official vendor sources and checked claim by claim before publishing.