the architecture of truth

The output of an LLM is not a fact. It is a lossy decompression of the internet.¹

When you ask for a quote, a citation, or a function argument, the model is not retrieving it from a database. It is reconstructing a high-probability pattern from the compressed representation in its weights. Sometimes, the pattern is perfect. Sometimes, it is missing a few pixels.

We call these missing pixels “hallucinations.”² Since the release of GPT-3 we’ve been trying to fix them by shouting at the compression algorithm, but it rarely works for long.

The Conservation of Complexity

Every engineering task has a fixed amount of inherent complexity.³ To build a reliable system, you must handle that complexity somewhere.

In traditional software, you handle it in the code. You write distinct logic for every edge case.

In AI engineering, we try to offload that complexity to the model. We write a prompt that says: “Be careful, check your work, and don’t make up citations.”

Then you change the prompt to fix a tone issue, and suddenly the citations break. You fix the citations, and the JSON output becomes malformed. It turns into a game of Prompt Whac-A-Mole.

Open-Loop vs. Closed-Loop

We often make the mistake of treating LLMs as open-loop systems.

In an open-loop system, you send an input, you get an output, and you show it to the user. You cross your fingers and hope the prompt was good enough.

Reliable engineering requires closed-loop systems (control theory⁵). You assume the output will be wrong (noisy), so you measure it against a reference and feed the error back into the system.

• Open Loop: “Call the weather API.” -> Model returns tool call -> Execute it. (Risk: High)
• Closed Loop: “Call the weather API.” -> Model returns tool call -> Verify (Schema Check) -> Invalid? -> Feed error back -> Model retries. (Risk: Managed)

The Three Layers of Grounding

To architect for truth, we must stop asking the model to “be true” and start constraining the search space so that truth is the only valid output.

1. Semantic Grounding (RAG)

Constraint: The Knowledge Base

Don’t ask the model to remember facts. Its memory is lossy. Inject the facts into the context window. This shifts the task from recall (hard, hallucinatory) to processing (easier, grounded).

2. Structural Grounding (Schemas)

Constraint: The Format

Don’t ask the model to “be consistent.” Force it. Use JSON modes, grammars, or tool definitions to constrain the output tokens to a valid syntax. If the model tries to output a token that violates the schema, the client (or the inference engine) should reject it immediately.

3. Execution Grounding (The Feedback Loop)

Constraint: Reality

This is the most important layer. The model’s output should never touch the user until it has passed a verifier.

In a robust architecture (like the Model Context Protocol), the tool execution happens on a server. This server acts as the verifier. It doesn’t just run the code; it validates the intent against the schema before execution.

If the validation fails, you don’t crash. You return the error to the model as a payload. This allows the model to “read” its own mistake and self-correct in the next turn.

1
import { z } from "zod";
2

3
const WeatherSchema = z.object({
4
  city: z.string(),
5
  unit: z.enum(["celsius", "fahrenheit"]),
6
});
7

8
async function getWeather(args: unknown) {
9
  /**
10
   * Raw MCP tool handler. We use safeParse to avoid throwing
11
   * and return an error payload for the LLM to read.
12
   */
13
  const result = WeatherSchema.safeParse(args);
14

15
  if (!result.success) {
16
    // 1. Structural Verification Failed
17
    // Return the error to the LLM to trigger self-correction
18
    return {
19
      content: [{
20
        type: "text",
21
        text: `Invalid arguments: ${result.error.message}`
22
      }],
23
      isError: true,
24
    };
25
  }
26

27
  const { city, unit } = result.data;
28

29
  // 2. Logic & Execution Success
30
  return {
31
    content: [{
32
      type: "text",
33
      text: `The weather in ${city} is 72 ${unit}`
34
    }],
35
    isError: false,
36
  };
37
}

1
from pydantic import BaseModel, ValidationError
2

3
class WeatherArgs(BaseModel):
4
city: str
5
unit: str
6

7
def get_weather(arguments: dict):
8
"""
9
A raw MCP tool handler. It parses arguments manually
10
and returns a structured response to the LLM.
11
"""
12
try: # 1. Structural Verification (Pydantic) # We don't throw, we catch and return the error as data
13
params = WeatherArgs.model_validate(arguments)
14

15
        # 2. Logic Verification
16
        if params.unit not in ['celsius', 'fahrenheit']:
17
             return {
18
                "content": [{"type": "text", "text": "Error: unit must be 'celsius' or 'fahrenheit'"}],
19
                "isError": True
20
            }
21

22
        # Success
23
        return {
24
            "content": [{"type": "text", "text": f"The weather in {params.city} is 72 {params.unit}"}],
25
            "isError": False
26
        }
27

28
    except ValidationError as e:
29
        # 3. The Feedback Loop
30
        # Return the validation error as data so the LLM can self-correct
31
        return {
32
            "content": [{"type": "text", "text": f"Invalid arguments: {e}"}],
33
            "isError": True
34
        }

Conclusion: The Pit of Success

We started this series by talking about tokens. We end it by talking about architectures.

In my experience, the journey usually looks like this: You start by being amazed that the rock can talk, spend a few months trying to “prompt” it into being a reliable employee, and eventually realize that reliability comes from the system you build around the rock.

The future of AI isn’t better prompting, it’s about deterministic skeletons with stochastic sprinkles.

It’s building architectures where the model can be wrong, and the system still works.