Summary  
The chapter outlines techniques for validating AI-generated programming advice by spotting plausible fabrications, verifying code by running it, checking official documentation, and prompting the model to argue against its own recommendations.

General domain of usage  
AI-assisted software development tools

## Fluency is not accuracy

Claude's most dangerous output is not the obviously wrong answer. It's the answer that
is 95% right, delivered in the same confident register as everything else, with one
invented detail in the middle.

There is no tone change when a model is wrong. It does not hedge more, slow down, or
sound uncertain. **Confidence carries zero information about correctness** — and that
is genuinely counter-intuitive, because with humans confidence is a real signal.


**Calibration**

The match between how confident a claim sounds and how likely it is to be true. A
well-calibrated source hedges when uncertain. Language models are poorly calibrated in
their prose — they sound equally sure whether they're right or wrong — which is why the
checking has to come from you rather than from their tone.


Definition

## Three tells

Learn to feel these as friction.

**1. The method that should exist**

`df.smart_merge()`, `array.sortBy()`, `str.titlecase()` — names so reasonable you
assume you forgot them. Plausible naming is exactly what a language model is best at.

**2. The over-specific citation**

"As shown in Chapter 7 of *Designing Data-Intensive Applications*, page 214..."
Precision is cheap to generate. The more exact a reference is, the more suspicious it
should make you.

**3. The confident version number**

"In pandas 2.1 this was renamed to..." Anything about *current* state — versions,
defaults, deprecations, pricing, who runs what — sits past the knowledge cutoff and is
guesswork dressed as fact.

Also watch: exact statistics with no source, quotes attributed to real people, and
anything about very recent events.
## Three moves

Match the move to the claim:

- **Code, syntax, method names** → **run it.** Seconds, and definitive.
- **API behaviour, arguments, defaults** → **official docs.** Not a blog, not Claude.
- **Reasoning, architecture, trade-offs** → **ask it to argue against itself.**
- **Facts, numbers, current state** → **an independent source.** 

Two, if it matters.
The third move is underused:

```text
Now argue the opposite of what you just told me. Give me the strongest case
against that recommendation, and tell me under what conditions I should
ignore your original advice.
```

If the counter-argument is strong, you were handed one option and told it was the
answer. If it's weak, your confidence is now earned rather than borrowed.
### The pre-emptive version

Ask before you're misled:

```text
Answer my question, then add a section: "LOW CONFIDENCE — VERIFY THIS."
List every specific name, number, version or citation in your answer
that I should independently check before relying on it.
```

## Build your prompt library

You've now met about a dozen prompts worth keeping. Don't rely on remembering them.

Make one file — `prompts.md`, in your notes app or repo — with sections:

```
## Understanding      → the Explanation Ladder (Ch3)
## Stuck on a bug     → escalating hints (Ch4)
## Testing myself     → Feynman flip, question generation (Ch5)
## Starting a project → spec + milestones (Ch6)
## Reviewing my code  → named-rubric review (Ch6)
## Checking an answer → argue-the-opposite, low-confidence list (Ch7)
```


**Organise by problem, not by feature**

Sections named after the situation you're in (`stuck on a bug`) get opened. Sections
named after a technique (`Socratic prompting`) don't, because when you're stuck you're
thinking about your problem, not about terminology. Add a prompt whenever one works
unusually well; delete ones you never open.

Within a month it'll be worth more to you than this course, because it'll be shaped to
how *you* actually get stuck.

Study More

Master the skills needed to turn any lecture, PDF, or course into knowledge that
survives the week. You will build prompts that explain at your level, retrieval
loops that quiz you instead of lecturing you, flashcards that test recall, and a
verification habit that catches confident mistakes. Every technique runs on
material you already have to study, so by the end you have a weekly system
instead of a pile of chat history.

Verify Before You Believe

Fluency is not accuracy

Three tells

Three moves

The pre-emptive version

Build your prompt library