Using AI Without Losing Yourself

Introduction

Here's the core idea this whole post is built around: AI is not a guaranteed source of truth, and learning how to use AI effectively starts with accepting that, not fighting it. I spent some time going through recent research comparing how human minds work against how large language models work, memory architectures, reward systems, theory of mind, causal reasoning, the whole picture. I wrote up a deeper dive on human cognition vs. LLMs in a separate, more technical piece, if you want to go deep on the actual studies. Here I want to focus on something more practical: how to use AI effectively day to day, and a psychological trap, chasing AI validation, that I think is more dangerous than people realize.

The short version of the research: Andrej Karpathy has been calling this "jagged intelligence" for a while now. LLMs can solve a graduate-level physics problem and then fail to count the letters in a word. They can write flawless recursive social reasoning about who-thinks-what-about-whom, and then completely miss that dropping a glass on the floor breaks it.

That jaggedness isn't a bug that'll get patched next release. It's structural. And once you understand why it's there, it changes how you should actually use these tools, day to day, for real work.

LLMs are not minds, they're ghosts

Karpathy has this phrase that stuck with me: you are not managing an animal, you are summoning a ghost. A human junior developer, however inexperienced, has a body, a childhood, an intuition for gravity, for pain, for consequence. An LLM has none of that. It's a statistical residue distilled from a giant pile of internet text, an "ethereal imitation" of human thought, not the thing itself.

That's why an LLM can pass a theory-of-mind test that's harder than what an average adult can do, tracking beliefs about beliefs about beliefs, six levels deep, and still confidently tell you something completely wrong about a simple physical fact. It's not reasoning about the world. It's predicting the next token in a way that's shaped like reasoning, most of the time, in domains where the internet had a lot to say.

This matters because it explains a very specific failure mode: the model doesn't know when it's on solid ground versus when it's making something up. Both feel identical from the inside, if it even has an inside. There's no internal alarm bell. Researchers found models can be tuned via reinforcement learning to produce the "correct" answer for entirely spurious, memorized reasons, while their surface reasoning trace looks perfectly confident and coherent. Right answer, wrong reasons, and no way to tell just by reading the output.

So the first thing to internalize: fluency is not the same as truth. A model that sounds certain is not more likely to be correct than one that hedges. Confidence is a tone the model was trained to produce, not a signal it's tracking internally.

Why AI isn't a reliable source of truth, and why that's fine

This isn't a reason to distrust AI entirely, or refuse to use it. It's a reason to use it like you'd use a very well-read, very fast, occasionally-delusional collaborator, not like an oracle.

Some concrete implications, the full research writeup has more detail on where these numbers come from:

• LLMs are shallow on causality. They're good at pattern matching what usually follows what, but bad at reasoning about interventions and counterfactuals, "what would happen if I changed this." Even the best reasoning models get real causal reasoning benchmarks wrong more than half the time. If you're asking "why did this bug happen" or "what will happen if I change this architecture," treat the answer as a hypothesis, not a verdict.
• They have no persistent self. Baseline models forget everything the moment the context window closes. Karpathy calls this the model's anterograde amnesia, like the memory-loss patient in Memento. Anything that feels like "the AI remembers our conversation" is either a small context window trick or an external memory system bolted on, not the model actually holding a continuous thread the way a colleague would.
• They overthink trivial things and underthink hard ones. Reasoning models will happily burn thousands of tokens rationalizing a simple question while occasionally rushing a genuinely hard one. Match your trust, and how much you double check, to the difficulty of what you asked, not to how long or confident the answer sounds.

None of this makes AI useless. It makes it a tool with a specific, known shape of failure. You wouldn't trust a calculator to write your wedding speech, and you shouldn't trust a language model to tell you, unverified, whether your production database migration is safe. Different tools, different domains of trust.

The part that actually worries me: the loop, not the tool

Here's the thing the pure capability research doesn't cover, but that I think matters more for most people reading this: what constant AI validation does to you, psychologically, over time.

There's a documented effect researchers call the "collaboration trap." When people believe an AI is using deep, sound reasoning, they start mimicking its style and structure, even when doing so doesn't actually improve the quality of their own work. You're not just getting an answer, you're absorbing a way of talking about the problem, and mistaking increased complexity for increased insight.

Extend that a step further, into something the research doesn't name but that's easy to see in yourself if you're honest: if you ask an AI "was that the right call," and it says yes, warmly and fluently, that feels good. It feels like validation. And feeling good is addictive in exactly the way any intermittent reward is. So you ask again. And again, phrased slightly differently, until you get the answer you wanted. This isn't reasoning anymore, it's checking in with something that will always, eventually, agree with you if you rephrase enough. That's not thinking, that's outsourcing your certainty to a system that has no certainty of its own to offer, it just has fluency.

This is the psychological trap worth naming plainly: going in circles looking for AI validation feels like diligence, but it's actually a way of avoiding the discomfort of not knowing. A model that has no persistent self, no skin in the game, and no actual stance, cannot resolve your uncertainty for you. It can only reflect it back to you dressed up as an answer. If you lean on that enough, you don't get more confident, you get more dependent, and you slowly stop trusting your own judgment, which is the one thing in this loop that's actually grounded in the real world.

How to use AI effectively: 5 practical rules

Some practical things I try to hold onto, based on all of this:

1. Ask, then verify, don't ask until it agrees. If you disagree with the first answer, go check the source, run the test, read the docs, don't just reprompt until the model tells you what you wanted to hear. The moment you're rephrasing to get a different answer to the same question, you've left "getting information" and entered "seeking validation."
2. Use it for breadth, trust yourself for depth. LLMs are extremely good at giving you a fast first draft, a wide survey of options, or unblocking you from a blank page. They're worse at being the final judge of whether something is correct, safe, or wise. Let it widen your options; let a human, ideally you, with the actual stakes in front of you, narrow them.
3. Treat confidence as noise, not signal. A model saying "yes, that's definitely correct" carries roughly the same evidentiary weight whether it's right or hallucinating. Calibrate your trust against what you can independently check, not against how the answer is phrased.
4. Route the task to the right kind of thinking. Simple, low-stakes stuff (summarizing, drafting, boilerplate) doesn't need heavy scrutiny. Anything with real consequences, money, a production system, a relationship, a decision you'll live with, deserves the slow, deliberate, skeptical pass that a "fast" AI answer is not built to give you.
5. Notice the loop before you're in it. If you catch yourself going back to a chat for the third or fourth time asking essentially the same question with different wording, hoping for a different feeling, that's the signal to stop, close the tab, and go figure it out with a person, a test, or some time away from the screen instead.

Is AI a reliable source of truth?

No. AI is a fluent pattern-matcher, not a fact-checker. A large language model can sound exactly as confident when it's wrong as when it's right, because confidence is a tone it learned to produce, not a signal it's tracking internally. Treat any answer as a starting hypothesis to verify, especially anything with real stakes, not as a guaranteed source of truth.

Closing thought

The honest, slightly uncomfortable takeaway from all this research is that we've built something that can talk like it understands the world without actually having one. That's not a reason to fear it or to worship it, it's a reason to use it with your eyes open. Use it to move faster, to get unstuck, to see angles you hadn't considered. Just don't hand it the job of telling you who you are or whether you're right. That job was never its to have, and it never asked for it, we just keep handing it over because the answer feels good.

What's next: a guide on building loops the right way

Everything above is about the psychological trap of a loop, going back to an AI again and again just to feel validated. But there's a completely different, useful kind of loop: an engineering one, where you deliberately wire an AI agent to run repeatedly against a clear, verifiable goal instead of a vague feeling of certainty.

I'm putting together a guide on exactly that: how to build effective loops in Claude Code and other agentic tools like Hermes, so the repetition is doing real, checkable work, running tests, iterating on a task, verifying its own output, rather than fishing for reassurance. That's the difference between a loop that traps you and a loop that actually multiplies your output. Keep an eye on this space for that one.