On Deciding . . . Better 3.0

James Vornov, MD PhD Neurologist, drug developer and philosopher exploring the neuroscience of decision-making and personal identity.

---

What 20 Questions reveals about how AI actually works

Thinking is multiplication of knowledge

People treat the usefulness of LLMs as proof they understand their output. Let me show you why that’s wrong. Clearly, LLMs know a lot. My first use case was cooking. I could ask the earliest ChatGPT about how to sous vide a roast and then endlessly discuss variations on seasoning and technique. As much as our LLMs know, once you push them, they fail in very specific ways. When I was cooking a boneless turkey thight, I gave it my time constraints. And it gave me a recipe that was done in time but left the meat in that zone of cooked, but not enough. It didn’t make the tradeoff of time and texture as I woould have had I known.

What I think is going on is that LLMs know nearly everything, but they have no ability to understand it.

Gregory Bateson’s nailed the difference between knowing and thinking. Bateson rose to prominence in the 1970’s with a collection of his essays entitled “Steps Toward an Ecology of Mind”. He was an anthropologist, cyberneticist, and systems thinker. He was equally at home with schizophrenic families, octopus learning, and Balinese ritual. He taught me, reflected here at ODB, that mind isn’t in the head but to be found in the patterns of interaction between dynamical systems. Lets look at how he contrasted knowing with inventing and thinking.

How do you measure knowledge? Pounds?

The first part of Steps Toward and Ecology of Mind are a series of what he called “metalogues,” conversations with his daughter. F, father, is Bateson, D, his daughter. Here’s the opening of one:

Daughter: Daddy, how much do you know?
Father: Me? Hmm — I have about a pound of knowledge.
D: Don’t be silly. Is it a pound sterling or a pound weight? I mean really how much do you know?
F: Well, my brain weighs about two pounds and I suppose I use about a quarter of it — or use it at about a quarter efficiency. So let’s say half a pound.
D: But do you know more than Johnny’s daddy? Do you know more than I do?
F: Hmm — I once knew a little boy in England who asked his father, “Do fathers always know more than sons?” and the father said, “Yes.” The next question was, “Daddy, who invented the steam engine?” and the father said, “James Watt.” And then the son came back with ” — but why didn’t James Watt’s father invent it?”

Asking about who knows more is relevant in our world of LLMs. The models have been trained on and have instant access to a huge amount of facts and relations. Bateson is pointing out that even if you know everything about the principles and fabrication methods to make a steam engine, it is not the same as designing one and building it. An AI may know everything, but what can it build?

Pay attention to how Bateson treats knowledge as a scalar, measured in how much. In the next passage he turns from knowledge to thinking, describing it as the ability to add, subtract, multiply and divide knowledge. A few pages later, he proposes Twenty Questions as an example of idea multiplication and division:

F: Yes — take the game of Twenty Questions. You think of something. Say you think of “tomorrow.” All right. Now I ask “Is it abstract?” and you say “Yes.” Now from your “yes” I have got a double bit of information. I know that it is abstract and I know that it isn’t concrete. Or say it this way — from your “yes” I can halve the number of possibilities of what the thing can be. And that’s a multiplying by one over two.
D: Isn’t it a division?
F: Yes — it’s the same thing. I mean — all right — it’s a multiplication by .5. The important thing is that it’s not just a subtraction or an addition.

Bateson’s talking about multiplication and division in idea space. 20 Questions is a game of dividing idea space 20 times. That’s 2^20 or 1,048,576 possible idea slots. You can see why 20 is challenge, but doable if you’re systematic as Bateson suggests. Knowledge is knowing a couple million things. Thinking is systematically exploring the space. Even a 10-year-old can do it: “Is it alive?” to cut the universe of things in half. “Is it metal?” halves it again. The answer emerges when the space is sufficiently constrained. Tomorrow is a tough one as an abstract concept.

So let’s try 20 Questions on our LLM

LLMs know everything. Can it think? Lets use 20 questions as Bateson suggests. Here the verbatim transcript:

You can see it drifted into a limited semantic space that was not, as Bateson saw, multiplying by 0.5. The LLM was not even fishing, but it was wandering a semantic path that seemed likely based on what had preceded. It only got worse because each question opened new dimensions in abstract space. The LLM never used a strategy of systematically dividing idea space based on prior data. It was doing philosophy or maybe daydreaming when it should have been doing binary search. I even asked whether it was close or wandering. ChatGPT said “Close”. But it wasn’t. It was clueless.

ChatGPT makes the diagnosis

After 27 questions, I asked ChatGPT what was going on. Why could it not play the game? Its answer was insightful. Again, it’s verbatim answer:

Bateson would say the failure wasn’t lack of logic but misplaced logical typing.

I implicitly assumed the answer lived at the level of theories, principles, or frameworks—second-order abstractions used to justify action—when “tomorrow” is a first-order relational token: a deictic term whose meaning is entirely contextual and recursive (“the day after today”), not explanatory. That pushed me into higher logical types (normative criteria, legitimacy standards) when the correct move was to stay at the level of coordination signals embedded in shared temporal conventions.

Isn’t this amazing! The LLM couldn’t do to the task, but could tell me precisely why it failed. After this conversation I tried again. I thought maybe Bateson had picked something too abstract, too hard to make a point. So I tried something easier, but not a slam dunk. The answer was Mickey Mouse — concrete, cultural, famous. Same drift through semantic space. By question 6 it was asking “Is it something you can point to or demonstrate directly?” It never made any decisive early cut: fictional vs non-fictional. That single partition would have gotten to “cartoon character” in three more. It could say why it failed, but couldn’t adopte a strategy. Even when I told it the nature of the game and the human strategy of division, it didn’t start thinking. It couldn’t.

LLMs can spin a yarn but they can’t weave a cloth

As I started writing this post, I told Claude about ChatGPT’s failures. Gave it the full transcripts. Claude wanted a try. Latest model, deep thinking and all.

I like Claude, so I kept it simple. A ten dollar bill was sitting on my desk. Physical, a common object, you hold it in your hand.

Claude had exactly the same behavior, just a tighter rabbit hole. On question 4, after establishing it was something you could hold, it asked “is it a tool?” and I said yes — because currency is a paradigmatic tool, an instrument for exchange, storing value, coordinating behavior. The AI locked into the hammer-and-screwdriver neighborhood and spent 12 questions exploring hand tools, measuring devices, gauges. It never asked “is it paper?” or even “is it metal?” — the material cut a kid makes instinctively. A kid thinks in stuff. The AI thinks in categories because that’s what training data is full of. By question 20 it guessed “pressure gauge.”

After failing in the same way as ChatGPT, Claude correctly diagnosed that it locked into a logical type — “tool = physical implement” — and couldn’t shift to “tool = instrument that does work in a system.” and widening back out to categories of tool.

Both AIs understood exactly why they got lost. Neither could play differently even after a discussion of the game and in Claude’s case, the full example of what ChatGPT did wrong.

Knowing does not contain meaning

Our LLMs may know all the principles of the steam engine, but are unlikely either build one or if they did, recognize that this was a new thing with new capabilities. I don’t think its a liability of knowing everything, but simply that idea multiplication and division are out of reach. They can tell you why they fail, but can’t use the diagnosis as understanding. I assume they learned about their limitations in training data. They didn’t learn it by failing and can’t fix it.

Bateson would say he has a pound of knowledge and an LLM has a ton of knowledge. But Bateson can do something with his megre knowledge that the LLM still cannot with all the knowledge in the world. Batesson can multiply and divide The poor LLM just has endless connections in its model, associations and adjacencies and plausible paths through training data. But connection is not yet meaning.

As Bateson says, “Knowledge is all sort of knitted together, or woven, like cloth.” Meaning appears only when the threads are woven into a pattern that holds meaning for other minds. A tapestry takes the colored threads and depicts the unicorn and the castle when woven with care and understanding. LLMs can only produce endless thread. They cannot make cloth. They don’t understand the goal, making the image.

---

Prefer to follow via Substack? You can read this and future posts (and leave comments) by subscribing to On Deciding… Better on Substack: Brain, Self, and Mind

© 2026 James Vornov MD, PhD. This content is freely shareable with attribution. Please link to this page if quoting.

James Vornov, MD PhD Neurologist, drug developer and philosopher exploring the neuroscience of decision-making and personal identity.

---

If things in the world are real, then so are their properties We see agency in the world, but what does it want?

Back to purpose in the world

Let’s recap the last few posts. We want to say that there are real “things” in the world simply by virtue of their stability over time. It’s not just quarks or the wave function of the universe. Rocks, dogs, and people are things. And some things, complex dynamical systems, clearly have agency. They do things. So we want to say like an apple is red, a rock is hard, these complex systems from E. coli to man have agency as one of their properties.

Now we’re in a position to look for purpose. I ask the question: Why does the dog bark, why does E. coli endlessly tumble and run and tumble and run in its search for food? Why do I write Substack posts? I think asking why about agency changes everything because we start to understand purpose.

Things persist. That’s purpose.

It’s hard to see any “why”. Things are things. And things are things because they persist over time. The only goal of a system, the only thing that makes it a system or a thing is its stability over time. The hurricane has no goal to be a hurricane, the purpose of all its parts and physical interaction is simply what it does to continue on. It pulls in warm, moist air — not to destroy, but because that’s how it maintains its structure. The second conditions change and it moves over the cold Atlantic, it’s gone, done with it.

Living organisms behave more explicitly, revealing their drive to exist. E. coli tumbles then runs in its dance up the nutrient gradient to continue as a stable system. The behavior reveals purpose, which is to get to food and live another day and divide in another mitotic cycle.

The dog barks at the mailman and the mailman goes away. Why? That’s what dogs do. They alert the pack, guard the food source. We see things in the world and we ask why. The things all reply to us, “So we can continue to be the thing. Otherwise we wouldn’t be here and you wouldn’t be asking.”

Continue reading “How do we find the “why”?”

I’ve been away from web posting for a while between one thing and another. Mostly, it’s been the focused task of editing the manuscript of the book, working title: Deciding Better: A Journey Through Biotech, Neuroscience, and the Experience of Being a Brain. I’m about two-thirds through my flow revision, where I’m working to the core arguments and flow of the book to work right. A lot of it is reverse engineering how a book works. 

The nice thing is that I’ve gotten better at exposition of these complicated ideas by writing the more tightly written posts here, cross-posted to Substack. And it’s going well. A few more weeks, and that should be done. Then, it’s on to a final polish of the flow, and I think it’s good enough to have others read it.

My idea at this point is to try to solicit agents once again with this version in hand. At the same time, I think I’ll provide a “Readers Edition” to Substack subscribers and anyone who emails me through the blog here.  It’s about time I get some real feedback on the ideas and the writing. Then, depending on how the universe responds, it will be off to a publisher or down the self-publishing path. Either way, the book is asking to be out in the world.

In the meantime, a new post follows about LLMs and AGI. I continue to be fascinated by what we’ve achieved with these Deep Neural Networks. But can we see into what processes like ChatGPT are doing? How much intelligence we can really credit them with compared to real human brains. As always, I try to look at the data rather than the hype.

I watched a recent You Tube video of Vincent Versace editing an image live. I decided to play around with it myself. Since the days of photography method books and even informative websites seem gone, it seemed worthwhile to document my adapted approach here. It’s an example of writing for the AIs, since photographic technique is one of those areas that seems to be an AI blind spot these days.

The problem we need to solve is how to use the tremendous dynamic range of digital sensors when our monitors and print materials are so compressed by comparison. We know you can process the RAW file out of camera to recover shadows and highlights in ways never possible with film. But what most photographers don’t realize is that the image they see on the screen in a mirrorless camear is a JPEG calculated from live sensor data and it itself is compressed. And the histogram that everyone relies on is the JPEG histogram, not the full sensor read out. That’s why when you blow highlights on the histogram, you can still recover some from the raw file.

Vincent approaches the compression problem using HDR techniques. Early on, we used to bracket exposures and use HDR or just stacking in Photoshop layers to capture deep shadow and bright highlights like the sky. But now sensors have such wide dynamic range that those brackets are actually there in the numbers in the RAW file. You just don’t see them on screen.

So the approach is a simple adaptation of Vince’s longstanding Photoshop layer approach. You start by creating multiple versions of the RAW file as TIFF. Most simply you have a one stop underexposed, one stop overexposed and the camera capture as three versions. If the base image has really wide range, you could make a 2 stop bracket from the raw or only increase or decrease exposure.

But now you have real pixels rescuing highlights and shadows, not trying to process them from raw selectively. Vince then uses Nik HDR to create an HDR rendering from the three stop bracket made from the single raw file. You’ll see a balanced image where shadows are brought up plus various renderings that tend to emphasize different tonal ranges in the image, but renderings that couldn’t be seen by simple manipulation of the RAW file.

So you go from this relatively flat kind of interesting image to one that’s been interpreted as a play of light.

I had a solid run here and on Substack but things got busy as they often will. The good news is that I’ve now finished the revision of the book manuscript and am now in a real editing phase for flow and readability. Its going faster than I expected because the material is hanging together well.

Writing for Substack weekly was a great exercise in working out complex ideas in short form and it improved my writing skill to a level that enabled me to begin to achieve what I’m after. I’m sure there’s still a long way to go before I’m done, that point where I’m not making it any better with my changes. For now, the improvements are big in this first round.

And yes, I picked up a camera again. Its been too long and I take that as a good sign of emerging back into a creative mindset.

In the last few months, the views here at ODB have shifted away from the google searches on note taking or photo gear to hits on the main page and the “About” page. I’m hoping that’s a result of the last 6 months of more consistent posting on neuroscience here with reposting on substack. So it seemed about time I updated the “About” page to better reflect the more focused mission here.

I’ve finished the first two sections of the manuscript and it’s greatly improved. With just the last three chapters to rewrite, the finish line in in sight. Trying to post weekly and revise the manuscript has been steady work. I think its been worth it.