The bottlenecks in our brains create awareness, meaning and coherence
Does it make sense to simulate the brain in a computer to produce a human-like intelligence that’s way more capable than a biological hum? I’ve make the argument that machine minds are different kinds of minds, and we should focus on intelligent function of machine minds based on their unique advantages rather than emulating how people think.
During the discussions of uploading a human mind, I repeatedly asked myself why we were interested in doing this in the first place. The obvious sci-fi intent is digital immortality. Perhaps an expansion of the nature of being human by replacing biological embodiment with a digital embodiment that had the potential for greater bandwidth and augmentation of sensation and physical ability. We know the abilities of the human brain are severely limited in bandwidth and speed.
The processing speed and storage limits
So let’s start with the brain’s limitations. You may have seen a recent paper showing that the brain processes speech at the remarkably low rate of 39 bits per second. The paper by Coupe et al. analyzed 17 spoken languages and found that while speech rates and information density per syllable vary between languages, the overall information transmission rate is remarkably consistent at 39 bits per second.
Now, the first acoustic coupling modems, you know those ancient modems for dial-up where you put the telephone handset into cups on top, and the device used squeals and sounds, started out at 300 bits per second. Modern networks transmit information in the gigabit range, of course. An LLM like ChatGPT is capable of processing hundreds of kilobytes per second. Of course, no single human can chat at that rate, even typing at 100 words per minute.
Typing! Many of us with long histories of computer use are pretty fast touch typists, and dictation is somewhat slower. And I’ll always prefer to read or, better yet, skim a transcript than listen if my goal is getting some information rather than some idle entertainment from interesting chatter. That’s more a social benefit than one of information transfer.
The 39 bps limitation limits just the intrinsically noisy speech channel. Spoken words are ephemeral, serial, and ambiguous. Lose a word or two, and you may lose the thread of the thoughts being expressed. There has to be a certain robustness to speech and plus emotional signaling of attitude that gets put in as verbal cues. Plus, there are non-verbal cues added to speech at least in person. So we can see why it’s a social turn-taking cooperative enterprise that is actually way more information-dense than that 39 bps rate.
Other systems are clearly more capable. Dr. Edward Tufte would argue that our ability to use spatial reasoning to acquire information about relationships is the most rapid data transfer rate of all. Information density gets higher when we abandon serial narrative forms like speech and writing. How high? If you look at time to acquire a target in a crowded visual field or characterize a scene in a photograph, the rate is probably up to a few thousand bits per second. But no greater. We have to admit that our biologically based processing system is way slower than our current digital one.
Physical contraints
Some constraints are physical in biological systems. For example, the speed of transmission using ion-based conduction along leaky axons puts a limit on how fast information can move from one part of the brain to another, from the toe to the sensory cortex many feet away inside the skull. Clearly, faster is better in the competition for resources. So we see evolutionary solutions to speed conduction. There’s the increase in axonal diameter as seen in the squid giant axon I mentioned last week. Vertebrates developed myelin sheathing of axons which acts as insulation and speeds transmission by allowing the action potential to jump from node to node. Faster is better, but there are limits to the physical system. Evolution never came up with the idea of using copper wires and nanometer spacing to really accelerate processing speed. Well, evolution came up with humans and their technology which did surpass biological speed limits by orders of magnitude.
But it’s not just speed, it’s storage capacity. Famously, George Miller, a foundational figure in cognitive psychology, got interested in human information processing. He took ideas like Shannon’s analysis of signal and noise seriously for human cognitive processing. I’m happy we’re coming full circle here in our thinking. In 1956, he published the famous seven plus or minus two paper on memory as a limited buffer. Now we understand that it’s a real limit based on the architecture of the prefrontal cortex and posterior parietal cortex.
By the way, I personally have a pretty solid 6-item limit. I’m great with Microsoft’s and Shopify’s 6-number codes when I need to transfer them from one screen to another. But I’ve never been able to do with 7-digit phone numbers unless I can chunk the area code to one item, the 3-digit prefix to another because it’s common in my area, then the 4 numbers. Unless, of course, the final 4 digits are chunkable as, say, 6000. I do know Jenny’s number from pure repetition and can sing Tommy Tutone’s song to myself: 8675309.
Missile Command and Tetris
But when I think of cognitive bottlenecks in real life, I think of video games. In particular, “Missile Command”. Now, if you didn’t spend hours in front of arcade games in your misspent youth as I did, this was a game of pure cognitive overload.
You had 6 cities to defend from incoming threats with 3 missile bases at the bottom of the screen. There was a trackball to aim and then three buttons to fire, one for each of the bases. The missile was launched by pushing one of the fire buttons. When it reached its target position, it would explode above the cities and take out any incoming missiles close enough to its kill range. As with any of these games, it starts out positively leisurely so you scan the top of the screen until you see a threat coming in, you place the targeting cursor in its path and press fire so that by the time the threat reaches the target spot your missile has also gotten there, explodes and takes out the threat. Clear all the threats for a round and advance to the next, faster level.
As the game gets faster it becomes a game of prioritizing threats, rapidly switching between threat detection, aiming and firing then cycling back to threat detection. At the highest level, the task of course becomes impossible as you reach a cognitive limit of how fast you can switch between the tasks. The hardest part is when your targeting fails to be timed or aimed to take out the threat and when you scan you have a threat very close to destroying one of your cities.
The military knows this as the OODA loop, described by John Boyd: observe ? orient ? decide ? act and repeat. In missile command, this is scan for incoming threats (observe), identify the most urgent (orient), pick the right spot to aim at and which base to use (decide), then place the cursor and fire (act). As you play the game, you learn that the worst thing to do is think about which threat and which base or to watch the outcome of your action. You have to move on to looking for the next target before the outcome of your action is determined.
Maybe you’ve played Tetris. It’s a similar setup of assessing your current array of tiles, the shape and location of the new incoming piece, then the more difficult task of mental rotation and optimal destination of the tile. But it’s the same struggle against cognitive bottlenecks as the game speeds up.
Unlike Missile Command, Tetris has competitions, and certain individuals can play much faster than most of the rest of us. As you might imagine, it’s not just reaction time, it’s speed at visual processing, particularly linking shape to rotational possibilities. Tetris also has the added challenge that you need to plan ahead to be really good, so you have to prepare motor plans in parallel, where you select but then chunk motor moves without going back and replanning. Experts in the flow lose explicit thinking about moves; they “just know” what to do next at every moment. There’s no self-monitoring in the sense of metacognition or inner speech, and of course, time slows down. Subjectively, of course.
Exploring bottlenecks in the lab
Just like language, these non-verbal games of perception and motor activation are too complicated to break down as cognitive bottlenecks to judge speed. But in the lab, the speed of these operations can be measured. And let’s not call them “mental calculations”. They are done outside of awareness, and in fact, self-monitoring only slows them down.
One of the biggest constraints is Multiple Object Tracking (MOT). Most people can only track 3 or 4 objects simultaneously. So there’s a tight limit to the sequencing of targets you can achieve playing these games. We can actually identify the brain network responsible for the task by imaging volunteers with fMRI. This is how the dorsal attention network was identified, consisting of the frontal eye fields directing attention and the posterior parietal cortex to track objects. I should mention that the rest of the visual system is doing its part from V1 detecting shape to other areas specialized for other visual cues, but we identified the areas specifically involved in object tracking. And interestingly, some of these abilities are lateralized, that is to say that some tasks are specialized to the right or left hemisphere. So we nicely multiply ability with further lateralized specialization of the cortex.
In Tetris, mental rotation is key. At least when I’m playing as a rank amateur. Looking into the literature going back to the 70s, it’s surprisingly slow. If you ask people to pick out a rotated block figure or a mirror image, it takes a few milliseconds for every degree of rotation. So that’s like 60 degrees per second or 3 seconds for mental rotation through 180 degrees. It’s as if rotation is not instantaneous like your drawing program; it’s a continuous analog translation through 2D space in the brain. No wonder I’m so bad at Tetris.
Of course, if you play Tetris long enough or deal with rotation of common shapes, other mechanisms like pattern recognition come to your rescue. Seeing an upside-down man is really fast because we have context-dependent learning about that kind of shape. We understand the normal body position of the head up and feet down. We understand what are often called “affordances” like a pull shows how to open a drawer or our many conventional symbols like pointing arrows. So we can short-circuit the analog part of mental rotation when looking at an upside-down human figure. And that’s why if you play Tetris enough, you eventually work out the patterns of how a straight piece is rotated vs an T-piece. These are templates that we use to avoid bottlenecks.
The multitasking limitation
Perhaps the biggest bottleneck compared to our computers is our inability to multitask. Attention is a single-threaded, serial process in playing these games. In Missile Command, I can scan for threats, I can move the trackball, I can fire, but only one at a time. I’m not thinking or directing my activity with awareness, but still, I can’t scan while aiming. You learn quickly to give undivided attention to the task at hand, then switch as quickly as possible. Once I select an action, I can’t move on to selecting the next until the action is completed.
It goes deeper than that. I think I can see everything at once on the game screen, but this is an illusion, and it too is built up sequentially. We only have a tiny region of sharp visual acuity at the macula. It’s just a degree or two. Four or five times a second, our eyes move in short, quick jumps to a new area of interest. These saccades build up a scene that we perceive, but just like motion in a movie is an illusion from rapid sequential presentation of frames, our visual world is built out of tiny regions of interest. The rest is image decay, working memory, predictive models, and context that convinces us of a real, stable world out there.
At some point in these games, the task just outruns the serial ability to run through the OODA loop, and we fail. After all, even the simplest, slowest microprocessor that was ever built can run more tasks faster than the best human brain. And by orders of magnitude. And as computing has become more capable, with more storage, faster storage, and massive parallelism.
All just to once again make the point that no one would think that the way to reach human-level intelligence for any particular task would be to do it the way humans do it. Programming a world-champion Tetris or Missile Command player was a relatively easy task. Tetris bots like StackRabbit are a few thousand lines of code with optimized search heuristics and look-ahead algorithms. You don’t even need fancy machine learning, although of course, it’s been done. Just the sheer speed of a computer beats any human by orders of magnitude.
The best human can place 500 or 1,000 pieces in a game lasting 5 or 10 minutes. A bot could play forever if allowed. At the fastest usual game rate of 60 frames/second, a million pieces would be a game of about 4 1/2 hours.
So again, even if you want to emulate human ability at games, at recognizing visual scenes, at just about any task, it doesn’t make sense to start by emulating how the brain achieves these tasks, at least from a speed or bandwidth perspective. I heard Sam Altman of OpenAI say that these new AI capabilities “make me feel small”. I know what he means. It seemed a blow when it was clear that even relatively simple look-ahead search programs could beat the best human chess players. Then, of course, we had ML algorithms able to beat us at Go, an open, unstructured game.
The purpose of single channel attention
I think it’s clear why our processing is limited when it comes to speed, bandwidth, and multitasking. Clearly, speed is ultimately limited by conduction velocities in axons getting information from one area of cortex to another. It looks like connections are about as dense as possible. I’d venture to say that some of the storage constraints like the limits on working memory or object tracking must be more or less due to functional tradeoffs.
But why is the attentional system single-channel? Why not allow for true multitasking? I was reminded just this morning that I can’t listen to a podcast and read an email at the same time. There was a gap in the podcast that was clearly due to interference in the language system. It seems natural that language processing would be a single stream. But it’s true I also can’t fully listen to a podcast and use the visuospatial system to scan a scene, like finding the striped red shirt of Waldo in a complex, ambiguous visual scene. This seems to be an inherent limitation in the central executive system where we have to focus on one task at a time.
Experimentally, there is a single-channel limitation in response selection, semantic integration, object recognition, and working memory gating. This is why when we test short-term memory, we use a distracting task like subtracting 7 from 100, subtracting 7 from that number (serial subtraction) to wipe the working memory, putting the burden on the hippocampus and short-term memory.
My suspicion is that this single-threaded feature is necessary for the kind of awareness we experience. Even though we subjectively perceive a stable world out there, the mysterious coherence of experience is actually painted bit by bit. Our visual system scans the world with the macula just a degree or two at a time. We don’t see the entire visual world, but we experience it. We interact with the world in a similar piecemeal manner when the executive system is in control during a task like playing Missile Command or Tetris.
We do have systems for parallelism that are older evolutionarily and specialized for automatic perception and action like the basal ganglia and brainstem. This is how we walk, chew gum, and talk about brain science simultaneously. How we hear our name called when reading a book. The sound is processed, but brought into awareness only when needed, interrupting the ongoing processing stream. Only current attention and action are controlled by the brain’s executive system; the rest is automated, preprogrammed motor control, outside of awareness. Somehow we need to avoid conflict to achieve integration in conscious awareness.
I wouldn’t trade my slow, limited human cognition for the multitasking of a machine mind. I believe that what makes us slow is the very essence of experience of being aware of the world, its significance, and our feeling about what happens. Because we can only experience one thing at a time, we are forced to live in the world in continuity. Maybe machines can handle everything at once with efficiency and speed, but without experience, there’s no point of view. There’s nothing but shifting bits.
