Stephen Wolfram is the kind of thinker that sees things as simply as possible. His work on emergent phenomenon, physics and metamathematics reveal how what we see as mysterious can be built up from simple parts. I’ve written about how he takes notes, a process that is more about capturing that thought process than cataloging facts.
He’s written a clear and very simple explanation of ChatGPT ,/a> that provided a fundamental understanding of the technology for me.
The first thing to explain is that what ChatGPT is always fundamentally trying to do is to produce a “reasonable continuation†of whatever text it’s got so far, where by “reasonable†we mean “what one might expect someone to write after seeing what people have written on billions of webpages, etc.â€
It had an unfortunate side effect for me last week. I was about to get up and give a lecture a faculty member for the NINDS sponsored course, Training in Neurotherapeutics Discovery for Academic Scientists, when I suddenly realized that I myself am a Large Language Model, knowing that I would soon stand up and for an hour allow my left temporal and frontal lobe language areas to chose words sequentially to go along with the planned talk and accompanying slides. I would be aware of listening to what I was saying and providing oversight, but those language areas would just be generating word strings autonomously.
Of course once I said good morning, all I could think about was what I was saying, so all good in the end.