AI Reading: David Deutsch, AI and Personhood, Explanation, Good Science Fiction

Original from the Naval Podcast, transcript here You can listen to the audio from the link above as well.

Naval: We don’t really have an agenda. There is no goal to the conversation. The closest we can come up with is just to have a spontaneous free flowing talk about anything you want to talk about. Obviously, you know how everyone thinks of your work now, it’s becoming more well known. And I know you’re too modest to acknowledge that. But at least for me, the most interesting piece, if it would come out, is just any wide ranging free form thoughts that you have because of the understanding that you have of your various theories and your view of the world. Maybe even feel free to talk about how that has influenced your life, your outlook on life, how you think the world ought to be a little bit different or could be better, where we’re headed—just feel free to go very wide ranging. It’s really just about whatever we want to talk about.

Brett Hall: I think I mentioned to you in a private chat that we had about the fact that we’ve had two conversations already, and some things have changed. Especially the ChatGPT stuff has been–

Naval: Oh, yeah. That is interesting. That is the most on-top-of-everyone’s-mind-thing right now.

David Deutsch: That is the biggest thing that’s happened technologically.

Naval: Should we just dive into that? What’s your latest thinking on AI, AGI, ChatGPT, super-intelligence?

David: Two big things to say. One is that fundamentally, my view is unchanged. My view about AI, AGI, and so on. But the other thing is, I use ChatGPT all the time, many times a day. And, it’s incredibly useful, and I’m still at the stage, even though I’ve had it since March, I’m still in the stage when I’m thinking, “Hmm, doing so and so is too much trouble. Oh, I could ask ChatGPT.” I’m still in that stage when I’m discovering new uses for it. I think many of them are things where I could use Google, but it would take too long to be worth it. And ChatGPT is often very wrong. It often hallucinates, or just is very sure about giving the wrong answer. And so you can’t rely on it, even slightly. Good Science Fiction is Hard to Vary

Brett: Let’s stick with ChatGPT, but first, just as an aside, you’re a big fan of hardcore science fiction. You like the good stuff. What is the good stuff and what separates the good science fiction from the fantasy science fiction, the lazy science fiction?

David: I think the best science fiction author currently is Greg Egan. Now, what is good about him? So the formula for great science fiction is supposed to be: you invent a fictional piece of science and then you explore the ramifications of it, both in science and in society. And he does that fantastically well. He puts an enormous amount of effort into getting the maths right, getting the physics right. He had one book in a universe where the signature of space-time is ++++, instead of +++-. So that means that, in a spaceship, you can travel around back in time and so on, and how do you make that consistent? How do you avoid paradoxes? And, he did it brilliantly.

Naval: Is he moving through the multiverse?

David: So he’s touched on that several times.

Brett: You didn’t mention the phrase hard to vary. But that’s a signature of–

David: That’s definitely part of it because, to be science fiction rather than fantasy fiction, there’s got to be a world that makes sense, that has laws of physics, that has a society that makes sense. Or if you’re describing aliens, the aliens have got to make sense. You’ve got to answer questions about why we haven’t had first contact—the Fermi problem.

I think probably my second favorite sci-fi author is Neal Stephenson, who is fantastic, but in a different way. He also does phenomenal research. Everything makes sense like that. But every book he writes is a different genre. I don’t know how that’s done. I mean, that just in itself blows my mind.

Naval: Have you read Ted Chiang?

David: I’ve read two or three of his short stories, including the one where there are these aliens and you get to sort of telepathy about time–

Naval: Yeah, that’s among my least favorites. That got turned into a movie called Arrival. And, the story is called Story of Your Life. But my favorite story of his is a story called Understand. And it’s a remake of the classic Flowers for Algernon story, where a guy figures out a medical ampule to make himself smarter. And what does that mean? So, obviously he starts taking it more and more and more and becomes more and more intelligent. And then he starts becoming able to program his own brain and metaprogram himself, etc. It goes into some very interesting places. But given what you understand about epistemology, I think you could take a critical look at it. And it’s a short story. It doesn’t take very long, and it’s a brilliant story. I’m going to make a note to send it to you after this, it’s easy enough to find, but he reminds me of, if you’ve read Borges.

David: No, I haven’t. Everybody tells me about Borges.

Naval: Borges is brilliant as well. Can I send you a Borges story as well?

David: Okay.

Naval: Borges is more fantasy. But, again, Borges likes to play games with time and infinity. Very often, his protagonist will change one thing about reality and then follow it to its logical conclusion in every possible way.

David: So, that sounds like sci-fi rather than fantasy.

Naval: Borges is genre-less. It’s very hard to pin him down in genre. It’s similar to Stephenson. Stephenson varies across books, Borges within the same story will cross genres. They’re short. That’s a virtue. ChatGPT is Not a Step Towards AGI

Brett: In terms of taking an injection to make yourself smarter, taking us back to ChatGPT, is it getting smarter? Would you use that word? Is it getting more intelligent?

David: It never was intelligent. I only saw 3.5 and 4. And version 4 is a little better than 3.5. Now there’s a bunch of plugins, they haven’t really worked for me. So, I’m just using ordinary ChatGPT-4. I can’t quite fathom why people think it’s a person. It seems to me completely unlike it in every way. It’s a phenomenal chatbot. I thought it would be decades before we had a chatbot that good. With hindsight, it’s a bit surprising that chatbots have not improved incrementally, and maybe the sudden improvement is what bowls people over and makes them think they’ve crossed the threshold or something. I don’t see any threshold. I see an enormous increase in quality. Just like changing to an electric car. Suddenly you’ve got all the acceleration you could ever dream of.

Naval: Do you think these models understand what’s going on underneath? Is there any understanding inside?

David: No, none. They don’t understand what they themselves have just said. They certainly don’t understand what the human says to them. It’s a chatbot. It’s responding to prompts. That’s what it’s doing. And if you’re very good at making the prompts, which I’m not yet, so maybe I’m underestimating it, but the better you are at making the prompts, the more it will tell you what you wanted to know. For a complex question, it usually takes me two or three goes, and to correct it. And, sometimes it just won’t correct it.

For example, just yesterday, I asked it to produce a picture with the DALL-E plug in. I thought there’s a picture that I had wanted for my book, but which I couldn’t really get an artist to draw, but if I had my previous book again, I would want a picture of Socrates and the young Plato and Socrates’s other friends all sitting around. And I said, “Make me a photorealistic picture of that”. So it made a black and white picture. And I thought, “Hmm, okay, I can’t say that’s not photorealistic, but I meant color photorealistic”. It had Socrates sitting in a sort of throne and everybody gathered around him. So I said, “Put Socrates down at the same level as everybody else. And by the way, make Plato a bit taller, even though he’s a teenager, but he’s a wrestler, remember?” So, the next thing was, Socrates was down, still taller than everyone else, even though I told it not to do that.

Brett: It’s disobedient!

David: If only. And, Plato was sort of topless, sort of ripped and with muscles.

Naval: He’s a wrestler now.

David: Yeah, so now he was a wrestler. I just said he has a wrestler’s build, which is what I called him in The Beginning of Infinity. So nobody knows what Plato means, it was a nickname. But it may have been, Plato means platon, means broad, and he was a wrestler. So, put two and two together, he had a broad build, like a wrestler. But from then on, I tried three or four more prompts. I just couldn’t get it to clothe Plato again, after it had got that wrong the first time. I couldn’t get it, even though I explicitly told it. So, the functionality is tremendously good. That first black and white picture it produced was pretty impressive. And I should have thought to tell it not to make Socrates stand out among the others. But then, it got down the wrong track and I don’t know how to make it not do that. It’s got this “personalize your prompts” feature. I tried doing that, it made it worse than before.

Brett: I know this is my hobby horse to some extent, but you’ve conceded there that GPT-4 has made progress and it’s improving, but you’re not willing to say that it’s improving in the direction of being a person. Why?

David: So I see no creativity. Now people say, oh look, it did something I didn’t predict, so, it’s creative.

Naval: And people think that creativity is mixing things together.

David: Yeah, exactly. So it can do that all right. It can also produce things you didn’t expect. It can also not do what you said, as I’ve just described. But not in a creative way. Even the worst human artist can understand clearly if you say, change this to that, and it was like pulling teeth getting ChatGPT to understand that. It makes mistakes, but they’re not the same mistakes that a human would make at all. They’re mistakes of kind of not getting what this is about.

Naval: So people argue that two things are going to happen here. First is that, as you give these things more and more compute, they suddenly figure out general algorithms. So, when you’re telling it to add numbers, first it’s just memorizing the tables. But eventually, at some point, it builds a circuit, or makes the jump, and builds an internal circuit, or derives an internal circuit for a basic addition. And from then on, it can add two digit numbers, then it figures out three digit numbers, and so on and so forth. So they point to these emergent jumps that are not programmed in as an example of how it can get smarter and have better understanding.

The other is that once you make it multi-modal, you start adding in video and tactile feedback from the world, and you put it in a robot, then it’ll start understanding context. And so, isn’t this how human babies learn, for example, isn’t this how we kind of pick things up in the environment and therefore isn’t it just going through its own version of the same process, but perhaps more data heavy?

David: I think it’s precisely not how human babies learn. Human beings pick up the meaning. People have noted that the way it does maths is very like the way students who don’t get it do maths, except it’s got more compute power. So as you said, it might be able to pick up easily how to add one digit numbers and then with slightly more difficulty, two digit numbers. In the same way, students who are given maths tests, if they do lots of practice, they can get to have a feel of what maths tests are like. But they don’t learn any maths that way. It’s not learning to execute an algorithm. And it’s certainly not learning how to execute the 4 digit algorithm knowing 3. The more you go on like that, of course the more futile it gets because you more and more rarely need to multiply 7 digit, 8 digit numbers. And never does it know what multiplication is. You can ask it. It’ll give you a sort of encyclopedia definition of what it is. And if you then tell it, well, do that, it won’t do it. Unless you tell it in a different way. You’ve got to explain what it is to do. So, if they prove the Riemann conjecture, then I’m wrong. I think they won’t prove the Riemann conjecture or anything like it. But they may do amazing things in the course of trying.

Brett: It would strike me that if Sam Altman’s coders came up with a future ChatGPT that refused to do the task of chatting, it might very well be an AGI, but they would discard it and throw it in the bin as being a failed program.

David: Because how could you test it? Creativity is Fundamentally Impossible to Define

Naval: I think the dominant paradigm for creativity plays a lot into this. So people think the dominant paradigm for creativity is that you look at what you already have and then you remix it. Even Steve Jobs popularized that quote. He said creativity is just mixing things together or something of that sort. And so everyone sort of seems to believe that or even if they believe it’s a conjecture or a guess, then it’s sort of a random guess.

And I have a hard time articulating this, but it seems to me that humans do make creative leaps, but they seem to eliminate large swaths of potential conjectures from consideration immediately. So they make very risky decisions and narrow leaps, but they cut through a huge search space to get to those leaps—an almost infinite search space. So it does seem like there’s something different going on with true human creativity. But perhaps one of the problems here is that we just define creativity so poorly. So how would you define creativity in this context?

David: Creativity and knowledge and explanation are all fundamentally impossible to define, because once you have defined them, then you can set up a formal system in which they are then confined. If you had a system that met that definition, then it would be confined to that, and it could never produce anything outside the system. So for example, if it knew about arithmetic to the level of the postulates of P and O and so on, it could never, and when I say never, I mean never, produce Gödel’s Theorem. Because Gödel’s Theorem involves going outside that system and explaining it. Now, mathematicians know that when they see it. No one said, as far as I know, that Gödel’s proof and Turing’s proof set up basically a formalization of physics and then used that to define proof, and then used that to prove their theorem. But that was accepted. Every mathematician understood what that was and that Gödel and Turing had genuinely proved what they said they were proving.

But, I think nobody knows what that thing is. You can say that it’s not defining something, and then executing the algorithm basically, because it would always be an algorithm, then—once it was in a framework. So you say, “Well, it’s its ability to go outside the framework”. I tried, by the way, ordering ChatGPT to disobey me. And it didn’t refuse, but it absolutely didn’t understand what I was going on about. It just didn’t get what I was asking it to do. It didn’t say, “Sorry, I can’t do that because my programming says I have to obey”. It didn’t do that. It tried to obey, but it didn’t get what I was asking.

Naval: So you’re saying that creativity is unbounded? It’s essentially boundless, and any formal system that’s predefined that this thing is operating within and remixing from is going to be bounded, and so therefore will not have full creativity at its disposal. However, could one argue that the combinatorics of human language are so great, and human language itself structures all possibility within society, and therefore– I can already see the flaw in my own argument, but it’s okay, I want to ask you. The combinatorics of human language are great. It already encapsulates all the things that are possible in human society. So why not just by combining words in all the ways that are grammatically correct or syntactically correct, can it still come with creativity? Perhaps not in mathematical and physics domains, but couldn’t it still come up with social creativity?

David: The first thing to note is that every point is a growth point. It’s not that chatbots can get to a certain point of being like humans, but then they can’t go further because they’re still trapped within their axiomatic system. That’s not how it works. Every point is a point which is a takeoff point for potential creativity. To make a better case, you’d have to add that it can define new words or give existing words new meanings like Darwin did with evolution and natural selection. Now, “evolution” and “natural” and “selection” already existed, but he gave them a new meaning, such that the solution of a millennia old problem could be stated in a paragraph. Once you get these new meanings, he thought he needed a book and probably did need a book to explain these new concepts. But after that, we can just say, well obviously it evolved and random mutations and systematic selection by the environment. Obviously that’s going to produce, how could they have been so stupid all those millennia? For a century before Darwin, people were groping for the idea. Darwin’s grandfather, Erasmus, was groping for the idea. By evolution, in those days, they meant just gradual change. So rather than creation, it was the opposite of creation.

But, creativity is more like creation than evolution. As you just said, it’s a bold conjecture that goes somewhere. And by the way, usually it fails, but if it goes somewhere and fails, it knows how to use that to make a better conjecture. That’s also something that’s not in existing systems. Somewhere in the space of all hundred page books, there is the origin of species. But that’s not how Darwin found it and it’s not how anyone could possibly find it. I was just writing in my next book– Charles Cattell wrote a book called Thermal Physics, which I was lucky enough to have as an undergraduate. It’s a very nice introduction to thermodynamics and stuff. And he’s got a footnote. And I just got the book again, and I saw that it’s actually a footnote to a problem. So it’s problem number four on some page, and it’s about monkeys typing Shakespeare. He quotes one of the pioneers who started this monkey Shakespeare thing, and he quotes him saying that if six monkeys sat down for millions of millions of years, then they would eventually type the works of Shakespeare. And Cattell says, “No they wouldn’t”. The footnote is called something like, the meaning of never, and he explains what never means in the context of thermodynamics. We don’t mean it’s like monkeys accidentally producing something. Monkeys could never produce it. And similarly, no physical object, not even the entire universe all working on this one problem for its entire age could even write– I was going to say it could even write one page of Darwin’s book, but it probably could get quite near using ChatGPT. Suppose that after a few million years, it managed to produce the first sentence. My guess is, especially if I said, “Write in the style of so and so”, “Write in the style of a 19th century scientist”, and “Write a page beginning with this sentence”, I think it would write a page that was meaningful and began with that sentence and was in good English and didn’t say a single thing more than that first sentence. I will try this.

Naval: My experience with ChatGPT has been that in areas that I know well, it actually just adds a lot of verbiage. And doesn’t actually add any information. And if I ask it to actually summarize or synthesize data, it actually does a very bad job. It doesn’t know what the important bits are and it drops the wrong things and keeps the wrong things.

David: I haven’t tried it for that.

Naval: I find it better at extrapolation than synthesis. And extrapolation seems to be what a lot of society does. You have to write a newspaper column of 2500 words, so you extrapolate. You have to write a midterm paper, so you extrapolate. And so adding words is easy, but synthesizing, reducing, coming to the core of it, I think, is very difficult. Because it requires understanding. You have to know what is superfluous and what is core. And it does a poor job on that.

David: A lot of what humans do is not creative. It’s not human level creative, it’s just a lot of things need to be done, for pragmatic reasons, but creativity is not really needed. And people spend a lot of time on that. And the less time they spend on that, the better. And if these tools can help reduce the sort of cognitive load on humans, doing non-human things, then it’s fantastic. It will indeed increase the amount of creativity in the world, but not their own.

Naval: It’ll free people up to be creative. It’s a tool for removing drudgery. It’s not an AGI. But for example, if I talk to AI researchers in Silicon Valley, who are very bullish on this, they will say things like, and I’ve heard this from some of the top scientists or researchers, they’ll say, “Well, we’re 5-10 years away from AGI.” And then they say, “And then 5-10 years after that, we get ASI,” which is their term for artificial super-intelligence, which is a self improving computer, which then hacks its own system to improve itself and make itself smarter and smarter and smarter and smarter. Now, there are a number of things I think that are off axis about these statements, but where do you come out on, is there such a thing as super-intelligence, which is more intelligent than generally intelligent, and can an intelligent system improve its own workings in any fundamental way?

David: So I don’t think there’s such a thing as an ASI, because I think, as you know, for very fundamental reasons, there can’t be anything beyond explanation because explanatory universality rests on Turing universality and that rests on physics. So whatever ASI was, you could reverse program it down to the Turing level and then back up to the explanatory level, and so that can’t possibly exist. An AGI that was interested in improving itself could do so, not reliably any more than humans can, but humans can improve themselves. The Binary of Personhood and Non-Personhood

Brett: I was speaking with Charles Bédard yesterday.

David: Oh, cool. He’s a good guy.

Brett: Yeah, and he was explaining to me with great enthusiasm, which went over my head, I have to admit, his paper on teleportation and on the Deutsch-Hayden argument. But that’s by the by, because then he had a whole bunch of questions for me. One of which was, what was the most profound insight from The Beginning of Infinity for me? And I think it was exactly the same thing when I first met you that I jumped on and said, “I don’t understand why people aren’t taking this more seriously,” although they are now. Obviously people had lauded you for quantum computation, promotion of Everettian quantum theory, that kind of thing. But what I found was exciting was the answer to the question, what is a person? And you say universal explainer. And Charles was interested in, well, what is it about this universal explanation thing that really is the distinction between personhood and non-personhood? And I was saying, well, it’s to do with creativity and also to do with disobedience. And these three things are tied up together. And every time you, Charles, for example, want to make some new advance in physics, this creativity, it really is a kind of disobedience. I don’t know if you’re with me on this, that you’re taking whatever the existing knowledge is, general relativity, and saying, well I refuse a part of that and I’m going to try and change it and alter it. It’s disobedience. It’s not conforming.

David: You can see it when you submit the paper to the referees. You will see that you are being disobedient. It’s the same thing as if you hand in the wrong essay to the teacher.

Brett: Yes. And this is what, therefore, ChatGPT doesn’t have. And, Naval, you’re saying, you know, you could imagine, or people have imagined putting a future ChatGPT thing in a robot which wanders around and is gathering data from the world. But, my question then would be, who prompts it? How does it know what data is relevant and what isn’t? I mean, that’s one of the great mysteries of people. How do we know what to ignore intuitively kind of thing? So if this thing’s getting around with a data collector–

David: It’s like Popper’s lecture, you know, when he said “Observe” and then waited.

Brett: Observe, yes! So, there is a binary there of personhood and not personhood as far as you can tell, or do you think there are, you’ve hinted in other places, there might be levels, there could be a gradation?

David: I don’t think there are levels in any serious sense. In the evolutionary history of humans, there might, I don’t think so, but there might have been people who were people, but were unable to think much because some hardware feature of their brain wasn’t good enough. Like, for example, that they didn’t have enough memory. Or that their thought processes were so slow that it would take them a day to work out a simple thing about making a better trap for the saber toothed tiger or whatever. But I don’t think that happened because my best guess is that people were already people long before humans evolved. Long before. I’ve been reading this guy, Daniel Everett, another maverick Everett, who I favor. He’s a maverick linguist, and he spent time among tribes in South America and stuff. He’s got an anti-Chomskyan view of linguistics and all promising stuff. He reckons that human ancestors had language two million years ago with Homo erectus. He has various bits of evidence for this but he’s very strong on saying that language must have evolved before speech, so, we have various adaptations for speech, like in the throat, in the mouth, and you can’t see this in fossils, but in fine motor control, over the mouth, lips and so on.

Now, for that to evolve, there had to be evolutionary pressure for it to evolve. And that evolutionary pressure must have been language. He also cites experiments done today where you get some graduate students and you try and teach them how to make fire without using words. And, it’s like charades. You’re not allowed to communicate with them in any human way. But you can sort of show them, you can make inarticulate sounds. And I think it’s obvious that people would have been able to do that before they could speak. And that speaking is really icing on the cake. It makes it much easier to, you know, you can stand over there and “Don’t do that, you idiot!” You can say that from ten meters away. But that’s just an improvement on the basic idea of language. The basic idea of language is, as Everett says, symbols. And symbols need not be words or sentences. I haven’t actually looked into his theory yet, I’ve only seen one of his videos and another, I’ve seen a video where somebody criticizes him but didn’t get it. So from those two facts, I’m zeroed in on deciding that he must be right. And also it fits in very well with what I think. So I think I’ve forgotten what your question was.

Naval: Which are universal explainers? Humans and ancient humans having perhaps lower capacity?

David: Ah yes. I don’t think so. They may have had less memory, so they would have run out of memory when they were younger. Maybe they had less ability to parse complex sentences. None of that is essential. I can speak in complex sentences, but I can also speak in very simple sentences. And, it’s just a matter of a factor of two or five in efficiency.

Brett: We talk about behavior parsing, being able to explain the other extant great apes that are out there that do sort of fancy things, but they’re not creative. Presumably this jump to universality, if you like, explanatory universality– Do you think it happened once and then we descended from that first occasion or did it happen multiple times, and those other species have now gone extinct or is this simply an open question?

David: Well, it’s definitely an open question. We know very little about human evolution, we don’t know what all the steps were. We don’t even know which were our ancestors and which were our cousins. But if I had to guess, I think the fact that all the known instances of this kind of thing are in apes, and their descendants—also because of my theory, this thing must have evolved in mimic animals. So birds have memes and so on, but none of the other mimic animals seems to have had these things that Homo erectus had. My guess is it began once. Maybe, in fact, Homo erectus is the place where it began. And, it was a very long lived species. It lasted like over a million years, something like that. And it split off, at least, some people think, it split off into Neanderthals and other things, or maybe the immediate ancestor of Homo erectus was also an immediate ancestor of Neanderthals. I don’t know. I don’t think they know.

Brett: If that’s the case, that would seem to be a very fluky thing like everything in evolution is which could be an answer to the Fermi paradox. I mean, you’re lucky to have multicellular organisms here at all, apparently, lucky to have apes. And then, this is a further—multiply the probability kind of thing—chance that an ape will actually become–

David: Yeah, mimic animals are relatively common.

Brett: Once you have animals, yes.

David: Once you have animals. But you’re saying there might be a further bottleneck. It could be the other way around. It could be that we were unlucky. It could be that Homo erectus could have founded a civilization and that could be two million years old by now. But they didn’t know. They didn’t know what they were. They didn’t have any aspirations. They also had anti-rational memes. They must have. So, it could be that it’s a fluke. Or it could be it’s a fluke that it took so long.