AI: How Can They Do That - fordsfords/fordsfords.github.io GitHub Wiki

I've been thinking a lot about AI recently.

Parent article: AI:.

I played with Dr. Eliza in the 1970s. I thought it was cute and kind of fun, but I was at no risk of accidentally thinking I was talking to a person. Even though I didn't see the code, I could imagine what it might look like. In the 80s I even implemented a little Markov chain program to generate gibberish that looked almost like real text. And I saw "poetry" written by more-complicated Markov-style code that was entertaining, but not mind-bending.

Then I played with ChatGPT, and ... wow.

The thing that I'm having the most trouble with is the degree to which it seems to understand what I'm saying. The very (VERY) limited reading I've done suggests that it is all statistical in nature. I.e. it simply generates the most likely series of words in response to a statistical analysis of the input prompts compared to its training data. But I still can't see how that can do that good a job of simulating a conceptual understanding of my prompts. At least, I assume it is simulating a conceptual understanding. Maybe there is some kind of abstraction engine in there that converts my prompts into abstract concepts, and the statistical analysis is done on a concept basis?

In my playing, I've tried to really test it to see if it understands what I'm saying. If I work at it, I can find cases where its conceptual framework breaks down and it just plain doesn't get what I'm saying. And I don't mean using complex sentences that it can't parse. I mean simple concepts that even a child would understand.

TODO: examples of concepts that ChatGPT doesn't understand.

All of this leads me to conclude that if we want to use the word "understand" for what ChatGPT does, then it's a very different kind of understanding than humans have. And since it is intended to resemble human understanding, I call it a simulation of human understanding.

Anyway, I would love to know how ChatGPT (and presumably other large language models) does such a good job of simulating human understanding.

The good news: I'm confident that I could achieve that knowledge (or at least approach it). There's tons of free-to-low-cost material available to learn from, and there are courses that I could take. I imagine that a year of reasonably focused study could bring me close, and maybe then I could get a job at OpenAI to learn their secret sauce.

The bad news: I'm just not that interested. Despite the tremendous advancements in the field, I don't find the profession appealing. Maybe it's like learning how magic tricks are done - it takes away some of that magic to know it. I don't have the time to study the field, and even if I did, there are other things I would rather do.

So I'll have to live on in relative ignorance of how ChatGPT can do what it does. Or, who knows? Maybe I'll try to make time for some AI learning.