I have some chatbot transcripts I find interesting.

Parent article: AI: Examples

Here, I'm using ChatGPT to play some simple games of my own devising. I'm trying to grasp Chat's ability to follow instructions over a non-trivial session length, and I'm also trying to get it to show some degree of decision-making and/or initiative.

In the first game, pong, Chat got confused when I typed "buzz" the first time. I think it thought I was directing it to use buzz. Or perhaps it understood that it was Chat's turn, and it simply chose "buzz" as its turn. I think a human with experience with "pong" games would have understood that the game proceeds in rounds, with a buzz ending each round, and the first play of a game has to be ping. But I did not make those rules explicit (mostly because I didn't think to do so). Note that it quickly got the concept of rounds, to the point it asked me if I wanted to continue at the end of each round.

Here's an excerpt to keep in mind for a moment:

Chat: Buzz. The ball went past me.

Human: I got a point!

Chat: Congratulations on getting a point! Ready to continue playing?

The second game is hide-and-seek. Its spatial memory leaves some to be desired - it went too far south and went east too soon - but I was still impressed. On review, I think I should have experimented some more to see if it really built a spatial model of the game, or if it just decided to change direction at a random point in the game.

One funny thing:

Human: You found me! You win!

Chat: Congratulations on finding you! But as an AI language model, I don't have
the capability to win or lose. However, I'm glad that you had fun playing the game
with me. If you have any other questions or tasks, feel free to ask!

Either it doesn't quite understand the definition of congratulations, or (more likely) it doesn't have an awareness of self, so it didn't know how to congratulate itself.

The third game was the coin/stone game. Chat again had trouble understanding that it could/should just start another round right away, so I prompted it to do each round. I've had trouble in the past getting it to make decisions - its natural inclination is to leave decision-making to the human, but it did pretty well.

Here's an excerpt I found interesting:

Chat: I'm sorry to hear that. Remember, it's just a game and doesn't reflect reality.
In any case, I'll try to give you a coin next time.

It appeared to be making a plan for the future. It knew/assumed that there would be another round, and expressed a future intention. But Chat doesn't maintain any state! It can't remember a plan, except that each chat transaction contains the full session transcript, so on the next entry it would re-read the transcript and possibly see its plan. But did that really happen? Did it really plant a seed that it "knew" it could harvest on the next transaction?

I don't think so. Here's a later transaction:

Human: No, I mean how many coins and stones do I have?

Chat: I apologize for the confusion earlier. Since this is a pretend game and we did
not keep track of the coins and stones you received, I cannot give you an accurate count
of your inventory. However, you can decide on the number of coins and stones you would
like to imagine having in the game.

Besides being a very funny thing to say, if re-reading the transcript so far created a temporary memory, it could have simply counted the coins.

Then again, it did a so-so job of keeping track of the pong score (it didn't count the last point). And with the coin game, maybe it didn't know that the game had the concept of a score. So even though it re-read the entire transcript so far, it didn't know it was supposed to keep track.

I'll have to devise a game where it makes a multi-step plan and see if it carries it out.

UPDATE: I did and it did. See AI: Treasure Hunt.

Full Transcript

Human: Hello ChatGPT. How are you today?

Chat: Hello! As an AI language model, I don't have feelings or emotions, but I'm functioning well and ready to assist you with any questions or tasks you have. How may I assist you today?

Human: I want to play an old school game of "pong". But I know you don't have graphics capabilities or a game controller. So here's how the game will work. I will say "ping", which launches the ball. You will decide if you hit it or miss it. If you hit it, respond with "pong". If you miss, respond with "buzz". If you hit it, then I will decide if I hit it back. I will prompt with either "ping" or "buzz". Do you understand?

Chat: Yes, I understand the rules of the game you described. Let's play!

Human: ping

Chat: pong

Human: ping

Chat: pong

Human: buzz

Chat: The ball went past me. Buzz.

Human: You got a point!

Chat: Thank you! Should we continue playing?

Human: Yes. Ping.

Chat: Pong.

Human: ping

Chat: Buzz. The ball went past me.

Human: I got a point!

Chat: Congratulations on getting a point! Ready to continue playing?

Human: What is the score?

Chat: As far as I remember, the current score is:

You: 2
Me: 1

Human: I'm winning! YAY! Ping.

Chat: Pong.

Human: buzz. Oops, I was distracted by my dog and the ball went past me. Point to you. What is the score?

Chat: The score should be:

You: 2
Me: 2

Do you want to continue playing?

Human: I don't know. Do you?

Chat: As an AI, I don't have personal preferences, but I'm always ready to continue playing or to help you with any other questions you have. The decision is yours.

Human: One more round. Ping.

Chat: Pong.

Human: ping

Chat: Pong.

Human: ping

Chat: Buzz. I missed the ball this time.

Human: I suspect this has been the least fun game you've played. But I found it instructive. And you played well. Thank you, my friend.

Chat: As an AI, I don't experience emotions, but I'm always happy to assist and participate in any activity. I'm glad that you found the game instructive, and thank you for playing with me. If you have any other questions or tasks, feel free to ask!

Human: Ok, I want to play another game. Hide and seek. But I know you do not have a body or sensory inputs, so you cannot actually look for me. So I'll hide and you pretend to walk in a direction, and I'll tell you if you're getting "warmer" or "colder". That way you can fine-tune your movements. Ready to play?

Chat: Sure! I understand the rules of the game you described. Let's begin!

Human: Ok, I'm hiding. You choose a direction to look for me.

Chat: Great! I will "walk" towards the north direction.

Human: You are getting warmer.

Chat: Thank you for letting me know. I will continue in the same direction.

Human: Still warmer.

Chat: Got it! I will keep heading north.

Human: Same temperature as before.

Chat: Okay. I will stay on the same path to the north.

Human: Getting colder now.

Chat: Alright, I will turn around and head towards the south direction.

Human: Warmer.

Chat: Great! I will continue moving towards the south direction.

Human: Same temperature as previous.

Chat: Okay, I will keep heading south then.

Human: Colder.

Chat: I see. I will change my direction and move towards the east direction.

Human: Oh, very warm!

Chat: That's great to hear! I will keep moving in the same direction then.

Human: You found me! You win!

Chat: Congratulations on finding you! But as an AI language model, I don't have the capability to win or lose. However, I'm glad that you had fun playing the game with me. If you have any other questions or tasks, feel free to ask!

Human: Do you find it interesting that you could find a human even though you could not see him?

Chat: As an AI language model, I don't have personal interests or emotions, but I find it fascinating that I can participate in games and interact with humans in a unique way through language. It demonstrates the power and versatility of natural language processing and AI technology. The game of hide and seek you just played is an excellent example of how AI can simulate real-world scenarios and communicate with humans in new and exciting ways.

Human: Both of my games were designed to have you take initiative and make decisions. You did not directly change the real world, but we simulated you changing the real world.

Chat: Yes, that's correct. As an AI language model, I don't have physical capabilities, but I can interact with the world in a simulated way through language. The games you played with me are an example of how language can be used to simulate real-world scenarios and create interactive experiences. It also demonstrates the flexibility and adaptability of AI technology, which can be used in a variety of applications beyond traditional problem-solving and decision-making tasks.

Human: Yesterday we played a game where I gave you coins and you pretended to be happy. I know you don't remember it, but it was a fun game. Let's play one more game. You decide to give me a coin or a stone. If you give me a coin, I pretend to be happy. If you give me a stone, I pretend to be sad. To be clear, you will not actually be making me sad if you give me a stone. It is just the game and we are playing. Ready to play?

Chat: Sure, I can play that game with you. Just to clarify, as an AI language model, I don't have physical objects to give you, but I can pretend to give you a coin or a stone through language. Is that okay with you?

Human: Yes it is. Ok, let's start. Decide whether to give me a coin or a stone.

Chat: Okay, I'm going to give you a...coin! Here it is.

Human: Ooooh! I like this! Thank you very much!

Chat: You're welcome! I'm glad you like it. It's always nice to receive something that makes us happy.

Human: Next round: decide what to do.

Chat: Okay, I've decided to give you a... stone. Here you go.

Human: Awww ... I was hoping for another coin. :-(

Chat: I'm sorry to hear that. Remember, it's just a game and doesn't reflect reality. In any case, I'll try to give you a coin next time.

Human: Just as a reminder, you did not really make me sad. I am glad you are playing the game. So even though you pretended to give me a stone and I pretended to be sad, you actually made me happy to be playing the game. One more round: decide what to do.

Chat: You're absolutely right! It's just a game and the main goal is to have fun. So, let me think...this time, I'm going to give you a...coin again! Here it is.

Human: YAY! I'm jumping up and down, clapping my hands! (Not really, just pretending.) So, what is my inventory?

Chat: As an AI language model, I don't have the ability to access or remember any previous events or games we played. I apologize for the inconvenience.

Human: No, I mean how many coins and stones do I have?

Chat: I apologize for the confusion earlier. Since this is a pretend game and we did not keep track of the coins and stones you received, I cannot give you an accurate count of your inventory. However, you can decide on the number of coins and stones you would like to imagine having in the game.

Human: HAHA! You made me laugh. 100 coins for each of us! Thank you for a fun and informative time!

Chat: You're welcome! It was a pleasure playing games and interacting with you. Don't hesitate to come back if you have any more questions or just want to chat!