We, Ruben and Charlie, are Artificial Intelligence Students on the Utrecht University of Applied sciences. Our job is to introduce you to Artificial Intelligence with a minigame made by us. This minigame shows you how AI coding looks like and what it can do. The minigame is about fighting a robot that predicts and dodges your moves. This Wiki page will explain every code piece of the AI part detailed.

• Description
• AI-Representation
• Deployment
• Environment

• Scoreboard
• Sky
• Arena
• Floor

• Hitting and Moving
• Dodge Algoritm
• MQTT communication

• Leaderboard
• Databoard

• Bugs
• Version History

In the Boxing Minigame you can fight an AI-bot which predicts your movements. The AI-bot won't fight back, but you can calculate how much punches you can hit within a minute. The bot can dodge your movement, so use unpredictable punches. Eventually you get a score and your highscore will be presented in a Scoreboard. This scoreboard is updated on the highscoreboard everytime someone is in the top 10 with their animal Username. You also get extra information on the Databoard.

~Update

During the game you will notice that Robot aka Bender is able to predict and dodge punches. This is complety done with AI algoritms, we will talk about this deeper in the algoritm section.

Run the server.py:

   python server.py

Let the highscorepublisher.py run in the background

   cd WebVR-Demo\scripts
   python highscorepublisher.py

Let the databoardupdater.py run in the background

   cd WebVR-Demo\scripts
   python databoardupdater.py

Run the main file aka index.html, in the html ide, Chrome or Firefox (preable).

Connected Issues: A-frame Basics & Javascripts basics

Connected Sources: Learn A-frame, Controllers, &

The scoreboard consists of block- and text entities. There are 5 parts:

The username (left-up) is the name given to the player when he or she starts the event. Whenever the players gets a highscore, this is the name of the player given on the board.

The leaderboard (left-down) is where the top 10 scores are. These are added with MQTT. Whenever your game is ended and your score is high enough, the score will be put on the highscoreboard.

The timer (middle-up) will show the player when the warming up time starts and when the player can start hitting the robot.

The score counter (middle-down) is score that gets added with +1 after the player hits the robot.

The databoard (right-down) is the board where the after game movements are shown. The current data showing is: moving time, current position of controllers, calories burned, controller move length and all games controller move length.

Connected Sources: Add scoring system & Learning Receiving Data From VR sensors

Connected Sources: Box Collider,

You can move freely whenever you want and hit the robot, but this will not increase the score. Hitting and receiving points only happens when the robot is hitable, which is activated when game start. There is a red button on the pillar in the middle of the room. If pressed the game will count to zero and the timer will show a blue number decreasing. This is the warm-up time. After that the timer will show "GO!" and the game will start and the robot is hitable. After the game event after x seconds the timer will say gameover and the points will reset. If the score is high enough it will be added to the scoreboard

Both the Camera and the VR Controllers are connected with the parent entity, rig. Each controller has a couple of functions within it: aabb-colider, oculus-touch-controls, thumbsticklogging & hit.

• Oculus-touch-controls makes sure that the controllers are visible in the game
• Thumbsticklogging logs the thumbsticks when pressed
• The aabb-colider gives complex True value back when the controller is collide with a other entity type, in our case a-box (hitbox)
• When the aabb-colider gives this true value, the function Hit is ran. This function wil increase the score value of the entity score with 1.

Seen on the picture above, to let the robot dodge an attack, we need to predict the trajectory of both controllers. If the trajectory is within a hitbox then the robot needs to move away from it. The direction and where are determined by the dodgeMovement function.

How all of this works is explained below.

Connected User Story and Branch.

To be able to predict a trajectory we need to get the positions of the controllers. This is done using the getPositions function. The function needs one input and that is the rig. The position of the rig, the controllers, and the headset is called and these positions are summed up with the rig to get the right positions (source). These positions are converted to the right format and returned in an array.

This block of code below is how your rig setup should look like.

<a-entity id="rig">
    <a-entity id="camera"></a-entity>
    <a-entity id="left_controller" oculus-touch-controls="hand: left"></a-entity>
    <a-entity id="right_controller" oculus-touch-controls="hand: right"></a-entity>
</a-entity>

Connected commit.

The next thing to add was a delay that shows the coordinates every so often. We save the position of both controllers every 5 ticks and after 2 positions are saved, the trajectory and dodge movement algorithms are run. This means that between 12 and 24 positions are saved every second because one second contains between 60 to 120 ticks. This means the algorithms are run between 6 and 12 times a second.

Connected User Story and File.

Now that we can obtain the hand positions, we could calculate the trajectory. How we did this is by calculating the difference between the two coordinates and dividing that by 10, to get small steps. These steps are added to the second measured position of the hand until it reached a length of 0.3 meters. This means the algorithm predicts the trajectory with a distance of 0.3m from the last saved controller position.

To better understand what is done we have an array with only the second measured position of the hand. Then we take the last coordinate of the array and add the step to it and append that to the array until the difference (source) between the second measured position and the last coordinate of the array is larger than 0.3m. Now we have an array with steps.

Connected Issue.

Bender has two hitboxes at the moment. They can be seen in the image above where they were transparent but now are invisible. To check if the opponent is in the trajectories path we need to know where the hitboxes are and how big they are. To do this we call the positionsOfBox() function. In this function, to get the right positions we call the positions of the hitboxes and the position of the parent entity, which is in this case bender. We calculate the world position of the hitboxes by adding the coordinates of the hitbox with the coordinates of the parent. This is also done with other attributes like scale, rotation, width, height, and depth. With these attributes, we can calculate the two outer corners of every hitbox.

Now that we know the coordinates of the opponent, we can filter the array with coordinates to only have coordinates that are inside of the opponent's hitboxes. This is done using the checkIfGonnaHit() function. This function runs through every item in the array, checks if the x, y, and z coordinates are inside the opponent's hitboxes, and returns these items.

Now that we know how to check if the opponent will be hit, it needs to dodge the hit. To calculate this we first need to know what way the opponent needs to be moved towards. To do this we use the array of hits (which was returned from the checkIfGonnaHit() function) and calculate which way the punch is going. Now that we know the direction we also know if the opponent needs to move left, right or backward. Then we calculate the distance between the point that is farthest away from the edge of the hitbox array so we know how far the opponent needs to move to dodge the punch. After we calculate that, we move the opponent.

We wanted to upgrade the trajectory algorithm to predict a curved punch way better. After some research, we came across some predictor-corrector algorithms. The Kalman Filter was one that was mentioned a lot, but after some more research, we saw that it was most useful with noise measurements. Because we use a VR, the positions of the gear are not noise which means that this algorithm didn't match our problem. (If you do want to learn more about this algorithm you can find a good tutorial here.) We advise using an algorithm like Heun's method or the RK4 if you want to add a predictor-corrector algorithm. These two algorithms both use the Euler method that we already coded for a future programmer to use, here. Because we had no time to implement these algorithms we thought of another way to calculate a curve, using the y=ax^2+bx formula. We made a function that (given two points) solves the a and b variables of this formula. We also made a function that then gives use the point of the y variable given the x, a and b variables. So future contributors could use this to expand and fix. The function works but has a little problem and that's that the curve (using this formula) always goes through the point (0,0) which is not what we want. So if people want to use this, they need to fix it so that has a new middle-point (where it goes through). For example, it uses a new measurement of the controller to assign the middle-point to. Then there is a problem that it's a formula that solves a 2D curve and not a 3D curve.

All of this code is in the ai-game-upgrade-trajectory-algorithm branch.

Connected Issues: Mqtt Client & Scoring System

Connected Sources: Mqtt client tutorial & broker official site

Javascript & Html have difficulties with reading other files so we use a python file for this, called Highscorepublisher.py . Python can open/read/write txt files. It's not very efficient to send the leaderboard as a dictionary, instead we use a JSON string which we can convert later back to a dictionary or list.

To communicate the python file with javascript files and vicaversa, we use a mqtt client called Paho. To send messages to each other you need a broker and topics. A broker is a server with a IP and a port, which can be changed depending on the security level, where you can send your message temporarly to. They are a lot of free open source servers which you can use, we use broker.emqx.io. To send a message there are 2 parties, one receiver and one sender. The receiver, aka the subscriber, subscribes to a topic (with a while loop) and receives every message that is send. The sender, aka the publisher, publishes a message on a topic. If the python file is a publisher and the javascript file a subscriber, then they can communicate with each other. To make sure that our javascript file receives only the message when needed, we also need to have a request topic. The python file subscribes to the request topic and the javascripts publishes its request messages. In the python code for example, it will send the leaderboard message when the request topic message has been received, this will be further explained in the next paragraph.

The python file consists of 5 functions and a client connector which connects it to the broker and its port.

• on_publish: prints a check value if the data has been published
• on_connect: if connected to the broker, then subscribe to the topics
• on_message: determines which functions needs to be ran depending on the topic of the message

The highscoreboard is a board that shows the top 10 scores of all time. The score is settled with the amount of punches hit on bender, after that your current (animal) username will be shown on the leaderboard. The highscores are received with MQTT when the entire game is powered up and when the game event is ended & to update if someone got a highscore. This is all done within a python file called highscorepublisher.py. Besides the standard connect function described above it also consists of extra functions:

• updateleaderboard: changes the leaderboard txt with the new score input
• getleaderboard: reads the leaderboard.txt, turns it into a readable string and returns it

The statistics, which are mentioned earlier, are the controller position, the controller travel distance, moving time, calories burned, slowest punch and fastest punch. All these data is calculated within a database and a txt file called lastmovement.txt. The txt file is filled with movement every half a second. This databoard.csv file is filled from this txt file with the python file, databaseinteracter.py. Besides the standard connect function described above it also consists of extra functions:

• csv_add_move_data: moves data from lastmovement.txt to databoard.csv with the current name and time
• csv_header: resets and empties the entire database
• csv_test_data: adds test data to the database to debug
• txt_updatelastmovement: rewrites the txt file with the current position
• txt_getlastmovement:get the last position from the txt and send it back with mqtt for the "Current controller position" value
• csv_getsorted_data: get and calculates simplified data, like total moving time, and send it back with mqtt for the rest of the databoard values

• V1: 26-10-21 Created a Boxing environment using Danilo's Tutorial with a simple robot Model.
• V2: 1-11-21 VR Controllers interacts with the Bender model
• V3: 4-11-21 Interactions now adds score + Scoreboard material and placement changed
• V4: 8-11-21 Score gets saved in txt file and leaderboard gets updated with mqtt client and python

• V1: 29-10-21 Added component that can move an entity on command.
• V2: 1-11-21 Added component that shows the real-time coordinates of the VR gear.
• V3: 2-11-21 Added component that shows the coordinates of the VR gear every 0.5 seconds.
• V4: 7-11-21 Added component that can calculate the trajectory.
• V5: 10-11-21 Added check if the opponent is in the trajectories path.
• V6: 10-11-21 Added that the opponent moves away from the trajectory.