Project Proposal

Nicholas Miklaucic, Joshua Qin, Garrett Smith

Introduction

Our proposal is to make an AI agent for a 2D one-on-one air combat game of our own creation, modeled after games like Hyperspace Dogfights. Fundamentally, the core gameplay idea is controlling a plane shooting in a particular direction and being controlled via thrust and turning under gravity. We think that this type of physics provides an interesting space to explore computer control within. Hopefully, without the limits of human reaction time and human speed limitations, we can achieve superhuman play: this is the ideal outcome.

Algorithms Used

Our plan is to use Q-learning as the main underpinning of the AI agent—this seems appropriate given the similar use of reinforcement learning in other physics-based control problems and other adversarial realtime games. We plan on using approximate Q-learning with features to deal with the dimensionality of the state space and modeling the continuous state space discretely, with some time delta chosen to maximize performance while retaining as precise control as possible. Adversarial search may be used in a final agent given a sufficiently well-trained value function, although we haven't gotten that far yet.

Problem Specification

Specifically, we want an AI agent to, given a game state (the position, orientation, velocity, and acceleration of the agent and its opponent, the bullets on screen), take a next action (some combination of shooting, thrusting, and turning for the next frame). The goal is to destroy the opponent without having the same done to you, both by staying on screen and firing at the opponent.

Team Composition

We think three people is appropriate for this project in order to split the time necessary to implement a game for the AI to play and the time necessary to develop the AI. We have no specific formal labor distribution: our skill sets are similar enough that all of us expect to contribute to all parts of the final product.

Weekly Plan

Week	Plan
March 13-20	Beginning game implementation, overall software design, reviewing relevant literature
March 20-27	Game implementation, further literature review
March 27-April 3	Game implementation, testing
April 3-10	Finishing game backend, beginning work on AI
April 10-17	AI training, optimization, and further tweaking
April 17-24	Further work on optimizing AI, writing draft report
April 24-30	Finishing final report

Our halfway milestone is to have the game itself finished, with rudimentary AI implemented on top of that (e.g., a simple heuristic function and basic search).

Technologies

The game backend and AI will be written in Java. Our plan is to completely separate the backend and frontend: as of now the plan is to implement a web frontend, although we might use a debug implementation in a different language for testing purposes. The plan is not to use any specific game frameworks, although this is subject to change. We will ideally not need to heavily use any existing libraries or projects and can implement everything from the ground up.