Introduction to Machine Learning - UM-LPM/GIANT GitHub Wiki

Machine Learning (ML) is a subset of artificial intelligence (AI) that enables computers to learn from data and improve their performance without being explicitly programmed. Instead of following a fixed set of rules, ML systems identify patterns, adapt to new situations, and refine their decision-making processes over time.

In game development, ML offers a way to create more dynamic, intelligent, and engaging AI agents. Unlike traditional AI methods that rely on predefined logic (such as Finite State Machines), ML-driven AI can adapt based on player actions, making gameplay less predictable and more immersive.

Types of Machine Learning and Their Role in Game AI

There are three primary types of machine learning, each with its applications in gaming:

1. Supervised Learning

Supervised learning involves training an AI model on labeled data, where each input is associated with a correct output. The AI learns by minimizing errors between its predictions and the actual results.

Game AI Applications:

  • NPC Behavior Prediction: Predicting player movement patterns to improve AI reaction time.
  • Player Profiling: Analyzing player styles (aggressive, defensive, exploratory) for personalized game difficulty.
  • Cheat Detection: Identifying anomalies in player behavior that indicate unfair play.

Example: A racing game AI could learn from human player inputs to optimize driving strategies for NPC opponents.

2. Unsupervised Learning

In unsupervised learning, AI is given unlabeled data and must find hidden patterns or structures without explicit instructions.

Game AI Applications:

  • Procedural Content Generation: Automatically generating levels, maps, or quests based on discovered patterns in previous designs.
  • Player Grouping & Matchmaking: Identifying players with similar skill levels for fair matchmaking.
  • Dynamic Difficulty Adjustment (DDA): Clustering player behaviors to create adaptive AI that adjusts difficulty based on player skill.

Example: An open-world RPG using AI to analyze exploration patterns and suggest quests tailored to individual playstyles.

3. Reinforcement Learning (RL)

Reinforcement Learning is a trial-and-error approach where an AI agent interacts with an environment, receives rewards or penalties, and adjusts its actions to maximize long-term rewards.

Game AI Applications:

  • AI Agents Learning to Play Games: Training NPCs to optimize strategies (e.g., DeepMind’s AI mastering StarCraft II).
  • Self-Improving AI Opponents: Bots in fighting games learning new combos and counterplays based on player behavior.
  • Automated Game Testing: AI playing a game thousands of times to detect balance issues.

Example: AlphaGo and OpenAI Five—AI models trained using RL to outperform human players in complex strategic games.

Genetic Programming in Game AI

A unique approach to machine learning in games is Genetic Programming (GP). Instead of using datasets for training, GP evolves AI behavior using principles of natural selection—similar to how biological evolution works.

How Genetic Programming Works

  1. Create a population of AI agents with random behaviors.
  2. Evaluate performance based on a fitness function (e.g., survival time, damage dealt).
  3. Select the best-performing agents and apply mutations/crossovers.
  4. Repeat the process for multiple generations until AI evolves optimal strategies.

How GIANT Uses Genetic Programming

GIANT is designed to evolve AI agents without pre-programmed rules by:

  • Allowing multi-agent competition to drive strategy development.
  • Using different Rating systems and Tournament organizations to evaluate AI fitness.
  • Enabling agents to learn, adapt, and improve over multiple generations.

Example: In a Robocode-like battle simulation, agents can evolve aiming, shooting, dodging, and movement strategies purely through the optimization with genetic programming.

Machine Learning vs. Traditional Game AI

Traditional AI (Rule-Based Systems)

Most game AI before ML relied on hand-coded logic, such as:

  • Finite State Machines (FSMs): AI operates within predefined states (e.g., patrolling → chasing → attacking).
  • Behavior Trees (BTs): AI follows a hierarchical decision-making structure.
  • Decision Trees & If-Else Logic: AI makes choices based on static conditions.

While these approaches work well for many games, they often result in predictable and static AI behavior.

Advantages of ML-Based AI in Games

Machine learning offers several improvements over traditional rule-based AI:

✅ Adaptability: ML-based AI can adjust to new situations and learn from experience.

✅ Unpredictability: Unlike scripted AI, ML-driven agents can develop emergent behaviors.

✅ Scalability: ML AI can handle complex decision-making with ease.

✅ Less Manual Tuning: Instead of manually coding every behavior, ML allows AI to learn optimal strategies on its own.

How Machine Learning is Used in Games Today

1. Adaptive AI Opponents

  • AI that learns from the player and adjusts difficulty accordingly.
  • Example: The AI Director in Left 4 Dead dynamically adjusting enemy spawns based on player performance.

2. Procedural Content Generation

  • AI models generating levels, quests, maps, and characters dynamically.
  • Example: No Man’s Sky uses ML-inspired algorithms to generate billions of unique planets.

3. Player Behavior Analysis & Personalization

  • AI analyzing player interactions to create customized experiences.
  • Example: Adaptive difficulty systems that tweak game balance based on playstyle.

4. Game Balancing & Testing

  • AI models simulating thousands of matches to refine game balance.
  • Example: Fighting games using ML-trained AI to identify overpowered characters.

Conclusion

Machine Learning has fundamentally changed how AI behaves in games. Whether through reinforcement learning, unsupervised clustering, or genetic programming, these techniques allow developers to create more intelligent, adaptable, and engaging game experiences.

GIANT embraces advances in Machine Learning, providing a platform where AI agents can evolve and improve in single-agent multi-agent environments. By leveraging machine learning, GIANT helps push the boundaries of AI in gaming.