Integrating Pygame with AI Runner

Overview

This guide explains how to integrate a Pygame project with AI Runner, enabling seamless interaction between PySide6 UI components, AI-generated responses, and Pygame-rendered visuals. The goal is to establish a clear and structured interface, making integration straightforward for your projects.

See src/airunner/pygame_example to see working code samples.

Components

1. PygameManager (Abstract Base Class)

• Manages Pygame initialization, screen setup, event handling, and integration with the AI Runner's API.
• Developers create subclasses of PygameManager to define game-specific behaviors.

2. PygameWidget

• A PySide6 widget responsible for embedding the Pygame display within a PySide6 application.
• Uses a timer (QTimer) for updating the Pygame screen at ~60 FPS.

3. PygameWindow

• Extends PySide6's QMainWindow and incorporates multiple mixins from AI Runner (Mediator, Settings, Styles, Pipeline, AIModel).
• Handles window creation, layout management, and threading for the Pygame loop.

4. Example Integration (ExampleGame)

• Demonstrates how to extend PygameManager to create a basic interactive experience that leverages AI Runner's capabilities.

Integration Steps

Step 1: Extend PygameManager

Implement game-specific logic by subclassing PygameManager. You must override these abstract methods:

• _handle_llm_response(self, response: LLMResponse): Handles responses from the AI (e.g., display messages, trigger game events).

• _start(self): Initializes game-specific elements.

• run(self): Contains your main game loop and event processing.

• quit(self): Cleans up resources upon exiting.

Example:

class ExampleGame(PygameManager):
    def _handle_llm_response(self, response: LLMResponse):
        print(response.message)

    def _start(self):
        self.set_screen_color()

    def _process_events(self):
        for event in pygame.event.get():
            if event.type == QUIT:
                self.running = False
            elif event.type == KEYDOWN and event.key == K_ESCAPE:
                self.running = False
            elif event.type == KEYDOWN and event.key == K_SPACE:
                self.api.send_llm_request("Tell me a joke.")

    def run(self):
        self.running = True
        while self.running:
            self._process_events()
            pygame.time.delay(100)

    def quit(self):
        self.running = False
        pygame.quit()

Step 2: Setup PygameWindow

Integrate your PygameManager subclass into the PySide6 main application window (PygameWindow).

if __name__ == "__main__":
    airunner_api = API(
        main_window_class=PygameWindow,
        window_class_params={
            "width": 800,
            "height": 600,
            "game_class": ExampleGame,
        }
    )

Step 3: Running Your Game

Launch your Python script. The PySide6 application window will open, embedding your Pygame project. Press SPACE to send a request to the AI model and receive a response printed to the console.

Handling AI Interactions

To send AI requests:

self.api.send_llm_request("Your prompt here")

Handle the response in:

def _handle_llm_response(self, response: LLMResponse):
    # Handle AI response here

Best Practices

• Always handle pygame.quit() within your quit() method.
• Implement clear event handling in _process_events().
• Keep your AI interactions responsive by integrating AI calls thoughtfully within your game loop.

This structured integration allows developers to rapidly prototype games that harness AI capabilities directly within their Pygame applications, leveraging the power of PySide6 and AI Runner.