Release 2 - CSC4790-Fall2024-Org/Distant-Lands GitHub Wiki

(Skeleton)

Focuses

  1. Audio Integration
  2. Landscape Generation

Velocity

  • As of Sprint 6, still refining approach to calculating project velocity due to ongoing development and integration of multiple AI components (Gaia AI, Skybox AI, and voice recognition)

  • We will start tracking time spent per task and assign story points to each task to improve velocity tracking for future sprints. This will help us better plan and estimate our workload for Release 3

AI Integration

  1. Gaia AI --> procedurally generate landscapes based on the user’s voice description (if a player says, "mountainous terrain with rivers," Gaia adjusts the terrain, elevation, and vegetation to match the prompt)
  • Pros:
    1. provides a vast range of customization, allowing for highly varied environments
    2. integration with Unity is smooth and gives us the flexibility to tweak parameters
  • Cons:
    1. some landscape parameters are difficult to fine-tune based solely on voice prompts, and additional work is needed to ensure the landscape aligns perfectly with user expectations
  1. Skybox AI --> used to dynamically generate skyboxes that reflect the player’s voice prompt ("a starry night sky over a desert"); working on using AI image generation models to create skyboxes that match the environment generated by Gaia
  • Pros:
    1. adds another layer of immersion, allowing users to experience unique skies that change with the landscape
  • Cons:
    1. generating high-quality skyboxes quickly is challenging
    2. AI image generation requires optimization to ensure skyboxes are applied without long delay
  1. Voice Integration --> Unity Microphone API to convert the player’s voice commands into text triggering the landscape and skybox generation processes
  • Pros:
    1. creates a natural way for players to interact with the system, allowing for seamless environment generation based on user input
  • Cons:
    1. speech recognition accuracy can vary depending on the complexity of the user's input, requiring us to implement error handling and fallback options

Audio

  1. Set Up Audio Input for Voice Prompts (Unity microphone API)
  2. Link with Gaia AI

Gaia AI interaction (Landscape Creation)

  1. OpenAI GPT w/ Landscape descriptors used with Gaia Pro 2021
  2. Google Dialogflow w/ Structured Landscape Queries