Technical Implementation

The Brain Dance Game combines several technical components to create an accessible, effective movement-based experience. The implementation prioritizes standard hardware compatibility, real-time feedback, and engaging visual representation of movement.

Motion Capture System

Video Capture

• Input Source: Standard laptop webcams
• Resolution: Compatible with common webcam resolutions (720p minimum recommended)
• Frame Rate: 30fps minimum for smooth motion tracking
• Field of View: Adjustable to accommodate different room sizes and player positions

User Impact:

• Alex: No special equipment needed reduces barriers to entry
• Maya: Works in limited apartment space
• Jamie: Familiar technology reduces tech anxiety
• Dr. Taylor: Accessible technology for diverse client families

Pose Detection

Video

The game uses Movement4Motion (m4m), a command-line tool for real-time pose detection and motion recording that:

• Detects poses in existing videos and live camera feed
• Records motions for playback in 2D and 3D environments
• Provides testable outputs for alignment with Brain Dance principles
• Analyzes movement patterns with sufficient accuracy for therapeutic feedback

Implementation Details:

• Skeletal tracking identifies key body points (joints, head, torso)
• Movement pattern recognition compares user motion to reference patterns
• Adjustable sensitivity accommodates different body types and movement abilities
• Error tolerance prevents frustration while maintaining therapeutic value

User Impact:

• Alex: Forgiving tracking reduces frustration with coordination challenges
• Jamie: Clear feedback on correct movements supports learning
• Maya: Accurate tracking provides satisfying feedback
• Dr. Taylor: Sufficient precision for therapeutic assessment

Depth Mapping

https://github.com/user-attachments/assets/b0d24bcb-76aa-472d-b172-b5a530c80408

https://github.com/user-attachments/assets/5c02f6aa-fe99-403a-810f-1b4130c00ed7

Depth Anything CLI creates 3D depth-mapped visuals for understanding movement dynamics:

• Converts 2D video into 3D depth-mapped information
• Provides enhanced understanding of spatial relationships
• Creates visual feedback that highlights proper movement execution
• Offers options for different visual representations (grayscale, heatmap, etc.)

Technical Samples:

• Depth Test Video Grayscale - Shows movement depth in grayscale gradient
• Depth Test Video Heatmap - Shows movement intensity through color mapping

User Impact:

• Jamie: Visual representation enhances understanding of movements
• Alex: Depth visualization helps with spatial awareness challenges
• Maya: Enhanced feedback maintains engagement
• Dr. Taylor: Detailed movement visualization aids assessment

Game Environment

Development Platform

• Primary Engine: Unity-based environment
• Cross-Platform: Compatible with Windows and macOS
• Browser Support: Future implementation planned for web-based access
• Hardware Requirements: Standard laptop/desktop with webcam, no special controllers needed

Technical Components

Initial Test Scene

• Prototype environment for validating fundamental elements:
  • Camera input processing
  • Skeleton tracking accuracy
  • Movement pattern recognition
  • Audio sync with movement
  • User interface responsiveness

3D Character Models

• Avatar models that replicate player movements:
  • Simplified skeletal representation for movement clarity
  • Customizable appearance options
  • Real-time mirroring of user movements
  • Visual indicators for correct/incorrect movement execution

3D Motion Playback

• System for demonstrating and recording movements:
  • Pre-recorded reference movements by professional dancers
  • Recorded user movements for review and comparison
  • Side-by-side comparison capability
  • Playback controls for learning at preferred pace

User Impact:

• Jamie: Visual representation of body aids spatial understanding
• Alex: Replay functionality supports learning at own pace
• Maya: Professional demonstrations provide clear guidance
• Dr. Taylor: Recording capability allows progress monitoring

Data Management

User Progress Tracking

• Local Storage: Progress data saved locally on user's device
• Metrics Tracked:
  • Completion status for each Brain Dance component
  • Accuracy scores for movement patterns
  • Time spent in each activity
  • Achievement unlocks and milestones
  • Custom difficulty settings

Optional Research Data Collection

• Anonymous Data: IRB-approved framework for collecting anonymized usage data
• Metrics:
  • Movement improvement over time
  • Component usage patterns
  • Engagement duration and frequency
  • Self-reported benefit ratings

User Impact:

• Dr. Taylor: Data supports clinical assessment and research
• All Users: Privacy-focused approach protects sensitive information

Technical Roadmap

Current Technical Demo Elements

• Video capture integration
• Basic pose detection functionality
• Concept art implementation
• Audio integration (Aaron's music compositions)
• Proof-of-concept Unity environment

Phase 1: Vertical Slice (Cross-Lateral Component)

• Core functionality for single Brain Dance component
• Complete motion detection for cross-body movements
• Full audio integration with feedback sounds
• Basic UI implementation
• Simplified progress tracking

Phase 2: Technical Foundation Expansion

• Enhanced motion detection accuracy
• Expanded movement pattern library
• Complete Hub World implementation
• Full data storage and retrieval system
• Additional visualization options

Phase 3: Full Implementation

• All eight Brain Dance components
• Advanced motion analysis
• Complete achievement system
• Performance optimization
• Extended hardware compatibility testing

Phase 4: User Testing & Refinement

• Technical adjustments based on user feedback
• Performance optimization for varied hardware
• Accessibility enhancements
• Bug fixing and stability improvements

Accessibility Considerations

Hardware Flexibility

• Webcam Positioning: Accommodates different camera placements and angles
• Processing Requirements: Optimized for mid-range computers to ensure wide accessibility
• Space Requirements: Adjustable detection zone for varied play spaces

Technical Accommodations

• Movement Sensitivity: Adjustable thresholds for movement detection
• Feedback Timing: Customizable delay for processing cognitive differences
• Visual Options: Multiple visualization modes for different processing needs
• Audio Settings: Separate volume controls for music, effects, and instructions

User Impact:

• Alex: Adjustable sensitivity accommodates coordination challenges
• Maya: Works in limited apartment space with standard laptop
• Jamie: Visual and audio customization prevents sensory overwhelm
• Dr. Taylor: Adaptability for diverse client needs and environments