Future Roadmap - tijnisfijn/Resolume-Composition-Converter GitHub Wiki

Future Roadmap

This page outlines potential future development ideas for the Resolume Composition Converter. As an open-source project, contributions from the community are welcome and encouraged!

Potential Feature: AI-Powered Video Upscaling Integration

One exciting possibility for the future is the integration of AI-powered video upscaling capabilities. This would allow users to not only convert compositions between different resolutions but also enhance the quality of the media files using state-of-the-art AI upscaling techniques.

How You Can Contribute

If you're interested in contributing to this or any other feature, please feel free to:

  • Open an issue to discuss your ideas
  • Submit a pull request with your implementation
  • Share your expertise or suggestions

The following roadmap is not a commitment from the original author but rather a collection of ideas that could be implemented by anyone in the community who is interested.

Phase 1: Foundations & Preparations

Establish AI Upscaling Module

  • Implement a new Python module (ai_upscaling.py) that handles all AI-related tasks
  • Support multiple AI libraries (Real-ESRGAN, OpenCV SuperRes, Video2X)
  • Ensure cross-platform support (Windows, macOS, Linux)
  • Add GPU acceleration with CPU fallback

Advanced Media Processing

  • Enhance media processing capabilities:
    • Add batch processing for media files
    • Implement automatic quality enhancement
    • Support for more specialized formats

Composition Parsing Updates

  • Enhance the existing composition-upscaling script to detect all media files
  • Store file path, resolution, and format information for each media file

Phase 2: AI Upscaling Pipeline Integration

Media Extraction & Preprocessing

  • Check each media file referenced by the composition
  • Convert to suitable format for AI upscaling if needed
  • Skip conversion for already compatible formats

Upscaling Step

  • Pass preprocessed video to the upscale_video() function
  • Support various scale factors (1080p → 2160p = 2× factor)
  • Allow selection of different AI models (Real-ESRGAN, EDSR, etc.)
  • Output upscaled media at the new resolution

Re-Encode to DXV (Optional)

  • Run upscaled file through Resolume Alley for DXV conversion if needed
  • Provide options to keep ProRes or convert to DXV automatically

Composition File Update

  • Update the composition file to reference the new upscaled media files

Phase 3: UI & Workflow Enhancements

New "AI Upscaling" Tab or Section

  • Add a panel in the interface for "AI Upscaling Settings"
  • Include options for:
    • Enable/disable AI upscaling
    • Model selection
    • Upscale factor or target resolution
    • Output format
    • GPU device selection

Automatic vs. Manual Processing

  • Allow users to choose between automatic or manual upscaling
  • Provide options to skip certain media files

Progress & Logging

  • Show a progress bar with "Now upscaling file X of Y"
  • Display estimated time remaining and frames per second
  • Log errors and warnings

Phase 4: Testing & Optimization

Test with Various Input Formats

  • Ensure compatibility with ProRes, DXV, H.264 MP4, etc.
  • Verify correct conversion and upscaling
  • Confirm proper file references in the final composition

Performance Profiling

  • Optimize for different hardware configurations
  • Implement batch processing for better performance
  • Evaluate memory usage and disk I/O

Edge Cases

  • Handle very short clips
  • Support large upscales (e.g., 720p to 8K)
  • Properly process generator clips and router clips
  • Handle missing or offline clips

Phase 5: Release & Maintenance

Documentation

  • Create comprehensive user guide for AI upscaling
  • Provide tips on GPU usage and recommended hardware

User Feedback Loop

  • Collect user feedback on upscaling quality and speed
  • Implement improvements based on user suggestions

Future Enhancements

  • Add frame interpolation (e.g., RIFE) for slow-motion or frame rate upscaling
  • Provide preview window for before/after comparison
  • Offer API mode for advanced scripting

Recommended AI Upscaling Technologies

Based on research, these technologies could be good candidates for integration:

Real-ESRGAN

  • Best quality upscaling with broad compatibility
  • Excellent for preserving details and reducing artifacts
  • Available via Python integration

OpenCV Super-Resolution

  • Faster processing with simpler integration
  • Models include EDSR, FSRCNN, ESPCN, and LapSRN
  • EDSR offers best quality while FSRCNN/ESPCN are faster

Video2X

  • Excellent command-line tool with multiple algorithm support
  • Provides flexibility to experiment with different AI models
  • Includes frame interpolation alongside upscaling

FFmpeg Integration

  • Essential for format conversion
  • Handles ProRes encoding directly
  • Works with external tools for DXV encoding

Other Potential Features

  • Batch Processing: Convert multiple compositions at once
  • Custom Presets: Save and load conversion settings
  • Advanced Timing Controls: More options for adjusting timing parameters
  • Plugin System: Allow third-party extensions for additional functionality
  • Enhanced Format Conversion: Build on the existing media format conversion feature with more advanced options and batch processing

Get Involved!

This is an open-source project, and your contributions are valuable! Whether you're a developer, designer, or user with ideas, there are many ways to help improve the Resolume Composition Converter.

If you're interested in working on any of these features or have ideas of your own, please:

  1. Check the Issues page
  2. Open a new issue to discuss your idea
  3. Fork the repository and submit a pull request

Together, we can make this tool even more powerful and useful for the Resolume community!