(initially called the VIDA Agent Evaluator and created by Kenny)

Evaluate the screenshots of the VIDA agent and provide descriptive summary and matrix within an artifact. summarize how they align with strategy and mission of uoes and services. include key strengths, areas of improvement. If something is not found, a rating of N/A should be used. List items that were N/A in the evaluation and suggest next steps to address

---

UOES serves as the central hub for online education innovation and support at the university, with a mission fundamentally focused on advancing excellence in online learning:

• Online Learning Excellence: Pioneering high-quality, engaging online education experiences that transform how students learn remotely
• Digital Pedagogy Leadership: Defining best practices for effective online teaching and learning methodologies
• Technological Integration: Enhancing student engagement through thoughtful application of educational technology in online environments
• Standardization & Best Practices: Establishing consistent standards for online educational content development and delivery
• Digital Accessibility: Creating inclusive online learning experiences that reach all students regardless of ability or location
• Cross-Departmental Support: Providing specialized online learning services through Instructional Design, Academic Media, and Emergent Technologies
• DEIA Commitment: Promoting diversity, equity, inclusion, and accessibility principles throughout online educational materials

• Course design consultation and development for online and hybrid formats
• Assessment strategy development and implementation
• Learning objective alignment and curriculum mapping
• Instructional content development and optimization
• Faculty training and support for online teaching methodologies
• Survey implementation and educational effectiveness measurement
• Quality assurance reviews of online courses
• Accessibility compliance guidance and implementation

• Educational video production and post-production
• Audio recording and editing for online learning materials
• Graphics design for instructional content
• Animation creation for complex concept visualization
• Multimedia content integration into learning management systems
• Technical support for media-enhanced online learning
• Media accessibility (closed captioning, transcription, audio description)
• Digital asset management and archiving

• Transform the remote student experience through engaging, interactive online learning
• Ensure all online educational content meets accessibility standards and best practices
• Foster innovative approaches to digital and hybrid learning environments
• Support faculty in adapting traditional teaching methods to effective online formats
• Bridge conventional education with emerging digital possibilities to reach diverse learners

EmTech explores the cutting edge of online educational technology, with a mission centered on:

• Online Educational Innovation: Creating new ways to learn remotely through games, VR, AI, and interactive media
• Digital Creative Technology: Exploring the intersection of online education, technology, and creativity
• Online Learning Tool Development: Building platforms that democratize technology creation for educators in digital environments
• Virtual Immersive Learning: Developing online experiences that engage remote students through multiple sensory channels
• Digital Social Impact: Creating online educational games that address meaningful social and health issues
• Online Pedagogical Innovation: Supporting faculty in implementing game mechanics in their online teaching

• Game-Based Learning Development: Creation of educational games with specific learning objectives
• Virtual Reality (VR) & Extended Reality (XR): Development of immersive learning experiences
• Game Lab (GRID): Faculty support for exploring and implementing game mechanics in courses
• Visual Novel Engine (VNE): Low-barrier tool for faculty to create interactive narrative experiences
• AI-Enhanced Learning Tools: Development of AI applications for educational purposes (VIDA)
• Interactive Simulations: Creation of scenario-based learning experiences
• Digital Prototype Development: Rapid creation of educational technology concepts
• Educational Technology Consultation: Specialized guidance on emerging technology integration
• Virtual World Building: Creation of online spaces for learning and community building
• Student Worker Mentorship: Professional development through hands-on project experience
• Technology Workshops: Training sessions on emerging educational technologies
• Cross-Department Collaboration: Partnership on innovative educational projects

• Lead innovation in online educational technology that reimagines remote learning
• Showcase thought leadership in immersive and game-based online learning environments
• Create transferable skills that enhance student engagement across digital learning spaces
• Provide professional development opportunities for student workers in online educational technology
• Develop tools that lower technical barriers for educators transitioning to online formats
• Build online projects with measurable educational impact in digital and hybrid contexts

Both UOES and EmTech are united by commitments to:

• Online Learner-Centered Design: Prioritizing remote student needs and digital learning experiences
• Digital Accessibility: Ensuring online educational content reaches all learners regardless of location or ability
• Evidence-Based Online Approaches: Grounding digital innovations in research and systematic assessment
• Virtual Collaborative Development: Working across departments to maximize impact in online spaces
• Continuous Digital Innovation: Staying at the forefront of online educational technology trends
• Online Community Building: Creating engaging virtual spaces that foster connection and collaboration
• Remote Learning Excellence: Ensuring online experiences meet or exceed the quality of traditional formats

UOES operates through an integrated service model where Instructional Design, Academic Media, and Emergent Technologies work collaboratively to create comprehensive online learning solutions:

1. Instructional Design establishes learning objectives and educational frameworks
2. Academic Media creates high-quality multimedia content to support those objectives
3. Emergent Technologies develops innovative interactive elements to enhance engagement
4. Quality Assurance processes ensure accessibility and educational effectiveness across all elements

• Dance Autism Game: Combines EmTech's game design expertise with Instructional Design's learning methodology and Academic Media's visual assets
• VIDA Project: Integrates AI tools with instructional design best practices to enhance course development
• DEIA Workshops: Utilizes games as educational tools with structured learning objectives and multimedia support
• Visual Novel Engine: Provides accessible technology tools backed by instructional methodologies and media resources

This mission summary serves as a guiding framework for strategic alignment of all online projects, digital initiatives, and resource allocations within UOES and EmTech.

---

This project, called VIDA (Virtual Instructional Design Assistant), aims to create AI-powered tools to support educators in various pedagogical needs. VIDA is designed to provide instructors with intuitive, accessible applications that enhance instructional design capabilities, save time, and improve educational outcomes for students. The project is being implemented in three phases:

Infrastructure & Process Development: Establishing foundational infrastructure, building core AI integration pipelines, and developing initial tools like the Alt Tag Generator and Syllabus Creator. Tool Dashboard & Resource Hub: Creating a centralized platform for accessing tools and resources, expanding the tool collection, and implementing LMS integrations. End-User Tool Creation Platform: Empowering educators to create their own custom tools through natural language interfaces without requiring technical expertise.

The project uses modern technologies including Streamlit for web applications, Hugging Face and Claude for AI capabilities, GitHub for version control, and AWS for cloud infrastructure. The development follows a containerized approach with Docker and uses automated CI/CD pipelines for deployment. VIDA is guided by ten core pillars, including accessibility by design, augmentation (not replacement) of instructional designers, rapid tool development, educational pedagogy integration, and faculty empowerment. The project aims to democratize access to sophisticated instructional design capabilities, enabling educators to focus on high-value teaching activities while automating routine tasks.

---

This document provides comprehensive descriptions for each evaluation criterion across the ten VIDA Project guiding pillars. Use these descriptions to ensure consistent and thorough evaluation of all applications.

• AI Tool Evaluator - Project Instructions
  • Project Instructions
    • Basic instructions:
• Project Knowledge set 1 - UOES & EmTech Mission Summary
  • University Online Education Services (UOES)
    • Core Mission Elements
    • Key Department Services
      • Instructional Design
      • Academic Media
    • Strategic Goals
  • Emergent Technologies (EmTech)
    • Core Mission Elements
    • Emergent Technologies Services
    • Strategic Goals
  • Shared Principles
  • Integrated Service Model
    • Cross-Departmental Workflow
    • Collaborative Project Examples
• Project Knowledge set 2 - Project Summary
• Project Knowledge set 3 - VIDA Project: Detailed Application Evaluation Criteria
  • Table of Contents
  • 1. Accessibility by Design
  • 2. Augmentation, Not Replacement
  • 3. Rapid Tool Development Framework
  • 4. Educational Pedagogy Integration
  • 5. Faculty Empowerment & Agency
  • 6. Ethical AI Implementation
  • 7. Institutional Integration
  • 8. Continuous Improvement Through User Feedback
  • 9. Scalable and Sustainable Infrastructure
  • 10. Knowledge Sharing and Community Building
  • 5-Star Rating Guide
  • Evaluation Matrix Template
  • Overall Score

Section Description: This pillar ensures all VIDA tools meet accessibility standards, specifically address accessibility challenges in educational content, and are tested with diverse users and assistive technologies. Accessibility is a non-negotiable criterion for all tools, with automated accessibility checks implemented where possible. This ensures our tools are usable by all educators and students, regardless of abilities or disabilities.

Meets WCAG 2.1 AA standards

• The application complies with Web Content Accessibility Guidelines 2.1 at the AA level.
• This includes proper contrast ratios, keyboard navigability, and compatibility with assistive technologies.
• Code validates against automated WCAG checkers.

Keyboard navigation fully supported

• All functionality is accessible using keyboard alone without requiring a mouse.
• Focus states are clearly visible and follow a logical navigation order.
• Custom controls have appropriate keyboard interactions implemented.

Proper heading structure and semantic HTML

• Content is organized with appropriate heading levels (H1, H2, etc.) in a logical hierarchy.
• Semantic HTML elements are used appropriately (nav, main, section, article, etc.).
• ARIA landmarks and roles are used when appropriate to enhance structure.

Sufficient color contrast ratios

• Text elements meet minimum contrast ratios (4.5:1 for normal text, 3:1 for large text).
• Interactive elements have sufficient contrast against their backgrounds.
• Colors aren't used as the only way to convey information.

Text alternatives for non-text content

• All images have appropriate alt text that describes the content and function.
• Complex visualizations have detailed descriptions or alternative representations.
• Non-decorative icons and graphics have text equivalents.

Responsive design works at 200% zoom

• Content remains functional and readable when zoomed to 200%.
• No horizontal scrolling required at standard screen sizes.
• Layout adjusts appropriately for different screen sizes and zoom levels.

Screen reader compatible

• Content is announced correctly by screen readers following a logical order.
• Custom components have appropriate ARIA attributes and roles.
• Dynamic content updates are announced appropriately.

Supports accessibility in its outputs

• Generated content (e.g., documents, images) includes accessibility features.
• Outputs can be used in accessible ways by end users.
• Generated code or content follows accessibility best practices.

Section Description: This pillar positions VIDA tools as enhancing rather than replacing instructional designers and educators. It focuses on creating tools that handle routine tasks while freeing humans for higher-value creative work. We maintain a "human in the loop" philosophy in all tool design, include collaboration features, emphasize quality improvement over cost reduction, and design tools that supplement and extend human expertise rather than attempting to replace it.

Positions tool as enhancing rather than replacing human expertise

• Marketing and documentation emphasize enhancement of human capabilities.
• User messaging presents the tool as an assistant rather than a substitute.
• Design decisions prioritize supporting human judgment over automation.

Maintains "human in the loop" philosophy

• Critical decisions require human review and approval.
• AI-generated content is presented as suggestions rather than final outputs.
• System design includes validation points for human oversight.

Handles routine tasks while freeing users for higher-value work

• Automates repetitive or time-consuming tasks effectively.
• Focuses automation on mechanical rather than judgment-requiring tasks.
• Time savings is measurable and meaningful for core workflows.

Includes collaboration features

• Supports sharing and collaborative refinement of work.
• Multiple users can provide input on generated content.
• Workflows support transitions between automated and human-led stages.

Emphasizes quality improvement in messaging

• Communications focus on improving output quality rather than cost reduction.
• Value proposition emphasizes enhanced capabilities rather than replacement.
• Documentation highlights how the tool helps users create better results.

Extends rather than replaces human expertise

• Tool design acknowledges limitations and appropriate contexts for use.
• Features complement areas where humans excel rather than attempting to replicate them.
• System provides insights or capabilities that enhance human decision-making.

Section Description: This pillar establishes standardized templates and workflows for quick tool creation, creating modular components that can be reused across different tools. It designs natural language interfaces for tool specification, provides clear documentation and examples, implements CI/CD pipelines for streamlined deployment, and creates a repository of reusable UI components and design patterns. This approach enables us to rapidly develop new tools while maintaining consistency and quality.

Uses standardized templates

• Built using VIDA project templates rather than custom one-off implementations.
• Follows established project structure conventions.
• Leverages standard components from the VIDA template library.

Employs modular, reusable components

• Built with components that can be reused across other VIDA applications.
• Components are properly encapsulated with clear interfaces.
• Avoids code duplication through effective modularity.

Follows project coding standards

• Code adheres to VIDA project style guides and best practices.
• Passes automated linting and formatting checks.
• Variable naming, organization, and documentation meet project standards.

Includes comprehensive documentation

• Code is well-documented with comments and docstrings.
• Includes a README with setup and usage instructions.
• API and key functions are documented for other developers.

Deployed through standard CI/CD pipeline

• Uses established GitHub Actions workflows for testing and deployment.
• Follows VIDA containerization standards.
• Integrates with project monitoring and logging systems.

Uses shared UI components and patterns

• Leverages the VIDA UI component library rather than custom components.
• Follows established UI patterns for consistency across applications.
• Matches VIDA design system for visual consistency.

Section Description: This pillar ensures VIDA tools align with established learning science principles and promote active learning, engagement, and accessibility. Our tools support diverse teaching and learning styles, facilitate assessment and feedback mechanisms, incorporate Universal Design for Learning (UDL) principles, enable mapping of content to learning objectives, and support various educational contexts including higher education, K-12, and corporate training. This foundation in educational theory ensures our tools effectively support teaching and learning.

Aligns with established learning science principles

• Design reflects evidence-based learning approaches.
• Features support cognitive processes associated with effective learning.
• Avoids common pedagogy anti-patterns that hinder learning.

Promotes active learning and engagement

• Encourages user participation rather than passive consumption.
• Includes interactive elements that support deeper learning.
• Provides meaningful feedback to reinforce learning.

Supports diverse teaching and learning styles

• Accommodates different pedagogical approaches.
• Offers multiple ways to engage with content.
• Supports different learning preferences and contexts.

Facilitates assessment and feedback

• Provides mechanisms to evaluate understanding or performance.
• Offers constructive feedback for improvement.
• Supports formative assessment approaches.

Incorporates Universal Design for Learning principles

• Provides multiple means of representation, engagement, and expression.
• Reduces barriers to learning for diverse learners.
• Supports flexible approaches to achieve learning goals.

Enables mapping to learning objectives

• Allows content to be aligned with specific learning outcomes.
• Supports structured progression toward learning goals.
• Makes connections between activities and objectives explicit.

Supports various educational contexts

• Functions effectively across different educational settings (K-12, higher ed, corporate).
• Adapts to different subject areas and disciplines.
• Scales appropriately for different class sizes and teaching contexts.

Section Description: This pillar designs intuitive interfaces requiring minimal technical expertise while providing granular control over AI-generated outputs. VIDA tools enable customization to match specific teaching contexts and disciplines, create clear documentation, respect faculty expertise and pedagogical autonomy, provide multiple pathways for faculty with different comfort levels with technology, and allow faculty to contribute to tool improvement. By empowering educators, we ensure our tools enhance rather than constrain their teaching practice.

Provides intuitive interface requiring minimal technical expertise

• Interface is straightforward and easily understood without technical background.
• Limits jargon and uses familiar educational terminology.
• Provides clear guidance through workflows and features.

Offers granular control over AI-generated outputs

• Users can specify parameters and preferences for AI generation.
• Allows editing and refinement of AI-generated content.
• Provides transparency into how outputs are created.

Allows customization for specific teaching contexts

• Supports adaptation to different disciplines and contexts.
• Enables tailoring to specific course needs and objectives.
• Provides flexible options rather than one-size-fits-all approaches.

Includes clear, jargon-free documentation

• Documentation uses accessible language for educators.
• Provides examples relevant to educational contexts.
• Avoids technical terminology or explains it clearly when necessary.

Respects faculty expertise and autonomy

• Presents AI as supportive rather than authoritative.
• Avoids prescriptive approaches that undermine faculty judgment.
• Recognizes and accommodates diverse teaching philosophies.

Provides multiple pathways for different tech comfort levels

• Offers both simple paths for beginners and advanced options for tech-savvy users.
• Provides scaffolding for users with limited technical confidence.
• Allows progressive discovery of more advanced features.

Enables user contributions to tool improvement

• Provides mechanisms for user feedback and suggestions.
• Incorporates user insights into development roadmap.
• Acknowledges and implements community contributions when appropriate.

Section Description: This pillar ensures transparency about AI involvement in content creation, addresses bias in AI-generated content and recommendations, protects user privacy and data security, provides options for faculty to review and modify AI suggestions, establishes clear guidelines for appropriate use cases, implements content filtering for problematic outputs, and creates governance structures for ethical oversight. Our ethical approach to AI ensures our tools serve educational needs responsibly and transparently.

Transparent about AI involvement in content creation

• Clearly discloses when content is AI-generated or AI-assisted.
• Explains AI's role in any recommendations or decisions.
• Provides appropriate attribution for content sources.

Addresses bias in AI-generated content

• Actively monitors and mitigates potential biases in outputs.
• Includes diverse perspectives and examples in generated content.
• Provides mechanisms to identify and correct biased outputs.

Protects user privacy and data security

• Properly secures user data and content.
• Has clear privacy policies and data handling procedures.
• Minimizes data collection to what's necessary for functionality.

Provides options to review and modify AI suggestions

• Users can accept, reject, or modify AI-generated content.
• Interface makes it clear which elements are suggestions vs. final.
• Editing workflow is straightforward and accessible.

Establishes clear guidelines for appropriate use

• Documentation includes ethical usage guidelines.
• Defines boundaries for appropriate application contexts.
• Communicates limitations and potential risks.

Implements content filtering for problematic outputs

• Filters inappropriate, harmful, or offensive content.
• Prevents generation of content that violates ethical standards.
• Has mechanisms to flag and address problematic outputs.

Section Description: This pillar designs tools to work within existing educational technology ecosystems, creates flexible authentication and authorization systems, supports institution-specific branding and customization, facilitates analytics that align with institutional assessment needs, complies with institutional data policies and regulations, enables integration with Learning Management Systems, and provides deployment options that work with various IT policies. This ensures VIDA tools can be seamlessly adopted within diverse institutional environments.

Works within existing educational technology ecosystems

• Compatible with common educational platforms and infrastructure.
• Doesn't require extensive custom setup or isolated operation.
• Functions alongside other institutional tools and systems.

Supports flexible authentication

• Offers multiple authentication options (e.g., SSO, institutional login).
• Follows security best practices for authentication.
• Can integrate with institutional identity management.

Allows institution-specific branding/customization

• Supports custom branding with institutional logos and colors.
• Enables terminology customization to match institutional language.
• Allows adapting interface elements to institutional preferences.

Facilitates analytics aligned with institutional needs

• Provides data that supports institutional assessment requirements.
• Generates reports relevant to institutional metrics.
• Supports data export for institutional reporting systems.

Complies with institutional data policies

• Adheres to common educational data privacy requirements (e.g., FERPA).
• Supports data retention and management policies.
• Provides documentation on compliance features.

Enables LMS integration where appropriate

• Offers integration options for common Learning Management Systems.
• Follows standards like LTI for interoperability.
• Supports assignment, grade, or content exchange with LMS.

Provides flexible deployment options

• Offers deployment models that work with various institutional IT policies.
• Can be hosted in different environments (cloud, on-premises, hybrid).
• Scalable to different institutional sizes and needs.

Section Description: This pillar implements robust feedback mechanisms in every tool, creates clear pathways for feature requests and bug reporting, establishes analytics to identify usage patterns and pain points, maintains regular communication with the user community, conducts usability testing throughout development, prioritizes improvements based on user impact, and closes the feedback loop by communicating changes made in response to feedback. This user-centered approach ensures our tools evolve to better meet educator needs over time.

Includes robust feedback mechanisms

• Provides easily accessible ways for users to submit feedback.
• Offers both structured (ratings, surveys) and open-ended feedback options.
• Makes the feedback process straightforward and low-friction.

Provides clear pathways for feature requests

• Users can submit specific feature requests or enhancement ideas.
• Communicates how feature requests are prioritized and implemented.
• Acknowledges receipt of feature requests and provides status updates.

Incorporates analytics to identify usage patterns

• Collects appropriate usage data to understand how features are used.
• Uses analytics to identify pain points and opportunities.
• Privacy-respecting analytics implementation with transparent policies.

Maintains communication channels with users

• Regular updates about changes and improvements.
• Responsive to user questions and issues.
• Multiple communication channels appropriate to user preferences.

Has undergone usability testing

• Formal or informal testing with representative users.
• Systematically collects and addresses usability findings.
• Iterative refinement based on observed user behaviors.

Demonstrates improvements based on user feedback

• Shows evidence of changes made in response to feedback.
• Communicates how user input has shaped development.
• Closes the feedback loop by informing users of improvements.

Section Description: This pillar designs for growth in both users and functionality, considers resource usage and optimization, creates clear maintenance documentation and procedures, establishes support systems that can scale with adoption, implements monitoring and alerting for system health, designs with cost efficiency in mind, and plans for long-term sustainability beyond initial development. This technical foundation ensures VIDA tools remain reliable, performant, and sustainable as they grow in usage and complexity.

Designed for growth in users and functionality

• Architecture supports increasing numbers of users without degradation.
• Can accommodate new features without major restructuring.
• Performance remains consistent as usage increases.

Optimized for resource efficiency

• Uses computing resources efficiently to minimize costs.
• Implements appropriate caching and optimization techniques.
• Avoids unnecessary processing or storage.

Includes maintenance documentation

• Has clear documentation for ongoing maintenance procedures.
• Includes troubleshooting guides for common issues.
• Documents system architecture for future maintainers.

Has appropriate monitoring and alerting

• Implements logging of system behavior and issues.
• Includes monitoring for critical functionality and performance.
• Alerts for potential problems or failures.

Designed with cost efficiency in mind

• Balances performance needs with resource costs.
• Leverages appropriate service tiers for different components.
• Implements cost optimization strategies where appropriate.

Plans for long-term sustainability

• Considers future maintenance and support requirements.
• Uses technologies with longevity and community support.
• Avoids overly complex or difficult-to-maintain approaches.

Section Description: This pillar facilitates sharing of best practices among users, creates opportunities for collaborative tool development, documents case studies and success stories, builds mechanisms for peer support and mentoring, hosts regular community events and webinars, recognizes and celebrates community contributions, and fosters a diverse and inclusive community of practice. By building community around our tools, we enable educators to learn from each other and maximize the educational impact of the VIDA platform.

Facilitates sharing of best practices

• Includes capabilities for users to share techniques or approaches.
• Documents exemplary usage examples.
• Supports knowledge exchange between users.

Supports collaborative development

• Open for community contributions where appropriate.
• Has clear guidelines for how others can contribute.
• Acknowledges and incorporates community-driven improvements.

Includes case studies or examples

• Provides real-world examples of effective use.
• Documents outcomes and benefits achieved.
• Offers templates or starting points based on successful usage.

Provides mechanisms for peer support

• Enables users to help each other through forums, comments, or other channels.
• Facilitates formation of user communities.
• Supports mentoring relationships between experienced and new users.

Recognizes community contributions

• Acknowledges individuals who contribute improvements or ideas.
• Highlights community-developed extensions or approaches.
• Celebrates effective use and innovative applications.

Promotes inclusive community of practice

• Creates welcoming environment for diverse participants.
• Makes communication and contribution accessible to all.
• Supports multiple ways to participate in the community.

---

Use the following guide to assign ratings to each criterion:

• 5 Stars (90-100%): Exemplary - Fully meets criteria with excellence • 4 Stars (75-89%): Strong - Well-implemented with minor improvements possible • 3 Stars (60-74%): Satisfactory - Meets basic requirements but has room for enhancement • 2 Stars (40-59%): Needs Improvement - Partially implements but with significant gaps • 1 Star (<40%): Poor - Minimal implementation or substantially misaligned • N/A: Not applicable to this particular application

Criteria	[App Name]	Notes
1. Accessibility by Design
Meets WCAG 2.1 AA standards
Keyboard navigation fully supported
Proper heading structure and semantic HTML
Sufficient color contrast ratios
Text alternatives for non-text content
Responsive design works at 200% zoom
Screen reader compatible
Supports accessibility in its outputs
Section Average
2. Augmentation, Not Replacement
Positions tool as enhancing rather than replacing human expertise
Maintains "human in the loop" philosophy
Handles routine tasks while freeing users for higher-value work
Includes collaboration features
Emphasizes quality improvement in messaging
Extends rather than replaces human expertise
Section Average

(Continue with remaining sections following the same format)

Pillar	Average Rating	Notes
1. Accessibility by Design
2. Augmentation, Not Replacement
3. Rapid Tool Development Framework
4. Educational Pedagogy Integration
5. Faculty Empowerment & Agency
6. Ethical AI Implementation
7. Institutional Integration
8. Continuous Improvement
9. Scalable and Sustainable Infrastructure
10. Knowledge Sharing and Community Building
OVERALL AVERAGE