IAVA: Interactive and Adaptive Virtual Agent System

Overview

IAVA is an interactive virtual agent system integrated into the Greta system that generates real-time adaptive behaviors in response to its human interlocutor.

It ensures the two aspects

• generating real-time adaptive behavior,
• managing natural dialogue.

The agent adapts to its interlocutor linguistically (choosing its next conversational move via the dialogue manager including a BERT-based (LLM-based) automatic thought classifier) and nonverbally (displaying reciprocally adaptive facial gestures via the ASAP model).

IAVA system uses Cognitive Behavioral Therapy (CBT) as its proof-of-concept but can be easily employed for other applications by changing the dialogue manager.

Demo video of the IAVA system:

A dyadic interaction between a human user and an adaptive CBT agent simulated via our IAVA system. The system loop consisting of user multimodal signal perception, agent behavior generation, signal communication, and visualization is assured to the frame level of 25 fps.

Please click to see the full demo video.

Usage

The IAVA system is currently maintained on the system's GitHub. Please check the GitHub repository located at: https://github.com/jieywoo/IAVA

CBT-HAI Database

A database of 60 CBT human-agent interactions has been collected using the IAVA system.

If you are interested in the CBT HAI DB, please contact the authors of the IAVA system.