Behavioral Imaging Toolbox (Bitbox) is a free and open-source Python library including a comprehensive set of tools for the computational analysis of nonverbal human behavior. The provided tools enable analysis of face, head, and body movements, expressions, and actions from videos and images. Included algorithms and metrics have been validated using clinically vetted datasets and published extensively, therefore, can be reliably used by behavioral, social, and medical scientists in their human subject research. As we closely follow state-of-the-art in computer vision and machine learning, provided methodologies can also be relied upon by computer vision researchers and other engineers as well.

Purpose and Audience

Bitbox is an easy-to-use computational toolbox to fill the gap between engineering advances in AI and their accessibility by behavioral scientists for quantifying nonverbal human behavior. While numerous successful computational approaches for quantifying human behavior from videos exist, their use is nearly always restricted to specialized engineers, rendering computational behavior analysis out of reach for those most interested in studying human behavior— behavioral, social, and medical scientists. The main goal of Bitbox is to provide computational tools that researchers with no engineering background can use and interpret, to democratize computational behavior analysis and significantly advance science in this area. The sophisticated design of Bitbox, as well as its state-of-the-art algorithms, make it a valuable tool for engineers and computer vision researchers as well. Moreover, its modular architecture makes it easy for developers to contribute their algorithms.

Provided Functionality

The toolbox provides the following functionality for quantifying numerous aspects of facial and bodily behavior. We are continuously updating and expanding the functionalities.

Level of Analysis	Modality	Function	Available	Tested	Published
Detection	Face	Face	:heavy_check_mark:	:heavy_check_mark:
Detection	Face	Landmarks	:heavy_check_mark:	:heavy_check_mark:
Detection	Face	Pose and identity normalized landmarks	:heavy_check_mark:	:heavy_check_mark:
Detection	Body	Joints	:construction:	:construction:
Kinematics	Face/Body	Spatial extent	:construction:	:x:
Kinematics	Face/Body	Speed	:construction:	:x:
Kinematics	Face/Body	Smoothness	:construction:	:x:
Expressions	Face/Body	Symmetry	:heavy_check_mark:	:heavy_check_mark:
Expressions	Face/Body	Expressivity	:heavy_check_mark:	:construction:
Expressions	Face/Body	Diversity	:heavy_check_mark:	:heavy_check_mark:
Expressions	Face	Temporal phases	:x:	:x:
Expressions	Face	Expressive movements	:heavy_check_mark:	:heavy_check_mark:
Expressions	Face	Localized expressive units	:heavy_check_mark:	:heavy_check_mark:
Coordination	Face/Body	Intra-personal coordination	:heavy_check_mark:	:heavy_check_mark:
Coordination	Face/Body	Inter-personal imitation	:heavy_check_mark:	:heavy_check_mark:

Team

Bitbox is being developed at Children’s Hospital Philadelphia and the University of Pennsylvania. The core development team consists of computational scientists and engineers, psychologists and behavioral scientists, and software developers, who design, develop, implement, test, and document all tools in collaboration.

Funding

The development of Bitbox is supported by the National Institutes of Health (NIH) Office of the Director (OD) and by the National Institute of Mental Health (NIMH) of the US, under the grant R01MH122599.

Citations

Please use the following citations…

Contributions

Bitbox is a free and open-source project, and we welcome contributions of new algorithms, implementations, and models to the toolbox. Each included algorithm and functionality in Bitbox goes through extensive testing and validation steps. Therefore, proposed contributions are expected to be justified through prior testing/validation results and published works.

License

Bitbox is provided under the …