List of Components - chitsaw/psi GitHub Wiki

This document contains a list of components available in the Platform for Situated Intelligence repository, as well as pointers to other third-party repositories containing other Platform for Situated Intelligence components.

The table below contains the list of \psi components that are available in the current release, together with the namespace in which the component can be found (in general, the NuGet packages in which you can find the component follow the same naming convention, potentially with additional platform suffixes)

Name / Namespace	Description	Windows	Linux
Sensors
AudioCapture Microsoft.Psi.Audio	Component that captures and streams audio from an input device such as a microphone	AnyCPU	Yes
MediaCapture Microsoft.Psi.Media	Component that captures and streams video and audio from a video camera (audio is currently supported only on the Windows version)	X64	Yes
WindowCapture Microsoft.Psi.Media	Component that captures and streams screen shots of a window specified by a window handle to a stream of Shared<Image> (defaults to primary screen)	X64	No
VisualCapture Microsoft.Psi.Media	Component that captures and streams renderings of a specified System.Windows.Media.Visual to a stream of Shared<Image>	X64	No
RealSenseSensor Microsoft.Psi.RealSense	Component that captures and streams video and depth from an Intel RealSense camera	X64	No
AzureKinectSensor Microsoft.Psi.AzureKinect	Component that captures and streams information (video, depth, IMU, tracked bodies etc.) from an Azure Kinect sensor	x64	Yes
AzureKinectBodyTracker Microsoft.Psi.AzureKinect	Component that performs body tracking from the depth/IR images captured by the Azure Kinect sensor.	x64	Yes
KinectSensor Microsoft.Psi.Kinect	Component that captures and streams information (video, depth, audio, tracked bodies, etc.) from a Kinect One (v2) sensor	AnyCPU	No
File sources
FFMPEGMediaSource Microsoft.Psi.Media	Component that streams video and audio from an MPEG file	No	Yes
MediaSource Microsoft.Psi.Media	Component that streams video and audio from a media file	X64	No
WaveFileAudioSource Microsoft.Psi.Audio	Component that streams audio from a WAVE file	AnyCPU	Yes
WaveStreamSampleSource Microsoft.Psi.Audio	Component that streams audio from a System.IO.Stream when triggered by a boolean input stream.	AnyCPU	Yes
File writers
WaveFileWriter Microsoft.Psi.Audio	Component that writes an audio stream into a WAVE file	AnyCPU	Yes
MPEG4Writer Microsoft.Psi.Media	Component that writes video and audio streams into an MPEG-4 file	X64	No
Imaging
DepthImageEncoder Microsoft.Psi.Imaging	Component that encodes a depth image using a specified encoder (e.g. PNG)	AnyCPU	Yes
ImageEncoder Microsoft.Psi.Imaging	Component that encodes an image using a specified encoder (e.g. JPEG, PNG, GZip)	AnyCPU	Yes
DepthImageDecoder Microsoft.Psi.Imaging	Component that decodes a depth image using a specified decoder (e.g. JPEG, PNG, GZip)	AnyCPU	Yes
ImageDecoder Microsoft.Psi.Imaging	Component that decodes an image using a specified decoder (e.g. JPEG, PNG)	AnyCPU	Yes
ImageTransformer Microsoft.Psi.Imaging	Component that transforms an image given a specified transformer	AnyCPU	Yes
Vision
ImageAnalyzer Microsoft.Psi.CognitiveServices.Vision	Component that performs image analysis via Microsoft Cognitive Services Vision API.	AnyCPU	No
FaceRecognizer Microsoft.Psi.CognitiveServices.Face	Component that performs face recognition via Microsoft Cognitive Services Face API.	AnyCPU	No
Machine Learning
OnnxModelRunner Microsoft.Psi.Onnx	Component that runs an ONNX model.	AnyCPU	Yes
TinyYoloV2OnnxModelRunner Microsoft.Psi.Onnx	Component that runs the Tiny Yolo V2 object detection model.	AnyCPU	Yes
ImageNetModelRunner Microsoft.Psi.Onnx	Component that runs an ImageNet image classification model.	AnyCPU	Yes
Audio processing
AudioResampler Microsoft.Psi.Audio	Component that resamples an audio stream into a different format	AnyCPU	No
AcousticFeaturesExtractor Microsoft.Psi.Audio	Component that extracts acoustic features (e.g. LogEnergy, ZeroCrossing, FFT) from an audio stream	AnyCPU	Yes
Reframe Microsoft.Psi.Audio	Component that reframes a stream of audio buffers to fixed size chunks	AnyCPU	Yes
Speech processing
SystemVoiceActivityDetector Microsoft.Psi.Speech	Component that performs voice activity detection by using the desktop speech recognition engine from `System.Speech__	AnyCPU	No
SimpleVoiceActivityDetector Microsoft.Psi.Speech	Component that performs voice activity detection via a simple heuristic using the energy in the audio stream	AnyCPU	Yes
SystemSpeechRecognizer Microsoft.Psi.Speech	Component that performs speech recognition using the desktop speech recognition engine from System.Speech.	AnyCPU	No
SystemSpeechIntentDetector Microsoft.Psi.Speech	Component that performs grammar-based intent detection using the desktop speech recognition engine from System.Speech.	AnyCPU	No
AzureSpeechRecognizer Microsoft.Psi.CognitiveServices.Speech	Component that performs speech recognition using the Microsoft Cognitive Services Speech to Text Service.	AnyCPU	Yes
LUISIntentDetector Microsoft.Psi.CognitiveServices.Language	Component that performs intent detection and entity extraction using the Microsoft Cognitive Services LUIS API.	AnyCPU	Yes
PersonalityChat Microsoft.Psi.CognitiveServices.Language	Component that generates dialogue responses to textual inputs using the Microsoft Cognitive Services Personality Chat API.	AnyCPU	Yes
Calibration
ProjectTo3D Microsoft.Psi.Calibration	Component that projects 2D color-space points into 3D camera-space points in the depth camera's coordinate system.	AnyCPU	Yes
Output
AudioPlayer Microsoft.Psi.Audio	Component that plays back an audio stream to an output device such as the speakers.	AnyCPU	Yes
SystemSpeechSynthesizer Microsoft.Psi.Speech	Component that performs speech synthesis via the desktop speech synthesis engine from System.Speech.	AnyCPU	No
SpeechSynthesizer Microsoft.Psi.CognitiveServices.Speech	Component that synthesizes speech from text strings into audio buffers.	AnyCPU	Yes
Remoting
RemoteExporter Microsoft.Psi.Remoting	Component that acts as an exporter for remoting streams over a network transport.	AnyCPU	Yes
RemoteImporter Microsoft.Psi.Remoting	Component that acts as an importer for remoting streams over a network transport.	AnyCPU	Yes
RemoteClockExporter Microsoft.Psi.Remoting	Component that exports pipeline clock information over TCP to enable synchronization.	AnyCPU	Yes
RemoteClockImporter Microsoft.Psi.Remoting	Component that imports pipeline clock information over TCP to enable synchronization.	AnyCPU	Yes
Interop
FileWriter Microsoft.Psi.Interop	Component that writes persisted files.	AnyCPU	Yes
FileSource Microsoft.Psi.Interop	Component that reads persisted files.	AnyCPU	Yes
NetMQWriter Microsoft.Psi.Interop	Component that publishes to NetMQ (ZeroMQ) topics.	AnyCPU	Yes
NetMQSource Microsoft.Psi.Interop	Component that subscribes to NetMQ (ZeroMQ) topics.	AnyCPU	Yes
TcpWriter Microsoft.Psi.Interop	Component that serializes and writes messages to a remote server over TCP.	AnyCPU	Yes
TcpSource Microsoft.Psi.Interop	Component that reads and deserializes from a remote server over TCP.	AnyCPU	Yes
Mixed Reality
PhotoVideoCamera Microsoft.Psi.MixedReality.MediaCapture	Component that captures and streams RGB images from the mixed reality device's PV (photo-video) camera.	UWP	No
Microphone Microsoft.Psi.MixedReality.MediaCapture	Component that captures and streams audio from the mixed reality device's microphone.	UWP	No
SceneUnderstanding Microsoft.Psi.MixedReality	Component that captures and streams Scene Understanding information.	UWP	No
HandsSensor Microsoft.Psi.MixedReality.OpenXR	Component that provides access to the user's hand poses via OpenXR.	AnyCPU	No
GazeSensor Microsoft.Psi.MixedReality.WinRT	Component that tracks the user's head and gaze poses	UWP	No
DepthCamera Microsoft.Psi.MixedReality.ResearchMode	Component that captures and streams depth images from the mixed reality device's depth camera.	UWP	No
VisibleLightCamera Microsoft.Psi.MixedReality.ResearchMode	Component that captures and streams grayscale images from the mixed reality device's visible light cameras.	UWP	No
Gyroscope Microsoft.Psi.MixedReality.ResearchMode	Component that captures and streams a mixed reality device's gyroscope data.	UWP	No
Accelerometer Microsoft.Psi.MixedReality.ResearchMode	Component that captures and streams a mixed reality device's accelerometer data.	UWP	No
Magnetometer Microsoft.Psi.MixedReality.ResearchMode	Component that captures and streams a mixed reality device's magnetometer data.	UWP	No
Microphone Microsoft.Psi.MixedReality.StereoKit	Component that captures and streams audio from the mixed reality device's microphone.	AnyCPU	No
HeadSensor Microsoft.Psi.MixedReality.StereoKit	Component that tracks the user's head pose.	AnyCPU	No
EyesSensor Microsoft.Psi.MixedReality.StereoKit	Component that tracks the user's eye gaze.	AnyCPU	No
HandsSensor Microsoft.Psi.MixedReality.StereoKit	Component that tracks the user's hand poses.	AnyCPU	No
Handle Microsoft.Psi.MixedReality.StereoKit	Component that implements a moveable StereoKit UI handle.	AnyCPU	No
SpatialSound Microsoft.Psi.MixedReality.StereoKit	Component that renders audio at a particular 3D location.	AnyCPU	No
MeshRenderer Microsoft.Psi.MixedReality.StereoKit	Component that renders a single StereoKit mesh.	AnyCPU	No
Mesh3DRenderer Microsoft.Psi.MixedReality.StereoKit	Component that renders a Mesh3D using StereoKit.	AnyCPU	No
Rectangle3DRenderer Microsoft.Psi.MixedReality.StereoKit	Component that renders a Rectangle3D using StereoKit.	AnyCPU	No
Box3DRenderer Microsoft.Psi.MixedReality.StereoKit	Component that renders a Box3D using StereoKit.	AnyCPU	No
EncodedImageRectangle3DRenderer Microsoft.Psi.MixedReality.StereoKit	Component that renders an encoded image placed on a Rectangle3D using StereoKit.	AnyCPU	No
TextRenderer Microsoft.Psi.MixedReality.StereoKit	Component that renders text on a billboard using StereoKit.	AnyCPU	No
HandsRenderer Microsoft.Psi.MixedReality.StereoKit	Component that controls how StereoKit renders the user's hands.	AnyCPU	No
OpenXRHandRenderer Microsoft.Psi.MixedReality.StereoKit	Component that controls how StereoKit renders the user's hands.	AnyCPU	No

You might also be interested in exploring the repositories below containing components for the Platform for Situated Intelligence ecosystem written by third parties, not affiliated with Microsoft. Microsoft makes NO WARRANTIES about these components, including about their usability or reliability.

Repo	Description
https://github.com/bsu-slim/psi-components	Components developed by the SLIM research group at Boise State University.
https://github.com/CMU-TBD/TBD.Psi.Components	Components developed by the Transportation, Bots, & Disability Lab at the Carnegie Mellon University.