General

• Format
• Message

Data Types

• Error
• Success
• Pong
• Session
• Participant
• ParticipantSummary
• Note
• ChatMessage
• CanvasElement
• FilterDict
• Filter
• RotationFilter
• GroupFilter
• SetFiltersRequest
• SetGroupFiltersRequest
• KickRequest
• KickNotification
• SessionIdRequest
• MuteRequest
• ConnectionProposal
• ConnectionOffer
• ConnectionAnswer
• RTCSessionDescription
• AddIceCandidate
• RTCIceCandidate
• GetFiltersDataRequest
• GetFiltersDataSendToParticipantRequest
• FiltersData
• FilterData
• FilterConfig

Required variable: <name>: <type>

Optional variable: <name>?: <type>

Object: { … }

Array: [ … ]

{
	type: string,
	data: any
}

• type: API endpoint. Possible types see bellow. For endpoint details, see Backend API.
• data: contents of the message. e.g. for type “saveSession” data: Session

For API details, see Backend API.

{
	code: number,
	type: string,
	description: string
}

• code: HTTP response status code. See mdn web docs
• type: error type, see bellow.
• description: error description

• NOT_IMPLEMENTED
• INTERNAL_SERVER_ERROR
• INVALID_REQUEST
• INVALID_DATATYPE
• UNKNOWN_ID
• DUPLICATE_ID
• UNKNOWN_SESSION
• UNKNOWN_EXPERIMENT
• UNKNOWN_PARTICIPANT
• UNKNOWN_USER
• UNKNOWN_SUBCONNECTION_ID
• BANNED_PARTICIPANT
• PARTICIPANT_ALREADY_CONNECTED
• EXPERIMENT_ALREADY_STARTED
• INVALID_PARAMETER
• NOT_CONNECTED_TO_EXPERIMENT
• EXPERIMENT_RUNNING
• ALREADY_JOINED_EXPERIMENT

{
	handled_time: number
	ping_data: any
}

• handled_time: time the PING message was handled which triggered the sending of this PONG message.
• ping_data: data from a PING message.

{
	type: string
	description: string
}

• type: success types, could be used to identify the success message with a previous API request. E.g. "SAVED_SESSION". Can be used internally to identify what happened
• description: Description of successful request that can be used to inform the user. e.g. “Successfully saved session”.

{
	id: string,
	title: string,
	description: string,
	date: number,
	time_limit: number,
	record: boolean,
	participants: [ Participant ],
	creation_time: number,
	start_time: number,
	end_time: number,
	notes: [ Note ],
	log: WIP,
}

• id: unique session id generated by the backend after saving the session. When creating a new session, this field is initially left blank.
  • Default: "", read only for client
• title: experiment title set by the experimenter
• description: experiment description set by the experimenter
• date: planned starting date / time of the experiment in milliseconds since January 1, 1970, 00:00:00 (UTC)
• time_limit: experiment time limit in milliseconds since January 1, 1970, 00:00:00 (UTC)
• record: whether the experiment is to be recorded
• participants: list of invited participants (see Participant)
• creation_time: time the experiment was created
  • Default: 0, read only for client
• start_time: time the experiment started in milliseconds since January 1, 1970, 00:00:00 (UTC).
  • Default: 0, read only for client
• end_time: time the experiment ended in milliseconds since January 1, 1970, 00:00:00 (UTC).
  • Default: 0, read only for client
• notes: list of Notes the experimenter added during the experiment.
  • Default: [], read only for client
• log: log created by the backend.
  • Default: [], read only for client

• Define log

{
	id: string,
	participant_name: string,
	muted_video: boolean,
	muted_audio: boolean,
	position: {
		x: number,
		y: number,
		z: number
	},
	size: {
		width: number,
		height: number
	},
	chat: [ ChatMessage ],
	banned: boolean,
	view: [ CanvasElement ],
        video_filters: [ Filter ],
        audio_filters: [ Filter ],
        video_group_filters: [ GroupFilter ],
        audio_group_filters: [ GroupFilter ],
}

• id: unique id for this participant in a Session, generated by the backend after saving the Participant / Session. When creating a new Participant in a Session, this field is initially left blank.
  • Default: "", read only for client
• name: name of the participant
• muted_video: whether the participants' video is forcefully muted by the experimenter.
• muted_audio: whether the participants' audio is forcefully muted by the experimenter.
• position: x, y, z coordinates of the participant's video feed on the canvas
• size: width and height of the participant's video feed on the canvas
• chat: chat log between experimenter and participant. See ChatMessage
• banned: whether the participant is banned from the experiment
• view: the asymmetric view that the participant sees in a session. See CanvasElement
• video_filters: Active video Filters for this participant
• audio_filters: Active audio Filters for this participant
• video_group_filters: Active video GroupFilters for this participant
• audio_group_filters: Active audio GroupFilters for this participant

{
	participant_name: string,
	position: {
		x: number,
		y: number,
		z: number,
	},
	size: {
		width: number,
		height: number
	},
	chat: [ ChatMessage ],
}

• participant_name: name of the participant
• position: x, y, z coordinates of the participant's video feed on the canvas
• size: width and height of the participant's video feed on the canvas
• chat: chat log between experimenter and participant. See ChatMessage

{
	time: number,
	speakers: [ string ],
	content: string
}

• time: time the note was saved in milliseconds since January 1, 1970, 00:00:00 (UTC)
• speakers: list of Participant IDs for participants speaking when the note was saved
• content: text content the experimenter wrote

Message format for sending messages between users.

{
	message: string,
	time: number,
	author: string
	target: string
}

• message: Message contents
• time: Time the message was send in milliseconds since January 1, 1970, 00:00:00 (UTC)
• author: Author of the message. Participant ID or "experimenter"
• target: Intended receiver
  • For participant: always "experimenter"
  • For experimenter: specific participant ID or "participants" for broadcast

Object that contains the information of one video stream on a canvas

{
	id: string,
	participant_name: string,
	size: {
		width: number,
		height: number
	},
	position: {
		x: number,
		y: number,
		z: number,
	},
}

• id: id of the participant
• participant_name: name of the participant
• size: width and height of the participant's video feed on the canvas
• position: x, y, z coordinates of the participant's video feed on the canvas

{
	id: string,
	name: string,
	channel: string,
	groupFilter: boolean,
	config: FilterConfig
}

FilterConfig = {
  [key: string]: FilterConfigArray | FilterConfigNumber;
}

FilterConfigNumber = {
	min: number;
	max: number;
	step: number;
	value: number;
	defaultValue: number;
}

FilterConfigArray = {
	value: string;
	defaultValue: string[];
	requiresOtherFilter: boolean;
}

• id: filter id, can be used to access this filter from another filter
• name: filter name (unique identifier)
• channel: filter channel, describes if the filter is either a "video", "audio" or "both"
• groupFilter: True for a GroupFilter, False for a Filter
• config: Filter configuration. Contains all variables of the filter that need to be configured upon experiment start. Necessary for dynamic filters
  • each config has a unique key with either
    • a FilterConfigNumber or
    • a FilterConfigArray

FilterDict with groupFilter = False.

{
	id: string,
	name: string,
	channel: string,
	groupFilter: boolean = False,
	config: FilterConfig
}

• MUTE_AUDIO
• MUTE_VIDEO
• DELAY
• ROTATION
• EDGE_OUTLINE
• FILTER_API_TEST
• SPEAKING_TIME
• GLASESS_DETECTION

Extends Filter. This filter needs two variables as input (direction, angle). Direction can be either clockwise or anti-clockwise. Angle needs a number as input between 1 and 180. The filter config then looks like:

{
	"name": name,
	"id": id,
	"channel": "video",
	"groupFilter": False,
	"config": {
		"direction": {
			"defaultValue": ["clockwise", "anti-clockwise"],
			"value": "clockwise",
			"requiresOtherFilter": "false",
		},
		"angle": {
			"min": 1,
			"max": 180,
			"step": 1,
			"value": 45,
			"defaultValue": 45,
		},
	},
}

FilterDict with groupFilter = True.

{
	id: string,
	name: string,
	channel: string,
	groupFilter: boolean = True,
	config: FilterConfig
}

• SYNC_SCORE

{
	participant_id: string,
	audio_filters: [ Filter ],
	video_filters: [ Filter ]
}

• participant_id: all or Participant ID for the requested endpoint. If participant_id is all, filters will be applied to all participants
• audio_filters: list of audio filters. See Filter
• video_filters: list of video filters. See Filter

{
	audio_group_filters: [ GroupFilter ],
	video_group_filters: [ GroupFilter ]
}

• audio_group_filters: list of audio group filters. See Group Filter
• video_group_filters: list of video group filters. See Group Filter

{
	participant_id: string
	reason: string
}

• participant_id: ID of the participant that should be kicked
• reason: Reason for kicking the participant

{
	reason: string
}

• reason: Reason for being kicked

{
	session_id: string
}

• session_id: Session ID for the requested endpoint

{
	participant_id: string
	mute_video: boolean
	mute_audio: boolean
}

• participant_id: Participant ID for the requested endpoint
• mute_video: Whether the participants video should be muted
• mute_audio: Whether the participants audio should be muted

{
	filter_name: string
	filter_channel: string
	filter_id: string
}

• filter_name: Name of the requested filter.
• filter_id: Filter ID of the requested filter. Can be either the id or 'all' for all filters with this name.
• filter_channel: Filter channel of the requested filter. Can be either 'audio', 'video' or 'both'.

{
	participant_id: string
	filter_name: string
	filter_channel: string
	filter_id: string
}

• participant_id: Participant ID for the requested endpoint. Can be either id of one participant         or 'all' for all participants.
• filter_name: Name of the requested filter.
• filter_id: Filter ID of the requested filter. Can be either the id or 'all' for all filters with this name.
• filter_channel: Filter channel of the requested filter. Can be either 'audio', 'video' or 'both'.

{
	video: List[ FilterData ]
        audio: List[ FilterData ]
}

• video: list of video filter data.
• audio: list of audio filter data.

{
	id : string
        data : any
}

• id: filter id
• data: filter data. Contains the data to be sent.

{
	id: '9b977403-639b-44d0-995a-f061a52c6170',
        data: {"Glasses Detected": True}
}

{
	TEST: Filter[],
	SESSION: Filter[]
}

• TEST: includes all filter configs for Testing
• SESSION: includes all filter configs for the Session

{
	id: string,
	participant_summary: ParticipantSummary | null,
}

• id: Identifier of this offer. The ID should not be related to any other ID, e.g. participant ID. Must be used in the ConnectionAnswer to identify the answer
• participant_summary: Optional summary for the participant the new offer is for. See ParticipantSummary

{
	id: string,
	offer: RTCSessionDescription,
	participant_summary: ParticipantSummary | null,
}

• id: Identifier of this offer. The ID should not be related to any other ID, e.g. participant ID. Must be an ID received from a ConnectionProposal and used in the ConnectionAnswer to identify the answer
• offer: WebRtc offer, see RTCSessionDescription

{
	id: string,
	answer: RTCSessionDescription,
}

• id: ID from the initial ConnectionProposal and ConnectionOffer
• answer: WebRtc answer, see RTCSessionDescription

{
	sdp: string,
	type: "offer" | "pranswer" | "answer" | "rollback",
}

• sdp: Session Description Protocol
• type: Type of the session description

{
	id: string,
	candidate: RTCIceCandidate
}

• id: ID of the sub-connection which is associated with the candidate
• candidate: candidate sent by client, see RTCIceCandidate

{
	candidate: string,
	sdpMid: string,
 	sdpLineIndex: integer,
	usernameFragment: string
}

• candidate: A string describing the properties of the candidate
• sdpMid: A string containing the identification tag of the media stream with which the candidate is associated,
• sdpLineIndex: A number property containing the zero-based index of the m-line with which the candidate is associated, within the SDP of the media description,
• usernameFragment: A string containing the username fragment

See MDN docs for more details.