Basic_Function - aopacloud/aopa-rtc GitHub Wiki

Take Android as an example

Set_audio_encoding_properties
Set_video_encoding_properties

Set_audio_encoding_properties

Different apps need to set different audio coding attributes or application scenarios. This article describes how to set appropriate audio coding properties and application scenarios in your app with the Opa RTC SDK.

Technical Principle

The SDK uses the 'AUDIO_PROFILE_DEFAULT' encoding attribute and the 'AUDIO_SCENARIO_DEFAULT' application scenario by default. If the default settings cannot meet your needs, call the following API to set audio coding properties and application scenarios.

Precondition

Before setting audio coding properties and application scenarios, please ensure that basic real-time audio and video functions have been implemented in your project. See [Realize audio and video interaction] for details

Implementation method

This section describes how to set audio coding properties and application scenarios for common applications. You can add the following sample code to your project's `/app/java/com. example< Projectname>/MainActivity 'file.

1-to-1 interactive teaching

The main requirements of one-to-one interactive teaching are to ensure the quality of calls and smooth transmission. Add the following code to your project:

Java

config.mAudioScenario = Constants.AudioScenario.getValue(Constants.AudioScenario.DEFAULT);  
engine = RtcEngine.create(config);  
RtcEngine.setAudioProfile(Constants.AUDIO_PROFILE_DEFAULT);

Game in black

This scenario mainly requires voice transmission, no noise, and code rate saving. Opa recommends the following settings:

Java

config.mAudioScenario = Constants.AudioScenario.getValue(Constants.AudioScenario.CHATROOM);  
engine = RtcEngine.create(config);  
RtcEngine.setAudioProfile(Constants.AUDIO_PROFILE_SPEECH_STANDARD);

Script killing

The main requirements of this scene are good voice performance, no volume and sound quality changes when loading and unloading microphones. Opa recommends the following settings:

Java

config.mAudioScenario = Constants.AudioScenario.getValue(Constants.AudioScenario.CHATROOM);  
engine = RtcEngine.create(config);  
RtcEngine.setAudioProfile(Constants.AUDIO_PROFILE_MUSIC_STANDARD);

KTV

KTV mainly requires high sound quality and good expression of music and singing. Add the following code to your project:

Java

config.mAudioScenario = Constants.AudioScenario.getValue(Constants.AudioScenario.GAME_STREAMING);  
engine = RtcEngine.create(config);  
RtcEngine.setAudioProfile(Constants.AUDIO_PROFILE_MUSIC_HIGH_QUALITY);

Voice radio

Voice radio generally uses professional audio equipment, which mainly requires high sound quality and stereo sound. Add the following code to your project:

Java

config.mAudioScenario = Constants.AudioScenario.getValue(Constants.AudioScenario.GAME_STREAMING);  
engine = RtcEngine.create(config);   
RtcEngine.setAudioProfile(Constants.AUDIO_PROFILE_MUSIC_HIGH_QUALITY_STEREO);

Music Teaching

This scene mainly requires high sound quality, and supports the transmission of sound effects played by speakers to the remote end. Opa recommends the following settings:

Java

config.mAudioScenario = Constants.AudioScenario.getValue(Constants.AudioScenario.GAME_STREAMING);  
engine = RtcEngine.create(config);  
RtcEngine.setAudioProfile(Constants.AUDIO_PROFILE_MUSIC_STANDARD_STEREO);

Double teacher class

This scene mainly requires high sound quality, presenting rich sound effects, and there is no change in volume and sound quality when loading and unloading. Opa recommends the following settings:

Java

  
config.mAudioScenario = Constants.AudioScenario.getValue(Constants.AudioScenario.CHATROOM);  
engine = RtcEngine.create(config);  
RtcEngine.setAudioProfile(Constants.AUDIO_PROFILE_MUSIC_STANDARD_STEREO);

Related information

This section provides information that may be required when setting audio coding properties and application scenarios.

Sample project

Opa provides an open source sample project to set audio coding properties and application scenario functions for your reference. You can download or view the source code.

GitHub: AopaApi-Example

Set_video_encoding_properties

This article describes how to adjust the API provided by Opa to set video encoding properties.

Function description

Video coding attributes include video resolution, frame rate, bit rate and other parameter settings that affect video quality. You can control how video streams are displayed under different network conditions by setting video coding properties.

The Orpa SDK provides the 'setVideoEncoderConfiguration' method to set video encoding properties. You can call this method before 'enableVideo' to speed up the first frame plotting time; This method can also be called in the channel to flexibly adjust the video coding properties.

Precondition

Before operating, please ensure that you have implemented the basic real-time audio and video functions in the project. See [Realize audio and video interaction] for details Implementation method

You can refer to the following sample code to set various video coding parameters in your project:

Java

VideoEncoderConfiguration.VideoDimensions value = VD_640x360;  
try {  
    Field tmp = VideoEncoderConfiguration.class.getDeclaredField(dimension.getSelectedItem(toString()); 
    tmp.setAccessible(true);
    value = (VideoEncoderConfiguration.VideoDimensions)
    tmp.get(null);} catch (NoSuchFieldException e) {  
    Log.e("Field", "Can not find field " + dimension.getSelectedItem().toString());} catch (IllegalAccessException e) {  
    Log.e("Field", "Could not access field " + dimension.getSelectedItem().toString());}  
  
    engine.setVideoEncoderConfiguration(new VideoEncoderConfiguration(  
        value,        VideoEncoderConfiguration.FRAME_RATE.valueOf(framerate.getSelectedItem().toString()),        
    Integer.valueOf(et_bitrate.getText().toString()),        
    VideoEncoderConfiguration.ORIENTATION_MODE.valueOf(orientation.getSelectedItem().toStri())));

Notes on development

The parameter values in 'setVideoEncoderConfiguration' are the maximum values in the ideal network state. The OPA SDK will make adaptive adjustments to the set parameters according to the real-time network environment and equipment, and usually will lower the parameters.
The parameter settings in 'setVideoEncoderConfiguration' may affect billing. If the parameter is lowered due to adaptation, the charging shall be based on the video resolution actually subscribed by the user. See [Billing Policy] for details

Reference information

More information about the methods used in this article and links to related pages.

Sample project

Opa has provided an open source example project of real-time audio and video interaction for your reference. You can go to download or view the source code.

*GitHub: [AopaApi Example]（ https://github.com/olaola-chat/AopaApi-Example/ ）

Resolution, frame rate and code rate

The parameters of video coding attributes are as follows:

'dimensions': video encoding resolution (px). The default value is 960 × 540. Generally, the higher the resolution, the better the video definition. The value of this parameter does not represent the direction of the final video output. Click [Rotation Direction Mode] (about: blank #% E6% 97% 8B% E8% BD% AC% E6% 96% B9% E5% 90% 91% E6% A8% A1% E5% BC% 8F) to learn how to set the direction mode of video output.
'frameRate': the frame rate (fps) of video encoding, that is, how many frames are encoded per second. The default value is 15. Generally, the larger the frame rate, the smoother the picture. This parameter can be set to 20 or 25 in scenes with high requirements for video smoothness. It is not recommended to set the 'frameRate' to be greater than 30.
bitrate：video coding rate (Kbps). The default value is' STANDARD_BITRATE ', that is, the standard bit rate mode. In the standard bit rate mode, the SDK will set a suitable bit rate for you according to the set channel scene, resolution and frame rate.
minBitrate：If you have special requirements for video quality, you can set it through the' minBitrate 'parameter` MinBitrate 'is the lowest video coding rate (Kbps). Opa SDK will adapt the bit rate according to the network conditions. When this parameter is set to be greater than the default value, the video encoder will be forced to output high-quality video images, but it may lead to higher packet loss rate and affect the smoothness of video playback.

information

The default value of 'minBitrate' can meet the requirements of most real-time scenarios. In general, Opa recommends that you do not change the default value.
In order to obtain high-quality video, it is necessary to keep the resolution, bit rate and frame rate in a relatively balanced state. Higher resolution requires higher bit rate. In the case of a certain bit rate, too high a frame rate will reduce the resolution.
When the video cannot reach the maximum value of the set resolution, frame rate or code rate due to network environment and other reasons, the value will be as close to the maximum value as possible.

Opa SDK provides a variety of resolutions and frame rates for selection. You can also define them according to the table below.

take care

After you set the resolution and frame rate, the SDK will automatically match the corresponding benchmark code rate. Opa does not recommend you to modify the code rate yourself.

Resolution (W × H)	Frame rate (fps)
160 × 120	15
120 × 120	15
320 × 180	15
180 × 180	15
240 × 180	15
320 × 240	15
240 × 240	15
424 × 240	15
640 × 360	15
360 × 360	15
640 × 360	30
360 × 360	30
480 × 360	15
480 × 360	30
640 × 480	15
480 × 480	15
640 × 480	30
480 × 480	30
848 × 480	15
848 × 480	30
640 × 480	10
960 × 540	15
960 × 540	30
1280 × 720	15
1280 × 720	30
960 × 720	15
960 × 720	30
1920 × 1080	15
1920 × 1080	30
1920 × 1080	60
2560 × 1440	30
2560 × 1440	60
3840 × 2160	30
3840 × 2160	60

Video mode

The video mode includes a variety of settings. You can adjust the direction, image quality and image effect of the video according to business needs.

Rotation direction mode

In the video rotation scene, we mainly focus on the behavior of the acquisition end and the playback end. Including:

The acquisition terminal collects video and outputs it. That is, the relative position of the video and the Status Bar.
The player renders the received video image and rotates the video according to the received rotation information and the relative position of its own Status Bar.

In order to prevent video from big head, scaling or cutting due to rotation, the OrpaSDK provides the 'orientationMode' parameter in the 'setVideoEncoderConfiguration'. You can use this parameter to obtain the desired video rendering effect in combination with the needs of the video scene.

`The orientationMode parameter provides three modes to meet different user needs: 'ORIENTATION_MODE_ADAPTIVE', 'ORIENTATION_MODE_FIXED_LANDSCAPE', and 'ORIENTATION_MODE_FIXED_PORTRAIT'.

No matter which mode is adopted, the OPA SDK ensures that the relative positions of the video and the Status Bar are always the same at the acquisition end and the playback end.

ORIENTATION _MODE _ADAPTIVE

In the 'ORIENTATION_MODE_ADAPTIVE' mode, the video output from the SDK is in the same direction as the captured video. The receiving end will rotate the video according to the received video rotation information. This mode is applicable to scenes where the receiver can adjust the video direction:

When using the rear camera to capture video, the video direction of the video collector and player is shown in the following figure. Note that the video direction varies depending on whether the UI is locked.

UI lock (or disable automatic rotation of app screen)

The Status Bar is consistent with the relative direction of the screen and has nothing to do with the mobile phone's gravity sensing (such as WeChat). Therefore, the relative position of the video and the screen is always the same at the video capture end and the playback end.

When the acquisition end is horizontal:
When the acquisition terminal is vertical:

UI is not locked (enable automatic rotation of app screen)

Regardless of the direction of the screen (for example, in Facetime), the Status Bar in App is always in the horizontal direction. Therefore, the relative direction of video and gravity is always the same at the acquisition end and the playback end.

When the acquisition end is horizontal:
When the acquisition terminal is vertical:

ORIENTATION _MODE _FIXED _LANDSCAPE

In the 'ORIENTATION_MODE_FIXED_LANDSCAPE' mode, the output video is always in the horizontal screen mode relative to the Status Bar. If the captured video is in vertical screen mode, the video encoder will crop it. This mode is applicable to the case where the receiver cannot process the rotation information.

When the rear camera is used as the video collector, the video direction of the video collector and player is shown in the following figure.

Video captured in landscape mode (video clipping is not required):
Video captured in portrait mode (video clipping is required):

ORIENTATION _MODE _FIXED _PORTRAIT

In the 'ORIENTATION_MODE_FIXED_PORTRAIT' mode, the output video is always in vertical screen mode relative to the Status Bar. If the captured video is in horizontal screen mode, the video encoder will cut it. This mode is applicable to the case where the receiver cannot process the rotation information.

When the rear camera is used as the video collector, the video direction of the video collector and player is shown in the following figure.

The captured video is in vertical screen mode (video clipping is not required):
The captured video is in horizontal screen mode (video clipping is required):

Demote preference

To ensure the video experience of users in weak networks, the Opa SDK also provides the 'degradationPreference' parameter to set the video coding degradation preference when bandwidth is limited.

Mirror mode

By default, the SDK does not mirror videos during encoding. You can use the 'mirrorMode' parameter to set the image mode of video coding to determine the video picture seen by remote users.

Next chapter:advanced function

Basic_Function - aopacloud/aopa-rtc GitHub Wiki

Set_audio_encoding_properties

Technical Principle

Precondition

Implementation method

1-to-1 interactive teaching

Game in black

Script killing

KTV

Voice radio

Music Teaching

Double teacher class

Related information

Related document

Sample project

Set_video_encoding_properties

Function description

Precondition

Before operating, please ensure that you have implemented the basic real-time audio and video functions in the project. See [Realize audio and video interaction] for details Implementation method

Notes on development

Reference information

Sample project

Recommended value of video parameters

Resolution, frame rate and code rate

Video mode

Rotation direction mode

Demote preference

Mirror mode

Basic_Function - aopacloud/aopa-rtc GitHub Wiki

Set_audio_encoding_properties

Technical Principle​

Precondition​

Implementation method​

1-to-1 interactive teaching​

Game in black​

Script killing​

KTV​

Voice radio​

Music Teaching​

Double teacher class​

Related information​

Related document ​

Sample project​

Set_video_encoding_properties

Function description​

Precondition​

Before operating, please ensure that you have implemented the basic real-time audio and video functions in the project. See [Realize audio and video interaction] for details Implementation method​

Notes on development​

Reference information​

Sample project​

Recommended value of video parameters​

Resolution, frame rate and code rate​

Video mode ​

Rotation direction mode​

Demote preference​

Mirror mode​

Technical Principle

Precondition

Implementation method

1-to-1 interactive teaching

Game in black

Script killing

KTV

Voice radio

Music Teaching

Double teacher class

Related information

Related document

Sample project

Function description

Precondition

Before operating, please ensure that you have implemented the basic real-time audio and video functions in the project. See [Realize audio and video interaction] for details Implementation method

Notes on development

Reference information

Sample project

Recommended value of video parameters

Resolution, frame rate and code rate

Video mode

Rotation direction mode

Demote preference

Mirror mode