Basic_Function - aopacloud/aopa-rtc GitHub Wiki

Take Android as an example

Set_audio_encoding_properties

Different apps need to set different audio coding attributes or application scenarios. This article describes how to set appropriate audio coding properties and application scenarios in your app with the Opa RTC SDK.

Technical Principle​

The SDK uses the 'AUDIO_PROFILE_DEFAULT' encoding attribute and the 'AUDIO_SCENARIO_DEFAULT' application scenario by default. If the default settings cannot meet your needs, call the following API to set audio coding properties and application scenarios.

|API | Description| |: -- |: --| |'create (config. mAudioScenario)' | Set the audio application scenario when creating the 'RtcEngine' instance. The default value is' AUDIO_SCENARIO_DEFAULT '| |setAudioProfile (profile) | Audio coding properties can be set before and after adding channels| |'setAudioScenario '| Audio application scenarios can be set before and after adding channels|

Precondition​

Before setting audio coding properties and application scenarios, please ensure that basic real-time audio and video functions have been implemented in your project. See [Realize audio and video interaction] for details

Implementation method​

This section describes how to set audio coding properties and application scenarios for common applications. You can add the following sample code to your project's `/app/java/com. example< Projectname>/MainActivity 'file.

1-to-1 interactive teaching​

The main requirements of one-to-one interactive teaching are to ensure the quality of calls and smooth transmission. Add the following code to your project:

Java

config.mAudioScenario = Constants.AudioScenario.getValue(Constants.AudioScenario.DEFAULT);  
engine = RtcEngine.create(config);  
RtcEngine.setAudioProfile(Constants.AUDIO_PROFILE_DEFAULT);  

Game in black​

This scenario mainly requires voice transmission, no noise, and code rate saving. Opa recommends the following settings:

Java

config.mAudioScenario = Constants.AudioScenario.getValue(Constants.AudioScenario.CHATROOM);  
engine = RtcEngine.create(config);  
RtcEngine.setAudioProfile(Constants.AUDIO_PROFILE_SPEECH_STANDARD);  

Script killing​

The main requirements of this scene are good voice performance, no volume and sound quality changes when loading and unloading microphones. Opa recommends the following settings:

Java

config.mAudioScenario = Constants.AudioScenario.getValue(Constants.AudioScenario.CHATROOM);  
engine = RtcEngine.create(config);  
RtcEngine.setAudioProfile(Constants.AUDIO_PROFILE_MUSIC_STANDARD);  

KTV​

KTV mainly requires high sound quality and good expression of music and singing. Add the following code to your project:

Java

config.mAudioScenario = Constants.AudioScenario.getValue(Constants.AudioScenario.GAME_STREAMING);  
engine = RtcEngine.create(config);  
RtcEngine.setAudioProfile(Constants.AUDIO_PROFILE_MUSIC_HIGH_QUALITY);  

Voice radio​

Voice radio generally uses professional audio equipment, which mainly requires high sound quality and stereo sound. Add the following code to your project:

Java

config.mAudioScenario = Constants.AudioScenario.getValue(Constants.AudioScenario.GAME_STREAMING);  
engine = RtcEngine.create(config);   
RtcEngine.setAudioProfile(Constants.AUDIO_PROFILE_MUSIC_HIGH_QUALITY_STEREO);  

Music Teaching​

This scene mainly requires high sound quality, and supports the transmission of sound effects played by speakers to the remote end. Opa recommends the following settings:

Java

config.mAudioScenario = Constants.AudioScenario.getValue(Constants.AudioScenario.GAME_STREAMING);  
engine = RtcEngine.create(config);  
RtcEngine.setAudioProfile(Constants.AUDIO_PROFILE_MUSIC_STANDARD_STEREO);  

Double teacher class​

This scene mainly requires high sound quality, presenting rich sound effects, and there is no change in volume and sound quality when loading and unloading. Opa recommends the following settings:

Java

  
config.mAudioScenario = Constants.AudioScenario.getValue(Constants.AudioScenario.CHATROOM);  
engine = RtcEngine.create(config);  
RtcEngine.setAudioProfile(Constants.AUDIO_PROFILE_MUSIC_STANDARD_STEREO);  

Related information​

This section provides information that may be required when setting audio coding properties and application scenarios.

Related document ​

  • [How do I distinguish between media volume and call volume?]
  • [Achieve high sound quality]

Sample project​

Opa provides an open source sample project to set audio coding properties and application scenario functions for your reference. You can download or view the source code.

Set_video_encoding_properties

This article describes how to adjust the API provided by Opa to set video encoding properties.

Function description​

Video coding attributes include video resolution, frame rate, bit rate and other parameter settings that affect video quality. You can control how video streams are displayed under different network conditions by setting video coding properties.

The Orpa SDK provides the 'setVideoEncoderConfiguration' method to set video encoding properties. You can call this method before 'enableVideo' to speed up the first frame plotting time; This method can also be called in the channel to flexibly adjust the video coding properties.

Precondition​

Before operating, please ensure that you have implemented the basic real-time audio and video functions in the project. See [Realize audio and video interaction] for details Implementation method​

You can refer to the following sample code to set various video coding parameters in your project:

Java

VideoEncoderConfiguration.VideoDimensions value = VD_640x360;  
try {  
    Field tmp = VideoEncoderConfiguration.class.getDeclaredField(dimension.getSelectedItem(toString()); 
    tmp.setAccessible(true);
    value = (VideoEncoderConfiguration.VideoDimensions)
    tmp.get(null);} catch (NoSuchFieldException e) {  
    Log.e("Field", "Can not find field " + dimension.getSelectedItem().toString());} catch (IllegalAccessException e) {  
    Log.e("Field", "Could not access field " + dimension.getSelectedItem().toString());}  
  
    engine.setVideoEncoderConfiguration(new VideoEncoderConfiguration(  
        value,        VideoEncoderConfiguration.FRAME_RATE.valueOf(framerate.getSelectedItem().toString()),        
    Integer.valueOf(et_bitrate.getText().toString()),        
    VideoEncoderConfiguration.ORIENTATION_MODE.valueOf(orientation.getSelectedItem().toStri())));  

Notes on development​

  • The parameter values in 'setVideoEncoderConfiguration' are the maximum values in the ideal network state. The OPA SDK will make adaptive adjustments to the set parameters according to the real-time network environment and equipment, and usually will lower the parameters.
  • The parameter settings in 'setVideoEncoderConfiguration' may affect billing. If the parameter is lowered due to adaptation, the charging shall be based on the video resolution actually subscribed by the user. See [Billing Policy] for details

Reference information​

More information about the methods used in this article and links to related pages.

Sample project​

Opa has provided an open source example project of real-time audio and video interaction for your reference. You can go to download or view the source code.

*GitHub: [AopaApi Example]( https://github.com/olaola-chat/AopaApi-Example/ οΌ‰

Recommended value of video parameters​

Generally speaking, the selection of video coding parameters should be determined according to the actual situation and scene of the product. For example, in one-on-one teaching scenarios, teachers and students have relatively large windows, requiring higher resolution, and then higher frame rate and code rate; If it is a one-to-four scene, the windows of teachers and students are relatively small, the resolution can be lower, and the corresponding code rate and frame rate will also be lower, so as to reduce the resource consumption of encoding and decoding and relieve the pressure on downlink bandwidth.

Generally, it can be set according to the recommended values in the following scenarios.

  • Two person video call scenario:
    • Resolution 320 Γ— 240, frame rate 15 fps, code rate 200 Kbps
    • Resolution 640 Γ— 360, frame rate 15 fps, code rate 400 Kbps
  • One to many video call scenario:
    • Resolution 160 Γ— 120, frame rate 15 fps, code rate 65 Kbps
    • Resolution 320 Γ— 180, frame rate 15 fps, code rate 140 Kbps
    • Resolution 320 Γ— 240, frame rate 15 fps, code rate 200 Kbps

Resolution, frame rate and code rate​

The parameters of video coding attributes are as follows:

  • 'dimensions': video encoding resolution (px). The default value is 960 Γ— 540. Generally, the higher the resolution, the better the video definition. The value of this parameter does not represent the direction of the final video output. Click [Rotation Direction Mode] (about: blank #% E6% 97% 8B% E8% BD% AC% E6% 96% B9% E5% 90% 91% E6% A8% A1% E5% BC% 8F) to learn how to set the direction mode of video output.

  • 'frameRate': the frame rate (fps) of video encoding, that is, how many frames are encoded per second. The default value is 15. Generally, the larger the frame rate, the smoother the picture. This parameter can be set to 20 or 25 in scenes with high requirements for video smoothness. It is not recommended to set the 'frameRate' to be greater than 30.

  • bitrate:video coding rate (Kbps). The default value is' STANDARD_BITRATE ', that is, the standard bit rate mode. In the standard bit rate mode, the SDK will set a suitable bit rate for you according to the set channel scene, resolution and frame rate.

  • minBitrate:If you have special requirements for video quality, you can set it through the' minBitrate 'parameter` MinBitrate 'is the lowest video coding rate (Kbps). Opa SDK will adapt the bit rate according to the network conditions. When this parameter is set to be greater than the default value, the video encoder will be forced to output high-quality video images, but it may lead to higher packet loss rate and affect the smoothness of video playback.

information

  • The default value of 'minBitrate' can meet the requirements of most real-time scenarios. In general, Opa recommends that you do not change the default value.
  • In order to obtain high-quality video, it is necessary to keep the resolution, bit rate and frame rate in a relatively balanced state. Higher resolution requires higher bit rate. In the case of a certain bit rate, too high a frame rate will reduce the resolution.
  • When the video cannot reach the maximum value of the set resolution, frame rate or code rate due to network environment and other reasons, the value will be as close to the maximum value as possible.

Opa SDK provides a variety of resolutions and frame rates for selection. You can also define them according to the table below.

take care

After you set the resolution and frame rate, the SDK will automatically match the corresponding benchmark code rate. Opa does not recommend you to modify the code rate yourself.

Resolution (W Γ— H) Frame rate (fps)
160 Γ— 120 15
120 Γ— 120 15
320 Γ— 180 15
180 Γ— 180 15
240 Γ— 180 15
320 Γ— 240 15
240 Γ— 240 15
424 Γ— 240 15
640 Γ— 360 15
360 Γ— 360 15
640 Γ— 360 30
360 Γ— 360 30
480 Γ— 360 15
480 Γ— 360 30
640 Γ— 480 15
480 Γ— 480 15
640 Γ— 480 30
480 Γ— 480 30
848 Γ— 480 15
848 Γ— 480 30
640 Γ— 480 10
960 Γ— 540 15
960 Γ— 540 30
1280 Γ— 720 15
1280 Γ— 720 30
960 Γ— 720 15
960 Γ— 720 30
1920 Γ— 1080 15
1920 Γ— 1080 30
1920 Γ— 1080 60
2560 Γ— 1440 30
2560 Γ— 1440 60
3840 Γ— 2160 30
3840 Γ— 2160 60

Video mode ​

The video mode includes a variety of settings. You can adjust the direction, image quality and image effect of the video according to business needs.

Rotation direction mode​

In the video rotation scene, we mainly focus on the behavior of the acquisition end and the playback end. Including:

  • The acquisition terminal collects video and outputs it. That is, the relative position of the video and the Status Bar.
  • The player renders the received video image and rotates the video according to the received rotation information and the relative position of its own Status Bar.

In order to prevent video from big head, scaling or cutting due to rotation, the OrpaSDK provides the 'orientationMode' parameter in the 'setVideoEncoderConfiguration'. You can use this parameter to obtain the desired video rendering effect in combination with the needs of the video scene.

`The orientationMode parameter provides three modes to meet different user needs: 'ORIENTATION_MODE_ADAPTIVE', 'ORIENTATION_MODE_FIXED_LANDSCAPE', and 'ORIENTATION_MODE_FIXED_PORTRAIT'.

No matter which mode is adopted, the OPA SDK ensures that the relative positions of the video and the Status Bar are always the same at the acquisition end and the playback end.

ORIENTATION _MODE _ADAPTIVE

In the 'ORIENTATION_MODE_ADAPTIVE' mode, the video output from the SDK is in the same direction as the captured video. The receiving end will rotate the video according to the received video rotation information. This mode is applicable to scenes where the receiver can adjust the video direction:

When using the rear camera to capture video, the video direction of the video collector and player is shown in the following figure. Note that the video direction varies depending on whether the UI is locked.

UI lock (or disable automatic rotation of app screen)

The Status Bar is consistent with the relative direction of the screen and has nothing to do with the mobile phone's gravity sensing (such as WeChat). Therefore, the relative position of the video and the screen is always the same at the video capture end and the playback end.

  • When the acquisition end is horizontal:

    ../_images/rotation_adaptive_uilock_landscape.jpg

  • When the acquisition terminal is vertical:

    ../_images/rotation_adaptive_uilock_portrait.jpg

UI is not locked (enable automatic rotation of app screen)

Regardless of the direction of the screen (for example, in Facetime), the Status Bar in App is always in the horizontal direction. Therefore, the relative direction of video and gravity is always the same at the acquisition end and the playback end.

  • When the acquisition end is horizontal:

    ../_images/rotation_adaptive_uiunlock_landscape.jpg

  • When the acquisition terminal is vertical:

    UI is not locked and acquisition end is vertical

ORIENTATION _MODE _FIXED _LANDSCAPE

In the 'ORIENTATION_MODE_FIXED_LANDSCAPE' mode, the output video is always in the horizontal screen mode relative to the Status Bar. If the captured video is in vertical screen mode, the video encoder will crop it. This mode is applicable to the case where the receiver cannot process the rotation information.

When the rear camera is used as the video collector, the video direction of the video collector and player is shown in the following figure.

  • Video captured in landscape mode (video clipping is not required):

    ../_images/rotation_fixed_landscape.jpg

  • Video captured in portrait mode (video clipping is required):

    ../_images/rotation_fixed_landscape_cut.jpg

ORIENTATION _MODE _FIXED _PORTRAIT

In the 'ORIENTATION_MODE_FIXED_PORTRAIT' mode, the output video is always in vertical screen mode relative to the Status Bar. If the captured video is in horizontal screen mode, the video encoder will cut it. This mode is applicable to the case where the receiver cannot process the rotation information.

When the rear camera is used as the video collector, the video direction of the video collector and player is shown in the following figure.

  • The captured video is in vertical screen mode (video clipping is not required):

    ../_images/rotation_fixed_portrait.jpg

  • The captured video is in horizontal screen mode (video clipping is required):

    ../_images/rotation_fixed_portrait_cut.jpg

Demote preference​

To ensure the video experience of users in weak networks, the Opa SDK also provides the 'degradationPreference' parameter to set the video coding degradation preference when bandwidth is limited.

Mirror mode​

By default, the SDK does not mirror videos during encoding. You can use the 'mirrorMode' parameter to set the image mode of video coding to determine the video picture seen by remote users.

Next chapter:advanced function