Basic_Function - aopacloud/aopa-rtc GitHub Wiki
Take Android as an example
Set_audio_encoding_properties
Different apps need to set different audio coding attributes or application scenarios. This article describes how to set appropriate audio coding properties and application scenarios in your app with the Opa RTC SDK.
β
Technical PrincipleThe SDK uses the 'AUDIO_PROFILE_DEFAULT' encoding attribute and the 'AUDIO_SCENARIO_DEFAULT' application scenario by default. If the default settings cannot meet your needs, call the following API to set audio coding properties and application scenarios.
|API | Description|
|: -- |: --|
|'create (config. mAudioScenario)' | Set the audio application scenario when creating the 'RtcEngine' instance. The default value is' AUDIO_SCENARIO_DEFAULT '|
|setAudioProfile (profile)
| Audio coding properties can be set before and after adding channels|
|'setAudioScenario '| Audio application scenarios can be set before and after adding channels|
β
PreconditionBefore setting audio coding properties and application scenarios, please ensure that basic real-time audio and video functions have been implemented in your project. See [Realize audio and video interaction] for details
β
Implementation methodThis section describes how to set audio coding properties and application scenarios for common applications. You can add the following sample code to your project's `/app/java/com. example< Projectname>/MainActivity 'file.
β
1-to-1 interactive teachingThe main requirements of one-to-one interactive teaching are to ensure the quality of calls and smooth transmission. Add the following code to your project:
Java
config.mAudioScenario = Constants.AudioScenario.getValue(Constants.AudioScenario.DEFAULT);
engine = RtcEngine.create(config);
RtcEngine.setAudioProfile(Constants.AUDIO_PROFILE_DEFAULT);
β
Game in blackThis scenario mainly requires voice transmission, no noise, and code rate saving. Opa recommends the following settings:
Java
config.mAudioScenario = Constants.AudioScenario.getValue(Constants.AudioScenario.CHATROOM);
engine = RtcEngine.create(config);
RtcEngine.setAudioProfile(Constants.AUDIO_PROFILE_SPEECH_STANDARD);
β
Script killingThe main requirements of this scene are good voice performance, no volume and sound quality changes when loading and unloading microphones. Opa recommends the following settings:
Java
config.mAudioScenario = Constants.AudioScenario.getValue(Constants.AudioScenario.CHATROOM);
engine = RtcEngine.create(config);
RtcEngine.setAudioProfile(Constants.AUDIO_PROFILE_MUSIC_STANDARD);
β
KTVKTV mainly requires high sound quality and good expression of music and singing. Add the following code to your project:
Java
config.mAudioScenario = Constants.AudioScenario.getValue(Constants.AudioScenario.GAME_STREAMING);
engine = RtcEngine.create(config);
RtcEngine.setAudioProfile(Constants.AUDIO_PROFILE_MUSIC_HIGH_QUALITY);
β
Voice radioVoice radio generally uses professional audio equipment, which mainly requires high sound quality and stereo sound. Add the following code to your project:
Java
config.mAudioScenario = Constants.AudioScenario.getValue(Constants.AudioScenario.GAME_STREAMING);
engine = RtcEngine.create(config);
RtcEngine.setAudioProfile(Constants.AUDIO_PROFILE_MUSIC_HIGH_QUALITY_STEREO);
β
Music TeachingThis scene mainly requires high sound quality, and supports the transmission of sound effects played by speakers to the remote end. Opa recommends the following settings:
Java
config.mAudioScenario = Constants.AudioScenario.getValue(Constants.AudioScenario.GAME_STREAMING);
engine = RtcEngine.create(config);
RtcEngine.setAudioProfile(Constants.AUDIO_PROFILE_MUSIC_STANDARD_STEREO);
β
Double teacher classThis scene mainly requires high sound quality, presenting rich sound effects, and there is no change in volume and sound quality when loading and unloading. Opa recommends the following settings:
Java
config.mAudioScenario = Constants.AudioScenario.getValue(Constants.AudioScenario.CHATROOM);
engine = RtcEngine.create(config);
RtcEngine.setAudioProfile(Constants.AUDIO_PROFILE_MUSIC_STANDARD_STEREO);
β
Related informationThis section provides information that may be required when setting audio coding properties and application scenarios.
β
Related document- [How do I distinguish between media volume and call volume?]
- [Achieve high sound quality]
β
Sample projectOpa provides an open source sample project to set audio coding properties and application scenario functions for your reference. You can download or view the source code.
- GitHub: AopaApi-Example
Set_video_encoding_properties
This article describes how to adjust the API provided by Opa to set video encoding properties.
β
Function descriptionVideo coding attributes include video resolution, frame rate, bit rate and other parameter settings that affect video quality. You can control how video streams are displayed under different network conditions by setting video coding properties.
The Orpa SDK provides the 'setVideoEncoderConfiguration' method to set video encoding properties. You can call this method before 'enableVideo' to speed up the first frame plotting time; This method can also be called in the channel to flexibly adjust the video coding properties.
β
Preconditionβ
Before operating, please ensure that you have implemented the basic real-time audio and video functions in the project. See [Realize audio and video interaction] for details Implementation methodYou can refer to the following sample code to set various video coding parameters in your project:
Java
VideoEncoderConfiguration.VideoDimensions value = VD_640x360;
try {
Field tmp = VideoEncoderConfiguration.class.getDeclaredField(dimension.getSelectedItem(toString());
tmp.setAccessible(true);
value = (VideoEncoderConfiguration.VideoDimensions)
tmp.get(null);} catch (NoSuchFieldException e) {
Log.e("Field", "Can not find field " + dimension.getSelectedItem().toString());} catch (IllegalAccessException e) {
Log.e("Field", "Could not access field " + dimension.getSelectedItem().toString());}
engine.setVideoEncoderConfiguration(new VideoEncoderConfiguration(
value, VideoEncoderConfiguration.FRAME_RATE.valueOf(framerate.getSelectedItem().toString()),
Integer.valueOf(et_bitrate.getText().toString()),
VideoEncoderConfiguration.ORIENTATION_MODE.valueOf(orientation.getSelectedItem().toStri())));
β
Notes on development- The parameter values in 'setVideoEncoderConfiguration' are the maximum values in the ideal network state. The OPA SDK will make adaptive adjustments to the set parameters according to the real-time network environment and equipment, and usually will lower the parameters.
- The parameter settings in 'setVideoEncoderConfiguration' may affect billing. If the parameter is lowered due to adaptation, the charging shall be based on the video resolution actually subscribed by the user. See [Billing Policy] for details
β
Reference informationMore information about the methods used in this article and links to related pages.
β
Sample projectOpa has provided an open source example project of real-time audio and video interaction for your reference. You can go to download or view the source code.
*GitHub: [AopaApi Example]οΌ https://github.com/olaola-chat/AopaApi-Example/ οΌ
β
Recommended value of video parametersGenerally speaking, the selection of video coding parameters should be determined according to the actual situation and scene of the product. For example, in one-on-one teaching scenarios, teachers and students have relatively large windows, requiring higher resolution, and then higher frame rate and code rate; If it is a one-to-four scene, the windows of teachers and students are relatively small, the resolution can be lower, and the corresponding code rate and frame rate will also be lower, so as to reduce the resource consumption of encoding and decoding and relieve the pressure on downlink bandwidth.
Generally, it can be set according to the recommended values in the following scenarios.
- Two person video call scenario:
- Resolution 320 Γ 240, frame rate 15 fps, code rate 200 Kbps
- Resolution 640 Γ 360, frame rate 15 fps, code rate 400 Kbps
- One to many video call scenario:
- Resolution 160 Γ 120, frame rate 15 fps, code rate 65 Kbps
- Resolution 320 Γ 180, frame rate 15 fps, code rate 140 Kbps
- Resolution 320 Γ 240, frame rate 15 fps, code rate 200 Kbps
β
Resolution, frame rate and code rateThe parameters of video coding attributes are as follows:
-
'dimensions': video encoding resolution (px). The default value is 960 Γ 540. Generally, the higher the resolution, the better the video definition. The value of this parameter does not represent the direction of the final video output. Click [Rotation Direction Mode] (about: blank #% E6% 97% 8B% E8% BD% AC% E6% 96% B9% E5% 90% 91% E6% A8% A1% E5% BC% 8F) to learn how to set the direction mode of video output.
-
'frameRate': the frame rate (fps) of video encoding, that is, how many frames are encoded per second. The default value is 15. Generally, the larger the frame rate, the smoother the picture. This parameter can be set to 20 or 25 in scenes with high requirements for video smoothness. It is not recommended to set the 'frameRate' to be greater than 30.
-
bitrate
οΌvideo coding rate (Kbps). The default value is' STANDARD_BITRATE ', that is, the standard bit rate mode. In the standard bit rate mode, the SDK will set a suitable bit rate for you according to the set channel scene, resolution and frame rate. -
minBitrate
οΌIf you have special requirements for video quality, you can set it through the' minBitrate 'parameter` MinBitrate 'is the lowest video coding rate (Kbps). Opa SDK will adapt the bit rate according to the network conditions. When this parameter is set to be greater than the default value, the video encoder will be forced to output high-quality video images, but it may lead to higher packet loss rate and affect the smoothness of video playback.
information
- The default value of 'minBitrate' can meet the requirements of most real-time scenarios. In general, Opa recommends that you do not change the default value.
- In order to obtain high-quality video, it is necessary to keep the resolution, bit rate and frame rate in a relatively balanced state. Higher resolution requires higher bit rate. In the case of a certain bit rate, too high a frame rate will reduce the resolution.
- When the video cannot reach the maximum value of the set resolution, frame rate or code rate due to network environment and other reasons, the value will be as close to the maximum value as possible.
Opa SDK provides a variety of resolutions and frame rates for selection. You can also define them according to the table below.
take care
After you set the resolution and frame rate, the SDK will automatically match the corresponding benchmark code rate. Opa does not recommend you to modify the code rate yourself.
Resolution (W Γ H) | Frame rate (fps) |
---|---|
160 Γ 120 | 15 |
120 Γ 120 | 15 |
320 Γ 180 | 15 |
180 Γ 180 | 15 |
240 Γ 180 | 15 |
320 Γ 240 | 15 |
240 Γ 240 | 15 |
424 Γ 240 | 15 |
640 Γ 360 | 15 |
360 Γ 360 | 15 |
640 Γ 360 | 30 |
360 Γ 360 | 30 |
480 Γ 360 | 15 |
480 Γ 360 | 30 |
640 Γ 480 | 15 |
480 Γ 480 | 15 |
640 Γ 480 | 30 |
480 Γ 480 | 30 |
848 Γ 480 | 15 |
848 Γ 480 | 30 |
640 Γ 480 | 10 |
960 Γ 540 | 15 |
960 Γ 540 | 30 |
1280 Γ 720 | 15 |
1280 Γ 720 | 30 |
960 Γ 720 | 15 |
960 Γ 720 | 30 |
1920 Γ 1080 | 15 |
1920 Γ 1080 | 30 |
1920 Γ 1080 | 60 |
2560 Γ 1440 | 30 |
2560 Γ 1440 | 60 |
3840 Γ 2160 | 30 |
3840 Γ 2160 | 60 |
β
Video modeThe video mode includes a variety of settings. You can adjust the direction, image quality and image effect of the video according to business needs.
β
Rotation direction modeIn the video rotation scene, we mainly focus on the behavior of the acquisition end and the playback end. Including:
- The acquisition terminal collects video and outputs it. That is, the relative position of the video and the Status Bar.
- The player renders the received video image and rotates the video according to the received rotation information and the relative position of its own Status Bar.
In order to prevent video from big head, scaling or cutting due to rotation, the OrpaSDK provides the 'orientationMode' parameter in the 'setVideoEncoderConfiguration'. You can use this parameter to obtain the desired video rendering effect in combination with the needs of the video scene.
`The orientationMode parameter provides three modes to meet different user needs: 'ORIENTATION_MODE_ADAPTIVE', 'ORIENTATION_MODE_FIXED_LANDSCAPE', and 'ORIENTATION_MODE_FIXED_PORTRAIT'.
No matter which mode is adopted, the OPA SDK ensures that the relative positions of the video and the Status Bar are always the same at the acquisition end and the playback end.
ORIENTATION _MODE _ADAPTIVE
In the 'ORIENTATION_MODE_ADAPTIVE' mode, the video output from the SDK is in the same direction as the captured video. The receiving end will rotate the video according to the received video rotation information. This mode is applicable to scenes where the receiver can adjust the video direction:
When using the rear camera to capture video, the video direction of the video collector and player is shown in the following figure. Note that the video direction varies depending on whether the UI is locked.
UI lock (or disable automatic rotation of app screen)
The Status Bar is consistent with the relative direction of the screen and has nothing to do with the mobile phone's gravity sensing (such as WeChat). Therefore, the relative position of the video and the screen is always the same at the video capture end and the playback end.
-
When the acquisition end is horizontal:
-
When the acquisition terminal is vertical:
UI is not locked (enable automatic rotation of app screen)
Regardless of the direction of the screen (for example, in Facetime), the Status Bar in App is always in the horizontal direction. Therefore, the relative direction of video and gravity is always the same at the acquisition end and the playback end.
-
When the acquisition end is horizontal:
-
When the acquisition terminal is vertical:
ORIENTATION _MODE _FIXED _LANDSCAPE
In the 'ORIENTATION_MODE_FIXED_LANDSCAPE' mode, the output video is always in the horizontal screen mode relative to the Status Bar. If the captured video is in vertical screen mode, the video encoder will crop it. This mode is applicable to the case where the receiver cannot process the rotation information.
When the rear camera is used as the video collector, the video direction of the video collector and player is shown in the following figure.
-
Video captured in landscape mode (video clipping is not required):
-
Video captured in portrait mode (video clipping is required):
ORIENTATION _MODE _FIXED _PORTRAIT
In the 'ORIENTATION_MODE_FIXED_PORTRAIT' mode, the output video is always in vertical screen mode relative to the Status Bar. If the captured video is in horizontal screen mode, the video encoder will cut it. This mode is applicable to the case where the receiver cannot process the rotation information.
When the rear camera is used as the video collector, the video direction of the video collector and player is shown in the following figure.
-
The captured video is in vertical screen mode (video clipping is not required):
-
The captured video is in horizontal screen mode (video clipping is required):
β
Demote preferenceTo ensure the video experience of users in weak networks, the Opa SDK also provides the 'degradationPreference' parameter to set the video coding degradation preference when bandwidth is limited.
β
Mirror modeBy default, the SDK does not mirror videos during encoding. You can use the 'mirrorMode' parameter to set the image mode of video coding to determine the video picture seen by remote users.