Advanced_function - aopacloud/aopa-rtc GitHub Wiki

Take Android as an example During a call, the RTC SDK will trigger callbacks related to call and live broadcast quality. From these callbacks, you can understand the user's interactive experience, troubleshoot problems and optimize the user experience.

This article introduces the details of these callbacks and how to use them.

Technical Principle​

After the user joins the channel, the SDK will trigger a series of callbacks every 2 seconds to report the following information: uplink and downlink network quality, real-time interaction statistics, and statistics of local and remote audio and video streams.

When the user's audio or video status changes, the SDK will trigger a callback to report the latest audio or video status and the reason for the status change.

Uplink and downlink network quality report​

onNetworkQuality callback reports the last mile network quality of each user (' COMMUNICATION 'scenario) or anchor (' LIVE_BROADCASTING 'scenario) in the current call. For details, see [Quality Scoring] (about: blank # rate). Including:

  • Last mile refers to the network from your device to the Opa edge server.
  • The uplink last mile network quality scoring is based on the actual transmission code rate, uplink network packet loss rate, average round trip delay and uplink network jitter calculation.
  • Downlink last mile network quality scoring is based on downlink network packet loss rate, average round trip delay and downlink network jitter calculation.

information

  • The higher the ratio of the actual transmission rate to the target transmission rate, the better the call quality under the network, and the higher the network quality score.
  • Average round trip delay refers to the average value of multiple round trip delay.

Statistical information report​

OnRtcStats The callback is triggered every 2 seconds to report local call statistics. You can learn the call duration, the number of people in the current call channel, the CPU utilization of the current system, the CPU utilization of the current App and other important parameters.

Audio quality report​

Audio quality report includes local audio stream statistics report, local audio stream status monitoring, remote audio stream statistics report and remote audio stream status monitoring.

Local audio stream statistics report​

OnLocalAudioStats The callback is triggered every 2 seconds to report the statistics of the audio stream sent by the local device. You can learn the following important statistics:

  • Current call channel number
  • Sample rate for sending audio
  • *Average value of transmitted audio code rate

Local audio stream status monitoring​

When the status of the local audio changes, such as the audio recording status and audio encoding status, the SDK will trigger the 'onLocalAudioStateChanged' callback to report the current local audio status. When the local audio fails, you can check the problem through the error code.

Remote audio stream statistics report​

The following figure shows the audio transmission process between App clients:

img

`The onRemoteAudioStats' callback is triggered every 2 seconds to report the statistics of each remote user ('COMMUNICATION' scenario) or remote anchor ('LIVE_BROADCASTING' scenario) audio stream in the current call, including the quality of the audio stream sent by the remote end (see [Quality Scoring] (about: blank # rate)), the number of channels and other important parameter information.

`OnRemoteAudioStats' focuses on reporting the full link audio quality of remote audio streams, which is closer to your subjective feelings. Even if packet loss occurs in the network, due to FEC (Forward Error Correction), retransmission recovery, bandwidth estimation and other * * anti packet loss * * and congestion control technologies, the final audio frame loss rate at the receiver may not be high, so the perceived audio quality is good.

onRemoteAudioStats focuses on reporting the full link audio quality of remote audio streams, which is closer to your subjective feelings. Even if packet loss occurs in the network, due to anti packet loss and congestion control technologies such as FEC (Forward Error Correction), retransmission recovery and bandwidth estimation, the final audio frame loss rate at the receiver may not be high, so the perceived audio quality is good.

Remote audio stream status monitoring ​

When the audio stream status of a remote user (the 'COMMUNICATION' scenario) or a remote anchor (the 'LIVE_BROADCASTING' scenario) changes, the SDK will trigger the 'onRemoteAudioStateChanged' callback to report the current status of the remote audio stream and the reason for the change.

Video quality report​

The video quality report includes local video stream statistics report, local video stream status monitoring, remote video stream statistics report and remote video stream status monitoring.

Local video stream statistics report​

OnLocalVideoStats Callback reports the statistics of the video stream sent by the local device, including important parameter information such as video coding width/height, target coding rate, and the bit rate of the sent video.

information

If you previously called the 'enableDualStreamMode' method to enable [Dual Stream Mode]（ https://docs.agora.io/en/Agora%20Platform/dual_stream_mode ）, this callback describes the statistics of the video stream sent by the local device.

Local video stream status monitoring​

When the status of the local video changes, the SDK will trigger the 'onLocalVideoStateChanged' callback to report the current local video status. When the local video fails, you can check the problem through the error code.

Remote video stream statistics report ​

The following figure shows the video transmission process between app clients:

img

`OnRemoteVideoStats' callback reports the video stream statistics of each remote user ('COMMUNICATION' scenario) or remote anchor ('LIVE_BROADCASTING' scenario) in the current call. You can learn the video width/height of each remote user or remote anchor, the bit rate of the received video and other important parameter information.

Remote video stream status monitoring​

When the video stream status of a remote user (the 'COMMUNICATION' scenario) or a remote anchor (the 'LIVE_BROADCASTING' scenario) changes, the SDK will trigger the 'onRemoteVideoStateChanged' callback to report the current status of the remote video stream and the reason for the change. Precondition​

Please ensure that your project has realized basic real-time audio and video functions. See [Realize audio and video interaction] for details（ https://doc.shengwang.cn/doc/rtc/android/get-started/quick-start ）. Implementation method ​

In 'IRtcEngineEventHandler', the following real-time interaction quality statistics callback and audio or video status monitoring callback are implemented to understand user interaction experience:

  • onNetworkQuality：reports the last mile network quality of the uplink and downlink.
  • onRtcStats： reports real-time interaction statistics.
  • onLocalAudioStats：reports the statistics of sent audio streams.
  • onLocalAudioStateChanged: reports the status monitoring of local audio streams.
  • onRemoteAudioStats：Reports the statistics of received remote audio streams.
  • onRemoteAudioStateChanged：Reports remote audio stream status monitoring.
  • onLocalVideoStats：reports statistics of sent video streams.
  • onLocalVideoStateChanged：reports the status monitoring of local video streams.
  • onRemoteVideoStats：Reports the statistics of received remote video streams.
  • onRemoteVideoStateChanged：Reports remote video stream status monitoring.

Java

private final IRtcEngineEventHandler iRtcEngineEventHandler = new IRtcEngineEventHandler() {  
  
  public void onRemoteAudioStateChanged(int uid, IRtcEngineEventHandler.REMOTE_AUDIO_STATE state, IRtcEngineEventHandler.REMOTE_AUDIO_STATE_REASON reason, int elapsed) {    super.onRemoteAudioStateChanged(uid, state, reason, elapsed);    Log.i(TAG, "onRemoteAudioStateChanged->" + uid + ", state->" + state + ", reason->" + reason);  }  
  @Override  public void onRemoteVideoStateChanged(int uid, int state, int reason, int elapsed) {    super.onRemoteVideoStateChanged(uid, state, reason, elapsed);    Log.i(TAG, "onRemoteVideoStateChanged->" + uid + ", state->" + state + ", reason->" + reason);  }  
  @Override  public void onRemoteAudioStats(RemoteAudioStats remoteAudioStats) {    statisticsInfo.setRemoteAudioStats(remoteAudioStats);    updateRemoteStats();  }  
  @Override  public void onLocalAudioStats(LocalAudioStats localAudioStats) {    statisticsInfo.setLocalAudioStats(localAudioStats);    updateLocalStats();  }  
  @Override  public void onRemoteVideoStats(RemoteVideoStats remoteVideoStats) {    statisticsInfo.setRemoteVideoStats(remoteVideoStats);    updateRemoteStats();  }  
  @Override  public void onLocalVideoStats(LocalVideoStats localVideoStats) {    statisticsInfo.setLocalVideoStats(localVideoStats);    updateLocalStats();  }  
  @Override  public void onRtcStats(RtcStats rtcStats) {    statisticsInfo.setRtcStats(rtcStats);  }};

Reference information​

This section provides the reference information that may be required to implement the quality monitoring function during a call.

API Reference​

Quality scoring​

Code Description
QUALITY_UNKNOWN (0) The network quality is unknown。
QUALITY_EXCELLENT(1) The network quality is excellent
QUALITY_GOOD(2) The user's subjective feeling is similar to 'QUALITY_EXCELLENT' (1), but the bit rate may be slightly lower than 'QUALITY_EXCELLENT' (1)
QUALITY_POOR (3) The user's subjective perception is flawed but does not affect communication
QUALITY_BAD (4) Communication is barely possible but not smooth
QUALITY_VBAD (5) The network quality is very poor, and communication is basically impossible
QUALITY_DOWN (6) Communication is completely unavailable

Next chapter:android api