WEBRTC 視訊接收到解碼
一. 視訊傳輸到解碼
VideoCodingModuleImpl類其實並沒有使用
函式呼叫次順:
VideoReceiveStream::DeliverRtp->RtpStreamReceiver::DeliverRtp->RtpReceiverImpl::IncomingRtpPacket->RTPReceiverVideo::ParseRtpPacket->
RtpStreamReceiver::OnReceivedPayloadData->VideoReceiver::IncomingPacket[進入視訊編解碼模組]
Module/video_coding/video_coding_impl.h
VideoReceiver
1. int32_t RegisterReceiveCallback(VCMReceiveCallback*receiveCallback);
註冊解碼後的視訊幀回撥物件，通過其虛擬函式FrameToRender回撥，在VideoStreamDecoder建構函式，註冊回撥物件 video_receiver_->RegisterReceiveCallback(this);
2. Video/video_stream_decoder.h
使用者物件
VideoStreamDecoder: publicVCMReceiveCallback
1. int32_t
FrameToRender
視訊解碼執行緒
VideoReceiveStream::Start->
VideoReceiver::Decode(const VCMEncodedFrame& frame)->
VideoReceiveStream::Encoded[解複用完整幀通過VideoReceiveStream建構函式中VideoReceiver的構造傳入VideoReceiveStream物件作為回撥]->VCMGenericDecoder::Decode(constVCMEncodedFrame&
frame, int64_t nowMs)->
[VCMCodecDataBase::GetDecoder->VCMGenericDecoder::RegisterDecodeCompleteCallback->
VP9DecoderImpl::RegisterDecodeCompleteCallback註冊解碼後資料回撥對]->
VP9DecoderImpl::Decode(const EncodedImage& input_image,
bool missing_frames,
constRTPFragmentationHeader* fragmentation,
constCodecSpecificInfo* codec_specific_info,
int64_t/*render_time_ms*/) ->DecodedImageCallback::Decoded(父)->VCMDecodedFrameCallback::Decoded (VideoFrame&decodedImage) (子)-> VCMDecodedFrameCallback::Decoded(VideoFrame&decodedImage,int64_t decode_time_ms)->
VideoStreamDecoder::FrameToRender(VideoFrame&video_frame)->
[接收解碼物件通過VideoStreamDecoder::VideoStreamDecoder ->VideoReceiver::RegisterReceiveCallback(this)–>VCMDecodedFrameCallback::SetUserReceiveCallback]
IncomingVideoStream::OnFrame[將幀放入佇列VideoRenderFrames]->
視訊顯示|錄影執行緒
VideoReceiveStream::Start->IncomingVideoStream::Start->
IncomingVideoStream::IncomingVideoStreamThreadFun[從VideoRenderFrames佇列獲取幀]->IncomingVideoStream::DeliverFrame->VideoReceiveStream::OnFrame[該回調函式由VideoReceiveStream::VideoReceiveStream中註冊incoming_video_stream_.SetExternalCallback(this)]-> WebRtcVideoReceiveStream::OnFrame(constwebrtc::VideoFrame& frame) [ WebRtcVideoReceiveStream::WebRtcVideoReceiveStream建構函式賦值config_.renderer = this]
[設定廣播接收者PeerConnection::SetRemoteDescription -> PeerConnection::OnRemoteTrackSeen-> PeerConnection::CreateVideoReceiver[new VideoRtpReceiver]->VideoRtpReceiver::VideoRtpReceiver[provider_->SetVideoPlayout(ssrc_,true,&broadcaster_)]->WebRtcSession::SetVideoPlayout-> VideoChannel::SetSink->
WebRtcVideoChannel2::SetSink-> WebRtcVideoReceiveStream::SetSink(sink_)]
總結:
- 從網路接收執行緒輸入到顯示執行緒輸出(中間解碼執行緒)
VideoReceiveStream::DeliverRtp[已複用的視訊RTP]->
VideoReceiveStream::OnFrame[已解碼的視訊幀]
- 從網路接收RTP到解複用完整幀
VideoReceiveStream::DeliverRtp[已複用的視訊RTP]->
int32_t VideoReceiveStream::Encoded(
constEncodedImage& encoded_image,
constCodecSpecificInfo* codec_specific_info,
const RTPFragmentationHeader*fragmentation)
RtpStreamReceiver作為Video_Coding模組和RTP/RTCP 模組的中介者
#include"webrtc/video/rtp_stream_receiver.h"
RtpStreamReceiver
1. 將RTP/RTCP模組解複用的RTP包送入Video_Coding中的VideoReceiver::IncomingPacket
2. 回撥RTP/RTCP中的RequestKeyFrame、SliceLossIndicationRequest、ResendPackets[關鍵字請求、重傳幀請求]
RtpReceiver
#include"webrtc/modules/rtp_rtcp/source/rtp_receiver_impl.h
一個PeerConnection擁有一個MediaControllerInterface、一個WebRtcSession、
一個WebRtcSession擁有一個VoiceChannel、一個VideoChannel、一個DataChannel
整個WEBRTC擁有一個ChannelManager 和AudioDeviceModule和MediaEngineInterface(包括一個WebRtcVideoEngine2和一個WebRtcVoiceEngine)