FFMPEG Tips (3) 如何讀取每一幀的資訊
1. 碼流中每一幀的哪些資訊值得關注 ?
[ ] 音訊幀還是視訊幀
[ ] 關鍵幀還是非關鍵幀
[ ] 幀的資料和大小
[ ] 時間戳資訊
2. 為什麼要關注這些資訊 ?
[ ] 音訊幀還是視訊幀 -> 分別送入音訊/視訊×××
[ ] 關鍵幀還是非關鍵幀 -> 追幀優化
[ ] 幀的資料和大小 -> 取出幀的內容
[ ] 時間戳資訊 -> 音視訊同步
3. 如何從 ffmpeg 取出這些資訊 ?
ffmpeg 提供了一個函式 av_read_frame 來完成解封裝的過程,它會從碼流裡面提取每一個音訊、視訊幀,它使用了結構體 AVPacket 來記錄每一幀的資訊。
讀取一幀資料的程式碼示例如下(ic 即為 AVFormatContext 物件,碼流的上下文控制代碼):
AVPacket avpkt;
av_init_packet(&avpkt);
while (!interrupt) {
int ret = av_read_frame(ic, &avpkt);
if (ret < 0) {
break;
}
// processing
}
av_free_packet(&avpkt);
每迴圈一次,就從碼流中解封裝並且提取了一幀資料,並存放在了 AVPacket 結構體中。
3.1 如何判斷是音訊幀還是視訊幀
上一篇文章我們提到過,使用下面的方法,獲取碼流中的 video_stream_idx 和 audio_stream_idx
int video_stream_idx = av_find_best_stream(ic, AVMEDIA_TYPE_VIDEO, -1, -1, NULL, 0);
int audio_stream_idx = av_find_best_stream(ic, AVMEDIA_TYPE_AUDIO, -1, -1, NULL, 0);
那麼,此時就派上用場了,每一個 AVPacket 都有一個成員變數:stream_index,由該成員變數即可判斷這個 Packet 到底是音訊還是視訊了:
if (avpkt.stream_index == video_stream_idx) {
LOGD("read a video frame");
} else if (avpkt.stream_index == audio_stream_idx) {
LOGD("read audio frame);
}
3.2 如何判斷是否為關鍵幀
判斷是否為關鍵幀的方法也比較簡單,示例如下:
if (avpkt.flags & AV_PKT_FLAG_KEY) {
LOGD("read a key frame");
}
3.3 如何獲取幀的資料和大小
幀的資料和大小直接定義在 AVPacket 結構體中,對應的成員變數如下:
// 壓縮編碼的資料,一幀音訊/視訊
uint8_t *data;
// 資料的大小
int size;
www.imdb.com/list/ls083027164/
www.imdb.com/list/ls083027164/?ref_=otl_Z
m.imdb.com/list/ls083027164/
www.imdb.com/list/ls083027164/?ref_=otl_5x
www.imdb.com/list/ls083027164/?ref_=otl_7
www.imdb.com/list/ls083027164/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027164/?fjxzl=5x
www.imdb.com/list/ls083027164/?mode=desktop
www.imdb.com/list/ls083027128/
www.imdb.com/list/ls083027128/?ref_=otl_K
m.imdb.com/list/ls083027128/
www.imdb.com/list/ls083027128/?ref_=otl_e6
www.imdb.com/list/ls083027128/?ref_=otl_0
www.imdb.com/list/ls083027128/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027128/?kywkw=e6
www.imdb.com/list/ls083027128/?mode=desktop
www.imdb.com/list/ls083027143/
www.imdb.com/list/ls083027143/?ref_=otl_z
m.imdb.com/list/ls083027143/
www.imdb.com/list/ls083027143/?ref_=otl_FJ
www.imdb.com/list/ls083027143/?ref_=otl_6
www.imdb.com/list/ls083027143/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027143/?kueef=FJ
www.imdb.com/list/ls083027143/?mode=desktop
www.imdb.com/list/ls083027199/
www.imdb.com/list/ls083027199/?ref_=otl_7
m.imdb.com/list/ls083027199/
www.imdb.com/list/ls083027199/?ref_=otl_X3
www.imdb.com/list/ls083027199/?ref_=otl_5
www.imdb.com/list/ls083027199/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027199/?btnrd=X3
www.imdb.com/list/ls083027199/?mode=desktop
www.imdb.com/list/ls083027184/
www.imdb.com/list/ls083027184/?ref_=otl_F
m.imdb.com/list/ls083027184/
www.imdb.com/list/ls083027184/?ref_=otl_R1
www.imdb.com/list/ls083027184/?ref_=otl_7
www.imdb.com/list/ls083027184/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027184/?znnnr=R1
www.imdb.com/list/ls083027184/?mode=desktop
www.imdb.com/list/ls083027304/
www.imdb.com/list/ls083027304/?ref_=otl_u
m.imdb.com/list/ls083027304/
www.imdb.com/list/ls083027304/?ref_=otl_Sg
www.imdb.com/list/ls083027304/?ref_=otl_8
www.imdb.com/list/ls083027304/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027304/?kyyco=Sg
www.imdb.com/list/ls083027304/?mode=desktop
www.imdb.com/list/ls083027357/
www.imdb.com/list/ls083027357/?ref_=otl_P
m.imdb.com/list/ls083027357/
www.imdb.com/list/ls083027357/?ref_=otl_zt
www.imdb.com/list/ls083027357/?ref_=otl_5
www.imdb.com/list/ls083027357/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027357/?lpxld=zt
www.imdb.com/list/ls083027357/?mode=desktop
www.imdb.com/list/ls083027371/
www.imdb.com/list/ls083027371/?ref_=otl_g
m.imdb.com/list/ls083027371/
www.imdb.com/list/ls083027371/?ref_=otl_cH
www.imdb.com/list/ls083027371/?ref_=otl_7
www.imdb.com/list/ls083027371/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027371/?mcrgg=cH
www.imdb.com/list/ls083027371/?mode=desktop
www.imdb.com/list/ls083027374/
www.imdb.com/list/ls083027374/?ref_=otl_l
m.imdb.com/list/ls083027374/
www.imdb.com/list/ls083027374/?ref_=otl_53
www.imdb.com/list/ls083027374/?ref_=otl_5
www.imdb.com/list/ls083027374/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027374/?lpnnp=53
www.imdb.com/list/ls083027374/?mode=desktop
www.imdb.com/list/ls083027313/
www.imdb.com/list/ls083027313/?ref_=otl_7
m.imdb.com/list/ls083027313/
www.imdb.com/list/ls083027313/?ref_=otl_x5
www.imdb.com/list/ls083027313/?ref_=otl_3
www.imdb.com/list/ls083027313/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027313/?vxxzz=x5
www.imdb.com/list/ls083027313/?mode=desktop
www.imdb.com/list/ls083027319/
www.imdb.com/list/ls083027319/?ref_=otl_C
m.imdb.com/list/ls083027319/
www.imdb.com/list/ls083027319/?ref_=otl_eC
www.imdb.com/list/ls083027319/?ref_=otl_4
www.imdb.com/list/ls083027319/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027319/?cecee=eC
www.imdb.com/list/ls083027319/?mode=desktop
www.imdb.com/list/ls083027339/
www.imdb.com/list/ls083027339/?ref_=otl_B
m.imdb.com/list/ls083027339/
www.imdb.com/list/ls083027339/?ref_=otl_fB
www.imdb.com/list/ls083027339/?ref_=otl_1
www.imdb.com/list/ls083027339/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027339/?ddbbd=fB
www.imdb.com/list/ls083027339/?mode=desktop
www.imdb.com/list/ls083027361/
www.imdb.com/list/ls083027361/?ref_=otl_O
m.imdb.com/list/ls083027361/
www.imdb.com/list/ls083027361/?ref_=otl_62
www.imdb.com/list/ls083027361/?ref_=otl_0
www.imdb.com/list/ls083027361/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027361/?oycss=62
www.imdb.com/list/ls083027361/?mode=desktop
www.imdb.com/list/ls083027326/
www.imdb.com/list/ls083027326/?ref_=otl_7
m.imdb.com/list/ls083027326/
www.imdb.com/list/ls083027326/?ref_=otl_Hn
www.imdb.com/list/ls083027326/?ref_=otl_5
www.imdb.com/list/ls083027326/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027326/?jrfbp=Hn
www.imdb.com/list/ls083027326/?mode=desktop
www.imdb.com/list/ls083027329/
www.imdb.com/list/ls083027329/?ref_=otl_x
m.imdb.com/list/ls083027329/
www.imdb.com/list/ls083027329/?ref_=otl_7T
www.imdb.com/list/ls083027329/?ref_=otl_3
www.imdb.com/list/ls083027329/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027329/?rphhh=7T
www.imdb.com/list/ls083027329/?mode=desktop
www.imdb.com/list/ls083027391/
www.imdb.com/list/ls083027391/?ref_=otl_2
m.imdb.com/list/ls083027391/
www.imdb.com/list/ls083027391/?ref_=otl_2W
www.imdb.com/list/ls083027391/?ref_=otl_6
www.imdb.com/list/ls083027391/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027391/?agcie=2W
www.imdb.com/list/ls083027391/?mode=desktop
www.imdb.com/list/ls083027399/
www.imdb.com/list/ls083027399/?ref_=otl_P
m.imdb.com/list/ls083027399/
www.imdb.com/list/ls083027399/?ref_=otl_95
www.imdb.com/list/ls083027399/?ref_=otl_9
www.imdb.com/list/ls083027399/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027399/?tpddz=95
www.imdb.com/list/ls083027399/?mode=desktop
www.imdb.com/list/ls083027384/
www.imdb.com/list/ls083027384/?ref_=otl_W
m.imdb.com/list/ls083027384/
www.imdb.com/list/ls083027384/?ref_=otl_W4
www.imdb.com/list/ls083027384/?ref_=otl_8
www.imdb.com/list/ls083027384/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027384/?skssy=W4
www.imdb.com/list/ls083027384/?mode=desktop
www.imdb.com/list/ls083027607/
www.imdb.com/list/ls083027607/?ref_=otl_p
m.imdb.com/list/ls083027607/
www.imdb.com/list/ls083027607/?ref_=otl_15
www.imdb.com/list/ls083027607/?ref_=otl_9
www.imdb.com/list/ls083027607/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027607/?jjxfj=15
www.imdb.com/list/ls083027607/?mode=desktop
www.imdb.com/list/ls083027651/
www.imdb.com/list/ls083027651/?ref_=otl_1
m.imdb.com/list/ls083027651/
www.imdb.com/list/ls083027651/?ref_=otl_82
www.imdb.com/list/ls083027651/?ref_=otl_8
www.imdb.com/list/ls083027651/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027651/?wvvwl=82
www.imdb.com/list/ls083027651/?mode=desktop
www.imdb.com/list/ls083027659/
www.imdb.com/list/ls083027659/?ref_=otl_8
m.imdb.com/list/ls083027659/
www.imdb.com/list/ls083027659/?ref_=otl_ai
www.imdb.com/list/ls083027659/?ref_=otl_2
www.imdb.com/list/ls083027659/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027659/?iayai=ai
www.imdb.com/list/ls083027659/?mode=desktop
www.imdb.com/list/ls083027679/
www.imdb.com/list/ls083027679/?ref_=otl_1
m.imdb.com/list/ls083027679/
www.imdb.com/list/ls083027679/?ref_=otl_37
www.imdb.com/list/ls083027679/?ref_=otl_7
www.imdb.com/list/ls083027679/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027679/?hlhlj=37
www.imdb.com/list/ls083027679/?mode=desktop
www.imdb.com/list/ls083027615/
www.imdb.com/list/ls083027615/?ref_=otl_5
m.imdb.com/list/ls083027615/
www.imdb.com/list/ls083027615/?ref_=otl_f8
www.imdb.com/list/ls083027615/?ref_=otl_9
www.imdb.com/list/ls083027615/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027615/?ppquu=f8
www.imdb.com/list/ls083027615/?mode=desktop
www.imdb.com/list/ls083027631/
www.imdb.com/list/ls083027631/?ref_=otl_C
m.imdb.com/list/ls083027631/
www.imdb.com/list/ls083027631/?ref_=otl_cg
www.imdb.com/list/ls083027631/?ref_=otl_8
www.imdb.com/list/ls083027631/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027631/?mwymq=cg
www.imdb.com/list/ls083027631/?mode=desktop
www.imdb.com/list/ls083027634/
www.imdb.com/list/ls083027634/?ref_=otl_M
m.imdb.com/list/ls083027634/
www.imdb.com/list/ls083027634/?ref_=otl_88
www.imdb.com/list/ls083027634/?ref_=otl_6
www.imdb.com/list/ls083027634/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027634/?eoekq=88
www.imdb.com/list/ls083027634/?mode=desktop
www.imdb.com/list/ls083027625/
www.imdb.com/list/ls083027625/?ref_=otl_0
m.imdb.com/list/ls083027625/
www.imdb.com/list/ls083027625/?ref_=otl_2w
www.imdb.com/list/ls083027625/?ref_=otl_6
www.imdb.com/list/ls083027625/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027625/?uyekm=2w
www.imdb.com/list/ls083027625/?mode=desktop
www.imdb.com/list/ls083027626/
www.imdb.com/list/ls083027626/?ref_=otl_7
m.imdb.com/list/ls083027626/
www.imdb.com/list/ls083027626/?ref_=otl_77
www.imdb.com/list/ls083027626/?ref_=otl_7
www.imdb.com/list/ls083027626/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027626/?zbzhf=77
www.imdb.com/list/ls083027626/?mode=desktop
www.imdb.com/list/ls083027648/
www.imdb.com/list/ls083027648/?ref_=otl_W
m.imdb.com/list/ls083027648/
www.imdb.com/list/ls083027648/?ref_=otl_uO
www.imdb.com/list/ls083027648/?ref_=otl_6
www.imdb.com/list/ls083027648/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027648/?uwwqk=uO
www.imdb.com/list/ls083027648/?mode=desktop
www.imdb.com/list/ls083027697/
www.imdb.com/list/ls083027697/?ref_=otl_a
m.imdb.com/list/ls083027697/
www.imdb.com/list/ls083027697/?ref_=otl_2y
www.imdb.com/list/ls083027697/?ref_=otl_4
www.imdb.com/list/ls083027697/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027697/?agykw=2y
www.imdb.com/list/ls083027697/?mode=desktop
www.imdb.com/list/ls083027687/
www.imdb.com/list/ls083027687/?ref_=otl_H
m.imdb.com/list/ls083027687/
www.imdb.com/list/ls083027687/?ref_=otl_3j
www.imdb.com/list/ls083027687/?ref_=otl_9
www.imdb.com/list/ls083027687/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027687/?hfzpb=3j
www.imdb.com/list/ls083027687/?mode=desktop
www.imdb.com/list/ls083027686/
www.imdb.com/list/ls083027686/?ref_=otl_O
m.imdb.com/list/ls083027686/
www.imdb.com/list/ls083027686/?ref_=otl_8s
www.imdb.com/list/ls083027686/?ref_=otl_4
www.imdb.com/list/ls083027686/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027686/?oasyu=8s
www.imdb.com/list/ls083027686/?mode=desktop
www.imdb.com/list/ls083027204/
www.imdb.com/list/ls083027204/?ref_=otl_5
m.imdb.com/list/ls083027204/
www.imdb.com/list/ls083027204/?ref_=otl_34
www.imdb.com/list/ls083027204/?ref_=otl_5
www.imdb.com/list/ls083027204/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027204/?gssor=34
www.imdb.com/list/ls083027204/?mode=desktop
www.imdb.com/list/ls083027208/
www.imdb.com/list/ls083027208/?ref_=otl_4
m.imdb.com/list/ls083027208/
www.imdb.com/list/ls083027208/?ref_=otl_WG
www.imdb.com/list/ls083027208/?ref_=otl_8
www.imdb.com/list/ls083027208/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027208/?oaqug=WG
www.imdb.com/list/ls083027208/?mode=desktop
www.imdb.com/list/ls083027275/
www.imdb.com/list/ls083027275/?ref_=otl_2
m.imdb.com/list/ls083027275/
www.imdb.com/list/ls083027275/?ref_=otl_Y6
www.imdb.com/list/ls083027275/?ref_=otl_4
www.imdb.com/list/ls083027275/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027275/?siokk=Y6
www.imdb.com/list/ls083027275/?mode=desktop
www.imdb.com/list/ls083027272/
www.imdb.com/list/ls083027272/?ref_=otl_1
m.imdb.com/list/ls083027272/
www.imdb.com/list/ls083027272/?ref_=otl_FB
www.imdb.com/list/ls083027272/?ref_=otl_7
www.imdb.com/list/ls083027272/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027272/?hxpdd=FB
www.imdb.com/list/ls083027272/?mode=desktop
www.imdb.com/list/ls083027216/
www.imdb.com/list/ls083027216/?ref_=otl_s
m.imdb.com/list/ls083027216/
www.imdb.com/list/ls083027216/?ref_=otl_8c
www.imdb.com/list/ls083027216/?ref_=otl_2
www.imdb.com/list/ls083027216/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027216/?sgwak=8c
www.imdb.com/list/ls083027216/?mode=desktop
www.imdb.com/list/ls083027235/
www.imdb.com/list/ls083027235/?ref_=otl_J
m.imdb.com/list/ls083027235/
www.imdb.com/list/ls083027235/?ref_=otl_XL
www.imdb.com/list/ls083027235/?ref_=otl_1
www.imdb.com/list/ls083027235/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027235/?vlltj=XL
www.imdb.com/list/ls083027235/?mode=desktop
www.imdb.com/list/ls083027238/
www.imdb.com/list/ls083027238/?ref_=otl_L
m.imdb.com/list/ls083027238/
www.imdb.com/list/ls083027238/?ref_=otl_3l
www.imdb.com/list/ls083027238/?ref_=otl_7
www.imdb.com/list/ls083027238/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027238/?xtzhj=3l
www.imdb.com/list/ls083027238/?mode=desktop
www.imdb.com/list/ls083027263/
www.imdb.com/list/ls083027263/?ref_=otl_E
m.imdb.com/list/ls083027263/
www.imdb.com/list/ls083027263/?ref_=otl_Y6
www.imdb.com/list/ls083027263/?ref_=otl_0
www.imdb.com/list/ls083027263/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027263/?mqawo=Y6
www.imdb.com/list/ls083027263/?mode=desktop
www.imdb.com/list/ls083027225/
www.imdb.com/list/ls083027225/?ref_=otl_4
m.imdb.com/list/ls083027225/
www.imdb.com/list/ls083027225/?ref_=otl_qw
www.imdb.com/list/ls083027225/?ref_=otl_2
www.imdb.com/list/ls083027225/?mode=desktop&ref_=m_ft_dsk
www.imdb.com/list/ls083027225/?qsaug=qw
www.imdb.com/list/ls083027225/?mode=desktop
3.4 如何獲取幀的時間戳資訊
每一個幀都可能攜帶有 2 個時間戳資訊,一個是解碼時間戳 dts,一個是顯示時間戳 pts,解碼時間戳告訴我們什麼時候需要解碼,顯示時間戳告訴我們什麼時候需要顯示,只有在碼流中存在 B 幀的情況下,這兩個時間戳才會不一致。
這些時間戳資訊不一定存在於碼流中(取決於生產端),如果不存在,則其值為:AV_NOPTS_VALUE
一定要選擇正確地方式列印時間戳,時間戳是使用 long long 來表示的,即 int64_t,因此列印的時候,需要使用 “%lld” 來列印,例如:
while (!interrupt) {
int ret = av_read_frame(player->ic, &avpkt);
if (ret < 0) {
break;
}
if (avpkt.stream_index == video_stream_idx) {
LOGD("read video frame, timestamp = %lld \n”, avpkt.pts);
} else if (avpkt.stream_index == audio_stream_idx) {
LOGD("read audio frame, timestamp = %lld \n”, avpkt.pts);
}
}
由此,我們就可以通過這些 log 資訊除錯一下某一段音視訊流的時間戳是否正確,比如是否出現了時間戳的回滾和錯亂,則必然會導致播放端出現音視訊不同步或者顯示異常等情況。