ReplayKit2 採集音視訊回撥格式分析
阿新 • • 發佈:2018-11-05
一、iOS中的音視訊都是採用一個叫做CMSampleBuffer的格式封裝的
比如回撥的App音訊
Printing description of sampleBuffer: CMSampleBuffer 0x143d06560 retainCount: 1 allocator: 0x25bae95e0 invalid = NO dataReady = YES makeDataReadyCallback = 0x0 makeDataReadyRefcon = 0x0 formatDescription = <CMAudioFormatDescription 0x281940990 [0x25bae95e0]> { mediaType:'soun' mediaSubType:'lpcm' mediaSpecific: { ASBD: { mSampleRate: 44100.000000 mFormatID: 'lpcm' mFormatFlags: 0xe mBytesPerPacket: 2 mFramesPerPacket: 1 mBytesPerFrame: 2 mChannelsPerFrame: 1 mBitsPerChannel: 16 } cookie: {(null)} ACL: {(null)} FormatList Array: {(null)} } extensions: {(null)} } sbufToTrackReadiness = 0x0 numSamples = 22528 sampleTimingArray[1] = { {PTS = {619877934457375/1000000000 = 619877.934}, DTS = {INVALID}, duration = {1/44100 = 0.000}}, } dataBuffer = 0x281940360
比如回撥的micphone音訊
Printing description of sampleBuffer: CMSampleBuffer 0x149e053b0 retainCount: 1 allocator: 0x25bae95e0 invalid = NO dataReady = YES makeDataReadyCallback = 0x0 makeDataReadyRefcon = 0x0 formatDescription = <CMAudioFormatDescription 0x281b47b10 [0x25bae95e0]> { mediaType:'soun' mediaSubType:'lpcm' mediaSpecific: { ASBD: { mSampleRate: 44100.000000 mFormatID: 'lpcm' mFormatFlags: 0xc mBytesPerPacket: 2 mFramesPerPacket: 1 mBytesPerFrame: 2 mChannelsPerFrame: 1 mBitsPerChannel: 16 } cookie: {(null)} ACL: {(null)} FormatList Array: {(null)} }extensions: {(null)} } sbufToTrackReadiness = 0x0 numSamples = 1024 sampleTimingArray[1] = { {PTS = {27273930501/44100 = 618456.474, rounded}, DTS = {INVALID}, duration = {1/44100 = 0.000}}, } dataBuffer = 0x281b47c30
從這個結構中,可以得到這個音訊資料的描述資訊
// Get samples. CMBlockBufferRef audioBuffer = CMSampleBufferGetDataBuffer(sampleBuffer); size_t lengthAtOffset; size_t totalLength; char *samples; CMBlockBufferGetDataPointer(audioBuffer, 0, &lengthAtOffset, &totalLength, &samples); // Get format. CMAudioFormatDescriptionRef format = CMSampleBufferGetFormatDescription(sampleBuffer); const AudioStreamBasicDescription *description = CMAudioFormatDescriptionGetStreamBasicDescription(format);
輸出為格式的描述資訊
Printing description of format: <CMAudioFormatDescription 0x28282f7b0 [0x25bae95e0]> { mediaType:'soun' mediaSubType:'lpcm' mediaSpecific: { ASBD: { mSampleRate: 44100.000000 mFormatID: 'lpcm' mFormatFlags: 0xc mBytesPerPacket: 2 mFramesPerPacket: 1 mBytesPerFrame: 2 mChannelsPerFrame: 1 mBitsPerChannel: 16 } cookie: {(null)} ACL: {(null)} FormatList Array: {(null)} } extensions: {(null)} }
關於音訊格式有幾個關鍵資訊:格式、取樣率、取樣精度、聲道數
關於音訊資料除了上面的格式資訊之外,還有 PTS 、 samples(幀數)描述資訊
比如上面的PCM資料中,取樣率是44100hz、取樣深度是16位、聲道數是1,samples是1024幀,最後計算這段聲音的長度資訊是 (1/44100) * 1024 = 23ms , 這個聲音的資料大小是 1024 * 2byte = 2014 Byte
如果是錄製聲音的話,需要先設定需要錄製的音訊格式資訊
AudioStreamBasicDescription format; memset(&format, 0, sizeof(format)); format.mSampleRate = 44100; format.mFormatID = kAudioFormatLinearPCM; format.mFormatFlags = kLinearPCMFormatFlagIsSignedInteger | kLinearPCMFormatFlagIsPacked; format.mChannelsPerFrame = 1; format.mBitsPerChannel = 16; format.mBytesPerFrame = (format.mBitsPerChannel/8) * format.mChannelsPerFrame; format.mFramesPerPacket = 1; format.mBytesPerPacket = format.mBytesPerFrame * format.mFramesPerPacket;
需要設定關鍵引數、取樣率、取樣精度、聲道數、資料排列格式、 一個packet的幀數
附錄:
關於取樣精度
關於取樣率