未完待續

最初的夢想

哈哈哈哈哈哈，我就是想了解下Android上錄音是怎麼實現的，寫了個簡單的錄音demo，一路跟下去，瞅瞅這傢伙都幹了些啥。基於Android 7.1, s905x 平臺。

冰山上面的部分

按照官方我Android大文件寫了下面的錄音程式碼，只打log不幹事也是厲害。

// 這個方法執行在子執行緒，要不那個死迴圈不得把應用搞ANR了。
private fun record() {
   val minSize = AudioRecord.getMinBufferSize(48000,
           AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT)
   var record: AudioRecord? = null
 

   try {
       record = AudioRecord.Builder()
               .setAudioSource(MediaRecorder.AudioSource.DEFAULT)
               .setAudioFormat(AudioFormat.Builder()
                       .setEncoding(AudioFormat.ENCODING_PCM_16BIT)
                       .setSampleRate(48000)
                       .setChannelMask(AudioFormat.CHANNEL_IN_MONO)
                       .build())
               .setBufferSizeInBytes(minSize * 2
 
)
               .build()
       record.startRecording()
       val buffer = ShortArray(minSize)
       while (true) {
           Log.d(TAG, "read ....")
           val read = record.read(buffer, 0, minSize)
           Log.d(TAG, "read $read bytes data...")
       }
   } catch (e: Throwable) {
       e.printStackTrace()
   } finally
 
 {
       record?.stop()
   }
}

AudioRecord程式碼追蹤

• 第一步的builder模式最重要的是建立了java層的AudioRecord物件

// frameworks/base/media/java/android/media/AudioRecord.java

public AudioRecord build() throws UnsupportedOperationException {
    // 前面都是一堆的引數構造和校驗，略過
    try {
        // If the buffer size is not specified,
        // use a single frame for the buffer size and let the
        // native code figure out the minimum buffer size.
        if (mBufferSizeInBytes == 0) {
            mBufferSizeInBytes = mFormat.getChannelCount()
                    * mFormat.getBytesPerSample(mFormat.getEncoding());
        }
        final AudioRecord record = new AudioRecord(
                mAttributes, mFormat, mBufferSizeInBytes, mSessionId);
        if (record.getState() == STATE_UNINITIALIZED) {
            // release is not necessary
            throw new UnsupportedOperationException("Cannot create AudioRecord");
        }
        return record;
    } catch (IllegalArgumentException e) {
        throw new UnsupportedOperationException(e.getMessage());
    }
}

• AudioRecord構造時最重要的是呼叫native的native_setup來初始化c++層的物件，進行真正的錄音初始化。

// frameworks/base/media/java/android/media/AudioRecord.java

public AudioRecord(AudioAttributes attributes, AudioFormat format, int bufferSizeInBytes,
            int sessionId) throws IllegalArgumentException {
......
int initResult = native_setup( new WeakReference<AudioRecord>(this),
        mAudioAttributes, sampleRate, mChannelMask, mChannelIndexMask,
        mAudioFormat, mNativeBufferSizeInBytes,
        session, ActivityThread.currentOpPackageName(), 0 /*nativeRecordInJavaObj*/);
......

• 這下進入jni層了，對於jni層程式碼的位置，有個6的不行的辦法，google直接搜尋java層類名稱加jni關鍵字，找到網址是android.googlesource.com的就是了。jni層的android_media_AudioRecord_setup中會建立c++層的AudioRecord物件

// frameworks/base/core/jni/android_media_AudioRecord.cpp

// 底下有jni方法和native方法的對應表
{"native_setup",         "(Ljava/lang/Object;Ljava/lang/Object;[IIIII[ILjava/lang/String;J)I",
                                  (void *)android_media_AudioRecord_setup},

// 本體出現
static jint
android_media_AudioRecord_setup(JNIEnv *env, jobject thiz, jobject weak_this,
        jobject jaa, jintArray jSampleRate, jint channelMask, jint channelIndexMask,
        jint audioFormat, jint buffSizeInBytes, jintArray jSession, jstring opPackageName,
        jlong nativeRecordInJavaObj)
{
// 前面都是亂七八糟的引數校驗
...
// create an uninitialized AudioRecord object
lpRecorder = new AudioRecord(String16(opPackageNameStr.c_str()));
...
// AudioRecord構造方法也會呼叫set
const status_t status = lpRecorder->set(paa->source,
    sampleRateInHertz,
    format,        // word length, PCM
    localChanMask,
    frameCount,
    recorderCallback,// callback_t
    lpCallbackData,// void* user
    0,             // notificationFrames,
    true,          // threadCanCallJava
    sessionId,
    AudioRecord::TRANSFER_DEFAULT,
    flags,
    -1, -1,        // default uid, pid
    paa);

• AudioRecord.cpp構造方法就只幹了一件事set，至於那是幹啥的，進去瞅瞅。（c++的語法不熟啊，恩恩，有朝一日看的煩死它，我大概也就學會了）

// frameworks/av/media/libmedia/AudioRecord.cpp

mStatus = set(inputSource, sampleRate, format, channelMask, frameCount, cbf, user,
        notificationFrames, false /*threadCanCallJava*/, sessionId, transferType, flags,
        uid, pid, pAttributes);

• set裡面可就神奇了，主要就建立了IAudioRecord例項，這傢伙一看就是個aidl的角色。

// frameworks/av/media/libmedia/AudioRecord.cpp

......
// 對於我們的音訊獲取一個session id，這個AudioSystem是什麼鬼，瞅瞅去。
if (sessionId == AUDIO_SESSION_ALLOCATE) {
    mSessionId = (audio_session_t) AudioSystem::newAudioUniqueId(AUDIO_UNIQUE_ID_USE_SESSION);
} else {
    mSessionId = sessionId;
}
ALOGV("set(): mSessionId %d", mSessionId);
......
// 建立了一個執行緒，搞毛的，一會兒瞅瞅
if (cbf != NULL) {
    mAudioRecordThread = new AudioRecordThread(*this, threadCanCallJava);
    mAudioRecordThread->run("AudioRecord", ANDROID_PRIORITY_AUDIO);
    // thread begins in paused state, and will not reference us until start()
}

// 這可就是重點了，敲黑板，方法名字加個l，那明顯是lock的意思。
// create the IAudioRecord
status_t status = openRecord_l(0 /*epoch*/, mOpPackageName);

if (status != NO_ERROR) {
    if (mAudioRecordThread != 0) {
        mAudioRecordThread->requestExit();   // see comment in AudioRecord.h
        mAudioRecordThread->requestExitAndWait();
        mAudioRecordThread.clear();
    }
    return status;
}

• AudioSystem裡面的神奇東西, 鬧了半天，AudioSystem是個傀儡，實際就調到AudioFlinger或者AudioPolicyService裡面去了，這個和java中的用法一樣的。system/media/audio/include/system/audio.h包含了Android 中對於聲音的一些常量定義，影響到Android上層，aps, audio-hal等多個地方。類似java中對於抽象介面的宣告和常量的宣告。

// 這就是上面的那個newAudioUniqueId，調到AudioFlinger裡去了。
audio_unique_id_t AudioSystem::newAudioUniqueId(audio_unique_id_use_t use)
{
    const sp<IAudioFlinger>& af = AudioSystem::get_audio_flinger();
    if (af == 0) return AUDIO_UNIQUE_ID_ALLOCATE;
    return af->newAudioUniqueId(use);
}
// 熟悉的配方，從ServiceManager中通過名稱獲取server的binder物件。
// establish binder interface to AudioFlinger service
const sp<IAudioFlinger> AudioSystem::get_audio_flinger()
{
    sp<IAudioFlinger> af;
    sp<AudioFlingerClient> afc;
    {
        Mutex::Autolock _l(gLock);
        if (gAudioFlinger == 0) {
            sp<IServiceManager> sm = defaultServiceManager();
            sp<IBinder> binder;
            do {
                binder = sm->getService(String16("media.audio_flinger"));
                if (binder != 0)
                    break;
                ALOGW("AudioFlinger not published, waiting...");
                usleep(500000); // 0.5 s
            } while (true);
            if (gAudioFlingerClient == NULL) {
                gAudioFlingerClient = new AudioFlingerClient();
            } else {
                if (gAudioErrorCallback) {
                    gAudioErrorCallback(NO_ERROR);
                }
            }
            binder->linkToDeath(gAudioFlingerClient);
            gAudioFlinger = interface_cast<IAudioFlinger>(binder);
            LOG_ALWAYS_FATAL_IF(gAudioFlinger == 0);
            afc = gAudioFlingerClient;
        }
        af = gAudioFlinger;
    }
    if (afc != 0) {
        af->registerClient(afc);
    }
    return af;
}
// AudioPolicyService 和上面那個一樣
// establish binder interface to AudioPolicy service
const sp<IAudioPolicyService> AudioSystem::get_audio_policy_service()

• 可以瞅瞅audio.h中關於audio_unique_id_use_t和audio_unique_id_t的定義，感覺就像是java中的介面。

// system/media/audio/include/system/audio.h

/* a unique ID allocated by AudioFlinger for use as an audio_io_handle_t, audio_session_t,
 * effect ID (int), audio_module_handle_t, and audio_patch_handle_t.
 * Audio port IDs (audio_port_handle_t) are allocated by AudioPolicy
 * in a different namespace than AudioFlinger unique IDs.
 */
typedef int audio_unique_id_t;

/* Possible uses for an audio_unique_id_t */
typedef enum {
    AUDIO_UNIQUE_ID_USE_UNSPECIFIED = 0,
    AUDIO_UNIQUE_ID_USE_SESSION = 1,    // for allocated sessions, not special AUDIO_SESSION_*
    AUDIO_UNIQUE_ID_USE_MODULE = 2,
    AUDIO_UNIQUE_ID_USE_EFFECT = 3,
    AUDIO_UNIQUE_ID_USE_PATCH = 4,
    AUDIO_UNIQUE_ID_USE_OUTPUT = 5,
    AUDIO_UNIQUE_ID_USE_INPUT = 6,
    // 7 is available
    AUDIO_UNIQUE_ID_USE_MAX = 8,  // must be a power-of-two
    AUDIO_UNIQUE_ID_USE_MASK = AUDIO_UNIQUE_ID_USE_MAX - 1
} audio_unique_id_use_t;

• AudioFlinger中關於newAudioUniqueId方法的實現

// frameworks/av/services/audioflinger/AudioFlinger.cpp
// AudioFlinger的aidl的宣告在
// frameworks/av/include/media/IAudioFlinger.h

// Binder方法的實現，這裡只做校驗，真實實現是在nextUniqueId中
audio_unique_id_t AudioFlinger::newAudioUniqueId(audio_unique_id_use_t use)
{
    // This is a binder API, so a malicious client could pass in a bad parameter.
    // Check for that before calling the internal API nextUniqueId().
    if ((unsigned) use >= (unsigned) AUDIO_UNIQUE_ID_USE_MAX) {
        ALOGE("newAudioUniqueId invalid use %d", use);
        return AUDIO_UNIQUE_ID_ALLOCATE;
    }
    return nextUniqueId(use);
}

// 真正的實現在這裡，哈哈哈哈
audio_unique_id_t AudioFlinger::nextUniqueId(audio_unique_id_use_t use)
{
    // This is the internal API, so it is OK to assert on bad parameter.
    LOG_ALWAYS_FATAL_IF((unsigned) use >= (unsigned) AUDIO_UNIQUE_ID_USE_MAX);
    const int maxRetries = use == AUDIO_UNIQUE_ID_USE_SESSION ? 3 : 1;
    for (int retry = 0; retry < maxRetries; retry++) {
        // The cast allows wraparound from max positive to min negative instead of abort
        // 這個和java中的cas一個套路呀，都是原子操作
        // audio_unique_id_t包含兩部分（2進位制模式表示）： xxxxxx... 後面3位是usage，前面（左邊）的是真正的id，每次+1，因為是從8開始，所以每次+8，也就是AUDIO_UNIQUE_ID_USE_MAX。
        uint32_t base = (uint32_t) atomic_fetch_add_explicit(&mNextUniqueIds[use],
                (uint_fast32_t) AUDIO_UNIQUE_ID_USE_MAX, memory_order_acq_rel);
        ALOG_ASSERT(audio_unique_id_get_use(base) == AUDIO_UNIQUE_ID_USE_UNSPECIFIED);
        // allow wrap by skipping 0 and -1 for session ids
        if (!(base == 0 || base == (~0u & ~AUDIO_UNIQUE_ID_USE_MASK))) {
            ALOGW_IF(retry != 0, "unique ID overflow for use %d", use);
            return (audio_unique_id_t) (base | use);
        }
    }
    // We have no way of recovering from wraparound
    LOG_ALWAYS_FATAL("unique ID overflow for use %d", use);
    // TODO Use a floor after wraparound.  This may need a mutex.
}

• 回到剛才AudioRecord中的 openRecord_l()

// frameworks/av/media/libmedia/AudioRecord.cpp

// must be called with mLock held
status_t AudioRecord::openRecord_l(const Modulo<uint32_t> &epoch, const String16& opPackageName)
{
    // 獲取AudioFlinger例項（binder代理）
    const sp<IAudioFlinger>& audioFlinger = AudioSystem::get_audio_flinger();
    if (audioFlinger == 0) {
        ALOGE("Could not get audioflinger");
        return NO_INIT;
    }
    ......

    audio_io_handle_t input;

    // mFlags (not mOrigFlags) is modified depending on whether fast request is accepted.
    // After fast request is denied, we will request again if IAudioRecord is re-created.
    status_t status;

    // Not a conventional loop, but a retry loop for at most two iterations total.
    // Try first maybe with FAST flag then try again without FAST flag if that fails.
    // Exits loop normally via a return at the bottom, or with error via a break.
    // The sp<> references will be dropped when re-entering scope.
    // The lack of indentation is deliberate, to reduce code churn and ease merges.
    for (;;) {
    status = AudioSystem::getInputForAttr(&mAttributes, &input,
                                        mSessionId,
                                        // FIXME compare to AudioTrack
                                        mClientPid,
                                        mClientUid,
                                        mSampleRate, mFormat, mChannelMask,
                                        mFlags, mSelectedDeviceId);
    // 
    sp<IAudioRecord> record = audioFlinger->openRecord(input,
                                                       mSampleRate,
                                                       mFormat,
                                                       mChannelMask,
                                                       opPackageName,
                                                       &temp,
                                                       &flags,
                                                       mClientPid,
                                                       tid,
                                                       mClientUid,
                                                       &mSessionId,
                                                       &notificationFrames,
                                                       iMem,
                                                       bufferMem,
                                                       &status);
    .......
    }
}

• 看看AudioSystem::getInputForAttr的實現,走到IAudioPolicyService裡頭去了。

// frameworks/av/media/libmedia/AudioSystem.cpp

status_t AudioSystem::getInputForAttr(const audio_attributes_t *attr,
                                audio_io_handle_t *input,
                                audio_session_t session,
                                pid_t pid,
                                uid_t uid,
                                uint32_t samplingRate,
                                audio_format_t format,
                                audio_channel_mask_t channelMask,
                                audio_input_flags_t flags,
                                audio_port_handle_t selectedDeviceId)
{
    const sp<IAudioPolicyService>& aps = AudioSystem::get_audio_policy_service();
    if (aps == 0) return NO_INIT;
    return aps->getInputForAttr(
            attr, input, session, pid, uid,
            samplingRate, format, channelMask, flags, selectedDeviceId);
}

• AudioPolicyService這傢伙不單純呀，裡面好複雜呀

程式碼位置

• android.media.AudioRecord: frameworks/base/media/java/android/media/
• android_media_AudioRecord.cpp: frameworks/base/core/jni/
• AudioRecord.h: frameworks/av/include/media/
• AudioRecord.cpp: frameworks/av/media/libmedia/
• AudioSystem.cpp: frameworks/av/media/libmedia/

追蹤中的一些工具，技巧

tinycap

tinycap是alsa的使用者空間的幫助工具，原始碼參見external/tinyalsa/tinycap.c，可以用這個快速驗證硬體是否正常，driver是否正常，Android的AudioFlinger中會載入hal的實現，在s905x這個平臺上是使用alsa來完成聲音的回放、錄製、音量控制等的硬體控制的。

列出alsa識別到的音效卡

cat /proc/asound/cards

 0 [AMLM8AUDIO     ]: AML-M8AUDIO - AML-M8AUDIO
                      AML-M8AUDIO
 1 [Audio          ]: USB-Audio - J-Make USB Audio
                      J-Make J-Make USB Audio at usb-xhci-hcd.0.auto-2.4, full speed