Android 語音遙控器的整體分析
阿新 • • 發佈：2019-01-30
今天藍芽遙控器的匯入終於完成了，分別梳理和記錄一下部分語音和藍芽相關的知識，首先是主機端上層語音部分：
一、應用層使用MediaRecorder的過程（應用層）
1.建立一個MediaRecorder
mRecorder = new MediaRecorder(); mRecorder.setAudioSource(MediaRecorder.AudioSource.MIC); mRemainingTimeCalculator.setBitRate(SoundRecorder.BITRATE_3GPP);5.設定音訊輸出格式mRecorder.setAudioSamplingRate(highQuality ? 44100 : 22050);
mRecorder.setOutputFormat(outputfileformat); mRecorder.setAudioEncoder(MediaRecorder.AudioEncoder.AAC);mRecorder.setOutputFile(path); mRecorder.setOnErrorListener(this);mRecorder.prepare();10.正式開始錄音
mRecorder.start();12.可以通過廣播改變UI狀態mWakeLock.acquire();
sendStateBroadcast();1.mRecorder.setOutputFormat(outputfileformat);中的outputfileformat是MediaRecorder.OutputFormat型別，新的音訊格式可以在這裡追加
/** * Defines the output format. These constants are used with * {@link MediaRecorder#setOutputFormat(int)}. */ public final class OutputFormat { /* Do not change these values without updating their counterparts * in include/media/mediarecorder.h! */ private OutputFormat() {} public static final int DEFAULT = 0; /** 3GPP media file format*/ public static final int THREE_GPP = 1; /** MPEG4 media file format*/ public static final int MPEG_4 = 2; /** The following formats are audio only .aac or .amr formats */ /** * AMR NB file format * @deprecated Deprecated in favor of MediaRecorder.OutputFormat.AMR_NB */ public static final int RAW_AMR = 3; /** AMR NB file format */ public static final int AMR_NB = 3; /** AMR WB file format */ public static final int AMR_WB = 4; /** @hide AAC ADIF file format */ public static final int AAC_ADIF = 5; /** AAC ADTS file format */ public static final int AAC_ADTS = 6; /** @hide Stream over a socket, limited to a single stream */ public static final int OUTPUT_FORMAT_RTP_AVP = 7; /** @hide H.264/AAC data encapsulated in MPEG2/TS */ public static final int OUTPUT_FORMAT_MPEG2TS = 8; };
2. mRecorder.setAudioSamplingRate(highQuality ? 44100 : 22050);是設定音訊的取樣頻率在prepare（）之前呼叫，因為在prepare中會對取樣頻率的可行性做檢測。取樣頻率和音訊的編碼格式有關，比如AAC 編碼的介於8k到96kHZ，AMRNB編碼的為8kHZ，AMRWB為16kHZ，根據實際需要來設定。
/**
* Sets the audio sampling rate for recording. Call this method before prepare().
* Prepare() may perform additional checks on the parameter to make sure whether
* the specified audio sampling rate is applicable. The sampling rate really depends
* on the format for the audio recording, as well as the capabilities of the platform.
* For instance, the sampling rate supported by AAC audio coding standard ranges
* from 8 to 96 kHz, the sampling rate supported by AMRNB is 8kHz, and the sampling
* rate supported by AMRWB is 16kHz. Please consult with the related audio coding
* standard for the supported audio sampling rate.
*
* @param samplingRate the sampling rate for audio in samples per second.
*/
public void setAudioSamplingRate(int samplingRate) {
if (samplingRate <= 0) {
throw new IllegalArgumentException("Audio sampling rate is not positive");
}
setParameter("audio-param-sampling-rate=" + samplingRate);
} /**
* Defines the audio encoding. These constants are used with
* {@link MediaRecorder#setAudioEncoder(int)}.
*/
public final class AudioEncoder {
/* Do not change these values without updating their counterparts
* in include/media/mediarecorder.h!
*/
private AudioEncoder() {}
public static final int DEFAULT = 0;
/** AMR (Narrowband) audio codec */
public static final int AMR_NB = 1;
/** AMR (Wideband) audio codec */
public static final int AMR_WB = 2;
/** AAC Low Complexity (AAC-LC) audio codec */
public static final int AAC = 3;
/** High Efficiency AAC (HE-AAC) audio codec */
public static final int HE_AAC = 4;
/** Enhanced Low Delay AAC (AAC-ELD) audio codec */
public static final int AAC_ELD = 5;
}
/**
* Defines the video encoding. These constants are used with
* {@link MediaRecorder#setVideoEncoder(int)}.
*/
public final class VideoEncoder {
/* Do not change these values without updating their counterparts
* in include/media/mediarecorder.h!
*/
private VideoEncoder() {}
public static final int DEFAULT = 0;
public static final int H263 = 1;
public static final int H264 = 2;
public static final int MPEG_4_SP = 3;
}4. mRecorder.prepare();中又做了什麼？這裡是呼叫jni函式去實現了。前面說過可以自定義編碼格式，
/**
* Prepares the recorder to begin capturing and encoding data. This method
* must be called after setting up the desired audio and video sources,
* encoders, file format, etc., but before start().
*
* @throws IllegalStateException if it is called after
* start() or before setOutputFormat().
* @throws IOException if prepare fails otherwise.
*/
public void prepare() throws IllegalStateException, IOException
{
_prepare();
}
status_t MediaRecorder::prepare()
{
ALOGV("prepare");
if (mMediaRecorder == NULL) {
ALOGE("media recorder is not initialized yet");
return INVALID_OPERATION;
}
if (!(mCurrentState & MEDIA_RECORDER_DATASOURCE_CONFIGURED)) {
ALOGE("prepare called in an invalid state: %d", mCurrentState);
return INVALID_OPERATION;
}
if (mIsAudioSourceSet != mIsAudioEncoderSet) {
if (mIsAudioSourceSet) {
ALOGE("audio source is set, but audio encoder is not set");
} else { // must not happen, since setAudioEncoder checks this already
ALOGE("audio encoder is set, but audio source is not set");
}
return INVALID_OPERATION;
}
if (mIsVideoSourceSet != mIsVideoEncoderSet) {
if (mIsVideoSourceSet) {
ALOGE("video source is set, but video encoder is not set");
} else { // must not happen, since setVideoEncoder checks this already
ALOGE("video encoder is set, but video source is not set");
}
return INVALID_OPERATION;
}
status_t ret = mMediaRecorder->prepare();
if (OK != ret) {
ALOGE("prepare failed: %d", ret);
mCurrentState = MEDIA_RECORDER_ERROR;
return ret;
}
mCurrentState = MEDIA_RECORDER_PREPARED;
return ret;
}這個mMediaRecorder是個Mediarecorder的成員變數，是個強指標（注意，不是智慧指標）
sp<IMediaRecorder> mMediaRecorder;5.現在要看這個IMediaRecorder型別的 mMediaRecorder->prepare()呼叫到了哪裡，我們需要弄清楚MediaRecorder這一部分的類圖：
顯然是呼叫到了MediaRecorderClient.prepare()中
status_t MediaRecorderClient::prepare()
{
ALOGV("prepare");
Mutex::Autolock lock(mLock);
if (mRecorder == NULL) {
ALOGE("recorder is not initialized");
return NO_INIT;
}
return mRecorder->prepare();
}這裡的mRecorder是一個MediaRecorderBase型別的成員，整個Android系統中目前還只有一個派生類StagefightRecorder。
所以根據繼承關係知道最終是呼叫到了StagefightRecorder.prepare()和StagefrightRecorder.start()中，這裡就正式開始錄音咯！！可以看到前面設定的音訊型別在這裡會用到
status_t StagefrightRecorder::start() {
// Get UID here for permission checking
mClientUid = IPCThreadState::self()->getCallingUid();
if (mWriter != NULL) {
ALOGE("File writer is not avaialble");
return UNKNOWN_ERROR;
}
status_t status = OK;
switch (mOutputFormat) {
case OUTPUT_FORMAT_DEFAULT:
case OUTPUT_FORMAT_THREE_GPP:
case OUTPUT_FORMAT_MPEG_4:
status = startMPEG4Recording();
break;
case OUTPUT_FORMAT_AMR_NB:
case OUTPUT_FORMAT_AMR_WB:
status = startAMRRecording();
break;
case OUTPUT_FORMAT_AAC_ADIF:
case OUTPUT_FORMAT_AAC_ADTS:
status = startAACRecording();
break;
case OUTPUT_FORMAT_RTP_AVP:
status = startRTPRecording();
break;
case OUTPUT_FORMAT_MPEG2TS:
status = startMPEG2TSRecording();
break;
default:
ALOGE("Unsupported output file format: %d", mOutputFormat);
status = UNKNOWN_ERROR;
break;
}
if ((status == OK) && (!mStarted)) {
mStarted = true;
uint32_t params = IMediaPlayerService::kBatteryDataCodecStarted;
if (mAudioSource != AUDIO_SOURCE_CNT) {
params |= IMediaPlayerService::kBatteryDataTrackAudio;
}
if (mVideoSource != VIDEO_SOURCE_LIST_END) {
params |= IMediaPlayerService::kBatteryDataTrackVideo;
}
addBatteryData(params);
}
return status;
}
現在分析下startAMRRecording（）;的具體實現
status_t StagefrightRecorder::startAMRRecording() {
CHECK(mOutputFormat == OUTPUT_FORMAT_AMR_NB ||
mOutputFormat == OUTPUT_FORMAT_AMR_WB);
if (mOutputFormat == OUTPUT_FORMAT_AMR_NB) {
if (mAudioEncoder != AUDIO_ENCODER_DEFAULT &&
mAudioEncoder != AUDIO_ENCODER_AMR_NB) {
ALOGE("Invalid encoder %d used for AMRNB recording",
mAudioEncoder);
return BAD_VALUE;
}
} else { // mOutputFormat must be OUTPUT_FORMAT_AMR_WB
if (mAudioEncoder != AUDIO_ENCODER_AMR_WB) {
ALOGE("Invlaid encoder %d used for AMRWB recording",
mAudioEncoder);
return BAD_VALUE;
}
}
mWriter = new AMRWriter(mOutputFd);
status_t status = startRawAudioRecording();
if (status != OK) {
mWriter.clear();
mWriter = NULL;
}
return status;
}
正在幹活的是這裡：
status_t StagefrightRecorder::startRawAudioRecording() {
if (mAudioSource >= AUDIO_SOURCE_CNT) {
ALOGE("Invalid audio source: %d", mAudioSource);
return BAD_VALUE;
}
status_t status = BAD_VALUE;
if (OK != (status = checkAudioEncoderCapabilities())) {
return status;
}
sp<MediaSource> audioEncoder = createAudioSource();
if (audioEncoder == NULL) {
return UNKNOWN_ERROR;
}
CHECK(mWriter != 0);
mWriter->addSource(audioEncoder);
if (mMaxFileDurationUs != 0) {
mWriter->setMaxFileDuration(mMaxFileDurationUs);
}
if (mMaxFileSizeBytes != 0) {
mWriter->setMaxFileSize(mMaxFileSizeBytes);
}
mWriter->setListener(mListener);
mWriter->start();
return OK;
}
6.startRawAudioRecording（)
(1)首先檢測波特率取樣率和通道數是否符合要求
status_t StagefrightRecorder::checkAudioEncoderCapabilities() {
clipAudioBitRate();
clipAudioSampleRate();
clipNumberOfAudioChannels();
return OK;
}（2）然後設定一個MediaWriter的物件，真正錄音的物件就是這個mWriter
可以分析其中一個派生類的具體實現來看怎麼錄音的。
正在的錄音是在這裡：
status_t AMRWriter::threadFunc() {
mEstimatedDurationUs = 0;
mEstimatedSizeBytes = 0;
bool stoppedPrematurely = true;
int64_t previousPausedDurationUs = 0;
int64_t maxTimestampUs = 0;
status_t err = OK;
prctl(PR_SET_NAME, (unsigned long)"AMRWriter", 0, 0, 0);
while (!mDone) {
MediaBuffer *buffer;
err = mSource->read(&buffer);
if (err != OK) {
break;
}
if (mPaused) {
buffer->release();
buffer = NULL;
continue;
}
mEstimatedSizeBytes += buffer->range_length();
if (exceedsFileSizeLimit()) {
buffer->release();
buffer = NULL;
notify(MEDIA_RECORDER_EVENT_INFO, MEDIA_RECORDER_INFO_MAX_FILESIZE_REACHED, 0);
break;
}
int64_t timestampUs;
CHECK(buffer->meta_data()->findInt64(kKeyTime, ×tampUs));
if (timestampUs > mEstimatedDurationUs) {
mEstimatedDurationUs = timestampUs;
}
if (mResumed) {
previousPausedDurationUs += (timestampUs - maxTimestampUs - 20000);
mResumed = false;
}
timestampUs -= previousPausedDurationUs;
ALOGV("time stamp: %lld, previous paused duration: %lld",
timestampUs, previousPausedDurationUs);
if (timestampUs > maxTimestampUs) {
maxTimestampUs = timestampUs;
}
if (exceedsFileDurationLimit()) {
buffer->release();
buffer = NULL;
notify(MEDIA_RECORDER_EVENT_INFO, MEDIA_RECORDER_INFO_MAX_DURATION_REACHED, 0);
break;
}
ssize_t n = write(mFd,
(const uint8_t *)buffer->data() + buffer->range_offset(),
buffer->range_length());
if (n < (ssize_t)buffer->range_length()) {
buffer->release();
buffer = NULL;
err = ERROR_IO;
break;
}
if (err != OK) {
break;
}
if (stoppedPrematurely) {
stoppedPrematurely = false;
}
buffer->release();
buffer = NULL;
}
if ((err == OK || err == ERROR_END_OF_STREAM) && stoppedPrematurely) {
err = ERROR_MALFORMED;
}
close(mFd);
mFd = -1;
mReachedEOS = true;
if (err == ERROR_END_OF_STREAM) {
return OK;
}
return err;
}
我們關注的是兩個點：
1.MediaBuffer *buffer;音訊最後都是來源於這裡：err = mSource->read(&buffer);。分析音訊格式，可以從這個buffer入手。
2. mFd = open(filename, O_CREAT | O_LARGEFILE | O_TRUNC | O_RDWR, S_IRUSR | S_IWUSR);
最後音訊內容是儲存在這個裡面了。
後面會針對這個buffer來分析Android 的音訊底層實現機制並新增自己的音訊編解碼。