iOS人臉識別、自定義相機、影象掃描、系統自帶二維碼識別
前段時間遇到一個需求,需要實時掃描影象,並且設定攝像頭的尺寸為1080x1920,然後從中間截取出512x512的圖片傳給第三方SDK做進一步業務處理,直到SDK返回正確的處理結果。
一頓Google,發現影象預覽、人臉識別、二維碼識別這些蘋果都幫我們做好了,而且它們都是基於AVFoundation框架實現的。
話不多說,上程式碼~!
用到的類,主要有這些:
//硬體裝置
@property (nonatomic, strong) AVCaptureDevice *device;
//輸入流
@property (nonatomic, strong) AVCaptureDeviceInput *input;
//協調輸入輸出流的資料
@property (nonatomic, strong) AVCaptureSession *session;
//預覽層
@property (nonatomic, strong) AVCaptureVideoPreviewLayer *previewLayer;
//輸出流
@property (nonatomic, strong) AVCaptureStillImageOutput *stillImageOutput; //用於捕捉靜態圖片
@property (nonatomic, strong) AVCaptureVideoDataOutput *videoDataOutput; //原始視訊幀,用於獲取實時影象以及視訊錄製
@property (nonatomic, strong) AVCaptureMetadataOutput *metadataOutput; //用於二維碼識別以及人臉識別
-
首先我們需要在手機上把影象顯示出來
1.1 獲取硬體裝置
-(AVCaptureDevice *)device{ if (_device == nil) { _device = [AVCaptureDevice defaultDeviceWithMediaType:AVMediaTypeVideo]; if ([_device lockForConfiguration:nil]) { //自動閃光燈
device有很多屬性可以調整(注意調整device屬性的時候需要上鎖, 調整完再解鎖):
閃光燈typedef NS_ENUM(NSInteger, AVCaptureFlashMode) { AVCaptureFlashModeOff = 0, AVCaptureFlashModeOn = 1, AVCaptureFlashModeAuto = 2 } NS_AVAILABLE(10_7, 4_0) __TVOS_PROHIBITED;
前後置攝像頭
typedef NS_ENUM(NSInteger, AVCaptureDevicePosition) { AVCaptureDevicePositionUnspecified = 0, AVCaptureDevicePositionBack = 1, AVCaptureDevicePositionFront = 2 } NS_AVAILABLE(10_7, 4_0) __TVOS_PROHIBITED;
手電筒
typedef NS_ENUM(NSInteger, AVCaptureTorchMode) { AVCaptureTorchModeOff = 0, AVCaptureTorchModeOn = 1, AVCaptureTorchModeAuto = 2, } NS_AVAILABLE(10_7, 4_0) __TVOS_PROHIBITED;
對焦
typedef NS_ENUM(NSInteger, AVCaptureFocusMode) { AVCaptureFocusModeLocked = 0, AVCaptureFocusModeAutoFocus = 1, AVCaptureFocusModeContinuousAutoFocus = 2, } NS_AVAILABLE(10_7, 4_0) __TVOS_PROHIBITED;
曝光
typedef NS_ENUM(NSInteger, AVCaptureExposureMode) { AVCaptureExposureModeLocked = 0, AVCaptureExposureModeAutoExpose = 1, AVCaptureExposureModeContinuousAutoExposure = 2, AVCaptureExposureModeCustom NS_ENUM_AVAILABLE_IOS(8_0) = 3, } NS_AVAILABLE(10_7, 4_0) __TVOS_PROHIBITED;
白平衡
typedef NS_ENUM(NSInteger, AVCaptureWhiteBalanceMode) { AVCaptureWhiteBalanceModeLocked = 0, AVCaptureWhiteBalanceModeAutoWhiteBalance = 1, AVCaptureWhiteBalanceModeContinuousAutoWhiteBalance = 2, } NS_AVAILABLE(10_7, 4_0) __TVOS_PROHIBITED;
1.2 獲取硬體的輸入流
-(AVCaptureDeviceInput *)input{ if (_input == nil) { _input = [[AVCaptureDeviceInput alloc] initWithDevice:self.device error:nil]; } return _input; }
建立輸入流的時候,會彈出alert向用戶獲取相機許可權
1.3 需要一個用來協調輸入和輸出資料的會話,然後把input新增到會話中-(AVCaptureSession *)session{ if (_session == nil) { _session = [[AVCaptureSession alloc] init]; if ([_session canAddInput:self.input]) { [_session addInput:self.input]; } } return _session; }
1.4 然後我們需要一個預覽影象的層
-(AVCaptureVideoPreviewLayer *)previewLayer{ if (_previewLayer == nil) { _previewLayer = [[AVCaptureVideoPreviewLayer alloc] initWithSession:self.session]; _previewLayer.frame = self.view.layer.bounds; } return _previewLayer; }
1.5 最後把previewLayer新增到self.view.layer上
[self.view.layer addSublayer:self.previewLayer];
1.6 找個合適的位置,讓session執行起來,比如viewWillAppear
-(void)viewWillAppear:(BOOL)animated{ [super viewWillAppear:animated]; [self.session startRunning]; }
-
搞一個按鈕用來控制手電筒
#pragma mark - 手電筒 -(void)openTorch:(UIButton*)button{ button.selected = !button.selected; Class captureDeviceClass = NSClassFromString(@"AVCaptureDevice"); if (captureDeviceClass != nil) { if ([self.device hasTorch] && [self.device hasFlash]){ [self.device lockForConfiguration:nil]; if (button.selected) { [self.device setTorchMode:AVCaptureTorchModeOn]; } else { [self.device setTorchMode:AVCaptureTorchModeOff]; } [self.device unlockForConfiguration]; } } }
-
再搞一個按鈕來切換前後置攝像頭
#pragma mark - 切換前後攝像頭 -(void)switchCamera{ NSUInteger cameraCount = [[AVCaptureDevice devicesWithMediaType:AVMediaTypeVideo] count]; if (cameraCount > 1) { AVCaptureDevice *newCamera = nil; AVCaptureDeviceInput *newInput = nil; AVCaptureDevicePosition position = [[self.input device] position]; if (position == AVCaptureDevicePositionFront){ newCamera = [self cameraWithPosition:AVCaptureDevicePositionBack]; }else { newCamera = [self cameraWithPosition:AVCaptureDevicePositionFront]; } newInput = [AVCaptureDeviceInput deviceInputWithDevice:newCamera error:nil]; if (newInput != nil) { [self.session beginConfiguration]; [self.session removeInput:self.input]; if ([self.session canAddInput:newInput]) { [self.session addInput:newInput]; self.input = newInput; }else { [self.session addInput:self.input]; } [self.session commitConfiguration]; } } }
-(AVCaptureDevice *)cameraWithPosition:(AVCaptureDevicePosition)position{ NSArray *devices = [AVCaptureDevice devicesWithMediaType:AVMediaTypeVideo]; for ( AVCaptureDevice *device in devices ) if ( device.position == position ) return device; return nil; }
-
使用AVCaptureStillImageOutput捕獲靜態圖片
4.1 建立一個AVCaptureStillImageOutput物件
-(AVCaptureStillImageOutput *)stillImageOutput{ if (_stillImageOutput == nil) { _stillImageOutput = [[AVCaptureStillImageOutput alloc] init]; } return _stillImageOutput; }
4.2 將stillImageOutput新增到session中
if ([_session canAddOutput:self.stillImageOutput]) { [_session addOutput:self.stillImageOutput]; }
4.3 搞個拍照按鈕,擷取靜態圖片
//AVCaptureStillImageOutput擷取靜態圖片,會有快門聲 -(void)screenshot{ AVCaptureConnection * videoConnection = [self.stillImageOutput connectionWithMediaType:AVMediaTypeVideo]; if (!videoConnection) { NSLog(@"take photo failed!"); return; } [self.stillImageOutput captureStillImageAsynchronouslyFromConnection:videoConnection completionHandler:^(CMSampleBufferRef imageDataSampleBuffer, NSError *error) { if (imageDataSampleBuffer == NULL) { return; } NSData * imageData = [AVCaptureStillImageOutput jpegStillImageNSDataRepresentation:imageDataSampleBuffer]; UIImage *image = [UIImage imageWithData:imageData]; [self saveImageToPhotoAlbum:image]; }]; }
-
使用AVCaptureVideoOutput實時獲取預覽影象,這也是樓主的專案需求所在
5.1 建立AVCaptureVideoOutput物件
-(AVCaptureVideoDataOutput *)videoDataOutput{ if (_videoDataOutput == nil) { _videoDataOutput = [[AVCaptureVideoDataOutput alloc] init]; [_videoDataOutput setSampleBufferDelegate:self queue:dispatch_get_main_queue()]; } return _videoDataOutput; }
5.2 將videoDataOutput新增session中
if ([_session canAddOutput:self.videoDataOutput]) { [_session addOutput:self.videoDataOutput]; }
5.3 遵守AVCaptureVideoDataOutputSampleBufferDelegate協議,並實現它的代理方法
#pragma mark - AVCaptureVideoDataOutputSampleBufferDelegate //AVCaptureVideoDataOutput獲取實時影象,這個代理方法的回撥頻率很快,幾乎與手機螢幕的重新整理頻率一樣快 -(void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection{ largeImage = [self imageFromSampleBuffer:sampleBuffer]; }
5.4 實現imageFromSampleBuffer:方法,將CMSampleBufferRef轉為NSImage
//CMSampleBufferRef轉NSImage -(UIImage *)imageFromSampleBuffer:(CMSampleBufferRef)sampleBuffer{ // 為媒體資料設定一個CMSampleBuffer的Core Video影象快取物件 CVImageBufferRef imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer); // 鎖定pixel buffer的基地址 CVPixelBufferLockBaseAddress(imageBuffer, 0); // 得到pixel buffer的基地址 void *baseAddress = CVPixelBufferGetBaseAddress(imageBuffer); // 得到pixel buffer的行位元組數 size_t bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer); // 得到pixel buffer的寬和高 size_t width = CVPixelBufferGetWidth(imageBuffer); size_t height = CVPixelBufferGetHeight(imageBuffer); // 建立一個依賴於裝置的RGB顏色空間 CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB(); // 用抽樣快取的資料建立一個位圖格式的圖形上下文(graphics context)物件 CGContextRef context = CGBitmapContextCreate(baseAddress, width, height, 8, bytesPerRow, colorSpace, kCGBitmapByteOrder32Little | kCGImageAlphaPremultipliedFirst); // 根據這個點陣圖context中的畫素資料建立一個Quartz image物件 CGImageRef quartzImage = CGBitmapContextCreateImage(context); // 解鎖pixel buffer CVPixelBufferUnlockBaseAddress(imageBuffer,0); // 釋放context和顏色空間 CGContextRelease(context); CGColorSpaceRelease(colorSpace); // 用Quartz image建立一個UIImage物件image UIImage *image = [UIImage imageWithCGImage:quartzImage]; // 釋放Quartz image物件 CGImageRelease(quartzImage); return (image); }
眼看大功告成,結果一執行,建立core graphic上下文的時候報錯:
CGBitmapContextCreate: invalid data bytes/row: should be at least 7680 for 8 integer bits/component, 3 components, kCGImageAlphaPremultipliedFirst.** CGBitmapContextCreateImage: invalid context 0x0. If you want to see the backtrace, please set CG_CONTEXT_SHOW_BACKTRACE environmental variable.**
又是一通Google,發現stackoverflow上很多這種問答,樓主英語不好,又是一通翻譯,發現大家都是在說畫素元件位數什麼的,摸索半天找到解決辦法,設定videoDataOutput的畫素格式:
[_videoDataOutput setVideoSettings:[NSDictionary dictionaryWithObject:[NSNumber numberWithInt:kCVPixelFormatType_32BGRA] forKey:(id)kCVPixelBufferPixelFormatTypeKey]];
再執行,還有問題,獲取到的圖片是顛倒的,尼瑪,真是多災多難,不過這個簡單,很快找到解決方法,設定一下視訊的方向:
#pragma mark - AVCaptureVideoDataOutputSampleBufferDelegate //AVCaptureVideoDataOutput獲取實時影象,這個代理方法的回撥頻率很快,幾乎與手機螢幕的重新整理頻率一樣快 -(void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection{ [connection setVideoOrientation:AVCaptureVideoOrientationPortrait]; largeImage = [self imageFromSampleBuffer:sampleBuffer]; }
5.5 還記得樓主一開始的需求嗎,設定攝像頭的尺寸為1080x1920,然後從中間截取出512x512的圖片傳給第三方SDK做進一步業務處理:
[_session setSessionPreset:AVCaptureSessionPreset1920x1080];
smallImage = [largeImage imageCompressTargetSize:CGSizeMake(512.0f, 512.0f)];
到這裡為止,樓主的需求就大功告成啦
-
使用AVCaptureMetadataOutput識別二維碼
6.1 建立AVCaptureMetadataOutput物件
-(AVCaptureMetadataOutput *)metadataOutput{ if (_metadataOutput == nil) { _metadataOutput = [[AVCaptureMetadataOutput alloc]init]; [_metadataOutput setMetadataObjectsDelegate:self queue:dispatch_get_main_queue()]; //設定掃描區域 _metadataOutput.rectOfInterest = self.view.bounds; } return _metadataOutput; }
6.2 將metadataOutput新增到session中,並且設定掃描型別
if ([_session canAddOutput:self.metadataOutput]) { [_session addOutput:self.metadataOutput]; //設定掃碼格式 self.metadataOutput.metadataObjectTypes = @[ AVMetadataObjectTypeQRCode, AVMetadataObjectTypeEAN13Code, AVMetadataObjectTypeEAN8Code, AVMetadataObjectTypeCode128Code ]; }
6.3 遵守AVCaptureMetadataOutputObjectsDelegate協議,並實現其代理方法
#pragma mark - AVCaptureMetadataOutputObjectsDelegate -(void)captureOutput:(AVCaptureOutput *)captureOutput didOutputMetadataObjects:(NSArray *)metadataObjects fromConnection:(AVCaptureConnection *)connection{ if (metadataObjects.count>0) { [self.session stopRunning]; AVMetadataMachineReadableCodeObject *metadataObject = [metadataObjects objectAtIndex :0]; NSLog(@"二維碼內容 : %@",metadataObject.stringValue); } }
-
關於人臉識別
人臉識別也是基於AVCaptureMetadataOutput實現的,跟二維碼識別的區別在於,掃描型別:
self.metadataOutput.metadataObjectTypes = @[AVMetadataObjectTypeFace];
#pragma mark - AVCaptureMetadataOutputObjectsDelegate -(void)captureOutput:(AVCaptureOutput *)captureOutput didOutputMetadataObjects:(NSArray *)metadataObjects fromConnection:(AVCaptureConnection *)connection{ if (metadataObjects.count>0) { [self.session stopRunning]; AVMetadataMachineReadableCodeObject *metadataObject = [metadataObjects objectAtIndex :0]; if (metadataObject.type == AVMetadataObjectTypeFace) { AVMetadataObject *objec = [self.previewLayer transformedMetadataObjectForMetadataObject:metadataObject]; NSLog(@"%@",objec); } } }
至於怎麼利用它來實現具體的功能需求,樓主也很方哈,這裡有個連結可以參考一下:基於 OpenCV 的人臉識別
好啦,就這麼多了,程式碼在這裡,水平有限,有不對的地方還請多多指教
參考資料:iOS 上的相機捕捉
文/戴尼瑪(簡書作者)
原文連結:http://www.jianshu.com/p/61ca3a917fe5
著作權歸作者所有,轉載請聯絡作者獲得授權,並標註“簡書作者”。