1. 程式人生 > >[opencv] lk光流法小結

[opencv] lk光流法小結

本文記錄了LK光流法的原理和opencv的使用例項。

一、理論部分

參考:http://www.cnblogs.com/andyanut/p/5893168.html

二、在opencv中的使用

void cvCalcOpticalFlowPyrLK(  
                            const CvArr* prev,  
                            const CvArr* curr,  
                            CvArr* prevPyr,  
                            CvArr* currPyr,  
                            const CvPoint2D32f* prevFeatures,  
                            CvPoint2D32f* currFeatures,  
                            int count,  
                            CvSize winSize,  
                            int level,  
                            char* status,  
                            float* track error,  
                            CvTermCriteria criteria,  
                            int flags );

引數介紹:

第一個8位輸入影象或者通過
  • nextImg – second input image or pyramid of the same size and the same type as prevImg.
第二個輸入影象或者和prevImg相同尺寸和型別的金字塔
  • prevPts – vector of 2D points for which the flow needs to be found; point coordinates must be single-precision floating-point numbers.
二維點向量儲存找到的光流;點座標必須是單精度浮點數
  • nextPts – output vector of 2D points (with single-precision floating-point coordinates) containing the calculated new positions of input features in the second image; when OPTFLOW_USE_INITIAL_FLOW flag is passed, the vector must have the same size as in the input.
輸出二維點向量(用單精度浮點座標)包括第二幅影象中計算的輸入特徵的新點位置;當OPTFLOW_USE_INITIAL_FLOW
 標誌通過,向量必須有和輸入一樣的尺寸。
  • status – output status vector (of unsigned chars); each element of the vector is set to 1 if the flow for the corresponding features has been found, otherwise, it is set to 0.
輸出狀態向量(無符號char);如果相應的流特徵被發現,向量的每個元素被設定為1,否則,被置為0.
  • err – output vector of errors; each element of the vector is set to an error for the corresponding feature, type of the error measure can be set in flags parameter; if the flow wasn’t found then the error is not defined (use the status parameter to find such cases).
輸出錯誤向量;向量的每個元素被設為相應特徵的一個錯誤,誤差測量的型別可以在flags引數中設定;如果流不被發現然後錯誤未被定義(使用status(狀態)引數找到此情形)。
  • winSize – size of the search window at each pyramid level.
在每個金字塔水平搜尋視窗的尺寸。
  • maxLevel – 0-based maximal pyramid level number; if set to 0, pyramids are not used (single level), if set to 1, two levels are used, and so on; if pyramids are passed to input then algorithm will use as many levels as pyramids have but no more than maxLevel.
  • criteria – parameter, specifying the termination criteria of the iterative search algorithm (after the specified maximum number of iterations criteria.maxCount or when the search window moves by less than criteria.epsilon.
  • flags –

    operation flags:

    • OPTFLOW_USE_INITIAL_FLOW uses initial estimations, stored in nextPts; if the flag is not set, then prevPts is copied to nextPts and is considered the initial estimate.
    • OPTFLOW_LK_GET_MIN_EIGENVALS use minimum eigen values as an error measure (seeminEigThreshold description); if the flag is not set, then L1 distance between patches around the original and a moved point, divided by number of pixels in a window, is used as a error measure.
  • minEigThreshold – the algorithm calculates the minimum eigen value of a 2x2 normal matrix of optical flow equations (this matrix is called a spatial gradient matrix in [Bouguet00]), divided by number of pixels in a window; if this value is less than minEigThreshold, then a corresponding feature is filtered out and its flow is not processed, so it allows to remove bad points and get a performance boost.

函式示例:

下面的程式實現了利用LK光流法對視訊中的運動目標進行跟蹤

#include <opencv2/video/video.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/core/core.hpp>
#include <iostream>
#include <cstdio>

using namespace std;
using namespace cv;

void tracking(Mat &frame, Mat &output);
bool addNewPoints();
bool acceptTrackedPoint(int i);

string window_name = "optical flow tracking";
Mat gray;	// 當前圖片
Mat gray_prev;	// 預測圖片
vector<Point2f> points[2];	// point0為特徵點的原來位置,point1為特徵點的新位置
vector<Point2f> initial;	// 初始化跟蹤點的位置
vector<Point2f> features;	// 檢測的特徵
int maxCount = 500;		// 檢測的最大特徵數
double qLevel = 0.01;	// 特徵檢測的等級
double minDist = 10.0;	// 兩特徵點之間的最小距離
vector<uchar> status;	// 跟蹤特徵的狀態,特徵的流發現為1,否則為0
vector<float> err;
int main()
{
	Mat frame;
	Mat result;

	VideoCapture capture("E:\\data\\hog_svm\\pedestrian.avi");

	if (capture.isOpened()/*capture*/)	// 攝像頭讀取檔案開關
	{
		while (true)
		{
			capture >> frame;

			if (!frame.empty())
			{
				tracking(frame, result);
			}
			else
			{
				printf(" --(!) No captured frame -- Break!");
				break;
			}

			int c = waitKey(100);
			if ((char)c == 27)
			{
				break;
			}
		}
	}
	return 0;
}

//////////////////////////////////////////////////////////////////////////
// function: tracking
// brief: 跟蹤
// parameter: frame	輸入的視訊幀
//			  output 有跟蹤結果的視訊幀
// return: void
//////////////////////////////////////////////////////////////////////////
void tracking(Mat &frame, Mat &output)
{
	cvtColor(frame, gray, CV_BGR2GRAY);
	frame.copyTo(output);
	// 新增特徵點
	if (addNewPoints())
	{
		goodFeaturesToTrack(gray, features, maxCount, qLevel, minDist);
		points[0].insert(points[0].end(), features.begin(), features.end());
		initial.insert(initial.end(), features.begin(), features.end());
	}

	if (gray_prev.empty())
	{
		gray.copyTo(gray_prev);
	}
	// l-k光流法運動估計
	calcOpticalFlowPyrLK(gray_prev, gray, points[0], points[1], status, err);
	// 去掉一些不好的特徵點
	int k = 0;
	for (size_t i = 0; i<points[1].size(); i++)
	{
		if (acceptTrackedPoint(i))
		{
			initial[k] = initial[i];
			points[1][k++] = points[1][i];
		}
	}
	points[1].resize(k);
	initial.resize(k);
	// 顯示特徵點和運動軌跡
	for (size_t i = 0; i<points[1].size(); i++)
	{
		line(output, initial[i], points[1][i], Scalar(0, 0, 255));
		circle(output, points[1][i], 3, Scalar(255, 0, 0), -1);
	}

	// 把當前跟蹤結果作為下一此參考
	swap(points[1], points[0]);
	swap(gray_prev, gray);

	imshow(window_name, output);
}

//////////////////////////////////////////////////////////////////////////
// function: addNewPoints
// brief: 檢測新點是否應該被新增
// parameter:
// return: 是否被新增標誌
//////////////////////////////////////////////////////////////////////////
bool addNewPoints()
{
	return points[0].size() <= 10;
}

//////////////////////////////////////////////////////////////////////////
// function: acceptTrackedPoint
// brief: 決定哪些跟蹤點被接受
// parameter:
// return:
//////////////////////////////////////////////////////////////////////////
bool acceptTrackedPoint(int i)
{
	return status[i] && ((abs(points[0][i].x - points[1][i].x) + abs(points[0][i].y - points[1][i].y)) > 2);
}


第一個8位輸入影象或者通過
  • nextImg – second input image or pyramid of the same size and the same type as prevImg.
第二個輸入影象或者和prevImg相同尺寸和型別的金字塔
  • prevPts – vector of 2D points for which the flow needs to be found; point coordinates must be single-precision floating-point numbers.
二維點向量儲存找到的光流;點座標必須是單精度浮點數
  • nextPts – output vector of 2D points (with single-precision floating-point coordinates) containing the calculated new positions of input features in the second image; when OPTFLOW_USE_INITIAL_FLOW flag is passed, the vector must have the same size as in the input.
輸出二維點向量(用單精度浮點座標)包括第二幅影象中計算的輸入特徵的新點位置;當OPTFLOW_USE_INITIAL_FLOW 標誌通過,向量必須有和輸入一樣的尺寸。
  • status – output status vector (of unsigned chars); each element of the vector is set to 1 if the flow for the corresponding features has been found, otherwise, it is set to 0.
輸出狀態向量(無符號char);如果相應的流特徵被發現,向量的每個元素被設定為1,否則,被置為0.
  • err – output vector of errors; each element of the vector is set to an error for the corresponding feature, type of the error measure can be set in flags parameter; if the flow wasn’t found then the error is not defined (use the status parameter to find such cases).
輸出錯誤向量;向量的每個元素被設為相應特徵的一個錯誤,誤差測量的型別可以在flags引數中設定;如果流不被發現然後錯誤未被定義(使用status(狀態)引數找到此情形)。
  • winSize – size of the search window at each pyramid level.
在每個金字塔水平搜尋視窗的尺寸。
  • maxLevel – 0-based maximal pyramid level number; if set to 0, pyramids are not used (single level), if set to 1, two levels are used, and so on; if pyramids are passed to input then algorithm will use as many levels as pyramids have but no more than maxLevel.
  • criteria – parameter, specifying the termination criteria of the iterative search algorithm (after the specified maximum number of iterations criteria.maxCount or when the search window moves by less than criteria.epsilon.
  • flags –

    operation flags:

    • OPTFLOW_USE_INITIAL_FLOW uses initial estimations, stored in nextPts; if the flag is not set, then prevPts is copied to nextPts and is considered the initial estimate.
    • OPTFLOW_LK_GET_MIN_EIGENVALS use minimum eigen values as an error measure (seeminEigThreshold description); if the flag is not set, then L1 distance between patches around the original and a moved point, divided by number of pixels in a window, is used as a error measure.
  • minEigThreshold – the algorithm calculates the minimum eigen value of a 2x2 normal matrix of optical flow equations (this matrix is called a spatial gradient matrix in [Bouguet00]), divided by number of pixels in a window; if this value is less than minEigThreshold, then a corresponding feature is filtered out and its flow is not processed, so it allows to remove bad points and get a performance boost.