Most similar container image to the given image of an object - Weekly Report 2

Last week, I started the image processing part of the project and used SURF method which is implemented in OpenCV library (2.0). I prepared the proper environment to develop the sample code which was presented with OpenCV version 2.0.

What Does This Program Actually Do?

This week I had the opportunity of examining and improving that sample. The code begins with specifying parameters for extracting Speeded Up Robust Features using the line


CvSURFParams params = cvSURFParams(500, 1);

and parameters correspond to:

hessianThreshold (Double)
only features with keypoint.hessian larger than that are extracted. good default value is ~300-500 (can depend on the average local contrast and sharpness of the image). user can further filter out some features based on their hessian values and other characteristics

extended (Int32)
0 means basic descriptors (64 elements each), 1 means extended descriptors (128 elements each)

and feature extracting process starts with

cvExtractSURF( object, 0, &objectKeypoints, &objectDescriptors, storage, params );

which correspond to:

void cvExtractSURF( const CvArr* image, const CvArr* mask,
CvSeq** keypoints, CvSeq** descriptors,
CvMemStorage* storage, CvSURFParams params );
image
The input 8-bit grayscale image.
mask
The optional input 8-bit mask. The features are only found in the areas that contain more than 50% of non-zero mask pixels.
keypoints
The output parameter; double pointer to the sequence of keypoints. This will be the sequence of CvSURFPoint structures:
typedef struct CvSURFPoint
{
CvPoint2D32f pt; // position of the feature within the image
int laplacian; // -1, 0 or +1. sign of the laplacian at the point.
// can be used to speedup feature comparison
// (normally features with laplacians of different signs can not match)
int size; // size of the feature
float dir; // orientation of the feature: 0..360 degrees
float hessian; // value of the hessian (can be used to approximately estimate the feature strengths;
// see also params.hessianThreshold)
}
CvSURFPoint;

descriptors
The optional output parameter; double pointer to the sequence of descriptors; Depending on the params.extended value, each element of the sequence will be either 64-element or 128-element floating-point (CV_32F) vector. If the parameter is NULL, the descriptors are not computed.
storage
Memory storage where keypoints and descriptors will be stored.
params
Various algorithm parameters put to the structure CvSURFParams.

both reference image and experimental image is processed with the cvExtractSURF function.

Using the keypoints and descriptors of the subject and reference images, in flannFindPairs function defined in the sample code, the points, which are calculated as pairs according to the approximate nearest neighbour search, are specified. Then, lines between those point pairs are drawn as coded.

How Did I Modify The Program?

With the help of the informations above, I tried to form a program which takes n+1 images as input, calculates the similarity of the first image to the subsequent images, introduce the similarity ratios to the user and show the most similar image together with the reference image in a seperate window.

I took the n+1 arguments from command line. After loading images, I used cvExtractSURF to extract images' keypoints and descriptors. Immediately after,with flannFindPairs function I found the point pairs of the reference image and other images, calculater similarity ratios with the function:


Similarity = (point pairs) / (reference feature points * subject image size) *100000000


The similarty function is just an experimental one and it will be improved later.