current position:Home>ORB-SLAM3 Algorithm Learning—Frame Construction—ORB Feature Extraction and BRIEF Descriptor Calculation

ORB-SLAM3 Algorithm Learning—Frame Construction—ORB Feature Extraction and BRIEF Descriptor Calculation

2022-11-24 22:51:43I am a happy little party dish


Characteristics of the detector realizing>ORBextractor::operator()
输入一张图像,Returns the store feature pointsvector向量,Store description the sonmat,特征点数量,Steps are roughly divided into the following4步

  1. 构建图像金字塔
  2. 提取FastFeature points and useOctTreeHomogenization distribution
  3. 利用灰度质心法,计算特征点角度
  4. Calculation with rotation invariance isrBRIEF描述子


Role is set related parameters of the characteristics of the detector,Mainly feature points extraction cap on the number and the related parameters in the image pyramid,These parameters according to theORB-SLAM3的配置文件设置,Generally do not need to change
mvScaleFactor: To hold each layer of the scale of the image pyramid information,The scale factor of each layer are made of a layer of scale*缩放因子,At the bottom of the scale as1
mvLevelSigma2: To hold the square of each layer pyramid image scale factor
mvInvScaleFactormvInvLevelSigma2The reciprocal of storing the scale information,方便计算
mvImagePyramid:To hold each layer pyramid images,Size of the pyramid layers
mnFeaturesPerLevel: Record each layer pyramid images can detect the number of the biggest characteristic,Number of feature points between each layer satisfy geometric relationship,According to the configuration file setting the maximum number of feature points can be calculated according to the geometric relationship between each layer can detect the number of feature points

1.构建图像金字塔,Each layer of the pyramid image deposited in the member variablesmvImagePyramid

函数:ORBextractor::ComputePyramid(cv::Mat image)
比较简单,According to the scale of information to calculate the width and height of each layer of pyramid images,然后resize放到vector容器里

2. 提取FastFeature points and useOctTreeHomogenization distribution

2.1 特征点提取

使用的是cv自带的fastCorner extraction functioncv::FAST(...),But here the author set up a strict time thresholdiniThFAST(20)And a loose thresholdminThFAST(7)
This calculation is the center pixel with a circle around the pixel gray value differences,Less than the threshold think gray feature point is not same,That is greater than that with different characteristics point

2.2 Characteristics of uniformity distribution


  1. Number should be retained first calculate each layer of the pyramid image feature points
  2. Secondly, based on quadtree principle,Choose best response for each layer image feature points and ensure the uniform distribution of feature points

1.Number should be retained first calculate each layer of the pyramid image feature points

基本原理: Now the number of known need to extract the feature points of each image,The area of each layer of the image pyramid,Can calculate a pyramid image number of feature points and the area of unit area.After according to each layer of the pyramid in proportion to the total area of the image and the calculation number should assign each layer image feature points(吐槽:Feeling is a geometric sequence,Ratio is the proportion between two layer image area that the square of the scale factor)

我们假设第0层图像的宽为W,长为L ,缩放因子为s(这里的 0<s<1 ).The whole pyramid total areaS
那么,The number of feature points per unit area asN/S
那么,第0Layer should be allocated for the number of feature points
Then so,推出了第αLayer should be allocated for the number of feature points

2.Secondly, based on quadtree principle,Choose best response for each layer image feature points and ensure the uniform distribution of feature points

  1. 如果图片的宽度比较宽,就先把分成左右w/h份.一般的640×480的图像
  2. At the start of the only onenode.
  3. 如果node里面的点数>1,把每个node分成四个node,如果node里面的特征点为空,就不要了,删掉.
  4. 新分的node的点数>1,就再分裂成4个node.
  5. 如此,一直分裂.终止条件为:node的总数量> N_a,或者无法再进行分裂.然后从每个node里面选择一个质量最好的FAST点

图示:Assume a certain layer image feature pointsN=25
(2)node数量4<25,Continue to split

(1)4个node分裂成15个node,因为其中一个nodeThere is no point so remove thenode
(2)node数量16<25,Continue to split

Step4:The third division
(1)15个node分裂为30个node,Deleted without feature pointsnode

Step5:从每个nodeSelect the response value of one of the biggest point



ORBComputing Angle is simple,First of all a circular area of gray centroid,Connect centroid and the center of the circle to form a vector,The vector Angle is the Angle of the angular point.
Circle circle isfast特征点,圆的半径取为15,因为整个patchGenerally take is31×31的.
Gray centroid can be calculated by the following formula:

The Angle of:


至此,Not only to extractFAST角点,Also find out the Angle of the Angle point.This Angle can be used to guide describes the extraction of child,Ensure every time calculation descriptor in the same direction,Realize the Angle invariance.
这就是Oriented FAST.


4.1 BRIEF描述子

BRIEFThe description of the is a binary sub,计算和匹配的速度都很快.计算步骤如下:

  1. 以关键点为中心,选择一个31×31的像素块

  2. In this block according to certain methods selectionN对点,N一般取256,The method here is generally through early learning effect is the best choice256个位置

  3. 对于点对{A,B},By comparing the gray size of these two points determine a binary results,0或者1.

  4. 这样把256Points to the results of the row,就形成了BRIEF描述子.
    [1 0 1 0 0 ... 1 0 0]

BRIEFThe matching of using hamming distance,非常快,Simple said is have a look at how many different digits.The following two descriptor,不同的位数为4.In fact we selected points of logarithm to256,Then distance scope is0~256.
[1 0 1 1 1 0]
[1 1 0 0 1 1]

4.2 rBRIEF

思想简述: The direction of the feature points or whether to calculate the descriptor in the direction of the,The ultimate purpose is in order to more accurately match the corresponding feature points in two images,The basis of a match is in the judgement by describing the son.Two feature points around the smaller pixel information distribution differ,The less the descriptor distance,Two feature points aregoodMatch的可能性就越高.

这样,Determine whether two feature points matching problem is converted into computing feature points around the pixel distribution difference problem,And then again into the surrounding pixels of selection and the design of this kind of difference calculation.

对于BRIEFDescribe the child is,Calculation is calculating hamming distance,Point to the choice of the above mentioned also use learning method to carry on the design,现在就剩下一个问题了,How to determine the feature points involved with the pixels around .

BRIEF把关键点"周围像素"Become a pixel is fastpatch,If you don't consider rotating chosepatch就简单了,Direct pixel coordinates with feature points as the center,Choose the length and width of fixed apatch就可以了,如下图所示.

但是存在一个问题,As the camera movement,特征点在x方向和yChange direction of the pixels on the,This selection of pixel in the image block will happen very big change,As abovePATCH1PATCH2,If take the two pixels fast to calculate distance to delineate the son,There must be a big difference.

基于这一现象,rBRIEFOn the basis of original increased rotary information,When the image rotation occurs,Feature points must occur in the direction of the spin,Select the image block scope also gave him a whole a spin,就变成了下图所示,使用PATCH1’The pixels andPATCH2The pixel to compare.

BRIEFRotation invariance principle is roughly such,下面看一下具体的实现.

BRIEFDescriptor is not consider rotating invariance,rBRIEF根据Oriented FASTCalculate the Angle of the,把原始的256The coordinates of a point to rotate after,In gray.So as to realize the rotation invariance.

原始的256A point to coordinate asS,旋转后的为S_θ


// At the end of the feature extraction,会在computeOrientationFunction of calculating the Angle of the characteristic points
        float angle = (float)kpt.angle*factorPI;// 将角度转换为弧度
        float a = (float)cos(angle), b = (float)sin(angle);// Computing cosine and sine value
        // 和之前一样,Here for the pixel feature point is a pointer to theuchar类型的指针,Is not the feature points corresponding pixel gray value,需要注意
        // 如果把imgThe address-of operator in front of the&去掉,Access is the position corresponding to the grey value of the
        // In other words is tocenterIndexes, to get the value of the other elements,This is feature in the macro definition under the
        const uchar* center = &<uchar>(cvRound(, cvRound(;
        // 这里的step是Mat的属性之一,Said of a data byte length,Skip the specified step when convenient later iterations
        // For example, a single channel grayscale image pixel values in my bytes is here700,而RGBThe length of the image a pixel is2100
        const int step = (int)img.step;
// 这里定义了一个宏,Used in subsequent calculations easy
// 这里的center就是上面获取的uchar的指针,而pattern就是传入的参数
// It a long list of are in the calculation of index,而对centerTake the index is the index to get(位置)所对应的灰度值
// Random spot setx轴方向,Rotate to the direction of feature points
#define GET_VALUE(idx) \ center[cvRound(pattern[idx].x*b + pattern[idx].y*a)*step + \ cvRound(pattern[idx].x*a - pattern[idx].y*b)]

Need to compute the pixel position of the corresponding relations between before and after the rotation as shown in the figure below,The pixel coordinates spins a feature point the direction.


-θ:The key point of
-φ:Pixel block of a pixel before spinning withx轴夹角
-r:Pixel block the distance to the origin
-p(x,y):Before rotating pixel coordinates
-p'(x,y):After rotating pixel coordinates

Now became knownθ,p(x,y)p'(x,y),推导公式如下:

以上就是ORB-SLAM3Image frame structure in the process of feature points extraction and the content of the descriptor computation part.

copyright notice
author[I am a happy little party dish],Please bring the original link to reprint, thank you.

Random recommended