Dynamic feature detection using virtual correction and camera oscillations

Visual SLAM is Simultaneous Localization And Mapping using visual information obtained from a camera. SLAM algorithms have gained much popularity due to their importance for mobile robot applications in unstructured environments. The SLAM objective is to localise a robot moving into an unknown environment and build a map of this environment. In the feature-based SLAM approach, the map is described by a set of features (landmarks), where these landmarks are supposed to be static. The assumption of static scene landmarks is not valid for all scene points, since the robot has to work in a dynamic environment and select only static features.

To detect the dynamic target using a mono-vision equipped moving platform, the observer (camera) needs to perform more manoeuvres than the target [1]. In this work, the re-projection error, RE, is used to distinguish the dynamic points in the image plane. The re-projection error is the distance between the measured point and the projection of the estimation of the same point, and it is represented by the Mahalanobis Distance, MD. The RE was used to distinguish the dynamic point, DP or the inconsistent point. If the RE for this feature is less than a defined threshold, the point is classified as inlier and used for the correction. If the feature’s RE is more than the threshold, the point is classified as inconsistent and deleted [2]. The RE method faces the problem of the static points, that may change their positions whenever a new observation is available, are misclassified and can not be recovered.

To solve the camera observability issue, we are using a camera with superimposed oscillations, which was shown to improve depth estimates of the features. RE is the main metric to judge if the feature is well classified by the VSLAM algorithm. Our core approach is that, if we can check the points RE, then we can isolate a set of points for which the RE is above a threshold and is believed to include the dynamic points together with the static points having irregular and out of model image motions. The set of uncertain points is then checked through a technique we call “virtual correction” for each point separately. This isolation is useful since it can recover misclassified static points. Such points are informative for the SLAM system, especially for distant points, which are good for robot orientation estimation [3].

VSLAM system with the proposed virtual correction method.

The flowchart shows the steps of The VC method. The red parts represent the VC added parts over the RE method

Simulation Experiments:

The VC method will be compared with the standard RE method. In the experiments, a robot goes through an environment of landmarks in an area of 16 m x 16 m. This environment contains 162 landmarks configured into two layers of landmarks. The same features configuration is used throughout all our experiments to avoid the scene features composition effect [4]. The experiments are made using the Monte Carlo method, 25 runs using random process and measurement noises.

Static Environment Testing

Experiments were made for different types of robot motions: forward, curved, and lateral. Each experiment was made once using the RE method and another using the VC method for comparison. Each VC experiment was made, once using the steady camera and another using the oscillating camera. The localization errors and the number of recovered points were registered.

The table below shows the results of the static environment testing. It can be observed that the localisation errors of all types of motions for the VC method are lower than in the RE case. The VC method successfully recovered the misclassified features in all experiments. The VC method experiments using an oscillating camera, show the same number of the recovered features, besides smaller errors for all motion types.

The position and the orientation errors and the recovered features number for various motion types in a static environment. The RE method results are done using a steady camera, and the VC method results are done using steady and oscillating cameras.

Dynamic Environment Testing

Experiments were made for the basic robot motion type, the forward motion and with two cases of the dynamic points linear motion, the lateral motion and the most difficult motion, the inverse forward motion. The figure below shows the motion of the robot and the dynamic points in the two cases of dynamic points motion. The experiments were made using both the RE and the VC methods. Each VC experiment was made one time using the steady camera and another using the oscillating camera.

The dotted blue arrow represents the robot motion and the solid red one represents the DP motion.

In the case of lateral motion of the dynamic points, the steady camera and the oscillating camera detected the dynamic points successfully. On the other hand, the VC method experiment, using an oscillating camera, shows better performance than the RE method. In the difficult inverse forward motion of the dynamic points, the steady camera could not detect any dynamic point, but the oscillating camera still can detect the dynamic points.

The results showed that the VC method can differentiate between static and dynamic points from knowledge of the relative motion between the camera and the points. However, if there is no relative motion, it will be difficult to identify dynamic features. It is very less probable that the feature will oscillate similar to the camera, and this is an advantage of camera oscillation. For more details about this work, please refer to this paper[5].

References:

1- Joan Solá, “Towards Visual Localization Mapping and Moving Objects Tracking by a Mobile Robot: a Geometric and Probabilistic Approach”, Institut National Polytechnique de Toulouse — INPT, 2007.

2- Joan Solá, Teresa Vidal-Calleja, Javier Civera and José María Martínez Montiel, “Impact of landmark parametrization on monocular EKF-SLAM with points and lines”, International journal of computer vision, vol. 97, no. 3, pp. 339–368, 2012.

3- Civera Javier, G Grasa Oscar, J Davison Andrew and J M M Montiel, “l-point RANSAC for extended Kalman filtering: Application to real-time structure from motion and visual odometry”, Journal of Field Robotics, vol. 27, no. 5, pp. 609–631, 2010.

4- M. Heshmat and M. Abdellatif, “The effect of feature composition on the localization accuracy of visual SLAM systems”, International Conference on Computer Vision Theory and Applications, VISAPP, pp. 419–424, 2012.

5- M. Heshmat, M. Abdellatif, K. Nakamura, A. A. Abouelsoud and N. Babaguchi, “Dynamic feature detection using virtual correction and camera oscillations,” 2014 International Conference on 3D Imaging (IC3D), 2014, pp. 1–8.