A mockup (steel, imitating a shrinked ship part with real sized features) is shown during real-time tracking with a moving camera after automatic initialization.
The mockup was created and delivered by Odense Steel Shipyard Ltd. (OSS), Odense/Denmark, including a CAD model. The convex, planar polygons of the CAD data were triangulated and converted to IITB's internal data format, which is used by the real-time tracking system VISDOM.
The initial pose is determined by a constrained hierarchical search in six dimensions of the scene space using a sophisticated strategy for constraining the search space and evaluating a few thousand different poses. A Kalman filter is used for updating the pose relative to the camera while the camera moves.
The initial localization as well as the subsequent Kalman filtering during the tracking phase assign image edge elements to projected model segments and minimize a mahalanobis distance between both of them.
In the video example (MPEG, 9.21 MB) each evaluated image half-frame is shown. The camera is rigidly mounted on the hand of a 6-joint robot. During the presented video sequence, the hand moves about 0.7 m towards the mockup and back again. The monocular tracking is performed at a frame rate of 25 Hz (half frames, 584 x 264). In this experiment the cycle time of about 40 ms is limited to the duration of the image acquisition. The mockup is partially occluded by other objects during the tracking. Two other examples demonstrate the possibility to deal with circular trajectories (MPEG, 8.08 MB), occlusions and changing lighting conditions (MPEG, 14.71 MB).
The yellow lines represent the model of the mockup (after hidden line removal) projected into the image plane. The blue pixels are edge elements extracted in real-time by the special purpose hardware MiniVista. They turn to red, provided they are successfully matched to model edge segments.
This work is done in cooperation between the Bereich Kognitive Systeme, Fraunhofer-Institut für Informations- und Datenverarbeitung (IITB) and the Institut für Algorithmen und Kognitive Systeme (IAKS), Universität Karlsruhe (TH) at Karlsruhe, Germany.