Photo-model-based stereo-vision 3D perception for marine creatures catching by ROV

By incorporating visual information obtained from the installed vision sensor into the feedback loop, visual servoing enables the robot to be able to operate in a changing environment or an unknown environment. In the past, the authors have proposed a real-time monocular fish-catching robot. However, it is insufficient for new demands, especially fish-catching in 3D space. To overcome these problems, a binocular stereo-vision system with photo-model-based recognition method has been proposed for picking and placing clothes. Then, this previous research has been the base for extending the stereo-vision pick-and-place system to a three-dimensional (3D) object visual servoing system, which used the genetic algorithm (GA) of the fishing-catching robot. This paper verifies the visual servoing ability of the system for a moving 3D object through visual servoing experiment. In the experiment, marine creature toy floated on the water in the pool without pose constraints. It is confirmed from the experimental results that the proposed system can be used to capture a moving marine creature target and is not susceptible to partial occlusion conditions.