Stereo vision-based human–robot interaction for weld seam detection in robotic welding

Robotic welding has been widely used to improve manufacturing efficiency. However, for small batches or one-off jobs, the time and effort required for robotic welding may be economically prohibitive. An intuitive ‘mimicking’-like robotic welding approach can help non-robotics experts instruct the robot to perform the welding operations by letting them demonstrate the welding path via hand gestures without requiring any programming. In this paper, we present a human–robot interaction (HRI)-based weld seam detection for such a robotic welding application. Our approach consists of a vision-based hand detection and tracking method for the user to demonstrate the welding path to the robot’s vision system. We then isolated the welding seam lines and developed a search algorithm to identify the welding paths for the robot. The continuous weld seam path is observed within the bounds of the seam edge in the image and is projected to the robot coordinate space. The seam detection is thoroughly tested with a real-world application on the UR10e robot. The evaluation has revealed an accuracy of 1 pixel in the image plane, equivalent to 1 mm in physical space in our setup.