Autonomous weld joint detection and localisation using computer vision in robotic arc welding

Abstract

At present, robotic welding is a rigid process which requires the robot to be taught by a human operator using teach and playback methods. A significant challenge for robotic welding to be widely adopted is the time taken to program the robot path for new parts. For example, in low to medium volume manufacturing and repair work, robotic welding is not always a viable option, as it can often be quicker and cheaper to weld the parts manually. Some flexibility has been achieved in industry through offline programming software and vision systems that can be used to recognise a pre-taught part and shift the program to accommodate for any positional offsets. However these systems still require a human operator to program the robot path. To achieve true flexibility and make robotic welding a viable option across the wider manufacturing industry, systems should be able to automatically identify the weld joints and then locate its position in the robot's workspace. Computer vision can be used to achieve this kind of autonomy. However the welding environment presents unique challenges for computer vision. These challenges include poor contrast, reflections from metallic surfaces, and imperfections on the work piece such as rust, mill scale and scratches which are not consistent from part to part. The system should also be adaptable to a variety of surface finishes and base material such as paint, steel and aluminium. This thesis develops an autonomous robotic arc welding system that is capable of detecting realistic weld joints and calculating their position in the robot workspace with minimal human interaction. This is accomplished using a pair of calibrated robot mounted stereo cameras combined with image processing algorithms. An automated calibration method for the robot and stereo vision system and image processing algorithms are developed in this thesis. To obtain accurate positional information from the vision system, an automatic camera calibration method is integrated with simultaneous calibration of the welding robot and robot mounted stereo vision system. This provides accurate geometrical transformations between the robot, cameras and the robot workspace which form the foundation of the computer vision algorithms for both the weld joint detection and localisation. The calibration algorithm is designed for economical and practical implementation. The automatic calibration not only allows for a faster initial calibration, but also reduces the machine down-time for any subsequent calibrations after an accidental collision or if the camera fixture is relocated. Unlike existing methods which require expensive 3D co-ordinate measuring devices or laser scanners, the optimised calibration method developed in this thesis is capable of achieving the sub-millimetre accuracy required for robotic arc welding using only the robot mounted stereo cameras, a mechanical pointer and a calibration board. This makes the proposed calibration method very practical, economical and easy to implement, hence making it highly desirable for industrial applications. This thesis introduces new methods for autonomous identification for both butt and fillet weld joints regardless of the base material, surface finish and surface imperfections. The detection methods analyse the image from a global perspective and are able to identify the weld joint without prior knowledge of the shape of location of the weld joint in the image. The method to detect both butt and fillet joints introduces an approach for the detection of weld joints in realistic work pieces using a novel adaptive line growing technique developed in this thesis. Image matching and triangulation of the weld joint is introduced using 2D homography for planar butt welds and an epipolar geometry based method for fillet welds. In the welding environment, traditional image matching is not reliable, as the images of weld joints typically contain similar shades of grey and are featureless and textureless. New image matching methods for weld joint matching are developed to provide reliable and accurate matching which is invariant to the environmental conditions of the welding environment. The proposed algorithms are validated through experiments using an industrial welding robot in a realistic workshop environment. The results show that the methods introduced in this thesis can provide robust and accurate identification and localisation of weld joints which can be implemented in industry.

Date of Award	2013
Original language	English