Intentions prediction for human–robot collaboration in utility tunnel maintenance

Purpose – This paper introduces a novel hybrid deep learning model aimed at enhancing human intention prediction in human–robot collaboration for utility tunnel maintenance. Recognizing the inherent dangers and the confined nature of utility tunnels, the study aims to advance safety and operational efficiency by improving robot assistants’ ability to accurately interpret and predict human-worker intentions. The purpose is to reduce human exposure to hazardous environments and optimize task execution through precise and timely robot actions. Design/methodology/approach – Our approach involves a hybrid deep learning architecture combining traditional time-series image classification with advanced semantic information extraction. The proposed hybrid intentions prediction deep learning model (HIPM) utilizes convolutional neural networks, long short-term memory networks and the contrastive language–image pre-training model for a comprehensive feature extraction and intention prediction. This integration enables the model to process visual and contextual data from dynamic and challenging tunnel environments, addressing the inadequacies of traditional vision-based and physical-based intention prediction methods. Findings – Empirical validation conducted on real-world utility tunnel data from Jiangsu Province, China, demonstrated that HIPM significantly outperforms traditional models. HIPM achieved a precision of 91.52% and a recall of 91.20%, indicating a high level of accuracy in predicting human intentions. The results underscore the model’s robustness and reliability, confirming its effectiveness in understanding and responding to complex human-worker behaviors in utility tunnel settings. Originality/value – The originality of this research lies in its novel integration of multimodal deep learning techniques to enhance the interpretability and adaptability of robots in human-robot collaborative environments. The HIPM model’s ability to interpret both human actions and environmental contexts presents a significant advancement over existing models, offering a more reliable and efficient approach to managing the safety and efficiency of utility tunnel maintenance operations. This study contributes a pioneering solution to the challenges of human intention prediction in one of the most hazardous and demanding industrial settings.