Computer vision enabled building digital twin using building information model

A building digital twin (BDT) can maintain an up-to-date digital model reflecting physical world conditions and has become necessary for building applications. Recent studies on the BDT employed the Internet of Things to sense physical-world conditions. Although cameras are one of the most widely used facilities in buildings, their adoption in the BDT remains unexplored. This study proposes a novel computer-vision (CV)-enabled BDT scheme using building information modeling (BIM) taking camera videos as input, which addresses the dimension, coordinate system, and object inconsistencies between BIM and camera videos. First, the proposed BDT scheme detects objects' locations and rotations jointly using a 2-D object detection network and a 3-D object estimation network. Then, theorem and lemmas are presented to compute the 3-D locations in BCS using detected 2-D locations. Thirdly, both cold-start object matching and run-time object matching schemes are proposed to address the object inconsistency between camera videos and BIM. Finally, experiments were conducted in the real-world environment. The experiment results showed that the proposed BDT scheme maintained average location errors of 0.181 m with distortions preserved and 0.165 m with distortions removed in the manual calibration scenario, 0.166 m with distortions preserved, and 0.195 m with distortions removed in automatic calibration scenario. This finding proved the effectiveness of the proposed BDT scheme. This study is the first to explore a BDT scheme on top of BIM using CV. It is anticipated that this study will inspire more intelligent studies in smart buildings jointly employing both CV and BIM.