Automatic feature-based markerless calibration and navigation method for augmented reality assisted dental treatment

Augmented reality (AR) is gaining traction in the field of computer-assisted treatment (CAT). Head-mounted display (HMD)-based AR in CAT provides dentists with enhanced visualisation by directly overlaying a three-dimensional (3D) model on a real patient during dental treatment. However, conventional AR-based treatments rely on optical markers and trackers, which makes them tedious, expensive, and uncomfortable for dentists. Therefore, a markerless image-to-patient tracking system is necessary to overcome these challenges and enhance system efficiency. This paper proposes a novel feature-based markerless calibration and navigation method for an HMD-based AR visualisation system. The authors address three sub-challenges: firstly, synthetic RGB-D data for anatomical landmark detection is generated to train a deep convolutional neural network (DCNN); secondly, the HMD is automatically calibrated using detected anatomical landmarks, eliminating the need for user input or optical trackers; and thirdly, a multi-iterative closest point (ICP) algorithm is developed for effective 3D-3D real-time navigation. The authors conduct several experiments on a commercially available HMD (HoloLens 2). Finally, the authors compare and evaluate the approach against state-of-the-art methods that employ HoloLens. The proposed method achieves a calibration virtual-to-real re-projection distance of (1.09 ± 0.23) mm and navigation projection errors and accuracies of approximately (0.53 ± 0.19) mm and 93.87%, respectively.