Sketch-to-3D: Transforming Hand-Sketched Floorplans into 3D Layouts

The conventional approach for analyzing built environments' spatial relations and structural components utilizes 2D floorplans as the primary representation method. However, these 2D blueprints can be challenging and potentially misleading for end-users who lack expertise in architecture or design, as they rely on low-dimensional abstractions to convey complex, high-dimensional information. This paper introduces an automated pipeline that transforms hand-sketched 2D floorplans into immersive 3D layouts, addressing this issue through a supervised learning approach. Initially, a dataset of diverse hand-sketched floorplans is compiled, each labeled with semantics for the object and structural element. Next, supervised deep learning models were developed for layout and object detection using this dataset. The performance of the supervised models was evaluated on a test dataset. The 2D digital designs were then converted to 3D immersive layouts through automated mapping. The entire pipeline is an interactive application that allows novice users to capture a picture of their hand-sketched floorplan and view their design ideas come to life as immersive 3D layouts populated with virtual crowds.