Damith Tennakoon, Aman U. Usmani, Muhammad Usman, Apostolos Vasileiou, Sasha Latchaev, Melanie Baljko, Usman T. Khan, Matthew A. Perras, Mojgan Jadidi

Research output: Contribution to journalConference articlepeer-review


In the methods of traditional education, students have had to learn complex concepts within the subjects of earth-science and engineering beginning with paper-based topographic maps, a pencil and a ruler. Elaborative observations, such as understanding of the landscape elevations using a contour map, had to be performed by reading accurate contour intervals for heights on a given map. Such paper-based concepts delivered by instructors are perplexing and hard for students to grasp, despite that they play a fundamental role towards earth-science and engineering education. However, using advanced technology and enhanced visualization techniques, instructors can deliver such complicated concepts in a more step-by-step interactive fashion, which is more straight forward and easier to understand by the students. Our team is working on changing this traditional norm of education into a more visual-based experiential learning process. By using depth sensors and a physical sandbox structure, we have developed a mechanism to capture self-created landscapes in a sandbox environment and recreate respective Digital Terrain Model (DTM). The Augmented Reality (AR) Sandbox by UC Davis is a 3D visualization application where individuals can simply mould the sand into mountains and valleys, and a real-time topographic map is projected onto the sand to bring the landscape to life. However, this technology has its own shortcomings in terms of requiring physical access to the AR Sandbox itself, and furthermore absence of performing complex geological, hydrological, and geotechnical applications on top of existing software. We are extending this immersive technology by developing a virtual sandbox (called XR Sandbox) linked with the AR Sandbox. The extended process extracts a 3D point cloud of the self-designed terrain (landscape) in the AR Sandbox and converts this point cloud into a colour-mapped point cloud based on spectral elevation. Furthermore, using a 3D modelling framework in the Python environment, the colour-mapped point cloud is converted into a DTM (a 3D mesh file). To furnish the application of our extensive work, we have developed a prototype virtual sandbox, a web-based application, that imports these DTM's for further digital analysis. The virtual sandbox application is an interactive computer program developed using the Unity 3D game engine, geared towards teaching complex earth-science concepts. The current prototype allows generation of automatic contour maps, overlayed on the terrain model and it is integrated into a learning exercise for students in Civil Engineering to understand the orientation of a planar structure using the 3-Point Problem approach. This exercise has been traditionally delivered using pen and paper, aided with physical 3D models and now it is possible to visualize the problem with a few clicks. The latest release of the Virtual Sandbox application was implemented into a 2000-level Geological process course in the Civil Engineering Program where students were tasked with performing an assessment to demonstrate their 3-dimensional visualization ability. A Google Forms survey, aimed towards learning about the individual user experiences using the Virtual Sandbox application, was distributed the students that took part in this lab activity. Our project also intended towards Virtual Reality (VR) development, therefore, provisioning towards a complete Mixed Reality (XR) experience in Earth system engineering education. The future of such a platform enables enhanced learning experiences for students and innovative solutions for instructors. Here users can import various self-created 3D terrain models, visualize, explore landscapes using first-person perspective, seek extensive new features, and potentials beyond traditional education.

Original languageEnglish
JournalASEE Annual Conference and Exposition, Conference Proceedings
StatePublished - 23 Aug 2022

Bibliographical note

Publisher Copyright:
© American Society for Engineering Education, 2022.


  • Augmented Reality
  • Earth systems
  • Engineering Education
  • Experiential Education
  • Virtual Reality

ASJC Scopus subject areas

  • General Engineering


Dive into the research topics of 'TEACHING EARTH SYSTEMS BEYOND THE CLASSROOM: DEVELOPING A MIXED REALITY (XR) SANDBOX'. Together they form a unique fingerprint.

Cite this