AROutcrop
An Augmented Reality Mobile Application for Teaching Geology in the Field
PROJECT OBJECTIVE/GOAL: POKEMON GO MEETS GOOGLE EARTH PRO
Figure 1. Illustrations of how AR apps like Pokemon Go (A) and visualization tools like Google Earth Pro (B) integrate different types of media into the user experience.
Figure 2. App features are accessed via standard Android menu system and keyboard. An About this App (A) describes the app, a Quickstart guide (B) provides instructions on using the app, and a Preferences page (C) controls settings regarding GPS information and render distance for AR objects.
AUGMENTING REALITY IN THE FIELD: DEPAUW NATURE PARK QUARRY, GREENCASTLE, IN
DePauw Nature Park Quarry with augmented reality location labeled.
APP DEVELOPMENT: GEOTECHNICAL CHALLENGES
1. GPS ACCURACY INSUFFICIENT FOR USING PLACEMARKS : Mobile GPS accuracy (typically ~10 m or more) prevents using “placemark”-style hotspots for informational windows (or for downloading 3-D models of rock samples from outcrop-specific placemarks). SOLUTION: Implemented auto-recognition of geologic features to download/visualize AR overlays, 3-D photogrammetry models, and informational windows.
2. IMAGE RECOGNITION UNDER DIFFERENT CONDITIONS: Outcrop auto-recognition affected by changes in lighting/shadows, seasonal variations in vegetation, weather conditions, position relative to original photo, etc. SOLUTION: Interfaced with mobile device’s camera and photo library to 1) define target image dynamically in real time or 2) place AR objects manually.
3. ACCESSING PHOTO LIBRARY AFTER AR SCENE INITIATED : ARCore image library not originally designed to interactively update after app starts. SOLUTION: Developed technique for accessing the photo library after the AR scene is initiated.
4. 2-D OVERLAY PLACEMENT WITHOUT WELL-DEFINED PLANES: ARCore functionality works best in areas with well-defined planes (e.g., walls, floors, etc.) that outcrops with irregular surfaces/textures commonly lack. SOLUTION: Rewrote ARCore functionality to place images relative to camera view (significantly reducing response time by skipping the need to identify planar surfaces).
5. GEOREFERENCING 2-D OVERLAY: Auto-scaling AR images requires specific control points for both target and overlay images. SOLUTION: Developed manual georeferencing tools to scale, translate (without a plane), rotate, and change the transparancy of the overlay image.
6. POSTIONING 3-D PHOTOGRAMMETRY MODELS: AROutcrop places 3-D models in the camera scene, but 1) scales the objects relative to planes in the field of view and distances from them, and 2) orients the objects relative to the AR origin and not the user’s camera. SOLUTION: Created manual georeferencing tools to scale, translate (without a plane), and rotate 3-D photogrammetry models.
Auto-Recognizing Outcrops
Figure 3. Students specify target outcrop image and 2-D image overlay from the device’s photo library. Students can use the camera & photo library to overcome shadows, weather conditions, and slight differences in position.
Placing/Georeferencing 2-D Interpretative Image Overlays
Figure 4. 2-D interpretive overlays highlight critical geologic features (e.g., rock layers, cross-beds, faults, folds, etc.) directly in the camera view. Students can georeference overlays by scaling, translating, and rotating. Rotation can skew the overlay to account for perspective differences.
Placing 3-D Photogrammetry Models
Figure 5. Students specify target outcrop image and a 3-D model from the app library. 3-D photogrammetry models enable students to view and interact with outcrops & rock samples digitally, without needing to collect physical samples from inaccessible locations (allowing students to obtain more detailed information about rock lithology, fossils, small-scale features, etc.).
Viewing Informational Windows
Figure 6. Double-tapping any type of AR media (2-D or 3-D) pops up a formatted informational window containing GPS location, elevation, labeled photos/diagrams, textural descriptions, etc. to augment user comprehension of the outcrop. Students can readily examine 3-D rock samples/outcrops with detailed contextural information related to the outcrop being studied, effectively walking around the sample and reading about its geology.
AUGMENTING REALITY IN THE FIELD:VAN HISE ROCK, BARABOO, WI AREA
Van Hise Rock near Baraboo, WI with augmented reality location labeled.
Figure 7. Van Hise Rock near Baraboo, WI showing near-vertical beds of quartzite and phyllite with refracted cleavage and cross-beds (above). Schematic diagram depicting overlay process, overlay georeferenced by a student, and 3-D photogrammetry model of Van Hise Rock shown below.
AUGMENTING REALITY IN THE CLASSROOM: TEXTBOOK IMAGE RECOGNITION
Figure 8. AROutcrop can associate AR media with any image, including textbook figures (Fossen, 2016). Because planes & dimensions of textbook images can be easily defined, AROutcrop can auto-scale objects such that AR media literally “jump off the page” . Dots indicate a planar surface.
AROutcrop: FUTURE DIRECTIONS
Figure 9. AROutcrop’s automatic image recognition functionality works with any type of image. Here, the iconic East College on DePauw University’s campus serves as the target image to place a 3-D photogrammetry model of the DePauw Tiger sculpture, providing an engaging tool for prospective students and their families who participate on Admissions campus tours.
1. Create an iOS version of AROutcrop.
2. Develop an AROutcrop database backend to a) store relationships between target image and AR objects, textural data, & numeric information (e.g., GPS coordinates),b) save and retrieve 3-D model/overlay data, andc) enable crowdsourcing for new 3-D models/overlays/etc.
3. Use GPS coordinates to filter AROutcrop database by location.
4. Populate informational windows with HTML- and CSS-based window layouts from AROutcrop database.
5. Customize AROutcrop for other uses (e.g., Admissions campus tours; Figure 9).
