GeoAI: Machine Learning & Deep Learning with GIS

Welcome to our GeoAI workshop! If you are new to machine learning, deep learning, and/or artificial intelligence, this is the workshop for you. We have prepared a quick overview of the fundamental concepts and principles necessary to undertake these techniques using GIS.

If you are familiar with artificial intelligence, but have never used them for geospatial applications, then we hope this workshop will also be useful to you.

The learning objectives of this workshop are:

• To learn the differences between Artificial Intelligence (AI), Machine Learning (ML), and Deep Learning (DL)
• To understand the value that GeoArtificial Intelligence (GeoAI) brings to the scientific community and its many applications
• To learn the different types of ML and DL capabilities available in ArcGIS
• To explore different ML workflows and get hands-on experience on ML with ArcGIS
• To understand the fundamental principles of DL in GIS and get hands-on experience on DL with ArcGIS
• To increase awareness of AI resources for GIS available to the Clemson community

Artificial Intelligence (AI) refers to the development of computer systems capable of performing tasks that typically require human intelligence. These tasks include learning, reasoning, problem-solving, perception, understanding natural language, and decision-making. Successful examples include improved maps and navigation, facial detection and recognition, text editors (e.g. Grammarly), ChatGPT, and speech detection amongst others.

AI can be categorized into two types:

1. Narrow/Weak AI: This type of AI is designed and trained for a particular task. It operates within a limited context and doesn't possess general intelligence. Examples of narrow AI include virtual personal assistants like Siri and Alexa, recommendation systems in streaming services, image or speech recognition systems, and language models (ChatGPT).
2. General AI: This represents the idea of a machine that possesses the ability to understand, learn, and apply its intelligence to any problem that a human being can. General AI aims to have the capacity to understand, learn, reason, and adapt to its environment, much like a human.

AI includes the subtype of machine learning, and machine learning includes the subtype of deep learning. Each subtype has an increasing level of complexity and system autonomy.

Both machine learning and deep learning analyze information and predict outcomes with limited human involvement. The way that the computer learns to analyze the data is what distinguishes deep learning from general machine learning.

Machine Learning (ML) involves algorithms and models that can learn from and make predictions or decisions based on data. Machine learning algorithms are designed to learn patterns from data and make predictions or decisions without explicit programming.

Deep Learning (DL) is a subset of machine learning that uses neural networks with multiple layers to learn from data. Deep learning algorithms, inspired by the structure of the human brain, consist of artificial neural networks with multiple layers (hence the term "deep"). These networks are capable of learning representations of data with multiple levels of abstraction. Deep learning excels in handling large volumes of data and can automatically discover patterns within it.

The key differences between machine learning and deep learning lie in their approach to learning and the types of algorithms used. Machine learning includes various techniques, including decision trees, support vector machines, and random forests, among others. Deep learning specifically focuses on neural networks with many layers and is exceptionally effective when dealing with complex data such as images, audio, and natural language processing.

Geospatial Artificial Intelligence (GeoAI) refers to the integration of AI technologies with geospatial data and systems (VoPham et al, 2018). Geospatial data involves information that can be linked to specific geographic locations. This could include satellite imagery, GPS data, maps, and other location-based data.

The combination of AI with geospatial data offers numerous possibilities for analyzing and interpreting spatial information, allowing for more sophisticated decision-making and problem-solving in various fields. Some key applications of geospatial AI include:

1. Remote Sensing and Image Analysis: AI algorithms can analyze satellite and aerial imagery to identify patterns, objects, changes in terrain, and more. This has applications in agriculture, urban planning, disaster response, and environmental monitoring.

2. Location-Based Services (LBS): AI-driven location-based services personalize experiences for users by leveraging location data. Examples include navigation apps, location-based advertising, and recommendation systems.

3. Urban Planning and Development: Geospatial AI can aid in urban planning by predicting traffic patterns, identifying suitable locations for infrastructure, and managing resources efficiently.

4. Natural Resource Management: AI can assist in managing and optimizing natural resources by monitoring forestry, water resources, and land use.

5. Disaster Response and Management: Geospatial AI helps in predicting and managing natural disasters by analyzing historical data, weather patterns, and vulnerable areas, improving emergency responses.

6. Environmental Monitoring: It enables the monitoring and analysis of environmental changes, such as deforestation, climate change impacts, and biodiversity conservation.

Machine learning has been a core component of spatial analysis in GIS for decades. These data-driven algorithms and techniques have been used to solve problems in three broad categories: automate prediction, classification, and clustering of data.

Classification techniques determine which category an object should be assigned to based on a training dataset.

Clustering groups observations based on similarities in value or location.

Prediction tools use the known to estimate the unknown.

In this workshop, we will be focusing on using classification tools to automatically extract meaningful information from imagery.

In general, we use image classification to extract information from imagery. Below are some examples typically used in GIS:

Image Classification Types: Pixel Vs. Object

Image classification types are defined by the unit of classification. There are two main types for image classification: pixel-based and object-base.

Pixel-based image classification considers each pixel individually and does not consider any information from neighboring features. Each pixel is assigned a class independent of it's spatial context. This type of classification works well with coarse-spatial resolution raster data where real-world features are difficult to recognize.

Object-based image classification uses a process called segmentation to group neighboring pixels into segments or objects based on similar spectral and spatial characteristics. Unlike pixel-based image classification, classes are assigned to segments, rather than the individual pixels. Object-based classification works well with fine-spatial resolution rasters.

Image Classification Methods: Unsupervised Vs. Supervised

Image classification methods explain the overall workflow or approach to image classification. There are two main methods in ArcGIS:

In this case, the computer automatically determine which classes are present in an image based on the spectral characteristics and statistics of the pixels (or segments).

Pixels (or segments) are assigned to clusters based on their distribution in multivariate data space.

The GIS analyst assigns classes according to the desired classification schema after the unsupervised method is executed.

Unsupervised classification is useful for quickly assigning labels to uncomplicated, broad land cover classes such as water, vegetation/non-vegetation, forested/non-forested. Unsupervised classification may reduce analyst bias.

Supervised classification initially relies on input from an analyst to determine the classes within the imagery.

The analyst creates training samples from pixels (or segments) that represent the target classes in the classification schema. These samples are used to train a classifier to identify all other pixels (or segments) that fall into one of the target classes.

Choosing Your Image Classification Technique

As you can see, image classification is a powerful tool to transform imagery into information. One of the challenges is to decide which image classification technique (method + type) is best suited for the analysis question.

You can apply either classification method to either type to create an image classification technique.

There is not a one-size-fits-all solution for every situation. Each analysis question is affected by many factors, including spatial resolution, what is being classified, and geographical context.

Nevertheless, there are some use cases where one workflow may work better than another. Familiarization and practice will help you decide which technique is best suited for each situation. Let's talk about a few scenarios below:

Scenario 1: As part of the Facilities Management team at Clemson, you have been tasked with measure the impervious surface area within our main campus. You have been given fine-resolution raster data from our drone team at CCGT to ensure that your classification is precise. Which image classification technique is most appropriate for this situation?

Answer: Supervised object-based image classification or supervised pixel-based image classification are most appropriate.

Scenario 2: As part of the South Carolina Wildlife Federation, you have been tasked with determining potential areas for the group to purchase to create habitat for an endangered species: the American Bald Eagle. You know that eagles prefer open grasslands and salt marshes to hunt. You have access to coarse spatial resolution raster data that covers the entire state. Which image classification technique is most appropriate for this situation? Answer: Unsupervised pixel-based image classification is most appropriate.

 Workshop Instructions 

1. In ArcGIS Pro, open the Catalog pane.
2. Go to Portal - Living Atlas. Search for NAIP.
3. Add the USA NAIP Imagery: Color Infrared to your map.
4. In the top ribbon, go to the Imagery tab, and click Raster Functions. In the dropdown menu, select Raster Functions. Raster Functions are a set of tools that work in-memory to perform temporal operations on raster data. This way, you can see what the operation results are before committing to a final dataset.
5. The Raster Functions pane appears. In the search bar at the top, type clip.
6. Follow the clip settings below.

7. Click on Create New Layer at the bottom. The operation creates a temporary clip file. 8. To make this file permanent, right-click on it in the Table of Contents, and select Data, Export Raster.

As mentioned earlier, both machine learning and deep learning analyze information and predict outcomes with limited human involvement. The way that the computer learns to analyze the data is what distinguishes deep learning from general machine learning.

Deep learning teaches the system what to look for from the data to conduct the desired analysis. This task is accomplished using neural networks, which mimic the ways in which a human brain processes information. These neural networks pass information through multiple layers to filter and classify data based on recognizable patterns.

Current deep learning models in ArcGIS use a specific type of neural network called the Convolutional Neural Network (CNN).

In this workshop, we focus on using neural networks to analyze raster images and interpret their content. This is also commonly known as Computer Vision.

In general, all deep learning workflows follow the same overall steps:

1. Generate training samples of features (labeled imagery datasets)
2. Use those training samples to train a deep learning model
3. Using the resulting deep learning model, run the inferencing tools to extract information from other images

There are other deep learning workflows that can be used in GIS apart from the ones used in working with imagery. Below is an overview of the types of tasks that can be done with deep learning in GIS depending on the input data.

To implement deep learning in your workflow, you can use or fine-tune pre-trained deep learning packages or train your own deep learning models. ArcGIS also integrates with third-party deep learning frameworks to extract features from single images, imagery collections, point clouds, or videos.

Below is a typical workflow for computer vision using object detection as the deep learning model:

 Here  is a complete list of deep learning model types, tasks, and associated tools available in ArcGIS Pro.

In this workshop, we focus on tools that extract meaningful information from imagery. In the case of Deep learning, the most common tasks for image interpretation can be categorized into 4 types:

There are other types of deep learning applications for computer vision, such as panoptic segmentation, image translation, and change detection, amongst others.

ArcGIS pretrained deep learning models allow you to accelerate your geospatial workflows by providing robust models that eliminate the need for:

• Creation of high volume of training data
• Massive compute resources
• Extensive deep learning knowledge

Pretrained models allow you to automate the way you extract meaningful insights from imagery, point clouds, and video for very specific scenarios such as land cover classification or rooftop extraction.

Pretrained models are available through the  ArcGIS Living Atlas , where new models get added frequently. For a complete, updated list, go  here .

To understand the benefits of geoAI, check out this  land cover explorer  using Sentinel data:

 Workshop Instructions 

Requirements:

1. ArcGIS Pro Your computer must have ArcGIS Pro installed. You can check ArcGIS Pro system requirements on this  link . Inside this page, there is a free app that can verify for you if your computer has the ability to run ArcGIS Pro. Inside the Scan your computer for compatibility section, click on  Verify your computer's ability to run ArcGIS Pro  and download the file. Run it and confirm all checking points are green.
2. ArcGIS Image Analyst Extension
3. Deep Learning Libraries
4. GPU Verification
5. NVIDIA Drivers Update

Overall requirements to run Deep Learning:  https://www.esri.com/arcgis-blog/products/arcgis-pro/imagery/deep-learning-with-arcgis-pro-tips-tricks/ 

Guide to install deep learning frameworks for ArcGIS:  https://pro.arcgis.com/en/pro-app/latest/help/analysis/deep-learning/install-deep-learning-frameworks.htm 

GeoAI-Related Resources:

•  Shifting to Modern Remote Sensing and Image Analys... - Esri Community 
•  ArcGIS API for Python: Sample Notebooks 

Clemson Research Computer and Data Services Workshops:

•  Deep Learning and Object Detection in Python 
•  Deep Learning in Pytorch for Beginners