Extending Segment Anything Model (SAM) for GeoAI

Extending SAM for extracting features of interest from geospatial imagery

Agricultural Land Parcels

Airplanes

Cargo Ships

Center-Pivot Farms

Yachts

Agricultural Land Parcels

Everything

Combining SAM with pretrained ArcGIS Object Detection models

SAM can be prompted using bounding boxes to produce segmentation masks. We have combined several pretrained ArcGIS object detection models with SAM, so they can extract the polygonal boundary of features instead of bounding boxes.

Turning Object Detection models into Object Segmentation models with SAM

Tree Segmentation

Combines Tree Detection model with SAM.

Swimming Pool Segmentation Combines Swimming Pool Detection model with SAM.

Text Prompt support for SAM (Grounded SAM)

The Grounded SAM model combines SAM with Grounding DINO, an open set object detection model, so it can be prompted using free form text prompts to extract features of various kinds.

Prompting SAM with Text Prompts to extract features

Here are a few examples of features extracted using text prompts:

Text Prompt: Airplanes

Text Prompt: Ships

Text Prompt: Cars

Text Prompt: Cars

Text Prompt: Yellow Cars

Text Prompt: Clouds

Text Prompt: Windows

Fine tuning SAM for Geospatial AI tasks

The SAMLoRA model in arcgis.learn applies the low-rank-adaptation (LoRA) technique to the SAM model and fine-tunes it for geospatial imagery segmentation.

Segmenting Geospatial imagery using low rank adaptation of SAM model

Building footprint extraction using fine-tuned SAMLoRA model

Segmenting Geospatial imagery using low rank adaptation of SAM model