Skip to main content

Applying Deep Learning Models to BTAA GeoPortal Collections

By Mech Frazier, Geographic Information Systems (GIS) Specialist, Northwestern University Libraries

 

The boom of Artificial Intelligence (AI) in recent years has made a significant impact on Geographic Information Systems (GIS), including applications of Deep Learning (DL). Applications of DL have revolutionized GIS tasks, such as detecting sensitive mangrove environments1, identifying global building footprints2, and even locating beaver complexes in complex forest landscapes3. DL, a subset of Machine Learning (ML), uses neural networks to detect objects, classify pixels, and perform segmentation4. 

Recent advancements in DL have led to significant improvements in both model quality and usability, with prebuilt models and tools that simplify workflows. GIS applications like ArcGIS Pro and QGIS have made available tools and plug-ins to streamline the process of DL. 

This blog post will focus on one aspect of DL: object detection using aerial imagery. We’ll explore how to use predefined models in ArcGIS Pro provided by ESRI. The aim is to introduce you to how the BTAA Geoportal collection can support your DL workflows and serve as a resource for geospatial objects.

For this tutorial, we’ll be using ArcGIS Pro version 3.4 (ESRI, Inc). We’ll also need to grab the web services link of our aerial imagery of Carver County in 2023 from the BTAA Geoportal (Figure 1). Navigate to the item record, select “Web Services” and copy the link.

Start a new project in ArcGIS Pro. From the "Add Data" button's drop-down menu, choose "From Path" and paste the copied web services link. Once the layer is added, right-click it in the Contents panel and select "Zoom to Layer." You should now see something like this:

 

Figure 1. Aerial imagery of Carver County, Minnesota.

 

Check out the (amazing!) detail in this image. Zooming in and out, explore where you can spot cropland, roads, houses, lakes, and other natural and artificial features. Opening the “Properties” description tells us this image is projected into US survey feet and has a cell size of 0.5 – meaning each cell of the dataset represents a 0.5 by 0.5 square feet on the ground. This is considered high resolution imagery, which is often difficult to find for free. Save your project.

Now that the imagery is set up, we need to download the Deep Learning Libraries Installer for ArcGIS Pro version 3.4 (make sure to match the installer version with your version of ArcGIS Pro). Once downloaded, run the .msi file to begin the installation.

Next, we need to clone the Python environment in ArcGIS Pro. To do this, go to the “Project” tab and select “Package Manager”. Since we can’t modify the original environment, click the gear icon and choose “Clone” to create a new environment (Figure 2). 

Figure 2. Cloning the Python environment in the Environment Manager is necessary before installing the Deep Learning essentials package.

If needed, activate the cloned environment. Once this is done, navigate to “Add Packages” and search for deep-learning-essentials (Figure 3). Select and install.

Figure 3. After cloning the environment and running the Deep Learning Libraries installer, the next step is to install the deep-learning-essentials package before proceeding with the analysis.

We’re now ready to select our area of interest. This tutorial will explore a DL model used to extract building footprints. Download the .dplk file for the building footprint extraction from the ArcGIS Living Atlas database. Take a moment to look at the metadata on the model. A couple things here are important to consider before running our model: input requirements, applicable geographies, model accuracy, and model architecture. While I’ve vetted this beforehand, it’s crucial to verify your imagery satisfies the requirements listed for the model you’re using. For our next step, zoom to a small area of houses. Open the Tools box under the Analysis tab and search for “Detect Objects Using Deep Learning”.

Important note: It is recommended to have a NVIDIA GPU with CUDA compute capability. While you can run this using a CPU, it is not recommended. Please consult with the system requirements documentation provided by ESRI.

 For the input raster, select the aerial imagery from the drop-down menu. You’ll see a red X appear next to the box. Navigate to the Environments tab and under “Extent” choose the first icon that looks like a small, square world. You may need to adjust the Zoom level if you continue to see the red X (it means the web layer extent is too large to process as is). For Model Definition, navigate to the building footprint .dplk file. Once added, a model parameter box appears. For this example, we’ll leave everything as the default, but typically we adjust these parameters when using the tool depending on our desired outcome. Run the tool.

Figure 4. The input and output of residential units. Residential units, outlined in red, detected by the model using a 95% threshold.

 The model was able to detect buildings with incredible accuracy (Figure 4)! Applying DL models is approachable in ArcGIS Pro where installers and tools are already built out for you. Try this out on other parts of Carver County, or try another image from our collection. For fun, I tested another part of Carver County, this time using a model to detect cars and ended up with great results after adjusting the confidence threshold to 0.5 (Figure 5).

Figure 5. A parking lot with various types of cars in eastern Carver County, Minnesota. Vehicles, outlined in red, after running the detect objects tool.

The BTAA Geoportal is a useful data resource for your next research project. Check out the database for your next project. This tutorial was tested by researchers at Northwestern University, who are also available to discuss Deep Learning approaches in more detail. Happy modeling!

 

Citations

  1. Microsoft. (n.d.). Microsoft/GlobalMLBuildingFootprints: Worldwide Building footprints derived from satellite imagery. GitHub. https://github.com/microsoft/GlobalMLBuildingFootprints 
  2. Ivanovic, A., Misra, A., & Dong, D. (2022, June 21). Identify mangrove forests using satellite image features using Amazon SageMaker studio and Amazon Sagemaker Autopilot – part 1 | AWS machine learning blog. Artificial Intelligence and Machine Learning. https://aws.amazon.com/blogs/machine-learning/part-1-identify-mangrove-forests-using-satellite-image-features-using-amazon-sagemaker-studio-and-amazon-sagemaker-autopilot/
  3. Fairfax, E., Zhu, E., Clinton, N., Maiman, S., Shaikh, A., Macfarlane, W. W., Wheaton, J. M., Ackerstein, D., & Corwin, E. (2023). EEAGER: A Neural Network Model for Finding Beaver Complexes in Satellite and Aerial Imagery. Journal of Geophysical Research. Biogeosciences, 128(6). https://doi.org/10.1029/2022JG007196
  4. ESRI, Inc. (n.d.). Introduction to deep learning. Introduction to deep learning-ArcGIS Pro | Documentation. https://pro.arcgis.com/en/pro-app/latest/help/analysis/deep-learning/what-is-deep-learning-.htm 

 

 

 

 

Labels: