QGIS Plugin for GeoAI¶

A QGIS plugin that brings the geoai models into dockable panels (Moondream VLM, segmentation training/inference, SamGeo, DeepForest) so you can keep QGIS as your main workspace while experimenting with GeoAI.

Quick Start¶

Create a Pixi project and install the dependencies.
Install the QGIS plugin from the QGIS Plugin Manager.
Enable the GeoAI plugin in QGIS.
Restart QGIS.
Open a GeoAI toolbar panel and try the sample datasets below.

Video Tutorials¶

Installation Tutorial¶

You can follow this video tutorial to install the GeoAI QGIS Plugin on Linux/Windows:

Usage Tutorial¶

Check out this short video demo and full video tutorial on how to use the GeoAI plugin in QGIS.

Requirements¶

QGIS 3.28 or later
Python 3.10+ (Pixi recommended)
PyTorch (CUDA if you want GPU acceleration)
geoai, samgeo, and deepforest packages

Features¶

Each tool lives inside a dockable panel that can be attached to either side of the QGIS interface, so you can keep layers, maps, and models visible simultaneously.

Moondream Vision-Language Model Panel¶

Caption: Generate descriptions of geospatial imagery (short, normal, or long)
Query: Ask questions about images using natural language
Detect: Detect and locate objects with bounding boxes
Point: Locate specific objects with point markers

Segmentation Panel (Combined Training & Inference)¶

Tab 1 - Create Training Data: Export GeoTIFF tiles from raster and vector data
Tab 2 - Train Models: Train custom segmentation models (U-Net, DeepLabV3+, FPN, etc.)
Tab 3 - Run Inference: Apply trained models to new imagery and vectorize results. Vector outputs can optionally be smoothed or simplified for immediate use in GIS workflows.

SamGeo Panel (Segment Anything Model)¶

Model Tab: Load SAM models (SAM1, SAM2, or SAM3) with configurable backend and device settings
Text Tab: Segment objects using text prompts (e.g., "tree", "building", "road")
Interactive Tab: Segment using point prompts (foreground/background) or box prompts drawn on the map
Batch Tab: Process multiple points interactively or from vector files/layers
Output Tab: Save results as raster (GeoTIFF) or vector (GeoPackage, Shapefile) with optional regularization (orthogonalize polygons, filter by minimum area)

DeepForest Panel (Object Detection)¶

Model Tab: Load pretrained DeepForest models from Hugging Face for tree crown, bird, livestock, nest, and dead tree detection
Predict Tab: Run predictions on single images or large geospatial tiles with configurable patch size, overlap, IoU threshold, and score filtering
Output Tab: Save detection results as georeferenced vector (GeoPackage, Shapefile, GeoJSON) or raster (GeoTIFF), and export training data in COCO or YOLO format

GPU Memory Management¶

Clear GPU Memory: Release GPU memory and clear CUDA cache for all loaded models

Installation¶

1. Set up the environment¶

Installing the GeoAI QGIS plugin on can be challenging due to the complicated pytorch/cuda dependencies. Conda or mamba might take a while to resolve the dependencies, while pip might fail to install the dependencies properly. It is recommended to use pixi to install the dependencies to avoid these issues.

1) Install Pixi¶

Linux/macOS (bash/zsh)¶

curl -fsSL https://pixi.sh/install.sh | sh

Close and re-open your terminal (or reload your shell) so pixi is on your PATH. Then confirm:

1	`pixi --version`

Windows (PowerShell)¶

Open PowerShell (preferably as a normal user, Admin not required), then run:

powershell -ExecutionPolicy Bypass -c "irm -useb https://pixi.sh/install.ps1 | iex"

Close and re-open PowerShell, then confirm:

pixi --version

2) Create a Pixi project¶

Navigate to a directory where you want to create the project and run:

pixi init geo
cd geo

3) Configure `pixi.toml`¶

Open pixi.toml in the geo directory and replace its contents with the following depending on your system.

If you have a NVIDIA GPU with CUDA, run nvidia-smi to check the CUDA version.

For GPU with CUDA 12.x:

[workspace]
channels = ["https://prefix.dev/conda-forge"]
name = "geo"
platforms = ["linux-64", "win-64"]

[system-requirements]
cuda = "12.0"

[dependencies]
python = "3.12.*"
pytorch-gpu = ">=2.7.1,<3"
qgis = "3.42.*"
pyqt = "5.15.*"
geoai = ">=0.24.0"
segment-geospatial = ">=1.2.0"
sam3 = ">=0.1.0.20251211"
deepforest = ">=2.0.0"
libopenblas = ">=0.3.30"

For GPU with CUDA 13.x:

[workspace]
channels = ["https://prefix.dev/conda-forge"]
name = "geo"
platforms = ["linux-64", "win-64"]

[system-requirements]
cuda = "13.0"

[dependencies]
python = "3.12.*"
pytorch-gpu = ">=2.7.1,<3"
qgis = "3.42.*"
pyqt = "5.15.*"
geoai = ">=0.24.0"
segment-geospatial = ">=1.2.0"
sam3 = ">=0.1.0.20251211"
deepforest = ">=2.0.0"

For CPU:

[workspace]
channels = ["https://prefix.dev/conda-forge"]
name = "geo"
platforms = ["linux-64", "win-64"]

[dependencies]
python = "3.12.*"
pytorch-cpu = ">=2.7.1,<3"
qgis = "3.42.*"
pyqt = "5.15.*"
geoai = ">=0.24.0"
segment-geospatial = ">=1.2.0"
sam3 = ">=0.1.0.20251211"
deepforest = ">=2.0.0"
libopenblas = ">=0.3.30"

4) Install the environment¶

From the geo folder:

1	`pixi install`

This step may take several minutes on first install depending on your internet connection and system.

5) Verify PyTorch + CUDA¶

If you have a NVIDIA GPU with CUDA, run the following command to verify the PyTorch + CUDA installation:

pixi run python -c "import torch; print('PyTorch:', torch.__version__); print('CUDA available:', torch.cuda.is_available()); print('GPU:', (torch.cuda.get_device_name(0) if torch.cuda.is_available() else 'None'))"

Expected output should be like this:

PyTorch: 2.9.1
CUDA available: True
GPU: NVIDIA RTX 4090

If CUDA is False, check:

nvidia-smi works in PowerShell
NVIDIA driver is up to date

Request access to SAM 3¶

To use SAM 3, you will need to request access by filling out this form on Hugging Face at https://huggingface.co/facebook/sam3. Once your request has been approved, run the following command in the terminal to authenticate:

pixi run hf auth login

After authentication, you can download the SAM 3 model from Hugging Face:

pixi run hf download facebook/sam3

Important Note: SAM 3 currently requires a NVIDIA GPU with CUDA support. You won't be able to use SAM 3 if you have a CPU only system (source). You will get an error message like this: Failed to load model: Torch not compiled with CUDA enabled.

2. Install the QGIS plugin¶

Option A — use QGIS Plugin Manager (recommended):

GeoAI is available as a QGIS plugin in the official QGIS plugin repository. To install:

Launch QGIS: pixi run qgis
Go to Plugins → Manage and Install Plugins...
Switch to the All tab, search for GeoAI, select it, and click Install Plugin

If you encounter an error message like this after installing the plugin, click Close to dismiss the dialog. Next, in the QGIS Plugin Manager on the Installed tab, toggle the checkbox next to the GeoAI plugin to enable it. If the error dialog appears again, close it once more, then restart QGIS to reload the plugin. After restarting, the GeoAI plugin should appear in the QGIS toolbar.

Option B — use the helper script:

git clone https://github.com/opengeos/geoai.git
cd geoai/qgis_plugin
python install.py

This links/copies the plugin into your active QGIS profile. Re-run after pulling updates. Remove with:

1	`python install.py --remove`

Option C — manual copy:

Copy the qgis_plugin folder to your QGIS plugins directory:
- Linux: ~/.local/share/QGIS/QGIS3/profiles/default/python/plugins/
- Windows: C:\Users\<username>\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\
- macOS: ~/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/

3. Enable in QGIS¶

Launch QGIS: pixi run qgis

QGIS → Plugins → Manage and Install Plugins... → enable GeoAI. After updates, toggle the plugin off/on or restart QGIS to reload.

Usage¶

Moondream Vision-Language Model¶

Sample dataset: parking_lot.tif

Steps:

Click the Moondream button in the GeoAI toolbar (or GeoAI menu → Moondream VLM)
Load a Moondream model (default: vikhyatk/moondream2)
Select a raster layer or browse for an image file
Choose a mode:
- Caption: Generate a description of the image
- Query: Ask a question about the image
- Detect: Detect objects by type (e.g., "building", "car")
- Point: Locate specific objects
Click "Run"
Results are displayed and optionally added to the map. You can drag the panel to any side of QGIS to keep it out of the way while browsing results. Save the output table or vector layer if you want to reuse detections later.

Segmentation Panel (Create Data, Train, Inference)¶

Sample datasets:

Steps:

Download the sample datasets (links above) or prepare your own imagery/vector labels. Store them in a folder that is accessible to pixi project.
Click the Segmentation button in the GeoAI toolbar (or GeoAI menu → Segmentation)
Use the tabs at the top of the panel to switch between:
- Create Training Data: Select input raster and vector labels, configure tile size and stride, and export tiles to a directory.
- Train Model: Select the images and labels directories, choose model architecture (U-Net, DeepLabV3+, etc.), configure training parameters, and start training.
- Run Inference: Select input raster layer or file, specify the trained model path, configure inference parameters, run inference, and optionally vectorize the results.

SamGeo Panel (Segment Anything Model)¶

Sample dataset:

Steps:

Click the SamGeo button in the GeoAI toolbar (or GeoAI menu → SamGeo)
In the Model tab:
- Select the SAM model version (SamGeo3/SAM3, SamGeo2/SAM2, or SamGeo/SAM1)
- Configure backend (meta or transformers) and device (auto, cuda, cpu)
- Click "Load Model" to initialize the model
- Select a raster layer or browse for an image file and click "Set Image"
Choose a segmentation method:
- Text Tab: Enter text prompts describing objects to segment (e.g., "tree, building")
- Interactive Tab:
  - Click "Add Foreground Points" or "Add Background Points" and click on the map
  - Or click "Draw Box" and drag a rectangle on the map
  - Click "Segment by Points" or "Segment by Box"
- Batch Tab: Add multiple points interactively or load from a vector file/layer
In the Output tab:
- Select output format (Raster GeoTIFF, Vector GeoPackage, or Vector Shapefile)
- For vector output, optionally enable regularization:
  - Check "Regularize polygons (orthogonalize)"
  - Set Epsilon (simplification tolerance) and Min Area (filter small polygons)
- Click "Save Masks" to export results

DeepForest Panel (Object Detection)¶

The DeepForest panel provides object detection in remote sensing imagery using pretrained deep learning models from the DeepForest library.

Supported Pretrained Models¶

Model	Hugging Face ID	Description
Tree Crown Detection	`weecology/deepforest-tree`	Detect individual tree crowns in RGB imagery
Bird Detection	`weecology/deepforest-bird`	Detect birds in aerial/satellite imagery
Livestock Detection	`weecology/deepforest-livestock`	Detect livestock in aerial imagery
Nest Detection	`weecology/everglades-nest-detection`	Detect bird nests in Everglades imagery
Dead Tree Detection	`weecology/cropmodel-deadtrees`	Detect dead trees in forest imagery

Steps:

Click the DeepForest button in the GeoAI toolbar (or GeoAI menu → DeepForest)
In the Model tab:
- Select a pretrained model from the dropdown (e.g., Tree Crown Detection)
- Configure device (auto, cuda, cpu)
- Click "Load Model" to download and initialize the model
- Select a raster layer or browse for an image file and click "Set Image from Layer" or "Set Image from File"
In the Predict tab:
- Choose prediction mode:
  - Single Image: Predict on the entire image at once (best for small images)
  - Large Tile: Split large geospatial tiles into overlapping patches for prediction
- For Large Tile mode, configure:
  - Patch Size: Size of each prediction window (default 400px; experiment with 400–800 for 0.1m data)
  - Patch Overlap: Overlap between adjacent patches (default 0.25)
  - IoU Threshold: Overlap threshold for suppressing duplicate detections (default 0.15)
  - Dataloader Strategy: Memory management strategy (single, batch, or window)
- Set the Score Threshold to filter low-confidence detections (default 0.3)
- Click "Run Prediction"
In the Output tab:
- Select output format:
  - Vector: GeoPackage, Shapefile, or GeoJSON (saves bounding boxes as polygons with score and label attributes)
  - Raster: GeoTIFF (rasterizes bounding boxes onto the source raster)
- Click "Save Results" to export
- To export training data for model fine-tuning:
  - Select, PASCAL_VOC, COCO or YOLO format
  - Choose an output directory
  - Click "Export Training Data"

Clear GPU Memory¶

Click the GPU button in the GeoAI toolbar to release GPU memory from all loaded models (Moondream, SamGeo, etc.) and clear CUDA cache. Use this frequently when switching between large models to prevent out-of-memory errors.

Plugin Update Checker¶

Go to GeoAI menu → Check for Updates... to see if a newer version of the GeoAI plugin is available. Click on the Check for Updates button to fetch the latest version info from GitHub. If an update is found, click the Download and Install Update button to download and install the latest version automatically. Restart QGIS to apply the update.

Supported Model Architectures (Segmentation)¶

The QGIS plugin supports any models supported by Pytorch Segmentation Models, including:

U-Net
U-Net++
DeepLabV3
DeepLabV3+
FPN (Feature Pyramid Network)
PSPNet
LinkNet
MANet
PAN
UperNet
SegFormer
DPT

Supported Encoders (Segmentation)¶

ResNet (34, 50, 101, 152)
EfficientNet (b0-b4)
MobileNetV2
VGG (16, 19)

Supported SAM Models (SamGeo)¶

SamGeo3 (SAM3): Latest version with text prompts, point prompts, and box prompts
SamGeo2 (SAM2): Improved version with better performance
SamGeo (SAM1): Original Segment Anything Model

Troubleshooting¶

Plugin missing after install: confirm the plugin folder exists in your QGIS profile path and that you restarted QGIS.
CUDA OOM: use the GPU button to clear cache, lower batch sizes, or switch to CPU for smaller runs.
Model download failures: check network/firewall, then retry loading models from the panel.

License¶

MIT License - see LICENSE for details.

QGIS Plugin for GeoAI¶

Quick Start¶

Video Tutorials¶

Installation Tutorial¶

Usage Tutorial¶

Requirements¶

Features¶

Moondream Vision-Language Model Panel¶

Segmentation Panel (Combined Training & Inference)¶

SamGeo Panel (Segment Anything Model)¶

DeepForest Panel (Object Detection)¶

GPU Memory Management¶

Installation¶

1. Set up the environment¶

1) Install Pixi¶

Linux/macOS (bash/zsh)¶

Windows (PowerShell)¶

2) Create a Pixi project¶

3) Configure pixi.toml¶

4) Install the environment¶

5) Verify PyTorch + CUDA¶

Request access to SAM 3¶

2. Install the QGIS plugin¶

3. Enable in QGIS¶

Usage¶

Moondream Vision-Language Model¶

Segmentation Panel (Create Data, Train, Inference)¶

SamGeo Panel (Segment Anything Model)¶

DeepForest Panel (Object Detection)¶

Supported Pretrained Models¶

Clear GPU Memory¶

Plugin Update Checker¶

Supported Model Architectures (Segmentation)¶

Supported Encoders (Segmentation)¶

Supported SAM Models (SamGeo)¶

Troubleshooting¶

License¶

Links¶

3) Configure `pixi.toml`¶