# Technical Documentation: opengeos/segment-geospatial

> ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/opengeos/segment-geospatial)

### Overview (`README.md`)

# SamGeo

[](https://studiolab.sagemaker.aws/import/github/opengeos/segment-geospatial/blob/main/docs/examples/satellite.ipynb)
[](https://colab.research.google.com/github/opengeos/segment-geospatial/blob/main/docs/examples/satellite.ipynb)
[](https://pypi.python.org/pypi/segment-geospatial)
[](https://anaconda.org/conda-forge/segment-geospatial)
[](https://hub.docker.com/r/giswqs/segment-geospatial)
[](https://pepy.tech/project/segment-geospatial)
[](https://github.com/conda-forge/segment-geospatial-feedstock)
[](https://anaconda.org/conda-forge/segment-geospatial)
[](https://doi.org/10.21105/joss.05663)
[](https://github.com/opengeos/qgis-samgeo-plugin)

[](https://github.com/opengeos/segment-geospatial/blob/main/docs/assets/logo.png)

**A Python package for segmenting geospatial data with the Segment Anything Model (SAM)**

## Introduction

The **SamGeo** package draws its inspiration from [segment-anything-eo](https://github.com/aliaksandr960/segment-anything-eo) repository authored by [Aliaksandr Hancharenka](https://github.com/aliaksandr960). The primary objective of SamGeo is to simplify the process of leveraging SAM for geospatial data analysis by enabling users to achieve this with minimal coding effort. The source code of SamGeo was adapted from the [segment-anything-eo](https://github.com/aliaksandr960/segment-anything-eo) repository, and credit for its original version goes to Aliaksandr Hancharenka.

-   Free software: MIT license
-   Documentation: <https://samgeo.gishub.org>

## Citations

-   Wu, Q., & Osco, L. (2023). samgeo: A Python package for segmenting geospatial data with the Segment Anything Model (SAM). _Journal of Open Source Software_, 8(89), 5663. <https://doi.org/10.21105/joss.05663>
-   Osco, L. P., Wu, Q., de Lemos, E. L., Gonçalves, W. N., Ramos, A. P. M., Li, J., & Junior, J. M. (2023). The Segment Anything Model (SAM) for remote sensing applications: From zero to one shot. _International Journal of Applied Earth Observation and Geoinformation_, 124, 103540. <https://doi.org/10.1016/j.jag.2023.103540>

## Features

-   Download map tiles from Tile Map Service (TMS) servers and create GeoTIFF files
-   Segment GeoTIFF files using the Segment Anything Model ([SAM](https://github.com/facebookresearch/segment-anything)) and [HQ-SAM](https://github.com/SysCV/sam-hq)
-   Segment remote sensing imagery with text prompts
-   Create foreground and background markers interactively
-   Load existing markers from vector datasets
-   Save segmentation results as common vector formats (GeoPackage, Shapefile, GeoJSON)
-   Save input prompts as GeoJSON files
-   Visualize segmentation results on interactive maps
-   Segment objects from timeseries remote sensing imagery
-   REST API for serving segmentation over HTTP (see [API docs](https://samgeo.gishub.org/api))

## QGIS Plugin

SamGeo is also available as a [QGIS plugin](https://github.com/opengeos/qgis-samgeo-plugin). Check out this [short video demo](https://youtu.be/DKKrQKeU3Ik) and [full video tutorial](https://youtu.be/oPZc7BvDsHE) on how to use the plugin.

[](https://youtu.be/oPZc7BvDsHE)

## Installation

### Install with pixi (Recommended)

For the most reliable installation experience, especially on Windows or when dealing with complex dependencies like PyTorch/CUDA and SAM 3, we recommend using [pixi](https://pixi.prefix.dev/latest). Pixi provides faster and more reliable dependency resolution than conda/mamba and avoids common numpy version conflicts. See the [full pixi installation guide](https://samgeo.gishub.org/installation/#install-with-pixi-recommended) for detailed instructions.

Quick start with pixi:

```bash
# Install pixi (Linux/macOS)
curl -fsSL https://pixi.sh/install.sh | sh

# Or on Windows (PowerShell)
powershell -ExecutionPolicy Bypass -c "irm -useb https://pixi.sh/install.ps1 | iex"

# Create a new pixi project
pixi init geo
cd geo

# Edit pixi.toml with your configuration (see docs for GPU/CPU examples)
# Then install
pixi install

# Start Jupyter Lab
pixi run jupyter lab
```

### Install from PyPI

**segment-geospatial** is available on [PyPI](https://pypi.org/project/segment-geospatial/) and can be installed in several ways so that its dependencies can be controlled more granularly. This reduces package size for CI environments, since not every time all of the models will be used.

Depending on what tools you need to use, you might want to do:

-   `segment-geospatial` or `segment-geospatial[samgeo]`: Installs only the minimum required dependencies to run SAMGeo
-   `segment-geospatial[samgeo2]`: Installs the dependencies to run SAMGeo 2
-   `segment-geospatial[samgeo3]`: Installs the dependencies to run SAMGeo 3
-   `segment-geospatial[fast]`: Installs the dependencies to run Fast SAM
-   `segment-geospatial[hq]`: Installs the dependencies to run HQ-SAM
-   `segment-geospatial[text]`: Installs Grounding DINO to use SAMGeo 1 and 2 with text prompts
-   `segment-geospatial[fer]`: Installs the dependencies to run the feature
    edge reconstruction algorithm
-   `segment-geospatial[api]`: Installs FastAPI and Uvicorn for serving segmentation as a REST API

Additionally, these other two optional imports are defined:

-   `segment-geospatial[all]`: Installs the dependencies to run all of the SAMGeo models
-   `segment-geospatial[extra]`: Installs the dependencies to run all of the SAMGeo models and other utilities to run the examples like Jupyter notebook support, `leafmap`, etc.

Simply running the following should install the dependencies for each use case:

```bash
pip install "segment-geospatial[samgeo3]" # Or any other choice of the above
```

To see more in detail what packages come with each choice, please refer to `pyproject.toml`.

### Install from conda-forge

**segment-geospatial** is also available on [conda-forge](https://anaconda.org/conda-forge/segment-geospatial). If you have
[Anaconda](https://www.anaconda.com/distribution/#download-section) or [Miniconda](https://docs.conda.io/en/latest/miniconda.html) installed on your computer, you can install segment-geospatial using the following commands. It is recommended to create a fresh conda environment for **segment-geospatial**. The following commands will create a new conda environment named `geo` and install **segment-geospatial** and its dependencies:

```bash
conda create -n geo python
conda activate geo
conda install -c conda-forge segment-geospatial
```

If your system has a GPU, but the above commands do not install the GPU version of pytorch, you can force the installation of the GPU version of pytorch using the following command:

```bash
conda install -c conda-forge segment-geospatial "pytorch=*=cuda*"
```

segment-geospatial has some optional dependencies that are not included in the default conda environment. To install these dependencies, run the following command:

```bash
conda install -c conda-forge groundingdino-py segment-anything-fast
```

### Install SAM 3 on Windows

It is a bit tricky to install SAM 3 on Windows. Run the following commands on Windows to install SamGeo:

```bash
conda create -n geo python=3.12
conda activate geo
conda install pytorch torchvision pytorch-cuda=12.1 -c pytorch -c nvidia
pip install "segment-geospatial[samgeo3]"
pip install triton-windows ipykernel jupyterlab
```

## Examples

-   [Segmenting remote sensing imagery](https://samgeo.gishub.org/examples/satellite)
-   [Automatically generating object masks](https://samgeo.gishub.org/examples/automatic_mask_generator)
-   [Segmenting remote sensing imagery with input prompts](https://samgeo.gishub.org/examples/input_prompts)
-   [Segmenting remote sensing imagery with box prompts](https://samgeo.gishub.org/examples/box_prompts)
-   [Segmenting remote sensing imagery with text prompts](https://samgeo.gishub.org/examples/text_prompts)
-   [Batch segmentation with text prompts](https://samgeo.gishub.org/examples/text_prompts_batch)
-   [Using segment-geospatial with ArcGIS Pro](https://samgeo.gishub.org/examples/arcgis)
-   [Segmenting swimming pools with text prompts](https://samgeo.gishub.org/examples/swimming_pools)
-   [Segmenting satellite imagery from the Maxar Open Data Program](https://samgeo.gishub.org/examples/max_open_data)

## Demos

-   Automatic mask generator



-   Interactive segmentation with input prompts



-   Input prompts from existing files



-   Interactive segmentation with text prompts



## Tutorials

Video tutorials are available on my [YouTube Channel](https://youtube.com/@giswqs).

-   Automatic mask generation

[](https://www.youtube.com/playlist?list=PLAxJ4-o7ZoPcrg5RnZjkB_KY6tv96WO2h)

-   Using SAM with ArcGIS Pro

[](https://www.youtube.com/playlist?list=PLAxJ4-o7ZoPcrg5RnZjkB_KY6tv96WO2h)

-   Interactive segmentation with text prompts

[](https://www.youtube.com/playlist?list=PLAxJ4-o7ZoPcrg5RnZjkB_KY6tv96WO2h)

## Using SAM with Desktop GIS

-   **QGIS**: Check out the [SamGeo QGIS Plugin](https://github.com/opengeos/qgis-samgeo-plugin).
-   **ArcGIS**: Check out the [Segment Anything Model (SAM) Toolbox for ArcGIS](https://www.arcgis.com/home/item.html?id=9b67b441f29f4ce6810979f5f0667ebe) and the [Resources for Unlocking the Power of Deep Learning Applications Using ArcGIS](https://community.esri.com/t5/education-blog/resources-for-unlocking-the-power-of-deep-learning/ba-p/1293098).

## Computing Resources

The Segment Anything Model is computationally intensive, and a powerful GPU is recommended to process large datasets. It is recommended to have a GPU with at least 8 GB of GPU memory. You can utilize the free GPU resources provided by Google Colab. Alternatively, you can apply for [AWS Cloud Credit for Research](https://aws.amazon.com/government-education/research-and-technical-computing/cloud-credit-for-research), which offers cloud credits to support academic research. If you are in the Greater China region, apply for the AWS Cloud Credit [here](https://aws.amazon.com/cn/events/educate_cloud/research-credits).

## Legal Notice

This repository and its content are provided for educational purposes only. By using the information and code provided, users acknowledge that they are using the APIs and models at their own risk and agree to comply with any applicable laws and regulations. Users who intend to download a large number of image tiles from any basemap are advised to contact the basemap provider to obtain permission before doing so. Unauthorized use of the basemap or any of its components may be a violation of copyright laws or other applicable laws and regulations.

## Contributing

Please refer to the [contributing guidelines](https://samgeo.gishub.org/contributing) for more information.

## Acknowledgements

This project is based upon work partially supported by the National Aeronautics and Space Administration (NASA) under Grant No. 80NSSC22K1742 issued through the [Open Source Tools, Frameworks, and Libraries 2020 Program](https://bit.ly/3RVBRcQ).

This project is also supported by Amazon Web Services ([AWS](https://aws.amazon.com/)). In addition, this package was made possible by the following open source projects. Credit goes to the developers of these projects.

-   [segment-anything](https://github.com/facebookresearch/segment-anything)
-   [SAM 3](https://github.com/facebookresearch/sam3)
-   [segment-anything-eo](https://github.com/aliaksandr960/segment-anything-eo)
-   [tms2geotiff](https://github.com/gumblex/tms2geotiff)
-   [GroundingDINO](https://github.com/IDEA-Research/GroundingDINO)
-   [lang-segment-anything](https://github.com/luca-medeiros/lang-segment-anything)

---

### Overview (`qgis-samgeo-plugin/README.md`)

# SamGeo QGIS Plugin

[](https://doi.org/10.21105/joss.05663)
[](https://github.com/opengeos/qgis-samgeo-plugin)
[](https://github.com/opengeos/segment-geospatial)

A QGIS plugin for remote sensing image segmentation using [SamGeo](https://samgeo.gishub.org), powered by Meta's Segment Anything Model ([SAM](https://github.com/facebookresearch/sam3)).

## Features

- **Multiple SAM versions**: Support for SamGeo (SAM1), SamGeo2 (SAM2), and SamGeo3 (SAM3)
- **Text-based segmentation**: Describe what you want to segment (e.g., "building", "tree", "road")
- **Interactive point prompts**: Click on the map to add foreground/background points
- **Box prompts**: Draw rectangles to segment specific regions
- **GeoTIFF support**: Works with georeferenced images, preserving spatial reference
- **Multiple output formats**: Save as vector (GeoPackage, Shapefile) or raster (GeoTIFF)

## Demos

Check out this [short video demo](https://youtu.be/DKKrQKeU3Ik) and [full video tutorial](https://youtu.be/oPZc7BvDsHE) on how to use the plugin.

[](https://youtu.be/oPZc7BvDsHE)

## Requirements

- QGIS 3.22 or later
- Python 3.10+
- SamGeo package
- Hugging Face account (for model downloads)
- Access to Meta's SAM 3 model

## Installation

### 1. Install SamGeo

First, install SamGeo in your QGIS Python environment. It is recommended to create a new conda environment **and** install QGIS and SamGeo. Install [Miniconda ](https://www.anaconda.com/docs/getting-started/miniconda/install), then run the following commands to install SamGeo on macOS/Linux:

```bash
conda create -n geo python=3.12
conda activate geo
conda install -c conda-forge qgis segment-geospatial
```

Some SamGeo dependencies are only available on PyPI. Run the following command to install all dependencies:

```bash
pip install -U "segment-geospatial[samgeo3]"
```

It is a bit tricky to install SAM 3 on Windows. Run the following commands on Windows to install SamGeo:

```bash
conda create -n geo python=3.12
conda activate geo
conda install pytorch torchvision pytorch-cuda=12.1 -c pytorch -c nvidia
pip install "segment-geospatial[samgeo3]" triton-windows
conda install -c conda-forge qgis
```

### 2. Install the Plugin

#### Option A: Using the install script

```bash
# Clone the repository
git clone https://github.com/opengeos/qgis-samgeo-plugin.git
cd qgis-samgeo-plugin
conda activate geo

# Install (Linux/macOS)
python install_plugin.py

# Or use the shell script
chmod +x install_plugin.sh
./install_plugin.sh
```

#### Option B: Manual installation

1. Find your QGIS plugins directory:

   - Linux: `~/.local/share/QGIS/QGIS3/profiles/default/python/plugins`
   - macOS: `~/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins`
   - Windows: `%APPDATA%\QGIS\QGIS3\profiles\default\python\plugins`

2. Copy or symlink this entire folder (qgis-samgeo-plugin) to the plugins directory and rename it to `samgeo_plugin`:

   ```bash
   ln -s /path/to/qgis-samgeo-plugin ~/.local/share/QGIS/QGIS3/profiles/default/python/plugins/samgeo_plugin
   ```

3. Restart QGIS

4. Enable the plugin: Go to **Plugins > Manage and Install Plugins**, find "SamGeo", and enable it.

### 3. Set up Hugging Face authentication

Sign up for a free account at [Hugging Face](https://huggingface.co/join) if you don't have one. Then, generate an access token by following the instructions [here](https://huggingface.co/docs/hub/en/security-tokens).

Run the following command in your terminal to log in to Hugging Face:

```bash
hf auth login
```

Provide your access token when prompted.

### 4. Request access to SAM 3 model

To use SamGeo3 (SAM 3), you need to request access to the model. Go to the [SAM 3 model page](https://huggingface.co/facebook/sam3) and log in with your Hugging Face account. Fill out the access request form and click on the "Submit" button to send your request. The approval process might take a few hours. Once approved, you can use SamGeo3 in the plugin.

If your request is denied, you can try to download the model from [ModelScope](https://www.modelscope.cn/models/facebook/sam3/files), which does not require access permission. After downloading, place the model files in the following directory:

- Linux/macOS: `~/.cache/huggingface/hub/models--facebook--sam3`
- Windows: `%USERPROFILE%\.cache\huggingface\hub\models--facebook--sam3`

## Usage

### Start QGIS

Run the following commands to start QGIS and use the SamGeo plugin:

```bash
conda activate geo
qgis
```

### Opening the Plugin

1. Click the SamGeo icon in the toolbar, or
2. Go to **Raster > SamGeo > SamGeo Segmentation**

### Workflow

#### 1. Load the Model

1. In the **Model** tab, select your preferred SAM version:
   - **SamGeo3 (SAM3)**: Latest version with text prompts support
   - **SamGeo2 (SAM2)**: Second generation SAM
   - **SamGeo (SAM1)**: Original SAM model
2. Choose the backend (meta or transformers)
3. Select the device (auto, cuda, or cpu)
4. Set the confidence threshold (default: 0.5)
5. Enable interactive mode if you want to use point/box prompts
6. Click **Load Model**

   

#### 2. Set an Image

- Select a raster layer from the dropdown and click **Set Image from Layer**, or
- Browse for an image file and click **Set Image from File**
- You can download a sample image from [here](https://huggingface.co/datasets/giswqs/geospatial/resolve/main/uc_berkeley.tif)

  

#### 3. Segment the Image

##### Text-Based Segmentation (Text tab)

1. Enter a text prompt describing what to segment (e.g., "tree", "building")
2. Optionally set min/max size filters
3. Click **Segment by Text**

   

##### Point-Based Segmentation (Interactive tab)

1. Click **Add Foreground Points** and click on objects to include
2. Click **Add Background Points** and click on areas to exclude
3. Click **Segment by Points**
4. Right-click to finish adding points

   

##### Box-Based Segmentation (Interactive tab)

1. Click **Draw Box**
2. Draw a rectangle around the area to segment
3. Click **Segment by Box**

   

##### Batch Point Segmentation (Batch tab)

Use this mode to process many points at once (e.g., building centroids):

1. Select a point vector layer from the dropdown, or browse for a vector file (GeoJSON, Shapefile, etc.)
2. Optionally specify the CRS if not auto-detected (e.g., "EPSG:4326")
3. Set min/max size filters if needed
4. Optionally specify an output raster file path
5. Click **Run Batch Segmentation**

   

Each point generates a separate mask, making this efficient for segmenting many individual objects.

#### 4. Save Results (Output tab)

1. Choose output format (vector or raster)
2. Set the output file path
3. Click **Save Masks**

   

## Programmatic Usage

You can also use SamGeo programmatically in the QGIS Python console:

```python
from samgeo import SamGeo3

# Initialize the model
sam = SamGeo3(
    backend="meta",
    enable_inst_interactivity=True,
    confidence_threshold=0.5,
)

# Set an image (GeoTIFF)
sam.set_image("/path/to/your/image.tif")

# Text-based segmentation
sam.generate_masks("building")
sam.save_masks("buildings.tif", unique=True)

# Point-based segmentation
point_coords = [[500, 300], [600, 400]]  # pixel coordinates
point_labels = [1, 1]  # 1=foreground, 0=background
sam.generate_masks_by_points(point_coords, point_labels)
sam.save_masks("points_result.tif")

# Batch point segmentation (from vector file)
sam.generate_masks_by_points_patch(
    point_coords_batch="building_centroids.geojson",
    point_crs="EPSG:4326",
    output="building_masks.tif",
    unique=True,
)

# Box-based segmentation
boxes = [[100, 100, 500, 500]]  # [xmin, ymin, xmax, ymax] in pixels
sam.generate_masks_by_boxes(boxes)
sam.save_masks("box_result.tif")

# Convert raster mask to vector
from samgeo import common
common.raster_to_vector("buildings.tif", "buildings.gpkg")
```

## Troubleshooting

### Model fails to load

- Ensure segment-geospatial is installed with the appropriate SAM support
- Check that you have enough GPU memory (or switch to CPU)
- Try restarting QGIS

### Segmentation is slow

- Use a GPU if available (select "cuda" device)
- Reduce image resolution before processing
- Use the transformers backend which may be faster for some tasks

### No objects found

- Try a different text prompt
- Lower the confidence threshold
- Adjust min/max size filters
- Ensure the image is properly loaded

## License

This plugin is released under the MIT License.

## Citation

If you use this plugin in your research, please cite:

```bibtex
@article{wu2023samgeo,
  title     = "{samgeo: A Python package for segmenting geospatial data with the
               Segment Anything Model (SAM)}",
  author    = "Wu, Qiusheng and Osco, Lucas Prado",
  journal   = "Journal of Open Source Software",
  volume    =  8,
  number    =  89,
  pages     =  5663,
  year      =  2023,
  url       = "https://doi.org/10.21105/joss.05663",
  doi       = "10.21105/joss.05663",
  issn      = "2475-9066"
}
```

## Contributing

Contributions are welcome! Please see the [GitHub repository](https://github.com/opengeos/qgis-samgeo-plugin) for guidelines.

---

### Overview (`docs/assets/README.md`)

# Credits

Credits to [Khalil Misbah](https://www.linkedin.com/in/khalil-misbah) for the original design of the leafmap logo. The samgeo logo is a derivative of the leafmap logo.

---

### Overview (`agent-harness/cli_anything/samgeo/README.md`)

# cli-anything-samgeo

CLI harness for [segment-geospatial](https://github.com/opengeos/segment-geospatial) — segment geospatial imagery using SAM models from the command line.

## Prerequisites

- Python 3.10+
- segment-geospatial: `pip install segment-geospatial[all]`
- PyTorch with CUDA (recommended) or CPU

## Installation

```bash
cd agent-harness
pip install -e .
```

This installs the `cli-anything-samgeo` command in your PATH.

## Quick Start

```bash
# Create a project
cli-anything-samgeo project new -n my-seg -o project.json -s image.tif

# Run automatic segmentation
cli-anything-samgeo --project project.json segment automatic -o masks.tif

# Convert masks to vectors
cli-anything-samgeo --project project.json vector convert masks.tif output.gpkg

# Export as GeoJSON
cli-anything-samgeo --project project.json export render output.geojson -f geojson

# All commands support --json for machine-readable output
cli-anything-samgeo --json model list
```

## Command Groups

| Command | Description |
|---------|-------------|
| `project` | Create, open, inspect projects |
| `model` | List, inspect, check SAM models |
| `segment` | Automatic, point, box, text segmentation |
| `data` | Download tiles, raster info, reproject, split |
| `vector` | Convert masks to vectors, inspect, filter |
| `export` | Export masks to various formats |
| `session` | Session status and history |

## Interactive REPL

Run without arguments to enter the interactive REPL:

```bash
cli-anything-samgeo
```

## JSON Output

Add `--json` before any command for machine-readable output:

```bash
cli-anything-samgeo --json data info image.tif
cli-anything-samgeo --json model list
```

## Using with Claude Code

This CLI ships with a `SKILL.md` file that lets Claude Code discover and use all
commands automatically. There are two ways to enable it.

### Option 1: Add SKILL.md to your CLAUDE.md

Append a reference to the skill file in your project or user `CLAUDE.md`:

```markdown
# In your CLAUDE.md
Read the skill file at /path/to/agent-harness/cli_anything/samgeo/skills/SKILL.md
for the full cli-anything-samgeo command reference. Use `--json` for all
cli-anything-samgeo commands so output is machine-readable.
```

Replace `/path/to/` with the actual absolute path. Claude Code reads `CLAUDE.md`
at the start of every conversation, so it will know the CLI exists and how to
call it.

### Option 2: Point Claude Code at the skill on the fly

In any Claude Code conversation, paste:

```
Read agent-harness/cli_anything/samgeo/skills/SKILL.md and use that CLI
to segment this satellite image.
```

Claude Code will read the skill file, learn the command syntax, and start
using `cli-anything-samgeo` with `--json` output.

### Example Claude Code session

Once Claude Code knows about the skill, you can give it natural-language tasks:

```
> Segment all buildings in satellite.tif and export the results as a GeoPackage.

# Claude Code will run:
cli-anything-samgeo --json project new -n buildings -o project.json -s satellite.tif -t sam2
cli-anything-samgeo --json --project project.json segment automatic -o masks.tif
cli-anything-samgeo --json vector convert masks.tif buildings.gpkg
```

```
> Download OpenStreetMap tiles for downtown Portland and tell me about the image.

# Claude Code will run:
cli-anything-samgeo --json data download-tiles -o portland.tif -b "-122.68,45.51,-122.66,45.53" -z 17
cli-anything-samgeo --json data info portland.tif
```

### Tips for Claude Code usage

- The `--json` flag is essential — it gives Claude Code structured output it can
  parse and reason about, rather than human-formatted tables.
- The `--project` flag must appear *before* the command group (e.g.,
  `--project proj.json segment automatic`, not `segment automatic --project proj.json`).
- Claude Code can chain multiple commands in sequence to build full pipelines
  (download → segment → vectorize → export).
- Use `model check sam2` to let Claude Code verify a model backend is installed
  before attempting segmentation.

## Running Tests

```bash
cd agent-harness
python -m pytest cli_anything/samgeo/tests/ -v -s
```

## Supported Models

| Type | Models | Install |
|------|--------|---------|
| SAM v1 | vit_h, vit_l, vit_b | `pip install segment-geospatial` |
| SAM 2 | hiera-tiny/small/base-plus/large | `pip install segment-geospatial[samgeo2]` |
| SAM 3 | facebook/sam3 | `pip install segment-geospatial[samgeo3]` |
| FastSAM | FastSAM-x, FastSAM-s | `pip install segment-geospatial[fast]` |
| HQ-SAM | vit_h, vit_l, vit_b, vit_tiny | `pip install segment-geospatial[hq]` |
| LangSAM | text-based (SAM2 backend) | `pip install segment-geospatial[text]` |

---

### Index (`docs/index.md`)

# SamGeo

[](https://studiolab.sagemaker.aws/import/github/opengeos/segment-geospatial/blob/main/docs/examples/satellite.ipynb)
[](https://colab.research.google.com/github/opengeos/segment-geospatial/blob/main/docs/examples/satellite.ipynb)
[](https://pypi.python.org/pypi/segment-geospatial)
[](https://anaconda.org/conda-forge/segment-geospatial)
[](https://hub.docker.com/r/giswqs/segment-geospatial)
[](https://pepy.tech/project/segment-geospatial)
[](https://github.com/conda-forge/segment-geospatial-feedstock)
[](https://anaconda.org/conda-forge/segment-geospatial)
[](https://doi.org/10.21105/joss.05663)
[](https://github.com/opengeos/qgis-samgeo-plugin)

[](https://github.com/opengeos/segment-geospatial/blob/main/docs/assets/logo.png)

**A Python package for segmenting geospatial data with the Segment Anything Model (SAM)**

## Introduction

The **SamGeo** package draws its inspiration from [segment-anything-eo](https://github.com/aliaksandr960/segment-anything-eo) repository authored by [Aliaksandr Hancharenka](https://github.com/aliaksandr960). The primary objective of SamGeo is to simplify the process of leveraging SAM for geospatial data analysis by enabling users to achieve this with minimal coding effort. The source code of SamGeo was adapted from the [segment-anything-eo](https://github.com/aliaksandr960/segment-anything-eo) repository, and credit for its original version goes to Aliaksandr Hancharenka.

-   Free software: MIT license
-   Documentation: <https://samgeo.gishub.org>

## Citations

-   Wu, Q., & Osco, L. (2023). samgeo: A Python package for segmenting geospatial data with the Segment Anything Model (SAM). _Journal of Open Source Software_, 8(89), 5663. <https://doi.org/10.21105/joss.05663>
-   Osco, L. P., Wu, Q., de Lemos, E. L., Gonçalves, W. N., Ramos, A. P. M., Li, J., & Junior, J. M. (2023). The Segment Anything Model (SAM) for remote sensing applications: From zero to one shot. _International Journal of Applied Earth Observation and Geoinformation_, 124, 103540. <https://doi.org/10.1016/j.jag.2023.103540>

## Features

-   Download map tiles from Tile Map Service (TMS) servers and create GeoTIFF files
-   Segment GeoTIFF files using the Segment Anything Model ([SAM](https://github.com/facebookresearch/segment-anything)) and [HQ-SAM](https://github.com/SysCV/sam-hq)
-   Segment remote sensing imagery with text prompts
-   Create foreground and background markers interactively
-   Load existing markers from vector datasets
-   Save segmentation results as common vector formats (GeoPackage, Shapefile, GeoJSON)
-   Save input prompts as GeoJSON files
-   Visualize segmentation results on interactive maps
-   Segment objects from timeseries remote sensing imagery
-   REST API for serving segmentation over HTTP (see [API docs](https://samgeo.gishub.org/api))

## QGIS Plugin

SamGeo is also available as a [QGIS plugin](https://github.com/opengeos/qgis-samgeo-plugin). Check out this [short video demo](https://youtu.be/DKKrQKeU3Ik) and [full video tutorial](https://youtu.be/oPZc7BvDsHE) on how to use the plugin.

[](https://youtu.be/oPZc7BvDsHE)

## Installation

### Install from PyPI

**segment-geospatial** is available on [PyPI](https://pypi.org/project/segment-geospatial/) and can be installed in several ways so that its dependencies can be controlled more granularly. This reduces package size for CI environments, since not every time all of the models will be used.

Depending on what tools you need to use, you might want to do:

-   `segment-geospatial` or `segment-geospatial[samgeo]`: Installs only the minimum required dependencies to run SAMGeo
-   `segment-geospatial[samgeo2]`: Installs the dependencies to run SAMGeo 2
-   `segment-geospatial[samgeo3]`: Installs the dependencies to run SAMGeo 3
-   `segment-geospatial[fast]`: Installs the dependencies to run Fast SAM
-   `segment-geospatial[hq]`: Installs the dependencies to run HQ-SAM
-   `segment-geospatial[text]`: Installs Grounding DINO to use SAMGeo 1 and 2 with text prompts
-   `segment-geospatial[fer]`: Installs the dependencies to run the feature
    edge reconstruction algorithm
-   `segment-geospatial[api]`: Installs FastAPI and Uvicorn for serving segmentation as a REST API

Additionally, these other two optional imports are defined:

-   `segment-geospatial[all]`: Installs the dependencies to run all of the SAMGeo models
-   `segment-geospatial[extra]`: Installs the dependencies to run all of the SAMGeo models and other utilities to run the examples like Jupyter notebook support, `leafmap`, etc.

Simply running the following should install the dependencies for each use case:

```bash
pip install "segment-geospatial[samgeo3]" # Or any other choice of the above
```

To see more in detail what packages come with each choice, please refer to `pyproject.toml`.

### Install from conda-forge

**segment-geospatial** is also available on [conda-forge](https://anaconda.org/conda-forge/segment-geospatial). If you have
[Anaconda](https://www.anaconda.com/distribution/#download-section) or [Miniconda](https://docs.conda.io/en/latest/miniconda.html) installed on your computer, you can install segment-geospatial using the following commands. It is recommended to create a fresh conda environment for **segment-geospatial**. The following commands will create a new conda environment named `geo` and install **segment-geospatial** and its dependencies:

```bash
conda create -n geo python
conda activate geo
conda install -c conda-forge segment-geospatial
```

If your system has a GPU, but the above commands do not install the GPU version of pytorch, you can force the installation of the GPU version of pytorch using the following command:

```bash
conda install -c conda-forge segment-geospatial "pytorch=*=cuda*"
```

Samgeo-geospatial has some optional dependencies that are not included in the default conda environment. To install these dependencies, run the following command:

```bash
conda install -c conda-forge groundingdino-py segment-anything-fast
```

### Install SAM 3 on Windows

It is a bit tricky to install SAM 3 on Windows. Run the following commands on Windows to install SamGeo:

```bash
conda create -n geo python=3.12
conda activate geo
conda install pytorch torchvision pytorch-cuda=12.1 -c pytorch -c nvidia
pip install "segment-geospatial[samgeo3]"
pip install triton-windows ipykernel jupyterlab
```

## Examples

-   [Segmenting remote sensing imagery](https://samgeo.gishub.org/examples/satellite)
-   [Automatically generating object masks](https://samgeo.gishub.org/examples/automatic_mask_generator)
-   [Segmenting remote sensing imagery with input prompts](https://samgeo.gishub.org/examples/input_prompts)
-   [Segmenting remote sensing imagery with box prompts](https://samgeo.gishub.org/examples/box_prompts)
-   [Segmenting remote sensing imagery with text prompts](https://samgeo.gishub.org/examples/text_prompts)
-   [Batch segmentation with text prompts](https://samgeo.gishub.org/examples/text_prompts_batch)
-   [Using segment-geospatial with ArcGIS Pro](https://samgeo.gishub.org/examples/arcgis)
-   [Segmenting swimming pools with text prompts](https://samgeo.gishub.org/examples/swimming_pools)
-   [Segmenting satellite imagery from the Maxar Open Data Program](https://samgeo.gishub.org/examples/max_open_data)

## Demos

-   Automatic mask generator



-   Interactive segmentation with input prompts



-   Input prompts from existing files



-   Interactive segmentation with text prompts



## Tutorials

Video tutorials are available on my [YouTube Channel](https://youtube.com/@giswqs).

-   Automatic mask generation

[](https://www.youtube.com/playlist?list=PLAxJ4-o7ZoPcrg5RnZjkB_KY6tv96WO2h)

-   Using SAM with ArcGIS Pro

[](https://www.youtube.com/playlist?list=PLAxJ4-o7ZoPcrg5RnZjkB_KY6tv96WO2h)

-   Interactive segmentation with text prompts

[](https://www.youtube.com/playlist?list=PLAxJ4-o7ZoPcrg5RnZjkB_KY6tv96WO2h)

## Using SAM with Desktop GIS

-   **QGIS**: Check out the [SamGeo QGIS Plugin](https://github.com/opengeos/qgis-samgeo-plugin).
-   **ArcGIS**: Check out the [Segment Anything Model (SAM) Toolbox for ArcGIS](https://www.arcgis.com/home/item.html?id=9b67b441f29f4ce6810979f5f0667ebe) and the [Resources for Unlocking the Power of Deep Learning Applications Using ArcGIS](https://community.esri.com/t5/education-blog/resources-for-unlocking-the-power-of-deep-learning/ba-p/1293098).

## Computing Resources

The Segment Anything Model is computationally intensive, and a powerful GPU is recommended to process large datasets. It is recommended to have a GPU with at least 8 GB of GPU memory. You can utilize the free GPU resources provided by Google Colab. Alternatively, you can apply for [AWS Cloud Credit for Research](https://aws.amazon.com/government-education/research-and-technical-computing/cloud-credit-for-research), which offers cloud credits to support academic research. If you are in the Greater China region, apply for the AWS Cloud Credit [here](https://aws.amazon.com/cn/events/educate_cloud/research-credits).

## Legal Notice

This repository and its content are provided for educational purposes only. By using the information and code provided, users acknowledge that they are using the APIs and models at their own risk and agree to comply with any applicable laws and regulations. Users who intend to download a large number of image tiles from any basemap are advised to contact the basemap provider to obtain permission before doing so. Unauthorized use of the basemap or any of its components may be a violation of copyright laws or other applicable laws and regulations.

## Contributing

Please refer to the [contributing guidelines](https://samgeo.gishub.org/contributing) for more information.

## Acknowledgements

This project is based upon work partially supported by the National Aeronautics and Space Administration (NASA) under Grant No. 80NSSC22K1742 issued through the [Open Source Tools, Frameworks, and Libraries 2020 Program](https://bit.ly/3RVBRcQ).

This project is also supported by Amazon Web Services ([AWS](https://aws.amazon.com/)). In addition, this package was made possible by the following open source projects. Credit goes to the developers of these projects.

-   [segment-anything](https://github.com/facebookresearch/segment-anything)
-   [SAM 3](https://github.com/facebookresearch/sam3)
-   [segment-anything-eo](https://github.com/aliaksandr960/segment-anything-eo)
-   [tms2geotiff](https://github.com/gumblex/tms2geotiff)
-   [GroundingDINO](https://github.com/IDEA-Research/GroundingDINO)
-   [lang-segment-anything](https://github.com/luca-medeiros/lang-segment-anything)

---

### Api (`docs/api.md`)

# REST API

segment-geospatial includes a built-in REST API powered by [FastAPI](https://fastapi.tiangolo.com/) that allows you to run image segmentation over HTTP. This is useful for integrating segmentation into web applications, pipelines, and non-Python clients.

## Installation

Install the API dependencies with the `api` extra:

```bash
pip install "segment-geospatial[api]"
```

To also install a specific SAM model backend, combine extras:

```bash
pip install "segment-geospatial[api,samgeo3]"
```

## Starting the Server

Use the `samgeo-api` command:

```bash
samgeo-api
```

Options:

```bash
samgeo-api --host 0.0.0.0 --port 8000        # Custom host/port
samgeo-api --preload sam2:sam2-hiera-large     # Preload a model at startup
samgeo-api --reload                            # Auto-reload for development
```

Alternatively, use `uvicorn` directly:

```bash
uvicorn samgeo.api:app --host 0.0.0.0 --port 8000
```

Once running, interactive API docs (Swagger UI) are available at [http://localhost:8000/docs](http://localhost:8000/docs).

## Endpoints

### Health Check

```
GET /health
```

Returns the server status and version.

```bash
curl http://localhost:8000/health
```

```json
{"status": "ok", "version": "1.2.3"}
```

### List Models

```
GET /models
```

Returns available model versions/IDs and which models are currently loaded in memory.

```bash
curl http://localhost:8000/models
```

### Clear Models

```
DELETE /models
```

Clears the model cache and frees GPU memory.

```bash
curl -X DELETE http://localhost:8000/models
```

### Automatic Segmentation

```
POST /segment/automatic
```

Runs automatic mask generation on an uploaded image. Supports SAM, SAM2, and SAM3.

**Parameters (multipart form):**

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `file` | file | required | Image file (TIFF, PNG, JPEG) |
| `model_version` | string | `sam2` | One of `sam`, `sam2`, `sam3` |
| `model_id` | string | auto | Model identifier (e.g., `sam2-hiera-large`, `facebook/sam3.1`) |
| `output_format` | string | `geojson` | One of `geojson`, `geotiff`, `png`, `json`, `detections` |
| `foreground` | bool | `true` | Extract foreground objects only |
| `unique` | bool | `true` | Assign unique ID to each object |
| `min_size` | int | `0` | Minimum mask size in pixels |
| `max_size` | int | none | Maximum mask size in pixels |
| `points_per_side` | int | `32` | Points sampled per side (SAM/SAM2) |
| `pred_iou_thresh` | float | `0.8` | IoU threshold for filtering |
| `stability_score_thresh` | float | `0.95` | Stability score threshold |

**Example:**

```bash
curl -X POST http://localhost:8000/segment/automatic \
  -F "file=@image.tif" \
  -F "model_version=sam2" \
  -F "output_format=geojson"
```

### Prompt-based Segmentation

```
POST /segment/predict
```

Runs segmentation with point or bounding box prompts. Supports SAM, SAM2, and SAM3.

For SAM3 with bounding box prompts, the model finds **all similar objects** in the image (not just the object inside the box). Point prompts with SAM3 segment the specific object at the point location.

**Parameters (multipart form):**

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `file` | file | required | Image file (TIFF, PNG, JPEG) |
| `model_version` | string | `sam3` | One of `sam`, `sam2`, `sam3` |
| `model_id` | string | auto | Model identifier (e.g., `facebook/sam3.1`) |
| `output_format` | string | `geojson` | One of `geojson`, `geotiff`, `png`, `json`, `detections` |
| `point_coords` | string | none | JSON array of `[[x, y], ...]` |
| `point_labels` | string | none | JSON array of `[1, 0, ...]` (1=foreground, 0=background) |
| `boxes` | string | none | JSON array of `[[xmin, ymin, xmax, ymax], ...]` |
| `point_crs` | string | none | CRS string (e.g., `EPSG:4326`) for point/box coordinates |
| `multimask_output` | bool | `false` | Return multiple masks per prompt |
| `min_size` | int | `0` | Minimum mask size in pixels |
| `max_size` | int | none | Maximum mask size in pixels |

**Example with point prompts:**

```bash
curl -X POST http://localhost:8000/segment/predict \
  -F "file=@image.tif" \
  -F "point_coords=[[100, 200]]" \
  -F "point_labels=[1]" \
  -F "output_format=geojson"
```

**Example with box prompts (finds all similar objects):**

```bash
curl -X POST http://localhost:8000/segment/predict \
  -F "file=@image.tif" \
  -F "boxes=[[10, 20, 300, 400]]" \
  -F "output_format=geojson"
```

**Example with JSON output (pixel-coordinate bounding boxes):**

```bash
curl -X POST http://localhost:8000/segment/predict \
  -F "file=@image.jpg" \
  -F "boxes=[[10, 20, 300, 400]]" \
  -F "output_format=json"
```

```json
{
  "image_width": 2647,
  "image_height": 1464,
  "num_detections": 12,
  "detections": [
    {"id": 1, "value": 1, "bbox": [50, 80, 200, 250], "width": 150, "height": 170},
    {"id": 2, "value": 2, "bbox": [310, 45, 480, 210], "width": 170, "height": 165}
  ]
}
```

**Example with detections output (geographic-coordinate bounding boxes):**

```bash
curl -X POST http://localhost:8000/segment/predict \
  -F "file=@image.tif" \
  -F "boxes=[[10, 20, 300, 400]]" \
  -F "output_format=detections"
```

```json
{
  "type": "FeatureCollection",
  "crs": "EPSG:3857",
  "num_detections": 12,
  "features": [
    {
      "type": "Feature",
      "geometry": {
        "type": "Polygon",
        "coordinates": [[[-13609328.39, 4561446.23], [-13609284.55, 4561446.23], [-13609284.55, 4561389.77], [-13609328.39, 4561389.77], [-13609328.39, 4561446.23]]]
      },
      "properties": {"id": 1, "value": 1, "bbox_pixel": [50.0, 80.0, 200.0, 250.0]}
    }
  ]
}
```

### Text-prompt Segmentation

```
POST /segment/text
```

Runs text-prompt segmentation using SAM3.

**Parameters (multipart form):**

| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `file` | file | required | Image file (TIFF, PNG, JPEG) |
| `prompt` | string | required | Text description (e.g., `building`, `tree`) |
| `model_id` | string | auto | SAM3 model identifier (`facebook/sam3` or `facebook/sam3.1`) |
| `backend` | string | `meta` | One of `meta`, `transformers`; SAM 3.1 uses `meta` |
| `output_format` | string | `geojson` | One of `geojson`, `geotiff`, `png`, `json`, `detections` |
| `confidence_threshold` | float | `0.5` | Detection confidence threshold |
| `min_size` | int | `0` | Minimum mask size in pixels |
| `max_size` | int | none | Maximum mask size in pixels |

**Example (GeoJSON mask polygons):**

```bash
curl -X POST http://localhost:8000/segment/text \
  -F "file=@image.tif" \
  -F "prompt=building" \
  -F "output_format=geojson"
```

**Example with JSON output (pixel-coordinate bounding boxes):**

```bash
curl -X POST http://localhost:8000/segment/text \
  -F "file=@image.jpg" \
  -F "prompt=building" \
  -F "output_format=json"
```

```json
{
  "image_width": 2647,
  "image_height": 1464,
  "num_detections": 46,
  "detections": [
    {"id": 1, "bbox": [2506, 134, 2653, 324], "width": 147, "height": 190, "score": 0.887},
    {"id": 2, "bbox": [1200, 450, 1380, 620], "width": 180, "height": 170, "score": 0.862}
  ]
}
```

**Example with detections output (geographic-coordinate bounding boxes):**

```bash
curl -X POST http://localhost:8000/segment/text \
  -F "file=@image.tif" \
  -F "prompt=building" \
  -F "output_format=detections"
```

```json
{
  "type": "FeatureCollection",
  "crs": "EPSG:3857",
  "num_detections": 46,
  "features": [
    {
      "type": "Feature",
      "geometry": {
        "type": "Polygon",
        "coordinates": [[[-13609328.39, 4561446.23], [-13609284.55, 4561446.23], [-13609284.55, 4561389.77], [-13609328.39, 4561389.77], [-13609328.39, 4561446.23]]]
      },
      "properties": {"id": 1, "score": 0.887, "bbox_pixel": [2506.47, 134.43, 2653.27, 323.52]}
    }
  ]
}
```

## GeoJSON output

With `output_format=geojson`, every segmentation endpoint returns one polygon
feature per mask. Each feature's `properties` carry the mask's raster `value`
and, when the model reports one, its confidence `score`, so a single request
yields both geometry and confidence:

```json
{"type": "Feature", "geometry": {"type": "Polygon", "coordinates": [...]}, "properties": {"value": 1, "score": 0.887}}
```

## Caching

The API automatically caches models and image encodings for better performance:

- **Model cache**: Models are loaded once and reused across requests. Use `DELETE /models` to free GPU memory.
- **Image cache**: When the same image is sent multiple times (e.g., with different prompts), the expensive image encoding step is skipped. This makes subsequent requests significantly faster.

Example timing with a 13 MB GeoTIFF:

| Request | Description | Time |
|---------|-------------|------|
| 1st | Model load + image encoding | ~7s |
| 2nd | Same image, different prompt | ~0.4s |
| 3rd | Same image, another prompt | ~0.2s |

## Python Client Example

```python
import requests

url = "http://localhost:8000/segment/text"

# Get GeoJSON mask polygons
with open("image.tif", "rb") as f:
    response = requests.post(
        url,
        files={"file": ("image.tif", f, "image/tiff")},
        data={"prompt": "building", "output_format": "geojson"},
    )

geojson = response.json()
print(f"Found {len(geojson['features'])} features")
```

```python
# Get bounding boxes in pixel coordinates (suitable for non-georeferenced images)
with open("image.jpg", "rb") as f:
    response = requests.post(
        url,
        files={"file": ("image.jpg", f, "image/jpeg")},
        data={"prompt": "car", "output_format": "json"},
    )

result = response.json()
for det in result["detections"]:
    print(f"Object {det['id']}: bbox={det['bbox']}, score={det['score']:.3f}")
```

## API Reference

::: samgeo.api

---

### Changelog (`docs/changelog.md`)

# Changelog

## v0.10.2 - Nov 7, 2023

**What's Changed**

-   Add JOSS paper by @giswqs in [#197](https://github.com/opengeos/segment-geospatial/pull/197)
-   Add notebook for using Maxar Open Data by @giswqs in [#198](https://github.com/opengeos/segment-geospatial/pull/198)
-   Add checkpoint to textsam.LangSAM() by @forestbat in [#204](https://github.com/opengeos/segment-geospatial/pull/204)
-   Add workshop notebook by @giswqs in [#209](https://github.com/opengeos/segment-geospatial/pull/209)

**New Contributors**

-   @forestbat made their first contribution in [#204](https://github.com/opengeos/segment-geospatial/pull/204)

**Full Changelog**: [v0.10.1...v0.10.2](https://github.com/opengeos/segment-geospatial/compare/v0.10.1...v0.10.2)

## v0.10.1 - Sep 1, 2023

**What's Changed**

-   Fix basemap issue by @giswqs in [#190](https://github.com/opengeos/segment-geospatial/pull/190)

**Full Changelog**: [v0.10.0...v0.10.1)](https://github.com/opengeos/segment-geospatial/compare/v0.10.0...v0.10.1)

## v0.10.0 - Aug 24, 2023

### What's Changed

-   Added fastsam module by @giswqs #167
-   Update optional dependencies by @giswqs in #68
-   Improve contributing guidelines by @giswqs in #169
-   [FIX] Added missing conversions from BGR to RGB by @lbferreira in #171
-   Address JOSS review comments by @giswqs in #175

### New Contributors

-   @lbferreira made their first contribution in #171

## v0.9.1 - Aug 14, 2023

**New Features**

-   Added support for HQ-SAM (#161)
-   Added HQ-SAM notebooks (#162)

## v0.9.0 - Aug 6, 2023

**New Features**

-   Added support for multiple input boxes (#159)

**Improvements**

-   UpdateD groundingdino installation (#147)
-   Updated README (#152)

## v0.8.5 - Jul 19, 2023

**Improvements**

-   Updated installation docs (#146)
-   Updated leafmap and localtileserver to dependencies (#146)
-   Added info about libgl1 dependency install on Linux systems (#141)
-   Fixed save_masks bug without source image (#139)

## v0.8.4 - Jul 5, 2023

**Improvements**

-   Fixed model download bug (#136)
-   Added legal notice (#133)
-   Fixed image source bug for show_anns (#131)
-   Improved exception handling for LangSAM GUI (#130)
-   Added to return pixel coordinates of masks (#129)
-   Added text_sam to docs (#123)
-   Fixed file deletion error on Windows (#122)
-   Fixed mask bug in text_sam/predict when the input is PIL image (#117)

## v0.8.3 - Jun 20, 2023

**New Features**

-   Added support for batch segmentation (#116)
-   Added swimming pools example (#106)

**Improvements**

-   Removed 'flag' and 'param' arguments (#112)
-   Used sorted function instead of if statements (#109)

## v0.8.2 - Jun 14, 2023

**New Features**

-   Added regularized option for vector output (#104)
-   Added text prompt GUI (#80)

**Improvements**

-   Added more deep learning resources (#90)
-   Use the force_filename parameter with hf_hub_download() (#93)
-   Fixed typo (#94)

## v0.8.1 - May 24, 2023

**Improvements**

-   Added huggingface_hub and remove onnx (#87)
-   Added more demos to docs (#82)

## v0.8.0 - May 24, 2023

**New Features**

-   Added support for using text prompts with SAM (#73)
-   Added text prompt GUI (#80)

**Improvements**

-   Improved text prompt notebook (#79)
-   Fixed notebook typos (#78)
-   Added ArcGIS tutorial to docs (#72)

## v0.7.0 - May 20, 2023

**New Features**

-   Added unittest (#58)
-   Added JOSS paper draft (#61)
-   Added ArcGIS notebook example (#63)
-   Added text prompting segmentation (#65)
-   Added support for segmenting non-georeferenced imagery (#66)

**Improvements**

-   Added blend option for show_anns method (#59)
-   Updated ArcGIS installation instructions (#68, #70)

**Contributors**

@p-vdp @LucasOsco

## v0.6.2 - May 17, 2023

**Improvements**

-   Added jupyter-server-proxy to Dockerfile for supporting add_raster (#57)

## v0.6.1 - May 16, 2023

**New Features**

-   Added Dockerfile (#51)

## v0.6.0 - May 16, 2023

**New Features**

-   Added interactive GUI for creating foreground and background markers (#44)
-   Added support for custom projection bbox (#39)

**Improvements**

-   Fixed Colab Marker AwesomeIcon bug (#50)
-   Added info about using SAM with Desktop GIS (#48)
-   Use proper extension in the usage documentation (#43)

**Demos**

-   Interactive segmentation with input prompts



-   Input prompts from existing files



## v0.5.0 - May 10, 2023

**New Features**

-   Added support for input prompts (#30)

**Improvements**

-   Fixed the batch processing bug (#29)

**Demos**



## v0.4.0 - May 6, 2023

**New Features**

-   Added new methods to `SamGeo` class, including `show_masks`, `save_masks`, `show_anns`, making it much easier to save segmentation results in GeoTIFF and vector formats.
-   Added new functions to `common` module, including `array_to_image`, `show_image`, `download_file`, `overlay_images`, `blend_images`, and `update_package`
-   Added tow more notebooks, including [automatic_mask_generator](https://samgeo.gishub.org/examples/automatic_mask_generator/) and [satellite-predictor](https://samgeo.gishub.org/examples/satellite-predictor/)
-   Added `SamGeoPredictor` class

**Improvements**

-   Improved `SamGeo.generate()` method
-   Improved docstrings and API reference
-   Added demos to docs

**Demos**

-   Automatic mask generator



**Contributors**

@darrenwiens

## v0.3.0 - Apr 26, 2023

**New Features**

-   Added several new functions, including `get_basemaps`, `reproject`, `tiff_to_shp`, and `tiff_to_geojson`
-   Added hundereds of new basemaps through xyzservices

**Improvement**

-   Fixed `tiff_to_vector` crs bug #12
-   Add `crs` parameter to `tms_to_geotiff`

## v0.2.0 - Apr 21, 2023

**New Features**

-   Added notebook example
-   Added `SamGeo.generate` method
-   Added `SamGeo.tiff_to_vector` method

## v0.1.0 - Apr 19, 2023

**New Features**

-   Added `SamGeo` class
-   Added GitHub Actions
-   Added notebook example

## v0.0.1 - Apr 18, 2023

Initial release

---

### Contributing (`docs/contributing.md`)

# Contributing

Contributions are welcome, and they are greatly appreciated! Every
little bit helps, and credit will always be given.

You can contribute in many ways:

## Types of Contributions

### Report Bugs

Report bugs at <https://github.com/giswqs/segment-geospatial/issues>.

If you are reporting a bug, please include:

-   Your operating system name and version.
-   Any details about your local setup that might be helpful in troubleshooting.
-   Detailed steps to reproduce the bug.

### Fix Bugs

Look through the GitHub issues for bugs. Anything tagged with `bug` and
`help wanted` is open to whoever wants to implement it.

### Implement Features

Look through the GitHub issues for features. Anything tagged with
`enhancement` and `help wanted` is open to whoever wants to implement it.

### Write Documentation

segment-geospatial could always use more documentation,
whether as part of the official segment-geospatial docs,
in docstrings, or even on the web in blog posts, articles, and such.

### Submit Feedback

The best way to send feedback is to file an issue at
<https://github.com/giswqs/segment-geospatial/issues>.

If you are proposing a feature:

-   Explain in detail how it would work.
-   Keep the scope as narrow as possible, to make it easier to implement.
-   Remember that this is a volunteer-driven project, and that contributions are welcome :)

## Get Started!

Ready to contribute? Here's how to set up segment-geospatial for local development.

1.  Fork the segment-geospatial repo on GitHub.

2.  Clone your fork locally:

    ```shell
    $ git clone git@github.com:your_name_here/segment-geospatial.git
    ```

3.  Install your local copy into a virtualenv. Assuming you have
    virtualenvwrapper installed, this is how you set up your fork for
    local development:

    ```shell
    $ mkvirtualenv segment-geospatial
    $ cd segment-geospatial/
    $ python setup.py develop
    ```

4.  Create a branch for local development:

    ```shell
    $ git checkout -b name-of-your-bugfix-or-feature
    ```

    Now you can make your changes locally.

5.  When you're done making changes, check that your changes pass flake8
    and the tests, including testing other Python versions with tox:

    ```shell
    $ flake8 segment-geospatial tests
    $ python setup.py test or pytest
    $ tox
    ```

    To get flake8 and tox, just pip install them into your virtualenv.

6.  Commit your changes and push your branch to GitHub:

    ```shell
    $ git add .
    $ git commit -m "Your detailed description of your changes."
    $ git push origin name-of-your-bugfix-or-feature
    ```

7.  Submit a pull request through the GitHub website.

## Pull Request Guidelines

Before you submit a pull request, check that it meets these guidelines:

1.  The pull request should include tests (see the section below - [Unit Testing](https://samgeo.gishub.org/contributing/#unit-testing)).
2.  If the pull request adds functionality, the docs should be updated.
    Put your new functionality into a function with a docstring, and add
    the feature to the list in README.md.
3.  The pull request should work for Python 3.8, 3.9, 3.10, and 3.11. Check <https://github.com/giswqs/segment-geospatial/pull_requests> and make sure that the tests pass for all
    supported Python versions.

## Unit Testing

Unit tests are in the `tests` folder. If you add new functionality to the package, please add a unit test for it. You can either add the test to an existing test file or create a new one. For example, if you add a new function to `samgeo/samgeo.py`, you can add the unit test to `tests/test_samgeo.py`. If you add a new module to `samgeo/<MODULE-NAME>`, you can create a new test file in `tests/test_<MODULE-NAME>`. Please refer to `tests/test_samgeo.py` for examples. For more information about unit testing, please refer to this tutorial - [Getting Started With Testing in Python](https://realpython.com/python-testing/).

To run the unit tests, navigate to the root directory of the package and run the following command:

```bash
python -m unittest discover tests/
```

## Add new dependencies

If you PR involves adding new dependencies, please make sure that the new dependencies are available on both [PyPI](https://pypi.org/) and [conda-forge](https://conda-forge.org/). Search [here](https://conda-forge.org/feedstock-outputs/) to see if the package is available on conda-forge. If the package is not available on conda-forge, it can't be added as a required dependency in `requirements.txt`. Instead, it should be added as an optional dependency in `requirements_dev.txt`.

If the package is available on PyPI and conda-forge, but if it is challenging to install the package on some operating systems, we would recommend adding the package as an optional dependency in `requirements_dev.txt` rather than a required dependency in `requirements.txt`.

The dependencies required for building the documentation should be added to `requirements_docs.txt`. In most cases, contributors do not need to add new dependencies to `requirements_docs.txt` unless the documentation fails to build due to missing dependencies.

---

### Faq (`docs/faq.md`)

# Frequently Asked Questions (FAQ)

## Coordinate Reference Systems (CRS)

### Does samgeo automatically convert my imagery to EPSG:4326?

**No.** samgeo does **NOT** automatically reproject or convert your GeoTIFF files to EPSG:4326. Your imagery stays in its native coordinate system throughout the entire segmentation process.

**Key points:**
- When you load a GeoTIFF with `set_image()` or use `generate_masks_tiled()`, the image data is read directly without any CRS transformation
- The output masks inherit the same CRS as your input imagery
- The `reproject()` function in `samgeo.common` has EPSG:4326 as a default parameter, but it's **only used when you explicitly call that function** - it is NOT called automatically during segmentation

**Why might you see distortion?**
1. Your visualization tool is displaying the results in EPSG:4326 (like some web map libraries)
2. You're manually calling the `reproject()` function somewhere in your workflow
3. Your original imagery already has CRS-related issues

**Recommendation:**
- Verify your input TIF has the correct CRS metadata using `gdalinfo` or rasterio (`src.crs`)
- The output masks will preserve the same CRS as your input
- If you need to reproject for visualization, do it as a separate step AFTER segmentation:
  ```python
  from samgeo import common
  common.reproject("input_masks.tif", "output_epsg4326.tif", dst_crs="EPSG:4326")
  ```

### How can I check the CRS of my GeoTIFF?

Using GDAL:
```bash
gdalinfo your_file.tif
```

Using Python with rasterio:
```python
import rasterio
with rasterio.open("your_file.tif") as src:
    print(f"CRS: {src.crs}")
```

### My imagery looks distorted during segmentation

The segmentation algorithm works on pixel values, not geographic coordinates, so CRS should not affect segmentation quality. If you're seeing distortion:

1. **Check if it's a visualization issue:** The distortion might only appear when viewing results in a different CRS
2. **Verify your input data:** Use `gdalinfo` to check if the CRS metadata is correct
3. **Try reprojecting to a more suitable CRS:** Some projections preserve shape better than others for specific regions (e.g., UTM for local areas)

---

### Installation (`docs/installation.md`)

# Installation

## Install with pixi (Recommended)

Installing **segment-geospatial** with `uv` or `pip` can be challenging on some platforms (especially Windows) due to complicated pytorch/cuda dependencies and numpy version conflicts. [Pixi](https://pixi.prefix.dev/latest) is recommended to avoid these issues, as it provides faster and more reliable dependency resolution than conda or mamba.

### 1) Install Pixi

#### Linux/macOS (bash/zsh)

```bash
curl -fsSL https://pixi.sh/install.sh | sh
```

Close and re-open your terminal (or reload your shell) so `pixi` is on your `PATH`. Then confirm:

```bash
pixi --version
```

#### Windows (PowerShell)

Open **PowerShell** (preferably as a normal user, Admin not required), then run:

```powershell
powershell -ExecutionPolicy Bypass -c "irm -useb https://pixi.sh/install.ps1 | iex"
```

Close and re-open PowerShell, then confirm:

```powershell
pixi --version
```

---

### 2) Create a Pixi project

Navigate to a directory where you want to create the project and run:

```bash
pixi init geo
cd geo
```

---

### 3) Configure `pixi.toml`

Open `pixi.toml` in the `geo` directory and replace its contents with the following depending on your system.

If you have an NVIDIA GPU with CUDA, run `nvidia-smi` to check the CUDA version.

#### For GPU with CUDA 12.x:

```toml
[workspace]
channels = ["https://prefix.dev/conda-forge"]
name = "geo"
platforms = ["linux-64", "win-64"]

[system-requirements]
cuda = "12.0"

[dependencies]
python = "3.12.*"
pytorch-gpu = ">=2.7.1,<3"
segment-geospatial = ">=1.2.0"
sam3 = ">=0.1.0.20251211"
jupyterlab = "*"
ipykernel = "*"
libopenblas = ">=0.3.30"
```

#### For GPU with CUDA 13.x:

```toml
[workspace]
channels = ["https://prefix.dev/conda-forge"]
name = "geo"
platforms = ["linux-64", "win-64"]

[system-requirements]
cuda = "13.0"

[dependencies]
python = "3.12.*"
pytorch-gpu = ">=2.7.1,<3"
segment-geospatial = ">=1.2.0"
sam3 = ">=0.1.0.20251211"
jupyterlab = "*"
ipykernel = "*"
```

#### For CPU:

```toml
[workspace]
channels = ["https://prefix.dev/conda-forge"]
name = "geo"
platforms = ["linux-64", "win-64"]

[dependencies]
python = "3.12.*"
pytorch-cpu = ">=2.7.1,<3"
segment-geospatial = ">=1.2.0"
sam3 = ">=0.1.0.20251211"
jupyterlab = "*"
ipykernel = "*"
libopenblas = ">=0.3.30"
```

---

### 4) Install the environment

From the `geo` folder:

```bash
pixi install
```

This step may take several minutes on first install depending on your internet connection and system.

---

### 5) Verify PyTorch + CUDA

If you have an NVIDIA GPU with CUDA, run the following command to verify the PyTorch + CUDA installation:

```bash
pixi run python -c "import torch; print('PyTorch:', torch.__version__); print('CUDA available:', torch.cuda.is_available()); print('GPU:', (torch.cuda.get_device_name(0) if torch.cuda.is_available() else 'None'))"
```

Expected output should be like this:

-   `PyTorch: 2.7.1` (or higher)
-   `CUDA available: True`
-   `GPU: NVIDIA RTX 4090` (your GPU name)

If CUDA is `False`, check:

-   `nvidia-smi` works in your terminal
-   NVIDIA driver is up to date

---

### 6) Request access to SAM 3 and SAM 3.1 (Optional)

To use SAM 3, you will need to request access by filling out this form on Hugging Face at <https://huggingface.co/facebook/sam3>. To use SAM 3.1, request access at <https://huggingface.co/facebook/sam3.1>. Once your request has been approved, run the following command in the terminal to authenticate:

```bash
pixi run hf auth login
```

After authentication, you can download the SAM 3 model from Hugging Face:

```bash
pixi run hf download facebook/sam3
```

For SAM 3.1:

```bash
pixi run hf download facebook/sam3.1
```

When `segment-geospatial` is installed with a released `sam3` package whose
checkpoint helper only downloads `facebook/sam3`, SAM 3.1 still downloads
`facebook/sam3.1` directly through Hugging Face Hub.

**Important Note**: SAM 3 and SAM 3.1 currently require an NVIDIA GPU with CUDA support. You won't be able to use them if you have a CPU only system ([source](https://github.com/facebookresearch/sam3/issues/164)). You will get an error message like this: `Failed to load model: Torch not compiled with CUDA enabled`.

---

### 7) Start Jupyter Lab

To start using segment-geospatial in Jupyter Lab:

```bash
pixi run jupyter lab
```

This will open Jupyter Lab in your default browser. You can now create a new notebook and start using segment-geospatial!

---

## Install from PyPI

**segment-geospatial** is available on [PyPI](https://pypi.org/project/segment-geospatial/). To install **segment-geospatial**, run this command in your terminal:

```bash
pip install segment-geospatial
```

## Install from conda-forge

**segment-geospatial** is also available on [conda-forge](https://anaconda.org/conda-forge/segment-geospatial). If you have
[Anaconda](https://www.anaconda.com/distribution/#download-section) or [Miniconda](https://docs.conda.io/en/latest/miniconda.html) installed on your computer, you can install segment-geospatial using the following commands. It is recommended to create a fresh conda environment for **segment-geospatial**. The following commands will create a new conda environment named `geo` and install **segment-geospatial** and its dependencies:

```bash
conda create -n geo python
conda activate geo
conda install -c conda-forge mamba
mamba install -c conda-forge segment-geospatial
```

If your system has a GPU, but the above commands do not install the GPU version of pytorch, you can force the installation of the GPU version of pytorch using the following command:

```bash
mamba install -c conda-forge segment-geospatial "pytorch=*=cuda*"
```

Samgeo-geospatial has some optional dependencies that are not included in the default conda environment. To install these dependencies, run the following command:

```bash
mamba install -c conda-forge groundingdino-py segment-anything-fast
```

## Install from GitHub

To install the development version from GitHub using [Git](https://git-scm.com/), run the following command in your terminal:

```bash
pip install git+https://github.com/opengeos/segment-geospatial
```

## Use docker

You can also use [docker](https://hub.docker.com/r/giswqs/segment-geospatial/) to run segment-geospatial:

```bash
docker run -it -p 8888:8888 giswqs/segment-geospatial:latest
```

To enable GPU for segment-geospatial, run the following command to run a short benchmark on your GPU:

```bash
docker run --rm -it --gpus=all nvcr.io/nvidia/k8s/cuda-sample:nbody nbody -gpu -benchmark
```

The output should be similar to the following:

```text
Run "nbody -benchmark [-numbodies=<numBodies>]" to measure performance.
        -fullscreen       (run n-body simulation in fullscreen mode)
        -fp64             (use double precision floating point values for simulation)
        -hostmem          (stores simulation data in host memory)
        -benchmark        (run benchmark to measure performance)
        -numbodies=<N>    (number of bodies (>= 1) to run in simulation)
        -device=<d>       (where d=0,1,2.... for the CUDA device to use)
        -numdevices=<i>   (where i=(number of CUDA devices > 0) to use for simulation)
        -compare          (compares simulation results running once on the default GPU and once on the CPU)
        -cpu              (run n-body simulation on the CPU)
        -tipsy=<file.bin> (load a tipsy model file for simulation)

NOTE: The CUDA Samples are not meant for performance measurements. Results may vary when GPU Boost is enabled.

> Windowed mode
> Simulation data stored in video memory
> Single precision floating point simulation
> 1 Devices used for simulation
GPU Device 0: "Turing" with compute capability 7.5

> Compute 7.5 CUDA device: [Quadro RTX 5000]
49152 bodies, total time for 10 iterations: 69.386 ms
= 348.185 billion interactions per second
= 6963.703 single-precision GFLOP/s at 20 flops per interaction
```

If you encounter the following error:

```text
nvidia-container-cli: initialization error: load library failed: libnvidia-ml.so.1: cannot open shared object file: no such file or directory: unknown.
```

Try adding `sudo` to the command:

```bash
sudo docker run --rm -it --gpus=all nvcr.io/nvidia/k8s/cuda-sample:nbody nbody -gpu -benchmark
```

Once everything is working, you can run the following command to start a Jupyter Notebook server:

```bash
docker run -it -p 8888:8888 --gpus=all giswqs/segment-geospatial:latest
```

---

### Usage (`docs/usage.md`)

# Usage

To use segment-geospatial in a project:

```python
import samgeo
```

Here is a simple example of using segment-geospatial to generate a segmentation mask from a satellite image:

```python
import os
import torch
from samgeo import SamGeo
from samgeo.common import tms_to_geotiff

bbox = [-95.3704, 29.6762, -95.368, 29.6775]
image = 'satellite.tif'
tms_to_geotiff(output=image, bbox=bbox, zoom=20, source='Satellite')

out_dir = os.path.join(os.path.expanduser('~'), 'Downloads')
checkpoint = os.path.join(out_dir, 'sam_vit_h_4b8939.pth')

device = 'cuda' if torch.cuda.is_available() else 'cpu'
sam = SamGeo(
    checkpoint=checkpoint,
    model_type='vit_h',
    device=device,
    erosion_kernel=(3, 3),
    mask_multiplier=255,
    sam_kwargs=None,
)

mask = 'segment.tif'
sam.generate(image, mask)

vector = 'segment.gpkg'
sam.tiff_to_gpkg(mask, vector, simplify_tolerance=None)
```

---

--- METRICS ---
- Files Extracted: 11
- Estimated Token Budget: ~17328 tokens
- Canonical Reference: https://codewiki.google/github.com/opengeos/segment-geospatial