presidio Documentation

Technical Documentation: microsoft/presidio

ℹ️ Provenance: Multi-Tier Ingestion · CodeWiki Reference

Index (`docs/index.md`) Copy Section

<h1 class="dps-hero__title">Presidio</h1>
by Data Privacy Stack

Presidio (Origin from Latin praesidium ‘protection, garrison’)
helps to ensure sensitive data is properly managed and governed.
It provides fast identification and anonymization
modules for private entities in text and images such as
credit card numbers, names, locations, social security numbers,
bitcoin wallets, US phone numbers, financial data and more.

!!! info "Project transition update"
Presidio has transitioned to a community-owned project.
Read the full announcement here.

Goals Copy Chapter

• Allow organizations to preserve privacy in a simpler way by democratizing de-identification technologies and introducing transparency in decisions.
• Embrace extensibility and customizability to a specific business need.
• Facilitate both fully automated and semi-automated PII de-identification flows on multiple platforms.

How it works Copy Chapter

Main features Copy Chapter

1. Predefined or custom PII recognizers leveraging Named Entity Recognition, regular expressions, rule based logic and checksum with relevant context in multiple languages.
2. Options for connecting to external PII detection models.
3. Multiple usage options, from Python or PySpark workloads through Docker to Kubernetes.
4. Customizability in PII identification and anonymization.
5. Module for redacting PII text in images.

!!! warning "Warning"
Presidio can help identify sensitive/PII data in un/structured text. However, because it is using automated detection mechanisms, there is no guarantee that Presidio will find all sensitive information. Consequently, additional systems and protections should be employed.

Demo Copy Chapter

Link to demo: https://huggingface.co/spaces/presidio/presidio_demo

Presidio's modules Copy Chapter

1. Presidio analyzer: PII identification in text
2. Presidio anonymizer: De-identify detected PII entities using different operators
3. Presidio image redactor: Redact PII entities from images using OCR and PII identification
4. Presidio structured: PII identification in structured/semi-structured data

Installing Presidio Copy Chapter

1. Supported Python Versions
2. Using pip
3. Using Docker
4. From source
5. Migrating from V1 to V2

Running Presidio Copy Chapter

1. Samples for running Presidio via code
2. Running Presidio as an HTTP service
3. Setting up a development environment
4. Perform PII identification using presidio-analyzer
5. Perform PII de-identification using presidio-anonymizer
6. Perform PII identification and redaction in images using presidio-image-redactor
7. Example deployments

Support Copy Chapter

• Before you submit an issue, please go over the documentation. For general discussions, please use the GitHub repo's discussion board.
• If you have a usage question, found a bug or have a suggestion for improvement, please file a GitHub issue.
• For other matters, please email [email protected].

Analyzer Index (`docs/analyzer/index.md`) Copy Section

Presidio Analyzer

The Presidio analyzer is a Python based service for detecting PII entities in text.

During analysis, it runs a set of different PII Recognizers,
each one in charge of detecting one or more PII entities using different mechanisms.

Presidio analyzer comes with a set of predefined recognizers,
but can easily be extended with other types of custom recognizers.
Predefined and custom recognizers leverage regex,
Named Entity Recognition and other types of logic to detect PII in unstructured text.

Installation Copy Chapter

see Installing Presidio.

Getting started Copy Chapter

=== "Python"

Once the Presidio-analyzer package is installed, run this simple analysis script:

```python
from presidio_analyzer import AnalyzerEngine

# Set up the engine, loads the NLP module (spaCy model by default) and other PII recognizers
analyzer = AnalyzerEngine()

# Call analyzer to get results
results = analyzer.analyze(text="My phone number is 212-555-5555",
                           entities=["PHONE_NUMBER"],
                           language='en')
print(results)

```

=== "As an HTTP server"

You can run presidio analyzer as an http server using either python runtime or using a docker container.

#### Using docker container

```sh
cd presidio-analyzer
docker run -p 5002:3000 presidio-analyzer
```

#### Using python runtime

!!! note "Note"
    This requires the Presidio Github repository to be cloned.

```sh
cd presidio-analyzer
python app.py
curl -d '{"text":"John Smith drivers license is AC432223", "language":"en"}' -H "Content-Type: application/json" -X POST http://localhost:3000/analyze
```

Main concepts Copy Chapter

Presidio analyzer is a set of tools that are used to detect entities in text. The main object in Presidio Analyzer is the AnalyzerEngine. In the following section we'll describe the main concepts in Presidio Analyzer.

This simplified class diagram shows the main classes in Presidio Analyzer:

/* Detailed source-code truncated for AI context efficiency. */

`RecognizerResult` Copy Section

A RecognizerResult holds the type and span of a PII entity.

`EntityRecognizer` Copy Section

An entity recognizer is an object in Presidio that is responsible for detecting entities in text. An entity recognizer can be a rule-based recognizer, a machine learning model, or a combination of both.

`PatternRecognizer` Copy Section

A PatternRecognizer is a type of entity recognizer that uses regular expressions to detect entities in text. One can create new PatternRecognizer objects by providing a list of regular expressions, context words, validation and invalidation logic and additional parameters that facilitate the detection of entities.

`AnalyzerEngine` Copy Section

The AnalyzerEngine is the main object in Presidio Analyzer that is responsible for detecting entities in text. The AnalyzerEngine can be configured in various ways to fit the specific needs of the user.

`RecognizerRegistry` Copy Section

The RecognizerRegistry is a registry that contains all the entity recognizers that are available in Presidio. The AnalyzerEngine uses the RecognizerRegistry to detect entities in text.

`NlpEngine` Copy Section

An NLP Engine is an object that holds the NLP model that is used by the AnalyzerEngine to parse the input text and extract different features from it, such as tokens, lemmas, entities, and more. Note that Named Entity Recognition (NER) models can be added in two ways to Presidio: One is through the NlpEngine object, and the other is through a new EntityRecognizer object. By creating a Named Entity Recognition model through the NlpEngine, the named entities will be available to the different modules in Presidio. Furthermore, the NlpEngine object supports a batch mode (i.e., processing multiple texts at once) which allows for faster processing of large amounts of text.
It is possible to mix multiple NER models in Presidio, for instance, one model as the NlpEngine and others as additional EntityRecognizer objects.

Presidio has an off-the-shelf support for multiple NLP packages, such as spaCy, stanza, and huggingface. The simplest way to integrate a model from these packages is through the NlpEngine. More information on this can be found in the NlpEngine documentation. The samples gallery has several examples of leveraging NER models as new EntityRecognizer objects. For example, flair and spanmarker.
For a detailed flow of Named Entities within presidio, see the diagram in this document.

`Context Aware Enhancer` Copy Section

The ContextAwareEnhancer is a module that enhances the detection of entities by using the context of the text. The ContextAwareEnhancer can be used to improve the detection of entities that are dependent on the context of the text, such as dates, locations, and more. The default implementation is the LemmaContextAwareEnhancer which uses the lemmas of the tokens in the text to enhance the detection of entities. Note that it's possible (and sometimes recommended) to create custom ContextAwareEnhancer objects to fit the specific needs of the user, for example if the context should support more than one word, which is currently not supported by the default Lemma based enhancer.
More information on this can be found in this sample.

Creating PII recognizers Copy Chapter

Presidio analyzer can be easily extended to support additional PII entities.
See this tutorial on adding new PII recognizers
for more information.

Multi-language support Copy Chapter

Presidio can be used to detect PII entities in multiple languages.
Refer to the multi-language support for more information.

Filtering recognizers by country Copy Chapter

Predefined recognizers in Presidio are tagged with a country_code so
deployments can scope detection to a specific set of countries (e.g.
US-only, EU-only) without manually enumerating recognizer classes. See
the Filtering recognizers by country guide
for usage and the rules for tagging custom recognizers.

Outputting the analyzer decision process Copy Chapter

Presidio analyzer has a built in mechanism for tracing each decision made. This can be useful when attempting to understand a specific PII detection. For more info, see the decision process documentation.

Supported entities Copy Chapter

For a list of the current supported entities:
Supported entities.

API reference Copy Chapter

Follow the API Spec for the Analyzer REST API reference details and Analyzer Python API for Python API reference

Anonymizer Index (`docs/anonymizer/index.md`) Copy Section

Presidio Anonymizer

The Presidio anonymizer is a Python based module
for anonymizing detected PII text entities with desired values.
Presidio anonymizer supports both anonymization and deanonymization
by applying different operators. Operators are built-in text manipulation classes
which can be easily extended.

The Presidio-Anonymizer package contains both Anonymizers and Deanonymizers.

• Anonymizers are used to replace a PII entity text
  with some other value by applying a certain operator (e.g. replace, mask, redact, encrypt)
• Deanonymizers are used to revert the anonymization operation.
  (e.g. to decrypt an encrypted text).

Installation Copy Chapter

see Installing Presidio.

Getting started Copy Chapter

=== "Python"

Simple example:

```python
from presidio_anonymizer import AnonymizerEngine
from presidio_anonymizer.entities import RecognizerResult, OperatorConfig

# Initialize the engine:
engine = AnonymizerEngine()

# Invoke the anonymize function with the text, 
# analyzer results (potentially coming from presidio-analyzer) and
# Operators to get the anonymization output:
result = engine.anonymize(
    text="My name is Bond, James Bond",
    analyzer_results=[
        RecognizerResult(entity_type="PERSON", start=11, end=15, score=0.8),
        RecognizerResult(entity_type="PERSON", start=17, end=27, score=0.8),
    ],
    operators={"PERSON": OperatorConfig("replace", {"new_value": "BIP"})},
)

print(result)

```

This example takes the output of the `AnonymizerEngine` 
containing an encrypted PII entity, and decrypts it back to the original text:

```python
from presidio_anonymizer import DeanonymizeEngine
from presidio_anonymizer.entities import OperatorResult, OperatorConfig

# Initialize the engine:
engine = DeanonymizeEngine()

# Invoke the deanonymize function with the text, anonymizer results and
# Operators to define the deanonymization type.
result = engine.deanonymize(
    text="My name is S184CMt9Drj7QaKQ21JTrpYzghnboTF9pn/neN8JME0=",
    entities=[
        OperatorResult(start=11, end=55, entity_type="PERSON"),
    ],
    operators={"DEFAULT": OperatorConfig("decrypt", {"key": "WmZq4t7w!z%C&F)J"})},
)

print(result)
```

=== "As an HTTP server"

You can run presidio anonymizer as an http server using either python runtime or using a docker container.

#### Using docker container

```sh
cd presidio-anonymizer
docker run -p 5001:3000 presidio-anonymizer 
```

#### Using python runtime

!!! note "Note"
    This requires the Presidio Github repository to be cloned.

```

/* Detailed source-code truncated for AI context efficiency. */

## Main concepts

The following class diagram shows a simplified view of the main classes in Presidio Anonymizer:

/* Detailed source-code truncated for AI context efficiency. */

- The **AnonymizerEngine** is the main class in Presidio that is responsible for anonymizing PII entities in text. It uses the results from the **AnalyzerEngine** to perform the anonymization.
- The **DeanonymizerEngine** is a class in Presidio that is responsible for deanonymizing text that has been anonymized by the **AnonymizerEngine**, given that the operation is reversible (e.g. encryption).
- An **Operator** is an object in Presidio that is responsible for performing the anonymization operation on a PII entity. Presidio provides several built-in operators, such as **Replace**, **Redact**, and **Encrypt**, and allows users to create custom operators.
- The **BatchAnonymizerEngine** is a class in Presidio that is responsible for anonymizing PII entities in a batch of texts. It uses the **AnonymizerEngine** to perform the anonymization on each text in the batch. ([see more here](../samples/python/batch_processing.ipynb)).

## Built-in operators

| Operator type | Operator name | Description                                                         | Parameters                                                                                                                                                                                        |
|---------------|---------------|---------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Anonymize     | replace       | Replace the PII with desired value                                  | `new_value`: replaces existing text with the given value. If `new_value` is not supplied or empty, default behavior will be: <entity_type\> e.g:                               |
| Anonymize     | redact        | Remove the PII completely from text                                 | None                                                                                                                                                                                              |
| Anonymize     | hash          | Hashes the PII text using salted hashing for security              | `hash_type`: sets the type of hashing. Can be either `sha256` or `sha512`. The default is `sha256`.`salt`: Optional salt for reproducible hashing. If not provided, a random salt is generated per entity to prevent brute-force attacks. To maintain referential integrity across records/calls, provide a consistent salt (see example below). |
| Anonymize     | mask          | Replace the PII with a given character                              | `chars_to_mask`: the amount of characters out of the PII that should be replaced.  `masking_char`: the character to be replaced with.  `from_end`: Whether to mask the PII from it's end. |
| Anonymize     | encrypt       | Encrypt the PII using a given key                                   | `key`: a cryptographic key used for the encryption.                                                                                                                                               |
| Anonymize     | custom        | Replace the PII with the result of the function executed on the PII | `lambda`: lambda to execute on the PII data. The lambda return type must be a string.                                                                                                             |
| Anonymize     | surrogate_ahds | Generate realistic, medically-appropriate surrogates using Azure Health Data Services de-identification service surrogation | `endpoint`: AHDS endpoint (optional, uses AHDS_ENDPOINT env var)`entities`: List of entities detected by analyzer`input_locale`: Input locale (default: "en-US")`surrogate_locale`: Surrogate locale (default: "en-US")Requires: `pip install presidio-anonymizer[ahds]` |
| Anonymize     | keep          | Preserver the PII unmodified                                        | None                                                                                                                                                                                              |
| Deanonymize   | decrypt       | Decrypt the encrypted PII in the text using the encryption key      | `key`: a cryptographic key used for the encryption is also used for the decryption.                                                                                                               |

!!! note "Note"
    When performing anonymization, if anonymizers map is empty or "DEFAULT" key is not stated, the default
    anonymization operator is "replace" for all entities. The replacing value will be the entity type
    e.g.: 

### Hash operator with salt for referential integrity

!!! warning "BREAKING CHANGE in Hash Operator"
    **Starting from version 2.2.361, the hash operator uses random salt by default for security.**
    
    **Impact:**
    - Hash outputs are different from previous versions
    - Same PII value gets **different hashes** across different entities and calls (unless you provide a salt)
    - This prevents brute-force and dictionary attacks but breaks backward compatibility
    
    **Action Required:**
    - If you need the same hash for the same value (referential integrity), you **must** explicitly provide a `salt` parameter
    - If you're upgrading from a previous version and need to match existing hashes, this is not possible without the original salt
    
    !!! note: Note
        Presidio does not store or maintain stateful sessions. For referential integrity across records or calls, users must securely manage and provide their own salt.

The hash operator uses random salt by default for maximum security, which means the same PII value will get different hashes in different entities or calls. If you need referential integrity (same values getting the same hash), you must provide a consistent salt:

```python
import secrets
from presidio_anonymizer import AnonymizerEngine
from presidio_anonymizer.entities import RecognizerResult, OperatorConfig

# Generate a secure salt once (store this securely for reuse across calls)
salt = secrets.token_hex(32)  # 64 character hex string (256 bits)

engine = AnonymizerEngine()
text = "John Doe called John Doe yesterday"
analyzer_results = [
    RecognizerResult(start=0, end=8, score=0.8, entity_type="PERSON"),
    RecognizerResult(start=16, end=24, score=0.8, entity_type="PERSON"),
]

# Use the same salt for all anonymizations to maintain referential integrity
result = engine.anonymize(
    text, 
    analyzer_results, 
    {"DEFAULT": OperatorConfig("hash", {"salt": salt})}
)

# Both "John Doe" instances will have the same hash
print(result.text)

!!! warning "Security Considerations"
- Discard the salt after processing: The salt should be discarded once all data is anonymized to prevent re-identification if leaked
- Store salt only if you need to process additional data later for referential integrity (increases re-identification risk)
- If you must store salt, use secure storage (e.g., key vault or secrets manager) with strict access controls
- Use a salt of at least 128 bits (16 bytes) - enforced by the operator
- Never include the salt in anonymized output
- For maximum security without referential integrity needs, omit the salt parameter to use random per-entity salts

Handling overlaps between entities Copy Chapter

As the input text could potentially have overlapping PII entities, there are different
anonymization scenarios:

• No overlap (single PII): When there is no overlap in spans of entities,
  Presidio Anonymizer uses a given or default anonymization operator to anonymize
  and replace the PII text entity.

• Full overlap of PII entity spans: When entities have overlapping substrings,
  the PII with the higher score will be taken.
  Between PIIs with identical scores, the selection is arbitrary.

• One PII is contained in another: Presidio Anonymizer will use the PII with the larger text even if it's score is lower.

• Partial intersection: Presidio Anonymizer will anonymize each individually and will return a concatenation of the anonymized text.
  For example:
  For the text

  text Copy

  I'm George Washington Square Park.

  Assuming one entity is George Washington and the other is Washington State Park
  and assuming we're using the default anonymizer,
  the result would be:

  text Copy

  I'm <PERSON><LOCATION>.

Additional examples for overlapping PII scenarios Copy Section

Text:

My name is Inigo Montoya. You Killed my Father. Prepare to die. BTW my number is:
03-232323.

Results:

• No overlaps: Assuming only Inigo is recognized as NAME:

  text Copy

  My name is <NAME> Montoya. You Killed my Father. Prepare to die. BTW my number is:
  03-232323.

• Full overlap: Assuming the number is recognized as PHONE_NUMBER with score of 0.7 and as SSN
  with score of 0.6, the higher score would count:

  text Copy

  My name is Inigo Montoya. You Killed my Father. Prepare to die. BTW my number is: <
  PHONE_NUMBER>.

• One PII is contained is another: Assuming Inigo is recognized as FIRST_NAME and Inigo Montoya
  was recognized as NAME, the larger one will be used:

  text Copy

  My name is <NAME>. You Killed my Father. Prepare to die. BTW my number is: 03-232323.

• Partial intersection: Assuming the number 03-2323 is recognized as a PHONE_NUMBER but 232323
  is recognized as SSN:

  text Copy

  My name is Inigo Montoya. You Killed my Father. Prepare to die. BTW my number is: <
  PHONE_NUMBER><SSN>.

Creating a new `operator` Copy Chapter

Presidio anonymizer can be easily extended to support additional operators.
See this tutorial on adding new operators
for more information.

API reference Copy Chapter

Follow
the API Spec
for the Anonymizer REST API reference details
and Anonymizer Python API for Python API reference.

Evaluation Index (`docs/evaluation/index.md`) Copy Section

Evaluating PII detection with Presidio

Why evaluate PII detection? Copy Chapter

No de-identification system is perfect.
It is important to evaluate the performance of a PII detection system for your specific use case.
This evaluation can help you understand where the system makes mistakes and how to iteratively improve the detection mechanisms,
which recognizers and models to use, and how to configure them.

Common evaluation metrics Copy Chapter

The most common evaluation metrics are precision, recall, and Fβ score, which is a combination of precision and recall.
These metrics are calculated based on the number of true positives, false positives, and false negatives.
For every use case, the false positive and false negative rates should be balanced to achieve the desired level of accuracy.

• Precision measures the proportion of true positive results among the positive results: TP / (TP + FP).
• Recall measures the proportion of true positive results among the actual positives: TP / (TP + FN).
• Fβ score is a weighted harmonic mean of precision and recall: (1 + β^2) * (precision * recall) / (β^2 * precision + recall).

Click here for more definitions.

!!! note "Note"
In PII detection, recall is often more important than precision, as we'd like to avoid missing any PII.
In such cases, we recommend to use the β=2 score, which gives more importance to recall.

How to evaluate PII detection with Presidio Copy Chapter

Presidio provides a set of tools to evaluate the performance of the PII detection system.
In addition, it provides simple data generation tools to help you create a dataset for evaluation.

Evaluating the Presidio Analyzer using Presidio-Research Copy Section

Presidio-Research is a python package with a set of tools that help you evaluate the performance of the Presidio Analyzer.
To get started, follow the instructions in the Presidio-Research repository.

The easiest way to get started is by reviewing the notebooks:

• Notebook 1: Shows how to use the PII data generator.
• Notebook 2: Shows a simple analysis of the PII dataset.
• Notebook 3: Provides tools to split the dataset into train/test/validation sets while avoiding leakage due to the same pattern appearing in multiple folds (only applicable for synthetically generated data).
• Notebook 4: Shows how to use the evaluation tools to evaluate how well Presidio detects PII. Note that this is using the vanilla Presidio, and the results aren't very accurate.
• Notebook 5: Shows how one can configure Presidio to detect PII much more accurately, and boost the f score in ~30%.

For more information and advanced usage, refer to the Presidio-Research repository.

Evaluating DICOM redaction with Presidio Image Redactor Copy Section

See Evaluating DICOM redaction for more information.
For a full demonstration, see the evaluation notebook.

Image Redactor Index (`docs/image-redactor/index.md`) Copy Section

Presidio Image Redactor

Please notice, this package is still in beta and not production ready.

Description Copy Chapter

The Presidio Image Redactor is a Python based module for detecting and redacting PII
text entities in images.

This module may also be used on medical DICOM images. The DicomImageRedactorEngine class may be used to redact text PII present as pixels in DICOM images.

!!! note "Note"
This class only redacts pixel data and does not scrub text PII which may exist in the DICOM metadata.
We highly recommend using the DICOM image redactor engine to redact text from images BEFORE scrubbing metadata PII.*

Installation Copy Chapter

Pre-requisites:

• Install Tesseract OCR by following the
  instructions on how to install it for your operating system.

!!! attention "Attention"
For best performance, please use the most up-to-date version of Tesseract OCR. Presidio was tested with v5.2.0.

=== "Using pip"

!!! note "Note"
    Consider installing the Presidio python packages on a virtual environment like venv or conda.

To get started with Presidio-image-redactor,
download the package and the `en_core_web_lg` spaCy model:

```sh
pip install presidio-image-redactor
python -m spacy download en_core_web_lg
```

=== "Using Docker"

!!! note "Note"
    This requires Docker to be installed. [Download Docker](https://docs.docker.com/get-docker/).

```sh
# Download image from GitHub Container Registry
docker pull ghcr.io/data-privacy-stack/presidio-image-redactor

# Run the container with the default port
docker run -d -p 5003:3000 ghcr.io/data-privacy-stack/presidio-image-redactor:latest
```

=== "From source"

First, clone the Presidio repo. [See here for instructions](../installation.md#install-from-source).

Then, build the presidio-image-redactor container:

```sh
cd presidio-image-redactor
docker build . -t presidio/presidio-image-redactor
```

Getting started (standard image types) Copy Chapter

=== "Python"

Once the Presidio-image-redactor package is installed, run this simple script:

```python
from PIL import Image
from presidio_image_redactor import ImageRedactorEngine

# Get the image to redact using PIL lib (pillow)
image = Image.open("./docs/image-redactor/ocr_text.png")

# Initialize the engine
engine = ImageRedactorEngine()

# Redact the image with pink color
redacted_image = engine.redact(image, (255, 192, 203))

# Optional: Redact the image and return redacted regions
redacted_image, bboxes = engine.redact_and_return_bbox(image, (255, 192, 203))

# save the redacted image 
redacted_image.save("new_image.png")
# uncomment to open the image for viewing
# redacted_image.show()

```

=== "As an HTTP server"

You can run presidio image redactor as an http server using either python runtime or using a docker container.

#### Using docker container

```sh
cd presidio-image-redactor
docker run -p 5003:3000 presidio-image-redactor 
```

#### Using python runtime

!!! note "Note"
    This requires the Presidio Github repository to be cloned.

```sh
cd presidio-image-redactor
python app.py
# use ocr_test.png as the image to redact, and 255 as the color fill. 
# out.png is the new redacted image received from the server.
curl -XPOST "http://localhost:3000/redact" -H "content-type: multipart/form-data" -F "image=@ocr_test.png" -F "data=\"{'color_fill':'255'}\"" > out.png
```

Python script example can be found under:
/presidio/e2e-tests/tests/test_image_redactor.py

Getting started (DICOM images) Copy Chapter

=== "Python"

Once the Presidio-image-redactor package is installed, run this simple script:

```python
import pydicom
from presidio_image_redactor import DicomImageRedactorEngine

# Set input and output paths
input_path = "path/to/your/dicom/file.dcm"
output_dir = "./output"

# Initialize the engine
engine = DicomImageRedactorEngine()

# Option 1: Redact from a loaded DICOM image
dicom_image = pydicom.dcmread(input_path)
redacted_dicom_image = engine.redact(dicom_image, fill="contrast")

# Option 2: Redact from a loaded DICOM image and return redacted regions
redacted_dicom_image, bboxes = engine.redact_and_return_bbox(dicom_image, fill="contrast")

# Option 3: Redact from DICOM file and save redacted regions as json file
engine.redact_from_file(input_path, output_dir, padding_width=25, fill="contrast", save_bboxes=True)

# Option 4: Redact from directory and save redacted regions as json files
ocr_kwargs = {"ocr_threshold": 50}
engine.redact_from_directory("path/to/your/dicom", output_dir, fill="background", save_bboxes=True, ocr_kwargs=ocr_kwargs)
```

Getting started using the document intelligence OCR engine Copy Chapter

Presidio offers two engines for OCR based PII removal. The first is the default engine which uses Tesseract OCR. The second is the Document Intelligence OCR engine which uses Azure's Document Intelligence service, which requires an Azure subscription. The following sections describe how to setup and use the Document Intelligence OCR engine.

You will need to register with Azure to get an API key and endpoint. Perform the steps in the "Prerequisites" section of this page. Once your resource deploys, copy your endpoint and key values and save them for the next step.

The most basic usage of the engine can be setup like the following in python

diOCR = DocumentIntelligenceOCR(endpoint="<your_endpoint>", key="<your_key>")

If your environment uses Azure Identity instead of API keys, pass an Azure SDK
credential that implements TokenCredential, such as DefaultAzureCredential.
For example, install azure-identity and use DefaultAzureCredential:

from azure.identity import DefaultAzureCredential

diOCR = DocumentIntelligenceOCR(
    endpoint="<your_endpoint>",
    credential=DefaultAzureCredential(),
)

The DocumentIntelligenceOCR can also attempt to pull your endpoint and key values from environment variables.

export DOCUMENT_INTELLIGENCE_ENDPOINT=<your_endpoint>
export DOCUMENT_INTELLIGENCE_KEY=<your_key>

Document Intelligence Model Support Copy Section

There are numerous document processing models available, and currently we only support the most basic usage of the model. For an overview of the functionalities offered by Document Intelligence, see this page. Presidio offers only word-level processing on the result for PII redaction purposes, as all prebuilt document models support this interface. Different models support additional structured support for tables, paragraphs, key-value pairs, fields and other types of metadata in the response.

Additional metadata can be sent to the Document Intelligence API call, such as pages, locale, and features, which are documented here. You are encouraged to test each model to see which fits best to your use case.

Creating an image redactor engine in Python

diOCR = DocumentIntelligenceOCR()
ia_engine = ImageAnalyzerEngine(ocr=diOCR)
my_engine = ImageRedactorEngine(image_analyzer_engine=ia_engine)

Testing Document Intelligence

Follow the steps of running the tests

The test suite has a series of tests which are only exercised when the appropriate environment variables are populated. To run the test suite, to test the DocumentIntelligenceOCR engine, call the tests like this:

export DOCUMENT_INTELLIGENCE_ENDPOINT=<your_endpoint>
export DOCUMENT_INTELLIGENCE_KEY=<your_key>
pytest

Evaluating de-identification performance Copy Section

If you are interested in evaluating the performance of the DICOM de-identification against ground truth labels, please see the evaluating DICOM de-identification page.

Side note for Windows Copy Section

If you are using a Windows machine, you may run into issues if file paths are too long. Unfortunately, this is not rare when working with DICOM images that are often nested in directories with descriptive names.

To avoid errors where the code may not recognize a path as existing due to the length of the characters in the file path, please enable long paths on your system.

DICOM Data Citation Copy Section

The DICOM data used for unit and integration testing for DicomImageRedactorEngine are stored in this repository with permission from the original dataset owners. Please see the dataset information as follows:

Rutherford, M., Mun, S.K., Levine, B., Bennett, W.C., Smith, K., Farmer, P., Jarosz, J., Wagner, U., Farahani, K., Prior, F. (2021). A DICOM dataset for evaluation of medical image de-identification (Pseudo-PHI-DICOM-Data) [Data set]. The Cancer Imaging Archive. DOI: https://doi.org/10.7937/s17z-r072

API reference Copy Chapter

the API Spec
for the Image Redactor REST API reference details
and Image Redactor Python API for Python API reference

Learn Presidio Index (`docs/learn_presidio/index.md`) Copy Section

Learn Presidio

Presidio is a suite of tools for detecting and de-identifying PII in text, images, and structured data.

The recommended place to start is to follow the tutorial which will guide you through the process of setting up and using Presidio.
To learn about the different concepts in Presidio, visit the concepts page.

To go deeper into each component, visit the relevant docs:

• For the Presidio Analyzer, visit the Analyzer docs.
• For Presidio Anonymizer, visit the Anonymizer docs.
• For Presidio Image Redactor, visit the Image Redactor docs.
• For Presidio structured, visit the Structured docs.

The following diagrams provide a high level understanding of the Presidio components:

Analyzer Copy Chapter

Anonymizer Copy Chapter

Image Redactor Copy Chapter

Standard Image Types Copy Section

DICOM Images Copy Section

Recipes Index (`docs/recipes/index.md`) Copy Section

Recipes Gallery

Welcome to the Presidio Recipes Gallery! This section provides curated, end-to-end examples demonstrating how to customize Presidio for specific data privacy and de-identification scenarios.

What are Recipes? Copy Chapter

Recipes are comprehensive, reproducible examples tailored to common data domains and use cases. Each recipe goes beyond basic documentation to provide:

1. Real-world Context: Focused on specific scenarios like financial chatbot conversations, clinical notes, REST API logs, or multilingual content
2. Synthetic Data Generation: Methods for creating realistic test data that mimics your production environment
3. Performance Benchmarks: Evaluation metrics (precision, recall, F₂ score, and latency) across different Presidio configurations
4. Progressive Complexity: Examples ranging from out-of-the-box usage to advanced customization with transformers, LLMs, or hybrid approaches

Why Use Recipes? Copy Chapter

While Presidio's documentation covers the fundamentals, recipes bridge the gap between generic examples and production-ready implementations. They help you:

• Evaluate Performance: Understand Presidio's accuracy and speed for your specific domain before deployment
• Customize Effectively: Learn which recognizers, models, and configurations work best for different data types
• Compare Approaches: See side-by-side comparisons of different implementation strategies
• Reduce Development Time: Start with a working example close to your use case instead of building from scratch

Recipe Structure Copy Chapter

Each recipe typically includes:

• Scenario Description: The domain and data type
• Data Synthesis: Methods for generating test data using Presidio Evaluator or custom methods
• Configuration: Presidio setup with any custom recognizers or models
• Evaluation: Performance metrics (precision, recall, F₂ score, latency)
• Implementation: Jupyter notebook or Python scripts showing the end-to-end flow (see example)
• Key Findings: When to use this approach and trade-offs to consider

For complex flows, consider breaking into multiple notebooks or scripts for better organization.

Available Recipes Copy Chapter

Currently, the recipes gallery is being built. Check back soon for recipes covering:

• Financial Domain: Chat conversations, transaction logs, customer service interactions
• Healthcare Domain: Clinical notes, patient records, medical reports
• Retail/E-commerce: Customer data, order information, support tickets
• Enterprise: REST API logs, database exports, internal communications
• Multilingual: Examples for Spanish, French, German, and other languages

Recipe Performance Table (Coming Soon) Copy Chapter

We're developing a comprehensive benchmark table that will show Presidio's performance across different domains and implementation levels. The table will include:

Each cell will contain:

• P = Precision
• R = Recall
• F₂ = F₂ score (recall-weighted F-score)
• Latency = Average processing time per sample (milliseconds)
• Notebook = Link to interactive Jupyter notebook

How to Use a Recipe Copy Chapter

1. Browse the recipes to find one matching your domain or use case
2. Review the notebook to understand the approach and results
3. Run the notebook in your environment to reproduce the results
4. Adapt the configuration to your specific data and requirements
5. Evaluate performance on your own test dataset
6. Deploy the configuration that best meets your accuracy and performance needs

Contributing a Recipe Copy Chapter

We welcome community contributions! See our contribution guidelines for details.

Reference Examples:

• Evaluate Presidio Analyzer - Complete end-to-end evaluation workflow
• Generate Synthetic Data - Presidio Evaluator data generator

Follow the pattern: Data Synthesis → Configuration → Evaluation

For complex flows, break into multiple notebooks or scripts. Focus on getting working code first - we'll help refine documentation during review.

Related Resources Copy Chapter

• Presidio Samples: Additional usage examples and integration patterns
• Tutorial Series: Step-by-step guide to Presidio features
• Best Practices for Developing Recognizers: Deep dive into creating custom PII recognizers
• Presidio Research Repository: Evaluation tools and research datasets
• FAQ: Common questions about improving detection accuracy

Questions or Feedback? Copy Chapter

If you have questions about recipes or suggestions for new scenarios to cover, please:

• Open an issue on GitHub
• Email us at [email protected]
• Join the discussion in our community channels

Note: The recipes gallery demonstrates Presidio's flexibility and customization capabilities. The goal is to show that Presidio is designed to be adapted to your specific needs, not used as a one-size-fits-all solution. Each recipe illustrates best practices for customization in different contexts.

Samples Index (`docs/samples/index.md`) Copy Section

Samples

Structured Index (`docs/structured/index.md`) Copy Section

Presidio structured

Description Copy Chapter

The Presidio structured package is a flexible and customizable framework designed to identify and protect structured sensitive data.

This tool extends the capabilities of Presidio, focusing on structured data formats such as tabular formats and semi-structured formats (JSON).
It leverages the detection capabilities of Presidio-Analyzer to identify columns or keys containing personally identifiable information (PII), and establishes a mapping between these column/keys names and the detected PII entities.

Following the detection, Presidio-Anonymizer is used to apply de-identification techniques
to each value in columns identified as containing PII, ensuring the sensitive data is appropriately protected.

Note that sensitive data might not be automatically detected in some cases. Consequently, additional systems and protections should be employed.

Installation Copy Chapter

As a python package Copy Section

To install the presidio-structured package, run the following command:

pip install presidio-structured

Getting started

Example 1: Anonymizing DataFrames

import pandas as pd
from presidio_structured import StructuredEngine, PandasAnalysisBuilder
from presidio_anonymizer.entities import OperatorConfig
from faker import Faker # optionally using faker as an example

# Initialize the engine with a Pandas data processor (default)
pandas_engine = StructuredEngine()

# Create a sample DataFrame
sample_df = pd.DataFrame({'name': ['John Doe', 'Jane Smith'], 'email': ['[email protected]', '[email protected]']})

# Generate a tabular analysis which describes PII entities in the DataFrame.
tabular_analysis = PandasAnalysisBuilder().generate_analysis(sample_df)

# Define anonymization operators
fake = Faker()
operators = {
    "PERSON": OperatorConfig("replace", {"new_value": "REDACTED"}),
    "EMAIL_ADDRESS": OperatorConfig("custom", {"lambda": lambda x: fake.safe_email()})
}

# Anonymize DataFrame
anonymized_df = pandas_engine.anonymize(sample_df, tabular_analysis, operators=operators)
print(anonymized_df)

Example 2: Anonymizing JSON Data

/* Detailed source-code truncated for AI context efficiency. */

A more detailed sample can be found here:

• https://github.com/data-privacy-stack/presidio/blob/main/docs/samples/python/example_structured.ipynb

Selection Strategy for Entity Detection in Tabular Data

• Most Common (default): Identifies the most frequently occurring PII entity in a data column or field.
• Highest Confidence: Selects PII entities based on the highest confidence scores, irrespective of their occurrence frequency.
• Mixed: Combines the strengths of both the above strategies. It selects the entity with the highest confidence score if that score exceeds a specified threshold (controlled by mixed_strategy_threshold); otherwise, it defaults to the most common entity.

Usage

Specify the selection_strategy and optionally the mixed_strategy_threshold in the generate_analysis() method:

# Generate a tabular analysis using the most common strategy
tabular_analysis = PandasAnalysisBuilder().generate_analysis(sample_df)

# Generate a tabular analysis using the highest confidence strategy
tabular_analysis = PandasAnalysisBuilder().generate_analysis(sample_df, selection_strategy="highest_confidence")

# Generate a tabular analysis using the mixed strategy
tabular_analysis = PandasAnalysisBuilder().generate_analysis(sample_df, selection_strategy="mixed", mixed_strategy_threshold=0.75)

Future work

• Improve support for datasets with mixed free-text and structure data (e.g. some columns contain free text)
• Add support for the detection of sensitive column names
• PySpark implementation
• Integration of additional anonymization techniques such as K-Anonymity and Differential Privacy.

Contributions are welcome! Please refer to the Contributing Guide.

More information

• API documentation
• Sample code
• Join the discussion
• Relevant issues on GitHub

Tutorial Index (`docs/tutorial/index.md`) Copy Section

Tutorial: Customization in Presidio

This tutorial covers different customization use cases to:

1. Adapt Presidio to detect new types of PII entities.
2. Adapt Presidio to detect PII entities in a new language.
3. Embed new types of detection modules into Presidio, to improve the coverage of the service.
4. Operate on identified entities: simple de-identification, custom operators and encryption.

Table of contents Copy Chapter

• Getting started
• Deny-list based recognizers
• Regex based PII recognition
• Rule based logic recognizer
• Supporting new models and languages
• Calling an external service for PII detection
• Using context words
• Tracing the decision process
• Loading recognizers from file
• Ad-Hoc recognizers
• Simple anonymization
• Custom anonymization
• Encryption/Decryption

Recipes German Language Support README (`docs/recipes/german-language-support/README.md`) Copy Section

German Language Support

Domain: Healthcare, Legal, Finance, General
Data Type: German-language free text (medical documents, contracts, invoices, ID documents)
Goal: Detect German PII and sensitive identifiers using Presidio's built-in German recognizers with a bilingual spaCy NLP engine

Overview Copy Chapter

Domain: Healthcare / Legal / Finance
Data Type: German-language documents
Goal: Detect German PII — including healthcare identifiers, tax numbers, official document numbers, and vehicle plates — using the German spaCy model alongside the English model so that bilingual (EN + DE) documents are handled correctly.

This recipe provides:

• spacy_en_de.yaml — a ready-to-use NLP engine configuration that loads both en_core_web_lg and de_core_news_md
• An overview of all German-specific recognizers available in Presidio

Quick Start Copy Chapter

Prerequisites Copy Section

pip install presidio-analyzer presidio-anonymizer
python -m spacy download en_core_web_lg
python -m spacy download de_core_news_md

Sample Data Copy Section

sample_text = """
Sehr geehrter Herr Müller,

Ihre Krankenversicherungsnummer (KVNR): A123456780
Steuer-IdNr.: 86095742719
Arztnummer (LANR): 123456601
Betriebsstättennummer (BSNR): 021234568
USt-IdNr.: DE136695976
Führerscheinnummer: BO12345678A
Reisepassnummer: C01234565
"""

Basic Configuration Copy Section

from presidio_analyzer import AnalyzerEngine
from presidio_analyzer.nlp_engine import NlpEngineProvider
from presidio_anonymizer import AnonymizerEngine

# Load the bilingual EN + DE spaCy configuration
provider = NlpEngineProvider(conf_file="spacy_en_de.yaml")
nlp_engine = provider.create_engine()

analyzer = AnalyzerEngine(
    nlp_engine=nlp_engine,
    supported_languages=["en", "de"],
)
anonymizer = AnonymizerEngine()

# Analyze German text
results = analyzer.analyze(text=sample_text, language="de")
anonymized = anonymizer.anonymize(text=sample_text, analyzer_results=results)

print(anonymized.text)

Approach Copy Chapter

Presidio ships pattern-based recognizers for 13 German entity types (see table below).
Each recognizer targets a single entity, uses \b-anchored regex patterns with
base confidence between 0.2 and 0.5, and relies on:

1. Context words (German terminology near the match) to boost confidence
2. Check digit validation where the official specification defines one
   (KVNR, Rentenversicherungsnummer, Steuer-IdNr., LANR)

The spaCy de_core_news_md model adds named-entity recognition for PERSON,
LOCATION, and ORGANIZATION on top of the pattern recognizers.

Supported German Entities Copy Section

Results Copy Chapter

Formal evaluation against a labelled German dataset has not yet been performed.
To benchmark this recipe follow the Presidio Research evaluation workflow:

1. Generate synthetic German text with the data generator
2. Configure the analyzer with spacy_en_de.yaml
3. Run the evaluator and report precision / recall / F₂ / latency

Precision: TBD
Recall: TBD
F₂ Score: TBD
Latency: TBD

Key Findings Copy Section

• Recognizers with check digit validation (KVNR, RVNR, Steuer-IdNr., LANR) achieve
  very low false-positive rates even on ambiguous digit strings.
• Recognizers without a checksum (BSNR, PLZ, KFZ) rely heavily on context words;
  setting score_threshold=0.5 when no context is present is recommended.
• The DE_PLZ (postal code) and DE_KFZ (vehicle plate) recognizers overlap with
  generic patterns; use the entities parameter to restrict detection when only
  specific entity types are needed.

Tips for Others Copy Chapter

• Set score_threshold to 0.4–0.5 for production use to filter out low-confidence
  pattern-only matches from context-free digit strings.
• Use the entities parameter to limit detection to the entity types relevant to
  your domain (e.g. only healthcare identifiers in clinical notes).
• Pre-2013 Führerschein numbers use locally defined, non-standardized formats that
  are not covered by DE_FUEHRERSCHEIN; handle them with a custom recognizer if needed.
• DE_TELEMATIK_ID was evaluated and rejected as a generic recognizer: the format
  (\d{1,2}-) is too ambiguous for reliable free-text detection.

Author: MvdB
Date: 2026-03-19

Samples Deployments Index (`docs/samples/deployments/index.md`) Copy Section

Example deployments

• Azure App Service
• Kubernetes
• Spark/Azure Databricks
• Azure Data Factory
• Data Protection toolkit for OpenAI

Samples Docker Index (`docs/samples/docker/index.md`) Copy Section

Using Presidio in Docker

Description Copy Chapter

Presidio can expose REST endpoints for each service using Flask and Docker. Follow the installation guide to learn how to install and run presidio-analyzer and presidio-anonymizer using docker.

Postman collection Copy Chapter

This repository contains a postman collection with sample REST API request for each service. Follow this tutorial to learn how to export the sample requests into postman

1. Download Presidio Analyzer postman requests
2. Download Presidio Anonymizer postman requests

Sample API Calls Copy Chapter

Simple Text Analysis Copy Section

curl -X POST http://localhost:5002/analyze -H "Content-type: application/json" --data "{ \"text\": \"John Smith drivers license is AC432223\", \"language\" : \"en\"}"

Simple Text Anonymization Copy Section

curl -X POST http://localhost:5001/anonymize -H "Content-type: application/json" --data "{\"text\": \"hello world, my name is Jane Doe. My number is: 034453334\", \"analyzer_results\": [{\"start\": 24, \"end\": 32, \"score\": 0.8, \"entity_type\": \"NAME\"}, { \"start\": 48, \"end\": 57,  \"score\": 0.95,\"entity_type\": \"PHONE_NUMBER\" }],  \"anonymizers\": {\"DEFAULT\": { \"type\": \"replace\", \"new_value\": \"ANONYMIZED\" },\"PHONE_NUMBER\": { \"type\": \"mask\", \"masking_char\": \"*\", \"chars_to_mask\": 4, \"from_end\": true }}}"

Samples Fabric Index (`docs/samples/fabric/index.md`) Copy Section

Presidio with Fabric

This folder contains guides and samples for running Presidio in Microsoft Fabric notebooks with Spark for scalable PII detection and anonymization.

Contents Copy Chapter

1. Environment Setup - How to set up your Fabric environment with the required dependencies.
2. Presidio and Spark Notebook - The sample notebook demonstrating PII detection and anonymization using Presidio with Spark.
3. Sample Data - Example CSV data for testing the PII detection and anonymization workflow.

Overview Copy Chapter

Microsoft Fabric provides a unified analytics platform, and these samples demonstrate how to leverage Presidio's PII detection and anonymization capabilities at scale using Fabric's Spark processing. The integration enables data engineers and analysts to:

• Detect PII in large datasets using distributed Spark processing
• Anonymize sensitive information in a scalable manner
• Enhance data privacy compliance for analytics workloads

What You'll Accomplish Copy Chapter

By the end of this guide, you'll have a working Presidio implementation in Fabric that can:

• Automatically detect various types of PII in your data at scale
• Transform sensitive data while preserving its analytical value
• Scale to handle larger datasets through Spark's distributed processing
• Integrate with your existing Fabric data workflows

You can easily incorporate this solution into your Fabric pipelines for scheduled runs and integrated workflows, or extend it with custom detection rules and anonymization methods.

Results Preview Copy Chapter

Here's what you'll achieve when running the sample notebook:

PII Detection Results Copy Section

Quickly identify what types of sensitive information exist in your data, helping you understand your privacy exposure:

Anonymized Data Copy Section

Transform your data by masking or replacing sensitive information while keeping its structure intact:

Scale Testing Copy Section

See how the solution performs with larger datasets— for understanding how it will handle your production workloads:

Delta Table Output Copy Section

Save your anonymized data directly to a Delta table, making it immediately available for downstream analytics while maintaining privacy:

Samples Deployments App Service Index (`docs/samples/deployments/app-service/index.md`) Copy Section

Deploy presidio services to an Azure App Service

Presidio containers can be hosted on an Azure App Service.
Azure App Service provides a managed production environment, which supports docker containers and devops optimizations. It is a global scale service with built in security and compliance features that fits multiple cloud workloads. The presidio team uses Azure App Service for both its development environment and the presidio demo website.

Deploy Presidio services to Azure Copy Chapter

Use the following button to deploy presidio services to your Azure subscription.

Deploy using command-line script Copy Chapter

The following script can be used alternatively to the ARM template deployment above. It sets up the same components which are required for each of the presidio services (analyzer and anonymizer) as the template.

Basic setup Copy Chapter

RESOURCE_GROUP=<resource group name>
APP_SERVICE_NAME=<name of app service>
LOCATION=<location>
APP_SERVICE_SKU=<sku>

IMAGE_NAME=ghcr.io/data-privacy-stack/presidio-analyzer:latest
# the following parameters are only required if you build and deploy your own containers from a private registry
ACR_USER_NAME=<user name>
ACR_USER_PASSWORD=<password>

# create the resource group
az group create --name $RESOURCE_GROUP
# create the app service plan
az appservice plan create --name $APP_SERVICE_NAME-plan --resource-group $RESOURCE_GROUP  \
--is-linux --location $LOCATION --sku $APP_SERVICE_SKU
# create the web app using the official presidio images
az webapp create --name $APP_SERVICE_NAME --plan $APP_SERVICE_NAME-plan \
--resource-group $RESOURCE_GROUP -i $IMAGE_NAME

# or alternatively, if building presidio and deploying from a private container registry
az webapp create --name $APP_SERVICE_NAME --plan $APP_SERVICE_NAME-plan \
--resource-group $RESOURCE_GROUP -i $IMAGE_NAME -s $ACR_USER_NAME -w $ACR_USER_PASSWORD

Blocking network access Copy Chapter

Use the following script to restrict network access for a specific ip such as your computer, a front-end website or an API management.

FRONT_END_IP_RANGE=[front end ip range]
az webapp config access-restriction add --resource-group $RESOURCE_GROUP --name $APP_SERVICE_NAME \
  --rule-name 'Front-end allow rule' --action Allow --ip-address $FRONT_END_IP_RANGE --priority 100

Further network isolation, using virtual networks, is possible using an Isolated tier of Azure App Service.

Configure App Service Logging Copy Chapter

Logging to the App Service File System Copy Section

az webapp log config --name $APP_SERVICE_NAME --resource-group $RESOURCE_GROUP \
--application-logging filesystem --detailed-error-messages true \
--docker-container-logging filesystem --level information

Logging to Log Analytics Workspace Copy Section

LOG_ANALYTICS_WORKSPACE_RESROUCE_GROUP=<resource group of log analytics>
LOG_ANALYTICS_WORKSPACE_NAME=<log analytics name>

# create a log analytics workspace
az monitor log-analytics workspace create --resource-group $LOG_ANALYTICS_WORKSPACE_RESROUCE_GROUP --workspace-name $LOG_ANALYTICS_WORKSPACE_NAME

# query the log analytics workspace id
LOG_ANALYTICS_WORKSPACE_ID=$(az monitor log-analytics workspace show --resource-group $LOG_ANALYTICS_WORKSPACE_RESROUCE_GROUP --workspace-name $LOG_ANALYTICS_WORKSPACE_NAME --query id -o tsv)
# query the app service id
APP_SERVICE_ID=$(az monitor log-analytics workspace show --resource-group $RESOURCE_GROUP --name $APP_SERVICE_NAME --query id -o tsv)

# create the diagnostics settings
az monitor diagnostic-settings create --name $APP_SERVICE_NAME-diagnostics --resource /
$APP_SERVICE_ID --logs   '[{"category": "AppServicePlatformLogs","enabled": true}, {"category": "AppServiceConsoleLogs", "enabled": true}]' --metrics '[{"category": "AllMetrics","enabled": true}]' --workspace $LOG_ANALYTICS_WORKSPACE_ID

Using an ARM template Copy Chapter

Alternatively, you can use the provided ARM template which can deploy either both or any of the presidio services.
Note that while Log Analytics integration with Azure App Service is in preview, the ARM template deployment will not create a Log Analytics resource or configure the diagnostics settings from the App Service to a Log Analytics workspace.
To deploy the app services using the provided ARM template, fill in the provided values.json file with the required values and run the following script.

az deployment group create --resource-group $RESOURCE_GROUP --template-file presidio-services.json --parameters @values.json

Samples Deployments Data Factory Index (`docs/samples/deployments/data-factory/index.md`) Copy Section

Anonymize PII entities with Azure Data Factory

You can build data anonymization ETL pipelines using Azure Data Factory (ADF) and Presidio.
This section provides instructions on how to leverage presidio in Azure Data Factory:

1. Complete samples - Setup Azure Data Factory and Presidio in a single deployment and run the samples.
2. Using Azure Data Factory Template Gallery to anonymize text files - If you already have an instance of Azure Data Factory and would like to use the built in data-anonymization-with-presidio template to anonymize text files.
3. Using Azure Data Factory Template Gallery to anonymize datasets - If you already have an instance of Azure Data Factory and would like to use the built in data-anonymization-with-presidio template to anonymize csv datasets.

Samples Deployments K8s Index (`docs/samples/deployments/k8s/index.md`) Copy Section

Deploy presidio to Kubernetes

You can install Presidio locally using KIND, as a service in Kubernetes or AKS.

• Deploy locally using KIND
• Deploy with Kubernetes
  • Prerequisites
  • Step by Step Deployment with customizable parameters

Deploy locally with KIND Copy Chapter

KIND (Kubernetes IN Docker).

1. Install Docker.

2. Clone Presidio.

3. Run the following script, which will use KIND (Kubernetes emulation in Docker)

   sh Copy

   cd docs/samples/deployments/k8s/deployment/
   ./run-with-kind.sh

4. Wait and verify all pods are running:

   sh Copy

   kubectl get pod -n presidio

5. Port forwarding of HTTP requests to the API micro-service will be done automatically. In order to run manual:

   sh Copy

   kubectl port-forward <presidio-analyzer-pod-name> 8080:8080 -n presidio

Presidio As a Service with Kubernetes Copy Chapter

Prerequisites Copy Section

1. A Kubernetes 1.18+ cluster with RBAC enabled. If you are using AKS RBAC is enabled by default.

   !!! note: Note
   Note the pod's resource requirements (CPU and memory) and plan the cluster accordingly.

2. kubectl installed. Verify you can communicate with the cluster by running:

   sh Copy

   kubectl version

3. Local helm client.

4. Optional - Container Registry - such as ACR. Only needed if you are using your own Presidio images and not the default ones from GitHub Container Registry.

5. Recent presidio repo is cloned on your local machine.

Step by step deployment with customizable parameters Copy Section

1. Install Helm with RBAC.

2. Optional - Ingress controller for presidio API, e.g., NGINX.

   NOTE:
   Presidio is deployed with an ingress controller by default, and uses nginx as ingress.class. To change
   this behavior, deploy the helm chart with ingress.enabled=false.

3. Deploy from /docs/samples/deployments/k8s/charts/presidio

   sh Copy

   # Choose a namespace and ensure it is created
   NAMESPACE=presidio
   
   # Choose the tag from ghcr.io/data-privacy-stack, e.g. `latest`
   TAG=latest
   
   # Choose a name for the deployment
   NAME=<name>
   
   # Use Helm to install all required components
   helm install $NAME . --set tag=$TAG --namespace $NAMESPACE
   
   # If you have your own images in a separate ACR, run
   DOCKER_REGISTRY=<your_registry>
   helm install $NAME . --set registry=$DOCKER_REGISTRY,tag=$TAG --namespace $NAMESPACE

Samples Deployments Openai Anonymaztion And Deanonymaztion Best Practices Index (`docs/samples/deployments/openai-anonymaztion-and-deanonymaztion-best-practices/index.md`) Copy Section

Invisio - Data Protection toolkit for OpenAI

Sample Overview Copy Section

In this sample, we will demonstrate best practices for creating meaningful conversations with OpenAI while ensuring that all Personally Identifiable Information (PII) is anonymized before being sent to the OpenAI API. Additionally, we will cover the process of de-anonymizing the data upon receiving the response.

Problem Statement Copy Section

The primary challenge we aim to address is: how can we ensure that the responses processed by OpenAI, which are also anonymized, remain meaningful to the user?

To tackle this issue, we provide a reversible and repeatable AI Design Win which handles the anonymization process for sensitive data. In other words, we implemented a de-anonymization middleware that maps the anonymized entities, allowing us to de-anonymize the data effectively upon receiving it back.

Core Concept Copy Section

The core concept of the toolkit revolves around sessions, which encapsulate the anonymization context. This context defines how each Personally Identifiable Information (PII) entity is anonymized. By maintaining consistency within the same session, the toolkit ensures that the same PII is always anonymized in the same manner. Moreover, it facilitates the de-anonymization of data as needed.

This repository serves as a comprehensive sample that you can use as a foundation and extend according to your specific requirements.

The toolkit is particularly useful in "Chat with your data" scenarios within enterprises, ensuring that personally identifiable information (PII) present in corporate documents is not exposed to users during queries.

The following sections will describe the required infrastructure and code to achieve the above functionality.

Workflow Example Copy Section

The following is the desired output. We can input PII data anonymized to the LLM input and return the replaced entities back to the original in the response back to the user.

• User Input: "Hello world, my name is Jane Doe. My number is: 034453334"
• Anonymized LLM Input: "Hello world, my name is [PERSON]. My number is: [PHONE_NUMBER]"
• LLM Response: "Hey [PERSON], nice to meet you!"
• De-anonymized User Response: "Hey Jane Doe, nice to meet you!"

Demo Client Application Copy Section

The demo client application showcases the capabilities of the Anonymization Toolkit within the context of an "Anonymous Chat with a Large Language Model (LLM)" use case.

In this scenario, the toolkit anonymizes Personally Identifiable Information (PII) in the messages sent to the LLM and subsequently de-anonymizes the responses received from it. This approach ensures that the LLM never has access to actual PII, while still facilitating meaningful conversations.

Architecture and Infrastructure Copy Section

The solution consists of the following components:

• Api (Deployed in AKS)
• Client
• Redis

Code Copy Chapter

Anonymization Copy Section

First we need to setup our Presidio engines to aid with analysis, anonymization and deanonymization.
The following code can exist as part of a service available to the appropriate api endpoints.

configuration = {
            "nlp_engine_name": "spacy",
            "models": [
                {"lang_code": "en", "model_name": "en_core_web_lg"},
                {"lang_code": "nl", "model_name": "nl_core_news_sm"},
                {"lang_code": "es", "model_name": "es_core_news_sm"},
            ],
        }
        provider = NlpEngineProvider(nlp_configuration=configuration)
        nlp_engine = provider.create_engine()

        self.analyzer = AnalyzerEngine(
            nlp_engine=nlp_engine,
            supported_languages=["en", "nl", "es"]
        )
        self.anonymizer = AnonymizerEngine()
        self.anonymizer.add_anonymizer(InstanceCounterAnonymizer)
        self.deanonymizer = DeanonymizeEngine()
        self.deanonymizer.add_deanonymizer(InstanceCounterDeanonymizer)

InstanceCounterAnonymizer and InstanceCounterDeanonymizer taken from these examples.

We anonymize data with the following:

/* Detailed source-code truncated for AI context efficiency. */

and deanonymize with:

/* Detailed source-code truncated for AI context efficiency. */

The above code would be called with:

# anonymize example
    anonymized_text, new_entity_mappings = presidio_service.anonymize_text(
            request.session_id, request.text, request.language, entity_mappings
        )

    # deanonymize example
    deanonymized_text = presidio_service.deanonymize_text(
            request.session_id, request.text, entity_mappings
        )

Where:

• request.session_id
  • A unique session identifier, e.g. guid.
• request.text
  • The text to be anonymized or deanonymized.
• request.language
  • The language code to analyze with. Note this requires the appropriate language model to be setup during the configuration. See code above. Example codes, en, es, nl.
• entity_mappings

  • A dictionary with the structure of:

    python Copy

    {'LOCATION': {'Redmond': '<LOCATION_0>'}, 'PERSON': {'Joe Bloggs': '<PERSON_0>'}}

  • Each item is the entity type and the value contains each instance of the entity and its replacement.

  • This dictionary is optional. On the first request of a session we wont have any mappings, subsequent requests can then build upon this dictionary to maintain knowledge of previous request mappings.

Repository structure Copy Section

• src/api - The toolkit API, which is responsible for performing anonymization and deanonymization, as well as managing user sessions in a persistent storage.
• src/client_app/ - The demo client app, which enables users to engage in an anonymous chat with an LLM. The app utilizes the toolkit API to hide sensitive data from the LLM. It can be run either in a terminal or as a website.
• infrastructure/ - Bicep definition for Azure infrastructure required to support deployment to a Kubernetes (k8s) cluster.
• deployments/ - The manifests necessary to deploy the Toolkit API and the Demo to a k8s cluster.
• spikes/ - Experimental code, such as a Streamlit chat application that can interact with the toolkit API in a similar manner to the demo client app.

Samples Deployments Redacting Telemetry Index (`docs/samples/deployments/redacting-telemetry/index.md`) Copy Section

Redacting Telemetry with Presidio

A practical example of redacting PII in logs using Presidio.

⚠️ This is a proof-of-concept demonstration, not a production-ready solution.

What This Demo Shows Copy Chapter

This sample demonstrates client-side PII masking before logs are exported to observability platforms like Grafana/Loki. The application calls Presidio to detect and mask PII (names, emails, SSNs, phone numbers) before logging.

Considerations and Out of Scope Copy Chapter

This demo intentionally does NOT cover:

• Production deployment patterns
• Configurations, NLP model selection, or custom entity recognizers
• Reliability & Performance optimization
• Alternative architectures (sidecar, (otel-)collector-side masking, et cetera)

Architecture:

Application → Presidio (mask PII) → OpenTelemetry Collector → Loki/Tempo → Grafana

Quick Start Copy Chapter

Prerequisites Copy Section

• Docker and Docker Compose

Run the Demo Copy Section

cd docs/samples/deployments/redacting-telemetry
docker compose up -d

# Wait ~60 seconds for Presidio to download models
# Then open Grafana

Access Grafana: http://localhost:3000

Navigate to Dashboards → PII Redaction Demo to see:

• Unredacted Logs Panel: Raw PII (names, emails, SSNs)
• Redacted Logs Panel: PII replaced with types (, , ``)

The dashboard auto-refreshes every 5 seconds.

What's Running Copy Chapter

How It Works Copy Chapter

The demo app (app/main.py) generates logs with PII, then calls Presidio to mask them:

from presidio_client import mask_pii

# Original log
unredacted = "User registered: John Doe, SSN: 123-45-6789, Email: [email protected]"
logger.info(f"[UNREDACTED] {unredacted}")

# Redacted log
redacted = mask_pii(unredacted)  # Calls Presidio
logger.info(f"[REDACTED] {redacted}")
# Output: [REDACTED] User registered: <PERSON>, SSN: <US_SSN>, Email: <EMAIL_ADDRESS>

The presidio_client.py makes two HTTP calls:

1. Analyze: Detect PII entities
2. Anonymize: Replace with placeholders

Use in Your Application Copy Chapter

Python Example Copy Section

Copy app/presidio_client.py to your project:

from presidio_client import mask_pii

# Before logging sensitive data
sensitive = f"Customer {name} called from {phone}"
redacted = mask_pii(sensitive)
logger.info(redacted)  # "Customer <PERSON> called from <PHONE_NUMBER>"

Cleanup Copy Chapter

docker compose down       # Stop services
docker compose down -v    # Stop and delete data

Samples Deployments Spark Index (`docs/samples/deployments/spark/index.md`) Copy Section

Anonymize PII using Presidio on Spark

You can leverages presidio to perform data anonymization as part of spark notebooks.

The following sample uses Azure Databricks and simple text files hosted on Azure Blob Storage. However, it can easily change to fit any other scenario which requires PII analysis or anonymization as part of spark jobs.

Note that this code works for Databricks runtime 8.1 (spark 3.1.1) and the libraries described here.

The basics of working with Presidio in Spark Copy Chapter

A typical use case of Presidio in Spark is transforming a text column in a data frame, by anonymizing its content. The following code sample, a part of transform presidio notebook, is the basis of the e2e sample which uses Azure Databricks as the Spark environment.

anonymized_column = "value" # name of column to anonymize
analyzer = AnalyzerEngine()
anonymizer = AnonymizerEngine()

# broadcast the engines to the cluster nodes
broadcasted_analyzer = sc.broadcast(analyzer)
broadcasted_anonymizer = sc.broadcast(anonymizer)

# define a pandas UDF function and a series function over it.
def anonymize_text(text: str) -> str:
    analyzer = broadcasted_analyzer.value
    anonymizer = broadcasted_anonymizer.value
    analyzer_results = analyzer.analyze(text=text, language="en")
    anonymized_results = anonymizer.anonymize(
        text=text,
        analyzer_results=analyzer_results,
        operators={
            "DEFAULT": OperatorConfig("replace", {"new_value": "<ANONYMIZED>"})
        },
    )
    return anonymized_results.text


def anonymize_series(s: pd.Series) -> pd.Series:
    return s.apply(anonymize_text)


# define a the function as pandas UDF
anonymize = pandas_udf(anonymize_series, returnType=StringType())

# apply the udf
anonymized_df = input_df.withColumn(
    anonymized_column, anonymize(col(anonymized_column))
)

Pre-requisites Copy Chapter

If you do not have an instance of Azure Databricks, follow through with the following steps to provision and setup the required infrastructure.

If you do have a Databricks workspace and a cluster you wish to configure to run Presidio, jump over to the Configure an existing cluster section.

Deploy Infrastructure Copy Section

Provision the Azure resources by running the following script.

export RESOURCE_GROUP=[resource group name]
export STORAGE_ACCOUNT_NAME=[storage account name]
export STORAGE_CONTAINER_NAME=[blob container name]
export DATABRICKS_WORKSPACE_NAME=[databricks workspace name]
export DATABRICKS_SKU=[basic/standard/premium]
export LOCATION=[location]

# Create the resource group
az group create --name $RESOURCE_GROUP --location $LOCATION

# Use ARM template to build the resources and get back the workspace URL
deployment_response=$(az deployment group create -g $RESOURCE_GROUP --template-file ./docs/samples/deployments/spark/arm-template/databricks.json  --parameters location=$LOCATION workspaceName=$DATABRICKS_WORKSPACE_NAME storageAccountName=$STORAGE_ACCOUNT_NAME containerName=$STORAGE_CONTAINER_NAME)

export DATABRICKS_WORKSPACE_URL=$(echo $deployment_response | jq -r ".properties.outputs.workspaceUrl.value")
export DATABRICKS_WORKSPACE_ID=$(echo $deployment_response | jq -r ".properties.outputs.workspaceId.value")

Setup Databricks Copy Section

The following script will setup a new cluster in the databricks workspace and prepare it to run presidio anonymization jobs.
Once finished, the script will output an access key which you can use when working with databricks cli.

sh ./scripts/configure_databricks.sh

Configure an existing cluster Copy Section

Only follow through with the steps in this section if you have an existing databricks workspace and clsuter you wish to configure to run presidio. If you've followed through with the "Deploy Infrastructure" and "Setup Databricks" sections you do not have to run the script in this section.

Set up secret scope and secrets for storage account

Add an Azure Storage account key to secret scope.

STORAGE_PRIMARY_KEY=[Primary key of storage account]

databricks secrets create-scope --scope storage_scope --initial-manage-principal users
databricks secrets put --scope storage_scope --key storage_account_access_key --string-value "$STORAGE_PRIMARY_KEY"

Upload or update cluster init scripts

Presidio libraries are loaded to the cluster on init.
Upload the cluster setup script or add its content to the existing cluster's init script.

databricks fs cp "./setup/startup.sh" "dbfs:/FileStore/dependencies/startup.sh"

Setup the cluster to run the init script.

Upload presidio notebooks

databricks workspace import_dir "./notebooks" "/notebooks" --overwrite

Update cluster environment

Add the following environment variables to your databricks cluster:

"STORAGE_MOUNT_NAME": "/mnt/files"
"STORAGE_CONTAINER_NAME": [Blob container name]
"STORAGE_ACCOUNT_NAME": [Storage account name]

Mount the storage container

Run the notebook 00_setup to mount the storage account to databricks.

Running the sample Copy Chapter

Configure Presidio transformation notebook Copy Section

From Databricks workspace, under notebooks folder, open the provided 01_transform_presidio notebook and attach it to the cluster preisidio_cluster.
Run the first code-cell and note the following parameters on the top end of the notebook (notebook widgets) and set them accordingly

• Input File Format - text (selected).
• Input path - a folder on the container where input files are found.
• Output Folder - a folder on the container where output files will be written to.
• Column to Anonymize - value (selected).

Run the notebook Copy Section

Upload a text file to the blob storage input folder, using any preferred method (Azure Portal, Azure Storage Explorer, Azure CLI).

az storage blob upload --account-name $STORAGE_ACCOUNT_NAME  --container $STORAGE_CONTAINER_NAME --file ./[file name] --name input/[file name]

Run the notebook cells, the output should be csv files which contain two columns, the original file name, and the anonymized content of that file.

Samples Python Ahds Index (`docs/samples/python/ahds/index.md`) Copy Section

Azure Health Data Services de-identification Integration

Introduction Copy Chapter

The Azure Health Data Services (AHDS) de-identification Service is a cloud-based
service that provides advanced natural language processing over
raw text. One of its main functions includes Named Entity Recognition
(NER), which has the ability to identify different
entities in text and categorize them into pre-defined classes or types.
This document will demonstrate Presidio integration with the AHDS
De-Identification Service.

Supported entity categories in the Azure Health Data Services de-identification API Copy Chapter

Azure Health Data Services de-identification supports multiple PII entity categories.
The Azure Health Data Services de-identification service
runs a predictive model to identify and categorize named entities from an input
document. The service's latest version includes the ability to detect personal (PII)
and health (PHI) information. A list of all supported entities can be found in the
official documentation.

Prerequisites Copy Chapter

To use AHDS De-Identification with Presidio, an Azure De-Identification Service resource should
first be created under an Azure subscription. Follow the official documentation
for instructions. The endpoint, generated once the resource is created,
will be used when integrating with AHDS De-Identification, using a Presidio remote recognizer.

Authentication Setup Copy Section

The integration uses a secure-by-default authentication approach:

Production Mode (Default): Uses a restricted credential chain (EnvironmentCredential, WorkloadIdentityCredential, ManagedIdentityCredential)

Development Mode: Set ENV=development to use DefaultAzureCredential for local development with Azure CLI:

export ENV=development
az login

For more details, see the AHDS Integration Authentication documentation.

Azure Health Data Services de-identification Recognizer Copy Chapter

The implementation of a AzureHealthDeid recognizer can be found here.

How to integrate Azure Health Data Services de-identification into Presidio Copy Chapter

1. Install the package with the ahds extra:

pip install "presidio-analyzer[ahds]"

2. Define environment variables AHDS_ENDPOINT

3. Add the AzureHealthDeidRecognizer to the recognizer registry:

from presidio_analyzer import AnalyzerEngine
from presidio_analyzer.predefined_recognizers import AzureHealthDeidRecognizer
  
ahds = AzureHealthDeidRecognizer()

analyzer = AnalyzerEngine()
analyzer.registry.add_recognizer(ahds)

analyzer.analyze(text="My email is [email protected]", language="en")

See also:
For a full surrogate integration example, see example_ahds_surrogate.py

Samples Python Langextract Index (`docs/samples/python/langextract/index.md`) Copy Section

Language Model-based PII/PHI Detection (Experimental Feature)

Introduction Copy Chapter

Presidio supports language model-based PII/PHI detection for flexible entity recognition using language models (LLMs, SLMs, etc.). This approach enables detection of both:

• PII (Personally Identifiable Information): Names, emails, phone numbers, SSN, credit cards, etc.
• PHI (Protected Health Information): Medical records, health identifiers, etc.

(The default approach uses LangExtract under the hood to integrate with language model providers.)

Entity Detection Capabilities Copy Chapter

Unlike pattern-based recognizers, language model-based detection is flexible and depends on:

• The language model being used
• The prompt description provided
• The few-shot examples configured

The default configuration includes examples for common PII/PHI entities such as PERSON, EMAIL_ADDRESS, PHONE_NUMBER, US_SSN, CREDIT_CARD, MEDICAL_LICENSE, and more.
You can customize the prompts and examples to detect any entity types relevant to your use case.

For the default entity mappings and examples, see the default configuration.

Supported Language Model Providers Copy Chapter

Presidio supports the following language model providers through LangExtract:

1. Azure OpenAI - Cloud-based Azure OpenAI Service (GPT-4o, GPT-4, GPT-3.5-turbo, etc.)
2. Ollama - Local language model deployment (open-source models like Gemma, Llama, etc.)

Choosing Between Azure OpenAI and Ollama Copy Chapter

Recommendations:

• Use Azure OpenAI for production workloads requiring enterprise security, compliance (HIPAA, SOC 2, etc.), and managed infrastructure
• Use Ollama for local development, testing, or when data must stay on-premises

Language Model-based Recognizer Implementation Copy Chapter

Presidio provides a hierarchy of recognizers for language model-based PII/PHI detection:

• LMRecognizer: Abstract base class for all language model recognizers (LLMs, SLMs, etc.)
• LangExtractRecognizer: Abstract base class for LangExtract library integration (model-agnostic)
• AzureOpenAILangExtractRecognizer: Concrete implementation for Azure OpenAI Service
  • Implementation
• BasicLangExtractRecognizer: Concrete implementation where ModelConfig is configured from YAML (supporting Ollama, OpenAI, Gemini, and other providers)
  • Implementation

Using Azure OpenAI (Cloud Models) Copy Chapter

Azure OpenAI provides cloud-based access to OpenAI models (GPT-4o, GPT-4, GPT-3.5-turbo) with enterprise security and compliance features.

Prerequisites Copy Section

1. Install Presidio with LangExtract support:

   sh Copy

   pip install "presidio-analyzer[langextract]"

2. Set up Ollama

You have two options to set up Ollama:

Option 1: Docker Compose (recommended for CPU)

This option requires Docker to be installed on your system.

Where to run: From the root presidio directory (where docker-compose.yml is located)

docker compose up -d ollama
docker exec presidio-ollama-1 ollama pull qwen2.5:1.5b
docker exec presidio-ollama-1 ollama list

Platform differences:

• Linux/Mac: Commands above work as-is
• Windows: Use PowerShell or CMD, commands are the same

If you don't have Docker installed:

• Linux: Follow Docker installation guide
• Mac: Install Docker Desktop for Mac
• Windows: Install Docker Desktop for Windows

Option 2: Native installation (recommended for GPU acceleration)

Follow the official LangExtract Ollama guide.

After installation, pull and run the model:

ollama pull qwen2.5:1.5b
ollama run qwen2.5:1.5b

This option provides better performance with GPU acceleration (e.g., on Mac with Metal Performance Shaders or systems with NVIDIA GPUs).
The model must be pulled and run before using the recognizer. The default model is qwen2.5:1.5b.

3. Configuration (optional): Create your own ollama_config.yaml or use the default configuration

Usage Copy Section

Option 1: Enable in configuration file

Enable the recognizer in default_recognizers.yaml:

- name: BasicLangExtractRecognizer
  enabled: true  # Change from false to true

Then load the analyzer using this modified configuration file:

from presidio_analyzer import AnalyzerEngine
from presidio_analyzer.recognizer_registry import RecognizerRegistryProvider

# Point to your modified default_recognizers.yaml with Ollama enabled
provider = RecognizerRegistryProvider(
    conf_file="/path/to/your/modified/default_recognizers.yaml"
)
registry = provider.create_recognizer_registry()

# Create analyzer with the registry that includes Ollama recognizer
analyzer = AnalyzerEngine(registry=registry, supported_languages=["en"])

# Analyze text - Ollama recognizer will participate in detection
results = analyzer.analyze(text="My email is [email protected]", language="en")

Option 2: Add programmatically

from presidio_analyzer import AnalyzerEngine
from presidio_analyzer.predefined_recognizers.third_party.basic_langextract_recognizer import BasicLangExtractRecognizer

analyzer = AnalyzerEngine()
analyzer.registry.add_recognizer(BasicLangExtractRecognizer())

results = analyzer.analyze(text="My email is [email protected]", language="en")

!!! note "Note"
The recognizer is disabled by default in default_recognizers.yaml to avoid requiring Ollama for basic Presidio usage. Enable it when you have Ollama set up and running.

Custom Configuration Copy Section

To use a custom configuration file:

analyzer.registry.add_recognizer(
    BasicLangExtractRecognizer(config_path="/path/to/custom_config.yaml")
)

Configuration Options Copy Section

The langextract_config_ollama.yaml file supports the following options:

• model_id: The model to use.
• provider.name: The model provider (eg, ollama, openai)
• provider.kwargs: kwargs to pass to the model provider (eg, model_url for Ollama, base_url for OpenAI)
• provider.extract_params: Extraction parameters (eg, use_schema_constraints, fence_output, temperature)
• provider.language_model_params: Parameters for the model itself (eg, timeout, num_ctx)
• supported_entities: PII/PHI entity types to detect
• entity_mappings: Map LangExtract entity classes to Presidio entity names
• min_score: Minimum confidence score (default: 0.5)

See the configuration file for all options.

Troubleshooting Copy Chapter

ConnectionError: "Ollama server not reachable"

• Ensure Ollama is running: docker ps or check http://localhost:11434
• Verify the model_url in your configuration matches your Ollama server address

RuntimeError: "Model 'qwen2.5:1.5b' not found"

• Pull the model: docker exec -it presidio-ollama-1 ollama pull qwen2.5:1.5b
• Or for manual setup: ollama pull qwen2.5:1.5b
• Verify the model name matches the model_id in your configuration

Using Azure OpenAI (Cloud Models) Copy Chapter

Azure OpenAI provides cloud-based access to OpenAI models (GPT-4o, GPT-4, GPT-3.5-turbo) with enterprise security and compliance features.

Prerequisites Copy Section

1. Install Presidio with LangExtract support:

   sh Copy

   pip install "presidio-analyzer[langextract]"

   This installs langextract with OpenAI support, including the OpenAI Python SDK and Azure Identity libraries.

2. Azure Subscription: Create one at azure.microsoft.com

3. Azure OpenAI Resource:

   • Create an Azure OpenAI resource in Azure Portal
   • Request access if needed (some regions require approval)
   • Deploy a model and note the deployment name you choose (e.g., "gpt-4", "my-gpt-deployment")

4. Optional: Download config file (only if customizing entities/prompts):

   sh Copy

   # On macOS/Linux/PowerShell:
   wget https://raw.githubusercontent.com/data-privacy-stack/presidio/main/presidio-analyzer/presidio_analyzer/conf/langextract_config_azureopenai.yaml
   
   # Or download manually from:
   # https://github.com/data-privacy-stack/presidio/blob/main/presidio-analyzer/presidio_analyzer/conf/langextract_config_azureopenai.yaml

Authentication Options Copy Section

Azure OpenAI supports multiple authentication methods with flexible configuration:

Option 1: Direct Parameters (Recommended for Most Users)

Simplest approach - pass credentials and deployment name as parameters:

from presidio_analyzer import AnalyzerEngine
from presidio_analyzer.predefined_recognizers import AzureOpenAILangExtractRecognizer

# Initialize with deployment name and credentials
azure_openai = AzureOpenAILangExtractRecognizer(
    model_id="gpt-4",  # Your Azure deployment name
    azure_endpoint="https://your-resource.openai.azure.com/",
    api_key="your-api-key-here",
    api_version="2024-02-15-preview"  # Optional
)

analyzer = AnalyzerEngine()
analyzer.registry.add_recognizer(azure_openai)

results = analyzer.analyze(
    text="My email is [email protected] and my phone is 555-123-4567",
    language="en"
)

Option 2: Environment Variables

Use environment variables for credentials:

import os
from presidio_analyzer import AnalyzerEngine
from presidio_analyzer.predefined_recognizers import AzureOpenAILangExtractRecognizer

# Set environment variables
os.environ["AZURE_OPENAI_ENDPOINT"] = "https://your-resource.openai.azure.com/"
os.environ["AZURE_OPENAI_API_KEY"] = "your-api-key-here"

# Initialize with just deployment name
azure_openai = AzureOpenAILangExtractRecognizer(
    model_id="gpt-4"  # Your Azure deployment name
)

analyzer = AnalyzerEngine()
analyzer.registry.add_recognizer(azure_openai)

results = analyzer.analyze(
    text="My email is [email protected] and my phone is 555-123-4567",
    language="en"
)

Option 3: Managed Identity (Production)

More secure - No API keys in code, uses Azure RBAC:

import os
from presidio_analyzer import AnalyzerEngine
from presidio_analyzer.predefined_recognizers import AzureOpenAILangExtractRecognizer

# Set endpoint (no API key = uses managed identity)
os.environ["AZURE_OPENAI_ENDPOINT"] = "https://your-resource.openai.azure.com/"

# Initialize without API key (uses managed identity)
azure_openai = AzureOpenAILangExtractRecognizer(
    model_id="gpt-4"  # Your Azure deployment name
)

analyzer = AnalyzerEngine()
analyzer.registry.add_recognizer(azure_openai)

results = analyzer.analyze(
    text="Patient John Smith has SSN 123-45-6789",
    language="en"
)

Managed Identity Authentication Flow (Production):

When api_key is not provided, the provider automatically uses ChainedTokenCredential which tries credentials in order:

1. EnvironmentCredential - Service principal from environment variables
2. WorkloadIdentityCredential - Azure Kubernetes Service workload identity
3. ManagedIdentityCredential - Azure VM/App Service managed identity

For local development, set ENV=development to use DefaultAzureCredential instead:

import os
os.environ["ENV"] = "development"
os.environ["AZURE_OPENAI_ENDPOINT"] = "https://your-resource.openai.azure.com/"
# AZURE_OPENAI_API_KEY not set - uses DefaultAzureCredential in development mode
# (includes Azure CLI, VS Code, etc.)

azure_openai = AzureOpenAILangExtractRecognizer()

Setup Managed Identity:

1. Enable managed identity on your Azure resource (VM, App Service, Container Instance, etc.)
2. Grant the managed identity "Cognitive Services OpenAI User" role on your Azure OpenAI resource
3. No API keys needed - authentication is automatic

See Azure Managed Identity documentation for details.

Configuration File (Optional) Copy Section

The configuration file is optional for basic usage. You only need it to customize:

• Supported entity types
• Entity mappings (LangExtract → Presidio)
• Prompts and examples
• Detection parameters

For basic usage, just pass model_id as a parameter (see examples above).

When you need a custom config:

1. Download the default config:

   sh Copy

   wget https://raw.githubusercontent.com/data-privacy-stack/presidio/main/presidio-analyzer/presidio_analyzer/conf/langextract_config_azureopenai.yaml

2. Customize entities, prompts, or other settings in the file

3. Use the customized config:

   python Copy

   recognizer = AzureOpenAILangExtractRecognizer(
       model_id="gpt-4",  # Can override config's model_id
       config_path="./custom_config.yaml",
       azure_endpoint="...",
       api_key="..."
   )

Configuration Reference:

The config file contains two main sections:

lm_recognizer section (LLM recognizer settings):

• supported_entities: List of PII/PHI entity types to detect
• labels_to_ignore: Entity labels to skip during processing
• enable_generic_consolidation: Whether to consolidate unknown entities to GENERIC_PII_ENTITY
• min_score: Minimum confidence score threshold (0.0-1.0)

langextract section (LangExtract-specific settings):

• model.model_id: Azure OpenAI deployment name (e.g., "gpt-4o", "gpt-4", "gpt-35-turbo")
• model.temperature: Model temperature for generation (null = use model default)
• prompt_file: Path to custom prompt template file
• examples_file: Path to few-shot examples file
• entity_mappings: Map LangExtract entity classes to Presidio entity names

See the full config file for details.

Using Ollama (Local Models) Copy Chapter

Samples Python Streamlit Index (`docs/samples/python/streamlit/index.md`) Copy Section

Simple demo website for Presidio

Here's a simple app, written in pure Python, to create a demo website for Presidio.
The app is based on the streamlit package.

A live version can be found here: https://huggingface.co/spaces/presidio/presidio_demo

Requirements Copy Chapter

1. Clone the repo and move to the docs/samples/python/streamlit folder
2. Install dependencies (preferably in a virtual environment)

pip install -r requirements.txt

Note: This would install additional packages such as transformers and flair which are not mandatory for using Presidio.

3. Optional: Update the analyzer_engine and anonymizer_engine functions for your specific implementation (in presidio_helpers.py).
4. Start the app:

streamlit run presidio_streamlit.py

5. Consider adding an .env file with the following environment variables, for further customizability:

TA_KEY=YOUR_TEXT_ANALYTICS_KEY
TA_ENDPOINT=YOUR_TEXT_ANALYTICS_ENDPOINT
OPENAI_TYPE="Azure" #or "openai"
OPENAI_KEY=YOUR_OPENAI_KEY
OPENAI_API_VERSION = "2023-05-15"
AZURE_OPENAI_ENDPOINT=YOUR_AZURE_OPENAI_AZURE_OPENAI_ENDPOINT
AZURE_OPENAI_DEPLOYMENT=text-davinci-003
ALLOW_OTHER_MODELS=true #true if the user could download new models

Output Copy Chapter

Output should be similar to this screenshot:

Samples Python Text Analytics Index (`docs/samples/python/text_analytics/index.md`) Copy Section

Azure AI Language Integration

Introduction Copy Chapter

Azure Text Analytics is a cloud-based service that provides advanced natural
language processing over raw text. One of its main functions includes
Named Entity Recognition (NER), which has the ability to identify different
entities in text and categorize them into pre-defined classes or types.
This document will demonstrate Presidio integration with Azure Text Analytics.

Supported entity categories in the Text Analytics API Copy Chapter

Azure AI Language supports multiple PII entity categories. The Azure AI Laguage service
runs a predictive model to identify and categorize named entities from an input
document. The service's latest version includes the ability to detect personal (PII)
and health (PHI) information. A list of all supported entities can be found in the
official documentation.

Prerequisites Copy Chapter

To use Azure AI Language with Preisido, an Azure AI Language resource should
first be created under an Azure subscription. Follow the official documentation
for instructions. The key and endpoint, generated once the resource is created,
will be used when integrating with Text Analytics, using a Presidio Text Analytics recognizer.

Azure AI Language Recognizer Copy Chapter

The implementation of a AzureAILanguage recognizer can be found here.

How to integrate Azure AI Language into Presidio Copy Chapter

1. Install the package with the azure-ai-language extra:

pip install "presidio-analyzer[azure-ai-language]"

2. Define environment varibles AZURE_AI_KEY and AZURE_AI_ENDPOINT

3. Add the AzureAILanguageRecognizer to the recognizer registry:

from presidio_analyzer import AnalyzerEngine
from presidio_analyzer.predefined_recognizers import AzureAILanguageRecognizer
  
azure_ai_language = AzureAILanguageRecognizer()

analyzer = AnalyzerEngine()
analyzer.registry.add_recognizer(azure_ai_language)

analyzer.analyze(text="My email is [email protected]", language="en")

Samples Python Transformers Recognizer Index (`docs/samples/python/transformers_recognizer/index.md`) Copy Section

Add a Transformers model based EntityRecognizer

!!! note "Note"

This example demonstrates how to create a **Presidio Recognizer**.
To integrate a transformers model as a **Presidio NLP Engine**, see [this documentation](../../../analyzer/nlp_engines/transformers.md).

We allow these two options, as a user might want to have multiple NER models running in parallel. In this case, one can create multiple `EntityRecognizer` instances, each serving a different model. If you only plan to use one NER model, consider creating a [`TransformersNlpEngine`](../../../analyzer/nlp_engines/transformers.md) instead of the [`TransformersRecognizer`](https://github.com/data-privacy-stack/presidio/blob/main/docs/samples/python/transformers_recognizer/transformer_recognizer.py) described in this document.

When initializing the TransformersRecognizer, choose from the following options:

1. A string referencing an uploaded model to HuggingFace. See the different available options for models here.

2. Initialize your own TokenClassificationPipeline instance using your custom transformers model and use it for inference.

3. Provide the path to your own local custom trained model.

!!! note "Note"
For each combination of model & dataset, it is recommended to create a configuration object which includes setting necessary parameters for getting the correct results. Please reference this configuration.py file for examples.

Example Code Copy Chapter

This example code uses a TransformersRecognizer for NER, and removes the default SpacyRecognizer.
In order to be able to use spaCy features such as lemmas, we introduce the small (and faster) en_core_web_sm model.

link to full TransformersRecognizer code

from presidio_analyzer import AnalyzerEngine, RecognizerRegistry
from presidio_analyzer.nlp_engine import NlpEngineProvider
import spacy

model_path = "obi/deid_roberta_i2b2"
supported_entities = BERT_DEID_CONFIGURATION.get(
    "PRESIDIO_SUPPORTED_ENTITIES")
transformers_recognizer = TransformersRecognizer(model_path=model_path,
                                                 supported_entities=supported_entities)

# This would download a large (~500Mb) model on the first run
transformers_recognizer.load_transformer(**BERT_DEID_CONFIGURATION)

# Add transformers model to the registry
registry = RecognizerRegistry()
registry.add_recognizer(transformers_recognizer)
registry.remove_recognizer("SpacyRecognizer")

# Use small spacy model, for faster inference.
if not spacy.util.is_package("en_core_web_sm"):
    spacy.cli.download("en_core_web_sm")

nlp_configuration = {
    "nlp_engine_name": "spacy",
    "models": [{"lang_code": "en", "model_name": "en_core_web_sm"}],
}

nlp_engine = NlpEngineProvider(nlp_configuration=nlp_configuration).create_engine()

analyzer = AnalyzerEngine(registry=registry, nlp_engine=nlp_engine)

sample = "My name is John and I live in NY"
results = analyzer.analyze(sample, language="en",
                           return_decision_process=True,
                           )
print("Found the following entities:")
for result in results:
    print(result, '----', sample[result.start:result.end])

Samples Deployments Openai Anonymaztion And Deanonymaztion Best Practices Src Api README (`docs/samples/deployments/openai-anonymaztion-and-deanonymaztion-best-practices/src/api/README.md`) Copy Section

API Readme

Install requirements Copy Chapter

cd src/api
pip install -r requirements.txt

Set environment variables Copy Section

Add .env file in the /src/api folder:

cp .env.sample .env

Then edit this configuration according to your setup.

Run locally Copy Chapter

uvicorn main:app --host 0.0.0.0 --port 8080

Run as a docker container Copy Chapter

docker build -t api .
docker run -d -p 8080:80 --env-file .env api

Samples Deployments Openai Anonymaztion And Deanonymaztion Best Practices Src Client App README (`docs/samples/deployments/openai-anonymaztion-and-deanonymaztion-best-practices/src/client_app/README.md`) Copy Section

Demo client for Anonymizer API

Have an anonymous chat with the LLM right in your terminal or on a web page.

Prerequisites Copy Chapter

Install requirements Copy Section

cd src/client_app
pip install -r requirements.txt

Create .env file Copy Section

cp .env.sample .env

Then edit the configuration in the .env file, for example:

• add your OpenAI API settings if you'd like to chat with an LLM
• add anonymizer/deanonymizer API endpoints

Running the chat Copy Chapter

Chat with LLM in the terminal Copy Section

python client.py

Chat with yourself in the terminal Copy Section

python client.py --mode manual

Chat with LLM on a web page Copy Section

python serve.py

Dockerize chatting on a web page Copy Section

Build container:

docker build -t client .

And run it, e.g like this:

docker run -p 8081:8081 --env-file .env client

Note: anonymizer API (API_URL env variable) should not point to localhost, otherwise the docker container won't be able to access it.

Ahds Integration (`docs/ahds_integration.md`) Copy Section

Azure Health Data Services De-identification Service Integration

Presidio supports integration with Azure Health Data Services (AHDS) de-identification service for both entity recognition and anonymization with realistic surrogate generation.

Resources Copy Chapter

• Azure Health Data Services Documentation
• Azure Health Deidentification Python SDK

Overview Copy Chapter

The AHDS de-identification service integration provides two main capabilities:

1. AHDS Recognizer (in presidio-analyzer): Detects PHI entities using the Azure Health Data Services de-identification service
2. AHDS Surrogate Operator (in presidio-anonymizer): Replaces PHI entities with realistic surrogates using the de-identification service

Benefits of Surrogation Copy Chapter

• Maintains Data Utility: Preserves structure and format for downstream analytics
• Realistic Healthcare Context: Generates medically plausible names, dates, and identifiers
• Consistent Cross-References: Same entity gets same surrogate throughout document
• Format Preservation: Maintains original formatting and linguistic patterns

Installation Copy Chapter

For AHDS Recognizer Copy Section

pip install "presidio-analyzer[ahds]"

For AHDS Surrogate Operator Copy Section

pip install "presidio-anonymizer[ahds]"

Prerequisites Copy Chapter

• Azure Health Data Services de-identification service endpoint
• Azure role-based access control configured
• Environment variables:
  • AHDS_ENDPOINT: Your AHDS de-identification service endpoint

Complete Workflow Example Copy Chapter

import os
from presidio_analyzer import AnalyzerEngine
from presidio_anonymizer import AnonymizerEngine
from presidio_anonymizer.entities import OperatorConfig

# Set your AHDS endpoint
os.environ["AHDS_ENDPOINT"] = "https://your-ahds-endpoint.api.eus001.deid.azure.com"

# Initialize engines
analyzer = AnalyzerEngine()
anonymizer = AnonymizerEngine()

# Medical text with PHI
text = "Patient John Doe was seen by Dr. Smith on 2024-01-15 for diabetes treatment."

# Step 1: Detect entities using AHDS recognizer
analyzer_results = analyzer.analyze(
    text=text,
    entities=["PATIENT", "DOCTOR", "DATE"],
    language="en"
)

# Step 2: Anonymize using AHDS surrogate generation
result = anonymizer.anonymize(
    text=text,
    analyzer_results=analyzer_results,
    operators={
        "DEFAULT": OperatorConfig("surrogate", {
            "entities": analyzer_results,
            "input_locale": "en-US",
            "surrogate_locale": "en-US"
        })
    },
)

print(f"Original: {text}")
print(f"Anonymized: {result.text}")
# Output: "Patient Michael Johnson was seen by Dr. Brown on 1987-06-23 for diabetes treatment."

Note: This example uses the Azure Health Data Services de-identification service surrogation, which provides superior data utility by generating realistic, medically-appropriate surrogates while maintaining document structure and relationships.

Configuration Options Copy Chapter

AHDS Surrogate Operator Parameters Copy Section

• endpoint: AHDS de-identification service endpoint (optional, uses AHDS_ENDPOINT env var)
• entities: List of entities detected by analyzer
• input_locale: Input locale (default: "en-US")
• surrogate_locale: Surrogate locale (default: "en-US")

Authentication Copy Chapter

The AHDS de-identification service integration uses Azure authentication with a secure-by-default approach:

Production Mode (Default) Copy Section

By default, the integration uses a restricted credential chain that only includes:

1. EnvironmentCredential: Service Principal via environment variables
2. WorkloadIdentityCredential: Workload Identity (Kubernetes)
3. ManagedIdentityCredential: Managed Identity (Azure services)

This credential chain is more secure as it excludes interactive browser logins and developer credentials, making it suitable for production deployments.

Development Mode Copy Section

For local development, you can enable DefaultAzureCredential by setting the environment variable:

export ENV=development

In development mode, the integration uses DefaultAzureCredential, which supports additional authentication methods:

1. Environment variables (Service Principal)
2. Workload Identity
3. Managed Identity
4. Azure CLI (az login)
5. Azure PowerShell
6. Visual Studio/VS Code credentials
7. Interactive browser login

Local Development Setup Copy Section

For local development with Azure CLI authentication:

# Set development mode
export ENV=development

# Login with Azure CLI
az login

# Set your AHDS endpoint
export AHDS_ENDPOINT="https://your-ahds-endpoint.api.eus001.deid.azure.com"

# Now you can run Presidio with AHDS integration
python your_script.py

Production Deployment Recommendations Copy Section

For production deployments, we recommend:

1. Do not set ENV=development (use default production mode)
2. Use Managed Identity when running on Azure services (AKS, App Service, Functions, etc.)
3. Use Workload Identity for Kubernetes deployments
4. Use Service Principal with environment variables for other scenarios:
   bash Copy

   export AZURE_CLIENT_ID="<client-id>"
   export AZURE_CLIENT_SECRET="<client-secret>"
   export AZURE_TENANT_ID="<tenant-id>"

Environment Variables Summary Copy Section

• AHDS_ENDPOINT: Your AHDS de-identification service endpoint (required)
• ENV: Set to development for local dev, omit for production (optional, default: production mode)

Troubleshooting Copy Chapter

Common Issues Copy Section

1. ModuleNotFoundError: Install the AHDS optional dependencies

   bash Copy

   pip install "presidio-analyzer[ahds]" "presidio-anonymizer[ahds]"

2. Authentication errors: Ensure Azure credentials are properly configured

   bash Copy

   az login  # For local development

3. Endpoint not found: Verify the AHDS_ENDPOINT environment variable is set correctly

Testing without AHDS Copy Section

The AHDS operators gracefully handle missing dependencies and will be skipped if not available. Tests will be automatically skipped if the AHDS_ENDPOINT environment variable is not set.

See Also Copy Chapter

• Presidio Analyzer
• Presidio Anonymizer
• Azure Health Data Services de-identification service documentation

Api (`docs/api.md`) Copy Section

Presidio API

Api reference for Presidio's main python modules

• Presidio analyzer Python API reference
• Presidio anonymizer Python API reference
• Presidio image redactor Python API reference
• Presidio structured Python API reference

Build Release (`docs/build_release.md`) Copy Section

Build and release process

Presidio uses GitHub Actions workflows to validate, build, release, and deliver official artifacts.

Workflows Copy Chapter

The following workflows are maintained as part of the Presidio development process:

• CI - triggered for pull requests and merges to the main branch.
  • Linting
  • Dependency review
  • Unit tests
  • Package builds
  • Container builds and E2E tests
• Release - manually triggered for official releases.
  • Publishes Python packages to PyPI
  • Publishes container images to GitHub Container Registry
  • Creates a draft GitHub release

Container image publishing Copy Chapter

Official Presidio container images are now published to GitHub Container Registry (GHCR) under the Data Privacy Stack organization:

• ghcr.io/data-privacy-stack/presidio-analyzer
• ghcr.io/data-privacy-stack/presidio-anonymizer
• ghcr.io/data-privacy-stack/presidio-image-redactor

The legacy Microsoft Container Registry images at mcr.microsoft.com/presidio-* are no longer updated. Existing MCR tags can still be pulled for older deployments, but users should update image references to GHCR before upgrading. For example:

docker pull ghcr.io/data-privacy-stack/presidio-analyzer:latest

For production deployments, avoid relying on the moving latest tag. Pin the release tag shown on the relevant GitHub Packages page, for example:

docker pull ghcr.io/data-privacy-stack/presidio-analyzer:<release-version>

If an existing deployment uses mcr.microsoft.com/presidio-analyzer:latest, treat it as a legacy reference and replace it with either ghcr.io/data-privacy-stack/presidio-analyzer:latest for quick testing or a pinned GHCR release tag for production.

Docker registries do not provide a portable way to redirect docker pull mcr.microsoft.com/presidio-analyzer:latest to GHCR or show a deprecation warning during docker pull. To reduce confusion, each GitHub release includes the GHCR image names, the GHCR images include OCI metadata labels pointing to the source repository, package page, and installation guide, and the MCR tags are documented as legacy references.

PyPI publishing with OIDC Copy Chapter

The release workflow uses OIDC (OpenID Connect) trusted publishing to PyPI, which eliminates the need to manage PyPI API tokens. This requires:

1. PyPI configuration: Each package (presidio-analyzer, presidio-anonymizer, and the other Presidio packages) must be configured on PyPI to trust the GitHub repository and workflow.
2. GitHub workflow permissions: The workflow uses pypa/gh-action-pypi-publish@release/v1 with id-token: write.
3. No PyPI secrets: No PyPI API tokens need to be stored as GitHub secrets.

Benefits of OIDC:

• Enhanced security through short-lived tokens
• No manual token management
• Automatic token rotation
• Audit trail of publishing activities

GHCR publishing permissions Copy Chapter

The container publishing jobs authenticate to GHCR with the workflow GITHUB_TOKEN by default and require packages: write permissions. If the workflow is run from a repository outside the data-privacy-stack organization during the transition, set the GHCR_TOKEN secret and optional GHCR_USERNAME variable to credentials that can publish packages to the Data Privacy Stack organization, or run the release from the migrated repository.

Community (`docs/community.md`) Copy Section

Presidio eco-system

This section collects different resources developed with Presidio.

Resources Copy Chapter

• Please create a PR if you're interested in adding your tool to this list.

Design (`docs/design.md`) Copy Section

Presidio design

Analyzer Copy Chapter

Anonymizer Copy Chapter

Image Redactor Copy Chapter

Standard Image Types Copy Section

DICOM Images Copy Section

Development (`docs/development.md`) Copy Section

Setting Up a Development Environment

Getting started Copy Chapter

Cloning the repo Copy Section

To create a local copy of Presidio repository, follow Github instructions
on how to clone a project using git.
The project is structured so that:

• Each Presidio service has a designated directory:
  • The service logic.
  • Tests, both unit and integration.
  • Serving it as an HTTP service (found in app.py).
  • Python Packaging setup script (setup.py).
• In the project root directory, you will find common code for using, serving and testing Presidio
  as a cluster of services, as well as CI/CD pipelines codebase and documentation.

Setting up uv Copy Section

uv is the Python package manager used to manage dependencies and virtual
environments for Presidio services (the CI installs dependencies the same way).
Follow these steps when starting to work on a Presidio service with uv:

1. Install uv

   • Using the standalone installer (you can inspect install.sh before running it)

     sh Copy

     curl -LsSf https://astral.sh/uv/install.sh | sh

   • Using Homebrew (in MacOS)

     text Copy

     brew install uv

   Additional installation instructions for uv: https://docs.astral.sh/uv/getting-started/installation/

2. Have uv create a virtualenv for the project and install all requirements in the pyproject.toml,
   including dev requirements.

   For example, in the presidio-analyzer folder, run:

   text Copy

   uv sync --all-extras --group dev

   Add --locked to install the exact locked dependency graph (matching CI behavior).

3. Run all tests:

   text Copy

   uv run pytest

4. To run arbitrary scripts within the virtual env, start the command with
   uv run. For example:

   1. uv run ruff check
   2. uv run pip freeze
   3. uv run python -m spacy download en_core_web_lg

   Command 3 downloads the default spacy model needed for Presidio Analyzer.

Alternatively, [activate the virtual environment](https://docs.astral.sh/uv/pip/environments/#using-a-virtual-environment) and run the commands without the `uv run` prefix.

Development guidelines Copy Section

• A Github issue suggesting the change should be opened prior to a PR.
• All contributions should be documented, tested and linted. Please verify that all tests and lint checks pass successfully before proposing a change.
• To make the linting process easier, you can use pre-commit hooks to verify and automatically format code upon a git commit
• In order for a pull request to be accepted, the CI (containing unit tests, e2e tests and linting) needs to succeed, in addition to approvals from two maintainers.
• PRs should be small and solve/improve one issue at a time. If you have multiple suggestions for improvement, please open multiple PRs.

Local build process Copy Section

After modifying presidio codebase, you might want to build presidio cluster locally, and run tests to spot regressions.
The recommended way of doing so is using docker-compose (bundled with 'Docker Desktop' for Windows and Mac systems,
more information can be found here).
Once installed, to start presidio cluster with all of its services in HTTP mode, run from the project root:

docker-compose up --build -d

!!! note "Note"
Building for the first time might take some time,
mainly on downloading the default spacy models.

To validate that the services were built and started successfully,
and to see the designated port for each,
use docker-compose ps:

>docker-compose ps
CONTAINER ID   IMAGE                       COMMAND                  CREATED         STATUS         PORTS                    NAMES
6d5a258d19c2   presidio-anonymizer         "./entrypoint.sh"   6 minutes ago   Up 6 minutes   0.0.0.0:5001->5001/tcp   presidio_presidio-anonymizer_1
9aad2b68f93c   presidio-analyzer           "./entrypoint.sh"   2 days ago      Up 6 minutes   0.0.0.0:5002->5001/tcp   presidio_presidio-analyzer_1
1448dfb3ec2b   presidio-image-redactor     "./entrypoint.sh"   2 seconds ago   Up 2 seconds   0.0.0.0:5003->5001/tcp   presidio_presidio-image-redactor_1

Edit docker-compose.yml configuration file to change the default ports.

Starting part of the cluster, or one service only, can be done by stating its image name as argument for docker-compose.
For example for analyzer service:

docker-compose up --build -d presidio-analyzer

Testing Copy Section

We strive to have a full test coverage in Presidio, and expect every pull request to
include tests.

In each service directory, a 'test' directory can be found. In it, both unit tests,
for testing single files or classes, and integration tests, for testing integration
between the service components, or integration with external packages.

Basic conventions

For tests to be consistent and predictable, we use the following basic conventions:

1. Treat tests as production code. Keep the tests concise and readable, with descriptive namings.
2. Assert on one behavior at a time in each test.
3. Test names should follow a pattern of test_when_[condition_to_test]_then_[expected_behavior].
   For example: test_given_an_unknown_entity_then_anonymize_uses_defaults.
4. Use test doubles and mocks
   when writing unit tests. Make less use of them when writing integration tests.

Running tests

Presidio uses the pytest framework for testing.
See the pytest documentation
for more information.

Running the tests locally can be done in two ways:

1. Using cli, from each service directory, run:

   sh Copy

   uv run pytest

2. Using your IDE.
   See configuration examples for
   JetBrains PyCharm / IntelliJ IDEA
   and Visual Studio Code

End-to-end tests

Since Presidio services can function as HTTP servers, Presidio uses an additional
end-to-end (e2e) testing layer to test their REST APIs.
This e2e test framework is located under 'e2e-tests' directory.
In it, you can also find test scenarios testing the integration between
Presidio services through REST API.
These tests should be annotated with 'integration' pytest marker @pytest.mark.integration,
while tests calling a single service API layer should be annotated with 'api'
pytest marker @pytest.mark.api.

Running the e2e-tests locally can be done in two ways:

1. Using cli, from e2e-tests directory, run:

   On Mac / Linux / WSL:

   sh Copy

   # Create a virtualenv named presidio-e2e (needs to be done only on the first run)
   python -m venv presidio-e2e
   # Activate the virtualenv
   source presidio-e2e/bin/activate
   # Install e2e-tests requirements using pip
   pip install -r requirements.txt
   # Run pytest
   pytest
   # Deactivate the virtualenv
   deactivate

   On Windows CMD / Powershell:

   shell Copy

   # Create a virtualenv named presidio-e2e (needs to be done only on the first run)
   py -m venv presidio-e2e
   # Activate the virtualenv
   presidio-e2e\Scripts\activate
   # Install e2e-tests requirements using pip
   pip install -r requirements.txt
   # Run pytest
   pytest
   # Deactivate the virtualenv
   deactivate

2. Using your IDE

   See references in the section above.

!!! note "Note"
The e2e tests require a Presidio cluster to be up, for example using the containerized cluster with docker-compose.

Linting Copy Section

Presidio services are PEP8 compliant and continuously enforced on style guide issues during the build process using ruff, in turn running flake8 and other linters.

Running ruff locally, using uv run ruff check, you can check for those issues prior to committing a change.

Ruff runs linters in addition to the basic flake8 functionality, Presidio uses linters as part as ruff such as:

• pep8-naming: To check that variable names are PEP8 compliant.
• flake8-docstrings: To check that docstrings are compliant.

Automatically format code and check for code styling Copy Section

To make the linting process easier, you can use pre-commit hooks to verify and automatically format code upon a git commit, using ruff-format:

1. Install pre-commit package manager locally.

2. From the project's root, enable pre-commit, installing git hooks in the .git/ directory by running: pre-commit install.

3. Commit non PEP8 compliant code will cause commit failure and automatically
   format your code using, as well as checking code formatting using ruff

[INFO] Initializing environment for https://github.com/astral-sh/ruff-pre-commit.
[INFO] Installing environment for https://github.com/astral-sh/ruff-pre-commit.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
ruff.....................................................................Passed
ruff-format..............................................................Failed
- hook id: ruff-format
- files were modified by this hook
  5 files reformatted, 4 files left unchanged

4. Committing again will finish successfully, with a well-formatted code.

Faq (`docs/faq.md`) Copy Section

Frequently Asked Questions (FAQ)

• General
  • What is Presidio?
  • Why was Presidio created?
  • Who owns and maintains Presidio?
  • What is the difference between Presidio and different PII detection services like Azure AI Language, Azure Health Data Services, and Amazon Comprehend?
• Using Presidio
  • How can I start using Presidio?
  • What are the main building blocks in Presidio?
• Customizing Presidio
  • How can Presidio be customized to my needs?
  • What NLP frameworks does Presidio support?
  • Can Presidio be used for Pseudonymization?
  • Does Presidio work on structured/tabular data?
• Improving detection accuracy
  • What can I do if Presidio does not detect some of the PII entities in my data (False Negatives)?
  • What can I do if Presidio falsely detects text as PII entities (False Positives)?
  • How can I evaluate the performance of my Presidio instance?
• Deployment
  • How can I deploy Presidio into my environment?
• Contributing
  • How can I contribute to Presidio?
  • How can I report security vulnerabilities?

General Copy Chapter

What is Presidio? Copy Section

Presidio (Origin from Latin praesidium ‘protection, garrison’) helps to ensure sensitive data is properly managed and governed. It provides fast identification and anonymization modules for private entities in text and images. It is fully customizable and pluggable, can be adapted to your needs and be deployed into various environments.

!!! note "Note"
Presidio is a library or SDK rather than a service. It is meant to be customized to the user's or organization's specific needs.

!!! warning "Warning"
Presidio can help identify sensitive/PII data in un/structured text. However, because it is using automated detection mechanisms, there is no guarantee that Presidio will find all sensitive information. Consequently, additional systems and protections should be employed.

Why was Presidio created? Copy Section

By developing Presidio, our goals are:

1. Allow organizations to preserve privacy in a simpler way by democratizing de-identification technologies and introducing transparency in decisions.
2. Embrace extensibility and customizability to a specific business need.
3. Facilitate both fully automated and semi-automated PII de-identification flows on multiple platforms.

Who owns and maintains Presidio? Copy Section

Presidio is a community-governed open source project under the Data Privacy Stack organization. It was originally created at Microsoft and has since transitioned to an independent, vendor-neutral project maintained by contributors and volunteers from across the community. See the project transition update for more details.

!!! note "Note"
Presidio is not an official product of any company and comes with no warranty or SLA. Usage terms are defined in the repository's license.

What is the difference between Presidio and different PII detection services like Azure AI Language, Azure Health Data Services, and Amazon Comprehend? Copy Section

In a nutshell, Presidio is a library which is meant to be customized, whereas different SaaS tools for PII detection have less customization capabilities. Most of these SaaS offerings use dedicated ML models and other logic for PII detection and often have better entity coverage or accuracy than Presidio.

Based on our internal research, leveraging Presidio in parallel to 3rd party PII detection services like Azure AI Language can bring optimal results mainly when the data in hand has entity types or values not supported by the 3rd party service. (see example here).

Using Presidio Copy Chapter

How can I start using Presidio? Copy Section

1. Check out the installation docs.
2. Take a look at the different samples.
3. Try the demo website.

What are the main building blocks in Presidio? Copy Section

Presidio is a suite built of several packages and building blocks:

1. Presidio Analyzer: a package for detecting PII entities in natural language.
2. Presidio Anonymizer: a package for manipulating PII entities in text (e.g. remove, redact, hash, encrypt).
3. Presidio Image Redactor: A package for detecting PII entities in image using OCR.
4. Presidio Structured: A package for detecting PII entities in structured/semi-structured data.
5. A set of sample deployments as Python packages or Docker containers for Kubernetes, Azure Data Factory, Spark and more.

Customizing Presidio Copy Chapter

How can Presidio be customized to my needs? Copy Section

Users can customize Presidio in multiple ways:

1. Create new or updated PII recognizers (docs).
2. Adapt Presidio to new languages (docs).
3. Leverage state of the art Named Entity Recognition models (docs).
4. Add new types of anonymizers (docs).
5. Create PII analysis and anonymization pipelines on different environments using Docker or Python (samples).

And more.

What NLP frameworks does Presidio support? Copy Section

Presidio supports spaCy version 3+ for Named Entity Recognition, tokenization, lemmatization and more. We also support Stanza using the spacy-stanza package, and it is further possible to create PII recognizers leveraging other frameworks like transformers or Flair.

For more information, see the docs.

Can Presidio be used for Pseudonymization? Copy Section

Pseudonymization is a de-identification technique in which the real data is replaced with fake data in a reversible way. Since there are various ways and approaches for this, we provide a simple sample which can be extended for more sophisticated usage. If you have a question or a request on this topic, please open an issue on the repo.

Does Presidio work on structured/tabular data? Copy Section

Presidio-structured is a capability in Presidio for detecting PII entities in structured/semi-structured data. It scans datasets for PII using Presidio Analyzer, and supports the redaction of text, cells, or columns in a tabular dataset.

Improving detection accuracy Copy Chapter

What can I do if Presidio does not detect some of the PII entities in my data (False Negatives)? Copy Section

Presidio comes loaded with several PII recognizers (see list here),
however its main strength lies in its customization capabilities to new entities, specific datasets, file types, languages or use cases.

What can I do if Presidio falsely detects text as PII entities (False Positives)? Copy Section

Some PII recognizers are less specific than others. A driver's license number, for example, could be any 9-digit number.
While Presidio leverages context words and other logic to improve the detection quality,
it could still falsely detect non-entity values as PII entities.

In order to avoid false positives, one could try to:

1. Change the acceptance threshold, which defines what is the minimum confidence value for a detected entity to be returned.
2. Remove unnecessary PII recognizers, if the dataset does not contain these entities.
3. Update/replace the logic of specific recognizers to better suit a specific dataset or use case.
4. Replace PII recognizers with those coming from 3rd party services.

Every PII identification logic would have its errors, and there is a trade-off between false positives (falsely detected text) and false negatives (PII entities which are not detected).

How can I evaluate the performance of my Presidio instance? Copy Section

In addition to Presidio, we maintain a repo focused on evaluation of models and PII recognizers here. It also features a simple PII data generator.

Deployment Copy Chapter

How can I deploy Presidio into my environment? Copy Section

The main Presidio modules (analyzer, anonymizer, image-redactor) can be used both as a Python package and as a dockerized REST API. See the different deployment samples for example deployments.

!!! warning "Authentication and Authorization"
Presidio API endpoints do not include built-in authentication by design. The containers are intentionally kept lean to allow flexibility for different deployment scenarios. Authentication and authorization should be implemented at a separate infrastructure layer (e.g., an API gateway, reverse proxy, or service mesh) according to your specific use case and security requirements. It is strongly recommended not to expose Presidio services directly to untrusted networks without an authentication layer in front of them.

Contributing Copy Chapter

How can I contribute to Presidio? Copy Section

First, review the contribution guidelines, and feel free to reach out by opening an issue, posting a discussion or emailing us at

How can I report security vulnerabilities? Copy Section

Please see the security information.

Getting Started (`docs/getting_started.md`) Copy Section

Getting started with Presidio

The core functionality in Presidio is to detect PII in text. Presidio further contains a set of tools that build on top of text PII detection, for example in images, structured data, JSON and more.

• For a quickstart for PII detection and de-identification in text click here.
• For a quickstart for PII detection and de-identification in images click here.
• For a quickstart for PII detection and de-identification in structured and semi-structured data click here.

Installation (`docs/installation.md`) Copy Section

Installing Presidio

Description Copy Chapter

This document describes the installation of the entire
Presidio suite using pip (as Python packages) or using Docker (As containerized services).

Using pip Copy Chapter

!!! note "Note"

Consider installing the Presidio python packages
in a virtual environment like [venv](https://docs.python.org/3/tutorial/venv.html)
or [conda](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html).

Supported Python Versions Copy Section

Presidio is supported for the following python versions:

• 3.10
• 3.11
• 3.12
• 3.13
• 3.14

PII anonymization on text Copy Section

For PII anonymization on text, install the presidio-analyzer and presidio-anonymizer packages
with at least one NLP engine (spaCy, transformers or stanza):

===+ "spaCy (default)"

```
pip install presidio_analyzer
pip install presidio_anonymizer
python -m spacy download en_core_web_lg
```

=== "Transformers"

```
pip install "presidio_analyzer[transformers]"
pip install presidio_anonymizer
python -m spacy download en_core_web_sm
```

!!! note "Note"
    
    When using a transformers NLP engine, Presidio would still use spaCy for other capabilities,
    therefore a small spaCy model (such as en_core_web_sm) is required. 
    Transformers models would be loaded lazily. To pre-load them, see: [Downloading a pre-trained model](./analyzer/nlp_engines/transformers.md#downloading-a-pre-trained-model)

=== "Stanza"

```
pip install "presidio_analyzer[stanza]"
pip install presidio_anonymizer
```


!!! note "Note"
    
    Stanza models would be loaded lazily. To pre-load them, see: [Downloading a pre-trained model](./analyzer/nlp_engines/spacy_stanza.md#download-the-pre-trained-model).

GPU acceleration (optional) Copy Section

For GPU acceleration, install the appropriate dependencies for your hardware:

• Linux with NVIDIA GPU: pip install "spacy[cuda12x]" (or the version matching your CUDA installation)
• macOS with Apple Silicon: MPS is detected automatically, no additional dependencies required.

For detailed GPU setup, verification, and troubleshooting, see GPU Acceleration.

PII redaction in images Copy Section

For PII redaction in images

1. Install the presidio-image-redactor package:

   sh Copy

   pip install presidio_image_redactor
   
   # Presidio image redactor uses the presidio-analyzer
   # which requires a spaCy language model:
   python -m spacy download en_core_web_lg

2. Install an OCR engine. The default version uses the Tesseract OCR Engine.
   More information on installation can be found here.

Using Docker Copy Chapter

Presidio can expose REST endpoints for each service using Flask and Docker.
To download the Presidio Docker containers, run the following command:

!!! note "Note"

This requires Docker to be installed. [Download Docker](https://docs.docker.com/get-docker/).

!!! important "Container registry moved to GitHub Packages"

New Presidio container releases are published to [GitHub Container Registry](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry) under the [Data Privacy Stack packages organization](https://github.com/orgs/data-privacy-stack/packages). The legacy Microsoft Container Registry images at `mcr.microsoft.com/presidio-*` are no longer updated. If you previously used `mcr.microsoft.com/presidio-analyzer:latest`, switch to `ghcr.io/data-privacy-stack/presidio-analyzer:latest`; for production, prefer pinning an explicit release tag from the package page.

For PII anonymization in text Copy Section

For PII detection and anonymization in text, the presidio-analyzer
and presidio-anonymizer modules are required.

# Download Docker images
docker pull ghcr.io/data-privacy-stack/presidio-analyzer
docker pull ghcr.io/data-privacy-stack/presidio-anonymizer

# Run containers with default ports
docker run -d -p 5002:3000 ghcr.io/data-privacy-stack/presidio-analyzer:latest

docker run -d -p 5001:3000 ghcr.io/data-privacy-stack/presidio-anonymizer:latest

For PII redaction in images Copy Section

For PII detection in images, the presidio-image-redactor is required.

# Download Docker image
docker pull ghcr.io/data-privacy-stack/presidio-image-redactor

# Run container with the default port
docker run -d -p 5003:3000 ghcr.io/data-privacy-stack/presidio-image-redactor:latest

Once the services are running, their APIs are available.
API reference and example calls can be found here.

Install from source Copy Chapter

To install Presidio from source, first clone the repo:

• using HTTPS

git clone https://github.com/data-privacy-stack/presidio.git

• Using SSH

git clone [email protected]:data-privacy-stack/presidio.git

Then, build the containers locally.

!!! note "Note"
Presidio uses docker-compose to manage the different Presidio containers.

From the root folder of the repo:

docker-compose up --build

Alternatively, you can build and run individual services.
For example, for the presidio-anonymizer service:

docker build ./presidio-anonymizer -t presidio/presidio-anonymizer

And run:

docker run -d -p 5001:5001 presidio/presidio-anonymizer

For more information on developing locally,
refer to the setting up a development environment section.

Presidio V2 (`docs/presidio_V2.md`) Copy Section

Presidio Revamp (aka V2)

As of March 2021, Presidio underwent a revamp to a new version referred to as V2.

The main changes introduced in V2 are:

1. gRPC replaced with HTTP to allow more customizable APIs and easier debugging

2. Focus on the Analyzer and Anonymizer services.

   1. Presidio Anonymizer is now Python-based and pip-installable.
   2. Presidio Analyzer does not use templates and external recognizer store.
   3. Image Redactor (formerly presidio-image-anonymizer) is in early beta and is Python based and pip installable.
   4. Other services are deprecated and may be migrated over time to V2 with the help of the community.

3. Improved documentation, sample code, and build workflows.

4. Format-Preserving Encryption replaced with Advanced Encryption Standard (AES)

V1 Availability Copy Chapter

Version V1 (legacy) is still available for download. To continue using the previous version:

• For docker containers, use tag=v1
• For python packages, download version < 2 (e.g. pip install presidio-analyzer==0.95)

!!! note "Note"
The legacy V1 code base will continue to be available under branch V1 but will no longer be officially supported.

API Changes Copy Chapter

The move from gRPC to HTTP-based APIs included changes to the API requests.

1. Changed payload format – moving from structured objects to JSON.

2. Removed templates from the API, including flattening the JSON structure.

3. Using snake_case instead of camelCase .

Below is a detailed outline of all changes made to the Analyzer and Anonymizer.

Analyzer API Changes Copy Section

Legacy json request (gRPC)

{
    "text": "My phone number is 212-555-5555",
    "AnalyzeTemplateId": "1234",
    "AnalyzeTemplate": {
        "Fields": [
            {
                "Name": "PHONE_NUMBER",
                "MinScore": "0.5"
            }
        ],
        "AllFields": true,
        "Description": "template description",
        "CreateTime": "template creation time",
        "ModifiedTime": "template modification time",
        "Language": "fr",
        "ResultsScoreThreshold": 0.5
    }
}

V2 json request (HTTP)

{
    "text": "My phone number is 212-555-5555",
    "entities": ["PHONE_NUMBER"],
    "language": "en",
    "correlation_id": "213",
    "score_threshold": 0.5,
    "trace": true,
    "return_decision_process": true
}

Anonymizer API Changes Copy Section

Legacy json request (gRPC)

{
  "text": "hello world, my name is Jane Doe. My number is: 034453334",
  "template": {
    "description": "DEPRECATED",
    "create_time": "DEPRECATED",
    "modified_time": "DEPRECATED",
    "default_transformation": {
      "replace_value": {...},
      "redact_value": {...},
      "hash_value": {...},
      "mask_value": {...},
      "fpe_value": {...}
    },
    "field_type_transformations": [
      {
        "fields": [
          {
            "name": "FIRST_NAME",
            "min_score": "0.2"
          }
        ],
        "transformation": {
          "replace_value": {...},
          "redact_value": {...},
          "hash_value": {...},
          "mask_value": {...},
          "fpe_value": {...}
        }
      }
    ],
    "analyze_results": [
      {
        "text": "Jane",
        "field": {
          "name": "FIRST_NAME",
          "min_score": "0.5"
        },
        "location": {
          "start": 24,
          "end": 32,
          "length": 6
        },
        "score": 0.8
      }
    ]
  }
}

V2 json request (HTTP)

{
    "text": "hello world, my name is Jane Doe. My number is: 034453334",
    "anonymizers": {
        "DEFAULT": {
            "type": "replace",
            "new_value": "val"
        },
        "PHONE_NUMBER": {
            "type": "mask",
            "masking_char": "*",
            "chars_to_mask": 4,
            "from_end": true
        }
    },
    "analyzer_results": [
        {
            "start": 24,
            "end": 32,
            "score": 0.8,
            "entity_type": "NAME"
        },
        {
            "start": 24,
            "end": 28,
            "score": 0.8,
            "entity_type": "FIRST_NAME"
        },
        {
            "start": 29,
            "end": 32,
            "score": 0.6,
            "entity_type": "LAST_NAME"
        },
        {
            "start": 48,
            "end": 57,
            "score": 0.95,
            "entity_type": "PHONE_NUMBER"
        }
    ]
}

Specific for each anonymization type:

!!! note "Note"
The V2 API is continuously evolving. please follow the change log for updates.

Project Transition (`docs/project_transition.md`) Copy Section

Presidio has transitioned to a community-owned project

We want to share an important update regarding the future of Presidio.

The Presidio project has completed its transition from a Microsoft-owned project to an independent, community-governed open source project under the GitHub organization Data Privacy Stack. Its official URL is: https://github.com/data-privacy-stack/presidio

Microsoft supported this transition and the continued success of the Presidio project as an open source community initiative.

Our goal remains to ensure Presidio serves the privacy engineering and data protection community as an open, transparent, and vendor-neutral project.

What this means Copy Chapter

• Presidio continues to be open source under the MIT license.
• Existing functionality, APIs, and documentation remain available.
• The project is maintained by contributors and volunteers from the community.
• The project is not owned or operated by a commercial entity.
• The focus on privacy, responsible AI, and data protection tooling remains unchanged.

Microsoft and Azure users Copy Chapter

This transition does not change the experience for users of Presidio, including organizations using Presidio alongside Microsoft and Azure services.

Integrations, dependencies, and usage patterns continue to work as expected.

Why this change Copy Chapter

Over the years, Presidio has grown beyond its original scope and has been adopted by organizations, researchers, and developers across industries. Moving to an independent community initiative positions the project for long-term sustainability, broader collaboration, and open governance.

What to expect next Copy Chapter

Now that the migration is complete:

• Repositories, documentation links, and package references have moved to the new organization. Please update your references.
• Legacy GitHub URLs and documentation links redirect to the new location.
• New Docker image releases are published to GitHub Container Registry under the Data Privacy Stack organization. Legacy Microsoft Container Registry (MCR) images remain available for older tags but are no longer updated; update mcr.microsoft.com/presidio-* image references to ghcr.io/data-privacy-stack/presidio-*.
• Contribution and governance processes are now community-driven.
• The Technical Steering Committee (TSC) includes maintainers and contributors from across the community.

Maintainers and contributors Copy Chapter

Maintainers and contributors continue to actively drive the project. We welcome new contributors who care about privacy-preserving technologies and responsible data handling.

Details regarding contribution guidelines and Docker image publishing are available in the repository documentation.

Thank you to everyone who supported Presidio through this transition and helped make it a valuable resource for the privacy engineering community.

Requirements Docs (`docs/requirements-docs.txt`) Copy Section

Dependencies for building the Presidio documentation with Zensical, the

successor to Material for MkDocs. Build with scripts/zensical_build.py, which

pre-converts the notebook samples and emits a generated zensical.yml. See that

script's module docstring for the full flow.

pip install -r docs/requirements-docs.txt

python scripts/zensical_build.py # build to ./site

python scripts/zensical_build.py serve # local preview

Zensical does not run MkDocs plugins, so mkdocs / mkdocs-material /

mkdocs-jupyter are intentionally absent. nbconvert renders the notebooks and

mkdocstrings-python + the presidio packages back the API reference pages.

zensical
nbconvert
mkdocstrings-python
pymdown-extensions
markdown
pyyaml
presidio_analyzer
presidio_anonymizer
presidio_image_redactor
presidio_structured
pygments>=2.10
black

Supported Entities (`docs/supported_entities.md`) Copy Section

PII entities supported by Presidio

Presidio contains predefined recognizers for PII entities.
This page describes the different entities Presidio can detect and the method Presidio employs to detect those.

In addition, Presidio allows you to add custom entity recognizers.
For more information, refer to the adding new recognizers documentation.

List of supported entities Copy Chapter

Global Copy Section

USA Copy Section

UK Copy Section

Spain Copy Section

Italy Copy Section

Poland Copy Section

Singapore Copy Section

Australia Copy Section

India Copy Section

Finland Copy Section

Korea Copy Section

Nigeria Copy Section

Philippines Copy Section

Canada Copy Section

Sweden Copy Section

South Africa Copy Section

Thai Copy Section

Turkey Copy Section

Philippines Copy Section

Germany Copy Section

Medical / Clinical Copy Section

Detected using the MedicalNERRecognizer (requires the transformers extra). Uses the blaze999/Medical-NER model by default.

Adding a custom PII entity Copy Chapter

See this documentation for instructions on how to add a new Recognizer for a new type of PII entity.

Complementing Presidio with Azure AI Language PII Copy Chapter

Azure AI Language PII
is a cloud-based service that provides Natural Language Processing (NLP) features for detecting PII in text.

A list of supported entities by Azure AI Language PII can be found here.

To add Azure AI language into Presidio, see this sample.

Complementing Presidio with Azure Health Data Services PHI Copy Chapter

Azure Health Data Services PHI is a cloud-based service that provides Natural Language Processing (NLP) features for detecting PHI in text.

A list of supported entities by Azure Health Data Services PHI can be found here.

To add Azure AI language into Presidio, see this sample.

Connecting to 3rd party PII detectors Copy Section

See this documentation for instructions on how to implement an external PII detector for a new or existing type of PII entity.

Text Anonymization (`docs/text_anonymization.md`) Copy Section

Text anonymization

Presidio's features two main modules for anonymization PII in text:

• Presidio analyzer: Identification of PII in text
• Presidio anonymizer: De-identify detected PII entities using different operators

In most cases, we would run the Presidio analyzer to detect where PII entities exist,
and then the Presidio anonymizer to remove those using specific operators (such as redact, replace, hash or encrypt)

This figure presents the overall flow in high level:

• The Presidio Analyzer holds multiple recognizers, each one capable of detecting specific PII entities. These recognizers leverage regular expressions, deny lists, checksum, rule based logic, Named Entity Recognition ML models and context from surrounding words.
• The Presidio Anonymizer holds multiple operators, each one can be used to anonymize the PII entity in a different way. Additionally, it can be used to de-anonymize an already anonymized entity (For example, decrypt an encrypted entity)

Analyzer Adding Recognizers (`docs/analyzer/adding_recognizers.md`) Copy Section

Supporting detection of new types of PII entities

Presidio can be extended to support detection of new types of PII entities, and to support additional languages.
These PII recognizers could be added via code or ad-hoc as part of the request.

Introduction to recognizer development Copy Chapter

Entity recognizers are Python objects capable of detecting one or more entities in a specific language.
In order to extend Presidio's detection capabilities to new types of PII entities,
these EntityRecognizer objects should be added to the existing list of recognizers.

Types of recognizer classes in Presidio Copy Chapter

The following class diagram shows the different types of recognizer families Presidio contains.

• The EntityRecognizer is an abstract class for all recognizers.
• The RemoteRecognizer is an abstract class for calling external PII detectors.
  See more info here.
• The abstract class LocalRecognizer is implemented by all recognizers running within the Presidio-analyzer process.
• The PatternRecognizer is an class for supporting regex and deny-list based recognition logic,
  including validation (e.g., with checksum) and context support. See an example here.

Extending the analyzer for additional PII entities Copy Chapter

1. Create a new class based on EntityRecognizer.
2. Add the new recognizer to the recognizer registry so that the AnalyzerEngine can use the new recognizer during analysis.

Simple example Copy Section

For simple recognizers based on regular expressions or deny-lists,
we can leverage the provided PatternRecognizer:

from presidio_analyzer import PatternRecognizer
titles_recognizer = PatternRecognizer(supported_entity="TITLE",
                                      deny_list=["Mr.","Mrs.","Miss"])

Calling the recognizer itself:

titles_recognizer.analyze(text="Mr. Schmidt", entities="TITLE")

Adding it to the list of recognizers:

from presidio_analyzer import AnalyzerEngine, RecognizerRegistry

registry = RecognizerRegistry()
registry.load_predefined_recognizers()

# Add the recognizer to the existing list of recognizers
registry.add_recognizer(titles_recognizer)

# Set up analyzer with our updated recognizer registry
analyzer = AnalyzerEngine(registry=registry)

# Run with input text
text="His name is Mr. Jones"
results = analyzer.analyze(text=text, language="en")
print(results)

Alternatively, we can add the recognizer directly to the existing registry:

from presidio_analyzer import AnalyzerEngine

analyzer = AnalyzerEngine()

analyzer.registry.add_recognizer(titles_recognizer)

results = analyzer.analyze(text=text, language="en")
print(results)

For pattern based recognizers, it is possible to change the regex flags, either for
one recognizer or for all.
For one recognizer, use the global_regex_flags parameter
in the PatternRecognizer constructor.
For all recognizers, use the global_regex_flags parameter in the RecognizerRegistry constructor:

from presidio_analyzer import AnalyzerEngine, RecognizerRegistry

import regex as re

registry = RecognizerRegistry(global_regex_flags=re.DOTALL | re.MULTILINE | re.IGNORECASE)
engine = AnalyzerEngine(registry=registry)
engine.analyze(...)

Creating a new `EntityRecognizer` in code Copy Section

To create a new recognizer via code:

1. Create a new Python class which implements LocalRecognizer.
   (LocalRecognizer implements the base EntityRecognizer class)

   This class has the following functions:

   i. load: load a model / resource to be used during recognition

   python Copy

   def load(self)

   ii. analyze: The main function to be called for getting entities out of the new recognizer:

   python Copy

   def analyze(self, text, entities, nlp_artifacts)

   Notes:

   1. Each recognizer has access to different NLP assets such as tokens, lemmas, and more.
      These are given through the nlp_artifacts parameter.
      Refer to the source code for more information.

   2. The analyze method should return a list of RecognizerResult.

2. Add it to the recognizer registry using registry.add_recognizer(my_recognizer).

For more examples, see the Customizing Presidio Analyzer jupyter notebook.

Creating a remote recognizer Copy Section

A remote recognizer is an EntityRecognizer object interacting with an external service. The external service could be a 3rd party PII detection service or a custom service deployed in parallel to Presidio.

Sample implementation of a RemoteRecognizer.
In this example, an external PII detection service exposes two APIs: detect and supported_entities. The class implemented here, ExampleRemoteRecognizer, uses the requests package to call the external service via HTTP.

In this code snippet, we simulate the external PII detector by using the Presidio analyzer. In reality, we would adapt this code to fit the external PII detector we have in hand.

For an example of integrating a RemoteRecognizer with Presidio-Analyzer, see this example.

Creating pre-defined recognizers Copy Section

Once a recognizer is created, it can either be added to the RecognizerRegistry via the add_recognizer method, or it could be added into the list of predefined recognizers.
To add a recognizer to the list of pre-defined recognizers:

1. Clone the repo.
2. Create a file containing the new recognizer Python class.
3. Add the recognizer to the recognizers in the default_recognizers config. Details of recognizer parameters are given Here.
4. If the recognizer is country-specific (file lives under predefined_recognizers/country_specific/<country>/), declare its country by setting the class-level COUNTRY_CODE = "" attribute. This makes the recognizer participate in the country filter — see Filtering recognizers by country. Generic / locale-agnostic recognizers leave COUNTRY_CODE unset (defaults to None).
5. Optional: Update documentation (e.g., the supported entities list).

Azure AI Language recognizer Copy Section

On how to integrate Presidio with Azure AI Language PII detection service,
and a sample for a Text Analytics Remote Recognizer, refer to the
Azure Text Analytics Integration document.

Azure Health Data Services (AHDS) de-identification recognizer Copy Section

On how to integrate Presidio with AHDS De-Identification Protected Health Information (PHI) detection service,
and a sample for a ADHS Remote Recognizer, refer to the
AHDS de-Identification Integration document.

Language Model-based PII/PHI detection recognizer Copy Section

Presidio supports language model-based entity detection using LLMs and SLMs for flexible PII/PHI recognition.

The current implementation uses LangExtract with Ollama (local models). For full setup instructions and usage examples,
see the Language Model-based PII/PHI Detection guide.

Creating ad-hoc recognizers Copy Section

In addition to recognizers in code, it is possible to create ad-hoc recognizers via the Presidio Analyzer API for regex and deny-list based logic.
These recognizers, in JSON form, are added to the /analyze request and are only used in the context of this request.

• The json structure for a regex ad-hoc recognizer is the following:

  json Copy

  {
      "text": "John Smith drivers license is AC432223. Zip code: 10023",
      "language": "en",
      "ad_hoc_recognizers":[
          {
          "name": "Zip code Recognizer",
          "supported_language": "en",
          "patterns": [
              {
              "name": "zip code (weak)", 
              "regex": "(\\b\\d{5}(?:\\-\\d{4})?\\b)", 
              "score": 0.01
              }
          ],
          "context": ["zip", "code"],
          "supported_entity":"ZIP"
          }
      ]
  }

• The json structure for a deny-list based recognizers is the following:

  json Copy

  {
      "text": "Mr. John Smith's drivers license is AC432223",
      "language": "en",
      "ad_hoc_recognizers":[
          {
          "name": "Mr. Recognizer",
          "supported_language": "en",
          "deny_list": ["Mr", "Mr.", "Mister"],
          "supported_entity":"MR_TITLE"
          },
          {
          "name": "Ms. Recognizer",
          "supported_language": "en",
          "deny_list": ["Ms", "Ms.", "Miss", "Mrs", "Mrs."],
          "supported_entity":"MS_TITLE"
          }
      ]
  }

In both examples, the /analyze request is extended with a list of ad_hoc_recognizers, which could be either patterns, deny_list or both.

Additional examples can be found in the OpenAPI spec.

Reading pattern recognizers from YAML Copy Section

Recognizers can be loaded from a YAML file, which allows users to add recognition logic without writing code.
An example YAML file can be found here.

Once the YAML file is created, it can be loaded into the RecognizerRegistry instance.

This example creates a RecognizerRegistry holding only the recognizers in the YAML file:

from presidio_analyzer import AnalyzerEngine
from presidio_analyzer.recognizer_registry import RecognizerRegistryProvider

recognizer_registry_conf_file = "./analyzer/recognizers-config.yml"

provider = RecognizerRegistryProvider(
                conf_file=recognizer_registry_conf_file
            )
registry = provider.create_recognizer_registry()
analyzer = AnalyzerEngine(registry=registry)

results = analyzer.analyze(text="My name is Morris", language="en")
print(results)

This example adds the new recognizers to the predefined recognizers in Presidio:

from presidio_analyzer import AnalyzerEngine, RecognizerRegistry

yaml_file = "recognizers.yaml"
registry = RecognizerRegistry()
registry.load_predefined_recognizers()

registry.add_recognizers_from_yaml(yaml_file)

analyzer = AnalyzerEngine()
analyzer.analyze(text="Mr. and Mrs. Smith", language="en")

Further reading:

1. PII detection in different languages.
2. Customizing the NLP model.
3. Best practices for developing PII recognizers.
4. Code samples for customizing Presidio Analyzer with new recognizers.

Analyzer Analyzer Engine Provider (`docs/analyzer/analyzer_engine_provider.md`) Copy Section

Configuring the Analyzer Engine from file

Presidio uses AnalyzerEngineProvider to load AnalyzerEngine configuration from file.
Configuration can be loaded in three different ways:

Using a single file Copy Chapter

Create an AnalyzerEngineProvider using a single configuration file and set its path to analyzer_engine_conf_file, then create AnalyzerEngine based on it:

from presidio_analyzer import AnalyzerEngine, AnalyzerEngineProvider

analyzer_conf_file = "./analyzer/analyzer-config-all.yml"

provider = AnalyzerEngineProvider(
    analyzer_engine_conf_file=analyzer_conf_file
    )
analyzer = provider.create_engine()

results = analyzer.analyze(text="My name is Morris", language="en")
print(results)

An example configuration file:

supported_languages: 
- en
default_score_threshold: 0

nlp_configuration:
  nlp_engine_name: spacy
  models:
  -
    lang_code: en
    model_name: en_core_web_lg
  -
    lang_code: es
    model_name: es_core_news_md
  ner_model_configuration:
    model_to_presidio_entity_mapping:
      PER: PERSON
      PERSON: PERSON
      LOC: LOCATION
      LOCATION: LOCATION
      GPE: LOCATION
      ORG: ORGANIZATION
      DATE: DATE_TIME
      TIME: DATE_TIME
      NORP: NRP

    low_confidence_score_multiplier: 0.4
    low_score_entity_names:
    - ORGANIZATION
    - ORG
    default_score: 0.85

recognizer_registry:
  global_regex_flags: 26
  recognizers: 
  - name: CreditCardRecognizer
    supported_languages: 
      - en
    supported_entity: IT_FISCAL_CODE
    type: predefined
    score_thresholds:
      default: 0.4
      CREDIT_CARD: 0.7

  - name: ItFiscalCodeRecognizer
    type: predefined

The configuration file contains the following parameters:

• supported_languages: A list of supported languages that the analyzer will support.
• default_score_threshold: A score that determines the minimal threshold for detection.
• nlp_configuration: Configuration given to the NLP engine which will detect the PIIs and extract features for the downstream logic.
• recognizer_registry: All the recognizers that will be used by the analyzer. Each recognizer entry can define score_thresholds, using default as its fallback and entity names for overrides.

!!! note "Defining confidence thresholds"
Presidio supports multiple types of score thresholds: default ones (for the analyzer and each recognizer), per-request thresholds, and per-entity thresholds. Supplying analyzer.analyze(score_threshold=...) bypasses recognizer-level thresholds for that request and applies the supplied threshold to every result. The order of importance is: analyzer.analyze(score_threshold=...) > an entity specific threshold > a recognizer default threshold (default) > the Presidio Analyzer default_score_threshold.

!!! note "Note"

`supported_languages` must be identical to the same field in recognizer_registry

Using multiple files Copy Chapter

Create an AnalyzerEngineProvider using three different configuration files for each of the following components:

• Analyzer
• NLP Engine
• Recognizer Registry

!!! note "Note"

Each of these parameters is optional and in case it's not set, the default configuration will be used.

from presidio_analyzer import AnalyzerEngine, AnalyzerEngineProvider

analyzer_conf_file = "./analyzer/analyzer-config.yml"
nlp_engine_conf_file = "./analyzer/nlp-config.yml"
recognizer_registry_conf_file = "./analyzer/recognizers-config.yml"

provider = AnalyzerEngineProvider(
    analyzer_engine_conf_file=analyzer_conf_file,
    nlp_engine_conf_file=nlp_engine_conf_file,
    recognizer_registry_conf_file=recognizer_registry_conf_file,
    )
analyzer = provider.create_engine()

results = analyzer.analyze(text="My name is Morris", language="en")
print(results)

The structure of the configuration files is as follows:

• Analyzer engine configuration file:

  yaml Copy

  supported_languages: 
  - en
  default_score_threshold: 0

• NLP engine configuration file structure is examined thoroughly in the Customizing the NLP model section.

• Recognizer registry configuration file structure is examined thoroughly in the Customizing recognizer registry from file section.

Using the default configuration Copy Chapter

Create an AnalyzerEngineProvider without any parameters. This will load the default configuration:

from presidio_analyzer import AnalyzerEngine, AnalyzerEngineProvider

provider = AnalyzerEngineProvider().create_engine()

results = provider.analyze(text="My name is Morris", language="en")
print(results)

The default configuration of AnalyzerEngine is defined in the following files:

• Analyzer Engine
• NLP Engine
• Recognizer Registry

Enabling and disabling recognizers Copy Chapter

In general, recognizers that are not added to the configuration would not be created, with one exception.

Enabling/Disabling the NLP recognizer Copy Section

One exception to this is the recognizer which extracts the NlpEngine entities (e.g. SpacyRecognizer when the NlpEngine is SpacyNlpEngine; TransformersRecognizer when the engine is TransformersNlpEngine and StanzaRecognizer when the engine is StanzaNlpEngine).

Recognizers (including the NLP recognizer) could be disabled by defining enabled=false in the YAML configuration. For example:

recognizer_registry:
  global_regex_flags: 26
  recognizers:
    - name: SpacyRecognizer
      type: predefined
      enabled: false
    - name: CreditCardRecognizer
      type: predefined
      enabled: true

supported_languages:
  - en
default_score_threshold: 0.7

nlp_configuration:
  nlp_engine_name: spacy
  models:
    -
      lang_code: en
      model_name: en_core_web_lg

In this example, the SpacyRecognizer is disabled, and the CreditCardRecognizer is enabled, resulting in only the CREDIT_CARD PII entity to be returned if detected.

Adding context words in YAML recognizers Copy Section

Recognizers defined in YAML can also include a context field.
When used with AnalyzerEngine and a context enhancer, these words boost the score if they appear near the detected entity.

Example:

recognizers:
  - name: "Date of Birth Recognizer"
    supported_entity: "DATE_TIME"
    supported_language: "en"
    patterns:
      - name: "DOB without slashes"
        regex: "((19|20)\\d{2}(0[1-9]|1[0-2])(0[1-9]|[12]\\d|3[01]))"
        score: 0.8
    context:
      - DOB

/* Detailed source-code truncated for AI context efficiency. */

Analyzer Customizing Nlp Models (`docs/analyzer/customizing_nlp_models.md`) Copy Section

Customizing the NLP engine in Presidio Analyzer

Presidio uses NLP engines for two main tasks: NER based PII identification,
and feature extraction for downstream rule based logic (such as leveraging context words for improved detection).
While Presidio comes with an open-source model (the en_core_web_lg model from spaCy),
additional NLP models and frameworks could be plugged in, either public or proprietary.
These models can be trained or downloaded from existing NLP frameworks like spaCy,
Stanza and
transformers.

In addition, other types of NLP frameworks can be integrated into Presidio.

Setting up a custom NLP model Copy Chapter

• spaCy or stanza
• transformers

Configure Presidio to use the new model Copy Chapter

Configuration can be done in two ways:

• Via code: Create an NlpEngine using the NlpEnginerProvider class, and pass it to the AnalyzerEngine as input:

  python Copy

  from presidio_analyzer import AnalyzerEngine, RecognizerRegistry
  from presidio_analyzer.nlp_engine import NlpEngineProvider
  
  # Create configuration containing engine name and models
  configuration = {
      "nlp_engine_name": "spacy",
      "models": [{"lang_code": "es", "model_name": "es_core_news_md"},
                  {"lang_code": "en", "model_name": "en_core_web_lg"}],
  }
  
  # Create NLP engine based on configuration
  provider = NlpEngineProvider(nlp_configuration=configuration)
  nlp_engine_with_spanish = provider.create_engine()
  
  # Pass the created NLP engine and supported_languages to the AnalyzerEngine
  analyzer = AnalyzerEngine(
      nlp_engine=nlp_engine_with_spanish, 
      supported_languages=["en", "es"]
  )
  
  # Analyze in different languages
  results_spanish = analyzer.analyze(text="Mi nombre es Morris", language="es")
  print(results_spanish)
  
  results_english = analyzer.analyze(text="My name is Morris", language="en")
  print(results_english)

• Via configuration: Set up the models which should be used in the default conf file.

  An example Conf file:

  yaml Copy

  nlp_engine_name: spacy
  models:
      -
      lang_code: en
      model_name: en_core_web_lg
      -
      lang_code: es
      model_name: es_core_news_md 
  ner_model_configuration:
  labels_to_ignore:
  - O
  model_to_presidio_entity_mapping:
      PER: PERSON
      LOC: LOCATION
      ORG: ORGANIZATION
      AGE: AGE
      ID: ID
      DATE: DATE_TIME
  low_confidence_score_multiplier: 0.4
  low_score_entity_names:
  - ID
  - ORG

  The ner_model_configuration section contains the following parameters:

  • labels_to_ignore: A list of labels to ignore. For example, O (no entity) or entities you are not interested in returning.

  • model_to_presidio_entity_mapping: A mapping between the transformers model labels and the Presidio entity types.

  • low_confidence_score_multiplier: A multiplier to apply to the score of entities with low confidence.

  • low_score_entity_names: A list of entity types to apply the low confidence score multiplier to.

    The default conf file is read during the default initialization of the AnalyzerEngine. Alternatively, the path to a custom configuration file can be passed to the NlpEngineProvider:

    python Copy

    from presidio_analyzer import AnalyzerEngine, RecognizerRegistry
    from presidio_analyzer.nlp_engine import NlpEngineProvider
    
    LANGUAGES_CONFIG_FILE = "./docs/analyzer/languages-config.yml"
    
    # Create NLP engine based on configuration file
    provider = NlpEngineProvider(conf_file=LANGUAGES_CONFIG_FILE)
    nlp_engine_with_spanish = provider.create_engine()
    
    # Pass created NLP engine and supported_languages to the AnalyzerEngine
    analyzer = AnalyzerEngine(
        nlp_engine=nlp_engine_with_spanish, 
        supported_languages=["en", "es"]
    )
    
    # Analyze in different languages
    results_spanish = analyzer.analyze(text="Mi nombre es David", language="es")
    print(results_spanish)
    
    results_english = analyzer.analyze(text="My name is David", language="en")
    print(results_english)

    In this examples we:
    a. create an NlpEngine holding two spaCy models (one in English: en_core_web_lg and one in Spanish: es_core_news_md).
    b. define the supported_languages parameter accordingly.
    c. pass requests in each of these languages.

    !!! note "Note"
    Presidio can currently use one NER model per language via the NlpEngine. If multiple are required,
    consider wrapping NER models as additional recognizers (see sample here).

Leverage frameworks other than spaCy, Stanza and transformers for ML based PII detection Copy Chapter

In addition to the built-in spaCy/Stanza/transformers capabilities, it is possible to create new recognizers which serve as interfaces to other models.
For more information:

• Remote recognizer documentation and samples.
• Flair recognizer example

For considerations for creating such recognizers, see the best practices for adding ML recognizers documentation.

Analyzer Decision Process (`docs/analyzer/decision_process.md`) Copy Section

The Presidio-analyzer decision process

Background Copy Chapter

Presidio-analyzer's decision process exposes information on why a specific PII was detected. Such information could contain:

• Which recognizer detected the entity
• Which regex pattern was used
• Interpretability mechanisms in ML models
• Which context words improved the score
• Confidence scores before and after each step

And more.

Usage Copy Chapter

The decision process can be leveraged in two ways:

1. Presidio-analyzer can log its decision process into a designated logger, which allows you to investigate a specific api request, by exposing a correlation-id as part of the api response headers.
2. The decision process can be returned as part of the /analyze response.

Getting the decision process as part of the response Copy Section

The decision process result can be added to the response.
To enable it, call the analyze method with return_decision_process set as True.

For example:

=== "HTTP"

```sh
curl -d '{
    "text": "John Smith drivers license is AC432223", 
    "language": "en", 
    "return_decision_process": true}' -H "Content-Type: application/json" -X POST http://localhost:3000/analyze
```

=== "Python"

```python
from presidio_analyzer import AnalyzerEngine

# Set up the engine, loads the NLP module (spaCy model by default)
# and other PII recognizers
analyzer = AnalyzerEngine()

# Call analyzer to get results
results = analyzer.analyze(text='My phone number is 212-555-5555', 
                        entities=['PHONE_NUMBER'], 
                        language='en', 
                        return_decision_process=True)

# Get the decision process results for the first result
print(results[0].analysis_explanation)
```

Logging the decision process Copy Section

Logging of the decision process is turned off by default. To turn it on, create the AnalyzerEngine object with log_decision_process=True.

For example:

from presidio_analyzer import AnalyzerEngine

# Set up the engine, loads the NLP module (spaCy model by default)
# and other PII recognizers
analyzer = AnalyzerEngine(log_decision_process=True)

# Call analyzer to get results
results = analyzer.analyze(text='My phone number is 212-555-5555', 
                           entities=['PHONE_NUMBER'], 
                           language='en', 
                           correlation_id="xyz")

The decision process logs will be written to standard output.
Note that it is possible to define a correlation-id which is the trace identification. It will help you to query the stdout logs.
The id can be retrieved from each API response header: x-correlation-id.

By having the traces written into the stdout it's very easy to configure a monitoring solution to ease the process of reading processing the tracing logs in a distributed system.

Examples Copy Chapter

For the a request with the following text:

My name is Bart Simpson, my Credit card is: 4095-2609-9393-4932,  my phone is 425 8829090

The following traces will be written to log, with this format:

[Date Time][decision_process][Log Level][Unique Correlation ID][Trace Message]

/* Detailed source-code truncated for AI context efficiency. */

Writing custom decision process for a recognizer Copy Chapter

When creating new PII recognizers, it is possible to add information about the recognizer's decision process. This information will be traced or returned to the user, depending on the configuration.

For example, the spacy_recognizer.py implements a custom trace as follows:

SPACY_DEFAULT_EXPLANATION = "Identified as {} by Spacy's Named Entity Recognition"

def build_spacy_explanation(recognizer_name, original_score, entity):
    explanation = AnalysisExplanation(
        recognizer=recognizer_name,
        original_score=original_score,
        textual_explanation=SPACY_DEFAULT_EXPLANATION.format(entity))
    return explanation

The textual_explanation field in AnalysisExplanation class allows you to add your own custom text into the final trace which will be written.

!!! note "Note"
These traces leverage the Python logging mechanisms. In the default configuration, A StreamHandler is used to write these logs to sys.stdout.

!!! warning "Warning"
Decision-process traces explain why PIIs were detected,
but not why they were not detected!

Analyzer Developing Recognizers (`docs/analyzer/developing_recognizers.md`) Copy Section

Recognizers Development - Best Practices and Considerations

Recognizers are the main building blocks in Presidio. Each recognizer is in charge of detecting one or more entities in one or more languages.
Recognizers define the logic for detection, as well as the confidence a prediction receives and a list of words to be used when context is leveraged.

Implementation Considerations Copy Chapter

Accuracy Copy Section

Each recognizer, regardless of its complexity, could have false positives and false negatives. When adding new recognizers, we try to balance the effect of each recognizer on the entire system.
A recognizer with many false positives would affect the system's usability, while a recognizer with many false negatives might require more work before it can be integrated. For reproducibility purposes, it is be best to note how the recognizer's accuracy was tested, and on which datasets.
For tools and documentation on evaluating and analyzing recognizers, refer to the presidio-research GitHub repository.

!!! note "Note"
When contributing recognizers to the Presidio OSS,
new predefined recognizers should be added to the
supported entities list,
and follow the contribution guidelines.

Performance Copy Section

Make sure your recognizer doesn't take too long to process text. Anything above 100ms per request with 100 tokens is probably not good enough.

Environment Copy Section

When adding new recognizers that have 3rd party dependencies, make sure that the new dependencies don't interfere with Presidio's dependencies.
In the case of a conflict, one can create an isolated model environment (outside the main presidio-analyzer process) and implement a RemoteRecognizer on the presidio-analyzer side to interact with the model's endpoint.

Recognizer Types Copy Chapter

Generally speaking, there are three types of recognizers:

Deny Lists Copy Section

A deny list is a list of words that should be removed during text analysis. For example, it can include a list of titles (["Mr.", "Mrs.", "Ms.", "Dr."] to detect a "Title" entity.)

See this documentation on adding a new recognizer. The PatternRecognizer class has built-in support for a deny-list input.

Pattern Based Copy Section

Pattern based recognizers use regular expressions to identify entities in text.
See this documentation on adding a new recognizer via code.
The PatternRecognizer class should be extended.
See some examples here:

!!! example "Examples"
Examples of pattern based recognizers are the CreditCardRecognizer and EmailRecognizer.

Machine Learning (ML) Based or Rule-Based Copy Section

Many PII entities are undetectable using naive approaches like deny-lists or regular expressions.
In these cases, we would wish to utilize a Machine Learning model capable of identifying entities in free text, or a rule-based recognizer.

ML: Utilize SpaCy, Stanza or Transformers

Presidio currently uses spaCy as a framework for text analysis and Named Entity Recognition (NER), and stanza and huggingface transformers as an alternative. To avoid introducing new tools, it is recommended to first try to use spaCy, stanza or transformers over other tools if possible.
spaCy provides decent results compared to state-of-the-art NER models, but with much better computational performance.
spaCy, stanza and transformers models could be trained from scratch, used in combination with pre-trained embeddings, or be fine-tuned.

In addition to those, it is also possible to use other ML models. In that case, a new EntityRecognizer should be created.
See an example using Flair here.

Apply Custom Logic

In some cases, rule-based logic provides reasonable ways for detecting entities.
The Presidio EntityRecognizer API allows you to use spaCy extracted features like lemmas, part of speech, dependencies and more to create your logic.
When integrating such logic into Presidio, a class inheriting from the EntityRecognizer should be created.

!!! attention "Considerations for selecting one option over another"

- Accuracy.
- Ease of integration.
- Runtime considerations (For example if the new model requires a GPU).
- 3rd party dependencies of the new model vs. the existing `presidio-analyzer` package.

Analyzer Filtering By Country (`docs/analyzer/filtering_by_country.md`) Copy Section

Filtering recognizers by country

Many Presidio deployments only care about a subset of the world's PII —
e.g. a US-only healthcare app, or an EU-only data pipeline. Loading every
predefined country-specific recognizer in those scenarios means more
false positives, more memory, and more noise in the analyzer output.

RecognizerRegistry.load_predefined_recognizers accepts an optional
countries argument that narrows the country-specific recognizers to
the subset you care about, while always preserving locale-agnostic
recognizers (credit cards, emails, URLs, IBAN, dates, crypto, NER, NLP
engine recognizers, third-party integrations, …).

Quick example Copy Chapter

from presidio_analyzer import AnalyzerEngine, RecognizerRegistry

registry = RecognizerRegistry()
registry.load_predefined_recognizers(languages=["en"], countries=["us", "uk"])

analyzer = AnalyzerEngine(registry=registry)

The registry above contains all locale-agnostic recognizers (e.g. credit
card, email, URL) plus US- and UK-specific recognizers (US SSN, UK NHS,
UK NINO, …) — but excludes German, Spanish, Indian, Italian, Korean,
Polish, Singaporean, Swedish, Thai, Turkish, and other country
recognizers.

The same filter can be set globally in YAML, mirroring how
supported_languages works:

supported_languages: ["en"]
supported_countries: ["us", "uk"]
recognizers:
  # ...

When the registry is built from a YAML file, this top-level
supported_countries is applied automatically inside the loader; you
don't need to pass countries= again from Python.

How the filter works Copy Chapter

A recognizer can be tagged with a country in two ways, reconciled at
construction time:

• Class-level COUNTRY_CODE — the canonical declaration for
  predefined recognizers (UsSsnRecognizer.COUNTRY_CODE = "us", …).
• Constructor country_code= kwarg — for custom recognizers
  without a subclass, or YAML type: custom entries that flow through
  PatternRecognizer.from_dict. This is the path used by the
  no-code/YAML route.

If both are set, they must agree (case-insensitively); a constructor
value that conflicts with COUNTRY_CODE raises ValueError so a
predefined Polish-tax-ID recognizer can't be silently re-tagged as
British. The filter then reads the resolved tag through two instance
methods on EntityRecognizer:

• recognizer.country_code() returns the lower-cased ISO code, or
  None if the recognizer is locale-agnostic.
• recognizer.is_country_specific() is the named predicate equivalent
  to country_code() is not None.

To introspect a class without instantiating, read the
COUNTRY_CODE ClassVar directly (e.g. UsSsnRecognizer.COUNTRY_CODE);
the methods themselves are instance methods so they can also surface
constructor-tagged custom recognizers.

The filter has a single rule:

Code comparison is case-insensitive: "US", "Us", "us" all work.

Declaring a country on your own recognizers Copy Chapter

There are three equivalent paths, pick whichever fits your code shape.
All three end up at the same self._country_code and are filtered the
same way.

1. Subclass with a class-level `COUNTRY_CODE` (preferred for shared code) Copy Section

This is the path used by every predefined country-specific recognizer
in Presidio (UsSsnRecognizer, UkNinoRecognizer, EsNifRecognizer,
…). Best when your recognizer lives in code and you want the country
tag visible at the class definition for review:

from presidio_analyzer import Pattern, PatternRecognizer


class BrCpfRecognizer(PatternRecognizer):
    """Recognize Brazilian CPF numbers."""

    COUNTRY_CODE = "br"

    PATTERNS = [
        Pattern("BR CPF (medium)", r"\b\d{3}\.\d{3}\.\d{3}-\d{2}\b", 0.5),
    ]

    def __init__(self):
        super().__init__(
            supported_entity="BR_CPF",
            patterns=self.PATTERNS,
            supported_language="pt",
        )

BrCpfRecognizer().country_code() returns "br" and the registry's
country filter treats it the same as predefined country-specific
recognizers.

2. Constructor `country_code=` (preferred for one-offs) Copy Section

If you don't want to subclass — e.g. you're constructing a one-off
PatternRecognizer for an Armenian national ID — pass country_code
to the constructor:

from presidio_analyzer import Pattern, PatternRecognizer

am_recognizer = PatternRecognizer(
    supported_entity="AM_NATIONAL_ID",
    name="AmNationalIdRecognizer",
    patterns=[Pattern("AM 10-digit", r"\b\d{10}\b", 0.5)],
    country_code="am",
)
assert am_recognizer.country_code() == "am"

This also covers PatternRecognizer.from_dict({"country_code": "am", ...}), so any tooling that round-trips recognizer dicts (logs,
serialization, recognizer-store backends, …) preserves the tag.

3. YAML `type: custom` with `country_code:` (no-code path) Copy Section

YAML custom-recognizer entries forward country_code straight through
to PatternRecognizer.__init__ via from_dict, so the no-code path
gets the same tagging without writing a single line of Python:

recognizers:
  - name: AmNationalIdRecognizer
    type: custom
    supported_entity: AM_NATIONAL_ID
    supported_languages:
      - language: en
    country_code: am
    patterns:
      - name: "AM 10-digit"
        regex: '\b\d{10}\b'
        score: 0.5

Loading this registry with supported_countries: ["am"] (or the
Python-side countries=["am"]) keeps the recognizer; filtering to a
different country drops it.

Combining paths Copy Section

If you subclass and pass country_code= to the constructor, the two
must agree — passing a different value raises ValueError. This means
a predefined UsSsnRecognizer can never be silently re-tagged via the
constructor, and a YAML country_code: on a type: predefined entry
is cross-checked against the class attribute at load time (see the
"Annotating predefined recognizers in YAML" section below).

Annotating predefined recognizers in YAML Copy Chapter

When you wire up the registry from a YAML file (the no-code path), you
can — and should — also declare the country tag next to each predefined
country-specific entry. This is purely advisory metadata that documents
the entry for human readers; the class-level COUNTRY_CODE remains the
source of truth, and the loader cross-checks the two and refuses to
load on a mismatch:

recognizers:
  - name: UsSsnRecognizer
    supported_languages:
      - en
    type: predefined
    country_code: us       # must match UsSsnRecognizer.COUNTRY_CODE

  - name: UkNinoRecognizer
    supported_languages:
      - en
    type: predefined
    country_code: uk

  - name: CreditCardRecognizer
    type: predefined
    # no country_code: locale-agnostic

If you change one without the other, the registry will raise
ValueError at load time naming both the YAML value and the class
attribute, so the misconfiguration is fixable from the error message
alone.

When to leave `COUNTRY_CODE` unset Copy Chapter

Some PII patterns are not anchored to a single country and should
remain locale-agnostic:

• Credit card numbers, email addresses, URLs, MAC addresses, crypto
  wallet addresses, dates, phone numbers (PhoneNumbers cover multiple
  countries via phonenumbers), …
• IBAN — geographically European-ish but not anchored to one country.
• Generic NER models (spaCy, Stanza, transformer-based recognizers).

Leave COUNTRY_CODE unset (it inherits the default None from
EntityRecognizer) for these. They will always be loaded and the
country filter will not touch them.

Backwards compatibility Copy Chapter

The filter is fully backwards compatible:

1. Default behavior is unchanged. Calling
   load_predefined_recognizers() without countries (or with
   countries=None) loads every predefined recognizer exactly as before.
2. Untagged custom recognizers continue to work. Any recognizer you
   have already deployed that does not declare COUNTRY_CODE is treated
   as locale-agnostic and is never filtered out — regardless of the
   requested countries. The filter only excludes recognizers that have
   explicitly opted in via the class attribute.
3. Predefined recognizers are pre-tagged. Every recognizer under
   presidio_analyzer.predefined_recognizers.country_specific.<country>
   ships with its COUNTRY_CODE set to the appropriate ISO code, so
   countries=["us"] works on a fresh install with no further setup.

Debugging Copy Chapter

If you pass countries=["br"] and your custom Brazilian recognizer
disappears, the most likely cause is that the recognizer hasn't
declared COUNTRY_CODE = "br". Two diagnostic helpers:

• registry.get_country_codes() returns the sorted list of country
  codes currently represented in the registry. If "br" is missing, no
  recognizer is tagged.
• The filter logs a WARNING whenever a requested country matches no
  recognizer in the input set, with a hint to set country_code on
  custom recognizers.

import logging
logging.getLogger("presidio-analyzer").setLevel(logging.WARNING)

registry = RecognizerRegistry()
registry.load_predefined_recognizers(countries=["br"])
# WARNING ... Country filter: no recognizer matched country_code='br'.
#         If you have custom recognizers for 'br', set country_code='br'
#         on them to include them in country-filtered loads.
print(registry.get_country_codes())
# ['au', 'ca', 'de', 'es', 'fi', 'in', 'it', 'kr', 'ng', 'pl', 'se',
#  'sg', 'th', 'tr', 'uk', 'us']

Analyzer Languages (`docs/analyzer/languages.md`) Copy Section

PII detection in different languages

Presidio supports PII detection in multiple languages.
In its default configuration, it contains recognizers and models for English.

To extend Presidio to detect PII in an additional language, these modules require modification:

1. The NlpEngine containing the NLP model which performs tokenization,
   lemmatization, Named Entity Recognition and other NLP tasks.
2. PII recognizers (different EntityRecognizer objects) should be adapted or created.

!!! note "Note"
While different detection mechanisms such as regular expressions are language agnostic, the context words used to increase the PII detection confidence aren't. Consider updating the list of context words for each recognizer to leverage context words in additional languages.

Table of contents Copy Chapter

• Configuring the NLP Engine
• Set up language specific recognizers
• Automatically install NLP models into the Docker container

Configuring the NLP Engine Copy Section

Presidio's NLP engine can be adapted to support multiple languages and frameworks (such as spaCy, Stanza and transformers).
Configuring the NLP engine for a new language or NLP framework is done by downloading or using a model trained on a different language, and providing a configuration.
See the NLP model customization documentation for details on how to configure models for new languages.

Set up language specific recognizers Copy Section

Recognizers are language dependent either by their logic or by the context words used while scanning the surrounding of a detected entity.
As these context words are used to increase score, they should be in the expected input language.

Consider updating the context words of existing recognizers or add new recognizers to support new languages.
Each recognizer can support one language. For example:

from presidio_analyzer import AnalyzerEngine, RecognizerRegistry
from presidio_analyzer.predefined_recognizers import EmailRecognizer
from presidio_analyzer.nlp_engine import NlpEngineProvider

LANGUAGES_CONFIG_FILE = "./docs/analyzer/languages-config.yml"

# Create NLP engine based on configuration file
provider = NlpEngineProvider(conf_file=LANGUAGES_CONFIG_FILE)
nlp_engine_with_spanish = provider.create_engine()

# Setting up an English Email recognizer:
email_recognizer_en = EmailRecognizer(supported_language="en", context=["email", "mail"])

# Setting up a Spanish Email recognizer
email_recognizer_es = EmailRecognizer(supported_language="es", context=["correo", "electrónico"])

registry = RecognizerRegistry()

# Add recognizers to registry
registry.add_recognizer(email_recognizer_en)
registry.add_recognizer(email_recognizer_es)

# Set up analyzer with our updated recognizer registry
analyzer = AnalyzerEngine(
    registry=registry,
    supported_languages=["en","es"],
    nlp_engine=nlp_engine_with_spanish)

analyzer.analyze(text="My name is David", language="en")

Link to LANGUAGES_CONFIG_FILE=languages-config.yml

Automatically install NLP models into the Docker container Copy Section

When packaging the code into a Docker container, NLP models are automatically installed.
To define which models should be installed,
update the conf/default.yaml file. This file is read during
the docker build phase and the models defined in it are installed automatically.

For transformers based models, the configuration can be found here.
A docker file supporting transformers models can be found here.

Analyzer Recognizer Registry Provider (`docs/analyzer/recognizer_registry_provider.md`) Copy Section

Customizing recognizer registry from file

To load recognizers from file, use RecognizerRegistryProvider to instantiate the recognizer registry and then pass it through to the analyzer engine:

from presidio_analyzer import AnalyzerEngine
from presidio_analyzer.recognizer_registry import RecognizerRegistryProvider

recognizer_registry_conf_file = "./analyzer/recognizers-config.yml"

provider = RecognizerRegistryProvider(
                conf_file=recognizer_registry_conf_file
            )
registry = provider.create_recognizer_registry()
analyzer = AnalyzerEngine(registry=registry)

results = analyzer.analyze(text="My name is Morris", language="en")
print(results)

Configuration file structure Copy Chapter

global_regex_flags: 26

supported_languages: 
  - en

recognizers: 
...

The configuration file consists of two parts:

• global_regex_flags: regex flags to be used in regex matching (see regex flags).
• supported_languages: A list of supported languages that the registry will support.
• recognizers: a list of recognizers to be loaded by the recognizer registry. This list consists of two different types of recognizers:
  • Predefined: A set of already defined recognizer classes in presidio. This includes all recognizers defined in the codebase (along with user defined recognizers) that inherit from EntityRecognizer.
  • Custom: custom created pattern recognizers that are created based on the fields provided in the configuration file.

!!! note "Note"

supported_languages must be identical to the same field in analyzer_engine

Recognizer list Copy Chapter

The recognizer list comprises of both the predefined and custom recognizers, for example:

...
  - name: CreditCardRecognizer
    supported_languages:
    - language: en
      context: [credit, card, visa, mastercard, cc, amex, discover, jcb, diners, maestro, instapayment]
    - language: es
      context: [tarjeta, credito, visa, mastercard, cc, amex, discover, jcb, diners, maestro, instapayment]
    - language: it
    - language: pl
    type: predefined
    score_thresholds:
      default: 0.4
      CREDIT_CARD: 0.7

  - name: UsBankRecognizer
    supported_languages: 
    - en
    type: predefined

  - name: MedicalLicenseRecognizer
    type: predefined

  - name: ExampleCustomRecognizer
    patterns:
    - name: "zip code (weak)"
      regex: "(\\b\\d{5}(?:\\-\\d{4})?\\b)"
      score: 0.01
    supported_languages:
    - language: en
      context: [zip, code]
    - language: es
      context: [código, postal]
    supported_entity: "ZIP"
    type: custom
    enabled: true

  - name: "TitlesRecognizer"
    supported_language: "en"
    supported_entity: "TITLE"
    deny_list: [Mr., Mrs., Ms., Miss, Dr., Prof.]
    deny_list_score: 1

  - name: "HuggingFace NER"
    type: "predefined"
    class_name: "HuggingFaceNerRecognizer"
    model_name: "dslim/bert-base-NER"
    supported_languages:
      - en
    supported_entities: ["PERSON", "LOCATION", "ORGANIZATION"]
    aggregation_strategy: "simple"
    device: "cpu"

The recognizer parameters Copy Section

• supported_languages: A list of supported languages that the analyzer will support. In case this field is missing, a recognizer will be created for each supported language provided to the AnalyzerEngine.
  In addition to the language code, this field also contains a list of context words, which increases confidence in the detection in case it is found in the surroundings of a detected entity (as seen in the credit card example above).

• type: this could be either predefined or custom. As this is optional, if not stated otherwise, the default type is custom.

• name: Different per the type of the recognizer. For predefined recognizers, this is the class name as defined in presidio, while for custom recognizers, it will be set as the name of the recognizer.

• patterns: a list of objects of type Pattern that contains a name, score and regex that define matching patterns.

• enabled: enables or disables the recognizer.

• supported_entity: the detected entity associated by the recognizer.

• deny_list: A list of words to detect, in case the recognizer uses a predefined list of words.

• deny_list_score: confidence score for a term identified using a deny-list. If omitted, defaults to 1.0 (previously this silently defaulted to 0.0 when loaded through RecognizerRegistryProvider, which caused deny-list matches to be filtered out by any positive score_threshold).

• score_thresholds: optional score thresholds for this recognizer. Use default as the recognizer-wide threshold and entity names for overrides. Note that supplying analyzer_engine.analyze(score_threshold=...) bypasses recognizer-level thresholds for that request. The precedence is: Presidio Analyzer analyzer.analyze(score_threshold=...) > an entity specific threshold > a recognizer default threshold (default) > the Presidio Analyzer default_score_threshold.

• text_chunker: configures how long texts are split for NER recognizers (GLiNERRecognizer, HuggingFaceNerRecognizer). Accepts a dict with chunker_type and params. Available types: character (default) and tokenizer (uses the model's tokenizer for accurate token-based splitting). Example:

  yaml Copy

  - name: GLiNERRecognizer
    type: predefined
    model_name: urchade/gliner_multi_pii-v1
    text_chunker:
      chunker_type: tokenizer
      # max_tokens omitted: auto-derived from the model's tokenizer and
      # reduced to reserve room for special tokens ([CLS]/[SEP]). Set it
      # explicitly only if you account for those special tokens yourself.
      overlap_tokens: 32

!!! tip "Configuration Tip: Agglutinative languages (e.g., Korean)"

If spaCy-based NER produces noisy or redundant results, you can disable `SpacyRecognizer` and use `HuggingFaceNerRecognizer` as an alternative. Note that `HuggingFaceNerRecognizer` bypasses the spaCy tokenizer alignment mechanism.



```yaml
- name: "SpacyRecognizer"
  type: "predefined"
  class_name: "SpacyRecognizer"
  supported_languages: ["ko"]
  enabled: false
```

Using a custom registry file with the Presidio Analyzer server Copy Chapter

The Presidio Analyzer server (the presidio-analyzer Docker image) selects
its recognizer registry through the RECOGNIZER_REGISTRY_CONF_FILE
environment variable. It defaults to the bundled
presidio_analyzer/conf/default_recognizers.yaml; point it at your own YAML
file (mounted into the container) to load a modified registry without
rebuilding the image. The server reads ANALYZER_CONF_FILE and
NLP_CONF_FILE the same way.

!!! note "Note"

`supported_languages` in the registry file must match the same field in
the analyzer configuration file.

Example: enabling a pattern-based country recognizer for `en` Copy Section

Some predefined recognizers are only registered for their native language.
ItFiscalCodeRecognizer, for example, is registered with
supported_languages: [it] in default_recognizers.yaml, so the default
image — which loads the en NLP model and supported_languages: [en] —
cannot detect Italian fiscal codes. Because the recognizer is pattern-based,
it does not depend on the Italian NLP model and can be enabled for en by
overriding the registry file:

1. Copy the default registry file and change the recognizer's
   supported_languages (leave country_code unchanged — it must match the
   class declaration and is only the country tag, not a language):

   yaml Copy

   - name: ItFiscalCodeRecognizer
     supported_languages:
     - en
     type: predefined
     country_code: it

2. Run the analyzer with the file mounted and the environment variable set:

   bash Copy

   docker run --rm -p 3000:3000 \
     -v $(pwd)/my_recognizers.yaml:/app/my_recognizers.yaml \
     -e RECOGNIZER_REGISTRY_CONF_FILE=/app/my_recognizers.yaml \
     ghcr.io/data-privacy-stack/presidio-analyzer:<version>

Requests with entities: ["IT_FISCAL_CODE"] are now served for
language: en.

To restrict or extend which country-specific recognizers are loaded in
general, see Filtering recognizers by country.

Anonymizer Adding Operators (`docs/anonymizer/adding_operators.md`) Copy Section

Supporting new types of PII operators

Operators are the presidio-anonymizer actions over the text.

There are two types of operators:

• Anonymize (e.g., hash, replace, redact, encrypt, mask)
• Deanonymize (e.g., decrypt)

Presidio anonymizer can be easily extended to support additional anonymization and deanonymization methods (called Operators).

Extending presidio-anonymizer for additional PII operators Copy Chapter

1. Create new python class implementing the abstract Operator class.
2. Implement the methods:
   • operate - gets the data and returns a new text expected to replace the old one.
   • validate - validate the parameters entered for the anonymizer exists and valid.
   • operator_name - this method helps to automatically load the existing anonymizers.
   • operator_type - either Anonymize or Deanonymize. Will be mapped to the proper engine.
3. Call the AnonymizerEngine.add_anonymizer method to add a new operator to the anonymizer. Alternatively, call the DeanonymizeEngine.add_deanonymizer method to add a new deanonymizer.

See a detailed example here.

Api Analyzer Python (`docs/api/analyzer_python.md`) Copy Section

Presidio Analyzer API Reference

Objects at the top of the presidio-analyzer package Copy Chapter

::: presidio_analyzer.AnalyzerEngine
handler: python

::: presidio_analyzer.analyzer_engine_provider.AnalyzerEngineProvider
handler: python

::: presidio_analyzer.analysis_explanation.AnalysisExplanation
handler: python

::: presidio_analyzer.recognizer_result.RecognizerResult
handler: python

Batch modules Copy Chapter

::: presidio_analyzer.batch_analyzer_engine.BatchAnalyzerEngine
handler: python

::: presidio_analyzer.dict_analyzer_result.DictAnalyzerResult
handler: python

Recognizers and patterns Copy Chapter

::: presidio_analyzer.entity_recognizer.EntityRecognizer
handler: python

::: presidio_analyzer.local_recognizer.LocalRecognizer
handler: python

::: presidio_analyzer.pattern.Pattern
handler: python

::: presidio_analyzer.pattern_recognizer.PatternRecognizer
handler: python

::: presidio_analyzer.remote_recognizer.RemoteRecognizer
handler: python

Recognizer registry modules Copy Chapter

::: presidio_analyzer.recognizer_registry.RecognizerRegistry
handler: python

::: presidio_analyzer.recognizer_registry.RecognizerRegistryProvider
handler: python

Context awareness modules Copy Chapter

::: presidio_analyzer.context_aware_enhancers

NLP Engine modules Copy Chapter

::: presidio_analyzer.nlp_engine
handler: python

Predefined Recognizers Copy Chapter

::: presidio_analyzer.predefined_recognizers

Misc Copy Chapter

::: presidio_analyzer.analyzer_request.AnalyzerRequest
handler: python

Api Anonymizer Python (`docs/api/anonymizer_python.md`) Copy Section

Presidio Anonymizer API Reference

::: presidio_anonymizer
handler: python

Api Image Redactor Python (`docs/api/image_redactor_python.md`) Copy Section

Presidio Image Redactor API Reference

::: presidio_image_redactor
handler: python
options:
docstring_style: sphinx

Getting Started Getting Started Images (`docs/getting_started/getting_started_images.md`) Copy Section

Getting started with image de-identification with Presidio

Presidio provides a simple way to de-identify image data by detecting and anonymizing personally identifiable information (PII). This guide shows you how to get started with image de-identification using Presidio's Python packages.

Presidio has two main modules for image de-identification: General purpose, and specifically for DICOM (medical) images.

Simple flow - Python package Copy Chapter

=== "Anonymize PII in images"

1. Install presidio-image-redactor

    ```sh
    pip install presidio-image-redactor
    ```
   
2. Redact PII from image

    ```py
    from presidio_image_redactor import ImageRedactorEngine
    from PIL import Image
    
    image = Image.open(path_to_image_file)
    
    redactor = ImageRedactorEngine()
    redactor.redact(image=image)
    ```

=== "Redact text PII in DICOM images"

1. Install presidio-image-redactor

    ```sh
    pip install presidio-image-redactor
    ```
   
2. Redact text PII from DICOM image

    ```py
    import pydicom
    from presidio_image_redactor import DicomImageRedactorEngine

    # Set input and output paths
    input_path = "path/to/your/dicom/file.dcm"
    output_dir = "./output"

    # Initialize the engine
    engine = DicomImageRedactorEngine()

    # Option 1: Redact from a loaded DICOM image
    dicom_image = pydicom.dcmread(input_path)
    redacted_dicom_image = engine.redact(dicom_image, fill="contrast")

    # Option 2: Redact from DICOM file
    engine.redact_from_file(input_path, output_dir, padding_width=25, fill="contrast")

    # Option 3: Redact from directory
    engine.redact_from_directory("path/to/your/dicom", output_dir, padding_width=25, fill="contrast")
    ```

Simple flow - Docker container Copy Chapter

Presidio provides a Docker containers that you can use to de-identify image data.

1. Download Docker image

docker pull ghcr.io/data-privacy-stack/presidio-image-redactor

2. Run container

docker run -d -p 5003:3000 ghcr.io/data-privacy-stack/presidio-image-redactor:latest

3. Use the API

curl -XPOST "http://localhost:5003/redact" -H "content-type: multipart/form-data" -F "[email protected]" -F "data=\"{'color_fill':'255'}\"" > out.png

Read more Copy Chapter

• Installing Presidio
• PII detection in images
• Samples
• Python API reference - Image Redactor

Getting Started Getting Started Structured (`docs/getting_started/getting_started_structured.md`) Copy Section

Getting started with structured and semi-structured de-identification with Presidio

Presidio-structured is a package built on top of Presidio that provides a simple way to de-identify structured and semi-structured data by detecting and anonymizing personally identifiable information (PII).

Presidio-structured supports the detection and anonymization of PII in tables (e.g. Pandas DataFrames or SQL tables) and semi-structured data (e.g. JSON).

!!! warning "Warning"
Alpha: This package is currently in alpha, meaning it is in its early stages of development. Features and functionality may change as the project evolves.

Simple flow - structured data Copy Chapter

import pandas as pd
from presidio_structured import StructuredEngine, PandasAnalysisBuilder
from presidio_anonymizer.entities import OperatorConfig
from faker import Faker # optionally using faker as an example

# Initialize the engine with a Pandas data processor (default)
pandas_engine = StructuredEngine()

# Create a sample DataFrame
sample_df = pd.DataFrame({'name': ['John Doe', 'Jane Smith'], 'email': ['[email protected]', '[email protected]']})

# Generate a tabular analysis which detects the PII entities in the DataFrame.
tabular_analysis = PandasAnalysisBuilder().generate_analysis(sample_df)

# Define anonymization operators
fake = Faker()
operators = {
    "PERSON": OperatorConfig("replace", {"new_value": "REDACTED"}),
    "EMAIL_ADDRESS": OperatorConfig("custom", {"lambda": lambda x: fake.safe_email()})
}

# Anonymize DataFrame
anonymized_df = pandas_engine.anonymize(sample_df, tabular_analysis, operators=operators)
print(anonymized_df)

Read more Copy Chapter

• Presidio structured documentation
• Presidio structured sample notebook

Getting Started Getting Started Text (`docs/getting_started/getting_started_text.md`) Copy Section

Getting started with text de-identification with Presidio

Presidio provides a simple way to de-identify text data by detecting and anonymizing personally identifiable information (PII). This guide shows you how to get started with text de-identification using Presidio's Python packages.

Note that Presidio can leverage different NLP packages to analyze text data. The default engine is based on spaCy, but you can also use others. This guide shows two examples: one using spaCy and the other using transformers.

Simple flow - Python package Copy Chapter

Using Presidio's modules as Python packages to get started:

===+ "Anonymize PII in text (Default spaCy model)"

1. Install Presidio
    
    ```sh
    pip install presidio-analyzer
    pip install presidio-anonymizer
    python -m spacy download en_core_web_lg
    ```

2. Analyze + Anonymize

    ```py
    from presidio_analyzer import AnalyzerEngine
    from presidio_anonymizer import AnonymizerEngine
    
    text="My phone number is 212-555-5555"
    
    # Set up the engine, loads the NLP module (spaCy model by default) 
    # and other PII recognizers
    analyzer = AnalyzerEngine()
    
    # Call analyzer to get results
    results = analyzer.analyze(text=text,
                               entities=["PHONE_NUMBER"],
                               language='en')
    print(results)
    
    # Analyzer results are passed to the AnonymizerEngine for anonymization
    
    anonymizer = AnonymizerEngine()
    
    anonymized_text = anonymizer.anonymize(text=text,analyzer_results=results)
    
    print(anonymized_text)
    ```

=== "Anonymize PII in text (transformers)"

1. Install Presidio
    
    ```sh
    pip install "presidio-analyzer[transformers]"
    pip install presidio-anonymizer
    python -m spacy download en_core_web_sm
    ```

2. Analyze + Anonymize

    ```py
    from presidio_analyzer import AnalyzerEngine
    from presidio_analyzer.nlp_engine import TransformersNlpEngine
    from presidio_anonymizer import AnonymizerEngine
    
    text = "My name is Don and my phone number is 212-555-5555"
    
    # Define which transformers model to use
    model_config = [{"lang_code": "en", "model_name": {
        "spacy": "en_core_web_sm",  # use a small spaCy model for lemmas, tokens etc.
        "transformers": "dslim/bert-base-NER"
        }
    }]

    nlp_engine = TransformersNlpEngine(models=model_config)

    # Set up the engine, loads the NLP module (spaCy model by default) 
    # and other PII recognizers
    analyzer = AnalyzerEngine(nlp_engine=nlp_engine)
    
    # Call analyzer to get results
    results = analyzer.analyze(text=text, language='en')
    print(results)
    
    # Analyzer results are passed to the AnonymizerEngine for anonymization
    
    anonymizer = AnonymizerEngine()
    
    anonymized_text = anonymizer.anonymize(text=text, analyzer_results=results)
    
    print(anonymized_text)
    
    ```
    !!! tip "Tip: Downloading models"
        If not available, the transformers model and the spacy model would be downloaded on the first call to the `AnalyzerEngine`. To pre-download, see [this doc](../analyzer/nlp_engines/transformers.md#downloading-a-pre-trained-model).

=== "GPU Acceleration (Optional)"

For GPU acceleration, install the appropriate dependencies for your hardware:

- **Linux with NVIDIA GPU**: cupy-cuda12x (or the version matching your CUDA installation)
- **macOS with Apple Silicon**: MPS (Metal Performance Shaders) is currently not supported. The analyzer will use CPU for PyTorch operations.

Simple flow - Docker container Copy Chapter

Presidio provides Docker containers that you can use to de-identify text data. Each module, analyzer, and anonymizer, has its own Docker container. New releases are available on GitHub Container Registry; the legacy Microsoft Container Registry images are no longer updated.

1. Download Docker images

docker pull ghcr.io/data-privacy-stack/presidio-analyzer
docker pull ghcr.io/data-privacy-stack/presidio-anonymizer

2. Run containers

docker run -d -p 5002:3000 ghcr.io/data-privacy-stack/presidio-analyzer:latest

docker run -d -p 5001:3000 ghcr.io/data-privacy-stack/presidio-anonymizer:latest

3. Use the API

curl -X POST http://localhost:5002/analyze \
-H "Content-Type: application/json" \
-d '{
  "text": "My phone number is 555-123-4567.",
  "language": "en"
}'


curl -X POST http://localhost:5001/anonymize -H "Content-Type: application/json"  -d '
    {
        "text": "My phone number is 555-123-4567",
        "anonymizers": {
            "PHONE_NUMBER": {
            "type": "replace",
            "new_value": "--Redacted phone number--"
            }
        },
        "analyzer_results": [
        {
            "start": 19,
            "end": 31,
            "score": 0.95,
            "entity_type": "PHONE_NUMBER"
        }
    ]}'

Read more Copy Chapter

• Installing Presidio
• PII detection in text
• PII anonymization in text
• Tutorial
• Samples
• Python API reference - Analyzer
• Python API reference - Anonymizer
• REST API reference

Image Redactor Evaluating Dicom Redaction (`docs/image-redactor/evaluating_dicom_redaction.md`) Copy Section

Evaluating DICOM de-identification

Table of Contents Copy Chapter

• Introduction
• Ground truth format
• Creating ground truth files
• Evaluating de-identification performance

Introduction Copy Chapter

We can evaluate the performance of the DicomImageRedactorEngine DICOM de-identification by using the DicomImagePiiVerifyEngine. The evaluation results consist of:

• Image with bounding boxes identifying detected Personal Health Information (PHI)
• All positives (True Positives and False Positives)
• Precision
• Recall

Ground truth format Copy Chapter

Ground truth labels are stored as .json files containing filename as the highest level keys. Each filename object consists of an item for each individual entity.

{
    "your/dicom/dir/file_0.dcm": [
        {
            "label": "DAVIDSON",
            "left": 25,
            "top": 25,
            "width": 241,
            "height": 37
        },
        {
            "label": "DOUGLAS",
            "left": 287,
            "top": 25,
            "width": 230,
            "height": 36
        },
        {
            "label": "[M]",
            "left": 535,
            "top": 25,
            "width": 60,
            "height": 45
        },
        {
            "label": "01.09.2012",
            "left": 613,
            "top": 26,
            "width": 226,
            "height": 35
        },
        {
            "label": "06.16.1976",
            "left": 170,
            "top": 72,
            "width": 218,
            "height": 35
        }
    ],
    "your/dicom/dir/file_1.dcm": [
        ...
    ]
}

Return to the Table of Contents

Creating ground truth files Copy Chapter

The DicomImagePiiVerifyEngine class can be used to assist in ground truth label generation. Use the following code snippet to generate the verification image, OCR results, and NER (analyzer) results.

import pydicom
from presidio_image_redactor import DicomImagePiiVerifyEngine

# Initialize engine
dicom_engine = DicomImagePiiVerifyEngine()

# Choose your file to create ground truth for
filename = "path/to/your/file.dcm"
instance = pydicom.dcmread(filename)
padding_width = 25

# Get OCR and NER results
verification_image, ocr_results, analyzer_results = dicom_engine.verify_dicom_instance(instance, padding_width)

By looking at the output of verify_dicom_instance, we can create a ground truth labels json.

Save analyzer_results as a json file and then perform the following

1. Group the results into a new item with the file name set as the key.
2. For each item in this group:
   a. Remove the "entity_type" field and value.
   b. Add a new "label" field with the value set to the ground truth text PHI with matching coordinate as you can see in the formatted OCR results and verification image.

Then check that your ground truth json contains all the text PHI you can visually confirm in the DICOM image. If something is not detected by the OCR or NER, you will need to manually add the item yourself.

Pixel position and size data can be obtained using any labeling software or imaging processing software (e.g., MS Paint) on the verification image.

Note: When manually specifying pixel position, make sure to account for any padding introduced in the OCR process (default padding added is 25 pixels).

Example Copy Section

Let's say we ran the above code block and see the following for ocr_results and analyzer_results.

// OCR Results
[
    {
        "left": 25,
        "top": 25,
        "width": 241,
        "height": 37,
        "conf": 95.833916,
        "label": "DAVIDSON"
    },
    {
        "left": 287,
        "top": 25,
        "width": 230,
        "height": 36,
        "conf": 93.292221,
        "label": "DOUGLAS"
    }
]

// Analyzer Results
[
    {
        "entity_type": "PERSON",
        "score": 1.0,
        "left": 25,
        "top": 25,
        "width": 241,
        "height": 37
    },
    {
        "entity_type": "PERSON",
        "score": 1.0,
        "left": 287,
        "top": 25,
        "width": 230,
        "height": 36
    }
]

Looking at the position and size values of the ground truth and detected text from the analyzer results, we can see that the first item in the analyzer results is likely "DAVIDSON" and the second is likely "DOUGLAS".

With this, we set our ground truth json to the following:

// Ground truth json
{
    "path/to/your/file.dcm": [
        {
            "label": "DAVIDSON",
            "left": 25,
            "top": 25,
            "width": 241,
            "height": 37
        },
        {
            "label": "DOUGLAS",
            "left": 287,
            "top": 25,
            "width": 230,
            "height": 36
        }
    ]
}

Return to the Table of Contents

Evaluating de-identification performance Copy Chapter

The DicomImagePiiVerifyEngine can be used to evaluate DICOM de-identification performance.

# Load ground truth for one file
with open(gt_path) as json_file:
    all_ground_truth = json.load(json_file)
ground_truth = all_ground_truth[file_of_interest]

# Select your DICOM instance
instance = pydicom.dcmread(file_of_interest)

# Evaluate the DICOM de-identification performance
_, eval_results = dicom_engine.eval_dicom_instance(instance, ground_truth)

You can also set optional arguments to see the effect of padding width, ground-truth matching tolerance, and OCR confidence threshold (e.g., ocr_kwargs={"ocr_threshold": 50}).

For a full demonstration, please see the evaluation notebook.

Return to the Table of Contents

Learn Presidio Concepts (`docs/learn_presidio/concepts.md`) Copy Section

Concepts in Presidio

High-level concepts Copy Chapter

Main objects in Presidio Copy Chapter

Evaluation concepts Copy Chapter

Recipes CONTRIBUTING (`docs/recipes/CONTRIBUTING.md`) Copy Section

Contributing to Presidio Recipes Gallery

Thank you for your interest in contributing! Recipes help others learn how to customize Presidio for specific domains.

What Makes a Good Recipe? Copy Chapter

A good recipe should:

1. Focus on a Specific Domain: Target a well-defined scenario (e.g., "Financial chatbot logs", "Clinical notes", "Customer support tickets")

2. Be Reproducible: Include working code that others can run

3. Follow the Research Pattern: Build upon the end-to-end approach from presidio-research, showing:

   • Data Synthesis: How to generate or obtain test data
   • Configuration: Your Presidio setup with any custom recognizers
   • Evaluation: Metrics (precision, recall, F₂) showing performance

4. Keep It Simple: Focus on getting a working example first. Detailed documentation can come later.

Recipe Format Copy Chapter

Follow Presidio Research Examples Copy Section

Your recipe should follow the end-to-end evaluation approach from presidio-research.

Key reference notebooks:

• Evaluate Presidio Analyzer - Complete evaluation workflow example
• Generate Synthetic Data - Using Presidio Evaluator data generator

Option 1: Single Jupyter Notebook (Recommended for Simple Cases) Copy Section

For straightforward examples, use one notebook:

your-recipe-name/
├── recipe.ipynb          # Main notebook with data synthesis → evaluation
└── README.md             # Brief overview (optional)

Option 2: Multiple Files (Recommended for Complex Flows) Copy Section

For complex scenarios, break into separate files:

your-recipe-name/
├── 1_generate_data.ipynb # Data synthesis (use Presidio Evaluator or custom)
├── 2_configure.ipynb     # Presidio setup with custom recognizers
├── 3_evaluate.ipynb      # Run evaluation and analysis
└── README.md             # Overview and instructions

Or as Python scripts:

your-recipe-name/
├── generate_data.py      # Generate test data
├── configure.py          # Presidio setup
├── evaluate.py           # Run evaluation
└── README.md             # Overview

Required Components Copy Section

Your recipe should include:

• Data Synthesis: Generate synthetic data using Presidio Evaluator or your own method
• Presidio Configuration: Show your setup (default, custom recognizers, or custom models)
• Evaluation: Measure and report precision, recall, F₂ score, latency
• Key Findings: Brief summary of results and when to use this approach

Quick Contribution Steps Copy Chapter

1. Fork the repo and create a new branch

2. Create your recipe folder under docs/recipes/your-recipe-name/

3. Use the template: Copy template.md or start with a notebook

4. Focus on working code first: Don't worry about making it perfect

5. Submit a PR: We'll help refine it during review

Evaluation Metrics Copy Chapter

Include at minimum:

• Precision: Percentage of detected entities that were correct
• Recall: Percentage of actual PII that was detected
• F₂ Score: Recall-weighted F-score (emphasizes catching all PII)
• Latency: Average processing time per sample

Examples to Learn From Copy Chapter

Presidio Research Notebooks (recommended starting point):

• Evaluate Presidio Analyzer - Complete end-to-end evaluation example
• Generate Synthetic Data - Presidio Evaluator data generator
• Other presidio-research notebooks - Additional examples and tools

Additional Resources:

• Presidio Samples: Integration patterns and usage examples

Questions? Copy Chapter

• Open an issue with the recipe label
• Email [email protected]
• Tag @omri374 in your PR for guidance

License Copy Chapter

By contributing, you agree to license your work under the MIT License.

Recipes Template (`docs/recipes/template.md`) Copy Section

Recipe Template: [Recipe Name]

Note: This is a simplified template for contributing recipes. Focus on providing a working example - detailed documentation can come later.

Overview Copy Chapter

Domain: [e.g., Healthcare, Finance, Retail]
Data Type: [e.g., Clinical notes, Chat logs, API responses]
Goal: [What you're trying to detect and anonymize]

Quick Start Copy Chapter

Sample Data Copy Section

# Example data that demonstrates the scenario
sample_text = """
[Your example text with PII]
"""

Basic Configuration Copy Section

from presidio_analyzer import AnalyzerEngine
from presidio_anonymizer import AnonymizerEngine

# Your Presidio configuration
analyzer = AnalyzerEngine()
anonymizer = AnonymizerEngine()

# Analyze and anonymize
results = analyzer.analyze(text=sample_text, language="en")
anonymized = anonymizer.anonymize(text=sample_text, analyzer_results=results)

print(anonymized.text)

Approach Copy Chapter

Describe your approach in a few sentences. What recognizers, models, or techniques did you use?

Results Copy Chapter

Precision: [0.XX]
Recall: [0.XX]
F₂ Score: [0.XX]
Latency: [XX ms/sample]

Key Findings Copy Section

• [Main insight 1]
• [Main insight 2]
• [Main insight 3]

End-to-End Example Copy Chapter

Recommended Approach: Follow Presidio Research Patterns Copy Section

Provide an end-to-end example following the patterns from presidio-research:

Reference notebooks:

• Evaluate Presidio Analyzer - Complete evaluation workflow
• Generate Synthetic Data - Using Presidio Evaluator data generator

Structure your example:

1. Data Synthesis: Generate synthetic data using Presidio Evaluator or your own method
2. Configuration: Set up Presidio with your custom recognizers or models
3. Evaluation: Measure precision, recall, and F₂ score
4. Analysis: Discuss results and trade-offs

Format Options Copy Section

• Single Jupyter Notebook: Best for simple, focused examples (recommended for most recipes)
• Multiple Files: For complex flows, break into separate notebooks or Python scripts:
  text Copy

  your-recipe-name/
  ├── 1_generate_data.ipynb    # Data synthesis
  ├── 2_configure.ipynb         # Presidio setup
  ├── 3_evaluate.ipynb          # Evaluation
  └── README.md                 # Overview

Tips for Others Copy Chapter

• [Tip 1: When this approach works best]
• [Tip 2: Common pitfalls to avoid]
• [Tip 3: How to adapt to similar scenarios]

Author: [Your Name]
Date: [YYYY-MM-DD]

Tutorial 00 Getting Started (`docs/tutorial/00_getting_started.md`) Copy Section

Getting started

Installation Copy Chapter

First, let's install presidio using pip. For detailed documentation, see the installation docs.
Install from PyPI:

pip install presidio_analyzer
pip install presidio_anonymizer
python -m spacy download en_core_web_lg

Simple flow Copy Chapter

A simple call to Presidio Analyzer:

from presidio_analyzer import AnalyzerEngine

text = "His name is Mr. Jones and his phone number is 212-555-5555"

analyzer = AnalyzerEngine()
analyzer_results = analyzer.analyze(text=text, language="en")

print(analyzer_results)

Next, we'll go over ways to customize Presidio to specific needs by adding PII recognizers, using context words, NER models and more.

Tutorial 01 Deny List (`docs/tutorial/01_deny_list.md`) Copy Section

Example 1: Deny-list based PII recognition

In this example, we will pass a short list of tokens which should be marked as PII if detected.
First, let's define the tokens we want to treat as PII. In this case it would be a list of titles:

titles_list = [
    "Sir",
    "Ma'am",
    "Madam",
    "Mr.",
    "Mrs.",
    "Ms.",
    "Miss",
    "Dr.",
    "Professor",
]

Second, let's create a PatternRecognizer which would scan for those titles, by passing a deny_list:

from presidio_analyzer import PatternRecognizer

titles_recognizer = PatternRecognizer(supported_entity="TITLE", deny_list=titles_list)

At this point we can call our recognizer directly:

from presidio_analyzer import PatternRecognizer

text1 = "I suspect Professor Plum, in the Dining Room, with the candlestick"
result = titles_recognizer.analyze(text1, entities=["TITLE"])
print(f"Result:\n {result}")

Finally, let's add this new recognizer to the list of recognizers used by the Presidio AnalyzerEngine:

from presidio_analyzer import AnalyzerEngine

analyzer = AnalyzerEngine()
analyzer.registry.add_recognizer(titles_recognizer)

When initializing the AnalyzerEngine, Presidio loads all available recognizers,
including the NlpEngine used to detect entities, and extract tokens, lemmas and other linguistic features.

Let's run the analyzer with the new recognizer in place:

results = analyzer.analyze(text=text1, language="en")

print("Results:")
print(results)

As expected, both the name "Plum" and the title were identified as PII: