gravitee-io/gliner-pii-detection

Gravitee GliNER PII Detection 🚀

GLiNER is a Named Entity Recognition (NER) model capable of identifying any entity type using a bidirectional transformer encoder (BERT-like). It provides a practical alternative to traditional NER models, which are limited to predefined entities, and Large Language Models (LLMs) that, despite their flexibility, are costly and large for resource-constrained scenarios. The list of entities is provided in Evaluation results; however, due to the model's nature, it has the capability to identify other entity types as well.

Evaluation results:

Installation

To use this model, you must install the GLiNER Python library:

!pip install gliner

Usage

Once you've downloaded the GLiNER library, you can import the GLiNER class. You can then load this model using GLiNER.from_pretrained and predict entities with predict_entities.

from gliner import GLiNER

# if you want to use quant model put "model.quant.onnx" in onnx_model_file argument
model = GLiNER.from_pretrained(
    "gravitee-io/gliner-pii-detection", load_onnx_model=True,
    load_tokenizer=True, onnx_model_file="model.onnx"
)

text = """
Hey, just a quick update — I talked to David yesterday. 
He sent over the files from his private email (david.doe@example.com), and we should be careful with his SSN: 123-45-6789.
He mentioned his new address is 123 Maple Street in New York. 
His PC adress is 192.168.1.100.
"""

labels = ["name",
          "email",
          "ssn",
          "street_address",
          "date",
          "ipv4"]

entities = model.predict_entities(text, labels)

for entity in entities:
    print(entity["text"], "=>", entity["label"], "=>", entity["score"])

David => name => 0.9066112041473389
yesterday => date => 0.9482080340385437
david.doe@example.com => email => 0.9911587834358215
123-45-6789 => ssn => 0.8612598180770874
123 Maple Street in New York => street_address => 0.9869663715362549
192.168.1.100 => ipv4 => 0.9810121059417725