pitangent-ds/deberta-v3-nli-onnx-quantized
By pitangent-ds · ↓ 1 · ♥ 0 · Updated 2024-07-20
What's in this repo
This page lists the files in pitangent-ds/deberta-v3-nli-onnx-quantized, including model_quantized.onnx.
- 1 ONNX models (.onnx files)
- 10 files, including 6 .json, 1 .md, 1 .onnx, 1 .model
- Last updated 2024-07-20
ONNX files
| File | Size | Download |
|---|---|---|
| model_quantized.onnx | — | Download |
How to use
These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.
README
DeBERTa-v3-base-onnx-quantized
This model has been quantized using the base model: sileod/deberta-v3-base-tasksource-nli, To use this model you need to have onnxruntime installed on your machine.
To use this model, you can check out my Huggingface Spaces.
The source code for the Huggingface Application can be found on GitHub.
To run this model on your machine use the following code. Note that this model is optimized for CPU with AVX2 support.
- Install dependencies
pip install transformers optimum[onnxruntime]
- Run the model:
# load libraries
from transformers import AutoTokenizer
from optimum.onnxruntime import ORTModelForSequenceClassification
from optimum.pipelines import pipeline
# load model components
MODEL_ID = "pitangent-ds/deberta-v3-nli-onnx-quantized"
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = ORTModelForSequenceClassification.from_pretrained(MODEL_ID)
# load the pipeline
classifier = pipeline("zero-shot-classification", tokenizer=tokenizer, model=model)
# inference
text = "The jacket that I bought is awesome"
candidate_labels = ["positive", "negative"]
results = classifier(text, candidate_labels)
Model info
Created: 2024-07-17
Updated: 2024-07-20
View on Hugging Face