philschmid/quantized-distilbert-banking77
By philschmid · ↓ 4 · ♥ 0 · Updated 2022-06-08
What's in this repo
This page lists the files in philschmid/quantized-distilbert-banking77, including model.onnx.
- 1 ONNX models (.onnx files)
- 8 files, including 4 .json, 1 .md, 1 .onnx, 1 .txt
- Last updated 2022-06-08
ONNX files
| File | Size | Download |
|---|---|---|
| model.onnx | — | Download |
How to use
These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.
README
Quantized-distilbert-banking77
This model is a statically quantized version of optimum/distilbert-base-uncased-finetuned-banking77 on the banking77 dataset.
The model was created using the optimum-static-quantization notebook.
It achieves the following results on the evaluation set:
Accuracy
- Vanilla model: 92.5%
- Quantized model: 92.24%
The quantized model achieves 99.72% accuracy of the fp32 model
Latency
Payload sequence length: 128
Instance type: AWS c6i.xlarge
| latency | vanilla transformers | quantized optimum model | improvement |
|---|---|---|---|
| p95 | 75.69ms | 26.75ms | 2.83x |
| avg | 57.52ms | 24.86ms | 2.31x |
How to use
from optimum.onnxruntime import ORTModelForSequenceClassification
from transformers import pipeline, AutoTokenizer
model = ORTModelForSequenceClassification.from_pretrained("philschmid/quantized-distilbert-banking77")
tokenizer = AutoTokenizer.from_pretrained("philschmid/quantized-distilbert-banking77")
remote_clx = pipeline("text-classification",model=model, tokenizer=tokenizer)
remote_clx("What is the exchange rate like on this app?")
Model info
Created: 2022-06-07
Updated: 2022-06-08
View on Hugging Face