RedHatAI/sst2-distilbert-sparse-blog
By RedHatAI · ↓ 6 · ♥ 4 · Updated 2024-03-13
What's in this repo
This page lists the files in RedHatAI/sst2-distilbert-sparse-blog, including model.onnx.
- 1 ONNX models (.onnx files)
- 8 files, including 3 .json, 1 .md, 1 .py, 1 .onnx, 1 .txt
- Last updated 2024-03-13
ONNX files
| File | Size | Download |
|---|---|---|
| model.onnx | — | Download |
How to use
These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.
README
Repo includes all necessary files for staging an Inference Endpoints API with DeepSparse as discussed in this BLOG. This DistilBERT was sparsified using the SparseML library.
Sparse Transfer 80% VNNI Pruned DistilBERT
This model is the result of pruning the DistilBERT model to 80% using the VNNI blocking (semi-structured), followed by fine-tuning and quantization on the SST2 dataset. Pruning is performed with the GMP algorithm and using the masked language modeling task based on the BookCorpus and Wikipedia datasets. It achieves 90.5% accuracy on the validation dataset, recovering over 99% of the accuracy of the baseline model. See the included recipe for training instructions.
Model info
Created: 2022-12-19
Updated: 2024-03-13
View on Hugging Face