echarlaix/t5-small-int8-dynamic
By echarlaix · ↓ 4 · ♥ 4 · Updated 2023-10-26
What's in this repo
This page lists the files in echarlaix/t5-small-int8-dynamic, including decoder_model.onnx, decoder_with_past_model.onnx, encoder_model.onnx.
- 3 ONNX models (.onnx files)
- 8 files, including 3 .json, 3 .onnx, 1 .md
- Last updated 2023-10-26
ONNX files
| File | Size | Download |
|---|---|---|
| decoder_model.onnx | — | Download |
| decoder_with_past_model.onnx | — | Download |
| encoder_model.onnx | — | Download |
How to use
These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.
README
t5-small exported to the ONNX format and dynamically quantized.
Model description
T5 is an encoder-decoder model pre-trained on a multi-task mixture of unsupervised and supervised tasks and for which each task is converted into a text-to-text format.
For more information, please take a look at the original paper.
Paper: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Authors: Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu
Usage example
You can use this model with Transformers pipeline.
from transformers import AutoTokenizer, pipeline
from optimum.onnxruntime import ORTModelForSeq2SeqLM
tokenizer = AutoTokenizer.from_pretrained("echarlaix/t5-small-dynamic")
model = ORTModelForSeq2SeqLM.from_pretrained("echarlaix/t5-small-dynamic")
translator = pipeline("translation_en_to_fr", model=model, tokenizer=tokenizer)
text = "He never went out without a book under his arm, and he often came back with two."
results = translator(text)
print(results)
Model info
Created: 2022-08-01
Updated: 2023-10-26
View on Hugging Face