tarekziade/t5-small-headline-generator-sft-3-3

By tarekziade · ↓ 2 · ♥ 0 · Updated 2024-01-27

What's in this repo

This page lists the files in tarekziade/t5-small-headline-generator-sft-3-3, including decoder_model.onnx, decoder_model_merged.onnx, decoder_model_merged_quantized.onnx and more.

8 ONNX models (.onnx files)
19 files, including 8 .onnx, 7 .json, 1 .md, 1 .safetensors, 1 .model
Last updated 2024-01-27

ONNX files

File	Size	Download
onnx/decoder_model.onnx	—	Download
onnx/decoder_model_merged.onnx	—	Download
onnx/decoder_model_merged_quantized.onnx	—	Download
onnx/decoder_model_quantized.onnx	—	Download
onnx/decoder_with_past_model.onnx	—	Download
onnx/decoder_with_past_model_quantized.onnx	—	Download
onnx/encoder_model.onnx	—	Download
onnx/encoder_model_quantized.onnx	—	Download

How to use

These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.

README

Shrinked version of https://huggingface.co/JulesBelveze/t5-small-headline-generator

This model is a compression experiment. The original model is 242MiB and this model, once quantized, is down to ~50MiB with an accuracy of ~92%+

50% of the encoder and decoder layers were pruned, and the model was fine-tuned again on the original dataset for a single epoch.

The "shrink and fine-tune" strategy was inspired from https://arxiv.org/pdf/2010.13002.pdf

The model was then tested against the dataset using the ROUGE scoring, and the scoring was compared with the scoring of the original model to produce the accuracy scores below:

rouge-1 Accuracy:

F1 Accuracy: 92.27%
Precision Accuracy: 91.83%
Recall Accuracy: 93.95%

rouge-2 Accuracy:

F1 Accuracy: 94.48%
Precision Accuracy: 95.40%
Recall Accuracy: 92.01%

rouge-l Accuracy:

F1 Accuracy: 92.53%
Precision Accuracy: 92.11%
Recall Accuracy: 94.17%

You can find the train script here https://github.com/tarekziade/distill-t5/blob/main/sft.py and the evalutation script here https://github.com/tarekziade/distill-t5/blob/main/evaluation.py

Model info

transformers

transformers onnx safetensors t5 text2text-generation arxiv:2010.13002 license:apache-2.0 text-generation-inference endpoints_compatible region:us

↓ 2 ♥ 0

Created: 2024-01-27

Updated: 2024-01-27

View on Hugging Face

Files (19)

.gitattributes

README.md

added_tokens.json

config.json

generation_config.json

model.safetensors

onnx/decoder_model.onnx ONNX

onnx/decoder_model_merged.onnx ONNX

onnx/decoder_model_merged_quantized.onnx ONNX

onnx/decoder_model_quantized.onnx ONNX

onnx/decoder_with_past_model.onnx ONNX

onnx/decoder_with_past_model_quantized.onnx ONNX

onnx/encoder_model.onnx ONNX

onnx/encoder_model_quantized.onnx ONNX

quantize_config.json

special_tokens_map.json

spiece.model

tokenizer.json

tokenizer_config.json

Related models

More models

No task type declared.