fxmarty/20220911-h15m48s16_
By fxmarty · ↓ 0 · ♥ 0 · Updated 2022-09-11
What's in this repo
This page lists the files in fxmarty/20220911-h15m48s16_, including model.onnx, quantized_model.onnx and more.
- 9 ONNX models (.onnx files)
- 29 files, including 11 .json, 9 .onnx, 1 .md, 1 .1, 1 .2, 1 .3
- Last updated 2022-09-11
ONNX files
| File | Size | Download |
|---|---|---|
| 20220911-h15m49s08_0/model.onnx | — | Download |
| 20220911-h15m49s08_0/quantized_model.onnx | — | Download |
| 20220911-h15m49s45_1/model.onnx | — | Download |
| 20220911-h15m49s45_1/quantized_model.onnx | — | Download |
| 20220911-h15m50s20_2/model.onnx | — | Download |
| 20220911-h15m50s20_2/quantized_model.onnx | — | Download |
| 20220911-h15m50s55_3/model.onnx | — | Download |
| 20220911-h15m50s55_3/quantized_model.onnx | — | Download |
| 20220911-h15m51s28_4/model.onnx | — | Download |
How to use
These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.
README
task: text-classification
Backend: sagemaker-training
Backend args: {'instance_type': 'ml.m5.2xlarge', 'supported_instructions': 'avx512'}
Number of evaluation samples: All dataset
Fixed parameters:
- dataset: [{'path': 'glue', 'eval_split': 'validation', 'data_keys': {'primary': 'sentence'}, 'ref_keys': ['label'], 'name': 'sst2', 'calibration_split': None}]
- name_or_path:
distilbert-base-uncased-finetuned-sst-2-english - from_transformers:
True - quantization_approach:
dynamic - node_exclusion:
[]
Benchmarked parameters:
- framework:
onnxruntime,pytorch - operators_to_quantize:
['Add', 'MatMul'],['Add'] - per_channel:
False,True - framework_args:
{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4},{} - apply_quantization:
True,False
Evaluation
Non-time metrics
| framework | operators_to_quantize | per_channel | framework_args | apply_quantization | accuracy | |
|---|---|---|---|---|---|---|
onnxruntime |
None |
None |
{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4} |
False |
| | 0.911 |
onnxruntime |
['Add', 'MatMul'] |
False |
{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4} |
True |
| | 0.898 |
onnxruntime |
['Add', 'MatMul'] |
True |
{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4} |
True |
| | 0.490 |
onnxruntime |
['Add'] |
False |
{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4} |
True |
| | 0.911 |
onnxruntime |
['Add'] |
True |
{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4} |
True |
| | 0.911 |
pytorch |
None |
None |
{} |
None |
| | 0.911 |
Time metrics
Time benchmarks were run for 15 seconds per config.
Below, time metrics for batch size = 1, input length = 224.
| framework | operators_to_quantize | per_channel | framework_args | apply_quantization | latency_mean (ms) | throughput (/s) | ||
|---|---|---|---|---|---|---|---|---|
onnxruntime |
None |
None |
{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4} |
False |
| | 83.23 | | | 12.07 |
onnxruntime |
['Add', 'MatMul'] |
False |
{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4} |
True |
| | 64.31 | | | 15.60 |
onnxruntime |
['Add', 'MatMul'] |
True |
{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4} |
True |
| | 64.78 | | | 15.47 |
onnxruntime |
['Add'] |
False |
{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4} |
True |
| | 82.63 | | | 12.13 |
onnxruntime |
['Add'] |
True |
{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4} |
True |
| | 83.82 | | | 11.93 |
pytorch |
None |
None |
{} |
None |
| | 84.34 | | | 11.87 |
Model info
tensorboard
onnx
distilbert
text-classification
dataset:glue
region:us
↓ 0
♥ 0
Created: 2022-09-11
Updated: 2022-09-11
View on Hugging FaceFiles (29)
.gitattributes
20220911-h15m49s08_0/model.onnx
ONNX
20220911-h15m49s08_0/ort_config.json
20220911-h15m49s08_0/quantized_model.onnx
ONNX
20220911-h15m49s08_0/results.json
20220911-h15m49s45_1/model.onnx
ONNX
20220911-h15m49s45_1/ort_config.json
20220911-h15m49s45_1/quantized_model.onnx
ONNX
20220911-h15m49s45_1/results.json
20220911-h15m50s20_2/model.onnx
ONNX
20220911-h15m50s20_2/ort_config.json
20220911-h15m50s20_2/quantized_model.onnx
ONNX
20220911-h15m50s20_2/results.json
20220911-h15m50s55_3/model.onnx
ONNX
20220911-h15m50s55_3/ort_config.json
20220911-h15m50s55_3/quantized_model.onnx
ONNX
20220911-h15m50s55_3/results.json
20220911-h15m51s28_4/model.onnx
ONNX
20220911-h15m51s28_4/results.json
20220911-h15m52s11_5/results.json
README.md
runs.json
tensorboard/1662911537.5446625/events.out.tfevents.1662911537.ip-10-0-148-3.ec2.internal.1.1
tensorboard/1662911537.5460162/events.out.tfevents.1662911537.ip-10-0-148-3.ec2.internal.1.2
tensorboard/1662911537.5475845/events.out.tfevents.1662911537.ip-10-0-148-3.ec2.internal.1.3
tensorboard/1662911537.548684/events.out.tfevents.1662911537.ip-10-0-148-3.ec2.internal.1.4
tensorboard/1662911537.5497618/events.out.tfevents.1662911537.ip-10-0-148-3.ec2.internal.1.5
tensorboard/1662911537.550816/events.out.tfevents.1662911537.ip-10-0-148-3.ec2.internal.1.6
tensorboard/events.out.tfevents.1662911537.ip-10-0-148-3.ec2.internal.1.0