fxmarty/20220911-h13m59s08_vit_beans_quantization

By fxmarty · ↓ 0 · ♥ 0 · Updated 2022-09-11

What's in this repo

This page lists the files in fxmarty/20220911-h13m59s08_vit_beans_quantization, including augmented_model.onnx, model.onnx, quantized_model.onnx and more.

41 ONNX models (.onnx files)
105 files, including 41 .onnx, 35 .json, 8 .npy, 1 .md, 1 .1, 1 .2
Last updated 2022-09-11

ONNX files

File	Size	Download
20220911-h14m03s04_0/augmented_model.onnx	—	Download
20220911-h14m03s04_0/model.onnx	—	Download
20220911-h14m03s04_0/quantized_model.onnx	—	Download
20220911-h14m04s10_1/model.onnx	—	Download
20220911-h14m04s10_1/quantized_model.onnx	—	Download
20220911-h14m05s03_2/model.onnx	—	Download
20220911-h14m05s03_2/quantized_model.onnx	—	Download
20220911-h14m09s58_3/augmented_model.onnx	—	Download
20220911-h14m09s58_3/model.onnx	—	Download
20220911-h14m09s58_3/quantized_model.onnx	—	Download

Showing 10 of 41 .onnx files — see the full file list on the right.

How to use

These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.

README

task: image-classification
Backend: sagemaker-training
Backend args: {'instance_type': 'ml.m5.2xlarge', 'supported_instructions': 'avx512'}
Number of evaluation samples: All dataset

Fixed parameters:

dataset: [{'path': 'beans', 'eval_split': 'validation', 'data_keys': {'primary': 'image'}, 'ref_keys': ['labels'], 'name': None, 'calibration_split': 'train'}]
name_or_path: nateraw/vit-base-beans
from_transformers: True
node_exclusion: []
calibration:
- method: percentile
- num_calibration_samples: 128
- calibration_histogram_percentile: 99.999

Benchmarked parameters:

framework: onnxruntime, pytorch
quantization_approach: dynamic, static
operators_to_quantize: ['Add', 'MatMul'], ['Add']
per_channel: False, True
framework_args: {'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}, {}
reduce_range: True, False
apply_quantization: True, False

Evaluation

Non-time metrics

framework	quantization_approach	operators_to_quantize	per_channel	framework_args	reduce_range	apply_quantization		accuracy
`onnxruntime`	`None`	`None`	`None`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`None`	`False`	\|	0.977
`onnxruntime`	`dynamic`	`['Add', 'MatMul']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	0.977
`onnxruntime`	`dynamic`	`['Add', 'MatMul']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	0.977
`onnxruntime`	`dynamic`	`['Add', 'MatMul']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	0.977
`onnxruntime`	`dynamic`	`['Add', 'MatMul']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	0.977
`onnxruntime`	`dynamic`	`['Add']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	0.977
`onnxruntime`	`dynamic`	`['Add']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	0.977
`onnxruntime`	`dynamic`	`['Add']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	0.977
`onnxruntime`	`dynamic`	`['Add']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	0.977
`onnxruntime`	`static`	`['Add', 'MatMul']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	0.421
`onnxruntime`	`static`	`['Add', 'MatMul']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	0.421
`onnxruntime`	`static`	`['Add', 'MatMul']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	0.316
`onnxruntime`	`static`	`['Add', 'MatMul']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	0.451
`onnxruntime`	`static`	`['Add']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	0.361
`onnxruntime`	`static`	`['Add']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	0.361
`onnxruntime`	`static`	`['Add']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	0.361
`onnxruntime`	`static`	`['Add']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	0.361
`pytorch`	`None`	`None`	`None`	`{}`	`None`	`None`	\|	0.977

Time metrics

Time benchmarks were run for 15 seconds per config.

Below, time metrics for batch size = 1, input length = 224.

framework	quantization_approach	operators_to_quantize	per_channel	framework_args	reduce_range	apply_quantization		latency_mean (ms)		throughput (/s)
`onnxruntime`	`None`	`None`	`None`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`None`	`False`	\|	130.41	\|	7.73
`onnxruntime`	`dynamic`	`['Add', 'MatMul']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	102.44	\|	9.80
`onnxruntime`	`dynamic`	`['Add', 'MatMul']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	101.57	\|	9.87
`onnxruntime`	`dynamic`	`['Add', 'MatMul']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	102.37	\|	9.80
`onnxruntime`	`dynamic`	`['Add', 'MatMul']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	102.36	\|	9.80
`onnxruntime`	`dynamic`	`['Add']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	130.67	\|	7.67
`onnxruntime`	`dynamic`	`['Add']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	131.29	\|	7.67
`onnxruntime`	`dynamic`	`['Add']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	132.65	\|	7.60
`onnxruntime`	`dynamic`	`['Add']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	131.03	\|	7.67
`onnxruntime`	`static`	`['Add', 'MatMul']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	127.99	\|	7.87
`onnxruntime`	`static`	`['Add', 'MatMul']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	128.27	\|	7.80
`onnxruntime`	`static`	`['Add', 'MatMul']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	131.10	\|	7.67
`onnxruntime`	`static`	`['Add', 'MatMul']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	130.29	\|	7.73
`onnxruntime`	`static`	`['Add']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	164.55	\|	6.13
`onnxruntime`	`static`	`['Add']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	168.61	\|	5.93
`onnxruntime`	`static`	`['Add']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	`True`	\|	164.52	\|	6.13
`onnxruntime`	`static`	`['Add']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	`True`	\|	165.31	\|	6.07
`pytorch`	`None`	`None`	`None`	`{}`	`None`	`None`	\|	149.23	\|	6.73

Model info

image-classification

tensorboard onnx vit image-classification dataset:beans region:us

↓ 0 ♥ 0

Created: 2022-09-11

Updated: 2022-09-11

View on Hugging Face

Files (105)

.gitattributes

20220911-h14m03s04_0/augmented_model.onnx ONNX

20220911-h14m03s04_0/calibration_histograms.npy

20220911-h14m03s04_0/model.onnx ONNX

20220911-h14m03s04_0/ort_config.json

20220911-h14m03s04_0/quantized_model.onnx ONNX

20220911-h14m03s04_0/results.json

20220911-h14m04s10_1/model.onnx ONNX

20220911-h14m04s10_1/ort_config.json

20220911-h14m04s10_1/quantized_model.onnx ONNX

20220911-h14m04s10_1/results.json

20220911-h14m05s03_2/model.onnx ONNX

20220911-h14m05s03_2/ort_config.json

20220911-h14m05s03_2/quantized_model.onnx ONNX

20220911-h14m05s03_2/results.json

20220911-h14m09s58_3/augmented_model.onnx ONNX

20220911-h14m09s58_3/calibration_histograms.npy

20220911-h14m09s58_3/model.onnx ONNX

20220911-h14m09s58_3/ort_config.json

20220911-h14m09s58_3/quantized_model.onnx ONNX

20220911-h14m09s58_3/results.json

20220911-h14m14s51_4/augmented_model.onnx ONNX

20220911-h14m14s51_4/calibration_histograms.npy

20220911-h14m14s51_4/model.onnx ONNX

20220911-h14m14s51_4/ort_config.json

20220911-h14m14s51_4/quantized_model.onnx ONNX

20220911-h14m14s51_4/results.json

20220911-h14m18s36_5/augmented_model.onnx ONNX

20220911-h14m18s36_5/calibration_histograms.npy

20220911-h14m18s36_5/model.onnx ONNX

20220911-h14m18s36_5/ort_config.json

20220911-h14m18s36_5/quantized_model.onnx ONNX

20220911-h14m18s36_5/results.json

20220911-h14m19s29_6/model.onnx ONNX

20220911-h14m19s29_6/ort_config.json

20220911-h14m19s29_6/quantized_model.onnx ONNX

20220911-h14m19s29_6/results.json

20220911-h14m20s17_7/model.onnx ONNX

20220911-h14m20s17_7/ort_config.json

20220911-h14m20s17_7/quantized_model.onnx ONNX

20220911-h14m20s17_7/results.json

20220911-h14m25s21_8/augmented_model.onnx ONNX

20220911-h14m25s21_8/calibration_histograms.npy

20220911-h14m25s21_8/model.onnx ONNX

20220911-h14m25s21_8/ort_config.json

20220911-h14m25s21_8/quantized_model.onnx ONNX

20220911-h14m25s21_8/results.json

20220911-h14m29s08_9/augmented_model.onnx ONNX

20220911-h14m29s08_9/calibration_histograms.npy

20220911-h14m29s08_9/model.onnx ONNX

20220911-h14m29s08_9/ort_config.json

20220911-h14m29s08_9/quantized_model.onnx ONNX

20220911-h14m29s08_9/results.json

20220911-h14m29s56_10/model.onnx ONNX

20220911-h14m29s56_10/ort_config.json

20220911-h14m29s56_10/quantized_model.onnx ONNX

20220911-h14m29s56_10/results.json

20220911-h14m31s00_11/model.onnx ONNX

20220911-h14m31s00_11/ort_config.json

20220911-h14m31s00_11/quantized_model.onnx ONNX

20220911-h14m31s00_11/results.json

20220911-h14m34s46_12/augmented_model.onnx ONNX

20220911-h14m34s46_12/calibration_histograms.npy

20220911-h14m34s46_12/model.onnx ONNX

20220911-h14m34s46_12/ort_config.json

20220911-h14m34s46_12/quantized_model.onnx ONNX

20220911-h14m34s46_12/results.json

20220911-h14m35s34_13/model.onnx ONNX

20220911-h14m35s34_13/ort_config.json

20220911-h14m35s34_13/quantized_model.onnx ONNX

20220911-h14m35s34_13/results.json

20220911-h14m36s21_14/model.onnx ONNX

20220911-h14m36s21_14/ort_config.json

20220911-h14m36s21_14/quantized_model.onnx ONNX

20220911-h14m36s21_14/results.json

20220911-h14m41s28_15/augmented_model.onnx ONNX

20220911-h14m41s28_15/calibration_histograms.npy

20220911-h14m41s28_15/model.onnx ONNX

20220911-h14m41s28_15/ort_config.json

20220911-h14m41s28_15/quantized_model.onnx ONNX

20220911-h14m41s28_15/results.json

20220911-h14m42s13_16/model.onnx ONNX

20220911-h14m42s13_16/results.json

20220911-h14m42s53_17/results.json

README.md

runs.json

tensorboard/1662907378.8460534/events.out.tfevents.1662907378.ip-10-0-138-55.ec2.internal.1.1

tensorboard/1662907378.8475587/events.out.tfevents.1662907378.ip-10-0-138-55.ec2.internal.1.2

tensorboard/1662907378.848857/events.out.tfevents.1662907378.ip-10-0-138-55.ec2.internal.1.3

tensorboard/1662907378.8500683/events.out.tfevents.1662907378.ip-10-0-138-55.ec2.internal.1.4

tensorboard/1662907378.8512115/events.out.tfevents.1662907378.ip-10-0-138-55.ec2.internal.1.5

tensorboard/1662907378.8523538/events.out.tfevents.1662907378.ip-10-0-138-55.ec2.internal.1.6

tensorboard/1662907378.8535051/events.out.tfevents.1662907378.ip-10-0-138-55.ec2.internal.1.7

tensorboard/1662907378.8546307/events.out.tfevents.1662907378.ip-10-0-138-55.ec2.internal.1.8

tensorboard/1662907378.855868/events.out.tfevents.1662907378.ip-10-0-138-55.ec2.internal.1.9

tensorboard/1662907378.857226/events.out.tfevents.1662907378.ip-10-0-138-55.ec2.internal.1.10

tensorboard/1662907378.8584979/events.out.tfevents.1662907378.ip-10-0-138-55.ec2.internal.1.11

tensorboard/1662907378.8597944/events.out.tfevents.1662907378.ip-10-0-138-55.ec2.internal.1.12

tensorboard/1662907378.8610377/events.out.tfevents.1662907378.ip-10-0-138-55.ec2.internal.1.13

tensorboard/1662907378.8623042/events.out.tfevents.1662907378.ip-10-0-138-55.ec2.internal.1.14

tensorboard/1662907378.8636096/events.out.tfevents.1662907378.ip-10-0-138-55.ec2.internal.1.15

tensorboard/1662907378.8649743/events.out.tfevents.1662907378.ip-10-0-138-55.ec2.internal.1.16

tensorboard/1662907378.8668153/events.out.tfevents.1662907378.ip-10-0-138-55.ec2.internal.1.17

tensorboard/1662907378.868103/events.out.tfevents.1662907378.ip-10-0-138-55.ec2.internal.1.18

tensorboard/events.out.tfevents.1662907378.ip-10-0-138-55.ec2.internal.1.0

Related models

More image-classification models

Browse all image-classification models