返回模型

说明文档

任务: text-classification
后端: sagemaker-training
后端参数: {'instance_type': 'ml.m5.2xlarge', 'supported_instructions': 'avx512'}
评估样本数量: 全量数据集

固定参数:

数据集: [{'path': 'glue', 'eval_split': 'validation', 'data_keys': {'primary': 'sentence'}, 'ref_keys': ['label'], 'name': 'sst2', 'calibration_split': None}]
模型名称或路径: distilbert-base-uncased-finetuned-sst-2-english
来自transformers: True
量化方法: dynamic
节点排除: []

基准测试参数:

框架: onnxruntime, pytorch
待量化算子: ['Add', 'MatMul'], ['Add']
逐通道: False, True
框架参数: {'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}, {}
应用量化: True, False

评估

非时间指标

框架	待量化算子	逐通道	框架参数	应用量化		准确率
`onnxruntime`	`None`	`None`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	\|	0.911
`onnxruntime`	`['Add', 'MatMul']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	\|	0.898
`onnxruntime`	`['Add', 'MatMul']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	\|	0.490
`onnxruntime`	`['Add']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	\|	0.911
`onnxruntime`	`['Add']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	\|	0.911
`pytorch`	`None`	`None`	`{}`	`None`	\|	0.911

时间指标

时间基准测试每个配置运行 15 秒。

以下是批次大小 = 1、输入长度 = 224 的时间指标。

框架	待量化算子	逐通道	框架参数	应用量化		平均延迟 (毫秒)		吞吐量 (/秒)
`onnxruntime`	`None`	`None`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`False`	\|	83.23	\|	12.07
`onnxruntime`	`['Add', 'MatMul']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	\|	64.31	\|	15.60
`onnxruntime`	`['Add', 'MatMul']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	\|	64.78	\|	15.47
`onnxruntime`	`['Add']`	`False`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	\|	82.63	\|	12.13
`onnxruntime`	`['Add']`	`True`	`{'opset': 13, 'optimization_level': 1, 'intra_op_num_threads': 4}`	`True`	\|	83.82	\|	11.93
`pytorch`	`None`	`None`	`{}`	`None`	\|	84.34	\|	11.87

fxmarty/20220911-h15m48s16_

作者 fxmarty

text-classification

↓ 0 ♥ 0

创建时间: 2022-09-11 15:52:12+00:00

更新时间: 2022-09-11 15:52:34+00:00

在 Hugging Face 上查看

文件 (29)

.gitattributes

20220911-h15m49s08_0/model.onnx ONNX

20220911-h15m49s08_0/ort_config.json

20220911-h15m49s08_0/quantized_model.onnx ONNX

20220911-h15m49s08_0/results.json

20220911-h15m49s45_1/model.onnx ONNX

20220911-h15m49s45_1/ort_config.json

20220911-h15m49s45_1/quantized_model.onnx ONNX

20220911-h15m49s45_1/results.json

20220911-h15m50s20_2/model.onnx ONNX

20220911-h15m50s20_2/ort_config.json

20220911-h15m50s20_2/quantized_model.onnx ONNX

20220911-h15m50s20_2/results.json

20220911-h15m50s55_3/model.onnx ONNX

20220911-h15m50s55_3/ort_config.json

20220911-h15m50s55_3/quantized_model.onnx ONNX

20220911-h15m50s55_3/results.json

20220911-h15m51s28_4/model.onnx ONNX

20220911-h15m51s28_4/results.json

20220911-h15m52s11_5/results.json

README.md

runs.json

tensorboard/1662911537.5446625/events.out.tfevents.1662911537.ip-10-0-148-3.ec2.internal.1.1

tensorboard/1662911537.5460162/events.out.tfevents.1662911537.ip-10-0-148-3.ec2.internal.1.2

tensorboard/1662911537.5475845/events.out.tfevents.1662911537.ip-10-0-148-3.ec2.internal.1.3

tensorboard/1662911537.548684/events.out.tfevents.1662911537.ip-10-0-148-3.ec2.internal.1.4

tensorboard/1662911537.5497618/events.out.tfevents.1662911537.ip-10-0-148-3.ec2.internal.1.5

tensorboard/1662911537.550816/events.out.tfevents.1662911537.ip-10-0-148-3.ec2.internal.1.6

tensorboard/events.out.tfevents.1662911537.ip-10-0-148-3.ec2.internal.1.0