gibberish_detector_onnx: 量化 avx2

# pip install 'optimum[onnxruntime,exporters]'
from optimum.pipelines import pipeline


classifier = pipeline(
    \"text-classification\",
    model=\"pszemraj/gibberish_detector_onnx-quant-avx2\",
    accelerator=\"ort\",
)
classifier(\"ayy waddup\")
# [{'label': 'noise', 'score': 0.38642483949661255}]

量化参数的区别

带有 -pc 后缀的表示 per_channel=True

>>> src = 'quant_onnx_gibberish_detector' # avx2
>>> classifier = pipeline('text-classification', model=src, accelerator='ort')
>>> classifier('ayy waddup')
[{'label': 'noise', 'score': 0.34829846024513245}]
>>> src = 'quant_onnx_gibberish_detector-pc' # avx2 per channel (this model)
>>> classifier = pipeline('text-classification', model=src, accelerator='ort')
>>> classifier('ayy waddup')
[{'label': 'noise', 'score': 0.38642483949661255}]
>>> src = 'onnx_gibberish_detector' # unquantized onnx
>>> classifier = pipeline('text-classification', model=src, accelerator='ort')
>>> classifier('ayy waddup')
[{'label': 'noise', 'score': 0.6847617626190186}]