libryo-ai/BAAI-bge-m3-int8
By libryo-ai · ↓ 8 · ♥ 1 · Updated 2024-06-11
What's in this repo
This page lists the files in libryo-ai/BAAI-bge-m3-int8, including model.onnx.
- 1 ONNX models (.onnx files)
- 8 files, including 5 .json, 1 .md, 1 .onnx
- Last updated 2024-06-11
ONNX files
| File | Size | Download |
|---|---|---|
| model.onnx | — | Download |
How to use
These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.
README
Converted BAAI/bge-m3 model (dense retriever only) in onnx int8 format for use with Vespa Embedding.
- BAAI-bge-m3_quantized.onnx (int8 quantized)
The model was quantized using the optimum toolkit.
Tips: conver to int8 quantized
# https://github.com/vespa-engine/sample-apps/blob/master/simple-semantic-search/export_hf_model_from_hf.py
./export_hf_model_from_hf.py --hf_model BAAI/bge-m3 --output_dir bge-m3
optimum-cli onnxruntime quantize --onnx_model ./bge-m3 -o bge-m3-large_quantized --avx512_vnni
License
The license for this model is based on the original license (found in the LICENSE file in the project's root directory), which is the MIT License.
- https://huggingface.co/BAAI/bge-m3
Attribution
All credits for this model go to the authors of BAAI/bge-m3 and the associated researchers and organizations. When using this model, please be sure to attribute the original authors.
Model info
Created: 2024-06-11
Updated: 2024-06-11
View on Hugging Face