jian-mo/jina-reranker-m0-onnx

By jian-mo · ↓ 0 · ♥ 2 · Updated 2025-04-09

What's in this repo

This page lists the files in jian-mo/jina-reranker-m0-onnx, including jina-reranker-m0.onnx.

1 ONNX models (.onnx files)
287 files, including 58 .weight, 28 .bias, 2 .md, 1 .onnx
Last updated 2025-04-09

ONNX files

File	Size	Download
jina-reranker-m0.onnx	—	Download

How to use

These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.

README

Jina Reranker M0 - ONNX FP16 Version

This repository contains the jinaai/jina-reranker-m0 model converted to the ONNX format with FP16 precision.

Model Description

Jina Reranker is designed to rerank search results or document passages based on their relevance to a given query. It takes a query and a list of documents as input and outputs relevance scores.

This version is specifically exported for use with ONNX Runtime.

Original Model Card: jinaai/jina-reranker-m0

Technical Details

Format: ONNX
Opset: 14
Precision: FP16 (exported using .half())
External Data: Uses ONNX external data format due to model size. All files in this repository are required. huggingface_hub handles downloading them automatically.
Export Source: Exported from the Hugging Face transformers library using torch.onnx.export.

Usage

You can use this model with onnxruntime for inference. You will also need the transformers library to load the appropriate processor for input preparation and huggingface_hub to download the model files.

1. Installation:

pip install onnxruntime huggingface_hub transformers torch sentencepiece

2. Inference Script:

import onnxruntime as ort
from huggingface_hub import hf_hub_download
from transformers import AutoProcessor
import numpy as np
import torch # For processor output handling

# --- Configuration ---
# Replace with your repository ID if different
repo_id = "jian-mo/jina-reranker-m0-onnx"
onnx_filename = "jina-reranker-m0.onnx" # Main ONNX file name
# Use the original model ID to load the correct processor
original_model_id = "jinaai/jina-reranker-m0"
# --- End Configuration ---

# 1. Download ONNX model files from the Hub
# hf_hub_download automatically handles external data files linked via LFS
print(f"Downloading ONNX model from {repo_id}...")
local_onnx_path = hf_hub_download(
    repo_id=repo_id,
    filename=onnx_filename
)
print(f"ONNX model downloaded to: {local_onnx_path}")

# 2. Load ONNX Runtime session
print("Loading ONNX Inference Session...")
# You can choose execution providers, e.g., ['CUDAExecutionProvider', 'CPUExecutionProvider']
# if you have GPU support and the necessary onnxruntime build.
session_options = ort.SessionOptions()
# session_options.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_EXTENDED
providers = ['CPUExecutionProvider'] # Default to CPU
session = ort.InferenceSession(local_onnx_path, sess_options=session_options, providers=providers)
print(f"ONNX session loaded with provider: {session.get_providers()}")

# 3. Load the Processor
print(f"Loading processor from {original_model_id}...")
processor = AutoProcessor.from_pretrained(original_model_id, trust_remote_code=True)
print("Processor loaded.")

# 4. Prepare Input Data
query = "What is deep learning?"
document = "Deep learning is a subset of machine learning based on artificial neural networks with representation learning."
# Example with multiple documents (batch processing)
# documents = [
#     "Deep learning is a subset of machine learning based on artificial neural networks with representation learning.",
#     "Artificial intelligence refers to the simulation of human intelligence in machines.",
#     "A transformer is a deep learning model used primarily in the field of natural language processing."
# ]
# Use processor logic suitable for query + multiple documents if needed

print("Preparing input data...")
# Process query and document together as expected by the reranker model
inputs = processor(
    text=f"{query} {document}",
    images=None, # Assuming text-only reranking
    return_tensors="pt", # Get PyTorch tensors first
    padding=True,
    truncation=True,
    max_length=512 # Use a reasonable max_length
)

# Convert to NumPy for ONNX Runtime
inputs_np = {
    "input_ids": inputs["input_ids"].numpy(),
    "attention_mask": inputs["attention_mask"].numpy()
}
print("Input data prepared.")
# print("Input shapes:", {k: v.shape for k, v in inputs_np.items()})

# 5. Run Inference
print("Running inference...")
output_names = [output.name for output in session.get_outputs()]
outputs = session.run(output_names, inputs_np)
print("Inference complete.")

# 6. Process Output
# The exact interpretation depends on the model's output structure.
# For Jina Reranker, the output is typically a logit score.
# Higher values usually indicate higher relevance. Check the original model card.
print(f"Number of outputs: {len(outputs)}")
if len(outputs) > 0:
    logits = outputs[0]
    print(f"Output logits shape: {logits.shape}")
    # Often, the relevance score is associated

Model info

sentence-similarity onnx

onnx text-reranking jina fp16 sentence-similarity base_model:jinaai/jina-reranker-m0 base_model:quantized:jinaai/jina-reranker-m0 region:us

↓ 0 ♥ 2

Created: 2025-04-09

Updated: 2025-04-09

View on Hugging Face

Files (287)

.gitattributes

README.md

jina-reranker-m0.onnx ONNX

model.embed_tokens.weight

model.layers.0.input_layernorm.weight

model.layers.0.post_attention_layernorm.weight

model.layers.0.self_attn.q_proj.bias

model.layers.1.input_layernorm.weight

model.layers.1.post_attention_layernorm.weight

model.layers.1.self_attn.q_proj.bias

model.layers.10.input_layernorm.weight

model.layers.10.post_attention_layernorm.weight

model.layers.10.self_attn.q_proj.bias

model.layers.11.input_layernorm.weight

model.layers.11.post_attention_layernorm.weight

model.layers.11.self_attn.q_proj.bias

model.layers.12.input_layernorm.weight

model.layers.12.post_attention_layernorm.weight

model.layers.12.self_attn.q_proj.bias

model.layers.13.input_layernorm.weight

model.layers.13.post_attention_layernorm.weight

model.layers.13.self_attn.q_proj.bias

model.layers.14.input_layernorm.weight

model.layers.14.post_attention_layernorm.weight

model.layers.14.self_attn.q_proj.bias

model.layers.15.input_layernorm.weight

model.layers.15.post_attention_layernorm.weight

model.layers.15.self_attn.q_proj.bias

model.layers.16.input_layernorm.weight

model.layers.16.post_attention_layernorm.weight

model.layers.16.self_attn.q_proj.bias

model.layers.17.input_layernorm.weight

model.layers.17.post_attention_layernorm.weight

model.layers.17.self_attn.q_proj.bias

model.layers.18.input_layernorm.weight

model.layers.18.post_attention_layernorm.weight

model.layers.18.self_attn.q_proj.bias

model.layers.19.input_layernorm.weight

model.layers.19.post_attention_layernorm.weight

model.layers.19.self_attn.q_proj.bias

model.layers.2.input_layernorm.weight

model.layers.2.post_attention_layernorm.weight

model.layers.2.self_attn.q_proj.bias

model.layers.20.input_layernorm.weight

model.layers.20.post_attention_layernorm.weight

model.layers.20.self_attn.q_proj.bias

model.layers.21.input_layernorm.weight

model.layers.21.post_attention_layernorm.weight

model.layers.21.self_attn.q_proj.bias

model.layers.22.input_layernorm.weight

model.layers.22.post_attention_layernorm.weight

model.layers.22.self_attn.q_proj.bias

model.layers.23.input_layernorm.weight

model.layers.23.post_attention_layernorm.weight

model.layers.23.self_attn.q_proj.bias

model.layers.24.input_layernorm.weight

model.layers.24.post_attention_layernorm.weight

model.layers.24.self_attn.q_proj.bias

model.layers.25.input_layernorm.weight

model.layers.25.post_attention_layernorm.weight

model.layers.25.self_attn.q_proj.bias

model.layers.26.input_layernorm.weight

model.layers.26.post_attention_layernorm.weight

model.layers.26.self_attn.q_proj.bias

model.layers.27.input_layernorm.weight

model.layers.27.post_attention_layernorm.weight

model.layers.27.self_attn.q_proj.bias

model.layers.3.input_layernorm.weight

model.layers.3.post_attention_layernorm.weight

model.layers.3.self_attn.q_proj.bias

model.layers.4.input_layernorm.weight

model.layers.4.post_attention_layernorm.weight

model.layers.4.self_attn.q_proj.bias

model.layers.5.input_layernorm.weight

model.layers.5.post_attention_layernorm.weight

model.layers.5.self_attn.q_proj.bias

model.layers.6.input_layernorm.weight

model.layers.6.post_attention_layernorm.weight

model.layers.6.self_attn.q_proj.bias

model.layers.7.input_layernorm.weight

model.layers.7.post_attention_layernorm.weight

model.layers.7.self_attn.q_proj.bias

model.layers.8.input_layernorm.weight

model.layers.8.post_attention_layernorm.weight

model.layers.8.self_attn.q_proj.bias

model.layers.9.input_layernorm.weight

model.layers.9.post_attention_layernorm.weight

model.layers.9.self_attn.q_proj.bias

model.norm.weight

onnx__MatMul_8817

onnx__MatMul_8818

onnx__MatMul_8819

onnx__MatMul_8844

onnx__MatMul_8845

onnx__MatMul_8846

onnx__MatMul_8847

onnx__MatMul_8848

onnx__MatMul_8849

onnx__MatMul_8850

onnx__MatMul_8875

onnx__MatMul_8876

onnx__MatMul_8877

onnx__MatMul_8878

onnx__MatMul_8879

onnx__MatMul_8880

onnx__MatMul_8881

onnx__MatMul_8906

onnx__MatMul_8907

onnx__MatMul_8908

onnx__MatMul_8909

onnx__MatMul_8910

onnx__MatMul_8911

onnx__MatMul_8912

onnx__MatMul_8937

onnx__MatMul_8938

onnx__MatMul_8939

onnx__MatMul_8940

onnx__MatMul_8941

onnx__MatMul_8942

onnx__MatMul_8943

onnx__MatMul_8968

onnx__MatMul_8969

onnx__MatMul_8970

onnx__MatMul_8971

onnx__MatMul_8972

onnx__MatMul_8973

onnx__MatMul_8974

onnx__MatMul_8999

onnx__MatMul_9000

onnx__MatMul_9001

onnx__MatMul_9002

onnx__MatMul_9003

onnx__MatMul_9004

onnx__MatMul_9005

onnx__MatMul_9030

onnx__MatMul_9031

onnx__MatMul_9032

onnx__MatMul_9033

onnx__MatMul_9034

onnx__MatMul_9035

onnx__MatMul_9036

onnx__MatMul_9061

onnx__MatMul_9062

onnx__MatMul_9063

onnx__MatMul_9064

onnx__MatMul_9065

onnx__MatMul_9066

onnx__MatMul_9067

onnx__MatMul_9092

onnx__MatMul_9093

onnx__MatMul_9094

onnx__MatMul_9095

onnx__MatMul_9096

onnx__MatMul_9097

onnx__MatMul_9098

onnx__MatMul_9123

onnx__MatMul_9124

onnx__MatMul_9125

onnx__MatMul_9126

onnx__MatMul_9127

onnx__MatMul_9128

onnx__MatMul_9129

onnx__MatMul_9154

onnx__MatMul_9155

onnx__MatMul_9156

onnx__MatMul_9157

onnx__MatMul_9158

onnx__MatMul_9159

onnx__MatMul_9160

onnx__MatMul_9185

onnx__MatMul_9186

onnx__MatMul_9187

onnx__MatMul_9188

onnx__MatMul_9189

onnx__MatMul_9190

onnx__MatMul_9191

onnx__MatMul_9216

onnx__MatMul_9217

onnx__MatMul_9218

onnx__MatMul_9219

onnx__MatMul_9220

onnx__MatMul_9221

onnx__MatMul_9222

onnx__MatMul_9247

onnx__MatMul_9248

onnx__MatMul_9249

onnx__MatMul_9250

onnx__MatMul_9251

onnx__MatMul_9252

onnx__MatMul_9253

onnx__MatMul_9278

onnx__MatMul_9279

onnx__MatMul_9280

onnx__MatMul_9281

onnx__MatMul_9282

onnx__MatMul_9283

onnx__MatMul_9284

onnx__MatMul_9309

onnx__MatMul_9310

onnx__MatMul_9311

onnx__MatMul_9312

onnx__MatMul_9313

onnx__MatMul_9314

onnx__MatMul_9315

onnx__MatMul_9340

onnx__MatMul_9341

onnx__MatMul_9342

onnx__MatMul_9343

onnx__MatMul_9344

onnx__MatMul_9345

onnx__MatMul_9346

onnx__MatMul_9371

onnx__MatMul_9372

onnx__MatMul_9373

onnx__MatMul_9374

onnx__MatMul_9375

onnx__MatMul_9376

onnx__MatMul_9377

onnx__MatMul_9402

onnx__MatMul_9403

onnx__MatMul_9404

onnx__MatMul_9405

onnx__MatMul_9406

onnx__MatMul_9407

onnx__MatMul_9408

onnx__MatMul_9433

onnx__MatMul_9434

onnx__MatMul_9435

onnx__MatMul_9436

onnx__MatMul_9437

onnx__MatMul_9438

onnx__MatMul_9439

onnx__MatMul_9464

onnx__MatMul_9465

onnx__MatMul_9466

onnx__MatMul_9467

onnx__MatMul_9468

onnx__MatMul_9469

onnx__MatMul_9470

onnx__MatMul_9495

onnx__MatMul_9496

onnx__MatMul_9497

onnx__MatMul_9498

onnx__MatMul_9499

onnx__MatMul_9500

onnx__MatMul_9501

onnx__MatMul_9526

onnx__MatMul_9527

onnx__MatMul_9528

onnx__MatMul_9529

onnx__MatMul_9530

onnx__MatMul_9531

onnx__MatMul_9532

onnx__MatMul_9557

onnx__MatMul_9558

onnx__MatMul_9559

onnx__MatMul_9560

onnx__MatMul_9561

onnx__MatMul_9562

onnx__MatMul_9563

onnx__MatMul_9588

onnx__MatMul_9589

onnx__MatMul_9590

onnx__MatMul_9591

onnx__MatMul_9592

onnx__MatMul_9593

onnx__MatMul_9594

onnx__MatMul_9619

onnx__MatMul_9620

onnx__MatMul_9621

onnx__MatMul_9622

onnx__MatMul_9623

onnx__MatMul_9624

onnx__MatMul_9625

onnx__MatMul_9650

onnx__MatMul_9651

onnx__MatMul_9652

onnx__MatMul_9653

onnx__MatMul_9654

onnx__MatMul_9655

onnx__MatMul_9656

onnx__MatMul_9681

onnx__MatMul_9682

onnx__MatMul_9683

onnx__MatMul_9684

onnx__MatMul_9685

readme.md

Related models

More from jian-mo

No other models by this author.

More sentence-similarity models

Browse all sentence-similarity models