khaangnguyeen/ruri-v3-310m-onnx

Ruri v3 310M — ONNX Format

This is the ONNX version of the embedding model cl-nagoya/ruri-v3-310m.

Model Information

• Base Model: cl-nagoya/ruri-v3-310m
• Model Type: ModernBERT (encoder-only)
• Task: Feature Extraction / Sentence Embeddings
• ONNX Opset: 17
• Optimization: O2 (Medium)

Files

Usage

With Optimum (Recommended)

from optimum.onnxruntime import ORTModelForFeatureExtraction
from transformers import AutoTokenizer
import torch

# Load model and tokenizer
model = ORTModelForFeatureExtraction.from_pretrained(
    "khaangnguyeen/ruri-v3-310m-onnx",
    provider="CPUExecutionProvider"  # or "CUDAExecutionProvider" for GPU
)
tokenizer = AutoTokenizer.from_pretrained("khaangnguyeen/ruri-v3-310m-onnx")

# Encode text
texts = ["Example sentence", "Another example"]
inputs = tokenizer(texts, padding=True, truncation=True, return_tensors="pt")

# Get embeddings
with torch.no_grad():
    outputs = model(**inputs)
    embeddings = outputs.last_hidden_state.mean(dim=1)  # Mean pooling

print(embeddings.shape)  # (2, hidden_size)

With ONNX Runtime (Direct)

import onnxruntime as ort
import numpy as np
from transformers import AutoTokenizer

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained("khaangnguyeen/ruri-v3-310m-onnx")

# Create ONNX Runtime session
session = ort.InferenceSession("model.onnx")

# Prepare inputs
text = "Example sentence"
inputs = tokenizer(text, return_tensors="np", padding=True, truncation=True)

# Run inference
outputs = session.run(
    None,
    {
        "input_ids": inputs["input_ids"],
        "attention_mask": inputs["attention_mask"]
    }
)

embeddings = outputs[0].mean(axis=1) 
print(embeddings.shape)

Performance

ONNX Runtime provides:

• ⚡ Faster inference speed compared to PyTorch (typically 1.5-3x)
• 💾 Smaller footprint / optimized graph
• 🎯 Optimized for production deployment
• 🔧 Support for various hardware backends (CPU, GPU, NPU)

Benchmark

CPU Benchmark (example):

• PyTorch: ~XX.XX ms/sample
• ONNX Runtime: ~XX.XX ms/sample
• Speedup: ~X.XXx

License

Apache-2.0 (inherited from base model)

Citation

@misc{ruri-v3-onnx,
  author = {khaangnguyeen},
  title = {Ruri v3 310M ONNX},
  year = {2025},
  publisher = {Hugging Face},
  howpublished = {\url{[https://huggingface.co/khaangnguyeen/ruri-v3-310m-onnx](https://huggingface.co/khaangnguyeen/ruri-v3-310m-onnx)}}
}