Ashx098/Embedding-Gemma_Onnx-for-vespa

Embedding Gemma ONNX for Vespa

This is a Gemma-based embedding model converted to ONNX format, optimized for use with Vespa database.

Why Single ONNX File?

Vespa database expects a single ONNX model file for embeddings. Typically, ONNX models are exported with separate files:

• model.onnx - the model structure
• onnx_data/ - weight files

This model has been merged into a single ONNX file (model_merged.onnx) so it works directly with Vespa without needing to handle external weight files.

Model Details

Files

Usage with ONNX Runtime

import onnxruntime as ort
import sentencepiece as spm
import numpy as np

# Load ONNX model
session = ort.InferenceSession('model_merged.onnx')

# Load tokenizer
sp = spm.SentencePieceProcessor()
sp.load('tokenizer.model')

# Generate embedding
text = 'Your search query'
input_ids = sp.encode(text)
attention_mask = [1] * len(input_ids)

outputs = session.run(None, {
    'input_ids': np.array([input_ids], dtype=np.int64),
    'attention_mask': np.array([attention_mask], dtype=np.int64)
})

embedding = outputs[1][0]  # sentence_embedding output
print(embedding.shape)  # (768,)

Requirements

• onnx
• onnxruntime
• sentencepiece
• numpy