jina-embeddings-v2-base-code (INT8 量化版)

jinaai/jina-embeddings-v2-base-code 的 INT8 动态量化版本，适用于高效的 CPU 推理。

模型详情

使用方法

import onnxruntime as ort
from huggingface_hub import hf_hub_download
from tokenizers import Tokenizer
import numpy as np

# 加载
tokenizer = Tokenizer.from_file(hf_hub_download("nijaru/jina-code-int8", "tokenizer.json"))
tokenizer.enable_padding(pad_id=0, pad_token="[PAD]")
tokenizer.enable_truncation(max_length=512)
session = ort.InferenceSession(hf_hub_download("nijaru/jina-code-int8", "model_int8.onnx"))

def embed(texts):
    encoded = tokenizer.encode_batch(texts)
    input_ids = np.array([e.ids for e in encoded], dtype=np.int64)
    attention_mask = np.array([e.attention_mask for e in encoded], dtype=np.int64)
    outputs = session.run(None, {"input_ids": input_ids, "attention_mask": attention_mask})
    embeddings = outputs[0]
    mask = attention_mask[:, :, np.newaxis]
    return (embeddings * mask).sum(axis=1) / mask.sum(axis=1)

embeddings = embed(["def hello(): pass", "authentication flow"])

许可证

Apache-2.0（与基础模型相同）

致谢

由 Jina AI 基于 jinaai/jina-embeddings-v2-base-code 量化。