geoffsee/octen-embedding-0.6b-onnx

By geoffsee · ↓ 0 · ♥ 0 · Updated 2026-02-26

What's in this repo

This page lists the files in geoffsee/octen-embedding-0.6b-onnx, including model.fp16.onnx, model.int4.bnb4.onnx, model.int4.onnx and more.

4 ONNX models (.onnx files)
17 files, including 6 .json, 4 .onnx, 3 .data, 1 .md, 1 .onnx_data, 1 .txt
Last updated 2026-02-26

ONNX files

File	Size	Download
model.fp16.onnx	—	Download
model.int4.bnb4.onnx	—	Download
model.int4.onnx	—	Download
model.int8.onnx	—	Download

How to use

These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.

README

Octen-Embedding-0.6B ONNX

ONNX export of Octen/Octen-Embedding-0.6B for inference with ONNX Runtime (Python, Web/WASM, etc.).

Base model: Octen/Octen-Embedding-0.6B
Pooling: last token, L2-normalized
Max sequence length: 512
Dynamic batch: True
Hidden size: 1024

Files

File	Description
`model.fp16.onnx` (+ `.onnx.data`)	FP16 weights, ~1.1 GB
`model.int8.onnx`	INT8 quantized, ~560 MB
`tokenizer/`	Hugging Face tokenizer (same as base model)
`conversion-metadata.json`	Export config

Usage

Python (ONNX Runtime)

import numpy as np
from transformers import AutoTokenizer
from huggingface_hub import hf_hub_download
import onnxruntime as ort

# Download from this repo
repo_id = "geoffsee/octen-embedding-0.6b-onnx"
path_fp16 = hf_hub_download(repo_id=repo_id, filename="model.fp16.onnx")
path_int8 = hf_hub_download(repo_id=repo_id, filename="model.int8.onnx")
tokenizer_path = hf_hub_download(repo_id=repo_id, filename="tokenizer/tokenizer.json", repo_type="model")

tokenizer = AutoTokenizer.from_pretrained(repo_id)
session = ort.InferenceSession(path_int8, providers=["CPUExecutionProvider"])

encoded = tokenizer("Your text here", return_tensors="np", padding=True, truncation=True, max_length=512)
outputs = session.run(None, {"input_ids": encoded["input_ids"].astype(np.int64), "attention_mask": encoded["attention_mask"].astype(np.int64)})
embeddings = outputs[0]  # (batch, 1024)

JavaScript / ONNX Runtime Web

Use model.fp16.onnx or model.int8.onnx with onnxruntime-web. Load the tokenizer from tokenizer/ (e.g. with a compatible JS tokenizer).

Model info

sentence-transformers

sentence-transformers onnx embeddings Octen license:mit region:us

↓ 0 ♥ 0

Created: 2026-02-26

Updated: 2026-02-26

View on Hugging Face

Files (17)

.gitattributes

README.md

conversion-metadata.json

model.fp16.onnx ONNX

model.fp16.onnx.data

model.int4.bnb4.onnx ONNX

model.int4.bnb4.onnx.data

model.int4.onnx ONNX

model.int4.onnx.data

model.int4.onnx_data

model.int8.onnx ONNX

tokenizer/added_tokens.json

tokenizer/merges.txt

tokenizer/special_tokens_map.json

tokenizer/tokenizer.json

tokenizer/tokenizer_config.json

tokenizer/vocab.json

Related models

More models

No task type declared.