keisuke-miyako/h2o-danube3-4b-chat-onnx-int4-NG

By keisuke-miyako · ↓ 0 · ♥ 0 · Updated 2026-01-06

What's in this repo

This page lists the files in keisuke-miyako/h2o-danube3-4b-chat-onnx-int4-NG, including model.onnx.

1 ONNX models (.onnx files)
11 files, including 4 .json, 1 .md, 1 .jinja, 1 .onnx, 1 .data, 1 .model
Last updated 2026-01-06

ONNX files

File	Size	Download
model.onnx	—	Download

How to use

These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.

README

⚠️ Do not use with onnxruntime-genai; model architecture has a non-standard head size of 120 which is not divisible by 16.

To save memory during export & quantisation

hf download h2oai/h2o-danube3-4b-chat --local-dir ~/.onnx/h2o-danube3-4b-chat

from optimum.onnxruntime import ORTModelForCausalLM
import torch

model_id = "~/.onnx/h2o-danube3-4b-chat"

print(f"Loading {model_id} in FP16 to save RAM...")

# 1. Load model in FP16 directly (Crucial for preventing crashes)
model = ORTModelForCausalLM.from_pretrained(
    model_id,
    export=True,
    torch_dtype=torch.float16,  # This tells PyTorch to use half the memory
    trust_remote_code=True
)

# 2. Save the ONNX model
print("Exporting to ONNX...")
model.save_pretrained("~/.onnx/h2o-danube3-4b-chat-onnx")
print("Done!")

python -m onnxruntime_genai.models.builder \
  -m ~/.onnx/h2o-danube3-4b-chat-onnx \
  -o ~/.onnx/h2o-danube3-4b-chat-onnx-int4-cpu \
  -p int4 \
  -e cpu

Model info

onnx base_model:h2oai/h2o-danube3-4b-chat base_model:quantized:h2oai/h2o-danube3-4b-chat region:us

↓ 0 ♥ 0

Created: 2026-01-05

Updated: 2026-01-06

View on Hugging Face

Files (11)

.DS_Store

.gitattributes

README.md

chat_template.jinja

genai_config.json

model.onnx ONNX

model.onnx.data

special_tokens_map.json

tokenizer.json

tokenizer.model

tokenizer_config.json

Related models

More models

No task type declared.