Edison2ST/Qwen3-1.7B-Q4-ONNX

By Edison2ST · ↓ 0 · ♥ 0 · Updated 2026-01-29

What's in this repo

This page lists the files in Edison2ST/Qwen3-1.7B-Q4-ONNX, including model.onnx.

1 ONNX models (.onnx files)
13 files, including 4 .json, 3 .py, 1 .md, 1 .onnx, 1 .bin
Last updated 2026-01-29

ONNX files

File	Size	Download
model.onnx	—	Download

How to use

These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.

README

language:

en
zh
multilingual tags:
llm
qwen
onnx
onnxruntime
quantization
4-bit
ghost-assistant
android-ai
edge-computing license: apache-2.0 base_model: Qwen/Qwen3-1.7B model_creator: Edison dos Santos

Qwen3-1.7B-Q4 - Optimized 4-bit ONNX for Mobile

This repository contains a highly optimized 4-bit quantized version of Qwen3-1.7B, specifically tailored for local inference on mid-range Android hardware.

🚀 Mobile Optimization Highlights

Unlike standard exports, this version was built to maximize the throughput of the XNNPACK execution provider on ARM-based chipsets (like the Dimensity 6300).

Memory Mapping (mmap): Configured for zero-copy loading using MappedByteBuffer, allowing a 1.31GB model to run on devices with limited RAM without triggering the Android OOM (Out of Memory) killer.
Tensor Alignment: Optimized for block size 32 to ensure efficient cache line usage on Cortex-A76/A55 architectures.

📊 Performance Benchmark (MediaTek Dimensity 6300)

Metric	Performance
Quantization	4-bit Integer (Q4)
Inference Engine	ONNX Runtime (GenAI API)
Avg. Speed	~10.89 tokens/sec (MediaTek Dimensity 6300). Surpassing Google's Edge Gallery on same hardware according to the Technical Report.
RAM Footprint	Stable ~1.31 GB

🛠 Usage example

Run scripts/inference_example.py. Check the Ghost Assistant's Technical Report for more info and benchmarks.

🛠 Creating this ONNX File

Made it easier for you with a customized JSON file. Run the following command: olive run --config qwen_config.json with the model generated on the Technical Report.

Model info

onnx llm qwen onnxruntime quantization 4-bit ghost-assistant android-ai edge-computing en zh multilingual

↓ 0 ♥ 0

Created: 2026-01-25

Updated: 2026-01-29

View on Hugging Face

Files (13)

.gitattributes

LICENSE

NOTICE

README.md

genai_config.json

model.onnx ONNX

model.weights.bin

scripts/check_alignment_tensors.py

scripts/inference_example.py

scripts/qwen_config.json

scripts/tensor_alignment_fix.py

tokenizer.json

tokenizer_config.json

Related models

More models

No task type declared.