inference4j/phi-3-mini-4k-instruct
By inference4j · ↓ 12 · ♥ 0 · Updated 2026-02-16
What's in this repo
This page lists the files in inference4j/phi-3-mini-4k-instruct, including phi3-mini-4k-instruct-cpu-int4-rtn-block-32-acc-level-4.onnx.
- 1 ONNX models (.onnx files)
- 11 files, including 6 .json, 1 .md, 1 .onnx, 1 .data, 1 .model
- Last updated 2026-02-16
ONNX files
| File | Size | Download |
|---|---|---|
| phi3-mini-4k-instruct-cpu-int4-rtn-block-32-acc-level-4.onnx | — | Download |
How to use
These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.
README
Phi-3-mini-4k-instruct — ONNX (INT4)
INT4-quantized ONNX export of Phi-3-mini-4k-instruct, a 3.8B-parameter lightweight language model from Microsoft. Optimized for CPU inference with int4 RTN block-32 quantization.
Mirrored for use with inference4j, an inference-only AI library for Java.
Original Source
- Repository: Microsoft
- License: mit
Usage with inference4j
try (TextGenerator gen = TextGenerator.builder().build()) {
GenerationResult result = gen.generate("What is Java in one sentence?");
System.out.println(result.text());
}
Model Details
| Property | Value |
|---|---|
| Architecture | Phi-3 (3.8B parameters, 32 layers, 3072 hidden) |
| Task | Text generation / chat |
| Context length | 4096 tokens |
| Quantization | INT4 RTN block-32 acc-level-4 |
| Original framework | PyTorch (transformers) |
License
This model is licensed under the MIT License. Original model by Microsoft.
Model info
Created: 2026-02-16
Updated: 2026-02-16
View on Hugging Face