Irfanuruchi/SmolLM2-135M-Instruct-MLX-4bit

By Irfanuruchi · ↓ 7 · ♥ 0 · Updated 2026-01-24

What's in this repo

This page lists the files in Irfanuruchi/SmolLM2-135M-Instruct-MLX-4bit.

No .onnx files in this repository
9 files, including 5 .json, 1 .md, 1 .jinja, 1 .safetensors
Last updated 2026-01-24

ONNX files

This repository has no .onnx files. See the full file list on the right for other formats.

How to use

These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.

README

SmolLM2-135M Instruct (MLX, 4-bit)

This is an MLX conversion of HuggingFaceTB/SmolLM2-135M-Instruct quantized to 4-bit for fast on-device inference on Apple Silicon.

Quickstart

Install:

pip install -U mlx-lm

Run:

mlx_lm.generate \
  --model Irfanuruchi/SmolLM2-135M-Instruct-MLX-4bit \
  --prompt "Reply with exactly 3 bullet points, 4–8 words each: what can you do offline?" \
  --max-tokens 80

Benchmarks (MacBook Pro M3 Pro)

Disk: 76 MB
Peak RAM: 0.106 GB

Performance will vary across devices and prompts.

Notes

Converted/quantized with mlx_lm.convert.
This repo contains MLX weights and tokenizer/config files.

License & attribution

Upstream model: HuggingFaceTB/SmolLM2-135M-Instruct (Apache-2.0).
Please follow the upstream license and attribution requirements.

Model info

text-generation mlx

mlx safetensors llama text-generation onnx transformers.js apple-silicon quantized smollm2 conversational en base_model:HuggingFaceTB/SmolLM2-135M-Instruct

↓ 7 ♥ 0

Created: 2026-01-24

Updated: 2026-01-24

View on Hugging Face

Files (9)

.gitattributes

README.md

chat_template.jinja

config.json

generation_config.json

model.safetensors

model.safetensors.index.json

tokenizer.json

tokenizer_config.json

Related models

More text-generation models

Browse all text-generation models