saravanan-arjuna-ravi/useless-dog-178K-onnx

useless-dog-178K-onnx

A tiny 3-layer transformer trained from scratch (zero pretrained weights, no HuggingFace model hub dependencies) that maps natural-language commands to 9 continuous parameters controlling a fully animated German Shepherd SVG dog. Inference runs entirely in the browser via ONNX Runtime Web — no Python needed after page load.

How It Works

Raw text
  → TextPreprocessor   (lowercase, expand contractions, synonyms, stop-word removal)
  → HybridTokenizer    (word-level lookup + WordPiece fallback, 1,155 tokens)
  → Transformer        (3-layer encoder, embed_dim=64, 4 heads)
  → RegressionHead     (pool → Linear → SiLU → Linear → Tanh → rescale)
  → 9 float params
  → SVG German Shepherd animates

Inference Pipeline Detail

1. TextPreprocessor — lowercases, expands contractions ("don't" → "do not"), strips punctuation, applies a domain synonym map ("furious" → "angry", "doggo" → "dog"), removes stop words while preserving emotion-bearing terms ("not", "very", "never", etc.)

2. HybridTokenizer — checks a word-level vocab first (fast path for known words); falls back to greedy longest-match WordPiece for unknowns. Produces a fixed-length sequence: [START] <tokens> [SEP] [END] [PAD…] (max 32 tokens).

3. Transformer Encoder — 3 layers of multi-head self-attention (4 heads) + feedforward (SiLU activation, ff_dim=128). Learned positional embeddings. Previous dog state (9 floats) is prepended as an extra token so the model is stateful — each prediction is conditioned on the current dog pose.

4. RegressionHead — global average pool → Linear(64→32) → SiLU → Dropout → Linear(32→9) → Tanh → rescale [0,1] params from (-1,1) range.

Model Output — 9 Parameters

Neutral state: [0.0, 0.0, 0.5, 0.5, 0.0, 0.3, 0.0, 0.0, 0.0]

Architecture

DogEmotionModel
├── DogModelEmbeddings
│   ├── Token embedding       (1155 × 64)
│   ├── Learned positional    (32 × 64)
│   └── State projection      Linear(9 → 64)   ← previous dog state as token
├── MiniTransformerEncoder    (3 × TransformerEncoderLayer)
│   ├── MultiHeadAttention    (4 heads, embed_dim=64)
│   ├── FeedForward           (64 → 128 → 64, SiLU)
│   ├── LayerNorm × 2
│   └── Dropout (0.1)
└── RegressionHead
    ├── Global average pool
    ├── Linear(64 → 32) + SiLU + Dropout
    ├── Linear(32 → 9)  + Tanh
    └── Rescale [0,1] params: (x + 1) / 2

Training

Loss Curve

Training converged cleanly — val loss tracks train loss with no divergence, indicating the 178K parameter model is appropriately sized for the 20K synthetic dataset.

Tokenizer

A custom HybridTokenizer combining two strategies:

• Word-level tokenizer — fixed domain vocabulary of common emotion/action words. Fast O(1) lookup. Covers ~95% of inputs.
• WordPiece fallback — trained from scratch on the domain corpus. Greedy longest-match subword segmentation for out-of-vocabulary words (e.g. "wagging" → ["wag", "##g", "##ing"]).

Both vocabs are merged into a single conflict-free vocab.json (1,155 tokens). WordPiece token IDs are offset by the word vocab size to prevent collisions.

Special tokens: [PAD]=0 [UNK]=1 [START]=2 [END]=3 [SEP]=4

Browser Inference (ONNX Runtime Web)

The frontend (static/index.html) runs the full inference pipeline client-side using ONNX Runtime Web. No server round-trips after initial page load.

Page load
  → fetch vocab.json + contractions.json + synonym_map.json  (~50 KB total)
  → fetch model.onnx                                          (0.88 MB)
  → ort.InferenceSession.create()                             (WASM compile)
  → input unlocks, ready

Use the Demo Frontend Yourself

The static/index.html is a self-contained frontend. To run it with any HTTP server:

# Clone the repo
git clone https://huggingface.co/spaces/your-username/useless-dog-178K-onnx
cd useless-dog-178K-onnx

# Install and run
pip install flask
python server.py
# Open http://localhost:7860

The frontend features:

• Full-page nature scene (SVG sky, clouds, mountains, pine trees, animated grass)
• German Shepherd SVG with 9 independently animated regions
• Tail wagging via requestAnimationFrame sinusoidal loop
• Glass-morphism HUD with real-time parameter bars (3×3 grid)
• Floating emotion badge (HAPPY / EXCITED / ANGRY / SAD / SLEEPY / CURIOUS etc.)

ONNX Inference (Python)

import json, re, numpy as np
import onnxruntime as ort

# Load assets
with open("hf_export/tokenizer/vocab.json")            as f: vocab = json.load(f)
with open("hf_export/tokenizer/tokenizer_config.json") as f: cfg  = json.load(f)
with open("hf_export/preprocessor/contractions.json")  as f: contractions = json.load(f)
with open("hf_export/preprocessor/synonym_map.json")   as f:
    raw = json.load(f)
    synonym_map = {}
    for cat in raw.values(): synonym_map.update(cat)

MAX_SEQ_LEN = cfg["max_seq_len"]   # 32
PAD, START, END, SEP, UNK = (vocab[t] for t in ["[PAD]","[START]","[END]","[SEP]","[UNK]"])

STOP  = {"the","a","an","please","can","you","i","want","to","my","your","is","are",
          "was","be","have","has","do","will","would","of","in","on","at","for",
          "with","by","as","it","he","she","they","we"}
KEEP  = {"not","more","less","very","slightly","now","also","but","and","too","no","never","stop"}

def preprocess(text):
    text = text.lower()
    for c, e in contractions.items(): text = text.split(c).join(e) if c in text else text
    text = re.sub(r"[^a-z0-9\s]", " ", text).strip()
    words = [synonym_map.get(w, w) for w in text.split()]
    return [w for w in words if w in KEEP or w not in STOP]

def encode(words):
    ids  = [vocab.get(w, UNK) for w in words][: MAX_SEQ_LEN - 3]
    seq  = [START] + ids + [SEP, END]
    pad  = MAX_SEQ_LEN - len(seq)
    return seq + [PAD]*pad, [1]*len(seq) + [0]*pad

sess    = ort.InferenceSession("hf_export/model.onnx", providers=["CPUExecutionProvider"])
NEUTRAL = [0.0, 0.0, 0.5, 0.5, 0.0, 0.3, 0.0, 0.0, 0.0]
state   = NEUTRAL[:]
NAMES   = ["mouth","eyebrow","eyelid","pupil","ear","tail","tongue","blush","tilt"]

def predict(text):
    global state
    words       = preprocess(text) or ["neutral"]
    ids, mask   = encode(words)
    out         = sess.run(["params"], {
        "input_ids":      np.array([ids],   dtype=np.int64),
        "attention_mask": np.array([mask],  dtype=np.int64),
        "previous_state": np.array([state], dtype=np.float32),
    })
    state = out[0][0].tolist()
    return dict(zip(NAMES, state))

print(predict("make the dog happy and excited"))
# {'mouth': 0.72, 'eyebrow': 0.18, 'eyelid': 0.12, 'pupil': 0.81,
#  'ear': -0.43, 'tail': 0.91, 'tongue': 0.67, 'blush': 0.44, 'tilt': 0.09}

Re-export ONNX

pip install -r requirements.txt
python export_to_hf.py --verify
# Outputs: hf_export/model.onnx  (single self-contained file, no .data sidecar)

Project Structure

model/                  Transformer architecture (built from scratch)
  dog_model.py          Full model: embeddings + transformer + head
  transformer.py        MiniTransformerEncoder (3-layer)
  embeddings.py         Token + positional + state embeddings
  regression_head.py    Pool → MLP → Tanh → rescale

tokenizer/              Custom tokenizer (no HuggingFace tokenizers library)
  word_tokenizer.py     Word-level vocab lookup
  wordpiece_tokenizer.py WordPiece trained from scratch
  hybrid_tokenizer.py   Combines both into unified vocab
  vocab.json            Unified 1,155-token vocab

preprocessor/
  preprocessor.py       Text normalization pipeline
  contractions.json     Contraction expansion map
  synonym_map.json      Domain synonym normalization

training/
  trainer.py            Training loop (AdamW, cosine LR, weighted MSE, early stop)
  config.py             All hyperparameters

data/
  dataset_generator.py  Synthetic dataset generator (15 categories, 20K samples)
  generated/            train.json / val.json / test.json

hf_export/              Browser-ready package
  model.onnx            Self-contained ONNX model (0.88 MB)
  config.json           Architecture + param metadata
  tokenizer/            vocab.json + tokenizer_config.json
  preprocessor/         contractions.json + synonym_map.json

static/
  index.html            Full-page frontend (nature scene + SVG dog + HUD)

checkpoints/
  best_model.pt         PyTorch checkpoint (epoch 89)
  training_log.json     Full train/val loss history

server.py               Flask server (static file serving only)
export_to_hf.py         ONNX export + HF packaging script
requirements.txt        Python dependencies
Dockerfile              HuggingFace Docker Space config

Tech Choices

Citation

If you use this model in your research or project, please cite:

@misc{saravanan2025uselessdog,
  author    = {Saravanan Arjuna Ravi},
  title     = {useless-dog-178K-onnx: Natural Language to Animated German Shepherd via Tiny Transformer},
  year      = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/spaces/saravanan-arjuna-ravi/useless-dog-178K-onnx}}
}

License

Creative Commons Attribution 4.0 International (CC BY 4.0)

You are free to share and adapt this work for any purpose, including commercially, as long as you give appropriate credit.