mehmetkeremturkcan/Loom

Loom: A Scalable Computer Architecture for Looped Transformers

Loom is a general-purpose computer implemented as a looped transformer with analytically derived weights. Programs are written in C, compiled to a 21-opcode ISA, and executed as iterated matrix multiplications through 8 fixed-weight transformer layers.

Model Description

Loom is not a trained model. Every weight is derived analytically from the ISA specification. The transformer implements a programmable computer: each forward pass executes one compiled instruction, and the model is applied in a loop until the program halts.

The architecture supports 21 opcodes (arithmetic, logic, shifts, comparisons, branches, indirect memory access, conditional moves, and multiply-accumulate) in 8 transformer layers, down from the 10 layers required for the single-instruction baseline of Giannou et al.

Key Design Choices

• Argmax attention replaces softmax for numerically exact execution over arbitrary step counts.
• Opcode-as-operand-routing maps all 21 operations to operand preparation for a shared subtract core, requiring only one arithmetic layer.
• 6-threshold direct subtraction computes a-b in one layer (replacing the classical 3-layer approach).
• STORE opcode enables indirect memory writes, reducing the Sudoku solver from 1,085 to 284 instructions.

Configurations

ONNX Models

The repository includes pre-exported ONNX models with argmax attention expressed as GPU-native operations (ReduceMax, comparison, division). No TopK or OneHot operators. Full WebGPU acceleration.

• argmax_146x512.onnx (8.3 MB): compact config
• argmax_155x1024.onnx (14.8 MB): standard config
• argmax_164x2048.onnx (28.0 MB): large config

Input/Output

• Input: state tensor of shape [d_model, n], dtype float32
• Output: new_state tensor of shape [d_model, n], dtype float32

Each call executes one ISA instruction. Loop until PC (program counter) reaches 0.

Usage

Python

from loom_v1 import LoomConfig, LoomComputer, init_state, read_memory, get_pc
from c_compiler import compile_c
import torch

source = """
int main() {
    int a; int b; int t; int i;
    a = 0; b = 1; i = 0;
    while (i < 10) { t = a + b; a = b; b = t; i = i + 1; }
    return a;
}
"""

cfg, mem, cmds, meta = compile_c(source, s=32, m=160, n=512, N=8)
comp = LoomComputer(cfg)
X = init_state(cfg, mem, cmds)

with torch.no_grad():
    while get_pc(X, cfg) != 0:
        X = comp.step(X)

from subleq import signed_from_bipolar
a = signed_from_bipolar(X[cfg.idx_memory:cfg.idx_memory+cfg.N, cfg.s + meta['variables']['a']])
print(f"fib(10) = {a}")  # 55

ONNX Runtime (Browser)

const session = await ort.InferenceSession.create('argmax_146x512.onnx', {
    executionProviders: ['webgpu']
});
const input = new ort.Tensor('float32', stateData, [146, 512]);
const result = await session.run({ state: input });
const newState = result.new_state.data;

Validation

• 42 opcode unit tests (all pass)
• 19 SWAP integration tests (all pass)
• 50 compiled C program tests including Fibonacci, GCD, sorting, LOAD/STORE round-trips (all pass)
• FPGA hardware verification on Xilinx Alveo U200 (INC test pass)

Browser Demos

Interactive demos run entirely client-side via ONNX Runtime Web:

• Sorting with real-time architecture visualization (3D layer activations)
• C debugger with source highlighting and variable watch
• Playable Snake game (84 transformer steps per tick)
• 9x9 Sudoku solver
• DOOM raycasting

Technical Report

See technical_report.tex for the full paper including the 8-layer pipeline, L4 subtraction proof, STORE implementation, and FPGA results.

Limitations

• 8-bit signed integers (-128 to 127) by default
• One instruction per forward pass (no pipelining)
• No multiplication/division in hardware (software emulation via MULACC)
• FIND requires unique values in the search array
• LOAD/STORE require valid in-range pointers

Citation

@article{turkcan2026loom,
  title={Loom: A Scalable Computer Architecture for Looped Transformers},
  author={Turkcan, Mehmet Kerem},
  year={2026}
}