scasella91/timecapsulellm-pr1
By scasella91 · ↓ 0 · ♥ 0 · Updated 2026-01-12
What's in this repo
This page lists the files in scasella91/timecapsulellm-pr1, including model.onnx.
- 1 ONNX models (.onnx files)
- 18 files, including 10 .json, 1 .md, 1 .safetensors, 1 .onnx, 1 .onnx_data, 1 .bin
- Last updated 2026-01-12
ONNX files
| File | Size | Download |
|---|---|---|
| onnx/model.onnx | — | Download |
How to use
These ONNX files can be loaded with ONNX Runtime. To inspect a model graph, upload the file to the Netron viewer. Download links point to the original files on Hugging Face.
README
haykgrigorian/TimeCapsuleLLM-v2-London-1800-1875: Llama-Architecture 1.2B Model
Model Overview
v2 model, trained from scratch on 112GB of 1800-1875 london texts using a Llama-based Casual Language Model.
| Detail | Value |
|---|---|
| Model Architecture | LlamaForCausalLM (Decoder-Only Transformer) |
| Parameter Count | ~1.22B |
| Training Type | Trained from Scratch (Random Initialization) |
| Tokenizer | Custom BPE, Vocab Size 32,000 |
| Sequence Length | 2048 tokens |
| Attention Type | Grouped Query Attention (GQA) |
Configuration Details
This model is a custom size and configuration based on Llama:
| Parameter | Value |
|---|---|
| Number of Layers | 22 |
| Hidden Size (d) | 2048 |
| Intermediate Size ($\text{d}_{\text{ff}}$) | 5504 |
| Attention Heads | 16 (Query) / 8 (Key/Value) |
| Activation Function | SiLU (silu) |
| Normalization | RMS Norm (rms_norm_eps: 1e-06) |
| Position Embeddings | Rotary Positional Embeddings (RoPE) |
Training Info
This model was trained for 182,000 steps, about 0.5 epochs.
Training Metrics:
Final Training Loss: 3.3951
Start Training Loss: 10.7932
Training Steps: 182,000
Epochs: 0.4997
Gradient Norm Stability: Consistently stable between 0.50 and 0.60 in later stages.
Training time: 117 hours 51 minutes
Cost
This model was trained on an H100 SXM from RunPod
Total: $340.97
How to Load and Run the Model
Install all the files locally in a folder and run the test script. You will have to make some adjustments in the run script like updating the config/file path and test prompts
Test script
A run file for testing and evaluating this model is available on the main project repository:
- Test Script Link: run_v2.py on GitHub
Model info
Created: 2026-01-12
Updated: 2026-01-12
View on Hugging FaceFiles (18)
Related models
More from scasella91
No other models by this author.