Zeolit/lettuce-emb-512d-v2

Lettuce-512D-v2 (Mobile Optimized)

This is a mobile-first sentence-transformers model designed for Memory Systems and Roleplay RAG applications.

It maps sentences & paragraphs to a 512-dimensional dense vector space. Unlike standard models, this model has been surgically altered to support 4096 tokens of context and fine-tuned specifically for narrative flow and character interactions.

Model Details

Key Specifications

• Model Type: Sentence Transformer (Distilled & Fine-tuned)
• Base Architecture: all-MiniLM-L6-v2 (BERT) with custom surgery.
• Maximum Sequence Length: 4096 tokens (Expanded from 512).
• Output Dimensionality: 512 (Projected via Dense Layer).
• Size: ~23 MB (Int8 Quantized ONNX).
• Latency: ~1.3ms on CPU (Short text).
• Similarity Function: Cosine Similarity.

Why this model?

Standard mobile models (like MiniLM) fail at two things: Context (limited to 512 tokens) and Narrative Logic (they prefer keyword matching over storytelling vibe).

Lettuce-v2 solves this via a 4-step engineering process:

1. Surgery: Expanded absolute position embeddings to 4096 using Copy-Repeat initialization.
2. Projection: Added a trainable Dense Layer to project 384d -> 512d.
3. Distillation: Knowledge distilled from BAAI/bge-m3 (State-of-the-Art) to retain general semantic logic.
4. Fine-Tuning: Trained on the Augmental (Steins;Gate) dataset mixed with NLI data to prioritize Narrative/Roleplay semantic alignment.

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 4096, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, ...})
  (2): Dense({'in_features': 384, 'out_features': 512, 'bias': True, 'activation_function': 'torch.nn.modules.linear.Identity'})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("Zeolit/lettuce-emb-512d-v2")

# Run inference on Narrative/RP text
sentences = [
    '"You raised a flag."',
    '*I take a deep breath, my mind racing with possibilities...* "Leap to an even earlier time."',
    'The quick brown fox jumps over the lazy dog.'
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 512]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)

Training Details

Training Dataset

The model was trained on a hybrid dataset to balance Narrative Style with General Logic:

• Primary: Heralax/Augmental-Dataset (Visual Novel / Roleplay logs).
• Foundation: sentence-transformers/all-nli (Logic & Reasoning anchors).
• Total Size: 18,798 training samples.

Training Logs

The training shows a smooth convergence, stopping exactly at the point of diminishing returns (Epoch 2) to prevent overfitting.

Training Hyperparameters

• per_device_train_batch_size: 4 (Gradient Accumulation simulated larger batches)
• num_train_epochs: 2
• fp16: True
• loss: CosineSimilarityLoss

Citation

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}