Felladrin/onnx-TinyMistral-248M-SFT-v4

INT8 ONNX version of Felladrin/TinyMistral-248M-SFT-v4 to use with Transformers.js.

Usage

import { pipeline } from '@xenova/transformers';

const prompt = `<|im_start|>user
Invited some friends to come home today. Give me some ideas for games to play with them!<|im_end|>
<|im_start|>assistant`;
const generator = await pipeline('text-generation', 'Felladrin/onnx-TinyMistral-248M-SFT-v4');
const output = await generator(prompt, { add_special_tokens: true, max_new_tokens: 128, do_sample: true, penalty_alpha: 0.6, top_k: 4, repetition_penalty: 1.07 });
console.log(output[0].generated_text);
// Sure! Here are a few suggestions for activities that could be added to a game night: 
// 1. Play board games and puzzle games 
// 2. Watch the movie trailer or gameplay videos 
// 3. Play a board game tournament or tournament with friends 
// 4. Watch movies on video or TV shows like Netflix, Hulu, and IFC.<|im_end|>