说明文档

TalhaLLM

from transformers import AutoModelForCausalLM, AutoTokenizer
checkpoint = "HuggingFaceTB/SmolLM2-135M-Instruct"

device = "cuda" # 用于 GPU 运行，或使用 "cpu" 进行 CPU 运行
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
# 对于多 GPU 环境，请安装 accelerate 并使用 `model = AutoModelForCausalLM.from_pretrained(checkpoint, device_map="auto")`
model = AutoModelForCausalLM.from_pretrained(checkpoint).to(device)

messages = [{"role": "user", "content": "What is gravity?"}]
input_text=tokenizer.apply_chat_template(messages, tokenize=False)
print(input_text)
inputs = tokenizer.encode(input_text, return_tensors="pt").to(device)
outputs = model.generate(inputs, max_new_tokens=50, temperature=0.2, top_p=0.9, do_sample=True)
print(tokenizer.decode(outputs[0]))

Talha01/LLMModel

作者 Talha01

text-generation transformers

↓ 0 ♥ 0

创建时间: 2024-11-05 17:11:12+00:00

更新时间: 2024-11-05 17:20:35+00:00

在 Hugging Face 上查看

文件 (21)

.gitattributes

README.md

all_results.json

config.json

eval_results.json

generation_config.json

merges.txt

model.safetensors

onnx/model.onnx ONNX

onnx/model_fp16.onnx ONNX

onnx/model_q4.onnx ONNX

onnx/model_q4f16.onnx ONNX

runs/Oct31_10-14-22_ip-26-0-174-36/events.out.tfevents.1730370128.ip-26-0-174-36.3239327.0

runs/Oct31_10-14-22_ip-26-0-174-36/events.out.tfevents.1730376056.ip-26-0-174-36.3239327.1

special_tokens_map.json

tokenizer.json

tokenizer_config.json

train_results.json

trainer_state.json

training_args.bin

vocab.json

说明文档

TalhaLLM

目录

如何使用

Transformers

Talha01/LLMModel

文件 (21)