基于 BAAI/bge-base-en-v1.5 的 SentenceTransformer

这是一个基于 BAAI/bge-base-en-v1.5 微调的 sentence-transformers 模型。它将句子和段落映射到768维的稠密向量空间，可用于语义文本相似度、语义搜索、复述挖掘、文本分类、聚类等任务。

模型详情

模型描述

• 模型类型: Sentence Transformer
• 基础模型: BAAI/bge-base-en-v1.5 <!-- at revision a5beb1e3e68b9ab74eb54cfd186867f64f240e1a -->
• 最大序列长度: 512 tokens
• 输出维度: 768 维
• 相似度函数: 余弦相似度 <!-- - 训练数据集: 未知 --> <!-- - 语言: 未知 --> <!-- - 许可证: 未知 -->

模型来源

• 文档: Sentence Transformers 文档
• 代码库: Sentence Transformers GitHub
• Hugging Face: Sentence Transformers on Hugging Face

完整模型架构

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

使用方法

直接使用 (Sentence Transformers)

首先安装 Sentence Transformers 库：

pip install -U sentence-transformers

然后你可以加载此模型并运行推理。

from sentence_transformers import SentenceTransformer

# 从 🤗 Hub 下载
model = SentenceTransformer("sentence_transformers_model_id")
# 运行推理
sentences = [
    'Represent this sentence for searching relevant passages: When a unit is set up, what conditions must be met regarding coherency?',
    'Any time a unit is **set up** or **ends a move**, it must be in a single group. A unit is considered to be in a coherent group if each model in that unit is within **coherency range**, measured horizontally, of **at least 1 other model** in that unit (ignore differences in height between the two models).  \n\nFor the majority of units, **coherency range is 1/2"**, though some units (particularly those with large models with overhanging parts) have a longer coherency range noted on their warscroll for ease of play. While there are **7 or more** models in a unit, that unit is considered to be in a coherent group if each model in that unit is within coherency range of **at least 2 other models** in that unit.  \n\nIf it is not possible for a unit to end a move in a single coherent group, that move cannot be made.  \n\n\n*   After finishing a move, a unit must be in a single group.\n*   Coherency range is 1/2" horizontally.\n*   Each model must be within coherency range of a different model from the same unit.\n*   While a unit has 7+ models, each model must be in coherency with 2 other models in the unit.',
    'Armies are made up of one or more **regiments**, each of which is led by a Hero. You must have **at least 1** regiment in your army, and you can include a **maximum of 5** regiments. To add a regiment, pick 1 HERO from your faction, then pick up to **3** non‑HERO units to accompany them.\n\nEach HERO\'s battle profile lists which units can be added to their regiment, and each non-HERO unit\'s battle profile lists any relevant keywords it has. The battle profiles of some HEROES may say that they can be added to the regiment of another HERO in place of a non-HERO unit in that regiment.\n\nQ: If a HERO is able to join another HERO\'s regiment (e.g. The Shadow Queen joining Morathi-Khaine or an Assassin joining a Dreadlord on Black Dragon), do they take the place of a non-HERO unit in that regiment?_\nA: Yes.\n\nQ: Can I add units from other factions to my HERO\' regiments?_\nA: No. The only way to add units from other factions to your army is by taking an eligible Regiment of Renown.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# 获取嵌入的相似度分数
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.6561, 0.0181],
#         [0.6561, 1.0000, 0.1562],
#         [0.0181, 0.1562, 1.0000]])

<!--

直接使用 (Transformers)

<details><summary>点击查看 Transformers 中的直接用法</summary>

</details> -->

<!--

下游使用 (Sentence Transformers)

你可以在此模型上微调你自己的数据集。

<details><summary>点击展开</summary>

</details> -->

<!--

超出范围的使用

列出模型可能被预见滥用的方式，并说明用户不应使用该模型做什么。 -->

评估

指标

信息检索

• 数据集: warhammer-eval
• 使用 <code>InformationRetrievalEvaluator</code> 评估

<!--

偏见、风险和局限性

该模型存在哪些已知或可预见的问题？你也可以在此标记已知的失败案例或模型弱点。 -->

<!--

建议

针对可预见的问题有什么建议？例如，过滤显式内容。 -->

训练详情

训练数据集

未命名数据集

• 大小: 1,014 个训练样本
• 列: <code>sentence_0</code> 和 <code>sentence_1</code>
• 基于前1000个样本的近似统计:

  	sentence_0	sentence_1
  类型	字符串	字符串
  详情	<ul><li>最小值: 15 tokens</li><li>平均值: 25.73 tokens</li><li>最大值: 50 tokens</li></ul>	<ul><li>最小值: 40 tokens</li><li>平均值: 191.46 tokens</li><li>最大值: 408 tokens</li></ul>

• 样本:

  sentence_0	sentence_1
  <code>Represent this sentence for searching relevant passages: If a friendly unit has Strike-First, and a friendly unit without Strike-First can fight after, who fights first?</code>	<code>Q: Can I use an ability that allows a friendly unit that does not have STRIKE-FIRST to fight immediately after a friendly unit that has STRIKE-FIRST if there are one or more enemy units with STRIKE-FIRST that have not yet been picked to fight?_<br>A: No. As mentioned in the sidebar next to 19.0, abilities that allow a unit to use a FIGHT ability immediately after another unit do not override the STRIKE-FIRST constraints, so you cannot pick a unit that does not have STRIKE-FIRST to fight until all other units that have STRIKE-FIRST have fought.</code>
  <code>Represent this sentence for searching relevant passages: When do I check if I gain control of an objective?</code>	<code>* An objective marker is a 40mm round marker.<br>* A model contests an objective if the objective marker is within its combat range.<br>* A player gains control of an objective if the sum of the Control characteristics of friendly models contesting that objective is higher than that of enemy models.<br>* Check if you gain control of objectives at the start of the first battle round and at the end of each turn.<br>* An objective remains in your control until your opponent gains control of it.<br>* Terrain features are controlled in the same way as objective markers but do not remain in your control if no friendly models are contesting them.</code>
  <code>Represent this sentence for searching relevant passages: Is setting up faction terrain in the deployment phase optional?</code>	<code>Q: Is it mandatory for players to set up a faction terrain feature (if one is included on their roster) during the deployment phase?_<br>A: No. A player can choose not to use the 'Deploy Faction Terrain' ability. However, if both players choose to set up a faction terrain feature, the player who begins deployment must set up their faction terrain features first (as specified in Step 1 of 10.0).</code>

• 损失函数: <code>main.TrackingLoss</code>

训练超参数

非默认超参数

• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• num_train_epochs: 8
• multi_dataset_batch_sampler: round_robin

所有超参数

<details><summary>点击展开</summary>

• do_predict: False
• eval_strategy: no
• prediction_loss_only: True
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 5e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1
• num_train_epochs: 8
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: None
• warmup_ratio: None
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• enable_jit_checkpoint: False
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• use_cpu: False
• seed: 42
• data_seed: None
• bf16: False
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: -1
• ddp_backend: None
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch_fused
• optim_args: None
• group_by_length: False
• length_column_name: length
• project: huggingface
• trackio_space_id: trackio
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• hub_revision: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_for_metrics: []
• eval_do_concat_batches: True
• auto_find_batch_size: False
• full_determinism: False
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• include_num_input_tokens_seen: no
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• liger_kernel_config: None
• eval_use_gather_object: False
• average_tokens_across_devices: True
• use_cache: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: round_robin
• router_mapping: {}
• learning_rate_mapping: {}

</details>

训练日志

框架版本

• Python: 3.12.12
• Sentence Transformers: 5.2.2
• Transformers: 5.0.0
• PyTorch: 2.9.0+cu128
• Accelerate: 1.12.0
• Datasets: 4.0.0
• Tokenizers: 0.22.2

引用

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = \"Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks\",
    author = \"Reimers, Nils and Gurevych, Iryna\",
    booktitle = \"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing\",
    month = \"11\",
    year = \"2019\",
    publisher = \"Association for Computational Linguistics\",
    url = \"https://arxiv.org/abs/1908.10084\",
}

<!--

术语表

清晰地定义术语，以便让各领域的读者都能理解。 -->

<!--

模型卡作者

列出创建模型卡的人员，为模型卡构建中的详细工作提供认可和责任归属。 -->

<!--

模型卡联系方式

为想要更新模型卡、提出建议或有疑问的人提供联系模型卡作者的方式。 -->