基于 intfloat/multilingual-e5-large-instruct 的 SentenceTransformer

这是一个基于 intfloat/multilingual-e5-large-instruct 微调的 sentence-transformers 模型,在 d4-embeddings-multiple_negative 数据集上训练。它将句子和段落映射到1024维的稠密向量空间,可用于语义文本相似度、语义搜索、复述挖掘、文本分类、聚类等任务。

模型详情

模型描述

• 模型类型: Sentence Transformer
• 基础模型: intfloat/multilingual-e5-large-instruct <!-- at revision 274baa43b0e13e37fafa6428dbc7938e62e5c439 -->
• 最大序列长度: 512 个词元
• 输出维度: 1024 维
• 相似度函数: 余弦相似度
• 训练数据集:
  • d4-embeddings-multiple_negative <!-- - 语言: 未知 --> <!-- - 许可证: 未知 -->

模型来源

• 文档: Sentence Transformers 文档
• 代码库: Sentence Transformers GitHub
• Hugging Face: Sentence Transformers on Hugging Face

完整模型架构

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: XLMRobertaModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

使用方法

直接使用 (Sentence Transformers)

首先安装 Sentence Transformers 库:

pip install -U sentence-transformers

然后您可以加载此模型并运行推理。

from sentence_transformers import SentenceTransformer

# 从 🤗 Hub 下载
model = SentenceTransformer("Lauther/d4-embeddings-v1.0")
# 运行推理
sentences = [
    'ECOMP-GUARAREMA - BY-PASS',
    'What is a flow computer?\nA flow computer is a device used in measurement engineering. It collects analog and digital data from flow meters and other sensors.\n\nKey features of a flow computer:\n- It has a unique name, firmware version, and manufacturer information.\n- It is designed to record and process data such as temperature, pressure, and fluid volume (for gases or oils).',
    'What is a Fluid?\nA Fluid is the substance measured within a measurement system. It can be a gas or liquid, such as hydrocarbons, water, or other industrial fluids. Proper classification of fluids is essential for ensuring measurement accuracy, regulatory compliance, and operational efficiency. By identifying fluids correctly, the system applies the appropriate measurement techniques, processing methods, and reporting standards.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# 获取嵌入的相似度分数
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

<!--

直接使用 (Transformers)

<details><summary>点击查看 Transformers 中的直接使用方法</summary>

</details> -->

<!--

下游使用 (Sentence Transformers)

您可以在自己的数据集上微调此模型。

<details><summary>点击展开</summary>

</details> -->

<!--

超出范围的使用

列出模型可能被预见到的误用方式,并说明用户不应该用模型做什么。 -->

<!--

偏见、风险和局限性

这个模型有哪些已知或可预见的问题?您也可以在此标记已知的失败案例或模型的弱点。 -->

<!--

建议

针对可预见的问题有什么建议?例如,过滤显式内容。 -->

训练详情

训练数据集

d4-embeddings-multiple_negative

• 数据集: d4-embeddings-multiple_negative at 299ee20
• 大小: 3,632 个训练样本
• 列: <code>anchor</code> 和 <code>positive</code>
• 基于前1000个样本的近似统计:

  	anchor	positive
  type	string	string
  details	<ul><li>min: 3 tokens</li><li>mean: 8.91 tokens</li><li>max: 19 tokens</li></ul>	<ul><li>min: 80 tokens</li><li>mean: 208.8 tokens</li><li>max: 406 tokens</li></ul>

• 样本:

  anchor	positive
  <code>differential pressure</code>	<code>What is an Uncertainty Curve Point?<br>An Uncertainty Curve Point represents a data point used to construct the uncertainty curve of a measurement system. These curves help analyze how measurement uncertainty behaves under different flow rate conditions, ensuring accuracy and reliability in uncertainty assessments.<br><br>Key Aspects of an Uncertainty Curve Point:<br>- Uncertainty File ID: Links the point to the specific uncertainty dataset, ensuring traceability.<br>Equipment Tag ID: Identifies the equipment associated with the uncertainty measurement, crucial for system validation.<br>- Uncertainty Points: Represent a list uncertainty values recorded at specific conditions, forming part of the overall uncertainty curve. Do not confuse this uncertainty points with the calculated uncertainty. <br>- Flow Rate Points: Corresponding flow rate values at which the uncertainty was measured, essential for evaluating performance under varying operational conditions.<br>These points are fundamental for generating unce...</code>
  <code>FQI-4300.44-101</code>	<code>What is a Measurement Type?<br>Measurement types define the classification of measurements used within a system based on their purpose and regulatory requirements. These types include fiscal, appropriation, operational, and custody measurements. <br><br>- Fiscal measurements are used for tax and regulatory reporting, ensuring accurate financial transactions based on measured quantities. <br>- Appropriation measurements track resource allocation and ownership distribution among stakeholders. <br>- Operational measurements support real-time monitoring and process optimization within industrial operations. <br>- Custody measurements are essential for legal and contractual transactions, ensuring precise handover of fluids between parties. <br><br>These classifications play a crucial role in compliance, financial accuracy, and operational efficiency across industries such as oil and gas, water management, and energy distribution. </code>
  <code>FQI-4300-44116</code>	<code>What is a Meter Stream?<br>A Meter Stream represents a measurement system configured within a flow computer. It serves as the interface between the physical measurement system and the computational processes that record and analyze flow data.<br><br>Key Aspects of a Meter Stream:<br>- Status: Indicates whether the meter stream is active or inactive.<br>- Measurement System Association: Links the meter stream to a specific measurement system, ensuring that the data collected corresponds to a defined physical setup.<br>- Flow Computer Association: Identifies the flow computer responsible for managing and recording the measurement system's data.<br>Why is a Meter Stream Important?<br>A meter stream is a critical component in flow measurement, as it ensures that the measurement system is correctly integrated into the flow computer for accurate monitoring and reporting. Since each flow computer can handle multiple meter streams, proper configuration is essential for maintaining data integrity and traceability.</code>

• 损失函数: <code>MultipleNegativesRankingLoss</code> 参数如下:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

评估数据集

d4-embeddings-multiple_negative

• 数据集: d4-embeddings-multiple_negative at 299ee20
• 大小: 778 个评估样本
• 列: <code>anchor</code> 和 <code>positive</code>
• 基于前778个样本的近似统计:

  	anchor	positive
  type	string	string
  details	<ul><li>min: 3 tokens</li><li>mean: 8.92 tokens</li><li>max: 19 tokens</li></ul>	<ul><li>min: 27 tokens</li><li>mean: 214.51 tokens</li><li>max: 406 tokens</li></ul>

• 样本:

  anchor	positive
  <code>FQI-4150.6122A1</code>	<code>What is an Uncertainty Curve Point?<br>An Uncertainty Curve Point represents a data point used to construct the uncertainty curve of a measurement system. These curves help analyze how measurement uncertainty behaves under different flow rate conditions, ensuring accuracy and reliability in uncertainty assessments.<br><br>Key Aspects of an Uncertainty Curve Point:<br>- Uncertainty File ID: Links the point to the specific uncertainty dataset, ensuring traceability.<br>Equipment Tag ID: Identifies the equipment associated with the uncertainty measurement, crucial for system validation.<br>- Uncertainty Points: Represent a list uncertainty values recorded at specific conditions, forming part of the overall uncertainty curve. Do not confuse this uncertainty points with the calculated uncertainty. <br>- Flow Rate Points: Corresponding flow rate values at which the uncertainty was measured, essential for evaluating performance under varying operational conditions.<br>These points are fundamental for generating unce...</code>
  <code>FQI-4150.63122C</code>	<code>What is uncertainty?<br>Uncertainty is a measure of confidence in the precision and reliability of results obtained from equipment or measurement systems. It quantifies the potential error or margin of error in measurements.<br><br>Types of uncertainty:<br>There are two main types of uncertainty:<br>1. Uncertainty of magnitudes (variables):<br> - Refers to the uncertainty of specific variables, such as temperature or pressure.<br> - It is calculated after calibrating a device or obtained from the equipment manufacturer's manual.<br> - This uncertainty serves as a starting point for further calculations related to the equipment.<br><br>2. Uncertainty of the measurement system:<br> - Refers to the uncertainty calculated for the overall flow measurement.<br> - It depends on the uncertainties of the individual variables (magnitudes) and represents the combined margin of error for the entire system.<br><br>Key points:<br>- The uncertainties of magnitudes (variables) are the foundation for calculating the uncertainty of ...</code>
  <code>Pressão Absoluta</code>	<code>What is uncertainty?<br>Uncertainty is a measure of confidence in the precision and reliability of results obtained from equipment or measurement systems. It quantifies the potential error or margin of error in measurements.<br><br>Types of uncertainty:<br>There are two main types of uncertainty:<br>1. Uncertainty of magnitudes (variables):<br> - Refers to the uncertainty of specific variables, such as temperature or pressure.<br> - It is calculated after calibrating a device or obtained from the equipment manufacturer's manual.<br> - This uncertainty serves as a starting point for further calculations related to the equipment.<br><br>2. Uncertainty of the measurement system:<br> - Refers to the uncertainty calculated for the overall flow measurement.<br> - It depends on the uncertainties of the individual variables (magnitudes) and represents the combined margin of error for the entire system.<br><br>Key points:<br>- The uncertainties of magnitudes (variables) are the foundation for calculating the uncertainty...</code>

• 损失函数: <code>MultipleNegativesRankingLoss</code> 参数如下:

  {
      "scale": 20.0,
      "similarity_fct": "cos_sim"
  }
  

训练超参数

非默认超参数

• eval_strategy: steps
• per_device_train_batch_size: 100
• per_device_eval_batch_size: 100
• learning_rate: 2e-05
• weight_decay: 0.01
• max_grad_norm: 0.5
• num_train_epochs: 10
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• fp16: True
• dataloader_num_workers: 4

所有超参数

<details><summary>点击展开</summary>

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 100
• per_device_eval_batch_size: 100
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.01
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 0.5
• num_train_epochs: 10
• max_steps: -1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: True
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 4
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non-blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional

</details>

训练日志

框架版本

• Python: 3.11.0
• Sentence Transformers: 3.4.1
• Transformers: 4.49.0
• PyTorch: 2.6.0+cu124
• Accelerate: 1.4.0
• Datasets: 3.3.2
• Tokenizers: 0.21.0

引用

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

<!--

术语表

明确定义术语,以便让不同受众都能理解。 -->

<!--

模型卡片作者

列出创建模型卡片的人员,为模型卡片构建中的详细工作提供认可和责任归属。 -->

<!--

模型卡片联系方式

为有模型卡片更新、建议或问题的人提供联系方式,以便与模型卡片作者联系。 -->