基于 intfloat/multilingual-e5-large-instruct 的 SentenceTransformer

这是一个基于 intfloat/multilingual-e5-large-instruct 在 measuring-embeddings-v4 数据集上微调的 sentence-transformers 模型。它将句子和段落映射到 1024 维的稠密向量空间,可用于语义文本相似度、语义搜索、复述挖掘、文本分类、聚类等任务。

模型详情

模型描述

• 模型类型: Sentence Transformer
• 基础模型: intfloat/multilingual-e5-large-instruct <!-- at revision 84344a23ee1820ac951bc365f1e91d094a911763 -->
• 最大序列长度: 512 tokens
• 输出维度: 1024 维
• 相似度函数: 余弦相似度
• 训练数据集:
  • measuring-embeddings-v4 <!-- - 语言: 未知 --> <!-- - 许可证: 未知 -->

模型来源

• 文档: Sentence Transformers 文档
• 代码库: Sentence Transformers GitHub
• Hugging Face: Hugging Face 上的 Sentence Transformers

完整模型架构

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: XLMRobertaModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

使用方法

直接使用 (Sentence Transformers)

首先安装 Sentence Transformers 库:

pip install -U sentence-transformers

然后你可以加载此模型并运行推理。

from sentence_transformers import SentenceTransformer

# 从 🤗 Hub 下载
model = SentenceTransformer("Lauther/measuring-embeddings-v4.2")
# 运行推理
sentences = [
    'uncertainty points',
    'What is a Fluid?\nA Fluid is the substance measured within a measurement system. It can be a gas or liquid, such as hydrocarbons, water, or other industrial fluids. Proper classification of fluids is essential for ensuring measurement accuracy, regulatory compliance, and operational efficiency. By identifying fluids correctly, the system applies the appropriate measurement techniques, processing methods, and reporting standards.',
    'What is a Calibration Point?\nA Calibration Point represents a specific data entry in a calibration process, comparing an expected reference value to an actual measured value. These points are fundamental in ensuring measurement accuracy and identifying deviations.\n\nKey Aspects of Calibration Points:\n- Calibration Report Association: Each calibration point belongs to a specific calibration report, linking it to a broader calibration procedure.\n- Reference Values: Theoretical or expected values used as a benchmark for measurement validation.\n- Measured Values: The actual recorded values during calibration, reflecting the instrument's response.\n- Errors: The difference between reference and measured values, indicating possible measurement inaccuracies.\nCalibration points are essential for evaluating instrument performance, ensuring compliance with standards, and maintaining measurement reliability.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# 获取嵌入的相似度分数
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

<!--

直接使用 (Transformers)

<details><summary>点击查看 Transformers 中的直接使用方法</summary>

</details> -->

<!--

下游使用 (Sentence Transformers)

你可以在自己的数据集上微调此模型。

<details><summary>点击展开</summary>

</details> -->

<!--

超出范围的使用

列出模型可能被预见滥用的方式,并说明用户不应该用模型做什么。 -->

<!--

偏差、风险和局限性

这个模型有哪些已知或可预见的问题?你也可以在这里标记已知的失败案例或模型的弱点。 -->

<!--

建议

针对可预见的问题有什么建议?例如,过滤显式内容。 -->

训练详情

训练数据集

measuring-embeddings-v4

• 数据集: measuring-embeddings-v4 版本 1e3ca2c
• 大小: 3,075 个训练样本
• 列: <code>sentence1</code>, <code>sentence2</code>, 和 <code>score</code>
• 基于前 1000 个样本的近似统计:

  	sentence1	sentence2	score
  type	string	string	float
  details	<ul><li>min: 3 tokens</li><li>mean: 7.55 tokens</li><li>max: 17 tokens</li></ul>	<ul><li>min: 80 tokens</li><li>mean: 180.22 tokens</li><li>max: 406 tokens</li></ul>	<ul><li>min: 0.07</li><li>mean: 0.21</li><li>max: 0.95</li></ul>

• 样本:

  sentence1	sentence2	score
  <code>last calibrated span</code>	<code>What are historical report values?<br>These represent the recorded data points within flow computer reports. Unlike the report index, which serves as a reference to locate reports, these values contain the actual measurements and calculated data stored in the historical records.<br><br>Flow computer reports store two types of data values:<br><br>- Hourly data values: Contain measured or calculated values (e.g., operational minutes, alarms set, etc.) recorded on an hourly basis.<br>- Daily data values: Contain measured or calculated values (e.g., operational minutes, alarms set, etc.) recorded on a daily basis.<br>Each value is directly linked to its respective report index, ensuring traceability to the original flow computer record. These values maintain their raw integrity, providing a reliable source for analysis and validation.</code>	<code>0.1</code>
  <code>flow computer configuration</code>	<code>What is a Measurement Type?<br>Measurement types define the classification of measurements used within a system based on their purpose and regulatory requirements. These types include fiscal, appropriation, operational, and custody measurements. <br><br>- Fiscal measurements are used for tax and regulatory reporting, ensuring accurate financial transactions based on measured quantities. <br>- Appropriation measurements track resource allocation and ownership distribution among stakeholders. <br>- Operational measurements support real-time monitoring and process optimization within industrial operations. <br>- Custody measurements are essential for legal and contractual transactions, ensuring precise handover of fluids between parties. <br><br>These classifications play a crucial role in compliance, financial accuracy, and operational efficiency across industries such as oil and gas, water management, and energy distribution. </code>	<code>0.1</code>
  <code>uncertainty certificate number</code>	<code>What is an Uncertainty Composition?<br>An Uncertainty Composition represents a specific factor that contributes to the overall uncertainty of a measurement system. These components are essential for evaluating the accuracy and reliability of measurements by identifying and quantifying the sources of uncertainty.<br><br>Key Aspects of an Uncertainty Component:<br>- Component Name: Defines the uncertainty factor (e.g., diameter, density, variance, covariance) influencing the measurement system.<br>- Value of Composition: Quantifies the component's contribution to the total uncertainty, helping to analyze which factors have the greatest impact.<br>- Uncertainty File ID: Links the component to a specific uncertainty dataset for traceability and validation.<br>Understanding these components is critical for uncertainty analysis, ensuring compliance with industry standards and improving measurement precision.</code>	<code>0.1</code>

• 损失函数: <code>CoSENTLoss</code> 参数如下:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

评估数据集

measuring-embeddings-v4

• 数据集: measuring-embeddings-v4 版本 1e3ca2c
• 大小: 659 个评估样本
• 列: <code>sentence1</code>, <code>sentence2</code>, 和 <code>score</code>
• 基于前 659 个样本的近似统计:

  	sentence1	sentence2	score
  type	string	string	float
  details	<ul><li>min: 3 tokens</li><li>mean: 7.63 tokens</li><li>max: 17 tokens</li></ul>	<ul><li>min: 80 tokens</li><li>mean: 186.36 tokens</li><li>max: 406 tokens</li></ul>	<ul><li>min: 0.07</li><li>mean: 0.2</li><li>max: 0.9</li></ul>

• 样本:

  sentence1	sentence2	score
  <code>measurement system details</code>	<code>What is an Uncertainty Composition?<br>An Uncertainty Composition represents a specific factor that contributes to the overall uncertainty of a measurement system. These components are essential for evaluating the accuracy and reliability of measurements by identifying and quantifying the sources of uncertainty.<br><br>Key Aspects of an Uncertainty Component:<br>- Component Name: Defines the uncertainty factor (e.g., diameter, density, variance, covariance) influencing the measurement system.<br>- Value of Composition: Quantifies the component's contribution to the total uncertainty, helping to analyze which factors have the greatest impact.<br>- Uncertainty File ID: Links the component to a specific uncertainty dataset for traceability and validation.<br>Understanding these components is critical for uncertainty analysis, ensuring compliance with industry standards and improving measurement precision.</code>	<code>0.15</code>
  <code>measurement system tag EMED-3102-02-010</code>	<code>What is a report index or historic index?<br>Indexes represent the recorded reports generated by flow computers, classified into two types: <br>- Hourly reports Index: Store data for hourly events.<br>- Daily reports Index: Strore data for daily events.<br><br>These reports, also referred to as historical data or flow computer historical records, contain raw, first-hand measurements directly collected from the flow computer. The data has not been processed or used in any calculations, preserving its original state for analysis or validation.<br><br>The index is essential for locating specific values within the report.</code>	<code>0.24</code>
  <code>static pressure</code>	<code>What is a Meter Stream?<br>A Meter Stream represents a measurement system configured within a flow computer. It serves as the interface between the physical measurement system and the computational processes that record and analyze flow data.<br><br>Key Aspects of a Meter Stream:<br>- Status: Indicates whether the meter stream is active or inactive.<br>- Measurement System Association: Links the meter stream to a specific measurement system, ensuring that the data collected corresponds to a defined physical setup.<br>- Flow Computer Association: Identifies the flow computer responsible for managing and recording the measurement system's data.<br>Why is a Meter Stream Important?<br>A meter stream is a critical component in flow measurement, as it ensures that the measurement system is correctly integrated into the flow computer for accurate monitoring and reporting. Since each flow computer can handle multiple meter streams, proper configuration is essential for maintaining data integrity and traceability.</code>	<code>0.1</code>

• 损失函数: <code>CoSENTLoss</code> 参数如下:

  {
      "scale": 20.0,
      "similarity_fct": "pairwise_cos_sim"
  }
  

训练超参数

非默认超参数

• eval_strategy: steps
• per_device_train_batch_size: 4
• per_device_eval_batch_size: 4
• gradient_accumulation_steps: 4
• learning_rate: 2e-05
• num_train_epochs: 10
• warmup_ratio: 0.1

所有超参数

<details><summary>点击展开</summary>

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 4
• per_device_eval_batch_size: 4
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 4
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 10
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 42
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: False
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: False
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional

</details>

训练日志

<details><summary>点击展开</summary>

</details>

框架版本

• Python: 3.11.0
• Sentence Transformers: 3.4.1
• Transformers: 4.49.0
• PyTorch: 2.6.0+cu124
• Accelerate: 1.4.0
• Datasets: 3.3.2
• Tokenizers: 0.21.0

引用

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

CoSENTLoss

@online{kexuefm-8847,
    title={CoSENT: A more efficient sentence vector scheme than Sentence-BERT},
    author={Su Jianlin},
    year={2022},
    month={Jan},
    url={https://kexue.fm/archives/8847},
}

<!--

术语表

清晰地定义术语,以便让不同受众都能理解。 -->

<!--

模型卡作者

列出创建模型卡的人员,对模型卡详细工作的认可和责任。 -->

<!--

模型卡联系方式

为有模型卡更新、建议或问题的人提供联系模型卡作者的方式。 -->