aac_metrics.functional.sbert_sim module

class SBERTSimScores

Bases: dict

sbert_sim: Tensor

sbert_sim(

candidates: list[str],

mult_references: list[list[str]],

return_all_scores: bool = True,

*,

sbert_model: str | SentenceTransformer = 'paraphrase-TinyBERT-L6-v2',

device: str | device | None = 'cuda_if_available',

batch_size: int | None = 32,

reset_state: bool = True,

verbose: int = 0,

) → tuple[SBERTSimScores, SBERTSimScores] | Tensor

Cosine-similarity of the Sentence-BERT embeddings.

• Paper: https://arxiv.org/abs/1908.10084

• Original implementation: https://github.com/blmoistawinde/fense

Parameters:

• candidates – The list of sentences to evaluate.

• mult_references – The list of list of sentences used as target.

• return_all_scores – If True, returns a tuple containing the globals and locals scores. Otherwise returns a scalar tensor containing the main global score. defaults to True.

• sbert_model – The sentence BERT model used to extract sentence embeddings for cosine-similarity. defaults to “paraphrase-TinyBERT-L6-v2”.

• device – The PyTorch device used to run pre-trained models. If “cuda_if_available”, it will use cuda if available. defaults to “cuda_if_available”.

• batch_size – The batch size of the sBERT models. defaults to 32.

• reset_state – If True, reset the state of the PyTorch global generator after the initialization of the pre-trained models. defaults to True.

• verbose – The verbose level. defaults to 0.

Returns:

A tuple of globals and locals scores or a scalar tensor with the main global score.