aac_metrics.functional.spider module¶

class SPIDErScores¶

Bases: dict

cider_d : Tensor¶

spice : Tensor¶

spider : Tensor¶

spider(candidates: list[str], mult_references: list[list[str]], return_all_scores: bool = True, *, n: int = 4, sigma: float = 6.0, tokenizer: ~typing.Callable[[str], list[str]] = <method 'split' of 'str' objects>, return_tfidf: bool = False, cache_path: str | ~pathlib.Path | None = None, java_path: str | ~pathlib.Path | None = None, tmp_path: str | ~pathlib.Path | None = None, n_threads: int | None = None, java_max_memory: str = '8G', timeout: None | int | ~typing.Iterable[int] = None, verbose: int = 0) → tuple[SPIDErScores, SPIDErScores] | Tensor[source]¶

SPIDEr function.

Paper: https://arxiv.org/pdf/1612.00370.pdf

Warning

This metric requires at least 2 candidates with 2 sets of references, otherwise it will raises a ValueError.

Parameters:¶

candidates: The list of sentences to evaluate.
mult_references: The list of list of sentences used as target.
return_all_scores: If True, returns a tuple containing the globals and locals scores. Otherwise returns a scalar tensor containing the main global score. defaults to True.
n: Maximal number of n-grams taken into account. defaults to 4.
sigma: Standard deviation parameter used for gaussian penalty. defaults to 6.0.
tokenizer: The fast tokenizer used to split sentences into words. defaults to str.split.
return_tfidf: If True, returns the list of dictionaries containing the tf-idf scores of n-grams in the sents_score output. defaults to False.
cache_path: The path to the external code directory. defaults to the value returned by get_default_cache_path().
java_path: The path to the java executable. defaults to the value returned by get_default_java_path().
tmp_path: Temporary directory path. defaults to the value returned by get_default_tmp_path().
n_threads: Number of threads used to compute SPICE. None value will use the default value of the java program. defaults to None.
java_max_memory: The maximal java memory used. defaults to “8G”.
timeout: The number of seconds before killing the java subprogram. If a list is given, it will restart the program if the i-th timeout is reached. If None, no timeout will be used. defaults to None.
verbose: The verbose level. defaults to 0.

Returns:¶

A tuple of globals and locals scores or a scalar tensor with the main global score.