aac_metrics.functional.meteor module

class METEORScores

Bases: dict

meteor: Tensor
meteor(
candidates: list[str],
mult_references: list[list[str]],
return_all_scores: bool = True,
*,
cache_path: str | Path | None = None,
java_path: str | Path | None = None,
java_max_memory: str = '2G',
language: Literal['en', 'cz', 'de', 'es', 'fr'] = 'en',
use_shell: bool | None = None,
params: Iterable[float] | None = None,
weights: Iterable[float] | None = None,
verbose: int = 0,
) → tuple[METEORScores, METEORScores] | Tensor[source]

Metric for Evaluation of Translation with Explicit ORdering function.

Parameters:

  • candidates – The list of sentences to evaluate.

  • mult_references – The list of list of sentences used as target.

  • return_all_scores – If True, returns a tuple containing the globals and locals scores. Otherwise returns a scalar tensor containing the main global score. defaults to True.

  • cache_path – The path to the external code directory. defaults to the value returned by get_default_cache_path().

  • java_path – The path to the java executable. defaults to the value returned by get_default_java_path().

  • java_max_memory – The maximal java memory used. defaults to “2G”.

  • language – The language used for stem, synonym and paraphrase matching. Can be one of (“en”, “cz”, “de”, “es”, “fr”). defaults to “en”.

  • use_shell – Optional argument to force use os-specific shell for the java subprogram. If None, it will use shell only on Windows OS. defaults to None.

  • params – List of 4 parameters (alpha, beta gamma delta) used in METEOR metric. If None, it will use the default of the java program, which is (0.85, 0.2, 0.6, 0.75). defaults to None.

  • weights – List of 4 parameters (w1, w2, w3, w4) used in METEOR metric. If None, it will use the default of the java program, which is (1.0 1.0 0.6 0.8). defaults to None.

  • verbose – The verbose level. defaults to 0.

Returns:

A tuple of globals and locals scores or a scalar tensor with the main global score.