aac_metrics.functional.spice module

class SPICEScores

Bases: dict

spice: Tensor
check_spice_install(
cache_path: str,
) → None[source]

Check if SPICE is installed in cache directory.

Raises FileNotFoundError or NotADirectoryError exception if something is missing.

spice(
candidates: list[str],
mult_references: list[list[str]],
return_all_scores: bool = True,
*,
cache_path: str | Path | None = None,
java_path: str | Path | None = None,
tmp_path: str | Path | None = None,
n_threads: int | None = None,
java_max_memory: str = '8G',
timeout: None | int | Iterable[int] = None,
separate_cache_dir: bool = True,
use_shell: bool | None = None,
verbose: int = 0,
) → tuple[SPICEScores, SPICEScores] | Tensor[source]

Semantic Propositional Image Caption Evaluation function.

Parameters:

  • candidates – The list of sentences to evaluate.

  • mult_references – The list of list of sentences used as target.

  • return_all_scores – If True, returns a tuple containing the globals and locals scores. Otherwise returns a scalar tensor containing the main global score. defaults to True.

  • cache_path – The path to the external code directory. defaults to the value returned by get_default_cache_path().

  • java_path – The path to the java executable. defaults to the value returned by get_default_java_path().

  • tmp_path – Temporary directory path. defaults to the value returned by get_default_tmp_path().

  • n_threads – Number of threads used to compute SPICE. None value will use the default value of the java program. defaults to None.

  • java_max_memory – The maximal java memory used. defaults to “8G”.

  • timeout – The number of seconds before killing the java subprogram. If a list is given, it will restart the program if the i-th timeout is reached. If None, no timeout will be used. defaults to None.

  • separate_cache_dir – If True, the SPICE cache files will be stored into in a new temporary directory. This removes potential freezes when multiple instances of SPICE are running in the same cache dir. defaults to True.

  • use_shell – Optional argument to force use os-specific shell for the java subprogram. If None, it will use shell only on Windows OS. defaults to None.

  • verbose – The verbose level. defaults to 0.

Returns:

A tuple of globals and locals scores or a scalar tensor with the main global score.