aac_metrics.functional.evaluate module¶

dcase2023_evaluate(

candidates: list[str],

mult_references: list[list[str]],

preprocess: bool | Callable[[list[str]], list[str]] = True,

cache_path: str | Path | None = None,

java_path: str | Path | None = None,

tmp_path: str | Path | None = None,

device: str | device | None = 'cuda_if_available',

verbose: int = 0,

) → tuple[dict[str, Tensor], dict[str, Tensor]][source]¶

Evaluate candidates with multiple references with the DCASE2023 Audio Captioning metrics.

Parameters:¶

candidates: list[str]¶

The list of sentences to evaluate.

mult_references: list[list[str]]¶

The list of list of sentences used as target.

preprocess: bool | Callable[[list[str]], list[str]] = True¶

If True, the candidates and references will be passed as input to the PTB stanford tokenizer before computing metrics. defaults to True.

cache_path: str | Path | None = None¶

The path to the external code directory. defaults to the value returned by get_default_cache_path().

java_path: str | Path | None = None¶

The path to the java executable. defaults to the value returned by get_default_java_path().

tmp_path: str | Path | None = None¶

Temporary directory path. defaults to the value returned by get_default_tmp_path().

device: str | device | None = 'cuda_if_available'¶

The PyTorch device used to run FENSE and SPIDErFL models. If None, it will try to detect use cuda if available. defaults to “cuda_if_available”.

verbose: int = 0¶

The verbose level. defaults to 0.

Returns:¶

A tuple contains the corpus and sentences scores.

dcase2024_evaluate(

candidates: list[str],

mult_references: list[list[str]],

preprocess: bool | Callable[[list[str]], list[str]] = True,

cache_path: str | Path | None = None,

java_path: str | Path | None = None,

tmp_path: str | Path | None = None,

device: str | device | None = 'cuda_if_available',

verbose: int = 0,

) → tuple[dict[str, Tensor], dict[str, Tensor]][source]¶

Evaluate candidates with multiple references with the DCASE2024 Audio Captioning metrics.

Parameters:¶

candidates: list[str]¶

The list of sentences to evaluate.

mult_references: list[list[str]]¶

The list of list of sentences used as target.

preprocess: bool | Callable[[list[str]], list[str]] = True¶

If True, the candidates and references will be passed as input to the PTB stanford tokenizer before computing metrics. defaults to True.

cache_path: str | Path | None = None¶

The path to the external code directory. defaults to the value returned by get_default_cache_path().

java_path: str | Path | None = None¶

The path to the java executable. defaults to the value returned by get_default_java_path().

tmp_path: str | Path | None = None¶

Temporary directory path. defaults to the value returned by get_default_tmp_path().

device: str | device | None = 'cuda_if_available'¶

The PyTorch device used to run FENSE and SPIDErFL models. If None, it will try to detect use cuda if available. defaults to “cuda_if_available”.

verbose: int = 0¶

The verbose level. defaults to 0.

Returns:¶

A tuple contains the corpus and sentences scores.

evaluate(

candidates: list[str],

mult_references: list[list[str]],

preprocess: bool | Callable[[list[str]], list[str]] = True,

metrics: str | Iterable[str] | Iterable[Callable[[list, list], tuple]] = 'default',

cache_path: str | Path | None = None,

java_path: str | Path | None = None,

tmp_path: str | Path | None = None,

device: str | device | None = 'cuda_if_available',

verbose: int = 0,

) → tuple[dict[str, Tensor], dict[str, Tensor]][source]¶

Evaluate candidates with multiple references with custom metrics.

Parameters:¶

candidates: list[str]¶

The list of sentences to evaluate.

mult_references: list[list[str]]¶

The list of list of sentences used as target.

preprocess: bool | Callable[[list[str]], list[str]] = True¶

If True, the candidates and references will be passed as input to the PTB stanford tokenizer before computing metrics. defaults to True.

metrics: str | Iterable[str] | Iterable[Callable[[list, list], tuple]] = 'default'¶

The name of the metric list or the explicit list of metrics to compute. defaults to “default”.

cache_path: str | Path | None = None¶

The path to the external code directory. defaults to the value returned by get_default_cache_path().

java_path: str | Path | None = None¶

The path to the java executable. defaults to the value returned by get_default_java_path().

tmp_path: str | Path | None = None¶

Temporary directory path. defaults to the value returned by get_default_tmp_path().

device: str | device | None = 'cuda_if_available'¶

The PyTorch device used to run FENSE and SPIDErFL models. If None, it will try to detect use cuda if available. defaults to “cuda_if_available”.

verbose: int = 0¶

The verbose level. defaults to 0.

Returns:¶

A tuple contains the corpus and sentences scores.

get_argnames(

fn: Callable,

) → list[str][source]¶

Get arguments names of a method, function or callable object.