aac_metrics.functional.evaluate module¶
- dcase2023_evaluate(
- candidates: list[str],
- mult_references: list[list[str]],
- preprocess: bool | Callable[[list[str]], list[str]] = True,
- cache_path: str | Path | None = None,
- java_path: str | Path | None = None,
- tmp_path: str | Path | None = None,
- device: str | device | None = 'cuda_if_available',
- verbose: int = 0,
Evaluate candidates with multiple references with the DCASE2023 Audio Captioning metrics.
- Parameters:
candidates – The list of sentences to evaluate.
mult_references – The list of list of sentences used as target.
preprocess – If True, the candidates and references will be passed as input to the PTB stanford tokenizer before computing metrics. defaults to True.
cache_path – The path to the external code directory. defaults to the value returned by
get_default_cache_path().java_path – The path to the java executable. defaults to the value returned by
get_default_java_path().tmp_path – Temporary directory path. defaults to the value returned by
get_default_tmp_path().device – The PyTorch device used to run FENSE and SPIDErFL models. If None, it will try to detect use cuda if available. defaults to “cuda_if_available”.
verbose – The verbose level. defaults to 0.
- Returns:
A tuple contains the corpus and sentences scores.
- dcase2024_evaluate(
- candidates: list[str],
- mult_references: list[list[str]],
- preprocess: bool | Callable[[list[str]], list[str]] = True,
- cache_path: str | Path | None = None,
- java_path: str | Path | None = None,
- tmp_path: str | Path | None = None,
- device: str | device | None = 'cuda_if_available',
- verbose: int = 0,
Evaluate candidates with multiple references with the DCASE2024 Audio Captioning metrics.
- Parameters:
candidates – The list of sentences to evaluate.
mult_references – The list of list of sentences used as target.
preprocess – If True, the candidates and references will be passed as input to the PTB stanford tokenizer before computing metrics. defaults to True.
cache_path – The path to the external code directory. defaults to the value returned by
get_default_cache_path().java_path – The path to the java executable. defaults to the value returned by
get_default_java_path().tmp_path – Temporary directory path. defaults to the value returned by
get_default_tmp_path().device – The PyTorch device used to run FENSE and SPIDErFL models. If None, it will try to detect use cuda if available. defaults to “cuda_if_available”.
verbose – The verbose level. defaults to 0.
- Returns:
A tuple contains the corpus and sentences scores.
- evaluate(
- candidates: list[str],
- mult_references: list[list[str]],
- preprocess: bool | Callable[[list[str]], list[str]] = True,
- metrics: str | Iterable[str] | Iterable[Callable[[list, list], tuple]] = 'default',
- cache_path: str | Path | None = None,
- java_path: str | Path | None = None,
- tmp_path: str | Path | None = None,
- device: str | device | None = 'cuda_if_available',
- verbose: int = 0,
Evaluate candidates with multiple references with custom metrics.
- Parameters:
candidates – The list of sentences to evaluate.
mult_references – The list of list of sentences used as target.
preprocess – If True, the candidates and references will be passed as input to the PTB stanford tokenizer before computing metrics. defaults to True.
metrics – The name of the metric list or the explicit list of metrics to compute. defaults to “default”.
cache_path – The path to the external code directory. defaults to the value returned by
get_default_cache_path().java_path – The path to the java executable. defaults to the value returned by
get_default_java_path().tmp_path – Temporary directory path. defaults to the value returned by
get_default_tmp_path().device – The PyTorch device used to run FENSE and SPIDErFL models. If None, it will try to detect use cuda if available. defaults to “cuda_if_available”.
verbose – The verbose level. defaults to 0.
- Returns:
A tuple contains the corpus and sentences scores.