aac_metrics.functional.mult_cands module¶

mult_cands_metric(metric: ~typing.Callable, metric_out_name: str, mult_candidates: list[list[str]], mult_references: list[list[str]], return_all_scores: bool = True, *, return_all_cands_scores: bool = False, selection: ~typing.Literal['max', 'min', 'mean'] = 'max', reduction_fn: ~typing.Callable[[~torch.Tensor], ~torch.Tensor] = <built-in method mean of type object>, **kwargs) → tuple[dict[str, Tensor], dict[str, Tensor]] | Tensor[source]¶

Multiple candidates metric wrapper.

Parameters:¶

metric

Any Callable metric code. Take (candidates, mult_references, return_all_scores) and return the global and local scores.

metric_out_name

The name of the metric output. Should be one of the keys of the sentences local scores returned by the metric.

mult_candidates

The list of list of sentences to evaluate.

mult_references

The references input.

selection

The selection to apply. Can be “max”, “min” or “mean”. defaults to “max”.

reduction_fn

The reduction function to apply to local scores. defaults to torch.mean.

**kwargs

The keywords arguments given to the metric call.

Returns:¶

A tuple of globals and locals scores or a scalar tensor with the main global score.