kernelfoundry.testing

Testing utilities for kernel output validation.

Functions

all_close_with_slack(output_reference, ...)

Check the accuracy of the kernel output compared to the reference output.

assert_allclose(actual, expected, *[, ...])

Asserts that two arrays are close within a given relative tolerance.

cosine_similarity(output_reference, ...[, ...])

Compute cosine similarity of flattened output tensors.
kernelfoundry.testing.all_close_with_slack(output_reference: torch.Tensor, output_kernel: torch.Tensor, epsilon: float = 1e-07, max_rel_err: float = 0.01, ratio_below_max_err: float = 0.99) → bool[source]

Check the accuracy of the kernel output compared to the reference output.

This function computes the absolute relative error between the new and original outputs, and determines if the proportion of elements within a specified maximum relative error is above a given ratio.

Parameters:

  • output_reference (torch.Tensor) – The reference output tensor.

  • output_kernel (torch.Tensor) – The kernel output tensor to compare.

  • epsilon (float, optional) – A small constant to avoid division by zero. Default is 1e-7.

  • max_rel_err (float, optional) – The maximum relative error allowed. Default is 0.01.

  • ratio_below_max_err (float, optional) – The minimum required ratio of elements with error below the maximum relative error. Default is 0.99.

Returns:

True if the ratio of elements with a relative error below max_rel_err

is greater than ratio_below_max_err, False otherwise.

Return type:

bool

kernelfoundry.testing.cosine_similarity(output_reference: torch.Tensor, output_kernel: torch.Tensor, min_sim: float = 0.99985)[source]

Compute cosine similarity of flattened output tensors.

Returns:

True if similarity meets the threshold, False otherwise.

Return type:

bool

kernelfoundry.testing.assert_allclose(actual, expected, *, epsilon: float = 1e-07, rtol: float = 0.01, ratio_below_max_err: float = 0.99, msg: str | Callable[[str], str] | None = None, err_stats: bool = True) → None[source]

Asserts that two arrays are close within a given relative tolerance.

This function computes the absolute relative error between the new and original outputs, and determines if the proportion of elements within a specified maximum relative error is above a given ratio.

This function behaves like all_close_with_slack, but raises an AssertionError with a detailed message

Parameters:

  • actual (np.ndarray|torch.Tensor) – The output tensor to validate.

  • expected (np.ndarray|torch.Tensor) – The reference output tensor.

  • epsilon (float, optional) – A small constant to avoid division by zero. Default is 1e-7.

  • rtol (float, optional) – The maximum relative error allowed. Default is 0.01.

  • ratio_below_max_err (float, optional) – The minimum required ratio of elements with error below the maximum relative error. Default is 0.99.

  • msg (str | Callable[[str], str] | None, optional) – Optional custom error message.

  • err_stats (bool, optional) – Whether to include error statistics. Default is True.

Raises:

AssertionError – If the arrays are not close enough.