kernelfoundry.eval_pipeline.utils.performance

Runtime measurement utilities.

Functions

detect_profiler()	Detect whether the process is running under a profiler.
measure_runtime(target, sync_fn[, args, ...])	Measures the runtime of the target callable.
measure_runtime_torch(target, device[, ...])	Measures the runtime of the target callable on the specified torch device.
profiler_session(profiler_name[, stop_on_exit])	Context manager to resume, pause and stop profiler sessions.

kernelfoundry.eval_pipeline.utils.performance.detect_profiler() → str | None

Detect whether the process is running under a profiler.

Returns:

The detected profiler name, or None if not detected.

Return type:

str | None

kernelfoundry.eval_pipeline.utils.performance.profiler_session(profiler_name: str, stop_on_exit: bool = False)

Context manager to resume, pause and stop profiler sessions.

Supports unitrace and vtune.

Parameters:

• profiler_name (str) – The name of the profiler to manage.

• stop_on_exit (bool) – Whether to stop the profiler session on exit.

• False (If)

• stopped. (the session will be paused instead of)

kernelfoundry.eval_pipeline.utils.performance.measure_runtime(target: Callable, sync_fn: Callable, args: list[tuple] | list[list] | tuple | list | None = None, kwargs: list[dict] | dict | None = None, warmup_min_time: float = 1.0, warmup_min_iters: int = 10, inner_loop_min_time: float = 0.01, perf_trials_min_iters: int = 10, perf_trials_min_time: float = 1.0, use_itt: bool = False, reduce_iterations_for_external_profiler: bool = True, auto_replicate_inputs_size: int = 134217728, info_str: str = '', output: list[float] | None = None, profiler_label: str | None = None) → list[float]

Measures the runtime of the target callable.

The function assumes that all invocations of the target are run in order on the same device, and that the sync_fn function will synchronize the device to ensure all operations are complete.

Parameters:

• target (Callable) – The kernel function to be measured.

• sync_fn (Callable) – The synchronization function to be called after target execution.

• args – Positional arguments to pass to the target function. This can be a list of positional arguments to iterate over different inputs for each call to the target. Note that you must provide a list of kwargs of the same length if you provide a list of args. Use kwargs=len(args)*[{}] if there are no kwargs to pass. Note that arguments for each list entry should have the same shape and structure.

• kwargs – Keyword arguments to pass to the target function. If this is a list of keyword argument dictionaries, then this function will iterate through the list to use a different set of kwargs for each call to the target with the intent to avoid caching effects. Note that args must be a list of tuples/lists of the same length as kwargs in this case. Note that arguments for each list entry should have the same shape and structure.

• warmup_min_time (float) – Minimum total time for warmup phase in seconds.

• warmup_min_iters (int) – Minimum number of iterations for warmup phase.

• inner_loop_min_time (float) – Minimum time for inner loop trials in seconds.

• perf_trials_min_iters (int) – Minimum number of performance trials.

• perf_trials_min_time (float) – Minimum total time for performance trials in seconds.

• use_itt (bool) – Whether to use ITT annotations during profiling.

• reduce_iterations_for_external_profiler (bool) – If an external profiler is detected, reduce the number of iterations to avoid long profiling sessions.

• auto_replicate_inputs_size (int) – Replicate the inputs, args and kwargs, to this size to avoid caching effects for very small inputs. Set to 0 to disable replication. This option has no effect if args and kwargs are lists of arguments.

• info_str (str) – Additional info string added before the timing info about warmup and test iterations. Useful for adding information about the device.

• output (list[float]) – Optional list to store the measured runtimes.

• profiler_label (str | None) – Optional label embedded into ITT model run loop markers. If omitted, the current pytest node id is read from the environment when available.

Returns:

List of measured runtimes in milliseconds.

kernelfoundry.eval_pipeline.utils.performance.measure_runtime_torch(target: Callable, device: str | torch.device, args: tuple | list | None = None, kwargs: dict | None = None, warmup_min_time: float = 1.0, warmup_min_iters: int = 10, inner_loop_min_time: float = 0.01, perf_trials_min_iters: int = 10, perf_trials_min_time: float = 1.0, use_itt: bool = False, reduce_iterations_for_external_profiler: bool = True, auto_replicate_inputs_size: int = 134217728, output: list[float] | None = None, profiler_label: str | None = None) → list[float]

Measures the runtime of the target callable on the specified torch device.

Parameters:

• target (Callable) – The kernel function to be measured.

• device (Union[str, torch.device]) – The device to use for synchronization.

• args – Positional arguments to pass to the target function. This can be a list of positional arguments to iterate over different inputs for each call to the target. Note that you must provide a list of kwargs of the same length if you provide a list of args. Use kwargs=len(args)*[{}] if there are no kwargs to pass. Note that arguments for each list entry should have the same shape and structure.

• kwargs – Keyword arguments to pass to the target function. If this is a list of keyword argument dictionaries, then this function will iterate through the list to use a different set of kwargs for each call to the target with the intent to avoid caching effects. Note that args must be a list of tuples/lists of the same length as kwargs in this case. Note that arguments for each list entry should have the same shape and structure.

• warmup_min_time (float) – Minimum total time for warmup phase in seconds.

• warmup_min_iters (int) – Minimum number of iterations for warmup phase.

• inner_loop_min_time (float) – Minimum time for inner loop trials in seconds.

• perf_trials_min_iters (int) – Minimum number of performance trials.

• perf_trials_min_time (float) – Minimum total time for performance trials in seconds.

• use_itt (bool) – Whether to use ITT annotations during profiling.

• reduce_iterations_for_external_profiler (bool) – If an external profiler is detected, reduce the number of iterations to avoid long profiling sessions.

• auto_replicate_inputs_size (int) – Replicate the inputs, args and kwargs, to this size to avoid caching effects for very small inputs. Set to 0 to disable replication. This option has no effect if args and kwargs are lists of arguments.

• output (list[float]) – Optional list to store the measured runtimes.

• profiler_label (str | None) – Optional label embedded into ITT model run loop markers.

Returns:

List of measured runtimes in milliseconds.

Return type:

list[float]