kernelfoundry.eval_pipeline.utils.custom_task_helper

Functions for analyzing and manipulating custom tasks.

Functions

allowed_text_file_extensions()

Returns a list of allowed text file extensions for custom tasks.

dict_to_yaml_str(data[, indent])

Converts a dict to a yaml string.

find_path(task, key)

Find the path to the file containing the block annotated with the given key.

get_block(task_root, key)

Returns the text block annotated with the given key.

get_config(task_root[, config_stem])

Reads the configuration yaml with the specified name in the task root directory.

get_test_names(task_root)

Returns the names of the correctnes and profile tests defined in task.py.

update_block(task, key, path, content)

Updates the block annotated with the given key in the specified file.
kernelfoundry.eval_pipeline.utils.custom_task_helper.allowed_text_file_extensions() → list[str][source]

Returns a list of allowed text file extensions for custom tasks.

kernelfoundry.eval_pipeline.utils.custom_task_helper.get_test_names(task_root: Path | str) → dict[str, list[str]][source]

Returns the names of the correctnes and profile tests defined in task.py.

Parameters:

task_root (Path | str) – Path to the root directory of the custom task.

Returns:

A dictionary with three keys: ‘all_tests’, ‘correctness_tests’ and ‘profile_tests’.

Each key maps to a list of test names.

Return type:

dict[str,list[str]]

kernelfoundry.eval_pipeline.utils.custom_task_helper.get_block(task_root: Path | str | MemoryFileMap, key: str) → dict[str, str][source]

Returns the text block annotated with the given key.

Parameters:

  • task_root (Path | str | MemoryFileMap) – Path to the root directory of the custom task or the in-memory file map.

  • key (str) – The key used to annotate the block (e.g., ‘REFERENCE’, ‘EVOLVE’,..)

Returns:

A dictionary with the annotated blocks as values and keys

as the relative path to the file where the block was found.

Return type:

dict[str,str]

kernelfoundry.eval_pipeline.utils.custom_task_helper.get_config(task_root: Path | str | MemoryFileMap, config_stem: str = 'config') → dict[source]

Reads the configuration yaml with the specified name in the task root directory.

Parameters:

  • task_root (Path | str | MemoryFileMap) – Path to the root directory of the custom task or a MemoryFileMap.

  • config_name (str, optional) – Name of the config file without extension. Defaults to “config”.

Returns:

The task configuration as a dictionary.

Return type:

dict

kernelfoundry.eval_pipeline.utils.custom_task_helper.update_block(task: MemoryFileMap, key: str, path: str | Path, content: str) → None[source]

Updates the block annotated with the given key in the specified file.

This updates the block in-place within the in-memory file map.

Parameters:

  • task (MemoryFileMap) – The in-memory file map of the task.

  • key (str) – The key used to annotate the block (e.g., ‘REFERENCE’, ‘EVOLVE’,..)

  • path (str | Path) – The path to the file within the task.

  • content (str) – The new content to insert into the block.