problem_logger#

Logger for kernel generation and evaluation results.

Functions

`extract_result_from_log`(log)	Reverse engineer result from a log file :param log: str, log file loaded as a string
`initialize_or_load_results`(logdir)	Load results from a previous job or initialize an empty results dictionary.

Classes

ProblemLogger(level, problem_id, logdir, trial)

Saves and loads the logged files for one problem in a single trial of one job

kernelfoundry.algorithm.problem_logger.extract_result_from_log(log: str) → tuple[dict, EvalResult][source]#

Reverse engineer result from a log file :param log: str, log file loaded as a string

Returns:: dict, exec result as dict EvalResult, the result inferred based on the log

kernelfoundry.algorithm.problem_logger.initialize_or_load_results(logdir: str)[source]#: Load results from a previous job or initialize an empty results dictionary.

class kernelfoundry.algorithm.problem_logger.ProblemLogger(level: int, problem_id: int, logdir: str, trial: int)[source]#

Saves and loads the logged files for one problem in a single trial of one job

__init__(level: int, problem_id: int, logdir: str, trial: int)[source]#

Initialize the problem logger.

Parameters:

read_from_prior_run(fn) → str | None[source]#: Load prior stdout or gen from a prior trial.

read_prior_gen_code()[source]#: Reads the generated code from the previous trial.

log_prompt_list(prompt_list: list[str]) → None[source]#: Write prompt to a file.

log_llm_messages(llm_messages: list) → None[source]#: Log whole conversation with LLM in one file.

log_eval_results(eval_results: list) → None[source]#: Write eval results to a file

save_gen_kernel(custom_kernel: str) → None[source]#: Save the generated kernel code to a file.

save_stdout(console_output: str, version=None) → None[source]#: Save console log to a file.

save_gen_kernel_w_version(custom_kernel: str, language: str, version: int) → str[source]#: Save the generated kernel code with versioning.

save_diffs(custom_kernel_out_list) → None[source]#: Save the diffs between the generated kernel and the previous one.

load_kernel_from_other_run(run_path: str) → tuple[str, str][source]#: Find the best generated kernel for this problem in run_path

log_result(kernel_exec_result: EvalResult, results: dict, save: bool = True) → dict[source]#: Log the result of the kernel execution to a JSON file.