kernelfoundry.algorithm.prompts.meta_prompting

Meta-Prompting database and manager for evolving parts of the prompt.

This module implements Science-CodeEvolve style meta-prompting where prompts evolve as complete programs rather than as modular genes.

Architecture

• PromptProgram: A complete prompt with evolvable regions

• HolisticPromptDatabase: MAP-Elites style database for prompt diversity

• MetaPrompter: LLM-based prompt mutation via SEARCH/REPLACE diffs

• MetaPromptingManager: Integration layer with the kernel generation pipeline

Evolution strategy

1. Prompts contain designated evolvable regions (PROMPT-BLOCK-START/END markers)

2. An auxiliary LLM (meta-prompter) generates SEARCH/REPLACE diffs to evolve prompts

3. Prompt fitness equals the best solution fitness it ever generated

4. Selection uses same strategies as programs (roulette, tournament, etc.)

Key difference from gene-based evolution

• Gene-based: Prompts assembled from independent components

• Holistic: Prompts evolve as coherent units with internal structure

• Both approaches are complementary and can be used together

Thread safety

• All public methods acquire locks before accessing shared state

• Database operations are atomic

Inspired by: Science-CodeEvolve (https://github.com/inter-co/science-codeevolve)

Functions

create_meta_prompting_manager(config, output_dir)	Factory function to create a MetaPromptingManager.
get_default_evolvable_regions([language])	Get default content for evolvable prompt regions.

Classes

HolisticPromptDatabase([config, output_dir, ...])	Database for holistic prompt evolution following Science-CodeEvolve design.
MetaPromptStrategy(value)	Strategies for meta-prompting.
MetaPrompter(llm_server[, language, ...])	LLM-based prompt evolution via SEARCH/REPLACE diffs.
MetaPromptingManager(config, output_dir[, ...])	High-level manager for meta-prompting integration with kernel generation.
PromptEvolutionMode(value)	Modes for prompt evolution.
PromptProgram(id, template_overrides, ...)	A complete prompt that can evolve like a program.

class kernelfoundry.algorithm.prompts.meta_prompting.PromptEvolutionMode(value)

Modes for prompt evolution.

HOLISTIC = 1

GENE_BASED = 2

HYBRID = 3

class kernelfoundry.algorithm.prompts.meta_prompting.MetaPromptStrategy(value)

Strategies for meta-prompting.

IMPROVE = 1

SPECIALIZE = 2

GENERALIZE = 3

SIMPLIFY = 4

ELABORATE = 5

class kernelfoundry.algorithm.prompts.meta_prompting.PromptProgram(id: str, template_overrides: ~typing.Dict[str, str] = <factory>, base_template_name: str = 'main_prompt.j2', fitness: float = 0.0, generation: int = 0, parent_id: str | None = None, child_scores: ~typing.List[~typing.Tuple[float, float]] = <factory>, usage_count: int = 0, creation_time: float = <factory>, metadata: ~typing.Dict[str, ~typing.Any] = <factory>, best_child_id: str | None = None, best_child_code: str | None = None, best_child_metrics: ~typing.Dict[str, float] = <factory>)

A complete prompt that can evolve like a program.

Unlike gene-based prompts where components are assembled, this represents the full prompt as a coherent unit with designated evolvable regions.

id: str

Unique identifier for this prompt

template_overrides: Dict[str, str]

Dict mapping region_name -> evolved text

base_template_name: str = 'main_prompt.j2'

Name of the Jinja2 template this is based on

fitness: float = 0.0

Best fitness score from kernels generated with this prompt

generation: int = 0

Evolution generation (0 = seed)

parent_id: str | None = None

ID of parent prompt if evolved

child_scores: List[Tuple[float, float]]

History of child kernel scores for selection

usage_count: int = 0

Number of times this prompt was used

creation_time: float

Timestamp of prompt creation

metadata: Dict[str, Any]

Additional tracking information

best_child_id: str | None = None

best_child_code: str | None = None

best_child_metrics: Dict[str, float]

property content_hash: str

Generate a hash of the evolvable content for deduplication.

property mean_child_score: float

Mean score of children generated with this prompt.

update_fitness(child_score: float, child_id: str | None = None, child_code: str | None = None, child_metrics: Dict[str, float] | None = None)

Update prompt fitness based on child solution performance.

Following Science-CodeEvolve: prompt.fitness = max(child_scores)

get_evolvable_content() → str

Get the combined evolvable content for meta-prompting display.

apply_diff(diff: str) → PromptProgram | None

Apply a SEARCH/REPLACE diff to create a child prompt.

Supports two formats: 1. With section name: <<<<<<< SEARCH [section_name] 2. Without section name: <<<<<<< SEARCH (searches all sections)

Parameters:

diff – The diff text from meta-prompting

Returns:

New PromptProgram if diff applied successfully, None otherwise

to_dict() → Dict[str, Any]

Serialize to dictionary for persistence.

classmethod from_dict(data: Dict[str, Any]) → PromptProgram

Deserialize from dictionary.

__init__(id: str, template_overrides: ~typing.Dict[str, str] = <factory>, base_template_name: str = 'main_prompt.j2', fitness: float = 0.0, generation: int = 0, parent_id: str | None = None, child_scores: ~typing.List[~typing.Tuple[float, float]] = <factory>, usage_count: int = 0, creation_time: float = <factory>, metadata: ~typing.Dict[str, ~typing.Any] = <factory>, best_child_id: str | None = None, best_child_code: str | None = None, best_child_metrics: ~typing.Dict[str, float] = <factory>) → None

kernelfoundry.algorithm.prompts.meta_prompting.get_default_evolvable_regions(language: str = 'SYCL') → Dict[str, str]

Get default content for evolvable prompt regions.

These are the starting points for meta-prompting evolution.

Parameters:

language – Target kernel language (SYCL, CUDA, triton)

Returns:

Dict mapping region_name -> default content

class kernelfoundry.algorithm.prompts.meta_prompting.HolisticPromptDatabase(config: Dict[str, Any] | None = None, output_dir: str | None = None, language: str = 'SYCL')

Database for holistic prompt evolution following Science-CodeEvolve design.

Key differences from gene-based PromptEvolutionDatabase: - Prompts stored as complete units, not assembled from genes - Selection based on prompt-level fitness (best child score) - Genealogy tracking for evolution history - Migration support for island-based evolution

Thread-safe with proper locking.

__init__(config: Dict[str, Any] | None = None, output_dir: str | None = None, language: str = 'SYCL')

Initialize the holistic prompt database.

Parameters:

• config – Configuration dictionary

• output_dir – Directory for persistence

• language – Target kernel language

add(prompt: PromptProgram) → bool

Add a prompt to the database.

Parameters:

prompt – PromptProgram to add

Returns:

True if added, False if duplicate content

sample(rng=None) → Tuple[PromptProgram, str]

Sample a prompt for kernel generation.

Uses softmax selection with exploration bonus for less-used prompts.

Parameters:

rng – Optional random number generator

Returns:

Tuple of (selected prompt, session_id for fitness tracking)

update_fitness(session_id: str, kernel_score: float, kernel_id: str | None = None, kernel_code: str | None = None, kernel_metrics: Dict[str, float] | None = None)

Update prompt fitness based on generated kernel performance.

Parameters:

• session_id – Session ID from sample()

• kernel_score – Fitness score of the generated kernel

• kernel_id – Optional ID of the kernel

• kernel_code – Optional code of the kernel (for meta-prompting context)

• kernel_metrics – Optional detailed metrics

get_best_prompt() → PromptProgram | None

Get the prompt with highest fitness.

get_prompt_for_evolution() → PromptProgram | None

Get a prompt suitable for meta-prompting evolution.

Selects a prompt that: 1. Has been used enough to have reliable fitness estimates 2. Has stored child code for context 3. Balances exploitation (high fitness) with exploration (generation diversity)

get_statistics() → Dict[str, Any]

Get database statistics.

save(filepath: str | None = None)

Save database to file.

classmethod load(filepath: str) → HolisticPromptDatabase

Load database from file.

class kernelfoundry.algorithm.prompts.meta_prompting.MetaPrompter(llm_server: Callable, language: str = 'SYCL', max_retries: int = 2, timeout: float = 60.0)

LLM-based prompt evolution via SEARCH/REPLACE diffs.

Uses an auxiliary LLM to analyze prompt effectiveness and suggest improvements based on the kernels generated and their performance.

__init__(llm_server: Callable, language: str = 'SYCL', max_retries: int = 2, timeout: float = 60.0)

Initialize the meta-prompter.

Parameters:

• llm_server – LLM inference function (same interface as Controller.llm_server)

• language – Target kernel language

• max_retries – Maximum retries on failure

• timeout – Timeout for LLM calls

evolve_prompt(prompt: PromptProgram, strategy: MetaPromptStrategy = MetaPromptStrategy.IMPROVE) → PromptProgram | None

Evolve a prompt using meta-prompting.

Parameters:

• prompt – The prompt to evolve

• strategy – Evolution strategy to apply

Returns:

New evolved PromptProgram, or None if evolution failed

class kernelfoundry.algorithm.prompts.meta_prompting.MetaPromptingManager(config: Dict[str, Any], output_dir: str, llm_server: Callable | None = None, language: str = 'SYCL', load_database: bool = False)

High-level manager for meta-prompting integration with kernel generation.

Coordinates: - Prompt selection for kernel generation - Fitness tracking and attribution - Periodic prompt evolution via meta-prompting - Persistence of prompt database

This is the main integration point for the controller.

__init__(config: Dict[str, Any], output_dir: str, llm_server: Callable | None = None, language: str = 'SYCL', load_database: bool = False)

Initialize the meta-prompting manager.

Parameters:

• config – Configuration dictionary with meta-prompting settings

• output_dir – Directory for persistence

• llm_server – LLM server for meta-prompting (optional, can be set later)

• language – Target kernel language

set_llm_server(llm_server: Callable)

Set the LLM server for meta-prompting.

property database: HolisticPromptDatabase

Access the prompt database.

sample_prompt(rng=None) → Tuple[PromptProgram | None, str]

Sample a prompt for kernel generation.

Parameters:

rng – Optional random number generator

Returns:

Tuple of (prompt, session_id)

get_evolvable_content(prompt: PromptProgram) → Dict[str, str]

Get the evolvable content from a prompt.

Returns dict that can be passed to Jinja2 template.

report_fitness(session_id: str, kernel_score: float, kernel_id: str | None = None, kernel_code: str | None = None, kernel_metrics: Dict[str, float] | None = None)

Report kernel fitness for prompt evolution.

Parameters:

• session_id – Session ID from sample_prompt()

• kernel_score – Fitness score of the generated kernel

• kernel_id – Optional ID of the kernel

• kernel_code – Optional code of the kernel

• kernel_metrics – Optional detailed metrics

force_evolution(num_prompts: int = 1) → List[PromptProgram]

Force immediate evolution of prompts.

Useful for manual triggering or testing.

Parameters:

num_prompts – Number of prompts to evolve

Returns:

List of newly created prompts

get_statistics() → Dict[str, Any]

Get meta-prompting statistics.

save()

Save prompt database.

load(filepath: str | None = None)

Load prompt database from file.

kernelfoundry.algorithm.prompts.meta_prompting.create_meta_prompting_manager(config: Dict[str, Any], output_dir: str, llm_server: Callable | None = None, language: str = 'SYCL') → MetaPromptingManager

Factory function to create a MetaPromptingManager.

Parameters:

• config – Configuration dictionary

• output_dir – Directory for persistence

• llm_server – Optional LLM server for meta-prompting

• language – Target kernel language

Returns:

Configured MetaPromptingManager instance