KernelFoundry

Hardware-aware evolutionary GPU kernel optimization

Nina Wiedemann^*, Quentin Leboutet^*, Michael Paulitsch, Diana Wofk, Benjamin Ummenhofer
Intel Corporation
^*Equal contribution

Code Paper Docs

Abstract

Optimizing GPU kernels presents a significantly greater challenge for large language models (LLMs) than standard code generation tasks, as it requires understanding hardware architecture, parallel optimization strategies, and performance profiling outputs. Most existing LLM-based approaches to kernel generation rely on simple prompting and feedback loops, incorporating hardware awareness only indirectly through profiling feedback. We introduce KernelFoundry, an evolutionary framework that efficiently explores the GPU kernel design space through three key mechanisms: (1) MAP-Elites quality-diversity search with kernel-specific behavioral dimensions to sustain exploration across diverse optimization strategies; (2) meta-prompt evolution, which co-evolves prompts with kernels to uncover task-specific optimization strategies, and (3) template-based parameter optimization to tune kernels to inputs and hardware. We evaluate this framework on KernelBench, robust-kbench, and custom tasks, generating SYCL kernels as a cross-platform GPU programming model and CUDA kernels for comparison to prior work. Our approach consistently outperforms the baseline methods, achieving an average speedup of 2.3x on KernelBench for SYCL. Moreover, KernelFoundry is implemented as a distributed framework with remote access to diverse hardware, enabling rapid benchmarking and featuring a flexible user input layer that supports kernel generation for a wide range of real-world use cases beyond benchmarking.

Approach Overview

Our approach adapts MAP-Elites for Quality-Diversity evolutionary search across a 3D optimization feature space.

Behavioral descriptors used in our optimization feature space.
	Memory	Algorithm	Parallelism
0	scalar, strided, or uncoalesced	direct PyTorch translation	no sync (embarrassingly parallel)
1	coalesced or vectorized	fused operations	work-group barriers
2	shared/local memory with tiling	reformulated algorithm	sub-group primitives
3	multi-level hierarchy	novel, improved asymptotically	global coordination (e.g., atomics)

Key Features

We augment MAP-Elites with gradient signals from transition history. These inform the sampling weights in parent selection and mutation hints in prompt construction to help steer exploration.
We allow prompts to co-evolve with kernels via a dedicated Meta-Prompter that diagnoses and refines guidance, e.g., a "common pitfalls" section.
We enable explicit hardware-specific parameter tuning through configurable templated kernels that can be dispatched and evaluated in a sweep.

Distributed Infrastructure

We build a distributed execution framework for scalability across hardware.

We leverage a customizable task format where tasks are defined by a set of files with configuration hyperparameters as well as special markers demarcating reference code, optional user instructions, and optional initial kernel implementations. Our pipeline processes tasks via a lightweight main thread that interacts with four servers. The server processes can run on a single machine or on multiple remote machines in the same network:

LLM Server: Hosts the generation model (Cloud services with REST-API or local vLLM instances).
Compilation Workers: Executes compiler toolchains (DPC++, nvcc). Does not need a GPU.
Execution Workers: Run correctness tests and benchmarks on GPU nodes. Connected via a task queue ensuring single-task-per-GPU isolation.
Database Server: Stores all generated kernels, evaluation results, and evolutionary state for reproducibility and analysis.

The LLM server, compile worker, and execution worker are connected via a load balancer and queue system that allows for scaling the number of workers to increase parallelism.

Results

KernelFoundry excels across benchmarks, backends, and hardware platforms.

When testing on CUDA kernel generation, we evaluate across KernelBench representative sets and robust-kbench tasks. KernelFoundry consistently outperforms re-evaluated baselines, with especially strong gains on L2 and robust-kbench.

*on NVIDIA L40S, †on NVIDIA H100, all others on NVIDIA A6000
Task set	Method	LLMs in Ensemble	Avg. Speedup
Kernel Bench L1 (n=20)	Kernelsseum*	GPT-{o1,4o}, DeepSeek-Coder, Sonnet-3.5, LLama-3.1-405B	0.765
	AI CUDA Engineer†	GPT-{o1-preview,o1-high,o3-mini}, DeepSeek-{v3,R1}, Sonnet-3.5	1.422
	AI CUDA Engineer		1.005
	Ours	GPT-o3-mini	1.204
	+ parameter optimization	GPT-o3-mini	1.241
Kernel Bench L2 (n=20)	Kernelsseum*	GPT-{o1,4o}, DeepSeek-Coder, Sonnet-3.5	0.874
	AI CUDA Engineer†	GPT-{o1-preview,o1-high,o3-mini}, DeepSeek-{v3,R1}, Sonnet-3.5	1.589
	AI CUDA Engineer		1.606
	Ours	GPT-o3-mini	2.051
	+ parameter optimization	GPT-o3-mini	2.104
Robust kbench (n=12)	Robust-kbench†	GPT-{o3,o4-mini,4.1}, Sonnet-3.7	15.622
	Robust-kbench	GPT-{o3,o4-mini,4.1}, Sonnet-3.7	8.865
	Ours	GPT-{o3,o4-mini,4.1}	12.107
	+ parameter optimization	GPT-{o3,o4-mini,4.1}	12.081

On SYCL kernel generation, our evolutionary algorithm demonstrates strong performance and begins to exceed OpenEvolve speedups with significantly fewer iterations.

Task set: KernelBench L2 (n=20). All methods tested the same LLM ensemble (GPT-{4.1, 5-mini}, Sonnet-4.5).
Method	# Iter.	Avg. Speedup
OpenEvolve (open-source implementation of AlphaEvolve)	10	1.483
OpenEvolve (open-source implementation of AlphaEvolve)	40	2.535
Ours (with kernel-specific optimization strategies, meta-prompting, and gradient guidance)	10	2.059
	20	2.539
Ours + parameter optimization	40	2.732

BibTeX

If you reference this work, please cite:

@inproceedings{wiedemann2026kernelfoundry,
  title={KernelFoundry: Hardware-Aware Evolutionary GPU Kernel Optimization},
  author={Nina Wiedemann and Quentin Leboutet and Michael Paulitsch and Diana Wofk and Benjamin Ummenhofer},
  booktitle={Forty-third International Conference on Machine Learning},
  year={2026},
  url={https://openreview.net/forum?id=4wy2YAHKRF}
}