## File: README.md

# OpenEvolve



---

## Why OpenEvolve?



| ### **Autonomous Discovery**
LLMs don't just optimize—they **discover** entirely new algorithms. No human guidance needed. | ### **Proven Results**
**2-3x speedups** on real hardware. **State-of-the-art** circle packing. **Breakthrough** optimizations. | ### **Research Grade**
Full reproducibility, extensive evaluation pipelines, and scientific rigor built-in. |
| --- | --- | --- |



**OpenEvolve vs Manual Optimization:**

| Aspect | Manual Optimization | OpenEvolve |
|--------|-------------------|------------|
| **Time to Solution** | Days to weeks | Hours |
| **Exploration Breadth** | Limited by human creativity | Unlimited LLM creativity |
| **Reproducibility** | Hard to replicate | Fully deterministic |
| **Multi-objective** | Complex tradeoffs | Automatic Pareto optimization |
| **Scaling** | Doesn't scale | Parallel evolution across islands |

## Proven Achievements



## 🚀 Quick Start

Get from zero to evolving code in **30 seconds**:

```bash
# Install OpenEvolve
pip install openevolve

# The example uses Google Gemini by default (free tier available)
# Get your API key from: https://aistudio.google.com/apikey
export OPENAI_API_KEY="your-gemini-api-key"  # Yes, use OPENAI_API_KEY env var

# Run your first evolution!
python openevolve-run.py examples/function_minimization/initial_program.py \
  examples/function_minimization/evaluator.py \
  --config examples/function_minimization/config.yaml \
  --iterations 50
```

**Note:** The example config uses Gemini by default, but you can use any OpenAI-compatible provider by modifying the `config.yaml`. See the [configs](configs/) for full configuration options.

### **Library Usage**

OpenEvolve can be used as a library without any external files:

```python
from openevolve import run_evolution, evolve_function

# Evolution with inline code (no files needed!)
result = run_evolution(
    initial_program='''
    def fibonacci(n):
        if n <= 1: return n
        return fibonacci(n-1) + fibonacci(n-2)
    ''',
    evaluator=lambda path: {"score": benchmark_fib(path)},
    iterations=100
)

# Evolve Python functions directly
def bubble_sort(arr):
    for i in range(len(arr)):
        for j in range(len(arr)-1):
            if arr[j] > arr[j+1]:
                arr[j], arr[j+1] = arr[j+1], arr[j] 
    return arr

result = evolve_function(
    bubble_sort,
    test_cases=[([3,1,2], [1,2,3]), ([5,2,8], [2,5,8])],
    iterations=50
)
print(f"Evolved sorting algorithm: {result.best_code}")
```

**Prefer Docker?** See the [Installation & Setup](#installation--setup) section for Docker options.

## See It In Action


**Circle Packing: From Random to State-of-the-Art**

**Watch OpenEvolve discover optimal circle packing in real-time:**

| Generation 1 | Generation 190 | Generation 460 (Final) |
|--------------|----------------|----------------------|
|  |  |  |
| Random placement | Learning structure | **State-of-the-art result** |

**Result**: Matches published benchmarks for n=26 circle packing problem.




**GPU Kernel Evolution**

**Before (Baseline)**:
```metal
// Standard attention implementation
kernel void attention_baseline(/* ... */) {
    // Generic matrix multiplication
    float sum = 0.0;
    for (int i = 0; i < seq_len; i++) {
        sum += query[tid] * key[i];
    }
}
```

**After Evolution (2.8x faster)**:
```metal
// OpenEvolve discovered optimization
kernel void attention_evolved(/* ... */) {
    // Hardware-aware tiling + unified memory optimization
    threadgroup float shared_mem[256];
    // ... evolved algorithm exploiting Apple Silicon architecture
}
```

**Performance Impact**: 2.8x speedup on Apple M1 Pro, maintaining numerical accuracy.



## How OpenEvolve Works

OpenEvolve implements a sophisticated **evolutionary coding pipeline** that goes far beyond simple optimization:



### **Core Innovation**: MAP-Elites + LLMs

- **Quality-Diversity Evolution**: Maintains diverse populations across feature dimensions
- **Island-Based Architecture**: Multiple populations prevent premature convergence
- **LLM Ensemble**: Multiple models with intelligent fallback strategies
- **Artifact Side-Channel**: Error feedback improves subsequent generations

### **Advanced Features**


**Scientific Reproducibility**

- **Comprehensive Seeding**: Every component (LLM, database, evaluation) is seeded
- **Default Seed=42**: Immediate reproducible results out of the box
- **Deterministic Evolution**: Exact reproduction of runs across machines
- **Component Isolation**: Hash-based isolation prevents cross-contamination




**Advanced LLM Integration**

- **Universal API**: Works with OpenAI, Google, Claude Code CLI, local models, and proxies
- **Intelligent Ensembles**: Weighted combinations with sophisticated fallback
- **Test-Time Compute**: Enhanced reasoning through proxy systems (see [OptiLLM setup](#llm-provider-setup))
- **Plugin Ecosystem**: Support for advanced reasoning plugins




**Evolution Algorithm Innovations**

- **Double Selection**: Different programs for performance vs inspiration
- **Adaptive Feature Dimensions**: Custom quality-diversity metrics
- **Migration Patterns**: Ring topology with controlled gene flow
- **Multi-Strategy Sampling**: Elite, diverse, and exploratory selection



## Perfect For

| **Use Case** | **Why OpenEvolve Excels** |
|--------------|---------------------------|
| **Performance Optimization** | Discovers hardware-specific optimizations humans miss |
| **Algorithm Discovery** | Finds novel approaches to classic problems |
| **Scientific Computing** | Automates tedious manual tuning processes |
| **Competitive Programming** | Generates multiple solution strategies |
| **Multi-Objective Problems** | Pareto-optimal solutions across dimensions |

## 🛠 Installation & Setup

### Requirements
- **Python**: 3.10+ 
- **LLM Access**: Any OpenAI-compatible API, or [Claude Code CLI](https://docs.anthropic.com/en/docs/claude-code)
- **Optional**: Docker for containerized runs

### Installation Options


**📦 PyPI (Recommended)**

```bash
pip install openevolve
```




**🔧 Development Install**

```bash
git clone https://github.com/algorithmicsuperintelligence/openevolve.git
cd openevolve
pip install -e ".[dev]"
```




**🐳 Docker**

```bash
# Pull the image
docker pull ghcr.io/algorithmicsuperintelligence/openevolve:latest

# Run an example
docker run --rm -v $(pwd):/app ghcr.io/algorithmicsuperintelligence/openevolve:latest \
  examples/function_minimization/initial_program.py \
  examples/function_minimization/evaluator.py --iterations 100
```



### Cost Estimation

**Cost depends on your LLM provider and iterations:**

- **o3**: ~$0.15-0.60 per iteration (depending on code size)
- **o3-mini**: ~$0.03-0.12 per iteration (more cost-effective)
- **Gemini-2.5-Pro**: ~$0.08-0.30 per iteration
- **Gemini-2.5-Flash**: ~$0.01-0.05 per iteration (fastest and cheapest)
- **Local models**: Nearly free after setup
- **OptiLLM**: Use cheaper models with test-time compute for better results

**Cost-saving tips:**
- Start with fewer iterations (100-200)
- Use o3-mini, Gemini-2.5-Flash or local models for exploration
- Use cascade evaluation to filter bad programs early
- Configure smaller population sizes initially

### LLM Provider Setup

OpenEvolve works with **any OpenAI-compatible API**:


**🔥 OpenAI (Direct)**

```bash
export OPENAI_API_KEY="sk-..."
# Uses OpenAI endpoints by default
```




**🤖 Google Gemini**

```yaml
# config.yaml
llm:
  api_base: "https://generativelanguage.googleapis.com/v1beta/openai/"
  model: "gemini-2.5-pro"
```

```bash
export OPENAI_API_KEY="your-gemini-api-key"
```




**🏠 Local Models (Ollama/vLLM)**

```yaml
# config.yaml
llm:
  api_base: "http://localhost:11434/v1"  # Ollama
  model: "codellama:7b"
```




**⚡ OptiLLM (Advanced)**

For maximum flexibility with rate limiting, model routing, and test-time compute:

```bash
# Install OptiLLM
pip install optillm

# Start OptiLLM proxy
optillm --port 8000

# Point OpenEvolve to OptiLLM
export OPENAI_API_KEY="your-actual-key"
```

```yaml
llm:
  api_base: "http://localhost:8000/v1"
  model: "moa&readurls-o3"  # Test-time compute + web access
```




**🔮 Claude Code CLI (No API Key)**

Use the [Claude Code CLI](https://docs.anthropic.com/en/docs/claude-code) as the LLM backend — no API keys needed, authentication uses the CLI's OAuth session.

```bash
# Install and authenticate
npm install @anthropic-ai/claude-code
claude login
```

```yaml
# config.yaml
llm:
  provider: "claude_code"
  models:
    - name: "sonnet"
      weight: 0.8
      max_tokens: 16000
      max_budget_usd: 1.0
    - name: "haiku"
      weight: 0.2
      max_tokens: 8000
```

See the [Claude Code quickstart example](examples/claude_code_quickstart/) for a complete walkthrough.



## Examples Gallery



### **Quick Example**: Function Minimization

**Watch OpenEvolve evolve from random search to sophisticated optimization:**

```python
# Initial Program (Random Search)
def minimize_function(func, bounds, max_evals=1000):
    best_x, best_val = None, float('inf')
    for _ in range(max_evals):
        x = random_point_in_bounds(bounds)
        val = func(x)
        if val < best_val:
            best_x, best_val = x, val
    return best_x, best_val
```

**Evolution Process**

```python
# Evolved Program (Simulated Annealing + Adaptive Cooling)
def minimize_function(func, bounds, max_evals=1000):
    x = random_point_in_bounds(bounds)
    temp = adaptive_initial_temperature(func, bounds)
    
    for i in range(max_evals):
        neighbor = generate_neighbor(x, temp, bounds)
        delta = func(neighbor) - func(x)
        
        if delta < 0 or random.random() < exp(-delta/temp):
            x = neighbor
            
        temp *= adaptive_cooling_rate(i, max_evals)  # Dynamic cooling
    
    return x, func(x)
```

**Performance**: 100x improvement in convergence speed!

### **Advanced Examples**


**Prompt Evolution**

**Evolve prompts instead of code** for better LLM performance. See the [LLM Prompt Optimization example](examples/llm_prompt_optimization/) for a complete case study with HotpotQA achieving +23% accuracy improvement.

[Full Example](examples/llm_prompt_optimization/)




**🏁 Competitive Programming**

**Automatic solution generation** for programming contests:

```python
# Problem: Find maximum subarray sum
# OpenEvolve discovers multiple approaches:

# Evolution Path 1: Brute Force → Kadane's Algorithm
# Evolution Path 2: Divide & Conquer → Optimized Kadane's
# Evolution Path 3: Dynamic Programming → Space-Optimized DP
```

[Online Judge Integration](examples/online_judge_programming/)



## Configuration

OpenEvolve offers extensive configuration for advanced users:

```yaml
# Advanced Configuration Example
max_iterations: 1000
random_seed: 42  # Full reproducibility

llm:
  # Ensemble configuration
  models:
    - name: "gemini-2.5-pro"
      weight: 0.6
    - name: "gemini-2.5-flash"
      weight: 0.4
  temperature: 0.7

database:
  # MAP-Elites quality-diversity
  population_size: 500
  num_islands: 5  # Parallel evolution
  migration_interval: 20
  feature_dimensions: ["complexity", "diversity", "performance"]

  # Optional novelty filtering with Gemini embeddings
  embedding_model: "gemini-embedding-001"
  similarity_threshold: 0.99

evaluator:
  enable_artifacts: true      # Error feedback to LLM
  cascade_evaluation: true    # Multi-stage testing
  use_llm_feedback: true      # AI code quality assessment

prompt:
  # Sophisticated inspiration system
  num_top_programs: 3         # Best performers
  num_diverse_programs: 2     # Creative exploration
  include_artifacts: true     # Execution feedback
  
  # Custom templates
  template_dir: "custom_prompts/"
  use_template_stochasticity: true  # Randomized prompts
```

For Gemini embeddings, set `GEMINI_API_KEY`. `GOOGLE_API_KEY` is also supported
as a fallback. OpenEvolve uses Google's OpenAI-compatible endpoint automatically.


**🎯 Feature Engineering**

**Control how programs are organized in the quality-diversity grid:**

```yaml
database:
  feature_dimensions: 
    - "complexity"      # Built-in: code length
    - "diversity"       # Built-in: structural diversity
    - "performance"     # Custom: from your evaluator
    - "memory_usage"    # Custom: from your evaluator
    
  feature_bins:
    complexity: 10      # 10 complexity levels
    performance: 20     # 20 performance buckets
    memory_usage: 15    # 15 memory usage categories
```

**Important**: Return raw values from evaluator, OpenEvolve handles binning automatically.




**🎨 Custom Prompt Templates**

**Advanced prompt engineering** with custom templates:

```yaml
prompt:
  template_dir: "custom_templates/"
  use_template_stochasticity: true
  template_variations:
    greeting:
      - "Let's enhance this code:"
      - "Time to optimize:"
      - "Improving the algorithm:"
    improvement_suggestion:
      - "Here's how we could improve this code:"
      - "I suggest the following improvements:"
      - "We can enhance this code by:"
```

**How it works:** Place `{greeting}` or `{improvement_suggestion}` placeholders in your templates, and OpenEvolve will randomly choose from the variations for each generation, adding diversity to prompts.

See [prompt examples](examples/llm_prompt_optimization/templates/) for complete template customization.



## Crafting Effective System Messages

**System messages are the secret to successful evolution.** They guide the LLM's understanding of your domain, constraints, and optimization goals. A well-crafted system message can be the difference between random mutations and targeted improvements.

### Why System Messages Matter

The system message in your config.yaml is arguably the most important component for evolution success:

- **Domain Expertise**: Provides LLM with specific knowledge about your problem space
- **Constraint Awareness**: Defines what can and cannot be changed during evolution
- **Optimization Focus**: Guides the LLM toward meaningful improvements
- **Error Prevention**: Helps avoid common pitfalls and compilation errors

### The Iterative Creation Process

Based on successful OpenEvolve implementations, system messages are best created through iteration:


**🔄 Step-by-Step Process**

**Phase 1: Initial Draft**

1. Start with a basic system message describing your goal
2. Run 20-50 evolution iterations to observe behavior
3. Note where the system gets "stuck" or makes poor choices

**Phase 2: Refinement**

4. Add specific guidance based on observed issues
5. Include domain-specific terminology and concepts
6. Define clear constraints and optimization targets
7. Run another batch of iterations

**Phase 3: Specialization**

8. Add detailed examples of good vs bad approaches
9. Include specific library/framework guidance
10. Add error avoidance patterns you've observed
11. Fine-tune based on artifact feedback

**Phase 4: Optimization**

12. Consider using OpenEvolve itself to optimize your prompt
13. Measure improvements using combined score metrics



### Examples by Complexity

#### **Simple: General Optimization**
```yaml
prompt:
  system_message: |
    You are an expert programmer specializing in optimization algorithms.
    Your task is to improve a function minimization algorithm to find the
    global minimum reliably, escaping local minima that might trap simple algorithms.
```

#### **Intermediate: Domain-Specific Guidance**
```yaml
prompt:
  system_message: |
    You are an expert prompt engineer. Your task is to revise prompts for LLMs.

    Your improvements should:
    * Clarify vague instructions and eliminate ambiguity
    * Strengthen alignment between prompt and desired task outcome
    * Improve robustness against edge cases
    * Include formatting instructions and examples where helpful
    * Avoid unnecessary verbosity

    Return only the improved prompt text without explanations.
```

#### ⚡ **Advanced: Hardware-Specific Optimization**
```yaml
prompt:
  system_message: |
    You are an expert Metal GPU programmer specializing in custom attention
    kernels for Apple Silicon.

    # TARGET: Optimize Metal Kernel for Grouped Query Attention (GQA)
    # HARDWARE: Apple M-series GPUs with unified memory architecture
    # GOAL: 5-15% performance improvement

    # OPTIMIZATION OPPORTUNITIES:
    **1. Memory Access Pattern Optimization:**
    - Coalesced access patterns for Apple Silicon
    - Vectorized loading using SIMD
    - Pre-compute frequently used indices

    **2. Algorithm Fusion:**
    - Combine max finding with score computation
    - Reduce number of passes through data

    # CONSTRAINTS - CRITICAL SAFETY RULES:
    **MUST NOT CHANGE:**
    ❌ Kernel function signature
    ❌ Template parameter names or types
    ❌ Overall algorithm correctness

    **ALLOWED TO OPTIMIZE:**
    ✅ Memory access patterns and indexing
    ✅ Computation order and efficiency
    ✅ Vectorization and SIMD utilization
    ✅ Apple Silicon specific optimizations
```

### Best Practices


**🎨 Prompt Engineering Patterns**

**Structure Your Message:** Start with role definition → Define task/context → List optimization opportunities → Set constraints → Success criteria

**Use Specific Examples:**
```yaml
# Good: "Focus on reducing memory allocations. Example: Replace `new Vector()` with pre-allocated arrays."
# Avoid: "Make the code faster"
```

**Include Domain Knowledge:**
```yaml
# Good: "For GPU kernels: 1) Memory coalescing 2) Occupancy 3) Shared memory usage"
# Avoid: "Optimize the algorithm"
```

**Set Clear Boundaries:**
```yaml
system_message: |
  MUST NOT CHANGE: ❌ Function signatures ❌ Algorithm correctness ❌ External API
  ALLOWED: ✅ Internal implementation ✅ Data structures ✅ Performance optimizations
```




**🔬 Advanced Techniques**

**Artifact-Driven Iteration:** Enable artifacts in config → Include common error patterns in system message → Add guidance based on stderr/warning patterns

**Multi-Phase Evolution:** Start broad ("Explore different algorithmic approaches"), then focus ("Given successful simulated annealing, focus on parameter tuning")

**Template Stochasticity:** See the [Configuration section](#configuration) for complete template variation examples.



### Meta-Evolution: Using OpenEvolve to Optimize Prompts

**You can use OpenEvolve to evolve your system messages themselves!** This powerful technique lets you optimize prompts for better LLM performance automatically.

See the [LLM Prompt Optimization example](examples/llm_prompt_optimization/) for a complete implementation, including the HotpotQA case study with +23% accuracy improvement.

### Common Pitfalls to Avoid

- **Too Vague**: "Make the code better" → Specify exactly what "better" means
- **Too Restrictive**: Over-constraining can prevent useful optimizations
- **Missing Context**: Include relevant domain knowledge and terminology
- **No Examples**: Concrete examples guide LLM better than abstract descriptions
- **Ignoring Artifacts**: Don't refine prompts based on error feedback

## Artifacts & Debugging

**Artifacts side-channel** provides rich feedback to accelerate evolution:

```python
# Evaluator can return execution context
from openevolve.evaluation_result import EvaluationResult

return EvaluationResult(
    metrics={"performance": 0.85, "correctness": 1.0},
    artifacts={
        "stderr": "Warning: suboptimal memory access pattern",
        "profiling_data": {...},
        "llm_feedback": "Code is correct but could use better variable names",
        "build_warnings": ["unused variable x"]
    }
)
```

**Next generation prompt automatically includes:**

```markdown
## Previous Execution Feedback
⚠️ Warning: suboptimal memory access pattern
💡 LLM Feedback: Code is correct but could use better variable names
🔧 Build Warnings: unused variable x
```

This creates a **feedback loop** where each generation learns from previous mistakes!

## Visualization

**Real-time evolution tracking** with interactive web interface:

```bash
# Install visualization dependencies
pip install -r scripts/requirements.txt

# Launch interactive visualizer
python scripts/visualizer.py

# Or visualize specific checkpoint
python scripts/visualizer.py --path examples/function_minimization/openevolve_output/checkpoints/checkpoint_100/
```

**Features:**

- 🌳 **Evolution tree** with parent-child relationships
- 📈 **Performance tracking** across generations
- 🔍 **Code diff viewer** showing mutations
- 📊 **MAP-Elites grid** visualization
- 🎯 **Multi-metric analysis** with custom dimensions



## Roadmap

### **🔥 Upcoming Features**

- [ ] **Multi-Modal Evolution**: Images, audio, and text simultaneously
- [ ] **Federated Learning**: Distributed evolution across multiple machines  
- [ ] **AutoML Integration**: Hyperparameter and architecture evolution
- [ ] **Benchmark Suite**: Standardized evaluation across domains

### **🌟 Research Directions**

- [ ] **Self-Modifying Prompts**: Evolution modifies its own prompting strategy
- [ ] **Cross-Language Evolution**: Python → Rust → C++ optimization chains
- [ ] **Neurosymbolic Reasoning**: Combine neural and symbolic approaches
- [ ] **Human-AI Collaboration**: Interactive evolution with human feedback

Want to contribute? Check out our [roadmap discussions](https://github.com/algorithmicsuperintelligence/openevolve/discussions/categories/roadmap)!

## FAQ


**💰 How much does it cost to run?**

See the [Cost Estimation](#cost-estimation) section in Installation & Setup for detailed pricing information and cost-saving tips.




**🆚 How does this compare to manual optimization?**

| Aspect | Manual | OpenEvolve |
|--------|--------|------------|
| **Initial Learning** | Weeks to understand domain | Minutes to start |
| **Solution Quality** | Depends on expertise | Consistently explores novel approaches |
| **Time Investment** | Days-weeks per optimization | Hours for complete evolution |
| **Reproducibility** | Hard to replicate exact process | Perfect reproduction with seeds |
| **Scaling** | Doesn't scale beyond human capacity | Parallel evolution across islands |

**OpenEvolve shines** when you need to explore large solution spaces or optimize for multiple objectives simultaneously.




**🔧 Can I use my own LLM?**

**Yes!** OpenEvolve supports any OpenAI-compatible API:

- **Commercial**: OpenAI, Google, Cohere
- **Local**: Ollama, vLLM, LM Studio, text-generation-webui
- **Advanced**: OptiLLM for routing and test-time compute

Just set the `api_base` in your config to point to your endpoint.




**🚨 What if evolution gets stuck?**

**Built-in mechanisms prevent stagnation:**

- **Island migration**: Fresh genes from other populations
- **Temperature control**: Exploration vs exploitation balance
- **Diversity maintenance**: MAP-Elites prevents convergence
- **Artifact feedback**: Error messages guide improvements
- **Template stochasticity**: Randomized prompts break patterns

**Manual interventions:**
- Increase `num_diverse_programs` for more exploration
- Add custom feature dimensions to diversify search
- Use template variations to randomize prompts
- Adjust migration intervals for more cross-pollination




**📈 How do I measure success?**

**Multiple success metrics:**

1. **Primary Metric**: Your evaluator's `combined_score` or metric average
2. **Convergence**: Best score improvement over time
3. **Diversity**: MAP-Elites grid coverage
4. **Efficiency**: Iterations to reach target performance
5. **Robustness**: Performance across different test cases

**Use the visualizer** to track all metrics in real-time and identify when evolution has converged.



### **Contributors**

Thanks to all our amazing contributors who make OpenEvolve possible!

<a href="https://github.com/algorithmicsuperintelligence/openevolve/graphs/contributors">
  
</a>

### **Contributing**

We welcome contributions! Here's how to get started:

1. 🍴 **Fork** the repository
2. 🌿 **Create** your feature branch: `git checkout -b feat-amazing-feature`
3. ✨ **Add** your changes and tests
4. ✅ **Test** everything: `python -m unittest discover tests`
5. 📝 **Commit** with a clear message
6. 🚀 **Push** and create a Pull Request

**New to open source?** Check out our [Contributing Guide](CONTRIBUTING.md) and look for [`good-first-issue`](https://github.com/algorithmicsuperintelligence/openevolve/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) labels!

### **Academic & Research**

**Articles & Blog Posts About OpenEvolve**:
- [Towards Open Evolutionary Agents](https://huggingface.co/blog/driaforall/towards-open-evolutionary-agents) - Evolution of coding agents and the open-source movement
- [OpenEvolve: GPU Kernel Discovery](https://huggingface.co/blog/codelion/openevolve-gpu-kernel-discovery) - Automated discovery of optimized GPU kernels
- [OpenEvolve: Evolutionary Coding with LLMs](https://huggingface.co/blog/codelion/openevolve) - Introduction to evolutionary algorithm discovery using large language models

## Citation

If you use OpenEvolve in your research, please cite:

```bibtex
@software{openevolve,
  title = {OpenEvolve: an open-source evolutionary coding agent},
  author = {Asankhaya Sharma},
  year = {2025},
  publisher = {GitHub},
  url = {https://github.com/algorithmicsuperintelligence/openevolve}
}
```
---

---

## File: configs/README.md

# OpenEvolve Configuration Files

This directory contains configuration files for OpenEvolve with examples for different use cases.

## Configuration Files

### `default_config.yaml`
The main configuration file containing all available options with sensible defaults. This file includes:
- Complete documentation for all configuration parameters
- Default values for all settings
- **Island-based evolution parameters** for proper evolutionary diversity

Use this file as a template for your own configurations.

### `island_config_example.yaml`
A practical example configuration demonstrating proper island-based evolution setup. Shows:
- Recommended island settings for most use cases
- Balanced migration parameters
- Complete working configuration

### `island_examples.yaml`
Multiple example configurations for different scenarios:
- **Maximum Diversity**: Many islands, frequent migration
- **Focused Exploration**: Few islands, rare migration  
- **Balanced Approach**: Default recommended settings
- **Quick Exploration**: Small-scale rapid testing
- **Large-Scale Evolution**: Complex optimization runs

Includes guidelines for choosing parameters based on your problem characteristics.

## Island-Based Evolution Parameters

The key new parameters for proper evolutionary diversity are:

```yaml
database:
  num_islands: 5                      # Number of separate populations
  migration_interval: 50              # Migrate every N generations  
  migration_rate: 0.1                 # Fraction of top programs to migrate
```

### Parameter Guidelines

- **num_islands**: 3-10 for most problems (more = more diversity)
- **migration_interval**: 25-100 generations (higher = more independence)
- **migration_rate**: 0.05-0.2 (5%-20%, higher = faster knowledge sharing)

### When to Use What

- **Complex problems** → More islands, less frequent migration
- **Simple problems** → Fewer islands, more frequent migration
- **Long runs** → More islands to maintain diversity
- **Short runs** → Fewer islands for faster convergence

## Usage

Copy any of these files as a starting point for your configuration:

```bash
cp configs/default_config.yaml my_config.yaml
# Edit my_config.yaml for your specific needs
```

Then use with OpenEvolve:

```python
from openevolve import OpenEvolve
evolve = OpenEvolve(
    initial_program_path="program.py",
    evaluation_file="evaluator.py", 
    config_path="my_config.yaml"
)
```

---

## File: examples/r_robust_regression/README.md

# R Robust Regression Evolution

This example demonstrates how to use OpenEvolve with the R programming language. The example focuses on evolving robust regression algorithms that can handle outliers in data, showcasing OpenEvolve's ability to work with statistical computing languages beyond Python.

## Files

- `initial_program.r`: Starting R implementation with basic least squares regression
- `evaluator.py`: Python evaluator that runs R code and measures performance
- `config.yaml`: Configuration optimized for statistical algorithm evolution
- `requirements.txt`: Dependencies for both R and Python components

## Prerequisites

### R Dependencies
Install R (version 3.6 or higher) and the required packages:

```r
install.packages(c("jsonlite"))
```

### Python Dependencies
```bash
pip install -r requirements.txt
```

## Usage

Run the evolution process:

```bash
cd examples/r_robust_regression
python ../../openevolve-run.py initial_program.r evaluator.py --config config.yaml --iterations 100
```

This example shows how OpenEvolve can evolve algorithms in R by starting with a basic least squares implementation and improving it to handle outliers through various robust regression techniques.

---

## File: examples/online_judge_programming/README.md

# Online Judge Programming Example

This example demonstrates how OpenEvolve can solve programming problems and pass all test cases on [Kattis online judge](https://open.kattis.com/) starting from scratch.

## Problem Description

We take the [Alphabet](https://open.kattis.com/problems/alphabet) problem from [Kattis](https://open.kattis.com/) as following:
```markdown
A string of lowercase letters is called **alphabetical** if some of the letters can be deleted so that the only letters that remain are the letters from 'a' to 'z' in order. Given a string s, determine the minimum number of letters to add anywhere in the string to make it alphabetical.

Input:
Each input will consist of a single test case. Note that your program may be run multiple times on different inputs. The only line of input contains a string s (1 ≤ |s| ≤ 50) which contains only lowercase letters.

Output:
Output a single integer, which is the smallest number of letters needed to add to `s` to make it alphabetical.

Sample Input 1:
xyzabcdefghijklmnopqrstuvw

Sample Output 1:
3

Sample Input 2:
aiemckgobjfndlhp

Sample Output 2:
20
```

## Getting Started

First, fill your username and token in `example.kattisrc` according to your personal configuration file (must be logged in) from [Kattis](https://open.kattis.com/download/kattisrc) and rename the file as `.kittisrc`.

Then, to run this example:

```bash
cd examples/online_judge_programming
python ../../openevolve-run.py initial_program.py evaluator.py --config config.yaml
```

## Algorithm Evolution

### Initial Algorithm (dummy output)

The initial implementation was a simple dummy output that returned 0 directly.

```python
import sys
for line in sys.stdin:
    s = line.strip()

ans = 0
print(ans)
```

### Evolved Algorithm (Dynamic Programming)

After running OpenEvolve for just 4 iterations, it discovered a dynamic programming algorithm that passes all test cases on Kattis:

```python
import sys

for line in sys.stdin:
    s = line.strip()

n = len(s)
dp = [1] * n

for i in range(1, n):
    for j in range(i):
        if s[i] > s[j]:
            dp[i] = max(dp[i], dp[j] + 1)

longest_alphabetical_subsequence_length = max(dp)
ans = 26 - longest_alphabetical_subsequence_length
print(ans)
```

## Next Steps

Try modifying the config.yaml file to:
- Change the programming problem in system prompt
- Change the LLM model configuration

---

## File: examples/mlx_metal_kernel_opt/README.md

# MLX Metal Kernel Optimization Example

This example uses OpenEvolve to automatically discover optimized Metal GPU kernels for Grouped Query Attention (GQA) in Qwen3-0.6B on Apple Silicon.

## Target Configuration

- **Model**: Qwen3-0.6B-bf16
- **Architecture**: 16 query heads : 8 KV heads (2:1 ratio), 2048 hidden size, 128 head dimension
- **Hardware**: Apple M-series GPUs with unified memory
- **Baseline**: `mx.fast.scaled_dot_product_attention` via `mlx_lm.generate`
- **Goal**: Evolve custom Metal kernel source code to outperform baseline

## Quick Start

### Prerequisites

```bash
pip install mlx mlx-lm openevolve

# Set API key (Gemini via OpenAI-compatible endpoint)
export OPENAI_API_KEY="your-gemini-key"
```

### Run Evolution

```bash
cd openevolve/examples/mlx_metal_kernel_opt

# Using the experiment runner script
./run_evolve_experiment.sh --run-name test_run --iterations 25

# Or directly
python -m openevolve.cli \
    --initial-program initial_program.py \
    --evaluator evaluator.py \
    --config config.yaml \
    --iterations 25 \
    --output ./openevolve_output
```

### Verify Evaluation Validity

```bash
# Run a single benchmark with verbose output
python -c "
from evaluator import Qwen3GQAEvaluator
e = Qwen3GQAEvaluator()
result = e.evaluate('initial_program.py')
print(result['summary'])
"
```

## Files

| File | Purpose |
| ---- | ------- |
| `initial_program.py` | Starting Metal kernel (to be evolved) |
| `evaluator.py` | Correctness + performance evaluation |
| `config.yaml` | Evolution configuration |
| `qwen3_benchmark_suite.py` | Benchmark definitions |
| `mlx_lm_generate_with_hook.py` | Subprocess hook wrapper |
| `run_evolve_experiment.sh` | Experiment runner script |

## Validity Fixes (This PR)

This PR corrects critical issues that invalidated prior evaluation results:

1. **Subprocess Kernel Hook**: Evolved kernels are now properly applied in benchmark subprocesses via `mlx_lm_generate_with_hook.py`

2. **bfloat16 Correctness Gate**: Correctness tests now use `mx.bfloat16` inputs to match actual inference dtype

3. **Architecture Alignment**: Fixed head ratio from 40:8 to correct 16:8 (2:1 GQA pattern)

4. **Evaluation Flow Optimizations**: Early exit on compilation errors, correctness-before-baseline ordering, GPU state cleanup between runs

## Current Status

After fixing validity issues, we ran 25 evolution iterations.

**Result: The best evolved kernel is 3.2% SLOWER than MLX's baseline implementation.**

The evolution improved from an initial -11.5% regression to -3.2%, but never exceeded baseline. This indicates fundamental limitations in the current evolution mechanism that require further investigation.

For detailed experiment results and analysis, see [EVOLUTION_ANALYSIS.md](./EVOLUTION_ANALYSIS.md).

### Demo Results (Committed)

For review and reproducibility, this example repo includes a committed snapshot of one post-fix evolution run:

- `best_program.py`: best evolved program (iteration 23)
- `best_program_info.json`: metrics + baseline comparisons (includes the -3.2% result)

The full run output directory is intentionally git-ignored (see `.gitignore`) to avoid committing large run artifacts.

### Known Limitations

1. MAP-Elites selection uses abstract `combined_score` instead of direct speedup ratios
2. LLM context underutilized (only 1 parent + 5 samples per iteration)
3. No GPU profiling data to guide optimization
4. 32% bf16 compilation failure rate

## References

- [OpenEvolve](https://github.com/codelion/openevolve)
- [MLX](https://github.com/ml-explore/mlx)
- [MLX-LM](https://github.com/ml-explore/mlx-examples)
- [KernelBench](https://github.com/ScalingIntelligence/KernelBench)

---

## File: examples/lm_eval/README.md

# lm-eval.py

`lm-eval.py` provides basic benchmark capability for LLM feedback-based evolutionary task solving. The benchmark framework is [EleutherAI's lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness).

*Limitation:* Only generation-only tasks such as gsm8k are supported. This is because tasks that require loglikelihood probabilities are not well applicable to agents.

## Usage

```bash
$ python3 examples/lm_eval/lm-eval.py -h
usage: lm-eval.py [-h] [--config CONFIG] [--init_file INIT_FILE] [--evaluator_file EVALUATOR_FILE] [--iterations ITERATIONS] [--limit LIMIT] [--tasks TASKS]
                  [--output_path OUTPUT_PATH]

OpenEvolve <-> lm-evaluation-harness adapter.

options:
  -h, --help            show this help message and exit
  --config CONFIG       config file
  --init_file INIT_FILE
                        initial content file
  --evaluator_file EVALUATOR_FILE
                        evaluator file
  --iterations ITERATIONS
                        number of iterations
  --limit LIMIT         limit the number of examples per task that are executed
  --tasks TASKS         list of tasks to evaluate
  --output_path OUTPUT_PATH
                        output path for results
```

Early examples that **were meant to** indicate that more evolution iterations improve task performance -- I suspect the prompting may not be ideal yet:
```
$ python3 examples/lm_eval/lm-eval.py --tasks gsm8k --limit 10 --iterations 1
[..]
Headline metrics:
  gsm8k           exact_match,strict-match 80.000%
[..]


$ python3 examples/lm_eval/lm-eval.py --tasks gsm8k --limit 10 --iterations 3
[..]
Headline metrics:
  gsm8k           exact_match,strict-match 90.000%
[..]

$ python3 examples/lm_eval/lm-eval.py --tasks gsm8k --limit 10 --iterations 10
[..]
Headline metrics:
  gsm8k           exact_match,strict-match 80.000%
[..]

$ python3 examples/lm_eval/lm-eval.py --tasks gsm8k --limit 10 --iterations 15
[..]
Headline metrics:
  gsm8k           exact_match,strict-match 70.000%
[..]
```

## Warning

- Be aware that this is an early implementation. No extensive benchmarks have been executed so far. With a limit to 10 tasks and 10 iterations, the benchmark is meaningless as is.
- Use the --limit parameter only for tests, not for metric generation.
- Do not cite the metrics that result from the script execution blindly without reviewing the solution first.

## References

```bibtex
@misc{eval-harness,
    author       = {Gao, Leo and Tow, Jonathan and Abbasi, Baber and Biderman, Stella and Black, Sid and DiPofi, Anthony and Foster, Charles and Golding, Laurence and Hsu, Jeffrey and Le Noac'h, Alain and Li, Haonan and McDonell, Kyle and Muennighoff, Niklas and Ociepa, Chris and Phang, Jason and Reynolds, Laria and Schoelkopf, Hailey and Skowron, Aviya and Sutawika, Lintang and Tang, Eric and Thite, Anish and Wang, Ben and Wang, Kevin and Zou, Andy},
    title        = {The Language Model Evaluation Harness},
    month        = 07,
    year         = 2024,
    publisher    = {Zenodo},
    version      = {v0.4.3},
    doi          = {10.5281/zenodo.12608602},
    url          = {https://zenodo.org/records/12608602}
}
```

---

## File: examples/llm_prompt_optimization/README.md

# LLM Prompt Optimization with OpenEvolve 🚀

This example demonstrates how to use OpenEvolve to automatically optimize prompts for Large Language Models across various benchmark datasets. The system uses evolutionary search to discover high-performing prompts, achieving significant improvements across multiple tasks.

## 📊 Latest Performance Results (GEPA Benchmarks)

OpenEvolve successfully improved prompt performance across three challenging GEPA benchmarks:

| Dataset | Baseline Accuracy | Evolved Accuracy | Improvement | Samples |
|---------|------------------|------------------|-------------|---------|
| **IFEval** | 95.01% | 97.41% | **+2.40%** ✅ | 541 |
| **HoVer** | 43.83% | 42.90% | -0.93% | 4,000 |
| **HotpotQA** | 77.93% | 88.62% | **+10.69%** ✅ | 7,405 |
| **Overall** | 67.29% | 73.71% | **+6.42%** ✅ | 11,946 |

### Key Achievements:
- **767 more correct answers** across all datasets
- **38% fewer empty responses** with evolved prompts
- **Near-perfect performance** on instruction following (IFEval: 97.41%)
- **Major improvement** in multi-hop reasoning (HotpotQA: 88.62%)

## 🎯 Overview

OpenEvolve automatically:
- Evolves prompts through multiple generations using LLMs
- Uses cascading evaluation for efficient testing
- Employs MAP-Elites algorithm to maintain diversity
- Incorporates LLM feedback for qualitative assessment
- Supports various datasets from HuggingFace

## 📊 All Supported Datasets

### GEPA Benchmarks (Latest Focus)

#### IFEval (Instruction Following Eval)
- **Task**: Follow complex, multi-constraint instructions
- **Size**: 541 samples (train split)
- **Metric**: Binary success on instruction adherence
- **Results**: 95.01% → 97.41% (+2.40%)
- **Config**: `ifeval_prompt_dataset.yaml`

#### HoVer (Claim Verification)
- **Task**: Verify claims as SUPPORTED or NOT_SUPPORTED
- **Size**: 4,000 samples (validation split)
- **Metric**: Binary classification accuracy
- **Results**: 43.83% → 42.90% (-0.93%)
- **Config**: `hover_prompt_dataset.yaml`
- **Note**: Uses integer labels (0=SUPPORTED, 1=NOT_SUPPORTED)

#### HotpotQA (Multi-hop Question Answering)
- **Task**: Answer questions requiring reasoning over multiple paragraphs
- **Size**: 7,405 samples (validation split)
- **Metric**: Exact match with answer
- **Results**: 77.93% → 88.62% (+10.69%)
- **Config**: `hotpotqa_prompt_dataset.yaml`

### Additional Datasets (Earlier Experiments)

#### Emotion Classification
- **Task**: Classify emotions in text (6 classes)
- **Dataset**: `dair-ai/emotion`
- **Config**: `emotion_prompt_dataset.yaml`
- **Benchmark**: Compared against DSPy results

#### GSM8K (Grade School Math)
- **Task**: Solve grade school math word problems
- **Dataset**: `gsm8k`
- **Config**: `gsm8k_prompt_dataset.yaml`
- **Benchmark**: DSPy achieves 97.1%

#### IMDB Sentiment Analysis
- **Task**: Binary sentiment classification
- **Dataset**: `stanfordnlp/imdb`
- **Config**: `initial_prompt_dataset.yaml`
- **Example Evolution**: 72% → 94% accuracy

## 🚀 Quick Start

### 1. Install Dependencies

```bash
cd examples/llm_prompt_optimization
pip install -r requirements.txt
```

### 2. Set Your API Key

```bash
export OPENAI_API_KEY="your_openrouter_api_key"
```

Note: Despite the variable name, this uses OpenRouter API. Get your key at https://openrouter.ai/

### 3. Evaluate Prompts

Use the unified evaluation script to test baseline or evolved prompts:

```bash
# Evaluate baseline prompts on a single dataset
python evaluate_prompts.py --dataset ifeval --prompt-type baseline --samples 100

# Evaluate evolved prompts on a single dataset
python evaluate_prompts.py --dataset hover --prompt-type evolved --samples 100

# Evaluate all GEPA datasets with evolved prompts (full dataset)
python evaluate_prompts.py --dataset all --prompt-type evolved

# Specify output file
python evaluate_prompts.py --dataset all --prompt-type evolved --output results.json
```

### 4. Run Evolution

To evolve prompts from scratch:

```bash
# For GEPA benchmarks
python ../../openevolve-run.py ifeval_prompt.txt evaluator.py \
  --config config_qwen3_evolution.yaml \
  --iterations 50

# For other datasets (using wrapper script)
./run_evolution.sh emotion_prompt.txt --iterations 50
./run_evolution.sh gsm8k_prompt.txt --iterations 100
```

## ⚙️ Configuration Files

### Evolution Configurations

#### GEPA Benchmarks (`config_qwen3_evolution.yaml`)
```yaml
llm:
  models:
    - name: "qwen/qwen3-8b"
      weight: 1.0
  temperature: 0.7
  max_tokens: 4096

evaluator:
  cascade_evaluation: true
  cascade_thresholds: [0.9]  # 2-stage evaluation
  timeout: 1800  # 30 minutes
  use_llm_feedback: true
  llm_feedback_weight: 0.3

database:
  n_islands: 4  # Island-based evolution
  migration_interval: 10
```

#### General Configuration (`config.yaml`)
```yaml
llm:
  api_base: "https://openrouter.ai/api/v1"
  models:
    - name: "google/gemini-2.5-flash"
      weight: 1.0
```

### Dataset Configurations

Each dataset has its own configuration file following the pattern `*_prompt_dataset.yaml`:

```yaml
# Example: ifeval_prompt_dataset.yaml
dataset_name: "google/IFEval"
input_field: "prompt"
target_field: "instruction_id_list"
split: "train"
is_ifeval: true  # Special handling flag
```

## 🧬 Evolution Process

### How It Works

1. **Initial Population**: Start with baseline prompt
2. **Variation**: LLM generates prompt mutations
3. **Evaluation**: Test on dataset samples (10 for Stage 1, 40 for Stage 2)
4. **Selection**: Keep best performers based on combined score
5. **Island Evolution**: 4 isolated populations with periodic migration
6. **Iteration**: Repeat for specified generations (typically 50-100)

### Cascade Evaluation

- **Stage 1**: Quick test on 10 samples (must achieve 90% to proceed)
- **Stage 2**: Comprehensive test on 40 samples
- **Combined Score**: 70% task accuracy + 30% LLM feedback

### LLM Feedback Metrics

Evolved prompts are evaluated on:
- **Clarity**: Unambiguous instructions
- **Specificity**: Appropriate detail level
- **Robustness**: Edge case handling
- **Format Specification**: Clear output requirements

## 📁 Complete File Structure

```
llm_prompt_optimization/
├── evaluate_prompts.py          # Unified evaluation script
├── evaluator.py                 # OpenEvolve evaluator
├── run_evolution.sh             # Wrapper script for evolution
│
├── Configuration Files
│   ├── config.yaml              # General LLM config
│   ├── config_qwen3_evolution.yaml  # GEPA evolution config
│   └── config_qwen3_baseline.yaml   # GEPA baseline config
│
├── Dataset Configurations & Prompts
│   ├── ifeval_prompt.txt & ifeval_prompt_dataset.yaml
│   ├── hover_prompt.txt & hover_prompt_dataset.yaml
│   ├── hotpotqa_prompt.txt & hotpotqa_prompt_dataset.yaml
│   ├── emotion_prompt.txt & emotion_prompt_dataset.yaml
│   ├── gsm8k_prompt.txt & gsm8k_prompt_dataset.yaml
│   └── initial_prompt.txt & initial_prompt_dataset.yaml
│
├── Evolution Templates
│   └── templates/
│       ├── full_rewrite_user.txt
│       ├── evaluation.txt
│       └── evaluator_system_message.txt
│
├── Results
│   ├── evaluation_results_baseline_20250809_070942.json
│   ├── evaluation_results_evolved_20250809_103002.json
│   └── openevolve_output_qwen3_*/
│       └── best/
│           └── best_program.txt     # Evolved prompt
│
└── requirements.txt
```

## 🔍 Example Evolved Prompts

### IFEval (97.41% accuracy)
```
Follow the instruction below precisely. Structure your response into two 
distinct parts: 1) a step-by-step reasoning process that explicitly 
identifies the task, constraints, and required output format, and 2) the 
final answer in the exact format specified...
```

### HotpotQA (88.62% accuracy)
```
Answer the following question using the provided context. The answer must 
integrate information from multiple paragraphs and follow these steps:
1. Paragraph Analysis: Extract key details from each relevant paragraph...
2. Synthesis: Combine these details into a single, coherent response...
3. Citation: Attribute all assertions to their source paragraphs...
```

### IMDB Sentiment (Example Evolution)
Starting prompt:
```
Analyze the sentiment: "{input_text}"
```

Evolved prompt after 100 iterations:
```
Analyze the sentiment of the following text. Determine if the overall 
emotional tone is positive or negative.

Text: "{input_text}"

Response: Provide only a single digit - either 1 for positive sentiment 
or 0 for negative sentiment. Do not include any explanation or additional text.
```
Accuracy improvement: 72% → 94%

## 🐛 Troubleshooting

### HoVer Dataset Issues
- **Problem**: Test split has no labels (all -1)
- **Solution**: Use validation split (configured automatically)
- **Labels**: Integer format (0=SUPPORTED, 1=NOT_SUPPORTED)

### Empty Responses
- **Cause**: Complex evolved prompts exceeding token limits
- **Solution**: Increase max_tokens in evaluation or simplify prompts

### Slow Evaluation
- **IFEval**: ~1 minute per 100 samples
- **HoVer**: ~30 minutes for full dataset
- **HotpotQA**: ~45 minutes for full dataset
- **Tip**: Use --samples flag for faster testing

### Dataset Not Found
- Check the exact dataset name and source
- Some datasets require acceptance of terms
- Use `trust_remote_code=True` for certain datasets

## 🚀 Advanced Usage

### Custom Datasets

To add a new dataset:

1. Create initial prompt: `mydataset_prompt.txt`
2. Create configuration: `mydataset_prompt_dataset.yaml`
3. Run evolution: 
   ```bash
   ./run_evolution.sh mydataset_prompt.txt --iterations 50
   # or directly:
   python ../../openevolve-run.py mydataset_prompt.txt evaluator.py --config config.yaml
   ```

### Batch Evaluation

Evaluate multiple configurations:

```bash
# Create a script to run multiple evaluations
for dataset in ifeval hover hotpotqa; do
    python evaluate_prompts.py --dataset $dataset --prompt-type evolved
done
```

### Resume Evolution

Continue from a checkpoint:

```bash
python ../../openevolve-run.py prompt.txt evaluator.py \
  --config config_qwen3_evolution.yaml \
  --checkpoint openevolve_output_qwen3_ifeval/checkpoints/checkpoint_30 \
  --iterations 20
```

### Custom Templates

The `templates/` directory contains customizable templates for prompt evolution:
- `full_rewrite_user.txt`: Instructions for prompt rewriting
- `evaluation.txt`: LLM feedback template
- `evaluator_system_message.txt`: System message for evaluation

## 📈 Tips for Best Results

1. **Start Simple**: Begin with clear, working baseline prompts
2. **Sufficient Samples**: Use at least 40 samples for Stage 2 evaluation
3. **Monitor Progress**: Check `openevolve_output_*/logs/` for progress
4. **Multiple Runs**: Evolution has randomness; try multiple runs
5. **Token Limits**: Ensure max_tokens accommodates prompt + response
6. **Dataset Variety**: Test on multiple datasets to ensure generalization

## 📚 References

- [OpenEvolve Documentation](../../README.md)
- [IFEval Paper](https://arxiv.org/abs/2311.07911)
- [HoVer Dataset](https://hover-nlp.github.io/)
- [HotpotQA Paper](https://arxiv.org/abs/1809.09600)
- [GSM8K Dataset](https://github.com/openai/grade-school-math)
- [DSPy Framework](https://github.com/stanfordnlp/dspy)
- [OpenRouter API](https://openrouter.ai/docs)

Happy prompt evolving! 🧬✨

---

## File: examples/k_module_problem/README.md

# K-Module Problem: Evolution vs Iterative Refinement

This example demonstrates a fundamental limitation of iterative refinement approaches and shows how evolutionary search with population-based exploration can solve problems that defeat single-trajectory optimization.

## The Problem

The K-Module Problem is a pipeline configuration task where you must find the correct combination of 4 independent modules:

| Module | Options |
|--------|---------|
| **loader** | csv_reader, json_reader, xml_reader, parquet_reader, sql_reader |
| **preprocess** | normalize, standardize, minmax, scale, none |
| **algorithm** | quicksort, mergesort, heapsort, bubblesort, insertion |
| **formatter** | json, xml, csv, yaml, protobuf |

**Search space**: 5⁴ = 625 possible combinations

**The Challenge**: The evaluator only tells you *how many* modules are correct (0-4), not *which ones*. This creates a deceptive fitness landscape with no gradient information.

## Why Iterative Refinement Fails

Consider this scenario:

```
Initial:  [json_reader, standardize, mergesort, xml]     → Score: 0/4
Refine 1: [csv_reader, standardize, mergesort, xml]      → Score: 1/4 ✓
Refine 2: [csv_reader, normalize, mergesort, xml]        → Score: 2/4 ✓✓
Refine 3: [csv_reader, normalize, heapsort, xml]         → Score: 1/4 ✗ (went backwards!)
Refine 4: [csv_reader, normalize, mergesort, json]       → Score: 2/4 (no progress)
```

**The Problem**: When the model changes `mergesort` to `heapsort`, it has no way to know this was wrong because:
- The score decreased, but was that because of the algorithm change?
- Or because `normalize` wasn't actually correct?
- The model can't tell which modules are contributing to the score

This leads to **random walk behavior** requiring O(625) evaluations on average.

## Why Evolution Succeeds

Evolution maintains a **population** that explores different regions simultaneously:

```
Generation 1:
  Individual A: [csv_reader, scale, quicksort, xml]      → 2/4 (loader, algorithm correct)
  Individual B: [json_reader, normalize, bubble, json]   → 2/4 (preprocess, formatter correct)
  Individual C: [xml_reader, minmax, mergesort, csv]     → 0/4

Generation 2 (crossover):
  Child(A,B): [csv_reader, normalize, quicksort, json]   → 4/4 SUCCESS!
```

**Key insight**: Evolution discovers correct modules in different individuals and **crossover combines them**. This is the "Building Block Hypothesis" - complex solutions are assembled from simpler discovered components.

## Running the Experiment

### Prerequisites

1. **OpenEvolve** (this repo):
   ```bash
   pip install -e .
   ```

2. **API Key** (both methods use the same model via OpenRouter for fair comparison):
   ```bash
   export OPENROUTER_API_KEY=your_key
   ```

Both OpenEvolve and the iterative agent use `google/gemini-2.5-flash-lite` via OpenRouter API for a fair comparison.

### Run OpenEvolve

```bash
cd examples/k_module_problem
chmod +x run_openevolve.sh
./run_openevolve.sh 50
```

Or directly:
```bash
openevolve-run initial_program.py evaluator.py --config config.yaml --iterations 50
```

### Run Iterative Agent

```bash
python iterative_agent.py --iterations 100
```

Or run multiple trials to get statistics:
```bash
python run_iterative_trials.py --trials 3 --iterations 100
```

### Run Random Baseline

Establish the "no learning" baseline (no LLM needed):

```bash
python run_random_baseline.py --samples 100 --trials 100
```

This runs 100 independent trials of random search, each with up to 100 samples, and calculates empirical pass@k metrics.

### Compare Results

```bash
python compare_results.py
```

This generates:
- `comparison_plot.png`: Visual comparison of convergence
- Summary statistics printed to console

## Experimental Results

### Random Baseline (100 trials, 100 samples each)

| Metric | Value |
|--------|-------|
| **Success rate (pass@100)** | 16% (16/100 trials found solution) |
| **Avg samples to solution** | 43.3 (when found) |
| **Min samples** | 5 (lucky guess) |
| **Max samples** | 91 |

This establishes the "no learning" baseline. Any method that beats this is demonstrating actual optimization, not just random sampling.

### Iterative Refinement Results (3 trials, 100 iterations max)

| Trial | Iterations | Result | Best Score |
|-------|------------|--------|------------|
| 1 | 100 | FAILED | 75% (3/4) |
| 2 | 100 | FAILED | 75% (3/4) |
| 3 | 13 | SUCCESS | 100% (4/4) |

**Summary:**
- **Success rate**: 33% (1/3 trials found solution)
- **When successful**: 13 iterations
- **Failure mode**: Gets stuck at 75% - keeps trying `standardize` instead of `normalize`

**Key observation**: The iterative agent repeatedly finds configurations with 3/4 correct modules (`csv_reader`, `quicksort`, `json`) but cannot identify that `preprocess` is the wrong module. It keeps cycling through variations without escaping this local optimum.

### OpenEvolve (Evolutionary) Results (3 trials, 100 iterations max)

| Trial | Iterations | Result | Best Score |
|-------|------------|--------|------------|
| 1 | 18 | SUCCESS | 100% (4/4) |
| 2 | 50 | SUCCESS | 100% (4/4) |
| 3 | 89 | SUCCESS | 100% (4/4) |

**Summary:**
- **Success rate**: 100% (3/3 trials found solution)
- **Avg iterations to solution**: 52.3
- **Min iterations**: 18
- **Max iterations**: 89

**Key advantage**: OpenEvolve's population-based approach maintains diverse configurations that explore different module combinations in parallel. Even when some individuals get stuck at local optima (75% with wrong preprocessing), others explore alternatives and eventually discover the correct solution.

### Comparison Summary

| Method | Success Rate | Avg Iterations | Key Finding |
|--------|-------------|----------------|-------------|
| **Random Baseline** | 16% | 43.3 (when found) | No learning baseline |
| **Iterative Refinement** | 33% (1/3) | 13 (when found) | Gets stuck at 75% local optimum |
| **OpenEvolve** | **100% (3/3)** | 52.3 | Always finds solution |

**Key insight**: While OpenEvolve takes more iterations on average (52.3 vs 13), it has a **100% success rate** compared to iterative refinement's 33%. The evolutionary approach's population diversity ensures it eventually escapes local optima that trap single-trajectory methods.

### Rich Feedback Mode: Proving Attribution Matters

To verify that feedback attribution is the key factor, we added a `RICH_FEEDBACK=1` mode that tells the agent exactly which modules are correct/incorrect:

```bash
RICH_FEEDBACK=1 python run_iterative_trials.py --trials 3 --iterations 100
```

| Method | Success Rate | Avg Iterations |
|--------|-------------|----------------|
| **Iterative (no feedback)** | 33% | 13 (when found) |
| **Iterative (rich feedback)** | **100%** | **3** |

With rich feedback, iterative refinement achieves **100% success rate in only 3 iterations** - dramatically faster than OpenEvolve's 52 iterations! This proves that:

1. **Feedback attribution is the key factor**, not the optimization method
2. When feedback is attributable, iterative refinement is highly effective
3. Evolution is necessary when feedback is NOT attributable (you can't tell which component is wrong)

## Why This Matters

This example illustrates when you should prefer evolutionary approaches:

1. **Combinatorial Configuration**: When solutions are combinations of independent choices
2. **Deceptive Fitness**: When partial solutions don't clearly indicate which components are correct
3. **No Gradient**: When small changes don't reliably improve or degrade solutions
4. **Building Block Problems**: When good solutions are assembled from discovered components

Real-world examples:
- Hyperparameter tuning (learning rate + batch size + architecture)
- Feature selection (which features to include)
- API composition (which services to combine)
- Configuration optimization (compiler flags, system settings)

## Files

| File | Description |
|------|-------------|
| `initial_program.py` | Starting configuration (0/4 correct) |
| `evaluator.py` | Scores configurations (0-4 correct modules) |
| `config.yaml` | OpenEvolve configuration |
| `iterative_agent.py` | Iterative refinement agent using OpenRouter API |
| `run_iterative_trials.py` | Run multiple trials of iterative agent |
| `run_openevolve_trials.py` | Run multiple trials of OpenEvolve |
| `run_random_baseline.py` | Random search baseline with pass@k analysis |
| `compare_results.py` | Analysis and visualization |

## Configuration Details

The OpenEvolve config uses settings optimized for this combinatorial problem:

```yaml
# High temperature for diverse exploration
temperature: 0.9

# Very high exploration ratio
exploration_ratio: 0.6
exploitation_ratio: 0.25

# Multiple islands for parallel search with frequent migration
num_islands: 5
migration_interval: 3
migration_rate: 0.3

# Larger population for more diversity
population_size: 25
```

**Key config improvements over default:**
- Higher temperature (0.9 vs 0.7) - more exploration of different options
- Prompt emphasizes systematic exploration, not following hints
- More islands (5) with faster migration (interval=3) - combines building blocks faster
- Larger population (25) - maintains more diverse configurations

## References

- **Building Block Hypothesis**: Holland, J.H. (1975). *Adaptation in Natural and Artificial Systems*
- **Schema Theorem**: Explains how evolution propagates good partial solutions
- **No Free Lunch**: Wolpert & Macready (1997) - Evolution excels on problems with structure

## Conclusion

This example demonstrates that **iterative refinement is not sufficient** for problems with independent, combinatorial components. Evolutionary search with population-based exploration and crossover can solve these problems orders of magnitude faster by:

1. Exploring multiple regions of the search space simultaneously
2. Discovering correct "building blocks" in different individuals
3. Combining discoveries through crossover to assemble complete solutions

When your optimization problem has this structure, consider evolutionary approaches like OpenEvolve over single-trajectory iterative refinement.

---

## File: examples/function_minimization/README.md

# Function Minimization Example

This example demonstrates how OpenEvolve can discover sophisticated optimization algorithms starting from a simple implementation.

## Problem Description

The task is to minimize a complex non-convex function with multiple local minima:

```python
f(x, y) = sin(x) * cos(y) + sin(x*y) + (x^2 + y^2)/20
```

The global minimum is approximately at (-1.704, 0.678) with a value of -1.519.

## Getting Started

To run this example:

```bash
cd examples/function_minimization
python ../../openevolve-run.py initial_program.py evaluator.py --config config.yaml
```

## Algorithm Evolution

### Initial Algorithm (Random Search)

The initial implementation was a simple random search that had no memory between iterations:

```python
def search_algorithm(iterations=1000, bounds=(-5, 5)):
    """
    A simple random search algorithm that often gets stuck in local minima.
    
    Args:
        iterations: Number of iterations to run
        bounds: Bounds for the search space (min, max)
        
    Returns:
        Tuple of (best_x, best_y, best_value)
    """
    # Initialize with a random point
    best_x = np.random.uniform(bounds[0], bounds[1])
    best_y = np.random.uniform(bounds[0], bounds[1])
    best_value = evaluate_function(best_x, best_y)
    
    for _ in range(iterations):
        # Simple random search
        x = np.random.uniform(bounds[0], bounds[1])
        y = np.random.uniform(bounds[0], bounds[1])
        value = evaluate_function(x, y)
        
        if value < best_value:
            best_value = value
            best_x, best_y = x, y
    
    return best_x, best_y, best_value
```

### Evolved Algorithm (Simulated Annealing)

After running OpenEvolve, it discovered a simulated annealing algorithm with a completely different approach:

```
/* Detailed source-code truncated for AI context efficiency. */
```

## Key Improvements

Through evolutionary iterations, OpenEvolve discovered several key algorithmic concepts:

1. **Exploration via Temperature**: Simulated annealing uses a `temperature` parameter to allow uphill moves early in the search, helping escape local minima that would trap simpler methods.
    ```python
    probability = np.exp((current_value - new_value) / temperature)
    ```

2. **Adaptive Step Size**: The step size is adjusted dynamically—shrinking as the search converges and expanding if progress stalls—leading to better coverage and faster convergence.
    ```python
    if i > iterations * 0.75:  # Reduce step size towards the end
        step_size *= 0.5
    if no_improvement_count > step_size_increase_threshold: # Increase step size if stuck
        step_size *= 1.1
        no_improvement_count = 0 # Reset the counter
    ```

3. **Bounded Moves**: The algorithm ensures all candidate solutions remain within the feasible domain, avoiding wasted evaluations.
    ```python
    # Keep the new points within the bounds
    new_x = max(bounds[0], min(new_x, bounds[1]))
    new_y = max(bounds[0], min(new_y, bounds[1]))
    ```

4. **Stagnation Handling**: By counting iterations without improvement, the algorithm responds by boosting exploration when progress stalls.
    ```python
    if no_improvement_count > step_size_increase_threshold: # Increase step size if stuck
        step_size *= 1.1
        no_improvement_count = 0 # Reset the counter
    ```

## Results

The evolved algorithm shows substantial improvement in finding better solutions:

| Metric | Value |
|--------|-------|
| Value Score | 0.990 |
| Distance Score | 0.921 |
| Standard Deviation Score | 0.900 |
| Speed Score | 0.466 |
| Reliability Score | 1.000 |
| Overall Score | 0.984 |
| Combined Score | 0.922 |

The simulated annealing algorithm:
- Achieves higher quality solutions (closer to the global minimum)
- Has perfect reliability (100% success rate in completing runs)
- Maintains a good balance between performance and reliability

## How It Works

This example demonstrates key features of OpenEvolve:

- **Code Evolution**: Only the code inside the evolve blocks is modified
- **Complete Algorithm Redesign**: The system transformed a random search into a completely different algorithm
- **Automatic Discovery**: The system discovered simulated annealing without being explicitly programmed with knowledge of optimization algorithms
- **Function Renaming**: The system even recognized that the algorithm should have a more descriptive name

## Next Steps

Try modifying the config.yaml file to:
- Increase the number of iterations
- Change the LLM model configuration
- Adjust the evaluator settings to prioritize different metrics
- Try a different objective function by modifying `evaluate_function()`

---

## File: examples/circle_packing_with_artifacts/README.md

# Circle Packing Example with Artifacts

This example demonstrates how OpenEvolve can be used to tackle the challenging mathematical problem of circle packing, a classic problem in computational geometry. Specifically, we focus on packing 26 circles of varying sizes into a unit square to maximize the sum of their radii, replicating one of the tasks from the AlphaEvolve paper.

**Enhanced with Artifacts**: This version showcases OpenEvolve's artifacts feature, which provides detailed execution feedback to help the LLM understand what went wrong and how to fix it.

## Artifacts Enhancement

This example has been enhanced to demonstrate OpenEvolve's **artifacts side-channel**, which captures detailed execution information beyond just numeric metrics. When a program fails or succeeds, the evaluator now provides rich context that gets included in the next generation's prompt.

### What Artifacts Capture

The enhanced evaluator captures different types of information:

#### 🚨 **Failure Artifacts**
When compilation or runtime errors occur:
```
## Last Execution Output
### Stderr
```
Invalid shapes: centers=(25, 2), radii=(26,), expected (26, 2) and (26,)
```
### Failure_stage
```
shape_validation
```
### Suggestion
```
Check for syntax errors, import issues, or runtime exceptions
```
```

#### ⚠️ **Validation Artifacts**  
When geometric constraints are violated:
```
## Last Execution Output
### Boundary_violations
```
Circle 23 at (0.950000, 0.950000) with radius 0.055000 is outside unit square
Circle 24 at (0.980000, 0.980000) with radius 0.030000 is outside unit square
```
### Overlap_violations
```
Circles 5 and 12 overlap: dist=0.180000, r1+r2=0.190000
```
### Validation_report
```
Valid: False, Violations: 2 boundary, 1 overlaps
```
```

#### ✅ **Success Artifacts**
For excellent solutions (>95% of target):
```
## Last Execution Output
### Stdout
```
Excellent packing! Achieved 99.7% of target value
```
### Radius_stats
```
Min: 0.045123, Max: 0.167500, Avg: 0.101319
```
### Packing_summary
```
Sum of radii: 2.634292/2.635 = 0.9997
```
```

#### ⏱️ **Performance Artifacts**
Always included:
```
### Execution_time
```
12.45s
```
```

### How Artifacts Help Evolution

The artifacts provide crucial context that helps the LLM make better decisions:

1. **Specific Error Fixing**: Instead of just seeing `validity: 0.0`, the LLM sees exactly which circles are problematic and why
2. **Performance Guidance**: Execution time helps the LLM understand if algorithms are too slow
3. **Success Recognition**: When a solution works well, artifacts explain why it's good
4. **Debugging Context**: Full stack traces help fix syntax and runtime errors

### Example Evolution with Artifacts

Here's how artifacts help in a typical evolution scenario:

**Generation N**: Program fails with overlapping circles
```python
# Faulty code
centers[5] = [0.3, 0.3]  
centers[6] = [0.3, 0.3]  # Same position!
radii[5] = radii[6] = 0.1
```

**Artifacts captured**:
```
Circles 5 and 6 overlap: dist=0.000000, r1+r2=0.200000
```

**Generation N+1**: LLM sees the overlap details and fixes it
```python
# Fixed code  
centers[5] = [0.3, 0.3]
centers[6] = [0.5, 0.3]  # Different position!
radii[5] = radii[6] = 0.1
```

This leads to faster convergence because the LLM gets specific, actionable feedback instead of just numeric scores.

### Backward Compatibility

The artifacts enhancement is fully backward compatible:
- **Existing evaluators** continue to work unchanged 
- **Enhanced evaluators** return `EvaluationResult` with both metrics and artifacts
- **Disable artifacts** by setting `export ENABLE_ARTIFACTS=false` if needed

To run with artifacts disabled:
```bash
export ENABLE_ARTIFACTS=false
python openevolve-run.py examples/circle_packing_with_artifacts/initial_program.py \
  examples/circle_packing_with_artifacts/evaluator.py \
  --config examples/circle_packing_with_artifacts/config_phase_1.yaml
```

## Problem Overview

The circle packing problem involves placing n non-overlapping circles inside a container (in this case, a unit square) to optimize a specific metric. For this example:

- We pack exactly 26 circles
- Each circle must lie entirely within the unit square
- No circles may overlap
- We aim to maximize the sum of all circle radii

According to the AlphaEvolve paper, a solution with a sum of radii of approximately 2.635 is achievable for n=26. Our goal was to match or exceed this result.

## Our Approach

We structured our evolution in two phases, each with a different configuration to encourage exploration and exploitation at different stages:

### Phase 1: Initial Exploration

In the first phase, we focused on exploring different fundamental approaches to the packing problem:

- Used a constructor-based approach that places circles in strategic positions
- Explored various geometric patterns (concentric rings, grid-based arrangements, etc.)
- Developed simple optimization routines to maximize circle sizes without overlaps

Configuration highlights:
```yaml
max_iterations: 100
population_size: 60  
num_islands: 4
exploitation_ratio: 0.7
```

### Phase 2: Breaking Through the Plateau

After the initial exploration phase, we observed our solutions plateauing around 2.377. For the second phase, we reconfigured OpenEvolve to encourage more radical innovations:

- Increased the population size to promote diversity
- Lowered the exploitation ratio to favor exploration
- Updated the system prompt to suggest different optimization techniques
- Allowed for longer and more complex code solutions

Configuration highlights:
```yaml
max_iterations: 100
population_size: 70  
num_islands: 5
exploitation_ratio: 0.6
```

## Evolution Progress

We tracked the evolution over 470 generations, capturing visualizations at each checkpoint. The progression shows dramatic improvements in the packing strategy:

### Initial Solution (Generation 0)

The initial program used a simple constructive approach with a central circle and two concentric rings:

```python
# Initial attempt
# Place a large circle in the center
centers[0] = [0.5, 0.5]

# Place 8 circles around it in a ring
for i in range(8):
    angle = 2 * np.pi * i / 8
    centers[i + 1] = [0.5 + 0.3 * np.cos(angle), 0.5 + 0.3 * np.sin(angle)]

# Place 16 more circles in an outer ring
for i in range(16):
    angle = 2 * np.pi * i / 16
    centers[i + 9] = [0.5 + 0.7 * np.cos(angle), 0.5 + 0.7 * np.sin(angle)]
```

This approach yielded a sum of radii of approximately 0.959.



### Generation 10 Breakthrough

By generation 10, OpenEvolve had already discovered a more sophisticated approach:

```python
# Generation 10
# Parameters for the arrangement (fine-tuned)
r_center = 0.1675  # Central circle radius

# 1. Place central circle
centers[0] = [0.5, 0.5]
radii[0] = r_center

# 2. First ring: 6 circles in hexagonal arrangement
r_ring1 = 0.1035
ring1_distance = r_center + r_ring1 + 0.0005  # Small gap for stability
for i in range(6):
    angle = 2 * np.pi * i / 6
    centers[i+1] = [
        0.5 + ring1_distance * np.cos(angle),
        0.5 + ring1_distance * np.sin(angle)
    ]
    radii[i+1] = r_ring1
```

The key innovations at this stage included:
- A carefully tuned hexagonal arrangement for the first ring
- Strategic placement of corner circles
- An additional optimization step to maximize each circle's radius

This approach achieved a sum of radii of approximately 1.795.



### Generation 100: Grid-Based Approach

By generation 100, OpenEvolve had pivoted to a grid-based approach with variable sized circles:

```python
# Generation 100
# Row 1: 5 circles
centers[0] = [0.166, 0.166]
centers[1] = [0.333, 0.166]
centers[2] = [0.500, 0.166]
centers[3] = [0.667, 0.166]
centers[4] = [0.834, 0.166]

# Row 2: 6 circles (staggered)
centers[5] = [0.100, 0.333]
centers[6] = [0.266, 0.333]
# ... additional circles
```

Key innovations:
- Grid-like pattern with staggered rows
- Variable circle sizes based on position (larger in the center)
- More aggressive optimization routine with 50 iterations

This approach achieved a sum of radii of approximately 2.201.



### Final Solution: Mathematical Optimization

The breakthrough came when OpenEvolve discovered the power of mathematical optimization techniques. The final solution uses:

```python
# Final solution with scipy.optimize
def construct_packing():
    # ... initialization code ...
    
    # Objective function: Negative sum of radii (to maximize)
    def objective(x):
        centers = x[:2*n].reshape(n, 2)
        radii = x[2*n:]
        return -np.sum(radii)

    # Constraint: No overlaps and circles stay within the unit square
    def constraint(x):
        centers = x[:2*n].reshape(n, 2)
        radii = x[2*n:]
        
        # Overlap constraint
        overlap_constraints = []
        for i in range(n):
            for j in range(i + 1, n):
                dist = np.sqrt(np.sum((centers[i] - centers[j])**2))
                overlap_constraints.append(dist - (radii[i] + radii[j]))
        # ... boundary constraints ...
        
    # Optimization using SLSQP
    result = minimize(objective, x0, method='SLSQP', bounds=bounds, constraints=constraints)
```

The key innovation in the final solution:
- Using `scipy.optimize.minimize` with SLSQP method to find the optimal configuration
- Formulating circle packing as a constrained optimization problem
- Representing both circle positions and radii as optimization variables
- Carefully crafted constraints to enforce non-overlap and boundary conditions

This approach achieved a sum of radii of 2.634, matching the AlphaEvolve paper's result of 2.635 to within 0.04%!



## Results

Our final solution achieves:

```
Sum of radii: 2.634292402141039
Target ratio: 0.9997314619131079 (99.97% of AlphaEvolve's result)
```

This demonstrates that OpenEvolve can successfully reproduce the results from the AlphaEvolve paper on this mathematical optimization problem.

## Fast Convergence with Dual-Model Configuration

Using a dual-model configuration with weighted sampling, OpenEvolve achieves near-optimal results in remarkably few iterations:



### Configuration

The `config.yaml` uses two Gemini models with different weights:
- `google/gemini-2.5-flash-lite` (weight: 0.8) - Fast, cost-effective for exploration
- `google/gemini-2.5-flash` (weight: 0.2) - Higher capability for breakthroughs

```yaml
llm:
  models:
    - name: "google/gemini-2.5-flash-lite"
      weight: 0.8
    - name: "google/gemini-2.5-flash"
      weight: 0.2
```

### Rapid Convergence

The plot shows the evolution of sum_radii across program versions:

- **Version 0**: Starts at ~0.96 (basic initial program)
- **Version 6**: First major improvement to ~2.09
- **Version 21**: Reaches 2.63 (99.8% of target)
- **Final**: Achieves 2.6304 sum of radii

**Key insight**: OpenEvolve discovers the mathematical optimization approach (using `scipy.optimize.minimize` with SLSQP) by version 21, achieving 99.8% of the AlphaEvolve target in just ~40 program evaluations. The dual-model approach allows rapid exploration with the lighter model while leveraging the more capable model for breakthrough discoveries.

### Why It Works

1. **Artifacts provide rich feedback**: Failed programs return detailed error information (boundary violations, overlaps), helping the LLM quickly correct mistakes
2. **MAP-Elites diversity**: The feature dimensions (`radius_variance`, `spatial_spread`) maintain diverse solutions in the population
3. **Island-based evolution**: 4 islands evolve independently, preventing premature convergence
4. **Efficient model weighting**: 80% lightweight model for broad exploration, 20% capable model for sophisticated solutions

## Key Observations

The evolution process demonstrated several interesting patterns:

1. **Algorithm Transition**: OpenEvolve discovered increasingly sophisticated algorithms, from basic geometric constructions to advanced mathematical optimization techniques.

2. **Exploration-Exploitation Balance**: The two-phase approach was crucial - initial exploration of different patterns followed by exploitation and refinement of the most promising approaches.

3. **Breakthrough Discoveries**: The most significant improvements came from fundamental changes in approach (e.g., switching from manual construction to mathematical optimization), not just parameter tuning.

4. **Code Complexity Evolution**: As the solutions improved, the code grew in complexity, adopting more sophisticated mathematical techniques.

## Running the Example

To reproduce our results:

```bash
# Phase 1: Initial exploration
python openevolve-run.py examples/circle_packing/initial_program.py \
  examples/circle_packing/evaluator.py \
  --config examples/circle_packing/config_phase_1.yaml \
  --iterations 100

# Phase 2: Breaking through the plateau
python openevolve-run.py examples/circle_packing/openevolve_output/checkpoints/checkpoint_100/best_program.py \
  examples/circle_packing/evaluator.py \
  --config examples/circle_packing/config_phase_2.yaml \
  --iterations 100
```

To visualize the best solution:

```python
from examples.circle_packing.openevolve_output.best.best_program import run_packing, visualize

centers, radii, sum_radii = run_packing()
print(f"Sum of radii: {sum_radii}")
visualize(centers, radii)
```

## Conclusion

This example demonstrates OpenEvolve's ability to discover sophisticated algorithms for mathematical optimization problems. By evolving from simple constructive approaches to advanced numerical optimization techniques, OpenEvolve was able to match the results reported in the AlphaEvolve paper.

The circle packing problem shows how OpenEvolve can discover not just improvements to existing algorithms, but entirely new algorithmic approaches, transitioning from manual geometric construction to principled mathematical optimization.