As part of my ongoing research evaluating GPE-A (Goal, Performance, Exclusion-Architecture), a structured LLM prompting framework for simulation software development, I generate a large number of algorithm implementations. These span multiple dimensions: baselines, GPE-A prompts, hybrid prompting strategies, and different LLM models.

To make meaningful, apples-to-apples comparisons between these implementations, the benchmarking tests must remain strictly constant. In practice, however, the structure of the generated algorithms often varies significantly. They cannot be benchmarked directly without an additional layer of code.

This is where wrappers come in.

By wrapper, I mean a small piece of generated code that sits around an algorithm implementation and handles input/output normalization, timing, setup, and teardown. The wrapper allows structurally different implementations to be benchmarked uniformly, without modifying the benchmark itself.

Given the sheer number of algorithms involved, manually writing wrappers is inefficient. The obvious solution is to use an LLM to generate wrappers automatically. However, the variable nature of generative models introduces an immediate concern:

Is the wrapper consistent across implementations?

Even small inconsistencies (extra logging, slightly different timing logic, additional checks) can invalidate performance results when milliseconds matter. This naturally raises a follow-up question:

How do you enforce reliable consistency in LLM-generated wrappers?

My Approach to Enforcing Consistency

I’ve experimented with this using both ChatGPT 5.1 and ChatGPT 5.2 in browser-based sessions. What follows is the workflow that has proven most reliable.

Method 1: Single Dedicated Chat Session

1. Use one chat session exclusively for wrapper generation
   Do not mix in unrelated tasks or conversations.
2. Define strict rules at the start of the session
   These rules specify exactly what the wrapper may and may not do.
3. Provide the benchmark test and algorithm implementation
   Ask the model to generate a wrapper that adapts the implementation to the benchmark.
4. Manually evaluate the returned wrapper
   • If correct, respond with a confirmation instructing the model to enforce those rules going forward.
   • If incorrect, provide a precise critique and request a revised wrapper.
5. Continue generating wrappers within the same session
   Perform quick manual checks to ensure the rules remain adhered to.
6. Issue reminder prompts when drift occurs
   If the model starts relaxing constraints, explicitly remind it of the original rules.

Over time, the session develops a strong internal rhythm and produces increasingly consistent results – until the browser sessions seems to reach capacity. After extended use, the session seems to inevitably degrades where new prompts crash the browser, and its unfortunately time to start a new session.

Method 2: Bootstrapping a New Session Using the Old One

When starting a fresh chat session as a continuation to one that seems to reach capacity, I use a slightly different but effective technique:

1. Start a new chat session
2. Define the same enforcement rules upfront
3. Provide the benchmark and an algorithm implementation
4. Generate a wrapper
5. Send that wrapper to the previous chat session
   Ask whether it adheres to the established rules. The old session typically produces a detailed critique identifying violations.
6. Feed that critique back into the new session
   The new session usually acknowledges the feedback and adjusts its understanding accordingly.
7. Proceed with wrapper generation as usual

This effectively transfers the “discipline” of the old session into the new one and dramatically reduces the warm-up time needed to reach consistent outputs.

Closing Thoughts

LLMs are powerful tools for automating repetitive glue code, but when benchmarking for characteristics like performance in algorithms, consistency matters as much as correctness. Treating wrapper generation as a controlled, stateful process rather than a series of isolated prompts has been essential to producing reliable results at scale.