In January of 2025, the second semester of my sophomore year of my undergraduate studies, I began a line of research exploring whether generative AI could be used for performance-critical simulation software development. While much prior work emphasized syntactical correctness or developer productivity, this research focused on an arguably more pressing question:

Can LLM-generated code meet performance requirements, not just compile and run?

That question ultimately led to my first conference paper, accepted and presented at the Winter Simulation Conference (WSC) 2025.

Performance Prioritized

The core contribution of this work was a metric-based evaluation of LLM-generated simulation code against a high-performance baseline. Rather than evaluating only syntactic or functional correctness, the study measured runtime performance, a key requirement in quality simulation software.

The goal was to determine whether performance constraints could be systematically enforced in code generated by large language models.

Early Results: Roles Assignments Didn’t Help

Initial experiments examined role assignments for an LLM across different stages of a software development pipeline. Using a heapsort simulation primitive as the test case, the results were clear:

• Role assignments had no measurable impact on code quality or performance.
• Generated code showed an average ~50% performance degradation compared to the baseline.

For simulation workloads, this level of degradation is unacceptable.

Iteration Through Metrics

In response, I adopted an iterative feedback loop using ChatGPT 4o. The process was simple:

• Generate code from a prompt
• Benchmark it using a test harness
• Feed the performance metrics back to the LLM
• Ask the model to revise the prompt based on the results

A key constraint was that the prompt could not become heapsort-specific; the objective was a generalizable prompting approach.

GPE-A Prompting

After several iterations, this process produced a prompt that consistently generated code meeting or exceeding the baseline. From this, I abstracted a structured prompting framework called: GPE-A: Goal, Performance, Exclusion – Architecture

GPE-A embeds explicit performance goals and architectural constraints directly into the prompt, treating it as a performance contract rather than relying on role-play or verbosity.

The Conference

These results were published in LLM Prompt Engineering for Performance in Simulation Software Development: A Metric-Based Approach to Using LLMs. The paper was accepted to Winter Simulation Conference 2025, presented on December 8th, 2025, and published in the conference proceedings.

Looking Forward

I have since continued working on advancing the work with GPE-A toward integration into simulation software development processes. The work has currently shown promising results for random number generation for both classical and quantum programming and runge-kutta methods for classical programming. Next steps include an evaluation GPE-A across a multitude of LLMs and extensive testing for repeatability.