ProgramBench: Can AI Rebuild Software?

Imagine handing over a compiled program, its documentation, and saying, “Rebuild this.” Not by looking at the source, not by searching the web, but by understanding the essence of what it does and recreating it from scratch. This isn’t a hypothetical for the future; it’s the challenge posed by ProgramBench, a new benchmark designed to stress-test the current frontier of AI agents and language models in software creation. The results? Frankly, they’re a stark reminder of how far we still have to go.

The core problem ProgramBench tackles is fundamental: can AI truly architect and implement software, or is it merely a sophisticated pattern-matcher capable of stitching together existing code snippets? Current AI agents, powered by advanced LLMs, often fall into the latter category. They excel at tasks like code completion, generating boilerplate, or even patching existing codebases, as seen in benchmarks like SWE-bench. ProgramBench, however, demands more. It requires agents to synthesize a complete, executable program that mirrors the behavior of a reference executable, using only the executable itself and its documentation. Critically, it disallows decompilation and internet access, forcing AI to infer and construct without shortcuts.

The technical execution of ProgramBench is elegant in its rigor. An AI agent is tasked with:

1. Architecting: Devising a project structure and design.
2. Implementing: Writing source code to fulfill the program’s documented functionality.
3. Generating a Build Script: Creating the necessary Makefile, CMakeLists.txt, or similar to compile the project.

The evaluation relies on agent-driven fuzzing to generate comprehensive end-to-end behavioral tests. This approach bypasses the need to prescribe specific implementation details, allowing the AI agent full latitude in its architectural choices. To participate, developers typically interact with the benchmark via Python:

# Example of evaluating a submission
import programbench

# Assume 'your_submission_dir' contains the agent's generated code and build script
results = programbench.eval("your_submission_dir")
print(results)

# Or directly run an evaluation command
# $ pip install programbench && programbench eval <your submission>

Under the hood, these agents leverage LLM APIs (like Gemini or GPT) orchestrated by agent frameworks (e.g., mini-SWE-agent). The observation from this benchmark is telling: models consistently produce monolithic, single-file implementations. This is a stark departure from human-written code, which typically involves modularity, clear separation of concerns, and idiomatic practices.

On platforms like Reddit and Hacker News, initial discussions in May 2026 lauded ProgramBench as a “frontier stress test.” However, the strict constraints—particularly the ban on decompilation—sparked debate. Some argue that a more realistic scenario would involve access to source code or at least the ability to inspect intermediate representations. Yet, this constraint is precisely what makes ProgramBench so valuable: it forces AI to bridge the semantic gap between low-level execution and high-level intent, a gap that remains a significant hurdle.

The critical verdict is clear: despite the hype surrounding AI in software development, LLMs are not currently capable of reliably rebuilding full software projects from scratch. ProgramBench’s results confirm this. The best performing model, Claude Opus 4.7, achieved a mere 95% test pass rate on only 3% of the tasks. This indicates profound limitations in high-level architectural design, system decomposition, and the generation of maintainable, human-quality code.

We should avoid tasks where LLMs are expected to autonomously handle holistic software development from the ground up, especially for complex or security-critical systems. The current strengths of LLMs lie in augmentation—assisting developers with specific, well-defined tasks—rather than independent creation. ProgramBench serves as a vital diagnostic tool, highlighting fundamental challenges that must be addressed before AI can truly be considered a partner in rebuilding the software that underpins our digital world. It’s a benchmark for research, not a reflection of current production-ready capabilities.