LLMs as a Compiler

Kadhir Manickam

(3 minutes)

<section data-section-id='8c758025-bac1-4de1-941b-6b6374ba9b01'>[2025-07-02] So far, I've only used LLMs as assistants, where I'm doing something and an LLM helps me along the way. Code autocomplete feels like a great example of how useful it can be when it gets it right. I don't doubt that this will improve over time, but I'm excited to see a more significant transition from this assistant mode to a compiler mode, at least for coding. It will be exciting when we focus solely on the context we fed the LLM, then test the features it generates rather than the code. And importantly, we let the LLM handle integrating new features into the existing codebase. That means we no longer examine the code. Our time as engineers will be spent handling context, testing features, and iterating on them. The consequence of that seems to be:<ul><li value="1">Democratize access to engineering<ul><li value="1">You don't need as specialized skillsets to build complex apps, you just need to know how to put context together and iterate</li></ul></li><li value="2">Increase the velocity of feature development<ul><li value="1">My gut says dealing with context will result in a better ratio of engineering time into features shipped than dealing with code directly</li></ul></li></ul> The obvious pushback here is, well, compilers are provable. There's a straightforward mapping between inputs and outputs, and we can prove the outputs are the same each time. We can also write tests to ensure the outputs are optimized. But if we squint, a compiler transforms an input into an output. If we treat the code as an intermediate layer, viewing the input as context and the output as features, we can demonstrate the compiler's reliability through evaluation and testing. And importantly, we don't have to get the output right in the first go, we can let it iterate over and over until it gets it right. A new kind of compiler. So I propose that if we get LLM-as-a-compiler, as a software engineer, I will go through this cycle:<ol><li value="1">Put together the context<ol><li value="1">Which includes a series of tests for the final behavior (perhaps I use an LLM for this)</li></ol></li><li value="2">I put it through the LLM compiler<ol><li value="1">Which is probably a system composed of several things</li></ol></li><li value="3">Which continually iterates on the output until all the tests pass<ol><li value="1">Ideally, as the LLM compiler gets better, the latency gets lower and lower</li></ol></li><li value="4">We cache the output (code) for performance improvements</li><li value="5">I decide how I need to edit the context, and go back to step 1</li></ol><mermaid data-height="320"> flowchart LR A[Put togetherthe context] --> B[LLM compiler] B --> C{Tests pass?} C -->|No| B C -->|Yes| D[Cache the output] D --> E[Edit the context] E --> A </mermaid> SWE agents feel like they're the right abstraction on this path; they convert context into features, iterating in the background. They feel like they'll be an integral part of the LLM compiler system, which I think will have the following pieces:<ul><li value="1">A way to specify the context of my app<ul><li value="1">And a way to specify which part of my context to focus on</li></ul></li><li value="2">Mechanism for specifying my reward signal (my tests)</li><li value="3">A system for monitoring the changes happening<ul><li value="1">And a way to redirect parts of the compiler if it's not doing what I expect</li><li value="2">Over time, I'd expect this part to evolve and the need to see the code to reduce</li></ul></li></ul> <h3>Resources</h3><ol><li value="1">Vivek Haldar - LLMs are compilers<web-citation data-url="https://vivekhaldar.com/articles/llms-are-compilers/" data-title="Vivek Haldar - LLMs are compilers"></web-citation></li><li value="2">Dave Hoover - LLM as Compiler<web-citation data-url="https://medium.com/redsquirrel-tech/llm-as-compiler-2a2f79d30f0b" data-title="Dave Hoover - LLM as Compiler"></web-citation></li><li value="3">Avik Das - LLMs are like compilers, sort of<web-citation data-url="https://avikdas.com/2025/05/05/llms-are-like-compilers-sort-of.html" data-title="Avik Das - LLMs are like compilers, sort of"></web-citation></li></ol></section>