AI for Program Optimization

In his blog, Chris Lattner called out “Some have criticized CCC for learning from this prior art, but I find that ridiculous - I certainly learned from GCC when building Clang!”. In regard to efficiency of the code generated by CCC, Nicholas Carlini had acknowledged in the original CCC blog: 

“The generated code is not very efficient. Even with all optimizations enabled, it outputs less efficient code than GCC with all optimizations disabled.”

There’s over 60 years of rich history of compiler optimization research (dating back to the 1957 seminal paper “The FORTRAN automatic coding system” by Backus et al). Out of curiosity, I dabbled a bit with Gemini to check what all it can surface in regard to loop taxonomy and how each type can be optimized. It did well with a bit of nudging (see the attached tabular summary). 

Likewise, with a few nudges, GPT-5.2 could summarize the classical loop transformations well (see the attached tabulation).

The question is whether the LLMs can “understand” optimizations (it’s a large set), come up with cost models and heuristics to determine when to apply which optimization and actually yield speedups in real-life programs. 

Recently, frameworks such as Magellan - it combines an LLM-powered coding agent (AlphaEvolve) with evolutionary search and autotuning - have been proposed to help discover new compiler optimization heuristics. How do we make sure that systems do not re-invent the wheel and leverage the rich literature? Or are such systems already past the “Move 37” moment?

Given the hardness associated with code optimization (see below for a snapshot) and the recent success of AI systems with longstanding problems in computer science [1, 2], mathematics [3] and biology [4, 5, 6], perhaps it’s paramount to leverage AI to drive training and inference efficiency in the sprawling data centers.

Would love to hear the community’s thoughts on the above.