Rule Based Approach to Code Transformations

Software systems must continuously evolve to remain business-relevant, maintainable, and secure.

This article's focus will be on tech-driven transformations. When a tech-driven transformation process is on the table, there are different approaches to consider.

Different Approaches To Code Transformations

Why Scale The Change?

How Rule-Based Systems Work?

For rule-based systems to work, a structured representation of the source code is needed rather than raw text. In rule-based systems the most common representation types are Abstract Syntax Trees (ASTs), and Lossless Semantic Trees (LSTs).

After a clear representation is acquired, the rules are applied to it in order to acquire a modified tree.

Finally the modified tree is converted to the output code in the targeted format and language.

A Parse Tree?

AST (Abstract Syntax Tree), and LST (Lossless Semantic Tree) are two different conceptual structures which are used widely in rule-based systems.

• Because of its focus on meaning, ASTs are preferred for cross-language transformations.
• Because of its lossless structure, LSTs are preferred in Large Scale Refactoring.

Generating these structures from source code in a way that supports safe and large-scale transformations is non-trivial. Tools such as ANTLR (by ANTLR ) and OpenRewrite (by Moderne ) address this challenge using different abstractions.

• ANTLR creates parse trees from text-formatted source code, from which ASTs can be derived;
• OpenRewrite, on the other hand, parses source code directly into a LSTs.

Cross-Language Transformations with ANTLR

ANTLR is a parser generator that converts formal grammars into parsers capable of building ASTs.

First step of ANTLR-based transformations is defining the grammar. This is the way we make ANTLR aware of the type of input it should expect on the run.

The grammar allows ANTLR to generate parser and lexer.

Then, ANTLR produces AST out of the input based on the lexer and parser generated on the previous step.

ANTLR also creates visitor / listener interfaces that allow developers to traverse the parse tree, and perform transformations to generate the desired output.

OpenRewrite's LST-Based Approach To Large-Scale Refactoring

OpenRewrite refactor engine parses the source code and converts it into an LST. In the core of making the real changes to the original code reside "recipes". A recipe is a Java code, implemented using visitors to traverse the LST, and apply the dictated changes into it.

These recipes run against the generated LST, and the transformed LST is printed back into source code as an output, preserving original formatting and style.

The OpenRewrite ecosystem includes a large collection of open-source recipes maintained by the community, as well as proprietary recipes developed by Moderne. Moderne also provides training sessions and educational resources focused on writing and maintaining custom recipes.

Summary

Rule-based approach to code transformations:

•  Is Effective when determinism and scalability and repeatability are required
•  Is Powerful when tree generation and change rules are well-defined
• Works Well With ANTLR when cross-language code transformation is needed.
• Works Well With OpenRewrite when large scale refactoring is needed.

*1 : https://docs.moderne.io/administrator-documentation/moderne-platform/references/lossless-semantic-trees/