Skip to content

feat: Add project parser for OpenRewrite #832

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 52 commits into from
Jul 25, 2023
Merged

Conversation

fabapp2
Copy link
Contributor

@fabapp2 fabapp2 commented Jun 30, 2023

This PR provides a first implementation of a Project parser to parse Java projects using Maven to OpenRewrite AST.
The AST together with the provided ExecutionContext can then be used to execute OpenRewrite recipes.
To find OpenRewrite recipes, a RewriteRecipeDiscovery component can be used to discover existing recipes, e.g. from the classpath.

@Component
@RequiredArgsConstructor
public class MyTool {

    // The parser is a Spring bean
    private final RewriteProjectParser parser;
    // RecipeDiscovery is a Spring bean
    private final RewriteRecipeDiscovery recipeDiscovery;
    // ProjectScanner is a Spring bean
    private final ProjectScanner projectScanner;

    public RecipeResult runRecipe(Path baseDir, String recipeName) {
        ExecutionContext ctx = new InMemoryExecutionContext();
        List<Resource> resources = projectScanner.scan(baseDir);
        List<SourceFile> ast = parser.parse(baseDir, resources, ctx);
        Xml.Document rootPom = findRootPom(ast);
        recipeDiscovery.discoverFilteredRecipe(rootPom, recipeName)
            .ifPresent(recipe -> recipe.run(new InMemoryLargeSourceSet(ast), ctx));
    }
}

The parser hooks into the Maven execution and provides the AST as it would be created when running the OpenRewrite Maven plugin.

This implementation programmatically starts the Maven goals clean and install and hooks into the execution with a listener providing access to the MavenSession and MavenProject.
These instances are used to call the MavenMojoProjectParser which does the heavy lifting of parsing the MavenProject to OpenRewrite's AST.

This implementation requires file IO. An attempt to use JIMFS was unsuccessful because at multiple places File is used which is not supported by JIMFS.

Replacing the Maven classes that depend on File (ordering the Maven modules in a new BuildFileGraph impl.) would free the parser from access to File allowing the usage of potentially in-memory Resourcess.

In MavenMojoProjectParser:402 the results of a call to MavenParser.parse() are collected into a List<SourceFile>. This is a problem when a pom contains a parser error which will be returned as ParseError which is a SourceFile but can't be casted into Xml.Document and thus a CCE is thrown. From this exception it is impossible to understand which pom had a problem.

The ParsingListener that can be registered is called for each parsed resource. This would not allow a countdown of resources as the total number of resources is unknown.

@fabapp2 fabapp2 added the spike label Jun 30, 2023
@fabapp2 fabapp2 linked an issue Jul 20, 2023 that may be closed by this pull request
25 tasks
Two implementations currently exist:
- RewriteMavenProjectParser
- RewriteProjectParser

RewriteMavenProjectParser
creates Maven Plexus container to retrieve the Maven instance.
A Maven instance is used to call `clean` and `package` while registering a listener
to the lifecycle event when the goals are finished.
The MavenSession provided to the listener allows executing the same OR classes as used when
running in the rewrite-maven plugin.
The created AST should be exactly the same as with the Maven plugin.

RewriteProjectParser
replicates the RewriteMavenProjectParser while split into smaller components.
The RewriteMavenProjectParser is used to create the target AST which should be
identically when parsing the same project with RewriteProjectParser.
The RewriteProjectParser is meant to replace the RewriteMavenProjectParser.
@fabapp2 fabapp2 force-pushed the spike/rewrite-maven-parser branch from 0247fbb to 14405e7 Compare July 21, 2023 10:48
@fabapp2 fabapp2 self-assigned this Jul 21, 2023
@fabapp2 fabapp2 requested a review from BoykoAlex July 21, 2023 11:08
@fabapp2 fabapp2 force-pushed the spike/rewrite-maven-parser branch from 46e8727 to f497c26 Compare July 21, 2023 13:05
fabapp2 added 20 commits July 21, 2023 18:45
Two implementations currently exist:
- RewriteMavenProjectParser
- RewriteProjectParser

RewriteMavenProjectParser
creates Maven Plexus container to retrieve the Maven instance.
A Maven instance is used to call `clean` and `package` while registering a listener
to the lifecycle event when the goals are finished.
The MavenSession provided to the listener allows executing the same OR classes as used when
running in the rewrite-maven plugin.
The created AST should be exactly the same as with the Maven plugin.

RewriteProjectParser
replicates the RewriteMavenProjectParser while split into smaller components.
The RewriteMavenProjectParser is used to create the target AST which should be
identically when parsing the same project with RewriteProjectParser.
The RewriteProjectParser is meant to replace the RewriteMavenProjectParser.
…esult

- RewriteProjectParser uses Maven's GraphBuilder to analyze the build file. This requires file access and thus binds the component to the File system.

- Fix assertions about resulting ExecutionContext
@fabapp2 fabapp2 force-pushed the spike/rewrite-maven-parser branch from 3011a73 to 5d9d30e Compare July 24, 2023 23:23
@fabapp2 fabapp2 added openrewrite Marks issues that might contain code worth contributing to OpenRewrite in: sbm-support-rewrite Issue is related to the sbm-support-rewrite compionent type: enhancement New feature or request and removed openrewrite Marks issues that might contain code worth contributing to OpenRewrite spike labels Jul 25, 2023
@fabapp2 fabapp2 requested a review from BoykoAlex July 25, 2023 08:34
@fabapp2 fabapp2 mentioned this pull request Jul 20, 2023
25 tasks
@fabapp2 fabapp2 merged commit 7f3f6ff into main Jul 25, 2023
@fabapp2 fabapp2 deleted the spike/rewrite-maven-parser branch July 25, 2023 12:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
in: sbm-support-rewrite Issue is related to the sbm-support-rewrite compionent type: enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Epic: Extract OpenRewrite parser
2 participants