Skip to content

Commit 04160b1

Browse files
suztomosuztomo
authored
ci: GitHub Actions to validate a Maven BOM (#5928)
Fixes #5922 It worked in https://togithub.com/googleapis/gapic-generator-java/pull/1629 Todo: - Merge this pull request 5928. - Update the branch name in https://togithub.com/googleapis/gapic-generator-java/pull/1629 to "main" - Add another check in https://togithub.com/googleapis/google-cloud-java with "main"
1 parent 28f172d commit 04160b1

File tree

9 files changed

+342
-10
lines changed

9 files changed

+342
-10
lines changed

‎.github/workflows/ci-release-note-generation.yaml

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,8 +10,9 @@ jobs:
1010
- uses: actions/checkout@v3
1111
- uses: actions/setup-java@v3
1212
with:
13-
distribution: zulu
13+
distribution: temurin
1414
java-version: 11
15+
cache: maven
1516
- run: java -version
1617
- name: Run test in release-note-generation
1718
shell: bash
@@ -27,8 +28,9 @@ jobs:
2728
- uses: actions/checkout@v3
2829
- uses: actions/setup-java@v3
2930
with:
30-
distribution: zulu
31+
distribution: temurin
3132
java-version: 11
33+
cache: maven
3234
- run: java -version
3335
- name: Dry-run release-note-generation
3436
shell: bash

‎.github/workflows/dashboard.yaml

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,8 +11,9 @@ jobs:
1111
- uses: actions/checkout@v3
1212
- uses: actions/setup-java@v3
1313
with:
14-
distribution: zulu
14+
distribution: temurin
1515
java-version: 8
16+
cache: maven
1617
- run: java -version
1718
- run: .kokoro/dashboard.sh
1819
env:
@@ -24,8 +25,9 @@ jobs:
2425
- uses: actions/checkout@v3
2526
- uses: actions/setup-java@v3
2627
with:
27-
distribution: zulu
28+
distribution: temurin
2829
java-version: 8
30+
cache: maven
2931
- run: java -version
3032
- run: .kokoro/dashboard.sh
3133
env:

‎.github/workflows/full-convergence-check.yaml

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -12,16 +12,17 @@ jobs:
1212
- uses: actions/checkout@v3
1313
- uses: actions/setup-java@v3
1414
with:
15-
distribution: zulu
16-
java-version: 8
15+
distribution: temurin
16+
java-version: 11
17+
cache: maven
1718
- run: java -version
1819
- name: Install BOMs
1920
run: |
2021
mvn -B -V -ntp install
2122
- name: Ensure the members of the Libraries BOM exist in Maven Central
22-
run: |
23-
mvn -B -V -ntp verify -Dtest="BomContentTest#testLibrariesBomReachable"
24-
working-directory: tests
23+
uses: ./tests/validate-bom
24+
with:
25+
bom-path: libraries-bom/pom.xml
2526
- name: Ensure the BOM has valid content (at releases)
2627
if: github.head_ref == 'release-please--branches--main'
2728
run: |

‎.github/workflows/release-note-generation.yaml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,8 +17,9 @@ jobs:
1717
- uses: actions/checkout@v3
1818
- uses: actions/setup-java@v3
1919
with:
20-
distribution: zulu
20+
distribution: temurin
2121
java-version: 11
22+
cache: maven
2223
- run: java -version
2324
- name: Pick Libraries BOM version
2425
id: pick-version

‎tests/validate-bom/README.md

Lines changed: 80 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,80 @@
1+
# Validate Maven BOM GitHub Action
2+
3+
This action validates a [Maven BOM](https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#bill-of-materials-bom-poms
4+
) specified as argument.
5+
6+
This action performs the following steps:
7+
8+
- It reads the BOM and gets all artifacts.
9+
- It may filter out "testlib" artifacts if they cause problems in subsequent steps
10+
- It creates a canary Maven project (a directory with a pom.xml file) with the artifacts as the dependencies.
11+
The canary project uses the BOM and declares the artifacts in the BOM as dependencies.
12+
- It runs `mvn install` in the canary project.
13+
If the BOM is valid, it should fetch dependencies (the artifacts in the BOM) without an error.
14+
15+
## Usage
16+
17+
You can use this action via `uses: googleapis/java-cloud-bom/tests/validate-bom@main`
18+
in one of the steps in a job in your GitHub repository.
19+
20+
Note that before running this action the caller needs to make the BOM and its
21+
listing artifacts available in Maven Central or local Maven repository.
22+
23+
Here is a concrete example to define a job to use this "validate-bom" action in
24+
a GitHub Actions workflow file:
25+
26+
```
27+
validate-bom:
28+
runs-on: ubuntu-latest
29+
steps:
30+
- uses: actions/checkout@v3
31+
- uses: actions/setup-java@v3
32+
with:
33+
java-version: 11
34+
distribution: temurin
35+
cache: maven
36+
- name: Install Maven artifacts locally
37+
run: |
38+
mvn install -B -ntp -DskipTests
39+
- uses: googleapis/java-cloud-bom/tests/validate-bom@main
40+
with:
41+
path: <path_to_bom_pom.xml>
42+
```
43+
44+
### Results
45+
46+
If there's an error in building the canary project, the check fails.
47+
You see errors in the log:
48+
49+
```
50+
[INFO] ------------------------------------------------------------------------
51+
[INFO] BUILD FAILURE
52+
[INFO] ------------------------------------------------------------------------
53+
[INFO] Total time: 14.253 s
54+
[INFO] Finished at: 2023-04-14T20:41:59Z
55+
[INFO] ------------------------------------------------------------------------
56+
Error: Failed to execute goal on project bom-validation-canary-project: Could n
57+
ot resolve dependencies for project com.google.cloud:bom-validation-canary-proje
58+
ct:jar:0.0.1-SNAPSHOT: The following artifacts could not be resolved: com.google
59+
.analytics.api.grpc:grpc-google-analytics-admin-v1alpha:jar:0.24.0 ...
60+
```
61+
62+
In this error message, there were invalid artifacts defined in the BOM
63+
(wrong group IDs).
64+
65+
If there's no error, the check passes with a successful message:
66+
67+
```
68+
[INFO] Installing /tmp/bom-validation/pom.xml to /home/runner/.m2/repository/...
69+
[INFO] ------------------------------------------------------------------------
70+
[INFO] BUILD SUCCESS
71+
[INFO] ------------------------------------------------------------------------
72+
[INFO] Total time: 5.147 s
73+
[INFO] Finished at: 2023-04-14T20:35:58Z
74+
[INFO] ------------------------------------------------------------------------
75+
```
76+
77+
# Disclaimer
78+
79+
This is not an official Google product.
80+
This is intended for Google-internal usages only.

‎tests/validate-bom/action.yml

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
name: 'Maven BOM Validation'
2+
description: 'Validation for the content of a Maven BOM'
3+
inputs:
4+
bom-path:
5+
description: "The relative path from the repository root to the pom.xml file"
6+
required: true
7+
runs:
8+
using: "composite"
9+
steps:
10+
- uses: actions/setup-java@v3
11+
with:
12+
distribution: temurin
13+
java-version: 11
14+
cache: maven
15+
- name: Set up Maven
16+
uses: stCarolas/[email protected]
17+
with:
18+
maven-version: 3.8.4
19+
- name: Create temporary directory /tmp/bom-validation
20+
shell: bash
21+
run: mkdir -p /tmp/bom-validation
22+
- name: Create a canary project that uses the BOM
23+
shell: bash
24+
run: |
25+
if [ ! -r "${{ inputs.bom-path }}" ]; then
26+
echo "The input bom-path ${{ inputs.bom-path }} is not readable"
27+
exit 1
28+
fi
29+
30+
bom_absolute_path=$(realpath "${{ inputs.bom-path }}")
31+
32+
# Before this "cd", the working directory is the repository that calls
33+
# this action. To use validate-bom classes, it needs to change directory
34+
# to the directory that defines this action.
35+
cd ${{ github.action_path }}
36+
37+
echo "Compiling CreateBomCanaryProject.java in $(pwd)"
38+
mvn -V -ntp compile
39+
echo "Running CreateBomCanaryProject with ${bom_absolute_path}"
40+
mvn -V -ntp -B exec:java -DoutputPath=/tmp/bom-validation -DbomPath="${bom_absolute_path}"
41+
- name: Build the canary project that uses the BOM
42+
shell: bash
43+
working-directory: /tmp/bom-validation
44+
run: |
45+
echo "working directory: $(pwd)"
46+
mvn -ntp -B install
47+

‎tests/validate-bom/pom.xml

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
3+
<project xmlns="http://maven.apache.org/POM/4.0.0"
4+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
5+
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
6+
<parent>
7+
<artifactId>java-cloud-bom-root</artifactId>
8+
<groupId>com.google.cloud</groupId>
9+
<version>0.1.0</version>
10+
<relativePath>../../</relativePath>
11+
</parent>
12+
<modelVersion>4.0.0</modelVersion>
13+
14+
<artifactId>bom-canary-project-creation</artifactId>
15+
16+
<name>BOM Canary Project Creation</name>
17+
18+
<properties>
19+
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
20+
<maven.compiler.source>11</maven.compiler.source>
21+
<maven.compiler.target>11</maven.compiler.target>
22+
</properties>
23+
24+
<dependencies>
25+
<dependency>
26+
<groupId>com.google.cloud.tools</groupId>
27+
<artifactId>dependencies</artifactId>
28+
<version>1.5.13</version>
29+
</dependency>
30+
<dependency>
31+
<groupId>junit</groupId>
32+
<artifactId>junit</artifactId>
33+
<version>4.13.1</version>
34+
<scope>test</scope>
35+
</dependency>
36+
<dependency>
37+
<groupId>com.google.truth</groupId>
38+
<artifactId>truth</artifactId>
39+
<version>1.1.3</version>
40+
<scope>test</scope>
41+
</dependency>
42+
</dependencies>
43+
44+
<build>
45+
<plugins>
46+
<plugin>
47+
<groupId>org.codehaus.mojo</groupId>
48+
<artifactId>exec-maven-plugin</artifactId>
49+
<configuration>
50+
<skip>false</skip>
51+
<mainClass>com.google.cloud.CreateBomCanaryProject</mainClass>
52+
</configuration>
53+
</plugin>
54+
</plugins>
55+
</build>
56+
</project>

‎tests/validate-bom/src/main/java/com/google/cloud/CreateBomCanaryProject.java

Lines changed: 129 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,129 @@
1+
package com.google.cloud;
2+
3+
import static com.google.common.base.Preconditions.checkNotNull;
4+
5+
import com.google.cloud.tools.opensource.dependencies.Bom;
6+
import com.google.cloud.tools.opensource.dependencies.MavenRepositoryException;
7+
import com.google.common.base.Verify;
8+
import java.io.IOException;
9+
import java.io.InputStream;
10+
import java.nio.file.Files;
11+
import java.nio.file.Path;
12+
import java.nio.file.Paths;
13+
import java.util.Map;
14+
import org.eclipse.aether.artifact.Artifact;
15+
16+
/**
17+
* Creates a Maven project that uses the specified BOM at the specified directory. This class reads
18+
* the following system properties:
19+
*
20+
* <ul>
21+
* <li>outputPath: the path to the directory to create the Maven project
22+
* <li>bomPath: the path to the BOM
23+
* </ul>
24+
*/
25+
public class CreateBomCanaryProject {
26+
27+
public static void main(String[] arguments) throws Exception {
28+
String outputPathProperty = System.getProperty("outputPath");
29+
checkNotNull(outputPathProperty, "System property outputPath should not be null");
30+
Path outputProjectDirectory = Paths.get(outputPathProperty);
31+
String bomPathProperty = System.getProperty("bomPath");
32+
checkNotNull(bomPathProperty, "System property bomPath should not be null");
33+
Path bomPath = Paths.get(bomPathProperty);
34+
35+
Bom bom;
36+
try {
37+
bom = Bom.readBom(bomPath);
38+
} catch (MavenRepositoryException exception) {
39+
throw new IOException(
40+
"Could not read the BOM: "
41+
+ bomPath
42+
+ ". Please ensure all artifacts in the BOM are available in Maven Central or local"
43+
+ " Maven repository.",
44+
exception);
45+
}
46+
47+
String pomTemplate = readPomTemplate();
48+
49+
String dependencyManagementSection = calculateDependencyManagementSection(bom);
50+
String dependenciesSection = calculateDependenciesSection(bom);
51+
52+
String replacedContent =
53+
pomTemplate
54+
.replace("DEPENDENCY_MANAGEMENT", dependencyManagementSection)
55+
.replace("DEPENDENCIES", dependenciesSection);
56+
57+
Path pomToWrite = outputProjectDirectory.resolve("pom.xml");
58+
Files.write(pomToWrite, replacedContent.getBytes());
59+
System.out.println("Wrote " + pomToWrite);
60+
}
61+
62+
/** Returns the pom.xml template content. */
63+
private static String readPomTemplate() throws IOException {
64+
try (InputStream inputStream =
65+
CreateBomCanaryProject.class.getClassLoader().getResourceAsStream("template.pom.xml")) {
66+
Verify.verifyNotNull(inputStream);
67+
return new String(inputStream.readAllBytes());
68+
}
69+
}
70+
71+
/** Returns the dependencyManagement section to import {@code bom}. */
72+
private static String calculateDependencyManagementSection(Bom bom) {
73+
String[] coordinatesElements = bom.getCoordinates().split(":");
74+
Verify.verify(coordinatesElements.length == 3);
75+
String groupId = coordinatesElements[0];
76+
String artifactId = coordinatesElements[1];
77+
String version = coordinatesElements[2];
78+
79+
StringBuilder builder = new StringBuilder();
80+
builder.append(" <dependencyManagement>\n");
81+
builder.append(" <dependencies>\n");
82+
builder.append(" <dependency>\n");
83+
builder.append(" <groupId>").append(groupId).append("</groupId>\n");
84+
builder.append(" <artifactId>").append(artifactId).append("</artifactId>\n");
85+
builder.append(" <version>").append(version).append("</version>\n");
86+
builder.append(" <type>pom</type>\n");
87+
builder.append(" <scope>import</scope>\n");
88+
builder.append(" </dependency>\n");
89+
builder.append(" </dependencies>\n");
90+
builder.append(" </dependencyManagement>\n");
91+
return builder.toString();
92+
}
93+
94+
/** Returns the "dependencies" section that would declare all artifacts appear in {@code bom}. */
95+
private static String calculateDependenciesSection(Bom bom) {
96+
StringBuilder builder = new StringBuilder();
97+
builder.append(" <dependencies>\n");
98+
99+
for (Artifact managedDependency : bom.getManagedDependencies()) {
100+
Map<String, String> properties = managedDependency.getProperties();
101+
String classifier = managedDependency.getClassifier();
102+
if ("tests".equals(classifier)) {
103+
// Tests classifier artifacts are not for customers
104+
continue;
105+
}
106+
String type = properties.get("type");
107+
if ("pom".equals(type)) {
108+
// Some artifacts have :pom" type, such as io.grpc:protoc-gen-grpc-java
109+
// and com.google.api-client:google-api-client-assembly. We are only interested
110+
// in "jar" artifacts.
111+
continue;
112+
}
113+
114+
builder.append(" <dependency>\n");
115+
builder
116+
.append(" <groupId>")
117+
.append(managedDependency.getGroupId())
118+
.append("</groupId>\n");
119+
builder
120+
.append(" <artifactId>")
121+
.append(managedDependency.getArtifactId())
122+
.append("</artifactId>\n");
123+
builder.append(" </dependency>\n");
124+
}
125+
builder.append(" </dependencies>\n");
126+
127+
return builder.toString();
128+
}
129+
}

0 commit comments

Comments
 (0)