feat(v2): Add GraalVM reachability metadata for core utilities #1753

rr-on-gh · 2024-12-03T13:16:39Z

Issue #, if available: #764

Description of changes:

Adding GraalVM reachability metadata files for logging, metrics and tracing modules. Also added a SAM based sample application to demonstrate the usage.

Checklist

Meet tenets criteria
Update tests
Update docs
PR title follows conventional commit semantics

Breaking change checklist

RFC issue #:

Migration process documented
Implement warnings (if it can live side by side)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

sonarqubecloud · 2024-12-05T11:30:48Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

phipag · 2025-02-20T09:32:07Z

Hi @rr-on-gh, I tested the @Metrics part of your PR today and found it working with one minor change. I created a basic Quarkus app following the AWS Lambda documentation and compiled it to a native image.

For reference, this is my test Lambda handler:

@Named("test")
public class TestLambda implements RequestHandler<InputObject, OutputObject> {

    @Inject
    ProcessingService service;

    private final static Logger logger = Logger.getLogger(TestLambda.class);
    private final static MetricsLogger metricsLogger = MetricsUtils.metricsLogger();

    @Override
    @Metrics(namespace = "QuarkusApp", service = "powertools")
    public OutputObject handleRequest(InputObject input, Context context) {
        logger.info("Quarkus Jboss Log.");  // Logging with Jboss is not yet supported by Powertools

        long startTime = System.currentTimeMillis();
        final OutputObject output = service.process(input).setRequestId(context.getAwsRequestId());
        metricsLogger.putMetric("ExecutionTime", System.currentTimeMillis() - startTime, Unit.MILLISECONDS);
        metricsLogger.putMetric("Invocation", 1, Unit.COUNT);
        return output;
    }
}

When I compile this using Java version: 21.0.2+13, vendor version: GraalVM CE 21.0.2+13.1 it will raise an error coming from CloudWatch EMF:

2025-02-20 09:26:02,585 INFO  [qua.TestLambda] (Lambda Thread (NORMAL)) Quarkus Jboss Log.
2025-02-20 09:26:02,585 INFO  [sof.ama.clo.emf.env.AgentBasedEnvironment] (Lambda Thread (NORMAL)) Endpoint is not defined. Using default: tcp://127.0.0.1:25888
2025-02-20 09:26:02,585 WARN  [sof.ama.clo.emf.env.AgentBasedEnvironment] (Lambda Thread (NORMAL)) Unknown ServiceName.
2025-02-20 09:26:02,585 WARN  [sof.ama.clo.emf.env.AgentBasedEnvironment] (Lambda Thread (NORMAL)) Unknown ServiceName.
2025-02-20 09:26:02,585 WARN  [sof.ama.clo.emf.env.AgentBasedEnvironment] (Lambda Thread (NORMAL)) Unknown ServiceName.
2025-02-20 09:26:02,585 INFO  [sof.ama.clo.emf.env.DefaultEnvironment] (Lambda Thread (NORMAL)) Unknown ServiceType

The CloudWatch EMF dependency seems to think we are running in an agent-based environment and tries to connect to a CloudWatch agent endpoint on localhost. If I instruct the MetricsLogger manually to use the LambdaEnvironment the error is gone and the metrics are emitted as expected when deployed as native image.

--- a/powertools-metrics/src/main/java/software/amazon/lambda/powertools/metrics/MetricsUtils.java
+++ b/powertools-metrics/src/main/java/software/amazon/lambda/powertools/metrics/MetricsUtils.java
@@ -22,8 +22,10 @@ import static software.amazon.lambda.powertools.metrics.internal.LambdaMetricsAs
 import java.util.Arrays;
 import java.util.Optional;
 import java.util.function.Consumer;
+
 import software.amazon.cloudwatchlogs.emf.config.SystemWrapper;
 import software.amazon.cloudwatchlogs.emf.environment.EnvironmentProvider;
+import software.amazon.cloudwatchlogs.emf.environment.LambdaEnvironment;
 import software.amazon.cloudwatchlogs.emf.logger.MetricsLogger;
 import software.amazon.cloudwatchlogs.emf.model.DimensionSet;
 import software.amazon.cloudwatchlogs.emf.model.MetricsContext;
@@ -37,7 +39,7 @@ import software.amazon.cloudwatchlogs.emf.model.Unit;
  * {@see Metrics}
  */
 public final class MetricsUtils {
-    private static final MetricsLogger metricsLogger = new MetricsLogger();
+    private static final MetricsLogger metricsLogger = new MetricsLogger(new LambdaEnvironment());
     private static DimensionSet[] defaultDimensions;
 
     private MetricsUtils() {

Output:

2025-02-20 09:30:14,196 INFO  [qua.TestLambda] (Lambda Thread (NORMAL)) Quarkus Jboss Log.
{"_aws":{"Timestamp":1740043814196,"CloudWatchMetrics":[{"Namespace":"QuarkusApp","Metrics":[{"Name":"ExecutionTime","Unit":"Milliseconds"},{"Name":"Invocation","Unit":"Count"}],"Dimensions":[["Service"]]}]},"function_request_id":"f724bc2d-3c25-4721-b24c-291b086f822e","ExecutionTime":0.0,"xray_trace_id":"1-67b6f625-3fc6683351ac598520a2ce43","functionVersion":"$LATEST","Invocation":1.0,"Service":"powertools","logStreamId":"2025/02/20/[$LATEST]941343d9f3924a079878a93813e8ae84"}

Note: This behavior does not occur when using JVM.

phipag

Thanks @rr-on-gh for updating the PR and providing the GraalVM.md as documentation. It is clear now how you generated the GraalVM files. I only added a few minor comments regarding markdown issues.

I also tested the example you provided without problems.

phipag · 2025-03-10T09:59:10Z

examples/powertools-examples-core-utilities/sam-graalvm/README.md

+```shell 
+      export JAVA_HOME=<path to GraalVM>
+````


There is one backtick too much at the end. And can you remove the indentation inside the code block?

phipag · 2025-03-10T09:59:24Z

examples/powertools-examples-core-utilities/sam-graalvm/README.md

+
+- Build the Docker image that will be used as the environment for SAM build:
+```shell
+       docker build --platform linux/amd64 . -t powertools-examples-core-sam-graalvm


Please remove indentation.

phipag · 2025-03-10T09:59:29Z

examples/powertools-examples-core-utilities/sam-graalvm/README.md

+
+- Build the SAM project using the docker image
+```shell
+    sam build --use-container --build-image powertools-examples-core-sam-graalvm


Please remove indentation.

phipag · 2025-03-10T10:07:34Z

I also resolved the log4j version conflict with the v2 branch.

…tEnvironmentVariable annotation to override env variables.

…-logging-v2

phipag

This is great. Removing the static mocks is a big improvement and allows us to avoid skipping the unit tests using static mock.

Next step is to update the GraalVM.md documentation with the updated process of generating the metadata files.

phipag · 2025-03-12T09:37:45Z

...rc/test/java/software/amazon/lambda/powertools/logging/internal/LambdaLoggingAspectTest.java

@@ -62,6 +59,9 @@
 import org.junit.jupiter.api.BeforeEach;
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.params.ParameterizedTest;
+import org.junitpioneer.jupiter.ClearEnvironmentVariable;
+import org.junitpioneer.jupiter.SetEnvironmentVariable;
+import org.junitpioneer.jupiter.SetSystemProperty;
 import org.mockito.Mock;
 import org.mockito.MockedStatic;


This import is unused now.

The same for SystemWrapper in line 73.

...n/src/main/java/software/amazon/lambda/powertools/common/internal/UserAgentConfigurator.java

…ive image test.

phipag · 2025-03-12T13:04:50Z

I pushed a small fix to this PR e879e06. This unit test was failing in native mode (mvn -Pgraalvm-native clean test). I assume GraalVM has problems with the getContextClassLoader inside the test – I replaced with a temp file creation which passes both on JVM and native image mode.

Now, all tests pass when I run

mvn -Pgenerate-graalvm-files clean test && mvn -Pgraalvm-native clean test

phipag · 2025-03-12T13:09:51Z

Next steps for this PR:

Finalize documentation in GraalVM.md (all tests should now pass without skipping any such as StaticMock like in the initial version of the PR).
Create follow-up issues:
- Integrate GraalVM compilation and unit tests with v2 GitHub workflows (we should run them for both JVM and GraalVM)
- Update end-to-end tests to run both against JVM and GraalVM (for the modules that already support GraalVM)
Merge PR into v2 branch

…x/improve typos/language in documentation.

sonarqubecloud · 2025-03-13T11:45:48Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

phipag · 2025-03-13T11:47:10Z

I pushed some smaller cosmetic changes. This PR is ready to be merged as an initial version of GraalVM support for the core utilities. We need to iterate more on GraalVM, such as debugging support with Quarkus native images and adding GraalVM as additional tests in the GitHub workflows and end-to-end tests. I will create follow-up issues for these topics.

rr-on-gh added 11 commits November 29, 2024 18:35

Initial version of PT common GraalVM reachability metadata

fd40160

Initial version of PT logging GraalVM reachability metadata

ee03ae4

Initial version of PT serialization GraalVM reachability metadata

866fd82

Initial version of PT logging log4j GraalVM reachability metadata

b72280f

Initial version of PT logging log4j GraalVM reachability metadata

97650c0

Initial version of PT logging logback GraalVM reachability metadata

276984d

Example of PT logging with GraalVM

63105bf

Added GRM for powertools-metrics and update the example

1a2cad2

Added GRM for powertools-metrics and update the example

90fe66e

Cleanup of junit and mockito references

617a57c

Cleanup of junit and mockito references

a0791a4

pull-request-size bot added the size/XXL label Dec 3, 2024

Added documentation

7bfedb3

phipag self-requested a review February 20, 2025 09:32

rr-on-gh added 2 commits March 7, 2025 16:34

Added GraalVM documentation

e867523

Added GraalVM documentation

9b0b623

phipag mentioned this pull request Mar 10, 2025

GraalVM Support for Core Utilities (Logging, Metrics, Tracing) #764

Closed

phipag reviewed Mar 10, 2025

View reviewed changes

Merge branch 'v2' into graalvm-logging-v2

9759aad

phipag added this to Powertools for AWS Lambda (Java) Mar 10, 2025

phipag moved this to Pending review in Powertools for AWS Lambda (Java) Mar 10, 2025

phipag added v2 Version 2 feature-request New feature or request and removed feature-request New feature or request v2 Version 2 labels Mar 10, 2025

phipag removed the status in Powertools for AWS Lambda (Java) Mar 10, 2025

Added GRM files for tracing module. Also changed the tests to use @Se…

13f5cc4

…tEnvironmentVariable annotation to override env variables.

Merge remote-tracking branch 'origin/graalvm-logging-v2' into graalvm…

c45325a

…-logging-v2

phipag removed this from Powertools for AWS Lambda (Java) Mar 11, 2025

rr-on-gh added 4 commits March 11, 2025 16:05

Switch commons module to use JUnit Pioneer

a969e3c

Switch logging module to use JUnit Pioneer

ea9a1a2

Switch logging-log4j module to use JUnit Pioneer

d96539b

Switch metrics module to use JUnit Pioneer

d186e91

phipag reviewed Mar 12, 2025

View reviewed changes

rr-on-gh commented Mar 12, 2025

View reviewed changes

...n/src/main/java/software/amazon/lambda/powertools/common/internal/UserAgentConfigurator.java Show resolved Hide resolved

Fix testGetVersionFromProperties_InvalidFile test when running as nat…

e879e06

…ive image test.

phipag assigned phipag and rr-on-gh Mar 12, 2025

rr-on-gh and others added 2 commits March 12, 2025 18:38

Cleaned up documentation and fixed review comments

a1651ae

Cosmetic fixes/formatting. Re-order imports in changed test files, fi…

9b2b6e8

…x/improve typos/language in documentation.

phipag linked an issue Mar 13, 2025 that may be closed by this pull request

GraalVM Support for Core Utilities (Logging, Metrics, Tracing) #764

Closed

phipag merged commit 26f8b10 into aws-powertools:v2 Mar 13, 2025
13 of 14 checks passed

phipag mentioned this pull request Mar 13, 2025

Powertools Metrics CloudWatch EMF dependency does not work with Quarkus native image #1802

Open

phipag mentioned this pull request Mar 25, 2025

Bug: UserAgentConfigurator does not detect Powertools version when packaged as JAR #1812

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(v2): Add GraalVM reachability metadata for core utilities #1753

feat(v2): Add GraalVM reachability metadata for core utilities #1753

rr-on-gh commented Dec 3, 2024 •

edited

Loading

sonarqubecloud bot commented Dec 5, 2024

phipag commented Feb 20, 2025

phipag left a comment

phipag Mar 10, 2025

phipag Mar 10, 2025

phipag Mar 10, 2025

phipag commented Mar 10, 2025

phipag left a comment

phipag Mar 12, 2025

phipag Mar 12, 2025

phipag commented Mar 12, 2025

phipag commented Mar 12, 2025

sonarqubecloud bot commented Mar 13, 2025

phipag commented Mar 13, 2025

feat(v2): Add GraalVM reachability metadata for core utilities #1753

feat(v2): Add GraalVM reachability metadata for core utilities #1753

Conversation

rr-on-gh commented Dec 3, 2024 • edited Loading

Description of changes:

Breaking change checklist

sonarqubecloud bot commented Dec 5, 2024

Quality Gate passed

phipag commented Feb 20, 2025

phipag left a comment

Choose a reason for hiding this comment

phipag Mar 10, 2025

Choose a reason for hiding this comment

phipag Mar 10, 2025

Choose a reason for hiding this comment

phipag Mar 10, 2025

Choose a reason for hiding this comment

phipag commented Mar 10, 2025

phipag left a comment

Choose a reason for hiding this comment

phipag Mar 12, 2025

Choose a reason for hiding this comment

phipag Mar 12, 2025

Choose a reason for hiding this comment

phipag commented Mar 12, 2025

phipag commented Mar 12, 2025

sonarqubecloud bot commented Mar 13, 2025

Quality Gate passed

phipag commented Mar 13, 2025

rr-on-gh commented Dec 3, 2024 •

edited

Loading