Skip to content

Powertools Metrics CloudWatch EMF dependency does not work with Quarkus native image #1802

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
phipag opened this issue Mar 13, 2025 · 0 comments
Labels
bug Something isn't working graalvm help-wanted v2 Version 2

Comments

@phipag
Copy link
Contributor

phipag commented Mar 13, 2025

This issue was first raised in #764. When compiling a native image in Quarkus using the GraalVM Metadata files introduced as of #1753 CloudWatch EMF does not understand that it is running in a Lambda environment. It tries to ping a CloudWatch agent on localhost which breaks in Lambda.

This happens irrespective of setting environment variables at compile time and/or build time to indicated to the CloudWatch EMF library that it is running in a Lambda environment.

What were you trying to accomplish?
Running Powertools Metrics in Lambda when building with quarkus build --native.

Current Behavior

For reference, this is my test Lambda handler following the Quarkus AWS Lambda tutorial here https://quarkus.io/guides/aws-lambda:

@Named("test")
public class TestLambda implements RequestHandler<InputObject, OutputObject> {

    @Inject
    ProcessingService service;

    private final static Logger logger = Logger.getLogger(TestLambda.class);
    private final static MetricsLogger metricsLogger = MetricsUtils.metricsLogger();

    @Override
    @Metrics(namespace = "QuarkusApp", service = "powertools")
    public OutputObject handleRequest(InputObject input, Context context) {
        logger.info("Quarkus Jboss Log.");  // Logging with Jboss is not yet supported by Powertools

        long startTime = System.currentTimeMillis();
        final OutputObject output = service.process(input).setRequestId(context.getAwsRequestId());
        metricsLogger.putMetric("ExecutionTime", System.currentTimeMillis() - startTime, Unit.MILLISECONDS);
        metricsLogger.putMetric("Invocation", 1, Unit.COUNT);
        return output;
    }
}

When I compile this using Java version: 21.0.6+8-LTS, vendor version: Oracle GraalVM 21.0.6+8.1 it will raise an error at runtime coming from CloudWatch EMF. I compile on Amazon Linux 2 using quarkus build --native:

2025-03-10 09:08:37,172 INFO  [sof.ama.clo.emf.env.AgentBasedEnvironment] (Lambda Thread (NORMAL)) Endpoint is not defined. Using default: tcp://127.0.0.1:25888
2025-03-10 09:08:37,172 WARN  [sof.ama.clo.emf.env.AgentBasedEnvironment] (Lambda Thread (NORMAL)) Unknown ServiceName.
2025-03-10 09:08:37,190 WARN  [sof.ama.clo.emf.env.AgentBasedEnvironment] (Lambda Thread (NORMAL)) Unknown ServiceName.
2025-03-10 09:08:37,190 WARN  [sof.ama.clo.emf.env.AgentBasedEnvironment] (Lambda Thread (NORMAL)) Unknown ServiceName.
2025-03-10 09:08:37,230 INFO  [qua.TestLambda] (Lambda Thread (NORMAL)) Quarkus Jboss Log.
2025-03-10 09:08:37,230 WARN  [sof.ama.clo.emf.env.AgentBasedEnvironment] (Lambda Thread (NORMAL)) Unknown ServiceName.
2025-03-10 09:08:37,230 WARN  [sof.ama.clo.emf.env.AgentBasedEnvironment] (Lambda Thread (NORMAL)) Unknown ServiceName.

Possible solution and expected behavior

The CloudWatch EMF dependency seems to think we are running in an agent-based environment and tries to connect to a CloudWatch agent endpoint on localhost. If I instruct the MetricsLogger manually to use the LambdaEnvironment the error is gone and the metrics are emitted as expected when deployed as native image.

--- a/powertools-metrics/src/main/java/software/amazon/lambda/powertools/metrics/MetricsUtils.java
+++ b/powertools-metrics/src/main/java/software/amazon/lambda/powertools/metrics/MetricsUtils.java
@@ -22,8 +22,10 @@ import static software.amazon.lambda.powertools.metrics.internal.LambdaMetricsAs
 import java.util.Arrays;
 import java.util.Optional;
 import java.util.function.Consumer;
+
 import software.amazon.cloudwatchlogs.emf.config.SystemWrapper;
 import software.amazon.cloudwatchlogs.emf.environment.EnvironmentProvider;
+import software.amazon.cloudwatchlogs.emf.environment.LambdaEnvironment;
 import software.amazon.cloudwatchlogs.emf.logger.MetricsLogger;
 import software.amazon.cloudwatchlogs.emf.model.DimensionSet;
 import software.amazon.cloudwatchlogs.emf.model.MetricsContext;
@@ -37,7 +39,7 @@ import software.amazon.cloudwatchlogs.emf.model.Unit;
  * {@see Metrics}
  */
 public final class MetricsUtils {
-    private static final MetricsLogger metricsLogger = new MetricsLogger();
+    private static final MetricsLogger metricsLogger = new MetricsLogger(new LambdaEnvironment());
     private static DimensionSet[] defaultDimensions;
 
     private MetricsUtils() {

Output:

2025-02-20 09:30:14,196 INFO  [qua.TestLambda] (Lambda Thread (NORMAL)) Quarkus Jboss Log.
{"_aws":{"Timestamp":1741600482250,"CloudWatchMetrics":[{"Namespace":"QuarkusApp","Metrics":[{"Name":"ExecutionTime","Unit":"Milliseconds"},{"Name":"Invocation","Unit":"Count"}],"Dimensions":[["Service"]]}]},"function_request_id":"e9fd1c6a-5e48-459e-9516-95fe0b551e7d","ExecutionTime":0.0,"xray_trace_id":"1-67ceb6e1-3d2500803059e1091f383cb4","functionVersion":"$LATEST","Invocation":1.0,"Service":"powertools","logStreamId":"2025/03/10/[$LATEST]692c10c397eb41c0a32e41be73a85dbb"}

Setting the environment variables at both at runtime and at compile time does not change this behavior. Only overwriting the environment directly in the code seems to resolve this issue when using quarkus build --native.

AWS_LAMBDA_FUNCTION_NAME=PowertoolsNative
AWS_EMF_ENVIRONMENT=Lambda

Environment

  • Powertools for AWS Lambda (Java) version used: 2.0.0-SNAPSHOT
  • Packaging format (Layers, Maven/Gradle): Maven Quarkus Native Image
  • AWS Lambda function runtime: provided.al2023
@phipag phipag added bug Something isn't working graalvm help-wanted triage v2 Version 2 labels Mar 13, 2025
@phipag phipag moved this from Triage to Backlog in Powertools for AWS Lambda (Java) Mar 13, 2025
@phipag phipag added this to the v2 milestone Mar 13, 2025
@phipag phipag removed the triage label Mar 13, 2025
@phipag phipag changed the title Powertools Metrics CloudWatch EMF does not work with Quarkus native image Powertools Metrics CloudWatch EMF dependency does not work with Quarkus native image Mar 14, 2025
@phipag phipag removed this from the v2 milestone Apr 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working graalvm help-wanted v2 Version 2
Projects
Status: Backlog
Development

No branches or pull requests

1 participant