Skip to content

feat: Default dimensions from powertools instead of emf library #317

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Mar 10, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 29 additions & 1 deletion docs/core/metrics.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,7 +108,6 @@ if no metrics are provided no exception will be raised. If metrics are provided,
not met, `ValidationException` exception will be raised.

!!! tip "Metric validation"
* Minimum of 1 dimension
* Maximum of 9 dimensions

If you want to ensure that at least one metric is emitted, you can pass `raiseOnEmptyMetrics = true` to the **@Metrics** annotation:
Expand Down Expand Up @@ -183,6 +182,35 @@ You can use `putMetadata` for advanced use cases, where you want to metadata as

This will be available in CloudWatch Logs to ease operations on high cardinal data.

## Overriding default dimension set

By default, all metrics emitted via module captures `Service` as one of the default dimension. This is either specified via
`POWERTOOLS_SERVICE_NAME` environment variable or via `service` attribute on `Metrics` annotation. If you wish to override the default
Dimension, it can be done via `#!java MetricsUtils.defaultDimensionSet()`.

=== "App.java"

```java hl_lines="8 9 10"
import software.amazon.lambda.powertools.metrics.Metrics;
import static software.amazon.lambda.powertools.metrics.MetricsUtils;

public class App implements RequestHandler<Object, Object> {

MetricsLogger metricsLogger = MetricsUtils.metricsLogger();

static {
MetricsUtils.defaultDimensionSet(DimensionSet.of("CustomDimension", "booking"));
}

@Override
@Metrics(namespace = "ExampleApplication", service = "booking")
public Object handleRequest(Object input, Context context) {
...
MetricsUtils.withSingleMetric("Metric2", 1, Unit.COUNT, log -> {});
}
}
```

## Creating a metric with a different dimension

CloudWatch EMF uses the same dimensions across all your metrics. Use `withSingleMetric` if you have a metric that should have different dimensions.
Expand Down
4 changes: 3 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,9 @@ You can use [SAM](https://aws.amazon.com/serverless/sam/) to quickly setup a ser

For more information about the project and available options refer to this [repository](https://github.com/aws-samples/cookiecutter-aws-sam-powertools-java/blob/main/README.md)

!!! note "Using Java 9 or later?"
If you are working with lambda function on runtime **Java 9 or later**, please refer **[issue](https://github.com/awslabs/aws-lambda-powertools-java/issues/50)** for a workaround.

=== "Maven"

```xml hl_lines="3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55"
Expand Down Expand Up @@ -98,7 +101,6 @@ For more information about the project and available options refer to this [repo
</plugins>
</build>
```
**Note:** If you are working with lambda function on runtime post java8, please refer [issue](https://github.com/awslabs/aws-lambda-powertools-java/issues/50) for workaround.

=== "Gradle"

Expand Down
3 changes: 3 additions & 0 deletions docs/utilities/batch.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@ are returned to the queue.

To install this utility, add the following dependency to your project.

!!! note "Using Java 9 or later?"
If you are working with lambda function on runtime **Java 9 or later**, please refer **[issue](https://github.com/awslabs/aws-lambda-powertools-java/issues/50)** for a workaround.

=== "Maven"
```xml hl_lines="3 4 5 6 7 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36"
<dependencies>
Expand Down
30 changes: 14 additions & 16 deletions docs/utilities/sqs_large_message_handling.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,23 +33,21 @@ This utility is compatible with versions *[1.1.0+](https://github.com/awslabs/am

To install this utility, add the following dependency to your project.

=== "Maven"
```xml
<dependency>
<groupId>software.amazon.lambda</groupId>
<artifactId>powertools-sqs</artifactId>
<version>1.3.0</version>
</dependency>
```
!!! note "Using Java 9 or later?"
If you are working with lambda function on runtime **Java 9 or later**, please refer **[issue](https://github.com/awslabs/aws-lambda-powertools-java/issues/50)** for a workaround.

=== "Maven Configuration"

Configure the aspectj-maven-plugin to compile-time weave (CTW) the
aws-lambda-powertools-java aspects into your project. You may already have this
plugin in your pom. In that case add the dependency to the `aspectLibraries`
section.

```xml hl_lines="13 14 15 16"
=== "Maven"
```xml hl_lines="3 4 5 6 7 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36"
<dependencies>
...
<dependency>
<groupId>software.amazon.lambda</groupId>
<artifactId>powertools-sqs</artifactId>
<version>1.3.0</version>
</dependency>
...
</dependencies>
<!-- configure the aspectj-maven-plugin to compile-time weave (CTW) the aws-lambda-powertools-java aspects into your project -->
<build>
<plugins>
...
Expand Down
20 changes: 9 additions & 11 deletions docs/utilities/validation.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,23 +15,21 @@ This utility provides JSON Schema validation for payloads held within events and

To install this utility, add the following dependency to your project.

!!! note "Using Java 9 or later?"
If you are working with lambda function on runtime **Java 9 or later**, please refer **[issue](https://github.com/awslabs/aws-lambda-powertools-java/issues/50)** for a workaround.

=== "Maven"
```xml
```xml hl_lines="3 4 5 6 7 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36"
<dependencies>
...
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>powertools-validation</artifactId>
<version>1.3.0</version>
</dependency>
```

=== "Maven Configuration"

Configure the aspectj-maven-plugin to compile-time weave (CTW) the
aws-lambda-powertools-java aspects into your project. You may already have this
plugin in your pom. In that case add the dependency to the `aspectLibraries`
section.

```xml hl_lines="13 14 15 16"
...
</dependencies>
<!-- configure the aspectj-maven-plugin to compile-time weave (CTW) the aws-lambda-powertools-java aspects into your project -->
<build>
<plugins>
...
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,14 @@
import java.util.function.Consumer;

import software.amazon.cloudwatchlogs.emf.config.SystemWrapper;
import software.amazon.cloudwatchlogs.emf.environment.EnvironmentProvider;
import software.amazon.cloudwatchlogs.emf.logger.MetricsLogger;
import software.amazon.cloudwatchlogs.emf.model.DimensionSet;
import software.amazon.cloudwatchlogs.emf.model.MetricsContext;
import software.amazon.cloudwatchlogs.emf.model.MetricsLoggerHelper;
import software.amazon.cloudwatchlogs.emf.model.Unit;

import static java.util.Objects.requireNonNull;
import static java.util.Optional.ofNullable;
import static software.amazon.lambda.powertools.core.internal.LambdaHandlerProcessor.getXrayTraceId;
import static software.amazon.lambda.powertools.metrics.internal.LambdaMetricsAspect.REQUEST_ID_PROPERTY;
Expand All @@ -22,6 +25,7 @@
*/
public final class MetricsUtils {
private static final MetricsLogger metricsLogger = new MetricsLogger();
private static DimensionSet defaultDimensionSet;

private MetricsUtils() {
}
Expand All @@ -35,11 +39,22 @@ public static MetricsLogger metricsLogger() {
return metricsLogger;
}

/**
* Configure default dimension to be used by logger.
* By default, @{@link Metrics} annotation captures configured service as a dimension <i>Service</i>
* @param dimensionSet Default value of dimension set for logger
*/
public static void defaultDimensionSet(final DimensionSet dimensionSet) {
requireNonNull(dimensionSet, "Null dimension set not allowed");
MetricsUtils.defaultDimensionSet = dimensionSet;
}


/**
* Add and immediately flush a single metric. It will use the default namespace
* specified either on {@link Metrics} annotation or via POWERTOOLS_METRICS_NAMESPACE env var.
* It by default captures AwsRequestId as property if used together with {@link Metrics} annotation. It will also
* capture XrayTraceId as property if tracing is enabled.
* It by default captures function_request_id as property if used together with {@link Metrics} annotation. It will also
* capture xray_trace_id as property if tracing is enabled.
*
* @param name the name of the metric
* @param value the value of the metric
Expand All @@ -50,7 +65,8 @@ public static void withSingleMetric(final String name,
final double value,
final Unit unit,
final Consumer<MetricsLogger> logger) {
MetricsLogger metricsLogger = new MetricsLogger();
MetricsLogger metricsLogger = logger();

try {
metricsLogger.setNamespace(defaultNameSpace());
metricsLogger.putMetric(name, value, unit);
Expand All @@ -63,8 +79,8 @@ public static void withSingleMetric(final String name,

/**
* Add and immediately flush a single metric.
* It by default captures AwsRequestId as property if used together with {@link Metrics} annotation. It will also
* capture XrayTraceId as property if tracing is enabled.
* It by default captures function_request_id as property if used together with {@link Metrics} annotation. It will also
* capture xray_trace_id as property if tracing is enabled.
*
* @param name the name of the metric
* @param value the value of the metric
Expand All @@ -77,7 +93,8 @@ public static void withSingleMetric(final String name,
final Unit unit,
final String namespace,
final Consumer<MetricsLogger> logger) {
MetricsLogger metricsLogger = new MetricsLogger();
MetricsLogger metricsLogger = logger();

try {
metricsLogger.setNamespace(namespace);
metricsLogger.putMetric(name, value, unit);
Expand All @@ -88,6 +105,14 @@ public static void withSingleMetric(final String name,
}
}

public static DimensionSet defaultDimensionSet() {
return defaultDimensionSet;
}

public static boolean hasDefaultDimension() {
return defaultDimensionSet.getDimensionKeys().size() > 0;
}

private static void captureRequestAndTraceId(MetricsLogger metricsLogger) {
awsRequestId().
ifPresent(requestId -> metricsLogger.putProperty(REQUEST_ID_PROPERTY, requestId));
Expand All @@ -107,4 +132,14 @@ private static Optional<String> awsRequestId() {
return ofNullable(context.getProperty(REQUEST_ID_PROPERTY))
.map(Object::toString);
}

private static MetricsLogger logger() {
MetricsContext metricsContext = new MetricsContext();

if (hasDefaultDimension()) {
metricsContext.setDefaultDimensions(defaultDimensionSet());
}

return new MetricsLogger(new EnvironmentProvider(), metricsContext);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@
import static software.amazon.lambda.powertools.core.internal.LambdaHandlerProcessor.placedOnRequestHandler;
import static software.amazon.lambda.powertools.core.internal.LambdaHandlerProcessor.placedOnStreamHandler;
import static software.amazon.lambda.powertools.core.internal.LambdaHandlerProcessor.serviceName;
import static software.amazon.lambda.powertools.metrics.MetricsUtils.defaultDimensionSet;
import static software.amazon.lambda.powertools.metrics.MetricsUtils.hasDefaultDimension;
import static software.amazon.lambda.powertools.metrics.MetricsUtils.metricsLogger;

@Aspect
Expand All @@ -47,8 +49,9 @@ public Object around(ProceedingJoinPoint pjp,

MetricsLogger logger = metricsLogger();

logger.setNamespace(namespace(metrics))
.putDimensions(DimensionSet.of("Service", service(metrics)));
refreshMetricsContext(metrics);

logger.setNamespace(namespace(metrics));

extractContext(pjp).ifPresent((context) -> {
coldStartSingleMetricIfApplicable(context.getAwsRequestId(), context.getFunctionName(), metrics);
Expand All @@ -65,7 +68,7 @@ public Object around(ProceedingJoinPoint pjp,
coldStartDone();
validateMetricsAndRefreshOnFailure(metrics);
logger.flush();
refreshMetricsContext();
refreshMetricsContext(metrics);
}
}

Expand All @@ -92,8 +95,8 @@ private void validateBeforeFlushingMetrics(Metrics metrics) {
throw new ValidationException("No metrics captured, at least one metrics must be emitted");
}

if (dimensionsCount() == 0 || dimensionsCount() > 9) {
throw new ValidationException(String.format("Number of Dimensions must be in range of 1-9." +
if (dimensionsCount() > 9) {
throw new ValidationException(String.format("Number of Dimensions must be in range of 0-9." +
" Actual size: %d.", dimensionsCount()));
}
}
Expand All @@ -102,25 +105,32 @@ private String namespace(Metrics metrics) {
return !"".equals(metrics.namespace()) ? metrics.namespace() : NAMESPACE;
}

private String service(Metrics metrics) {
private static String service(Metrics metrics) {
return !"".equals(metrics.service()) ? metrics.service() : serviceName();
}

private void validateMetricsAndRefreshOnFailure(Metrics metrics) {
try {
validateBeforeFlushingMetrics(metrics);
} catch (ValidationException e){
refreshMetricsContext();
refreshMetricsContext(metrics);
throw e;
}
}

// This can be simplified after this issues https://github.com/awslabs/aws-embedded-metrics-java/issues/35 is fixed
private static void refreshMetricsContext() {
public static void refreshMetricsContext(Metrics metrics) {
try {
Field f = metricsLogger().getClass().getDeclaredField("context");
f.setAccessible(true);
f.set(metricsLogger(), new MetricsContext());
MetricsContext context = new MetricsContext();

DimensionSet defaultDimensionSet = hasDefaultDimension() ? defaultDimensionSet()
: DimensionSet.of("Service", service(metrics));

context.setDefaultDimensions(defaultDimensionSet);

f.set(metricsLogger(), context);
} catch (NoSuchFieldException | IllegalAccessException e) {
throw new RuntimeException(e);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.assertj.core.api.Assertions;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.BeforeEach;
Expand All @@ -16,6 +17,7 @@
import software.amazon.cloudwatchlogs.emf.model.Unit;

import static java.util.Collections.*;
import static org.assertj.core.api.Assertions.*;
import static org.assertj.core.api.Assertions.assertThat;
import static org.mockito.Mockito.mockStatic;
import static software.amazon.lambda.powertools.core.internal.SystemWrapper.getenv;
Expand Down Expand Up @@ -43,6 +45,31 @@ static void beforeAll() {
}
}

@Test
void singleMetricsCaptureUtilityWithDefaultDimension() {
try (MockedStatic<SystemWrapper> mocked = mockStatic(SystemWrapper.class);
MockedStatic<software.amazon.lambda.powertools.core.internal.SystemWrapper> internalWrapper = mockStatic(software.amazon.lambda.powertools.core.internal.SystemWrapper.class)) {
mocked.when(() -> SystemWrapper.getenv("AWS_EMF_ENVIRONMENT")).thenReturn("Lambda");
internalWrapper.when(() -> getenv("_X_AMZN_TRACE_ID")).thenReturn("Root=1-5759e988-bd862e3fe1be46a994272793;Parent=53995c3f42cd8ad8;Sampled=1\"");

MetricsUtils.defaultDimensionSet(DimensionSet.of("Service", "Booking"));

MetricsUtils.withSingleMetric("Metric1", 1, Unit.COUNT, "test",
metricsLogger -> {});

assertThat(out.toString())
.satisfies(s -> {
Map<String, Object> logAsJson = readAsJson(s);

assertThat(logAsJson)
.containsEntry("Metric1", 1.0)
.containsEntry("Service", "Booking")
.containsKey("_aws")
.containsEntry("xray_trace_id", "1-5759e988-bd862e3fe1be46a994272793");
});
}
}

@Test
void singleMetricsCaptureUtility() {
try (MockedStatic<SystemWrapper> mocked = mockStatic(SystemWrapper.class);
Expand Down Expand Up @@ -96,6 +123,13 @@ void singleMetricsCaptureUtilityWithDefaultNameSpace() {
}
}

@Test
void shouldThrowExceptionWhenDefaultDimensionIsNull() {
assertThatNullPointerException()
.isThrownBy(() -> MetricsUtils.defaultDimensionSet(null))
.withMessage("Null dimension set not allowed");
}

private Map<String, Object> readAsJson(String s) {
try {
return mapper.readValue(s, Map.class);
Expand Down
Loading