feat(redis) - merging from develop

Author: leandrodamascena

.github/workflows/publish_v2_layer.yml

@@ -65,7 +65,7 @@ jobs:
         # NOTE: we need QEMU to build Layer against a different architecture (e.g., ARM)
       - name: Set up Docker Buildx
         id: builder
-        uses: docker/setup-buildx-action@dc7b9719a96d48369863986a06765841d7ea23f6 # v2.0.0
+        uses: docker/setup-buildx-action@f03ac48505955848960e80bbb68046aa35c7b9e7 # v2.4.1
         with:
           install: true
           driver: docker

.github/workflows/run-e2e-tests.yml

@@ -26,6 +26,7 @@ jobs:
       id-token: write # needed to request JWT with GitHub's OIDC Token endpoint. docs: https://bit.ly/3MNgQO9
       contents: read
     strategy:
+      fail-fast: false # needed so if a version fails, the others will still be able to complete and cleanup
       matrix:
         version: ["3.7", "3.8", "3.9"]
     if: ${{ github.actor != 'dependabot[bot]' }}

CHANGELOG.md

@@ -6,23 +6,72 @@
 
 ## Bug Fixes
 
+* **idempotency:** make idempotent_function decorator thread safe ([#1899](https://github.com/awslabs/aws-lambda-powertools-python/issues/1899))
+
+## Documentation
+
+* **homepage:** set url for end-of-support in announce block ([#1893](https://github.com/awslabs/aws-lambda-powertools-python/issues/1893))
+* **idempotency:** add IAM permissions section ([#1902](https://github.com/awslabs/aws-lambda-powertools-python/issues/1902))
+
+## Features
+
+* **metrics:** add default_dimensions to single_metric ([#1880](https://github.com/awslabs/aws-lambda-powertools-python/issues/1880))
+
+## Maintenance
+
+* **deps:** bump docker/setup-buildx-action from 2.4.0 to 2.4.1 ([#1903](https://github.com/awslabs/aws-lambda-powertools-python/issues/1903))
+* **deps-dev:** bump mypy-boto3-appconfig from 1.26.0.post1 to 1.26.63 ([#1895](https://github.com/awslabs/aws-lambda-powertools-python/issues/1895))
+* **deps-dev:** bump types-requests from 2.28.11.8 to 2.28.11.12 ([#1906](https://github.com/awslabs/aws-lambda-powertools-python/issues/1906))
+* **deps-dev:** bump pytest-xdist from 3.1.0 to 3.2.0 ([#1905](https://github.com/awslabs/aws-lambda-powertools-python/issues/1905))
+* **deps-dev:** bump aws-cdk-lib from 2.63.0 to 2.63.2 ([#1904](https://github.com/awslabs/aws-lambda-powertools-python/issues/1904))
+* **deps-dev:** bump mkdocs-material from 9.0.10 to 9.0.11 ([#1896](https://github.com/awslabs/aws-lambda-powertools-python/issues/1896))
+* **deps-dev:** bump mkdocs-material from 9.0.9 to 9.0.10 ([#1888](https://github.com/awslabs/aws-lambda-powertools-python/issues/1888))
+* **deps-dev:** bump mypy-boto3-s3 from 1.26.58 to 1.26.62 ([#1889](https://github.com/awslabs/aws-lambda-powertools-python/issues/1889))
+* **deps-dev:** bump black from 22.12.0 to 23.1.0 ([#1886](https://github.com/awslabs/aws-lambda-powertools-python/issues/1886))
+* **deps-dev:** bump aws-cdk-lib from 2.62.2 to 2.63.0 ([#1887](https://github.com/awslabs/aws-lambda-powertools-python/issues/1887))
+* **maintainers:** fix release workflow rename
+* **pypi:** add new links to Pypi package homepage ([#1912](https://github.com/awslabs/aws-lambda-powertools-python/issues/1912))
+
+
+<a name="v2.7.1"></a>
+## [v2.7.1] - 2023-02-01
+## Bug Fixes
+
 * parallel_run should fail when e2e tests fail
 * bump aws-cdk version
 * **ci:** scope e2e tests by python version
+* **ci:** add auth to API HTTP Gateway and Lambda Function Url ([#1882](https://github.com/awslabs/aws-lambda-powertools-python/issues/1882))
+* **license:** correction to MIT + MIT-0 (no proprietary anymore) ([#1883](https://github.com/awslabs/aws-lambda-powertools-python/issues/1883))
+* **license:** add MIT-0 license header ([#1871](https://github.com/awslabs/aws-lambda-powertools-python/issues/1871))
+* **tests:** make logs fetching more robust ([#1878](https://github.com/awslabs/aws-lambda-powertools-python/issues/1878))
+* **tests:** remove custom workers
+* **tests:** make sure multiple e2e tests run concurrently ([#1861](https://github.com/awslabs/aws-lambda-powertools-python/issues/1861))
 
 ## Documentation
 
 * **event-source:**  fix incorrect method in example CloudWatch Logs ([#1857](https://github.com/awslabs/aws-lambda-powertools-python/issues/1857))
+* **homepage:** add banner for end-of-support v1 ([#1879](https://github.com/awslabs/aws-lambda-powertools-python/issues/1879))
+* **parameters:** snippets split, improved, and lint ([#1564](https://github.com/awslabs/aws-lambda-powertools-python/issues/1564))
 
 ## Maintenance
 
+* update v2 layer ARN on documentation
+* **deps:** bump docker/setup-buildx-action from 2.0.0 to 2.4.0 ([#1873](https://github.com/awslabs/aws-lambda-powertools-python/issues/1873))
 * **deps:** bump dependabot/fetch-metadata from 1.3.5 to 1.3.6 ([#1855](https://github.com/awslabs/aws-lambda-powertools-python/issues/1855))
+* **deps-dev:** bump mypy-boto3-s3 from 1.26.0.post1 to 1.26.58 ([#1868](https://github.com/awslabs/aws-lambda-powertools-python/issues/1868))
+* **deps-dev:** bump isort from 5.11.4 to 5.11.5 ([#1875](https://github.com/awslabs/aws-lambda-powertools-python/issues/1875))
+* **deps-dev:** bump aws-cdk-lib from 2.62.1 to 2.62.2 ([#1869](https://github.com/awslabs/aws-lambda-powertools-python/issues/1869))
+* **deps-dev:** bump mkdocs-material from 9.0.6 to 9.0.8 ([#1874](https://github.com/awslabs/aws-lambda-powertools-python/issues/1874))
 * **deps-dev:** bump aws-cdk-lib from 2.62.0 to 2.62.1 ([#1866](https://github.com/awslabs/aws-lambda-powertools-python/issues/1866))
 * **deps-dev:** bump mypy-boto3-cloudformation from 1.26.35.post1 to 1.26.57 ([#1865](https://github.com/awslabs/aws-lambda-powertools-python/issues/1865))
 * **deps-dev:** bump coverage from 7.0.5 to 7.1.0 ([#1862](https://github.com/awslabs/aws-lambda-powertools-python/issues/1862))
 * **deps-dev:** bump aws-cdk-lib from 2.61.1 to 2.62.0 ([#1863](https://github.com/awslabs/aws-lambda-powertools-python/issues/1863))
+* **deps-dev:** bump flake8-bugbear from 22.12.6 to 23.1.20 ([#1854](https://github.com/awslabs/aws-lambda-powertools-python/issues/1854))
 * **deps-dev:** bump mypy-boto3-lambda from 1.26.49 to 1.26.55 ([#1856](https://github.com/awslabs/aws-lambda-powertools-python/issues/1856))
 
+## Reverts
+* fix(tests): remove custom workers
+
 
 <a name="v2.7.0"></a>
 ## [v2.7.0] - 2023-01-24
@@ -2801,7 +2850,8 @@
 * Merge pull request [#5](https://github.com/awslabs/aws-lambda-powertools-python/issues/5) from jfuss/feat/python38
 
 
-[Unreleased]: https://github.com/awslabs/aws-lambda-powertools-python/compare/v2.7.0...HEAD
+[Unreleased]: https://github.com/awslabs/aws-lambda-powertools-python/compare/v2.7.1...HEAD
+[v2.7.1]: https://github.com/awslabs/aws-lambda-powertools-python/compare/v2.7.0...v2.7.1
 [v2.7.0]: https://github.com/awslabs/aws-lambda-powertools-python/compare/v2.6.0...v2.7.0
 [v2.6.0]: https://github.com/awslabs/aws-lambda-powertools-python/compare/v2.5.0...v2.6.0
 [v2.5.0]: https://github.com/awslabs/aws-lambda-powertools-python/compare/v2.4.0...v2.5.0

LICENSE

@@ -1,3 +1,5 @@
+MIT No Attribution
+
 Copyright 2019 Amazon.com, Inc. or its affiliates. All Rights Reserved.
 
 Permission is hereby granted, free of charge, to any person obtaining a copy of

MAINTAINERS.md

@@ -218,7 +218,7 @@ These are some questions to keep in mind when drafting your first or future rele
 
 Once you're happy, hit `Publish release` 🎉🎉🎉.
 
-This will kick off the [Publishing workflow](https://github.com/awslabs/aws-lambda-powertools-python/actions/workflows/on_release_notes.yml) and within a few minutes you should see the latest version in PyPi, and all issues labeled as `pending-release` will be closed and notified.
+This will kick off the [Publishing workflow](https://github.com/awslabs/aws-lambda-powertools-python/actions/workflows/release.yml) and within a few minutes you should see the latest version in PyPi, and all issues labeled as `pending-release` will be closed and notified.
 
 > TODO: Include information to verify SAR and Lambda Layers deployment; we're still finalizing Lambda Layer automated deployment in GitHub Actions - ping @am29d when in doubt.
 

aws_lambda_powertools/event_handler/api_gateway.py

@@ -836,7 +836,6 @@ def route(
     # Override _compile_regex to exclude trailing slashes for route resolution
     @staticmethod
     def _compile_regex(rule: str, base_regex: str = _ROUTE_REGEX):
-
         return super(APIGatewayRestResolver, APIGatewayRestResolver)._compile_regex(rule, "^{}/*$")
 
 

aws_lambda_powertools/logging/utils.py

@@ -12,7 +12,6 @@ def copy_config_to_registered_loggers(
     exclude: Optional[Set[str]] = None,
     include: Optional[Set[str]] = None,
 ) -> None:
-
     """Copies source Logger level and handler to all registered loggers for consistent formatting.
 
     Parameters

aws_lambda_powertools/metrics/base.py

@@ -463,7 +463,11 @@ def add_metric(self, name: str, unit: Union[MetricUnit, str], value: float) -> N
 
 @contextmanager
 def single_metric(
-    name: str, unit: MetricUnit, value: float, namespace: Optional[str] = None
+    name: str,
+    unit: MetricUnit,
+    value: float,
+    namespace: Optional[str] = None,
+    default_dimensions: Optional[Dict[str, str]] = None,
 ) -> Generator[SingleMetric, None, None]:
     """Context manager to simplify creation of a single metric
 
@@ -516,6 +520,11 @@ def single_metric(
     try:
         metric: SingleMetric = SingleMetric(namespace=namespace)
         metric.add_metric(name=name, unit=unit, value=value)
+
+        if default_dimensions:
+            for dim_name, dim_value in default_dimensions.items():
+                metric.add_dimension(name=dim_name, value=dim_value)
+
         yield metric
         metric_set = metric.serialize_metric_set()
     finally:

aws_lambda_powertools/utilities/feature_flags/schema.py

@@ -272,7 +272,6 @@ def __init__(self, rule: Dict[str, Any], rule_name: str, logger: Optional[Union[
         self.logger = logger or logging.getLogger(__name__)
 
     def validate(self):
-
         if not self.conditions or not isinstance(self.conditions, list):
             self.logger.debug(f"Condition is empty or invalid for rule={self.rule_name}")
             raise SchemaValidationError(f"Invalid condition, rule={self.rule_name}")

aws_lambda_powertools/utilities/idempotency/persistence/dynamodb.py

@@ -4,7 +4,9 @@
 from typing import Any, Dict, Optional
 
 import boto3
+from boto3.dynamodb.types import TypeDeserializer
 from botocore.config import Config
+from botocore.exceptions import ClientError
 
 from aws_lambda_powertools.shared import constants
 from aws_lambda_powertools.utilities.idempotency import BasePersistenceLayer
@@ -79,13 +81,14 @@ def __init__(
 
         self._boto_config = boto_config or Config()
         self._boto3_session = boto3_session or boto3.session.Session()
+        self._client = self._boto3_session.client("dynamodb", config=self._boto_config)
+
         if sort_key_attr == key_attr:
             raise ValueError(f"key_attr [{key_attr}] and sort_key_attr [{sort_key_attr}] cannot be the same!")
 
         if static_pk_value is None:
             static_pk_value = f"idempotency#{os.getenv(constants.LAMBDA_FUNCTION_NAME_ENV, '')}"
 
-        self._table = None
         self.table_name = table_name
         self.key_attr = key_attr
         self.static_pk_value = static_pk_value
@@ -95,31 +98,15 @@ def __init__(
         self.status_attr = status_attr
         self.data_attr = data_attr
         self.validation_key_attr = validation_key_attr
-        super(DynamoDBPersistenceLayer, self).__init__()
 
-    @property
-    def table(self):
-        """
-        Caching property to store boto3 dynamodb Table resource
+        self._deserializer = TypeDeserializer()
 
-        """
-        if self._table:
-            return self._table
-        ddb_resource = self._boto3_session.resource("dynamodb", config=self._boto_config)
-        self._table = ddb_resource.Table(self.table_name)
-        return self._table
-
-    @table.setter
-    def table(self, table):
-        """
-        Allow table instance variable to be set directly, primarily for use in tests
-        """
-        self._table = table
+        super(DynamoDBPersistenceLayer, self).__init__()
 
     def _get_key(self, idempotency_key: str) -> dict:
         if self.sort_key_attr:
-            return {self.key_attr: self.static_pk_value, self.sort_key_attr: idempotency_key}
-        return {self.key_attr: idempotency_key}
+            return {self.key_attr: {"S": self.static_pk_value}, self.sort_key_attr: {"S": idempotency_key}}
+        return {self.key_attr: {"S": idempotency_key}}
 
     def _item_to_data_record(self, item: Dict[str, Any]) -> DataRecord:
         """
@@ -136,36 +123,39 @@ def _item_to_data_record(self, item: Dict[str, Any]) -> DataRecord:
             representation of item
 
         """
+        data = self._deserializer.deserialize({"M": item})
         return DataRecord(
-            idempotency_key=item[self.key_attr],
-            status=item[self.status_attr],
-            expiry_timestamp=item[self.expiry_attr],
-            in_progress_expiry_timestamp=item.get(self.in_progress_expiry_attr),
-            response_data=item.get(self.data_attr),
-            payload_hash=item.get(self.validation_key_attr),
+            idempotency_key=data[self.key_attr],
+            status=data[self.status_attr],
+            expiry_timestamp=data[self.expiry_attr],
+            in_progress_expiry_timestamp=data.get(self.in_progress_expiry_attr),
+            response_data=data.get(self.data_attr),
+            payload_hash=data.get(self.validation_key_attr),
         )
 
     def _get_record(self, idempotency_key) -> DataRecord:
-        response = self.table.get_item(Key=self._get_key(idempotency_key), ConsistentRead=True)
-
+        response = self._client.get_item(
+            TableName=self.table_name, Key=self._get_key(idempotency_key), ConsistentRead=True
+        )
         try:
             item = response["Item"]
-        except KeyError:
-            raise IdempotencyItemNotFoundError
+        except KeyError as exc:
+            raise IdempotencyItemNotFoundError from exc
         return self._item_to_data_record(item)
 
     def _put_record(self, data_record: DataRecord) -> None:
         item = {
             **self._get_key(data_record.idempotency_key),
-            self.expiry_attr: data_record.expiry_timestamp,
-            self.status_attr: data_record.status,
+            self.key_attr: {"S": data_record.idempotency_key},
+            self.expiry_attr: {"N": str(data_record.expiry_timestamp)},
+            self.status_attr: {"S": data_record.status},
         }
 
         if data_record.in_progress_expiry_timestamp is not None:
-            item[self.in_progress_expiry_attr] = data_record.in_progress_expiry_timestamp
+            item[self.in_progress_expiry_attr] = {"N": str(data_record.in_progress_expiry_timestamp)}
 
-        if self.payload_validation_enabled:
-            item[self.validation_key_attr] = data_record.payload_hash
+        if self.payload_validation_enabled and data_record.payload_hash:
+            item[self.validation_key_attr] = {"S": data_record.payload_hash}
 
         now = datetime.datetime.now()
         try:
@@ -199,8 +189,8 @@ def _put_record(self, data_record: DataRecord) -> None:
             condition_expression = (
                 f"{idempotency_key_not_exist} OR {idempotency_expiry_expired} OR ({inprogress_expiry_expired})"
             )
-
-            self.table.put_item(
+            self._client.put_item(
+                TableName=self.table_name,
                 Item=item,
                 ConditionExpression=condition_expression,
                 ExpressionAttributeNames={
@@ -210,22 +200,28 @@ def _put_record(self, data_record: DataRecord) -> None:
                     "#status": self.status_attr,
                 },
                 ExpressionAttributeValues={
-                    ":now": int(now.timestamp()),
-                    ":now_in_millis": int(now.timestamp() * 1000),
-                    ":inprogress": STATUS_CONSTANTS["INPROGRESS"],
+                    ":now": {"N": str(int(now.timestamp()))},
+                    ":now_in_millis": {"N": str(int(now.timestamp() * 1000))},
+                    ":inprogress": {"S": STATUS_CONSTANTS["INPROGRESS"]},
                 },
             )
-        except self.table.meta.client.exceptions.ConditionalCheckFailedException:
-            logger.debug(f"Failed to put record for already existing idempotency key: {data_record.idempotency_key}")
-            raise IdempotencyItemAlreadyExistsError
+        except ClientError as exc:
+            error_code = exc.response.get("Error", {}).get("Code")
+            if error_code == "ConditionalCheckFailedException":
+                logger.debug(
+                    f"Failed to put record for already existing idempotency key: {data_record.idempotency_key}"
+                )
+                raise IdempotencyItemAlreadyExistsError from exc
+            else:
+                raise
 
     def _update_record(self, data_record: DataRecord):
         logger.debug(f"Updating record for idempotency key: {data_record.idempotency_key}")
         update_expression = "SET #response_data = :response_data, #expiry = :expiry, " "#status = :status"
         expression_attr_values = {
-            ":expiry": data_record.expiry_timestamp,
-            ":response_data": data_record.response_data,
-            ":status": data_record.status,
+            ":expiry": {"N": str(data_record.expiry_timestamp)},
+            ":response_data": {"S": data_record.response_data},
+            ":status": {"S": data_record.status},
         }
         expression_attr_names = {
             "#expiry": self.expiry_attr,
@@ -235,7 +231,7 @@ def _update_record(self, data_record: DataRecord):
 
         if self.payload_validation_enabled:
             update_expression += ", #validation_key = :validation_key"
-            expression_attr_values[":validation_key"] = data_record.payload_hash
+            expression_attr_values[":validation_key"] = {"S": data_record.payload_hash}
             expression_attr_names["#validation_key"] = self.validation_key_attr
 
         kwargs = {
@@ -245,8 +241,8 @@ def _update_record(self, data_record: DataRecord):
             "ExpressionAttributeNames": expression_attr_names,
         }
 
-        self.table.update_item(**kwargs)
+        self._client.update_item(TableName=self.table_name, **kwargs)
 
     def _delete_record(self, data_record: DataRecord) -> None:
         logger.debug(f"Deleting record for idempotency key: {data_record.idempotency_key}")
-        self.table.delete_item(Key=self._get_key(data_record.idempotency_key))
+        self._client.delete_item(TableName=self.table_name, Key={**self._get_key(data_record.idempotency_key)})

docs/core/metrics.md

@@ -216,6 +216,20 @@ CloudWatch EMF uses the same dimensions across all your metrics. Use `single_met
     --8<-- "examples/metrics/src/single_metric_output.json"
     ```
 
+By default it will skip all previously defined dimensions including default dimensions. Use `default_dimensions` keyword argument if you want to reuse default dimensions or specify custom dimensions from a dictionary.
+
+=== "single_metric_default_dimensions_inherit.json"
+
+    ```json hl_lines="10 15"
+    --8<-- "examples/metrics/src/single_metric_default_dimensions_inherit.py"
+    ```
+
+=== "single_metric_default_dimensions.py"
+
+    ```python hl_lines="12"
+    --8<-- "examples/metrics/src/single_metric_default_dimensions.py"
+    ```
+
 ### Flushing metrics manually
 
 If you prefer not to use `log_metrics` because you might want to encapsulate additional logic when doing so, you can manually flush and clear metrics as follows: