fix: metrics not being flushed on every invocation (#45)

* [Sync Master] 0.7.0 release (#22)

* docs: add pypi badge

* fix: add missing single_metric example; test var name

* chore: pypi monthly download badge

* chore: fix github badge typo

* feat: add docs to CI

* fix: CI attempt 2

* fix: CI attempt 3

* fix: CI attempt 3

* fix: CI attempt 4

* chore: clean up CI workflows

* Decorator factory Feat: Create your own middleware (#17)

* feat(utils): add decorator factory

* improv: use partial to reduce complexity

* improv: add error handling

* chore: type hint

* docs: include pypi downloads badge

* feat: opt in to trace each middleware that runs

* improv: add initial util tests

* improv: test explicit and implicit trace_execution

* improv: test decorator with params

* chore: linting

* docs: include utilities

* improv: correct tests, dec_factory only for func

* improv: make util name more explicit

* improv: doc trace_execution, fix casting

* docs: add limitations, improve syntax

* docs: use new docs syntax

* fix: remove middleware decorator from libs

* feat: build docs in CI

* chore: linting

* fix: CI python-version type

* chore: remove docs CI

* chore: kick CI

* chore: include build badge master branch

* chore: refactor naming

* fix: rearrange tracing tests

* improv(tracer): toggle default auto patching

* feat(tracer): retrieve registered class instance

* fix(Makefile):  make cov target more explicit

* improv(Register): support multiple classes reg.

* improv(Register): inject class methods correctly

* docs: add how to reutilize Tracer

* improv(tracer): test auto patch method

* improv: address nicolas feedback

* improv: update example to reflect middleware feat

* fix: metric dimension in root blob

* chore: version bump

Co-authored-by: heitorlessa <[email protected]>

Co-authored-by: heitorlessa <[email protected]>

* feat: add algolia search for docs and api ref (#39) (#40)

Signed-off-by: heitorlessa <[email protected]>

* fix: revert makefile build-docs-api

Signed-off-by: heitorlessa <[email protected]>

* fix: metric_set reuse #43

Signed-off-by: heitorlessa <[email protected]>

* fix: clear metrics

Signed-off-by: heitorlessa <[email protected]>

* fix: update serialize_metrics helper function to use MetricManager instead of Metrics

* fix: clear metrics after lambda invocation #43

Signed-off-by: heitorlessa <[email protected]>

* improv: document metrics tests, remove redundants

Signed-off-by: heitorlessa <[email protected]>

#43

* chore: linting

Signed-off-by: heitorlessa <[email protected]>

* docs: add section to flush metrics manually

Signed-off-by: heitorlessa <[email protected]>

* docs: include EMF Json object

Signed-off-by: heitorlessa <[email protected]>

* chore: bump version 0.9.4

Signed-off-by: heitorlessa <[email protected]>

Co-authored-by: Tom McCarthy <[email protected]>

Author: heitorlessa

Diff for: CHANGELOG.md

@@ -1,5 +1,11 @@
 # HISTORY 
 
+## May 29th
+
+**0.9.4**
+
+* **Metrics**: Bugfix - Metrics were not correctly flushed, and cleared on every invocation
+
 ## May 16th
 
 **0.9.3**

Diff for: Makefile

@@ -33,9 +33,8 @@ build-docs:
 	@$(MAKE) build-docs-website
 	@$(MAKE) build-docs-api
 
-build-docs-api:
-	pip install pdoc3~=0.7.5
-	pdoc3 --html --output-dir dist/api/ ./aws_lambda_powertools --force
+build-docs-api: dev
+	poetry run pdoc --html --output-dir dist/api/ ./aws_lambda_powertools --force
 	mv dist/api/aws_lambda_powertools/* dist/api/
 	rm -rf dist/api/aws_lambda_powertools
 

Diff for: aws_lambda_powertools/metrics/base.py

@@ -50,8 +50,8 @@ class MetricManager:
     """
 
     def __init__(self, metric_set: Dict[str, str] = None, dimension_set: Dict = None, namespace: str = None):
-        self.metric_set = metric_set or {}
-        self.dimension_set = dimension_set or {}
+        self.metric_set = metric_set if metric_set is not None else {}
+        self.dimension_set = dimension_set if dimension_set is not None else {}
         self.namespace = os.getenv("POWERTOOLS_METRICS_NAMESPACE") or namespace
         self._metric_units = [unit.value for unit in MetricUnit]
         self._metric_unit_options = list(MetricUnit.__members__)
@@ -116,7 +116,10 @@ def add_metric(self, name: str, unit: MetricUnit, value: Union[float, int]):
             logger.debug(f"Exceeded maximum of {MAX_METRICS} metrics - Publishing existing metric set")
             metrics = self.serialize_metric_set()
             print(json.dumps(metrics))
-            self.metric_set = {}
+
+            # clear metric set only as opposed to metrics and dimensions set
+            # since we could have more than 100 metrics
+            self.metric_set.clear()
 
     def serialize_metric_set(self, metrics: Dict = None, dimensions: Dict = None) -> Dict:
         """Serializes metric and dimensions set

Diff for: aws_lambda_powertools/metrics/metrics.py

@@ -65,8 +65,15 @@ def do_something():
     _metrics = {}
     _dimensions = {}
 
-    def __init__(self, metric_set=None, dimension_set=None, namespace=None):
-        super().__init__(metric_set=self._metrics, dimension_set=self._dimensions, namespace=namespace)
+    def __init__(self):
+        self.metric_set = self._metrics
+        self.dimension_set = self._dimensions
+        super().__init__(metric_set=self.metric_set, dimension_set=self.dimension_set)
+
+    def clear_metrics(self):
+        logger.debug("Clearing out existing metric set from memory")
+        self.metric_set.clear()
+        self.dimension_set.clear()
 
     def log_metrics(self, lambda_handler: Callable[[Any, Any], Any] = None):
         """Decorator to serialize and publish metrics at the end of a function execution.
@@ -101,6 +108,7 @@ def decorate(*args, **kwargs):
                 response = lambda_handler(*args, **kwargs)
             finally:
                 metrics = self.serialize_metric_set()
+                self.clear_metrics()
                 logger.debug("Publishing metrics", {"metrics": metrics})
                 print(json.dumps(metrics))
 

Diff for: docs/content/core/metrics.mdx

@@ -122,6 +122,25 @@ def lambda_handler(evt, ctx):
     ...
 ```
 
+## Flushing metrics manually
+
+If you prefer not to use `log_metrics` because you might want to encapsulate additional logic when doing so, you can manually flush and clear metrics as follows:
+
+```python:title=manual_metric_serialization.py
+import json
+from aws_lambda_powertools.metrics import Metrics, MetricUnit
+
+metrics = Metrics()
+metrics.add_metric(name="ColdStart", unit="Count", value=1)
+metrics.add_dimension(name="service", value="booking")
+
+# highlight-start
+your_metrics_object = metrics.serialize_metric_set() 
+metrics.clear_metrics()
+print(json.dumps(your_metrics_object))
+# highlight-end
+```
+
 ## Testing your code
 
 Use `POWERTOOLS_METRICS_NAMESPACE` env var when unit testing your code to ensure a metric namespace object is created, and your code doesn't fail validation.

Diff for: pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "aws_lambda_powertools"
-version = "0.9.3"
+version = "0.9.4"
 description = "Python utilities for AWS Lambda functions including but not limited to tracing, logging and custom metric"
 authors = ["Amazon Web Services"]
 classifiers=[