Merge branch 'develop' of github.com:awslabs/aws-lambda-powertools-python into develop

Author: Alexander Melnyk

.github/workflows/python_build.yml

@@ -37,7 +37,7 @@ jobs:
       - name: Complexity baseline
         run: make complexity-baseline
       - name: Upload coverage to Codecov
-        uses: codecov/[email protected].1
+        uses: codecov/[email protected].2
         with:
           file: ./coverage.xml
           # flags: unittests

.github/workflows/python_docs.yml

@@ -4,6 +4,10 @@ on:
   push:
     branches:
       - develop
+    paths:
+      - 'docs/**'
+      - 'CHANGELOG.md'
+      - 'mkdocs.yml'
 
 jobs:
   docs:

CHANGELOG.md

@@ -7,6 +7,51 @@ This project follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) fo
 
 ## [Unreleased]
 
+## [1.19.0] - 2021-08-11
+
+### Bug Fixes
+
+* **deps:** bump poetry to latest ([#592](https://github.com/awslabs/aws-lambda-powertools-python/issues/592))
+* **feature-flags:** bug handling multiple conditions ([#599](https://github.com/awslabs/aws-lambda-powertools-python/issues/599))
+* **parser:** API Gateway WebSocket validation under check_message_id; plus some housekeeping ([#553](https://github.com/awslabs/aws-lambda-powertools-python/issues/553))
+* **feature-toggles:** correct cdk example ([#601](https://github.com/awslabs/aws-lambda-powertools-python/issues/601))
+
+### Code Refactoring
+
+* **feature-flags:** add debug statements for all feature evaluations ([#590](https://github.com/awslabs/aws-lambda-powertools-python/issues/590))
+* **feature-flags:** optimize UX and maintenance ([#563](https://github.com/awslabs/aws-lambda-powertools-python/issues/563))
+
+### Documentation
+
+* **event-handler:** new custom serializer option
+* **feature-flags:** create concrete documentation ([#594](https://github.com/awslabs/aws-lambda-powertools-python/issues/594))
+* **feature-flags:** correct docs and typing ([#588](https://github.com/awslabs/aws-lambda-powertools-python/issues/588))
+* **parameters:** auto-transforming values based on suffix ([#573](https://github.com/awslabs/aws-lambda-powertools-python/issues/573))
+* **readme:** add code coverage badge ([#577](https://github.com/awslabs/aws-lambda-powertools-python/issues/577))
+* **tracer:** update wording that it auto-disables on non-Lambda env
+* **feature-flags:** fix SAM infra, convert CDK to Python
+* **feature-flags:** fix sample feature name in evaluate method
+* **feature-flags:** add guidance when to use vs env vars vs parameters
+### Features
+
+* **api-gateway:** add support for custom serializer ([#568](https://github.com/awslabs/aws-lambda-powertools-python/issues/568))
+* **data-classes:** decode json_body if based64 encoded ([#560](https://github.com/awslabs/aws-lambda-powertools-python/issues/560))
+* **feature-flags:** Add not_in action and rename contains to in ([#589](https://github.com/awslabs/aws-lambda-powertools-python/issues/589))
+* **params:** expose params `max_age`, `raise_on_transform_error` to high level functions ([#567](https://github.com/awslabs/aws-lambda-powertools-python/issues/567))
+* **tracer:** auto-disable tracer for non-Lambda environments to ease testing ([#598](https://github.com/awslabs/aws-lambda-powertools-python/issues/598))
+
+### Maintenance
+
+* **deps:** bump boto3 from 1.18.1 to 1.18.15 ([#591](https://github.com/awslabs/aws-lambda-powertools-python/issues/591))
+* **deps:** bump codecov/codecov-action from 2.0.1 to 2.0.2 ([#558](https://github.com/awslabs/aws-lambda-powertools-python/issues/558))
+* **deps:** bump boto3 from 1.18.15 to 1.18.17 ([#597](https://github.com/awslabs/aws-lambda-powertools-python/issues/597))
+* **deps-dev:** bump mkdocs-material from 7.2.2 to 7.2.3 ([#596](https://github.com/awslabs/aws-lambda-powertools-python/issues/596))
+* **deps-dev:** bump mkdocs-material from 7.2.1 to 7.2.2 ([#582](https://github.com/awslabs/aws-lambda-powertools-python/issues/582))
+* **deps-dev:** bump pdoc3 from 0.9.2 to 0.10.0 ([#584](https://github.com/awslabs/aws-lambda-powertools-python/issues/584))
+* **deps-dev:** bump isort from 5.9.2 to 5.9.3 ([#574](https://github.com/awslabs/aws-lambda-powertools-python/issues/574))
+* **deps-dev:** bump mkdocs-material from 7.2.0 to 7.2.1 ([#566](https://github.com/awslabs/aws-lambda-powertools-python/issues/566))
+* **deps-dev:** bump mkdocs-material from 7.1.11 to 7.2.0 ([#551](https://github.com/awslabs/aws-lambda-powertools-python/issues/551))
+* **deps-dev:** bump flake8-black from 0.2.1 to 0.2.3 ([#541](https://github.com/awslabs/aws-lambda-powertools-python/issues/541))
 ## [1.18.1] - 2021-07-23
 
 ### Bug Fixes

Makefile

@@ -5,7 +5,7 @@ target:
 	@$(MAKE) pr
 
 dev:
-	pip install --upgrade pip pre-commit poetry==1.1.4
+	pip install --upgrade pip pre-commit poetry
 	poetry install --extras "pydantic"
 	pre-commit install
 

README.md

@@ -25,6 +25,7 @@ A suite of Python utilities for AWS Lambda functions to ease adopting best pract
 * **[Event source data classes](https://awslabs.github.io/aws-lambda-powertools-python/latest/utilities/data_classes/)** - Data classes describing the schema of common Lambda event triggers
 * **[Parser](https://awslabs.github.io/aws-lambda-powertools-python/latest/utilities/parser/)** - Data parsing and deep validation using Pydantic
 * **[Idempotency](https://awslabs.github.io/aws-lambda-powertools-python/latest/utilities/idempotency/)** - Convert your Lambda functions into idempotent operations which are safe to retry
+* **[Feature Flags](./utilities/feature_flags.md)** - A simple rule engine to evaluate when one or multiple features should be enabled depending on the input
 
 ### Installation
 

aws_lambda_powertools/shared/constants.py

@@ -12,10 +12,9 @@
 
 EVENT_HANDLER_DEBUG_ENV: str = "POWERTOOLS_EVENT_HANDLER_DEBUG"
 
-SAM_LOCAL_ENV: str = "AWS_SAM_LOCAL"
-CHALICE_LOCAL_ENV: str = "AWS_CHALICE_CLI_MODE"
 SERVICE_NAME_ENV: str = "POWERTOOLS_SERVICE_NAME"
 XRAY_TRACE_ID_ENV: str = "_X_AMZN_TRACE_ID"
+LAMBDA_TASK_ROOT_ENV: str = "LAMBDA_TASK_ROOT"
 
 XRAY_SDK_MODULE: str = "aws_xray_sdk"
 XRAY_SDK_CORE_MODULE: str = "aws_xray_sdk.core"

aws_lambda_powertools/tracing/tracer.py

@@ -719,16 +719,15 @@ def _is_tracer_disabled() -> Union[bool, str]:
         Union[bool, str]
         """
         logger.debug("Verifying whether Tracing has been disabled")
-        is_lambda_sam_cli = os.getenv(constants.SAM_LOCAL_ENV)
-        is_chalice_cli = os.getenv(constants.CHALICE_LOCAL_ENV)
+        is_lambda_env = os.getenv(constants.LAMBDA_TASK_ROOT_ENV)
         is_disabled = resolve_truthy_env_var_choice(env=os.getenv(constants.TRACER_DISABLED_ENV, "false"))
 
         if is_disabled:
             logger.debug("Tracing has been disabled via env var POWERTOOLS_TRACE_DISABLED")
             return is_disabled
 
-        if is_lambda_sam_cli or is_chalice_cli:
-            logger.debug("Running under SAM CLI env or not in Lambda env; disabling Tracing")
+        if not is_lambda_env:
+            logger.debug("Running outside Lambda env; disabling Tracing")
             return True
 
         return False

aws_lambda_powertools/utilities/feature_flags/appconfig.py

@@ -21,7 +21,7 @@ def __init__(
         environment: str,
         application: str,
         name: str,
-        cache_seconds: int,
+        max_age: int = 5,
         sdk_config: Optional[Config] = None,
         envelope: Optional[str] = "",
         jmespath_options: Optional[Dict] = None,
@@ -36,8 +36,8 @@ def __init__(
             AppConfig application name, e.g. 'powertools'
         name: str
             AppConfig configuration name e.g. `my_conf`
-        cache_seconds: int
-            cache expiration time, how often to call AppConfig to fetch latest configuration
+        max_age: int
+            cache expiration time in seconds, or how often to call AppConfig to fetch latest configuration
         sdk_config: Optional[Config]
             Botocore Config object to pass during client initialization
         envelope : Optional[str]
@@ -49,7 +49,7 @@ def __init__(
         self.environment = environment
         self.application = application
         self.name = name
-        self.cache_seconds = cache_seconds
+        self.cache_seconds = max_age
         self.config = sdk_config
         self.envelope = envelope
         self.jmespath_options = jmespath_options

aws_lambda_powertools/utilities/feature_flags/feature_flags.py

@@ -24,7 +24,7 @@ def __init__(self, store: StoreProvider):
             environment="test",
             application="powertools",
             name="test_conf_name",
-            cache_seconds=300,
+            max_age=300,
             envelope="features"
         )
 
@@ -46,7 +46,8 @@ def _match_by_action(action: str, condition_value: Any, context_value: Any) -> b
             schema.RuleAction.EQUALS.value: lambda a, b: a == b,
             schema.RuleAction.STARTSWITH.value: lambda a, b: a.startswith(b),
             schema.RuleAction.ENDSWITH.value: lambda a, b: a.endswith(b),
-            schema.RuleAction.CONTAINS.value: lambda a, b: a in b,
+            schema.RuleAction.IN.value: lambda a, b: a in b,
+            schema.RuleAction.NOT_IN.value: lambda a, b: a not in b,
         }
 
         try:
@@ -63,6 +64,13 @@ def _evaluate_conditions(
         rule_match_value = rule.get(schema.RULE_MATCH_VALUE)
         conditions = cast(List[Dict], rule.get(schema.CONDITIONS_KEY))
 
+        if not conditions:
+            logger.debug(
+                f"rule did not match, no conditions to match, rule_name={rule_name}, rule_value={rule_match_value}, "
+                f"name={feature_name} "
+            )
+            return False
+
         for condition in conditions:
             context_value = context.get(str(condition.get(schema.CONDITION_KEY)))
             cond_action = condition.get(schema.CONDITION_ACTION, "")
@@ -75,9 +83,8 @@ def _evaluate_conditions(
                 )
                 return False  # context doesn't match condition
 
-            logger.debug(f"rule matched, rule_name={rule_name}, rule_value={rule_match_value}, name={feature_name}")
-            return True
-        return False
+        logger.debug(f"rule matched, rule_name={rule_name}, rule_value={rule_match_value}, name={feature_name}")
+        return True
 
     def _evaluate_rules(
         self, *, feature_name: str, context: Dict[str, Any], feat_default: bool, rules: Dict[str, Any]
@@ -237,6 +244,7 @@ def get_enabled_features(self, *, context: Optional[Dict[str, Any]] = None) -> L
             logger.debug(f"Failed to fetch feature flags from store, returning empty list, reason={err}")
             return features_enabled
 
+        logger.debug("Evaluating all features")
         for name, feature in features.items():
             rules = feature.get(schema.RULES_KEY, {})
             feature_default_value = feature.get(schema.FEATURE_DEFAULT_VAL_KEY)

aws_lambda_powertools/utilities/feature_flags/schema.py

@@ -20,7 +20,8 @@ class RuleAction(str, Enum):
     EQUALS = "EQUALS"
     STARTSWITH = "STARTSWITH"
     ENDSWITH = "ENDSWITH"
-    CONTAINS = "CONTAINS"
+    IN = "IN"
+    NOT_IN = "NOT_IN"
 
 
 class SchemaValidator(BaseValidator):
@@ -79,7 +80,7 @@ class SchemaValidator(BaseValidator):
     The value MUST contain the following members:
 
     * **action**: `str`. Operation to perform to match a key and value.
-    The value MUST be either EQUALS, STARTSWITH, ENDSWITH, CONTAINS
+    The value MUST be either EQUALS, STARTSWITH, ENDSWITH, IN, NOT_IN
     * **key**: `str`. Key in given context to perform operation
     * **value**: `Any`. Value in given context that should match action operation.
 

docs/core/event_handler/api_gateway.md

@@ -730,6 +730,49 @@ This will enable full tracebacks errors in the response, print request and respo
 		return app.resolve(event, context)
 	```
 
+### Custom serializer
+
+You can instruct API Gateway handler to use a custom serializer to best suit your needs, for example take into account Enums when serializing.
+
+=== "custom_serializer.py"
+	```python hl_lines="19-20 24"
+	import json
+	from enum import Enum
+	from json import JSONEncoder
+	from typing import Dict
+
+	class CustomEncoder(JSONEncoder):
+		"""Your customer json encoder"""
+		def default(self, obj):
+			if isinstance(obj, Enum):
+				return obj.value
+			try:
+				iterable = iter(obj)
+			except TypeError:
+				pass
+			else:
+				return sorted(iterable)
+			return JSONEncoder.default(self, obj)
+
+	def custom_serializer(obj) -> str:
+		"""Your custom serializer function ApiGatewayResolver will use"""
+		return json.dumps(obj, cls=CustomEncoder)
+
+	# Assigning your custom serializer
+	app = ApiGatewayResolver(serializer=custom_serializer)
+
+	class Color(Enum):
+		RED = 1
+		BLUE = 2
+
+	@app.get("/colors")
+	def get_color() -> Dict:
+		return {
+			# Color.RED will be serialized to 1 as expected now
+			"color": Color.RED,
+			"variations": {"light", "dark"},
+		}
+	```
 
 ## Testing your code
 

docs/core/tracer.md

@@ -10,8 +10,7 @@ Tracer is an opinionated thin wrapper for [AWS X-Ray Python SDK](https://github.
 ## Key features
 
 * Auto capture cold start as annotation, and responses or full exceptions as metadata
-* Run functions locally with SAM CLI without code change to disable tracing
-* Explicitly disable tracing via env var `POWERTOOLS_TRACE_DISABLED="true"`
+* Auto-disable when not running in AWS Lambda environment
 * Support tracing async methods, generators, and context managers
 * Auto patch supported modules by AWS X-Ray
 
@@ -357,11 +356,7 @@ Tracer keeps a copy of its configuration after the first initialization. This is
 
 ## Testing your code
 
-You can safely disable Tracer when unit testing your code using `POWERTOOLS_TRACE_DISABLED` environment variable.
-
-```bash
-POWERTOOLS_TRACE_DISABLED=1 python -m pytest
-```
+Tracer is disabled by default when not running in the AWS Lambda environment - This means no code changes or environment variables to be set.
 
 ## Tips
 

docs/index.md

@@ -212,6 +212,7 @@ aws serverlessrepo list-application-versions \
 [Event source data classes](./utilities/data_classes.md) | Data classes describing the schema of common Lambda event triggers
 [Parser](./utilities/parser.md) | Data parsing and deep validation using Pydantic
 [Idempotency](./utilities/idempotency.md) | Idempotent Lambda handler
+[Feature Flags](./utilities/feature_flags.md) | A simple rule engine to evaluate when one or multiple features should be enabled depending on the input
 
 ## Environment variables
 
@@ -222,7 +223,7 @@ aws serverlessrepo list-application-versions \
 | ------------------------------------------------- | --------------------------------------------------------------------------------- | --------------------------------------------------------------------------------- | ------------------------------------------------- |
 | **POWERTOOLS_SERVICE_NAME** | Sets service name used for tracing namespace, metrics dimension and structured logging | All | `"service_undefined"` |
 | **POWERTOOLS_METRICS_NAMESPACE** | Sets namespace used for metrics | [Metrics](./core/metrics) | `None` |
-| **POWERTOOLS_TRACE_DISABLED** | Disables tracing | [Tracing](./core/tracer) | `false` |
+| **POWERTOOLS_TRACE_DISABLED** | Explicitly disables tracing | [Tracing](./core/tracer) | `false` |
 | **POWERTOOLS_TRACER_CAPTURE_RESPONSE** | Captures Lambda or method return as metadata. | [Tracing](./core/tracer) | `true` |
 | **POWERTOOLS_TRACER_CAPTURE_ERROR** | Captures Lambda or method exception as metadata. | [Tracing](./core/tracer) | `true` |
 | **POWERTOOLS_TRACE_MIDDLEWARES** | Creates sub-segment for each custom middleware | [Middleware factory](./utilities/middleware_factory) | `false` |