feat(parameters): transform = "auto"

[michaelbrewer]

Issue #, if available: #136

When fetching multiple values from the parameter store or dynamodb often each record might have different encoding and not all have uniform encoding. eg:

key	value
A	json string
B	base64 decoded image
C	might be a plain string

Description of changes:

Add a new tranform type called "auto", which looks at the
key name to autodetect the encoding of the values in dynamodb or
the parameter store.

• Keys ending with ".json" are assumed to be "json" string
• Keys ending with ".binary" are assumed to be "binary" base64 decode
  strings
• Other Keys are assumed to be plain text values.

key	value
A.json	["one", "two"]
B.binary	b'base64 decoded image'
C	plain text string

Checklist

• Meet tenets criteria
• Update tests
• Update docs
• PR title follows conventional commit semantics

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[codecov-commenter]

Codecov Report

Merging #133 into develop will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##           develop      #133   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           28        28           
  Lines          793       806   +13     
  Branches        72        76    +4     
=========================================
+ Hits           793       806   +13     

Impacted Files	Coverage Δ
aws_lambda_powertools/utilities/parameters/base.py	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b28c47e...01e991f. Read the comment docs.

[michaelbrewer]

@nmoutschen - if it is a matter of flexibility on the naming pattern of the keys, we could create patterns for them.

Another optional it giving the new user an option to set a customer converter that accepts "key" and "value". So you could provide an auto-formatter, a yaml one, xml etc.

[nmoutschen]

Hey @michaelbrewer !

Thanks a lot for your contribution! 😄 I have one open question regarding the user experience for this feature: should we rename the key by dropping the extension?

It would also be good to parameterise the transforms just to have a single place where to add/change transforms to prevent errors in the future if we want to add new transforms. However, this aspect is good as-is for my point of view.

[michaelbrewer]

@nmoutschen would this patch make this more flexibile?

diff --git a/aws_lambda_powertools/utilities/parameters/base.py b/aws_lambda_powertools/utilities/parameters/base.py
index e1ba9ac..1269432 100644
--- a/aws_lambda_powertools/utilities/parameters/base.py
+++ b/aws_lambda_powertools/utilities/parameters/base.py
@@ -15,6 +15,9 @@ DEFAULT_MAX_AGE_SECS = 5
 ExpirableValue = namedtuple("ExpirableValue", ["value", "ttl"])
 # These providers will be dynamically initialized on first use of the helper functions
 DEFAULT_PROVIDERS = {}
+TRANSFORM_METHOD_JSON = "json"
+TRANSFORM_METHOD_BINARY = "binary"
+SUPPORTED_TRANSFORM_METHODS = [TRANSFORM_METHOD_JSON, TRANSFORM_METHOD_BINARY]
 
 
 class BaseProvider(ABC):
@@ -194,12 +197,10 @@ def get_transform_method(key: str, transform: str) -> Optional[str]:
     if transform != "auto":
         return transform
 
-    if key.endswith(".json"):
-        return "json"
-    elif key.endswith(".binary"):
-        return "binary"
-    else:
-        return None
+    for transform_method in SUPPORTED_TRANSFORM_METHODS:
+        if key.endswith("." + transform_method):
+            return transform_method
+    return None
 
 
 def transform_value(value: str, transform: str, raise_on_transform_error: bool = True) -> Union[dict, bytes, None]:
@@ -223,9 +224,9 @@ def transform_value(value: str, transform: str, raise_on_transform_error: bool =
     """
 
     try:
-        if transform == "json":
+        if transform == TRANSFORM_METHOD_JSON:
             return json.loads(value)
-        elif transform == "binary":
+        elif transform == TRANSFORM_METHOD_BINARY:
             return base64.b64decode(value)
         else:
             raise ValueError(f"Invalid transform type '{transform}'")

[nmoutschen]

LGTM!