Updated Idempotency examples and highlight importance of JMESPath for JSON APIs

Author: walmsles

docs/utilities/idempotency.md

@@ -206,6 +206,11 @@ In this example, we have a Lambda handler that creates a payment for a user subs
 
 Imagine the function executes successfully, but the client never receives the response due to a connection issue. It is safe to retry in this instance, as the idempotent decorator will return a previously saved response.
 
+!!! warning "Making JSON APIs Idempotent"
+    To ensure JSON structured APIs are fully idempotent we must ensure the idempotent key is a valid JSON object.
+
+    To do this we can [customize the default behaviour](#customizing-the-default-behavior) with *event_key_jmespath* and the [JMESPath built-in function](/utilities/jmespath_functions) *powertools_json()*.
+
 === "payment.py"
 
     ```python hl_lines="2-4 10 12 15 20"
@@ -218,7 +223,7 @@ Imagine the function executes successfully, but the client never receives the re
 
     # Treat everything under the "body" key
     # in the event json object as our payload
-    config = IdempotencyConfig(event_key_jmespath="body")
+    config = IdempotencyConfig(event_key_jmespath="powertools_json(body)")
 
     @idempotent(config=config, persistence_store=persistence_layer)
     def handler(event, context):
@@ -270,6 +275,7 @@ Imagine the function executes successfully, but the client never receives the re
     }
     ```
 
+
 ### Idempotency request flow
 
 This sequence diagram shows an example flow of what happens in the payment scenario:

docs/utilities/validation.md

@@ -429,129 +429,7 @@ For each format defined in a dictionary key, you must use a regex, or a function
 
 You might have events or responses that contain non-encoded JSON, where you need to decode before validating them.
 
-You can use our built-in JMESPath functions within your expressions to do exactly that to decode JSON Strings, base64, and uncompress gzip data.
+You can use our built-in [JMESPath functions](/utilities/jmespath_functions) within your expressions to do exactly that to decode JSON Strings, base64, and uncompress gzip data.
 
 !!! info
     We use these for built-in envelopes to easily to decode and unwrap events from sources like Kinesis, CloudWatch Logs, etc.
-
-#### powertools_json function
-
-Use `powertools_json` function to decode any JSON String.
-
-This sample will decode the value within the `data` key into a valid JSON before we can validate it.
-
-=== "powertools_json_jmespath_function.py"
-
-    ```python hl_lines="9"
-    from aws_lambda_powertools.utilities.validation import validate
-
-    import schemas
-
-    sample_event = {
-        'data': '{"payload": {"message": "hello hello", "username": "blah blah"}}'
-    }
-
-    validate(event=sample_event, schema=schemas.INPUT, envelope="powertools_json(data)")
-    ```
-
-=== "schemas.py"
-
-    ```python hl_lines="7 14 16 23 39 45 47 52"
-    --8<-- "docs/shared/validation_basic_jsonschema.py"
-    ```
-
-#### powertools_base64 function
-
-Use `powertools_base64` function to decode any base64 data.
-
-This sample will decode the base64 value within the `data` key, and decode the JSON string into a valid JSON before we can validate it.
-
-=== "powertools_json_jmespath_function.py"
-
-    ```python hl_lines="12"
-    from aws_lambda_powertools.utilities.validation import validate
-
-    import schemas
-
-    sample_event = {
-        "data": "eyJtZXNzYWdlIjogImhlbGxvIGhlbGxvIiwgInVzZXJuYW1lIjogImJsYWggYmxhaCJ9="
-    }
-
-    validate(
-          event=sample_event,
-          schema=schemas.INPUT,
-          envelope="powertools_json(powertools_base64(data))"
-    )
-    ```
-
-=== "schemas.py"
-
-    ```python hl_lines="7 14 16 23 39 45 47 52"
-    --8<-- "docs/shared/validation_basic_jsonschema.py"
-    ```
-
-#### powertools_base64_gzip function
-
-Use `powertools_base64_gzip` function to decompress and decode base64 data.
-
-This sample will decompress and decode base64 data, then use JMESPath pipeline expression to pass the result for decoding its JSON string.
-
-=== "powertools_json_jmespath_function.py"
-
-    ```python hl_lines="12"
-    from aws_lambda_powertools.utilities.validation import validate
-
-    import schemas
-
-    sample_event = {
-        "data": "H4sIACZAXl8C/52PzUrEMBhFX2UILpX8tPbHXWHqIOiq3Q1F0ubrWEiakqTWofTdTYYB0YWL2d5zvnuTFellBIOedoiyKH5M0iwnlKH7HZL6dDB6ngLDfLFYctUKjie9gHFaS/sAX1xNEq525QxwFXRGGMEkx4Th491rUZdV3YiIZ6Ljfd+lfSyAtZloacQgAkqSJCGhxM6t7cwwuUGPz4N0YKyvO6I9WDeMPMSo8Z4Ca/kJ6vMEYW5f1MX7W1lVxaG8vqX8hNFdjlc0iCBBSF4ERT/3Pl7RbMGMXF2KZMh/C+gDpNS7RRsp0OaRGzx0/t8e0jgmcczyLCWEePhni/23JWalzjdu0a3ZvgEaNLXeugEAAA=="
-    }
-
-    validate(
-        event=sample_event,
-        schema=schemas.INPUT,
-        envelope="powertools_base64_gzip(data) | powertools_json(@)"
-    )
-    ```
-
-=== "schemas.py"
-
-    ```python hl_lines="7 14 16 23 39 45 47 52"
-    --8<-- "docs/shared/validation_basic_jsonschema.py"
-    ```
-
-### Bring your own JMESPath function
-
-!!! warning
-    This should only be used for advanced use cases where you have special formats not covered by the built-in functions.
-
-    This will **replace all provided built-in functions such as `powertools_json`, so you will no longer be able to use them**.
-
-For special binary formats that you want to decode before applying JSON Schema validation, you can bring your own [JMESPath function](https://github.com/jmespath/jmespath.py#custom-functions){target="_blank"} and any additional option via `jmespath_options` param.
-
-=== "custom_jmespath_function.py"
-
-    ```python hl_lines="2 6-10 14"
-    from aws_lambda_powertools.utilities.validation import validator
-    from jmespath import functions
-
-    import schemas
-
-    class CustomFunctions(functions.Functions):
-
-        @functions.signature({'types': ['string']})
-        def _func_special_decoder(self, s):
-            return my_custom_decoder_logic(s)
-
-    custom_jmespath_options = {"custom_functions": CustomFunctions()}
-
-    @validator(schema=schemas.INPUT, jmespath_options=**custom_jmespath_options)
-    def handler(event, context):
-        return event
-    ```
-
-=== "schemas.py"
-
-    ```python hl_lines="7 14 16 23 39 45 47 52"
-    --8<-- "docs/shared/validation_basic_jsonschema.py"
-    ```

mkdocs.yml

@@ -26,6 +26,7 @@ nav:
       - utilities/parser.md
       - utilities/idempotency.md
       - utilities/feature_flags.md
+      - utilities/jmespath_functions.md
 
 theme:
   name: material