|
| 1 | +--- |
| 2 | +title: Validation |
| 3 | +description: Utility |
| 4 | +--- |
| 5 | + |
| 6 | + |
| 7 | +import Note from "../../src/components/Note" |
| 8 | + |
| 9 | +This utility provides JSON Schema validation for events and responses, including JMESPath support to unwrap events before validation. |
| 10 | + |
| 11 | +**Key features** |
| 12 | + |
| 13 | +* Validate incoming event and response |
| 14 | +* JMESPath support to unwrap events before validation applies |
| 15 | +* Built-in envelopes to unwrap popular event sources payloads |
| 16 | + |
| 17 | +## Validating events |
| 18 | + |
| 19 | +You can validate inbound and outbound events using `validator` decorator. |
| 20 | + |
| 21 | +You can also use the standalone `validate` function, if you want more control over the validation process such as handling a validation error. |
| 22 | + |
| 23 | +We support any JSONSchema draft supported by [fastjsonschema](https://horejsek.github.io/python-fastjsonschema/) library. |
| 24 | + |
| 25 | +<Note type="warning"> |
| 26 | + Both <code>validator</code> decorator and <code>validate</code> standalone function expects your JSON Schema to be |
| 27 | + a <strong>dictionary</strong>, not a filename. |
| 28 | +</Note> |
| 29 | + |
| 30 | + |
| 31 | +### Validator decorator |
| 32 | + |
| 33 | +**Validator** decorator is typically used to validate either inbound or functions' response. |
| 34 | + |
| 35 | +It will fail fast with `SchemaValidationError` exception if event or response doesn't conform with given JSON Schema. |
| 36 | + |
| 37 | +```python:title=validator_decorator.py |
| 38 | +from aws_lambda_powertools.utilities.validation import validator |
| 39 | + |
| 40 | +json_schema_dict = {..} |
| 41 | +response_json_schema_dict = {..} |
| 42 | + |
| 43 | +@validator(inbound_schema=json_schema_dict, outbound_schema=response_json_schema_dict) |
| 44 | +def handler(event, context): |
| 45 | + return event |
| 46 | +``` |
| 47 | + |
| 48 | +**NOTE**: It's not a requirement to validate both inbound and outbound schemas - You can either use one, or both. |
| 49 | + |
| 50 | +### Validate function |
| 51 | + |
| 52 | +**Validate** standalone function is typically used within the Lambda handler, or any other methods that perform data validation. |
| 53 | + |
| 54 | +You can also gracefully handle schema validation errors by catching `SchemaValidationError` exception. |
| 55 | + |
| 56 | +```python:title=validator_decorator.py |
| 57 | +from aws_lambda_powertools.utilities.validation import validate |
| 58 | +from aws_lambda_powertools.utilities.validation.exceptions import SchemaValidationError |
| 59 | + |
| 60 | +json_schema_dict = {..} |
| 61 | + |
| 62 | +def handler(event, context): |
| 63 | + try: |
| 64 | + validate(event=event, schema=json_schema_dict) |
| 65 | + except SchemaValidationError as e: |
| 66 | + # do something before re-raising |
| 67 | + raise |
| 68 | + |
| 69 | + return event |
| 70 | +``` |
| 71 | + |
| 72 | +## Unwrapping events prior to validation |
| 73 | + |
| 74 | +You might want to validate only a portion of your event - This is where the `envelope` parameter is for. |
| 75 | + |
| 76 | +Envelopes are [JMESPath expressions](https://jmespath.org/tutorial.html) to extract a portion of JSON you want before applying JSON Schema validation. |
| 77 | + |
| 78 | +Here is a sample custom EventBridge event, where we only validate what's inside the `detail` key: |
| 79 | + |
| 80 | +```json:title=sample_wrapped_event.json |
| 81 | +{ |
| 82 | + "id": "cdc73f9d-aea9-11e3-9d5a-835b769c0d9c", |
| 83 | + "detail-type": "Scheduled Event", |
| 84 | + "source": "aws.events", |
| 85 | + "account": "123456789012", |
| 86 | + "time": "1970-01-01T00:00:00Z", |
| 87 | + "region": "us-east-1", |
| 88 | + "resources": ["arn:aws:events:us-east-1:123456789012:rule/ExampleRule"], |
| 89 | + "detail": {"message": "hello hello", "username": "blah blah"}, // highlight-line |
| 90 | +} |
| 91 | +``` |
| 92 | + |
| 93 | +Here is how you'd use the `envelope` parameter to extract the payload inside the `detail` key before validating: |
| 94 | + |
| 95 | +```python:title=unwrapping_events.py |
| 96 | +from aws_lambda_powertools.utilities.validation import validator, validate |
| 97 | + |
| 98 | +json_schema_dict = {..} |
| 99 | + |
| 100 | +@validator(inbound_schema=json_schema_dict, envelope="detail") # highlight-line |
| 101 | +def handler(event, context): |
| 102 | + validate(event=event, schema=json_schema_dict, envelope="detail") # highlight-line |
| 103 | + return event |
| 104 | +``` |
| 105 | + |
| 106 | +This is quite powerful because you can use JMESPath Query language to extract records from [arrays, slice and dice](https://jmespath.org/tutorial.html#list-and-slice-projections), to [pipe expressions](https://jmespath.org/tutorial.html#pipe-expressions) and [function expressions](https://jmespath.org/tutorial.html#functions), where you'd extract what you need before validating the actual payload. |
| 107 | + |
| 108 | +## Built-in envelopes |
| 109 | + |
| 110 | +This utility comes with built-in envelopes to easily extract the payload from popular event sources. |
| 111 | + |
| 112 | +```python:title=unwrapping_popular_event_sources.py |
| 113 | +from aws_lambda_powertools.utilities.validation import envelopes, validate, validator |
| 114 | + |
| 115 | +json_schema_dict = {..} |
| 116 | + |
| 117 | +@validator(inbound_schema=json_schema_dict, envelope=envelopes.EVENTBRIDGE) # highlight-line |
| 118 | +def handler(event, context): |
| 119 | + validate(event=event, schema=json_schema_dict, envelope=envelopes.EVENTBRIDGE) # highlight-line |
| 120 | + return event |
| 121 | +``` |
| 122 | + |
| 123 | +Here is a handy table with built-in envelopes along with their JMESPath expressions in case you want to build your own. |
| 124 | + |
| 125 | +Envelope name | JMESPath expression |
| 126 | +------------------------------------------------- | --------------------------------------------------------------------------------- |
| 127 | +**API_GATEWAY_REST** | "powertools_json(body)" |
| 128 | +**API_GATEWAY_HTTP** | "powertools_json(body)" |
| 129 | +**SQS** | "Records[*].powertools_json(body)" |
| 130 | +**SNS** | "Records[0].Sns.Message | powertools_json(@)" |
| 131 | +**EVENTBRIDGE** | "detail" |
| 132 | +**CLOUDWATCH_EVENTS_SCHEDULED** | "detail" |
| 133 | +**KINESIS_DATA_STREAM** | "Records[*].kinesis.powertools_json(powertools_base64(data))" |
| 134 | +**CLOUDWATCH_LOGS** | "awslogs.powertools_base64_gzip(data) | powertools_json(@).logEvents[*]" |
| 135 | + |
| 136 | +## Built-in JMESPath functions |
| 137 | + |
| 138 | +You might have events or responses that contain non-encoded JSON, where you need to decode before validating them. |
| 139 | + |
| 140 | +You can use our built-in JMESPath functions within your expressions to do exactly that to decode JSON Strings, base64, and uncompress gzip data. |
| 141 | + |
| 142 | +<Note type="info"> |
| 143 | + We use these for built-in envelopes to easily to decode and unwrap events from sources like Kinesis, CloudWatch Logs, etc. |
| 144 | +</Note> |
| 145 | + |
| 146 | +### powertools_json function |
| 147 | + |
| 148 | +Use `powertools_json` function to decode any JSON String. |
| 149 | + |
| 150 | +This sample will decode the value within the `data` key into a valid JSON before we can validate it. |
| 151 | + |
| 152 | +```python:title=powertools_json_jmespath_function.py |
| 153 | +from aws_lambda_powertools.utilities.validation import validate |
| 154 | + |
| 155 | +json_schema_dict = {..} |
| 156 | +sample_event = { |
| 157 | + 'data': '{"payload": {"message": "hello hello", "username": "blah blah"}}' |
| 158 | +} |
| 159 | + |
| 160 | +def handler(event, context): |
| 161 | + validate(event=event, schema=json_schema_dict, envelope="powertools_json(data)") # highlight-line |
| 162 | + return event |
| 163 | + |
| 164 | +handler(event=sample_event, context={}) |
| 165 | +``` |
| 166 | + |
| 167 | +### powertools_base64 function |
| 168 | + |
| 169 | +Use `powertools_base64` function to decode any base64 data. |
| 170 | + |
| 171 | +This sample will decode the base64 value within the `data` key, and decode the JSON string into a valid JSON before we can validate it. |
| 172 | + |
| 173 | +```python:title=powertools_json_jmespath_function.py |
| 174 | +from aws_lambda_powertools.utilities.validation import validate |
| 175 | + |
| 176 | +json_schema_dict = {..} |
| 177 | +sample_event = { |
| 178 | + "data": "eyJtZXNzYWdlIjogImhlbGxvIGhlbGxvIiwgInVzZXJuYW1lIjogImJsYWggYmxhaCJ9=" |
| 179 | +} |
| 180 | + |
| 181 | +def handler(event, context): |
| 182 | + validate(event=event, schema=json_schema_dict, envelope="powertools_json(powertools_base64(data))") # highlight-line |
| 183 | + return event |
| 184 | + |
| 185 | +handler(event=sample_event, context={}) |
| 186 | +``` |
| 187 | + |
| 188 | +### powertools_base64_gzip function |
| 189 | + |
| 190 | +Use `powertools_base64_gzip` function to decompress and decode base64 data. |
| 191 | + |
| 192 | +This sample will decompress and decode base64 data, then use JMESPath pipeline expression to pass the result for decoding its JSON string. |
| 193 | + |
| 194 | +```python:title=powertools_json_jmespath_function.py |
| 195 | +from aws_lambda_powertools.utilities.validation import validate |
| 196 | + |
| 197 | +json_schema_dict = {..} |
| 198 | +sample_event = { |
| 199 | + "data": "H4sIACZAXl8C/52PzUrEMBhFX2UILpX8tPbHXWHqIOiq3Q1F0ubrWEiakqTWofTdTYYB0YWL2d5zvnuTFellBIOedoiyKH5M0iwnlKH7HZL6dDB6ngLDfLFYctUKjie9gHFaS/sAX1xNEq525QxwFXRGGMEkx4Th491rUZdV3YiIZ6Ljfd+lfSyAtZloacQgAkqSJCGhxM6t7cwwuUGPz4N0YKyvO6I9WDeMPMSo8Z4Ca/kJ6vMEYW5f1MX7W1lVxaG8vqX8hNFdjlc0iCBBSF4ERT/3Pl7RbMGMXF2KZMh/C+gDpNS7RRsp0OaRGzx0/t8e0jgmcczyLCWEePhni/23JWalzjdu0a3ZvgEaNLXeugEAAA==" |
| 200 | +} |
| 201 | + |
| 202 | +def handler(event, context): |
| 203 | + validate(event=event, schema=json_schema_dict, envelope="powertools_base64_gzip(data) | powertools_json(@)") # highlight-line |
| 204 | + return event |
| 205 | + |
| 206 | +handler(event=sample_event, context={}) |
| 207 | +``` |
| 208 | + |
| 209 | +## Bring your own JMESPath function |
| 210 | + |
| 211 | +<Note type="warning"> |
| 212 | + This should only be used for advanced use cases where you have special formats not covered by the built-in functions. |
| 213 | + <br/><br/> |
| 214 | + This will <strong>replace all built-in functions provided with what you provide</strong>. |
| 215 | +</Note> |
| 216 | + |
| 217 | +For special binary formats that you want to decode before applying JSON Schema validation, you can bring your own [JMESPath function](https://github.com/jmespath/jmespath.py#custom-functions) and any additional option via `jmespath_options` param. |
| 218 | + |
| 219 | +```python:title=custom_jmespath_function |
| 220 | +from aws_lambda_powertools.utilities.validation import validate |
| 221 | +from jmespath import functions |
| 222 | +
|
| 223 | +json_schema_dict = {..} |
| 224 | +
|
| 225 | +class CustomFunctions(functions.Functions): |
| 226 | +
|
| 227 | + @functions.signature({'types': ['string']}) |
| 228 | + def _func_special_decoder(self, s): |
| 229 | + return my_custom_decoder_logic(s) |
| 230 | +
|
| 231 | +custom_jmespath_options = {"custom_functions": CustomFunctions()} |
| 232 | +
|
| 233 | +def handler(event, context): |
| 234 | + validate(event=event, schema=json_schema_dict, envelope="", jmespath_options=**custom_jmespath_options) # highlight-line |
| 235 | + return event |
| 236 | +``` |
0 commit comments