Maintenance: review e2e test utilities

[dreamorosi]

Note
This issue was originally opened specifically for the Access Grant Aspect mentioned here, however there are more items that we should review from the integration tests so I have expanded the scope of the issue.

Summary

The integration / e2e tests are the result of many iterations over almost two years. With more utilities being added to the toolkit and more AWS resources to integrate with, some aspects of the utilities written to support the tests can be reviewed and improved.

As part of this task we should survey the code related to the deployment, running, invocation, and assertions of the integration tests and find areas of improvement.

Why is this needed?

The current implementation works, but readability and maintainability can be improved.

Which area does this relate to?

Tests

Solution

Not known yet, however it's likely that we'll have to create a number of smaller issues from this one.

Acknowledgment

• This request meets Lambda Powertools Tenets
• Should this be considered in other Lambda Powertools languages? i.e. Python, Java

[dreamorosi]

Currently as part of the integration tests of the Parameters utility we are creating a number of resources. These resources need to be accessed by one or more Lambda functions that is using the utility. This allows us to test the utility in a real world scenario and increase our confidence on its correctness.

Due to how the test harness is structured, we initially opted for a CDK Aspect that would grant access to said resources to a given function.

The logic and types of the aspect have however been sprawling and if we were to add any additional resource type both readability and maintainability would decrease.

While working on this, maybe also simplify/review the names of the tests in the AppConfigProvider test, i.e.:

test: 'get-freeform-json-binary',

could become:

test: 'get-json-freeform-transform-json',

as well as the test numbers.

[dreamorosi]

Another area to improve is the function currently called createStackWithLambdaFunction (here) which is used to create a CDK stack with one Lambda function in it.

The function was written at a time when only the core utilities existed, and as such, we didn't have the necessity of deploying other AWS resources.

Now that we have to deploy different types of resources for other utilities that interact with AWS, the function could be extended or rewritten into an utility that allows to easily create a stack with a function as well as any additional resource needed for that specific test

Ideally this helper should be able to also handle naming the resources with unique names that share a prefix and also handle resource access grant - both tasks currently handled in the specific tests and repeated multiple times.

[dreamorosi]

An additional area of improvement could be to revisit the invokeFunction (here) which is used to call the Lambda function for a specific test.

Similar to the comment above, the function was written at a time where the payload sent to the function was standard across the test surface. With more utilities being added we needed more flexibility, and for this reason the code has become hard to maintain and convoluted.

We should streamline the function and make it more flexible (or more simple by delegating more responsibility to the caller).

[dreamorosi]

Another area of improvement would be to move all the test utilities in their own dedicated workspace folder.

Currently they are part of the commons package, this was decided at a time in which we didn't have a good grasp of how the workspace worked and so that package became the "everything bin".

We should create a workspace folder dedicated to this type of utilities that are never to be published to npmjs.com and make it part of the workspace, so that all other packages can use it. Related to #1354

[dreamorosi]

Another area to look at and to investigate is whether we can leverage the new CDK package instead of the cdk-cli one that we are monkey-patching.

The aws-cdk-lib has a new module @aws-cdk/cli-lib-alpha module (link) currently in alpha that could replace our implementation in the future.

While I'm not necessarily advocating for the usage of an alpha module, I think we should dedicate some time to understand how it work and if it can help us.

---

EDIT:
See here.

[dreamorosi]

I've started looking into the @aws-cdk/cli-lib-alpha module and while it looks promising, it doesn't work yet with our use case.

I have created a PoC here and tried to setup the minimum amount of settings/resources to reproduce a similar setup to our e2e tests: https://github.com/dreamorosi/cdk-cli-poc

I ran into a potential bug / limitation and opened an issue on the CDK repo: aws/aws-cdk#26376
The issue was not necessarily a bug but rather, it was caused by the way vite (which I was using in my PoC) threats inputs from stdin and how the NodejsFunction construct expects them. Using a specific vite flag, or another test framework suppresses the issue.

[dreamorosi]

I have opened an issue to track the work needed to adopt the new AWS CDK cli library #1623.

[dreamorosi]

All the areas mentioned above were addressed in #1661, specifically:

• Access management to resources created as part of the Parameters utility has been simplified and usage of Aspects removed
• All the testing utilities have been moved to a dedicated internal package and out of commons
• Resource creation has been standardized across utilities
• Usage of the CDK CLI package has been rolled out replacing the workaround adopted so far

[github-actions]

⚠️ COMMENT VISIBILITY WARNING ⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.