chore(tests): build and deploy Lambda Layer stack once

[heitorlessa]

Issue number: #1464

Summary

This PR introduces LambdaLayerStack that is deployed per test session. It coordinates all parallel workers across CPU to wait for completion before they begin deploying their respective stacks and test run.

This diagram may better explain. More details on a few differences at the comment section.

graph TD
    A[make e2e test] -->Spawn{"Split and group tests <br>by feature and CPU"}

    Spawn -->|Worker0| Worker0_Start["Load tests"]
    Spawn -->|Worker1| Worker1_Start["Load tests"]
    Spawn -->|WorkerN| WorkerN_Start["Load tests"]

    Worker0_Start -->|Wait| LambdaLayerStack["Lambda Layer Stack Deployment"]
    Worker1_Start -->|Wait| LambdaLayerStack["Lambda Layer Stack Deployment"]
    WorkerN_Start -->|Wait| LambdaLayerStack["Lambda Layer Stack Deployment"]

    LambdaLayerStack -->|Worker0| Worker0_Deploy["Launch feature stack"]
    LambdaLayerStack -->|Worker1| Worker1_Deploy["Launch feature stack"]
    LambdaLayerStack -->|WorkerN| WorkerN_Deploy["Launch feature stack"]

    Worker0_Deploy -->|Worker0| Worker0_Tests["Run tests"]
    Worker1_Deploy -->|Worker1| Worker1_Tests["Run tests"]
    WorkerN_Deploy -->|WorkerN| WorkerN_Tests["Run tests"]

    Worker0_Tests --> ResultCollection
    Worker1_Tests --> ResultCollection
    WorkerN_Tests --> ResultCollection

    ResultCollection{"Wait for workers<br/>Collect test results"}
    ResultCollection --> TestEnd["Report results"]
    ResultCollection --> DeployEnd["Delete Stacks"]

Loading

Changes

Please provide a summary of what's being changed

User experience

Please share what the user experience looks like before and after this change

Checklist

If your change doesn't seem to apply, please leave them unchecked.

• Meet tenets criteria
• I have performed a self-review of this change
• Changes have been tested
• Changes are documented
• PR title follows conventional commit semantics

Is this a breaking change?

RFC issue number:

Checklist:

• Migration process documented
• Implement warnings (if it can live side by side)

Acknowledgment

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

[codecov-commenter]

Codecov Report

Merging #1466 (127b093) into develop (6930b42) will increase coverage by 0.00%.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop    #1466   +/-   ##
========================================
  Coverage    99.89%   99.89%           
========================================
  Files          123      123           
  Lines         5497     5499    +2     
  Branches       629      629           
========================================
+ Hits          5491     5493    +2     
  Misses           2        2           
  Partials         4        4           

Impacted Files	Coverage Δ
aws_lambda_powertools/utilities/batch/sqs.py	96.38% <0.00%> (+0.08%)	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[heitorlessa]

I've increased test distribution parallelization algorithm from module (loadscope) to per test file (loadist).

• Sequential test: 279.52s
• Parallel test + Layer per stack: 254.53s
• Parallel + Single Layer: 243.58s

Numbers are slightly higher now as I've included all extra dependencies to futureproof when we test Parser.

[heitorlessa]

Coordinating across shared queue memory required some changes in how we define infrastructure and their dependencies to run.

infrastructure fixtures per feature (e.g., Tracer, Logger, etc.) must accept layer_arn argument. This dependency injection also ensures all workers scattered across CPUs will wait for a session fixture. We then moved away from deploy_once as this is controlled up a level, making it more explicit (slightly more verbose too :/)

BEFORE

from pathlib import Path

import pytest

from tests.e2e.logger.infrastructure import LoggerStack


@pytest.fixture(autouse=True, scope="module")
def infrastructure(request: pytest.FixtureRequest, tmp_path_factory: pytest.TempPathFactory, worker_id: str):
    yield from deploy_once(stack=LoggerStack, request=request, tmp_path_factory=tmp_path_factory, worker_id=worker_id)

AFTER

from pathlib import Path

import pytest

from tests.e2e.logger.infrastructure import LoggerStack


@pytest.fixture(autouse=True, scope="module")
def infrastructure(request: pytest.FixtureRequest, lambda_layer_arn: str):
    stack = LoggerStack(handlers_dir=Path(f"{request.path.parent}/handlers"), layer_arn=lambda_layer_arn)
    try:
        yield stack.deploy()
    finally:
        stack.delete()

[heitorlessa]

Stacks changed signature to accept a new optional argument (layer_arn). To reduce some level of verbosity I also added a class attribute (FEATURE_NAME) to make it easier for newcomers to copy/paste as only create_resources and the feature name will change.

BEFORE

from pathlib import Path

from tests.e2e.utils.infrastructure import BaseInfrastructureV2


class LoggerStack(BaseInfrastructureV2):
    def __init__(self, handlers_dir: Path, feature_name: str = "logger") -> None:
        super().__init__(feature_name, handlers_dir)

    def create_resources(self):
        self.create_lambda_functions()

AFTER

from pathlib import Path

from tests.e2e.utils.infrastructure import BaseInfrastructureV2


class LoggerStack(BaseInfrastructureV2):
    FEATURE_NAME = "logger"

    def __init__(self, handlers_dir: Path, feature_name: str = FEATURE_NAME, layer_arn: str = "") -> None:
        super().__init__(feature_name, handlers_dir, layer_arn)

    def create_resources(self):
        self.create_lambda_functions()

[rubenfonseca]

Amazing work Heitor, no comments from me. I was expecting a lot of logic to handle the "map-reduce", but it seems the pytest plugin already handles this for you. So no comments here :)

[heitorlessa]

I'm gonna remove the old BaseInfrastructure and rename V2 so we solidify this thing now that all stacks are migrated.

[heitorlessa]

thanks for the approval @rubenfonseca !! Merging now, and will send one last PR to enable E2E at CI level and document it in the contributing guide.