Bug: High import time when importing ApiGatewayResolver because of CodePipelineJobEvent

[darnley]

Expected Behaviour

Do not import CodePipelineJobEvent when using ApiGatewayResolver, because the CodePipelineJobEvent will import boto3 and huge increase the AWS Lambda Cold Start or prevent boto3 from being called unexpectedly.

Current Behaviour

When importing ApiGatewayResolver, it is importing CodePipelineJobEvent and boto3. However, ApiGatewayResolver does not need boto3.

This behaviour is increasing the AWS Lambda cold start. When I use a profiling tool in my machine, it seems that just to import it is costing ~200ms. When running on AWS with a 800MB runtime, it is costing ~400ms.

According to the profiling in this use case, importing CodePipelineJobEvent costed 61.9% of time, but it will not be used. That is weird.

profile.log

Code snippet

from aws_lambda_powertools.logging import Logger
from aws_lambda_powertools.metrics import Metrics
from aws_lambda_powertools.tracing import Tracer
from aws_lambda_powertools.event_handler.api_gateway import ApiGatewayResolver

Possible Solution

No response

Steps to Reproduce

1. Put this code on a Python file;
2. Install tuna package using pip install tuna;
3. Run command

python -X importtime MY_SCRIPT.py 2> profile.log && tuna profile.log

Browser will appear with the profiling graph.

Powertools for AWS Lambda (Python) version

latest

AWS Lambda function runtime

3.9

Packaging format used

PyPi

Debugging logs

No response

[boring-cyborg]

Thanks for opening your first issue here! We'll come back to you as soon as we can.
In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link

[darnley]

I do not know if this is a really "bug". I am just want help with this problem. :) Thank you.

[leandrodamascena]

Hi @darnley! Tudo bom? 😃

Thank you very much for opening this issue, I really appreciate it!! Actually, I consider this a bug because it affects performance and you could demonstrate this. The problem here is how things happen in Python and not exactly a bug generated because of a wrong code or something like this. Let me try to explain what is going on and a possible solution.

We have the Event Source Data Class utility. This utility helps developers to transform the Lambda Payload into a dictionary-like object, what makes easy to access properties and nested items. The ApiGatewayResolver takes advantage of the DataClass utility to make the code cleaner and this is where the problem occurs.

We are not directly importing the CodePipelineJobEvent class in the ApiGatewayResolver, but In this line we import 4 DataClass to use it in all event_handler resolvers, and the way Python works will always run __init__.py which exposes our entire DataClass module and consequently import boto3 from CodePipelineJobEvent.

I think we can solve this problem by changing the boto3 import in CodePipelineJobEvent class. We just need to import boto3 if the customer explicitly wants to download an artifact from a CodePipeline Job and this is not a common case.

I've made some changes in my local environment and check this:

If you have bandwidth, please send a PR to fix this, if not, don't worry, I can fix it. We are planning to release a version this week and this issue makes it more urgent than I expected.

Muito obrigado 🚀

[darnley]

Hello, @leandrodamascena! Tudo bem e com você? 😊

That sounds perfect and thanks for the explanation. I am sending a PR to fix this.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.