Bug: logger.remove_keys("xray_trace_id") is not removing xray_trace_id field for lambdas

[bandapatideepak]

Expected Behaviour

should not have xray_trace_id property when add it in logger.remove_keys

Current Behaviour

xray_trace_id property is dispalyed in the object even though it is added in logger.remove_keys

Code snippet

logger = Logger(level=INFO)
logger.remove_keys("xray_trace_id") 

def lambda_handler(event,context)
     log_obj= {
                         "somekey1":"somevalue1",
                         "somekey2":"somevalue2"}
    logger.info("this is info message",extra=log_obj)


above code is print structured object along with xray_trace_id property
even though  i removed it, it still displays

Possible Solution

exclude xray_trace_id value

Steps to Reproduce

print the coed snippet in any aws lambda

Powertools for AWS Lambda (Python) version

latest

AWS Lambda function runtime

3.7

Packaging format used

Lambda Layers, PyPi

Debugging logs

not available

[boring-cyborg]

Thanks for opening your first issue here! We'll come back to you as soon as we can.
In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link

[heitorlessa]

Hi @bandapatideepak - that's expected behaviour. If you don't want that key despite having X-Ray enabled, you can remove it at Logger initialisation, as described in our documentation:

https://docs.powertools.aws.dev/lambda/python/latest/core/logger/#overriding-log-records

logger = Logger(xray_trace_id=None)

---

Why doesn't remove_keys work?

Because xray_trace_id and other keys are added during log record creation. remove_keys would remove existing keys registered via append_keys, or during Logger initialisation.

Thank you for taking the time to create an issue regardless.

Feel free to reopen if this doesn't work for you.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.

[bandapatideepak]

Hi @heitorlessa
the issue still exists
below is the code snippet from sample lambda function,

import json
from logging import WARN, INFO, ERROR, DEBUG
from aws_lambda_powertools import Logger
from aws_lambda_powertools.logging.formatter import LambdaPowertoolsFormatter

def lambda_handler(event, context):
logger = Logger(level=INFO, xray_trace_id=None)
log_obj = {
"Testkey1":"Testvalue1","Testkey2":"Testvalue2","Testkey3":"Testvalue3"
}
logger.info("test info message",extra=log_obj)
# TODO implement
return {
'statusCode': 200,
'body': json.dumps('Hello from Lambda!')
}
below is the output for above execution

{
"level": "INFO",
"location": "lambda_handler:11",
"message": "test info message",
"timestamp": "2023-07-26 09:51:57,344+0000",
"service": "service_undefined",
"xray_trace_id": "1-64c0ecbc-14cb1e0a740d14c94b233b70",
"Testkey1": "Testvalue1",
"Testkey2": "Testvalue2",
"Testkey3": "Testvalue3"
}

you can quickly copy paste and add below link for lambda layer which is aws lambda power tools lambda layer
arn:aws:lambda:{region}:017000801446:layer:AWSLambdaPowertoolsPythonV2:40

[heitorlessa]

@bandapatideepak sorry to hear that. I'll look into it today

[heitorlessa]

confirmed, sending a fix shortly and sending a quick workaround you can use until we make the release.

[heitorlessa]

Done. Until we make a release (we do every two weeks), you can use the following custom formatter:

from aws_lambda_powertools import Logger
from aws_lambda_powertools.logging.formatter import LambdaPowertoolsFormatter
from aws_lambda_powertools.logging.types import LogRecord


class CustomFormatter(LambdaPowertoolsFormatter):
    def serialize(self, log: LogRecord) -> str:
        """Serialize final structured log dict to JSON str"""
        log.pop("xray_trace_id", None)  # removes X-Ray Trace ID key until PR #2852 is released
        return self.json_serializer(log)


logger = Logger(service="payment", xray_trace_id=None, logger_formatter=CustomFormatter())
logger.info("blah")

After the release, you can get rid of this custom formatter and simply use:

from aws_lambda_powertools import Logger

logger = Logger(service="payment", xray_trace_id=None)
logger.info("blah")

Thank you for your patience and truly appreciate you coming back to report it was still an issue @bandapatideepak

[heitorlessa]

Automation didn't kick in somehow - this was released last week. @bandapatideepak please let us know if you still find that in production in the latest release: https://github.com/aws-powertools/powertools-lambda-python/releases/tag/v2.22.0

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.

[github-actions]

This is now released under 2.23.0 version!