Feature request: allow custom timestamps for metrics

[pseidel-kcf]

Use case

I'm creating a lambda to report metrics from an external source to Cloudwatch for observability within AWS. I'm looking for a way to set a custom timestamp for the metrics emitted each invocation.

The metrics I'm querying from the external source will basically be a list of (metric name, value) tuples for a specific time. This specific time will be in the recent past, maybe a few minutes old, but I'm not sure that matters here.

I can easily add the key and values to metrics, however I cannot set an explicit time because Metrics does not expose a way and the the time ultimately always set to datetime.now() when rendered to emf.

Solution/User Experience

I'm imagining a simple api to set the timestamp on the Metrics object. If a timestamp is not specific then the current time should be used.

metrics = Metrics()

@metrics.log_metrics 
def lambda_handler(event: dict, context: LambdaContext):
    time = ...
    metric_values: list[(string, string)] = ...
    for (name, value) in metric_values:
            metrics.add_metric(name=name unit=MetricUnit.Count, value=value)
   metrics.set_timestamp(time) # <---

Alternative solutions

aws-embedded-metrics-python supports setting the timestamp on their MetricLogger class.

Acknowledgment

• This feature request meets Powertools for AWS Lambda (Python) Tenets
• Should this be considered in other Powertools for AWS Lambda languages? i.e. Java, TypeScript, and .NET

[boring-cyborg]

Thanks for opening your first issue here! We'll come back to you as soon as we can.
In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link

[heitorlessa]

Hey! Thank you so much for opening a feature request — we should do this, it wasn’t allowed by CloudWatch EMF in the earlier implementation but I checked and it is now! We’re at reinvent this week hence the delay. If you’d like to take a stab at it and open up a PR, we’d be more than happy to review and/or improve it.

…

On Tue, 28 Nov 2023 at 12:31, boring-cyborg[bot] ***@***.***> wrote: Thanks for opening your first issue here! We'll come back to you as soon as we can. In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link <https://discord.gg/B8zZKbbyET> — Reply to this email directly, view it on GitHub <#3429 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAZPQBARKKJCR5XCDX4K65DYGZC2VAVCNFSM6AAAAAA76LLMDCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMZQGY3TOMZQGA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.*** .com>

[rubenfonseca]

@pseidel-kcf just checking to see if you would like to work on this one?

[pseidel-kcf]

Thanks for the follow up.

I don't have capacity to work on this.

This is no longer a blocker for me because the external source from my example is implementing a built-in way to export metrics.

[rubenfonseca]

No problem! We'll keep it in the backlog to see if there's more interest in this feature. Thank you so much for opening this!

[MatasNed]

@rubenfonseca Would it be possible for me to contribute to this? I'd be interested. Let me know

[leandrodamascena]

Hey everyone! I submitted a PullRequest to add this new feature, but I have some questions.

1 - Timestamp type

According to the AWS Documentation timestamp is a number representing the time stamp used for metrics extracted from the event. Values MUST be expressed as the number of milliseconds after Jan 1, 1970 00:00:00 UTC.

That said, should we only allow int inputs with epoch value, or should customers be able to pass datetime objects and have them converted internally?

2 - Invalid values

If a customer passes an invalid number, we will emit a warning and fallback to the current timestamp instead of raising an exception.

I'm still working on the pull request and I expect to have it merged by next week.

Thank you.

[rubenfonseca]

1 - I agree that we should start simple with "int", as we can always overload the function later if there's the demand for other types

Great job picking this up Leandro

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.

[github-actions]

This is now released under 2.36.0 version!