Feature request: Shared Cache for Parameters

[alexbaileyuk]

Use case

I'm interested in using App Config for feature flags via the @aws-lambda-powertools/parameters library. I'm not sure it's 100% the right use case or there should be a separate module like @aws-lambda-powertools/features which is more specific and can handle more advanced use cases.

Regardless, this (and the regular parameters) would almost certainly benefit from a shared cache of some kind. Right now, if I have low traffic, caching is virtually irrelevant. As traffic increases, caching only helps if you have relatively low concurrency. In some batch handling cases, my functions scale concurrency significantly for a short time which again renders caching virtually irrelevant due to it being in function memory.

By allowing a central cache through DynamoDB or similar, we can effectively reduce the number of requests being sent to services like SSM and AppConfig which would reduce costs for users. For SSM, it's pretty cheap but App Config and similar services start to add up pretty quickly if functions are spinning up and down regularly.

One thing that I'm not sure about right now is if that would prevent some of the App Config functionality like roll-out percentages working properly. If that's the case, then maybe this is a non-starter.

Edit

I just found https://docs.powertools.aws.dev/lambda/python/3.13.0/utilities/feature_flags/ in the Python library. Something like this would be good. Though I still wonder about caching.

Solution/User Experience

I imagine that something similar to the persistence store approach from the idempotency module could be used here. Providing sensible caching options out of the box with an interface that would allow customers to add custom cache providers if required.

Alternative solutions

Alternative solutions could include the App Config agent (https://docs.aws.amazon.com/appconfig/latest/userguide/appconfig-integration-lambda-extensions-how-it-works.html) but again, that uses a local cache so comes with similar limitations.

Acknowledgment

• This feature request meets Powertools for AWS Lambda (TypeScript) Tenets
• Should this be considered in other Powertools for AWS Lambda languages? i.e. Python, Java, and .NET

Future readers

Please react with 👍 and your use case to help us understand customer demand.

[boring-cyborg]

Thanks for opening your first issue here! We'll come back to you as soon as we can.
In the meantime, check out the #typescript channel on our Powertools for AWS Lambda Discord: Invite link

[dreamorosi]

Hi @alexbaileyuk thank you for opening the issue.

You're right that the current caching solution in Parameters is primarily aimed at reducing costs/calls for AWS Lambda execution environments that handle multiple requests. If your traffic patterns are at the two ends of the spectrum then as you described, each new environment that is spun up will result in a request to the parameter store.

I'd like to understand a bit more about how you'd imagine this to work before making a decision, and specifically how do you see this utility interacting with both the cache and with the parameter store.

Right now cache invalidation is not a concern because of the same reasons you mentioned as downsides, but if we were to introduce a feature in this area we'd need to have a solid story around cache invalidation to prevent functions from consuming stale/invalid parameters.

Regarding AppConfig and the Feature Flag feature, it's definitely something that we're considering adding to this version of Powertools for AWS, however you're correct in assuming that any additional persistent caching layer would invalidate the deployment strategies offered by the service.

On another note, if you're open to replicating your configs/parameters in Amazon DynamoDB, then why not storing them in a DDB Table in the first place, and skip SSM/AppConfig entirely? The current feature set of Parameters already allows you to get values from Amazon DynamoDB, so in practice you could already either use it as primary store, or create your own caching logic and cache your values there.

[alexbaileyuk]

@dreamorosi I imagine that we'd have a caching layer within the parameters package. If we take DynamoDB as an example for a cache provider, the flow would look something like:

1. Check cache for parameter value
2. If found, return parameter
3. If not found, retrieve from SSM and store in DynamoDB with TTL

I do not think that cache invalidation is strictly required here. We already have an idea of TTL in the parameter package when working with local caching within function memory. I would suggest using that. We probably need to add jitter when checking against DynamoDB for a value so that we avoid a cache stampede.

If we did want to deal with cache invalidation, I would suggest it is done through the same method as currently import { clearCaches } from '@aws-lambda-powertools/parameters';. This would just trigger the provider to clear cache.

It's a good point about just using DynamoDB out of the box to store the parameters. I think it makes sense given the lack of feature package available today. However, it does restrict us from being able to use some of the more advanced app-config features around gradual roll outs etc.

[dreamorosi]

However, it does restrict us from being able to use some of the more advanced app-config features around gradual roll outs etc.

this proposed feature would not be different though. Rebuilding rollout mechanism is most definitely out of the scope here.

[dreamorosi]

With this said, I'd like to leave the feature request open for a while and see if there's interest also from other customers as well as hear the opinion of other maintainers.

For full transparency, it's most likely not something we will be addressing this nor next quarter and we'll likely revisit the decision towards the end of the year, unless there's significant interest or new details/info that make the decision more clear cut.

[dreamorosi]

Something I should add is that we should also do a cost estimate / model of usage to see if this actually makes sense from a pricing perspective. Things like SSM API calls are free for standard throughput & Secrets Manager charges you $0.05 x 10K requests - unsure if DynamoDB storage and WCU/RCU would come out cheaper at scale and depending on what you're storing.

[alexbaileyuk]

Understood thanks. It's more aimed towards App Config since that carries different costs. Looking at the estimates on https://aws.amazon.com/systems-manager/pricing/

Cost of configurations received =1 (configurations) * 2000 (servers) * 3 (updates a day) * 30 (days) * $0.0008 (price per configuration received)
=$144

It's doesn't feel unreasonable to assume that 2,000 new functions could be spun up per day, for 30 days and receive 3 updates. Each individual instance of a function would receive the configuration at start up as a minimum. I'd expect use cases much higher than this for production workloads running on serverless.

I think you're right that in the scope of SSM this is a non-issue. Secrets Manager I guess depends again on scale. Will stay subscribed to the issue and see if there is any more traction in the future. Thanks for your timely responses!