Skip to content

Support for approximate trimming in XAddOptions #2144

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
TunaYagci opened this issue Aug 12, 2021 · 3 comments
Closed

Support for approximate trimming in XAddOptions #2144

TunaYagci opened this issue Aug 12, 2021 · 3 comments
Labels
type: enhancement A general enhancement

Comments

@TunaYagci
Copy link

XADD command offers another option called approximateTrimming along with MAXLEN

About XADD mystream MAXLEN ~ 1000 in this link

However trimming with MAXLEN can be expensive: streams are represented by macro nodes into a radix tree, in order to be very memory efficient. Altering the single macro node, consisting of a few tens of elements, is not optimal. So it's possible to use the command in the following special form:

XADD mystream MAXLEN ~ 1000 * ... entry fields here ...

The ~ argument between the MAXLEN option and the actual count means, I don't really need this to be exactly 1000 items. It can be 1000 or 1010 or 1030, just make sure to save at least 1000 items. With this argument, the trimming is performed only when we can remove a whole node. This makes it much more efficient, and it is usually what you want.

Lettuce driver supports with this issue: #redis/lettuce#846

It would be nice to chain it with maxlen:

RedisStreamCommands.XAddOptions
  .maxlen(STREAM_LENGHT)
  .approximateTrimming(true)

also for Reactive:

ReactiveStreamCommands.AddStreamRecord
.maxlen(STREAM_LENGTH)
.approximateTrimming(true)

we can check this option in: org.springframework.data.redis.connection.lettuce.LettuceStreamCommands#xAdd

@spring-projects-issues spring-projects-issues added the status: waiting-for-triage An issue we've not yet triaged label Aug 12, 2021
@mp911de mp911de added status: ideal-for-contribution An issue that a contributor can help us with type: enhancement A general enhancement and removed status: waiting-for-triage An issue we've not yet triaged labels Sep 20, 2021
@mp911de
Copy link
Member

mp911de commented Sep 20, 2021

Thanks for your suggestion. Would you be interested in contributing to Spring Data by submitting a pull request?

@TunaYagci
Copy link
Author

@mp911de Yes, why not. I'm interested with it but cannot promise to any date yet.

@mp911de
Copy link
Member

mp911de commented Apr 24, 2023

Fixed via #2247

@mp911de mp911de closed this as completed Apr 24, 2023
@mp911de mp911de removed the status: ideal-for-contribution An issue that a contributor can help us with label Apr 24, 2023
@mp911de mp911de added this to the 2.7 M4 (2021.2.0) milestone Apr 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: enhancement A general enhancement
Projects
None yet
Development

No branches or pull requests

3 participants