Skip to content

Investigate dplyr rolling joins as alternative to data.table's for archive operations #265

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
brookslogan opened this issue Feb 13, 2023 · 1 comment
Open

Investigate dplyr rolling joins as alternative to data.table's for archive operations #265

brookslogan opened this issue Feb 13, 2023 · 1 comment
Labels
cleanup improvements to developing&building experience&quality, not directly to built pkg&docs P2 low priority performance
Milestone
[CSTE] Improve sc...

Comments

@brookslogan
Copy link
Contributor

If dplyr's rolling joins are performant, this may be one approach to getting away from reference semantics altogether.
@brookslogan brookslogan added P2 low priority cleanup improvements to developing&building experience&quality, not directly to built pkg&docs performance labels Feb 13, 2023
@brookslogan brookslogan self-assigned this Oct 20, 2023
@brookslogan
Copy link
Contributor Author

Cursory initial testing revealed that dplyr took 2x the time, and polars and duckdb took similar time to, data.table, for as_of operations. polars and duckdb also involve more installation time. I still need to check if there are indexing features that might change the as_of performance comparison or extra features to consider.

This was referenced Jan 31, 2024
Closed
Open
@brookslogan brookslogan removed their assignment Feb 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
cleanup improvements to developing&building experience&quality, not directly to built pkg&docs P2 low priority performance
Projects
None yet
Milestone
[CSTE] Improve scaling of epiprocess
Development

No branches or pull requests
2 participants
@dsweber2 @brookslogan