-
-
Notifications
You must be signed in to change notification settings - Fork 18.5k
Use date32 for pyarrow date #52332
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use date32 for pyarrow date #52332
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think this necessarily should unequivocally be date32. There may be a use case for storing with increased precision.
@mroeschke the precision is always the same, it's unexpected to store non-fullday value in pa.date64 and date64 is always milliseconds: https://lists.apache.org/thread/q036r1q3cw5ysn3zkpvljx3s9ho18419 The only gain of using that type is the cheaper conversion for frameworks / languages using millisecond based timestamps, but it's slower and uses more memory. |
Owee, I'm MrMeeseeks, Look at me. There seem to be a conflict, please backport manually. Here are approximate instructions:
And apply the correct labels and milestones. Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon! Remember to remove the If these instructions are inaccurate, feel free to suggest an improvement. |
Thanks for the explanation @alippai |
Thanks for the swift merge and backport @mroeschke |
) Backport PR #52332: Use date32 for pyarrow date Co-authored-by: Ádám Lippai <[email protected]>
doc/source/whatsnew/vX.X.X.rst
file if fixing a bug or adding a new feature.