Skip to content

BUG: date_range inclusive parameter behavior doesn't match interval notation #55319

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 22 commits into from
Closed

BUG: date_range inclusive parameter behavior doesn't match interval notation #55319

wants to merge 22 commits into from

Conversation

PiotrekB416
Copy link

@PiotrekB416 PiotrekB416 commented Sep 28, 2023

This pr fixes the issues with inclusive in date_range

Got rid of the problematic expression which seems to be i8values[] == start/end_i8, also the and in line 498 should have been an or. However after those fixes the if-statement was over complicated, so I simplified it.

@github-actions
Copy link
Contributor

This pull request is stale because it has been open for thirty days with no activity. Please update and respond to this comment if you're still interested in working on this.

@github-actions github-actions bot added the Stale label Oct 31, 2023
@PiotrekB416
Copy link
Author

This pr is ready for review

@datapythonista datapythonista added Bug Datetime Datetime data dtype and removed Stale labels Nov 17, 2023
Copy link
Member

@datapythonista datapythonista left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution @PiotrekB416, and sorry it too so long to review, seems like this PR was forgotten in the long list of PRs we have.

It's a bit difficult to understand all possible cases reading the diff, and I'm worried that by fixing the bug we may be introducing new ones.

What I'd like to see in general is that you keep existing tests as they are (in general we shouldn't have wrong cases as tests), and that you add one new test that it reproduces the fixed issue.

In this case the test seems a bit cumbersome, was it asserting cases with the behavior we consider a bug in this PR?

@MarcoGorelli do you have an opinion here?

@MarcoGorelli MarcoGorelli self-requested a review November 17, 2023 14:59
@PiotrekB416
Copy link
Author

In this case the test seems a bit cumbersome, was it asserting cases with the behavior we consider a bug in this PR?

Yes, as far as I could tell the tests had a few mistakes in them, also they were written in a really strange and unnecessarily complex way.
For example this line:

expected = date_range(bday_start, bday_end, freq="D")

should get the expected range, but the inclusive argument is not passed to it

@datapythonista
Copy link
Member

Thanks for the feedback and the work on this @PiotrekB416

Marco will be able to provide better feedback when he has the time to review, but what makes sense to me is:

  • Leave the existing tests as they are in this PR
  • Add a clearer brand new test that hits the specific bug you are fixing. If it makes sense we can test more on it
  • Only if there is any specific part of the existing tests that fail after your changes, remove that part, in a ver controlled way of what's the behavior we were testing to be true, that we now want to be changed
  • In a follow up, we can do a proper refactoring of the tests of this functionality

While what you are doing here seems reasonable, it feels strange that the complex logic you are making much simpler here wasn't implemented for a reason, and that we may be missing some use case where your changes fix a bug, but introduce a new one. Changing too much here makes difficult to see if that will be the case. I feel like the approach I'm proposing should minimize the chances of it happening. What do you think?

Comment on lines -597 to +608
begintz = Timestamp("2011/1/1", tz="US/Eastern")
endtz = Timestamp("2014/1/1", tz="US/Eastern")
# begintz = Timestamp("2011/1/1", tz="US/Eastern")
# endtz = Timestamp("2014/1/1", tz="US/Eastern")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this part of the test comment out?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The shortest answer would be that those lines are not needed and can be safely deleted. I just commented them out when I was testing and then forgot to remove them.

And as for why they were there in the first place. They were used by _get_expected_range, but they are not needed for the function to work

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok thanks - if they're not needed, then to simplify reviews, could you please first open a precursor PR in which you remove anything that's not needed, and then we keep this PR focused on the logic changes (+ relevant tests)? thanks 🙏

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm afraid that might not be possible. The code is complicated because of its logic errors, thus I can't simplify it without making logic changes.
For example:
in test_range_closed_boundary

expected_right = both_boundary[1:]
expected_left = both_boundary[:-1]
expected_both = both_boundary[:]
expected_neither = both_boundary[1:-1]

this simplification doesn't work if there are no logic changes in the _generate_range function, because then the expected values are compared to incorrectly generated ranges.

@mroeschke
Copy link
Member

Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.

@mroeschke mroeschke closed this Apr 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Datetime Datetime data dtype
Projects
None yet
4 participants