Skip to content

DatetimeIndex._maybe_cast_slice_bound with duplicates #16515

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
TomAugspurger opened this issue May 26, 2017 · 2 comments
Closed

DatetimeIndex._maybe_cast_slice_bound with duplicates #16515

TomAugspurger opened this issue May 26, 2017 · 2 comments
Labels
Algos Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff Datetime Datetime data dtype Indexing Related to indexing on series/frames, not to indexes themselves
Milestone

Comments

@TomAugspurger
Copy link
Contributor

DatetimeIndex._maybe_cast_slice_bound can return the wrong value when you have an index that is monotonically decreasing, but not strictly monotonically decreasing.

In [1]: import pandas as pd

In [2]: pd.DatetimeIndex(['2017', '2017'])
Out[2]: DatetimeIndex(['2017-01-01', '2017-01-01'], dtype='datetime64[ns]', freq=None)

In [3]: pd.DatetimeIndex(['2017', '2017'])._maybe_cast_slice_bound('2017-01-01', 'left', 'loc')
Out[3]: Timestamp('2017-01-01 23:59:59.999999999')

The issue is here That check really needs to check for strict monotonic decreasing, not just monotonic decreasing.

Have we hit any other needs for strict monotonic algos? This is such an extreme edge case, not sure it's worthwhile on its own.

Came up in dask/dask#2389

@TomAugspurger TomAugspurger added Algos Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff Difficulty Intermediate Indexing Related to indexing on series/frames, not to indexes themselves Datetime Datetime data dtype labels May 26, 2017
@TomAugspurger TomAugspurger added this to the Next Major Release milestone May 26, 2017
@TomAugspurger
Copy link
Contributor Author

TomAugspurger commented May 29, 2017

Actually, we don't need any additional cython code. We can just do

@property
def is_strictly_monotonically_increasing(self):
    return self.is_unique and self.is_monotonic_increasing

Do we want to add that as a method? I'm slightly in favor of adding it.

@jreback
Copy link
Contributor

jreback commented May 31, 2017

@TomAugspurger yeah that sound fine (add tests as well), do this in base and it should just work for all Index sub-classes (maybe add is_strictly_monotonicaly_decreasing as well)

TomAugspurger added a commit to TomAugspurger/pandas that referenced this issue May 31, 2017
Fixed an edge case in partial string indexing where we would incorrectly flip
the endpoint on a slice, since we checked for monotonicity when we needed strict
monotonicity.

Closes pandas-dev#16515
xref dask/dask#2389
TomAugspurger added a commit to TomAugspurger/pandas that referenced this issue May 31, 2017
Fixed an edge case in partial string indexing where we would incorrectly flip
the endpoint on a slice, since we checked for monotonicity when we needed strict
monotonicity.

Closes pandas-dev#16515
xref dask/dask#2389
TomAugspurger added a commit to TomAugspurger/pandas that referenced this issue May 31, 2017
Fixed an edge case in partial string indexing where we would incorrectly flip
the endpoint on a slice, since we checked for monotonicity when we needed strict
monotonicity.

Closes pandas-dev#16515
xref dask/dask#2389
TomAugspurger added a commit to TomAugspurger/pandas that referenced this issue May 31, 2017
Fixed an edge case in partial string indexing where we would incorrectly flip
the endpoint on a slice, since we checked for monotonicity when we needed strict
monotonicity.

Closes pandas-dev#16515
xref dask/dask#2389
@jreback jreback modified the milestones: 0.20.2, Next Major Release Jun 1, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Algos Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff Datetime Datetime data dtype Indexing Related to indexing on series/frames, not to indexes themselves
Projects
None yet
Development

No branches or pull requests

2 participants