BUG: fixes indexing with monotonic decreasing DTI (#19362) #20677

mapehe · 2018-04-13T06:30:02Z

closes BUG: indexing with monotonic decreasing DTI #19362
tests added / passed
passes git diff upstream/master -u -- "*.py" | flake8 --diff
whatsnew entry

Seems that #19362 is caused by the fact that np.any([0]) is false. Here's a suggested solution. Also added a test.

jreback

pls add a whatsnew note (indexing bug fix section)

jreback · 2018-04-14T12:37:14Z

pandas/core/indexes/datetimelike.py

@@ -342,7 +342,7 @@ def _format_with_header(self, header, **kwargs):
    def __contains__(self, key):
        try:
            res = self.get_loc(key)
-            return is_scalar(res) or type(res) == slice or np.any(res)
+            return is_scalar(res) or type(res) == slice or np.any(res + 1)


can you change type(res) == slice to isinstance(res, slice).

I think condition could be (for the last clause)
is_list_like(res) and len(res). just need to account for an indexer, not the character of it

jreback · 2018-04-14T12:37:35Z

pandas/tests/indexing/test_datetime.py

+        df = pd.DataFrame({'A': [1, 2, 3]},
+                          index=pd.date_range('20170101',
+                                              periods=3)[::-1])
+        df.loc['2017-01-03']


can you assert on the result of this

jreback · 2018-04-14T12:40:29Z

pandas/tests/indexing/test_datetime.py

@@ -252,3 +252,11 @@ def test_series_partial_set_period(self):
                                        check_stacklevel=False):
            result = ser.loc[keys]
        tm.assert_series_equal(result, exp)
+
+    def test_monotone_DTI_indexing_bug(self):


this test belongs in one of:

pandas/tests/indexes/datetimes/test_datetime.py pandas/tests/indexes/datetimes/test_partial_slicing.py

codecov · 2018-04-14T14:40:05Z

Codecov Report

❗ No coverage uploaded for pull request base (master@8756f55). Click here to learn what that means.
The diff coverage is 100%.

@@            Coverage Diff            @@
##             master   #20677   +/-   ##
=========================================
  Coverage          ?   91.81%           
=========================================
  Files             ?      153           
  Lines             ?    49279           
  Branches          ?        0           
=========================================
  Hits              ?    45247           
  Misses            ?     4032           
  Partials          ?        0

Flag	Coverage Δ
#multiple	`90.2% <100%> (?)`
#single	`41.9% <100%> (?)`

Impacted Files	Coverage Δ
pandas/core/indexes/datetimelike.py	`96.72% <100%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8756f55...ecff6e4. Read the comment docs.

mapehe · 2018-04-14T14:46:06Z

Thanks for the helpful comments @jreback, I think the updated commit should address the points.

jreback

lgtm. some small comments. ping on green.

jreback · 2018-04-14T15:00:23Z

pandas/tests/indexes/datetimes/test_datetime.py

@@ -368,3 +368,14 @@ def test_factorize_dst(self):
    def test_unique(self, arr, expected):
        result = arr.unique()
        tm.assert_index_equal(result, expected)
+


can you move near the other tests?

also can you add all of the examples from the original issue (e.g. the working ones). so this is a complete example.

can you add a comment on what is being tested here.

can you move near the other tests?

Sorry, didn't quite get this one, but tried to move this closer to other tests that seem a little similar.

also can you add all of the examples from the original issue (e.g. the working ones). so this is a complete example.

There wasn't a lot of stuff in the original issue, but I added what the person who posted that issue was attempting to do.

can you add a comment on what is being tested here.

Added a short comment.

TomAugspurger

Small style comment. @mapehe can you update fixing that & @jreback's suggestions (and fix the merge conflict). Doing a release this week.

TomAugspurger · 2018-04-15T19:27:49Z

pandas/core/indexes/datetimelike.py

@@ -342,7 +342,8 @@ def _format_with_header(self, header, **kwargs):
    def __contains__(self, key):
        try:
            res = self.get_loc(key)
-            return is_scalar(res) or type(res) == slice or np.any(res)
+            return is_scalar(res) or isinstance(res, slice) or \


Typically we use parenthesis to break long lines, not \.

Changed that, also fixed the conflict.

mapehe · 2018-04-16T11:32:31Z

3c4f38b was all green @jreback

jreback · 2018-04-20T10:39:18Z

thanks @mapehe!

mapehe · 2018-04-20T10:52:46Z

Np!

…andas-dev#20677)

mapehe mentioned this pull request Apr 14, 2018

BUG: indexing with monotonic decreasing DTI #19362

Closed

jreback requested changes Apr 14, 2018

View reviewed changes

jreback added Bug Indexing Related to indexing on series/frames, not to indexes themselves Datetime Datetime data dtype labels Apr 14, 2018

jreback reviewed Apr 14, 2018

View reviewed changes

mapehe force-pushed the datetime_bug branch from 251ae24 to 3c84724 Compare April 14, 2018 14:39

mapehe force-pushed the datetime_bug branch from 3c84724 to db669b5 Compare April 14, 2018 14:43

jreback requested changes Apr 14, 2018

View reviewed changes

jreback added this to the 0.23.0 milestone Apr 14, 2018

TomAugspurger reviewed Apr 15, 2018

View reviewed changes

mapehe force-pushed the datetime_bug branch from db669b5 to 79638c2 Compare April 16, 2018 09:57

BUG: fixes indexing with monotonic decreasing DTI (pandas-dev#19362)

3c4f38b

mapehe force-pushed the datetime_bug branch from 79638c2 to 3c4f38b Compare April 16, 2018 10:11

jreback added 2 commits April 16, 2018 06:38

Merge branch 'master' into PR_TOOL_MERGE_PR_20677

274cddc

doc typos & move tests

ecff6e4

jreback merged commit 3a2e9e6 into pandas-dev:master Apr 20, 2018

mapehe deleted the datetime_bug branch April 20, 2018 12:46

fjetter pushed a commit to fjetter/pandas that referenced this pull request Apr 21, 2018

BUG: fixes indexing with monotonic decreasing DTI (pandas-dev#19362) (p…

cde3cab

…andas-dev#20677)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BUG: fixes indexing with monotonic decreasing DTI (#19362) #20677

BUG: fixes indexing with monotonic decreasing DTI (#19362) #20677

mapehe commented Apr 13, 2018 •

edited

Loading

jreback left a comment

jreback Apr 14, 2018

jreback Apr 14, 2018

jreback Apr 14, 2018

codecov bot commented Apr 14, 2018 •

edited

Loading

mapehe commented Apr 14, 2018

jreback left a comment

jreback Apr 14, 2018

mapehe Apr 16, 2018

TomAugspurger left a comment

TomAugspurger Apr 15, 2018

mapehe Apr 16, 2018 •

edited

Loading

mapehe commented Apr 16, 2018

jreback commented Apr 20, 2018

mapehe commented Apr 20, 2018

BUG: fixes indexing with monotonic decreasing DTI (#19362) #20677

BUG: fixes indexing with monotonic decreasing DTI (#19362) #20677

Conversation

mapehe commented Apr 13, 2018 • edited Loading

jreback left a comment

Choose a reason for hiding this comment

jreback Apr 14, 2018

Choose a reason for hiding this comment

jreback Apr 14, 2018

Choose a reason for hiding this comment

jreback Apr 14, 2018

Choose a reason for hiding this comment

codecov bot commented Apr 14, 2018 • edited Loading

Codecov Report

mapehe commented Apr 14, 2018

jreback left a comment

Choose a reason for hiding this comment

jreback Apr 14, 2018

Choose a reason for hiding this comment

mapehe Apr 16, 2018

Choose a reason for hiding this comment

TomAugspurger left a comment

Choose a reason for hiding this comment

TomAugspurger Apr 15, 2018

Choose a reason for hiding this comment

mapehe Apr 16, 2018 • edited Loading

Choose a reason for hiding this comment

mapehe commented Apr 16, 2018

jreback commented Apr 20, 2018

mapehe commented Apr 20, 2018

mapehe commented Apr 13, 2018 •

edited

Loading

codecov bot commented Apr 14, 2018 •

edited

Loading

mapehe Apr 16, 2018 •

edited

Loading