Skip to content

BUG: Cleaner exception when .iloc called with non-integer list #25759

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Mar 26, 2019

Conversation

kykosic
Copy link
Contributor

@kykosic kykosic commented Mar 18, 2019

Issue #25753 was raised requesting more verbose error messages when calling loc and iloc with mixed dtype arguments. In this pull request, I've added a new error message when calling iloc with a non-numeric dtype list-like object.

This fixes numpy ultimately raising an ambiguous exception when, for example, calling .iloc with a list of string column names:

>>> print(data.iloc[['Colorado', 'Utah'], [3, 0, 1]])
Old:
TypeError: cannot perform reduce with flexible type
New:
IndexError: .iloc requires integer indexers, got ['Colorado' 'Utah']

However, the current exception when calling .loc with integer column names when the column names are actually strings I feel is already accurate:

>>> print(data.loc[['Colorado', 'Utah'], [3, 0, 1]])
KeyError: "None of [Int64Index([3, 0, 1], dtype='int64')] are in the [columns]"

@kykosic kykosic changed the title Cleaner exception when .iloc called with non-integer list BUG: Cleaner exception when .iloc called with non-integer list Mar 18, 2019
Copy link
Member

@mroeschke mroeschke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs a new test to catch that this error message is raised.

@codecov
Copy link

codecov bot commented Mar 18, 2019

Codecov Report

Merging #25759 into master will decrease coverage by <.01%.
The diff coverage is 50%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #25759      +/-   ##
==========================================
- Coverage   91.25%   91.24%   -0.01%     
==========================================
  Files         172      172              
  Lines       52977    52979       +2     
==========================================
+ Hits        48342    48343       +1     
- Misses       4635     4636       +1
Flag Coverage Δ
#multiple 89.82% <50%> (-0.01%) ⬇️
#single 41.74% <50%> (ø) ⬆️
Impacted Files Coverage Δ
pandas/core/indexing.py 90.8% <50%> (-0.08%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 707c720...4cb4667. Read the comment docs.

@codecov
Copy link

codecov bot commented Mar 18, 2019

Codecov Report

Merging #25759 into master will increase coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #25759      +/-   ##
==========================================
+ Coverage   91.26%   91.26%   +<.01%     
==========================================
  Files         172      172              
  Lines       52965    52967       +2     
==========================================
+ Hits        48337    48339       +2     
  Misses       4628     4628
Flag Coverage Δ
#multiple 89.82% <100%> (ø) ⬆️
#single 41.74% <50%> (ø) ⬆️
Impacted Files Coverage Δ
pandas/core/indexing.py 90.9% <100%> (+0.01%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c975fc4...a8caad0. Read the comment docs.

@WillAyd WillAyd added the Error Reporting Incorrect or improved errors from pandas label Mar 18, 2019
Copy link
Member

@WillAyd WillAyd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm @mroeschke

Copy link
Member

@mroeschke mroeschke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One small whatsnew change otherwise looks good to me.

@@ -2079,6 +2079,10 @@ def _validate_key(self, key, axis):
arr = np.array(key)
len_axis = len(self.obj._get_axis(axis))

if not np.issubdtype(arr.dtype, np.number):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

use is_integer_dtype

Copy link
Contributor Author

@kykosic kykosic Mar 19, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will cause a new exception when an array of float values is passed to .iloc. Currently float values are allowed and the array is cast to integer dtype. This will cause some regression.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

then you need to thread this in the appropriate place

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

alt you could check is_numeric_dtype

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright I tried to reorganize it a bit, let me know if I misunderstood what you meant.

@jreback jreback added this to the 0.25.0 milestone Mar 19, 2019
@jreback
Copy link
Contributor

jreback commented Mar 19, 2019

also merge master

@jreback
Copy link
Contributor

jreback commented Mar 19, 2019

lgtm. @mroeschke over to you

@jreback jreback merged commit 51c6a05 into pandas-dev:master Mar 26, 2019
@jreback
Copy link
Contributor

jreback commented Mar 26, 2019

thanks @kykosic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Error Reporting Incorrect or improved errors from pandas
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ERR: Raise a better message when mixing positional and label indexing in loc/iloc
4 participants