Skip to content

BUG: np.inf now causes Index to upcast from int to float #16996

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jul 18, 2017
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v0.21.0.txt
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,7 @@ Bug Fixes
~~~~~~~~~

- Fixes regression in 0.20, :func:`Series.aggregate` and :func:`DataFrame.aggregate` allow dictionaries as return values again (:issue:`16741`)
- Fixes inf upcast from integer indices in 0.20, previously :func:`Int64Index.__contains__` and `DataFrame.loc.__setitem__` raised an `OverflowError` when given `np.inf` (:issue:`16957`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right location, though I would shorten to this:

- Fixes bug where indexing with `np.inf` caused an `OverflowError` to be raised (:issue:`16957`)


Conversion
^^^^^^^^^^
Expand Down
6 changes: 3 additions & 3 deletions pandas/core/indexes/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -666,7 +666,7 @@ def _try_convert_to_int_index(cls, data, copy, name):
res = data.astype('u8', copy=False)
if (res == data).all():
return UInt64Index(res, copy=copy, name=name)
except (TypeError, ValueError):
except (OverflowError, TypeError, ValueError):
pass

raise ValueError
Expand Down Expand Up @@ -1640,7 +1640,7 @@ def __contains__(self, key):
hash(key)
try:
return key in self._engine
except (TypeError, ValueError):
except (OverflowError, TypeError, ValueError):
return False

_index_shared_docs['contains'] = """
Expand Down Expand Up @@ -3365,7 +3365,7 @@ def _maybe_cast_indexer(self, key):
ckey = int(key)
if ckey == key:
key = ckey
except (ValueError, TypeError):
except (OverflowError, ValueError, TypeError):
pass
return key

Expand Down
41 changes: 41 additions & 0 deletions pandas/tests/indexing/test_indexing.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,35 @@ def f():

pytest.raises(ValueError, f)

def test_inf_upcast(self):
# GH 16957
# We should be able to use np.inf as a key
# np.inf should cause an index to convert to float

# Test with np.inf in rows
df = pd.DataFrame(columns=[0])
df.loc[1] = 1
df.loc[2] = 2
df.loc[np.inf] = 3

# make sure we can look up the value
result = df.loc[np.inf, 0]
tm.assert_almost_equal(result, 3)

result = df.index
expected = pd.Float64Index([1, 2, np.inf])
tm.assert_index_equal(result, expected)

# Test with np.inf in columns
df = pd.DataFrame()
df.loc[0, 0] = 1
df.loc[1, 1] = 2
df.loc[0, np.inf] = 3

result = df.columns
expected = pd.Float64Index([0, 1, np.inf])
tm.assert_index_equal(result, expected)

def test_setitem_dtype_upcast(self):

# GH3216
Expand Down Expand Up @@ -542,6 +571,18 @@ def test_astype_assignment_with_dups(self):
# result = df.get_dtype_counts().sort_index()
# expected = Series({'float64': 2, 'object': 1}).sort_index()

def test_coercion_with_contains(self):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a couple of examples where the expected result is True.

# Related to GH 16957
# Checking if Int64Index contains np.inf should catch the OverflowError
for val in [np.inf, np.nan]:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can avoid this for-loop using pytest parametrization, which you can see here

index = pd.Int64Index([1, 2, 3])
result = val in index
tm.assert_almost_equal(result, False)
Copy link
Member

@gfyoung gfyoung Jul 17, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you should be able to write:

assert val not in index

instead of the two lines above.


index = pd.UInt64Index([1, 2, 3])
result = np.inf in index
tm.assert_almost_equal(result, False)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment as above


def test_index_type_coercion(self):

with catch_warnings(record=True):
Expand Down