Skip to content

API: allow scalar setting/getting via float indexer on integer indexes #12370

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 7 additions & 4 deletions doc/source/advanced.rst
Original file line number Diff line number Diff line change
Expand Up @@ -717,6 +717,10 @@ values NOT in the categories, similarly to how you can reindex ANY pandas index.
Int64Index and RangeIndex
~~~~~~~~~~~~~~~~~~~~~~~~~

.. warning::

Indexing on an integer-based Index with floats has been clarified in 0.18.0, for a summary of the changes, see :ref:`here <whatsnew_0180.float_indexers>`.

``Int64Index`` is a fundamental basic index in *pandas*. This is an Immutable array implementing an ordered, sliceable set.
Prior to 0.18.0, the ``Int64Index`` would provide the default index for all ``NDFrame`` objects.

Expand All @@ -736,7 +740,6 @@ Float64Index
operations by about 30x and boolean indexing operations on the
``Float64Index`` itself are about 2x as fast.


.. versionadded:: 0.13.0

By default a ``Float64Index`` will be automatically created when passing floating, or mixed-integer-floating values in index creation.
Expand Down Expand Up @@ -797,12 +800,12 @@ In non-float indexes, slicing using floats will raise a ``TypeError``

.. warning::

Using a scalar float indexer has been removed in 0.18.0, so the following will raise a ``TypeError``
Using a scalar float indexer for ``.iloc`` has been removed in 0.18.0, so the following will raise a ``TypeError``

.. code-block:: python

In [3]: pd.Series(range(5))[3.0]
TypeError: cannot do label indexing on <class 'pandas.indexes.range.RangeIndex'> with these indexers [3.0] of <type 'float'>
In [3]: pd.Series(range(5)).iloc[3.0]
TypeError: cannot do positional indexing on <class 'pandas.indexes.range.RangeIndex'> with these indexers [3.0] of <type 'float'>

Further the treatment of ``.ix`` with a float indexer on a non-float index, will be label based, and thus coerce the index.

Expand Down
4 changes: 4 additions & 0 deletions doc/source/indexing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,10 @@ advanced indexing.
but instead subclass ``PandasObject``, similarly to the rest of the pandas objects. This should be
a transparent change with only very limited API implications (See the :ref:`Internal Refactoring <whatsnew_0150.refactoring>`)

.. warning::

Indexing on an integer-based Index with floats has been clarified in 0.18.0, for a summary of the changes, see :ref:`here <whatsnew_0180.float_indexers>`.

See the :ref:`MultiIndex / Advanced Indexing <advanced>` for ``MultiIndex`` and more advanced indexing documentation.

See the :ref:`cookbook<cookbook.selection>` for some advanced strategies
Expand Down
64 changes: 44 additions & 20 deletions doc/source/whatsnew/v0.18.0.txt
Original file line number Diff line number Diff line change
Expand Up @@ -1024,11 +1024,11 @@ Removal of deprecated float indexers
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

In :issue:`4892` indexing with floating point numbers on a non-``Float64Index`` was deprecated (in version 0.14.0).
In 0.18.0, this deprecation warning is removed and these will now raise a ``TypeError``. (:issue:`12165`)
In 0.18.0, this deprecation warning is removed and these will now raise a ``TypeError``. (:issue:`12165`, :issue:`12333`)

.. ipython:: python

s = pd.Series([1,2,3])
s = pd.Series([1, 2, 3], index=[4, 5, 6])
s
s2 = pd.Series([1, 2, 3], index=list('abc'))
s2
Expand All @@ -1037,15 +1037,18 @@ Previous Behavior:

.. code-block:: python

In [2]: s[1.0]
# this is label indexing
In [2]: s[5.0]
FutureWarning: scalar indexers for index type Int64Index should be integers and not floating point
Out[2]: 2

# this is positional indexing
In [3]: s.iloc[1.0]
FutureWarning: scalar indexers for index type Int64Index should be integers and not floating point
Out[3]: 2

In [4]: s.loc[1.0]
# this is label indexing
In [4]: s.loc[5.0]
FutureWarning: scalar indexers for index type Int64Index should be integers and not floating point
Out[4]: 2

Expand All @@ -1062,33 +1065,54 @@ Previous Behavior:

New Behavior:

For iloc, getting & setting via a float scalar will always raise.

.. code-block:: python

In [2]: s[1.0]
TypeError: cannot do label indexing on <class 'pandas.indexes.range.RangeIndex'> with these indexers [1.0] of <type 'float'>
In [3]: s.iloc[2.0]
TypeError: cannot do label indexing on <class 'pandas.indexes.numeric.Int64Index'> with these indexers [2.0] of <type 'float'>

In [3]: s.iloc[1.0]
TypeError: cannot do label indexing on <class 'pandas.indexes.range.RangeIndex'> with these indexers [1.0] of <type 'float'>
Other indexers will coerce to a like integer for both getting and setting. The ``FutureWarning`` has been dropped for ``.loc``, ``.ix`` and ``[]``.

In [4]: s.loc[1.0]
TypeError: cannot do label indexing on <class 'pandas.indexes.range.RangeIndex'> with these indexers [1.0] of <type 'float'>
.. ipython:: python

# .ix will now cause this to be a label lookup and coerce to and Index
In [5]: s2.ix[1.0] = 10
s[5.0]
s.loc[5.0]
s.ix[5.0]

In [6]: s2
Out[3]:
a 1
b 2
c 3
1.0 10
dtype: int64
and setting

.. ipython:: python

s_copy = s.copy()
s_copy[5.0] = 10
s_copy
s_copy = s.copy()
s_copy.loc[5.0] = 10
s_copy
s_copy = s.copy()
s_copy.ix[5.0] = 10
s_copy

Slicing will also coerce integer-like floats to integers for a non-``Float64Index``.

.. ipython:: python

s.loc[5.0:6]
s.ix[5.0:6]

Note that for floats that are NOT coercible to ints, the label based bounds will be excluded

.. ipython:: python

s.loc[5.1:6]
s.ix[5.1:6]

Float indexing on a ``Float64Index`` is unchanged.

.. ipython:: python

s = pd.Series([1,2,3],index=np.arange(3.))
s = pd.Series([1, 2, 3], index=np.arange(3.))
s[1.0]
s[1.0:2.5]

Expand Down
2 changes: 1 addition & 1 deletion pandas/core/frame.py
Original file line number Diff line number Diff line change
Expand Up @@ -2016,7 +2016,7 @@ def _getitem_array(self, key):
# with all other indexing behavior
if isinstance(key, Series) and not key.index.equals(self.index):
warnings.warn("Boolean Series key will be reindexed to match "
"DataFrame index.", UserWarning)
"DataFrame index.", UserWarning, stacklevel=3)
elif len(key) != len(self.index):
raise ValueError('Item wrong length %d instead of %d.' %
(len(key), len(self.index)))
Expand Down
4 changes: 4 additions & 0 deletions pandas/core/indexing.py
Original file line number Diff line number Diff line change
Expand Up @@ -995,6 +995,10 @@ def _getitem_axis(self, key, axis=0):

return self._getitem_iterable(key, axis=axis)
else:

# maybe coerce a float scalar to integer
key = labels._maybe_cast_indexer(key)

if is_integer(key):
if axis == 0 and isinstance(labels, MultiIndex):
try:
Expand Down
Loading