Skip to content

DEPR: deprecate returning a tuple from a callable in iloc indexing #53769

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Nov 7, 2023
18 changes: 18 additions & 0 deletions doc/source/user_guide/indexing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,8 @@ of multi-axis indexing.
* A boolean array (any ``NA`` values will be treated as ``False``).
* A ``callable`` function with one argument (the calling Series or DataFrame) and
that returns valid output for indexing (one of the above).
* A tuple of row (and column) indices whose elements are one of the
above inputs.

See more at :ref:`Selection by Label <indexing.label>`.

Expand All @@ -78,13 +80,21 @@ of multi-axis indexing.
* A boolean array (any ``NA`` values will be treated as ``False``).
* A ``callable`` function with one argument (the calling Series or DataFrame) and
that returns valid output for indexing (one of the above).
* A tuple of row (and column) indices whose elements are one of the
above inputs.

See more at :ref:`Selection by Position <indexing.integer>`,
:ref:`Advanced Indexing <advanced>` and :ref:`Advanced
Hierarchical <advanced.advanced_hierarchical>`.

* ``.loc``, ``.iloc``, and also ``[]`` indexing can accept a ``callable`` as indexer. See more at :ref:`Selection By Callable <indexing.callable>`.

.. note::

Destructuring tuple keys into row (and column) indexes occurs
*before* callables are applied, so you cannot return a tuple from
a callable to index both rows and columns.

Getting values from an object with multi-axes selection uses the following
notation (using ``.loc`` as an example, but the following applies to ``.iloc`` as
well). Any of the axes accessors may be the null slice ``:``. Axes left out of
Expand Down Expand Up @@ -450,6 +460,8 @@ The ``.iloc`` attribute is the primary access method. The following are valid in
* A slice object with ints ``1:7``.
* A boolean array.
* A ``callable``, see :ref:`Selection By Callable <indexing.callable>`.
* A tuple of row (and column) indexes, whose elements are one of the
above types.

.. ipython:: python

Expand Down Expand Up @@ -553,6 +565,12 @@ Selection by callable
``.loc``, ``.iloc``, and also ``[]`` indexing can accept a ``callable`` as indexer.
The ``callable`` must be a function with one argument (the calling Series or DataFrame) that returns valid output for indexing.

.. note::

For ``.iloc`` indexing, returning a tuple from the callable is
not supported, since tuple destructuring for row and column indexes
occurs *before* applying callables.

.. ipython:: python

df1 = pd.DataFrame(np.random.randn(6, 4),
Expand Down
26 changes: 24 additions & 2 deletions pandas/core/indexing.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
import sys
from typing import (
TYPE_CHECKING,
Any,
TypeVar,
cast,
final,
)
Expand All @@ -25,6 +27,7 @@
_chained_assignment_msg,
)
from pandas.util._decorators import doc
from pandas.util._exceptions import find_stack_level

from pandas.core.dtypes.cast import (
can_hold_element,
Expand Down Expand Up @@ -90,6 +93,7 @@
Series,
)

T = TypeVar("T")
# "null slice"
_NS = slice(None, None)
_one_ellipsis_message = "indexer may only contain one '...' entry"
Expand Down Expand Up @@ -153,6 +157,10 @@ def iloc(self) -> _iLocIndexer:
"""
Purely integer-location based indexing for selection by position.

.. deprecated:: 2.2.0

Returning a tuple from a callable is deprecated.

``.iloc[]`` is primarily integer position based (from ``0`` to
``length-1`` of the axis), but may also be used with a boolean
array.
Expand All @@ -166,7 +174,8 @@ def iloc(self) -> _iLocIndexer:
- A ``callable`` function with one argument (the calling Series or
DataFrame) and that returns valid output for indexing (one of the above).
This is useful in method chains, when you don't have a reference to the
calling object, but would like to base your selection on some value.
calling object, but would like to base your selection on
some value.
- A tuple of row and column indexes. The tuple elements consist of one of the
above inputs, e.g. ``(0, 1)``.

Expand Down Expand Up @@ -878,7 +887,8 @@ def __setitem__(self, key, value) -> None:
key = tuple(list(x) if is_iterator(x) else x for x in key)
key = tuple(com.apply_if_callable(x, self.obj) for x in key)
else:
key = com.apply_if_callable(key, self.obj)
maybe_callable = com.apply_if_callable(key, self.obj)
key = self._check_deprecated_callable_usage(key, maybe_callable)
indexer = self._get_setitem_indexer(key)
self._has_valid_setitem_indexer(key)

Expand Down Expand Up @@ -1137,6 +1147,17 @@ def _contains_slice(x: object) -> bool:
def _convert_to_indexer(self, key, axis: AxisInt):
raise AbstractMethodError(self)

def _check_deprecated_callable_usage(self, key: Any, maybe_callable: T) -> T:
# GH53533
if self.name == "iloc" and callable(key) and isinstance(maybe_callable, tuple):
warnings.warn(
"Returning a tuple from a callable with iloc "
"is deprecated and will be removed in a future version",
FutureWarning,
stacklevel=find_stack_level(),
)
return maybe_callable

@final
def __getitem__(self, key):
check_dict_or_set_indexers(key)
Expand All @@ -1151,6 +1172,7 @@ def __getitem__(self, key):
axis = self.axis or 0

maybe_callable = com.apply_if_callable(key, self.obj)
maybe_callable = self._check_deprecated_callable_usage(key, maybe_callable)
return self._getitem_axis(maybe_callable, axis=axis)

def _is_scalar_access(self, key: tuple):
Expand Down
9 changes: 9 additions & 0 deletions pandas/tests/frame/indexing/test_indexing.py
Original file line number Diff line number Diff line change
Expand Up @@ -1017,6 +1017,15 @@ def test_single_element_ix_dont_upcast(self, float_frame):
result = df.loc[[0], "b"]
tm.assert_series_equal(result, expected)

def test_iloc_callable_tuple_return_value(self):
# GH53769
df = DataFrame(np.arange(40).reshape(10, 4), index=range(0, 20, 2))
msg = "callable in iLocation indexing is deprecated"
with tm.assert_produces_warning(FutureWarning, match=msg):
df.iloc[lambda _: (0,)]
with tm.assert_produces_warning(FutureWarning, match=msg):
df.iloc[lambda _: (0,)] = 1

def test_iloc_row(self):
df = DataFrame(
np.random.default_rng(2).standard_normal((10, 4)), index=range(0, 20, 2)
Expand Down