Skip to content

CLN: preempt TypeError for EAs in groupby agg_series #29186

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Oct 24, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion pandas/core/groupby/generic.py
Original file line number Diff line number Diff line change
Expand Up @@ -264,7 +264,12 @@ def aggregate(self, func=None, *args, **kwargs):
return self._python_agg_general(func, *args, **kwargs)
except (AssertionError, TypeError):
raise
except Exception:
except (ValueError, KeyError, AttributeError, IndexError):
# TODO: IndexError can be removed here following GH#29106
# TODO: AttributeError is caused by _index_data hijinx in
# libreduction, can be removed after GH#29160
# TODO: KeyError is raised in _python_agg_general,
# see see test_groupby.test_basic
result = self._aggregate_named(func, *args, **kwargs)

index = Index(sorted(result), name=self.grouper.names[0])
Expand Down
10 changes: 10 additions & 0 deletions pandas/core/groupby/ops.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
is_complex_dtype,
is_datetime64_any_dtype,
is_datetime64tz_dtype,
is_extension_array_dtype,
is_integer_dtype,
is_numeric_dtype,
is_sparse,
Expand Down Expand Up @@ -659,6 +660,12 @@ def _transform(
return result

def agg_series(self, obj, func):
if is_extension_array_dtype(obj.dtype) and obj.dtype.kind != "M":
# _aggregate_series_fast would raise TypeError when
# calling libreduction.Slider
# TODO: is the datetime64tz case supposed to go through here?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know why it is working for datetimes?

return self._aggregate_series_pure_python(obj, func)

try:
return self._aggregate_series_fast(obj, func)
except AssertionError:
Expand All @@ -683,6 +690,8 @@ def agg_series(self, obj, func):
def _aggregate_series_fast(self, obj, func):
func = self._is_builtin_func(func)

# TODO: pre-empt this, also pre-empt get_result raising TypError if we pass a EA
# for EAs backed by ndarray we may have a performant workaround
if obj.index._has_complex_internals:
raise TypeError("Incompatible index for Cython grouper")

Expand Down Expand Up @@ -717,6 +726,7 @@ def _aggregate_series_pure_python(self, obj, func):
result[label] = res

result = lib.maybe_convert_objects(result, try_float=0)
# TODO: try_cast back to EA?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is done in the code calling this (via agg_series):

for name, obj in self._iterate_slices():
try:
result, counts = self.grouper.agg_series(obj, f)
except TypeError:
continue
else:
output[name] = self._try_cast(result, obj, numeric_only=True)

return result, counts


Expand Down