BUG: GH12824 fixed apply() returns different result depending on whet… #12977

adneu · 2016-04-25T07:36:38Z

closes groupby().apply() returns different result depends on the first result is None or not. #12824
tests added / passed
passes git diff upstream/master | flake8 --diff
whatsnew entry

In _wrap_applied_output for NDFrameGroupBy, if the first result of apply is None, find the first non-None result to determine behavior

jreback · 2016-04-25T12:54:33Z

pandas/core/groupby.py

-        if isinstance(values[0], DataFrame):
+        # GH12824.
+        v = values[0]
+        if v is None:


make this a helper function in core/groupby.py (and consolidate the other one which has the same code)

jreback · 2016-05-07T19:13:43Z

can you rebase / update

adneu · 2016-05-09T09:59:35Z

Rebased and made the changes you suggested but there was a new error with the test. It is introduced by #12743 which now throws an error in _concat_objects when any of the values are None (see below). The new PR makes a small change to deal with this, can you check if ok?

======================================================================
ERROR: test_groupby_apply_none_first (pandas.tests.test_groupby.TestGroupBy)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../pandas/tests/test_groupby.py", line 6292, in test_groupby_apply_none_first
    result = test_df.groupby('groups').apply(test_func)
  File ".../pandas/core/groupby.py", line 651, in apply
    return self._python_apply_general(f)
  File ".../pandas/core/groupby.py", line 660, in _python_apply_general
    not_indexed_same=mutated or self.mutated)
  File ".../pandas/core/groupby.py", line 3241, in _wrap_applied_output
    not_indexed_same=not_indexed_same)
  File ".../pandas/core/groupby.py", line 824, in _concat_objects
    values = reset_identity(values)
  File ".../pandas/core/groupby.py", line 809, in reset_identity
    ax = v._get_axis(self.axis)
AttributeError: 'NoneType' object has no attribute '_get_axis'

----------------------------------------------------------------------

jreback · 2016-05-09T12:37:50Z

pandas/core/groupby.py

@@ -3256,12 +3271,7 @@ def _wrap_applied_output(self, keys, values, not_indexed_same=False):

            # make Nones an empty object
            if com._count_not_none(*values) != len(values):


I think we can remove the _count_not_none(....)

codecov-io · 2016-05-09T12:38:59Z

Current coverage is 84.12%

Merging #12977 into master will increase coverage by <.01%

@@             master     #12977   diff @@
==========================================
  Files           138        138          
  Lines         50388      50390     +2   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits          42386      42388     +2   
  Misses         8002       8002          
  Partials          0          0

4 files (not in diff) in pandas were modified. more
- Hits -4

Powered by Codecov. Last updated by 4e4a7d9...c630c8b

jreback · 2016-05-16T12:18:23Z

can you squash

…her first result is None BUG: GH12824 fixed apply() returns different result depending on whether first result is None BUG: GH12824 rebased and made requested fixes BUG: GH12824 made requested change and added tests

jreback · 2016-05-20T14:09:46Z

thanks @adneu

jreback added Bug Groupby Reshaping Concat, Merge/Join, Stack/Unstack, Explode labels Apr 25, 2016

jreback reviewed Apr 25, 2016
View reviewed changes

jreback reviewed May 9, 2016
View reviewed changes

jreback added this to the 0.18.2 milestone May 16, 2016

jreback closed this in cc25040 May 20, 2016

adneu deleted the GH12824 branch May 21, 2016 03:43

fjetter mentioned this pull request Feb 3, 2020

Groupby apply result shape depends on internal apply path #31612

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BUG: GH12824 fixed apply() returns different result depending on whet… #12977

BUG: GH12824 fixed apply() returns different result depending on whet… #12977

adneu commented Apr 25, 2016

jreback Apr 25, 2016

jreback commented May 7, 2016

adneu commented May 9, 2016

jreback May 9, 2016

codecov-io commented May 9, 2016 •

edited

Loading

jreback commented May 16, 2016

jreback commented May 20, 2016

		@@ -3256,12 +3271,7 @@ def _wrap_applied_output(self, keys, values, not_indexed_same=False):

		# make Nones an empty object
		if com._count_not_none(*values) != len(values):

BUG: GH12824 fixed apply() returns different result depending on whet… #12977

BUG: GH12824 fixed apply() returns different result depending on whet… #12977

Conversation

adneu commented Apr 25, 2016

jreback Apr 25, 2016

Choose a reason for hiding this comment

jreback commented May 7, 2016

adneu commented May 9, 2016

jreback May 9, 2016

Choose a reason for hiding this comment

codecov-io commented May 9, 2016 • edited Loading

Current coverage is 84.12%

jreback commented May 16, 2016

jreback commented May 20, 2016

codecov-io commented May 9, 2016 •

edited

Loading