ENH: Allow multi values for index and columns in df.pivot #30928

charlesdong1991 · 2020-01-11T21:01:38Z

closes BUG: DataFrame.pivot fails on multiple columns to set as index #21425
tests added / passed
passes black pandas
passes git diff upstream/master -u -- "*.py" | flake8 --diff
whatsnew entry

pep8speaks · 2020-01-11T21:01:44Z

Hello @charlesdong1991! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2020-02-02 19:24:06 UTC

charlesdong1991 · 2020-01-18T19:36:55Z

would be nice if someone could take a look here, i just rebased ^^

jreback

can you add an example in the docs as well

jreback · 2020-01-18T19:40:04Z

doc/source/whatsnew/v1.0.0.rst

@@ -228,6 +228,7 @@ Other enhancements
 - Added an experimental :attr:`~DataFrame.attrs` for storing global metadata about a dataset (:issue:`29062`)
 - :meth:`Timestamp.fromisocalendar` is now compatible with python 3.8 and above (:issue:`28115`)
 - :meth:`DataFrame.to_pickle` and :func:`read_pickle` now accept URL (:issue:`30163`)
+- :meth:`DataFrame.pivot` can now take lists for ``index`` and ``columns`` arguments (:issue:`21425`)


move to 1.1

jreback · 2020-01-18T19:40:44Z

pandas/core/frame.py

@@ -5847,11 +5847,19 @@ def groupby(

        Parameters
        ----------%s
-        index : str or object, optional
+        index : str or object or a list of the previous, optional


list of str, this really takes object? what exactly does that mean

oh, my bad, i copied the description from values, will update a bit to make it clearer

charlesdong1991 · 2020-01-20T21:21:12Z

just in case if you'd like to take a look too @WillAyd thanks! ^^

WillAyd · 2020-01-20T22:11:59Z

pandas/core/frame.py

            Column to use to make new frame's index. If None, uses
            existing index.
-        columns : str or object
+
+            .. versionchanged:: 1.0.0


Suggested change

.. versionchanged:: 1.0.0

.. versionchanged:: 1.1.0

nice catch!! my bad 😅

WillAyd · 2020-01-20T22:12:06Z

pandas/core/frame.py

            Column to use to make new frame's columns.
+
+            .. versionchanged:: 1.0.0


Suggested change

.. versionchanged:: 1.0.0

.. versionchanged:: 1.1.0

WillAyd · 2020-01-20T22:12:52Z

pandas/core/reshape/pivot.py


    if values is None:
-        cols = [columns] if index is None else [index, columns]
+        cols: List[str] = []


I think you want List[Label] (import from pandas._typing) to allow anything column name, unless this really does have to be string-only

i don't think this will matter until we type the signature, so ok with handling this as a followon.

jreback · 2020-01-20T22:57:50Z

pandas/tests/reshape/test_pivot.py

@@ -1007,6 +1007,192 @@ def test_pivot_no_level_overlap(self):
        expected = grouped.unstack("b").unstack("c").dropna(axis=1, how="all")
        tm.assert_frame_equal(table, expected)

+    @pytest.mark.parametrize(


can you split this out (and other multiindex uses of pivot) to test_pivot_multilevel.py

we can do this in a followon.

thanks! I think a bit, and feel maybe it's better to make this change here to make the scope of PR clearer, so i made a new file test_pivot_multilevel.py and moved related tests there.

jreback · 2020-01-20T22:59:45Z

pandas/core/reshape/pivot.py

@@ -431,17 +431,31 @@ def _convert_by(by):
 def pivot(data: "DataFrame", index=None, columns=None, values=None) -> "DataFrame":
    if columns is None:
        raise TypeError("pivot() missing 1 required argument: 'columns'")
+    columns = columns if is_list_like(columns) else [columns]


asa followon can you try to type tings in the signature

jreback · 2020-01-20T23:00:22Z

pandas/core/reshape/pivot.py


    if values is None:
-        cols = [columns] if index is None else [index, columns]
+        cols: List[str] = []


i don't think this will matter until we type the signature, so ok with handling this as a followon.

charlesdong1991 · 2020-01-21T09:12:30Z

HI, @jreback and @WillAyd thanks for both of yours reviews, appreciate a lot!!

One thing to mention: i'd like to have signature added as a follow-up to make this clearer, since we have PRs addressing TYPING issues only if you agree. Otherwise, I think I have addressed both of your reviews in the change ^^

Feel free for further reviews! Thanks!

charlesdong1991 · 2020-01-23T15:27:39Z

just updated with latest master

charlesdong1991 · 2020-01-24T23:12:56Z

gentle ping @WillAyd for feedback? ^^

WillAyd

lgtm @jreback

charlesdong1991 · 2020-01-26T07:31:52Z

gentle ping @jreback ^^

charlesdong1991 · 2020-02-02T20:14:40Z

sorry for pushing for a review, I know you are so busy with so many PRs. This PR has been rebased several times, so some feedbacks would be appreciated @jreback 👍

jreback · 2020-02-09T21:57:34Z

thanks @charlesdong1991
very nice

charlesdong1991 added 5 commits December 3, 2018 17:43

remove \n from docstring

7e461a1

fix conflicts

1314059

Merge remote-tracking branch 'upstream/master'

8bcb313

Merge remote-tracking branch 'upstream/master' into multi_pivot

7bd097d

Allow multi values for index and columns in pivot

f7b37d6

charlesdong1991 added 7 commits January 11, 2020 22:02

better naming

180194d

fix linting

98e9730

Add whatsnew note

8739957

Update docstring

8a2af11

fix pep8

4cdd17a

fix pep

ced4ec7

fix linting

7f0ea51

gfyoung added Bug Reshaping Concat, Merge/Join, Stack/Unstack, Explode labels Jan 15, 2020

charlesdong1991 added 2 commits January 18, 2020 17:38

rebase and fix conflict

abe991b

Merge remote-tracking branch 'upstream/master' into multi_pivot

d312f56

jreback requested changes Jan 18, 2020

View reviewed changes

charlesdong1991 added 3 commits January 18, 2020 20:49

move from 1.0.0 to 1.1

3ed3a60

update doc and add example

cc1826e

fix pep8

311670b

charlesdong1991 requested a review from jreback January 18, 2020 21:20

WillAyd requested changes Jan 20, 2020

View reviewed changes

jreback requested changes Jan 20, 2020

View reviewed changes

charlesdong1991 added 3 commits January 21, 2020 09:10

Merge remote-tracking branch 'upstream/master' into multi_pivot

61d32b0

code change on reviews

3aa04fa

rename

9f5f170

charlesdong1991 requested review from WillAyd and jreback January 21, 2020 09:12

charlesdong1991 added 2 commits January 21, 2020 13:39

merge master and resolve conflict

ce0e85d

Merge remote-tracking branch 'upstream/master' into multi_pivot

20a54ba

WillAyd approved these changes Jan 24, 2020

View reviewed changes

Merge remote-tracking branch 'upstream/master' into multi_pivot

ab20be2

Merge remote-tracking branch 'upstream/master' into multi_pivot

c70230b

jreback added this to the 1.1 milestone Feb 9, 2020

jreback approved these changes Feb 9, 2020

View reviewed changes

jreback merged commit 11f1500 into pandas-dev:master Feb 9, 2020

This was referenced Feb 23, 2020

TYP: Add annotation for df.pivot #32197

Merged

Feature request Multi Index pivot #32129

Closed

simonjayhawkins mentioned this pull request Aug 18, 2020

BUG: pivot broken on the latest version 1.1.0 #35785

Open

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ENH: Allow multi values for index and columns in df.pivot #30928

ENH: Allow multi values for index and columns in df.pivot #30928

charlesdong1991 commented Jan 11, 2020 •

edited

Loading

pep8speaks commented Jan 11, 2020 •

edited

Loading

charlesdong1991 commented Jan 18, 2020

jreback left a comment

jreback Jan 18, 2020

jreback Jan 18, 2020

charlesdong1991 Jan 18, 2020

charlesdong1991 commented Jan 20, 2020

WillAyd Jan 20, 2020

charlesdong1991 Jan 21, 2020 •

edited

Loading

WillAyd Jan 20, 2020

charlesdong1991 Jan 21, 2020

WillAyd Jan 20, 2020

jreback Jan 20, 2020

jreback Jan 20, 2020

jreback Jan 20, 2020

charlesdong1991 Jan 21, 2020

jreback Jan 20, 2020

jreback Jan 20, 2020

charlesdong1991 commented Jan 21, 2020

charlesdong1991 commented Jan 23, 2020

charlesdong1991 commented Jan 24, 2020

WillAyd left a comment

charlesdong1991 commented Jan 26, 2020

charlesdong1991 commented Feb 2, 2020

jreback commented Feb 9, 2020

		Column to use to make new frame's columns.

		.. versionchanged:: 1.0.0

ENH: Allow multi values for index and columns in df.pivot #30928

ENH: Allow multi values for index and columns in df.pivot #30928

Conversation

charlesdong1991 commented Jan 11, 2020 • edited Loading

pep8speaks commented Jan 11, 2020 • edited Loading

Comment last updated at 2020-02-02 19:24:06 UTC

charlesdong1991 commented Jan 18, 2020

jreback left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

charlesdong1991 commented Jan 20, 2020

Choose a reason for hiding this comment

charlesdong1991 Jan 21, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

charlesdong1991 commented Jan 21, 2020

charlesdong1991 commented Jan 23, 2020

charlesdong1991 commented Jan 24, 2020

WillAyd left a comment

Choose a reason for hiding this comment

charlesdong1991 commented Jan 26, 2020

charlesdong1991 commented Feb 2, 2020

jreback commented Feb 9, 2020

charlesdong1991 commented Jan 11, 2020 •

edited

Loading

pep8speaks commented Jan 11, 2020 •

edited

Loading

charlesdong1991 Jan 21, 2020 •

edited

Loading