BUG/API: pivot_table with multi-index columns only

[chris-b1]

Code Sample, a copy-pastable example if possible

In [21]: df = pd.DataFrame({'k': [1, 2, 3], 'v': [4, 5, 6]})

In [22]: df.pivot_table(values='v', columns='k')
Out[22]: 
k  1  2  3
v  4  5  6

In [23]: df.pivot_table(values='v', index='k')
Out[23]: 
   v
k   
1  4
2  5
3  6

In [24]: df2 = pd.DataFrame({'k1': [1, 2, 3], 'k2': [1, 2, 3], 'v': [4, 5, 6]})

In [25]: df2.pivot_table(values='v', index=('k1','k2'))
Out[25]: 
       v
k1 k2   
1  1   4
2  2   5
3  3   6

In [26]: df2.pivot_table(values='v', columns=('k1','k2'))
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-26-80d7fdeb9743> in <module>()
----> 1 df2.pivot_table(values='v', columns=('k1','k2'))

~\Anaconda\envs\py36\lib\site-packages\pandas\core\reshape\pivot.py in pivot_table(data, values, index, columns, aggfunc, fill_value, margins, dropna, margins_name)
    172     # discard the top level
    173     if values_passed and not values_multi and not table.empty and \
--> 174        (table.columns.nlevels > 1):
    175         table = table[values[0]]
    176 

~\Anaconda\envs\py36\lib\site-packages\pandas\core\generic.py in __getattr__(self, name)
   3075         if (name in self._internal_names_set or name in self._metadata or
   3076                 name in self._accessors):
-> 3077             return object.__getattribute__(self, name)
   3078         else:
   3079             if name in self._info_axis:

AttributeError: 'Series' object has no attribute 'columns'

Expected Output

No error, symmetrical between rows/columns and single/multi case

Output of pd.show_versions()

pandas 0.20.2

[chris-b1]

This did sort of work in previous versions - arguably returning the wrong shape, but didn't error.

In [5]: df = pd.DataFrame({'k1': [1, 2, 3], 'k2': [1, 2, 3], 'v': [4, 5, 6]})

In [6]: df.pivot_table(values='v', columns=('k1','k2'))
Out[6]:
k1  k2
1   1     4
2   2     5
3   3     6
dtype: int64

In [7]: df.pivot_table(values='v', index=('k1','k2'))
Out[7]:
k1  k2
1   1     4
2   2     5
3   3     6
Name: v, dtype: int64

In [8]: pd.__version__
Out[8]: '0.18.1'

[sroodhorst]

Have the same problem after updated from pandas 0.19.2 to 0.20.3. Python 3.5.2

The problem starts when adding more than 1 column to the columns parameter.
pt2 = pd.pivot_table(data=aa, columns=['lr','optimizer','batchsize','imsize','batchnorm','dropout'], values='val_DSC')
returns:

Traceback (most recent call last):
  File "F:\Steven\Software\Anaconda3-4.2.0\lib\site-packages\IPython\core\interactiveshell.py", line 2881, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-6-0e5f95cd9e00>", line 1, in <module>
    pt2 = pd.pivot_table(data=aa,columns=['lr','optimizer','batchsize','imsize','batchnorm','dropout'],values='val_DSC')
  File "F:\Steven\Software\Anaconda3-4.2.0\lib\site-packages\pandas\core\reshape\pivot.py", line 174, in pivot_table
    (table.columns.nlevels > 1):
  File "F:\Steven\Software\Anaconda3-4.2.0\lib\site-packages\pandas\core\generic.py", line 3077, in __getattr__
    return object.__getattribute__(self, name)
AttributeError: 'Series' object has no attribute 'columns'

While:
pt2 = pd.pivot_table(data=aa,columns=['lr'],values='val_DSC')
returns
lr 1E-01 1E-02 1E-03 1E-05 5E-02
val_DSC 0.25 0.5575 0.545593 0.01 0.14

Currently downgrading to get the wanted result back.

[jorisvandenbossche]

@sroodhorst are you sure that is the correct output on 0.19.2? (or can you provide a small reproducible example)
As @chris-b1 noted, this kind of works on 0.19.2, but giving a wrong shape. So even when the bug gets fixed, the result will be different from the one in 0.19.2

[jorisvandenbossche]

By the way, a PR to try to fix this is very welcome!

[BBQHarvey]

I am working on this.

[bryaan]

Will this be fixed soon? Seems like it would affect a lot of code for some.

[jreback]

@bryaan best way to have things fixed is to submit a PR

[bryaan]

@jreback Sure is, I don't mean to complain. Just surprised a widely used lib like pandas would have this issue in a release branch.

[TomAugspurger]

As you can see, we have a lot of open issues. It comes down to who in the community has the time to fix them (do you have time?).

[benjello]

Just seeking info: is this bug still not solved or is there a non released version which fixes it ?

[TomAugspurger]

Still open. Issues are closed when the fix is merged into master, and assigned that release as the milestone.

…

On Mon, Jul 2, 2018 at 3:25 PM, Mahdi Ben Jelloul ***@***.***> wrote: Just seeking info: is this bug still not solved or is there a non released version which fixes it ? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#17038 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABQHIsbRIcpr8I0jM1kEWOwZWSpwQtmwks5uCoHPgaJpZM4OeFpw> .