Skip to content

BUG: values argument ignored when also supplied to index/columns in pivot_table #61292

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
3 tasks done
it176131 opened this issue Apr 15, 2025 · 0 comments · May be fixed by #61293
Open
3 tasks done

BUG: values argument ignored when also supplied to index/columns in pivot_table #61292

it176131 opened this issue Apr 15, 2025 · 0 comments · May be fixed by #61293
Labels
Bug Reshaping Concat, Merge/Join, Stack/Unstack, Explode

Comments

@it176131
Copy link
Contributor

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import numpy as np
import pandas as pd
from pandas import Index, MultiIndex
import pandas.testing as tm


def test_pivot_table_values_in_columns():
    """``values`` arg is shared between ``values`` and ``columns``."""
    data = [
        ["A", 1, 50, -1],
        ["B", 1, 100, -2],
        ["A", 2, 100, -2],
        ["B", 2, 200, -4],
    ]
    df = pd.DataFrame(data=data, columns=["index", "col", "value", "extra"])
    result = df.pivot_table(values="value", index="index", columns=["col", "value"])
    nan = np.nan
    e_data = [
        [50.0, nan, 100.0, nan],
        [nan, 100.0, nan, 200.0],
    ]
    e_index = Index(data=["A", "B"], name="index")
    e_cols = MultiIndex.from_arrays(
        arrays=[[1, 1, 2, 2], [50, 100, 100, 200]], names=["col", "value"]
    )
    expected = pd.DataFrame(data=e_data, index=e_index, columns=e_cols)
    tm.assert_frame_equal(left=result, right=expected)


def test_pivot_table_values_in_index():
    """``values`` arg is shared between ``values`` and ``index``."""
    data = [
        ["A", 1, 50, -1],
        ["B", 1, 100, -2],
        ["A", 2, 100, -2],
        ["B", 2, 200, -4],
    ]
    df = pd.DataFrame(data=data, columns=["index", "col", "value", "extra"])
    result = df.pivot_table(values="value", index=["index", "value"], columns="col")
    nan = np.nan
    e_data = [
        [50.0, nan],
        [nan, 100.0],
        [100.0, nan],
        [nan, 200.0],
    ]
    e_index = MultiIndex.from_arrays(
        arrays=[["A", "A", "B", "B"], [50, 100, 100, 200]], names=["index", "value"]
    )
    e_cols = Index(data=[1, 2], name="col")
    expected = pd.DataFrame(data=e_data, index=e_index, columns=e_cols)
    tm.assert_frame_equal(left=result, right=expected)


test_pivot_table_values_in_columns()  # Fails.
test_pivot_table_values_in_index()  # Fails.

Issue Description

When the column supplied to values in pandas.DataFrame.pivot_table is also supplied to index or columns, the resulting DataFrame does not contain the aggregations of the values argument. If any extra column(s) are present, those columns are aggregated instead of those supplied to values. This is similar to issue #57876, but the additional columns result in a non-empty DataFrame.

Expected Behavior

I would expect the two tests above to pass, i.e., the values arg is aggregated instead of the non-supplied "extra" column.

# Expected output of ``test_pivot_table_values_in_columns``:
col       1             2       
value   50     100    100    200
index                           
A      50.0    NaN  100.0    NaN
B       NaN  100.0    NaN  200.0
# Expected output of ``test_pivot_table_values_in_index``:
col              1      2
index value              
A     50      50.0    NaN
      100      NaN  100.0
B     100    100.0    NaN
      200      NaN  200.0

Installed Versions

INSTALLED VERSIONS

commit : 0691c5c
python : 3.13.3
python-bits : 64
OS : Windows
OS-release : 11
Version : 10.0.22631
machine : AMD64
processor : AMD64 Family 25 Model 116 Stepping 1, AuthenticAMD
byteorder : little
LC_ALL : None
LANG : None
LOCALE : English_United States.1252

pandas : 2.2.3
numpy : 2.2.4
pytz : 2025.2
dateutil : 2.9.0.post0
pip : 25.0.1
Cython : None
sphinx : None
IPython : 9.1.0
adbc-driver-postgresql: None
adbc-driver-sqlite : None
bs4 : 4.13.3
blosc : None
bottleneck : None
dataframe-api-compat : None
fastparquet : None
fsspec : None
html5lib : None
hypothesis : None
gcsfs : None
jinja2 : 3.1.6
lxml.etree : None
matplotlib : None
numba : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
psycopg2 : None
pymysql : None
pyarrow : None
pyreadstat : None
pytest : 8.3.5
python-calamine : None
pyxlsb : None
s3fs : None
scipy : 1.15.2
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlsxwriter : None
zstandard : None
tzdata : 2025.2
qtpy : None
pyqt5 : None

@it176131 it176131 added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Apr 15, 2025
it176131 added a commit to it176131/pandas that referenced this issue Apr 15, 2025
	- Added GH issue pandas-dev#61292 as comment to test :func:`test_pivot_table_multiindex_values_as_two_params`.
@it176131 it176131 linked a pull request Apr 15, 2025 that will close this issue
6 tasks
@rhshadrach rhshadrach added Reshaping Concat, Merge/Join, Stack/Unstack, Explode and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Apr 16, 2025
it176131 added a commit to it176131/pandas that referenced this issue Apr 16, 2025
	- Added GH issue pandas-dev#61292 as comment to test :func:`test_pivot_table_multiindex_values_as_two_params`.
it176131 added a commit to it176131/pandas that referenced this issue Apr 16, 2025
	- Added GH issue pandas-dev#61292 as comment to test :func:`test_pivot_table_multiindex_values_as_two_params`.
it176131 added a commit to it176131/pandas that referenced this issue Apr 16, 2025
	- Added pivot_table bug to Bugs/Reshaping section referencing issues pandas-dev#57876 and pandas-dev#61292.
it176131 added a commit to it176131/pandas that referenced this issue Apr 19, 2025
	- Added GH issue pandas-dev#61292 as comment to test :func:`test_pivot_table_multiindex_values_as_two_params`.
it176131 added a commit to it176131/pandas that referenced this issue Apr 19, 2025
	- Added pivot_table bug to Bugs/Reshaping section referencing issues pandas-dev#57876 and pandas-dev#61292.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants