Skip to content

BUG: pivot_table with overlapping values #61293

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 22 commits into from
Apr 23, 2025

Conversation

it176131
Copy link
Contributor

@it176131 it176131 commented Apr 15, 2025

@it176131
Copy link
Contributor Author

Should I update the doc/source/whatsnew/v2.3.0.rst or doc/source/whatsnew/v3.0.0.rst file?

@rhshadrach
Copy link
Member

rhshadrach commented Apr 16, 2025

@it176131 - update v3.0.0. Also due to an erroneous commit with a large file, we've had to force-push to the main branch recently. You'll need to fix this up by removing the erroneous commit from your history as well. See the unexpected changes and file in the diff - those need to be reverted.

@datapythonista - if we're going to force-push to a contributors branch, I think a comment detailing what's going on would be helpful.

@rhshadrach rhshadrach added Bug Reshaping Concat, Merge/Join, Stack/Unstack, Explode labels Apr 16, 2025
@rhshadrach rhshadrach changed the title Bug pivot table BUG: pivot_table with overlapping values Apr 16, 2025
@it176131 it176131 force-pushed the bug-pivot-table branch 2 times, most recently from 1509123 to 32c0b78 Compare April 16, 2025 15:32
@it176131
Copy link
Contributor Author

@rhshadrach I believe I've reverted the changes as requested. Please lmk if I did it incorrectly.

@datapythonista
Copy link
Member

Oh, so GitHub added the image we removed to the open PRs? That explains why it was in that PR...

@it176131
Copy link
Contributor Author

@rhshadrach this check appears to have failed due to a timeout—is there a way to restart it?

Copy link
Member

@rhshadrach rhshadrach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good! Mostly nit-picks below. I'd ignore the free-threading workflow for now.

it176131 and others added 17 commits April 19, 2025 13:42
	- Added two tests, :func:`test_pivot_table_values_in_columns` and :func:`test_pivot_table_values_in_index`, to ensure that the `values` param is still used when the argument is shared between the `columns` and `values` params, and `index` and `values` params.
	- Added condition to :func:`__internal_pivot_table` to aggregate `values` explicitly if `values` were passed, otherwise aggregate all remaining columns. This allows the tests :func:`test_pivot_table_values_in_columns` and :func:`test_pivot_table_values_in_index` in test_pivot_multilevel.py to pass.
	- Added test :func:`test_pivot_table_values_as_two_params` to test that the updates in pivot.py result in expected results, satisfying GH issue pandas-dev#57876.
	- Added GH issue comment to test :func:`test_pivot_table_values_as_two_params`.
	- Combined tests :func:`test_pivot_table_values_in_columns` and :func:`test_pivot_table_values_in_index` into a single parametrized test, :func:`test_pivot_table_multiindex_values_as_two_params` to reduce duplicate setup code.
	- Added GH issue pandas-dev#61292 as comment to test :func:`test_pivot_table_multiindex_values_as_two_params`.
	- Simplified proposed logic in :func:`__internal_pivot_table`.
	- Added GH issue numbers to new logic in :func:`__internal_pivot_table`.
	- Added ignore-comment to silence mypy error in :func:`__internal_pivot_table`.
	- Added TODO-comment stating that the :meth:`DataFrameGroupBy.__getitem__` should be overloaded to match the pandas-stubs type declarations, informing mypy that the type is correct given `values` is a list.
	- Added pivot_table bug to Bugs/Reshaping section referencing issues pandas-dev#57876 and pandas-dev#61292.
	- Moved and simplified mypy comment per feedback.
	- Removed comment about explicit aggregation per feedback.
	- Removed param names and updated `argnames` arg per feedback in parametrized marker.
	- Removed param names in favor of implicit args per feedback.
	- Removed param names and updated arg for `argnames` in parametrized marker per feedback.
	- Reduced `expected` assignments from two to one per feedback.
	- Removed param names in favor of implicit args per feedback.
@it176131 it176131 requested a review from rhshadrach April 19, 2025 18:42
Copy link
Member

@rhshadrach rhshadrach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@rhshadrach rhshadrach added this to the 3.0 milestone Apr 22, 2025
it176131 and others added 3 commits April 23, 2025 12:44
	- Moved e_data, e_index, and e_cols to parametrized marker instead of declaring inside the test :func:`test_pivot_table_multiindex_values_as_two_params`.
	- Moved `expected` setup to parametrized marker instead of in the test :meth:`TestPivotTable.test_pivot_table_values_as_two_params`.
@it176131 it176131 requested a review from mroeschke April 23, 2025 18:13
	- Removed walrus operator declarations in parametrized marker for test :meth:TestPivotTable.test_pivot_table_values_as_two_params`. Appears related to this mypy issue -> python/mypy#17377.
	- Removed walrus operator declarations as I'm sure mypy would raise an issue with it given that it did in test_pivot.py (see commit 52cf560).
@mroeschke mroeschke merged commit 20d5b1c into pandas-dev:main Apr 23, 2025
42 checks passed
@mroeschke
Copy link
Member

Thanks @it176131

@it176131 it176131 deleted the bug-pivot-table branch April 23, 2025 22:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
4 participants