Skip to content

BUG: pd.crosstab fails when passed multiple columns, margins True and normalize True #35150

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Aug 19, 2020

Conversation

CloseChoice
Copy link
Member

@CloseChoice CloseChoice commented Jul 6, 2020

@simonjayhawkins simonjayhawkins changed the title correct wrong if statement in crosstab and add tests for #35144 BUG: pd.crosstab fails when passed multiple columns, margins True and normalize True Jul 7, 2020
@simonjayhawkins simonjayhawkins added Bug Reshaping Concat, Merge/Join, Stack/Unstack, Explode labels Jul 7, 2020
Copy link
Member

@simonjayhawkins simonjayhawkins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @CloseChoice for the PR. needs a release note.

@@ -698,3 +698,46 @@ def test_margin_normalize(self):
names=["A", "B"],
)
tm.assert_frame_equal(result, expected)

def test_crosstab_multiple_columns_normalize(self):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it possible to parametrize the tests added in #27663, would the results be the expected transposed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't that result in a lot of overhead defining the parameters (inputs and results)? I actually see no easy way to refactor these usiing @pytest.mark.parametrize.

What do you mean by would the results be the expected transposed??

Copy link
Member

@charlesdong1991 charlesdong1991 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the fix! just a couple nitpicks.

and as @simonjayhawkins said, you need to add a whatsnew note in 1.1.0.rst

Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

merge master and ping on green

@jreback
Copy link
Contributor

jreback commented Aug 7, 2020

can you merge master and will look again

@jreback jreback added this to the 1.2 milestone Aug 7, 2020
@@ -1166,6 +1166,7 @@ Reshaping
- Fixed bug in :func:`melt` where melting MultiIndex columns with ``col_level`` > 0 would raise a ``KeyError`` on ``id_vars`` (:issue:`34129`)
- Bug in :meth:`Series.where` with an empty Series and empty ``cond`` having non-bool dtype (:issue:`34592`)
- Fixed regression where :meth:`DataFrame.apply` would raise ``ValueError`` for elements whth ``S`` dtype (:issue:`34529`)
- Bug in func :meth:`crosstab` when using multiple columns with ``margins=True`` and ``normalize=True`` (:issue:`35144`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you move to 1.2

# index/column and save the column and index margin
if (margins_name not in table.iloc[-1, :].name) | (
if (margins_name not in table.iloc[-1, :].name) & (
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you pull out
table_name = table.iloc[-1, :].name and use that as a comparison

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@jreback jreback removed this from the 1.2 milestone Aug 13, 2020
@CloseChoice
Copy link
Member Author

CloseChoice commented Aug 19, 2020

can you merge master and will look again

Done. @jreback I'll go on vacation on Friday would be great if we can get this done before that. Otherwise I'll finish this at the beginning of September.

@jreback jreback added this to the 1.2 milestone Aug 19, 2020
@jreback
Copy link
Contributor

jreback commented Aug 19, 2020

thanks @CloseChoice lgtm.

@jreback jreback merged commit a1ac051 into pandas-dev:master Aug 19, 2020
@CloseChoice CloseChoice deleted the fix_normalize_crosstabbing branch August 20, 2020 00:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: pd.crosstab fails when passed multiple columns, margins True and normalize True
4 participants