BUG: hash_pandas_object ignores optional arguments when the input is a DataFrame. #42049
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mzeitlin11
reviewed
Jun 16, 2021
pandas/tests/util/test_hashing.py
Outdated
| @@ -255,6 +255,16 @@ def test_hash_keys(): | |||
| assert (a != b).all() | |||
|
|
|||
|
|
|||
| def test_df_hash_keys(): | |||
| # DataFrame version of the test_hash_keys | |||
There was a problem hiding this comment.
Can you please link the issue here?
There was a problem hiding this comment.
@mzeitlin11 Thank you for your comment. I added a link to the issue.
| @@ -139,7 +139,10 @@ def hash_pandas_object( | |||
| ser = Series(h, index=obj.index, dtype="uint64", copy=False) | |||
|
|
|||
| elif isinstance(obj, ABCDataFrame): | |||
| hashes = (hash_array(series._values) for _, series in obj.items()) | |||
| hashes = ( | |||
| hash_array(series._values, encoding, hash_key, categorize) | |||
There was a problem hiding this comment.
Since encoding also wasn't passed before, can you add a test that ensures we respect encoding too?
There was a problem hiding this comment.
I added a test_df_encoding to confirm that hash values can change when the different encodings are specified.
|
lgtm @i-aki-y can you add a release note bug fix section (Other is fine) for 1.3 ping on green. |
|
@jreback Thanks, I have added a whatsnew entry. |
|
thanks @i-aki-y |
|
@meeseeksdev backport 1.3.x |
meeseeksmachine
pushed a commit
to meeseeksmachine/pandas
that referenced
this pull request
Jun 18, 2021
…l arguments when the input is a DataFrame.
|
Something went wrong ... Please have a look at my logs. |
simonjayhawkins
pushed a commit
that referenced
this pull request
Jun 18, 2021
…s when the input is a DataFrame. (#42108) Co-authored-by: i-aki-y <[email protected]>
JulianWgs
pushed a commit
to JulianWgs/pandas
that referenced
this pull request
Jul 3, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In this PR, I will try to solve issue #41404.
I would appreciate it if you could review this.
About the issue
In the current
pandas.util.hash_pandas_objectfunction implementation, the hash_key argument is ignored when the input object type is a DataFrame.This is a reproducible example.
The output is here.
The last output should be
About the fix
When the input object is a DataFrame, the hash values are calculated by applying
hash_arrayfunction for each Series.However, optional arguments, including
hash_key, are not passed to the hash_array function in the current implementation.To fix this, I passed the optional arguments to the hash_array function.