BUG: Outer/right merge with EA dtypes cast to object #43152
Conversation
debnathshoham
commented
Aug 21, 2021
debnathshoham
commented
- closes BUG: Outer merge casts EA dtype to object #40073
- tests added / passed
- Ensure all linting tests pass, see here for how to run them
- whatsnew entry
jreback
added
Bug
ExtensionArray
|
https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=64682&view=logs&j=bf526160-cc38-5046-7b49-5141c64a483a&t=6a92c4fe-2b0b-5acd-cfba-e7901042d9bd is failing. maybe restrict the test to only EA types e.g. any_float_dtype and any_int_ea_dtype |
|
Hi @jreback - The failure is somehow in TestMerge.test_handle_join_key_pass_array. And not the ones I added. Surprisingly, its not failing in my local |
Since these are windows and 32-bit failures, would guess it has to do with these platforms defaulting to i32 instead of i64 |
doc/source/whatsnew/v1.3.3.rst
Outdated
| @@ -17,7 +17,7 @@ Fixed regressions | |||
| - Fixed regression in :class:`DataFrame` constructor failing to broadcast for defined :class:`Index` and len one list of :class:`Timestamp` (:issue:`42810`) | |||
| - Performance regression in :meth:`core.window.ewm.ExponentialMovingWindow.mean` (:issue:`42333`) | |||
| - Fixed regression in :meth:`.GroupBy.agg` incorrectly raising in some cases (:issue:`42390`) | |||
| - | |||
| - Fixed regression in :meth:`merge` where columns with ``ExtensionDtype`` was cast to ``object`` in ``left`` and ``outer`` merge (:issue:`40073`) | |||
There was a problem hiding this comment.
Maybe specify that the cast only occurs when those columns were being merged on?
There was a problem hiding this comment.
IIUC what happened here was that left and outer were always broken, so fixing those could be moved down into the bug fix section of the whatsnew.
Then the regression portion could just focus on right
There was a problem hiding this comment.
my bad, this bug was pertaining to right(regression) and outer (had mentioned in the PR subject), also discussed in the bug comments.
I will keep right in fixed regression, and move outer down to bug fix.
If these are defaulting to int32 instead of int64, shouldn't the assertion pass? (the way both defaults to int64 in my local). |
| @@ -364,6 +369,9 @@ def test_merge_join_key_dtype_cast(self): | |||
| df = merge(df1, df2, left_on=lkey, right_on=rkey, how="outer") | |||
| assert df["key_0"].dtype == "int64" | |||
|
|
|||
| @pytest.mark.xfail( | |||
There was a problem hiding this comment.
not able to figure why this test is failing in windows and 32bit in azure pipelines here.
Was just wondering if xfailing is an option..
There was a problem hiding this comment.
note generally, is the comparison failing? construct the initial dataframe with int64 (rather than int) should help.
There was a problem hiding this comment.
yes, that worked. Thanks!
Let me know if this looks fine..
|
@mzeitlin11 if you can give a quick relook |
Co-authored-by: Matthew Zeitlin <[email protected]>
|
thanks @debnathshoham |
|
@meeseeksdev backport 1.3.x |
|
Something went wrong ... Please have a look at my logs. |
…ypes cast to object) (#43389)
