Skip to content

BUG: Fix combine_first converts other columns type into floats unexpectedly #20965

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Oct 9, 2018
Merged

BUG: Fix combine_first converts other columns type into floats unexpectedly #20965

merged 4 commits into from
Oct 9, 2018

Conversation

Licht-T
Copy link
Contributor

@Licht-T Licht-T commented May 6, 2018

@codecov
Copy link

codecov bot commented May 6, 2018 •
edited
Loading

Codecov Report

Merging #20965 into master will decrease coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #20965      +/-   ##
==========================================
- Coverage   92.19%   92.19%   -0.01%     
==========================================
  Files         169      169              
  Lines       50873    50879       +6     
==========================================
+ Hits        46904    46908       +4     
- Misses       3969     3971       +2
Flag Coverage Δ
#multiple 90.61% <100%> (-0.01%) ⬇️
#single 42.32% <0%> (-0.01%) ⬇️
Impacted Files Coverage Δ
pandas/core/frame.py 97.11% <100%> (-0.1%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 66c2e5f...df89c71. Read the comment docs.

jreback
jreback requested changes May 7, 2018
View reviewed changes
@jreback jreback added Bug Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate labels May 7, 2018
@Licht-T
Copy link
Contributor Author

Licht-T commented May 7, 2018

Thanks @jreback, fixed!

@jreback
Copy link
Contributor

jreback commented May 8, 2018

can you add a whatsnew note. also rebase on master; I cleaned some older code out of here (shouldn't impact but just to be sure)

@jorisvandenbossche
Copy link
Member

@Licht-T Do you have time to update this?

jreback
jreback requested changes Jun 19, 2018
View reviewed changes
Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you update and add a whatnew (0.23.2) is ok

@jreback
Copy link
Contributor

jreback commented Sep 25, 2018

can you rebase / update?

@gfyoung if not, can you fix the note

@gfyoung gfyoung force-pushed the fix-combine_first-behavior branch from b4753f1 to 361113a Compare September 29, 2018 21:11
@pep8speaks
Copy link

Hello @Licht-T! Thanks for updating the PR.

@gfyoung gfyoung added this to the 0.24.0 milestone Sep 29, 2018
@gfyoung gfyoung force-pushed the fix-combine_first-behavior branch from 361113a to 37053e2 Compare October 6, 2018 22:48
@gfyoung
Copy link
Member

gfyoung commented Oct 6, 2018

@jreback : Comments addressed, and all is rebased and green. PTAL.

jreback
jreback requested changes Oct 7, 2018
View reviewed changes
pandas/core/frame.py Outdated
# try to promote series, which is all NaN, as other_dtype.
new_dtype = other_dtype
try:
series = series.astype(new_dtype)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add copy=False

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

pandas/tests/frame/test_combine_concat.py Outdated
@@ -750,6 +750,16 @@ def test_combine_first_int(self):
tm.assert_frame_equal(res, df1)
assert res['a'].dtype == 'int64'

def test_combine_first_with_asymmetric_other(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you try the same when IsInt is actually a float as well (maybe call it something else)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as an additional test case (you can parametrize)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@gfyoung gfyoung force-pushed the fix-combine_first-behavior branch from 37053e2 to df89c71 Compare October 7, 2018 22:44
@gfyoung
Copy link
Member

gfyoung commented Oct 8, 2018

@jreback : Comments addressed, and all is rebased and green. PTAL.

@jreback jreback merged commit a4482db into pandas-dev:master Oct 9, 2018
@jreback
Copy link
Contributor

jreback commented Oct 9, 2018

thanks @Licht-T (and @gfyoung )!

note that this may close other combine / combine_first issues. maybe have a look if you have a chance.

tm9k1 pushed a commit to tm9k1/pandas that referenced this pull request Nov 19, 2018
@Licht-T @tm9k1
BUG: Fix combine_first converts other columns type into floats unexpe…
7d8fd4b 
…ctedly (pandas-dev#20965)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gfyoung gfyoung gfyoung left review comments

@jreback jreback jreback approved these changes

Assignees
No one assigned
Labels
Bug Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
Projects
None yet
Milestone
0.24.0
Development

Successfully merging this pull request may close these issues.

Combine_first method converts booleans into floats
5 participants
@Licht-T @jreback @jorisvandenbossche @pep8speaks @gfyoung