Skip to content

.isin implicitly converts data types #19356

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
caerner opened this issue Jan 23, 2018 · 7 comments · Fixed by #37770
Closed

.isin implicitly converts data types #19356

caerner opened this issue Jan 23, 2018 · 7 comments · Fixed by #37770
Labels
Bug Dtype Conversions Unexpected or buggy dtype conversions Indexing Related to indexing on series/frames, not to indexes themselves isin isin method
Milestone

Comments

@caerner
Copy link

caerner commented Jan 23, 2018

Code Sample, a copy-pastable example if possible

import pandas as pd
pd.__version__
Out[1]: '0.22.0'

data = pd.Series([-9.0, 0.0])
data.isin([-9, -0.5])

Out[2]: 
0     True
1    False
dtype: bool

data = pd.Series([-9, 0])
data.isin([-9, -0.5])

Out[3]: 
0    True
1    True
dtype: bool

Problem description

.isin seems to implicitly convert the data type of what is passed in values depending on the data type of the Series. In the above example, I would expect the output to be (True, False) in both cases. There seems to have been a change in behavior after pandas version 0.19.2. With version 0.19.2 the output is in both cases (True, False).

Expected Output

import pandas as pd
pd.__version__
Out[1]: '0.19.2'

data = pd.Series([-9.0, 0.0])
data.isin([-9, -0.5])

Out[2]: 
0     True
1    False
dtype: bool

data = pd.Series([-9, 0])
data.isin([-9, -0.5])

Out[3]: 
0    True
1    False
dtype: bool
@jreback
Copy link
Contributor

jreback commented Jan 23, 2018

cross data type conversion is hard. Its somewhere: https://github.com/pandas-dev/pandas/blob/master/pandas/core/algorithms.py#L66. pull requests welcome.

@jreback jreback added Bug Indexing Related to indexing on series/frames, not to indexes themselves Dtype Conversions Unexpected or buggy dtype conversions Difficulty Intermediate labels Jan 23, 2018
@jreback jreback added this to the Next Major Release milestone Jan 23, 2018
@jdoepfert
Copy link
Contributor

Will take a look today

@jreback
Copy link
Contributor

jreback commented Nov 15, 2020

this issue seems to overlap with: #21804

cc @phofl @avinashpancham

@avinashpancham
Copy link
Contributor

I see @phofl started earlier with the PR, so I dont mind leaving this to him.

@phofl
Copy link
Member

phofl commented Nov 15, 2020

Thx, I will look into this and give feedback if mine fixes both

@avinashpancham
Copy link
Contributor

Issues look similar and (our proposed modifications too!), so I guess it will fix both :)

@phofl
Copy link
Member

phofl commented Nov 15, 2020

It is exactly the same :)

@jreback jreback modified the milestones: 1.2, Contributions Welcome Nov 19, 2020
@jreback jreback removed this from the Contributions Welcome milestone Dec 29, 2020
@jreback jreback added this to the 1.3 milestone Dec 29, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Dtype Conversions Unexpected or buggy dtype conversions Indexing Related to indexing on series/frames, not to indexes themselves isin isin method
Projects
None yet
6 participants