Skip to content

BUG: None and np.nan behave differently for .replace() #38510

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
2 of 3 tasks
d-kz opened this issue Dec 15, 2020 · 2 comments
Closed
2 of 3 tasks

BUG: None and np.nan behave differently for .replace() #38510

d-kz opened this issue Dec 15, 2020 · 2 comments
Labels
Closing Candidate May be closeable, needs more eyeballs Duplicate Report Duplicate issue or pull request replace replace method Usage Question

Comments

@d-kz
Copy link

d-kz commented Dec 15, 2020

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.
    pip freeze | grep pandas -> pandas==1.1.5

  • (optional) I have confirmed this bug exists on the master branch of pandas.


Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Code Sample, a copy-pastable example

In [9]: df = pd.DataFrame([1, None, np.nan])

In [10]: df
Out[10]:
     0
0  1.0
1  NaN
2  NaN

In [11]: df.replace(1, None)
Out[11]:
     0
0  1.0
1  NaN
2  NaN

In [12]: df.replace(1, np.nan)
Out[12]:
    0
0 NaN
1 NaN
2 NaN

Problem description

This is a problem because when creating a dataframe, both None and np.nan are converted into NaN. Therefore, one expect replacing will also treat them equivalently.

Expected Output

df = pd.DataFrame([1, None, np.nan])
df.replace(1, None)
     0
0  1.0
1  NaN
2  NaN

should return the same as

df = pd.DataFrame([1, None, np.nan])
df.replace(1, np.nan)
    0
0 NaN
1 NaN
2 NaN

Output of pd.show_versions()

[paste the output of pd.show_versions() here leaving a blank line after the details tag]

@d-kz d-kz added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Dec 15, 2020
@simonjayhawkins
Copy link
Member

This look like a duplicate of #36984 and can often cause confusion. see #36984 (comment)

>>> import numpy as np
>>> import pandas as pd
>>>
>>> pd.__version__
'1.3.0.dev0+83.g122d50246b'
>>>
>>> df = pd.DataFrame([1, None, np.nan, 1])
>>>
>>> df.replace(1, None)
     0
0  1.0
1  NaN
2  NaN
3  NaN
>>>
>>> df.replace({1:None})
      0
0  None
1   NaN
2   NaN
3  None
>>>
>>>
>>> df.replace(1, None, method="bfill")
     0
0  NaN
1  NaN
2  NaN
3  1.0
>>>

@simonjayhawkins simonjayhawkins added Closing Candidate May be closeable, needs more eyeballs Duplicate Report Duplicate issue or pull request replace replace method Usage Question and removed Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Dec 16, 2020
@mroeschke
Copy link
Member

Agreed this looks related to #36984 and could be better documented

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Closing Candidate May be closeable, needs more eyeballs Duplicate Report Duplicate issue or pull request replace replace method Usage Question
Projects
None yet
Development

No branches or pull requests

3 participants