BUG: Replace methods fills value from previous row when replacing with None

[DataSolveProblems]

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• (optional) I have confirmed this bug exists on the master branch of pandas.

---

Code Sample, a copy-pastable example

import numpy as np
import pandas as pd

df = pd.DataFrame({'x': [10, 20, np.nan], 'y': [30, 40, 50]})
print(df.replace(np.NaN, None))

#       x   y
# 0  10.0  30
# 1  20.0  40
# 2  20.0  50

Problem description

When replacing NaN with None using replace method, value, NaN value is replaced with value from previous row instead of None.

Expected Output

#       x   y
# 0  10.0  30
# 1  20.0  40
# 2  None  50

Output of pd.show_versions()

INSTALLED VERSIONS
------------------
commit           : None
python           : 3.7.4.final.0
python-bits      : 64
OS               : Windows
OS-release       : 10
machine          : AMD64
processor        : Intel64 Family 6 Model 158 Stepping 13, GenuineIntel
byteorder        : little
LC_ALL           : None
LANG             : en_US.UTF-8
LOCALE           : None.None
pandas           : 1.0.0rc0
numpy            : 1.17.2
pytz             : 2019.2
dateutil         : 2.8.0
pip              : 19.3.1
setuptools       : 40.8.0
Cython           : None
pytest           : None
hypothesis       : None
sphinx           : None
blosc            : None
feather          : None
xlsxwriter       : None
lxml.etree       : None
html5lib         : None
pymysql          : None
psycopg2         : None
jinja2           : None
IPython          : None
pandas_datareader: None
bs4              : None
bottleneck       : None
fastparquet      : None
gcsfs            : None
lxml.etree       : None
matplotlib       : None
numexpr          : None
odfpy            : None
openpyxl         : None
pandas_gbq       : None
pyarrow          : None
pytables         : None
pytest           : None
s3fs             : None
scipy            : None
sqlalchemy       : None
tables           : None
tabulate         : 0.8.6
xarray           : None
xlrd             : None
xlwt             : None
xlsxwriter       : None
numba            : None

[simonjayhawkins]

@DataSolveProblems Thanks for the report. I can confirm that the issue persists on master '1.1.0.dev0+1113.g8415f9ba2'

[simonjayhawkins]

I think the more idiomatic way would be to use DataFrame.fillna here, but that would raise ValueError: Must specify a fill 'value' or 'method'. Maybe a solution to this issue would be to allow None in fillna instead.

[DataSolveProblems]

I was able to workaround by passing a dictionary. Not ideal, but works.

print(df.replace({np.NaN: None}))

[simonjayhawkins]

The issue occurs with other missing values not present in the DataFrame..

>>> import numpy as np
>>>
>>> import pandas as pd
>>>
>>> pd.__version__
'1.1.0.dev0+1113.g8415f9ba2'
>>>
>>> df = pd.DataFrame({"x": [10, 20, np.nan], "y": [30, 40, 50]})
>>> print(df.replace(np.NaN, None))
      x   y
0  10.0  30
1  20.0  40
2  20.0  50
>>>
>>> df.replace(None, None)
      x   y
0  10.0  30
1  20.0  40
2  20.0  50
>>>
>>> df.replace(pd.NA, None)
      x   y
0  10.0  30
1  20.0  40
2  20.0  50
>>>

[simonjayhawkins]

from the examples in https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.replace.html, it looks like this maybe expected behaviour as the default value for method is pad

This issue also looks like a duplicate of #19998, so closing. ping to reopen if I missed something.