BUG: Reindex doesn't add NaN when using level=1

[gurukiran07]

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• (optional) I have confirmed this bug exists on the master branch of pandas.

---

Code Sample, a copy-pastable example

index = pd.MultiIndex.from_tuples([('bar', 'one'),
            ('bar', 'two'),
            ('baz', 'one'),
            ('baz', 'two'),
            ('foo', 'one'),
            ('foo', 'two'),
            ('qux', 'one'),
            ('qux', 'two')],
           names=['first', 'second'])
np.random.seed(0)
s = pd.Series(np.random.randn(8), index=index)
first  second
bar    one       1.764052
       two       0.400157
baz    one       0.978738
       two       2.240893
foo    one       1.867558
       two      -0.977278
qux    one       0.950088
       two      -0.151357
dtype: float64

When I reindex by level 1.
Current output:

s.reindex(['two','one','three'],level=1)
first  second
bar    two       0.400157
       one       1.764052
baz    two       2.240893
       one       0.978738
foo    two      -0.977278
       one       1.867558
qux    two      -0.151357
       one       0.950088
dtype: float64

Expected Output

This output can reproduced by doing s.unstack().reindex(['two','one','three'],axis=1).stack(dropna=False) or s.reindex(pd.MultiIndex.from_product([s.index.levels[0],['two','one','three']]))

first  second
bar    two       0.400157
       one       1.764052
       three       NaN
baz    two       2.240893
       one       0.978738
       three        NaN
foo    two      -0.977278
       one       1.867558
       three        NaN
qux    two      -0.151357
       one       0.950088
       three        NaN
dtype: float64

Problem description

reindex is dropping NaN while using it on a MultiIndex over level 1. The normal behaviour of reindex produces NaN for new indices, It should also in the above case.

Output of pd.show_versions()

INSTALLED VERSIONS

commit : None
python : 3.7.4.final.0
python-bits : 64
OS : Windows
OS-release : 10
machine : AMD64
processor : Intel64 Family 6 Model 94 Stepping 3, GenuineIntel
byteorder : little
LC_ALL : None
LANG : None
LOCALE : None.None

pandas : 1.0.3
numpy : 1.18.3
pytz : 2019.3
dateutil : 2.8.1
pip : 20.1.1
setuptools : 46.1.3
Cython : None
pytest : None
hypothesis : None
sphinx : 3.0.4
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.11.2
IPython : 7.13.0
pandas_datareader: None
bs4 : None
bottleneck : None
fastparquet : None
gcsfs : None
lxml.etree : None
matplotlib : 3.2.1
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pytables : None
pytest : None
pyxlsb : None
s3fs : None
scipy : 1.4.1
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : 1.2.0
xlwt : None
xlsxwriter : None
numba : None

[simonjayhawkins]

Thanks @gurukiran07 for the report. This appears to be a duplicate of #25460