Pandas 0.23.4 reindexing multiindexed frame with ffill confusing output

[skatenerd]

Code Sample, a copy-pastable example if possible

# The last assertion fails here
import pandas as pd

def go():
    sparse_index = pd.MultiIndex.from_product([['FOO'], ['a','b'], [0,3,6]])
    full_index = pd.MultiIndex.from_product([['FOO'], ['a','b'], range(9)])

    sparse_frame = pd.DataFrame(index=sparse_index, columns=['support', 'oppose']).apply(makefake, axis=1)
    reindexed_with_gaps = sparse_frame.reindex(full_index)                                                                                                             
    successfull_bfill_reindex = sparse_frame.reindex(full_index, method='bfill')
    failed_ffill_reindex = sparse_frame.reindex(full_index, method='ffill')
    assert reindexed_with_gaps.isna().sum().sum() == 24
    assert successfull_bfill_reindex.isna().sum().sum() == 0
    assert failed_ffill_reindex.isna().sum().sum() == 0

def makefake(row):
    row['support'] = 0.5 
    row['oppose'] = 0.5 
    return row 

Problem description

This issue is very similar to #5569, except in the context of a MultiIndex. The behavior in question appears to have changed between 0.22.0 and 0.23.4`.

The explanation given in #5569 makes sense for regular indexes, but I'm having trouble seeing how it applies in the context of a MultiIndex.

Expected Output

The actual value of failed_ffill_reindex looks like

         support  oppose
FOO a 0      0.5     0.5
      1      NaN     NaN
      2      NaN     NaN
      3      0.5     0.5
      4      NaN     NaN
      5      NaN     NaN
      6      NaN     NaN
      7      NaN     NaN
      8      0.5     0.5
    b 0      0.5     0.5
      1      0.5     0.5
      2      0.5     0.5
      3      0.5     0.5
      4      0.5     0.5
      5      0.5     0.5
      6      0.5     0.5
      7      0.5     0.5
      8      0.5     0.5

I'm not sure how to understand the fact that the later rows successfully take on the 0.5 value, but the earlier rows do not.

Specifically, I'm interested in the second row (at ('Foo', 'a', 1)). I would expect this row to contain non-NaN data.

Is the idea here that a multi-index can't be seen as "monotonically increasing", so I shouldn't be passing any "fill methods" into my reindex call? If that's the case, should we institute some sort of warning or exception?

Output of pd.show_versions()

[paste the output of pd.show_versions() here below this line]
INSTALLED VERSIONS

commit: None
python: 3.6.6.final.0
python-bits: 64
OS: Linux
OS-release: 4.15.0-36-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: C.UTF-8
LANG: C.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.23.4
pytest: None
pip: 18.0
setuptools: 40.0.0
Cython: 0.25.2
numpy: 1.15.2
scipy: 1.1.0
pyarrow: None
xarray: None
IPython: 7.0.1
sphinx: None
patsy: 0.5.0
dateutil: 2.7.3
pytz: 2018.5
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 3.0.0
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: 1.0.1
sqlalchemy: None
pymysql: None
psycopg2: 2.7.5 (dt dec pq3 ext lo64)
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

[jreback]

all issues r fair game

[skatenerd]

@MaricheloGv Just a heads up, this may actually be Intended Behavior