multiindex with stats.api.ols: unexpected formatting

[DSLituiev]

With MultiIndex, stats.api.ols returns prediction of a different shape than outcome variable:

"set up the example"
x1 = np.arange(0,4)
x2 = 2*np.random.rand(4)
y = x1+x2 +  .2*np.random.randn(4)

df = pd.DataFrame( np.c_[y,x1, x2] ,
             columns = ["y", "x1","x2"],
             index = pd.MultiIndex.from_tuples([("chr1", 1), ("chr1",2), ("chr2",2), ("chr2", 4)]))
df.index.names = ["chr", "pos"]
df["y"]

"set up the model"
from pandas.stats.api import ols
res = ols(y = df["y"] , x = df[["x1","x2"]])
unexpected_output = res.predict()
unexpected_output

following post-formating recovers expected shape:

cols = unexpected_output.columns
unexpected_output.reset_index()
expected_output = pd.melt(unexpected_output.reset_index(), 
                     id_vars= res.y.index.names[0]  \
                    ).dropna().set_index(res.y.index.names).loc[res.y.index.get_values()]
expected_output

[jreback]

pandas.stats.api is basically deprecated (and will be in the next version)

use statsmodels instead

[jreback]

see #6077