BUG: _flex_binary_moment() problems with different index sets

[seth-p]

There are several issues with _flex_binary_moment().

A. When pairwise=True, it doesn't properly handle two DataFrames with different index sets. In the following example, I believe [6], [7], and [8] should all produce the result in [9].

In [1]: from pandas import DataFrame, expanding_corr

In [2]: df1 = DataFrame([[1,2], [3, 2], [3,4]], columns=['A','B'])

In [3]: df1a = DataFrame([[1,2], [3,4]], columns=['A','B'], index=[0,2])

In [4]: df2 = DataFrame([[5,6], [None,None], [2,1]], columns=['X','Y'])

In [5]: df2a = DataFrame([[5,6], [2,1]], columns=['X','Y'], index=[0,2])

In [6]: expanding_corr(df1, df2, pairwise=True)[2]
Out[6]:
          X         Y
A -1.224745 -1.224745
B -1.224745 -1.224745

In [7]: expanding_corr(df1, df2a, pairwise=True)[2]
Out[7]:
    X   Y
A NaN NaN
B NaN NaN

In [8]: expanding_corr(df1a, df2, pairwise=True)[2]
Out[8]:
    X   Y
A NaN NaN
B NaN NaN

In [9]: expanding_corr(df1a, df2a, pairwise=True)[2]
Out[9]:
   X  Y
A -1 -1
B -1 -1

[seth-p]

My proposed code above depends on #7516 being fixed (due to the line "results.iloc[:, i, j] = results.iloc[:, j, i]").

[jreback]

it is quite inefficient to start with an empty pandas structure and fill it up. The reason dicts are used in the orgiinal is to avoid this problem. the issue in #7516 is separate and indepedent of this. You can do a pull-request to see what has changed in your proposed fix. Which should start with tests which replicate the issue.

[seth-p]

I'm trying to set things up to build pandas on Windows. Will see how it goes...

[jreback]

just found this nice link: http://cowboyprogrammer.org/building-python-wheels-for-windows/

[seth-p]

Yep, was working through it...

On Jun 19, 2014, at 7:03 PM, jreback [email protected] wrote:

just found this nice link: http://cowboyprogrammer.org/building-python-wheels-for-windows/

—
Reply to this email directly or view it on GitHub.

[seth-p]

... am not having much luck getting things to build on Windows. :-( Will try this weekend on another machine.

[seth-p]

I'm going to break this out into two issues: one (this one) having to do with the bug with different index sets; and one (#7542) having to do with the column orders not being preserved.

[seth-p]

Merged into #7512.