PERF: perf regression with mixed-type ops using numexpr (GH5481)

[jreback]

closes #5481

BUG: non-unique ops not aligning correctly

these are bascially a trivial op in numpy, so numexpr is slightly slower (but the
the dtype inference issue is fixed). Essentially the recreation of an int64 ndarray had to check if its a datetime-like. In this case just passing in the dtype on the reconstructed series fixes it.

Also handles non-unique columns now (no tests before, and it would fail).

In [1]: df = pd.DataFrame({"A": np.arange(1000000), "B": np.arange(1000000, 0, -1), "C": np.random.randn(1000000)})

In [2]: pd.computation.expressions.set_use_numexpr(False)

In [3]: %timeit df*df
100 loops, best of 3: 11 ms per loop

In [4]: pd.computation.expressions.set_use_numexpr(True)

In [5]: %timeit df*df
100 loops, best of 3: 15.7 ms per loop

In [6]: df = df.astype(float)

In [7]: pd.computation.expressions.set_use_numexpr(False)

In [8]: %timeit df*df
100 loops, best of 3: 5.16 ms per loop

In [9]: pd.computation.expressions.set_use_numexpr(True)

In [10]: %timeit df*df
100 loops, best of 3: 5.37 ms per loop

[jreback]

@jtratner , cc @dsm054

pls give a try with this...should fix the perf issue

[dsm054]

Works for me; in fact numexpr is somewhat faster--

In [4]: pd.computation.expressions.set_use_numexpr(False)

In [5]: %timeit df*df
10 loops, best of 3: 28.3 ms per loop

In [6]: pd.computation.expressions.set_use_numexpr(True)

In [7]: %timeit df*df
10 loops, best of 3: 26.9 ms per loop

[jtratner]

Why is this a non-unique index issue? Do you mean that we're special casing unique ops?

[jreback]

no the non unique issue is separate
but was convient to fix

[jtratner]

So the actual fix was just passing the dtype explicitly?

[jreback]

well had to create the series with an explicit dtype (as opposed to passing a dict of ndarrays)

[jtratner]

Got it - makes sense.