bool DataFrame arith operations error when size is large enough

[dalejung]

import pandas as pd
import numpy as np

N = 1000000
bools = np.random.randn(N, 1) > 0

df = pd.DataFrame(bools)

df * df

/python2.7/site-packages/numexpr/necompiler.pyc in typeCompileAst(ast)
    147                 raise NotImplementedError(
    148                     "couldn't find matching opcode for '%s'"
--> 149                     % (ast.value + '_' + retsig+basesig))
    150         # First just cast constants, then cast variables if necessary:
    151         for i, (have, want)  in enumerate(zip(basesig, sig)):

NotImplementedError: couldn't find matching opcode for 'mul_bbb'

I'm guessing there's a size trigger that throws ops to numexpr since this doesn't fail for smaller frames.

[jtratner]

yes, that's correct. Let me see if I get the same thing.

[cpcloud]

You cannot multiply boolean arrays using numexpr...you can use & instead and | for +

[cpcloud]

numexpr has no op code for it, hence that (i guess somewhat cryptic) error message

[jtratner]

Yeah, but don't we need to account for this?

[cpcloud]

yes....wasn't dismissing it ... was just offering an alternative for @dalejung if he is awake hacking at 1:30 AM EST and needs a quick solution

[cpcloud]

@jtratner can u put this in your arith ops refactor?

[cpcloud]

actually kind of annoying ... since u have to convert bools to ints for arith ops ... good eval test case as well

[cpcloud]

same error, shocker 😄

[jtratner]

yeah - what should we do - just check for bool dtypes in _can_use?

On Sun, Sep 22, 2013 at 1:28 AM, Phillip Cloud [email protected]:

@jtratner https://github.com/jtratner can u put this in your arith ops
refactor?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/4934#issuecomment-24876373
.

[dalejung]

@cpcloud No worries. Using the .values works fine.

[cpcloud]

@jtratner disallow for arith ops allow for &, |, ~

[jreback]

this was closed by #6803 (raises ``NotImplementedError)