Query errors in Windows

[jlopezpena]

We are observing some inconsistent behaviour of the dataframe query method under windows.

I enclose a smallish (3600 rows) dataframe where we find the issue.

The following code fails randomly under Windows, but seems to work smoothly on Mac:

import pandas as pd

df = pd.read_csv("pd_bug.txt")

for x in df.A.unique():
    try:
        assert len(df.query('(A == @x) | (B == @x)')) == len(df[(df.A == x) | (df.B == x)])
    except AssertionError:
        print x

We are observing a failure every other run (often with two or three values failing).
Here is the output of pd.show_versions() on the failing machine:

INSTALLED VERSIONS
------------------
commit: None
python: 2.7.11.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 94 Stepping 3, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None

pandas: 0.17.1
nose: 1.3.7
pip: 7.1.2
setuptools: 19.1.1
Cython: 0.23.4
numpy: 1.10.1
scipy: 0.16.0
statsmodels: 0.6.1
IPython: 4.0.1
sphinx: 1.3.1
patsy: 0.4.0
dateutil: 2.4.2
pytz: 2015.7
blosc: None
bottleneck: 1.0.0
tables: 3.2.2
numexpr: 2.4.4
matplotlib: 1.5.0
openpyxl: 2.2.6
xlrd: 0.9.4
xlwt: 1.0.0
xlsxwriter: 0.7.7
lxml: 3.4.4
bs4: 4.4.1
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: 1.0.9
pymysql: None
psycopg2: None
Jinja2: None

pd_bug.txt

[sinhrks]

Thanks for the report.

I've got a similar report personally which seems to be caused by a local variable using @. Unfortunately I couldn't reproduce the issue (I've used Mac at that time).

Is your problem only caused by queries includes @?

[jlopezpena]

We have only observed it on queries using @, since the failure happens at random values it is very hard to test whether it fails without using the @ operator.

[jreback]

this is numexpr 2.4.4 which is a bit buggy

see #11743

update to 2.4.6 and all will be well

[jlopezpena]

According to your comment on #11743 shouldn't the indexing issue only happen for >500K elements?
My example here is much smaller.

Anyway, if if is a numexpr issue, I cannot fix it in the offending machine until 2.4.6 gets pushed to conda (the pip version requires a different numpy that fails pip-installing on windows).

[jreback]

you can remove numexpr and it wlll work

[jreback]

note that numexpr was just updated in conda to 2.4.6

[jlopezpena]

Just noticed it a few minutes ago. Will test with the upgraded version and check that everything works. Thanks!