Skip to content

df.eval fails for Unary operation involving float32 column #19697

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
vmuriart opened this issue Feb 14, 2018 · 2 comments
Closed

df.eval fails for Unary operation involving float32 column #19697

vmuriart opened this issue Feb 14, 2018 · 2 comments
Labels
Duplicate Report Duplicate issue or pull request

Comments

@vmuriart
Copy link

vmuriart commented Feb 14, 2018

Code Sample, a copy-pastable example if possible

import pandas as pd

df = pd.DataFrame({
    'Foo64': [1.2, 2.4, 7.24],
})

df['Foo32'] = df['Foo64'].astype('float32')

df.eval('-1 + Foo64')   # Works
df.eval('-1 + Foo32')   # Throws Exception
Traceback (most recent call last):
  File "C:\Users\vmuriart\Desktop\bug_eval.py", line 10, in <module>
    df.eval('-1 + Foo32')  # Throws Exception
  File "C:\Anaconda\lib\site-packages\pandas\core\frame.py", line 2186, in eval
    return _eval(expr, inplace=inplace, **kwargs)
  File "C:\Anaconda\lib\site-packages\pandas\core\computation\eval.py", line 262, in eval
    truediv=truediv)
  File "C:\Anaconda\lib\site-packages\pandas\core\computation\expr.py", line 727, in __init__
    self.terms = self.parse()
  File "C:\Anaconda\lib\site-packages\pandas\core\computation\expr.py", line 744, in parse
    return self._visitor.visit(self.expr)
  File "C:\Anaconda\lib\site-packages\pandas\core\computation\expr.py", line 313, in visit
    return visitor(node, **kwargs)
  File "C:\Anaconda\lib\site-packages\pandas\core\computation\expr.py", line 319, in visit_Module
    return self.visit(expr, **kwargs)
  File "C:\Anaconda\lib\site-packages\pandas\core\computation\expr.py", line 313, in visit
    return visitor(node, **kwargs)
  File "C:\Anaconda\lib\site-packages\pandas\core\computation\expr.py", line 322, in visit_Expr
    return self.visit(node.value, **kwargs)
  File "C:\Anaconda\lib\site-packages\pandas\core\computation\expr.py", line 313, in visit
    return visitor(node, **kwargs)
  File "C:\Anaconda\lib\site-packages\pandas\core\computation\expr.py", line 417, in visit_BinOp
    left, right = self._maybe_downcast_constants(left, right)
  File "C:\Anaconda\lib\site-packages\pandas\core\computation\expr.py", line 365, in _maybe_downcast_constants
    name = self.env.add_tmp(np.float32(left.value))
AttributeError: 'UnaryOp' object has no attribute 'value'

Problem description

df.eval(...) operations fail if one of the columns involved is of type float32 and some unary operations are involved. In the example above df.eval('-1 + Foo32') failed, but df.eval('-Foo32') will succeed, and df.eval('-1*Foo32') will also fail.

Originally I tested fixing this by just adding self.value = operand.value to the UnaryOp object, before I realized that this issue only affects float32 objects. I haven't looked to see where the code branches off to cause this misbehavior.

This was tested and reproduced on both 0.20.3 and 0.22.0

Expected Output

No exception thrown.

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.6.3.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 79 Stepping 1, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.None

pandas: 0.22.0
pytest: 3.2.1
pip: 9.0.1
setuptools: 36.5.0.post20170921
Cython: 0.26.1
numpy: 1.12.1
scipy: 0.19.1
pyarrow: 0.7.1
xarray: None
IPython: 6.1.0
sphinx: 1.6.3
patsy: 0.4.1
dateutil: 2.6.1
pytz: 2017.2
blosc: None
bottleneck: 1.2.1
tables: 3.4.2
numexpr: 2.6.2
feather: None
matplotlib: 2.1.0
openpyxl: 2.4.8
xlrd: 1.1.0
xlwt: 1.3.0
xlsxwriter: 1.0.2
lxml: 4.1.0
bs4: 4.6.0
html5lib: 0.999999999
sqlalchemy: 1.1.13
pymysql: None
psycopg2: None
jinja2: 2.9.6
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

@TomAugspurger
Copy link
Contributor

Looks like a duplicate of #16363. Some more details there if you want to try submitting a fix.

@TomAugspurger TomAugspurger added the Duplicate Report Duplicate issue or pull request label Feb 14, 2018
@TomAugspurger TomAugspurger added this to the No action milestone Feb 14, 2018
@vmuriart
Copy link
Author

Thanks! didn't find it this morning, though I only searched from my phone... thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Duplicate Report Duplicate issue or pull request
Projects
None yet
Development

No branches or pull requests

2 participants