Skip to content

Invalid calculation of MINfloat16/MAXfloat16 in pandas/algos.pyx #10046

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
vonosmas opened this issue May 2, 2015 · 3 comments
Closed

Invalid calculation of MINfloat16/MAXfloat16 in pandas/algos.pyx #10046

vonosmas opened this issue May 2, 2015 · 3 comments
Labels
Build Library building on various platforms Numeric Operations Arithmetic, Comparison, and Logical operations

Comments

@vonosmas
Copy link

vonosmas commented May 2, 2015

The code generated by Cython from pandas/algos.pyx contains an undefined behavior (invalid cast of floating-point number to integer):

easy_install -U Cython
git clone git://github.com/pydata/pandas.git
python setup.py install

The generated code for the line from pandas/algos.pyx

cdef np.float16_t MINfloat16 = np.NINF

in pandas/algos.c contains:

typedef npy_float16 __pyx_t_5numpy_float16_t;
//... more code
__pyx_t_5numpy_float16_t __pyx_t_10;
//... more code
__pyx_t_10 = __pyx_PyFloat_AsFloat(__pyx_t_1);

The file numpy-1.9.2-py2.7-linux-x86_64.egg/numpy/core/include/numpy/npy_common.h
defines npy_float16 as 16-bit integer type (I guess numpy implementation has a custom code that carefully represents these 16 bits as mantiss+exponent). Casting floating-point value -inf to a 16-bit integer type is an undefined behavior according to C standard.

I wasn't able to find any place in pandas code that uses MINfloat16 (and MAXfloat16). Is it safe to just delete them?

@jreback
Copy link
Contributor

jreback commented May 2, 2015

can u show an example with the error?

the float16 constants are prob hanging around - as it's not an explicitly supported type (though u can actually pass an array in) - it's just that all of the algos will cast

@vonosmas
Copy link
Author

vonosmas commented May 2, 2015

Unfortunately, I see this error only if I compile generated .c files with clang -fsanitize=float-cast-overflow (http://clang.llvm.org/docs/UsersManual.html#controlling-code-generation). Looks like Cython directives are somewhat limited... Let me know if I should craft reproduction steps.

@jreback jreback added Build Library building on various platforms Numeric Operations Arithmetic, Comparison, and Logical operations labels May 5, 2015
@jreback
Copy link
Contributor

jreback commented Jun 18, 2015

dupe of #10382

@jreback jreback closed this as completed Jun 18, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Build Library building on various platforms Numeric Operations Arithmetic, Comparison, and Logical operations
Projects
None yet
Development

No branches or pull requests

2 participants