Skip to content

astype(np.int32) converts np.inf to int32 min value #14265

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
JonathanMace opened this issue Sep 20, 2016 · 2 comments · Fixed by #14343
Closed

astype(np.int32) converts np.inf to int32 min value #14265

JonathanMace opened this issue Sep 20, 2016 · 2 comments · Fixed by #14343
Labels
Enhancement Numeric Operations Arithmetic, Comparison, and Logical operations
Milestone

Comments

@JonathanMace
Copy link

JonathanMace commented Sep 20, 2016

It seems that converting np.inf to int32 and int64 using pd.astype converts them to the min int value rather than throwing an exception. Presumably this is because they raise OverflowError and not ValueError

Code Sample, a copy-pastable example if possible

for typ in [np.int32, np.int64]:
    for badval in [np.nan, np.inf]:
        # Print the exception thrown by casting to numpy directly
        try:
            typ(badval)
        except Exception as e:
            print "np.%s(np.%s) throws %s: %s" % (typ.__name__, badval, type(e).__name__, e)

        # Print the exception or resulting value by casting using pd.astype
        try:
            typed = pd.Series([badval]).astype(typ)
            print "pd.Series([np.%s]).astype(np.%s) returns dataframe containing %s" % (badval, typ.__name__, typed[0])
            print "Returned DataFrame:"
            print typed
        except Exception as e:
            print "pd.Series([np.%s]).astype(np.%s) throws %s: %s" % (badval, typ.__name__, type(e).__name__, e)

        print ""

Expected Output

np.int32(np.nan) throws ValueError: cannot convert float NaN to integer
pd.Series([np.nan]).astype(np.int32) throws ValueError: Cannot convert NA to integer

np.int32(np.inf) throws OverflowError: cannot convert float infinity to integer
pd.Series([np.inf]).astype(np.int32) returns dataframe containing -2147483648
Returned DataFrame:
0   -2147483648
dtype: int32

np.int64(np.nan) throws ValueError: cannot convert float NaN to integer
pd.Series([np.nan]).astype(np.int64) throws ValueError: Cannot convert NA to integer

np.int64(np.inf) throws OverflowError: cannot convert float infinity to integer
pd.Series([np.inf]).astype(np.int64) returns dataframe containing -9223372036854775808
Returned DataFrame:
0   -9223372036854775808
dtype: int64

output of pd.show_versions()

INSTALLED VERSIONS
------------------
commit: None
python: 2.7.12.final.0
python-bits: 64
OS: Linux
OS-release: 4.0.9-63_fbk11_3989_g7d5ca22
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8

pandas: 0.18.1
nose: 1.3.4
pip: 8.1.2
setuptools: 25.1.6
Cython: 0.22
numpy: 1.11.1
scipy: 0.18.0
statsmodels: 0.6.1
xarray: None
IPython: 5.0.0
sphinx: 1.2.3
patsy: 0.3.0
dateutil: 2.4.1
pytz: 2015.2
blosc: None
bottleneck: None
tables: 3.1.1
numexpr: 2.6.1
matplotlib: 1.4.3
openpyxl: 1.8.5
xlrd: 0.9.3
xlwt: 0.7.5
xlsxwriter: 0.6.7
lxml: 3.4.2
bs4: 4.3.2
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: 0.9.9
pymysql: None
psycopg2: None
jinja2: 2.8
boto: 2.36.0
pandas_datareader: None
@jreback
Copy link
Contributor

jreback commented Sep 20, 2016

to be honest this is not a pandas issue and a numpy issue. though I suppose we could intercept / fix this.

@jreback jreback added Enhancement Numeric Operations Arithmetic, Comparison, and Logical operations Difficulty Intermediate labels Sep 20, 2016
@jreback jreback added this to the Next Major Release milestone Sep 20, 2016
@shawnheide
Copy link
Contributor

Encountered this once in my own work and thought I'd close it up with a PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement Numeric Operations Arithmetic, Comparison, and Logical operations
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants