Skip to content

Unexpected behaviour of uint dtype #29290

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
nvm1 opened this issue Oct 30, 2019 · 3 comments
Closed

Unexpected behaviour of uint dtype #29290

nvm1 opened this issue Oct 30, 2019 · 3 comments
Labels

Comments

@nvm1
Copy link

nvm1 commented Oct 30, 2019

Code Sample, a copy-pastable example if possible

import pandas as pd
# generate some data and create dataframe
col = {'x': []}
for i in range(10):
    col['x'].append(18446744073709500000 - i)
df = pd.DataFrame.from_dict(col, dtype='uint64')
# see the dtype:
print(df.dtypes, end='\n\n')
# see correct and expected behaviour:
print(df.x * -1, end='\n\n')
# see possibly inconsistent behaviour:
print(-df.x, end='\n\n')

Problem description

please see the example: in two cases which are expected (sorry if the expectation is wrong) to generate the same results we see huge difference. In our case we didn't check the type of a column and assigned it to itself but with a minus value.

Expected Output

Probably the expected behaviour should match the logic here: #15832

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.7.3.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 142 Stepping 10, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.None

pandas: 0.24.2
pytest: 4.3.1
pip: 19.0.3
setuptools: 40.8.0
Cython: 0.29.6
numpy: 1.16.2
scipy: 1.2.1
pyarrow: None
xarray: None
IPython: 7.4.0
sphinx: 1.8.5
patsy: 0.5.1
dateutil: 2.8.0
pytz: 2018.9
blosc: None
bottleneck: 1.2.1
tables: 3.5.1
numexpr: 2.6.9
feather: None
matplotlib: 3.0.3
openpyxl: 2.6.1
xlrd: 1.2.0
xlwt: 1.3.0
xlsxwriter: 1.1.5
lxml.etree: 4.3.2
bs4: 4.7.1
html5lib: 1.0.1
sqlalchemy: 1.3.1
pymysql: None
psycopg2: 2.7.6.1 (dt dec pq3 ext lo64)
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
gcsfs: None
None

@nvm1 nvm1 changed the title Unexpected behaviour of assigning uint dtype Unexpected behaviour of uint dtype Oct 30, 2019
@jorisvandenbossche
Copy link
Member

jorisvandenbossche commented Oct 31, 2019

This seems to be an issue in numpy as well (so I suppose we inherit this behaviour):

In [21]: np.array([1, 2, 3], dtype='uint64') * -1 
Out[21]: array([-1., -2., -3.])

In [22]: -np.array([1, 2, 3], dtype='uint64') 
Out[22]: 
array([18446744073709551615, 18446744073709551614, 18446744073709551613],
      dtype=uint64)

Would you like to open an issue with numpy?

@R1j1t
Copy link
Contributor

R1j1t commented Oct 31, 2019

The reason for the dtype change I think is because uint cant take negative values, so numpy library handles multiply by negitive (x*-1) request by changing the dtype to float but does not change dtype for (-x). To convince myself I tried the following:

For uint dtype

>>> a=np.array([1, 2, 3], dtype='uint64')
>>> a.dtype
dtype('uint64')
>>> -a
array([18446744073709551615, 18446744073709551614, 18446744073709551613],
      dtype=uint64)
>>> a=a*-1
>>> a.dtype
dtype('float64')
>>> a
array([-1., -2., -3.])

for int dtype

>>> b=np.array([1, 2, 3], dtype='int64')
>>> b.dtype
dtype('int64')
>>> -b
array([-1, -2, -3])
>>> b=b*-1
>>> b.dtype
dtype('int64')
>>> b
array([-1, -2, -3])

@mroeschke
Copy link
Member

Appears to be a numpy issue as discussed. Closing as an upstream issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants