Skip to content

BUG: DataFrame.reindex sometimes promotes float16 / float32 #15517

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
adbull opened this issue Feb 27, 2017 · 2 comments
Closed

BUG: DataFrame.reindex sometimes promotes float16 / float32 #15517

adbull opened this issue Feb 27, 2017 · 2 comments
Labels
Bug Dtype Conversions Unexpected or buggy dtype conversions Reshaping Concat, Merge/Join, Stack/Unstack, Explode

Comments

@adbull
Copy link
Contributor

adbull commented Feb 27, 2017

Code Sample, a copy-pastable example if possible

>>> x = pd.DataFrame([[0]], dtype='float32')
>>> [x.reindex(i, [1]).dtypes[1] for i in [[0], [1]]]
[dtype('float64'), dtype('float32')]

Problem description

When a float16 / float32 DataFrame gets reindexed, new columns of nans are sometimes created as float64, depending on the specific indices requested. I think the expected behaviour is to create nans at the largest type of existing columns? In any case, the type of an empty column probably shouldn't depend on which indices it couldn't find.

Expected Output

[dtype('float32'), dtype('float32')]

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.5.2.final.0
python-bits: 64
OS: Linux
OS-release: 4.9.8-100.fc24.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: C
LANG: C
LOCALE: None.None

pandas: 0.19.2
nose: 1.3.7
pip: 9.0.1
setuptools: 27.2.0
Cython: 0.25.2
numpy: 1.11.3
scipy: 0.18.1
statsmodels: 0.8.0
xarray: 0.9.1
IPython: 4.2.0
sphinx: 1.5.1
patsy: 0.4.1
dateutil: 2.6.0
pytz: 2016.10
blosc: None
bottleneck: 1.2.0
tables: 3.3.0
numexpr: 2.6.2
matplotlib: 2.0.0
openpyxl: 2.4.1
xlrd: 1.0.0
xlwt: 1.2.0
xlsxwriter: 0.9.6
lxml: 3.7.2
bs4: 4.5.3
html5lib: 0.999
httplib2: None
apiclient: None
sqlalchemy: 1.1.5
pymysql: None
psycopg2: None
jinja2: 2.9.4
boto: 2.45.0
pandas_datareader: None

@jorisvandenbossche
Copy link
Member

I agree this is inconsistent, and both cases should give the same dtype.
However, I think the expected result should be 'float64' in both cases, as this is creating a new column with NaN, and the default dtype for that in pandas is float64

@jreback
Copy link
Contributor

jreback commented Feb 27, 2017

note that float16 is almost completely unsupported (it may happen to work in some instances though).

@jreback jreback added Difficulty Intermediate Dtype Conversions Unexpected or buggy dtype conversions Reshaping Concat, Merge/Join, Stack/Unstack, Explode labels Feb 27, 2017
@jreback jreback added this to the Next Major Release milestone Feb 27, 2017
@jreback jreback added the Bug label Feb 27, 2017
@adbull adbull changed the title DataFrame.reindex sometimes promotes float16 / float32 BUG: DataFrame.reindex sometimes promotes float16 / float32 Feb 28, 2017
@jreback jreback closed this as completed Jan 12, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Dtype Conversions Unexpected or buggy dtype conversions Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

No branches or pull requests

3 participants