Skip to content

fillna can't process np.float16 type #19205

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
MarsGarai opened this issue Jan 12, 2018 · 4 comments
Closed

fillna can't process np.float16 type #19205

MarsGarai opened this issue Jan 12, 2018 · 4 comments
Labels
Compat pandas objects compatability with Numpy or Python functions Dtype Conversions Unexpected or buggy dtype conversions

Comments

@MarsGarai
Copy link

Code Sample, a copy-pastable example if possible

p_s = pd.Series(data=[47.97,   np.NAN,  np.NAN, 47.16, 47.97], dtype=np.float16, 
                index=pd.DatetimeIndex(['2015-07-24', '2015-07-25', '2015-07-26', \
                                                          '2015-07-27','2015-07-28']))
p_s
Out:
2015-07-24    47.96875
2015-07-25         NaN
2015-07-26         NaN
2015-07-27    47.15625
2015-07-28    47.96875
dtype: float16

Problem description

p_s.fillna(method='ffill')
get the error:
ValueError: Invalid dtype for pad_2d [float16]
But:
p_s.astype(np.float64).fillna(method='ffill')
or:
p_s.astype(np.float32).fillna(method='ffill')
works well:
2015-07-24 47.96875
2015-07-25 47.96875
2015-07-26 47.96875
2015-07-27 47.15625
2015-07-28 47.96875

Expected Output

p_s..fillna(method='ffill')
Out:
2015-07-24 47.96875
2015-07-25 47.96875
2015-07-26 47.96875
2015-07-27 47.15625
2015-07-28 47.96875
dtype: float16

Output of pd.show_versions()

[paste the output of pd.show_versions() here below this line]

  1. On WINDOWS
    INSTALLED VERSIONS

commit: None
python: 3.6.3.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 58 Stepping 9, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.None

pandas: 0.20.3
pytest: 3.2.1
pip: 9.0.1
setuptools: 36.5.0.post20170921
Cython: 0.26.1
numpy: 1.14.0
scipy: 1.0.0
xarray: None
IPython: 6.1.0
sphinx: 1.6.3
patsy: 0.4.1
dateutil: 2.6.1
pytz: 2017.2
blosc: None
bottleneck: 1.2.1
tables: 3.4.2
numexpr: 2.6.2
feather: None
matplotlib: 2.1.0
openpyxl: 2.4.8
xlrd: 1.1.0
xlwt: 1.3.0
xlsxwriter: 1.0.2
lxml: 4.1.0
bs4: 4.6.0
html5lib: 0.9999999
sqlalchemy: 1.1.13
pymysql: None
psycopg2: None
jinja2: 2.9.6
s3fs: None
pandas_gbq: None
pandas_datareader: None

  1. On UBUNTU 16.04
    INSTALLED VERSIONS

commit: None
python: 3.6.2.final.0
python-bits: 64
OS: Linux
OS-release: 4.4.0-108-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: ru_RU.UTF-8
LOCALE: ru_RU.UTF-8

pandas: 0.22.0
pytest: 3.2.1
pip: 9.0.1
setuptools: 36.5.0.post20170921
Cython: 0.26.1
numpy: 1.14.0
scipy: 1.0.0
pyarrow: None
xarray: None
IPython: 6.1.0
sphinx: 1.6.3
patsy: 0.4.1
dateutil: 2.6.1
pytz: 2017.2
blosc: None
bottleneck: 1.2.1
tables: 3.4.2
numexpr: 2.6.2
feather: None
matplotlib: 2.0.2
openpyxl: 2.4.8
xlrd: 1.1.0
xlwt: 1.3.0
xlsxwriter: 0.9.8
lxml: 3.8.0
bs4: 4.6.0
html5lib: 0.999999999
sqlalchemy: 1.1.13
pymysql: None
psycopg2: None
jinja2: 2.9.6
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

@jreback jreback added Dtype Conversions Unexpected or buggy dtype conversions Compat pandas objects compatability with Numpy or Python functions labels Jan 12, 2018
@jreback jreback added this to the won't fix milestone Jan 12, 2018
@jreback
Copy link
Contributor

jreback commented Jan 12, 2018

float16 have almost no support anywhere as a first class type.

@jreback jreback closed this as completed Jan 12, 2018
@jreback
Copy link
Contributor

jreback commented Jan 12, 2018

we have a master issue about this

@TomAugspurger TomAugspurger modified the milestones: won't fix, No action Jul 6, 2018
@gustavz
Copy link

gustavz commented Nov 6, 2019

So this issue won't be addressed in future updates?

@jreback
Copy link
Contributor

jreback commented Nov 6, 2019

see #9220

it would take pull requests from the community
pandas is all volunteer and there are 3000 issues open

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Compat pandas objects compatability with Numpy or Python functions Dtype Conversions Unexpected or buggy dtype conversions
Projects
None yet
Development

No branches or pull requests

4 participants