Skip to content

BUG: pd.to_timedelta() fails when arg is a Series with Int64 dtype #35574

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
3 tasks done
egoddard opened this issue Aug 5, 2020 · 3 comments · Fixed by #35582
Closed
3 tasks done

BUG: pd.to_timedelta() fails when arg is a Series with Int64 dtype #35574

egoddard opened this issue Aug 5, 2020 · 3 comments · Fixed by #35582
Assignees
Labels
Bug Datetime Datetime data dtype NA - MaskedArrays Related to pd.NA and nullable extension arrays
Milestone

Comments

@egoddard
Copy link
Contributor

egoddard commented Aug 5, 2020

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • (optional) I have confirmed this bug exists on the master branch of pandas.


Code Sample, a copy-pastable example

import pandas as pd


series = pd.Series([pd.NA], dtype="Int64")

pd.to_timedelta(series, errors="coerce")

Problem description

When converting a Series with an Int64 dtype to a timedelta, to_timedelta fails with

ValueError: cannot convert to 'int64'-dtype NumPy array with missing values. Specify an appropriate 'na_value' for this dtype.

Passing a Python list containing a NA value works, as does passing an Int64 Series that does not contain NA values. The functionality between these cases should be equivalent.

Expected Output

pd.to_timedelta() output returns a timedelta64 Series containing NA values when the input is an Int64 Series containg NA values:

0   <NA>
dtype: timedelta64[ns]

Output of pd.show_versions()

INSTALLED VERSIONS

commit : 3b6915e
python : 3.7.7.final.0
python-bits : 64
OS : Linux
OS-release : 5.4.0-7634-generic
Version : #38159534531720.04~a8480ad-Ubuntu SMP Wed Jul 22 15:13:45 UTC
machine : x86_64
processor :
byteorder : little
LC_ALL : C.UTF-8
LANG : C.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.2.0.dev0+28.g3b6915ef9
numpy : 1.18.5
pytz : 2020.1
dateutil : 2.8.1
pip : 20.2.1
setuptools : 45.2.0.post20200210
Cython : 0.29.21
pytest : 6.0.1
hypothesis : 5.23.11
sphinx : 3.1.1
blosc : None
feather : None
xlsxwriter : 1.3.1
lxml.etree : 4.4.1
html5lib : 1.1
pymysql : None
psycopg2 : None
jinja2 : 2.11.2
IPython : 7.17.0
pandas_datareader: None
bs4 : 4.9.1
bottleneck : 1.3.2
fsspec : 0.8.0
fastparquet : 0.4.1
gcsfs : 0.6.2
matplotlib : 3.2.1
numexpr : 2.7.1
odfpy : None
openpyxl : 3.0.4
pandas_gbq : None
pyarrow : 0.16.0
pytables : None
pyxlsb : None
s3fs : 0.4.2
scipy : 1.5.2
sqlalchemy : 1.3.18
tables : 3.6.1
tabulate : 0.8.7
xarray : 0.16.0
xlrd : 1.2.0
xlwt : 1.3.0
numba : 0.50.1

@egoddard egoddard added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Aug 5, 2020
@TomAugspurger
Copy link
Contributor

Thanks for the report. sequence_to_td64ns should probably handle masked arrays like IntegerArray specially. It could be done with a to_numpy("int64", na_value=tslibs.iNaT).

Are you interested in working on this?

@TomAugspurger TomAugspurger added NA - MaskedArrays Related to pd.NA and nullable extension arrays Datetime Datetime data dtype and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Aug 5, 2020
@TomAugspurger TomAugspurger added this to the Contributions Welcome milestone Aug 5, 2020
@egoddard
Copy link
Contributor Author

egoddard commented Aug 5, 2020

@TomAugspurger I am interested in working on it. I attempted a quick fix by trying to force ints_to_td64ns to work with the Int64Dtype(), but that lead to other issues :). I'll try a specific case in sequence_to_td64ns and report back. Thanks!

@egoddard
Copy link
Contributor Author

egoddard commented Aug 5, 2020

take

@jreback jreback modified the milestones: Contributions Welcome, 1.2, 1.1.1 Aug 6, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Datetime Datetime data dtype NA - MaskedArrays Related to pd.NA and nullable extension arrays
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants