BUG: to_datetime with xarray DataArray and specifie unit errors

[jorisvandenbossche]

When passing the values as a xarray.DataArray and specifying a unit, we get an error:

In [1]: import xarray as xr

In [2]: arr = xr.DataArray([1, 2, 3])

In [3]: pd.to_datetime(arr, unit='ns')
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-547fdd08888f> in <module>
----> 1 pd.to_datetime(arr, unit='ns')

~/miniconda3/envs/DS-geospatial/lib/python3.8/site-packages/pandas/core/tools/datetimes.py in to_datetime(arg, errors, dayfirst, yearfirst, utc, format, exact, unit, infer_datetime_format, origin, cache)
    910             result = _convert_and_box_cache(arg, cache_array)
    911         else:
--> 912             result = convert_listlike(arg, format)
    913     else:
    914         result = convert_listlike(np.array([arg]), format)[0]

~/miniconda3/envs/DS-geospatial/lib/python3.8/site-packages/pandas/core/tools/datetimes.py in _convert_listlike_datetimes(arg, format, name, tz, unit, errors, infer_datetime_format, dayfirst, yearfirst, exact)
    354         if format is not None:
    355             raise ValueError("cannot specify both format and unit")
--> 356         return _to_datetime_with_unit(arg, unit, name, tz, errors)
    357     elif getattr(arg, "ndim", 1) > 1:
    358         raise TypeError(

~/miniconda3/envs/DS-geospatial/lib/python3.8/site-packages/pandas/core/tools/datetimes.py in _to_datetime_with_unit(arg, unit, name, tz, errors)
    526         tz_parsed = None
    527     else:
--> 528         arr, tz_parsed = tslib.array_with_unit_to_datetime(arg, unit, errors=errors)
    529 
    530     if errors == "ignore":

TypeError: Argument 'values' has incorrect type (expected numpy.ndarray, got DataArray)

Since this is an array-like, I expect that we coerce that to a numpy array before passing to our cython routines (eg passing a list or Series does work this way).

And actually when not specifying the unit, it does work:

In [4]: pd.to_datetime(arr)
Out[4]: 
DatetimeIndex(['1970-01-01 00:00:00.000000001',
               '1970-01-01 00:00:00.000000002',
               '1970-01-01 00:00:00.000000003'],
              dtype='datetime64[ns]', freq=None)

Installed Versions

In [5]: pd.show_versions()

INSTALLED VERSIONS
------------------
commit           : 73c68257545b5f8530b7044f56647bd2db92e2ba
python           : 3.8.12.final.0
python-bits      : 64
OS               : Linux
OS-release       : 5.4.0-88-generic
Version          : #99-Ubuntu SMP Thu Sep 23 17:29:00 UTC 2021
machine          : x86_64
processor        : x86_64
byteorder        : little
LC_ALL           : None
LANG             : en_US.UTF-8
LOCALE           : en_US.UTF-8

pandas           : 1.3.3
numpy            : 1.21.2
pytz             : 2021.1
dateutil         : 2.8.2
pip              : 21.2.4
setuptools       : 58.0.4
Cython           : None
pytest           : None
hypothesis       : None
sphinx           : None
blosc            : None
feather          : None
xlsxwriter       : None
lxml.etree       : None
html5lib         : None
pymysql          : None
psycopg2         : None
jinja2           : 3.0.1
IPython          : 7.28.0
pandas_datareader: None
bs4              : None
bottleneck       : None
fsspec           : 2021.09.0
fastparquet      : None
gcsfs            : None
matplotlib       : 3.4.3
numexpr          : None
odfpy            : None
openpyxl         : None
pandas_gbq       : None
pyarrow          : 5.0.0
pyxlsb           : None
s3fs             : 2021.09.0
scipy            : 1.7.1
sqlalchemy       : None
tables           : None
tabulate         : None
xarray           : 0.19.0
xlrd             : None
xlwt             : None
numba            : 0.53.1

[rocabrera]

Hi, I would like to get into the world of contributions to open source software. Can I take a look at this bug? If I can't solve it, I'll report it here.

thanks in advance

[rocabrera]

Apparently, I've fixed the problem, but I haven't run all the tests yet. Do I just need to run the pytest tests/ command? I got some errors in the tests before changing the source code.

I am trying to follow this guideline https://pandas.pydata.org/docs/dev/development/contributing.html

[jorisvandenbossche]

@rocabrera thanks for taking a look at this!

It might be that the errors you get are unrelated (difficult to say without seeing some output).
I would recommend to open a PR so it is easier to see what you did and comment on that. Also make sure to add a new test for this specific bug.

[shubham11941140]

I am on it. I will do local testing and then open a PR.

[rocabrera]

I am still learning about the package and the correct guidelines to contribute. I made a pr with the code a previously wrote.

Thank you for the awesome package. I love working with pandas. I hope I can help as much as possible.