Skip to content

BUG: parsing mixed-offset Timestamps with errors='ignore' and no format raises #50585

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
3 tasks done
MarcoGorelli opened this issue Jan 5, 2023 · 0 comments · Fixed by #50586
Closed
3 tasks done
Labels
Bug Datetime Datetime data dtype

Comments

@MarcoGorelli
Copy link
Member

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

In [1]: to_datetime([Timestamp('2020').tz_localize('Europe/London'), Timestamp('2020').tz_localize('US/Pacific')], errors='ignore')
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[1], line 1
----> 1 to_datetime([Timestamp('2020').tz_localize('Europe/London'), Timestamp('2020').tz_localize('US/Pacific')], errors='ignore')

File ~/pandas-dev/pandas/core/tools/datetimes.py:1063, in to_datetime(arg, errors, dayfirst, yearfirst, utc, format, exact, unit, infer_datetime_format, origin, cache)
   1061         result = _convert_and_box_cache(argc, cache_array)
   1062     else:
-> 1063         result = convert_listlike(argc, format)
   1064 else:
   1065     result = convert_listlike(np.array([arg]), format)[0]

File ~/pandas-dev/pandas/core/tools/datetimes.py:437, in _convert_listlike_datetimes(arg, format, name, utc, unit, errors, dayfirst, yearfirst, exact)
    434 if format is not None:
    435     return _array_strptime_with_fallback(arg, name, utc, format, exact, errors)
--> 437 result, tz_parsed = objects_to_datetime64ns(
    438     arg,
    439     dayfirst=dayfirst,
    440     yearfirst=yearfirst,
    441     utc=utc,
    442     errors=errors,
    443     allow_object=True,
    444 )
    446 if tz_parsed is not None:
    447     # We can take a shortcut since the datetime64 numpy array
    448     # is in UTC
    449     dta = DatetimeArray(result, dtype=tz_to_dtype(tz_parsed))

File ~/pandas-dev/pandas/core/arrays/datetimes.py:2158, in objects_to_datetime64ns(data, dayfirst, yearfirst, utc, errors, allow_object)
   2156 order: Literal["F", "C"] = "F" if flags.f_contiguous else "C"
   2157 try:
-> 2158     result, tz_parsed = tslib.array_to_datetime(
   2159         data.ravel("K"),
   2160         errors=errors,
   2161         utc=utc,
   2162         dayfirst=dayfirst,
   2163         yearfirst=yearfirst,
   2164     )
   2165     result = result.reshape(data.shape, order=order)
   2166 except OverflowError as err:
   2167     # Exception is raised when a part of date is greater than 32 bit signed int

File ~/pandas-dev/pandas/_libs/tslib.pyx:441, in pandas._libs.tslib.array_to_datetime()
    439 @cython.wraparound(False)
    440 @cython.boundscheck(False)
--> 441 cpdef array_to_datetime(
    442     ndarray[object] values,
    443     str errors="raise",

File ~/pandas-dev/pandas/_libs/tslib.pyx:521, in pandas._libs.tslib.array_to_datetime()
    519 else:
    520     found_naive = True
--> 521 tz_out = convert_timezone(
    522     val.tzinfo,
    523     tz_out,

File ~/pandas-dev/pandas/_libs/tslibs/conversion.pyx:725, in pandas._libs.tslibs.conversion.convert_timezone()
    723                      "datetime64 unless utc=True")
    724 elif tz_out is not None and not tz_compare(tz_out, tz_in):
--> 725     raise ValueError("Tz-aware datetime.datetime "
    726                      "cannot be converted to "
    727                      "datetime64 unless utc=True")

ValueError: Tz-aware datetime.datetime cannot be converted to datetime64 unless utc=True

In [2]: to_datetime([Timestamp('2020').tz_localize('Europe/London'), Timestamp('2020').tz_localize('US/Pacific')], errors='ignore', format='%Y-%m
   ...: -%d')
Out[2]: Index([2020-01-01 00:00:00+00:00, 2020-01-01 00:00:00-08:00], dtype='object')

Issue Description

It shouldn't raise with errors='ignore'

Expected Behavior

Index([2020-01-01 00:00:00+00:00, 2020-01-01 00:00:00-08:00], dtype='object')

Installed Versions

INSTALLED VERSIONS

commit : 8acd314
python : 3.8.16.final.0
python-bits : 64
OS : Linux
OS-release : 5.10.102.1-microsoft-standard-WSL2
Version : #1 SMP Wed Mar 2 00:30:59 UTC 2022
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_GB.UTF-8
LOCALE : en_GB.UTF-8

pandas : 2.0.0.dev0+1049.g8acd31495a
numpy : 1.23.5
pytz : 2022.7
dateutil : 2.8.2
setuptools : 65.6.3
pip : 22.3.1
Cython : 0.29.32
pytest : 7.2.0
hypothesis : 6.61.0
sphinx : 5.3.0
blosc : 1.11.1
feather : None
xlsxwriter : 3.0.6
lxml.etree : 4.9.2
html5lib : 1.1
pymysql : 1.0.2
psycopg2 : 2.9.5
jinja2 : 3.1.2
IPython : 8.8.0
pandas_datareader: None
bs4 : 4.11.1
bottleneck : 1.3.5
brotli :
fastparquet : 2022.12.0
fsspec : 2022.11.0
gcsfs : 2022.11.0
matplotlib : 3.6.2
numba : 0.56.4
numexpr : 2.8.4
odfpy : None
openpyxl : 3.0.10
pandas_gbq : None
pyarrow : 10.0.1
pyreadstat : 1.2.0
pyxlsb : 1.0.10
s3fs : 2022.11.0
scipy : 1.10.0
snappy :
sqlalchemy : 1.4.46
tables : 3.8.0
tabulate : 0.9.0
xarray : 2022.12.0
xlrd : 2.0.1
zstandard : 0.19.0
tzdata : 2022.7
qtpy : None
pyqt5 : None
None

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Datetime Datetime data dtype
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant