REF: require listlike in maybe_cast_to_datetime #38505

jbrockmendel · 2020-12-15T17:51:59Z

No description provided.

jreback · 2020-12-16T00:51:46Z

are the fails unrelated?

jbrockmendel · 2020-12-16T01:15:19Z

are the fails unrelated?

no, there are some int32 wrinkles to figure out

…f-cast-scalars

jbrockmendel · 2020-12-17T22:36:12Z

@simonjayhawkins IIRC you use windows. can you reproduce the failures here?

simonjayhawkins · 2020-12-18T10:41:37Z

@simonjayhawkins IIRC you use windows. can you reproduce the failures here?

Not been successful running test suite locally on Windows for a long time. now use Ubuntu in WSL.

last time I tried to debug Windows issues, I seemed to be hit with #38214. (also concerns listlike)

simonjayhawkins · 2020-12-18T12:18:14Z

can confirm running full test suite still failing for me (with master)

here as a baseline. note the duration. although I think it had hung and I may have interrupted. will run on same hardware with Ubuntu for comparison and also with this branch on Windows shortly.

=========================== short test summary info ===========================
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other3]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other7]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other0]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other4]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other8]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other2]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other6]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other1]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other5]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other9]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_mixed_invalid[tzlocal()]
FAILED pandas/tests/dtypes/cast/test_promote.py::test_maybe_promote_datetimetz_with_datetimetz[tzlocal()-tzlocal()]
FAILED pandas/tests/extension/json/test_json.py::TestConstructors::test_series_constructor_no_data_with_index
FAILED pandas/tests/extension/json/test_json.py::TestConstructors::test_series_constructor_scalar_na_with_index
FAILED pandas/tests/plotting/test_backend.py::test_register_entrypoint - pkg_...
FAILED pandas/tests/plotting/frame/test_frame_subplots.py::TestDataFramePlotsSubplots::test_bar_barwidth_position_int
ERROR pandas/tests/io/excel/test_readers.py::TestReaders::test_read_from_py_localpath[engine_and_read_ext1]
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! KeyboardInterrupt !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
C:\Users\simon\anaconda3\envs\pandas-dev\lib\threading.py:306: KeyboardInterrupt
(to show a full traceback on KeyboardInterrupt use --full-trace)
= 16 failed, 147074 passed, 3542 skipped, 1074 xfailed, 6 xpassed, 115 warnings, 1 error in 4577.43s (1:16:17) =

with a splattering of...

.......................................................Windows fatal exception: stack overflow

Thread 0x000031ac (most recent call first):
  File "C:\Users\simon\anaconda3\envs\pandas-dev\lib\site-packages\execnet\gateway_base.py", line 400 in read
  File "C:\Users\simon\anaconda3\envs\pandas-dev\lib\site-packages\execnet\gateway_base.py", line 432 in from_io
  File "C:\Users\simon\anaconda3\envs\pandas-dev\lib\site-packages\execnet\gateway_base.py", line 967 in _thread_receiver
  File "C:\Users\simon\anaconda3\envs\pandas-dev\lib\site-packages\execnet\gateway_base.py", line 220 in run
  File "C:\Users\simon\anaconda3\envs\pandas-dev\lib\site-packages\execnet\gateway_base.py", line 285 in _perform_spawn

Current thread 0x00002fe8 (most recent call first):
  File "C:\Users\simon\pandas\pandas\core\construction.py", line 652 in create_series_with_explicit_dtype
  File "C:\Users\simon\pandas\pandas\core\series.py", line 382 in _init_dict
  File "C:\Users\simon\pandas\pandas\core\series.py", line 289 in __init__
  File "C:\Users\simon\pandas\pandas\core\construction.py", line 656 in create_series_with_explicit_dtype
  File "C:\Users\simon\pandas\pandas\core\series.py", line 382 in _init_dict
  File "C:\Users\simon\pandas\pandas\core\series.py", line 289 in __init__
  File "C:\Users\simon\pandas\pandas\core\construction.py", line 656 in create_series_with_explicit_dtype
  File "C:\Users\simon\pandas\pandas\core\series.py", line 382 in _init_dict
  File "C:\Users\simon\pandas\pandas\core\series.py", line 289 in __init__
  File "C:\Users\simon\pandas\pandas\core\construction.py", line 656 in create_series_with_explicit_dtype

simonjayhawkins · 2020-12-18T12:41:36Z

same hardware. WSL...

=========================================================== short test summary info ============================================================
FAILED pandas/tests/io/test_clipboard.py::test_raw_roundtrip[\U0001f44d...] - pandas.io.clipboard.PyperclipException:
FAILED pandas/tests/io/test_clipboard.py::test_raw_roundtrip[\u03a9\u0153\u2211\xb4...] - pandas.io.clipboard.PyperclipException:
FAILED pandas/tests/io/test_clipboard.py::test_raw_roundtrip[abcd...] - pandas.io.clipboard.PyperclipException:
====================== 3 failed, 148578 passed, 3410 skipped, 1077 xfailed, 2 xpassed, 105 warnings in 607.40s (0:10:07) =====================

simonjayhawkins · 2020-12-18T13:19:45Z

can you reproduce the failures here?

appears so. (95-16 = 79)

=========================== short test summary info ===========================
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other3]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other7]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other0]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other4]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other8]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other2]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other6]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other1]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other5]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_arraylike_invalid[tzlocal()-other9]
FAILED pandas/tests/arithmetic/test_datetime64.py::TestDatetime64ArrayLikeComparisons::test_dt64arr_cmp_mixed_invalid[tzlocal()]
FAILED pandas/tests/dtypes/cast/test_promote.py::test_maybe_promote_datetimetz_with_datetimetz[tzlocal()-tzlocal()]
FAILED pandas/tests/extension/test_integer.py::TestGroupby::test_groupby_extension_transform[Int8Dtype]
FAILED pandas/tests/extension/test_integer.py::TestGroupby::test_groupby_extension_transform[Int16Dtype]
FAILED pandas/tests/extension/test_integer.py::TestGroupby::test_groupby_extension_transform[Int32Dtype]
FAILED pandas/tests/extension/test_integer.py::TestGroupby::test_groupby_extension_transform[Int64Dtype]
FAILED pandas/tests/extension/test_integer.py::TestGroupby::test_groupby_extension_transform[UInt8Dtype]
FAILED pandas/tests/extension/test_integer.py::TestGroupby::test_groupby_extension_transform[UInt16Dtype]
FAILED pandas/tests/extension/test_integer.py::TestGroupby::test_groupby_extension_transform[UInt32Dtype]
FAILED pandas/tests/extension/test_integer.py::TestGroupby::test_groupby_extension_transform[UInt64Dtype]
FAILED pandas/tests/extension/test_interval.py::TestGrouping::test_groupby_extension_transform
FAILED pandas/tests/frame/test_stack_unstack.py::TestDataFrameReshape::test_stack_mixed_level
FAILED pandas/tests/extension/test_numpy.py::TestGroupby::test_groupby_extension_transform[float]
FAILED pandas/tests/extension/test_numpy.py::TestGroupby::test_groupby_extension_transform[object]
FAILED pandas/tests/extension/test_boolean.py::TestGroupby::test_groupby_extension_transform
FAILED pandas/tests/extension/test_datetime.py::TestGroupby::test_groupby_extension_transform[US/Central]
FAILED pandas/tests/extension/test_period.py::TestGroupby::test_groupby_extension_transform
FAILED pandas/tests/extension/test_floating.py::TestGroupby::test_groupby_extension_transform[Float32Dtype]
FAILED pandas/tests/extension/test_floating.py::TestGroupby::test_groupby_extension_transform[Float64Dtype]
FAILED pandas/tests/extension/test_integer.py::TestDtype::test_check_dtype[Int32Dtype]
FAILED pandas/tests/extension/test_integer.py::TestDtype::test_check_dtype[Int64Dtype]
FAILED pandas/tests/frame/methods/test_count.py::TestDataFrameCount::test_count
FAILED pandas/tests/frame/methods/test_count.py::TestDataFrameCount::test_count_level_corner
FAILED pandas/tests/frame/methods/test_dtypes.py::TestDataFrameDataTypes::test_empty_frame_dtypes
FAILED pandas/tests/extension/test_string.py::TestGroupBy::test_groupby_extension_transform[StringDtype]
FAILED pandas/tests/extension/test_string.py::TestGroupBy::test_groupby_extension_transform[ArrowStringDtype]
FAILED pandas/tests/extension/decimal/test_decimal.py::TestGroupby::test_groupby_extension_transform
FAILED pandas/tests/frame/methods/test_get_numeric_data.py::TestGetNumericData::test_get_numeric_data
FAILED pandas/tests/extension/json/test_json.py::TestConstructors::test_series_constructor_no_data_with_index
FAILED pandas/tests/indexes/datetimes/test_partial_slicing.py::TestSlicing::test_monotone_DTI_indexing_bug
FAILED pandas/tests/series/indexing/test_delitem.py::TestSeriesDelItem::test_delitem_object_index
FAILED pandas/tests/series/indexing/test_setitem.py::TestSetitemWithExpansion::test_setitem_empty_series
FAILED pandas/tests/series/indexing/test_setitem.py::TestSetitemWithExpansion::test_setitem_empty_series_datetimeindex_preserves_freq
FAILED pandas/tests/resample/test_deprecated.py::test_resample_loffset_count
FAILED pandas/tests/resample/test_deprecated.py::test_resample_float_base - A...
FAILED pandas/tests/series/methods/test_unstack.py::test_unstack_tuplename_in_multiindex
FAILED pandas/tests/series/methods/test_unstack.py::test_unstack_mixed_type_name_in_multiindex[unstack_idx0-expected_values0-expected_index0-expected_columns0]
FAILED pandas/tests/series/methods/test_unstack.py::test_unstack_mixed_type_name_in_multiindex[unstack_idx1-expected_values1-expected_index1-expected_columns1]
FAILED pandas/tests/resample/test_time_grouper.py::test_upsample_sum[sum-method_args0-expected_values0]
FAILED pandas/tests/resample/test_time_grouper.py::test_upsample_sum[sum-method_args1-expected_values1]
FAILED pandas/tests/resample/test_time_grouper.py::test_upsample_sum[prod-method_args4-expected_values4]
FAILED pandas/tests/resample/test_time_grouper.py::test_upsample_sum[prod-method_args5-expected_values5]
FAILED pandas/tests/reshape/test_crosstab.py::TestCrosstab::test_crosstab_tuple_name[names0]
FAILED pandas/tests/reshape/test_crosstab.py::TestCrosstab::test_crosstab_tuple_name[names1]
FAILED pandas/tests/frame/methods/test_to_csv.py::TestDataFrameToCSV::test_to_csv_with_dst_transitions
FAILED pandas/tests/reshape/merge/test_join.py::TestJoin::test_join_index_mixed_overlap
FAILED pandas/tests/reshape/merge/test_join.py::test_join_inner_multiindex_deterministic_order
FAILED pandas/tests/reshape/merge/test_merge.py::TestMerge::test_merge_nocopy
FAILED pandas/tests/groupby/test_apply.py::test_time_field_bug - AssertionErr...
FAILED pandas/tests/groupby/test_categorical.py::test_apply[True] - Assertion...
FAILED pandas/tests/groupby/test_categorical.py::test_apply[False] - Assertio...
FAILED pandas/tests/reshape/merge/test_multi.py::TestMergeMulti::test_merge_multiple_cols_with_mixed_cols_index
FAILED pandas/tests/groupby/test_filters.py::test_filter_multiple_timestamp
FAILED pandas/tests/groupby/test_filters.py::test_filter_and_transform_with_non_unique_int_index
FAILED pandas/tests/groupby/test_filters.py::test_filter_and_transform_with_multiple_non_unique_int_index
FAILED pandas/tests/groupby/test_filters.py::test_filter_and_transform_with_non_unique_float_index
FAILED pandas/tests/groupby/test_filters.py::test_filter_and_transform_with_non_unique_timestamp_index
FAILED pandas/tests/groupby/test_filters.py::test_filter_and_transform_with_non_unique_string_index
FAILED pandas/tests/series/test_arithmetic.py::TestSeriesArithmetic::test_arithmetic_with_duplicate_index
FAILED pandas/tests/groupby/test_groupby_dropna.py::test_slice_groupby_then_transform[False-df_expected1-s_expected1]
FAILED pandas/tests/groupby/test_grouping.py::TestGrouping::test_grouper_getting_correct_binner
FAILED pandas/tests/extension/json/test_json.py::TestConstructors::test_series_constructor_scalar_na_with_index
FAILED pandas/tests/indexes/datetimes/test_shift.py::TestDatetimeIndexShift::test_dti_shift_near_midnight[0-2014-11-14 00:00:00]
FAILED pandas/tests/indexes/datetimes/test_shift.py::TestDatetimeIndexShift::test_dti_shift_near_midnight[-1-2014-11-13 23:00:00]
FAILED pandas/tests/indexes/datetimes/test_shift.py::TestDatetimeIndexShift::test_dti_shift_near_midnight[1-2014-11-14 01:00:00]
FAILED pandas/tests/io/json/test_normalize.py::TestJSONNormalize::test_nonetype_record_path[None]
FAILED pandas/tests/io/json/test_normalize.py::TestJSONNormalize::test_nonetype_record_path[nan0]
FAILED pandas/tests/io/json/test_normalize.py::TestJSONNormalize::test_nonetype_record_path[NaT]
FAILED pandas/tests/io/json/test_normalize.py::TestJSONNormalize::test_nonetype_record_path[nan1]
FAILED pandas/tests/io/json/test_normalize.py::TestJSONNormalize::test_nonetype_record_path[<NA>]
FAILED pandas/tests/indexing/test_datetime.py::TestDatetimeIndex::test_indexing_with_datetimeindex_tz
FAILED pandas/tests/indexing/test_datetime.py::TestDatetimeIndex::test_nanosecond_getitem_setitem_with_tz
FAILED pandas/tests/indexing/test_floats.py::TestFloatIndexers::test_scalar_integer[DataFrame-makeIntIndex-<lambda>-True]
FAILED pandas/tests/indexing/test_floats.py::TestFloatIndexers::test_scalar_integer[DataFrame-makeRangeIndex-<lambda>-True]
FAILED pandas/tests/indexing/test_iloc.py::TestiLoc2::test_iloc_setitem - Ass...
FAILED pandas/tests/indexing/test_loc.py::TestLocSetitemWithExpansion::test_loc_setitem_incremental_with_dst
FAILED pandas/tests/indexing/test_scalar.py::TestScalar2::test_imethods_with_dups
FAILED pandas/tests/frame/test_constructors.py::TestDataFrameConstructors::test_constructor_scalar_inference
FAILED pandas/tests/frame/test_constructors.py::TestDataFrameConstructors::test_constructor_list_of_series_aligned_index
FAILED pandas/tests/frame/test_constructors.py::TestDataFrameConstructors::test_constructor_with_datetimes
FAILED pandas/tests/frame/test_constructors.py::TestDataFrameConstructors::test_constructor_for_list_with_dtypes
FAILED pandas/tests/io/pytables/test_timezones.py::test_py2_created_with_datetimez
FAILED pandas/tests/plotting/test_backend.py::test_register_entrypoint - pkg_...
FAILED pandas/tests/plotting/frame/test_frame_subplots.py::TestDataFramePlotsSubplots::test_bar_barwidth_position_int
FAILED pandas/tests/resample/test_datetime_index.py::test_resample_upsampling_picked_but_not_correct
ERROR pandas/tests/io/excel/test_readers.py::TestExcelFileRead::test_excel_passes_na_filter[engine_and_read_ext0-False]
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! KeyboardInterrupt !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
C:\Users\simon\anaconda3\envs\pandas-dev\lib\threading.py:306: KeyboardInterrupt
(to show a full traceback on KeyboardInterrupt use --full-trace)
= 95 failed, 147109 passed, 3555 skipped, 1074 xfailed, 6 xpassed, 115 warnings, 1 error in 1912.78s (0:31:52) =

jbrockmendel · 2020-12-18T16:09:21Z

Windows fatal exception: stack overflow

yikes. Thanks for taking a look. I'll keep poking at this on the CI for now.

note the duration [4577.43s]

yikes

same hardware. WSL...

re test_clipboard, I saw the same failures on a headless Ubuntu machine that I sometimes use, fixed that by

adding export DISPLAY=":0" to .bash_profile
amending the alias I use for tests to alias ptest="xvfb-run python3 -m pytest pandas/tests --skip-slow --skip-db"

YMMV

simonjayhawkins · 2020-12-18T16:22:23Z

re test_clipboard, I saw the same failures on a headless Ubuntu machine that I sometimes use, fixed that by

there's an open issue #38527 and a potential fix #38546

I'm using an (headless) desktop machine to run the tests (in WSL) using Windows remote desktop (from laptop).

i've not been too concerned since when I used to use the machine as my primary desktop machine, I always had clipboard errors. (the only difference being is that reruning the tests with --lf and they always then passed)

…f-cast-scalars

jreback · 2020-12-22T00:20:33Z

@jbrockmendel good on this?

jbrockmendel · 2020-12-22T00:51:27Z

yep

REF: require listlike in maybe_cast_to_datetime

2fbcab7

jreback added the Refactor Internal refactoring of code label Dec 16, 2020

jbrockmendel added 4 commits December 15, 2020 17:30

Merge branch 'master' of https://github.com/pandas-dev/pandas into re…

8fa2d03

…f-cast-scalars

troubleshoot 32bit build failures

30dea45

Merge branch 'master' of https://github.com/pandas-dev/pandas into re…

57e065a

…f-cast-scalars

troubleshoot 32bit builds

8f696de

jbrockmendel mentioned this pull request Dec 17, 2020

BUG: require arraylike in infer_dtype_from_array #38473

Merged

5 tasks

jbrockmendel added 2 commits December 17, 2020 10:28

Merge branch 'master' of https://github.com/pandas-dev/pandas into re…

495fd8d

…f-cast-scalars

Merge branch 'master' of https://github.com/pandas-dev/pandas into re…

2e5b422

…f-cast-scalars

Merge branch 'master' of https://github.com/pandas-dev/pandas into re…

ca5fc13

…f-cast-scalars

jreback added this to the 1.3 milestone Dec 22, 2020

jreback merged commit cd0fb05 into pandas-dev:master Dec 22, 2020

jbrockmendel deleted the ref-cast-scalars branch December 22, 2020 01:02

luckyvs1 pushed a commit to luckyvs1/pandas that referenced this pull request Jan 20, 2021

REF: require listlike in maybe_cast_to_datetime (pandas-dev#38505)

07aba3f

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

REF: require listlike in maybe_cast_to_datetime #38505

REF: require listlike in maybe_cast_to_datetime #38505

jbrockmendel commented Dec 15, 2020

jreback commented Dec 16, 2020

jbrockmendel commented Dec 16, 2020

jbrockmendel commented Dec 17, 2020

simonjayhawkins commented Dec 18, 2020

simonjayhawkins commented Dec 18, 2020

simonjayhawkins commented Dec 18, 2020

simonjayhawkins commented Dec 18, 2020

jbrockmendel commented Dec 18, 2020

simonjayhawkins commented Dec 18, 2020

jreback commented Dec 22, 2020

jbrockmendel commented Dec 22, 2020

REF: require listlike in maybe_cast_to_datetime #38505

REF: require listlike in maybe_cast_to_datetime #38505

Conversation

jbrockmendel commented Dec 15, 2020

jreback commented Dec 16, 2020

jbrockmendel commented Dec 16, 2020

jbrockmendel commented Dec 17, 2020

simonjayhawkins commented Dec 18, 2020

simonjayhawkins commented Dec 18, 2020

simonjayhawkins commented Dec 18, 2020

simonjayhawkins commented Dec 18, 2020

jbrockmendel commented Dec 18, 2020

simonjayhawkins commented Dec 18, 2020

jreback commented Dec 22, 2020

jbrockmendel commented Dec 22, 2020