DatetimeIndex.union with tz-aware and tz-naive raises TypeError

[mroeschke]

xref #11351

In [28]: s = pd.Series([pd.Timestamp('2011-01-01'), pd.NaT])

In [29]: stz = pd.Series(pd.DatetimeIndex(['2012-01-01', '2012-01-02'], tz='Asia/Tokyo'))

In [30]: pd.Index(s).union(pd.Index(stz))

TypeError: Cannot join tz-naive with tz-aware DatetimeIndex

In [32]: pd.__version__
Out[32]: '0.24.0.dev0+192.g0b63e81.dirty'

Problem description

Per #11351 (comment), this operation should just convert the data to object and not raise a TypeError.

Expected

In [33]: pd.Index(s).append(pd.Index(stz))
Out[33]:
Index([2011-01-01 00:00:00, NaT, 2012-01-01 00:00:00+09:00,
       2012-01-02 00:00:00+09:00],
      dtype='object')

[gfyoung]

Don't forget though that the union will do object comparison at the element-level, where that timezone incompatibility still exists so "casting to object" will issue warnings like this:

Traceback (most recent call last):
  File "pandas/_libs/tslibs/timestamps.pyx", line 171, in pandas._libs.tslibs.timestamps._Timestamp.__richcmp__
  File "pandas/_libs/tslibs/timestamps.pyx", line 231, in pandas._libs.tslibs.timestamps._Timestamp._assert_tzawareness_compat
TypeError: Cannot compare tz-naive and tz-aware timestamps
Exception ignored in: 'pandas._libs.lib.array_equivalent_object'
Traceback (most recent call last):
  File "pandas/_libs/tslibs/timestamps.pyx", line 171, in pandas._libs.tslibs.timestamps._Timestamp.__richcmp__
  File "pandas/_libs/tslibs/timestamps.pyx", line 231, in pandas._libs.tslibs.timestamps._Timestamp._assert_tzawareness_compat
TypeError: Cannot compare tz-naive and tz-aware timestamps
sys:1: RuntimeWarning: Cannot compare tz-naive and tz-aware timestamps, sort order is undefined for incomparable objects
Index([2011-01-01 00:00:00, NaT, 2012-01-01 00:00:00+09:00,
       2012-01-02 00:00:00+09:00],
      dtype='object')

Do we just want to swallow those or suppress them? I don't want to overhaul __richcmp__ just for this (corner) case admittedly.

Here is the current attempt:

forking-repos@392bd53

[gfyoung]

Actually, are we even able to redirect the Cython stderr without doing some kind of dance?

See this SO post here.

[gfyoung]

@jreback : Thoughts?

[jbrockmendel]

@mroeschke this appears to work in master. closeable?

[mroeschke]

Probably could use a unit test. I don't think we have one for this use case.