tz-aware DatetimeIndex intersection raises AttributeError when values are the same but tz differs

[TomAugspurger]

In [25]: a = pd.date_range('2000', periods=4, tz="US/Central")

In [26]: a & a.tz_convert("US/Eastern")
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-26-d8e6c109715c> in <module>
----> 1 a & a.tz_convert("US/Eastern")

~/sandbox/pandas/pandas/core/indexes/base.py in __and__(self, other)
   2221
   2222     def __and__(self, other):
-> 2223         return self.intersection(other)
   2224
   2225     def __or__(self, other):

~/sandbox/pandas/pandas/core/indexes/datetimes.py in intersection(self, other)
    575             result = Index.intersection(self, other)
    576             result = self._shallow_copy(result._values, name=result.name,
--> 577                                         tz=result.tz, freq=None)
    578             if result.freq is None:
    579                 result.freq = to_offset(result.inferred_freq)

AttributeError: 'Index' object has no attribute 'tz'

If the values differ, no error is raised

In [27]: a & a._time_shift(1).tz_convert("US/Eastern")
Out[27]:
DatetimeIndex(['2000-01-02 01:00:00-05:00', '2000-01-03 01:00:00-05:00',
               '2000-01-04 01:00:00-05:00'],
              dtype='datetime64[ns, US/Eastern]', freq='D')

[cbertinato]

It looks like the issue here is that, initially, the DTIs are not equal nor are their dtypes because of the different timezones, so they both converted to Index.

pandas/pandas/core/indexes/base.py

Line 2388 in 08c920e

if not is_dtype_equal(self.dtype, other.dtype):

This actually starts in the intersection method for DTI, but gets dropped into the Index intersection method when it is determined that they're not equal.

Is the expected behavior that, in this case of the intersection, the DTIs are considered equal if the underlying indices are equal, regardless of timezone?

If so, then we can make a change to the equality check in the DTI intersection method.

The other option I see is to return an Index rather than a DTI in the case that they're equal and avoid or modify the offending shallow copy.

[jbrockmendel]

for other intersection-like operations with mixed timezones we convert to UTC (in particular im thinking of NDFrame._align_frame). would that be a better behavior than returning object-dtype?

[jbrockmendel]

or even convert other to self's tz might make more sense. when i patch this to just return object dtype:

>>> a & a.tz_convert("US/Eastern")
Index([2000-01-01 00:00:00-06:00, 2000-01-02 00:00:00-06:00,
       2000-01-03 00:00:00-06:00, 2000-01-04 00:00:00-06:00],
      dtype='object')

i.e. we get an object index of Timestamps that are all the same tz

[jbrockmendel]

This now returns an object-dtype Index. The analogous union also returns object dtype, but has a deprecation saying it will be UTC in a future version. We probably should do the same for all setops right?