API: 'D' and offset.Day always represents calendar day

doc/source/timeseries.rst

@@ -408,7 +408,7 @@ In practice this becomes very cumbersome because we often need a very long
 index with a large number of timestamps. If we need timestamps on a regular
 frequency, we can use the :func:`date_range` and :func:`bdate_range` functions
 to create a ``DatetimeIndex``. The default frequency for ``date_range`` is a
-**day** while the default for ``bdate_range`` is a **business day**:
+**calendar day** while the default for ``bdate_range`` is a **business day**:
 
 .. ipython:: python
 
@@ -927,27 +927,6 @@ in the operation).
 
 .. _relativedelta documentation: https://dateutil.readthedocs.io/en/stable/relativedelta.html
 
-.. _timeseries.dayvscalendarday:
-
-Day vs. CalendarDay
-~~~~~~~~~~~~~~~~~~~
-
-:class:`Day` (``'D'``) is a timedelta-like offset that respects absolute time
-arithmetic and is an alias for 24 :class:`Hour`. This offset is the default
-argument to many pandas time related function like :func:`date_range` and :func:`timedelta_range`.
-
-:class:`CalendarDay` (``'CD'``) is a relativedelta-like offset that respects
-calendar time arithmetic. :class:`CalendarDay` is useful preserving calendar day
-semantics with date times with have day light savings transitions, i.e. :class:`CalendarDay`
-will preserve the hour before the day light savings transition.
-
-.. ipython:: python
-
-   ts = pd.Timestamp('2016-10-30 00:00:00', tz='Europe/Helsinki')
-   ts + pd.offsets.Day(1)
-   ts + pd.offsets.CalendarDay(1)
-
-
 Parametric Offsets
 ~~~~~~~~~~~~~~~~~~
 
@@ -1243,8 +1222,7 @@ frequencies. We will refer to these aliases as *offset aliases*.
 
     "B", "business day frequency"
     "C", "custom business day frequency"
-    "D", "day frequency"
-    "CD", "calendar day frequency"
+    "D", "calendar day frequency"
     "W", "weekly frequency"
     "M", "month end frequency"
     "SM", "semi-month end frequency (15th and end of month)"

doc/source/whatsnew/v0.24.0.rst

@@ -589,46 +589,6 @@ that the dates have been converted to UTC
     pd.to_datetime(["2015-11-18 15:30:00+05:30",
                     "2015-11-18 16:30:00+06:30"], utc=True)
 
-.. _whatsnew_0240.api_breaking.calendarday:
-
-CalendarDay Offset
-^^^^^^^^^^^^^^^^^^
-
-:class:`Day` and associated frequency alias ``'D'`` were documented to represent
-a calendar day; however, arithmetic and operations with :class:`Day` sometimes
-respected absolute time instead (i.e. ``Day(n)`` and acted identically to ``Timedelta(days=n)``).
-
-*Previous Behavior*:
-
-.. code-block:: ipython
-
-
-    In [2]: ts = pd.Timestamp('2016-10-30 00:00:00', tz='Europe/Helsinki')
-
-    # Respects calendar arithmetic
-    In [3]: pd.date_range(start=ts, freq='D', periods=3)
-    Out[3]:
-    DatetimeIndex(['2016-10-30 00:00:00+03:00', '2016-10-31 00:00:00+02:00',
-                   '2016-11-01 00:00:00+02:00'],
-                  dtype='datetime64[ns, Europe/Helsinki]', freq='D')
-
-    # Respects absolute arithmetic
-    In [4]: ts + pd.tseries.frequencies.to_offset('D')
-    Out[4]: Timestamp('2016-10-30 23:00:00+0200', tz='Europe/Helsinki')
-
-:class:`CalendarDay` and associated frequency alias ``'CD'`` are now available
-and respect calendar day arithmetic while :class:`Day` and frequency alias ``'D'``
-will now respect absolute time (:issue:`22274`, :issue:`20596`, :issue:`16980`, :issue:`8774`)
-See the :ref:`documentation here <timeseries.dayvscalendarday>` for more information.
-
-Addition with :class:`CalendarDay` across a daylight savings time transition:
-
-.. ipython:: python
-
-   ts = pd.Timestamp('2016-10-30 00:00:00', tz='Europe/Helsinki')
-   ts + pd.offsets.Day(1)
-   ts + pd.offsets.CalendarDay(1)
-
 .. _whatsnew_0240.api_breaking.period_end_time:
 
 Time values in ``dt.end_time`` and ``to_timestamp(how='end')``
@@ -1132,7 +1092,7 @@ Deprecations
 - Passing a string alias like ``'datetime64[ns, UTC]'`` as the `unit` parameter to :class:`DatetimeTZDtype` is deprecated. Use :class:`DatetimeTZDtype.construct_from_string` instead (:issue:`23990`).
 - In :meth:`Series.where` with Categorical data, providing an ``other`` that is not present in the categories is deprecated. Convert the categorical to a different dtype or add the ``other`` to the categories first (:issue:`24077`).
 - :meth:`Series.clip_lower`, :meth:`Series.clip_upper`, :meth:`DataFrame.clip_lower` and :meth:`DataFrame.clip_upper` are deprecated and will be removed in a future version. Use ``Series.clip(lower=threshold)``, ``Series.clip(upper=threshold)`` and the equivalent ``DataFrame`` methods (:issue:`24203`)
-
+- Operations where the offset alias ``'D'`` or :class:`Day` acts as a fixed, absolute duration of 24 hours are deprecated. This includes :class:`Timestamp` arithmetic and all operations with :class:`Timedelta`. :class:`Day` will always represent a calendar day in a future version (:issue:`22864`)
 
 .. _whatsnew_0240.deprecations.datetimelike_int_ops:
 

pandas/_libs/tslibs/timestamps.pyx

@@ -360,6 +360,9 @@ cdef class _Timestamp(datetime):
 
         elif PyDelta_Check(other) or hasattr(other, 'delta'):
             # delta --> offsets.Tick
+            if self.tz is not None and getattr(other, '_prefix') == 'D':
+                warnings.warn("Day arithmetic will respect calendar day in a"
+                              "future release", DeprecationWarning)
             nanos = delta_to_nanoseconds(other)
             result = Timestamp(self.value + nanos,
                                tz=self.tzinfo, freq=self.freq)

pandas/core/arrays/datetimes.py

@@ -27,7 +27,7 @@
 import pandas.core.common as com
 
 from pandas.tseries.frequencies import get_period_alias, to_offset
-from pandas.tseries.offsets import Tick, generate_range
+from pandas.tseries.offsets import Day, Tick, _Day, generate_range
 
 _midnight = time(0, 0)
 
@@ -289,6 +289,9 @@ def _generate_range(cls, start, end, periods, freq, tz=None,
                 end, end.tz, start.tz, freq, tz
             )
         if freq is not None:
+            # TODO: Remove when _Day replaces Day
+            if isinstance(freq, Day) and tz is not None:
+                freq = _Day(freq.n)
             # TODO: consider re-implementing _cached_range; GH#17914
             index = _generate_regular_range(cls, start, end, periods, freq)
 
@@ -322,6 +325,9 @@ def _generate_range(cls, start, end, periods, freq, tz=None,
         if not right_closed and len(index) and index[-1] == end:
             index = index[:-1]
 
+        # TODO: Remove when _Day replaces Day
+        if isinstance(freq, _Day):
+            freq = Day(freq.n)
         return cls._simple_new(index.asi8, freq=freq, tz=tz)
 
     # -----------------------------------------------------------------

pandas/core/indexes/datetimes.py

@@ -1353,7 +1353,7 @@ def date_range(start=None, end=None, periods=None, freq=None, tz=None,
         Right bound for generating dates.
     periods : integer, optional
         Number of periods to generate.
-    freq : str or DateOffset, default 'D'
+    freq : str or DateOffset, default 'D' (calendar daily)
         Frequency strings can have multiples, e.g. '5H'. See
         :ref:`here <timeseries.offset_aliases>` for a list of
         frequency aliases.

pandas/core/indexes/interval.py

@@ -1174,7 +1174,7 @@ def interval_range(start=None, end=None, periods=None, freq=None,
     freq : numeric, string, or DateOffset, default None
         The length of each interval. Must be consistent with the type of start
         and end, e.g. 2 for numeric, or '5H' for datetime-like.  Default is 1
-        for numeric and 'D' for datetime-like.
+        for numeric and 'D' (calendar daily) for datetime-like.
     name : string, default None
         Name of the resulting IntervalIndex
     closed : {'left', 'right', 'both', 'neither'}, default 'right'

pandas/core/indexes/period.py

@@ -984,7 +984,7 @@ def period_range(start=None, end=None, periods=None, freq='D', name=None):
         Right bound for generating periods
     periods : integer, default None
         Number of periods to generate
-    freq : string or DateOffset, default 'D'
+    freq : string or DateOffset, default 'D' (calendar daily)
         Frequency alias
     name : string, default None
         Name of the resulting PeriodIndex

pandas/core/resample.py

@@ -1589,8 +1589,16 @@ def _get_range_edges(first, last, offset, closed='left', base=0):
 
         # #1165 and #24127
         if (is_day and not offset.nanos % day_nanos) or not is_day:
-            return _adjust_dates_anchored(first, last, offset,
-                                          closed=closed, base=base)
+            first, last = _adjust_dates_anchored(first, last, offset,
+                                                 closed=closed, base=base)
+            # TODO: Remove when _Day replaces Day and just return first, last
+            if offset == 'D' and first.tz is not None:
+                # We need to make Tick 'D' flexible to DST (23H, 24H, or 25H)
+                # _adjust_dates_anchored assumes 'D' means 24H, so ensure
+                # first and last snap to midnight.
+                first = first.normalize()
+                last = last.normalize()
+            return first, last
 
     else:
         first = first.normalize()

pandas/tests/indexes/datetimes/test_date_range.py

@@ -351,18 +351,18 @@ def test_range_tz_pytz(self):
          Timestamp(datetime(2013, 11, 6), tz='US/Eastern')]
     ])
     def test_range_tz_dst_straddle_pytz(self, start, end):
-        dr = date_range(start, end, freq='CD')
+        dr = date_range(start, end, freq='D')
         assert dr[0] == start
         assert dr[-1] == end
         assert np.all(dr.hour == 0)
 
-        dr = date_range(start, end, freq='CD', tz='US/Eastern')
+        dr = date_range(start, end, freq='D', tz='US/Eastern')
         assert dr[0] == start
         assert dr[-1] == end
         assert np.all(dr.hour == 0)
 
         dr = date_range(start.replace(tzinfo=None), end.replace(
-            tzinfo=None), freq='CD', tz='US/Eastern')
+            tzinfo=None), freq='D', tz='US/Eastern')
         assert dr[0] == start
         assert dr[-1] == end
         assert np.all(dr.hour == 0)
@@ -596,9 +596,9 @@ def test_mismatching_tz_raises_err(self, start, end):
         with pytest.raises(TypeError):
             pd.date_range(start, end, freq=BDay())
 
-    def test_CalendarDay_range_with_dst_crossing(self):
+    def test_date_range_with_dst_crossing(self):
         # GH 20596
-        result = date_range('2018-10-23', '2018-11-06', freq='7CD',
+        result = date_range('2018-10-23', '2018-11-06', freq='7D',
                             tz='Europe/Paris')
         expected = date_range('2018-10-23', '2018-11-06',
                               freq=pd.DateOffset(days=7), tz='Europe/Paris')
@@ -758,8 +758,7 @@ def test_cdaterange_weekmask_and_holidays(self):
                         holidays=['2013-05-01'])
 
     @pytest.mark.parametrize('freq', [freq for freq in prefix_mapping
-                                      if freq.startswith('C')
-                                      and freq != 'CD'])  # CalendarDay
+                                      if freq.startswith('C')])
     def test_all_custom_freq(self, freq):
         # should not raise
         bdate_range(START, END, freq=freq, weekmask='Mon Wed Fri',

pandas/tests/indexes/datetimes/test_indexing.py

@@ -134,47 +134,46 @@ def test_where_tz(self):
 
 
 class TestTake(object):
-    def test_take(self):
-        # GH#10295
-        idx1 = pd.date_range('2011-01-01', '2011-01-31', freq='D', name='idx')
-        idx2 = pd.date_range('2011-01-01', '2011-01-31', freq='D',
-                             tz='Asia/Tokyo', name='idx')
-
-        for idx in [idx1, idx2]:
-            result = idx.take([0])
-            assert result == Timestamp('2011-01-01', tz=idx.tz)
 
-            result = idx.take([0, 1, 2])
-            expected = pd.date_range('2011-01-01', '2011-01-03', freq='D',
-                                     tz=idx.tz, name='idx')
-            tm.assert_index_equal(result, expected)
-            assert result.freq == expected.freq
+    @pytest.mark.parametrize('tz', [None, 'Asia/Tokyo'])
+    def test_take(self, tz):
+        # GH#10295
+        idx = pd.date_range('2011-01-01', '2011-01-31', freq='D', name='idx',
+                            tz=tz)
+        result = idx.take([0])
+        assert result == Timestamp('2011-01-01', tz=idx.tz)
+
+        result = idx.take([0, 1, 2])
+        expected = pd.date_range('2011-01-01', '2011-01-03', freq='D',
+                                 tz=idx.tz, name='idx')
+        tm.assert_index_equal(result, expected)
+        assert result.freq == expected.freq
 
-            result = idx.take([0, 2, 4])
-            expected = pd.date_range('2011-01-01', '2011-01-05', freq='2D',
-                                     tz=idx.tz, name='idx')
-            tm.assert_index_equal(result, expected)
-            assert result.freq == expected.freq
+        result = idx.take([0, 2, 4])
+        expected = pd.date_range('2011-01-01', '2011-01-05', freq='2D',
+                                 tz=idx.tz, name='idx')
+        tm.assert_index_equal(result, expected)
+        assert result.freq == expected.freq
 
-            result = idx.take([7, 4, 1])
-            expected = pd.date_range('2011-01-08', '2011-01-02', freq='-3D',
-                                     tz=idx.tz, name='idx')
-            tm.assert_index_equal(result, expected)
-            assert result.freq == expected.freq
+        result = idx.take([7, 4, 1])
+        expected = pd.date_range('2011-01-08', '2011-01-02', freq='-3D',
+                                 tz=idx.tz, name='idx')
+        tm.assert_index_equal(result, expected)
+        assert result.freq == expected.freq
 
-            result = idx.take([3, 2, 5])
-            expected = DatetimeIndex(['2011-01-04', '2011-01-03',
-                                      '2011-01-06'],
-                                     freq=None, tz=idx.tz, name='idx')
-            tm.assert_index_equal(result, expected)
-            assert result.freq is None
+        result = idx.take([3, 2, 5])
+        expected = DatetimeIndex(['2011-01-04', '2011-01-03',
+                                  '2011-01-06'],
+                                 freq=None, tz=idx.tz, name='idx')
+        tm.assert_index_equal(result, expected)
+        assert result.freq is None
 
-            result = idx.take([-3, 2, 5])
-            expected = DatetimeIndex(['2011-01-29', '2011-01-03',
-                                      '2011-01-06'],
-                                     freq=None, tz=idx.tz, name='idx')
-            tm.assert_index_equal(result, expected)
-            assert result.freq is None
+        result = idx.take([-3, 2, 5])
+        expected = DatetimeIndex(['2011-01-29', '2011-01-03',
+                                  '2011-01-06'],
+                                 freq=None, tz=idx.tz, name='idx')
+        tm.assert_index_equal(result, expected)
+        assert result.freq is None
 
     def test_take_invalid_kwargs(self):
         idx = pd.date_range('2011-01-01', '2011-01-31', freq='D', name='idx')

pandas/tests/indexes/datetimes/test_timezones.py

@@ -436,7 +436,7 @@ def test_dti_tz_localize_utc_conversion(self, tz):
 
     @pytest.mark.parametrize('idx', [
         date_range(start='2014-01-01', end='2014-12-31', freq='M'),
-        date_range(start='2014-01-01', end='2014-12-31', freq='CD'),
+        date_range(start='2014-01-01', end='2014-12-31', freq='D'),
         date_range(start='2014-01-01', end='2014-03-01', freq='H'),
         date_range(start='2014-08-01', end='2014-10-31', freq='T')
     ])
@@ -1072,7 +1072,7 @@ def test_date_range_span_dst_transition(self, tzstr):
 
         dr = date_range('2012-11-02', periods=10, tz=tzstr)
         result = dr.hour
-        expected = Index([0, 0, 0, 23, 23, 23, 23, 23, 23, 23])
+        expected = Index([0] * 10)
         tm.assert_index_equal(result, expected)
 
     @pytest.mark.parametrize('tzstr', ['US/Eastern', 'dateutil/US/Eastern'])

pandas/tests/indexes/timedeltas/test_timedelta_range.py

@@ -4,7 +4,7 @@
 import pandas as pd
 import pandas.util.testing as tm
 from pandas import timedelta_range, to_timedelta
-from pandas.tseries.offsets import Day, Second
+from pandas.tseries.offsets import Second
 
 
 class TestTimedeltas(object):
@@ -19,7 +19,8 @@ def test_timedelta_range(self):
         result = timedelta_range('0 days', '10 days', freq='D')
         tm.assert_index_equal(result, expected)
 
-        expected = to_timedelta(np.arange(5), unit='D') + Second(2) + Day()
+        expected = to_timedelta(np.arange(5), unit='D')
+        expected = expected + Second(2) + pd.Timedelta(days=1)
         result = timedelta_range('1 days, 00:00:02', '5 days, 00:00:02',
                                  freq='D')
         tm.assert_index_equal(result, expected)
@@ -49,9 +50,7 @@ def test_timedelta_range(self):
         result = df.loc['0s':, :]
         tm.assert_frame_equal(expected, result)
 
-        with pytest.raises(ValueError):
-            # GH 22274: CalendarDay is a relative time measurement
-            timedelta_range('1day', freq='CD', periods=2)
+        timedelta_range('1day', freq='D', periods=2)
 
     @pytest.mark.parametrize('periods, freq', [
         (3, '2D'), (5, 'D'), (6, '19H12T'), (7, '16H'), (9, '12H')])

pandas/tests/resample/test_datetime_index.py

@@ -1278,7 +1278,7 @@ def test_resample_dst_anchor(self):
         # 5172
         dti = DatetimeIndex([datetime(2012, 11, 4, 23)], tz='US/Eastern')
         df = DataFrame([5], index=dti)
-        assert_frame_equal(df.resample(rule='CD').sum(),
+        assert_frame_equal(df.resample(rule='D').sum(),
                            DataFrame([5], index=df.index.normalize()))
         df.resample(rule='MS').sum()
         assert_frame_equal(
@@ -1332,14 +1332,14 @@ def test_resample_dst_anchor(self):
 
         df_daily = df['10/26/2013':'10/29/2013']
         assert_frame_equal(
-            df_daily.resample("CD").agg({"a": "min", "b": "max", "c": "count"})
+            df_daily.resample("D").agg({"a": "min", "b": "max", "c": "count"})
             [["a", "b", "c"]],
             DataFrame({"a": [1248, 1296, 1346, 1394],
                        "b": [1295, 1345, 1393, 1441],
                        "c": [48, 50, 48, 48]},
                       index=date_range('10/26/2013', '10/29/2013',
-                                       freq='CD', tz='Europe/Paris')),
-            'CD Frequency')
+                                       freq='D', tz='Europe/Paris')),
+            'D Frequency')
 
     def test_downsample_across_dst(self):
         # GH 8531

pandas/tests/resample/test_period_index.py

@@ -276,6 +276,8 @@ def test_with_local_timezone_dateutil(self):
         expected = Series(1, index=expected_index)
         assert_series_equal(result, expected)
 
+    @pytest.mark.xfail(reason='Day as calendar day will raise on '
+                              'NonExistentTimeError')
     def test_resample_nonexistent_time_bin_edge(self):
         # GH 19375
         index = date_range('2017-03-12', '2017-03-12 1:45:00', freq='15T')