Skip to content

API: Implement set_freq for DTI/TDI, deprecate freq setter #20892

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions doc/source/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1775,6 +1775,7 @@ Conversion
.. autosummary::
:toctree: generated/

DatetimeIndex.set_freq
DatetimeIndex.to_period
DatetimeIndex.to_perioddelta
DatetimeIndex.to_pydatetime
Expand Down Expand Up @@ -1808,6 +1809,7 @@ Conversion
.. autosummary::
:toctree: generated/

TimedeltaIndex.set_freq
TimedeltaIndex.to_pytimedelta
TimedeltaIndex.to_series
TimedeltaIndex.round
Expand Down
2 changes: 2 additions & 0 deletions doc/source/whatsnew/v0.23.0.txt
Original file line number Diff line number Diff line change
Expand Up @@ -524,6 +524,7 @@ Other Enhancements
- Added new writer for exporting Stata dta files in version 117, ``StataWriter117``. This format supports exporting strings with lengths up to 2,000,000 characters (:issue:`16450`)
- :func:`to_hdf` and :func:`read_hdf` now accept an ``errors`` keyword argument to control encoding error handling (:issue:`20835`)
- :func:`date_range` now returns a linearly spaced ``DatetimeIndex`` if ``start``, ``stop``, and ``periods`` are specified, but ``freq`` is not. (:issue:`20808`)
- :meth:`DatetimeIndex.set_freq` and :meth:`TimedeltaIndex.set_freq` are now available for setting the ``.freq`` attribute (:issue:`20886`)

.. _whatsnew_0230.api_breaking:

Expand Down Expand Up @@ -998,6 +999,7 @@ Deprecations
- Setting ``PeriodIndex.freq`` (which was not guaranteed to work correctly) is deprecated. Use :meth:`PeriodIndex.asfreq` instead (:issue:`20678`)
- ``Index.get_duplicates()`` is deprecated and will be removed in a future version (:issue:`20239`)
- The previous default behavior of negative indices in ``Categorical.take`` is deprecated. In a future version it will change from meaning missing values to meaning positional indices from the right. The future behavior is consistent with :meth:`Series.take` (:issue:`20664`).
- Setting the ``.freq`` attribute is deprecated for :class:`DatetimeIndex` and :class:`TimedeltaIndex`. Use the associated ``.set_freq()`` method instead (:issue:`20886`)


.. _whatsnew_0230.prior_deprecations:
Expand Down
32 changes: 32 additions & 0 deletions pandas/core/indexes/datetimelike.py
Original file line number Diff line number Diff line change
Expand Up @@ -236,12 +236,44 @@ def freq(self):

@freq.setter
def freq(self, value):
msg = ('Setting {obj}.freq has been deprecated and will be removed '
'in a future version; use {obj}.set_freq instead.'
).format(obj=type(self).__name__)
warnings.warn(msg, FutureWarning, stacklevel=2)
if value is not None:
value = frequencies.to_offset(value)
self._validate_frequency(self, value)

self._freq = value

def set_freq(self, freq):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you make this available on the .dt accessors as well

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This currently does nothing when used with the dt accessor due to #20937; the underlying frequency gets changed with set_freq, but upon Series construction the new frequency gets lost.

I've pushed a commit to add this to the dt accessor, and xfailed the corresponding tests I've added. Can revert the commit if this is no longer deemed important.

"""
Set the frequency of the DatetimeIndex or TimedeltaIndex to the
specified frequency `freq`.

Parameters
----------
freq: str or Offset
The frequency to set on the DatetimeIndex or TimedeltaIndex

Returns
-------
new: DatetimeIndex or TimedeltaIndex with the new frequency

Raises
------
ValueError
If the values of the DatetimeIndex or TimedeltaIndex are not
compatible with the new frequency
"""
if freq is not None:
freq = frequencies.to_offset(freq)
self._validate_frequency(self, freq)

new = self.copy()
new._freq = freq
return new


class DatetimeIndexOpsMixin(object):
""" common ops mixin to support a unified interface datetimelike Index """
Expand Down
25 changes: 15 additions & 10 deletions pandas/core/indexes/datetimes.py
Original file line number Diff line number Diff line change
Expand Up @@ -250,6 +250,7 @@ class DatetimeIndex(DatelikeOps, TimelikeOps, DatetimeIndexOpsMixin,
normalize
strftime
snap
set_freq
tz_convert
tz_localize
round
Expand Down Expand Up @@ -319,7 +320,7 @@ def _add_comparison_methods(cls):
_other_ops = ['date', 'time']
_datetimelike_ops = _field_ops + _object_ops + _bool_ops + _other_ops
_datetimelike_methods = ['to_period', 'tz_localize',
'tz_convert',
'tz_convert', 'set_freq',
'normalize', 'strftime', 'round', 'floor',
'ceil', 'month_name', 'day_name']

Expand Down Expand Up @@ -460,7 +461,7 @@ def __new__(cls, data=None,
if freq_infer:
inferred = subarr.inferred_freq
if inferred:
subarr.freq = to_offset(inferred)
subarr._freq = to_offset(inferred)

return subarr._deepcopy_if_needed(ref_to_data, copy)

Expand Down Expand Up @@ -759,7 +760,7 @@ def _cached_range(cls, start=None, end=None, periods=None, freq=None,
arr = tools.to_datetime(list(xdr), box=False)

cachedRange = DatetimeIndex._simple_new(arr)
cachedRange.freq = freq
cachedRange._freq = freq
cachedRange = cachedRange.tz_localize(None)
cachedRange.name = None
drc[freq] = cachedRange
Expand Down Expand Up @@ -794,7 +795,7 @@ def _cached_range(cls, start=None, end=None, periods=None, freq=None,

indexSlice = cachedRange[startLoc:endLoc]
indexSlice.name = name
indexSlice.freq = freq
indexSlice._freq = freq

return indexSlice

Expand Down Expand Up @@ -1184,7 +1185,7 @@ def union(self, other):
result._tz = timezones.tz_standardize(this.tz)
if (result.freq is None and
(this.freq is not None or other.freq is not None)):
result.freq = to_offset(result.inferred_freq)
result._freq = to_offset(result.inferred_freq)
return result

def to_perioddelta(self, freq):
Expand Down Expand Up @@ -1232,7 +1233,7 @@ def union_many(self, others):
this._tz = timezones.tz_standardize(tz)

if this.freq is None:
this.freq = to_offset(this.inferred_freq)
this._freq = to_offset(this.inferred_freq)
return this

def join(self, other, how='left', level=None, return_indexers=False,
Expand Down Expand Up @@ -1393,7 +1394,7 @@ def intersection(self, other):
result = Index.intersection(self, other)
if isinstance(result, DatetimeIndex):
if result.freq is None:
result.freq = to_offset(result.inferred_freq)
result._freq = to_offset(result.inferred_freq)
return result

elif (other.freq is None or self.freq is None or
Expand All @@ -1404,7 +1405,7 @@ def intersection(self, other):
result = self._shallow_copy(result._values, name=result.name,
tz=result.tz, freq=None)
if result.freq is None:
result.freq = to_offset(result.inferred_freq)
result._freq = to_offset(result.inferred_freq)
return result

if len(self) == 0:
Expand Down Expand Up @@ -1738,9 +1739,13 @@ def offset(self):
def offset(self, value):
"""get/set the frequency of the Index"""
msg = ('DatetimeIndex.offset has been deprecated and will be removed '
'in a future version; use DatetimeIndex.freq instead.')
'in a future version; use DatetimeIndex.set_freq instead.')
warnings.warn(msg, FutureWarning, stacklevel=2)
self.freq = value
if value is not None:
value = to_offset(value)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

call set_freq here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is for the deprecated offset setter which is just an alias for the freq setter, so I don't think set_freq should be used here since setting is to happen inplace (as was the previous convention). This previously would call the freq setter, but since that's also being deprecated it was subsequently raising the freq setter warning too, so I just reused the freq setter code to avoid the additional warning.

self._validate_frequency(self, value)

self._freq = value

year = _field_accessor('year', 'Y', "The year of the datetime")
month = _field_accessor('month', 'M',
Expand Down
9 changes: 5 additions & 4 deletions pandas/core/indexes/timedeltas.py
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,7 @@ class TimedeltaIndex(DatetimeIndexOpsMixin, TimelikeOps, Int64Index):

Methods
-------
set_freq
to_pytimedelta
to_series
round
Expand Down Expand Up @@ -177,7 +178,7 @@ def _join_i8_wrapper(joinf, **kwargs):
_field_ops = ['days', 'seconds', 'microseconds', 'nanoseconds']
_datetimelike_ops = _field_ops + _object_ops + _bool_ops
_datetimelike_methods = ["to_pytimedelta", "total_seconds",
"round", "floor", "ceil"]
"round", "floor", "ceil", "set_freq"]

@classmethod
def _add_comparison_methods(cls):
Expand Down Expand Up @@ -253,14 +254,14 @@ def __new__(cls, data=None, unit=None, freq=None, start=None, end=None,
if freq is not None and not freq_infer:
index = cls._simple_new(data, name=name)
cls._validate_frequency(index, freq)
index.freq = freq
index._freq = freq
return index

if freq_infer:
index = cls._simple_new(data, name=name)
inferred = index.inferred_freq
if inferred:
index.freq = to_offset(inferred)
index._freq = to_offset(inferred)
return index

return cls._simple_new(data, name=name, freq=freq)
Expand Down Expand Up @@ -598,7 +599,7 @@ def union(self, other):
result = Index.union(this, other)
if isinstance(result, TimedeltaIndex):
if result.freq is None:
result.freq = to_offset(result.inferred_freq)
result._freq = to_offset(result.inferred_freq)
return result

def join(self, other, how='left', level=None, return_indexers=False,
Expand Down
2 changes: 1 addition & 1 deletion pandas/core/resample.py
Original file line number Diff line number Diff line change
Expand Up @@ -904,7 +904,7 @@ def _downsample(self, how, **kwargs):
if not len(ax):
# reset to the new freq
obj = obj.copy()
obj.index.freq = self.freq
obj.index = obj.index.set_freq(self.freq)
return obj

# do we have a regular frequency
Expand Down
49 changes: 22 additions & 27 deletions pandas/tests/categorical/test_constructors.py
Original file line number Diff line number Diff line change
Expand Up @@ -256,36 +256,31 @@ def test_constructor_with_generator(self):
cat = Categorical([0, 1, 2], categories=xrange(3))
tm.assert_categorical_equal(cat, exp)

def test_constructor_with_datetimelike(self):

@pytest.mark.parametrize('dtl', [
date_range('1995-01-01', periods=5, freq='s'),
date_range('1995-01-01', periods=5, freq='s', tz='US/Eastern'),
timedelta_range('1 day', periods=5, freq='s')])
def test_constructor_with_datetimelike(self, dtl):
# 12077
# constructor wwth a datetimelike and NaT
s = Series(dtl)
c = Categorical(s)
expected = dtl._constructor(s).set_freq(None)
tm.assert_index_equal(c.categories, expected)
tm.assert_numpy_array_equal(c.codes, np.arange(5, dtype='int8'))

# with NaT
s2 = s.copy()
s2.iloc[-1] = NaT
c = Categorical(s2)
expected = dtl._constructor(s2.dropna()).set_freq(None)
tm.assert_index_equal(c.categories, expected)

exp = np.array([0, 1, 2, 3, -1], dtype=np.int8)
tm.assert_numpy_array_equal(c.codes, exp)

for dtl in [date_range('1995-01-01 00:00:00', periods=5, freq='s'),
date_range('1995-01-01 00:00:00', periods=5,
freq='s', tz='US/Eastern'),
timedelta_range('1 day', periods=5, freq='s')]:

s = Series(dtl)
c = Categorical(s)
expected = type(dtl)(s)
expected.freq = None
tm.assert_index_equal(c.categories, expected)
tm.assert_numpy_array_equal(c.codes, np.arange(5, dtype='int8'))

# with NaT
s2 = s.copy()
s2.iloc[-1] = NaT
c = Categorical(s2)
expected = type(dtl)(s2.dropna())
expected.freq = None
tm.assert_index_equal(c.categories, expected)

exp = np.array([0, 1, 2, 3, -1], dtype=np.int8)
tm.assert_numpy_array_equal(c.codes, exp)

result = repr(c)
assert 'NaT' in result
result = repr(c)
assert 'NaT' in result

def test_constructor_from_index_series_datetimetz(self):
idx = date_range('2015-01-01 10:00', freq='D', periods=3,
Expand Down
14 changes: 13 additions & 1 deletion pandas/tests/indexes/datetimelike.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ def test_map_dictlike(self, mapper):

# don't compare the freqs
if isinstance(expected, pd.DatetimeIndex):
expected.freq = None
expected = expected.set_freq(None)

result = self.index.map(mapper(expected, self.index))
tm.assert_index_equal(result, expected)
Expand All @@ -88,3 +88,15 @@ def test_asobject_deprecated(self):
with tm.assert_produces_warning(FutureWarning):
i = d.asobject
assert isinstance(i, pd.Index)

def test_freq_setter_deprecated(self):
# GH 20678/20886
idx = self.create_index()

# no warning for getter
with tm.assert_produces_warning(None):
idx.freq

# warning for setter
with tm.assert_produces_warning(FutureWarning):
idx.freq = pd.offsets.Day()
4 changes: 2 additions & 2 deletions pandas/tests/indexes/datetimes/test_date_range.py
Original file line number Diff line number Diff line change
Expand Up @@ -396,6 +396,7 @@ def test_misc(self):
assert len(dr) == 20
assert dr[0] == firstDate
assert dr[-1] == end
assert dr.freq == BDay()

def test_date_parse_failure(self):
badly_formed_date = '2007/100/1'
Expand All @@ -416,7 +417,6 @@ def test_daterange_bug_456(self):
# GH #456
rng1 = bdate_range('12/5/2011', '12/5/2011')
rng2 = bdate_range('12/2/2011', '12/5/2011')
rng2.freq = BDay()
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is redundant, as bdate_range should already set this as the frequency. I couldn't find a test that verified this though, so I added a check to a test a few lines above. Same deal with the CDay() changes below.


result = rng1.union(rng2)
assert isinstance(result, DatetimeIndex)
Expand Down Expand Up @@ -658,12 +658,12 @@ def test_misc(self):
assert len(dr) == 20
assert dr[0] == firstDate
assert dr[-1] == end
assert dr.freq == CDay()

def test_daterange_bug_456(self):
# GH #456
rng1 = bdate_range('12/5/2011', '12/5/2011', freq='C')
rng2 = bdate_range('12/2/2011', '12/5/2011', freq='C')
rng2.freq = CDay()

result = rng1.union(rng2)
assert isinstance(result, DatetimeIndex)
Expand Down
Loading