Skip to content

CLN: enforce deprecation of frequencies deprecated for offsets #57986

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
10f8ba6
enforce deprecation of offset deprecated freqstr
natmokval Mar 24, 2024
428fd06
fix tests
natmokval Mar 25, 2024
363395a
fix mypy error
natmokval Mar 25, 2024
bba27f4
Merge branch 'main' into depr-enforce-deprecation-offset-depr-freqstr
natmokval Mar 25, 2024
d6092b3
add a note to v3.0.0
natmokval Mar 25, 2024
f5fd5ff
remove c_REVERSE_OFFSET_REMOVED_FREQSTR, correct tests
natmokval Mar 28, 2024
6a124ae
fix test_to_period_offsets_not_supported
natmokval Mar 28, 2024
19f35cc
correct to_offset, fix tests
natmokval Apr 1, 2024
8b9edf4
add dict PERIOD_TO_OFFSET_FREQSTR, corect meth to_offset, fix tests
natmokval Apr 12, 2024
df65a84
Merge branch 'main' into depr-enforce-deprecation-offset-depr-freqstr
natmokval Apr 12, 2024
30b7765
add a comment
natmokval Apr 12, 2024
49af3e7
Merge branch 'main' into depr-enforce-deprecation-offset-depr-freqstr
natmokval May 2, 2024
81fe8f0
create dictionary PERIOD_AND_OFFSET_ALIASES
natmokval May 2, 2024
727c6ae
correct def to_offset
natmokval May 2, 2024
2ae8533
Merge branch 'main' into depr-enforce-deprecation-offset-depr-freqstr
natmokval May 15, 2024
c730be0
fixup
natmokval May 17, 2024
81ad386
fixup
natmokval May 21, 2024
dcc1c92
replace c_OFFSET_RENAMED_FREQSTR with c_PERIOD_TO_OFFSET_FREQSTR
natmokval May 31, 2024
5616c3d
add cimport c_PERIOD_TO_OFFSET_FREQSTR
natmokval May 31, 2024
163d05b
Merge branch 'main' into depr-enforce-deprecation-offset-depr-freqstr
natmokval May 31, 2024
be6c0bc
resolve conflict
natmokval Jun 1, 2024
74ae136
Merge branch 'depr-enforce-deprecation-offset-depr-freqstr' of https:…
natmokval Jun 1, 2024
d7a5037
add a helper function for error reporting
natmokval Jun 1, 2024
d9b3a5e
minor whatsnew fix
MarcoGorelli Jun 1, 2024
46bcd86
Merge remote-tracking branch 'upstream/main' into depr-enforce-deprec…
MarcoGorelli Jun 6, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 29 additions & 1 deletion doc/source/whatsnew/v3.0.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -200,6 +200,34 @@ Other Deprecations

Removal of prior version deprecations/changes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Enforced deprecation of aliases ``M``, ``Q``, ``Y``, etc. in favour of ``ME``, ``QE``, ``YE``, etc. for offsets
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Removed the following frequency aliases (:issue:`57986`):

+-------------------------------+------------------+------------------+
|offsets |removed aliases |new aliases |
+===============================+==================+==================+
|:class:`MonthEnd` | ``M`` | ``ME`` |
+-------------------------------+------------------+------------------+
|:class:`BusinessMonthEnd` | ``BM`` | ``BME`` |
+-------------------------------+------------------+------------------+
|:class:`SemiMonthEnd` | ``SM`` | ``SME`` |
+-------------------------------+------------------+------------------+
|:class:`CustomBusinessMonthEnd`| ``CBM`` | ``CBME`` |
+-------------------------------+------------------+------------------+
|:class:`QuarterEnd` | ``Q`` | ``QE`` |
+-------------------------------+------------------+------------------+
|:class:`BQuarterEnd` | ``BQ`` | ``BQE`` |
+-------------------------------+------------------+------------------+
|:class:`YearEnd` | ``Y`` | ``YE`` |
+-------------------------------+------------------+------------------+
|:class:`BYearEnd` | ``BY`` | ``BYE`` |
+-------------------------------+------------------+------------------+

Other Removal
^^^^^^^^^^^^^
- :class:`.DataFrameGroupBy.idxmin`, :class:`.DataFrameGroupBy.idxmax`, :class:`.SeriesGroupBy.idxmin`, and :class:`.SeriesGroupBy.idxmax` will now raise a ``ValueError`` when used with ``skipna=False`` and an NA value is encountered (:issue:`10694`)
- :func:`concat` no longer ignores empty objects when determining output dtypes (:issue:`39122`)
- :func:`read_excel`, :func:`read_json`, :func:`read_html`, and :func:`read_xml` no longer accept raw string or byte representation of the data. That type of data must be wrapped in a :py:class:`StringIO` or :py:class:`BytesIO` (:issue:`53767`)
Expand Down Expand Up @@ -241,7 +269,7 @@ Removal of prior version deprecations/changes
- Enforced deprecation of string ``A`` denoting frequency in :class:`YearEnd` and strings ``A-DEC``, ``A-JAN``, etc. denoting annual frequencies with various fiscal year ends (:issue:`57699`)
- Enforced deprecation of string ``BAS`` denoting frequency in :class:`BYearBegin` and strings ``BAS-DEC``, ``BAS-JAN``, etc. denoting annual frequencies with various fiscal year starts (:issue:`57793`)
- Enforced deprecation of string ``BA`` denoting frequency in :class:`BYearEnd` and strings ``BA-DEC``, ``BA-JAN``, etc. denoting annual frequencies with various fiscal year ends (:issue:`57793`)
- Enforced deprecation of strings ``T``, ``L``, ``U``, and ``N`` denoting frequencies in :class:`Minute`, :class:`Second`, :class:`Milli`, :class:`Micro`, :class:`Nano` (:issue:`57627`)
- Enforced deprecation of strings ``T``, ``L``, ``U``, and ``N`` denoting frequencies in :class:`Minute`, :class:`Milli`, :class:`Micro`, :class:`Nano` (:issue:`57627`)
- Enforced deprecation of strings ``T``, ``L``, ``U``, and ``N`` denoting units in :class:`Timedelta` (:issue:`57627`)
- Enforced deprecation of the behavior of :func:`concat` when ``len(keys) != len(objs)`` would truncate to the shorter of the two. Now this raises a ``ValueError`` (:issue:`43485`)
- Enforced deprecation of values "pad", "ffill", "bfill", and "backfill" for :meth:`Series.interpolate` and :meth:`DataFrame.interpolate` (:issue:`57869`)
Expand Down
4 changes: 2 additions & 2 deletions pandas/_libs/tslibs/dtypes.pxd
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ cdef NPY_DATETIMEUNIT get_supported_reso(NPY_DATETIMEUNIT reso)
cdef bint is_supported_unit(NPY_DATETIMEUNIT reso)

cdef dict c_OFFSET_TO_PERIOD_FREQSTR
cdef dict c_OFFSET_DEPR_FREQSTR
cdef dict c_REVERSE_OFFSET_DEPR_FREQSTR
cdef dict c_PERIOD_TO_OFFSET_FREQSTR
cdef dict c_OFFSET_REMOVED_FREQSTR
cdef dict c_DEPR_ABBREVS
cdef dict attrname_to_abbrevs
cdef dict npy_unit_to_attrname
Expand Down
85 changes: 52 additions & 33 deletions pandas/_libs/tslibs/dtypes.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,10 @@ OFFSET_TO_PERIOD_FREQSTR: dict = {
"EOM": "M",
"BME": "M",
"SME": "M",
"BMS": "M",
"CBME": "M",
"CBMS": "M",
"SMS": "M",
"BQS": "Q",
"QS": "Q",
"BQE": "Q",
Expand Down Expand Up @@ -245,7 +249,7 @@ OFFSET_TO_PERIOD_FREQSTR: dict = {
"YS": "Y",
"BYS": "Y",
}
cdef dict c_OFFSET_DEPR_FREQSTR = {
cdef dict c_OFFSET_REMOVED_FREQSTR = {
"M": "ME",
"Q": "QE",
"Q-DEC": "QE-DEC",
Expand Down Expand Up @@ -273,40 +277,55 @@ cdef dict c_OFFSET_DEPR_FREQSTR = {
"Y-SEP": "YE-SEP",
"Y-OCT": "YE-OCT",
"Y-NOV": "YE-NOV",
"BY": "BYE",
"BY-DEC": "BYE-DEC",
"BY-JAN": "BYE-JAN",
"BY-FEB": "BYE-FEB",
"BY-MAR": "BYE-MAR",
"BY-APR": "BYE-APR",
"BY-MAY": "BYE-MAY",
"BY-JUN": "BYE-JUN",
"BY-JUL": "BYE-JUL",
"BY-AUG": "BYE-AUG",
"BY-SEP": "BYE-SEP",
"BY-OCT": "BYE-OCT",
"BY-NOV": "BYE-NOV",
"BM": "BME",
"CBM": "CBME",
"SM": "SME",
"BQ": "BQE",
"BQ-DEC": "BQE-DEC",
"BQ-JAN": "BQE-JAN",
"BQ-FEB": "BQE-FEB",
"BQ-MAR": "BQE-MAR",
"BQ-APR": "BQE-APR",
"BQ-MAY": "BQE-MAY",
"BQ-JUN": "BQE-JUN",
"BQ-JUL": "BQE-JUL",
"BQ-AUG": "BQE-AUG",
"BQ-SEP": "BQE-SEP",
"BQ-OCT": "BQE-OCT",
"BQ-NOV": "BQE-NOV",
}
cdef dict c_OFFSET_TO_PERIOD_FREQSTR = OFFSET_TO_PERIOD_FREQSTR
cdef dict c_REVERSE_OFFSET_DEPR_FREQSTR = {
v: k for k, v in c_OFFSET_DEPR_FREQSTR.items()
PERIOD_TO_OFFSET_FREQSTR = {
"D": "D",
"B": "B",
"W": "W",
"W-SUN": "W-SUN",
"W-MON": "W-MON",
"W-TUE": "W-TUE",
"W-WED": "W-WED",
"W-THU": "W-THU",
"W-FRI": "W-FRI",
"W-SAT": "W-SAT",
"M": "ME",
"Q": "QE",
"Q-DEC": "QE-DEC",
"Q-JAN": "QE-JAN",
"Q-FEB": "QE-FEB",
"Q-MAR": "QE-MAR",
"Q-APR": "QE-APR",
"Q-MAY": "QE-MAY",
"Q-JUN": "QE-JUN",
"Q-JUL": "QE-JUL",
"Q-AUG": "QE-AUG",
"Q-SEP": "QE-SEP",
"Q-OCT": "QE-OCT",
"Q-NOV": "QE-NOV",
"Y": "YE",
"Y-DEC": "YE-DEC",
"Y-JAN": "YE-JAN",
"Y-FEB": "YE-FEB",
"Y-MAR": "YE-MAR",
"Y-APR": "YE-APR",
"Y-MAY": "YE-MAY",
"Y-JUN": "YE-JUN",
"Y-JUL": "YE-JUL",
"Y-AUG": "YE-AUG",
"Y-SEP": "YE-SEP",
"Y-OCT": "YE-OCT",
"Y-NOV": "YE-NOV",
"Min": "min",
"min": "min",
"s": "s",
"ms": "ms",
"us": "us",
"ns": "ns",
"h": "h",
}
cdef dict c_OFFSET_TO_PERIOD_FREQSTR = OFFSET_TO_PERIOD_FREQSTR
cdef dict c_PERIOD_TO_OFFSET_FREQSTR = PERIOD_TO_OFFSET_FREQSTR

# Map deprecated resolution abbreviations to correct resolution abbreviations
cdef dict c_DEPR_ABBREVS = {
Expand Down
66 changes: 21 additions & 45 deletions pandas/_libs/tslibs/offsets.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,9 @@ from pandas._libs.tslibs.ccalendar cimport (
from pandas._libs.tslibs.conversion cimport localize_pydatetime
from pandas._libs.tslibs.dtypes cimport (
c_DEPR_ABBREVS,
c_OFFSET_DEPR_FREQSTR,
c_REVERSE_OFFSET_DEPR_FREQSTR,
c_OFFSET_REMOVED_FREQSTR,
c_OFFSET_TO_PERIOD_FREQSTR,
c_PERIOD_TO_OFFSET_FREQSTR,
periods_per_day,
)
from pandas._libs.tslibs.nattype cimport (
Expand Down Expand Up @@ -4831,54 +4832,29 @@ cpdef to_offset(freq, bint is_period=False):

tups = zip(split[0::4], split[1::4], split[2::4])
for n, (sep, stride, name) in enumerate(tups):
if not is_period and name.upper() in c_OFFSET_DEPR_FREQSTR:
warnings.warn(
f"\'{name}\' is deprecated and will be removed "
f"in a future version, please use "
f"\'{c_OFFSET_DEPR_FREQSTR.get(name.upper())}\' instead.",
FutureWarning,
stacklevel=find_stack_level(),
)
name = c_OFFSET_DEPR_FREQSTR[name.upper()]
if (not is_period and
name != name.upper() and
name.lower() not in {"s", "ms", "us", "ns"} and
name.upper().split("-")[0].endswith(("S", "E"))):
warnings.warn(
f"\'{name}\' is deprecated and will be removed "
f"in a future version, please use "
f"\'{name.upper()}\' instead.",
FutureWarning,
stacklevel=find_stack_level(),
)
name = name.upper()
if is_period and name.upper() in c_REVERSE_OFFSET_DEPR_FREQSTR:
if name.upper().startswith("Y"):
if not is_period:
if name.upper() in c_OFFSET_REMOVED_FREQSTR:
raise ValueError(
f"for Period, please use \'Y{name.upper()[2:]}\' "
f"instead of \'{name}\'"
f"\'{name}\' is no longer supported for offsets. Please "
f"use \'{c_OFFSET_REMOVED_FREQSTR.get(name.upper())}\' "
f"instead."
)
if (name.upper().startswith("B") or
name.upper().startswith("S") or
name.upper().startswith("C")):
# below we raise for lowrecase monthly and bigger frequencies
if (name.upper() != name and
name.lower() not in {"h", "min", "s", "ms", "us", "ns"} and
name.upper() not in c_PERIOD_TO_OFFSET_FREQSTR and
name.upper() in c_OFFSET_TO_PERIOD_FREQSTR):
raise ValueError(INVALID_FREQ_ERR_MSG.format(name))
else:
if is_period:
if name in c_PERIOD_TO_OFFSET_FREQSTR:
name = c_PERIOD_TO_OFFSET_FREQSTR[name]
elif name in {"d", "b"}:
name = name.upper()
elif (name.upper() not in {"B", "D"} and
not name.upper().startswith("W")):
raise ValueError(
f"for Period, please use "
f"\'{c_REVERSE_OFFSET_DEPR_FREQSTR.get(name.upper())}\' "
f"instead of \'{name}\'"
)
elif is_period and name.upper() in c_OFFSET_DEPR_FREQSTR:
if name.upper() != name:
warnings.warn(
f"\'{name}\' is deprecated and will be removed in "
f"a future version, please use \'{name.upper()}\' "
f"instead.",
FutureWarning,
stacklevel=find_stack_level(),
f"\'{name}\' is not supported as period frequency."
)
name = c_OFFSET_DEPR_FREQSTR.get(name.upper())

if sep != "" and not sep.isspace():
raise ValueError("separator must be spaces")
prefix = _lite_rule_alias.get(name) or name
Expand Down
45 changes: 17 additions & 28 deletions pandas/tests/arrays/test_datetimes.py
Original file line number Diff line number Diff line change
Expand Up @@ -764,29 +764,14 @@ def test_iter_zoneinfo_fold(self, tz):
assert left.utcoffset() == right2.utcoffset()

@pytest.mark.parametrize(
"freq, freq_depr",
[
("2ME", "2M"),
("2SME", "2SM"),
("2SME", "2sm"),
("2QE", "2Q"),
("2QE-SEP", "2Q-SEP"),
("1YE", "1Y"),
("2YE-MAR", "2Y-MAR"),
("2ME", "2m"),
("2QE-SEP", "2q-sep"),
("2YE", "2y"),
],
"freq",
["2M", "2SM", "2sm", "2Q", "2Q-SEP", "1Y", "2Y-MAR", "2m", "2q-sep", "2y"],
)
def test_date_range_frequency_M_Q_Y_A_deprecated(self, freq, freq_depr):
# GH#9586, GH#54275
depr_msg = f"'{freq_depr[1:]}' is deprecated and will be removed "
f"in a future version, please use '{freq[1:]}' instead."
def test_date_range_frequency_M_Q_Y_raises(self, freq):
msg = f"Invalid frequency: {freq}"

expected = pd.date_range("1/1/2000", periods=4, freq=freq)
with tm.assert_produces_warning(FutureWarning, match=depr_msg):
result = pd.date_range("1/1/2000", periods=4, freq=freq_depr)
tm.assert_index_equal(result, expected)
with pytest.raises(ValueError, match=msg):
pd.date_range("1/1/2000", periods=4, freq=freq)

@pytest.mark.parametrize("freq_depr", ["2H", "2CBH", "2MIN", "2S", "2mS", "2Us"])
def test_date_range_uppercase_frequency_deprecated(self, freq_depr):
Expand All @@ -800,7 +785,7 @@ def test_date_range_uppercase_frequency_deprecated(self, freq_depr):
tm.assert_index_equal(result, expected)

@pytest.mark.parametrize(
"freq_depr",
"freq",
[
"2ye-mar",
"2ys",
Expand All @@ -811,17 +796,21 @@ def test_date_range_uppercase_frequency_deprecated(self, freq_depr):
"2bms",
"2cbme",
"2me",
"2w",
],
)
def test_date_range_lowercase_frequency_deprecated(self, freq_depr):
def test_date_range_lowercase_frequency_raises(self, freq):
msg = f"Invalid frequency: {freq}"

with pytest.raises(ValueError, match=msg):
pd.date_range("1/1/2000", periods=4, freq=freq)

def test_date_range_lowercase_frequency_deprecated(self):
# GH#9586, GH#54939
depr_msg = f"'{freq_depr[1:]}' is deprecated and will be removed in a "
f"future version, please use '{freq_depr.upper()[1:]}' instead."
depr_msg = "'w' is deprecated and will be removed in a future version"

expected = pd.date_range("1/1/2000", periods=4, freq=freq_depr.upper())
expected = pd.date_range("1/1/2000", periods=4, freq="2W")
with tm.assert_produces_warning(FutureWarning, match=depr_msg):
result = pd.date_range("1/1/2000", periods=4, freq=freq_depr)
result = pd.date_range("1/1/2000", periods=4, freq="2w")
tm.assert_index_equal(result, expected)

@pytest.mark.parametrize("freq", ["1A", "2A-MAR", "2a-mar"])
Expand Down
2 changes: 1 addition & 1 deletion pandas/tests/dtypes/test_dtypes.py
Original file line number Diff line number Diff line change
Expand Up @@ -445,7 +445,7 @@ def test_construction(self):

def test_cannot_use_custom_businessday(self):
# GH#52534
msg = "C is not supported as period frequency"
msg = "Invalid frequency: C"
msg1 = "<CustomBusinessDay> is not supported as period frequency"
msg2 = r"PeriodDtype\[B\] is deprecated"
with pytest.raises(ValueError, match=msg):
Expand Down
18 changes: 8 additions & 10 deletions pandas/tests/frame/methods/test_asfreq.py
Original file line number Diff line number Diff line change
Expand Up @@ -236,32 +236,30 @@ def test_asfreq_2ME(self, freq, freq_half):
"freq, freq_depr",
[
("2ME", "2M"),
("2ME", "2m"),
("2QE", "2Q"),
("2QE-SEP", "2Q-SEP"),
("1BQE", "1BQ"),
("2BQE-SEP", "2BQ-SEP"),
("1YE", "1Y"),
("2BQE-SEP", "2bq-sep"),
("1YE", "1y"),
("2YE-MAR", "2Y-MAR"),
],
)
def test_asfreq_frequency_M_Q_Y_deprecated(self, freq, freq_depr):
# GH#9586, #55978
depr_msg = f"'{freq_depr[1:]}' is deprecated and will be removed "
f"in a future version, please use '{freq[1:]}' instead."
def test_asfreq_frequency_M_Q_Y_raises(self, freq, freq_depr):
msg = f"Invalid frequency: {freq_depr}"

index = date_range("1/1/2000", periods=4, freq=f"{freq[1:]}")
df = DataFrame({"s": Series([0.0, 1.0, 2.0, 3.0], index=index)})
expected = df.asfreq(freq=freq)
with tm.assert_produces_warning(FutureWarning, match=depr_msg):
result = df.asfreq(freq=freq_depr)
tm.assert_frame_equal(result, expected)
with pytest.raises(ValueError, match=msg):
df.asfreq(freq=freq_depr)

@pytest.mark.parametrize(
"freq, error_msg",
[
(
"2MS",
"MS is not supported as period frequency",
"Invalid frequency: 2MS",
),
(
offsets.MonthBegin(),
Expand Down
Loading