DEPR: enforce a bunch (#49552)

* DEPR: enforce a bunch

* docstring fixup

* disable pylint check

* pylint fixup

* Fix gh ref in whatsnew

Author: jbrockmendel

doc/source/whatsnew/v2.0.0.rst

@@ -299,13 +299,15 @@ Removal of prior version deprecations/changes
 - Removed deprecated :meth:`Index.is_all_dates` (:issue:`36697`)
 - Enforced deprecation disallowing passing a timezone-aware :class:`Timestamp` and ``dtype="datetime64[ns]"`` to :class:`Series` or :class:`DataFrame` constructors (:issue:`41555`)
 - Enforced deprecation disallowing passing a sequence of timezone-aware values and ``dtype="datetime64[ns]"`` to to :class:`Series` or :class:`DataFrame` constructors (:issue:`41555`)
+- Enforced deprecation disallowing ``numpy.ma.mrecords.MaskedRecords`` in the :class:`DataFrame` constructor; pass ``"{name: data[name] for name in data.dtype.names}`` instead (:issue:`40363`)
 - Enforced deprecation disallowing unit-less "datetime64" dtype in :meth:`Series.astype` and :meth:`DataFrame.astype` (:issue:`47844`)
 - Enforced deprecation disallowing using ``.astype`` to convert a ``datetime64[ns]`` :class:`Series`, :class:`DataFrame`, or :class:`DatetimeIndex` to timezone-aware dtype, use ``obj.tz_localize`` or ``ser.dt.tz_localize`` instead (:issue:`39258`)
 - Enforced deprecation disallowing using ``.astype`` to convert a timezone-aware :class:`Series`, :class:`DataFrame`, or :class:`DatetimeIndex` to timezone-naive ``datetime64[ns]`` dtype, use ``obj.tz_localize(None)`` or ``obj.tz_convert("UTC").tz_localize(None)`` instead (:issue:`39258`)
 - Enforced deprecation disallowing passing non boolean argument to sort in :func:`concat` (:issue:`44629`)
 - Removed Date parser functions :func:`~pandas.io.date_converters.parse_date_time`,
   :func:`~pandas.io.date_converters.parse_date_fields`, :func:`~pandas.io.date_converters.parse_all_fields`
   and :func:`~pandas.io.date_converters.generic_parser` (:issue:`24518`)
+- Removed argument ``index`` from the :class:`core.arrays.SparseArray` constructor (:issue:`43523`)
 - Remove argument ``squeeze`` from :meth:`DataFrame.groupby` and :meth:`Series.groupby` (:issue:`32380`)
 - Removed deprecated ``apply``, ``apply_index``, ``__call__``, ``onOffset``, and ``isAnchored`` attributes from :class:`DateOffset` (:issue:`34171`)
 - Removed ``keep_tz`` argument in :meth:`DatetimeIndex.to_series` (:issue:`29731`)
@@ -315,6 +317,7 @@ Removal of prior version deprecations/changes
 - Removed argument ``line_terminator`` from :meth:`DataFrame.to_csv` and :meth:`Series.to_csv`, use ``lineterminator`` instead (:issue:`45302`)
 - Removed argument ``inplace`` from :meth:`DataFrame.set_axis` and :meth:`Series.set_axis`, use ``obj = obj.set_axis(..., copy=False)`` instead (:issue:`48130`)
 - Disallow passing positional arguments to :meth:`MultiIndex.set_levels` and :meth:`MultiIndex.set_codes` (:issue:`41485`)
+- Disallow parsing to Timedelta strings with components with units "Y", "y", or "M", as these do not represent unambiguous durations (:issue:`36838`)
 - Removed :meth:`MultiIndex.is_lexsorted` and :meth:`MultiIndex.lexsort_depth` (:issue:`38701`)
 - Removed argument ``how`` from :meth:`PeriodIndex.astype`, use :meth:`PeriodIndex.to_timestamp` instead (:issue:`37982`)
 - Removed argument ``try_cast`` from :meth:`DataFrame.mask`, :meth:`DataFrame.where`, :meth:`Series.mask` and :meth:`Series.where` (:issue:`38836`)
@@ -375,6 +378,7 @@ Removal of prior version deprecations/changes
 - Removed :attr:`Rolling.win_type` returning ``"freq"`` (:issue:`38963`)
 - Removed :attr:`Rolling.is_datetimelike` (:issue:`38963`)
 - Removed deprecated :meth:`Timedelta.delta`, :meth:`Timedelta.is_populated`, and :attr:`Timedelta.freq` (:issue:`46430`, :issue:`46476`)
+- Removed deprecated :attr:`NaT.freq` (:issue:`45071`)
 - Removed deprecated :meth:`Categorical.replace`, use :meth:`Series.replace` instead (:issue:`44929`)
 - Removed the ``numeric_only`` keyword from :meth:`Categorical.min` and :meth:`Categorical.max` in favor of ``skipna`` (:issue:`48821`)
 - Changed behavior of :meth:`DataFrame.median` and :meth:`DataFrame.mean` with ``numeric_only=None`` to not exclude datetime-like columns THIS NOTE WILL BE IRRELEVANT ONCE ``numeric_only=None`` DEPRECATION IS ENFORCED (:issue:`29941`)
@@ -433,6 +437,7 @@ Removal of prior version deprecations/changes
 - Changed behavior of :class:`DataFrame` constructor when passed a ``dtype`` (other than int) that the data cannot be cast to; it now raises instead of silently ignoring the dtype (:issue:`41733`)
 - Changed the behavior of :class:`Series` constructor, it will no longer infer a datetime64 or timedelta64 dtype from string entries (:issue:`41731`)
 - Changed behavior of :class:`Timestamp` constructor with a ``np.datetime64`` object and a ``tz`` passed to interpret the input as a wall-time as opposed to a UTC time (:issue:`42288`)
+- Changed behavior of :meth:`Timestamp.utcfromtimestamp` to return a timezone-aware object satisfying ``Timestamp.utcfromtimestamp(val).timestamp() == val`` (:issue:`45083`)
 - Changed behavior of :class:`Index` constructor when passed a ``SparseArray`` or ``SparseDtype`` to retain that dtype instead of casting to ``numpy.ndarray`` (:issue:`43930`)
 - Changed behavior of setitem-like operations (``__setitem__``, ``fillna``, ``where``, ``mask``, ``replace``, ``insert``, fill_value for ``shift``) on an object with :class:`DatetimeTZDtype` when using a value with a non-matching timezone, the value will be cast to the object's timezone instead of casting both to object-dtype (:issue:`44243`)
 - Changed behavior of :class:`Index`, :class:`Series`, :class:`DataFrame` constructors with floating-dtype data and a :class:`DatetimeTZDtype`, the data are now interpreted as UTC-times instead of wall-times, consistent with how integer-dtype data are treated (:issue:`45573`)
@@ -441,6 +446,7 @@ Removal of prior version deprecations/changes
 - Change the default argument of ``regex`` for :meth:`Series.str.replace` from ``True`` to ``False``. Additionally, a single character ``pat`` with ``regex=True`` is now treated as a regular expression instead of a string literal. (:issue:`36695`, :issue:`24804`)
 - Changed behavior of :meth:`DataFrame.any` and :meth:`DataFrame.all` with ``bool_only=True``; object-dtype columns with all-bool values will no longer be included, manually cast to ``bool`` dtype first (:issue:`46188`)
 - Changed behavior of comparison of a :class:`Timestamp` with a ``datetime.date`` object; these now compare as un-equal and raise on inequality comparisons, matching the ``datetime.datetime`` behavior (:issue:`36131`)
+- Changed behavior of comparison of ``NaT`` with a ``datetime.date`` object; these now raise on inequality comparisons (:issue:`39196`)
 - Enforced deprecation of silently dropping columns that raised a ``TypeError`` in :class:`Series.transform` and :class:`DataFrame.transform` when used with a list or dictionary (:issue:`43740`)
 - Change behavior of :meth:`DataFrame.apply` with list-like so that any partial failure will raise an error (:issue:`43740`)
 - Removed ``na_sentinel`` argument from :func:`factorize`, :meth:`.Index.factorize`, and :meth:`.ExtensionArray.factorize` (:issue:`47157`)

pandas/_libs/tslibs/nattype.pyx

@@ -1,7 +1,3 @@
-import warnings
-
-from pandas.util._exceptions import find_stack_level
-
 from cpython.datetime cimport (
     PyDate_Check,
     PyDateTime_Check,
@@ -128,14 +124,7 @@ cdef class _NaT(datetime):
                 return False
             if op == Py_NE:
                 return True
-            warnings.warn(
-                "Comparison of NaT with datetime.date is deprecated in "
-                "order to match the standard library behavior. "
-                "In a future version these will be considered non-comparable.",
-                FutureWarning,
-                stacklevel=find_stack_level(),
-            )
-            return False
+            raise TypeError("Cannot compare NaT with datetime.date object")
 
         return NotImplemented
 
@@ -374,15 +363,6 @@ class NaTType(_NaT):
 
         return base
 
-    @property
-    def freq(self):
-        warnings.warn(
-            "NaT.freq is deprecated and will be removed in a future version.",
-            FutureWarning,
-            stacklevel=find_stack_level(),
-        )
-        return None
-
     def __reduce_ex__(self, protocol):
         # python 3.6 compat
         # https://bugs.python.org/issue28730
@@ -566,12 +546,17 @@ class NaTType(_NaT):
         """
         Timestamp.utcfromtimestamp(ts)
 
-        Construct a naive UTC datetime from a POSIX timestamp.
+        Construct a timezone-aware UTC datetime from a POSIX timestamp.
+
+        Notes
+        -----
+        Timestamp.utcfromtimestamp behavior differs from datetime.utcfromtimestamp
+        in returning a timezone-aware object.
 
         Examples
         --------
         >>> pd.Timestamp.utcfromtimestamp(1584199972)
-        Timestamp('2020-03-14 15:32:52')
+        Timestamp('2020-03-14 15:32:52+0000', tz='UTC')
         """,
     )
     fromtimestamp = _make_error_func(

pandas/_libs/tslibs/timedeltas.pyx

@@ -1,7 +1,4 @@
 import collections
-import warnings
-
-from pandas.util._exceptions import find_stack_level
 
 cimport cython
 from cpython.object cimport (
@@ -688,11 +685,9 @@ cdef inline timedelta_from_spec(object number, object frac, object unit):
 
     unit = ''.join(unit)
     if unit in ["M", "Y", "y"]:
-        warnings.warn(
-            "Units 'M', 'Y' and 'y' do not represent unambiguous "
-            "timedelta values and will be removed in a future version.",
-            FutureWarning,
-            stacklevel=find_stack_level(),
+        raise ValueError(
+            "Units 'M', 'Y' and 'y' do not represent unambiguous timedelta "
+            "values and are not supported."
         )
 
     if unit == 'M':

pandas/_libs/tslibs/timestamps.pyx

@@ -1303,24 +1303,20 @@ class Timestamp(_Timestamp):
         """
         Timestamp.utcfromtimestamp(ts)
 
-        Construct a naive UTC datetime from a POSIX timestamp.
+        Construct a timezone-aware UTC datetime from a POSIX timestamp.
+
+        Notes
+        -----
+        Timestamp.utcfromtimestamp behavior differs from datetime.utcfromtimestamp
+        in returning a timezone-aware object.
 
         Examples
         --------
         >>> pd.Timestamp.utcfromtimestamp(1584199972)
-        Timestamp('2020-03-14 15:32:52')
+        Timestamp('2020-03-14 15:32:52+0000', tz='UTC')
         """
         # GH#22451
-        warnings.warn(
-            "The behavior of Timestamp.utcfromtimestamp is deprecated, in a "
-            "future version will return a timezone-aware Timestamp with UTC "
-            "timezone. To keep the old behavior, use "
-            "Timestamp.utcfromtimestamp(ts).tz_localize(None). "
-            "To get the future behavior, use Timestamp.fromtimestamp(ts, 'UTC')",
-            FutureWarning,
-            stacklevel=find_stack_level(),
-        )
-        return cls(datetime.utcfromtimestamp(ts))
+        return cls.fromtimestamp(ts, tz="UTC")
 
     @classmethod
     def fromtimestamp(cls, ts, tz=None):

pandas/core/arrays/sparse/array.py

@@ -378,7 +378,6 @@ def __init__(
         self,
         data,
         sparse_index=None,
-        index=None,
         fill_value=None,
         kind: SparseIndexKind = "integer",
         dtype: Dtype | None = None,
@@ -413,26 +412,8 @@ def __init__(
                 fill_value = dtype.fill_value
             dtype = dtype.subtype
 
-        if index is not None:
-            warnings.warn(
-                "The index argument has been deprecated and will be "
-                "removed in a future version. Use a function like np.full "
-                "to construct an array with the desired repeats of the "
-                "scalar value instead.\n\n",
-                FutureWarning,
-                stacklevel=find_stack_level(),
-            )
-
-        if index is not None and not is_scalar(data):
-            raise Exception("must only pass scalars with an index")
-
         if is_scalar(data):
-            if index is not None and data is None:
-                data = np.nan
-
-            if index is not None:
-                npoints = len(index)
-            elif sparse_index is None:
+            if sparse_index is None:
                 npoints = 1
             else:
                 npoints = sparse_index.length

pandas/core/frame.py

@@ -686,33 +686,22 @@ def __init__(
 
             # masked recarray
             if isinstance(data, mrecords.MaskedRecords):
-                mgr = rec_array_to_mgr(
-                    data,
-                    index,
-                    columns,
-                    dtype,
-                    copy,
-                    typ=manager,
-                )
-                warnings.warn(
-                    "Support for MaskedRecords is deprecated and will be "
-                    "removed in a future version.  Pass "
-                    "{name: data[name] for name in data.dtype.names} instead.",
-                    FutureWarning,
-                    stacklevel=find_stack_level(),
+                raise TypeError(
+                    "MaskedRecords are not supported. Pass "
+                    "{name: data[name] for name in data.dtype.names} "
+                    "instead"
                 )
 
             # a masked array
-            else:
-                data = sanitize_masked_array(data)
-                mgr = ndarray_to_mgr(
-                    data,
-                    index,
-                    columns,
-                    dtype=dtype,
-                    copy=copy,
-                    typ=manager,
-                )
+            data = sanitize_masked_array(data)
+            mgr = ndarray_to_mgr(
+                data,
+                index,
+                columns,
+                dtype=dtype,
+                copy=copy,
+                typ=manager,
+            )
 
         elif isinstance(data, (np.ndarray, Series, Index, ExtensionArray)):
             if data.dtype.names:

pandas/core/internals/construction.py

@@ -6,11 +6,9 @@
 
 from collections import abc
 from typing import (
-    TYPE_CHECKING,
     Any,
     Hashable,
     Sequence,
-    cast,
 )
 import warnings
 
@@ -32,7 +30,6 @@
     maybe_cast_to_datetime,
     maybe_convert_platform,
     maybe_infer_to_datetimelike,
-    maybe_upcast,
 )
 from pandas.core.dtypes.common import (
     is_1d_only_ea_dtype,
@@ -91,10 +88,6 @@
     create_block_manager_from_column_arrays,
 )
 
-if TYPE_CHECKING:
-    from numpy.ma.mrecords import MaskedRecords
-
-
 # ---------------------------------------------------------------------
 # BlockManager Interface
 
@@ -163,7 +156,7 @@ def arrays_to_mgr(
 
 
 def rec_array_to_mgr(
-    data: MaskedRecords | np.recarray | np.ndarray,
+    data: np.recarray | np.ndarray,
     index,
     columns,
     dtype: DtypeObj | None,
@@ -184,24 +177,9 @@ def rec_array_to_mgr(
         columns = ensure_index(columns)
     arrays, arr_columns = to_arrays(fdata, columns)
 
-    # fill if needed
-    if isinstance(data, np.ma.MaskedArray):
-        # GH#42200 we only get here with MaskedRecords, but check for the
-        #  parent class MaskedArray to avoid the need to import MaskedRecords
-        data = cast("MaskedRecords", data)
-        new_arrays = fill_masked_arrays(data, arr_columns)
-    else:
-        # error: Incompatible types in assignment (expression has type
-        # "List[ExtensionArray]", variable has type "List[ndarray]")
-        new_arrays = arrays  # type: ignore[assignment]
-
     # create the manager
 
-    # error: Argument 1 to "reorder_arrays" has incompatible type "List[ndarray]";
-    # expected "List[Union[ExtensionArray, ndarray]]"
-    arrays, arr_columns = reorder_arrays(
-        new_arrays, arr_columns, columns, len(index)  # type: ignore[arg-type]
-    )
+    arrays, arr_columns = reorder_arrays(arrays, arr_columns, columns, len(index))
     if columns is None:
         columns = arr_columns
 
@@ -212,24 +190,6 @@ def rec_array_to_mgr(
     return mgr
 
 
-def fill_masked_arrays(data: MaskedRecords, arr_columns: Index) -> list[np.ndarray]:
-    """
-    Convert numpy MaskedRecords to ensure mask is softened.
-    """
-    new_arrays = []
-
-    for col in arr_columns:
-        arr = data[col]
-        fv = arr.fill_value
-
-        mask = ma.getmaskarray(arr)
-        if mask.any():
-            arr, fv = maybe_upcast(arr, fill_value=fv, copy=True)
-            arr[mask] = fv
-        new_arrays.append(arr)
-    return new_arrays
-
-
 def mgr_to_mgr(mgr, typ: str, copy: bool = True):
     """
     Convert to specific type of Manager. Does not copy if the type is already

pandas/tests/arrays/sparse/test_constructors.py

@@ -218,28 +218,6 @@ def test_from_spmatrix_raises(self):
         with pytest.raises(ValueError, match="not '4'"):
             SparseArray.from_spmatrix(mat)
 
-    @pytest.mark.parametrize(
-        "scalar,dtype",
-        [
-            (False, SparseDtype(bool, False)),
-            (0.0, SparseDtype("float64", 0)),
-            (1, SparseDtype("int64", 1)),
-            ("z", SparseDtype("object", "z")),
-        ],
-    )
-    def test_scalar_with_index_infer_dtype(self, scalar, dtype):
-        # GH#19163
-        with tm.assert_produces_warning(
-            FutureWarning, match="The index argument has been deprecated"
-        ):
-            arr = SparseArray(scalar, index=[1, 2, 3], fill_value=scalar)
-        exp = SparseArray([scalar, scalar, scalar], fill_value=scalar)
-
-        tm.assert_sp_array_equal(arr, exp)
-
-        assert arr.dtype == dtype
-        assert exp.dtype == dtype
-
     def test_constructor_from_too_large_array(self):
         with pytest.raises(TypeError, match="expected dimension <= 1 data"):
             SparseArray(np.arange(10).reshape((2, 5)))