Skip to content

DEPR: Deprecate use of un-supported numpy dt64/td64 dtype for pandas.array #53439

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 25 commits into from
Closed
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
158ebd3
Adding logic to deprecate the use of dt64/td64 dtypes for pandas.array
rmhowe425 May 28, 2023
4ff6dc5
Updating unit tests
rmhowe425 May 29, 2023
c15123d
Fixing unit tests
rmhowe425 May 29, 2023
4360254
Updating documentation to reflect the deprecation of dt64/d64 dtype.
rmhowe425 May 29, 2023
48a59ab
Updating whatsnew/v2.1.0.rst
rmhowe425 May 29, 2023
5efe978
Updating unit tests based on reviewer recommendations.
rmhowe425 Jun 3, 2023
e5a085d
Updating unit tests based on reviewer recommendations.
rmhowe425 Jun 3, 2023
7addd43
Adding back documentation that should'nt have been removed
rmhowe425 Jun 3, 2023
f5e5e49
Merge branch 'main' into dev/array/depr_dt64
rmhowe425 Jun 3, 2023
d3ba003
Fixing formatting error, adding documentation back that shouldn't hav…
rmhowe425 Jun 3, 2023
73de691
Updating unit tests per reviewer recommendation.
rmhowe425 Jun 3, 2023
19835a4
Fixing reviewer recomendations.
rmhowe425 Jun 3, 2023
2a98a57
Fixing pylint error
rmhowe425 Jun 3, 2023
2af83d4
Merge branch 'main' into dev/array/depr_dt64
rmhowe425 Jun 11, 2023
427e6d6
Fixing error in whatsnew documentation
rmhowe425 Jun 11, 2023
e8afca9
Merge branch 'main' into dev/array/depr_dt64
rmhowe425 Jun 12, 2023
a9d6573
Merge branch 'main' into dev/array/depr_dt64
rmhowe425 Jun 12, 2023
78f5fbd
Merge branch 'main' into dev/array/depr_dt64
rmhowe425 Jun 15, 2023
33c7f72
Fixing documentation error after fixing merge conflict
rmhowe425 Jun 15, 2023
5ca499f
Updating PR based on reviewer recommendations
rmhowe425 Jun 16, 2023
c3bdbf8
Updated PR based on reviewer recommendations
rmhowe425 Jun 17, 2023
26da8a6
Merge branch 'main' into dev/array/depr_dt64
rmhowe425 Jun 18, 2023
6b2b089
Merge branch 'main' into dev/array/depr_dt64
rmhowe425 Jun 19, 2023
0ab08ec
Merge branch 'main' into dev/array/depr_dt64
rmhowe425 Jun 19, 2023
5de5af3
Merge branch 'pandas-dev:main' into dev/array/depr_dt64
rmhowe425 Jun 21, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v2.1.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -291,6 +291,7 @@ Deprecations
- Deprecated positional indexing on :class:`Series` with :meth:`Series.__getitem__` and :meth:`Series.__setitem__`, in a future version ``ser[item]`` will *always* interpret ``item`` as a label, not a position (:issue:`50617`)
- Deprecated the "method" and "limit" keywords on :meth:`Series.fillna`, :meth:`DataFrame.fillna`, :meth:`SeriesGroupBy.fillna`, :meth:`DataFrameGroupBy.fillna`, and :meth:`Resampler.fillna`, use ``obj.bfill()`` or ``obj.ffill()`` instead (:issue:`53394`)
- Deprecated the ``method`` and ``limit`` keywords in :meth:`DataFrame.replace` and :meth:`Series.replace` (:issue:`33302`)
- Deprecated the use of hour and minute dt64/td64 resolutions with :func:`pandas.array`. Supported resolutions are: "s", "ms", "us", "ns" resolutions (:issue:`53439`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there are others besides hour/minute, say "non-supported". similarly please write out "datetime64 and timedelta64"

- Deprecated values "pad", "ffill", "bfill", "backfill" for :meth:`Series.interpolate` and :meth:`DataFrame.interpolate`, use ``obj.ffill()`` or ``obj.bfill()`` instead (:issue:`53581`)
-

Expand Down
17 changes: 14 additions & 3 deletions pandas/core/construction.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
cast,
overload,
)
import warnings

import numpy as np
from numpy import ma
Expand All @@ -31,6 +32,7 @@
DtypeObj,
T,
)
from pandas.util._exceptions import find_stack_level

from pandas.core.dtypes.base import ExtensionDtype
from pandas.core.dtypes.cast import (
Expand Down Expand Up @@ -185,9 +187,8 @@ def array(
Length: 2, dtype: str32

Finally, Pandas has arrays that mostly overlap with NumPy

* :class:`arrays.DatetimeArray`
* :class:`arrays.TimedeltaArray`
* :class:`arrays.DatetimeArray`
* :class:`arrays.TimedeltaArray`

When data with a ``datetime64[ns]`` or ``timedelta64[ns]`` dtype is
passed, pandas will always return a ``DatetimeArray`` or ``TimedeltaArray``
Expand Down Expand Up @@ -379,6 +380,16 @@ def array(
):
return TimedeltaArray._from_sequence(data, dtype=dtype, copy=copy)

elif lib.is_np_dtype(dtype, "mM"):
warnings.warn(
r"dt/td64 dtypes with 'm' and 'h' resolutions are deprecated."
r"Supported resolutions are 's', 'ms','us', and 'ns'. "
r"In future releases, 'm' and 'h' resolutions will be cast to the closest "
r"supported unit.",
FutureWarning,
stacklevel=find_stack_level(),
)

return PandasArray._from_sequence(data, dtype=dtype, copy=copy)


Expand Down
17 changes: 17 additions & 0 deletions pandas/tests/arrays/test_array.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import datetime
import decimal
import re

import numpy as np
import pytest
Expand Down Expand Up @@ -28,6 +29,22 @@
)


def test_dt64_array():
# PR 53439
dtype_unit_lst = ["M8[h]", "M8[m]", "m8[h]", "M8[m]"]

for unit in dtype_unit_lst:
dtype_var = np.dtype(unit)
msg = (
r"dt/td64 dtypes with 'm' and 'h' resolutions are deprecated."
r"Supported resolutions are 's', 'ms','us', and 'ns'. "
r"In future releases, 'm' and 'h' resolutions will be cast to the "
r"closest supported unit."
)
with tm.assert_produces_warning(FutureWarning, match=re.escape(msg)):
pd.array([], dtype=dtype_var)


@pytest.mark.parametrize(
"data, dtype, expected",
[
Expand Down