DEPR: Deprecate use of un-supported numpy dt64/td64 dtype for pandas.array #53439

rmhowe425 · 2023-05-29T16:31:03Z

closes API: pd.array convert unsupported dt64/td64 to supported? #53058
Tests added and passed if fixing a bug or adding a new feature
All code checks passed.
Added type annotations to new arguments/methods/functions.
Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

rmhowe425 · 2023-05-29T21:21:26Z

@jbrockmendel Would you be able to take a look at this PR and let me know if its good to go?

jbrockmendel · 2023-05-31T18:26:43Z

doc/source/reference/arrays.rst

@@ -22,8 +22,6 @@ can be found at :ref:`basics.dtypes`.
 =================== ========================== ============================= =============================
 Kind of Data        pandas Data Type           Scalar                        Array
 =================== ========================== ============================= =============================
-TZ-aware datetime   :class:`DatetimeTZDtype`   :class:`Timestamp`            :ref:`api.arrays.datetime`
-Timedeltas          (none)                     :class:`Timedelta`            :ref:`api.arrays.timedelta`


why would this change?

@jbrockmendel Maybe I didn't completely understand #53058. Are we not removing all dt.td64 dtypes?

I removed the above, as it was part of the dt/td64 documentation

As I said here #53439 (comment), we aren't deprecating all dt64/td64, just the un-supported resolutions

@jbrockmendel Ah okay understood!

Looking at the implementation, it looks like the changes for this PR need to be made either in _ensure_nanosecond_dtype() in cast.py, or in the implementation for is_supported_unit() in dtypes.pyi.

Seeing that is_supported_unit() in dtypes.pyi is an interface, I'm assuming that there is a definition for this function for pd.array, but I don't see it. Would you be able to shed some light on how this interface method is being used?

jbrockmendel · 2023-05-31T18:27:22Z

Oh, i see. #53058 isnt about deprecating all dt64/td64 dtypes, just the unsupported units. we support s, ms, us, ns.

jbrockmendel · 2023-06-03T03:14:24Z

pandas/tests/arrays/test_array.py

@@ -203,6 +218,8 @@
    ],
 )
 def test_array(data, dtype, expected):
+    with open("/home/richard/Desktop/file.txt", "a+") as fil:


this shouldnt be needed

jbrockmendel · 2023-06-03T03:15:06Z

In pd.array, at the end where we have checks for is_supported_dtype, do an 'else' statement and issue a deprecation warning

…e been removed.

rmhowe425 · 2023-06-03T03:41:19Z

@jbrockmendel I should add a deprecation warning rather than a TypeError? I was thinking that we wanted to raise an error rather than a deprecation warning?

Is the image below what you're looking for? If we use an else statement I think that we'll be throwing a deprecation warning for other supported dtypes before we hit the return statement at the bottom.

I had my td64/dt64 check up top. Are we okay with me adding an elif statement at the bottom and checking for the unsupported types?

rmhowe425 · 2023-06-13T02:08:03Z

@jbrockmendel PR is ready for inspection. Failing test does not appear to be related to PR

rmhowe425 · 2023-06-15T04:02:17Z

@jbrockmendel PR is ready for inspection

jbrockmendel · 2023-06-15T15:10:46Z

doc/source/whatsnew/v2.1.0.rst

@@ -291,6 +291,7 @@ Deprecations
 - Deprecated positional indexing on :class:`Series` with :meth:`Series.__getitem__` and :meth:`Series.__setitem__`, in a future version ``ser[item]`` will *always* interpret ``item`` as a label, not a position (:issue:`50617`)
 - Deprecated the "method" and "limit" keywords on :meth:`Series.fillna`, :meth:`DataFrame.fillna`, :meth:`SeriesGroupBy.fillna`, :meth:`DataFrameGroupBy.fillna`, and :meth:`Resampler.fillna`, use ``obj.bfill()`` or ``obj.ffill()`` instead (:issue:`53394`)
 - Deprecated the ``method`` and ``limit`` keywords in :meth:`DataFrame.replace` and :meth:`Series.replace` (:issue:`33302`)
+- Deprecated the use of dt64/td64 dtypes with :func:`pandas.array` (:issue:`53439`)


only un-supported. we supports "s", "ms", "us", "ns" resolutions

jbrockmendel · 2023-06-15T15:11:07Z

pandas/core/construction.py

@@ -379,6 +378,9 @@ def array(
    ):
        return TimedeltaArray._from_sequence(data, dtype=dtype, copy=copy)

+    elif isinstance(dtype, np.dtype) and dtype.kind in "mM":


lib.is_np_dtype(dtype, "mM")

jbrockmendel · 2023-06-15T15:12:11Z

pandas/tests/arrays/test_array.py

+            f"dtype={dtype_var} is not supported. Supported resolutions are 's', 'ms', "
+            "'us', and 'ns'"
+        )
+        with pytest.raises(TypeError, match=re.escape(msg)):


this needs to issue a FutureWarning saying it will raise in the future (or cast to closest-supported-unit), not raise

rmhowe425 · 2023-06-16T03:22:22Z

@jbrockmendel Fixed PR based on recommendations and pinging on green!

jbrockmendel · 2023-06-17T18:00:12Z

doc/source/whatsnew/v2.1.0.rst

@@ -291,6 +291,7 @@ Deprecations
 - Deprecated positional indexing on :class:`Series` with :meth:`Series.__getitem__` and :meth:`Series.__setitem__`, in a future version ``ser[item]`` will *always* interpret ``item`` as a label, not a position (:issue:`50617`)
 - Deprecated the "method" and "limit" keywords on :meth:`Series.fillna`, :meth:`DataFrame.fillna`, :meth:`SeriesGroupBy.fillna`, :meth:`DataFrameGroupBy.fillna`, and :meth:`Resampler.fillna`, use ``obj.bfill()`` or ``obj.ffill()`` instead (:issue:`53394`)
 - Deprecated the ``method`` and ``limit`` keywords in :meth:`DataFrame.replace` and :meth:`Series.replace` (:issue:`33302`)
+- Deprecated the use of hour and minute dt64/td64 resolutions with :func:`pandas.array`. Supported resolutions are: "s", "ms", "us", "ns" resolutions (:issue:`53439`)


there are others besides hour/minute, say "non-supported". similarly please write out "datetime64 and timedelta64"

rmhowe425 · 2023-06-21T19:18:07Z

@jbrockmendel PR is ready for inspection.

Failing tests are unrelated to PR. Doctest and code checks are passing

rmhowe425 added 2 commits May 28, 2023 19:47

Adding logic to deprecate the use of dt64/td64 dtypes for pandas.array

158ebd3

Updating unit tests

4ff6dc5

rmhowe425 marked this pull request as draft May 29, 2023 16:33

Fixing unit tests

c15123d

rmhowe425 marked this pull request as ready for review May 29, 2023 19:24

rmhowe425 added 2 commits May 29, 2023 15:35

Updating documentation to reflect the deprecation of dt64/d64 dtype.

4360254

Updating whatsnew/v2.1.0.rst

48a59ab

jbrockmendel reviewed May 31, 2023

View reviewed changes

mroeschke added the Deprecate Functionality to remove in pandas label Jun 1, 2023

rmhowe425 and others added 4 commits June 2, 2023 22:48

Updating unit tests based on reviewer recommendations.

5efe978

Updating unit tests based on reviewer recommendations.

e5a085d

Adding back documentation that should'nt have been removed

7addd43

Merge branch 'main' into dev/array/depr_dt64

f5e5e49

jbrockmendel reviewed Jun 3, 2023

View reviewed changes

Fixing formatting error, adding documentation back that shouldn't hav…

d3ba003

…e been removed.

rmhowe425 and others added 4 commits June 2, 2023 23:42

Updating unit tests per reviewer recommendation.

73de691

Fixing reviewer recomendations.

19835a4

Fixing pylint error

2a98a57

Merge branch 'main' into dev/array/depr_dt64

2af83d4

rmhowe425 requested a review from jbrockmendel June 11, 2023 19:42

rmhowe425 and others added 3 commits June 11, 2023 15:46

Fixing error in whatsnew documentation

427e6d6

Merge branch 'main' into dev/array/depr_dt64

e8afca9

Merge branch 'main' into dev/array/depr_dt64

a9d6573

rmhowe425 and others added 2 commits June 14, 2023 21:15

Merge branch 'main' into dev/array/depr_dt64

78f5fbd

Fixing documentation error after fixing merge conflict

33c7f72

jbrockmendel reviewed Jun 15, 2023

View reviewed changes

Updating PR based on reviewer recommendations

5ca499f

rmhowe425 requested a review from jbrockmendel June 16, 2023 03:22

jbrockmendel reviewed Jun 17, 2023

View reviewed changes

rmhowe425 added 2 commits June 17, 2023 14:33

Updated PR based on reviewer recommendations

c3bdbf8

Merge branch 'main' into dev/array/depr_dt64

26da8a6

rmhowe425 mentioned this pull request Jun 18, 2023

DEPR: Deprecate DataFrame.last and Series.last #53710

Merged

5 tasks

rmhowe425 added 3 commits June 19, 2023 08:43

Merge branch 'main' into dev/array/depr_dt64

6b2b089

Merge branch 'main' into dev/array/depr_dt64

0ab08ec

Merge branch 'pandas-dev:main' into dev/array/depr_dt64

5de5af3

rmhowe425 requested a review from jbrockmendel June 21, 2023 19:17

rmhowe425 closed this by deleting the head repository Jun 22, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DEPR: Deprecate use of un-supported numpy dt64/td64 dtype for pandas.array #53439

DEPR: Deprecate use of un-supported numpy dt64/td64 dtype for pandas.array #53439

rmhowe425 commented May 29, 2023 •

edited

Loading

rmhowe425 commented May 29, 2023

jbrockmendel May 31, 2023

rmhowe425 May 31, 2023

jbrockmendel Jun 2, 2023

rmhowe425 Jun 3, 2023 •

edited

Loading

jbrockmendel commented May 31, 2023

jbrockmendel Jun 3, 2023

jbrockmendel commented Jun 3, 2023

rmhowe425 commented Jun 3, 2023 •

edited

Loading

rmhowe425 commented Jun 13, 2023

rmhowe425 commented Jun 15, 2023

jbrockmendel Jun 15, 2023

jbrockmendel Jun 15, 2023

jbrockmendel Jun 15, 2023

rmhowe425 commented Jun 16, 2023

jbrockmendel Jun 17, 2023

rmhowe425 commented Jun 21, 2023

DEPR: Deprecate use of un-supported numpy dt64/td64 dtype for pandas.array #53439

DEPR: Deprecate use of un-supported numpy dt64/td64 dtype for pandas.array #53439

Conversation

rmhowe425 commented May 29, 2023 • edited Loading

rmhowe425 commented May 29, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rmhowe425 Jun 3, 2023 • edited Loading

Choose a reason for hiding this comment

jbrockmendel commented May 31, 2023

Choose a reason for hiding this comment

jbrockmendel commented Jun 3, 2023

rmhowe425 commented Jun 3, 2023 • edited Loading

rmhowe425 commented Jun 13, 2023

rmhowe425 commented Jun 15, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rmhowe425 commented Jun 16, 2023

Choose a reason for hiding this comment

rmhowe425 commented Jun 21, 2023

rmhowe425 commented May 29, 2023 •

edited

Loading

rmhowe425 Jun 3, 2023 •

edited

Loading

rmhowe425 commented Jun 3, 2023 •

edited

Loading