Skip to content

API: Add pandas.api.typing #48578

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 37 commits into from
Apr 17, 2023
Merged
Show file tree
Hide file tree
Changes from 34 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
ea91291
API: Add pandas.api.typing
rhshadrach Sep 9, 2022
985627c
whatsnew note
rhshadrach Dec 31, 2022
1114428
Merge branch 'main' into groupby_in_api
rhshadrach Jan 1, 2023
d4e6e09
Don't import Styler
rhshadrach Jan 1, 2023
3dd6cf8
Test fixup for Styler
rhshadrach Jan 2, 2023
8f953a5
API: Add pandas.api.typing
rhshadrach Sep 9, 2022
d00861e
Merge branch 'main' into groupby_in_api
rhshadrach Jan 22, 2023
e94061f
Add back Styler
rhshadrach Jan 23, 2023
b3abf13
Merge branch 'main' of https://github.com/pandas-dev/pandas into grou…
rhshadrach Jan 23, 2023
236e99d
Merge branch 'groupby_in_api' of https://github.com/rhshadrach/pandas…
rhshadrach Jan 23, 2023
d4f39d4
Merge branch 'groupby_in_api' of https://github.com/rhshadrach/pandas…
rhshadrach Jan 23, 2023
3f63dd5
Merge branch 'main' of https://github.com/pandas-dev/pandas into grou…
rhshadrach Jan 24, 2023
44edcdb
Revert Styler changes
rhshadrach Jan 24, 2023
04a88d9
Point to pandas.api.typing for GroupBy objects
rhshadrach Jan 24, 2023
b11b21b
Merge branch 'main' of https://github.com/pandas-dev/pandas into grou…
rhshadrach Feb 6, 2023
7a8b1aa
Add references to pandas.api.typing
rhshadrach Feb 6, 2023
b0423af
fixup
rhshadrach Feb 11, 2023
79fe367
Merge branch 'main' of https://github.com/pandas-dev/pandas into grou…
rhshadrach Feb 11, 2023
940a33e
Refinements
rhshadrach Feb 11, 2023
b558c0a
Merge branch 'main' of https://github.com/pandas-dev/pandas into grou…
rhshadrach Feb 14, 2023
56a2480
Add references to User Guide; fixup docstrings
rhshadrach Feb 14, 2023
0e279f4
Add references
rhshadrach Feb 14, 2023
edaa9c3
Merge branch 'groupby_in_api' of https://github.com/rhshadrach/pandas…
rhshadrach Feb 14, 2023
1b8189c
fixup
rhshadrach Feb 14, 2023
3506ea6
Merge branch 'main' of https://github.com/pandas-dev/pandas into grou…
rhshadrach Feb 14, 2023
433040c
Merge branch 'main' of https://github.com/pandas-dev/pandas into grou…
rhshadrach Feb 24, 2023
0dd0e2e
Move whatsnew to 2.1.0
rhshadrach Feb 24, 2023
45de284
Merge branch 'main' of https://github.com/pandas-dev/pandas into grou…
rhshadrach Feb 26, 2023
781ff4a
Merge branch 'main' of https://github.com/pandas-dev/pandas into grou…
rhshadrach Mar 4, 2023
cd40744
Merge branch 'main' of https://github.com/pandas-dev/pandas into grou…
rhshadrach Mar 9, 2023
0e8cb7a
Docstring fixup
rhshadrach Mar 17, 2023
cf1d68b
Merge branch 'main' of https://github.com/pandas-dev/pandas into grou…
rhshadrach Mar 17, 2023
bb164e5
Merge branch 'main' into groupby_in_api
simonjayhawkins Mar 17, 2023
621b37e
Merge branch 'main' into groupby_in_api
rhshadrach Apr 8, 2023
df0d343
Test dir of api.typing
rhshadrach Apr 16, 2023
9f868f1
Merge branch 'main' of https://github.com/pandas-dev/pandas into grou…
rhshadrach Apr 16, 2023
4689965
Merge branch 'main' of https://github.com/pandas-dev/pandas into grou…
rhshadrach Apr 17, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion doc/source/reference/groupby.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,9 @@ GroupBy
=======
.. currentmodule:: pandas.core.groupby

GroupBy objects are returned by groupby calls: :func:`pandas.DataFrame.groupby`, :func:`pandas.Series.groupby`, etc.
:class:`pandas.api.typing.DataFrameGroupBy` and :class:`pandas.api.typing.SeriesGroupBy`
instances are returned by groupby calls :func:`pandas.DataFrame.groupby` and
:func:`pandas.Series.groupby` respectively.

Indexing, iteration
-------------------
Expand Down
23 changes: 18 additions & 5 deletions doc/source/reference/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,24 @@ API reference
This page gives an overview of all public pandas objects, functions and
methods. All classes and functions exposed in ``pandas.*`` namespace are public.

Some subpackages are public which include ``pandas.errors``,
``pandas.plotting``, and ``pandas.testing``. Public functions in
``pandas.io`` and ``pandas.tseries`` submodules are mentioned in
the documentation. ``pandas.api.types`` subpackage holds some
public functions related to data types in pandas.
The following subpackages are public.

- ``pandas.errors``: Custom exception and warnings classes that are raised by pandas.
- ``pandas.plotting``: Plotting public API.
- ``pandas.testing``: Functions that are useful for writing tests involving pandas objects.
- ``pandas.api.extensions``: Functions and classes for extending pandas objects.
- ``pandas.api.indexers``: Functions and classes for rolling window indexers.
- ``pandas.api.interchange``: DataFrame interchange protocol.
- ``pandas.api.types``: Datatype classes and functions.
- ``pandas.api.typing``: Classes that may be necessary for type-hinting. These are
classes that are encountered as intermediate results but should not be
instantiated directly by users. These classes are not to be confused with
classes from the `pandas-stubs <https://github.com/pandas-dev/pandas-stubs>`_
package which has classes in addition to those that occur in pandas for type-hinting.

In addition, public functions in ``pandas.io`` and ``pandas.tseries`` submodules
are mentioned in the documentation.


.. warning::

Expand Down
3 changes: 2 additions & 1 deletion doc/source/reference/resampling.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@ Resampling
==========
.. currentmodule:: pandas.core.resample

Resampler objects are returned by resample calls: :func:`pandas.DataFrame.resample`, :func:`pandas.Series.resample`.
:class:`pandas.api.typing.Resampler` instances are returned by
resample calls: :func:`pandas.DataFrame.resample`, :func:`pandas.Series.resample`.

Indexing, iteration
~~~~~~~~~~~~~~~~~~~
Expand Down
9 changes: 6 additions & 3 deletions doc/source/reference/window.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,12 @@
Window
======

Rolling objects are returned by ``.rolling`` calls: :func:`pandas.DataFrame.rolling` and :func:`pandas.Series.rolling`.
Expanding objects are returned by ``.expanding`` calls: :func:`pandas.DataFrame.expanding` and :func:`pandas.Series.expanding`.
ExponentialMovingWindow objects are returned by ``.ewm`` calls: :func:`pandas.DataFrame.ewm` and :func:`pandas.Series.ewm`.
:class:`pandas.api.typing.Rolling` instances are returned by ``.rolling`` calls:
:func:`pandas.DataFrame.rolling` and :func:`pandas.Series.rolling`.
:class:`pandas.api.typing.Expanding` instances are returned by ``.expanding`` calls:
:func:`pandas.DataFrame.expanding` and :func:`pandas.Series.expanding`.
:class:`pandas.api.typing.ExponentialMovingWindow` instances are returned by ``.ewm``
calls: :func:`pandas.DataFrame.ewm` and :func:`pandas.Series.ewm`.

.. _api.functions_rolling:

Expand Down
6 changes: 4 additions & 2 deletions doc/source/user_guide/groupby.rst
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,7 @@ consider the following ``DataFrame``:
df

On a DataFrame, we obtain a GroupBy object by calling :meth:`~DataFrame.groupby`.
This method returns a ``pandas.api.typing.DataFrameGroupBy`` instance.
We could naturally group by either the ``A`` or ``B`` columns, or both:

.. ipython:: python
Expand Down Expand Up @@ -1413,8 +1414,9 @@ Groupby a specific column with the desired frequency. This is like resampling.

df.groupby([pd.Grouper(freq="1M", key="Date"), "Buyer"])[["Quantity"]].sum()

You have an ambiguous specification in that you have a named index and a column
that could be potential groupers.
When ``freq`` is specified, the object returned by ``pd.Grouper`` will be an
instance of ``pandas.api.typing.TimeGrouper``. You have an ambiguous specification
in that you have a named index and a column that could be potential groupers.

.. ipython:: python

Expand Down
6 changes: 3 additions & 3 deletions doc/source/user_guide/io.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2069,7 +2069,7 @@ is ``None``. To explicitly force ``Series`` parsing, pass ``typ=series``
seconds, milliseconds, microseconds or nanoseconds respectively.
* ``lines`` : reads file as one json object per line.
* ``encoding`` : The encoding to use to decode py3 bytes.
* ``chunksize`` : when used in combination with ``lines=True``, return a JsonReader which reads in ``chunksize`` lines per iteration.
* ``chunksize`` : when used in combination with ``lines=True``, return a ``pandas.api.typing.JsonReader`` which reads in ``chunksize`` lines per iteration.
* ``engine``: Either ``"ujson"``, the built-in JSON parser, or ``"pyarrow"`` which dispatches to pyarrow's ``pyarrow.json.read_json``.
The ``"pyarrow"`` is only available when ``lines=True``

Expand Down Expand Up @@ -5984,15 +5984,15 @@ Reading from Stata format
'''''''''''''''''''''''''

The top-level function ``read_stata`` will read a dta file and return
either a ``DataFrame`` or a :class:`~pandas.io.stata.StataReader` that can
either a ``DataFrame`` or a :class:`pandas.api.typing.StataReader` that can
be used to read the file incrementally.

.. ipython:: python

pd.read_stata("stata.dta")

Specifying a ``chunksize`` yields a
:class:`~pandas.io.stata.StataReader` instance that can be used to
:class:`pandas.api.typing.StataReader` instance that can be used to
read ``chunksize`` lines from the file at a time. The ``StataReader``
object can be used as an iterator.

Expand Down
5 changes: 3 additions & 2 deletions doc/source/user_guide/timeseries.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1750,8 +1750,9 @@ We can instead only resample those groups where we have points as follows:
Aggregation
~~~~~~~~~~~

Similar to the :ref:`aggregating API <basics.aggregate>`, :ref:`groupby API <groupby.aggregate>`, and the :ref:`window API <window.overview>`,
a ``Resampler`` can be selectively resampled.
The ``resample()`` method returns a ``pandas.api.typing.Resampler`` instance. Similar to
the :ref:`aggregating API <basics.aggregate>`, :ref:`groupby API <groupby.aggregate>`,
and the :ref:`window API <window.overview>`, a ``Resampler`` can be selectively resampled.

Resampling a ``DataFrame``, the default will be to act on all columns with the same function.

Expand Down
16 changes: 8 additions & 8 deletions doc/source/user_guide/window.rst
Original file line number Diff line number Diff line change
Expand Up @@ -37,14 +37,14 @@ pandas supports 4 types of windowing operations:
#. Expanding window: Accumulating window over the values.
#. Exponentially Weighted window: Accumulating and exponentially weighted window over the values.

============================= ================= =========================== =========================== ======================== =================================== ===========================
Concept Method Returned Object Supports time-based windows Supports chained groupby Supports table method Supports online operations
============================= ================= =========================== =========================== ======================== =================================== ===========================
Rolling window ``rolling`` ``Rolling`` Yes Yes Yes (as of version 1.3) No
Weighted window ``rolling`` ``Window`` No No No No
Expanding window ``expanding`` ``Expanding`` No Yes Yes (as of version 1.3) No
Exponentially Weighted window ``ewm`` ``ExponentialMovingWindow`` No Yes (as of version 1.2) No Yes (as of version 1.3)
============================= ================= =========================== =========================== ======================== =================================== ===========================
============================= ================= ============================================= =========================== ======================== =================================== ===========================
Concept Method Returned Object Supports time-based windows Supports chained groupby Supports table method Supports online operations
============================= ================= ============================================= =========================== ======================== =================================== ===========================
Rolling window ``rolling`` ``pandas.typing.api.Rolling`` Yes Yes Yes (as of version 1.3) No
Weighted window ``rolling`` ``pandas.typing.api.Window`` No No No No
Expanding window ``expanding`` ``pandas.typing.api.Expanding`` No Yes Yes (as of version 1.3) No
Exponentially Weighted window ``ewm`` ``pandas.typing.api.ExponentialMovingWindow`` No Yes (as of version 1.2) No Yes (as of version 1.3)
============================= ================= ============================================= =========================== ======================== =================================== ===========================

As noted above, some operations support specifying a window based on a time offset:

Expand Down
1 change: 1 addition & 0 deletions doc/source/whatsnew/v2.1.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ Other enhancements
:meth:`Categorical.map` implicitly had a default value of ``"ignore"`` for ``na_action``. This has formally been deprecated and will be changed to ``None`` in the future.
Also notice that :meth:`Series.map` has default ``na_action=None`` and calls to series with categorical data will now use ``na_action=None`` unless explicitly set otherwise (:issue:`44279`)
- Implemented ``__pandas_priority__`` to allow custom types to take precedence over :class:`DataFrame`, :class:`Series`, :class:`Index`, or :class:`ExtensionArray` for arithmetic operations, :ref:`see the developer guide <extending.pandas_priority>` (:issue:`48347`)
- Classes that are useful for type-hinting have been added to the public API in the new submodule ``pandas.api.typing`` (:issue:`48577`)
- :meth:`MultiIndex.sort_values` now supports ``na_position`` (:issue:`51612`)
- :meth:`MultiIndex.sortlevel` and :meth:`Index.sortlevel` gained a new keyword ``na_position`` (:issue:`51612`)
- :meth:`arrays.DatetimeArray.map`, :meth:`arrays.TimedeltaArray.map` and :meth:`arrays.PeriodArray.map` can now take a ``na_action`` argument (:issue:`51644`)
Expand Down
2 changes: 2 additions & 0 deletions pandas/api/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,13 @@
indexers,
interchange,
types,
typing,
)

__all__ = [
"interchange",
"extensions",
"indexers",
"types",
"typing",
]
50 changes: 50 additions & 0 deletions pandas/api/typing/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
"""
Public API classes that store intermediate results useful for type-hinting.
"""

from pandas.core.groupby import (
DataFrameGroupBy,
SeriesGroupBy,
)
from pandas.core.resample import (
DatetimeIndexResamplerGroupby,
PeriodIndexResamplerGroupby,
Resampler,
TimedeltaIndexResamplerGroupby,
TimeGrouper,
)
from pandas.core.window import (
Expanding,
ExpandingGroupby,
ExponentialMovingWindow,
ExponentialMovingWindowGroupby,
Rolling,
RollingGroupby,
Window,
)

# TODO: Can't import Styler without importing jinja2
# from pandas.io.formats.style import Styler
from pandas.io.json._json import JsonReader
from pandas.io.stata import StataReader

__all__ = [
"DataFrameGroupBy",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just commenting that in a follow up PR it might make sense for these objects to have their own API Reference page?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea - agreed. I can open an issue to track.

"DatetimeIndexResamplerGroupby",
"Expanding",
"ExpandingGroupby",
"ExponentialMovingWindow",
"ExponentialMovingWindowGroupby",
"JsonReader",
"PeriodIndexResamplerGroupby",
"Resampler",
"Rolling",
"RollingGroupby",
"SeriesGroupBy",
"StataReader",
# See TODO above
# "Styler",
"TimedeltaIndexResamplerGroupby",
"TimeGrouper",
"Window",
]
2 changes: 1 addition & 1 deletion pandas/core/generic.py
Original file line number Diff line number Diff line change
Expand Up @@ -8699,7 +8699,7 @@ def resample(

Returns
-------
pandas.core.Resampler
pandas.api.typing.Resampler
:class:`~pandas.core.Resampler` object.

See Also
Expand Down
17 changes: 14 additions & 3 deletions pandas/core/groupby/groupby.py
Original file line number Diff line number Diff line change
Expand Up @@ -2618,8 +2618,11 @@ def resample(self, rule, *args, **kwargs):

Returns
-------
Grouper
Return a new grouper with our resampler appended.
pandas.api.typing.DatetimeIndexResamplerGroupby,
pandas.api.typing.PeriodIndexResamplerGroupby, or
pandas.api.typing.TimedeltaIndexResamplerGroupby
Return a new groupby object, with type depending on the data
being resampled.

See Also
--------
Expand Down Expand Up @@ -2783,7 +2786,7 @@ def rolling(self, *args, **kwargs) -> RollingGroupby:

Returns
-------
RollingGroupby
pandas.api.typing.RollingGroupby
Return a new grouper with our rolling appended.

See Also
Expand Down Expand Up @@ -2846,6 +2849,10 @@ def expanding(self, *args, **kwargs) -> ExpandingGroupby:
"""
Return an expanding grouper, providing expanding
functionality per group.

Returns
-------
pandas.api.typing.ExpandingGroupby
"""
from pandas.core.window import ExpandingGroupby

Expand All @@ -2862,6 +2869,10 @@ def expanding(self, *args, **kwargs) -> ExpandingGroupby:
def ewm(self, *args, **kwargs) -> ExponentialMovingWindowGroupby:
"""
Return an ewm grouper, providing ewm functionality per group.

Returns
-------
pandas.api.typing.ExponentialMovingWindowGroupby
"""
from pandas.core.window import ExponentialMovingWindowGroupby

Expand Down
4 changes: 3 additions & 1 deletion pandas/core/groupby/grouper.py
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,9 @@ class Grouper:

Returns
-------
A specification for a groupby instruction
Grouper or pandas.api.typing.TimeGrouper
A TimeGrouper is returned if ``freq`` is not ``None``. Otherwise, a Grouper
is returned.

Examples
--------
Expand Down
2 changes: 1 addition & 1 deletion pandas/core/shared_docs.py
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@

Returns
-------
%(klass)sGroupBy
pandas.api.typing.%(klass)sGroupBy
Returns a groupby object that contains information about the groups.

See Also
Expand Down
2 changes: 1 addition & 1 deletion pandas/core/window/ewm.py
Original file line number Diff line number Diff line change
Expand Up @@ -231,7 +231,7 @@ class ExponentialMovingWindow(BaseWindow):

Returns
-------
``ExponentialMovingWindow`` subclass
pandas.api.typing.ExponentialMovingWindow

See Also
--------
Expand Down
2 changes: 1 addition & 1 deletion pandas/core/window/expanding.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ class Expanding(RollingAndExpandingMixin):

Returns
-------
``Expanding`` subclass
pandas.api.typing.Expanding

See Also
--------
Expand Down
6 changes: 3 additions & 3 deletions pandas/core/window/rolling.py
Original file line number Diff line number Diff line change
Expand Up @@ -965,9 +965,9 @@ class Window(BaseWindow):

Returns
-------
``Window`` subclass if a ``win_type`` is passed

``Rolling`` subclass if ``win_type`` is not passed
pandas.api.typing.Window or pandas.api.typing.Rolling
An instance of Window is returned if ``win_type`` is passed. Otherwise,
an instance of Rolling is returned.

See Also
--------
Expand Down
5 changes: 3 additions & 2 deletions pandas/io/json/_json.py
Original file line number Diff line number Diff line change
Expand Up @@ -665,8 +665,9 @@ def read_json(

Returns
-------
Series or DataFrame
The type returned depends on the value of `typ`.
Series, DataFrame, or pandas.api.typing.JsonReader
A JsonReader is returned when ``chunksize`` is not ``0`` or ``None``.
Otherwise, the type returned depends on the value of ``typ``.

See Also
--------
Expand Down
2 changes: 1 addition & 1 deletion pandas/io/stata.py
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@

Returns
-------
DataFrame or StataReader
DataFrame or pandas.api.typing.StataReader

See Also
--------
Expand Down
2 changes: 1 addition & 1 deletion pandas/tests/api/test_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -236,7 +236,7 @@ def test_depr(self):


class TestApi(Base):
allowed = ["types", "extensions", "indexers", "interchange"]
allowed = ["types", "extensions", "indexers", "interchange", "typing"]

def test_api(self):
self.check(api, self.allowed)
Expand Down