Any other comments? I am currently getting errors on checking with mypy, The erased type of self "pandas.core.frame.DataFrame" is not a supertype of its class "pandas.core.generic.NDFrame".

If I understand the error correctly: any is a method of NDArray, but your overloads do now only cover DataFrame and Series - NDFrame is not covered by the overloads (since neither of them is a super-class for NDFrame).

I think you will need a third overload that is compatible with the return type of the Series and DataFrame overload. Not sure whether this works:

    @overload
    def any(
        self: NDFrame,
        axis: Axis = ...,
        bool_only: bool_t | None = ...,
        skipna: bool_t = ...,
        level: Level | None = ...,
        **kwargs,
    ) -> Series | bool_t:
        ...

@twoertwein Thanks for the suggestion.
But the error still persists after I make the suggested changes.

Why not define any in both pandas/core/frame.py and pandas/core/series.py, in which they each have the correct return type and call super().any?

Why not define any in both pandas/core/frame.py and pandas/core/series.py, in which they each have the correct return type and call super().any?

Thanks @MarcoGorelli
I was working on this approach exactly, once the @overload approach didn't seem to work.

Why not define any in both pandas/core/frame.py and pandas/core/series.py, in which they each have the correct return type and call super().any?

If the mypy error is a false-positive, I would prefer adding two ignore comments compared to "implementing" any in the sub-classes. This has the advantage that as soon as mypy is updated, we can remove the ignore comments. We don't get an automatic reminder if we implement any in the sub-classes.

At least in the toy example of python/mypy#11759, the return types are correctly inferred even though mypy complains.

@twoertwein true, but there's the same construct for replace, where both Series.replace and DataFrame.replace just call super().replace, if I recall correctly

This has the advantage that as soon as mypy is updated, we can remove the ignore comments.

Sure, but even then, what's the advantage of doing this via overloads rather than by overriding any in DataFrame and Series?

Even without the mypy false-positive, using overloads requires adding 3 overloads, while overriding in the child classes only requires 2 extra function definitions.

In terms of lines of code which need adding, overriding any adds fewer

Thank you @MarcoGorelli , I agree super().any(...) is the better solution. I was hoping that typing should ideally not change the implementation.

Nice, thanks! Could you also post screenshots of the docs for Series.any and DataFrame.any?

You can use --single for this, see https://pandas.pydata.org/docs/development/contributing_documentation.html#building-the-documentation

# compile the reference docs for a single function
python make.py clean
python make.py --single pandas.DataFrame.join

DataFrame.any

Series.any

There is a mismatch between the doc-string and the type annotations: The type annotations say that DataFrame.any returns a Series (doc: Series or DataFrame) and Seriesn.any returns a bool (doc: Series or scalar).

There is a mismatch between the doc-string and the type annotations: The type annotations say that DataFrame.any returns a Series (doc: Series or DataFrame) and Seriesn.any returns a bool (doc: Series or scalar).

Yes, you're right!

So, the this needs some overloads too, to indicate that the return type depends on whether level was specified

Overloaded function signatures 1 and 2 overlap with incompatible return types
What could be a way to define these overload definitions without overlapping signatures as per our requirement?

ah, it's because level takes default values, and can be passed both positionally and as keyword, and it has default arguments which come before it. This makes overloading complicated, check https://medium.com/analytics-vidhya/making-sense-of-typing-overload-437e6deecade

So, I'd suggest:

1. deprecate non-keyword arguments in any, see https://github.com/pandas-dev/pandas/pull/41523/files as an example
2. for now, the return types can be DataFrame | Series and Series | bool
3. in a follow-up PR (or, in the next version of pandas, once non-keyword arguments in any have been deprecated), overload the return types depending on the value of level

This has been a bit more challenging that I originally thought, but I hope you find it worthwhile and learn a few new things!

Level is defined as Union[Hashable, int] but None is also hashable, so they actually overlap.

@MarcoGorelli
Do I also need to add a what's new note and test?

What needs to be changed for the failing checks?

What needs to be changed for the failing checks?

You'll need to update those tests such that they don't throw a warning

=================================== FAILURES ===================================
__________________ TestDataFrameAnalytics.test_any_all_extra ___________________
[gw1] linux -- Python 3.10.4 /usr/share/miniconda/envs/pandas-dev/bin/python

self = <pandas.tests.frame.test_reductions.TestDataFrameAnalytics object at 0x7feb4634aa40>

    def test_any_all_extra(self):
        df = DataFrame(
            {
                "A": [True, False, False],
                "B": [True, True, False],
                "C": [True, True, True],
            },
            index=["a", "b", "c"],
        )
>       result = df[["A", "B"]].any(1)

pandas/tests/frame/test_reductions.py:1058: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = (       A      B
a   True   True
b  False   True
c  False  False, 1)
kwargs = {}, arguments = ''

    @wraps(func)
    def wrapper(*args, **kwargs):
        arguments = _format_argument_list(allow_args)
        if len(args) > num_allow_args:
>           warnings.warn(
                msg.format(arguments=arguments),
                FutureWarning,
                stacklevel=stacklevel,
            )
E           FutureWarning: In a future version of pandas all arguments of DataFrame.any and Series.any will be keyword-only.

pandas/util/_decorators.py:[31](https://github.com/pandas-dev/pandas/runs/5830982510?check_suite_focus=true#step:11:31)2: FutureWarning

So, change result = df[["A", "B"]].any(1) to result = df[["A", "B"]].any(axis=1), and likewise for the others

What is the tool used in the tests for docstring validation?

What is the tool used in the tests for docstring validation?

numpydoc

There was a failure on the main branch but it's been fixed now (see #46690)

I don't know how to solve the current failing checks.
Some of them show FileNotFoundError. Could the given paths be wrong 🤔 ? (in doc/source/user_guide/io.rst)

They're unrelated, don't worry about them

lgm @MarcoGorelli over to you

Thanks @MarcoGorelli @twoertwein @jreback for the reviews. 😄