small typing fixes

[Dr-Irv]

Some very small fixes to typing declarations to fix some errors found by pyright

Should not affect running code at all. mypy tests passed locally.

[jbrockmendel]

do we actually use this? grepping for ": Axes" im mostly (only?) seeing matplotlib types

[Dr-Irv]

It's in the type signature for DataFrame.__init__

[jreback]

we don't want to generically type like this

[Dr-Irv]

@jbrockmendel Only failure was in some IO testing. Do you want to merge in anyway?

[jreback]

we don't want to generically type

[Dr-Irv]

we don't want to generically type

@jreback Can you explain. The changes I made were to change

Axes = Collection to Axes = Collection[Any]

and in pandas/core/dtypes/base.py change the return type of the method type from Type to Type[Any]

So aren't my changes not generically typing by specifying that you can use any type? pyright is requiring the stuff in square brackets (as best as I can tell)

[jreback]

the Any for the collections should be more specific on what actually is allowed

[Dr-Irv]

the Any for the collections should be more specific on what actually is allowed

@jreback That's non-trivial

In pandas/_typing.py we declare Axes as a type. It is only used in the declaration of DataFrame.__init__() as follows (from pandas/core/frame.py):

    def __init__(
        self,
        data=None,
        index: Optional[Axes] = None,
        columns: Optional[Axes] = None,
        dtype: Optional[Dtype] = None,
        copy: bool = False,
    ):

So since an index or columns can contain any kind of object, I think Any is appropriate here. Unless we really want to say that only certain types can be used to define an index or columns in a constructor.

Furthermore, the correction in this PR is to change Collection to Collection[Any] which essentially are the same, but the latter is more "proper" python.

We could also change DataFrame.__init__() to use Optional[Union[ArrayLike, Index]] instead, and I can test that change and delete the Axes declaration.

Let me know.

[Dr-Irv]

@jreback pinging you on #39348 (comment)

[jreback]

thanks @Dr-Irv generally we don't want to use Any if at all possible, even generically, but ok

[simonjayhawkins]

pyright is requiring the stuff in square brackets (as best as I can tell)

FYI we have an issue related to this. #30539

mypy has strictness flags that you can enable at milestones throughout the typing journey. Requiring type parameters is often requested during review. Adding Any just to satisfy the type checker shouldn't be necessary, as we in general prefer to leave untyped for now to indicate that we have not yet established what types are permitted.

Just like Optional, Any is a confusingly named alias. Any means that the type checker will accept any type without reporting errors and is often used in heavily dynamic code that is difficult to statically check. If indeed, the code accepts any type, the type should be object not Any. Mypy will then report errors if we try to do anything with that type that is not available on all objects.

for instance, if we do a len(x), Any would not report any errors, but using object would give us error: Argument 1 to "len" has incompatible type "object"; expected "Sized"

with mypy we can turn these strictness flags on/off globally on for individual modules

so can we please, not add Any just for pyright at this stage and maybe discuss perhaps locking down the files that pass disallow_any_generic and tackling the existing annotations missing type parameters in some other way instead.

[Dr-Irv]

pyright is requiring the stuff in square brackets (as best as I can tell)

FYI we have an issue related to this. #30539

mypy has strictness flags that you can enable at milestones throughout the typing journey. Requiring type parameters is often requested during review. Adding Any just to satisfy the type checker shouldn't be necessary, as we in general prefer to leave untyped for now to indicate that we have not yet established what types are permitted.

Just like Optional, Any is a confusingly named alias. Any means that the type checker will accept any type without reporting errors and is often used in heavily dynamic code that is difficult to statically check. If indeed, the code accepts any type, the type should be object not Any. Mypy will then report errors if we try to do anything with that type that is not available on all objects.

for instance, if we do a len(x), Any would not report any errors, but using object would give us error: Argument 1 to "len" has incompatible type "object"; expected "Sized"

with mypy we can turn these strictness flags on/off globally on for individual modules

so can we please, not add Any just for pyright at this stage and maybe discuss perhaps locking down the files that pass disallow_any_generic and tackling the existing annotations missing type parameters in some other way instead.

@simonjayhawkins That's fair. We're all learning the best way of doing this as we move forward. It might be worth having a page in the developer's documentation section about the "rules" (such as avoiding Any) we want to use with respect to typing the pandas code. You seem to have the best handle on that at the moment