TYP: enable reportMissingImports

[twoertwein]

reportMissingImports checks whether all libraries can be imported. This requires all dependencies to be installed on the CI that runs pyright.

This flag should probably NOT be enabled, as it would force all contributors to install all dependencies. I'm enabling it temporarily to see whether the CI has all dependencies installed.

edit:
The typing CI has not all packages installed:

/home/runner/work/pandas/pandas/pandas/io/clipboard/__init__.py
  /home/runner/work/pandas/pandas/pandas/io/clipboard/__init__.py:150:14 - error: Import "qtpy.QtWidgets" could not be resolved (reportMissingImports)
  /home/runner/work/pandas/pandas/pandas/io/clipboard/__init__.py:155:18 - error: Import "PyQt4.QtGui" could not be resolved (reportMissingImports)
  /home/runner/work/pandas/pandas/pandas/io/clipboard/__init__.py:537:20 - error: Import "AppKit" could not be resolved (reportMissingImports)
  /home/runner/work/pandas/pandas/pandas/io/clipboard/__init__.py:538:20 - error: Import "Foundation" could not be resolved (reportMissingImports)
  /home/runner/work/pandas/pandas/pandas/io/clipboard/__init__.py:557:20 - error: Import "qtpy" could not be resolved (reportMissingImports)
  /home/runner/work/pandas/pandas/pandas/io/clipboard/__init__.py:564:28 - error: Import "PyQt4" could not be resolved (reportMissingImports)
/home/runner/work/pandas/pandas/pandas/io/excel/_pyxlsb.py
  /home/runner/work/pandas/pandas/pandas/io/excel/_pyxlsb.py:36:14 - error: Import "pyxlsb" could not be resolved (reportMissingImports)
  /home/runner/work/pandas/pandas/pandas/io/excel/_pyxlsb.py:41:14 - error: Import "pyxlsb" could not be resolved (reportMissingImports)

[twoertwein]

Install pyxlsb but don't bother installing all the platform-dependent code for the clipboard.

Should PyQt4 be removed? I don't find it on pip and "PyQt4 and Qt v4 are no longer supported and no new releases will be made."

[twoertwein]

could replace
from pandas._typing import npt
with
import numpy.typing as npt
in stub files, as I think they will not be executed at run-time - cannot fail even if an old version of numpy is installed.

[simonjayhawkins]

yep, since we don't need to guard the import at run-time for stub files sgtm.

[twoertwein]

One option could be to enable reportMissingImports but add the following pyright comment to pandas/io/clipboard/__init__.py

# pyright: reportMissingImports=false

xref #43751 (comment)

[jreback]

looks fine to me. cc @simonjayhawkins

[twoertwein]

Should probably wait until #43799 to not force contributors to install all dependencies.

[jreback]

if you'd rebase (but lgtm).

[simonjayhawkins]

One option could be to enable reportMissingImports but add the following pyright comment to pandas/io/clipboard/__init__.py

# pyright: reportMissingImports=false

xref #43751 (comment)

since clipboard is vendored, happy to exclude from all pyright checks and limit the use of module level comment syntax for pyright to pandas code

[simonjayhawkins]

Install pyxlsb but don't bother installing all the platform-dependent code for the clipboard.

in the pip secction of environment.yml we have some typing stubs imports

    - types-python-dateutil
    - types-PyMySQL
    - types-pytz
    - types-setuptools

if the package has stubs we should include it here.

if it is a typed library like numpy, then we already have it in environment.yaml so the missing import only becomes an issue if an older version of numpy is installed.

I'm not sure what is best for untyped optional libraries without stubs packages.

conda solver is already slow for our pandas-dev environment, but otoh happy to add pyxlsb as our dev env should probably have all optional dependencies installed to ensure local testing catches issues. (having some optional dependencies included in environment.yml but not all seems strange)

I assume that as pyxlsb is not a pytyped library and has no stubs, that the imports will resolve to Any. So we gain nothing from a typing perspective other than silencing the pyright warning by adding pyxlsb to environment.yml.

So am also happy not to add.

[twoertwein]

conda solver is already slow for our pandas-dev environment

Fortunately, pyxlsb does not seem to list any dependencies, so it should be quick to resolve :) But enabling the pyright option would also mean that future potentially difficult to resolve packages will be added.

I assume that as pyxlsb is not a pytyped library and has no stubs, that the imports will resolve to Any.

Unfortunately, pyxlsb has no type annotations.

So we gain nothing from a typing perspective other than silencing the pyright warning by adding pyxlsb to environment.yml.

We can catch some typos in pyi files. I was just trying flake8-pyi, it wouldn't have caught these two typos.

Let me know what you think is more important. Happy to remove pyxlsb and not disable MissingImports.

[simonjayhawkins]

I'm not against enabling the pyright option so that the typos are caught.

We can enable MissingImports without adding to environment.yaml using the module level pyright options syntax to disable MissingImports in just pandas/io/excel/_pyxlsb.py if any objections.

otherwise lgtm.

[jreback]

thanks @twoertwein happy being strict here.