CI: Check for file leaks in _all_ tests

[jbrockmendel]

Sits on top of #35693

Still have two tests failing locally, but I'm not seeing any socket leaks that I occasionally see in the CI logs.

[jbrockmendel]

Excellent, Travis is showing us where the socket leaks are coming from.

@simonjayhawkins thoughts on how to handle the mypy complaint?

[simonjayhawkins]

@simonjayhawkins thoughts on how to handle the mypy complaint?

not sure which is best until look at bigger cleanup. (can replace handles (or whatever used) in get_handle calls with _ in some cases, since it appears that not used in some cases)

IIUC we should only need to close the files in handles and not access them for any other purpose, so could use Protocol, say Closable (used in https://docs.python.org/3/library/typing.html#typing.runtime_checkable) or SupportsClose

from https://github.com/python/typeshed/blob/master/CONTRIBUTING.md#conventions

A few guidelines for protocol names below. In cases that don't fall into any of those categories, use your best judgement.

Use plain names for protocols that represent a clear concept (e.g. Iterator, Container).
Use SupportsX for protocols that provide callable methods (e.g. SupportsInt, SupportsRead, SupportsReadSeek).
Use HasX for protocols that have readable and/or writable attributes or getter/setter methods (e.g. HasItems, HasFileno).

or maybe could make _MMapWrapper a subclass (or virtual) of IO

or simpliest, to get mypy to green, is to change type declaration of handles to handles: List[Union[IO, "_MMapWrapper"]] = list(). The return of get_handle is not typed, so this simple solution shouldn't create any consistency issues for now.

[jbrockmendel]

The isort complaint looks like it is disagrees with black on how to format a runtime import.

[jbrockmendel]

Remaining test failures look like they're coming from moto/boto3, no luck tracking down where the left-open file is. @TomAugspurger do we have anyone knowledgeable about these packages?

[TomAugspurger]

Dunno, but @martindurant has a PR open changing moto to run as a server (in a separate process?), so it's possible those failures will go away.

[martindurant]

Correct, moto will run as a separate process - but the cleanup was apparently leaving something behind, so the latest iteration is to run with broader scope for the pytest fixture, so they should go away (worked in dask/dask#6528 ). I'm not sure those would show up in what you are testing here.

Maybe some "open files" will tend to show up in tests, if file closing is left to the garbage collector, which is fine in general use.

[jbrockmendel]

Following #35836 i think this has been pushed as far as it can ATM, closing.