STYLE use absolute imports

[MarcoGorelli]

• Ensure all linting tests pass, see here for how to run them

The code style guide reads

Imports (aim for absolute)

In Python 3, absolute imports are recommended. Using absolute imports, doing something
like import string will import the string module rather than string.py
in the same directory. As much as possible, you should try to write out
absolute imports that show the whole import chain from top-level pandas.

Explicit relative imports are also supported in Python 3 but it is not
recommended to use them. Implicit relative imports should never be used
and are removed in Python 3.

For example:

::

# preferred
import pandas.core.common as com

# not preferred
from .common import test_base

# wrong
from common import test_base

yet this isn't checked (nor followed).

So I've made a little package which does this - it's pretty simple and light (link to source), just standard Python libraries (ast.parse). Timing on my laptop:

$ time pre-commit run abs-imports --all-files
abs-imports..............................................................Passed

real    0m1.102s
user    0m10.050s
sys     0m0.373s

I asked about doing this on the gitter channel and got the response

in general we go for absolute imports, the exception being in places where that creates circular imports (generally init files)

I didn't find that this created any circular imports (at least, running ./test_fast.sh worked fine), but __init__ files can always be excluded if necessary (exclude: __init__\.py$)

[jbrockmendel]

I didn't find that this created any circular imports

i think this has been an issue in some of the cython code

Does this affect import speed?

[MarcoGorelli]

I didn't find that this created any circular imports

i think this has been an issue in some of the cython code

So if this is just run on Python files, it should be OK?

Does this affect import speed?

I didn't notice any difference from several runs of python -X importtime -c "import pandas"

[jbrockmendel]

is this added automatically by versioneer?

[MarcoGorelli]

yes, thanks for spotting this - I've updated versioneer.py to use an absolute import

[jbrockmendel]

LGTM

[jreback]

thanks @MarcoGorelli very nice.

[jorisvandenbossche]

Personally, I am -1 on general linting check for this. For example, this specific example where I am commenting on, I find the relative imports actually more readable.

[jbrockmendel]

I agree with Joris on this (more generally, i like relative imports for same-directory as it clarified dependency structure)

[MarcoGorelli]

I have no strong opinion here, I was just following the (existing) pandas style guide - should the style guide be changed so an exception is made for same-level imports?

Or this can be reverted if you wish, it was a really good learning exercise anyway.

My personal preference would be to have some kind of rule agreed on, but again, not a strong opinion

[jreback]

if we had easily automatic check able for relative imports that would be ok for a limited purpose

we cannot expreess 'more readable' in this way and have it enforced

this is why i am currently -1 on relative imports and why we cannot use them in the code base

[jorisvandenbossche]

I was just following the (existing) pandas style guide - should the style guide be changed so an exception is made for same-level imports?

The part of the style guide about general relative imports, of course fully agreed that we never want implicit relative imports. But personally I think we should reconsider explicit ones, yes.

more generally, i like relative imports for same-directory as it clarified dependency structure

+1, I also find this helpful. I think it should certainly be doable to have a check that relative imports are never outside the local directory

[jbrockmendel]

I just don't (yet) see why you'd make an exception for same-directory imports. Doesn't from ..foo import bar also clarify the dependency structure?

there are a bunch of places where we have dependency structure rules (imperfectly enforced, so maybe better thought of as aspirations) of the form "this file doesnt depend on anything else in pandas except other files in this directory or things explicitly higher up in the dependency structure":

pandas._config
pandas._libs.tslibs (depends only on _config)
pandas._libs (depends almost-only on tslibs)
pandas.tseries (depends only on tslibs)
pandas.core.dtypes (depends almost-only on _libs)
pandas.core.arrays (depends almost^2-only on dtypes)
pandas.core.indexes (doesnt depend in groupby or internals)

[MarcoGorelli]

Thanks for explaining, I hadn't appreciated that - I'll look into this and see how to adapt imports

[MarcoGorelli]

pandas.core.arrays (depends almost^2-only on dtypes)

@jbrockmendel just to make sure I've understood, are you suggesting that every file inside pandas/core/arrays should import from pandas/core/arrays in a relative manner? For example, in pandas/core/arrays/sparse/array.py there is

from pandas.core.arrays import ExtensionArray

, would you want that to be from .. import ExtensionArray instead?

[jbrockmendel]

would you want that to be from .. import ExtensionArray instead?

No, I would essentially never use from .. import, only from . import

Note that isort has a LOCALFOLDER section, so from . imports get put below everything else, which is consistent with the pattern of laying out imports in rough order of dependency structure

[MarcoGorelli]

OK, so you'd always use a relative import when importing from the same directory (even outside of the folders you listed above), and use absolute elsewhere? If so, this seems easily enforceable, I can make a PR later this week