Skip to content

CLN: Flake8 E741 #22913

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Oct 9, 2018
Merged

CLN: Flake8 E741 #22913

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
ed8fbbb
Flake8 E741
alimcmaster1 Sep 30, 2018
0779dfb
E741 update as per comments
alimcmaster1 Oct 2, 2018
869ca05
Remove pylint
alimcmaster1 Oct 2, 2018
9c72397
Add back gfx import
alimcmaster1 Oct 3, 2018
9474944
Use u literal
alimcmaster1 Oct 3, 2018
76c1f8f
Flake8 ignore multiple import on one line
alimcmaster1 Oct 3, 2018
04478f9
Fix import ordering
alimcmaster1 Oct 3, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 32 additions & 33 deletions pandas/core/frame.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# pylint: disable=E1101
# pylint: disable=W0212,W0703,W0622
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i would rather not disable specific items for a whole file. Is there a reason for this?

Copy link
Member

@datapythonista datapythonista Oct 9, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jreback 100% agree, but these lines are already there in the original files, they've just been moved.

@alimcmaster1 removed some of the errors that were being ignored, but the ones left are not trivial.

Didn't check in detail, but E1101 is probably to avoid false positives in attributes that are created dynamically and not found in the linting. It's disabled like this in around 80 files, so it may be worth to move it to setup.cfg and ignore it everywhere.

Didn't check the others, but I'd merge this PR as it is, and take care of all the pylint: disable in a separate PR.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok that's fine.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks both, had a brief discussion above with @jbrockmendel about these.

"E1101: frame.py:2577:40: E1101: Module 'pandas.core.common' has no '_unpickle_array' member (no-member)" this is as @datapythonista described above.

"W0622: frame.py:36:0: W0622: Redefining built-in 'zip' (redefined-builtin)"
We should be able to get rid of W0622, I can do a follow up PR with this.

I will follow and remove E741 from our lint CI setup and fix this error for the few remaining test scripts.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm merging #22863 as soon as the CI is green, so you may want to wait to remove E741, as it'll conflict.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merged. E741 should be removed from setup.cfg and .pep8speaks.yml. But not sure if the remaining tests are few. ;)

./pandas/tests/test_algos.py:375:9: E741 ambiguous variable name 'l'
./pandas/tests/test_multilevel.py:2065:9: E741 ambiguous variable name 'l'
./pandas/tests/frame/test_operators.py:796:9: E741 ambiguous variable name 'l'
./pandas/tests/frame/test_indexing.py:2079:9: E741 ambiguous variable name 'l'
./pandas/tests/frame/test_constructors.py:899:9: E741 ambiguous variable name 'l'
./pandas/tests/frame/test_constructors.py:926:9: E741 ambiguous variable name 'l'
./pandas/tests/frame/test_constructors.py:1747:9: E741 ambiguous variable name 'l'
./pandas/tests/frame/test_constructors.py:1752:9: E741 ambiguous variable name 'l'
./pandas/tests/plotting/common.py:260:24: E741 ambiguous variable name 'l'
./pandas/tests/plotting/test_frame.py:302:9: E741 ambiguous variable name 'l'
./pandas/tests/plotting/test_frame.py:310:9: E741 ambiguous variable name 'l'
./pandas/tests/plotting/test_datetimelike.py:545:9: E741 ambiguous variable name 'l'
./pandas/tests/plotting/test_datetimelike.py:560:9: E741 ambiguous variable name 'l'
./pandas/tests/plotting/test_datetimelike.py:575:9: E741 ambiguous variable name 'l'
./pandas/tests/plotting/test_datetimelike.py:595:9: E741 ambiguous variable name 'l'
./pandas/tests/plotting/test_datetimelike.py:611:9: E741 ambiguous variable name 'l'
./pandas/tests/plotting/test_datetimelike.py:642:9: E741 ambiguous variable name 'l'
./pandas/tests/plotting/test_datetimelike.py:953:17: E741 ambiguous variable name 'l'
./pandas/tests/plotting/test_datetimelike.py:966:17: E741 ambiguous variable name 'l'
./pandas/tests/plotting/test_datetimelike.py:984:17: E741 ambiguous variable name 'l'
./pandas/tests/plotting/test_datetimelike.py:998:17: E741 ambiguous variable name 'l'
./pandas/tests/indexing/test_loc.py:671:13: E741 ambiguous variable name 'l'
./pandas/tests/indexing/test_indexing.py:772:19: E741 ambiguous variable name 'l'
./pandas/tests/io/test_packers.py:515:9: E741 ambiguous variable name 'l'
./pandas/tests/io/test_packers.py:521:9: E741 ambiguous variable name 'l'
./pandas/tests/io/test_packers.py:529:9: E741 ambiguous variable name 'l'
./pandas/tests/io/test_pytables.py:2185:13: E741 ambiguous variable name 'l'
./pandas/tests/series/test_analytics.py:542:13: E741 ambiguous variable name 'l'
./pandas/tests/series/test_analytics.py:977:13: E741 ambiguous variable name 'l'
./pandas/tests/series/test_dtypes.py:246:9: E741 ambiguous variable name 'l'
./pandas/tests/series/test_dtypes.py:252:9: E741 ambiguous variable name 'l'
./pandas/tests/series/test_dtypes.py:273:9: E741 ambiguous variable name 'l'

"""
DataFrame
---------
Expand All @@ -9,11 +11,9 @@
labeling information
"""
from __future__ import division
# pylint: disable=E1101,E1103
# pylint: disable=W0212,W0231,W0703,W0622

import functools
import collections
import functools
import itertools
import sys
import warnings
Expand All @@ -22,7 +22,20 @@
import numpy as np
import numpy.ma as ma

from pandas.core.accessor import CachedAccessor
from pandas._libs import lib, algos as libalgos
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Internal convention is to group by dependency structure. This usually means libs first, then util, compat, then core, then ...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually the pytables module is a pretty good example

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool thanks for the example @jbrockmendel will fix this up!


from pandas.util._decorators import (Appender, Substitution,
rewrite_axis_style_signature,
deprecate_kwarg)
from pandas.util._validators import (validate_bool_kwarg,
validate_axis_style_args)

from pandas import compat
from pandas.compat import (range, map, zip, lrange, lmap, lzip, StringIO, u,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Combine with previous

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks done!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great, thanks, I know its nitpicky. For future reference in case you want to really match my habits, I usually put a newline after the compat imports, then collect core.dtypes imports in a section above the rest of core (lots of stuff depends on core.dtypes, but it depends on very little except for compat and _libs)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks much appreciated, I will update! I might add some kind of description to the contributing guide on the import layout you have described here, unless its already documented somewhere?

OrderedDict, PY36, raise_with_traceback,
string_and_binary_types)
from pandas.compat.numpy import function as nv

from pandas.core.dtypes.cast import (
maybe_upcast,
cast_scalar_to_array,
Expand Down Expand Up @@ -62,46 +75,32 @@
from pandas.core.dtypes.concat import _get_sliced_frame_result_type
from pandas.core.dtypes.missing import isna, notna


from pandas.core import algorithms
from pandas.core import common as com
from pandas.core import nanops
from pandas.core import ops
from pandas.core.accessor import CachedAccessor
from pandas.core.arrays import Categorical, ExtensionArray
from pandas.core.config import get_option
from pandas.core.generic import NDFrame, _shared_docs
from pandas.core.index import (Index, MultiIndex, ensure_index,
ensure_index_from_sequences)
from pandas.core.indexes import base as ibase
from pandas.core.indexes.datetimes import DatetimeIndex
from pandas.core.indexes.period import PeriodIndex
from pandas.core.indexes.timedeltas import TimedeltaIndex
from pandas.core.indexing import (maybe_droplevels, convert_to_index_sliceable,
check_bool_indexer)
from pandas.core.internals import (BlockManager,
create_block_manager_from_arrays,
create_block_manager_from_blocks)
from pandas.core.series import Series
from pandas.core.arrays import Categorical, ExtensionArray
import pandas.core.algorithms as algorithms
from pandas.compat import (range, map, zip, lrange, lmap, lzip, StringIO, u,
OrderedDict, raise_with_traceback,
string_and_binary_types)
from pandas import compat
from pandas.compat import PY36
from pandas.compat.numpy import function as nv
from pandas.util._decorators import (Appender, Substitution,
rewrite_axis_style_signature,
deprecate_kwarg)
from pandas.util._validators import (validate_bool_kwarg,
validate_axis_style_args)

from pandas.core.indexes.period import PeriodIndex
from pandas.core.indexes.datetimes import DatetimeIndex
from pandas.core.indexes.timedeltas import TimedeltaIndex
import pandas.core.indexes.base as ibase

import pandas.core.common as com
import pandas.core.nanops as nanops
import pandas.core.ops as ops
import pandas.io.formats.console as console
import pandas.io.formats.format as fmt
from pandas.io.formats import console
from pandas.io.formats import format as fmt
from pandas.io.formats.printing import pprint_thing
import pandas.plotting._core as gfx

from pandas._libs import lib, algos as libalgos

from pandas.core.config import get_option
import pandas.plotting._core as gfx

# ---------------------------------------------------------------------
# Docstring templates
Expand Down Expand Up @@ -1003,7 +1002,7 @@ def dot(self, other):
rvals = np.asarray(other)
if lvals.shape[1] != rvals.shape[0]:
raise ValueError('Dot product shape mismatch, '
'{l} vs {r}'.format(l=lvals.shape,
'{s} vs {r}'.format(s=lvals.shape,
r=rvals.shape))

if isinstance(other, DataFrame):
Expand Down
19 changes: 10 additions & 9 deletions pandas/core/indexes/range.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,28 +1,29 @@
from sys import getsizeof
import operator
from datetime import timedelta
from sys import getsizeof

import numpy as np

from pandas import compat

from pandas._libs import index as libindex
from pandas.util._decorators import Appender, cache_readonly

from pandas.compat import lrange, range, get_range_parameters
from pandas.compat.numpy import function as nv

from pandas.core.dtypes.common import (
is_integer,
is_scalar,
is_timedelta64_dtype,
is_int64_dtype)
from pandas.core.dtypes.generic import ABCSeries, ABCTimedeltaIndex

from pandas import compat
from pandas.compat import lrange, range, get_range_parameters
from pandas.compat.numpy import function as nv
from pandas.core.dtypes import concat as _concat

import pandas.core.common as com
import pandas.core.indexes.base as ibase
from pandas.core import ops
from pandas.core.indexes.base import Index, _index_shared_docs
from pandas.util._decorators import Appender, cache_readonly
import pandas.core.dtypes.concat as _concat
import pandas.core.indexes.base as ibase

from pandas.core.indexes.numeric import Int64Index


Expand Down
49 changes: 25 additions & 24 deletions pandas/core/indexing.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,16 @@
# pylint: disable=W0223
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you remove this warning? The general problem with these warnings is that they are added, but never removed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's fine too

import textwrap
import warnings

import numpy as np
from pandas.compat import range, zip
from pandas._libs.indexing import _NDFrameIndexerBase
from pandas.util._decorators import Appender

from pandas.errors import AbstractMethodError

import pandas.compat as compat
from pandas.core.dtypes.generic import ABCDataFrame, ABCPanel, ABCSeries
from pandas.compat import range, zip

from pandas.core.dtypes.common import (
is_integer_dtype,
is_integer, is_float,
Expand All @@ -14,14 +20,11 @@
is_scalar,
is_sparse,
ensure_platform_int)
from pandas.core.dtypes.generic import ABCDataFrame, ABCPanel, ABCSeries
from pandas.core.dtypes.missing import isna, _infer_fill_value
from pandas.errors import AbstractMethodError
from pandas.util._decorators import Appender

from pandas.core.index import Index, MultiIndex

import pandas.core.common as com
from pandas._libs.indexing import _NDFrameIndexerBase
from pandas.core.index import Index, MultiIndex


# the supported indexers
Expand Down Expand Up @@ -304,8 +307,7 @@ def _setitem_with_indexer(self, indexer, value):
self._has_valid_setitem_indexer(indexer)

# also has the side effect of consolidating in-place
# TODO: Panel, DataFrame are not imported, remove?
from pandas import Panel, DataFrame, Series # noqa
from pandas import Series
info_axis = self.obj._info_axis_number

# maybe partial set
Expand Down Expand Up @@ -553,14 +555,14 @@ def can_do_equal_len():
is_scalar(plane_indexer[0])):
return False

l = len(value)
item = labels[0]
index = self.obj[item].index

values_len = len(value)
# equal len list/ndarray
if len(index) == l:
if len(index) == values_len:
return True
elif lplane_indexer == l:
elif lplane_indexer == values_len:
return True

return False
Expand Down Expand Up @@ -717,8 +719,8 @@ def ravel(i):

# single indexer
if len(indexer) > 1 and not multiindex_indexer:
l = len(indexer[1])
ser = np.tile(ser, l).reshape(l, -1).T
len_indexer = len(indexer[1])
ser = np.tile(ser, len_indexer).reshape(len_indexer, -1).T

return ser

Expand Down Expand Up @@ -2077,9 +2079,9 @@ def _validate_key(self, key, axis):
elif is_list_like_indexer(key):
# check that the key does not exceed the maximum size of the index
arr = np.array(key)
l = len(self.obj._get_axis(axis))
len_axis = len(self.obj._get_axis(axis))

if len(arr) and (arr.max() >= l or arr.min() < -l):
if len(arr) and (arr.max() >= len_axis or arr.min() < -len_axis):
raise IndexError("positional indexers are out-of-bounds")
else:
raise ValueError("Can only index by location with "
Expand Down Expand Up @@ -2136,9 +2138,8 @@ def _validate_integer(self, key, axis):
If 'key' is not a valid position in axis 'axis'
"""

ax = self.obj._get_axis(axis)
l = len(ax)
if key >= l or key < -l:
len_axis = len(self.obj._get_axis(axis))
if key >= len_axis or key < -len_axis:
raise IndexError("single positional indexer is out-of-bounds")

def _getitem_tuple(self, tup):
Expand Down Expand Up @@ -2425,18 +2426,18 @@ def length_of_indexer(indexer, target=None):
"""return the length of a single non-tuple indexer which could be a slice
"""
if target is not None and isinstance(indexer, slice):
l = len(target)
target_len = len(target)
start = indexer.start
stop = indexer.stop
step = indexer.step
if start is None:
start = 0
elif start < 0:
start += l
if stop is None or stop > l:
stop = l
start += target_len
if stop is None or stop > target_len:
stop = target_len
elif stop < 0:
stop += l
stop += target_len
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 on all the changed names here; this is very clear

if step is None:
step = 1
elif step < 0:
Expand Down
44 changes: 23 additions & 21 deletions pandas/io/packers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,39 +38,41 @@
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
"""

from datetime import datetime, date, timedelta
from dateutil.parser import parse
import os
from textwrap import dedent
import warnings
from datetime import datetime, date, timedelta
from textwrap import dedent

import numpy as np
from dateutil.parser import parse

from pandas import compat
from pandas import (Timestamp, Period, Series, DataFrame, # noqa:F401
Index, MultiIndex, Float64Index, Int64Index,
Panel, RangeIndex, PeriodIndex, DatetimeIndex, NaT,
Categorical, CategoricalIndex, IntervalIndex, Interval,
TimedeltaIndex)

from pandas.util._move import (
BadMove as _BadMove,
move_into_mutable_buffer as _move_into_mutable_buffer,
)
from pandas.errors import PerformanceWarning

from pandas.compat import u, u_safe

from pandas.core.dtypes.common import (
is_categorical_dtype, is_object_dtype,
needs_i8_conversion, pandas_dtype)

from pandas import (Timestamp, Period, Series, DataFrame, # noqa
Index, MultiIndex, Float64Index, Int64Index,
Panel, RangeIndex, PeriodIndex, DatetimeIndex, NaT,
Categorical, CategoricalIndex, IntervalIndex, Interval,
TimedeltaIndex)
from pandas.core import internals
from pandas.core.arrays import IntervalArray
from pandas.core.generic import NDFrame
from pandas.core.internals import BlockManager, make_block, _safe_reshape
from pandas.core.sparse.api import SparseSeries, SparseDataFrame
from pandas.core.sparse.array import BlockIndex, IntIndex
from pandas.core.generic import NDFrame
from pandas.errors import PerformanceWarning
from pandas.io.common import get_filepath_or_buffer, _stringify_path
from pandas.core.internals import BlockManager, make_block, _safe_reshape
import pandas.core.internals as internals

from pandas.io.msgpack import Unpacker as _Unpacker, Packer as _Packer, ExtType
from pandas.util._move import (
BadMove as _BadMove,
move_into_mutable_buffer as _move_into_mutable_buffer,
)

# check which compression libs we have installed
try:
Expand Down Expand Up @@ -187,16 +189,16 @@ def read_msgpack(path_or_buf, encoding='utf-8', iterator=False, **kwargs):
return Iterator(path_or_buf)

def read(fh):
l = list(unpack(fh, encoding=encoding, **kwargs))
if len(l) == 1:
return l[0]
unpacked_obj = list(unpack(fh, encoding=encoding, **kwargs))
if len(unpacked_obj) == 1:
return unpacked_obj[0]

if should_close:
try:
path_or_buf.close()
except: # noqa: flake8
pass
return l
return unpacked_obj

# see if we have an actual file
if isinstance(path_or_buf, compat.string_types):
Expand Down
Loading