DEPS: drop numpy < 1.12 #23062

h-vetinari · 2018-10-09T18:25:59Z

closes DEPS: drop numpy < 1.12 #21242
tests modified / passed
passes git diff upstream/master -u -- "*.py" | flake8 --diff
whatsnew entry

Spurred on also partly by compat failure of #22725. ;-)

pep8speaks · 2018-10-09T18:26:24Z

Hello @h-vetinari! Thanks for updating the PR.

There are no PEP8 issues in the file pandas/compat/numpy/__init__.py !
There are no PEP8 issues in the file pandas/core/algorithms.py !
There are no PEP8 issues in the file pandas/core/computation/check.py !
There are no PEP8 issues in the file pandas/core/ops.py !
There are no PEP8 issues in the file pandas/plotting/_compat.py !
There are no PEP8 issues in the file pandas/plotting/_converter.py !
There are no PEP8 issues in the file pandas/plotting/_core.py !
There are no PEP8 issues in the file pandas/plotting/_style.py !
There are no PEP8 issues in the file pandas/tests/arithmetic/test_period.py !
There are no PEP8 issues in the file pandas/tests/frame/test_analytics.py !
There are no PEP8 issues in the file pandas/tests/frame/test_quantile.py !
There are no PEP8 issues in the file pandas/tests/indexes/datetimes/test_ops.py !
There are no PEP8 issues in the file pandas/tests/indexes/period/test_ops.py !
There are no PEP8 issues in the file pandas/tests/indexes/period/test_partial_slicing.py !
There are no PEP8 issues in the file pandas/tests/indexes/timedeltas/test_ops.py !
There are no PEP8 issues in the file pandas/tests/plotting/common.py !
There are no PEP8 issues in the file pandas/tests/plotting/test_boxplot_method.py !
There are no PEP8 issues in the file pandas/tests/plotting/test_datetimelike.py !
There are no PEP8 issues in the file pandas/tests/plotting/test_frame.py !
There are no PEP8 issues in the file pandas/tests/plotting/test_hist_method.py !
There are no PEP8 issues in the file pandas/tests/plotting/test_misc.py !
There are no PEP8 issues in the file pandas/tests/plotting/test_series.py !
There are no PEP8 issues in the file pandas/tests/series/test_analytics.py !
There are no PEP8 issues in the file pandas/tests/sparse/series/test_series.py !
There are no PEP8 issues in the file pandas/tests/test_algos.py !
There are no PEP8 issues in the file pandas/tests/test_expressions.py !
There are no PEP8 issues in the file pandas/tests/test_sorting.py !
There are no PEP8 issues in the file pandas/tests/test_window.py !
There are no PEP8 issues in the file pandas/util/_test_decorators.py !
There are no PEP8 issues in the file setup.py !

Comment last updated on October 12, 2018 at 16:26 Hours UTC

codecov · 2018-10-09T20:28:58Z

Codecov Report

Merging #23062 into master will increase coverage by 0.04%.
The diff coverage is 94.73%.

@@            Coverage Diff             @@
##           master   #23062      +/-   ##
==========================================
+ Coverage   92.14%   92.19%   +0.04%     
==========================================
  Files         170      170              
  Lines       51017    50963      -54     
==========================================
- Hits        47011    46984      -27     
+ Misses       4006     3979      -27

Flag	Coverage Δ
#multiple	`90.61% <94.73%> (+0.04%)`	⬆️
#single	`42.29% <31.57%> (-0.02%)`	⬇️

Impacted Files	Coverage Δ
pandas/plotting/_converter.py	`63.82% <ø> (-0.07%)`	⬇️
pandas/util/_test_decorators.py	`93.24% <ø> (+0.56%)`	⬆️
pandas/plotting/_style.py	`75% <100%> (+0.71%)`	⬆️
pandas/plotting/_compat.py	`87.5% <100%> (-3.81%)`	⬇️
pandas/plotting/_core.py	`83.63% <100%> (+0.27%)`	⬆️
pandas/core/computation/check.py	`90.9% <100%> (ø)`	⬆️
pandas/core/algorithms.py	`95.12% <100%> (+0.55%)`	⬆️
pandas/core/ops.py	`94.63% <100%> (+0.24%)`	⬆️
pandas/compat/numpy/__init__.py	`92.85% <80%> (-1.09%)`	⬇️
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cf11f71...d097b43. Read the comment docs.

WillAyd

Generally good; looks like there's a dependency conflict between bottleneck and numpy

WillAyd · 2018-10-09T20:25:35Z

ci/azure-macos-35.yaml

@@ -11,7 +11,7 @@ dependencies:
  - matplotlib
  - nomkl
  - numexpr
-  - numpy=1.10.4
+  - numpy=1.12.1


Most of the code / docs point to 1.12.0 but this is 1.12.1 - any reason for the minor version bump here?

Nope, that was an oversight...

WillAyd

Can you also add the minimum version to the appropriate requirements file(s) in the ci folder?

FYI I know that bumping Matplotlib to a min of 2.0 also calls for some code cleanup / removal. OK to tackle that in a separate PR - do you mind opening the issue for that?

h-vetinari · 2018-10-09T23:16:01Z

@WillAyd
I did it right here (where it belongs IMO). Not sure I got everything, because there are some Win32 errors in the plotting tests that were already there before.

jreback · 2018-10-09T23:18:17Z

i don’t think any real urgency to bump numexpr or bottleneck mins

we generally try to allow packages to continue to work as much as possible

h-vetinari · 2018-10-09T23:23:35Z

i don’t think any real urgency to bump numexpr or bottleneck mins
we generally try to allow packages to continue to work as much as possible

I only bumped them because anything lower is not compatible with numpy>=1.12 (you can check the respective builds to see that always something was failing due to unresolvable dependencies)

WillAyd · 2018-10-10T00:20:28Z

pandas/plotting/_style.py

@@ -72,18 +70,9 @@ def _maybe_valid_colors(colors):
        # check whether each character can be convertible to colors
        maybe_color_cycle = _maybe_valid_colors(list(colors))
        if maybe_single_color and maybe_color_cycle and len(colors) > 1:
-            # Special case for single str 'CN' match and convert to hex
-            # for supporting matplotlib < 2.0.0
-            if re.match(r'\AC[0-9]\Z', colors) and _mpl_ge_2_0_0():


Think we still need the condition to check the pattern match here, no?

It already said under else that "this may no longer be required". So I wanted to see if removing it still passes the test suite

WillAyd · 2018-10-10T00:24:10Z

pandas/tests/plotting/test_frame.py

@@ -854,7 +847,7 @@ def test_area_lim(self):
    @pytest.mark.slow
    def test_bar_colors(self):
        import matplotlib.pyplot as plt
-        default_colors = self._maybe_unpack_cycler(plt.rcParams)
+        default_colors = [v['color'] for v in plt.rcParams['axes.prop_cycle']]


Any reason you decided to replace the method with a list comp? The couple calls would probably be more readable if left alone

h-vetinari · 2018-10-10T05:33:35Z

pandas/plotting/_style.py

@@ -72,18 +70,9 @@ def _maybe_valid_colors(colors):
        # check whether each character can be convertible to colors
        maybe_color_cycle = _maybe_valid_colors(list(colors))
        if maybe_single_color and maybe_color_cycle and len(colors) > 1:
-            # Special case for single str 'CN' match and convert to hex
-            # for supporting matplotlib < 2.0.0
-            if re.match(r'\AC[0-9]\Z', colors) and _mpl_ge_2_0_0():


It already said under else that "this may no longer be required". So I wanted to see if removing it still passes the test suite

h-vetinari · 2018-10-10T05:34:49Z

pandas/tests/plotting/common.py

@@ -510,20 +493,6 @@ def is_grid_on():
                obj.plot(kind=kind, grid=True, **kws)
                assert is_grid_on()

-    def _maybe_unpack_cycler(self, rcParams, field='color'):
-        """
-        Compat layer for MPL 1.5 change to color cycle


@WillAyd This sounded like it was only for compat. But I'll add it back in as convenience instead of the list comps

jreback · 2018-10-10T10:12:34Z

doc/source/whatsnew/v0.24.0.txt

+We have updated our minimum supported versions of dependencies (:issue:`21242`).
+If installed, we now require:
+
+-----------------+-----------------+----------+---------------+


remove the issue part here it is not necessary

jbrockmendel · 2018-10-10T13:47:46Z

In core.ops there is a try/except for np.broadcast_to that can be simplified along with this PR.

jbrockmendel · 2018-10-10T13:50:47Z

pandas/compat/numpy/__init__.py

-            arr = tz_replacer(arr)
+    # is_list_like
+    if hasattr(arr, '__iter__') and not \
+       isinstance(arr, string_and_binary_types):


Can you do this without the backslash?

jbrockmendel · 2018-10-12T20:18:21Z

There's something wrong with the azure failure reporting (discovered by accident):
The last three runs reported success despite two failures in the Mac job

I've been troubleshooting the same problem here, please let me know if you figure it out

h-vetinari · 2018-10-12T20:58:18Z

I understand now why no-one touched the matplotlib compat for a long time (e.g. _mpl_le_1_2_1 still around, while the lowest tested version prior to this PR was 1.4.3)...

In any case, I've now got a passing CI on all mpl builds again in 079bdaf (after the false positive from azure; there are no pins or changed code paths for matplotlib in Circle, which is lagging behind due to not cancelling on new pushes):
2.0.0, 2.0.1, 2.1.0, 2.2.0, 2.2.2, 2.2.3, 3.0.0

Reverting the pins, then this thing should be done...

@TomAugspurger @jreback another quick look?

This reverts commit ededd73.

This reverts commit 9be3d10.

jreback · 2018-10-14T18:14:31Z

pls rebase

not you will have to rebase somewhat frequently until this is merged

h-vetinari · 2018-10-14T19:22:14Z

note you will have to rebase somewhat frequently until this is merged

What's the hold-up on getting this merged soon?

h-vetinari · 2018-10-14T19:22:40Z

PS. Can someone trigger the circleCI runs? not sure why they didn't start.

jreback · 2018-10-14T20:42:15Z

this. needs review for the plotting changes

TomAugspurger

Plotting changes look fine. Thanks for cleaning that up @h-vetinari.

TomAugspurger · 2018-10-14T21:10:17Z

I don't see any way to trigger a circle build from their UI. Probably easiest to add an empty commit.

jreback · 2018-10-15T12:34:50Z

lgtm. @WillAyd if ok, pls accept, but after that @h-vetinari will need a rebase then, just before merging to make sure nothing has changed. (you can rebase now if you want).

h-vetinari · 2018-10-15T12:56:03Z

Not much happened in the last 17h (3 commits), but ok.

WillAyd · 2018-10-15T15:58:49Z

pandas/plotting/_converter.py

-        # For mpl > 2.0 the format strings are controlled via rcparams
-        # so do not mess with them.  For mpl < 2.0 change the second
-        # break point and add a musec break point
-        if _mpl_le_2_0_0():


Since mpl has a min version of 2.0.0 don't we still need this check?

That was probably being used incorrectly. I don't think it would apply to 2.0.0 but not 2.0.1.

I made sure to test all available matplotlib versions, and while the removal of the code for 2.0.0 was an oversight, nothing in the test suite failed. As far as I can tell, 2.0.0 wasn't tested in the CI before, which is also the reason that several if-switches before this PR were strictly speaking wrong (when I removed the compat code in a way that just removed pre-2.0 branches, suddenly there were failures).

jreback · 2018-10-15T17:07:36Z

thanks @h-vetinari

h-vetinari · 2018-10-15T18:22:34Z

This was quite the effort I have to say. MPL has some really weird behaviour in that (e.g.) several times in test_datetimelike there's:

behaviour A in 2.0.0, 2.2.0 and 3.0.0
behaviour B in 2.0.1, 2.1.0, 2.2.2, 2.2.3.

Not 100% sure this is not overlaid with some numpy/system dependencies as well, but happy that right now everything passes and got merged.

DEPS: drop numpy < 1.12

22dab2d

h-vetinari mentioned this pull request Oct 9, 2018

CLN/ERR: str.cat internals #22725

Merged

3 tasks

Drop bottleneck < 1.2.0

a495db9

WillAyd requested changes Oct 9, 2018

View reviewed changes

Drop matplotlib < 2.0.0

124d511

WillAyd reviewed Oct 9, 2018

View reviewed changes

h-vetinari added 3 commits October 10, 2018 01:01

Drop numexpr < 2.6.2

12b9a79

Remove mpl compat code

c8bfe66

Remove redundant functions

b2cffb4

WillAyd requested changes Oct 10, 2018

View reviewed changes

h-vetinari commented Oct 10, 2018

View reviewed changes

h-vetinari added 4 commits October 10, 2018 08:50

Reinstante _unpack_cycler

e9c407e

Drop pytables < 3.4.2

aac634d

Bump numpy requirement in install.py

f18880d

Add output for debugging tests/plotting/test_datetimelike

f81ad9f

h-vetinari force-pushed the bump_numpy branch 2 times, most recently from f4f5b46 to f81ad9f Compare October 10, 2018 06:51

Drop scipy < 0.18.1

c29b478

h-vetinari force-pushed the bump_numpy branch from f4f5b46 to c29b478 Compare October 10, 2018 07:10

Fix debugging output

7ecc5f2

auvipy approved these changes Oct 10, 2018

View reviewed changes

jreback requested changes Oct 10, 2018

View reviewed changes

jbrockmendel reviewed Oct 10, 2018

View reviewed changes

Reduce min version for numexpr/pytables after conda investigation

6e533e7

h-vetinari mentioned this pull request Oct 12, 2018

CI: Improve CircleCI #22763

Closed

MPLMPLMPLMPLMPLMPLMPLMPLMPLMPLMPL

079bdaf

h-vetinari force-pushed the bump_numpy branch from 997dd9d to 079bdaf Compare October 12, 2018 20:08

h-vetinari added 2 commits October 12, 2018 22:58

Revert "shift debugging pin away from doc build"

162458b

This reverts commit ededd73.

Revert "Revert "Partially revert mpl-pins used for debugging""

cdd497d

This reverts commit 9be3d10.

Merge branch 'master' into bump_numpy

d8587a8

TomAugspurger reviewed Oct 14, 2018

View reviewed changes

Retrigger CI

8e37179

auvipy approved these changes Oct 15, 2018

View reviewed changes

jreback approved these changes Oct 15, 2018

View reviewed changes

Merge remote-tracking branch 'upstream/master' into bump_numpy

d097b43

WillAyd requested changes Oct 15, 2018

View reviewed changes

WillAyd approved these changes Oct 15, 2018

View reviewed changes

jreback merged commit aaa69d1 into pandas-dev:master Oct 15, 2018

h-vetinari deleted the bump_numpy branch October 15, 2018 18:28

tm9k1 pushed a commit to tm9k1/pandas that referenced this pull request Nov 19, 2018

DEPS: drop numpy < 1.12 (pandas-dev#23062)

fe3bc70

h-vetinari mentioned this pull request Nov 19, 2018

CLN/DEPS: Clean up post numpy bump to 1.12 #23796

Merged

3 tasks

h-vetinari mentioned this pull request Dec 19, 2018

CLN: post-post numpy bump #24365

Merged

h-vetinari mentioned this pull request Mar 6, 2019

DEPS: Bump numpy to 1.13.3 #25554

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

DEPS: drop numpy < 1.12 #23062

DEPS: drop numpy < 1.12 #23062

h-vetinari commented Oct 9, 2018 •

edited

Loading

pep8speaks commented Oct 9, 2018 •

edited

Loading

codecov bot commented Oct 9, 2018 •

edited

Loading

WillAyd left a comment

WillAyd Oct 9, 2018

h-vetinari Oct 9, 2018

WillAyd left a comment

h-vetinari commented Oct 9, 2018

jreback commented Oct 9, 2018

h-vetinari commented Oct 9, 2018

WillAyd Oct 10, 2018

h-vetinari Oct 10, 2018

WillAyd Oct 10, 2018

h-vetinari Oct 10, 2018

h-vetinari Oct 10, 2018

jreback Oct 10, 2018

jbrockmendel commented Oct 10, 2018

jbrockmendel Oct 10, 2018

jbrockmendel commented Oct 12, 2018

h-vetinari commented Oct 12, 2018 •

edited

Loading

jreback commented Oct 14, 2018

h-vetinari commented Oct 14, 2018

h-vetinari commented Oct 14, 2018

jreback commented Oct 14, 2018

TomAugspurger left a comment

TomAugspurger commented Oct 14, 2018

jreback commented Oct 15, 2018

h-vetinari commented Oct 15, 2018

WillAyd Oct 15, 2018

TomAugspurger Oct 15, 2018

h-vetinari Oct 15, 2018

jreback commented Oct 15, 2018

h-vetinari commented Oct 15, 2018

DEPS: drop numpy < 1.12 #23062

DEPS: drop numpy < 1.12 #23062

Conversation

h-vetinari commented Oct 9, 2018 • edited Loading

pep8speaks commented Oct 9, 2018 • edited Loading

Comment last updated on October 12, 2018 at 16:26 Hours UTC

codecov bot commented Oct 9, 2018 • edited Loading

Codecov Report

WillAyd left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

WillAyd left a comment

Choose a reason for hiding this comment

h-vetinari commented Oct 9, 2018

jreback commented Oct 9, 2018

h-vetinari commented Oct 9, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jbrockmendel commented Oct 10, 2018

Choose a reason for hiding this comment

jbrockmendel commented Oct 12, 2018

h-vetinari commented Oct 12, 2018 • edited Loading

jreback commented Oct 14, 2018

h-vetinari commented Oct 14, 2018

h-vetinari commented Oct 14, 2018

jreback commented Oct 14, 2018

TomAugspurger left a comment

Choose a reason for hiding this comment

TomAugspurger commented Oct 14, 2018

jreback commented Oct 15, 2018

h-vetinari commented Oct 15, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Oct 15, 2018

h-vetinari commented Oct 15, 2018

h-vetinari commented Oct 9, 2018 •

edited

Loading

pep8speaks commented Oct 9, 2018 •

edited

Loading

codecov bot commented Oct 9, 2018 •

edited

Loading

h-vetinari commented Oct 12, 2018 •

edited

Loading