RLS: 0.20.0 #16049

jorisvandenbossche · 2017-04-18T14:17:32Z

We are nearing the 0.20.0 release, so opening an issue to track this.

Remaining issues/PRs with the 0.20 tag: https://github.com/pandas-dev/pandas/milestones/0.20.0
If there are other remaining things, time to speak up

cc @pandas-dev/pandas-core

jorisvandenbossche · 2017-04-18T14:28:24Z

Current target for the release candidate is this Friday I think, @TomAugspurger ?

TomAugspurger · 2017-04-18T14:47:51Z

Yep, that's the goal.

jreback · 2017-04-18T14:52:17Z

this is just an RC this friday. We prob need 2 or more weeks for that.

jorisvandenbossche · 2017-04-18T14:54:19Z

yep, that is what I meant, sorry for the confusion

TomAugspurger · 2017-04-21T19:35:05Z

Doing the RC tonight or early tomorrow. Of the remaining issues, #16088 is a blocker and #16086 would be nice. Any I am missing?

@jreback do you have a preference on tagging v0.21.dev immediately after tagging the RC, versus waiting to tag v0.21.dev until the full 0.20 release? If we tag 0.21.dev now then we can merge whatever we want into master, and I'll just backport whatever needs to be backported between now and the 0.20 release.

jreback · 2017-04-21T21:55:43Z

no those are fine. I'll have a look in a few.

hmm it feels weird to tag 0.21.dev before 0.20.0.

jorisvandenbossche · 2017-04-21T23:08:15Z

hmm it feels weird to tag 0.21.dev before 0.20.0.

I don't think there is anything strange about that. It is just a practical choice to make about when we branch the 0.20.x branch (the tagging is just a consequence of that): now with the rc, or only after the final release.

For me both options are OK. Branching now makes it a bit easier for progressing / merging PRs (we have to be less concerned about which PR we merge, or which has to wait until 0.20 is out), but is more work for Tom to backport.

jorisvandenbossche · 2017-04-21T23:10:58Z

@TomAugspurger there are still some doc issues open for 0.20, but you don't have to be too concerned about them, as for rc's we typically point to the dev docs which are further updated (I mean, tagging the rc does not freeze the docs, as it is with the final release)

jreback · 2017-04-21T23:13:53Z

ok open to whatever tagging makes sense.

yeah doc issues certainly can be done after (as well as a small number of additional fixes, already marked for 0.20.0). mainly the sorting one.

jreback · 2017-04-22T02:59:26Z

@TomAugspurger ok all of the things on my list are merged.

TomAugspurger · 2017-04-22T03:03:47Z

👍 Building now and testing locally. I'll probably tag tonight.

Conda-forge hasn't settled on pre-release stuff yet, so I'll probably wait till tomorrow to make announcements (have to finish setting up this windows VM).

jreback · 2017-04-22T03:31:03Z

ok lmk if u need anything
though won't be around most of tomorrow

TomAugspurger · 2017-04-22T03:57:45Z

Ok, tagged and made the release on github. Doing all the conda stuff and mailing list announcements tomorrow.

TomAugspurger · 2017-04-23T01:46:07Z

Just sent out the announce email. All the wheels are on PyPI. Our conda channel has packages for OSX and 64-bit windows. I haven't setup a 32-bit windows environment yet. Linux packages should be up shortly (built 0.19.2 on accident).

jreback · 2017-04-23T13:34:57Z

thanks @TomAugspurger !

Since we didn't tag 0.21.0.dev (I think we should do this after tag of 0.20.0 final), then going to continue merging small things to master.

topper-123 · 2017-04-25T16:20:13Z

Could #16111 be fixed before 0.20.final? It's a bug that causes Pandas to crash the whole Python interpreter, leaving no traceback or clue as to what went wrong.

I do not have the capabilities to fix this myself, though, so consider this just a plea from a happy user. Also, I've made my own (non-optimal) workaround, but this could bring a negative surprise to others when it happens...

EDIT: This has been taken care of. Thanks!

dhirschfeld · 2017-04-26T02:48:48Z

The 0.20.0 release candidate breaks a large amount of my code and AFAICS the breakage was in previously non-deprecate modules so 3rd party libraries haven't had time to adapt to the new api.

xref: #16137, #16138

I think there should be serious consideration given to providing a compatibility layer for at least one minor release. It's not hard to do and doesn't need to stick around for very long, but will give 3rd party libraries a change to adapt. The required work is in PR #16140

jorisvandenbossche · 2017-04-26T08:51:45Z

Which is why we want people to test with the rc, thanks a lot for your feedback @dhirschfeld !

TomAugspurger · 2017-05-01T11:56:39Z

Any remaining issues / fixes people want in for 0.20? #16171 is the only blocker I see.

I'll probably do the release Tuesday or Wednesday if that works for everyone.

jreback · 2017-05-04T11:40:23Z

after #16223 and remaining doc fixes I think good to go. We could do an rc2 (maybe just conda packages). For a quick check? up to you @TomAugspurger

TomAugspurger · 2017-05-04T11:51:06Z

Yeah, I could go either way on an RC2... May as do one just for the conda packages, and test against dask, statsmodels, etc.

jreback · 2017-05-04T12:03:30Z

sure that works.

TomAugspurger · 2017-05-04T12:06:36Z

Should I even push an RC2 to PyPI? Or just tag, push to github, and then build the conda pkgs?

jreback · 2017-05-04T12:21:42Z

no, I would just make some conda packages.

TomAugspurger · 2017-05-04T14:52:54Z

Going to do the RC2 now.

We have the two doc issues outstanding, but those can go in between RC2 and the release.

TomAugspurger · 2017-05-04T15:50:25Z

@jreback a couple GBQ failures on master

Could the be related to googleapis/python-bigquery-pandas#34? (I haven't looked at that PR)

jreback · 2017-05-04T16:10:53Z

I restarted on master. this should work. I released a new version of gbq because was using PandasError (which we removed)

jreback · 2017-05-04T16:11:38Z

these gbq sometimes fail if multiple things are running against it (e.g. 0.20rc2 tag & master both launched at the same time).

TomAugspurger · 2017-05-04T16:28:22Z

Dask, statsmodels, and seaborn all looked OK on rc2. I did a windows build and test locally, and that passed as well.

jreback · 2017-05-04T16:38:54Z

great @TomAugspurger

yeah master is green now (and rc2 I just restarted those builds).

cgohlke · 2017-05-04T20:22:18Z

I get three test failures on win-amd64-py3.6.1 with numpy-1.11.3+mkl. The test_shim failure is fixed by #16239

running: pytest --skip-slow --skip-network X:\Python36\lib\site-packages\pandas
<snip>
================================== FAILURES ===================================
__________________________________ test_shim __________________________________

    def test_shim():
        # https://github.com/pandas-dev/pandas/pull/16059
        # Remove in 0.21
        with tm.assert_produces_warning(FutureWarning,
                                        check_stacklevel=False):
>           from pandas.formats.style import Styler as _styler  # noqa
E           ModuleNotFoundError: No module named 'pandas.formats'

X:\Python36\lib\site-packages\pandas\tests\io\formats\test_style.py:866: ModuleNotFoundError
______________________ TestSeriesAnalytics.test_overflow ______________________

self = <pandas.tests.series.test_analytics.TestSeriesAnalytics object at 0x0000019B01E52400>

    def test_overflow(self):
        # GH 6915
        # overflowing on the smaller int dtypes
        for dtype in ['int32', 'int64']:
            v = np.arange(5000000, dtype=dtype)
            s = Series(v)

            # no bottleneck
            result = s.sum(skipna=False)
            assert int(result) == v.sum(dtype='int64')
            result = s.min(skipna=False)
            assert int(result) == 0
            result = s.max(skipna=False)
            assert int(result) == v[-1]

            # use bottleneck if available
            result = s.sum()
>           assert int(result) == v.sum(dtype='int64')
E           AssertionError: assert 1642668640 == 12499997500000
E            +  where 1642668640 = int(1642668640)
E            +  and   12499997500000 = <built-in method sum of numpy.ndarray object at 0x0000019B061C0CB0>(dtype='int64')
E            +    where <built-in method sum of numpy.ndarray object at 0x0000019B061C0CB0> = array([      0,       1,       2, ..., 4999997, 4999998, 4999999], dtype=int64).sum

X:\Python36\lib\site-packages\pandas\tests\series\test_analytics.py:69: AssertionError
________________________ TestSeriesAnalytics.test_sum _________________________

self = <pandas.tests.series.test_analytics.TestSeriesAnalytics object at 0x0000019B69C32EB8>

    def test_sum(self):
>       self._check_stat_op('sum', np.sum, check_allna=True)

X:\Python36\lib\site-packages\pandas\tests\series\test_analytics.py:96:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
X:\Python36\lib\site-packages\pandas\tests\series\test_analytics.py:556: in _check_stat_op
    testit()
X:\Python36\lib\site-packages\pandas\tests\series\test_analytics.py:535: in testit
    assert_almost_equal(float(f(s)), float(alternate(s.values)))
X:\Python36\lib\site-packages\pandas\util\testing.py:177: in assert_almost_equal
    **kwargs)
pandas\_libs\testing.pyx:59: in pandas._libs.testing.assert_almost_equal (pandas\_libs\testing.c:4156)
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???
E   AssertionError: expected 1099511628275500.00000 but got 499500.00000, with decimal 5

pandas\_libs\testing.pyx:209: AssertionError
=========================== pytest-warning summary ============================
WC1 X:\Python36\lib\site-packages\pandas\tests\test_config.py cannot collect test class 'TestConfig' because it has a __init__ constructor
 3 failed, 9606 passed, 542 skipped, 21 xfailed, 1 xpassed, 1 pytest-warnings in 631.87 seconds

TomAugspurger · 2017-05-04T20:24:17Z

@cgohlke thanks. Is that on 32-bit windows?

cgohlke · 2017-05-04T20:25:30Z

It is 64-bit Python 3.6.1 from python.org on Windows 10.

cgohlke · 2017-05-04T20:36:50Z

test_overflow and test_sum pass after uninstalling bottleneck-1.2.0.

TomAugspurger · 2017-05-04T20:43:18Z

test_overflow and test_sum pass after uninstalling bottleneck-1.2.0.

Thanks, I'm able to reproduce on my windows VM with bottleneck installed (I didn't have it installed earlier). Taking a look at what's going on now.

jreback · 2017-05-04T20:45:32Z

we might not have bottleneck installed for windows test builds

just skip them for now

jreback · 2017-05-04T21:02:38Z

we are testing on 2.7 w/bottleneck on windows, but not 3.6....will add and skip/fix those.

TomAugspurger · 2017-05-04T21:14:14Z

we are testing on 2.7 w/bottleneck on windows, but not 3.6....will add and skip/fix those.

Yep, it is definitely in bottleneck. Are you doing a PR for skipping those, or should I?

jreback · 2017-05-04T21:17:45Z

testing now

jreback · 2017-05-04T21:17:56Z

https://ci.appveyor.com/project/jreback/pandas/build/1.0.3685/job/y1rv3i97bs1ruet6

xref pandas-dev#16049 (comment)

jreback · 2017-05-04T21:43:26Z

xfailed these tests: https://ci.appveyor.com/project/jreback/pandas/build/1.0.3686
merging on pass (I think we have an issue about this overflow in any event, here: #15453)

so when I fix the bottlneck things in 0.21.0 will fix this.

TomAugspurger · 2017-05-04T21:45:05Z

@jreback thanks bottleneck issue is here: pydata/bottleneck#163

* CI: add bottleneck for 3.6 on windows * TST: xfail some bottleneck tests on windows xref #16049 (comment)

jreback · 2017-05-04T23:32:58Z

@TomAugspurger ok everything lgtm. assuming master finishes and passes. cut when you are ready. (obviously if you see any doc things when you are doing, pls push fixes).

TomAugspurger · 2017-05-05T02:00:39Z

@jreback the whatsnew mentions a new hash_tuples. Do we want to expose that in pandas.util along with hash_pandas_object?

jreback · 2017-05-05T02:05:15Z

that's not public it could be i guess

TomAugspurger · 2017-05-05T02:06:56Z

I only ask since it was in a release note item. Though, dask isn't using it, so maybe keep it private for now?

jreback · 2017-05-05T02:08:13Z

yeah is meant to be private

OT - date of whatsnew note needs changing

TomAugspurger · 2017-05-05T02:09:08Z

Got it in #16245

I'll push one more commit removing that reference to hash_tuples, then tag it. Going to wait for that doc build to finish first.

topper-123 · 2017-05-06T20:37:33Z

Has 0.20 been released? pandas.pydata.org says 0.19.2 is the current release..

jreback · 2017-05-06T20:52:40Z

just refresh and you shall see.

* CI: add bottleneck for 3.6 on windows * TST: xfail some bottleneck tests on windows xref pandas-dev#16049 (comment)

jorisvandenbossche added the Release label Apr 18, 2017

jreback added this to the 0.20.0 milestone Apr 18, 2017

jreback added a commit to jreback/pandas that referenced this issue May 4, 2017

TST: xfail some bottleneck tests on windows

6055a72

xref pandas-dev#16049 (comment)

jreback mentioned this issue May 4, 2017

TST: xfail some bottleneck on windows #16240

Merged

jreback added a commit that referenced this issue May 4, 2017

TST: xfail some bottleneck on windows (#16240)

5dd3b43

* CI: add bottleneck for 3.6 on windows * TST: xfail some bottleneck tests on windows xref #16049 (comment)

TomSmithCGAT mentioned this issue May 5, 2017

{ts} include single chrom test CGATOxford/UMI-tools#110

Merged

jreback closed this as completed May 6, 2017

pcluo pushed a commit to pcluo/pandas that referenced this issue May 22, 2017

TST: xfail some bottleneck on windows (pandas-dev#16240)

428f06d

* CI: add bottleneck for 3.6 on windows * TST: xfail some bottleneck tests on windows xref pandas-dev#16049 (comment)

stangirala pushed a commit to stangirala/pandas that referenced this issue Jun 11, 2017

TST: xfail some bottleneck on windows (pandas-dev#16240)

3bcbc90

* CI: add bottleneck for 3.6 on windows * TST: xfail some bottleneck tests on windows xref pandas-dev#16049 (comment)

RLS: 0.20.0 #16049

RLS: 0.20.0 #16049

Comments

jorisvandenbossche commented Apr 18, 2017

jorisvandenbossche commented Apr 18, 2017 • edited Loading

TomAugspurger commented Apr 18, 2017

jreback commented Apr 18, 2017

jorisvandenbossche commented Apr 18, 2017

TomAugspurger commented Apr 21, 2017

jreback commented Apr 21, 2017

jorisvandenbossche commented Apr 21, 2017

jorisvandenbossche commented Apr 21, 2017

jreback commented Apr 21, 2017

jreback commented Apr 22, 2017

TomAugspurger commented Apr 22, 2017

jreback commented Apr 22, 2017

TomAugspurger commented Apr 22, 2017

TomAugspurger commented Apr 23, 2017

jreback commented Apr 23, 2017

topper-123 commented Apr 25, 2017 • edited Loading

dhirschfeld commented Apr 26, 2017

jorisvandenbossche commented Apr 26, 2017

TomAugspurger commented May 1, 2017

jreback commented May 4, 2017

TomAugspurger commented May 4, 2017

jreback commented May 4, 2017

TomAugspurger commented May 4, 2017 • edited Loading

jreback commented May 4, 2017

TomAugspurger commented May 4, 2017

TomAugspurger commented May 4, 2017

jreback commented May 4, 2017

jreback commented May 4, 2017

TomAugspurger commented May 4, 2017

jreback commented May 4, 2017

cgohlke commented May 4, 2017

TomAugspurger commented May 4, 2017

cgohlke commented May 4, 2017

cgohlke commented May 4, 2017

TomAugspurger commented May 4, 2017

jreback commented May 4, 2017

jreback commented May 4, 2017

TomAugspurger commented May 4, 2017

jreback commented May 4, 2017

jreback commented May 4, 2017

jreback commented May 4, 2017

TomAugspurger commented May 4, 2017

jreback commented May 4, 2017

TomAugspurger commented May 5, 2017

jreback commented May 5, 2017

TomAugspurger commented May 5, 2017

jreback commented May 5, 2017

TomAugspurger commented May 5, 2017 • edited Loading

topper-123 commented May 6, 2017

jreback commented May 6, 2017

jorisvandenbossche commented Apr 18, 2017 •

edited

Loading

topper-123 commented Apr 25, 2017 •

edited

Loading

TomAugspurger commented May 4, 2017 •

edited

Loading

TomAugspurger commented May 5, 2017 •

edited

Loading