Skip to content

CI: Fixed NumPy pinning in conda-build #19575

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 8 commits into from
Feb 9, 2018
Merged

CI: Fixed NumPy pinning in conda-build #19575

merged 8 commits into from
Feb 9, 2018

Conversation

TomAugspurger
Copy link
Contributor

@TomAugspurger TomAugspurger commented Feb 7, 2018 •
edited by jreback
Loading

Closes #19572
closes #19153

(testing locally as well too)

@TomAugspurger TomAugspurger added the CI Continuous Integration label Feb 7, 2018
@TomAugspurger TomAugspurger added this to the 0.23.0 milestone Feb 7, 2018
@TomAugspurger
Copy link
Contributor Author

Built fine locally.

@jreback
Copy link
Contributor

jreback commented Feb 7, 2018

local building has always worked fine, its branches that are failing only sometimes.

@TomAugspurger
Copy link
Contributor Author

Ah, OK.

Thoughts on failing that worker entirely if the conda-build command fails?

I also see that the command we run is time conda build ./conda.recipe --numpy 1.13 --python 3.5 -q --no-test

I suppose that should be bumped?

@jreback
Copy link
Contributor

jreback commented Feb 7, 2018

yes should bump that. I think there is an issue about 1.14 somewhere as well.

@TomAugspurger
Unpin NumPy
4de1e1e 
Quite install
@codecov
Copy link

codecov bot commented Feb 7, 2018 •
edited
Loading

Codecov Report

Merging #19575 into master will increase coverage by <.01%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #19575      +/-   ##
==========================================
+ Coverage   91.62%   91.62%   +<.01%     
==========================================
  Files         150      150              
  Lines       48781    48795      +14     
==========================================
+ Hits        44695    44708      +13     
- Misses       4086     4087       +1
Flag Coverage Δ
#multiple 89.99% <ø> (ø) ⬆️
#single 41.72% <ø> (-0.01%) ⬇️
Impacted Files Coverage Δ
pandas/core/indexes/category.py 97.26% <0%> (-0.26%) ⬇️
pandas/core/groupby.py 92.12% <0%> (-0.05%) ⬇️
pandas/core/indexes/datetimelike.py 97.05% <0%> (-0.05%) ⬇️
pandas/core/frame.py 97.42% <0%> (ø) ⬆️
pandas/core/sparse/frame.py 94.81% <0%> (ø) ⬆️
pandas/core/indexing.py 93.11% <0%> (ø) ⬆️
pandas/io/formats/format.py 96.25% <0%> (+0.01%) ⬆️
pandas/core/generic.py 95.93% <0%> (+0.01%) ⬆️
pandas/core/reshape/merge.py 94.25% <0%> (+0.03%) ⬆️
pandas/core/arrays/categorical.py 94.87% <0%> (+0.13%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6b0c7e7...6f5f87a. Read the comment docs.

@TomAugspurger
Copy link
Contributor Author

numba packages for numpy 1.14 are out now, so I removed that pin.

Trying without any pins for the conda build recipe. We'll see.

@jreback
Copy link
Contributor

jreback commented Feb 8, 2018

so this is still initially installing 1.13 and building, then upgrading to 1.14 after, not sure whey that is. I also think we DO need the pinnning on the conda build, it looks like its using a pretty old version (1.11) for building. It still works but is slightly odd.

@TomAugspurger
Copy link
Contributor Author

OK, I think I have it so we build against 1.11 and run against NumPy latest (lmk if you want something else for run). I think we do want to build against 1.11 though, since that's the oldest ABI compatible NumPy version that supports Python 3.5, and it's what we use for conda-forge / manylinux builds.

@jreback
Copy link
Contributor

jreback commented Feb 8, 2018

yeah that kind right
we want to build against the oldest version we support for ABI - i guess u can build with 1.9 but maybe we have something in the recipe which doesn’t allow that

@jreback
Copy link
Contributor

jreback commented Feb 8, 2018

this still looks odd. we are installing 1.14, then building with 1.11 (i guess min numpy version), but end up with 13.3 (see show_versions() at the end)

# Name                    Version                   Build  Channel
pandas                    0.23.0.dev0     np111py35_259_g5d52a3a    local

@TomAugspurger
Copy link
Contributor Author

Missed a pin. Hopefully now.

@TomAugspurger
Copy link
Contributor Author

Still ended up with 1.13. One of these packages is forcing a downgrade.

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    python-dateutil-2.6.1      |           py35_0         237 KB  conda-forge
    parquet-cpp-1.2.0          |                0         1.2 MB  conda-forge
    openssl-1.0.2n             |                0         3.5 MB  conda-forge
    numpy-1.13.3               |py35_nomklhb7e7d7c_0         3.8 MB
    pandas-0.22.0              |           py35_0        25.6 MB  conda-forge
    certifi-2017.11.5          |           py35_0         195 KB  conda-forge
    feather-format-0.4.0       |           py35_2           5 KB  conda-forge
    arrow-cpp-0.5.0            |      np113py35_2         2.0 MB  conda-forge
    pyarrow-0.5.0              |      np113py35_0         600 KB  conda-forge
    jemalloc-4.4.0             |                0         2.3 MB  conda-forge
    ca-certificates-2017.11.5  |                0         145 KB  conda-forge
    ------------------------------------------------------------
                                           Total:        39.6 MB

The following NEW packages will be INSTALLED:

    arrow-cpp:       0.5.0-np113py35_2           conda-forge
    feather-format:  0.4.0-py35_2                conda-forge
    jemalloc:        4.4.0-0                     conda-forge
    pandas:          0.22.0-py35_0               conda-forge
    parquet-cpp:     1.2.0-0                     conda-forge
    pyarrow:         0.5.0-np113py35_0           conda-forge
    python-dateutil: 2.6.1-py35_0                conda-forge

The following packages will be UPDATED:

    ca-certificates: 2017.08.26-h1d4fec5_0                   --> 2017.11.5-0                 conda-forge
    openssl:         1.0.2n-hb7f436b_0                       --> 1.0.2n-0                    conda-forge

The following packages will be DOWNGRADED:

    certifi:         2018.1.18-py35_0                        --> 2017.11.5-py35_0            conda-forge
    numpy:           1.14.0-py35_nomklh7cdd4dd_0             [nomkl] --> 1.13.3-py35_nomklhb7e7d7c_0             [nomkl]

pyarrow is pinned to 0.5 for this build. Do you have a preference for

  1. pyarrow 0.5 & NumPy 1.13
  2. pyarrow / NumPy latest

If 1, this can be merged. Else I'll remove that pyarrow pin.

@jreback
Copy link
Contributor

jreback commented Feb 9, 2018

pyarrow 0.5 & NumPy 1.13
pyarrow / NumPy latest

hmm would just bump the pyarrow pin to 0.6.0 i think.

@TomAugspurger
Copy link
Contributor Author

TomAugspurger commented Feb 9, 2018 via email

@TomAugspurger
Copy link
Contributor Author

OK, https://api.travis-ci.org/v3/job/339442588/log.txt did it correctly. To summarize

  1. Build against NumPy 1.11: https://travis-ci.org/pandas-dev/pandas/jobs/339442588#L1131
  2. Run in an env with NumPy 1.1.4: https://travis-ci.org/pandas-dev/pandas/jobs/339442588#L947
  3. Pyarrow is pinned to 0.7.1: https://travis-ci.org/pandas-dev/pandas/jobs/339442588#L1038

@TomAugspurger TomAugspurger merged commit a214915 into pandas-dev:master Feb 9, 2018
@TomAugspurger TomAugspurger deleted the conda-build branch February 9, 2018 16:11
harisbal pushed a commit to harisbal/pandas that referenced this pull request Feb 28, 2018
@TomAugspurger
CI: Fixed NumPy pinning in conda-build (pandas-dev#19575)
b5d4128 
* CI: Fixed NumPy pinning in conda-build

* Unpin NumPy

Quite install

* Pin numpy

* Unpin everywhere else

* Build vs. 1.11

* remove one more pin

* Remove one more pin

* bump pyarrow
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CI Continuous Integration
Projects
None yet
Milestone
0.23.0
Development

Successfully merging this pull request may close these issues.

CI: conda build test failing Revert CI: Pin NumPy in 3.6 build
2 participants
@TomAugspurger @jreback