CI: Build failures #19137

TomAugspurger · 2018-01-08T14:50:43Z

e.g. https://travis-ci.org/pandas-dev/pandas/jobs/326347971

[script_single]
pytest -s -m single --strict --cov=pandas --cov-report xml:/tmp/cov-single.xml --junitxml=/tmp/single.xml --skip-slow --skip-network pandas
Traceback (most recent call last):
  File "/home/travis/miniconda3/envs/pandas/lib/python3.6/site-packages/_pytest/config.py", line 328, in _getconftestmodules
    return self._path2confmods[path]
KeyError: local('/home/travis/build/pandas-dev/pandas/pandas')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/travis/miniconda3/envs/pandas/lib/python3.6/site-packages/_pytest/config.py", line 359, in _importconftest
    return self._conftestpath2mod[conftestpath]
KeyError: local('/home/travis/build/pandas-dev/pandas/pandas/conftest.py')
During handling of the above exception, another exception occurred:
RuntimeError: module compiled against API version 0xc but this version of numpy is 0xb
Traceback (most recent call last):
  File "/home/travis/miniconda3/envs/pandas/lib/python3.6/site-packages/_pytest/config.py", line 328, in _getconftestmodules
    return self._path2confmods[path]
KeyError: local('/home/travis/build/pandas-dev/pandas/pandas')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/travis/miniconda3/envs/pandas/lib/python3.6/site-packages/_pytest/config.py", line 359, in _importconftest
    return self._conftestpath2mod[conftestpath]
KeyError: local('/home/travis/build/pandas-dev/pandas/pandas/conftest.py')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/travis/build/pandas-dev/pandas/pandas/__init__.py", line 26, in <module>
    from pandas._libs import (hashtable as _hashtable,
  File "/home/travis/build/pandas-dev/pandas/pandas/_libs/__init__.py", line 4, in <module>
    from .tslib import iNaT, NaT, Timestamp, Timedelta, OutOfBoundsDatetime
  File "pandas/_libs/tslibs/conversion.pxd", line 11, in init pandas._libs.tslib
    cdef class _TSObject:
  File "pandas/_libs/tslibs/conversion.pyx", line 1, in init pandas._libs.tslibs.conversion
    # -*- coding: utf-8 -*-
ImportError: numpy.core.multiarray failed to import
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/home/travis/miniconda3/envs/pandas/lib/python3.6/site-packages/_pytest/config.py", line 365, in _importconftest
    mod = conftestpath.pyimport()
  File "/home/travis/miniconda3/envs/pandas/lib/python3.6/site-packages/py/_path/local.py", line 668, in pyimport
    __import__(modname)
  File "/home/travis/build/pandas-dev/pandas/pandas/__init__.py", line 35, in <module>
    "the C extensions first.".format(module))
ImportError: C extension: numpy.core.multiarray failed to import not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace --force' to build the C extensions first.
ERROR: could not load /home/travis/build/pandas-dev/pandas/pandas/conftest.py

Looking into it now.

The text was updated successfully, but these errors were encountered:

TomAugspurger · 2018-01-08T14:54:21Z

Looks like we compiled against numpy 1.14 but were downgraded to 1.11.

TomAugspurger · 2018-01-08T14:56:58Z

https://travis-ci.org/pandas-dev/pandas/jobs/326153864#L2059

The following packages will be DOWNGRADED:
    numpy:                         1.14.0-py36_blas_openblas_200 conda-forge [blas_openblas] --> 1.13.3-py36_blas_openblas_201 conda-forge [blas_openblas]

TomAugspurger · 2018-01-08T15:10:37Z

Looks like it's numba:

 numba-0.36.2               |np113py36h7a10136_0

TomAugspurger · 2018-01-08T15:12:38Z

Which is presumably being pulled in through fastparquet.

earthgecko · 2018-01-08T16:03:07Z

I got this error on 2018-01-05 as well with Python-2.7 pandas==0.22.0 and numpy==1.13.3 breaking with the same error

  File "/opt/python_virtualenv/projects/skyline-luminosity-py2714/lib/python2.7/site-packages/pandas/__init__.py", line 35, in <module>
    "the C extensions first.".format(module))
ImportError: C extension: numpy.core.multiarray failed to import not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace --force' to build the C extensions first.

As per:

ImportError: C extension: numpy.core.multiarray failed to import not built. #18281 - ImportError: C extension: numpy.core.multiarray failed to import not built
0.21.0 wheels broken for older (<1.13) numpy version #18530 (comment)

I rebuilt the virtualenv and explicitly installed numpy first and it was still broken and in my app log I found gunicorn report

RuntimeError: module compiled against API version 0xc but this version of numpy is 0xb
...
gunicorn.errors.HaltServer: <HaltServer 'Worker failed to boot.' 3>
RuntimeError: module compiled against API version 0xc but this version of numpy is 0xb
Traceback (most recent call last):
  File "/opt/skyline/github/skyline/skyline/crucible/agent.py", line 16, in <module>
    from crucible import Crucible
  File "/opt/skyline/github/skyline/skyline/crucible/crucible.py", line 41, in <module>
    from crucible_algorithms import run_algorithms
  File "/opt/skyline/github/skyline/skyline/crucible/crucible_algorithms.py", line 2, in <module>
    import pandas
  File "/opt/python_virtualenv/projects/skyline-luminosity-py2714/lib/python2.7/site-packages/pandas/__init__.py", line 35, in <module>
    "the C extensions first.".format(module))
ImportError: C extension: numpy.core.multiarray failed to import not built. If you want to import pandas from the source directory, you may need to run 'python setup.py build_ext --inplace --force' to build the C extensions first.
error - failed to start crucible

I thought seeing as it was said to have been fixed in 0.21 it must be me :)
I figured that I would roll back to 0.20.3 and keep an eye on "numpy.core.multiarray" - https://github.com/pandas-dev/pandas/issues?utf8=%E2%9C%93&q=is%3Aissue+is%3Aopen+numpy.core.multiarray
And I found this, so I thought I would share as this inflicted 2.7 as well and this suggests it was not me just me or my env :) but possibly something deeper.

TomAugspurger · 2018-01-08T16:18:25Z

@earthgecko this is just for our CI.

what version of python are you on, and how did you install numpy and pandas?

TomAugspurger · 2018-01-08T16:20:22Z

Oh, I see 2.7 in your virtualenv name. You installed numpy=1.13 before installing pandas?

TomAugspurger · 2018-01-08T16:24:36Z

Can you paste the output of pip install numpy and pip install pandas on your machine?

TomAugspurger · 2018-01-08T16:30:35Z

e.g.

$ docker run -it python bash                                                                                                                                                                                                                                                              root@63b72e7e2cf8:/# python
root@63b72e7e2cf8:/# python3 -m pip install numpy==1.13.3
Collecting numpy==1.13.3
  Downloading numpy-1.13.3-cp36-cp36m-manylinux1_x86_64.whl (17.0MB)
    100% |████████████████████████████████| 17.0MB 99kB/s
Installing collected packages: numpy
Successfully installed numpy-1.13.3
root@63b72e7e2cf8:/# python3 -m pip install pandas===0.22.0
Collecting pandas===0.22.0
  Downloading pandas-0.22.0-cp36-cp36m-manylinux1_x86_64.whl (26.2MB)
    100% |████████████████████████████████| 26.3MB 66kB/s
Collecting pytz>=2011k (from pandas===0.22.0)
  Downloading pytz-2017.3-py2.py3-none-any.whl (511kB)
    100% |████████████████████████████████| 512kB 2.0MB/s
Requirement already satisfied: numpy>=1.9.0 in /usr/local/lib/python3.6/site-packages (from pandas===0.22.0)
Collecting python-dateutil>=2 (from pandas===0.22.0)
  Downloading python_dateutil-2.6.1-py2.py3-none-any.whl (194kB)
    100% |████████████████████████████████| 194kB 2.5MB/s
Collecting six>=1.5 (from python-dateutil>=2->pandas===0.22.0)
  Downloading six-1.11.0-py2.py3-none-any.whl
Installing collected packages: pytz, six, python-dateutil, pandas
Successfully installed pandas-0.22.0 python-dateutil-2.6.1 pytz-2017.3 six-1.11.0
root@63b72e7e2cf8:/# python
Python 3.6.4 (default, Dec 21 2017, 01:35:12)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas

earthgecko · 2018-01-08T17:06:27Z

Yes thanks now 0.22.0 works for me now, why did it not work the other day?

I think I had done some change in my requirements and gone through 2 numpy version and not reinstalled pandas or seems used --no-cache-dir with pip which reflecting on this info and some more debugging is probably the case.

TomAugspurger added the CI Continuous Integration label Jan 8, 2018

TomAugspurger mentioned this issue Jan 8, 2018

CI: Pin NumPy in 3.6 build #19138

Merged

TomAugspurger closed this as completed in #19138 Jan 8, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CI: Build failures #19137

CI: Build failures #19137

TomAugspurger commented Jan 8, 2018

TomAugspurger commented Jan 8, 2018 •

edited

Loading

TomAugspurger commented Jan 8, 2018

TomAugspurger commented Jan 8, 2018

TomAugspurger commented Jan 8, 2018

earthgecko commented Jan 8, 2018

TomAugspurger commented Jan 8, 2018

TomAugspurger commented Jan 8, 2018

TomAugspurger commented Jan 8, 2018

TomAugspurger commented Jan 8, 2018

earthgecko commented Jan 8, 2018 •

edited

Loading

CI: Build failures #19137

CI: Build failures #19137

Comments

TomAugspurger commented Jan 8, 2018

TomAugspurger commented Jan 8, 2018 • edited Loading

TomAugspurger commented Jan 8, 2018

TomAugspurger commented Jan 8, 2018

TomAugspurger commented Jan 8, 2018

earthgecko commented Jan 8, 2018

TomAugspurger commented Jan 8, 2018

TomAugspurger commented Jan 8, 2018

TomAugspurger commented Jan 8, 2018

TomAugspurger commented Jan 8, 2018

earthgecko commented Jan 8, 2018 • edited Loading

TomAugspurger commented Jan 8, 2018 •

edited

Loading

earthgecko commented Jan 8, 2018 •

edited

Loading