CI: PyArrow import failing on windows with DLL error

[TomAugspurger]

From https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=1251&view=logs

2018-10-16T11:26:55.0746867Z ________________________________ test_pyarrow _________________________________
2018-10-16T11:26:55.0747259Z [gw0] win32 -- Python 3.6.6 C:\Miniconda\envs\pandas\python.exe
2018-10-16T11:26:55.0747479Z 
2018-10-16T11:26:55.0748447Z df =    A
2018-10-16T11:26:55.0748661Z 0  1
2018-10-16T11:26:55.0748840Z 1  2
2018-10-16T11:26:55.0748994Z 2  3
2018-10-16T11:26:55.0749185Z 
2018-10-16T11:26:55.0749378Z     @pytest.mark.filterwarnings("ignore:can't resolve:ImportWarning")
2018-10-16T11:26:55.0749589Z     def test_pyarrow(df):
2018-10-16T11:26:55.0749764Z     
2018-10-16T11:26:55.0749982Z >       pyarrow = import_module('pyarrow')  # noqa
2018-10-16T11:26:55.0750118Z 
2018-10-16T11:26:55.0750317Z pandas\tests\test_downstream.py:128: 
2018-10-16T11:26:55.0751826Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
2018-10-16T11:26:55.0752083Z pandas\tests\test_downstream.py:22: in import_module
2018-10-16T11:26:55.0752323Z     return importlib.import_module(name)
2018-10-16T11:26:55.0752816Z C:\Miniconda\envs\pandas\lib\importlib\__init__.py:126: in import_module
2018-10-16T11:26:55.0753309Z     return _bootstrap._gcd_import(name[level:], package, level)
2018-10-16T11:26:55.0753594Z <frozen importlib._bootstrap>:994: in _gcd_import
2018-10-16T11:26:55.0753802Z     ???
2018-10-16T11:26:55.0754357Z <frozen importlib._bootstrap>:971: in _find_and_load
2018-10-16T11:26:55.0754577Z     ???
2018-10-16T11:26:55.0755050Z <frozen importlib._bootstrap>:955: in _find_and_load_unlocked
2018-10-16T11:26:55.0755265Z     ???
2018-10-16T11:26:55.0755655Z <frozen importlib._bootstrap>:665: in _load_unlocked
2018-10-16T11:26:55.0755864Z     ???
2018-10-16T11:26:55.0756317Z <frozen importlib._bootstrap_external>:678: in exec_module
2018-10-16T11:26:55.0756524Z     ???
2018-10-16T11:26:55.0756978Z <frozen importlib._bootstrap>:219: in _call_with_frames_removed
2018-10-16T11:26:55.0757208Z     ???
2018-10-16T11:26:55.0758166Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
2018-10-16T11:26:55.0758470Z 
2018-10-16T11:26:55.0758928Z     from pkg_resources import get_distribution, DistributionNotFound
2018-10-16T11:26:55.0759216Z     try:
2018-10-16T11:26:55.0759463Z         __version__ = get_distribution(__name__).version
2018-10-16T11:26:55.0759792Z     except DistributionNotFound:
2018-10-16T11:26:55.0760036Z        # package is not installed
2018-10-16T11:26:55.0760243Z         try:
2018-10-16T11:26:55.0760749Z             # This code is duplicated from setup.py to avoid a dependency on each
2018-10-16T11:26:55.0760980Z             # other.
2018-10-16T11:26:55.0761371Z             def parse_version(root):
2018-10-16T11:26:55.0761897Z                 from setuptools_scm import version_from_scm
2018-10-16T11:26:55.0762378Z                 import setuptools_scm.git
2018-10-16T11:26:55.0762833Z                 describe = (setuptools_scm.git.DEFAULT_DESCRIBE +
2018-10-16T11:26:55.0763396Z                             " --match 'apache-arrow-[0-9]*'")
2018-10-16T11:26:55.0763802Z                 # Strip catchall from the commandline
2018-10-16T11:26:55.0764654Z                 describe = describe.replace("--match *.*", "")
2018-10-16T11:26:55.0765811Z                 version = setuptools_scm.git.parse(root, describe)
2018-10-16T11:26:55.0765988Z                 if not version:
2018-10-16T11:26:55.0766380Z                     return version_from_scm(root)
2018-10-16T11:26:55.0766570Z                 else:
2018-10-16T11:26:55.0766734Z                     return version
2018-10-16T11:26:55.0766914Z     
2018-10-16T11:26:55.0767073Z             import setuptools_scm
2018-10-16T11:26:55.0767318Z             __version__ = setuptools_scm.get_version('../', parse=parse_version)
2018-10-16T11:26:55.0767507Z         except (ImportError, LookupError):
2018-10-16T11:26:55.0767667Z             __version__ = None
2018-10-16T11:26:55.0767837Z     
2018-10-16T11:26:55.0768469Z     
2018-10-16T11:26:55.0768685Z >   from pyarrow.lib import cpu_count, set_cpu_count
2018-10-16T11:26:55.0768926Z E   ImportError: DLL load failed: The specified module could not be found.

[TomAugspurger]

I can't reproduce locally on a Mac.

Looking at the diff in environments, there's a few suspects

• arrow is 0.9 on both, but the build changed from py36_vc14_7 [vc14] conda-forge to py36h12fa3ca_7 conda-forge
• boost-cpp from 1.66.0 -> 1.67.0
• New libboost 1.67.0 , which (maybe troublingly) came from main, not conda-forge.

[TomAugspurger]

I'm not really sure what's going on here. Anyone able to reproduce locally (cc @chris-b1?)

[paddyhoran]

@TomAugspurger I was getting this issue with python 3.7 but once I upgraded to numpy 0.16.0 it resolved itself. Conda was installing numpy 0.15.x by default.

Might be a different issue but worth checking...

[xhochy]

• New libboost 1.67.0 , which (maybe troublingly) came from main, not conda-forge.

This is the issue. boost-cpp and libboost provide the same libs in a different package in different versions.

[jbrockmendel]

@xhochy any idea if this is closeable?

[xhochy]

@jbrockmendel Is this still an issue? How could I reproduce this?

[jbrockmendel]

No idea, I was hoping you'd know.

[jorisvandenbossche]

@xhochy I think this is fixed with your PR?

[xhochy]

@jorisvandenbossche @jbrockmendel Yes, this can be closed now.

[jorisvandenbossche]

OK, thanks!