ImportError: DLL load failed with Windows wheel for 1.0.2 and 1.0.3 #32857

J-Postma · 2020-03-20T09:39:09Z

When trying to install pandas 1.0.2 or newer versions on a docker, I run into the following import error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python\lib\site-packages\pandas\__init__.py", line 55, in <module>
    from pandas.core.api import (
  File "C:\Python\lib\site-packages\pandas\core\api.py", line 29, in <module>
    from pandas.core.groupby import Grouper, NamedAgg
  File "C:\Python\lib\site-packages\pandas\core\groupby\__init__.py", line 1, in <module>
    from pandas.core.groupby.generic import DataFrameGroupBy, NamedAgg, SeriesGroupBy
  File "C:\Python\lib\site-packages\pandas\core\groupby\generic.py", line 60, in <module>
    from pandas.core.frame import DataFrame
  File "C:\Python\lib\site-packages\pandas\core\frame.py", line 124, in <module>
    from pandas.core.series import Series
  File "C:\Python\lib\site-packages\pandas\core\series.py", line 4572, in <module>
    Series._add_series_or_dataframe_operations()
  File "C:\Python\lib\site-packages\pandas\core\generic.py", line 10349, in _add_series_or_dataframe_operations
    from pandas.core.window import EWM, Expanding, Rolling, Window
  File "C:\Python\lib\site-packages\pandas\core\window\__init__.py", line 1, in <module>
    from pandas.core.window.ewm import EWM  # noqa:F401
  File "C:\Python\lib\site-packages\pandas\core\window\ewm.py", line 5, in <module>
    import pandas._libs.window.aggregations as window_aggregations
ImportError: DLL load failed while importing aggregations: The specified module could not be found.

I've reproduced this error on a separate pc by running:

docker run -it python cmd.exe
pip install pandas
python
import pandas

This error only occurs when trying to import pandas from the docker. Python and pip versions:

python 3.8.2
pip 20.0.2

The text was updated successfully, but these errors were encountered:

TomAugspurger · 2020-03-20T12:04:02Z

Things work for you with pandas 1.0.1 and earlier?

J-Postma · 2020-03-20T12:10:35Z

Yes, things were fine with pandas 1.0.1 or any of the 0.* versions

TomAugspurger · 2020-03-20T13:22:43Z

Thanks. We switched our wheel building infrastructure for 1.0.3, but if it's happening on 1.0.2 as well that's likely not it.

I'm not aware of anyone else using pandas inside a windows docker container. You might be on your own here, so anything you can do to debug this would be welcome.

TomAugspurger · 2020-03-20T13:38:11Z

Can you create a new environment and post the full output of this?

pip install pandas
python -c "import pandas"

For both pandas 1.0.2 and 1.0.3 potentially.

AaryamanCodes · 2020-03-21T07:32:08Z

So, I too faced the exact error. I tried it on Jupyter notebooks and it worked though I could still not import some other pandas submodules.

TomAugspurger · 2020-03-23T18:46:00Z

From @mborus:

TLDR; 2 DLLs are missing that were included in 1.0.1

Code Sample, a copy-pastable example if possible

import pandas as pd

Problem description

I'm using Windows 7, German 64bit with a fresh installation of Python 3.8.2 64bit.
I updated pip, created a fresh virtual environment with "py -3 -m venv venv" and activated it.
This installation is in a restricted area with no direct, unfiltered internet access.

I installed Pandas 1.0.3 from a via "python -m pip download pandas" download made on an unrestricted similar Windows7 machine. I moved the download folder behind a firewall and installed Pandas like this: "python -m pip install pandas --no-index --find-links /path/to/downloads":

Pandas installs fine. It crashes on import with the error below (The last sentence says in English: The requested Module was not found)

As recommended by this stack overflow thread (https://stackoverflow.com/questions/60763529/unable-to-import-pandas-pandas-libs-window-aggregations) going back to 1.0.1 fixes the problem. (More below, after the error)

Error:

(venv) C:\my_program>python
Python 3.8.2 (tags/v3.8.2:7b3ab59, Feb 25 2020, 23:03:10) [MSC v.1916 64 bit (AM
D64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

import pandas as pd
Traceback (most recent call last):
File "", line 1, in
File "C:\my_program\venv\lib\site-packages\pandas_init_.py", line 5
5, in
from pandas.core.api import (
File "C:\my_program\venv\lib\site-packages\pandas\core\api.py", line 2
9, in
from pandas.core.groupby import Grouper, NamedAgg
File "C:\my_program\venv\lib\site-packages\pandas\core\groupby_init
_.py", line 1, in
from pandas.core.groupby.generic import DataFrameGroupBy, NamedAgg, SeriesGr
oupBy
File "C:\my_program\venv\lib\site-packages\pandas\core\groupby\generic
.py", line 60, in
from pandas.core.frame import DataFrame
File "C:\my_program\venv\lib\site-packages\pandas\core\frame.py", line
124, in
from pandas.core.series import Series
File "C:\my_program\venv\lib\site-packages\pandas\core\series.py", lin
e 4572, in
Series._add_series_or_dataframe_operations()
File "C:\my_program\venv\lib\site-packages\pandas\core\generic.py", li
ne 10349, in add_series_or_dataframe_operations
from pandas.core.window import EWM, Expanding, Rolling, Window
File "C:\my_program\venv\lib\site-packages\pandas\core\window_init
.py", line 1, in
from pandas.core.window.ewm import EWM # noqa:F401
File "C:\my_program\venv\lib\site-packages\pandas\core\window\ewm.py",
line 5, in
import pandas._libs.window.aggregations as window_aggregations
ImportError: DLL load failed while importing aggregations: Das angegebene Modul
wurde nicht gefunden.

I compared the installation of version 1.0.1 to 1.0.3;

The 1.0.3 installs these files in
"\venv\Lib\site-packages\pandas_libs\window"

    __init__.py
    aggregations.cp38-win_amd64.pyd
    indexers.cp38-win_amd64.pyd

The 1.0.1 version also installs these two DLLs in the same folder.

    concrt140.dll
    msvcp140.dll

When I manually place these DLLs into the above folder of the 1.0.3 version,
the import works without an error.

Expected Output

No error message

Output of `pd.show_versions()`

Not available, since pd did not import

TomAugspurger · 2020-03-23T18:47:04Z

To everyone in this issue, does the problem occur using wheels from pandas 1.0.2 and 1.0.3?

mborus · 2020-03-23T20:56:53Z

Can't say for 1.0.2, but I have installed the 1.0.3 wheel "pandas-1.0.3-cp38-cp38-win_amd64.whl" and extracted the DLLs from "pandas-1.0.1-cp38-cp38-win_amd64.whl"

TomAugspurger · 2020-03-23T21:10:06Z

@mborus can you try for 1.0.2? Our wheel building infrastructure changed for 1.0.3, but we've had users report issues with 1.0.2 as well.

mborus · 2020-03-23T21:17:22Z

I'll give it a try. This may take a while because the process of getting the Pandas wheel behind the firewall is a bit complicated. I'll post the result when I know more.

TomAugspurger · 2020-03-23T21:20:07Z

Thanks!

mborus · 2020-03-23T21:48:30Z

I installed the 1.0.2 (pandas-1.0.2-cp38-cp38-win_amd64.whl) briefly and it's the same problem. The two DLLs are not installed, so the import fails the same way.

TomAugspurger · 2020-03-24T01:30:23Z

OK thanks for checking!

@jbrockmendel or @WillAyd right now I'm guessing this is a wheel building issue, but do you know if we've had any changes to our build configuration between 1.0.1 and 1.0.2 that would cause this? #32857 (comment) is the clearest summary.

TomAugspurger · 2020-03-24T01:39:25Z

Hmm https://github.com/pandas-dev/pandas/pull/32386/files touched window/aggregations. Is there anything there that seems off? It seems innocuous.

AaryamanCodes · 2020-03-24T04:06:21Z

I installed it using pip. I am using windows 10. Should I go to Panda's official website and download it?

AaryamanCodes · 2020-03-24T04:10:18Z

@TomAugspurger I am unable to use the pandas module though I can use it on Jupyter Notebooks without an error. How do I need to download any other version because I have used pip install.

TomAugspurger · 2020-03-24T12:26:50Z

@AaryamanCodes this isn't something that would be affected by using it through the notebook or through the terminal. Most likely your notebook is using a different environment.

It sounds like pandas 1.0.1 is not affected, so you can pin to that version for now.

TomAugspurger · 2020-03-24T12:35:41Z

cc @cgohlke if you have any guesses. (see #32857 (comment)). Though the fact that it's failing with your wheels and the one we built on azure pipelines indicates that it's a problem with our configuration.

cgohlke · 2020-03-24T14:41:49Z

I think the C++ runtime DLLs need to be packaged in the _libs/window directory :

pandas/setup.py

Line 760 in f20331d

package_data={"": ["templates/*", "_libs/*.dll"]},

TomAugspurger · 2020-03-24T16:01:10Z

Thanks. Is it strange that this is only coming up for 1.0.2 and newer, since there weren't any changes to the setup.py on the 1.0.x branch after 1.0: https://github.com/pandas-dev/pandas/commits/1.0.x/setup.py

Can people running into this confirm that it only occurs with Python 3.8 (not Python 3.7)? 3.8 changed how DLL loading works IIRC. I've seen two people report using 3.8.2. Does anyone see it with 3.8.1?

jaydave1988 · 2020-03-24T17:56:03Z

I am facing the same error for 1.0.3 and 1.0.2 using python 3.6.2.
C:\Users\Administrator>python --version
Python 3.6.2

TomAugspurger · 2020-03-24T18:28:37Z

OK thanks. So it probably isn't related to Python 3.8's changes to DLL loading then.

Is there a windows equivalent of ldd or otool? If I run

$ otool -L pandas/_libs/window/aggregations.cpython-38-darwin.so
pandas/_libs/window/aggregations.cpython-38-darwin.so:
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 400.9.4)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)

Can people check the windows equivalent for aggregations.pyd for 1.0.1 and 1.0.2 (or 1.0.3)?

Any if anyone is able to compile pandas from source, @cgohlke's suggestion #32857 (comment) might provide a fix, even if we don't understand the cause.

mborus · 2020-03-24T18:33:50Z

One more thing about this: I looked close at the other Windows7 machine, where version 1.0.3 worked fine. On this machine I found the missing DLLs msvcp140.dll and concrt140.dll on the system path in c:\windows\system32 and dozens of other places. (like inside matplotlib, PySide2, numpy, dropbox and such).

My guess is that it's working for most Windows users if Pandas is missing the DLLs because they are found somewhere else. Only some pretty empty installations (like a Docker container or the firewall protected machine I had problems with) will not have those DLLs on the path.

I tried my Python 3.7.4 64bit installation, updated Pandas to 1.0.3 and the "c:\Python37\Lib\site-packages\pandas_libs\window" directory is also missing the DLLs. But import pandas as pd still works from Python 3.7.4.

WillAyd · 2020-03-25T15:10:15Z

Is this reproducible outside of Docker? AFAICT msvcp140.dll isn't necessarily a system file but at the same time is required to run C++ applications. There's a similar thread on MSFT forums about it missing:

https://answers.microsoft.com/en-us/windows/forum/windows_10-performance/msvcp140dll-is-missing-in-my-win-10/1c65d6b0-68b8-4b59-b720-3e6a33038389

So maybe the Docker container should just require that instead of changing anything with pandas?

chris-b1 · 2020-04-02T16:25:19Z

Hmm, kind of stuck here. Built a wheel with the DLLs packed, see below, but still get that import error. But, if I install the 1.0 wheel, with the same DLLs packed (was that built by you @cgohlke ?), works fine.

# inside Windows docker container

C:\>dir Python\Lib\site-packages\pandas\_libs\window\*.dll
 Volume in drive C has no label.
 Volume Serial Number is 4676-6F6D

 Directory of C:\Python\Lib\site-packages\pandas\_libs\window

04/02/2020  10:45 AM           333,632 concrt140.dll
04/02/2020  10:45 AM           633,152 msvcp140.dll
               2 File(s)        966,784 bytes
               0 Dir(s)  21,033,172,992 bytes free

C:\python
>>> import pandas
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python\lib\site-packages\pandas\__init__.py", line 54, in <module>
    from pandas.core.api import (
  File "C:\Python\lib\site-packages\pandas\core\api.py", line 29, in <module>
    from pandas.core.groupby import Grouper, NamedAgg
  File "C:\Python\lib\site-packages\pandas\core\groupby\__init__.py", line 1, in <module>
    from pandas.core.groupby.generic import DataFrameGroupBy, NamedAgg, SeriesGroupBy
  File "C:\Python\lib\site-packages\pandas\core\groupby\generic.py", line 65, in <module>
    from pandas.core.frame import DataFrame
  File "C:\Python\lib\site-packages\pandas\core\frame.py", line 131, in <module>
    from pandas.core.series import Series
  File "C:\Python\lib\site-packages\pandas\core\series.py", line 4598, in <module>
    Series._add_series_or_dataframe_operations()
  File "C:\Python\lib\site-packages\pandas\core\generic.py", line 10212, in _add_series_or_dataframe_operations
    from pandas.core.window import EWM, Expanding, Rolling, Window
  File "C:\Python\lib\site-packages\pandas\core\window\__init__.py", line 1, in <module>
    from pandas.core.window.ewm import EWM  # noqa:F401
  File "C:\Python\lib\site-packages\pandas\core\window\ewm.py", line 5, in <module>
    import pandas._libs.window.aggregations as window_aggregations
ImportError: DLL load failed: The specified module could not be found.

TomAugspurger · 2020-04-02T17:59:09Z

1.0.0 - 1.0.2 are from Christoph. 1.0.3 is the first to be built on azure-pipelines.

cgohlke · 2020-04-02T18:18:12Z

Hmm, kind of stuck here. Built a wheel with the DLLs packed, see below, but still get that import error. But, if I install the 1.0 wheel, with the same DLLs packed (was that built by you @cgohlke ?), works fine.

In case you are using VS 2019, msvcp140.dll now also depends on vcruntime140_1.dll.

chris-b1 · 2020-04-02T18:39:41Z

Wow, that's it - thanks!

ghost · 2020-04-15T13:21:22Z

I ran into this issue today on a Windows 2012 server after updating to Python 3.8 and installing pandas 1.0.3. I found concrt140.dll and msvcp140.dll in C:\Windows\System32 and copied them to the \pandas_libs\window folder as specified here to resolve.

jshridha · 2020-04-17T23:40:19Z

I just installed python3.8 on a completely fresh installation of Windows 10 1909 and ran into this issue. All I had to do was install the lastest x64 version of the visual c redistributable vc_redist from here and my import problem went away.

ali-cetin-4ss · 2020-04-28T07:41:21Z

We ran into the same issue on our production VM (Windows Server 2016 Data Center). Pandas 1.0.1 works fine, but anything later throws a "Could not load DLL" error. We use Python 3.7.6.

TomAugspurger · 2020-04-28T13:32:18Z

This is fixed in master. I think MacPython/pandas-wheels#82 is the only remaining change needed for our build infrastructure.

I'm not sure that anyone has time to make a release in the near-term, but these wheels are uploaded to https://anaconda.org/scipy-wheels-nightly/pandas nightly.

hanzigs · 2020-06-23T07:02:36Z

i'm getting for pandas, python 3.7.4 with pandas 1.0.5

Traceback (most recent call last):
  File "C:\app\FlaskAPI.py", line 14, in <module>
    import pandas as pd
  File "C:\Python\lib\site-packages\pandas\__init__.py", line 55, in <module>
    from pandas.core.api import (
  File "C:\Python\lib\site-packages\pandas\core\api.py", line 29, in <module>
    from pandas.core.groupby import Grouper, NamedAgg
  File "C:\Python\lib\site-packages\pandas\core\groupby\__init__.py", line 1, in <module>
    from pandas.core.groupby.generic import DataFrameGroupBy, NamedAgg, SeriesGroupBy
  File "C:\Python\lib\site-packages\pandas\core\groupby\generic.py", line 60, in <module>
    from pandas.core.frame import DataFrame
  File "C:\Python\lib\site-packages\pandas\core\frame.py", line 124, in <module>
    from pandas.core.series import Series
  File "C:\Python\lib\site-packages\pandas\core\series.py", line 4572, in <module>
    Series._add_series_or_dataframe_operations()
  File "C:\Python\lib\site-packages\pandas\core\generic.py", line 10349, in _add_series_or_dataframe_operations
    from pandas.core.window import EWM, Expanding, Rolling, Window
  File "C:\Python\lib\site-packages\pandas\core\window\__init__.py", line 1, in <module>
    from pandas.core.window.ewm import EWM  # noqa:F401
  File "C:\Python\lib\site-packages\pandas\core\window\ewm.py", line 5, in <module>
    import pandas._libs.window.aggregations as window_aggregations
ImportError: DLL load failed: The specified module could not be found.

may I know the fix?

TomAugspurger · 2020-06-23T14:57:35Z

@apremgeorge as far as I know this was fixed in 1.0.4, and it shouldb e fixed in 1.0.5 as well.

Can you post the outputs of

creating a clean virtualven
pip install pandas==1.0.4
python -c "import pandas"

and repeat that for 1.0.5?

hanzigs · 2020-06-24T01:00:04Z

@TomAugspurger Thanks for the reply
Here is the screenshots, created new env

hanzigs · 2020-06-24T02:21:26Z

pandas 1.0.1 worked for me
Thanks

jshridha · 2020-06-24T02:24:13Z

@apremgeorge Try out my suggestion above. It has worked for me on several fresh windows/python installs lately where I was seeing the same issue. #32857 (comment)

cgohlke · 2020-06-24T02:26:01Z

The VC++ runtime DLLs did not make it into the wheels on PyPI. It looks like the DLLs are copied to the pandas source directory after (not before) building the wheels. https://github.com/MacPython/pandas-wheels/blob/master/azure/windows.yml#L71-L74

TomAugspurger · 2020-06-24T11:34:29Z

Thanks for the pointer. Will see if MacPython/pandas-wheels#90 fixes it.

TomAugspurger · 2020-07-24T16:25:08Z

Could people here try out the 1.1.0rc0 wheel to see if the problem is fixed? pip install --pre pandas==1.1.0rc0.

AaryamanCodes · 2020-07-26T07:48:25Z

Hi, try reinstalling python and do pip install pandas it worked for me! (on windows 10)

TomAugspurger · 2020-07-27T15:43:38Z

Has anyone here been able to try the pre-release wheels for 1.1.0rc0? We're planning to release 1.1.0 final today or tomorrow and it would be nice to verify that this is fixed.

artkha1 · 2020-07-27T20:35:37Z

Has anyone here been able to try the pre-release wheels for 1.1.0rc0? We're planning to release 1.1.0 final today or tomorrow and it would be nice to verify that this is fixed.

Tried pip install --pre pandas==1.1.0rc0, the issue was fixed.

mborus · 2020-07-28T06:34:42Z

Tried this with Python3.8 64bit & Python3.8 32bit (the 32bit never had pandas installed) and it worked for me.

ghost · 2020-07-28T09:14:56Z

On a docker image with python 3.7.8 and Windows-10-10.0.17763-SP0, we can reproduce the DLL error with pandas==1.0.5, and in a fresh instance get no error with pandas==1.1.0.rc0 🥳

TomAugspurger · 2020-07-28T13:21:54Z

Thanks for confirming. 1.1.0 will be out later today.

TomAugspurger mentioned this issue Mar 20, 2020

Unable to import pandas #7517

Closed

TomAugspurger mentioned this issue Mar 23, 2020

Pandas 1.0.3 on Windows 7 64bit throws error "DLL load failed while importing aggregations" on import. Version 1.0.1 works fine. #32936

Closed

TomAugspurger changed the title ~~ImportError when installing pandas from docker~~ DLL load failed with Windows wheel Mar 23, 2020

TomAugspurger changed the title ~~DLL load failed with Windows wheel~~ ImportError: DLL load failed with Windows wheel Mar 23, 2020

TomAugspurger changed the title ~~ImportError: DLL load failed with Windows wheel~~ ImportError: DLL load failed with Windows wheel for 1.0.2 and 1.0.3 Mar 24, 2020

TomAugspurger mentioned this issue Mar 24, 2020

import is failing in windows in 1.0.3 version #32971

Closed

chris-b1 mentioned this issue Apr 2, 2020

BLD: recursive inclusion of DLLs in package data #33246

Merged

1 task

TomAugspurger mentioned this issue Apr 3, 2020

import pandas error. #33258

Closed

mroeschke mentioned this issue Apr 5, 2020

BUG:I am using Latest python 3.8 ,the pip install pandas is working and is successful but showing an when importing it is showing an error, need help #33305

Closed

3 tasks

aivarannamaa mentioned this issue Apr 20, 2020

pandas not working even though installed thonny/thonny#1168

Closed

cgohlke mentioned this issue Jun 15, 2020

Getting "ImportError: DLL load failed: The specified module could not be found." scikit-image/scikit-image#4780

Closed

TomAugspurger mentioned this issue Jun 24, 2020

Copy before building MacPython/pandas-wheels#90

Closed

TomAugspurger closed this as completed Jul 28, 2020

lithomas1 mentioned this issue Jul 27, 2021

BUG: v1.3.1 ImportError: DLL load failed #42733

Closed

3 tasks

ImportError: DLL load failed with Windows wheel for 1.0.2 and 1.0.3 #32857

ImportError: DLL load failed with Windows wheel for 1.0.2 and 1.0.3 #32857

Comments

J-Postma commented Mar 20, 2020

TomAugspurger commented Mar 20, 2020

J-Postma commented Mar 20, 2020

TomAugspurger commented Mar 20, 2020

TomAugspurger commented Mar 20, 2020 • edited Loading

AaryamanCodes commented Mar 21, 2020

TomAugspurger commented Mar 23, 2020

Code Sample, a copy-pastable example if possible

Problem description

Expected Output

Output of pd.show_versions()

TomAugspurger commented Mar 23, 2020

mborus commented Mar 23, 2020

TomAugspurger commented Mar 23, 2020

mborus commented Mar 23, 2020

TomAugspurger commented Mar 23, 2020

mborus commented Mar 23, 2020 • edited Loading

TomAugspurger commented Mar 24, 2020

TomAugspurger commented Mar 24, 2020

AaryamanCodes commented Mar 24, 2020

AaryamanCodes commented Mar 24, 2020

TomAugspurger commented Mar 24, 2020

TomAugspurger commented Mar 24, 2020

cgohlke commented Mar 24, 2020

TomAugspurger commented Mar 24, 2020

jaydave1988 commented Mar 24, 2020

TomAugspurger commented Mar 24, 2020

mborus commented Mar 24, 2020 • edited Loading

WillAyd commented Mar 25, 2020

chris-b1 commented Apr 2, 2020

TomAugspurger commented Apr 2, 2020

cgohlke commented Apr 2, 2020

chris-b1 commented Apr 2, 2020

ghost commented Apr 15, 2020

jshridha commented Apr 17, 2020

ali-cetin-4ss commented Apr 28, 2020

TomAugspurger commented Apr 28, 2020 • edited Loading

hanzigs commented Jun 23, 2020 • edited Loading

TomAugspurger commented Jun 23, 2020

hanzigs commented Jun 24, 2020 • edited Loading

hanzigs commented Jun 24, 2020

jshridha commented Jun 24, 2020

cgohlke commented Jun 24, 2020

TomAugspurger commented Jun 24, 2020

TomAugspurger commented Jul 24, 2020

AaryamanCodes commented Jul 26, 2020 • edited Loading

TomAugspurger commented Jul 27, 2020

artkha1 commented Jul 27, 2020

mborus commented Jul 28, 2020 • edited Loading

ghost commented Jul 28, 2020

TomAugspurger commented Jul 28, 2020

TomAugspurger commented Mar 20, 2020 •

edited

Loading

Output of `pd.show_versions()`

mborus commented Mar 23, 2020 •

edited

Loading

mborus commented Mar 24, 2020 •

edited

Loading

TomAugspurger commented Apr 28, 2020 •

edited

Loading

hanzigs commented Jun 23, 2020 •

edited

Loading

hanzigs commented Jun 24, 2020 •

edited

Loading

AaryamanCodes commented Jul 26, 2020 •

edited

Loading

mborus commented Jul 28, 2020 •

edited

Loading