ImportError: dlopen on MacOs chip M1

[tranchinh87]

Hi everyone !
I just migrated to a new M1 MacBook Pro, and i get an error while running a sample on macbook pro with M1 chip.

cd ~/aws-iot-device-sdk-python-v2/samples
# And run the Sample Code
python pubsub.py --endpoint a1nd5owl3rxxxxx-ats.iot.ap-southeast-1.amazonaws.com --root-ca ~/certs/Amazon-root-CA-1.pem --cert ~/certs/device.pem.crt --key ~/certs/private.pem.key

An error has occurred

Traceback (most recent call last):
  File "/Users/xxx/aws-iot-device-sdk-python-v2/samples/pubsub.py", line 5, in <module>
    from awscrt import io, mqtt, auth, http
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/awscrt/io.py", line 11, in <module>
    import _awscrt
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/_awscrt.cpython-310-darwin.so, 0x0002): symbol not found in flat namespace '_aws_hash_ptr'

My packages installed
Python version: 3.10.2
pip list
Package Version

---

awscrt 0.13.3
awsiotsdk 1.9.2
boto3 1.21.15
botocore 1.24.15
jmespath 0.10.0
pip 22.0.4
python-dateutil 2.8.2
s3transfer 0.5.2
setuptools 58.1.0
six 1.16.0
urllib3 1.26.8

[jmklix]

We have yet to test this sdk on M1 Macs. Thanks for letting us know about this ImportError and make sure to add any other errors that you run into below.

[tgamushet]

I have the same issue with awscrt.
And awsiot has missing import error but it's installed.

I am trying to run pubsub example too.

M1 MacBook Pro
Python 3.9.10 (installed with brew)

pip list

Package         Version
--------------- -------
awscrt          0.13.6
awsiot          0.1.3
boto3           1.21.25
botocore        1.24.26
click           8.0.4
jmespath        1.0.0
pip             22.0.4
python-dateutil 2.8.2
s3transfer      0.5.2
setuptools      60.5.0
six             1.16.0
urllib3         1.26.9

(venv) ➜  aws-iot-client python
Python 3.9.10 (main, Jan 15 2022, 11:40:36) 
[Clang 13.0.0 (clang-1300.0.29.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from awscrt import auth, http, io, mqtt
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/ym/tga/aws-iot-client/venv/lib/python3.9/site-packages/awscrt/auth.py", line 8, in <module>
    import _awscrt
ImportError: dlopen(/Users/ym/tga/aws-iot-client/venv/lib/python3.9/site-packages/_awscrt.cpython-39-darwin.so, 2): Symbol not found: _aws_hash_ptr
  Referenced from: /Users/ym/tga/aws-iot-client/venv/lib/python3.9/site-packages/_awscrt.cpython-39-darwin.so
  Expected in: flat namespace
 in /Users/ym/tga/aws-iot-client/venv/lib/python3.9/site-packages/_awscrt.cpython-39-darwin.so
>>> from awsiot import mqtt_connection_builder
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'awsiot'

When I install awsiotsdk missing import is fixed but _awscrt comes again with awsiot too.

(venv) ➜  aws-iot-client pip install awsiotsdk 
Collecting awsiotsdk
  Using cached awsiotsdk-1.10.0-py3-none-any.whl (59 kB)
Requirement already satisfied: awscrt==0.13.6 in ./venv/lib/python3.9/site-packages (from awsiotsdk) (0.13.6)
Installing collected packages: awsiotsdk
Successfully installed awsiotsdk-1.10.0
(venv) ➜  aws-iot-client python
Python 3.9.10 (main, Jan 15 2022, 11:40:36) 
[Clang 13.0.0 (clang-1300.0.29.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from awsiot import mqtt_connection_builder
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/ym/tga/aws-iot-client/venv/lib/python3.9/site-packages/awsiot/__init__.py", line 13, in <module>
    from awscrt import mqtt
  File "/Users/ym/tga/aws-iot-client/venv/lib/python3.9/site-packages/awscrt/mqtt.py", line 10, in <module>
    import _awscrt
ImportError: dlopen(/Users/ym/tga/aws-iot-client/venv/lib/python3.9/site-packages/_awscrt.cpython-39-darwin.so, 2): Symbol not found: _aws_hash_ptr
  Referenced from: /Users/ym/tga/aws-iot-client/venv/lib/python3.9/site-packages/_awscrt.cpython-39-darwin.so
  Expected in: flat namespace
 in /Users/ym/tga/aws-iot-client/venv/lib/python3.9/site-packages/_awscrt.cpython-39-darwin.so

[agodoyAI]

Following.

[tgamushet]

Manage to get it to work with python-alpine docker image and
awsiotsdk==1.10.0
awscrt==0.13.6

awsiotsdk works fine instead of awsiot

[graebm]

We are currently working on fixing this for Apple M1. Skip to the end for a short term workaround.

Background: awsiotsdk depends on awscrt, which contains some C code that needs to be compiled per-platform. We haven't done the work yet to provide precompiled wheels for Apple M1. It also appears that our precompiled wheels for Python 3.9 and 3.10 on macOS are mis-labeled as "universal2" wheels (meaning they should work on either M1 or Intel Macs), but in reality these files only work on Intel right now. The real fix is to provide wheels that work on M1 or Intel. We're currently working on this.

The short term workaround is to compile the code on your own Mac. You can do this by following these steps:

1. Ensure a C compiler is installed on your machine.
   • In the terminal run: xcode-select --install
2. Ensure CMake is installed on your machine
   • Either install from a download: https://cmake.org/download/
   • Or install via homebrew: brew install cmake
3. Re-install awscrt
   • Uninstall: python3 -m pip uninstall awscrt awsiotsdk
   • Re-install, but force it to compile locally: python3 -m pip install awsiotsdk --no-binary :all:

[tranchinh87]

Thank you for your support!

[graebm]

Ok, fixed!
Apple Silicon is now an official supported platform for us as of awsiotsdk v1.11.1
Best of luck