Bug: markdownlint not supported on ARM

[ran-isenberg]

Expected Behaviour

On a clean fetch from develop branch, make pr should work.

both mypy and markdown-cli linter either fail or raise warning.

Current Behaviour

First issue:
docker run -v /Users/xxxx/aws-lambda-powertools-python:/markdown 06kellyjac/markdownlint-cli "docs"
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

I'm running Mac studio M1 (hence the arm64).

Second issue:
poetry run mypy --pretty aws_lambda_powertools examples
examples/jmespath_functions/src/powertools_custom_jmespath_function.py:4:1: error: Cannot find implementation or library stub for module named "snappy" [import]
import snappy
^
examples/jmespath_functions/src/powertools_custom_jmespath_function.py:4:1: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
examples/jmespath_functions/src/powertools_json_idempotency_jmespath.py: note: In function "handler":
examples/jmespath_functions/src/powertools_json_idempotency_jmespath.py:24:31: error: "Dict[Any, Any]" has no attribute "id" [attr-defined]
return {"payment_id": payment.id, "message": "success", "statusCode": 200}
^
Found 2 errors in 2 files (checked 240 source files)
make: *** [mypy] Error 1

Code snippet

make pr

Possible Solution

No response

Steps to Reproduce

make pr

AWS Lambda Powertools for Python version

latest

AWS Lambda function runtime

3.8

Packaging format used

PyPi

Debugging logs

docker run -v /Users/xxxx/aws-lambda-powertools-python:/markdown 06kellyjac/markdownlint-cli "docs"
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
poetry run mypy --pretty aws_lambda_powertools examples
examples/jmespath_functions/src/powertools_custom_jmespath_function.py:4:1: error: Cannot find implementation or library stub for module named "snappy"  [import]
    import snappy
    ^
examples/jmespath_functions/src/powertools_custom_jmespath_function.py:4:1: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
examples/jmespath_functions/src/powertools_json_idempotency_jmespath.py: note: In function "handler":
examples/jmespath_functions/src/powertools_json_idempotency_jmespath.py:24:31: error: "Dict[Any, Any]" has no attribute "id"  [attr-defined]
            return {"payment_id": payment.id, "message": "success", "statusCode": 200}
                                  ^
Found 2 errors in 2 files (checked 240 source files)
make: *** [mypy] Error 1

[leandrodamascena]

Hi @ran-isenberg.. I'm sending a PR in a few minutes to fix the error in the jmespath examples.
Thanks for sharing this.

[heitorlessa]

Thanks Ran! None of the maintainers have a M1 Mac so we will delay looking into the first issue - the second is now fixed.

From the logs, it appears that markdownlint doesn't support ARM.

[eldritchideen]

Just testing this out on my M1 Mac. I'm not sure the issue with markdownlint is what is initially seems.

 % docker run -v /Users/awscook/projects/py/aws-lambda-powertools-python:/markdown 06kellyjac/markdownlint-cli "docs"
WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

The container runs okay, Docker on M1 will emulate linux/amd64, it just warns that it is not running an Arm container. The warning can be suppressed by adding --platform linux/amd64 to the docker run command. Eg:

docker run --platform linux/amd64 -v /Users/awscook/projects/py/aws-lambda-powertools-python:/markdown 06kellyjac/markdownlint-cli "docs"

I don't believe there is an issue here with docker on M1 Macs running the make pr.

However I have discovered, that if I use the container image from https://github.com/igorshubovych/markdownlint-cli
I get a different result on both M1 and Linux x86 builds. Replacing the markdownlint-cli in the lint targets of the Makefile with

docker run --platform linux/amd64 -v $PWD:/workdir ghcr.io/igorshubovych/markdownlint-cli:latest "docs"

Finds the following issues with docs.

docs/changelog.md:1:1 MD053/link-image-reference-definitions Link and image reference definitions should be needed [Unused link or image reference definition: "comment"] [Context: "[comment]: <> (Includes Change..."]
docs/tutorial/index.md:1044 MD043/required-headings/required-headers Required heading structure [Context: "## Key features"]
docs/utilities/idempotency.md:405:31 MD051/link-fragments Link fragments should be valid [Context: "[@idempotent_function decorator](#idempotentfunction-decorator)"]

This is the same whether running this on my M1 Mac or Linux EC2 instance.

Should we consider changing the markdown-cli container?

[heitorlessa]

Thank you so much @eldritchideen for digging into this. Yes, we should consider changing it.

I vaguely remember using that as a first option and having issues with GH Container Registry image pull due to token or being flaky - I'd appreciate the help if you have the bandwidth.

I'm rewriting the Integration Test RFC and some admin work this week.

[heitorlessa]

following up @eldritchideen if you still have some bandwidth to make this contribution :-).

[eldritchideen]

@heitorlessa Yes, I have some time in the coming week to take a look at this further.

[eldritchideen]

Looking at the alternate markdown-lint it reports the following issues:

docs/changelog.md:1:1 MD053/link-image-reference-definitions Link and image reference definitions should be needed [Unused link or image reference definition: "comment"] [Context: "[comment]: <> (Includes Change..."]
docs/tutorial/index.md:1044 MD043/required-headings/required-headers Required heading structure [Context: "## Key features"]
docs/utilities/idempotency.md:405:31 MD051/link-fragments Link fragments should be valid [Context: "[@idempotent_function decorator](#idempotentfunction-decorator)"]
docs/utilities/middleware_factory.md:96 MD043/required-headings/required-headers Required heading structure [Context: "## Getting started"]

Of these, only the issue reported in docs/utilities/idempotency.md is a real problem.

Happy to submit PR to fix this minor issue. I'm not sure it's worth changing the container image used, they both seem to work. @heitorlessa At the moment I'm leaning towards just fixing the one real MD issue and closing this out. Probably no real need to change from using the 06kellyjac/markdownlint-cli image.

[heitorlessa]

ah great! I forgot to share that I've just recently got a M1 myself (last week) and it's been smooth sailing so far as regards to this issue.

Please do send that small fix and we can keep the container image as-is.

Thank you @eldritchideen !!!

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

This issue is now closed. Please be mindful that future comments are hard for our team to see.

If you need more assistance, please either tag a team member or open a new issue that references this one.

If you wish to keep having a conversation with other community members under this issue feel free to do so.