Skip to content

Commit 8625ccb

Browse files
heitorlessaheitorlessa
authored
docs(process): explain our integration automated checks; revamp navigation (#2764)
1 parent 0523ff6 commit 8625ccb

File tree

8 files changed

+908
-859
lines changed

8 files changed

+908
-859
lines changed

‎MAINTAINERS.md

+2-803
Large diffs are not rendered by default.

‎docs/automation.md

+90
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
---
2+
title: Automation
3+
description: Automation practices and processes for Powertools for AWS Lambda (Python)
4+
---
5+
6+
<!-- markdownlint-disable MD043 MD013 -->
7+
8+
## Continuous integration practices
9+
10+
!!! note "We adhere to industry recommendations from the [OSSF Scorecard project](https://bestpractices.coreinfrastructure.org/en/criteria){target="_blank"}, among [others](https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions){target="_blank"}."
11+
12+
Since all code changes require a pull request (PR) along with one or more reviewers, we automate quality and security checks **before**, **during**, and **after** a PR is merged to trunk (`develop`).
13+
14+
We use a combination of tools coupled with peer review to increase its compound effect in detecting issues early.
15+
16+
This is a snapshot of our automated checks at a glance.
17+
18+
<!-- NOTE: mkdocs material doesn't render timeline mermaid diagrams hence png for now -->
19+
20+
![Continuous Integration practices](./media/continuous_integration_practices.png)
21+
22+
### Pre-commit checks
23+
24+
> [**Pre-commit configuration**](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/.pre-commit-config.yaml){target="_blank"}.
25+
26+
Pre-commit checks are crucial for a fast feedback loop while ensuring security practices at the individual change level.
27+
28+
To prevent scenarios where these checks are intentionally omitted at the client side, we run at [CI level too](#pull-request-checks).
29+
30+
!!! note "These run locally only for changed files"
31+
32+
* [**Merge conflict check**](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/.pre-commit-config.yaml#L10){target="_blank"}. Checks for merge strings in each individual change accidentally left unresolved to prevent breakage.
33+
* [**Code linting**](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/Makefile#L22){target="_blank"}. Linter checks for industry quality standards and known bad practices that could lead to abuse.
34+
* [**CloudFormation linting**](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/.pre-commit-config.yaml#L34){target="_blank"}. `cfn-lint` ensures [best practices](https://github.com/aws-cloudformation/cfn-lint/blob/86f0370bd43b400ed4c485180dbc2697f73367b2/docs/rules.md){target=""_blank"} at our documentation examples.
35+
* [**Markdown linting**](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/.pre-commit-config.yaml#L25){target="_blank}. Primarily [industry markdown practices](https://github.com/DavidAnson/markdownlint/blob/d01180ec5a014083ee9d574b693a8d7fbc1e566d/README.md#rules--aliases){target="_blank"} at this stage.
36+
* [**GitHub Actions linting**](https://github.com/rhysd/actionlint/blob/main/docs/checks.md){target="_blank"}. `actionlint` ensures workflows follow [GitHub Actions security practices](https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions){target="_blank"}. It guards against numerous [leading practices](https://github.com/rhysd/actionlint/blob/main/docs/checks.md){target="_blank"} to prevent common configuration mistakes, insecure inline scripts, among many others.
37+
* [**Terraform linting**](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/.pre-commit-config.yaml#L43){target="_blank"}. As of now, largely formatting until we increase our Terraform coverage in documentation examples.
38+
* [**Secrets linting**](https://github.com/aws-powertools/powertools-lambda-python/blob/develop/.pre-commit-config.yaml#L49){target="_blank"}. Detects industry credentials that might be accidentally leaked in source code.
39+
40+
### Pre-Pull Request checks
41+
42+
For an improved contributing experience, most of our checks can run locally. For maintainers, this also means increased focus on reviewing actual value instead of standards and security malpractices that can be caught earlier.
43+
44+
!!! note "These are in addition to [pre-commit checks](#pre-commit-checks)."
45+
46+
* [**Static typing analysis**](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/Makefile#L108). `mypy` checks for static typing annotations to prevent common bugs in Python that may or may not lead to abuse.
47+
* [**Tests**](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/Makefile#L31){target="_blank"}. We run `unit`, `functional`, and `performance` tests ([_see our definition_](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/CONTRIBUTING.md#testing-definition){target="_blank"}). Besides breaking changes, we are investing in mutation testing to find additional sources of bugs and potential abuse.
48+
* [**Security baseline**](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/CONTRIBUTING.md#testing-definition){target="_blank"}. `bandit` detects common security issues defined by Python Code Quality Authority (PyCQA).
49+
* [**Complexity baseline**](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/Makefile#L79){target="_blank"}. We run a series of maintenability and cyclomatic checks to reduce code and logic complexity. This aids reviewers' cognitive overhead and long-term maintainers revisiting legacy code at a later date.
50+
51+
### Pull Request checks
52+
53+
While we trust contributors and maintainers do go through pre-commit and pre-pull request due diligence, we verify them at CI level.
54+
55+
!!! note "Checks described earlier are omitted to improve reading experience."
56+
57+
* [**Semantic PR title**](https://github.com/Ezard/semantic-prs){target="_blank"}. We enforce PR titles follow semantic naming, for example `chore(category): change`. This benefits contributors with a lower entry bar, no need for semantic commits. It also benefits everyone looking for an [useful changelog message](https://docs.powertools.aws.dev/lambda/python/latest/changelog/){target="_blank"} on **what** changed and **where**.
58+
* [**Related issue check**](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/.github/workflows/on_opened_pr.yml#L44){target="_blank"}. [Every change require an issue](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/CONTRIBUTING.md#contributing-via-pull-requests){target="_blank"} describing its needs. This enforces a PR has a related issue by blocking merge operations if missing.
59+
* [**Acknowledgment check**](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/.github/workflows/on_opened_pr.yml#L63){target="_blank"}. [Ensures PR template](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/.github/PULL_REQUEST_TEMPLATE.md#L36){target="_blank"} is used and every contributor is aware of code redistribution.
60+
* [**Code coverage diff**](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/.github/workflows/quality_check.yml#L73){target="_blank"}. Educates contributors and maintainers about code coverage differences for a given change.
61+
* [**Contribution size check**](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/.github/workflows/on_label_added.yml#L44){target="_blank"}. Suggests contributors and maintainers to break up large changes (100-499 LOC) in smaller PRs. It helps reduce overlooking security and other practices due to increased cognitive overhead.
62+
* [**Dependency vulnerability check**](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/.github/workflows/dependency-review.yml#L22){target="_blank"}. Verifies any dependency changes for common vulnerability exposures (CVEs), in addition to our daily check on any dependencies used (e.g., Python, Docker, Go, etc.)
63+
* [**GitHub Actions security check**](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/.github/workflows/secure_workflows.yml#L35){target="_blank"}. Enforces use of immutable 3rd-party GitHub Actions (_e.g., `actions/checkout@<git-SHA>_`) to prevent abuse. Upgrades are handled by a [separate automated process](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/.github/dependabot.yml#L4){target="_blank"} that includes a maintainer review to also prevent unexpected behavior changes.
64+
65+
### After merge checks
66+
67+
!!! note "Checks described earlier are omitted to improve reading experience."
68+
69+
We strike a balance in security and contribution experience. These automated checks take several minutes to complete. Failures are reviewed by a maintainer on-call and before a release.
70+
71+
* [**End-to-end tests**](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/.github/workflows/run-e2e-tests.yml#L41){target="_blank"}. We run E2E with a [high degree of parallelization](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/MAINTAINERS.md#test-runner-parallelization). While it is designed to also run locally, it may incur AWS charges to contributors. For additional security, all infrastructure is ephemeral per change and per Python version.
72+
* [**SAST check**](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/.github/workflows/codeql-analysis.yml#L20){target="_blank"}. GitHub CodeQL runs ~30m static analysis in the entire codebase.
73+
* [**Security posture check**](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/.github/workflows/ossf_scorecard.yml#L14){target="_blank"}. OSSF Scorecard runs numerous automated checks upon changes, and raises security alerts if [OSSF security practices](https://bestpractices.coreinfrastructure.org/en/criteria){target="_blank"} are no longer followed.
74+
* [**Rebuild Changelog**](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/.github/workflows/build_changelog.yml#L23){target="_blank"}. We rebuild our entire changelog upon changes and create a PR for maintainers. This has the added benefit in keeping a [protected branch](https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches){target="_blank"} while keeping removing error-prone tasks from maintainers.
75+
* [**Stage documentation**](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/.github/workflows/on_push_docs.yml#L27C16-L27C16){target="_blank"}. We rebuild and deploy changes to the documentation to a [staged version](https://docs.powertools.aws.dev/lambda/python/stage/){target="_blank"}. This gives us safety that our docs can always be rebuilt, and ready to release to production when needed.
76+
* [**Update draft release**](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/.github/workflows/release-drafter.yml#L25){target="_blank"}. We use [Release Drafter](https://github.com/release-drafter/release-drafter){target="_blank"} to generate a portion of our release notes and to always keep a fresh draft upon changes. You can read our [thoughts on a good quality release notes here](https://github.com/aws-powertools/powertools-lambda-python/blob/0523ff64606514ea3e59c07c8c69c83d751f61fa/MAINTAINERS.md#drafting-release-notes){target="_blank"} (human readable changes + automation).
77+
78+
## Continuous deployment practices
79+
80+
!!! note "We adhere to industry recommendations from the [OSSF Scorecard project](https://bestpractices.coreinfrastructure.org/en/criteria){target="_blank"}, among [others](https://docs.github.com/en/actions/security-guides/security-hardening-for-github-actions){target="_blank"}."
81+
82+
Releases are triggered by maintainers along with a reviewer - [detailed info here](https://github.com/aws-powertools/powertools-lambda-python/blob/develop/MAINTAINERS.md#releasing-a-new-version){target="_blank"}. In addition to [checks that run for every code change](#continuous-integration-practices), our pipeline requires a manual approval before releasing.
83+
84+
We use a combination of provenance and signed attestation for our builds, source code sealing, SAST scanners, Python specific static code analysis, ephemeral credentials that last a given job step, and more.
85+
86+
This is a snapshot of our automated checks at a glance.
87+
88+
![Continuous Deployment practices](./media/continuous_deployment_practices.png)
89+
90+
!!! info "More details to come"

‎docs/diagram_src/cicd_steps.md

+106
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
<!-- markdownlint-disable MD041 MD043 -->
2+
<!-- separate file until mkdocs-material supports it: https://github.com/squidfunk/mkdocs-material/discussions/5708 -->
3+
```mermaid
4+
timeline
5+
title Powertools for AWS Lambda (Python) CI/CD pipeline
6+
7+
section Continuous Integration
8+
Project setup <br> (make dev) : Code checkout
9+
: Virtual environment
10+
: Dependencies
11+
: Git pre-commit hooks
12+
: Local branch
13+
: Local changes
14+
: Local tests
15+
16+
Pre-commit checks <br> (git commit) : Merge conflict check
17+
: Trailing whitespaces
18+
: TOML checks
19+
: Code linting (standards)
20+
: Markdown linting
21+
: CloudFormation linting
22+
: GitHub Actions linting
23+
: Terraform linting
24+
: Secrets linting
25+
26+
Pre-Pull Request <br> (make pr) : Code linting
27+
: Docs linting
28+
: Static typing analysis
29+
: Tests (unit|functional|perf)
30+
: Security baseline
31+
: Complexity baseline
32+
: +pre-commit checks
33+
34+
Pull Request <br> (CI checks) : Semantic PR title check
35+
: Related issue check
36+
: Acknowledgment check
37+
: Code coverage diff
38+
: Contribution size check
39+
: Contribution category check
40+
: Dependency vulnerability check
41+
: GitHub Actions security check
42+
: +pre-pull request checks
43+
44+
After merge <br> (CI checks) : End-to-end tests
45+
: Longer SAST check
46+
: Security posture check (scorecard)
47+
: GitHub Actions security check
48+
: Rebuild Changelog
49+
: Deploy staging docs
50+
: Update draft release
51+
52+
section Continuous Delivery
53+
54+
Source code anti-tampering : Checkout release commit code
55+
: Bump release version
56+
: Seal and upload artifact
57+
58+
Quality Assurance : Restore sealed code
59+
: +Continuous Integration checks
60+
61+
Build : Restore sealed code
62+
: Integrity check
63+
: Build release artifact
64+
: Seal and upload artifact
65+
66+
Provenance : Detect build environment
67+
: Generate SLSA Builder
68+
: Verify SLSA Builder provenance
69+
: Create and sign provenance
70+
: Seal and upload artifact
71+
: Write to public ledger
72+
73+
Release : Restore sealed build
74+
: Integrity check
75+
: PyPi ephemeral credentials
76+
: Publish PyPi
77+
: Baking time
78+
79+
Git tagging : Restore sealed code
80+
: Integrity check
81+
: Bump git tag
82+
: Create temporary branch
83+
: Create PR
84+
85+
Lambda Layers : Fetch PyPi release
86+
: Build x86 architecture
87+
: Build ARM architecture
88+
: Deploy Beta
89+
: Canary testing
90+
: Deploy Prod
91+
92+
Lambda Layers SAR : Deploy Beta
93+
: Deploy Prod
94+
95+
Documentation : Update Lambda Layer ARNs
96+
: Build User Guide
97+
: Build API Guide
98+
: Rebuild Changelog
99+
: Release new version
100+
: Update latest alias
101+
: Create temporary branch
102+
: Create PR
103+
104+
Post-release : Close pending-release issues
105+
: Notify customers
106+
```

0 commit comments

Comments
 (0)