Skip to content

docs: Add support for docs versioning (#1239) #1370

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 0 additions & 41 deletions .github/workflows/docs.yml

This file was deleted.

39 changes: 39 additions & 0 deletions .github/workflows/rebuild-latest-docs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: Rebuild latest docs

# PROCESS
#
# 1. Build User Guide and API docs
# 2. Publish to GitHub Pages
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it just describes steps of the process which is done here (except "and API docs" which can be removed). I think it might be helpful to understand what we do here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The step "# 2. Publish to GitHub Pages" - we don't need this anymore, do we - maybe it becomes "push versioned docs to gh-pages branch" or something?

# 3. Publish to S3 (new home)

# USAGE
#
# Only used for deploying a documentation hotfix to /latest and its associated version w/o a full release.
#
# Steps:
#
# 1. Trigger "Rebuild latest docs" workflow manually: https://docs.github.com/en/actions/managing-workflow-runs/manually-running-a-workflow
# 2. Use the latest version released under Releases e.g. 2.0.0

on:
workflow_dispatch:
inputs:
latest_published_version:
description: "Latest published version to rebuild latest docs for, e.g. 1.4.2"
default: "1.4.2"
required: true

permissions:
contents: read

jobs:
release-docs:
permissions:
contents: write # push to gh-pages
pages: write # deploy gh-pages website
id-token: write # trade JWT token for AWS credentials in AWS Docs account
secrets: inherit
uses: ./.github/workflows/reusable-publish-docs.yml
with:
version: ${{ inputs.latest_published_version }}
alias: latest
19 changes: 19 additions & 0 deletions .github/workflows/release-doc.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
name: Docs

on:
release:
types:
- published
workflow_dispatch: {}

jobs:
release-docs:
permissions:
contents: write
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're not doing github pages anymore, right? So we can drop pages. I don't believe we're pushing docs back to the repo either, so we can drop contents

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on gh-pages - probably we need contents but not pages ?

pages: write
id-token: write
secrets: inherit
uses: ./.github/workflows/reusable-publish-docs.yml
with:
version: main
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just found that version here is hardcoded. I need to check how it use here released version instead of "main".

Copy link
Contributor

@scottgerring scottgerring Aug 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at reusable-publish-docs.yml, it seems like it should be the actual version alias, not a git ref? It looks like it is dropping the files into there in S3 and using it for the version label 🧐

run: |
aws s3 sync \
site/ \
s3://${{ secrets.AWS_DOCS_BUCKET }}/lambda-java/${{ env.VERSION }}/
- name: Deploy Docs (Alias)
env:
VERSION: ${{ inputs.version }}
ALIAS: ${{ inputs.alias }}

I think we can use github.ref_name -->

https://docs.github.com/en/actions/learn-github-actions/contexts

alias: stage
118 changes: 118 additions & 0 deletions .github/workflows/reusable-publish-docs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
name: Reusable Publish docs

env:
BRANCH: main
ORIGIN: aws-powertools/powertools-lambda-java
VERSION: ""

on:
workflow_call:
inputs:
version:
description: "Version to build and publish docs (1.28.0, develop)"
required: true
type: string
alias:
description: "Alias to associate version (latest, stage)"
required: true
type: string
detached_mode:
description: "Whether it's running in git detached mode to ensure git is sync'd"
required: false
default: false
type: boolean

permissions:
contents: write
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above - I don't think we need contents or pages

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on gh-pages - probably we need contents but not pages ?

id-token: write
pages: write

jobs:
publish-docs:
runs-on: ubuntu-latest
environment: Docs
steps:
- name: Checkout code
uses: actions/checkout@8e5e7e5ab8b370d6c329ec480221332ada57f0ab # v3.5.2
with:
# While `fetch-depth` is used to allow the workflow to later commit & push the changes.
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@57ded4d7d5e986d7296eab16560982c6dd7c923b # v4.6.0
with:
python-version: "3.8"
- name: Install doc generation dependencies
run: |
pip install --upgrade pip
pip install -r docs/requirements.txt
- name: Setup doc deploy
run: |
git config --global user.name Docs deploy
git config --global user.email [email protected]
- name: Git refresh tip (detached mode)
# Git Detached mode (release notes) doesn't have origin
if: ${{ inputs.detached_mode }}
run: |
git config pull.rebase true
git config remote.origin.url >&- || git remote add origin https://github.com/"$ORIGIN"
git pull origin "$BRANCH"
- name: Normalize Version Number
run: echo "VERSION=$(echo ${{ inputs.version }} | sed 's/v//')" >> $GITHUB_ENV
- name: Build docs website and API reference
env:
ALIAS: ${{ inputs.alias }}
run: |
rm -rf site
mkdocs build
mike deploy --update-aliases --no-redirect ${{ env.VERSION }} ${{ env.ALIAS }} --branch backup-gh-pages
# Set latest version as a default
mike set-default latest --branch backup-gh-pages
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@e1e17a757e536f70e52b5a12b2e8d1d1c60e04ef # v2.0.0
with:
aws-region: us-east-1
role-to-assume: ${{ secrets.AWS_DOCS_ROLE_ARN }}
- name: Deploy Docs (Version)
env:
VERSION: ${{ inputs.version }}
ALIAS: ${{ inputs.alias }}
run: |
aws s3 sync \
site/ \
s3://${{ secrets.AWS_DOCS_BUCKET }}/lambda-java/${{ env.VERSION }}/
- name: Deploy Docs (Alias)
env:
VERSION: ${{ inputs.version }}
ALIAS: ${{ inputs.alias }}
run: |
aws s3 sync \
site/ \
s3://${{ secrets.AWS_DOCS_BUCKET }}/lambda-java/${{ env.ALIAS }}/
- name: Deploy Docs (Version JSON)
env:
VERSION: ${{ inputs.version }}
ALIAS: ${{ inputs.alias }}


# We originally used "mike" from PyPi to manage versions for us, but since we moved to S3, we can't use it to manage versions any more.
# Instead, we're using some shell script that manages the versions.
#
# Operations:
# 1. Download the versions.json file from S3
# 2. Find any reference to the alias and delete it from the versions file
# 3. This is voodoo (don't use JQ):
# - we assign the input as $o and the new version/alias as $n,
# - we check if the version number exists in the file already (for republishing docs)
# - if it's an alias (stage/latest/*) or old version, we do nothing and output $o (original input)
# - if it's a new version number, we add it at position 0 in the array.
# 4. Once done, we'll upload it back to S3.

run: |
aws s3 cp \
s3://${{ secrets.AWS_DOCS_BUCKET }}/lambda-java/versions.json \
versions_old.json
jq 'del(.[].aliases[] | select(. == "${{ env.ALIAS }}"))' < versions_old.json > versions_proc.json
jq '. as $o | [{"title": "${{ env.VERSION }}", "version": "${{ env.VERSION }}", "aliases": ["${{env.ALIAS}}"] }] as $n | $n | if .[0].title | test("[a-z]+") or any($o[].title == "${{ env.VERSION }}";.) then $o else $n + $o end' < versions_proc.json > versions.json
aws s3 cp \
versions.json \
s3://${{ secrets.AWS_DOCS_BUCKET }}/lambda-java/versions.json
10 changes: 4 additions & 6 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,11 @@ nav:
- Utilities:
- utilities/idempotency.md
- utilities/parameters.md
- utilities/large_messages.md
- utilities/sqs_large_message_handling.md
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These changes need to be rolled back - from L16-L25. We released new modules and changed the doc structure after you forked!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. I will rebase it onto current main.

- utilities/batch.md
- utilities/validation.md
- utilities/custom_resources.md
- utilities/serialization.md
- Deprecated:
- utilities/sqs_large_message_handling.md
- utilities/sqs_batch.md
- Processes:
- processes/maintainers.md

theme:
name: material
Expand Down Expand Up @@ -87,6 +82,9 @@ extra_javascript:
- javascript/extra.js

extra:
version:
provider: mike
default: latest
powertools:
version: 1.16.1 # to update after each release (we do not want snapshot version here)

Expand Down