Skip to content

Automate compile version & upload to S3 #179

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 23 commits into from
Apr 14, 2022
Merged

Automate compile version & upload to S3 #179

Changes from 1 commit
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
e48fe15
Automate compile version & upload to S3
humitos Apr 4, 2022
0522a81
Initial test using AWS orbs and matrix
humitos Apr 12, 2022
6876dc3
Config: define docker keys
humitos Apr 12, 2022
52ad5a9
Test
humitos Apr 12, 2022
5e7a436
Use orbs/aws to upload the file to -dev S3 bucket
humitos Apr 13, 2022
452cd6b
Define proper Docker images
humitos Apr 13, 2022
4e63957
Typo
humitos Apr 13, 2022
c22fbd8
Call the script via bash
humitos Apr 13, 2022
8368f9e
Better naming
humitos Apr 13, 2022
5c41f9a
Define AWS_REGION to avoid script entering in dev mode
humitos Apr 13, 2022
3e27b46
Remove names, they are not supported here
humitos Apr 13, 2022
8e115c3
Add extra comments
humitos Apr 13, 2022
c9a3847
Split docker image with bash before pulling it
humitos Apr 13, 2022
172176b
Tag the image so the script can find it
humitos Apr 13, 2022
1f3c6e2
Compile all tools/versions in the same job
humitos Apr 13, 2022
18e095f
Typo
humitos Apr 13, 2022
a9347a7
Changes on the bash script
humitos Apr 13, 2022
2f2b21e
Remove parameters
humitos Apr 13, 2022
226b303
Typo
humitos Apr 13, 2022
7a65115
Disable cache which is not supported in our plan
humitos Apr 13, 2022
0a40efb
Disable Python 3.6.15 and style $TOOLS
humitos Apr 13, 2022
335d40e
Re-enable Python 3.6.15 and build it with clang
humitos Apr 14, 2022
7ccd0fb
Update branches to use `main`
humitos Apr 14, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 69 additions & 0 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,48 @@
version: 2

jobs:
compile-upload-tools:
parameters:
os:
type: string
tool:
type: string

docker:
- image: cimg/python:3.10
- environment:
OS=<< parameters.os >>
# TODO: how do we set this for .org and/or .com?
AWS_REGION=
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
AWS_BUILD_TOOLS_BUCKET=

steps:
- checkout
- setup_remote_docker:
version: 20.10.11
docker_layer_caching: true
- run:
name: Install depedencies
command: |
pip install awscli==1.20.34
- run:
name: Download Docker image
command: |
docker pull readthedocs/build:<< parameters.os >>
- run:
name: Download compile_version_upload_s3.sh script
command: |
wget https://raw.githubusercontent.com/readthedocs/readthedocs.org/rel/scripts/compile_version_upload_s3.sh
- run:
name: Compile tool version
command: |
# TODO: How do we iterate over `parameterts.tool` properly here?
# TODO: How do we run only the command we want, avoiding compiling all the tools/versions each time?
./compile_version_upload_s3.sh << parameters.tool >> << parameters.tool.version >>


tests:
docker:
- image: cimg/python:3.10
Expand Down Expand Up @@ -29,3 +71,30 @@ workflows:
tests:
jobs:
- tests

compile:
jobs:
- compile-upload-tools:
matrix:
parameters:
os:
- ubuntu-20.04
- ubuntu-22.04
tool:
python:
- 2.7.18
- 3.6.15
- 3.7.12
- 3.8.12
- 3.9.7
- 3.10.0
- pypy3.7-7.3.5
- miniconda3-4.7.12
- mambaforge-4.10.3-10
nodejs:
- 14.17.6
- 16.9.1
rust:
- 1.55.0
golang:
- 1.17.1