Skip to content

Fix Circle CI builds: Tox 4 compatibility, add external commands to allowlist #9789

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Dec 8, 2022
Merged

Fix Circle CI builds: Tox 4 compatibility, add external commands to allowlist #9789

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
c7b3959
./manage.py now required to be in "allowlist_externals" so we invoke …
benjaoming Dec 8, 2022
4af2301
Add sh+git to allowlist_externals
benjaoming Dec 8, 2022
36568be
Try explicitly fetching main branch
benjaoming Dec 8, 2022
b43d7b8
Fail faster: Run pre-commit before anything else, move nvm installati…
benjaoming Dec 8, 2022
a2ea51c
Call with "base_python", that's the MOST correct way :100:
benjaoming Dec 8, 2022
2ea33be
Rename remaining white=>allow
benjaoming Dec 8, 2022
c6ab19f
Adjust tox.embedapi.ini for Tox 4
benjaoming Dec 8, 2022
c201580
Upper bound the testembedapi tox<4
benjaoming Dec 8, 2022
4cad872
Add upper bound for tox
benjaoming Dec 8, 2022
8c76c74
Revert a no-op change to env names in tox.embedapi.ini
benjaoming Dec 8, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 10 additions & 8 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ jobs:
- checkout
- run: git submodule sync
- run: git submodule update --init
- run: pip install --user tox
- run: pip install --user 'tox<5'
- run: tox -e py310
- codecov/upload

Expand All @@ -29,7 +29,7 @@ jobs:
- checkout
- run: git submodule sync
- run: git submodule update --init
- run: pip install --user tox
- run: pip install --user 'tox<4'
- run: tox -c tox.embedapi.ini

checks:
Expand All @@ -41,18 +41,20 @@ jobs:
- checkout
- run: git submodule sync
- run: git submodule update --init
- run: pip install --user tox
- run: git fetch origin main # needed for comparisson in pre-commit
- run: git branch --track main origin/main # needed for comparisson in pre-commit
Copy link
Contributor Author

@benjaoming benjaoming Dec 8, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is apparently now required. But not related to tox. I don't understand why pre-commit started failing 🤷 But I do understand if Circle CI doesn't fetch all the branches from the remote, it would seem like a good optimization.

Before this, I was getting:

[INFO] Restored changes from /home/circleci/.cache/pre-commit/patch1670506529-1455.
An unexpected error has occurred: CalledProcessError: command: ('/usr/bin/git', 'diff', '--name-only', '--no-ext-diff', '-z', 'main..HEAD')
return code: 128
expected return code: 0
stdout: (none)
stderr:
    fatal: ambiguous argument 'main..HEAD': unknown revision or path not in the working tree.
    Use '--' to separate paths from revisions, like this:
    'git <command> [<revision>...] -- [<file>...]'
    
Check the log at /home/circleci/.cache/pre-commit/pre-commit.log
pre-commit: exit 3 (15.94 seconds) /home/circleci/project> pre-commit run --from-ref main --to-ref HEAD pid=1455
  pre-commit: FAIL code 3 (19.02=setup[3.08]+cmd[15.94] seconds)
  evaluation failed :( (19.19 seconds)

Exited with code exit status 3

- run: pip install --user 'tox<5'
- run: tox -e pre-commit
- run: tox -e migrations
- run: tox -e lint
- run: tox -e docs
- run: tox -e docs-dev
- run: scripts/circle/install_node.sh
- run:
name: Add node to the path
command: |
echo 'export PATH=~/.nvm/versions/node/v${NODE_VERSION}/bin:$PATH' >> $BASH_ENV
source $BASH_ENV
- run: tox -e migrations
- run: tox -e pre-commit
- run: tox -e lint
- run: tox -e docs
- run: tox -e docs-dev
- run: tox -e eslint

workflows:
Expand Down
5 changes: 4 additions & 1 deletion tox.embedapi.ini
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
[tox]
# NOTE: Currently tox 3 is required, breaks with tox 4, possibly basepython 3.10 needs to be defined
# NOTE: Sphinx 3.5 and 4.x < 4.2 fails with Python 3.10 because of a typing issue
#
# NOTE: Sphinx 1.8 and 2.0 are not tested anymore because of some
Expand All @@ -7,10 +8,12 @@ envlist = sphinx-{21,22,23,24,30,31,32,33,34,42,43,44,45,50,latest,24-docutils-0

[testenv]
description = run test suite for the EmbedAPIv3
allowlist_externals =
sh
install_command =
# Install requirements in multiple steps because we don't want to install
# Sphinx from `requirements/pip.txt` but from the `deps=` field.
/bin/sh -c ' \
sh -c ' \
cat {toxinidir}/requirements/pip.txt | grep -v "Sphinx" > {toxinidir}/requirements/embedapi.txt; \
sed {toxinidir}/requirements/testing.txt -e "s|pip.txt|embedapi.txt|g" > {toxinidir}/requirements/testing.embedapi.txt; \
pip install -r {toxinidir}/requirements/testing.embedapi.txt; \
Expand Down
13 changes: 8 additions & 5 deletions tox.ini
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,13 +18,16 @@ deps =
basepython =
python3.10
commands =
/bin/sh -c '\
sh -c '\
export DJANGO_SETTINGS_MODULE=readthedocs.settings.test; \
pytest --cov-report=xml --cov-config .coveragerc --cov=. --pyargs readthedocs --suppress-no-test-exit-code -m "not proxito and not embed_api" {posargs:{env:TOX_POSARGS:-m "not search and not proxito and not embed_api"}}'

/bin/sh -c '\
sh -c '\
export DJANGO_SETTINGS_MODULE=readthedocs.settings.proxito.test; \
pytest --cov-report=xml --cov-config .coveragerc --cov=. --cov-append --pyargs readthedocs -m proxito --suppress-no-test-exit-code {posargs}'
allowlist_externals =
sh
git

[testenv:docs]
description = Build readthedocs user documentation
Expand All @@ -49,7 +52,7 @@ commands =
[testenv:migrations]
description = check for missing migrations
commands =
./manage.py makemigrations --check --dry-run
{base_python} manage.py makemigrations --check --dry-run

[testenv:lint]
description = run linter (prospector) to ensure the source code corresponds to our coding standards
Expand All @@ -75,15 +78,15 @@ commands =
pre-commit run --from-ref main --to-ref HEAD

[testenv:eslint]
whitelist_externals = npm
allowlist_externals = npm
description = run the JavaScript linter (requires `npm install`)
commands =
npm run lint

[testenv:coverage]
description = shows the coverage report
deps = coverage
whitelist_externals = echo
allowlist_externals = echo
commands =
coverage report --show-missing
coverage html
Expand Down