Skip to content

Commit a7ca190

Browse files
potiukjedcunningham
authored andcommitted
Add update-constraints command that allows to modify released ones (apache#33144)
* Add update-constraints command that allows to modify released ones This command can be used to semi-automatically update constraints in case we need to update historical version of them, because for some reason (like setuptools or pip or cython changes) we need to update individual constraints in historically released constraints. --------- Co-authored-by: Jed Cunningham <[email protected]>
1 parent a3c8ddc commit a7ca190

15 files changed

+926
-372
lines changed

BREEZE.rst

Lines changed: 44 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -455,7 +455,7 @@ Often errors during documentation generation come from the docstrings of auto-ap
455455
During the docs building auto-api generated files are stored in the ``docs/_api`` folder. This helps you
456456
easily identify the location the problems with documentation originated from.
457457

458-
Those are all available flags of ``build-docs`` command:
458+
These are all available flags of ``build-docs`` command:
459459

460460
.. image:: ./images/breeze/output_build-docs.svg
461461
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output_build-docs.svg
@@ -537,7 +537,7 @@ The above will run the check for the last 4 commits in your branch. You can use
537537
in ``--from-ref`` and ``--to-ref`` flags.
538538

539539

540-
Those are all available flags of ``static-checks`` command:
540+
These are all available flags of ``static-checks`` command:
541541

542542
.. image:: ./images/breeze/output_static-checks.svg
543543
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output_static-checks.svg
@@ -580,7 +580,7 @@ You can also use it to start any released version of Airflow from ``PyPI`` with
580580
581581
breeze start-airflow --python 3.8 --backend mysql --use-airflow-version 2.2.5
582582
583-
Those are all available flags of ``start-airflow`` command:
583+
These are all available flags of ``start-airflow`` command:
584584

585585
.. image:: ./images/breeze/output_start-airflow.svg
586586
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output_start-airflow.svg
@@ -605,7 +605,7 @@ to enter the running container. It's as easy as launching ``breeze exec`` while
605605
Breeze environment. You will be dropped into bash and environment variables will be read in the same
606606
way as when you enter the environment. You can do it multiple times and open as many terminals as you need.
607607

608-
Those are all available flags of ``exec`` command:
608+
These are all available flags of ``exec`` command:
609609

610610
.. image:: ./images/breeze/output_exec.svg
611611
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output_exec.svg
@@ -643,7 +643,7 @@ need to build the images again from scratch - pulling from the registry might ta
643643

644644
Breeze will ask you to confirm each step, unless you specify ``--answer yes`` flag.
645645

646-
Those are all available flags of ``cleanup`` command:
646+
These are all available flags of ``cleanup`` command:
647647

648648
.. image:: ./images/breeze/output_cleanup.svg
649649
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output_cleanup.svg
@@ -736,7 +736,7 @@ You can always stop it via:
736736
737737
breeze down
738738
739-
Those are all available flags of ``down`` command:
739+
These are all available flags of ``down`` command:
740740

741741
.. image:: ./images/breeze/output_down.svg
742742
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output_down.svg
@@ -1394,7 +1394,7 @@ download the latest images before rebuilding because this is usually faster than
13941394
Building CI image
13951395
.................
13961396
1397-
Those are all available flags of ``ci-image build`` command:
1397+
These are all available flags of ``ci-image build`` command:
13981398
13991399
.. image:: ./images/breeze/output_ci-image_build.svg
14001400
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output_ci-image_build.svg
@@ -1406,7 +1406,7 @@ Pulling CI image
14061406
14071407
You can also pull the CI images locally in parallel with optional verification.
14081408
1409-
Those are all available flags of ``pull`` command:
1409+
These are all available flags of ``pull`` command:
14101410
14111411
.. image:: ./images/breeze/output_ci-image_pull.svg
14121412
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output_ci-image_pull.svg
@@ -1419,7 +1419,7 @@ Verifying CI image
14191419
Finally, you can verify CI image by running tests - either with the pulled/built images or
14201420
with an arbitrary image.
14211421
1422-
Those are all available flags of ``verify`` command:
1422+
These are all available flags of ``verify`` command:
14231423
14241424
.. image:: ./images/breeze/output_ci-image_verify.svg
14251425
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output_ci-image_verify.svg
@@ -1488,7 +1488,7 @@ Same as above but uses python 3.8.
14881488
Building PROD image
14891489
...................
14901490
1491-
Those are all available flags of ``build-prod-image`` command:
1491+
These are all available flags of ``build-prod-image`` command:
14921492
14931493
.. image:: ./images/breeze/output_prod-image_build.svg
14941494
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output_prod-image_build.svg
@@ -1500,7 +1500,7 @@ Pulling PROD image
15001500
15011501
You can also pull PROD images in parallel with optional verification.
15021502
1503-
Those are all available flags of ``pull-prod-image`` command:
1503+
These are all available flags of ``pull-prod-image`` command:
15041504
15051505
.. image:: ./images/breeze/output_prod-image_pull.svg
15061506
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output_prod-image_pull.svg
@@ -1513,7 +1513,7 @@ Verifying PROD image
15131513
Finally, you can verify PROD image by running tests - either with the pulled/built images or
15141514
with an arbitrary image.
15151515
1516-
Those are all available flags of ``verify-prod-image`` command:
1516+
These are all available flags of ``verify-prod-image`` command:
15171517
15181518
.. image:: ./images/breeze/output_prod-image_verify.svg
15191519
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output_prod-image_verify.svg
@@ -1528,7 +1528,7 @@ Breeze has tools that you can use to configure defaults and breeze behaviours an
15281528
operations that might be necessary when you add new commands in Breeze. It also allows to configure your
15291529
host operating system for Breeze autocompletion.
15301530
1531-
Those are all available flags of ``setup`` command:
1531+
These are all available flags of ``setup`` command:
15321532
15331533
.. image:: ./images/breeze/output_setup.svg
15341534
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output_setup.svg
@@ -1552,7 +1552,7 @@ it off by passing ``--no-colour`` to config in which case the messages to the us
15521552
will be printed using different schemes (italic/bold/underline) to indicate different kind of messages
15531553
rather than colours.
15541554
1555-
Those are all available flags of ``setup config`` command:
1555+
These are all available flags of ``setup config`` command:
15561556
15571557
.. image:: ./images/breeze/output_setup_config.svg
15581558
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output_setup_config.svg
@@ -1570,7 +1570,7 @@ also force reinstalling the autocomplete via:
15701570
15711571
breeze setup autocomplete --force
15721572
1573-
Those are all available flags of ``setup-autocomplete`` command:
1573+
These are all available flags of ``setup-autocomplete`` command:
15741574
15751575
.. image:: ./images/breeze/output_setup_autocomplete.svg
15761576
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output_setup_autocomplete.svg
@@ -1583,7 +1583,7 @@ Breeze version
15831583
You can display Breeze version and with ``--verbose`` flag it can provide more information: where
15841584
Breeze is installed from and details about setup hashes.
15851585
1586-
Those are all available flags of ``version`` command:
1586+
These are all available flags of ``version`` command:
15871587
15881588
.. image:: ./images/breeze/output_setup_version.svg
15891589
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output_setup_version.svg
@@ -1594,7 +1594,7 @@ Those are all available flags of ``version`` command:
15941594
Breeze self-upgrade
15951595
...................
15961596
1597-
You can self-upgrade breeze automatically. Those are all available flags of ``self-upgrade`` command:
1597+
You can self-upgrade breeze automatically. These are all available flags of ``self-upgrade`` command:
15981598
15991599
.. image:: ./images/breeze/output_setup_self-upgrade.svg
16001600
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output_setup_self-upgrade.svg
@@ -1650,7 +1650,7 @@ Breeze requires certain resources to be available - disk, memory, CPU. When you
16501650
the resources are checked and information if there is enough resources is displayed. However you can
16511651
manually run resource check any time by ``breeze ci resource-check`` command.
16521652
1653-
Those are all available flags of ``resource-check`` command:
1653+
These are all available flags of ``resource-check`` command:
16541654
16551655
.. image:: ./images/breeze/output_ci_resource-check.svg
16561656
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output_ci_resource-check.svg
@@ -1664,7 +1664,7 @@ When our CI runs a job, it needs all memory and disk it can have. We have a Bree
16641664
the memory and disk space used. You can also use it clear space locally but it performs a few operations
16651665
that might be a bit invasive - such are removing swap file and complete pruning of docker disk space used.
16661666
1667-
Those are all available flags of ``free-space`` command:
1667+
These are all available flags of ``free-space`` command:
16681668
16691669
.. image:: ./images/breeze/output_ci_free-space.svg
16701670
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output_ci_free-space.svg
@@ -1684,7 +1684,7 @@ by the root user, you can fix the ownership of those files by running :
16841684
16851685
breeze ci fix-ownership
16861686
1687-
Those are all available flags of ``fix-ownership`` command:
1687+
These are all available flags of ``fix-ownership`` command:
16881688
16891689
.. image:: ./images/breeze/output_ci_fix-ownership.svg
16901690
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output_ci_fix-ownership.svg
@@ -1705,7 +1705,7 @@ from the context of the commit/PR to be merged via stderr output.
17051705
More details about the algorithm used to pick the right tests and the available outputs can be
17061706
found in `Selective Checks <dev/breeze/SELECTIVE_CHECKS.md>`_.
17071707
1708-
Those are all available flags of ``selective-check`` command:
1708+
These are all available flags of ``selective-check`` command:
17091709
17101710
.. image:: ./images/breeze/output_ci_selective-check.svg
17111711
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output_ci_selective-check.svg
@@ -1719,7 +1719,7 @@ When our CI runs a job, it might be within one of several workflows. Information
17191719
is stored in GITHUB_CONTEXT. Rather than using some jq/bash commands, we retrieve the necessary information
17201720
(like PR labels, event_type, where the job runs on, job description and convert them into GA outputs.
17211721
1722-
Those are all available flags of ``get-workflow-info`` command:
1722+
These are all available flags of ``get-workflow-info`` command:
17231723
17241724
.. image:: ./images/breeze/output_ci_get-workflow-info.svg
17251725
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output_ci_get-workflow-info.svg
@@ -2020,7 +2020,7 @@ while publishing the documentation.
20202020
The flag ``--airflow-site-directory`` takes the path of the cloned ``airflow-site``. The command will
20212021
not proceed if this is an invalid path.
20222022
2023-
Those are all available flags of ``release-management publish-docs`` command:
2023+
These are all available flags of ``release-management publish-docs`` command:
20242024
20252025
.. image:: ./images/breeze/output_release-management_publish-docs.svg
20262026
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output_release-management_publish-docs.svg
@@ -2064,7 +2064,7 @@ providers - you can mix apache-airflow, helm-chart and provider packages this wa
20642064
breeze release-management publish-docs --airflow-site-directory DIRECTORY apache.airflow apache.beam google
20652065
20662066
2067-
Those are all available flags of ``release-management add-back-references`` command:
2067+
These are all available flags of ``release-management add-back-references`` command:
20682068
20692069
.. image:: .images/breeze/output_release-management_add-back-references.svg
20702070
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output_release-management_add-back-references.svg
@@ -2109,7 +2109,7 @@ Constraints are generated separately for each python version and there are separ
21092109
providers. If you want to manage airflow separately and then add providers individually, you can
21102110
use those.
21112111
2112-
Those are all available flags of ``generate-constraints`` command:
2112+
These are all available flags of ``generate-constraints`` command:
21132113
21142114
.. image:: ./images/breeze/output_release-management_generate-constraints.svg
21152115
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output_release-management_generate-constraints.svg
@@ -2118,13 +2118,30 @@ Those are all available flags of ``generate-constraints`` command:
21182118
21192119
In case someone modifies setup.py, the scheduled CI Tests automatically upgrades and
21202120
pushes changes to the constraint files, however you can also perform test run of this locally using
2121-
the procedure described in `Refreshing CI Cache <dev/REFRESHING_CI_CACHE.md#manually-generating-constraint-files>`_
2121+
the procedure described in the
2122+
`Manually generating image cache and constraints <dev/MANUALLY_GENERATING_IMAGE_CACHE_AND_CONSTRAINTS.md>`_
21222123
which utilises multiple processors on your local machine to generate such constraints faster.
21232124
21242125
This bumps the constraint files to latest versions and stores hash of setup.py. The generated constraint
21252126
and setup.py hash files are stored in the ``files`` folder and while generating the constraints diff
21262127
of changes vs the previous constraint files is printed.
21272128
2129+
Updating constraints
2130+
""""""""""""""""""""
2131+
2132+
Sometimes (very rarely) we might want to update individual packages in constraints that we generated and
2133+
tagged already in the past. This can be done using ``breeze release-management update-constraints`` command.
2134+
2135+
These are all available flags of ``update-constraints`` command:
2136+
2137+
.. image:: ./images/breeze/output_release-management_update-constraints.svg
2138+
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output_release-management_update-constraints.svg
2139+
:width: 100%
2140+
:alt: Breeze update-constraints
2141+
2142+
You can read more details about what happens when you update constraints in the
2143+
`Manually generating image cache and constraints <dev/MANUALLY_GENERATING_IMAGE_CACHE_AND_CONSTRAINTS.md>`_
2144+
21282145
21292146
SBOM generation tasks
21302147
----------------------
@@ -2174,7 +2191,7 @@ keeps data in their own volume. Those volumes are persisted until ``breeze down`
21742191
You can also preserve the volumes by adding flag ``--preserve-volumes`` when you run the command.
21752192
Then, next time when you start Breeze, it will have the data pre-populated.
21762193
2177-
Those are all available flags of ``down`` command:
2194+
These are all available flags of ``down`` command:
21782195
21792196
.. image:: ./images/breeze/output-down.svg
21802197
:target: https://raw.githubusercontent.com/apache/airflow/main/images/breeze/output-down.svg

0 commit comments

Comments
 (0)