Publish code from Orion release 2.0a8 (#5318)

* Use a more realistic mock version

* Add debug print on failed assertion

* build with types

* Fix test

* Ensure version is correct during test runs by doing a deep fetch

* Remove unused threading lock

* Fixup comment

* Types working with a little hacking

* Return failed exit codes correctly

* Update vue-compositions

* Edits to installation and first steps tutorial, including updated UI image.

* Flow and task configuration edits and examples.

* Bundle everything in the components folder

* Task dependencies edits and examples.

* Orchestration components edits.

* FAQ edits.

* Fix some type errors

* Export radar nodes

* Remove name filter

* Add flow_id and task_id sorting options

* WIP on filters for read logs API

* Use flow run and task run IDs for logs

* Use flow_run_id, task_run_id in Log schema

* Use task run and flow run IDs for logs API

* format with black

* Add offset, empty case log API tests

* Update src/prefect/orion/database/interface.py

Co-authored-by: Mariia Kerimova <[email protected]>

* Add LogFilter tests

* Remove unused imports

* Use `schema.dict()` for comparisons so pytest reports accurate diffs

* Add check for `minItems` and `maxItems` for Pydantic 1.9.0+

* fix docs bug

* Use array response for logs, bulk insert

* WIP

* Update `State.__str__` to only include type if meaningful and drop "message="

* Remove some experimental code

* Cleanup

* Allow kwargs to be passed to `run_logger`

* Update `run_logger` docstring

* Use `send_to_orion`; drop `Flow/Task` loggers

* Update docs/getting-started/installation.md

Co-authored-by: Chris White <[email protected]>

* Drop extraneous stuff from logging config

* Update context tests

* Update docs/tutorials/orion.md

* Fixup more tests

* Fix logging for crashes

* Positional-only parameters is py3.8+ only

* Restore simple log format

* Simplify retrieval of parent logger

* Fixup context test

* Fixup output tests

* Test sorting logs API

* Fix log schema field descriptions

* Remove now-unused Logs schema

* Format with black

* Update docs/index.md

Co-authored-by: Michael Adkins <[email protected]>

* Add tests for run logger retrieval

* Review feedback

* Remove now-unused test helper

* Fix bundle error with vite

Not 100% sure why this is happening but using `any` here makes it stop erroring

* Implement `OrionHandler`

* Add `create_logs` to `OrionClient` and pass `dicts` around instead of `Log`

We cannot use `Log` because we need `LogCreate` objects to be passed to the API. Using `LogCreate` outside of the `Client` feels weird

* Fix note in handler

* Fix bare except

* Switch back to `LogCreate` for better error handling

* Fix `getsizeof` call

* Add error handlers

* Minor fixups

* Deployments overview and UI image.

* Add handling when `pending_logs` is empty

* Deployment specification

* Deployment registration

* Running deployments

* Deployment example

* Make inject_db async-aware

* Add log models tests

* Remove unnecessary uses of inject_db

* Misc formatting

* Use exc_info

* Do not emit records if `send_to_orion` is `False

* Fix context loading; add tests for `OrionHandler`

* Update miter to latest version

* Move `prefect flow-run create` to `prefect deployment run`

* Add 3.9+ workaround; add `OrionHandler.flush`

* Expand on regression test description

* Clean up stale docs

* Privatize `OrionLogWorker`

* Use a le, ge filter for log level

* Fix tests

* Fix bug where envion could be left modified if settings failed validation

* Set log interval to float so it can be fast in tests

* Add single log max size setting

* Add `read_logs` support to `OrionClient`

* Pass serialized logs to the `OrionLogWorker` for easier size checking

* Add test for single log size max

* Fixup comment

* Add rudimentary docstrings to client

* Update references to engine logger

* Rename to `get_run_logger()`

* Use a worker fixture to ensure shutdown

* Flush logs when a root flow run completes

* Remove unused imports

* Remove some more unused imports

* Rewriting based on review feedback.

* Reset the OrionHandler as a global fixture; add flow and task run integration tests

* Add logging fixtures file

* Further revsions and reorganizing topics.

* Refactor `OrionHandler` logging settings and add global toggle

* Disable OrionHandler by default during tests and allow enable with a mark

* Add docstring

* Fix mispelling of "Cancelled" state in UI

* Implement OrionAdapter for preserving extra fields

* Update src/prefect/utilities/logging.py

Co-authored-by: Michael Adkins <[email protected]>

* Update docs/concepts/deployments.md

Co-authored-by: Bill Palombi <[email protected]>

* Update docs/concepts/deployments.md

Co-authored-by: Bill Palombi <[email protected]>

* Update docs/concepts/deployments.md

Co-authored-by: Bill Palombi <[email protected]>

* Rename to `PrefectLogAdapter` because it is not Orion specific

* Link to Python issue

* Move `logging.yml` to new logging module

* Move logging into separate modules at 'prefect.logging'

* Remove deferred imports

* Move logging tests up a level

* Update manfiest and docs for move

* Remove old logging module

* Add `JsonFormatter` docstring

* Add more docstrings

* Move `AsyncMock` from compat to testing utility module

This allows us to drop the try/except import which felt weird. Things that are only used for testing belong in the testing utils module.

* Update miter to latest version

* Use an event instead instead of sleep to reduce brittleness

* Update docs/concepts/deployments.md

Co-authored-by: Michael Adkins <[email protected]>

* Update docs/concepts/deployments.md

Co-authored-by: Michael Adkins <[email protected]>

* Update docs/concepts/deployments.md

Co-authored-by: Michael Adkins <[email protected]>

* Update docs/concepts/deployments.md

Co-authored-by: Michael Adkins <[email protected]>

* Update docs/concepts/deployments.md

Co-authored-by: Michael Adkins <[email protected]>

* Revised description of ways to create deployment specs.

* Enable services by default with `prefect orion start`

* Revise options for deployment creation.

* Add flow runner concepts

* Add blurb for `FlowRunnerSettings`

* Fix universal flow runner import

* Add tutorial

* Fix `prefect deployment run` command

This command was not updated when the client interface changed

* Add note about custom image requirement

* Add `FROM` recommendation

* Whitespace changes

* Fix `prefect deployment run` command

This command was not updated when the client interface changed

* Add virtual environments tutorial

* Add venv tutorial to flow-runners concepts

* Update header

* Flow runner doc edits.

* Add flow run docs to nav.

* Only schedule with the scheduler

* Change router watchers to better reflect component lifecycle

* Update docs/concepts/flow-runners.md

Co-authored-by: Chris White <[email protected]>

* Update docs/concepts/flow-runners.md

Co-authored-by: Chris White <[email protected]>

* Update docs/tutorials/docker-flow-runner.md

Co-authored-by: Chris White <[email protected]>

* Update docs/tutorials/docker-flow-runner.md

Co-authored-by: Chris White <[email protected]>

* Feedback from review

* Minor edits based on feedback.

* Update bash examples, removing prompts.

* Revise explanation of flow runner operation.

* Additional revisions based on feedback.

* Additional revision based on feedback.

* Minor code revisions.

* Revise using a flow runner.

* Only retain python test docker images for 1 day

The default retention of 90 days is expensive if we build 3 images for every PR commit

* Use consistent casing for "task"

* Rename file name for clarity

* Add CODEOWNERS

* Add Dustin

* Add Zach and Andrew as Orion server owners

* Remove unecesssary conditional

It does the same evaluation as the while loop check, oops.

* Remove `=`, allows logs to be too big

* Disable caching for date_add

* Fix release PR to the public repository by resolving merge conflicts automatically

* Pull docker image before we create a container

* Add release notes for 2.0a8

* Add image pull policy to Docker flow runner

* Improve mocking hygiene

* Update release notes for implementation in #810

* Add log on image pull

* Use the string repr

* Use `AutoEnum` and move from orion to prefect utilities

* Fixup autoenum usage

* Update docs

* Update discussion of flow runner types and default flow runners.

* Restore circleci config

* Drop welcome orb

* Minimal circle config?

* Remove filter

* Fixup job name

* Set the asyncio mode to auto

* Fix pytest-asyncio failures; increase lower bound since we rely on auto now

See pytest-dev/pytest-asyncio#257

Co-authored-by: Craig Harshbarger <[email protected]>
Co-authored-by: Terrence Dorsey <[email protected]>
Co-authored-by: Andrew Brookins <[email protected]>
Co-authored-by: Zach Angell <[email protected]>
Co-authored-by: Chris White <[email protected]>
Co-authored-by: Andrew Brookins <[email protected]>
Co-authored-by: Mariia Kerimova <[email protected]>
Co-authored-by: Anna Geller <[email protected]>
Co-authored-by: Terrence Dorsey <[email protected]>
Co-authored-by: Zach Angell <[email protected]>
Co-authored-by: Bill Palombi <[email protected]>
Co-authored-by: nicholas <[email protected]>
Co-authored-by: Nicholas Brown <[email protected]>

Author: 12 people

.circleci/config.yml

@@ -1,9 +1,9 @@
 version: 2.1
 
-orbs:
-  welcome: circleci/[email protected]
+jobs:
+  build:  # must be called 'build' if not using a workflow
+    docker:
+      - image: alpine:latest
 
-workflows:
-  welcome:
-    jobs:
-      - welcome/run
+    steps:
+      - run: echo "looks good to me!"

.flake8

@@ -1,2 +1,20 @@
-# global owners
-*   @madkinsz @zangell44
+# global
+*                                    @madkinsz
+
+# release
+/.github/workflows/release.yaml      @cicdw
+
+# orion server
+/src/prefect/orion                   @zangell44 @madkinsz @abrookins
+
+# ui
+/orion-ui                            @pleek91
+
+# documentation
+/docs                                @tpdorsey
+
+# orchestration rules / policies
+/src/prefect/orion/orchestration     @anticorrelator
+
+# database configuration / models
+/src/prefect/orion/database          @anticorrelator

.github/CODEOWNERS

@@ -1,11 +1,11 @@
+# To get started with Dependabot version updates, you'll need to specify which
+# package ecosystems to update and where the package manifests are located.
+# Please see the documentation for all configuration options:
+# https://help.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
+
 version: 2
 updates:
-- package-ecosystem: pip
-  directory: "/"
-  schedule:
-    interval: weekly
-    day: saturday
-    time: "04:00"
-    timezone: America/New_York
-  open-pull-requests-limit: 10
-  rebase-strategy: disabled
+  - package-ecosystem: "pip" # See documentation for possible values
+    directory: "/" # Location of package manifests
+    schedule:
+      interval: "daily"