Skip to content

Router PRs builds to last queue where a build was executed #7912

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 15, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion readthedocs/builds/tasks.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
)
from readthedocs.builds.constants import (
BRANCH,
EXTERNAL,
BUILD_STATUS_FAILURE,
BUILD_STATUS_PENDING,
BUILD_STATUS_SUCCESS,
Expand All @@ -38,10 +39,11 @@ class TaskRouter:
Celery tasks router.

It allows us to decide which queue is where we want to execute the task
based on project's settings but also in queue availability.
based on project's settings.

1. the project is using conda
2. new project with less than N successful builds
3. version to be built is external

It ignores projects that have already set ``build_queue`` attribute.

Expand Down Expand Up @@ -92,6 +94,16 @@ def route_for_task(self, task, args, kwargs, **__):
)
return self.BUILD_LARGE_QUEUE

# Use last queue used for external versions
if version.type == EXTERNAL:
for build in last_builds.iterator():
Comment on lines +98 to +99
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I got confused here. This is wrong.

We need to find out the last_builds for the default Version here. The Version to be built (external) has no builds at all if it's new. I'll open a new PR to fix this.

if not build.builder:
continue

if 'default' in build.builder:
return self.BUILD_DEFAULT_QUEUE
return self.BUILD_LARGE_QUEUE

# We do not have enough builds for this version yet
if queryset.count() < self.N_BUILDS:
log.info(
Expand Down
21 changes: 21 additions & 0 deletions readthedocs/builds/tests/test_celery_task_router.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
from django.test import TestCase
from django.test.utils import override_settings

from readthedocs.builds.constants import EXTERNAL
from readthedocs.builds.models import Build, Version
from readthedocs.builds.tasks import TaskRouter
from readthedocs.projects.models import Project
Expand Down Expand Up @@ -67,3 +68,23 @@ def test_no_build_pk(self):
self.assertIsNone(
self.router.route_for_task(self.task, self.args, {}),
)

def test_external_version(self):
self.version.type = EXTERNAL
self.version.save()

self.build.builder = 'build-default-a1b2c3'
self.build.save()

self.assertEqual(
self.router.route_for_task(self.task, self.args, self.kwargs),
TaskRouter.BUILD_DEFAULT_QUEUE,
)

self.build.builder = 'build-large-a1b2c3'
self.build.save()

self.assertEqual(
self.router.route_for_task(self.task, self.args, self.kwargs),
TaskRouter.BUILD_LARGE_QUEUE,
)