Skip to content

Prefetch build and project on version list #11616

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions readthedocs/builds/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -207,6 +207,9 @@ class Meta:
unique_together = [("project", "slug")]
ordering = ["-verbose_name"]

# Property used for prefetching version related fields
LATEST_BUILD_CACHE = "_latest_build"

def __str__(self):
return self.verbose_name

Expand Down Expand Up @@ -291,6 +294,19 @@ def vcs_url(self):

@property
def last_build(self):
# TODO deprecated in favor of `latest_build`, which matches naming on
# the Project model
return self.latest_build

@property
def latest_build(self):
# Check if there is `_latest_build` prefetch in the Queryset.
# Used for database optimization.
if hasattr(self, self.LATEST_BUILD_CACHE):
if latest_build := getattr(self, self.LATEST_BUILD_CACHE):
Comment on lines +305 to +306
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need the first if here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I started to write it like that but we do need to test for hasattr first, as this attribute only exists after prefetch and it's what shortcuts this method to avoid the return self.builds.... below.

If this was just getattr, we'd always skip the return self.builds... below.

return latest_build[0]
return None

return self.builds.order_by("-date").first()

@property
Expand Down
26 changes: 25 additions & 1 deletion readthedocs/builds/querysets.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

import structlog
from django.db import models
from django.db.models import Q
from django.db.models import OuterRef, Prefetch, Q, Subquery
from django.utils import timezone

from readthedocs.builds.constants import (
Expand Down Expand Up @@ -141,6 +141,30 @@ def for_reindex(self):
.distinct()
)

def prefetch_subquery(self):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't we call it prefetch_latest_build() instead to keep consistency to what we are doing with Project at https://github.com/readthedocs/readthedocs.org/pull/11613/files#diff-05611882195df182851df952b312f031a4314bce04a104e10a09655e570515f3R124?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That method shouldn't be called prefetch_latest_build, it's doing more than that. Similar here, there are other fields to prefetch.

"""
Prefetch related objects via subquery for each version.

.. note::

This should come after any filtering.
"""
from readthedocs.builds.models import Build

# Prefetch the latest build for each project.
subquery_builds = Subquery(
Build.internal.filter(version=OuterRef("version_id"))
.order_by("-date")
.values_list("id", flat=True)[:1]
)
prefetch_builds = Prefetch(
"builds",
Build.internal.filter(pk__in=subquery_builds),
to_attr=self.model.LATEST_BUILD_CACHE,
)

return self.prefetch_related(prefetch_builds)


class VersionQuerySet(SettingsOverrideObject):
_default_class = VersionQuerySetBase
Expand Down
6 changes: 5 additions & 1 deletion readthedocs/projects/views/public.py
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,11 @@ def get_context_data(self, **kwargs):
queryset=versions,
project=project,
)
versions = self.get_filtered_queryset()
versions = (
self.get_filtered_queryset()
.prefetch_related("project")
.prefetch_subquery()
)
context["versions"] = versions

protocol = "http"
Expand Down