Skip to content

External versions: keep build log #7674

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
stsewd opened this issue Nov 17, 2020 · 6 comments · Fixed by #7679
Closed

External versions: keep build log #7674

stsewd opened this issue Nov 17, 2020 · 6 comments · Fixed by #7679
Labels
Needed: design decision A core team decision is required

Comments

@stsewd
Copy link
Member

stsewd commented Nov 17, 2020

Currently, when a PR is closed/merged, we delete the version, and it deletes the build objects with it, but we do keep the files in storage. The problem with this is that it feels weird lo lose the information from the build after the PR is closed.

  • One idea is to not cascade delete the build objects, but do we want to do the same behavior for normal versions?
  • If we delete the version and left it as null, we still need to save some extra information in the build model, like the version_slug and version_name (version.verbose_name). This is to allow us to link back to the PR (the view docs button wouldn't be available here, or we can save that as well).

With the last sentence there is the additional question... Do we want to delete the docs too? Currently, we don't delete the docs, they are kept forever. Should we delete them after x days? This probably something we want to do in .com, as we shouldn't keep private docs around forever.

Some ideas:

  • Delete the version, but don't delete the build object, the extra fields should be able to link to the original PR and build docs (docs are kept).
  • Don't delete the version when the PR is closed, but mark it as inactive. Have a task delete all external versions marked as inactive and where the last edited date is greater than x days, but set the relationship of the build to null, so it doesn't get deleted (here the docs are deleted as well).
  • The same as above, but delete the build objects too.
  • Keep both, version/builds and docs, but allow users to delete them manually, additionally have an automation rule to delete versions from closed PRs.
@stsewd stsewd added the Needed: design decision A core team decision is required label Nov 17, 2020
@ericholscher
Copy link
Member

ericholscher commented Nov 17, 2020

I think we want a DB record for all Build's, so we have history of it. I think we should probably remove the version on delete and just store a version_slug or similar.

I don't think we want to keep Version objects around forever, given the issues we have with too many of them. Keeping the Build's seem important.

@stsewd
Copy link
Member Author

stsewd commented Nov 17, 2020

What about the docs from external versions? Should we keep those around as well?

@ericholscher
Copy link
Member

ericholscher commented Nov 17, 2020

Edit: Misunderstood

I think we want to keep the docs around, but perhaps for some limited time? Eventually this will start costing us real money, so probably after 3-6 months we delete it? Maybe we keep Versions & Docs around for 3 months, and then delete them together.

@humitos
Copy link
Member

humitos commented Nov 18, 2020

Keeping the Build's seem important.

👍

Maybe we keep Versions & Docs around for 3 months, and then delete them together.

👍 --maybe we want to make this configurable by project? like 0, 1, 3, 6 or 12 months? or even keep it for X months but allow the user to "force delete a particular external version"?

If we delete the docs, all the GH status links will point to 404 pages. We will probably need to update those states to point back to the build when deleting the docs, I think.


I think we should consider this from the UI as well. Currently, trying to find "the latest/stable version build" is not trivial if you have many open PRs. So, we will need a way to filter by (non-)external versions, cc @agjohnson

@ericholscher
Copy link
Member

I think we should consider this from the UI as well. Currently, trying to find "the latest/stable version build" is not trivial if you have many open PRs. So, we will need a way to filter by (non-)external versions, cc @agjohnson

We already hide PR builds in the UI, but I think we should expose them. Having filters should be enough to solve most of the issues here. I originally had a different build listings for PR's, but I think filters are more useful.

stsewd added a commit that referenced this issue Nov 18, 2020
Currently we are deleting external versions
when they are closed, but their docs remain in storage.
With this change we mark them as inactive instead.
Later a task will delete all external versions that are inactive
and their last activity was from 3 months ago.

Before being deleted the status is updated to link to the build page
instead (the build will be deleted currently, but I'm changing that in
another PR).

Ref #7674
stsewd added a commit that referenced this issue Nov 18, 2020
Currently we are deleting external versions
when they are closed, but their docs remain in storage.
With this change we mark them as inactive instead.
Later a task will delete all external versions that are inactive
and their last activity was from 3 months ago.

Before being deleted the status is updated to link to the build page
instead (the build will be deleted currently, but I'm changing that in
another PR).

Ref #7674
stsewd added a commit that referenced this issue Nov 18, 2020
We lose information when a version is deleted.
Setting the version to null allow us to keep the builds around.
But now we need to save more data to be able to reproduce some links.

A data migration is needed to update old builds, but isn't required,
as we fallback to the value from the version.

Close #7674
stsewd added a commit that referenced this issue Nov 18, 2020
Currently we are deleting external versions
when they are closed, but their docs remain in storage.
With this change we mark them as inactive instead.
Later a task will delete all external versions that are inactive
and their last activity was from 3 months ago.

Before being deleted the status is updated to link to the build page
instead (the build will be deleted currently, but I'm changing that in
another PR).

Ref #7674
@agjohnson
Copy link
Contributor

Yeah, filters are already a part of the new theme on the build list page and version list page. I'd also like to make a special filter eventually which is latest/stable at the top of the queryset filter. I'll note to add a filter for pull request builds if there isn't already a filter on version/builds

stsewd added a commit that referenced this issue Dec 14, 2020
We lose information when a version is deleted.
Setting the version to null allow us to keep the builds around.
But now we need to save more data to be able to reproduce some links.

A data migration is needed to update old builds, but isn't required,
as we fallback to the value from the version.

Close #7674
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needed: design decision A core team decision is required
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants