Mismatch when updating latest via webhook and after clone

[nadyawilliams]

Details

I am building the docs for a repo that has not been changed today. Simply making sure that builds go ok.

First successful build at the timestamp Completed Nov. 20, 2023. 11:07 p.m.
According to the console logs, the first two commands of the build were :
git clone --depth 1 https://github.com/RCIC-UCI-Public/rcs3-docs .
git fetch origin --force --prune --prune-tags --depth 50

With no changes to the repo or RTD settings, from RTD console i execute a second build about 30 min later

Second build failed at the timestamp Completed Nov. 20, 2023. 11:33 p.m.
According to the console logs the first two commands of this build were different:
git clone --depth 1 https://github.com/RCIC-UCI-Public/rcs3-docs .
git fetch origin --force --prune --prune-tags --depth 50 refs/heads/master:refs/remotes/origin/master
fatal: couldn't find remote ref refs/heads/master

Expected Result

Considering no repo or RTD configuration was touched in any way I assume it is a result of some changes in
the automated build process.

The second git command additional parameters refs/heads/master:refs/remotes/origin/master
make this git fetch command fail.

Please advise how to avoid the addition of these extraneous arguments to git fetch command during the automated build.
No such parameters are present anywhere in my repo files

Thanks

[nadyawilliams]

Additional info. As of this morning with no changes to RTD or city repo the build succeeds. Here is the build history of
3 builds:
passed Build #22612469 Completed Nov. 20, 2023. 11:07 p.m.
failed Build #22612647 Completed Nov. 20, 2023. 11:33 p.m.
passed Build #22621615 Completed Nov. 21, 2023. 5 p.m.

Quite a similar situation is happening to another project/repo (no changes to either git files or rtd configuration)
failed Build #22612511 Completed Nov. 20, 2023. 11:12 p.m.
failed Build #22621623 Completed Nov. 21, 2023. 5:01 p.m.
passed Build #22623612 Completed Nov. 21, 2023. 8:12 p.m.

All successful builds have no extra "refs/heads/master:refs/remotes/origin/master" in the git fetch commands.

It's almost as if the successful builds land on a different server compare to the failed builds.
It would be nice to have a better predictability of what is going on so that one can depend on the build outcome.

[Qizhen-Xue]

Met the same fetch error when build documents on ReadtheDocs today.

The document for our project is https://nnabla.readthedocs.io/en/latest/. We use tags of the project to control the release document version. But the default git fetch command used by ReadtheDocs could not support tags now.

I tried to replace refs/heads/v1.37.0 with refs/tags/v1.37.0. Then the fetch command could run successfully.

I believe there must be many other projects would met the same error. Hope the maintainers of ReadtheDocs could handle this issue soon.

[nadyawilliams]

This morning, Nov 22, both of my projects builds are failing yet again after a successful build yesterday.
No tags, branches, etc changes to the git repo. NO changes to the RTD settings.
Build #22634012
Build #22633991
The same culprit is in git fetch command on the server where the builds are running.

[stsewd]

@nadyawilliams hi, did you change your project's default branch setting to be explicitly main? That should fix the problem.

Previously, what was happening is that your project's default version was being updated by the GitHub webbook (to point to main)

readthedocs.org/readthedocs/api/v2/views/integrations.py

Lines 315 to 320 in 1b38024

	if self.get_integration().secret and not self.project.default_branch:
	# Always check for the machine attribute, since latest can be user created.
	# RTD doesn't manage those.
	self.project.versions.filter(slug=LATEST, machine=True).update(
	identifier=default_branch
	)

And then again after triggering a build (to point to master)

readthedocs.org/readthedocs/projects/models.py

Lines 1284 to 1292 in 1b38024

	def get_default_branch(self):
	"""Get the version representing 'latest'."""
	if self.default_branch:
	return self.default_branch
	if self.remote_repository and self.remote_repository.default_branch:
	return self.remote_repository.default_branch
	return self.vcs_class().fallback_branch

This is kind of related to #10892 and how we handle latest.

[nadyawilliams]

Yes, i did this change earlier today before i saw your response. My git repos were initially on the "main" branch and never had any changes from master to main. Webhook behavior was not changed either with the exception of creating a new one with the secret and all the rest remaining the same. The webhook was changed before i run the builds and got pass/fail behavior. What throw me for a loop was an inconsistent behavior: i would get pass or fail without any changes to the repo or RTD settings. What settings i have should either make all builds work or all builds fail but failing or working within few minutes from each other with no changes on my side indicate that some of the build servers are probably not using the same build logic path. This is an inconsistency on RTD side that probably needs to be looked at.

[stsewd]

@nadyawilliams yep this is a bug in RTD, I just described the cause at #10915 (comment), there is a mismatch in how the webhook updates latest and how the sync after a clone does.

[nadyawilliams]

Ok, thanks. Can I assume that the update of a default branch to main will hold on for all the builds moving forward, not just some builds.

[stsewd]

@nadyawilliams yeah, explicitly defining your default branch will prevent this bug from re-appearing in your projects.

[nadyawilliams]

Thanks