Subprojects page is throwing 502s

[siddacious]

Details

This is the same issue as #6027, with the same project

• Read the Docs project URL:
  https://readthedocs.org/projects/circuitpython/
• Build URL (if applicable):
• Read the Docs username (if applicable):
  siddacious

Expected Result

Going to https://readthedocs.org/dashboard/circuitpython/subprojects/
should load within a second or two

Actual Result

Recently I will often get a 502 when loading the subprojects admin page. When I don't, I almost always get a 502 upon submitting the form, though the addition appears to be successful

[stsewd]

Just loading the page https://readthedocs.org/projects/circuitpython/ I can see it takes a long time to load... I think this is because we are calling the resolver for each subproject (to get the docs URL). We need to optimize this somehow.

[tannewt]

@stsewd Let us know if there is a sponsor tier we can join to help get this fixed. I'm personally a gold member but we aren't as a company.

[humitos]

Our resolve method is doing 7 db queries to resolve each URL. Some of them are duplicated when resolving one URL and most of them are duplicated when resolving multiple URLs for subprojects of the same project. This method is probably a good place to take a look and try to optimize.

[siddacious]

@humitos or whomever else; please let us know what we can do to help get this addressed. It's impacting part of our process for documenting our libraries

[humitos]

Hi @siddacious! I took a look at the code and it really needs a refactor. Although, it does not look as a small refactor or easy to attack. I got lost very quickly while trying to make some performance improvements. Unfortunately, I don't have the time or energy currently to focus 100% on this problem but I will keep thinking on a different approach for a solution (at least an acceptable hack) for now.

If someone else has some ideas about how we can optimize this code to avoid this problems, we would be happy to help.

[agjohnson]

@siddacious we're going to get this on our short term roadmap, it's a pretty clear bug of our subproject handling. I think we understand what needs to change, but if there is a short term fix that can fix the bug for your projects, we'll see about temporarily putting that in place for your projects.

[siddacious]

@agjohnson Awesome, great to hear. Thank you!

[siddacious]

Thank you @stsewd ! Looks like you folks have a roughly weekly release cadence? Much appreciated, great work :)

[stsewd]

@siddacious sorry for the delay resolving this issue, we are doing a release today, hopefully in a couple of hours we can check if the merged PRs were enough. #6867 is still missing, but I hope that whit what already have is enough to stop the 500.

[stsewd]

@siddacious it should be out now, can you please confirm it that fixed the problem?

[siddacious]

@stsewd Yes, it's fixed! Thank you :)