subprojects: Search returns broken links when used from subproject subdomain

[ltalirz]

Details

readthedocs.org allows to configure projects as subprojects of other projects.
In this example "aiida-core" is configured as a subproject of "aiida".

With the current default settings, going to the root url of the subproject, i.e. https://aiida-core.readthedocs.io, actually redirects automatically to https://aiida.readthedocs.io/projects/aiida-core/en/latest/.
On the aiida. subdomain everything works fine.

Via direct links such as https://aiida-core.readthedocs.io/en/latest/, however, one can still end up on the subproject domain (and there are a number of such links to our project documentation on the web.).
If one now uses the search field (e.g. for "batch scheduler"), the search results contain broken links of the form

https://aiida-core.readthedocs.io/projects/aiida-core/en/latest/topics/schedulers.html?highlight=batch%20scheduler

I.e. the links contain the subproject path but with the aiida-core subdomain. The correct link would be:
https://aiida.readthedocs.io/projects/aiida-core/en/latest/topics/schedulers.html?highlight=batch%20scheduler

Edit: or, alternatively:
https://aiida-core.readthedocs.io/en/latest/topics/schedulers.html?highlight=batch%20scheduler

• Read the Docs project URL: https://aiida.readthedocs.io/projects/aiida-core/en/latest/
• Build URL (if applicable): https://readthedocs.org/projects/aiida-core/builds/11893452/
• Read the Docs username (if applicable): ltalirz

Expected Result

Search results should not contain broken links, independently of whether search is used on the "parent" project subdomain or on the "child" project subdomain.

Actual Result

Search results on the "child" project subdomain (aiida-core) contain broken links.

[stsewd]

Thanks for reporting! This is because our resolver always resolves to the canonical domain. We could solve this by:

• A) Check in our js client if the current domain is different from the one we return in the API (this is easy, but kind of reverts the reason we went for using relative URLs).
• B) Modify our resolver to accept a keyword arg to resolve using the current project and not the cannonical project

  readthedocs.org/readthedocs/core/resolver.py

  Line 134 in 1b54393

  main_project, subproject_slug = self._get_canonical_project_data(project)

• C) Always redirect to the main domain.

I think C is the more "correct" solution here. Thoughts @ericholscher ?

[ltalirz]

I would just like to point out that if one goes with the solution that ends up returning the link
https://aiida-core.readthedocs.io/en/latest/topics/schedulers.html?highlight=batch%20scheduler
(is this is option A?), this link will continue to work even if, at some point in the future, the subproject "grows up" and is removed from its parent project.

The same is probably not true for
https://aiida.readthedocs.io/projects/aiida-core/en/latest/topics/schedulers.html?highlight=batch%20scheduler
(unless you keep automatic redirects forever).

P.S. Thanks a lot for looking into this!

[stsewd]

Removing the subproject will break the links yes, but you can create a redirect for it if that happens https://docs.readthedocs.io/en/latest/user-defined-redirects.html

[ltalirz]

Removing the subproject will break the links yes, but you can create a redirect for it if that happens https://docs.readthedocs.io/en/latest/user-defined-redirects.html

Thanks, it might actually be useful to add this case as an example to the docs of the exact redirects.
If I understand correctly, it would be:

Type: Exact Redirect
From URL: /projects/myproject/$rest
To URL: https://myproject.readthedocs.org/

[ltalirz]

@ericholscher @stsewd I guess what remains here is to decide among the options listed by @stsewd .

I withdraw my concern with option C (links breaking when a subproject moves away from a parent project). In this scenario, any link that includes the parent project prefix will break (not just the ones returned by search) and, as pointed out by @stsewd, this can be mitigated via setting up a redirect (edit: not quite true, the $rest keyword seems to be broken (?)).

It would, however, be important for us to fix the broken links in the search results (one way or another), since our users keep encountering this.

[ltalirz]

Ping

[ramirezfranciscof]

Hello @ericholscher @stsewd ; I am working with @ltalirz in aiida. We were wondering if there was any news regarding this issue. Would you know if you would be making a decision soon? It would really help us to have this sorted out. If you need any other information to make the decision, or maybe some help afterwards implementing the changes, we might be able to collaborate or lend a hand. Let us know!

In the meantime: do you have any suggestions as to how we could temporarily patch this on our end?

[ericholscher]

@stsewd I'm fine with C here. We should definitely be creating links that always work properly I think. Tricky problem :/

[stsewd]

Ok, we should add the redirect in proxito, hopefully it doesn't add another query for normal projects.

[ericholscher]

@stsewd I don't think we want to redirect, do we? Don't we just want to create the link with the full URL if the current domain doesn't match?

[ericholscher]

Oh I see what you mean -- actually redirect all URL's on the subproject to the subproject domain? Not just search results?

[ericholscher]

I think as a first step we should make sure we aren't generating broken search results, so lets do A first, and then work on C since that's a larger project.

[stsewd]

Ok, got it!

[ericholscher]

If we don't have a ticket for it, we should open one for properly redirecting subproject URL's though. 👍

[ericholscher]

Interestingly, if you hit the root domain, it already does this:

-> curl -IL https://aiida-core.readthedocs.io/
...
location: https://aiida.readthedocs.io/projects/aiida-core/en/latest/

But not for actual file serving requests.

-> curl -IL https://aiida-core.readthedocs.io/en/latest/
HTTP/2 200