Contextualize 404 page

[benjaoming]

Update: This PR has gone through manual testing and is now ready for review.

Work history:

• This PR is extends work from Adds more basic info to the default 404 page #9656
• Base branch was changed to Proxito: use unresolver in 404 handler #10074
• After Proxito: use unresolver in 404 handler #10074 was merged, base branch changed to main again
• New implementation for unresolver landed
• Added support in new unresolver
• Removed implementation in old/legacy El Proxito code
• Moved exception classes and 404 view to El Proxito

TODO

• Version not found
• Subproject page not found
• Handle exception types with instanceof or similar
• Remove too much debug logging
• For "project page not found", we could try to suggest searching because {% url "elastic_project_search" project.slug %} affords us this option across Documentation project types?
• Remember to remove errors/404/include_search.html in cases where searching isn't possible
• Display correct request.path, not the one with _proxito_404_ in the path...
• Add exception for "language not found"

What it looks like / scope

There are individual templates for different scenarios:

• Page not found
• Project not found (domain not found)
• Subproject not found Scoped out, not currently covered by new unresolver
• Translation not found
• Version not found

Each case is detected, but we resort to only giving very basic suggestions and information. I made that choice to avoid discussing details at this stage.

Here is the 404 for a missing page:

And here is the 404 for a missing version:

Where to go now?

We can do more work to help users and documentation owners troubleshoot 404s. I think we just need to make small improvised changes to the 404s, as we get inspired from real life.

Brainstorming ideas for contextualized messages

Suggestions for "project page not found"

• Inform people that project "xyz" does not contain this page, possibly because it got updated.
• Look at the URL to understand if it's the /latest/ branch and tell the user that projects are updated over time, so a link to "latest" may change and that's a good thing. They can look for the page in older versions if the latest version doesn't contain a similar page.
• Give people a search form for the project? That should be easy, right? <form method="GET" action="/en/{{ default_branch }}/search.html?> This needs a lot more context, i.e. "Is this a Sphinx project?" and "Which language are we looking at?". I don't find it feasible to implement.
• Update: Instead of in-doc search, we can give people our ES powered search using the project scope 💯 🎉
• A suggestion box for documentation developers
  • Use custom 404 pages: https://docs.readthedocs.io/en/latest/hosting.html#custom-not-found-404-pages
  • Use redirects: https://docs.readthedocs.io/en/stable/user-defined-redirects.html

For "project not found"

• Tell people that the URL is wrong
• Project might have been deleted

For "subproject page not found"

• Tell people that the URL is wrong
• Tell people that the page in the subproject did not exist
• That the page may have failed to build
• That a translation may not be available (not ready)
• A suggestion box for documentation developers

For "translation not found"

• Something like "help translate this project to new languages" :)

For "readthedocs.org page not found"

• The link you followed is broken
• We will look into this if it's a technical error

References #2551

---

📚 Documentation previews 📚

• User's documentation (docs): https://docs--9657.org.readthedocs.build/en/9657/

• Developer's documentation (dev): https://dev--9657.org.readthedocs.build/en/9657/

[benjaoming]

Note on the current state of the PR:

I'm still not looking for a code review :) But a general comment on the approach of contextualizing the Http404 exception is most welcome. I had the idea today and once I started implementing it, I saw a really nice pattern emerging.

[humitos]

I love where are we going with this. 404 pages will be way better with these changes 💯

[benjaoming]

We are missing lots of tests :)

Going over the tests now, hadn't realized they were failing, was only testing on a small subset locally.

[benjaoming]

Nice - the Django test client seems to have an option raise_view_exception that can test exception handling: https://code.djangoproject.com/ticket/18707#comment:14

[benjaoming]

@stsewd ready for review - I added a POC of using raise_view_exception without adding a new bunch of test cases, but I think it's nice.

[stsewd]

Much simpler implementation 👍

Left some general suggestions

[benjaoming]

Thanks for the great and detailed review @stsewd 💯

Will do some manual testing and get back ⏳

[benjaoming]

@stsewd changes work locally, the failure in "checks" is unrelated.

[stsewd]

There is some information that will (should) always be available to templates, other than that I'm fine moving forward with this.

[benjaoming]

Super happy to move forwards on this, thanks for all the feedback @stsewd 👍

I hope that anyone encountering a 404 in the future will quickly pitch in with improvements to texts and suggestions.

The new contents haven't been worked much with in the PR (although we did add some significant helpful texts and functions, as mentioned in the PR description). But it's great that we have now introduced a pattern that enables better 404 error handling with the option of using a targeted context, rather than unhelpfully broad messages.