Redirects: allow to redirect even if a page exists (#9243)

Co-authored-by: Eric Holscher <[email protected]>

Author: stsewd

docs/user/user-defined-redirects.rst

@@ -3,35 +3,39 @@ User-defined Redirects
 
 You can set up redirects for a project in your project dashboard's :guilabel:`Redirects` page.
 
-Quick Summary
+.. contents:: Table of contents
+   :local:
+
+Quick summary
 -------------
 
-* Log into your readthedocs.org account.
-* From your dashboard, select the project on which you wish to add redirects.
-* From the project's top navigation bar, select the :guilabel:`Admin` tab.
+* Go to the :guilabel:`Admin` tab of your project.
 * From the left navigation menu, select :guilabel:`Redirects`.
-* In the form box "Redirect Type" select the type of redirect you want. See below for detail.
-* Depending on the redirect type you select, enter FROM and/or TO URL as needed.
+* In the form box "Redirect Type" select the type of redirect you want.
+  :ref:`See below <user-defined-redirects:redirect types>` for detail.
+* Depending on the redirect type you select, enter ``From URL`` and/or ``To URL`` as needed.
 * When finished, click the :guilabel:`Add` button.
 
 Your redirects will be effective immediately.
 
-Limitations
-~~~~~~~~~~~
-
-Redirects are only implemented in case of a *404 File Not Found* error.
-If you need to redirect a large number of files that still exist,
-please reach out to :doc:`/support`.
+Features
+--------
 
-Page & Exact Redirects can redirect to URLs outside Read the Docs.
-Define the `To URL` as the absolute URL you want to redirect to.
+- By default, redirects are followed only if the requested page doesn't exist
+  (*404 File Not Found* error), if you need to apply a redirect for files that exist,
+  mark the :guilabel:`Force redirect` option.
+  **This option is only available on some plan levels**.
+  Please ask support if you need it for some reason.
+- :ref:`user-defined-redirects:page redirects` and :ref:`user-defined-redirects:exact redirects`
+  can redirect to URLs outside Read the Docs,
+  just include the protocol in ``To URL``, e.g ``https://example.com``.
 
-Redirect Types
+Redirect types
 --------------
 
 We offer a few different type of redirects based on what you want to do.
 
-Prefix Redirects
+Prefix redirects
 ~~~~~~~~~~~~~~~~
 
 The most useful and requested feature of redirects was when migrating to Read the Docs from an old host.
@@ -63,7 +67,7 @@ Where ``en`` and ``latest`` are the default language and version values for your
    which will prepend ``/$lang/$version/`` to all incoming URLs.
 
 
-Page Redirects
+Page redirects
 ~~~~~~~~~~~~~~
 
 A more specific case is when you move a page around in your docs.
@@ -82,14 +86,23 @@ You would set the following configuration::
 Because of this,
 the ``/`` at the start of the ``From URL`` doesn't include the ``/$lang/$version`` prefix (e.g.
 ``/en/latest``), but just the version-specific part of the URL.
-If you want to set directs only for some languages or some versions, you should use
+If you want to set redirects only for some languages or some versions, you should use
 :ref:`user-defined-redirects:exact redirects` with the fully-specified path.
 
-Exact Redirects
+Exact redirects
 ~~~~~~~~~~~~~~~
 
 *Exact Redirects* are for redirecting a single URL,
-and take into account the full URL (including language & version).
+taking into account the full URL (including language and version).
+
+You can also redirect a subset of URLs by including the ``$rest`` keyword
+at the end of the ``From URL``.
+
+Exact redirects examples
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+Redirecting a single URL
+````````````````````````
 
 Say you're moving ``docs.example.com`` to Read the Docs and want to redirect traffic
 from an old page at ``https://docs.example.com/dev/install.html`` to a new URL
@@ -109,6 +122,9 @@ Your users query would now redirect in the following manner::
 Note that you should insert the desired language for "en" and version for "latest" to
 achieve the desired redirect.
 
+Redirecting a whole sub-path to a different one
+```````````````````````````````````````````````
+
 *Exact Redirects* could be also useful to redirect a whole sub-path to a different one by using a special ``$rest`` keyword in the "From URL".
 Let's say that you want to redirect your readers of your version ``2.0`` of your documentation under ``/en/2.0/`` because it's deprecated,
 to the newest ``3.0`` version of it at ``/en/3.0/``.
@@ -128,8 +144,22 @@ Similarly, if you maintain several branches of your documentation (e.g. ``3.0``
 ``latest``) and decide to move pages in ``latest`` but not the older branches, you can use
 *Exact Redirects* to do so.
 
+Migrating your documentation to another domain
+``````````````````````````````````````````````
+
+You can use an exact redirect to migrate your documentation to another domain,
+for example::
+
+  Type: Exact Redirect
+  From URL: /$rest
+  To URL: https://newdocs.example.com/
+  Force Redirect: True
+
+Then all pages will redirect to the new domain, for example
+``https://docs.example.com/en/latest/install.html`` will redirect to
+``https://newdocs.example.com/en/latest/install.html``.
 
-Sphinx Redirects
+Sphinx redirects
 ~~~~~~~~~~~~~~~~
 
 We also support redirects for changing the type of documentation Sphinx is building.

readthedocs/projects/forms.py

@@ -585,21 +585,30 @@ class RedirectForm(forms.ModelForm):
 
     class Meta:
         model = Redirect
-        fields = ['redirect_type', 'from_url', 'to_url']
+        fields = ["redirect_type", "from_url", "to_url", "force"]
 
     def __init__(self, *args, **kwargs):
         self.project = kwargs.pop('project', None)
         super().__init__(*args, **kwargs)
 
+        if self.project.has_feature(Feature.ALLOW_FORCED_REDIRECTS):
+            # Remove the nullable option from the form.
+            # TODO: remove after migration.
+            self.fields["force"].widget = forms.CheckboxInput()
+            self.fields["force"].empty_value = False
+        else:
+            self.fields.pop("force")
+
     def save(self, **_):  # pylint: disable=arguments-differ
         # TODO this should respect the unused argument `commit`. It's not clear
         # why this needs to be a call to `create`, instead of relying on the
         # super `save()` call.
         redirect = Redirect.objects.create(
             project=self.project,
-            redirect_type=self.cleaned_data['redirect_type'],
-            from_url=self.cleaned_data['from_url'],
-            to_url=self.cleaned_data['to_url'],
+            redirect_type=self.cleaned_data["redirect_type"],
+            from_url=self.cleaned_data["from_url"],
+            to_url=self.cleaned_data["to_url"],
+            force=self.cleaned_data.get("force", False),
         )
         return redirect
 

readthedocs/projects/models.py

@@ -1774,6 +1774,7 @@ def add_features(sender, **kwargs):
     CDN_ENABLED = "cdn_enabled"
     DOCKER_GVISOR_RUNTIME = "gvisor_runtime"
     RECORD_404_PAGE_VIEWS = "record_404_page_views"
+    ALLOW_FORCED_REDIRECTS = "allow_forced_redirects"
 
     # Versions sync related features
     SKIP_SYNC_TAGS = 'skip_sync_tags'
@@ -1865,6 +1866,10 @@ def add_features(sender, **kwargs):
             RECORD_404_PAGE_VIEWS,
             _("Record 404s page views."),
         ),
+        (
+            ALLOW_FORCED_REDIRECTS,
+            _("Allow forced redirects."),
+        ),
 
         # Versions sync related features
         (

readthedocs/proxito/tests/test_full.py

@@ -26,6 +26,7 @@
 )
 from readthedocs.projects.models import Domain, Feature, Project
 from readthedocs.proxito.views.mixins import ServeDocsMixin
+from readthedocs.redirects.models import Redirect
 from readthedocs.rtd_tests.storage import BuildMediaFileSystemStorageTest
 from readthedocs.subscriptions.models import Plan, PlanFeature, Subscription
 
@@ -1223,6 +1224,23 @@ def _test_cache_control_header_project(self, expected_value, host=None):
         self.assertEqual(resp.headers['CDN-Cache-Control'], 'private', url)
         self.assertNotIn('Cache-Tag', resp.headers, url)
 
+        # Forced redirects will be cached only if the version is public.
+        get(
+            Redirect,
+            project=self.project,
+            redirect_type="exact",
+            from_url="/en/latest/install.html",
+            to_url="/en/latest/tutorial/install.html",
+            force=True,
+        )
+        url = "/en/latest/install.html"
+        resp = self.client.get(url, secure=True, HTTP_HOST=host)
+        self.assertEqual(
+            resp["Location"], f"https://{host}/en/latest/tutorial/install.html", url
+        )
+        self.assertEqual(resp.headers["CDN-Cache-Control"], expected_value, url)
+        self.assertEqual(resp.headers["Cache-Tag"], "project,project:latest", url)
+
     def _test_cache_control_header_subproject(self, expected_value, host=None):
         """
         Test the CDN-Cache-Control header on requests for `self.subproject`.