Merge tag '5.4.1' into rel

Author: humitos

CHANGELOG.rst

@@ -1,3 +1,26 @@
+Version 5.4.1
+-------------
+
+:Date: September 01, 2020
+
+* `@stsewd <https://github.com/stsewd>`__: Docs: update readthedcos-sphinx-search ext (`#7427 <https://github.com/readthedocs/readthedocs.org/pull/7427>`__)
+* `@humitos <https://github.com/humitos>`__: Upgrade Django to 2.2.16 (`#7426 <https://github.com/readthedocs/readthedocs.org/pull/7426>`__)
+* `@bmorrison4 <https://github.com/bmorrison4>`__: Fix typo in docs/guides/adding-custom-css.rst (`#7424 <https://github.com/readthedocs/readthedocs.org/pull/7424>`__)
+* `@stsewd <https://github.com/stsewd>`__: Test: set privacy level explicitly (`#7422 <https://github.com/readthedocs/readthedocs.org/pull/7422>`__)
+* `@stsewd <https://github.com/stsewd>`__: Pip: test new resolver (`#7412 <https://github.com/readthedocs/readthedocs.org/pull/7412>`__)
+* `@stsewd <https://github.com/stsewd>`__: Update common (`#7411 <https://github.com/readthedocs/readthedocs.org/pull/7411>`__)
+* `@stsewd <https://github.com/stsewd>`__: Release 5.4.0 (`#7410 <https://github.com/readthedocs/readthedocs.org/pull/7410>`__)
+* `@stsewd <https://github.com/stsewd>`__: Docker: install requirements from local changes (`#7409 <https://github.com/readthedocs/readthedocs.org/pull/7409>`__)
+* `@stsewd <https://github.com/stsewd>`__: ES: update dependencies (`#7408 <https://github.com/readthedocs/readthedocs.org/pull/7408>`__)
+* `@pyup-bot <https://github.com/pyup-bot>`__: pyup:  Scheduled weekly dependency update for week 34 (`#7406 <https://github.com/readthedocs/readthedocs.org/pull/7406>`__)
+* `@stsewd <https://github.com/stsewd>`__: API client: don't retry on POST (`#7383 <https://github.com/readthedocs/readthedocs.org/pull/7383>`__)
+* `@saadmk11 <https://github.com/saadmk11>`__: build_url added to all API v3 build endpoints (`#7373 <https://github.com/readthedocs/readthedocs.org/pull/7373>`__)
+* `@stsewd <https://github.com/stsewd>`__: Guide: deprecating content (`#7333 <https://github.com/readthedocs/readthedocs.org/pull/7333>`__)
+* `@humitos <https://github.com/humitos>`__: Auto-join email users field for Team model (`#7328 <https://github.com/readthedocs/readthedocs.org/pull/7328>`__)
+* `@stsewd <https://github.com/stsewd>`__: Guide: Cross-referencing with Sphinx (`#7326 <https://github.com/readthedocs/readthedocs.org/pull/7326>`__)
+* `@humitos <https://github.com/humitos>`__: Sync RemoteRepository and RemoteOrganization in all VCS providers (`#7310 <https://github.com/readthedocs/readthedocs.org/pull/7310>`__)
+* `@stsewd <https://github.com/stsewd>`__: Page views: use origin URL instead of page name (`#7293 <https://github.com/readthedocs/readthedocs.org/pull/7293>`__)
+
 Version 5.4.0
 -------------
 

common

@@ -39,8 +39,8 @@ RUN ln -s /usr/bin/python3 /usr/bin/python
 
 WORKDIR /tmp
 
-RUN curl -O https://raw.githubusercontent.com/readthedocs/readthedocs.org/master/requirements/pip.txt
-RUN curl -O https://raw.githubusercontent.com/readthedocs/readthedocs.org/master/requirements/docker.txt
+COPY requirements/pip.txt pip.txt
+COPY requirements/docker.txt docker.txt
 RUN pip3 install --no-cache-dir -r docker.txt
 
 # Install readthedocs-ext only if GITHUB_TOKEN is provided

dockerfiles/Dockerfile

@@ -42,7 +42,7 @@ A similar approach can be used to add JavaScript files::
 .. note::
 
     The Sphinx HTML options ``html_css_files`` and ``html_js_files``
-    where added in Sphinx 1.8.
+    were added in Sphinx 1.8.
     Unless you have a good reason to use an older version,
     you are strongly encouraged to upgrade.
     Sphinx is almost entirely backwards compatible.

docs/guides/adding-custom-css.rst

@@ -0,0 +1,100 @@
+Deprecating Content on Read the Docs
+====================================
+
+When you deprecate a feature from your project,
+you may want to deprecate its docs as well,
+and stop your users from reading that content.
+
+Deprecating content may sound as easy as delete it,
+but doing that will break existing links,
+and you don't necessary want to make the content inaccessible.
+Here you'll find some tips on how to use Read the Docs to deprecate your content
+progressively and in non harmful ways.
+
+Deprecating versions
+--------------------
+
+If you have multiple versions of your project,
+it makes sense to have its :doc:`documentation versioned </versions>` as well.
+For example, if you have the following versions and want to deprecate v1.
+
+- ``https://project.readthedocs.io/en/v1/``
+- ``https://project.readthedocs.io/en/v2/``
+- ``https://project.readthedocs.io/en/v3/``
+
+For cases like this you can *hide* a version.
+Hidden versions won't be listed in the versions menu of your docs,
+and they will be listed in a :ref:`robots.txt file <hosting:custom robots.txt pages>`
+to stop search engines of showing results for that version.
+
+Users can still see all versions in the dashboard of your project.
+To hide a version go to your project and click on :guilabel:`Versions` > :guilabel:`Edit`,
+and mark the :guilabel:`Hidden` option. Check :ref:`versions:Version States` for more information.
+
+.. note::
+
+   If the versions of your project follow the semver_ convention,
+   you can activate the :ref:`versions:version warning` option for your project.
+   A banner with a warning and linking to the stable version
+   will be shown on all versions that are lower than the stable one.
+
+   .. _semver: https://semver.org/
+
+Deprecating content
+-------------------
+
+You may not always want to deprecate a version, but deprecate some pages.
+For example, if you have documentation about two APIs and you want to deprecate v1:
+
+- ``https://project.readthedocs.io/en/latest/api/v1.html``
+- ``https://project.readthedocs.io/en/latest/api/v2.html``
+
+A simple way is just adding a warning at the top of the page,
+this will warn users visiting that page,
+but it won't stop users from being redirected to that page from search results.
+You can add an entry of that page in a :ref:`custom robots.txt <hosting:custom robots.txt pages>` file
+to avoid search engines of showing those results. For example::
+
+   # robots.txt
+
+   User-agent: *
+
+   Disallow: /en/latest/api/v1.html # Deprecated API
+
+But your users will still see search results from that page if they use the search from your docs.
+With Read the Docs you can set a :ref:`custom rank per pages <config-file/v2:search.ranking>`.
+For example:
+
+.. code-block:: yaml
+
+   # .readthedocs.yaml
+
+   version: 2
+   search:
+      ranking:
+         api/v1.html: -1
+
+This wont hide results from that page, but it will give priority to results from other pages.
+
+.. TODO: mention search.ignore when it's implemented.
+
+
+.. tip::
+
+   If you are using Sphinx with reStructuredText,
+   you can make use of some :doc:`directives <sphinx:usage/restructuredtext/directives>`
+   like ``warning``, ``deprecated``, ``versionchanged`` to warn your users about deprecated content.
+
+Moving and deleting pages
+-------------------------
+
+After you have deprecated a feature for a while,
+you may want to get rid of its documentation,
+that's OK, you don't have to maintain that content forever.
+But be aware that users may have links of that page saved,
+and it will be frustrating and confusing for them to get a 404.
+
+To solve that problem you can create a redirect to a page with a similar feature/content,
+like redirecting to the docs of the v2 of your API when your users visit the deleted docs from v1,
+this is a :ref:`page redirect <user-defined-redirects:page redirects>` from ``/api/v1.html`` to ``/api/v2.html``.
+See :doc:`/user-defined-redirects`.

docs/guides/deprecating-content.rst

@@ -4,19 +4,20 @@ Read the Docs Guides
 These guides will help you customize or tune aspects of Read the Docs.
 
 .. toctree::
-    :maxdepth: 1
+   :maxdepth: 1
 
-    autobuild-docs-for-pull-requests
-    build-notifications
-    build-using-too-many-resources
-    canonical
-    conda
-    environment-variables
-    feature-flags
-    google-analytics
-    hiding-a-version
-    searching-with-readthedocs
-    embedding-content
-    specifying-dependencies
-    technical-docs-seo-guide
-    wipe-environment
+   autobuild-docs-for-pull-requests
+   build-notifications
+   build-using-too-many-resources
+   canonical
+   conda
+   deprecating-content
+   environment-variables
+   feature-flags
+   google-analytics
+   hiding-a-version
+   searching-with-readthedocs
+   embedding-content
+   specifying-dependencies
+   technical-docs-seo-guide
+   wipe-environment

docs/guides/platform.rst

@@ -4,29 +4,41 @@
 from django.utils import timezone
 
 from readthedocs.api.v2.signals import footer_response
+from readthedocs.core.unresolver import unresolve_from_request
 from readthedocs.projects.models import Feature
 
 from .models import PageView
 
 
 @receiver(footer_response)
-def increase_page_view_count(sender, **kwargs):
+def increase_page_view_count(sender, *, request, context, absolute_uri, **kwargs):
     """Increase the page view count for the given project."""
-    del sender  # unused
-    context = kwargs['context']
+    # unused
+    del sender
+    del kwargs
 
     project = context['project']
     version = context['version']
-    # footer_response sends an empty path for the index
-    path = context['path'] or '/'
-
-    if project.has_feature(Feature.STORE_PAGEVIEWS):
-        page_view, _ = PageView.objects.get_or_create(
-            project=project,
-            version=version,
-            path=path,
-            date=timezone.now().date(),
-        )
-        PageView.objects.filter(pk=page_view.pk).update(
-            view_count=F('view_count') + 1
-        )
+
+    if not absolute_uri or not project.has_feature(Feature.STORE_PAGEVIEWS):
+        return
+
+    unresolved = unresolve_from_request(request, absolute_uri)
+    if not unresolved:
+        return
+
+    path = unresolved.filename
+
+    fields = dict(
+        project=project,
+        version=version,
+        path=path,
+        date=timezone.now().date(),
+    )
+
+    page_view = PageView.objects.filter(**fields).first()
+    if page_view:
+        page_view.view_count = F('view_count') + 1
+        page_view.save(update_fields=['view_count'])
+    else:
+        PageView.objects.create(**fields, view_count=1)