Merge tag '5.12.0' into rel

Author: ericholscher

CHANGELOG.rst

@@ -1,3 +1,41 @@
+Version 5.12.0
+--------------
+
+:Date: March 08, 2021
+
+* `@pyup-bot <https://github.com/pyup-bot>`__: pyup:  Scheduled weekly dependency update for week 10 (`#7995 <https://github.com/readthedocs/readthedocs.org/pull/7995>`__)
+* `@saadmk11 <https://github.com/saadmk11>`__: Remove json field from RemoteRepositoryRelation and RemoteOrganizationRelation model (`#7993 <https://github.com/readthedocs/readthedocs.org/pull/7993>`__)
+* `@humitos <https://github.com/humitos>`__: Use independent Docker image to build assets (`#7992 <https://github.com/readthedocs/readthedocs.org/pull/7992>`__)
+* `@Pradhvan <https://github.com/Pradhvan>`__: Fixes typo in getting-started-with-sphinx: (`#7991 <https://github.com/readthedocs/readthedocs.org/pull/7991>`__)
+* `@stsewd <https://github.com/stsewd>`__: Search: use doctype from indexed pages instead of the db (`#7984 <https://github.com/readthedocs/readthedocs.org/pull/7984>`__)
+* `@humitos <https://github.com/humitos>`__: Allow `donate` app to use Stripe Checkout for one-time donations (`#7983 <https://github.com/readthedocs/readthedocs.org/pull/7983>`__)
+* `@humitos <https://github.com/humitos>`__: Update development/standards guide (`#7981 <https://github.com/readthedocs/readthedocs.org/pull/7981>`__)
+* `@stsewd <https://github.com/stsewd>`__: Docs: update expand_tabs to work with the latest version of sphinx-tabs (`#7979 <https://github.com/readthedocs/readthedocs.org/pull/7979>`__)
+* `@ericholscher <https://github.com/ericholscher>`__: Fix build routing (`#7978 <https://github.com/readthedocs/readthedocs.org/pull/7978>`__)
+* `@stsewd <https://github.com/stsewd>`__: Builds: register tasks to delete inactive external versions (`#7975 <https://github.com/readthedocs/readthedocs.org/pull/7975>`__)
+* `@stsewd <https://github.com/stsewd>`__: Embed: fix join (`#7974 <https://github.com/readthedocs/readthedocs.org/pull/7974>`__)
+* `@stsewd <https://github.com/stsewd>`__: Embed: change proxied urls (`#7973 <https://github.com/readthedocs/readthedocs.org/pull/7973>`__)
+* `@ericholscher <https://github.com/ericholscher>`__: refactor footer, add jobs & status page (`#7970 <https://github.com/readthedocs/readthedocs.org/pull/7970>`__)
+* `@stsewd <https://github.com/stsewd>`__: Sphinx domain: remove API (`#7969 <https://github.com/readthedocs/readthedocs.org/pull/7969>`__)
+* `@humitos <https://github.com/humitos>`__: Upgrade `postgres-client` to v12 in Docker image (`#7967 <https://github.com/readthedocs/readthedocs.org/pull/7967>`__)
+* `@saadmk11 <https://github.com/saadmk11>`__: Add management command to Load Project and RemoteRepository Relationship from JSON file (`#7966 <https://github.com/readthedocs/readthedocs.org/pull/7966>`__)
+* `@astrojuanlu <https://github.com/astrojuanlu>`__: Update guide on conda support (`#7965 <https://github.com/readthedocs/readthedocs.org/pull/7965>`__)
+* `@stsewd <https://github.com/stsewd>`__: Embed: add more tests (`#7962 <https://github.com/readthedocs/readthedocs.org/pull/7962>`__)
+* `@humitos <https://github.com/humitos>`__: Lower rank of development/install.html (`#7960 <https://github.com/readthedocs/readthedocs.org/pull/7960>`__)
+* `@stsewd <https://github.com/stsewd>`__: Embed: refactor view (`#7955 <https://github.com/readthedocs/readthedocs.org/pull/7955>`__)
+* `@stsewd <https://github.com/stsewd>`__: Search: make --queue required for management command (`#7952 <https://github.com/readthedocs/readthedocs.org/pull/7952>`__)
+* `@ericholscher <https://github.com/ericholscher>`__: Add proxito healthcheck (`#7948 <https://github.com/readthedocs/readthedocs.org/pull/7948>`__)
+* `@Pradhvan <https://github.com/Pradhvan>`__: Docs: Adds Myst to the getting started with sphinx (`#7938 <https://github.com/readthedocs/readthedocs.org/pull/7938>`__)
+* `@ericholscher <https://github.com/ericholscher>`__: Add a support form to the website (`#7929 <https://github.com/readthedocs/readthedocs.org/pull/7929>`__)
+* `@humitos <https://github.com/humitos>`__: Use Stripe Checkout for Gold Users (`#7889 <https://github.com/readthedocs/readthedocs.org/pull/7889>`__)
+* `@stsewd <https://github.com/stsewd>`__: Docs: guide about reproducible builds (`#7888 <https://github.com/readthedocs/readthedocs.org/pull/7888>`__)
+* `@stsewd <https://github.com/stsewd>`__: Docs: update links from build images (`#7886 <https://github.com/readthedocs/readthedocs.org/pull/7886>`__)
+* `@stsewd <https://github.com/stsewd>`__: Install latest mkdocs by default as we do with sphinx (`#7869 <https://github.com/readthedocs/readthedocs.org/pull/7869>`__)
+* `@stsewd <https://github.com/stsewd>`__: Docs: document analytics (`#7857 <https://github.com/readthedocs/readthedocs.org/pull/7857>`__)
+* `@stsewd <https://github.com/stsewd>`__: Remove some feature flags (`#7846 <https://github.com/readthedocs/readthedocs.org/pull/7846>`__)
+* `@stsewd <https://github.com/stsewd>`__: Update requirements/deploy.txt (`#7843 <https://github.com/readthedocs/readthedocs.org/pull/7843>`__)
+* `@humitos <https://github.com/humitos>`__: Implementation of APIv3 (`#4863 <https://github.com/readthedocs/readthedocs.org/pull/4863>`__)
+
 Version 5.11.0
 --------------
 

common

@@ -20,14 +20,23 @@ RUN apt-get -y install \
         build-essential \
         python3-pip \
         python3-dev \
-        postgresql-client \
         libmysqlclient-dev \
         libfreetype6 \
         libjpeg-dev \
         sqlite \
         netcat \
         telnet
 
+# https://www.postgresql.org/download/linux/ubuntu/
+# Use PostgreSQL client 12 from official PostgreSQL Ubuntu's repository to avoid this WARNING
+# WARNING: psql major version 10, server major version 12.
+# Some psql features might not work.
+# TODO: remove this when upgrading to Ubuntu 20.04 LTS
+RUN sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
+RUN curl -s https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
+RUN apt-get -y update
+RUN apt-get -y install postgresql-client-12
+
 # Uncomment en_US.UTF-8 locale and generate it
 RUN sed -i -e 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen && \
     locale-gen

dockerfiles/Dockerfile

@@ -0,0 +1,8 @@
+version: '3'
+
+services:
+  assets:
+    image: node:8.16
+    volumes:
+      - ${PWD}:/usr/src/app/readthedocs.org
+    working_dir: /usr/src/app/readthedocs.org

dockerfiles/docker-compose-assets.yml

@@ -2,7 +2,7 @@
 #
 
 # You can set these variables from the command line.
-SPHINXOPTS    = -W
+SPHINXOPTS    =
 SPHINXBUILD   = sphinx-build
 PAPER         =
 BUILDDIR      = _build
@@ -53,7 +53,7 @@ clean:
 	rm -rf $(BUILDDIR)/*
 
 auto:
-	sphinx-autobuild -p 8888 $(ALLSPHINXOPTS) $(BUILDDIR)/html
+	sphinx-autobuild --port 8888 $(ALLSPHINXOPTS) $(BUILDDIR)/html
 	@echo
 	@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
 
@@ -63,7 +63,7 @@ html:
 	@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
 
 livehtml:
-	sphinx-autobuild -p 4444 -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
+	sphinx-autobuild --port 4444 -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
 
 
 dirhtml:

docs/Makefile

@@ -9,7 +9,7 @@ $( document ).ready(function() {
   const urlParams = new URLSearchParams(window.location.search);
   const tabName = urlParams.get('tab');
   if (tabName !== null) {
-    const tab = $('a.item > div:contains("' + tabName + '")');
+    const tab = $('button.sphinx-tabs-tab:contains("' + tabName + '")');
     if (tab.length > 0) {
       tab.click();
     }

docs/_static/images/traffic-analytics-demo.png

@@ -0,0 +1,24 @@
+Analytics
+---------
+
+Analytics lets you see *which* documents your users are reading.
+This allows you to understand how your documentation is being used,
+so you can focus on expanding and updating parts people are reading most.
+
+To see a list of the top pages from the last month,
+go to the :guilabel:`Admin` tab of your project,
+and then click on :guilabel:`Traffic Analytics`.
+
+.. figure:: /_static/images/traffic-analytics-demo.png
+   :width: 50%
+   :align: center
+   :alt: Traffic analytics demo
+
+   Traffic analytics demo
+
+You can also access to analytics data from :ref:`search results <server-side-search:analytics>`.
+
+.. note::
+
+   If you require more information about page views,
+   you can use a solution like :doc:`Google Analytics </guides/google-analytics>`.

docs/_static/js/expand_tabs.js

@@ -67,9 +67,7 @@ An example in code:
 .. note::
 
     Regardless of whether you build your docs with Sphinx or MkDocs,
-    we recommend you pin the version of Sphinx or Mkdocs you want us to use.
-    You can do this the same way other
-    :doc:`dependencies are specified <guides/specifying-dependencies>`.
+    we recommend you :ref:`pinning the version <guides/reproducible-builds:pinning dependencies>` of Sphinx or Mkdocs you want us to use.
     Some examples of pinning versions might be ``sphinx<2.0`` or ``mkdocs>=1.0``.
 
 Build environment

docs/analytics.rst

@@ -36,4 +36,3 @@ Advertising-free
    single-sign-on
    sharing
    privacy-level
-   analytics

docs/builds.rst

@@ -319,8 +319,8 @@ The Docker image used for building the docs.
 Each image support different Python versions and has different packages installed,
 as defined here:
 
-* `stable <https://github.com/readthedocs/readthedocs-docker-images/tree/releases/4.x>`_: :buildpyversions:`stable`
-* `latest <https://github.com/readthedocs/readthedocs-docker-images/tree/releases/5.x>`_: :buildpyversions:`latest`
+* `stable <https://github.com/readthedocs/readthedocs-docker-images/tree/releases/5.x>`_: :buildpyversions:`stable`
+* `latest <https://github.com/readthedocs/readthedocs-docker-images/tree/releases/6.x>`_: :buildpyversions:`latest`
 
 sphinx
 ~~~~~~

docs/commercial/analytics.rst

@@ -55,43 +55,22 @@ in.
 Getting Started
 ---------------
 
-You will need a working version of Node (tested with ``v10.17.0``) and NPM to get started.
-We won't cover that here, as it varies from platform to platform.
-
-To install these tools and dependencies::
-
-    npm install
-
-Next, install front end dependencies::
-
-    bower install
+You will need to follow our :doc:`guide to install a development Read the Docs instance </development/standards>` first.
 
 The sources for our bundles are found in the per-application path
 ``static-src``, which has the same directory structure as ``static``. Files in
 ``static-src`` are compiled to ``static`` for static file collection in Django.
 Don't edit files in ``static`` directly, unless you are sure there isn't a
 source file that will compile over your changes.
 
-To test changes while developing, which will watch source files for changes and
-compile as necessary, you can run `Gulp`_ with our development target::
-
-    npm run dev
+To compile your changes and make them available in the application you need to run::
 
-Once you are satisfied with your changes, finalize the bundles (this will
-minify library sources)::
+    inv docker.buildassets
 
-    npm run build
-
-If you updated any of our vendor libraries, compile those::
-
-    npm run vendor
-
-Make sure to check in both files under ``static`` and ``static-src``.
-
-.. note::
+Once you are happy with your changes,
+make sure to check in both files under ``static`` and ``static-src``,
+and commit those.
 
-    We run Gulp  through an ``npm`` script in order to ensure
-    that the correct version from ``package.json`` is used.
 
 Making Changes
 --------------

docs/commercial/index.rst

@@ -13,9 +13,6 @@ Core team members expect to have a development environment that closely
 approximates our production environment, in order to spot bugs and logical
 inconsistencies before they make their way to production.
 
-Currently, core team members are migrating to a Docker Compose based
-solution that is not yet recommended for contributing to development.
-
 This solution gives us many features that allows us to have an
 environment closer to production:
 
@@ -73,6 +70,12 @@ After cloning ``readthedocs.org`` repository, you need to
 
 #. install `Docker <https://www.docker.com/>`_ following `their installation guide <https://docs.docker.com/install/>`_.
 
+#. clone the ``readthedocs.org`` repository:
+
+   .. prompt:: bash
+
+      git clone --recurse-submodules https://github.com/readthedocs/readthedocs.org/
+
 #. install the requirements from ``common`` submodule:
 
    .. prompt:: bash
@@ -158,6 +161,10 @@ save some work while typing docker compose commands. This section explains these
        you can run ``inv docker.attach web`` and jump into a pdb session
        (it also works with ipdb and pdb++)
 
+    .. tip::
+
+       You can hit CTRL-p CTRL-p to detach it without stopping the running process.
+
 ``inv docker.test``
     Runs all the test suites inside the container.
 
@@ -168,10 +175,13 @@ save some work while typing docker compose commands. This section explains these
 
     * ``--only-latest`` does not pull ``stable`` and ``testing`` images.
 
+``inv docker.buildassets``
+    Build all the assets and "deploy" them to the storage.
+
 Adding a new Python dependency
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-The Docker image for the servers is built with the requirements defined in the ``master`` branch.
+The Docker image for the servers is built with the requirements defined in the current checked out branch.
 In case you need to add a new Python dependency while developing,
 you can use the ``common/dockerfiles/entrypoints/common.sh`` script as shortcut.
 
@@ -184,7 +194,7 @@ To do this, add the ``pip`` command required for your dependency in ``common.sh`
    # common.sh
    pip install my-dependency==1.2.3
 
-Once the PR that adds this dependency was merged into ``master``, you can rebuild the image
+Once the PR that adds this dependency was merged, you can rebuild the image
 so the dependency is added to the Docker image itself and it's not needed to be installed
 each time the container spins up.
 

docs/config-file/v2.rst

@@ -33,7 +33,7 @@ My documentation requires additional dependencies
 -------------------------------------------------
 
 For most Python dependencies, you can can specify a requirements file
-which details your dependencies. See our guide on :doc:`/guides/specifying-dependencies`.
+which details your dependencies. See our guide on :ref:`guides/reproducible-builds:using a configuration file`.
 You can also set your project documentation to install your project itself
 as a dependency.
 
@@ -216,8 +216,7 @@ and as a result, it tends to look a bit better with the default theme.
 .. note::
 
    To use these extensions you need to specify the dependencies on your project
-   by following this :doc:`guide <guides/specifying-dependencies>`.
-
+   by following this :ref:`guide <guides/reproducible-builds:using a configuration file>`.
 
 Can I document a python package that is not at the root of my repository?
 -------------------------------------------------------------------------