Merge tag '7.6.1' into rel

Author: agjohnson

CHANGELOG.rst

@@ -1,3 +1,13 @@
+Version 7.6.1
+-------------
+
+:Date: April 19, 2022
+
+* `@humitos <https://github.com/humitos>`__: Logging: reduce verbosity (`#9107 <https://github.com/readthedocs/readthedocs.org/pull/9107>`__)
+* `@stsewd <https://github.com/stsewd>`__: Analytics: Don't use full_path in get_or_create (`#9099 <https://github.com/readthedocs/readthedocs.org/pull/9099>`__)
+* `@humitos <https://github.com/humitos>`__: Build: do not upload `build.tool` to production S3 (`#9098 <https://github.com/readthedocs/readthedocs.org/pull/9098>`__)
+* `@ericholscher <https://github.com/ericholscher>`__: Cleanup Redirects docs (`#9095 <https://github.com/readthedocs/readthedocs.org/pull/9095>`__)
+
 Version 7.6.0
 -------------
 

docs/conf.py

@@ -67,7 +67,7 @@
 
 master_doc = "index"
 copyright = "2010, Read the Docs, Inc & contributors"
-version = "7.6.0"
+version = "7.6.1"
 release = version
 exclude_patterns = ["_build"]
 default_role = "obj"

docs/user/user-defined-redirects.rst

@@ -1,7 +1,7 @@
 User-defined Redirects
 ======================
 
-You can set up redirects for a project in your project dashboard's Redirects page.
+You can set up redirects for a project in your project dashboard's :guilabel:`Redirects` page.
 
 Quick Summary
 -------------
@@ -16,14 +16,21 @@ Quick Summary
 
 Your redirects will be effective immediately.
 
-.. note::
+Limitations
+~~~~~~~~~~~
+
+Redirects are only implemented in case of a *404 File Not Found* error.
+If you need to redirect a large number of files that still exist,
+please reach out to :doc:`/support`.
 
-    For the time being, redirects are only implemented in case of a
-    *404 File Not Found* error.
+Page & Exact Redirects can redirect to URLs outside Read the Docs.
+Define the `To URL` as the absolute URL you want to redirect to.
 
 Redirect Types
 --------------
 
+We offer a few different type of redirects based on what you want to do.
+
 Prefix Redirects
 ~~~~~~~~~~~~~~~~
 
@@ -50,12 +57,10 @@ Your users query would now redirect in the following manner::
 
 Where ``en`` and ``latest`` are the default language and version values for your project.
 
-
 .. note::
 
-   In other words, a *Prefix Redirect* removes a prefix from the original URL.
-   This prefix is removed from the rest of the URL's ``path`` after ``/$lang/$version``.
-   For example, if the URL is ``/es/1.0/guides/tutorial/install.html`` the "From URL's prefix" will be removed from ``/guides/tutorial/install.html`` part.
+   If you were hosting your docs without a prefix, you can create a ``/`` Prefix Redirect,
+   which will prepend ``/$lang/$version/`` to all incoming URLs.
 
 
 Page Redirects
@@ -73,23 +78,18 @@ You would set the following configuration::
     From URL: /example.html
     To URL: /examples/intro.html
 
-Note that the ``/`` at the start doesn't count the ``/$lang/$version`` prefix (e.g.
-``/en/latest``), but just the user-controlled section of the URL.
+**Page Redirects apply to all versions of you documentation.**
+Because of this,
+the ``/`` at the start of the ``From URL`` doesn't include the ``/$lang/$version`` prefix (e.g.
+``/en/latest``), but just the version-specific part of the URL.
 If you want to set directs only for some languages or some versions, you should use
 :ref:`user-defined-redirects:exact redirects` with the fully-specified path.
 
-.. tip::
-
-   *Page Redirects* can redirect URLs **outside** Read the Docs platform
-   just by defining the "To URL" as the absolute URL you want to redirect to.
-
-
 Exact Redirects
 ~~~~~~~~~~~~~~~
 
-If you're redirecting from an old host AND you aren't maintaining old paths for your
-documents, a Prefix Redirect won't suffice and you'll need to create *Exact Redirects*
-to redirect from a specific URL, to a specific page.
+*Exact Redirects* are for redirecting a single URL,
+and take into account the full URL (including language & version).
 
 Say you're moving ``docs.example.com`` to Read the Docs and want to redirect traffic
 from an old page at ``https://docs.example.com/dev/install.html`` to a new URL
@@ -128,11 +128,6 @@ Similarly, if you maintain several branches of your documentation (e.g. ``3.0``
 ``latest``) and decide to move pages in ``latest`` but not the older branches, you can use
 *Exact Redirects* to do so.
 
-.. tip::
-
-   *Exact Redirects* can redirect URLs **outside** Read the Docs platform
-   just by defining the "To URL" as the absolute URL you want to redirect to.
-
 
 Sphinx Redirects
 ~~~~~~~~~~~~~~~~

package.json

@@ -1,6 +1,6 @@
 {
   "name": "readthedocs",
-  "version": "7.6.0",
+  "version": "7.6.1",
   "description": "Read the Docs build dependencies",
   "author": "Read the Docs, Inc <[email protected]>",
   "scripts": {

readthedocs/__init__.py

@@ -1,4 +1,4 @@
 """Read the Docs."""
 
 
-__version__ = "7.6.0"
+__version__ = "7.6.1"

readthedocs/doc_builder/environments.py

@@ -646,7 +646,10 @@ def __exit__(self, exc_type, exc_value, tb):
                 container_id=self.container_id,
             )
         except DockerAPIError:
-            log.exception(
+            # Logging this as warning because it usually happens due memory
+            # limit or build timeout. In those cases, the container is not
+            # running and can't be killed
+            log.warning(
                 'Unable to kill container.',
                 container_id=self.container_id,
             )

readthedocs/projects/tasks/builds.py

@@ -136,7 +136,6 @@ def on_failure(self, exc, task_id, args, kwargs, einfo):
         if isinstance(exc, RepositoryError):
             log.warning(
                 'There was an error with the repository.',
-                exc_info=True,
             )
         elif isinstance(exc, SyncRepositoryLocked):
             log.warning(

scripts/compile_version_upload_s3.sh

@@ -23,52 +23,16 @@
 # PRODUCTION ENVIRONMENT
 #
 # To create a pre-compiled cached version and make it available on production,
-# **the script must be ran from a builder (build-default or build-large)** and
-# it's required to set several environment variables for an IAM user with
-# permissions on ``readthedocs(inc)-build-tools-prod`` S3's bucket. Also, note
-# that in production we need to install `aws` Python package to run the script.
-# We can do this in a different virtualenv to avoid collision with the builder's
-# code.
-#
-# The whole process would be something like:
-#
-#   ssh util01
-#   ssh `scaling status -s build-default -q | head -n 1`
-#
-#   sudo su - docs
-#   TOOL=python
-#   VERSION=3.10.0
-#
-#   cd /home/docs/checkouts/readthedocs.org/scripts
-#   virtualenv venv
-#   source venv/bin/activate
-#   pip install awscli==1.20.34
-#
-#   export AWS_REGION=...
-#   export AWS_ACCESS_KEY_ID=...
-#   export AWS_SECRET_ACCESS_KEY=...
-#   export AWS_BUILD_TOOLS_BUCKET=readthedocs(inc)-build-tools-prod
-#
-#   ./compile_version_upload.sh $TOOL $VERSION
-#
-#
-# ONE-LINE COMMAND FROM UTIL01 PRODUCTION
-#
-#   TOOL=python
-#   VERSION=3.10.0
-#   AWS_BUILD_TOOLS_BUCKET=readthedocs(inc)-build-tools-prod
-#
-#   ssh `scaling status -s build-default -q | head -n 1` \
-#     "cd /home/docs && \
-#     sudo -u docs virtualenv --python python3 /home/docs/buildtools && \
-#     sudo -u docs /home/docs/buildtools/bin/pip install awscli==1.20.34 && \
-#     sudo -u docs env AWS_REGION=$AWS_REGION \
-#       AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID \
-#       AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY \
-#       AWS_BUILD_TOOLS_BUCKET=$AWS_BUILD_TOOLS_BUCKET \
-#       PATH=/home/docs/buildtools/bin:${PATH} \
-#       /home/docs/checkouts/readthedocs.org/scripts/compile_version_upload_s3.sh $TOOL $VERSION"
+# we use a CircleCI job
+# (https://github.com/readthedocs/readthedocs-docker-images/blob/main/.circleci/config.yml)
+# It requires to set several environment variables for an IAM user with
+# permissions on ``readthedocs(inc)-build-tools-prod`` S3's bucket. These
+# variables are defined via the CircleCI UI under the `readthedocs-docker-image`
+# project.
 #
+# Note that if for some reason, you need to run this command *outside CircleCI*
+# you can find more information in this comment:
+# https://github.com/readthedocs/readthedocs-ops/issues/1155#issuecomment-1082615972
 #
 # USAGE
 #
@@ -90,6 +54,7 @@ set -x # Echo commands
 # Define variables
 SLEEP=350 # Container timeout
 OS="${OS:-ubuntu-22.04}" # Docker image name
+
 TOOL=$1
 VERSION=$2
 
@@ -103,7 +68,16 @@ echo "Running all the commands in Docker container: $CONTAINER_ID"
 # Install the tool version requested
 if [[ $TOOL == "python" ]]
 then
-    docker exec --env PYTHON_CONFIGURE_OPTS="--enable-shared" $CONTAINER_ID asdf install $TOOL $VERSION
+    if [[ $VERSION == "3.6.15" ]]
+    then
+        # Special command for Python 3.6.15
+        # See https://github.com/pyenv/pyenv/issues/1889#issuecomment-833587851
+        docker exec --user root $CONTAINER_ID apt-get update
+        docker exec --user root $CONTAINER_ID apt-get install --yes clang
+        docker exec --env PYTHON_CONFIGURE_OPTS="--enable-shared" --env CC=clang $CONTAINER_ID asdf install $TOOL $VERSION
+    else
+        docker exec --env PYTHON_CONFIGURE_OPTS="--enable-shared" $CONTAINER_ID asdf install $TOOL $VERSION
+    fi
 else
     docker exec $CONTAINER_ID asdf install $TOOL $VERSION
 fi
@@ -113,6 +87,10 @@ docker exec $CONTAINER_ID asdf global $TOOL $VERSION
 docker exec $CONTAINER_ID asdf reshim $TOOL
 
 # Install dependencies for this version
+#
+# TODO: pin all transitive dependencies with pip-tools or similar. We can find
+# the current versions by running `pip freeze` in production and stick with them
+# for now to avoid changing versions.
 if [[ $TOOL == "python" ]] && [[ ! $VERSION =~ (^miniconda.*|^mambaforge.*) ]]
 then
     RTD_PIP_VERSION=21.2.4
@@ -138,22 +116,21 @@ docker cp $CONTAINER_ID:/home/docs/$OS-$TOOL-$VERSION.tar.gz .
 # Kill the container
 docker container kill $CONTAINER_ID
 
-# Upload the .tar.gz to S3
-AWS_ENDPOINT_URL="${AWS_ENDPOINT_URL:-http://localhost:9000}"
-AWS_BUILD_TOOLS_BUCKET="${AWS_BUILD_TOOLS_BUCKET:-build-tools}"
-AWS_ACCESS_KEY_ID="${AWS_ACCESS_KEY_ID:-admin}"
-AWS_SECRET_ACCESS_KEY="${AWS_SECRET_ACCESS_KEY:-password}"
-
-if [[ -z $AWS_REGION ]]
+if [[ -z $CIRCLECI ]]
 then
-    # Development environment
+    # Upload the .tar.gz to S3 development environment
     echo "Uploading to dev environment"
+
+    AWS_ENDPOINT_URL="${AWS_ENDPOINT_URL:-http://localhost:9000}"
+    AWS_BUILD_TOOLS_BUCKET="${AWS_BUILD_TOOLS_BUCKET:-build-tools}"
+    AWS_ACCESS_KEY_ID="${AWS_ACCESS_KEY_ID:-admin}"
+    AWS_SECRET_ACCESS_KEY="${AWS_SECRET_ACCESS_KEY:-password}"
+
     aws --endpoint-url $AWS_ENDPOINT_URL s3 cp $OS-$TOOL-$VERSION.tar.gz s3://$AWS_BUILD_TOOLS_BUCKET
+
+    # Delete the .tar.gz file from the host
+    rm $OS-$TOOL-$VERSION.tar.gz
 else
-    # Production environment does not requires `--endpoint-url`
-    echo "Uploading to prod environment"
-    aws s3 cp $OS-$TOOL-$VERSION.tar.gz s3://$AWS_BUILD_TOOLS_BUCKET
+    echo "Skip uploading .tar.gz file because it's being run from inside CircleCI."
+    echo "It should be uploaded by orbs/aws automatically."
 fi
-
-# Delete the .tar.gz file from the host
-rm $OS-$TOOL-$VERSION.tar.gz

setup.cfg

@@ -29,7 +29,7 @@ github_owner = readthedocs
 github_repo = readthedocs.org
 
 [bumpver]
-current_version = "7.6.0"
+current_version = "7.6.1"
 version_pattern = "MAJOR.MINOR.PATCH[TAGNUM]"
 commit_message = "Bump version {old_version} -> {new_version}"
 commit = False