Merge branch 'master' into humitos/do-not-list-banned-projects

Author: ericholscher

.github/mergeable.yml

@@ -0,0 +1,28 @@
+# ProBot Mergeable Bot
+# https://github.com/jusx/mergeable
+
+mergeable:
+  pull_requests:
+    approvals:
+      # Minimum of approvals needed.
+      min: 1
+        message: 'The PR must have a minimum of 1 approvals.'
+
+    description:
+      no_empty:
+        # Do not allow empty descriptions on PR.
+        enabled: false
+        message: 'Description can not be empty.'
+
+      must_exclude:
+        # Do not allow 'DO NOT MERGE' phrase on PR's description.
+        regex: 'DO NOT MERGE'
+        message: 'Description says that the PR should not be merged yet.'
+
+    # Do not allow 'WIP' on PR's title.
+    title: 'WIP'
+
+    label:
+      # Do not allow PR with label 'PR: work in progress'
+      must_exclude: 'PR: work in progress'
+      message: 'This PR is work in progress.'

.gitignore

@@ -7,6 +7,7 @@
 .DS_Store
 .cache
 .coverage
+.coverage.*
 .idea
 .vagrant
 .vscode
@@ -22,7 +23,7 @@ celerybeat-schedule.*
 deploy/.vagrant
 dist/*
 local_settings.py
-locks/*
+locks/**
 logs/*
 media/dash
 media/epub

.readthedocs.yml

@@ -3,4 +3,4 @@ formats: all
 sphinx:
   configuration: docs/conf.py
 python:
-  requirements: requirements.txt
+  requirements: requirements/local-docs-build.txt

.travis.yml

@@ -1,11 +1,10 @@
 language: python
 python:
-  - 2.7
   - 3.6
-env:
-  - ES_VERSION=1.3.9 ES_DOWNLOAD_URL=https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-${ES_VERSION}.tar.gz
 matrix:
   include:
+    - python: 3.6
+      env: TOXENV=py36 ES_VERSION=1.3.9 ES_DOWNLOAD_URL=https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-${ES_VERSION}.tar.gz
     - python: 3.6
       env: TOXENV=docs
     - python: 3.6
@@ -45,6 +44,6 @@ notifications:
 
 branches:
   only:
-  - master 
+  - master
   - rel # Community release branch
   - relcorp # Corporate release branch

CHANGELOG.rst

@@ -1,3 +1,56 @@
+Version 2.8.5
+-------------
+
+:Date: January 15, 2019
+
+* `@stsewd <http://github.com/stsewd>`__: Use the python path from virtualenv in Conda (`#5110 <https://github.com/rtfd/readthedocs.org/pull/5110>`__)
+* `@humitos <http://github.com/humitos>`__: Feature flag to use `readthedocs/build:testing` image (`#5109 <https://github.com/rtfd/readthedocs.org/pull/5109>`__)
+* `@stsewd <http://github.com/stsewd>`__: Use python from virtualenv's bin directory when executing commands (`#5107 <https://github.com/rtfd/readthedocs.org/pull/5107>`__)
+* `@humitos <http://github.com/humitos>`__: Do not build projects from banned users (`#5096 <https://github.com/rtfd/readthedocs.org/pull/5096>`__)
+* `@agjohnson <http://github.com/agjohnson>`__: Fix common pieces (`#5095 <https://github.com/rtfd/readthedocs.org/pull/5095>`__)
+* `@rainwoodman <http://github.com/rainwoodman>`__: Suppress progress bar of the conda command. (`#5094 <https://github.com/rtfd/readthedocs.org/pull/5094>`__)
+* `@humitos <http://github.com/humitos>`__: Remove unused suggestion block from 404 pages (`#5087 <https://github.com/rtfd/readthedocs.org/pull/5087>`__)
+* `@humitos <http://github.com/humitos>`__: Remove header nav (Login/Logout button) on 404 pages (`#5085 <https://github.com/rtfd/readthedocs.org/pull/5085>`__)
+* `@stsewd <http://github.com/stsewd>`__: Fix little typo (`#5084 <https://github.com/rtfd/readthedocs.org/pull/5084>`__)
+* `@agjohnson <http://github.com/agjohnson>`__: Split up deprecated view notification to GitHub and other webhook endpoints (`#5083 <https://github.com/rtfd/readthedocs.org/pull/5083>`__)
+* `@humitos <http://github.com/humitos>`__: Install ProBot (`#5082 <https://github.com/rtfd/readthedocs.org/pull/5082>`__)
+* `@stsewd <http://github.com/stsewd>`__: Update docs about contributing to docs (`#5077 <https://github.com/rtfd/readthedocs.org/pull/5077>`__)
+* `@humitos <http://github.com/humitos>`__: Declare and improve invoke tasks (`#5075 <https://github.com/rtfd/readthedocs.org/pull/5075>`__)
+* `@davidfischer <http://github.com/davidfischer>`__: Fire a signal for domain verification (eg. for SSL) (`#5071 <https://github.com/rtfd/readthedocs.org/pull/5071>`__)
+* `@agjohnson <http://github.com/agjohnson>`__: Update copy on notifications for github services deprecation (`#5067 <https://github.com/rtfd/readthedocs.org/pull/5067>`__)
+* `@humitos <http://github.com/humitos>`__: Upgrade all packages with pur (`#5059 <https://github.com/rtfd/readthedocs.org/pull/5059>`__)
+* `@dojutsu-user <http://github.com/dojutsu-user>`__: Reduce logging to sentry (`#5054 <https://github.com/rtfd/readthedocs.org/pull/5054>`__)
+* `@discdiver <http://github.com/discdiver>`__: fixed missing apostrophe for possessive "project's" (`#5052 <https://github.com/rtfd/readthedocs.org/pull/5052>`__)
+* `@dojutsu-user <http://github.com/dojutsu-user>`__: Template improvements in "gold/subscription_form.html" (`#5049 <https://github.com/rtfd/readthedocs.org/pull/5049>`__)
+* `@merwok <http://github.com/merwok>`__: Fix link in features page (`#5048 <https://github.com/rtfd/readthedocs.org/pull/5048>`__)
+* `@stsewd <http://github.com/stsewd>`__: Update webhook docs (`#5040 <https://github.com/rtfd/readthedocs.org/pull/5040>`__)
+* `@stsewd <http://github.com/stsewd>`__: Remove sphinx static and template dir (`#5039 <https://github.com/rtfd/readthedocs.org/pull/5039>`__)
+* `@stephenfin <http://github.com/stephenfin>`__: Add temporary method for disabling shallow cloning (#5031) (`#5036 <https://github.com/rtfd/readthedocs.org/pull/5036>`__)
+* `@stsewd <http://github.com/stsewd>`__: Raise exception in failed checkout (`#5035 <https://github.com/rtfd/readthedocs.org/pull/5035>`__)
+* `@dojutsu-user <http://github.com/dojutsu-user>`__: Change default_branch value from Version.slug to Version.identifier (`#5034 <https://github.com/rtfd/readthedocs.org/pull/5034>`__)
+* `@humitos <http://github.com/humitos>`__: Make wipe view not CSRF exempt (`#5025 <https://github.com/rtfd/readthedocs.org/pull/5025>`__)
+* `@humitos <http://github.com/humitos>`__: Convert an IRI path to URI before setting as NGINX header (`#5024 <https://github.com/rtfd/readthedocs.org/pull/5024>`__)
+* `@safwanrahman <http://github.com/safwanrahman>`__: index project asynchronously (`#5023 <https://github.com/rtfd/readthedocs.org/pull/5023>`__)
+* `@stsewd <http://github.com/stsewd>`__: Keep command output when it's killed (`#5015 <https://github.com/rtfd/readthedocs.org/pull/5015>`__)
+* `@stsewd <http://github.com/stsewd>`__: More hints for invalid submodules (`#5012 <https://github.com/rtfd/readthedocs.org/pull/5012>`__)
+* `@ericholscher <http://github.com/ericholscher>`__: Release 2.8.4 (`#5011 <https://github.com/rtfd/readthedocs.org/pull/5011>`__)
+* `@stsewd <http://github.com/stsewd>`__: Remove `auto` doctype (`#5010 <https://github.com/rtfd/readthedocs.org/pull/5010>`__)
+* `@davidfischer <http://github.com/davidfischer>`__: Tweak sidebar ad priority (`#5005 <https://github.com/rtfd/readthedocs.org/pull/5005>`__)
+* `@stsewd <http://github.com/stsewd>`__: Replace git status and git submodules status for gitpython (`#5002 <https://github.com/rtfd/readthedocs.org/pull/5002>`__)
+* `@davidfischer <http://github.com/davidfischer>`__: Backport jquery 2432 to Read the Docs (`#5001 <https://github.com/rtfd/readthedocs.org/pull/5001>`__)
+* `@stsewd <http://github.com/stsewd>`__: Refactor remove_dir (`#4994 <https://github.com/rtfd/readthedocs.org/pull/4994>`__)
+* `@humitos <http://github.com/humitos>`__: Skip builds when project is not active (`#4991 <https://github.com/rtfd/readthedocs.org/pull/4991>`__)
+* `@dojutsu-user <http://github.com/dojutsu-user>`__: Make $ unselectable in docs (`#4990 <https://github.com/rtfd/readthedocs.org/pull/4990>`__)
+* `@dojutsu-user <http://github.com/dojutsu-user>`__: Remove deprecated "models.permalink" (`#4975 <https://github.com/rtfd/readthedocs.org/pull/4975>`__)
+* `@dojutsu-user <http://github.com/dojutsu-user>`__: Add validation for tags of length greater than 100 characters (`#4967 <https://github.com/rtfd/readthedocs.org/pull/4967>`__)
+* `@dojutsu-user <http://github.com/dojutsu-user>`__: Add test case for send_notifications on VersionLockedError (`#4958 <https://github.com/rtfd/readthedocs.org/pull/4958>`__)
+* `@dojutsu-user <http://github.com/dojutsu-user>`__: Remove trailing slashes on svn checkout (`#4951 <https://github.com/rtfd/readthedocs.org/pull/4951>`__)
+* `@stsewd <http://github.com/stsewd>`__: Safe symlink on version deletion (`#4937 <https://github.com/rtfd/readthedocs.org/pull/4937>`__)
+* `@humitos <http://github.com/humitos>`__: CRUD for EnvironmentVariables from Project's admin (`#4899 <https://github.com/rtfd/readthedocs.org/pull/4899>`__)
+* `@humitos <http://github.com/humitos>`__: Notify users about the usage of deprecated webhooks (`#4898 <https://github.com/rtfd/readthedocs.org/pull/4898>`__)
+* `@dojutsu-user <http://github.com/dojutsu-user>`__: Disable django guardian warning (`#4892 <https://github.com/rtfd/readthedocs.org/pull/4892>`__)
+* `@humitos <http://github.com/humitos>`__: Handle 401, 403 and 404 status codes when hitting GitHub for webhook (`#4805 <https://github.com/rtfd/readthedocs.org/pull/4805>`__)
+
 Version 2.8.4
 -------------
 

LICENSE

@@ -1,4 +1,4 @@
-Copyright (c) 2010-2017 Read the Docs, Inc & contributors
+Copyright (c) 2010-2019 Read the Docs, Inc & contributors
 
 Permission is hereby granted, free of charge, to any person
 obtaining a copy of this software and associated documentation

README.rst

@@ -78,6 +78,6 @@ when you push to GitHub.
 License
 -------
 
-`MIT`_ © 2010-2017 Read the Docs, Inc & contributors
+`MIT`_ © 2010-2019 Read the Docs, Inc & contributors
 
 .. _MIT: LICENSE

common

@@ -225,3 +225,8 @@ The *Sphinx* and *Mkdocs* builders set the following RTD-specific environment va
 +-------------------------+--------------------------------------------------+----------------------+
 | ``READTHEDOCS_PROJECT`` | The RTD name of the project which is being built | ``myexampleproject`` |
 +-------------------------+--------------------------------------------------+----------------------+
+
+.. tip::
+
+   In case extra environment variables are needed to the build process (like secrets, tokens, etc),
+   you can add them going to **Admin > Environment Variables** in your project. See :doc:`guides/environment-variables`.

docs/builds.rst

@@ -6,7 +6,6 @@
 import sys
 
 import sphinx_rtd_theme
-from recommonmark.parser import CommonMarkParser
 
 sys.path.insert(0, os.path.abspath('..'))
 sys.path.append(os.path.dirname(__file__))
@@ -29,13 +28,11 @@
     'doc_extensions',
     'sphinx_tabs.tabs',
     'sphinx-prompt',
+    'recommonmark',
 ]
 templates_path = ['_templates']
 
 source_suffix = ['.rst', '.md']
-source_parsers = {
-    '.md': CommonMarkParser,
-}
 
 master_doc = 'index'
 project = u'Read the Docs'

docs/conf.py

@@ -230,3 +230,49 @@ What commit of Read the Docs is in production?
 ----------------------------------------------
 
 We deploy readthedocs.org from the `rel` branch in our GitHub repository. You can see the latest commits that have been deployed by looking on GitHub: https://github.com/rtfd/readthedocs.org/commits/rel
+
+
+How can I avoid search results having a deprecated version of my docs?
+----------------------------------------------------------------------
+
+If readers search something related to your docs in Google, it will probably return the most relevant version of your documentation.
+It may happen that this version is already deprecated and you want to stop Google indexing it as a result,
+and start suggesting the latest (or newer) one.
+
+To accomplish this, you can add a ``robots.txt`` file to your documentation's root so it ends up served at the root URL of your project
+(for example, https://yourproject.readthedocs.io/robots.txt).
+
+
+Minimal example of ``robots.txt``
++++++++++++++++++++++++++++++++++
+
+::
+
+   User-agent: *
+   Disallow: /en/deprecated-version/
+   Disallow: /en/2.0/
+
+.. note::
+
+   See `Google's docs`_ for its full syntax.
+
+This file has to be served as is under ``/robots.txt``.
+Depending if you are using Sphinx or MkDocs, you will need a different configuration for this.
+
+
+Sphinx
+~~~~~~
+
+Sphinx uses `html_extra`_ option to add static files to the output.
+You need to create a ``robots.txt`` file and put it under the path defined in ``html_extra``.
+
+
+MkDocs
+~~~~~~
+
+MkDocs needs the ``robots.txt`` to be at the directory defined at `docs_dir`_ config.
+
+
+.. _Google's docs: https://support.google.com/webmasters/answer/6062608
+.. _html_extra: https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-html_extra_path
+.. _docs_dir: https://www.mkdocs.org/user-guide/configuration/#docs_dir

docs/faq.rst

@@ -0,0 +1,37 @@
+I Need Secrets (or Environment Variables) in my Build
+=====================================================
+
+It may happen that your documentation depends on an authenticated service to be built properly.
+In this case, you will require some secrets to access these services.
+
+Read the Docs provides a way to define environment variables for your project to be used in the build process.
+All these variables will be exposed to all the commands executed when building your documentation.
+
+To define an environment variable, you need to
+
+#. Go to your project **Admin > Environment Variables**
+#. Click on "Add Environment Variable" button
+#. Input a ``Name`` and ``Value`` (your secret needed here)
+#. Click "Save" button
+
+.. note::
+
+   Values will never be exposed to users, even to owners of the project. Once you create an environment variable you won't be able to see its value anymore because of security purposes.
+
+After adding an environment variable from your project's admin, you can access it from your build process using Python, for example:
+
+.. code-block:: python
+
+   # conf.py
+   import os
+   import requests
+
+   # Access to our custom environment variables
+   username = os.environ.get('USERNAME')
+   password = os.environ.get('PASSWORD')
+
+   # Request a username/password protected URL
+   response = requests.get(
+       'https://httpbin.org/basic-auth/username/password',
+       auth=(username, password),
+   )

docs/guides/environment-variables.rst

@@ -35,7 +35,7 @@ Using the project admin dashboard
 Once the requirements file has been created;
 
 - Login to Read the Docs and go to the project admin dashboard.
-- Go to ``Admin > Advanced Settings > Requirements file``.
+- Go to **Admin > Advanced Settings > Requirements file**.
 - Specify the path of the requirements file you just created. The path should be relative to the root directory of the documentation project.
 
 Using a conda environment file

docs/guides/specifying-dependencies.rst

@@ -161,9 +161,9 @@ that your VCS provider is not configured correctly. If there is payload
 information on your Read the Docs project, you might need to verify that your
 versions are configured to build correctly.
 
-Either way, it may help to either resync your webhook intergration (see
+Either way, it may help to either resync your webhook integration (see
 `Resyncing webhooks`_ for information on this process), or set up an entirely
-new webhook intergration.
+new webhook integration.
 
 .. _webhook-github-services:
 
@@ -180,7 +180,7 @@ In order for your project to continue automatically building, you will need to
 configure your GitHub repository with a new webhook. You can use either a
 connected GitHub account and a :ref:`GitHub webhook integration <webhook-integration-github>`
 on your Read the Docs project, or you can use a
-:ref:`generic webhook integraiton <webhook-integration-generic>` without a connected
+:ref:`generic webhook integration <webhook-integration-generic>` without a connected
 account.
 
 .. [1] https://developer.github.com/changes/2018-04-25-github-services-deprecation/

docs/webhooks.rst

@@ -1,9 +1,10 @@
 # -*- coding: utf-8 -*-
+
 """Read the Docs."""
 
 import os.path
 
-from future.moves.configparser import RawConfigParser
+from configparser import RawConfigParser
 
 
 def get_version(setupcfg_path):

readthedocs/__init__.py

@@ -1,3 +1,5 @@
-"""App init"""
+# -*- coding: utf-8 -*-
 
-default_app_config = 'readthedocs.analytics.apps.AnalyticsAppConfig'    # noqa
+"""App init."""
+
+default_app_config = 'readthedocs.analytics.apps.AnalyticsAppConfig'  # noqa

readthedocs/analytics/__init__.py

@@ -1,12 +1,13 @@
+# -*- coding: utf-8 -*-
+
 """Django app config for the analytics app."""
 
-from __future__ import absolute_import
 from django.apps import AppConfig
 
 
 class AnalyticsAppConfig(AppConfig):
 
-    """Analytics app init code"""
+    """Analytics app init code."""
 
     name = 'readthedocs.analytics'
     verbose_name = 'Analytics'