Merge branch 'master' into guilabel-reformatting

Author: ericholscher

.pyup.yml

@@ -0,0 +1,31 @@
+# configure updates globally
+# default: all
+# allowed: all, insecure, False
+update: all
+
+# configure dependency pinning globally
+# default: True
+# allowed: True, False
+pin: True
+
+# update schedule
+# default: empty
+# allowed: "every day", "every week", ..
+schedule: "every week"
+
+# search for requirement files
+# default: True
+# allowed: True, False
+search: True
+
+# configure the branch prefix the bot is using
+# default: pyup-
+branch_prefix: pyup/
+
+# set a global prefix for PRs
+# default: empty
+pr_prefix: pyup: 
+
+# allow to close stale PRs
+# default: True
+close_prs: True

CHANGELOG.rst

@@ -1,3 +1,50 @@
+Version 3.2.1
+-------------
+
+:Date: February 07, 2019
+
+* `@ericholscher <http://github.com/ericholscher>`__: Remove excluding files on search. (`#5246 <https://github.com/rtfd/readthedocs.org/pull/5246>`__)
+* `@ericholscher <http://github.com/ericholscher>`__: Don't update search on HTMLFile save (`#5244 <https://github.com/rtfd/readthedocs.org/pull/5244>`__)
+* `@ericholscher <http://github.com/ericholscher>`__: Be more defensive in our 404 handler (`#5243 <https://github.com/rtfd/readthedocs.org/pull/5243>`__)
+* `@humitos <http://github.com/humitos>`__: Install sphinx-notfound-page for building 404.html custom page (`#5242 <https://github.com/rtfd/readthedocs.org/pull/5242>`__)
+* `@humitos <http://github.com/humitos>`__: Remove py2 compatibility (`#5241 <https://github.com/rtfd/readthedocs.org/pull/5241>`__)
+* `@ericholscher <http://github.com/ericholscher>`__: Release 3.2.0 (`#5240 <https://github.com/rtfd/readthedocs.org/pull/5240>`__)
+
+Version 3.2.0
+-------------
+
+:Date: February 06, 2019
+
+* `@ericholscher <http://github.com/ericholscher>`__: Support passing an explicit `index_name` for search indexing (`#5239 <https://github.com/rtfd/readthedocs.org/pull/5239>`__)
+* `@davidfischer <http://github.com/davidfischer>`__: Tweak some ad styles (`#5237 <https://github.com/rtfd/readthedocs.org/pull/5237>`__)
+* `@stsewd <http://github.com/stsewd>`__: Fix conda issue link (`#5226 <https://github.com/rtfd/readthedocs.org/pull/5226>`__)
+* `@humitos <http://github.com/humitos>`__: Add Santos to the development team (`#5224 <https://github.com/rtfd/readthedocs.org/pull/5224>`__)
+* `@ericholscher <http://github.com/ericholscher>`__: Update our GSOC page for 2019 (`#5210 <https://github.com/rtfd/readthedocs.org/pull/5210>`__)
+* `@humitos <http://github.com/humitos>`__: Do not allow to merge 'Status: blocked' PRs (`#5205 <https://github.com/rtfd/readthedocs.org/pull/5205>`__)
+* `@stsewd <http://github.com/stsewd>`__: Inject user to middleware tests (`#5203 <https://github.com/rtfd/readthedocs.org/pull/5203>`__)
+* `@ericholscher <http://github.com/ericholscher>`__: Remove approvals requirement from mergable (`#5200 <https://github.com/rtfd/readthedocs.org/pull/5200>`__)
+* `@agjohnson <http://github.com/agjohnson>`__: Update project notification copy to past tense (`#5199 <https://github.com/rtfd/readthedocs.org/pull/5199>`__)
+* `@stsewd <http://github.com/stsewd>`__: Remove feature flag for v2 (`#5198 <https://github.com/rtfd/readthedocs.org/pull/5198>`__)
+* `@ericholscher <http://github.com/ericholscher>`__: Refactor search code (`#5197 <https://github.com/rtfd/readthedocs.org/pull/5197>`__)
+* `@stsewd <http://github.com/stsewd>`__: Update mergeable settings to v2 (`#5196 <https://github.com/rtfd/readthedocs.org/pull/5196>`__)
+* `@stsewd <http://github.com/stsewd>`__: Fix mergeable bot (`#5195 <https://github.com/rtfd/readthedocs.org/pull/5195>`__)
+* `@stsewd <http://github.com/stsewd>`__: Fix broken links for badges (`#5190 <https://github.com/rtfd/readthedocs.org/pull/5190>`__)
+* `@dojutsu-user <http://github.com/dojutsu-user>`__: Change badge style (`#5189 <https://github.com/rtfd/readthedocs.org/pull/5189>`__)
+* `@humitos <http://github.com/humitos>`__: Allow source_suffix to be a dictionary (`#5183 <https://github.com/rtfd/readthedocs.org/pull/5183>`__)
+* `@humitos <http://github.com/humitos>`__: Upgrade all packages removing py2 compatibility (`#5179 <https://github.com/rtfd/readthedocs.org/pull/5179>`__)
+* `@dojutsu-user <http://github.com/dojutsu-user>`__: Small docs fix (`#5176 <https://github.com/rtfd/readthedocs.org/pull/5176>`__)
+* `@stsewd <http://github.com/stsewd>`__: Sync all services even if  one social accoun fails (`#5171 <https://github.com/rtfd/readthedocs.org/pull/5171>`__)
+* `@ericholscher <http://github.com/ericholscher>`__: Release 3.1.0 (`#5170 <https://github.com/rtfd/readthedocs.org/pull/5170>`__)
+* `@rvmzes <http://github.com/rvmzes>`__: SyntaxError caused by comma in python3 (`#5156 <https://github.com/rtfd/readthedocs.org/pull/5156>`__)
+* `@humitos <http://github.com/humitos>`__: Use latest docker images as default (`#5155 <https://github.com/rtfd/readthedocs.org/pull/5155>`__)
+* `@stsewd <http://github.com/stsewd>`__:  Remove logic for guessing slug from an unregistered domain (`#5143 <https://github.com/rtfd/readthedocs.org/pull/5143>`__)
+* `@humitos <http://github.com/humitos>`__: Allow custom 404.html on projects (`#5130 <https://github.com/rtfd/readthedocs.org/pull/5130>`__)
+* `@dojutsu-user <http://github.com/dojutsu-user>`__: Docs for feature flag (`#5043 <https://github.com/rtfd/readthedocs.org/pull/5043>`__)
+* `@stsewd <http://github.com/stsewd>`__: Remove usage of project.documentation_type in tasks (`#4896 <https://github.com/rtfd/readthedocs.org/pull/4896>`__)
+* `@ericholscher <http://github.com/ericholscher>`__: Reapply the Elastic Search upgrade to `master` (`#4722 <https://github.com/rtfd/readthedocs.org/pull/4722>`__)
+* `@stsewd <http://github.com/stsewd>`__: Config file v2 docs (`#4451 <https://github.com/rtfd/readthedocs.org/pull/4451>`__)
+* `@stsewd <http://github.com/stsewd>`__: Set python3 as default interpreter (`#3581 <https://github.com/rtfd/readthedocs.org/pull/3581>`__)
+
 Version 3.1.0
 -------------
 

docs/builds.rst

@@ -18,7 +18,7 @@ if you provide a good reason your documentation needs more resources.
 You can see the current Docker build images that we use in our `docker repository <https://github.com/rtfd/readthedocs-docker-images>`_.
 `Docker Hub <https://hub.docker.com/r/readthedocs/build/>`_ also shows the latest set of images that have been built.
 
-Currently in production we're using the ``readthedocs/build:2.0`` docker image as our default image.
+Currently in production we're using the ``readthedocs/build:latest`` docker image as our default image.
 
 How we build documentation
 --------------------------
@@ -118,7 +118,7 @@ The build environment
 ---------------------
 
 The build process is executed inside Docker containers,
-by default the image used is ``readthedocs/build:2.0``,
+by default the image used is ``readthedocs/build:latest``,
 but you can change that using a :doc:`config-file/index`.
 
 .. note::
@@ -133,57 +133,7 @@ but you can change that using a :doc:`config-file/index`.
    If you want to know the specific version of a package that is installed in the image
    you can check the `Ubuntu package search page <https://packages.ubuntu.com/>`__.
 
-2.0 (stable)
-~~~~~~~~~~~~
-
-:O.S: Ubuntu 16.04
-:Conda: Miniconda 4.3.31
-:Python:
-  * ``m2crypto``
-  * ``matplolib``
-  * ``numpy``
-  * ``pandas``
-  * ``pip``
-  * ``scipy``
-:Other packages:
-  * ``doxygen``
-  * ``graphviz``
-  * ``libevent``
-  * ``libjpeg``
-  * ``libxml2-dev``
-  * ``libxslt1.1``
-  * ``pandoc``
-  * ``textlive-full``
-
-`More details <https://github.com/rtfd/readthedocs-docker-images/blob/releases/2.x/Dockerfile>`__
-
-3.0 (latest)
-~~~~~~~~~~~~
-
-:O.S: Ubuntu 16.04
-:Conda: Miniconda 4.4.10
-:Python:
-  * ``matplolib``
-  * ``numpy``,
-  * ``pandas``
-  * ``pip``
-  * ``scipy``
-:JavaScript:
-  * ``jsdoc``
-  * ``nodejs``
-  * ``npm``
-:Other packages:
-  * ``doxygen``
-  * ``libevent-dev``
-  * ``libgraphviz-dev``
-  * ``libjpeg``
-  * ``libxml2-dev``
-  * ``libxslt1-dev``
-  * ``pandoc``
-  * ``plantuml``
-  * ``textlive-full``
-
-`More details <https://github.com/rtfd/readthedocs-docker-images/blob/releases/3.x/Dockerfile>`__
+More details on software installed in images could be found by browsing specific branch in `rtfd/readthedocs-docker-images <https://github.com/rtfd/readthedocs-docker-images>`__ repository.
 
 Writing your own builder
 ------------------------

docs/conf.py

@@ -4,6 +4,7 @@
 
 import os
 import sys
+from configparser import RawConfigParser
 
 import sphinx_rtd_theme
 
@@ -18,6 +19,13 @@
 django.setup()
 
 
+def get_version():
+    """Return package version from setup.cfg."""
+    config = RawConfigParser()
+    config.read(os.path.join('..', 'setup.cfg'))
+    return config.get('metadata', 'version')
+
+
 sys.path.append(os.path.abspath('_ext'))
 extensions = [
     'sphinx.ext.autosectionlabel',
@@ -29,6 +37,7 @@
     'sphinx_tabs.tabs',
     'sphinx-prompt',
     'recommonmark',
+    'notfound.extension',
 ]
 templates_path = ['_templates']
 
@@ -39,7 +48,7 @@
 copyright = '2010-{}, Read the Docs, Inc & contributors'.format(
     timezone.now().year
 )
-version = '2.7'
+version = get_version()
 release = version
 exclude_patterns = ['_build']
 default_role = 'obj'
@@ -81,6 +90,19 @@
 # Activate autosectionlabel plugin
 autosectionlabel_prefix_document = True
 
+# sphinx-notfound-page
+# https://github.com/rtfd/sphinx-notfound-page
+notfound_context = {
+    'title': 'Page Not Found',
+    'body': '''
+<h1>Page Not Found</h1>
+
+<p>Sorry, we couldn't find that page.</p>
+
+<p>Try using the search box or go to the homepage.</p>
+''',
+}
+
 
 def setup(app):
     app.add_stylesheet('css/sphinx_prompt_css.css')

docs/config-file/v1.rst

@@ -112,8 +112,8 @@ The ``build`` block configures specific aspects of the documentation build.
 build.image
 ```````````
 
-* Default: :djangosetting:`DOCKER_IMAGE`
-* Options: ``1.0``, ``2.0``, ``latest``
+* Default: :djangosetting:`DOCKER_DEFAULT_VERSION`
+* Options: ``stable``, ``latest``
 
 The build image to use for specific builds.
 This lets users specify a more experimental build image,
@@ -122,9 +122,8 @@ if they want to be on the cutting edge.
 Certain Python versions require a certain build image,
 as defined here:
 
-* ``1.0``: 2, 2.7, 3, 3.4
-* ``2.0``: 2, 2.7, 3, 3.5
-* ``latest``: 2, 2.7, 3, 3.3, 3.4, 3.5, 3.6
+* ``stable``: :buildpyversions:`stable`
+* ``latest``: :buildpyversions:`latest`
 
 .. code-block:: yaml
 
@@ -147,8 +146,8 @@ used for building documentation.
 python.version
 ``````````````
 
-* Default: ``2.7``
-* Options: ``2.7``, ``2``, ``3.5``, ``3``
+* Default: ``3.7``
+* Options: :buildpyversions:`latest`
 
 This is the version of Python to use when building your documentation.
 If you specify only the major version of Python,
@@ -158,7 +157,7 @@ the highest supported minor version will be selected.
 
     The supported Python versions depends on the version of the build image your
     project is using. The default build image that is used to build
-    documentation contains support for Python ``2.7`` and ``3.5``.  See the
+    documentation contains support for Python ``2.7`` and ``3.7``.  See the
     :ref:`yaml__build__image` for more information on supported Python versions.
 
 .. code-block:: yaml

docs/development/search.rst

@@ -37,13 +37,14 @@ By default, Auto Indexing is turned off in development mode. To turn it on, chan
 After that, whenever a documentation successfully builds, or project gets added,
 the search index will update automatically.
 
-
 Architecture
 ------------
 The search architecture is devided into 2 parts.
-One part is responsible for **indexing** the documents and projects and
-the other part is responsible for querying the Index to show the proper results to users.
-We use the `django-elasticsearch-dsl`_ package mostly to the keep the search working.
+
+* One part is responsible for **indexing** the documents and projects (``documents.py``)
+* The other part is responsible for **querying** the Index to show the proper results to users (``faceted_search.py``)
+
+We use the `django-elasticsearch-dsl`_ package for our Document abstraction.
 `django-elasticsearch-dsl`_ is a wrapper around `elasticsearch-dsl`_ for easy configuration
 with Django.
 
@@ -72,11 +73,11 @@ and index/delete the documentation content from the `HTMLFile` instances.
 
 How we index projects
 ~~~~~~~~~~~~~~~~~~~~~
+
 We also index project information in our search index so that the user can search for projects
-from the main site. `django-elasticsearch-dsl`_ listen `post_create` and `post_delete` signals of
+from the main site. We listen to the `post_create` and `post_delete` signals of
 `Project` model and index/delete into Elasticsearch accordingly.
 
-
 Elasticsearch Document
 ~~~~~~~~~~~~~~~~~~~~~~
 
@@ -88,9 +89,7 @@ As per requirements of `django-elasticsearch-dsl`_, it is stored in the
     `django-elasticsearch-dsl`_ listens to the `post_save` signal of `Project` model and
     then index/delete into Elasticsearch.
 
-    **PageDocument**: It is used for indexing documentation of projects. By default, the auto
-    indexing is turned off by `ignore_signals = settings.ES_PAGE_IGNORE_SIGNALS`.
-    `settings.ES_PAGE_IGNORE_SIGNALS` is `False` both in development and production.
+    **PageDocument**: It is used for indexing documentation of projects. 
     As mentioned above, our `Search` app listens to the `bulk_post_create` and `bulk_post_delete`
     signals and indexes/deleted documentation into Elasticsearch. The signal listeners are in
     the `readthedocs/search/signals.py` file. Both of the signals are dispatched

docs/security.rst

@@ -84,6 +84,17 @@ __ https://pgp.mit.edu/pks/lookup?op=vindex&search=0xFEEF9FC2DD21D271
 Security issue archive
 ----------------------
 
+Version 3.2.0
+~~~~~~~~~~~~~
+
+:ref:`version-3.2.0` resolved an issue where a specially crafted request
+could result in a DNS query to an arbitrary domain.
+
+This issue was found by `Cyber Smart Defence <https://www.cybersmartdefence.com/>`_
+who reported it as part of a security audit to a firm running a local installation
+of Read the Docs.
+
+
 Release 2.3.0
 ~~~~~~~~~~~~~
 

docs/webhooks.rst

@@ -46,11 +46,13 @@ GitHub
 * Go to the :guilabel:`Settings` page for your project
 * Click :guilabel:`Webhooks` > :guilabel:`Add webhook`
 * For **Payload URL**, use the URL of the integration on Read the Docs,
-  found on the project's :guilabel:`Admin` > :guilabel:`Integrations` page
+  found on the project's :guilabel:`Admin` > :guilabel:`Integrations` page.
+  You may need to prepend *https://* to the URL.
 * For **Content type**, both *application/json* and
   *application/x-www-form-urlencoded* work
 * Select **Let me select individual events**,
   and mark **Pushes**, **Branch or tag creation**, and **Branch or tag deletion** events
+* Leave the **Secrets** field black
 * Finish by clicking **Add webhook**
 
 You can verify if the webhook is working at the bottom of the GitHub page under **Recent Deliveries**.

media/css/readthedocs-doc-embed.css

@@ -55,7 +55,7 @@ div.ethical-footer {
 }
 .ethical-sidebar,
 .ethical-footer {
-    padding: 1em;
+    padding: 0.5em;
     margin: 1em 0;
 }
 .ethical-sidebar img,
@@ -93,6 +93,30 @@ div.ethical-footer {
     line-height: 20px;
 }
 
+/* Techstack badging */
+.ethical-sidebar ul {
+    margin: 0 !important;
+    padding-left: 0;
+    list-style: none;
+}
+.ethical-sidebar ul li {
+    display: inline-block;
+    background-color: lightskyblue;
+    color: black;
+    padding: 0.25em 0.4em;
+    font-size: 75%;
+    font-weight: 700;
+    margin: 0.25em;
+    border-radius: 0.25rem;
+    text-align: center;
+    vertical-align: baseline;
+    white-space: nowrap;
+    line-height: 1.41;
+}
+.ethical-sidebar ul li:not(:last-child) {
+    margin-right: .25rem;
+}
+
 .ethical-sidebar a,
 .ethical-sidebar a:visited,
 .ethical-sidebar a:hover,
@@ -102,7 +126,7 @@ div.ethical-footer {
 .ethical-footer a:hover,
 .ethical-footer a:active {
     color: #0a0a0a;
-    text-decoration: underline !important;
+    text-decoration: none !important;
     border-bottom: 0 !important;
 }