Merge branch 'master' into sss-integration-guide

Author: stsewd

.travis.yml

@@ -10,7 +10,7 @@ addons:
 jobs:
   include:
     - python: 3.6
-      env: TOXENV=py36,codecov TOX_POSARGS='' ES_VERSION=6.2.4 ES_DOWNLOAD_URL=https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-${ES_VERSION}.tar.gz
+      env: TOXENV=py36,codecov TOX_POSARGS='' ES_VERSION=6.5.4 ES_DOWNLOAD_URL=https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-${ES_VERSION}.tar.gz
     - python: 3.6
       env: TOXENV=docs
     - python: 3.6

CHANGELOG.rst

@@ -1,3 +1,76 @@
+Version 5.4.2
+-------------
+
+:Date: September 09, 2020
+
+* `@humitos <https://github.com/humitos>`__: Show "Connected Services" form errors to the user (`#7469 <https://github.com/readthedocs/readthedocs.org/pull/7469>`__)
+* `@humitos <https://github.com/humitos>`__: Allow to extend OrganizationTeamBasicForm from -corporate (`#7467 <https://github.com/readthedocs/readthedocs.org/pull/7467>`__)
+* `@pyup-bot <https://github.com/pyup-bot>`__: pyup:  Scheduled weekly dependency update for week 36 (`#7465 <https://github.com/readthedocs/readthedocs.org/pull/7465>`__)
+* `@AvdN <https://github.com/AvdN>`__: correct invalid YAML (`#7458 <https://github.com/readthedocs/readthedocs.org/pull/7458>`__)
+* `@stsewd <https://github.com/stsewd>`__: Remote repository: filter by account before deleting (`#7454 <https://github.com/readthedocs/readthedocs.org/pull/7454>`__)
+* `@humitos <https://github.com/humitos>`__: Truncate the beggining of the commands' output (`#7449 <https://github.com/readthedocs/readthedocs.org/pull/7449>`__)
+* `@davidfischer <https://github.com/davidfischer>`__: Update links to advertising (`#7443 <https://github.com/readthedocs/readthedocs.org/pull/7443>`__)
+* `@stsewd <https://github.com/stsewd>`__: Revert "Don't retry on POST" (`#7442 <https://github.com/readthedocs/readthedocs.org/pull/7442>`__)
+* `@stsewd <https://github.com/stsewd>`__: Organizations: move signals (`#7441 <https://github.com/readthedocs/readthedocs.org/pull/7441>`__)
+* `@stsewd <https://github.com/stsewd>`__: Organizations: move forms (`#7438 <https://github.com/readthedocs/readthedocs.org/pull/7438>`__)
+* `@humitos <https://github.com/humitos>`__: Grab the correct name of RemoteOrganization to use in the query (`#7430 <https://github.com/readthedocs/readthedocs.org/pull/7430>`__)
+* `@stsewd <https://github.com/stsewd>`__: Revert "ES: update dependencies" (`#7429 <https://github.com/readthedocs/readthedocs.org/pull/7429>`__)
+* `@pyup-bot <https://github.com/pyup-bot>`__: pyup:  Scheduled weekly dependency update for week 35 (`#7423 <https://github.com/readthedocs/readthedocs.org/pull/7423>`__)
+* `@humitos <https://github.com/humitos>`__: Mark a build as DUPLICATED (same version) only it's close in time (`#7420 <https://github.com/readthedocs/readthedocs.org/pull/7420>`__)
+
+Version 5.4.1
+-------------
+
+:Date: September 01, 2020
+
+* `@stsewd <https://github.com/stsewd>`__: Docs: update readthedcos-sphinx-search ext (`#7427 <https://github.com/readthedocs/readthedocs.org/pull/7427>`__)
+* `@humitos <https://github.com/humitos>`__: Upgrade Django to 2.2.16 (`#7426 <https://github.com/readthedocs/readthedocs.org/pull/7426>`__)
+* `@bmorrison4 <https://github.com/bmorrison4>`__: Fix typo in docs/guides/adding-custom-css.rst (`#7424 <https://github.com/readthedocs/readthedocs.org/pull/7424>`__)
+* `@stsewd <https://github.com/stsewd>`__: Test: set privacy level explicitly (`#7422 <https://github.com/readthedocs/readthedocs.org/pull/7422>`__)
+* `@stsewd <https://github.com/stsewd>`__: Pip: test new resolver (`#7412 <https://github.com/readthedocs/readthedocs.org/pull/7412>`__)
+* `@stsewd <https://github.com/stsewd>`__: Update common (`#7411 <https://github.com/readthedocs/readthedocs.org/pull/7411>`__)
+* `@stsewd <https://github.com/stsewd>`__: Release 5.4.0 (`#7410 <https://github.com/readthedocs/readthedocs.org/pull/7410>`__)
+* `@stsewd <https://github.com/stsewd>`__: Docker: install requirements from local changes (`#7409 <https://github.com/readthedocs/readthedocs.org/pull/7409>`__)
+* `@stsewd <https://github.com/stsewd>`__: ES: update dependencies (`#7408 <https://github.com/readthedocs/readthedocs.org/pull/7408>`__)
+* `@pyup-bot <https://github.com/pyup-bot>`__: pyup:  Scheduled weekly dependency update for week 34 (`#7406 <https://github.com/readthedocs/readthedocs.org/pull/7406>`__)
+* `@stsewd <https://github.com/stsewd>`__: API client: don't retry on POST (`#7383 <https://github.com/readthedocs/readthedocs.org/pull/7383>`__)
+* `@saadmk11 <https://github.com/saadmk11>`__: build_url added to all API v3 build endpoints (`#7373 <https://github.com/readthedocs/readthedocs.org/pull/7373>`__)
+* `@stsewd <https://github.com/stsewd>`__: Guide: deprecating content (`#7333 <https://github.com/readthedocs/readthedocs.org/pull/7333>`__)
+* `@humitos <https://github.com/humitos>`__: Auto-join email users field for Team model (`#7328 <https://github.com/readthedocs/readthedocs.org/pull/7328>`__)
+* `@stsewd <https://github.com/stsewd>`__: Guide: Cross-referencing with Sphinx (`#7326 <https://github.com/readthedocs/readthedocs.org/pull/7326>`__)
+* `@humitos <https://github.com/humitos>`__: Sync RemoteRepository and RemoteOrganization in all VCS providers (`#7310 <https://github.com/readthedocs/readthedocs.org/pull/7310>`__)
+* `@stsewd <https://github.com/stsewd>`__: Page views: use origin URL instead of page name (`#7293 <https://github.com/readthedocs/readthedocs.org/pull/7293>`__)
+
+Version 5.4.0
+-------------
+
+:Date: August 25, 2020
+
+* `@stsewd <https://github.com/stsewd>`__: ES: match version used in production (`#7407 <https://github.com/readthedocs/readthedocs.org/pull/7407>`__)
+* `@davidfischer <https://github.com/davidfischer>`__: Advertising docs tweaks (`#7400 <https://github.com/readthedocs/readthedocs.org/pull/7400>`__)
+* `@stsewd <https://github.com/stsewd>`__: Docs: update readthedocs-sphinx-search (`#7399 <https://github.com/readthedocs/readthedocs.org/pull/7399>`__)
+* `@keewis <https://github.com/keewis>`__: document installing into a environment with pinned dependencies (`#7397 <https://github.com/readthedocs/readthedocs.org/pull/7397>`__)
+* `@pyup-bot <https://github.com/pyup-bot>`__: pyup:  Scheduled weekly dependency update for week 32 (`#7377 <https://github.com/readthedocs/readthedocs.org/pull/7377>`__)
+* `@stsewd <https://github.com/stsewd>`__: Builds: store build commands in storage (`#7356 <https://github.com/readthedocs/readthedocs.org/pull/7356>`__)
+* `@stsewd <https://github.com/stsewd>`__: Guide: Cross-referencing with Sphinx (`#7326 <https://github.com/readthedocs/readthedocs.org/pull/7326>`__)
+
+Version 5.3.0
+-------------
+
+:Date: August 18, 2020
+
+* `@humitos <https://github.com/humitos>`__: Remove the comma added in logs that breaks grep parsing (`#7393 <https://github.com/readthedocs/readthedocs.org/pull/7393>`__)
+* `@stsewd <https://github.com/stsewd>`__: GitLab webhook: don't fail on invalid payload (`#7391 <https://github.com/readthedocs/readthedocs.org/pull/7391>`__)
+* `@stsewd <https://github.com/stsewd>`__: Task router: improve logging (`#7389 <https://github.com/readthedocs/readthedocs.org/pull/7389>`__)
+* `@stsewd <https://github.com/stsewd>`__: External providers: better logging for GitLab (`#7385 <https://github.com/readthedocs/readthedocs.org/pull/7385>`__)
+* `@stsewd <https://github.com/stsewd>`__: Task router: small changes (`#7379 <https://github.com/readthedocs/readthedocs.org/pull/7379>`__)
+* `@stsewd <https://github.com/stsewd>`__: Search: return relatives URLS (`#7376 <https://github.com/readthedocs/readthedocs.org/pull/7376>`__)
+* `@stsewd <https://github.com/stsewd>`__: Sync versions: little optimization when deleting versions (`#7367 <https://github.com/readthedocs/readthedocs.org/pull/7367>`__)
+* `@agjohnson <https://github.com/agjohnson>`__: Add feature flag to just skip the sync version task entirely (`#7366 <https://github.com/readthedocs/readthedocs.org/pull/7366>`__)
+* `@agjohnson <https://github.com/agjohnson>`__: Convert zip to list for templates (`#7359 <https://github.com/readthedocs/readthedocs.org/pull/7359>`__)
+* `@stsewd <https://github.com/stsewd>`__: Search: implement stable API (`#7255 <https://github.com/readthedocs/readthedocs.org/pull/7255>`__)
+* `@stsewd <https://github.com/stsewd>`__: Search: improve parser (`#7233 <https://github.com/readthedocs/readthedocs.org/pull/7233>`__)
+
 Version 5.2.3
 -------------
 

common

@@ -39,8 +39,8 @@ RUN ln -s /usr/bin/python3 /usr/bin/python
 
 WORKDIR /tmp
 
-RUN curl -O https://raw.githubusercontent.com/readthedocs/readthedocs.org/master/requirements/pip.txt
-RUN curl -O https://raw.githubusercontent.com/readthedocs/readthedocs.org/master/requirements/docker.txt
+COPY requirements/pip.txt pip.txt
+COPY requirements/docker.txt docker.txt
 RUN pip3 install --no-cache-dir -r docker.txt
 
 # Install readthedocs-ext only if GITHUB_TOKEN is provided

dockerfiles/Dockerfile

@@ -71,8 +71,6 @@ Instead, we target based solely upon:
     targeting by state or by metro area (DMA specifically).
   * We geolocate a user's IP address to a country when a request is made.
 
-Read the Docs uses GeoLite2 data created by `MaxMind <http://maxmind.com>`_.
-
 
 Where ads are shown
 -------------------
@@ -97,6 +95,14 @@ For more details, see the :ref:`Do Not Track section <privacy-policy:Do Not Trac
 of our privacy policy.
 
 
+Ad serving infrastructure
+-------------------------
+
+Our `entire ad server <https://github.com/readthedocs/ethical-ad-server>`_ is open source,
+so you can inspect how we're doing things.
+We believe strongly in open source, and we practice what we preach.
+
+
 Analytics
 ---------
 

docs/advertising/advertising-details.rst

@@ -19,10 +19,13 @@ as our way of saying thanks to the open source community.
 We talk a bit below about :ref:`our worldview on advertising <advertising/ethical-advertising:Our Worldview>`,
 if you want to know more.
 
-.. important::
+.. admonition:: Are you a marketer?
+
+    We built a `whole business <https://www.ethicalads.io/?ref=rtd-docs>`_
+    around privacy-focused advertising.
+    If you're trying to reach developers, we have a network of hand-approved sites (including Read the Docs)
+    where your ads are shown.
 
-   **Are you a marketer?**
-   `Learn more <https://readthedocs.org/sustainability/advertising/>`_ about how you can connect with the millions of developers who Read the Docs each month.
 
 Feedback
 --------
@@ -74,6 +77,9 @@ They should run before your content,
 they should take over the page,
 the bigger, weirder, or flashier the better.
 
+.. _fake ad clicks: https://en.wikipedia.org/wiki/Click_fraud
+.. _massive downsides: http://idlewords.com/talks/what_happens_next_will_amaze_you.htm
+
 We opt out
 ~~~~~~~~~~
 
@@ -91,8 +97,6 @@ The ads won't flash or move.
 We run the ads we want to have on our site,
 in a way that makes us feel good.
 
-.. _fake ad clicks: https://en.wikipedia.org/wiki/Click_fraud
-
 Additional details
 ------------------
 
@@ -105,8 +109,11 @@ Additional details
 * Eric Holscher, one of our co-founders
   `talks a bit more <https://www.ericholscher.com/blog/2016/aug/31/funding-oss-marketing-money/>`_
   about funding open source this way on his blog.
+* After proving our ad model as a way to fund open source and building our ad serving infrastructure,
+  we launched the `EthicalAds network <https://www.ethicalads.io?ref=rtd-docs>`_
+  to help other projects be sustainable.
 
-.. _advertising FAQ: https://readthedocs.org/sustainability/advertising/faq/
+.. _advertising FAQ: https://www.ethicalads.io/advertisers/faq/?ref=rtd-docs
 
 Join us
 -------
@@ -118,9 +125,7 @@ We hope that others will join us in this mission:
   `talk to your marketing folks <https://www.ericholscher.com/blog/2016/aug/31/funding-oss-marketing-money/>`_ about using advertising that respects your privacy.
 * **If you're a marketer**,
   vote with your dollars and support us in building the ad model we want to exist.
-  `Get more information <https://readthedocs.org/sustainability/advertising/>`_ on what we offer.
-
-.. _massive downsides: http://idlewords.com/talks/what_happens_next_will_amaze_you.htm
+  `Get more information <https://www.ethicalads.io/advertisers/?ref=rtd-docs>`_ on what we offer.
 
 
 Community Ads

docs/advertising/ethical-advertising.rst

@@ -110,6 +110,8 @@ def get_version():
 # Activate autosectionlabel plugin
 autosectionlabel_prefix_document = True
 
+numfig = True
+
 # sphinx-notfound-page
 # https://github.com/readthedocs/sphinx-notfound-page
 notfound_context = {

docs/conf.py

@@ -468,6 +468,8 @@ Settings for more control over :doc:`/server-side-search`.
       ranking:
         api/v1/*: -1
         api/v2/*: 4
+      ignore:
+        - 404.html
 
 search.ranking
 ``````````````
@@ -490,20 +492,23 @@ Pages with a rank closer to -10 will appear further down the list of results,
 and pages with a rank closer to 10 will appear higher in the list of results.
 Note that 0 means *normal rank*, not *no rank*.
 
+If you are looking to completely ignore a page,
+check :ref:`config-file/v2:search.ignore`.
+
 .. code-block:: yaml
 
    version: 2
 
    search:
       ranking:
         # Match a single file
-        tutorial.hml: 2
+        tutorial.html: 2
 
         # Match all files under the api/v1 directory
         api/v1/*: -5
 
         # Match all files that end with tutorial.html
-        */tutorial.html: 3
+        '*/tutorial.html': 3
 
 .. note::
 
@@ -514,6 +519,43 @@ Note that 0 means *normal rank*, not *no rank*.
    Is better to decrease the rank of pages you want to deprecate,
    rather than increasing the rank of the other pages.
 
+search.ignore
+`````````````
+
+Don't index files matching a pattern.
+This is, you won't see search results from these files.
+
+:Type: ``list`` of patterns
+:Default: ``['search.html', 'search/index.html', '404.html', '404/index.html']``
+
+Patterns are matched against the final html pages produced by the build
+(you should try to match `index.html`, not `docs/index.rst`).
+Patterns can include some special characters:
+
+- ``*`` matches everything
+- ``?`` matches any single character
+- ``[seq]`` matches any character in ``seq``
+
+.. code-block:: yaml
+
+   version: 2
+
+   search:
+      ignore:
+        # Ignore a single file
+        - 404.html
+
+        # Ignore all files under the search/ directory
+        - search/*
+
+        # Ignore all files that end with ref.html
+        - '*/ref.html'
+
+.. note::
+
+   Since Read the Docs fallbacks to the original search engine when no results are found,
+   you may still see search results from ignored pages.
+
 Schema
 ------
 

docs/config-file/v2.rst

@@ -251,6 +251,23 @@ using a Read the Docs :doc:`config-file/index`.
 .. _pip requirements file: https://pip.pypa.io/en/stable/user_guide.html#requirements-files
 
 
+I need to install a package in a environment with pinned versions
+-----------------------------------------------------------------
+
+To ensure proper installation of a python package, the ``pip`` :ref:`install method <config-file/v2:python.install>` will automatically upgrade every dependency to its most recent version in case they aren't pinned by the package definition.
+If instead you'd like to pin your dependencies outside the package, you can add this line to your requirements or environment file (if you are using Conda).
+
+In your ``requirements.txt`` file::
+
+    # path to the directory containing setup.py relative to the project root
+    -e .
+
+In your Conda environment file (``environment.yml``)::
+
+    # path to the directory containing setup.py relative to the environment file
+    -e ..
+
+
 How can I avoid search results having a deprecated version of my docs?
 ----------------------------------------------------------------------
 

docs/faq.rst

@@ -42,7 +42,7 @@ A similar approach can be used to add JavaScript files::
 .. note::
 
     The Sphinx HTML options ``html_css_files`` and ``html_js_files``
-    where added in Sphinx 1.8.
+    were added in Sphinx 1.8.
     Unless you have a good reason to use an older version,
     you are strongly encouraged to upgrade.
     Sphinx is almost entirely backwards compatible.