Merge tag '3.7.5' into rel

Author: agjohnson

CHANGELOG.rst

@@ -1,3 +1,27 @@
+Version 3.7.5
+-------------
+
+:Date: September 26, 2019
+
+* `@davidfischer <http://github.com/davidfischer>`__: Remove if storage blocks (`#6191 <https://github.com/readthedocs/readthedocs.org/pull/6191>`__)
+* `@davidfischer <http://github.com/davidfischer>`__: Update security docs (`#6179 <https://github.com/readthedocs/readthedocs.org/pull/6179>`__)
+* `@davidfischer <http://github.com/davidfischer>`__: Add the private spamfighting module to INSTALLED_APPS (`#6177 <https://github.com/readthedocs/readthedocs.org/pull/6177>`__)
+* `@davidfischer <http://github.com/davidfischer>`__: Document connected account permissions (`#6172 <https://github.com/readthedocs/readthedocs.org/pull/6172>`__)
+* `@stsewd <http://github.com/stsewd>`__: Require login for old redirect (`#6170 <https://github.com/readthedocs/readthedocs.org/pull/6170>`__)
+* `@humitos <http://github.com/humitos>`__: Remove old and unused code (`#6167 <https://github.com/readthedocs/readthedocs.org/pull/6167>`__)
+* `@stsewd <http://github.com/stsewd>`__: Clean up views (`#6166 <https://github.com/readthedocs/readthedocs.org/pull/6166>`__)
+* `@stsewd <http://github.com/stsewd>`__: Update docs for sharing (`#6164 <https://github.com/readthedocs/readthedocs.org/pull/6164>`__)
+* `@pyup-bot <http://github.com/pyup-bot>`__: pyup:  Scheduled weekly dependency update for week 36 (`#6158 <https://github.com/readthedocs/readthedocs.org/pull/6158>`__)
+* `@saadmk11 <http://github.com/saadmk11>`__: Remove PR Builder Project Idea from RTD GSoC Docs (`#6147 <https://github.com/readthedocs/readthedocs.org/pull/6147>`__)
+* `@ericholscher <http://github.com/ericholscher>`__: Serialize time in search queries properly (`#6142 <https://github.com/readthedocs/readthedocs.org/pull/6142>`__)
+* `@humitos <http://github.com/humitos>`__: Allow to extend DomainCreate view (`#6139 <https://github.com/readthedocs/readthedocs.org/pull/6139>`__)
+* `@saadmk11 <http://github.com/saadmk11>`__: Integration Re-sync Bug Fix (`#6124 <https://github.com/readthedocs/readthedocs.org/pull/6124>`__)
+* `@stsewd <http://github.com/stsewd>`__: Don't log BuildEnvironmentWarning as error (`#6112 <https://github.com/readthedocs/readthedocs.org/pull/6112>`__)
+* `@dojutsu-user <http://github.com/dojutsu-user>`__: Add Search Guide (`#6101 <https://github.com/readthedocs/readthedocs.org/pull/6101>`__)
+* `@saadmk11 <http://github.com/saadmk11>`__: Add PR Builder guide to docs (`#6093 <https://github.com/readthedocs/readthedocs.org/pull/6093>`__)
+* `@dojutsu-user <http://github.com/dojutsu-user>`__: Record search queries smartly (`#6088 <https://github.com/readthedocs/readthedocs.org/pull/6088>`__)
+* `@dojutsu-user <http://github.com/dojutsu-user>`__: Remove 'highlight' URL param from search results (`#6087 <https://github.com/readthedocs/readthedocs.org/pull/6087>`__)
+
 Version 3.7.4
 -------------
 

docs/_static/images/guides/github-build-status-reporting.gif

@@ -0,0 +1,54 @@
+-----BEGIN PGP PUBLIC KEY BLOCK-----
+Comment: GPGTools - http://gpgtools.org
+
+mQINBFqNz1ABEADvgtp3LT1pV5wuTyBPaKrbWBFj10eKyQ15wfgyc2RR6Ix5QnBo
+6BcJ4fpgBhSwlngsrm0WU5kI/jH7ySwzbDpYCRiLvGJx+pEYLuBBOSm6r5M1N+FV
+xq3ShT4mHXhwPS1mKf9Xe+KlMdYa2e5TlBEr+TxGAmFFrOLjPxw6IDHgP3MVidr2
+iHA2PAATl6H9ZYvNzLkI2sP7h0V1/ADd43YpAK4yk6gdVjype5ez8lmoxDKNabMt
+dSfdOup8zy/fbC5KlxqrT9hHBkYfQWDLWXWcDW111q+ZvncujCrpONaY86bcQ3nn
+QgkeWCwj254vvqsrygEU93reC2onWaROUKoLlX1/1m2k2X3qze/hJRFZaljXVPKH
+jV/5q88EbjSUDgY5v9mdX8jhJAukx9HkOFdkMSh3RBgu1r+UPnCNd9K4T2nN0LBL
+c9NTG0HW7Di5ivEVq74SqDIeiVeOrfY/B6pRuUm/kNPcvZ+ZQPeNk6JUMqEemO9Q
+h0VHSkgkhCPWPO9c9wWJz7O6y6vXgsFG7BZg7mTVOiKbdgneGo/rKRvuBlQ7hCvP
+PklwyRn90SJSgv7NF6HMm4TA1R9mzp+90oXjrDXARXmGTsPtcDXFv7xqpK1+Mfcn
+ajEJYdIRNWVgx0E2RzHRipdG5MIQ5Plf4/GasVHl71nMGY06oIu1T+44MQARAQAB
+tFpSZWFkIHRoZSBEb2NzIFNlY3VyaXR5IFRlYW0gKGh0dHBzOi8vcmVhZHRoZWRv
+Y3Mub3JnL3NlY3VyaXR5LykgPHNlY3VyaXR5QHJlYWR0aGVkb2NzLm9yZz6JAk4E
+EwEIADgWIQRq+P453S2vjKvMbGn+75/C3SHScQUCWo3PUAIbAwULCQgHAgYVCgkI
+CwIEFgIDAQIeAQIXgAAKCRD+75/C3SHScYMMD/4z0TN08oJ57Krg+UODXPsT9U3l
+8fyKHhe6fJCTt5GQiWwBbkfa4M0YcxemIJGdgF1DpdSteWIL0tCwXbxHph+keYlg
+z+EmF+W7OlnwbmtDx/Rj9VNdzf636DkMusTQzYEB/+FdN4LtMVq7Al4CZ2Ca82F8
+h0TLceh2bRgNjeWPuAMj7kS8tw3D9LmYA8d8Lv2c2jN7ba9p+QNKdSa4ErdJ0kbz
+CSFcABPfc+LlYWFbm5j1ggzTONgR9R27mpAGMAtgSeAtxXLU0sQfLtCNaVkRyJ3C
+s0awUvJCuq11YUPjz4HAcTWM4baAxK5LliEDOdaOlTK0q8T0sPP+SWt5JRL6/Xc3
+SwaXnVfzzZyeaSmRGEHmGQYBTB3WMUcH1RNH6uhNPCF4x3t0jOHWP7Eka4B9IdfE
+cd+GDwqTKCHyddh8yUzTrmlSbdO7iuel6WVN0Xo1xzVrLUKpgDvB0UuPQXlxDLUc
+WVrKv9rcyDVGVpDjQSQ4l191NDzlfzmDFkZ69Qe3E5Ir8oWBCMtHX3C99ocIcbR3
+3mqOio2/QQCJzrMOWxgauF/q4JMKQRj5Qq8US2U32xlPzB8O09z1e3kUOEy4gbwE
+6LVMj6vxJqjV8/4AOcocGgJKLLC9nqhf2sq5zA4TjI7lI25pgDDYozaLF2ss5nk3
+t9hQmI5Q0MXGSsflAbkCDQRajc9QARAA30mNh2QaDVLcHEfJ9QKfqRfY3ddG6s6F
+AHLR7mQ2tmO7ygzxgEcM7+7W+lmc9L+mZ5WutK5PIae/MpAgOo220079aW3em2sz
++dIHdSE7CaajUahQaPqLY6W0bbZloGGDetPtOMKBTI1HtSNyKIsULsbyUA1SsEFn
+aWtOof1MqqVQvYDwwwRj6T+DHtV17yO33v98k01Nx1SSThVY9wQ4MOZDBOAqWhie
+iboDssrvtVZZihbQ9LM8TH/l81auodBDpp96tgWguzjM4eyutaYZ6ZOLhfVUuEX+
+gEqqJ7alXfDhh3NZUMHZ0SHVII7u7nqciTo7NS9rxBXfdGlKmC/9Z3ehIXSpCnPY
+JO42qMjPVM5/QDoeK9BWWX3rXmwnNzqK0D4L7zK/cVnt2q9EqPCUJgOITJWEGc9G
+crO0ni/8M+BuhO/4MeJJtrPtmq1b1BoeuYBzf1M7ARtnvtC5hLLrtxiy4UANlwSm
+HFcIEt5UViwEqRuQWr5ZO3mwaJP2R/foDHww7JYEqJ/GFI5RpT+3LWT5FXPC1QvU
+sbewD+ZmLSfifhC0WUzF002eadgXNyXSZKAirM8+yELM4xZAs0pJVlKVFRnis0OL
+Wxdzthp2gTg+agtMoz27belxVUEmRK9GDaXi9XtJSooSglt0xlTimgB40nDPniVB
+4h5S/gHsg8cAEQEAAYkCNgQYAQgAIBYhBGr4/jndLa+Mq8xsaf7vn8LdIdJxBQJa
+jc9QAhsMAAoJEP7vn8LdIdJxwswP/0oGlxUJZhDG8yCbTTTvxvKXd02AXw/GQKrq
+ptrLEXbhko6TOuZolEWsRrc1ObMiky97CicqQthg22Kf1K7g2UNlPS4LFtTrPXKL
+9iJMAgms0a0ul3cHqQh2XiuGc1bfDuGyNe/nE5/uvgpjxg0hvvBH/5xuiaMkf+gZ
+nJjF2ZcXm6a17MCuAcw/siox1/PeXn0At/wzOWD9qONg+BI/QUynzcSMg/coBe7V
+hUX1LU02n6laBwuQ6Q0KoD6CP43seYv3JaPyVP7+IkhtH/RDm8q3vs0qLpEBrJIb
+vBYBXLtyoGHxTkWueou0Ur1j2lLUMqnQkq5NAsckSfHtZEdPDy6T3NHMfVRmnXnW
+m/GM3BDE7DFe5BBYb+vJS4/JHNDoSpk+jNezaf3hdx9+fh2DIoL84fs1FRRAl3Od
+6LWPAt3twOQLS0KsQh0GSIZ+zdJf3xvlZ4ixAaPB4iAF8bXYzvsODN3LRQIGhet2
+NzjD41f5IrAlG/qFiC6s/YLj1DWanLw2nTzSi4x3v0Gc4DEXPebB3KvaNEmqoKGP
+5aXa9IPbvzEVCX82qjeqCPYAsYVOBQnFEAcnkrQ76363oJTeTHxK7kgewS2YCVyy
+7wVinR8eyrs+3AWrZ5Op817HgxGvAVDGOEK+1OX9g1wt+IdxX00s85/T+Zk9RF6H
+wtRaD9li
+=LjIC
+-----END PGP PUBLIC KEY BLOCK-----

docs/_static/images/guides/search-analytics-demo.png

@@ -2,30 +2,42 @@ Sharing
 -------
 
 .. note::
-    This feature only exists on `Read the Docs for Business <https://readthedocs.com/>`_.
 
-You can share your project with users outside of your company.
-There are two ways to do this:
+   This feature only exists on `Read the Docs for Business <https://readthedocs.com/>`__.
+
+You can share your project with users outside of your company:
 
 * by sending them a *secret link*,
 * by giving them a *password*.
 
 These methods will allow them to view a specific project inside your company.
 
+Additionally, you can use a HTTP Authorization Header.
+This is useful to have access from a script.
+
 Enabling
 ~~~~~~~~
 
 * Go into your *Project Admin* page and to the *Sharing* menu.
-* Under the *Share with someone new* heading, select the way you prefer (secret link or password), add an expiration date and a *Description* so you remember who you're sharing it with.
+* Under the *Share with someone new* heading, select the way you prefer (secret link, password, or HTTP header token),
+  add an expiration date and a *Description* so you remember who you're sharing it with.
 * Click *Share!* to create.
 * Get the info needed to share your documentation with other users:
 
   * If you have selected secret link, copy the link that is generated
   * In case of password, copy the link and password
+  * For HTTP header token, you need to pass the ``Authorization`` header in your HTTP request.
 
 * Give that information to the person who you want to give access.
 
-.. note:: You can always revoke access in the same panel.
+.. note::
+   
+   You can always revoke access in the same panel.
+
+.. note::
+   
+   Sharing using a password and a HTTP header token are currently in beta.
+   If you want access to these features, email us to [email protected].
 
 Effects
 ~~~~~~~
@@ -40,6 +52,17 @@ Password
 ********
 
 Once the person you send the link to clicks on the link, they will see
-a *Authorization required* page asking them for the password you
+an *Authorization required* page asking them for the password you
 generated. When the user enters the password, they will have access to
 view your project.
+
+HTTP Authorization Header
+*************************
+
+You need to send the ``Authorization`` header with the token on each HTTP request.
+The header has the form ``Authorization: Token <ACCESS_TOKEN>``.
+For example:
+
+.. prompt:: bash $
+   
+   curl -H "Authorization: Token 19okmz5k0i6yk17jp70jlnv91v" https://docs.example.com/en/latest/example.html

docs/_static/security/pgpkey.txt

@@ -1,5 +1,5 @@
 Connecting Your Account
------------------------
+=======================
 
 If you are going to import repositories from GitHub, Bitbucket, or GitLab,
 you should connect your Read the Docs account to your repository host first.
@@ -18,3 +18,69 @@ and select `Connected Services <https://readthedocs.org/accounts/social/connecti
 From here, you'll be able to connect to your GitHub, Bitbucket or GitLab
 account. This process will ask you to authorize a connection to Read the Docs,
 that allows us to read information about and clone your repositories.
+
+
+Permissions for connected accounts
+----------------------------------
+
+Read the Docs does not generally ask for write permission to your repositories' code
+(with one exception detailed below)
+and since we only connect to public repositories we don't need special permissions to read them.
+However, we do need permissions for authorizing your account
+so that you can login to Read the Docs with your connected account credentials
+and to setup :doc:`webhooks`
+which allow us to build your documentation on every change to your repository.
+
+
+GitHub
+~~~~~~
+
+Read the Docs requests the following permissions (more precisely, `OAuth scopes`_)
+when connecting your Read the Docs account to GitHub.
+
+.. _OAuth scopes: https://developer.github.com/apps/building-oauth-apps/understanding-scopes-for-oauth-apps/
+
+Read access to your email address (``user:email``)
+    We ask for this so you can create a Read the Docs account and login with your GitHub credentials.
+
+Administering webhooks (``admin:repo_hook``)
+    We ask for this so we can create webhooks on your repositories when you import them into Read the Docs.
+    This allows us to build the docs when you push new commits.
+
+Read access to your organizations (``read:org``)
+    We ask for this so we know which organizations you have access to.
+    This allows you to filter repositories by organization when importing repositories.
+
+Repository status (``repo:status``)
+    Repository statuses allow Read the Docs to report the status
+    (eg. passed, failed, pending) of pull requests to GitHub.
+    This is used for a feature currently in beta testing
+    that builds documentation on each pull request similar to a continuous integration service.
+
+.. note::
+
+    :doc:`Read the Docs for Business </commercial/index>`
+    asks for one additional permission (``repo``) to allow access to private repositories
+    and to allow us to setup SSH keys to clone your private repositories.
+    Unfortunately, this is the permission for read/write control of the repository
+    but there isn't a more granular permission
+    that only allows setting up SSH keys for read access.
+
+
+Bitbucket
+~~~~~~~~~
+
+For similar reasons to those above for GitHub, we request permissions for:
+
+* Reading your account information including your email address
+* Read access to your team memberships
+* Read access to your repositories
+* Read and write access to webhooks
+
+GitLab
+~~~~~~
+
+Like the others, we request permissions for:
+
+* Reading your account information (``read_user``)
+* API access (``api``) which is needed to create webhooks in GitLab

docs/commercial/sharing.rst

@@ -94,25 +94,6 @@ This could include:
 * Taking a swagger YAML file and generating HTML properly with Sphinx
 * Integration with our existing API to generate Swagger output
 
-Autobuild docs for Pull Requests
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-It would be great to automatically build docs for Pull Requests in GitHub repos that our users have.
-Currently we don't support this,
-and it's one of our most requested features.
-
-This would include:
-
-* Modeling Pull Requests as a type of version alongside Tags and Branches
-* Modifying how we upload HTML docs to store them in a place like S3 for long term storage
-* Build integration with GitHub to send the status notifications when a PR is building and complete
-
-More info here: 
-
-* https://github.com/readthedocs/readthedocs.org/issues/1340
-* https://github.com/readthedocs/readthedocs.org/issues/2465
-
-
 Build a new Sphinx theme
 ~~~~~~~~~~~~~~~~~~~~~~~~
 

docs/connected-accounts.rst

@@ -0,0 +1,52 @@
+Autobuild Documentation for Pull Requests
+=========================================
+
+Read the Docs allows autobuilding documentation for pull/merge requests for GitHub or GitLab projects.
+This feature is currently available under a :doc:`Feature Flag </guides/feature-flags>`.
+So, you can enable this feature by sending us an `email <mailto:[email protected]>`__ including your project URL.
+
+Features
+========
+
+- **Build on Pull/Merge Request Event:** We create an external version and trigger a build for that version
+  when we receive pull/merge request open event from the webhook.
+  We also trigger a new build when a new commit has been pushed to the Pull/Merge Request.
+
+- **Warning Banner for Pull/Merge Request Documentation:** While building documentation for pull/merge requests
+  we add a warning banner at the top of those documentations to let the users know that
+  this documentation was generated from pull/merge requests and is not the main documentation for the project.
+
+- **Send Build Status Notification:** We send build status reports to the status API of the provider (e.g. GitHub, GitLab).
+  When a build is triggered for a pull/merge request we send build pending notification with the build URL
+  and after the build has finished we send success notification if the build succeeded without any error
+  or failure notification if the build failed.
+
+.. figure:: ../_static/images/guides/github-build-status-reporting.gif
+    :align: center
+    :alt: GitHub Build Status Reporting for Pull Requests.
+    :figwidth: 80%
+    :target: ../_static/images/guides/github-build-status-reporting.gif
+
+    GitHub Build Status Reporting for Pull Requests
+
+Troubleshooting
+===============
+
+After the feature is enabled on your project if everything does not work as expected,
+some common causes might be:
+
+#. Project repository should be from GitHub or GitLab. This feature is only available for GitHub or GitLab.
+
+#. Social Account (GitHub, Gitlab) is not connected with Read the Docs account.
+   If your project repository provider is GitHub or GitLab you need to make sure
+   that you Read the Docs account is connected with that providers social account.
+   You can check this by going to your `profile settings`_.
+
+#. Webhook is not properly setup. You need to make sure your webhook is properly setup
+   to handle events. You can setup or ``re-sync`` the webhook from you projects admin dashboard.
+   Learn more about setting up webhooks from our :doc:`Webhook Documentation </webhooks>`.
+
+If you have tried all the above troubleshooting and still getting issues,
+please let us know by sending us an `email <mailto:[email protected]>`__.
+
+.. _profile settings: https://readthedocs.org/accounts/social/connections/

docs/gsoc.rst

@@ -32,6 +32,7 @@ These guides will help you customize or tune aspects of the Read the Docs build
 .. toctree::
     :maxdepth: 1
 
+    autobuild-docs-for-pull-requests
     build-notifications
     build-using-too-many-resources
     technical-docs-seo-guide
@@ -40,6 +41,7 @@ These guides will help you customize or tune aspects of the Read the Docs build
     environment-variables
     feature-flags
     google-analytics
+    searching-with-readthedocs
     sitemaps
     specifying-dependencies
     wipe-environment