Docs: Refactor and simplify our docs

docs/.rstcheck.cfg

@@ -1,6 +1,7 @@
 [rstcheck]
 ignore_directives=automodule,http:get,tabs,tab,prompt,http:patch,http:post,http:put,http:delete
 ignore_roles=djangosetting,setting,featureflags,buildpyversions
+ignore_substitutions=org_brand,com_brand
 # This error needs to be ignored for now
 # See: https://github.com/myint/rstcheck/issues/19
 ignore_messages=(Hyperlink target ".*" is not referenced)

docs/_ext/djangodocs.py

@@ -4,3 +4,9 @@ def setup(app):
         rolename="setting",
         indextemplate="pair: %s; setting",
     )
+
+    return {
+        'version': 'builtin',
+        'parallel_read_safe': True,
+        'parallel_write_safe': True,
+    }

docs/about.rst

@@ -0,0 +1,27 @@
+About Read the Docs
+===================
+
+Read the Docs is a C Corporation registered in Oregon.
+Our bootstrapped company is owned and fully controlled by the founders,
+and fully funded by our customers and advertisers.
+This allows us to **focus 100% on our users**. 
+
+We have two main sources of revenue:
+
+* |com_brand| - where we `provide <https://readthedocs.com>`__ a valuable paid service to companies.
+* |org_brand| - where we provide a free service to the open source community, funded via :doc:`/advertising/ethical-advertising`.
+
+We believe that having both paying customers and ethical advertising is the best way to create a sustainable platform for our users.
+We have built something that we expect to last a long time,
+and we are able to make decisions based only on the best interest of our community and customers.
+
+All of the source code for Read the Docs is open source.
+You are welcome to :doc:`contribute </contribute>` the features you want or run your own instance.
+We should note that we generally only support our hosted versions as a matter of :doc:`our philosophy </open-source-philosophy>`.
+
+We owe a great deal to the open source community that we are a part of,
+so we provide free ads via our :ref:`community ads <advertising/ethical-advertising:Community Ads>` program.
+This allows us to give back to the communities and projects that we support and depend on.
+
+We are proud about the way we manage our company and products,
+and are glad to have you on board with us in this :doc:`great documentation journey </story>`.

docs/automatic-redirects.rst

@@ -4,29 +4,44 @@ Automatic Redirects
 Read the Docs supports redirecting certain URLs automatically.
 This is an overview of the set of redirects that are fully supported and will work into the future.
 
+Redirecting to a Page
+---------------------
+
+You can link to a specific page and have it redirect to your default version.
+This is done with the ``/page/`` URL prefix.
+You can reach this page by going to https://docs.readthedocs.io/page/automatic-redirects.html.
+
+This allows you to create links that are always up to date.
+
+Another way to handle this is the ``latest`` version.
+You can set your ``latest`` version to a specific version and just always link to ``latest``.
+You can read more about this in our :doc:`versions </versions>` page.
+
 Root URL
 --------
 
 A link to the root of your documentation will redirect to the *default version*,
 as set in your project settings.
 For example::
 
-    pip.readthedocs.io -> pip.readthedocs.io/en/latest/
-    www.pip-installer.org -> www.pip-installer.org/en/latest
+    docs.readthedocs.io -> docs.readthedocs.io/en/latest/
+    www.pip-installer.org -> www.pip-installer.org/en/latest/
 
 This only works for the root URL, not for internal pages. It's designed to redirect people from http://pip.readthedocs.io/ to the default version of your documentation, since serving up a 404 here would be a pretty terrible user experience. (If your "develop" branch was designated as your default version, then it would redirect to http://pip.readthedocs.io/en/develop.) But, it's not a universal redirecting solution. So, for example, a link to an internal page like http://pip.readthedocs.io/usage.html doesn't redirect to http://pip.readthedocs.io/en/latest/usage.html.
 
 The reasoning behind this is that RTD organizes the URLs for docs so that multiple translations and multiple versions of your docs can be organized logically and consistently for all projects that RTD hosts. For the way that RTD views docs, http://pip.readthedocs.io/en/latest/ is the root directory for your default documentation in English, not http://pip.readthedocs.io/. Just like http://pip.readthedocs.io/en/develop/ is the root for your development documentation in English.
 
 Among all the multiple versions of docs, you can choose which is the "default" version for RTD to display, which usually corresponds to the git branch of the most recent official release from your project.
 
-rtfd.org
+rtfd.io
 ~~~~~~~~
 
-Links to rtfd.org are treated the same way as above.
+Links to rtfd.io are treated the same way as above.
 They redirect the root URL to the default version of the project.
 They are intended to be easy and short for people to type.
 
+You can reach these docs at https://docs.rtfd.io.
+
 Supported Top-Level Redirects
 -----------------------------
 
@@ -43,19 +58,3 @@ The main challenge of URL routing in Read the Docs is handling redirects correct
 
 The language redirect will work for any of the defined ``LANGUAGE_CODES`` we support.
 The version redirect will work for supported versions.
-
-Redirecting to a Page
----------------------
-
-You can link to a specific page and have it redirect to your default version.
-This is done with the ``/page/`` URL.
-For example::
-
-    pip.readthedocs.io/page/quickstart.html -> pip.readthedocs.io/en/latest/quickstart.html
-    www.pip-installer.org/page/quickstart.html -> www.pip-installer.org/en/latest/quickstart.html
-
-This allows you to create links that are always up to date.
-
-Another way to handle this is the *latest* version.
-You can set your ``latest`` version to a specific version and just always link to latest.
-

docs/builds.rst

@@ -4,62 +4,29 @@ Build Process
 Every documentation build has limited resources.
 Our current build limits are:
 
-* 15 minutes of CPU
-* 3GB of RAM memory
-* 2 concurrent builds
+.. tabs::
 
-We can increase build limits on a per-project basis,
-sending an email to [email protected] providing a good reason why your documentation needs more resources.
-If your business is hitting build limits hosting documentation on Read the Docs,
-please consider :doc:`Read the Docs for Business </commercial/index>`
-which has much higher build resources.
+   .. tab:: |org_brand|
 
-You can see the current Docker build images that we use in our `docker repository <https://github.com/readthedocs/readthedocs-docker-images>`_.
-`Docker Hub <https://hub.docker.com/r/readthedocs/build/>`_ also shows the latest set of images that have been built.
-
-Currently in production we're using the ``readthedocs/build:latest`` docker image as our default image.
-
-How we build documentation
---------------------------
-
-When we import your documentation, we look at two things first: your *Repository URL* and the *Documentation Type*.
-We will clone your repository,
-and then build your documentation using the *Documentation Type* specified.
-
-Sphinx
-~~~~~~
+      * 15 minutes build time
+      * 3GB of memory
+      * 2 concurrent builds
 
-When you choose *Sphinx* as your *Documentation Type*,
-we will first look for a ``conf.py`` file in your repository.
-If we don't find one,
-we will generate one for you.
-We will look inside a ``doc`` or ``docs`` directory first,
-and then look within your entire project.
+      We can increase build limits on a per-project basis.
+      Send an email to [email protected] providing a good reason why your documentation needs more resources.
 
-Then Sphinx will build any files with an ``.rst`` extension.
+      If your business is hitting build limits hosting documentation on Read the Docs,
+      please consider :doc:`Read the Docs for Business </commercial/index>`
+      which has much higher build resources.
 
-MkDocs
-~~~~~~
+   .. tab:: |com_brand|
 
-When you choose *Mkdocs* as your *Documentation Type*,
-we will first look for a ``mkdocs.yml`` file in the root of your repository.
-If we don't find one,
-we will generate one for you.
-
-Then MkDocs will build any files with a ``.md`` extension within the directory specified as ``docs_dir`` in the ``mkdocs.yml``.
-
-If no ``mkdocs.yml`` was found in the root of your repository and we generated one for you,
-Read the Docs will attempt to set ``docs_dir`` by sequentially searching for a  ``docs``, ``doc``, ``Doc``, or ``book`` directory.
-The first of these directories that exists and contains files with a ``.md`` extension will be set to ``docs_dir`` within ``mkdocs.yml``,
-and MkDocs will build the ``.md`` files in that directory.
-As MkDocs doesn't support automatic PDF generation,
-Read the Docs cannot create a PDF version of your documentation with the *Mkdocs* option.
-
-.. warning::
-
-   We strongly recommend to :ref:`pin the MkDocs version <guides/specifying-dependencies:Specifying Dependencies>`
-   used for your project to build the docs to avoid potential future incompatibilities.
+      * 30 minutes build time
+      * 7GB of memory
+      * 4 concurrent builds
 
+      If you are having trouble with your documentation builds,
+      you can reach our support at [email protected].
 
 Understanding what's going on
 -----------------------------
@@ -70,35 +37,32 @@ It should also allow you to take advantage of certain things that happen during
 The first step of the process is that we check out your code from the repository you have given us.
 If the code is already checked out, we update the copy to the branch that you have specified in your project's configuration.
 
-Then we build the proper backend code for the type of documentation you've selected.
+Then we build the proper backend code for the type of documentation you've selected,
+this is done inside a :ref:`Docker container <builds:Docker images>`.
 
 At this point, if you need extra requirements,
 or even install your own package in the virtual environment to build your documentation,
 you can use a :doc:`config-file/index`.
 
-When we build your Sphinx documentation, we run ``sphinx-build -b html . _build/html``,
-where ``html`` would be replaced with the correct backend.
+When we build your Sphinx documentation, we run ``sphinx-build -b <format> . _build/<format>``
 We also create pdf's and ePub's automatically based on your project.
 For MkDocs, we run ``mkdocs build``.
 
-Then these files are copied across to our application servers from the build server.
-Once on the application servers, they are served from nginx.
+Once these files are built,
+they are deployed to our file hosting backend and will be visible to users.
 
 An example in code:
 
 .. code-block:: python
 
     update_docs_from_vcs(version)
     config = get_config(project)
-    if config.python.install.method.setuptools:
-        run('python setup.py install')
     if config.python.install.method.pip:
         run('pip install .')
     if config.python.install.requirement:
         run('pip install -r %s' % config.python.install.requirement)
-    build_docs(version=version)
-    copy_files(artifact_dir)
-
+    build_docs(version)
+    deploy_docs(version)
 
 .. note::
 
@@ -108,79 +72,36 @@ An example in code:
     :doc:`dependencies are specified <guides/specifying-dependencies>`.
     Some examples of pinning versions might be ``sphinx<2.0`` or ``mkdocs>=1.0``.
 
-Builder responsibility
-----------------------
-
-Builders have a very specific job.
-They take the updated source code and generate the correct artifacts.
-The code lives in ``self.version.project.checkout_path(self.version.slug)``.
-The artifacts should end up in ``self.version.project.artifact_path(version=self.version.slug, type=self.type)``
-Where ``type`` is the name of your builder.
-All files that end up in the artifact directory should be in their final form.
-
-The build environment
----------------------
-
-The build process is executed inside Docker containers,
-by default the image used is ``readthedocs/build:latest``,
-but you can change that using a :doc:`config-file/index`.
-
-.. note::
-
-   The Docker images have a select number of C libraries installed,
-   because they are used across a wide array of python projects.
-   We can't install every C library out there,
-   but we try and support the major ones.
-
-.. tip::
-
-   If you want to know the specific version of a package that is installed in the image
-   you can check the `Ubuntu package search page <https://packages.ubuntu.com/>`__.
-
-More details on software installed in images could be found by browsing specific branch in `rtfd/readthedocs-docker-images <https://github.com/readthedocs/readthedocs-docker-images>`__ repository.
-
-Writing your own builder
-------------------------
-
-.. note:: Builds happen on a server using only the RTD Public API. There is no reason that you couldn't build your own independent builder that wrote into the RTD namespace. The only thing that is currently unsupported there is a saner way than uploading the processed files as a zip.
-
-The documentation build system in RTD is made pluggable, so that you can build out your own backend. If you have a documentation format that isn't currently supported, you can add support by contributing a backend.
-
-`The builder backends`_ detail the higher level parts of the API that you need to implement. A basic run goes something like this:
-
-.. sourcecode:: python
-
-    backend = get_backend(project.documentation_type)
-    if force:
-        backend.force(version)
-    backend.clean(version)
-    backend.build(version)
-    if success:
-        backend.move(version)
-
-.. _The builder backends: https://github.com/readthedocs/readthedocs.org/tree/master/readthedocs/doc_builder/backends
-
-Deleting a stale or broken build environment
---------------------------------------------
-
-If you're having trouble getting your version to build, try wiping out the existing build/environment files.  On your version list page ``/projects/[project]/versions`` there is a "Wipe" button that will remove all of the files associated with your documentation build, but not the documentation itself.
-
 Build environment
 -----------------
 
 The *Sphinx* and *Mkdocs* builders set the following RTD-specific environment variables when building your documentation:
 
-.. csv-table::
-   :header-rows: 1
+.. csv-table:: Environment Variables
+   :header: Environment variable, Description, Example value
+   :widths: 15, 10, 30
+
+   ``READTHEDOCS``, Whether the build is running inside RTD, ``True``
+   ``READTHEDOCS_VERSION``, The RTD name of the version which is being built, ``latest``
+   ``READTHEDOCS_PROJECT``, The RTD slug of the project which is being built, ``my-example-project``
+   ``READTHEDOCS_LANGUAGE``, The RTD language slug of the project which is being built, ``en``
 
- Environment variable, Description, Example value
- ``READTHEDOCS``, Whether the build is running inside RTD, ``True``
- ``READTHEDOCS_VERSION``, The RTD name of the version which is being built, ``latest``
- ``READTHEDOCS_PROJECT``, The RTD slug of the project which is being built, ``my-example-project``
- ``READTHEDOCS_LANGUAGE``, The RTD language slug of the project which is being built, ``en``
+If you want to learn more about how the build environment works as a low level,
+you can read about it in our :doc:`/development/buildenvironments` docs.
 
 .. tip::
 
    In case extra environment variables are needed to the build process (like secrets, tokens, etc),
    you can add them going to :guilabel:`Admin` > :guilabel:`Environment Variables` in your project.
    See :doc:`guides/environment-variables`.
+
+
+Docker images
+-------------
+
+The build process is executed inside Docker containers,
+by default the image used is ``readthedocs/build:latest``,
+but you can change that using a :doc:`/config-file/index`.
+
+You can see the current Docker build images that we use in our `docker repository <https://github.com/readthedocs/readthedocs-docker-images>`_.
+`Docker Hub <https://hub.docker.com/r/readthedocs/build/>`_ also shows the latest set of images that have been built.

docs/conf.py

@@ -36,6 +36,7 @@ def get_version():
     'notfound.extension',
     'hoverxref.extension',
     'sphinx_search.extension',
+    'sphinxemoji.sphinxemoji',
 ]
 
 templates_path = ['_templates']
@@ -101,6 +102,10 @@ def get_version():
     'confval': 'tooltip',  # for custom object
 }
 
+rst_epilog = """
+.. |org_brand| replace:: Read the Docs Community
+.. |com_brand| replace:: Read the Docs for Business
+"""
 
 # Activate autosectionlabel plugin
 autosectionlabel_prefix_document = True
@@ -132,4 +137,4 @@ def get_version():
 
 
 def setup(app):
-    app.add_stylesheet('css/sphinx_prompt_css.css')
+    app.add_css_file('css/sphinx_prompt_css.css')

docs/config-file/index.rst

@@ -3,7 +3,7 @@ Configuration File
 
 In addition to using the admin panel of your project to configure your project,
 you can use a configuration file in the root of your project.
-The configuration file can be named as:
+The configuration file can be named:
 
 - ``readthedocs.yml``
 - ``readthedocs.yaml``
@@ -12,10 +12,10 @@ The configuration file can be named as:
 
 The main advantages of using a configuration file over the web interface are:
 
+- Settings are per version rather than per project.
+- Settings live in your VCS.
+- They enable reproducible build environments over time.
 - Some settings are only available using a configuration file
-- The settings are per version rather than per project.
-- The settings live in your VCS.
-- Reproducible build environments over time.
 
 .. tip::