From b14e129bdf4a4a71dcfa75dcecffc0385a3e6a2f Mon Sep 17 00:00:00 2001 From: Benjamin Bach Date: Thu, 26 Jan 2023 16:57:51 +0100 Subject: [PATCH 01/18] Updated reference for hosting features, adding more features and re-organizing content --- docs/user/hosting.rst | 167 ++++++++++++++++++++++--------- docs/user/index.rst | 1 - docs/user/reference/features.rst | 1 + 3 files changed, 120 insertions(+), 49 deletions(-) diff --git a/docs/user/hosting.rst b/docs/user/hosting.rst index 355b0ea8737..eede75c961e 100644 --- a/docs/user/hosting.rst +++ b/docs/user/hosting.rst @@ -1,28 +1,71 @@ -Documentation Hosting Features -============================== +.. old label +.. _Documentation Hosting Features: -The main way that users interact with your documentation is via the hosted HTML that we serve. -We support a number of important features that you would expect for a documentation host. +Hosting features provided by Read the Docs +========================================== -.. contents:: Contents - :local: - :depth: 1 +This article provides a quick overview and reference to how documentation is *hosted* on Read the Docs. +Hosting is optimized with documentation in mind and a number of special features are provided, +some of which may be customized. + +.. seealso:: + + :doc:`/reference/features` + For a full list of Read the Docs features, see the features reference page. + +Overview of hosting features +---------------------------- Subdomain support ------------------ + Every project has a subdomain that is available to serve its documentation based on it's :term:`slug`. + If you go to ``.readthedocs.io``, it should show you the latest version of your documentation, + for example https://docs.readthedocs.io. + For :doc:`/commercial/index` the subdomain looks like ``.readthedocs-hosted.com``. -Every project has a subdomain that is available to serve its documentation based on it's :term:`slug`. -If you go to ``.readthedocs.io``, it should show you the latest version of your documentation, -for example https://docs.readthedocs.io. -For :doc:`/commercial/index` the subdomain looks like ``.readthedocs-hosted.com``. +Custom domain support + Documentation projects may also define their own domains. + A project may define multiple domains, + as well as define which is the *canonical domain*. -.. seealso:: + ⏩️ :doc:`Read more ` + +Multiple documentation versions + The built-in URL structure of Read the Docs is pre-defined to allow for multiple versions and translations to be hosted. + You can also switch off this mechanism with :doc:`Single-Version projects `. - :doc:`/custom-domains` - More information on using a custom domain. + ⏩️ :doc:`Read more ` + +Redirect support + Projects may define their own custom URL redirects rules that trigger in the HTTP layer. + + ⏩️ :doc:`Read more ` Content Delivery Network (CDN) ------------------------------- + Documentation projects are by nature static HTML pages and assets. + Hence contents are delivered through a speedy distributed cloud proxy. + + ⬇️ `Read more `__ + +Sitemaps + Sitemaps are generated and hosted automatically, + improving search engine crawling. + + ⬇️ `Read more `__ + +Custom 404s + A 404 page is provided by default. It may be replaced with a projects own 404 page. + + ⬇️ `Read more `__ + +Automatic and custom robots.txt + `robots.txt`_ files allow you to customize how your documentation is indexed in search engines. + We automatically generate one for you, but you can also customize it. + + ⬇️ `Read more `__ + + +Content Delivery Network (CDN) and caching +------------------------------------------ A CDN is used for making documentation pages faster for your users. This is done by caching the documentation page content in multiple data centers around the world, @@ -59,27 +102,16 @@ We support CDNs on both of our sites: .. _Cloudflare: https://www.cloudflare.com/ -Sitemaps --------- - -`Sitemaps `__ allows us to inform search engines about URLs that are available for crawling -and communicate them additional information about each URL of the project: +Built-in content +---------------- -* when it was last updated, -* how often it changes, -* how important it is in relation to other URLs in the site, and -* what translations are available for a page. - -Read the Docs automatically generates a sitemap for each project that hosts -to improve results when performing a search on these search engines. -This allow us to prioritize results based on the version number, for example -to show ``stable`` as the top result followed by ``latest`` and then all the project's -versions sorted following `semantic versioning`_. +A number of content files can be said to be "built-in", +since they are automatically generated and hosted together with your site. -.. _semantic versioning: https://semver.org/ +They are also possible to customize. -Custom Not Found (404) Pages ----------------------------- +Custom Not Found (404) pages +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If you want your project to use a custom page for not found pages instead of the "Maze Found" default, you can put a ``404.html`` at the top level of your project's HTML output. @@ -91,17 +123,34 @@ and uses it if it exists. Otherwise, it tries to fall back to the ``404.html`` page corresponding to the *default* version of the project. -We recommend the `sphinx-notfound-page`_ extension, -which Read the Docs maintains. -It automatically creates a ``404.html`` page for your documentation, -matching the theme of your project. -See its documentation_ for how to install and customize it. +Sphinx and Mkdocs both have different ways of outputting static files in the build: + +.. tabs:: + + .. tab:: Sphinx + + We recommend the `sphinx-notfound-page`_ extension, + which Read the Docs maintains. + It automatically creates a ``404.html`` page for your documentation, + matching the theme of your project. + See its documentation_ for how to install and customize it. + + If you want to write the entire ``404.html`` from scratch, + Sphinx uses `html_extra_path`_ option to add static files to the output. + You need to create a ``404.html`` file and put it under the path defined in ``html_extra_path``. + + .. tab:: MkDocs + + MkDocs generates a ``404.html`` which Read the Docs will use. + However, assets will not be loaded correctly unless you define the `site_url`_ configuration value as your site's + :ref:`canonical base URL `. .. _sphinx-notfound-page: https://pypi.org/project/sphinx-notfound-page .. _documentation: https://sphinx-notfound-page.readthedocs.io/ +.. _site_url: https://www.mkdocs.org/user-guide/configuration/#site_url -Custom robots.txt Pages ------------------------ +Custom robots.txt +~~~~~~~~~~~~~~~~~ `robots.txt`_ files allow you to customize how your documentation is indexed in search engines. We automatically generate one for you, @@ -114,17 +163,39 @@ The **default version** is the best place to look for it. Sphinx and Mkdocs both have different ways of outputting static files in the build: -Sphinx -~~~~~~ +.. tabs:: + + .. tab:: Sphinx -Sphinx uses `html_extra_path`_ option to add static files to the output. -You need to create a ``robots.txt`` file and put it under the path defined in ``html_extra_path``. + Sphinx uses the `html_extra_path`_ configuration value to add static files to its final HTML output. + You need to create a ``robots.txt`` file and put it under the path defined in ``html_extra_path``. -MkDocs -~~~~~~ + .. tab:: MkDocs -MkDocs needs the ``robots.txt`` to be at the directory defined at `docs_dir`_ config. + MkDocs needs the ``robots.txt`` to be at the directory defined by the `docs_dir`_ configuration value. .. _robots.txt: https://developers.google.com/search/reference/robots_txt .. _html_extra_path: https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-html_extra_path .. _docs_dir: https://www.mkdocs.org/user-guide/configuration/#docs_dir + +Sitemaps +~~~~~~~~ + +`Sitemaps `__ allows us to inform search engines about URLs that are available for crawling +and communicate them additional information about each URL of the project: + +* When it was last updated. +* How often it changes. +* How important it is in relation to other URLs in the site. +* What translations are available for a page. + +Read the Docs automatically generates a sitemap for each project that hosts +to improve results when performing a search on these search engines. +This allow us to prioritize results based on the version number, for example +to show ``stable`` as the top result followed by ``latest`` and then all the project's +versions sorted following `semantic versioning`_. + +If you need a custom sitemap, please let us know in `GitHub issue #5391`_. + +.. _semantic versioning: https://semver.org/ +.. _GitHub issue #5391: https://github.com/readthedocs/readthedocs.org/issues/5391 diff --git a/docs/user/index.rst b/docs/user/index.rst index 667b24ae713..3d121527389 100644 --- a/docs/user/index.rst +++ b/docs/user/index.rst @@ -154,7 +154,6 @@ and some of the core features of Read the Docs. /config-file/index /integrations /versions - /hosting /security-log /builds diff --git a/docs/user/reference/features.rst b/docs/user/reference/features.rst index 73c79ddc212..c1424f5a6ad 100644 --- a/docs/user/reference/features.rst +++ b/docs/user/reference/features.rst @@ -8,3 +8,4 @@ Features analytics /server-side-search/index + Special hosting features From 3c5a5695c49e6a79e1f916cb55e1d87abb9e6732 Mon Sep 17 00:00:00 2001 From: Benjamin Balder Bach Date: Wed, 1 Feb 2023 11:38:52 +0100 Subject: [PATCH 02/18] Apply suggestions from code review @ericholscher Co-authored-by: Eric Holscher <25510+ericholscher@users.noreply.github.com> --- docs/user/hosting.rst | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/docs/user/hosting.rst b/docs/user/hosting.rst index eede75c961e..7ea185b28cd 100644 --- a/docs/user/hosting.rst +++ b/docs/user/hosting.rst @@ -10,8 +10,8 @@ some of which may be customized. .. seealso:: - :doc:`/reference/features` - For a full list of Read the Docs features, see the features reference page. + :doc:`Feature reference ` + A complete list of our features. Overview of hosting features ---------------------------- @@ -23,15 +23,17 @@ Subdomain support For :doc:`/commercial/index` the subdomain looks like ``.readthedocs-hosted.com``. Custom domain support - Documentation projects may also define their own domains. + Documentation projects can also use their own domains. A project may define multiple domains, - as well as define which is the *canonical domain*. + as well as define the *canonical domain* where all other domains will redirect. ⏩️ :doc:`Read more ` Multiple documentation versions - The built-in URL structure of Read the Docs is pre-defined to allow for multiple versions and translations to be hosted. - You can also switch off this mechanism with :doc:`Single-Version projects `. + We allow for multiple versions and translations to be hosted, + integrated nicely into the URL of your documentation. + If you only have 1 version and translation, + we support :doc:`single version projects `. ⏩️ :doc:`Read more ` From 78ac176403c0074ba9b6376ec8d901351b161478 Mon Sep 17 00:00:00 2001 From: Benjamin Bach Date: Wed, 1 Feb 2023 11:49:20 +0100 Subject: [PATCH 03/18] Update name of Feature reference index to avoid explicit labels elsewhere --- docs/user/hosting.rst | 2 +- docs/user/reference/features.rst | 6 +++--- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/user/hosting.rst b/docs/user/hosting.rst index 7ea185b28cd..3de390e7d0c 100644 --- a/docs/user/hosting.rst +++ b/docs/user/hosting.rst @@ -10,7 +10,7 @@ some of which may be customized. .. seealso:: - :doc:`Feature reference ` + :doc:`/reference/features` A complete list of our features. Overview of hosting features diff --git a/docs/user/reference/features.rst b/docs/user/reference/features.rst index 7fa3c2936bd..7fde6a34c67 100644 --- a/docs/user/reference/features.rst +++ b/docs/user/reference/features.rst @@ -1,6 +1,6 @@ -======== -Features -======== +================= +Feature reference +================= .. The TOC here will be refactored once we reorganize the files in docs/user/. From e11f18087863c5bf1b7680477cedeeba6a20031f Mon Sep 17 00:00:00 2001 From: Benjamin Bach Date: Wed, 1 Feb 2023 13:24:32 +0100 Subject: [PATCH 04/18] Fix misalignments --- docs/user/hosting.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/user/hosting.rst b/docs/user/hosting.rst index 3de390e7d0c..188ae710168 100644 --- a/docs/user/hosting.rst +++ b/docs/user/hosting.rst @@ -30,8 +30,8 @@ Custom domain support ⏩️ :doc:`Read more ` Multiple documentation versions - We allow for multiple versions and translations to be hosted, - integrated nicely into the URL of your documentation. + We allow for multiple versions and translations to be hosted, + integrated nicely into the URL of your documentation. If you only have 1 version and translation, we support :doc:`single version projects `. From 9c38f276cc5a15c08855d39cb16c3e245a1927a5 Mon Sep 17 00:00:00 2001 From: Benjamin Bach Date: Wed, 1 Feb 2023 13:27:45 +0100 Subject: [PATCH 05/18] Redo in-page references --- docs/user/hosting.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/user/hosting.rst b/docs/user/hosting.rst index 188ae710168..22b10eb7845 100644 --- a/docs/user/hosting.rst +++ b/docs/user/hosting.rst @@ -46,24 +46,24 @@ Content Delivery Network (CDN) Documentation projects are by nature static HTML pages and assets. Hence contents are delivered through a speedy distributed cloud proxy. - ⬇️ `Read more `__ + ⬇️ :ref:`Read more ` Sitemaps Sitemaps are generated and hosted automatically, improving search engine crawling. - ⬇️ `Read more `__ + ⬇️ :ref:`Read more ` Custom 404s A 404 page is provided by default. It may be replaced with a projects own 404 page. - ⬇️ `Read more `__ + ⬇️ :ref:`Read more ` Automatic and custom robots.txt `robots.txt`_ files allow you to customize how your documentation is indexed in search engines. We automatically generate one for you, but you can also customize it. - ⬇️ `Read more `__ + ⬇️ :ref:`Read more ` Content Delivery Network (CDN) and caching From a5323b63c6135ac8ecc3bed7e142e5bfe10af4f8 Mon Sep 17 00:00:00 2001 From: Benjamin Bach Date: Wed, 1 Feb 2023 13:48:51 +0100 Subject: [PATCH 06/18] Update "read more" link for "multiple documentation versions" --- docs/user/hosting.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user/hosting.rst b/docs/user/hosting.rst index 22b10eb7845..e0fcd6ccdde 100644 --- a/docs/user/hosting.rst +++ b/docs/user/hosting.rst @@ -35,7 +35,7 @@ Multiple documentation versions If you only have 1 version and translation, we support :doc:`single version projects `. - ⏩️ :doc:`Read more ` + ⏩️ :doc:`Read more ` Redirect support Projects may define their own custom URL redirects rules that trigger in the HTTP layer. From c3ac6de26ff179d071f7edadc648b1a3c59a10b2 Mon Sep 17 00:00:00 2001 From: Benjamin Bach Date: Wed, 1 Feb 2023 13:57:46 +0100 Subject: [PATCH 07/18] Add "private documentation" to hosting feature list --- docs/user/hosting.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/user/hosting.rst b/docs/user/hosting.rst index e0fcd6ccdde..f31c789c5c9 100644 --- a/docs/user/hosting.rst +++ b/docs/user/hosting.rst @@ -65,6 +65,11 @@ Automatic and custom robots.txt ⬇️ :ref:`Read more ` +Private documentation + It is possible to host private documentation on Read the Docs for Business. + + ⬇️ :doc:`Read more ` + Content Delivery Network (CDN) and caching ------------------------------------------ From e885bc7c3726e4ea31938e78fce46f58f8ba73e1 Mon Sep 17 00:00:00 2001 From: Benjamin Bach Date: Wed, 1 Feb 2023 14:07:04 +0100 Subject: [PATCH 08/18] Revert to old title, modify introduction with less passive voice --- docs/user/hosting.rst | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-) diff --git a/docs/user/hosting.rst b/docs/user/hosting.rst index f31c789c5c9..4df547efde7 100644 --- a/docs/user/hosting.rst +++ b/docs/user/hosting.rst @@ -1,12 +1,9 @@ -.. old label -.. _Documentation Hosting Features: +Documentation hosting features +============================== -Hosting features provided by Read the Docs -========================================== - -This article provides a quick overview and reference to how documentation is *hosted* on Read the Docs. -Hosting is optimized with documentation in mind and a number of special features are provided, -some of which may be customized. +This page provides a quick overview of the *hosting features* on Read the Docs. +These features are designed especially with documentation in mind, +and several of the features can be customized for specific project needs. .. seealso:: @@ -68,7 +65,7 @@ Automatic and custom robots.txt Private documentation It is possible to host private documentation on Read the Docs for Business. - ⬇️ :doc:`Read more ` + ⏩️ :doc:`Read more ` Content Delivery Network (CDN) and caching From 0a36b8bf5c7d04c7b4ae156a86d0500c0efb95fc Mon Sep 17 00:00:00 2001 From: Benjamin Bach Date: Wed, 1 Feb 2023 14:13:42 +0100 Subject: [PATCH 09/18] Dot the list sentences, don't say "bust" but "invalidate and refresh" --- docs/user/hosting.rst | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/docs/user/hosting.rst b/docs/user/hosting.rst index 4df547efde7..b741325ac5f 100644 --- a/docs/user/hosting.rst +++ b/docs/user/hosting.rst @@ -85,11 +85,11 @@ We support CDNs on both of our sites: we are able to provide a CDN to all the projects that we host. This service is graciously sponsored by `Cloudflare`_. - We bust the cache on the CDN when the following actions happen: + We invalidate and refresh the cache on the CDN when the following actions happen: - * Your Project is saved - * Your Domain is saved - * A new version is built + * Your Project is saved. + * Your Domain is saved. + * A new version is built. .. tab:: |com_brand| @@ -100,19 +100,17 @@ We support CDNs on both of our sites: We invalidate and refresh the cache on the CDN when the following actions happen: - * Your project is saved - * Your domain is saved - * A version or branch is built + * Your project is saved. + * Your domain is saved. + * A version or branch is built. .. _Cloudflare: https://www.cloudflare.com/ Built-in content ---------------- -A number of content files can be said to be "built-in", -since they are automatically generated and hosted together with your site. - -They are also possible to customize. +A number of content files are automatically generated and hosted together with your site. +You can read more about these files and how to customize them in this section. Custom Not Found (404) pages ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ From 4d8dd32b4d5a9c0aafb631bc1c5b5a7514c76985 Mon Sep 17 00:00:00 2001 From: Eric Holscher Date: Wed, 1 Feb 2023 16:08:35 -0800 Subject: [PATCH 10/18] Refactor of feature section --- docs/user/hosting.rst | 43 ++++++++++++++++++++++--------------------- 1 file changed, 22 insertions(+), 21 deletions(-) diff --git a/docs/user/hosting.rst b/docs/user/hosting.rst index b741325ac5f..041575366cf 100644 --- a/docs/user/hosting.rst +++ b/docs/user/hosting.rst @@ -2,8 +2,8 @@ Documentation hosting features ============================== This page provides a quick overview of the *hosting features* on Read the Docs. -These features are designed especially with documentation in mind, -and several of the features can be customized for specific project needs. +These features are designed with documentation in mind, +and provide a good experience for your users. .. seealso:: @@ -13,57 +13,58 @@ and several of the features can be customized for specific project needs. Overview of hosting features ---------------------------- -Subdomain support - Every project has a subdomain that is available to serve its documentation based on it's :term:`slug`. - If you go to ``.readthedocs.io``, it should show you the latest version of your documentation, - for example https://docs.readthedocs.io. - For :doc:`/commercial/index` the subdomain looks like ``.readthedocs-hosted.com``. - Custom domain support - Documentation projects can also use their own domains. + Documentation projects can use their own domain name. A project may define multiple domains, as well as define the *canonical domain* where all other domains will redirect. ⏩️ :doc:`Read more ` Multiple documentation versions - We allow for multiple versions and translations to be hosted, + We support multiple versions and translations, integrated nicely into the URL of your documentation. + This is served at ``/en/latest/`` by default. If you only have 1 version and translation, - we support :doc:`single version projects `. + we also support :doc:`single version projects ` served at ``/``. ⏩️ :doc:`Read more ` -Redirect support - Projects may define their own custom URL redirects rules that trigger in the HTTP layer. +Custom URL redirects + Projects may define their own custom URL redirects, + with advanced functionality like folder redirects. ⏩️ :doc:`Read more ` Content Delivery Network (CDN) - Documentation projects are by nature static HTML pages and assets. - Hence contents are delivered through a speedy distributed cloud proxy. + Documentation projects are primarily static HTML pages along with media files. + This allows us to cache them with our CDN, + making them *load faster* for your users. ⬇️ :ref:`Read more ` Sitemaps Sitemaps are generated and hosted automatically, - improving search engine crawling. + improving search engine optimization. + This helps your users find content more effectively on your site. ⬇️ :ref:`Read more ` -Custom 404s - A 404 page is provided by default. It may be replaced with a projects own 404 page. +Custom ``404s Not Found`` pages + A 404 page is served when we can't find a page on your site. + We provide a default 404 page, + but you can also customize it. ⬇️ :ref:`Read more ` -Automatic and custom robots.txt +Custom robots.txt `robots.txt`_ files allow you to customize how your documentation is indexed in search engines. - We automatically generate one for you, but you can also customize it. + We provide a default robots.txt file, + but you can also customize it. ⬇️ :ref:`Read more ` Private documentation - It is possible to host private documentation on Read the Docs for Business. + It is possible to host private and password protected documentation on Read the Docs for Business. ⏩️ :doc:`Read more ` From 2022d5b9089342f7b687a30ce1b4fb4d884e9261 Mon Sep 17 00:00:00 2001 From: Eric Holscher Date: Thu, 9 Feb 2023 17:15:55 -0800 Subject: [PATCH 11/18] Initial refactor of features into dedicated pages. Ended up writing a bunch of additional content here, since these features were barely documented... This will be a big win for our users. --- docs/user/hosting.rst | 135 ----------------------------- docs/user/reference/cdn.rst | 39 +++++++++ docs/user/reference/custom-404.rst | 47 ++++++++++ docs/user/reference/features.rst | 65 ++++++++++++++ docs/user/reference/robots.rst | 52 +++++++++++ docs/user/reference/sitemaps.rst | 38 ++++++++ 6 files changed, 241 insertions(+), 135 deletions(-) create mode 100644 docs/user/reference/cdn.rst create mode 100644 docs/user/reference/custom-404.rst create mode 100644 docs/user/reference/robots.rst create mode 100644 docs/user/reference/sitemaps.rst diff --git a/docs/user/hosting.rst b/docs/user/hosting.rst index 041575366cf..9c87871fefd 100644 --- a/docs/user/hosting.rst +++ b/docs/user/hosting.rst @@ -67,138 +67,3 @@ Private documentation It is possible to host private and password protected documentation on Read the Docs for Business. ⏩️ :doc:`Read more ` - - -Content Delivery Network (CDN) and caching ------------------------------------------- - -A CDN is used for making documentation pages faster for your users. -This is done by caching the documentation page content in multiple data centers around the world, -and then serving docs from the data center closest to the user. - -We support CDNs on both of our sites: - -.. tabs:: - - .. tab:: |org_brand| - - On |org_brand|, - we are able to provide a CDN to all the projects that we host. - This service is graciously sponsored by `Cloudflare`_. - - We invalidate and refresh the cache on the CDN when the following actions happen: - - * Your Project is saved. - * Your Domain is saved. - * A new version is built. - - - .. tab:: |com_brand| - - On |com_brand|, - we offer a CDN as part of our **Pro plan** and above. - Please contact support@readthedocs.com to discuss how we can enable this for you. - - We invalidate and refresh the cache on the CDN when the following actions happen: - - * Your project is saved. - * Your domain is saved. - * A version or branch is built. - -.. _Cloudflare: https://www.cloudflare.com/ - -Built-in content ----------------- - -A number of content files are automatically generated and hosted together with your site. -You can read more about these files and how to customize them in this section. - -Custom Not Found (404) pages -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -If you want your project to use a custom page for not found pages instead of the "Maze Found" default, -you can put a ``404.html`` at the top level of your project's HTML output. - -When a 404 is returned, -Read the Docs checks if there is a ``404.html`` in the root of your project's output -corresponding to the *current* version -and uses it if it exists. -Otherwise, it tries to fall back to the ``404.html`` page -corresponding to the *default* version of the project. - -Sphinx and Mkdocs both have different ways of outputting static files in the build: - -.. tabs:: - - .. tab:: Sphinx - - We recommend the `sphinx-notfound-page`_ extension, - which Read the Docs maintains. - It automatically creates a ``404.html`` page for your documentation, - matching the theme of your project. - See its documentation_ for how to install and customize it. - - If you want to write the entire ``404.html`` from scratch, - Sphinx uses `html_extra_path`_ option to add static files to the output. - You need to create a ``404.html`` file and put it under the path defined in ``html_extra_path``. - - .. tab:: MkDocs - - MkDocs generates a ``404.html`` which Read the Docs will use. - However, assets will not be loaded correctly unless you define the `site_url`_ configuration value as your site's - :ref:`canonical base URL `. - -.. _sphinx-notfound-page: https://pypi.org/project/sphinx-notfound-page -.. _documentation: https://sphinx-notfound-page.readthedocs.io/ -.. _site_url: https://www.mkdocs.org/user-guide/configuration/#site_url - -Custom robots.txt -~~~~~~~~~~~~~~~~~ - -`robots.txt`_ files allow you to customize how your documentation is indexed in search engines. -We automatically generate one for you, -which automatically hides versions which are set to :ref:`versions:Hidden`. - -The ``robots.txt`` file will be served from the **default version** of your Project. -This is because the ``robots.txt`` file is served at the top-level of your domain, -so we must choose a version to find the file in. -The **default version** is the best place to look for it. - -Sphinx and Mkdocs both have different ways of outputting static files in the build: - -.. tabs:: - - .. tab:: Sphinx - - Sphinx uses the `html_extra_path`_ configuration value to add static files to its final HTML output. - You need to create a ``robots.txt`` file and put it under the path defined in ``html_extra_path``. - - .. tab:: MkDocs - - MkDocs needs the ``robots.txt`` to be at the directory defined by the `docs_dir`_ configuration value. - -.. _robots.txt: https://developers.google.com/search/reference/robots_txt -.. _html_extra_path: https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-html_extra_path -.. _docs_dir: https://www.mkdocs.org/user-guide/configuration/#docs_dir - -Sitemaps -~~~~~~~~ - -`Sitemaps `__ allows us to inform search engines about URLs that are available for crawling -and communicate them additional information about each URL of the project: - -* When it was last updated. -* How often it changes. -* How important it is in relation to other URLs in the site. -* What translations are available for a page. - -Read the Docs automatically generates a sitemap for each project that hosts -to improve results when performing a search on these search engines. -This allow us to prioritize results based on the version number, for example -to show ``stable`` as the top result followed by ``latest`` and then all the project's -versions sorted following `semantic versioning`_. - -If you need a custom sitemap, please let us know in `GitHub issue #5391`_. - -.. _semantic versioning: https://semver.org/ -.. _GitHub issue #5391: https://github.com/readthedocs/readthedocs.org/issues/5391 diff --git a/docs/user/reference/cdn.rst b/docs/user/reference/cdn.rst new file mode 100644 index 00000000000..579d31e24b5 --- /dev/null +++ b/docs/user/reference/cdn.rst @@ -0,0 +1,39 @@ +Content Delivery Network (CDN) and caching +========================================== + +A CDN is used for making documentation pages *fast* for your users. +This is done by caching the documentation page content in multiple data centers around the world, +and then serving docs from the data center closest to the user. + +We support CDNs on both of our sites: + +* On |org_brand|, + we are able to provide a CDN to all the projects that we host. + This service is graciously sponsored by `Cloudflare`_. +* On |com_brand|, + the CDN is included as part of our all of our plans. + We use `Cloudflare`_ for this as well. + +CDN Benefits +------------ + +Having a CDN in front of your documentation has many benefits: + +* **Improved reliability**: Since docs are served from multiple places, one can go down and the docs are still accessible. +* **Improved performance**: Data takes time to travel across space, so connecting to a server closer to the user makes documentation load faster. + +CDN Features +------------ + +Our integration between building and hosting documentation allows to do many smart things to make the experience seemless. + +We automatically refresh the cache on the CDN when the following actions happen: + +* Your project is saved. +* Your domain is saved. +* A new version of your documentation is built. + +This means that you should **never see outdated content**, +but it's always as fast a possible for your users. + +.. _Cloudflare: https://www.cloudflare.com/ diff --git a/docs/user/reference/custom-404.rst b/docs/user/reference/custom-404.rst new file mode 100644 index 00000000000..473443ea88c --- /dev/null +++ b/docs/user/reference/custom-404.rst @@ -0,0 +1,47 @@ +Custom ``404 Not Found`` pages +============================== + +If you want your project to use a custom or branded ``404 Not Found`` page, +you can put a ``404.html`` at the top level of your project's HTML output. + +How it works +------------ + +When our servers return a ``404 Not Found`` error, +we check if there is a ``404.html`` in the root of your project's output. + +The following locations are checked, in order: + +* ``/404.html`` in the *current* documentation version. +* ``/404.html`` in the *default* documentation version. + +Tool Integration +---------------- + +Documentation tools will have different ways of generating a ``404.html`` file. +We have examples for some of the most popular tools below. + +.. tabs:: + + .. tab:: Sphinx + + We recommend the `sphinx-notfound-page`_ extension, + which Read the Docs maintains. + It automatically creates a ``404.html`` page for your documentation, + matching the theme of your project. + See its documentation_ for how to install and customize it. + + If you want to create a custom ``404.html``, + Sphinx uses `html_extra_path`_ option to add static files to the output. + You need to create a ``404.html`` file and put it under the path defined in ``html_extra_path``. + + .. tab:: MkDocs + + MkDocs automatically generates a ``404.html`` which Read the Docs will use. + However, assets will not be loaded correctly unless you define the `site_url`_ configuration value as your site's + :ref:`canonical base URL `. + +.. _sphinx-notfound-page: https://pypi.org/project/sphinx-notfound-page +.. _html_extra_path: https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-html_extra_path +.. _documentation: https://sphinx-notfound-page.readthedocs.io/ +.. _site_url: https://www.mkdocs.org/user-guide/configuration/#site_url diff --git a/docs/user/reference/features.rst b/docs/user/reference/features.rst index 7fde6a34c67..1634b963c66 100644 --- a/docs/user/reference/features.rst +++ b/docs/user/reference/features.rst @@ -2,6 +2,65 @@ Feature reference ================= +.. TODO: Continue to add more features here. + +Hosting features +---------------- + +Custom domain support + Documentation projects can use their own domain name. + A project may define multiple domains, + as well as define the *canonical domain* where all other domains will redirect. + + ⏩️ :doc:`Read more ` + +Multiple documentation versions + We support multiple versions and translations, + integrated nicely into the URL of your documentation. + This is served at ``/en/latest/`` by default. + If you only have 1 version and translation, + we also support :doc:`single version projects ` served at ``/``. + + ⏩️ :doc:`Read more ` + +Custom URL redirects + Projects may define their own custom URL redirects, + with advanced functionality like folder redirects. + + ⏩️ :doc:`Read more ` + +Content Delivery Network (CDN) + Documentation projects are primarily static HTML pages along with media files. + This allows us to cache them with our CDN, + making them *load faster* for your users. + + ⬇️ :ref:`Read more ` + +Sitemaps + Sitemaps are generated and hosted automatically, + improving search engine optimization. + This helps your users find content more effectively on your site. + + ⬇️ :ref:`Read more ` + +Custom ``404s Not Found`` pages + A 404 page is served when we can't find a page on your site. + We provide a default 404 page, + but you can also customize it. + + ⬇️ :ref:`Read more ` + +Custom robots.txt + `robots.txt`_ files allow you to customize how your documentation is indexed in search engines. + We provide a default robots.txt file, + but you can also customize it. + + ⬇️ :ref:`Read more ` + +Private documentation + It is possible to host private and password protected documentation on Read the Docs for Business. + + ⏩️ :doc:`Read more ` .. The TOC here will be refactored once we reorganize the files in docs/user/. .. Probably, all feature reference should be in this directory! @@ -9,6 +68,8 @@ Feature reference .. In fact, this whole page will become a proper index page with more explanation of what sort of reference can .. be found. + + .. toctree:: :maxdepth: 1 @@ -18,3 +79,7 @@ Feature reference /server-side-search/index Special hosting features /badges + /reference/cdn + /reference/custom-404 + /reference/robots + /reference/sitemaps diff --git a/docs/user/reference/robots.rst b/docs/user/reference/robots.rst new file mode 100644 index 00000000000..49d83abb28a --- /dev/null +++ b/docs/user/reference/robots.rst @@ -0,0 +1,52 @@ +``robots.txt`` support +====================== + +The `robots.txt`_ files allow you to customize how your documentation is indexed in search engines. +It's useful for: + +* Hiding various pages from search engines +* Disabling certain web crawlers from accessing your documentation +* Disallowing any indexing of your documentation + +Read the Docs automatically generates one for you with a configuration that works for most projects. +By default, the automatically created ``robots.txt``:abbr: + +* Hides versions which are set to :ref:`versions:Hidden` from being indexed. +* Allows indexing of all other versions. + +.. warning:: ``robots.txt`` files are respected by search engines, but they are not a guarantee that your documentation will not be indexed. + Search engines may choose to ignore your ``robots.txt`` file and index your documentation anyway. + +How it works +------------ + +You can customize this file to add more rules to it. +The ``robots.txt`` file will be served from the **default version** of your project. +This is because the ``robots.txt`` file is served at the top-level of your domain, +so we must choose a version to find the file in. +The **default version** is the best place to look for it. + +Tool Integration +---------------- + +Documentation tools will have different ways of generating a ``robots.txt`` file. +We have examples for some of the most popular tools below. + +.. tabs:: + + .. tab:: Sphinx + + Sphinx uses the `html_extra_path`_ configuration value to add static files to its final HTML output. + You need to create a ``robots.txt`` file and put it under the path defined in ``html_extra_path``. + + .. tab:: MkDocs + + MkDocs needs the ``robots.txt`` to be at the directory defined by the `docs_dir`_ configuration value. + +.. _robots.txt: https://developers.google.com/search/reference/robots_txt +.. _html_extra_path: https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-html_extra_path +.. _docs_dir: https://www.mkdocs.org/user-guide/configuration/#docs_dir + +.. seealso:: + + :doc:`/guides/technical-docs-seo-guide` diff --git a/docs/user/reference/sitemaps.rst b/docs/user/reference/sitemaps.rst new file mode 100644 index 00000000000..23d938e32cf --- /dev/null +++ b/docs/user/reference/sitemaps.rst @@ -0,0 +1,38 @@ +Sitemaps +======== + +`Sitemaps `__ allow you to inform search engines about URLs that are available for crawling. +This helps make your content more discoverable, +and improve your Search Engine Optimization (SEO). + +How it works +------------ + +The ``sitemap.xml`` file is read by search engines in order to index your documentation. +It contains information such as: + +* When a URL was last updated. +* How often that URL changes. +* How important this URL is in relation to other URLs in the site. +* What translations are available for a page. + +Read the Docs automatically generates a sitemap for your project, +to improve results when performing a search on these search engines. + +By default our Sitemap includes: + +* Each version of your documentation and when it was last updated, sorted by version number. + +This allows search engines to prioritize results based on the version number, +sorted by `semantic versioning`_. + +Custom ``sitemap.xml`` +---------------------- + +If you need a custom sitemap, +please let us know by contacting :doc:`/support`. +It's on our long term roadmap, +but hearing that users want it is a great way to have us prioritize it. + +.. _semantic versioning: https://semver.org/ +.. _GitHub issue #5391: https://github.com/readthedocs/readthedocs.org/issues/5391 From 6f71b6419a71ae735e2015c9b16403bfe33bcdbb Mon Sep 17 00:00:00 2001 From: Eric Holscher Date: Thu, 9 Feb 2023 17:22:26 -0800 Subject: [PATCH 12/18] More refactoring --- docs/user/hosting.rst | 69 ------------------- docs/user/index.rst | 1 - .../{custom-404.rst => 404-not-found.rst} | 4 +- docs/user/reference/features.rst | 24 +++---- docs/user/reference/sitemaps.rst | 4 +- 5 files changed, 15 insertions(+), 87 deletions(-) delete mode 100644 docs/user/hosting.rst rename docs/user/reference/{custom-404.rst => 404-not-found.rst} (96%) diff --git a/docs/user/hosting.rst b/docs/user/hosting.rst deleted file mode 100644 index 9c87871fefd..00000000000 --- a/docs/user/hosting.rst +++ /dev/null @@ -1,69 +0,0 @@ -Documentation hosting features -============================== - -This page provides a quick overview of the *hosting features* on Read the Docs. -These features are designed with documentation in mind, -and provide a good experience for your users. - -.. seealso:: - - :doc:`/reference/features` - A complete list of our features. - -Overview of hosting features ----------------------------- - -Custom domain support - Documentation projects can use their own domain name. - A project may define multiple domains, - as well as define the *canonical domain* where all other domains will redirect. - - ⏩️ :doc:`Read more ` - -Multiple documentation versions - We support multiple versions and translations, - integrated nicely into the URL of your documentation. - This is served at ``/en/latest/`` by default. - If you only have 1 version and translation, - we also support :doc:`single version projects ` served at ``/``. - - ⏩️ :doc:`Read more ` - -Custom URL redirects - Projects may define their own custom URL redirects, - with advanced functionality like folder redirects. - - ⏩️ :doc:`Read more ` - -Content Delivery Network (CDN) - Documentation projects are primarily static HTML pages along with media files. - This allows us to cache them with our CDN, - making them *load faster* for your users. - - ⬇️ :ref:`Read more ` - -Sitemaps - Sitemaps are generated and hosted automatically, - improving search engine optimization. - This helps your users find content more effectively on your site. - - ⬇️ :ref:`Read more ` - -Custom ``404s Not Found`` pages - A 404 page is served when we can't find a page on your site. - We provide a default 404 page, - but you can also customize it. - - ⬇️ :ref:`Read more ` - -Custom robots.txt - `robots.txt`_ files allow you to customize how your documentation is indexed in search engines. - We provide a default robots.txt file, - but you can also customize it. - - ⬇️ :ref:`Read more ` - -Private documentation - It is possible to host private and password protected documentation on Read the Docs for Business. - - ⏩️ :doc:`Read more ` diff --git a/docs/user/index.rst b/docs/user/index.rst index bcb09978db1..a07358d571e 100644 --- a/docs/user/index.rst +++ b/docs/user/index.rst @@ -125,7 +125,6 @@ and some of the core features of Read the Docs. :doc:`/custom-domains` | :doc:`/versions` | :doc:`/downloadable-documentation` | - :doc:`/hosting` | :doc:`/server-side-search/index` | :doc:`/analytics` | :doc:`/pull-requests` | diff --git a/docs/user/reference/custom-404.rst b/docs/user/reference/404-not-found.rst similarity index 96% rename from docs/user/reference/custom-404.rst rename to docs/user/reference/404-not-found.rst index 473443ea88c..5bc9a4500d1 100644 --- a/docs/user/reference/custom-404.rst +++ b/docs/user/reference/404-not-found.rst @@ -1,5 +1,5 @@ -Custom ``404 Not Found`` pages -============================== +``404 Not Found`` pages +======================= If you want your project to use a custom or branded ``404 Not Found`` page, you can put a ``404.html`` at the top level of your project's HTML output. diff --git a/docs/user/reference/features.rst b/docs/user/reference/features.rst index 1634b963c66..a4825020cdb 100644 --- a/docs/user/reference/features.rst +++ b/docs/user/reference/features.rst @@ -12,7 +12,7 @@ Custom domain support A project may define multiple domains, as well as define the *canonical domain* where all other domains will redirect. - ⏩️ :doc:`Read more ` + ⏩️ :doc:`/custom-domains` Multiple documentation versions We support multiple versions and translations, @@ -21,46 +21,46 @@ Multiple documentation versions If you only have 1 version and translation, we also support :doc:`single version projects ` served at ``/``. - ⏩️ :doc:`Read more ` + ⏩️ :doc:`/versions` Custom URL redirects Projects may define their own custom URL redirects, with advanced functionality like folder redirects. - ⏩️ :doc:`Read more ` + ⏩️ :doc:`/user-defined-redirects` Content Delivery Network (CDN) Documentation projects are primarily static HTML pages along with media files. This allows us to cache them with our CDN, making them *load faster* for your users. - ⬇️ :ref:`Read more ` + ⬇️ :doc:`/reference/cdn` Sitemaps Sitemaps are generated and hosted automatically, improving search engine optimization. This helps your users find content more effectively on your site. - ⬇️ :ref:`Read more ` + ⬇️ :doc:`/reference/sitemaps` Custom ``404s Not Found`` pages A 404 page is served when we can't find a page on your site. We provide a default 404 page, but you can also customize it. - ⬇️ :ref:`Read more ` + ⬇️ :doc:`/reference/404-not-found` Custom robots.txt - `robots.txt`_ files allow you to customize how your documentation is indexed in search engines. + `robots.txt` files allow you to customize how your documentation is indexed in search engines. We provide a default robots.txt file, but you can also customize it. - ⬇️ :ref:`Read more ` + ⬇️ :doc:`/reference/robots` Private documentation It is possible to host private and password protected documentation on Read the Docs for Business. - ⏩️ :doc:`Read more ` + ⏩️ :doc:`/commercial/sharing` .. The TOC here will be refactored once we reorganize the files in docs/user/. .. Probably, all feature reference should be in this directory! @@ -68,18 +68,16 @@ Private documentation .. In fact, this whole page will become a proper index page with more explanation of what sort of reference can .. be found. - - .. toctree:: :maxdepth: 1 + :hidden: ../features analytics /feature-flags /server-side-search/index - Special hosting features /badges /reference/cdn - /reference/custom-404 + /reference/404-not-found /reference/robots /reference/sitemaps diff --git a/docs/user/reference/sitemaps.rst b/docs/user/reference/sitemaps.rst index 23d938e32cf..07ec4686ca6 100644 --- a/docs/user/reference/sitemaps.rst +++ b/docs/user/reference/sitemaps.rst @@ -1,5 +1,5 @@ -Sitemaps -======== +``sitemap.xml`` support +======================== `Sitemaps `__ allow you to inform search engines about URLs that are available for crawling. This helps make your content more discoverable, From c7e5fb2a3bd2669c63702214c04105312085b37f Mon Sep 17 00:00:00 2001 From: Eric Holscher Date: Thu, 9 Feb 2023 17:27:53 -0800 Subject: [PATCH 13/18] Remove redundancy --- docs/Makefile | 2 +- docs/user/reference/features.rst | 26 ++++++++------------------ 2 files changed, 9 insertions(+), 19 deletions(-) diff --git a/docs/Makefile b/docs/Makefile index a42f00bcef7..160b6f4799d 100644 --- a/docs/Makefile +++ b/docs/Makefile @@ -7,7 +7,7 @@ # (except for '--keep-going', failing fast is prefered on local builds) # We also remove '-E' to reduce the time of rebuilding reference indexes # on each build. -SPHINXOPTS = -T -j auto -W +SPHINXOPTS = -T -W SPHINXBUILD = sphinx-build PAPER = PROJECT ?= user diff --git a/docs/user/reference/features.rst b/docs/user/reference/features.rst index a4825020cdb..d570be3e06e 100644 --- a/docs/user/reference/features.rst +++ b/docs/user/reference/features.rst @@ -7,61 +7,51 @@ Feature reference Hosting features ---------------- -Custom domain support +⏩️ :doc:`/custom-domains` Documentation projects can use their own domain name. A project may define multiple domains, as well as define the *canonical domain* where all other domains will redirect. - ⏩️ :doc:`/custom-domains` -Multiple documentation versions +⏩️ :doc:`/versions` We support multiple versions and translations, integrated nicely into the URL of your documentation. This is served at ``/en/latest/`` by default. If you only have 1 version and translation, we also support :doc:`single version projects ` served at ``/``. - ⏩️ :doc:`/versions` -Custom URL redirects +⏩️ :doc:`/user-defined-redirects` Projects may define their own custom URL redirects, with advanced functionality like folder redirects. - ⏩️ :doc:`/user-defined-redirects` - -Content Delivery Network (CDN) +⬇️ :doc:`/reference/cdn` Documentation projects are primarily static HTML pages along with media files. This allows us to cache them with our CDN, making them *load faster* for your users. - ⬇️ :doc:`/reference/cdn` -Sitemaps +⬇️ :doc:`/reference/sitemaps` Sitemaps are generated and hosted automatically, improving search engine optimization. This helps your users find content more effectively on your site. - ⬇️ :doc:`/reference/sitemaps` -Custom ``404s Not Found`` pages +⬇️ :doc:`/reference/404-not-found` A 404 page is served when we can't find a page on your site. We provide a default 404 page, but you can also customize it. - ⬇️ :doc:`/reference/404-not-found` -Custom robots.txt +⬇️ :doc:`/reference/robots` `robots.txt` files allow you to customize how your documentation is indexed in search engines. We provide a default robots.txt file, but you can also customize it. - ⬇️ :doc:`/reference/robots` -Private documentation +⬇️ :doc:`/reference/robots` It is possible to host private and password protected documentation on Read the Docs for Business. - ⏩️ :doc:`/commercial/sharing` - .. The TOC here will be refactored once we reorganize the files in docs/user/. .. Probably, all feature reference should be in this directory! .. In upcoming work, redirects will be added for old URL destinations. From 32940544a7c38cf3b708b5e3c6badecdf4aa1d81 Mon Sep 17 00:00:00 2001 From: Eric Holscher Date: Fri, 10 Feb 2023 05:41:19 -0800 Subject: [PATCH 14/18] Remove stray Makefile commit --- docs/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/Makefile b/docs/Makefile index 160b6f4799d..a42f00bcef7 100644 --- a/docs/Makefile +++ b/docs/Makefile @@ -7,7 +7,7 @@ # (except for '--keep-going', failing fast is prefered on local builds) # We also remove '-E' to reduce the time of rebuilding reference indexes # on each build. -SPHINXOPTS = -T -W +SPHINXOPTS = -T -j auto -W SPHINXBUILD = sphinx-build PAPER = PROJECT ?= user From 2b192a5033fbc790fb3e3d0ebb13cd76b4573cc5 Mon Sep 17 00:00:00 2001 From: Benjamin Bach Date: Fri, 10 Feb 2023 17:00:32 +0100 Subject: [PATCH 15/18] Make a mention of this "404/index.html" structure in relation to Sphinx DirHTML builder. --- docs/user/reference/404-not-found.rst | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/user/reference/404-not-found.rst b/docs/user/reference/404-not-found.rst index 006394be1ad..c260a8402ee 100644 --- a/docs/user/reference/404-not-found.rst +++ b/docs/user/reference/404-not-found.rst @@ -35,6 +35,12 @@ We have examples for some of the most popular tools below. Sphinx uses `html_extra_path`_ option to add static files to the output. You need to create a ``404.html`` file and put it under the path defined in ``html_extra_path``. + If you are using the ``DirHTML`` builder, + no further steps are required. + Sphinx will automatically apply the ``/index.html`` folder structure to your 404 page: + ``404/index.html``. + Read the Docs also detects 404 pages named this way. + .. tab:: MkDocs MkDocs automatically generates a ``404.html`` which Read the Docs will use. From afa0ec887b1d040fe7610600eb3b9e507f2cfcfa Mon Sep 17 00:00:00 2001 From: Eric Holscher <25510+ericholscher@users.noreply.github.com> Date: Fri, 10 Feb 2023 16:45:59 -0800 Subject: [PATCH 16/18] Apply suggestions from code review Co-authored-by: Benjamin Balder Bach --- docs/user/reference/404-not-found.rst | 2 +- docs/user/reference/cdn.rst | 13 ++++++------- docs/user/reference/features.rst | 10 +++++----- docs/user/reference/robots.rst | 14 ++++++++++---- docs/user/reference/sitemaps.rst | 15 ++++++--------- 5 files changed, 28 insertions(+), 26 deletions(-) diff --git a/docs/user/reference/404-not-found.rst b/docs/user/reference/404-not-found.rst index c260a8402ee..1cfba4ba00c 100644 --- a/docs/user/reference/404-not-found.rst +++ b/docs/user/reference/404-not-found.rst @@ -15,7 +15,7 @@ The following locations are checked, in order: * ``/404.html`` or ``404/index.html`` in the *current* documentation version. * ``/404.html`` or ``404/index.html`` in the *default* documentation version. -Tool Integration +Tool integration ---------------- Documentation tools will have different ways of generating a ``404.html`` or ``404/index.html`` file. diff --git a/docs/user/reference/cdn.rst b/docs/user/reference/cdn.rst index 579d31e24b5..e7ed31e589b 100644 --- a/docs/user/reference/cdn.rst +++ b/docs/user/reference/cdn.rst @@ -2,7 +2,7 @@ Content Delivery Network (CDN) and caching ========================================== A CDN is used for making documentation pages *fast* for your users. -This is done by caching the documentation page content in multiple data centers around the world, +CDNs increase speed by caching documentation content in multiple data centers around the world, and then serving docs from the data center closest to the user. We support CDNs on both of our sites: @@ -22,10 +22,8 @@ Having a CDN in front of your documentation has many benefits: * **Improved reliability**: Since docs are served from multiple places, one can go down and the docs are still accessible. * **Improved performance**: Data takes time to travel across space, so connecting to a server closer to the user makes documentation load faster. -CDN Features ------------- - -Our integration between building and hosting documentation allows to do many smart things to make the experience seemless. +Automatic cache refresh +----------------------- We automatically refresh the cache on the CDN when the following actions happen: @@ -33,7 +31,8 @@ We automatically refresh the cache on the CDN when the following actions happen: * Your domain is saved. * A new version of your documentation is built. -This means that you should **never see outdated content**, -but it's always as fast a possible for your users. +By refreshing the cache according to these rules, +readers should **never see outdated content**. +This makes the end-user experience seamless, and fast. .. _Cloudflare: https://www.cloudflare.com/ diff --git a/docs/user/reference/features.rst b/docs/user/reference/features.rst index d45ee337eac..d69d4344180 100644 --- a/docs/user/reference/features.rst +++ b/docs/user/reference/features.rst @@ -25,31 +25,31 @@ Hosting features Projects may define their own custom URL redirects, with advanced functionality like folder redirects. -⬇️ :doc:`/reference/cdn` +⏩️ :doc:`/reference/cdn` Documentation projects are primarily static HTML pages along with media files. This allows us to cache them with our CDN, making them *load faster* for your users. -⬇️ :doc:`/reference/sitemaps` +⏩️ :doc:`/reference/sitemaps` Sitemaps are generated and hosted automatically, improving search engine optimization. This helps your users find content more effectively on your site. -⬇️ :doc:`/reference/404-not-found` +⏩️ :doc:`/reference/404-not-found` A 404 page is served when we can't find a page on your site. We provide a default 404 page, but you can also customize it. -⬇️ :doc:`/reference/robots` +⏩️ :doc:`/reference/robots` `robots.txt` files allow you to customize how your documentation is indexed in search engines. We provide a default robots.txt file, but you can also customize it. -⬇️ :doc:`/reference/robots` +⏩️ :doc:`/commerical/sharing` It is possible to host private and password protected documentation on Read the Docs for Business. .. The TOC here will be refactored once we reorganize the files in docs/user/. diff --git a/docs/user/reference/robots.rst b/docs/user/reference/robots.rst index 49d83abb28a..1c38d493657 100644 --- a/docs/user/reference/robots.rst +++ b/docs/user/reference/robots.rst @@ -9,13 +9,19 @@ It's useful for: * Disallowing any indexing of your documentation Read the Docs automatically generates one for you with a configuration that works for most projects. -By default, the automatically created ``robots.txt``:abbr: +By default, the automatically created ``robots.txt``: * Hides versions which are set to :ref:`versions:Hidden` from being indexed. * Allows indexing of all other versions. -.. warning:: ``robots.txt`` files are respected by search engines, but they are not a guarantee that your documentation will not be indexed. - Search engines may choose to ignore your ``robots.txt`` file and index your documentation anyway. +.. warning:: + + ``robots.txt`` files are respected by most search engines, + but they aren't a guarantee that your pages will not be indexed. + Search engines may choose to ignore your ``robots.txt`` file, + and index your docs anyway. + + If you require *private* documentation, please see :doc:`/commercial/sharing`. How it works ------------ @@ -26,7 +32,7 @@ This is because the ``robots.txt`` file is served at the top-level of your domai so we must choose a version to find the file in. The **default version** is the best place to look for it. -Tool Integration +Tool integration ---------------- Documentation tools will have different ways of generating a ``robots.txt`` file. diff --git a/docs/user/reference/sitemaps.rst b/docs/user/reference/sitemaps.rst index 07ec4686ca6..49483613a92 100644 --- a/docs/user/reference/sitemaps.rst +++ b/docs/user/reference/sitemaps.rst @@ -1,9 +1,9 @@ -``sitemap.xml`` support -======================== +``sitemap.xml`` support +======================= `Sitemaps `__ allow you to inform search engines about URLs that are available for crawling. -This helps make your content more discoverable, -and improve your Search Engine Optimization (SEO). +This makes your content more :term:`discoverable `, +and improves your :doc:`Search Engine Optimization (SEO) `. How it works ------------ @@ -16,10 +16,9 @@ It contains information such as: * How important this URL is in relation to other URLs in the site. * What translations are available for a page. -Read the Docs automatically generates a sitemap for your project, -to improve results when performing a search on these search engines. +Read the Docs automatically generates a ``sitemap.xml`` for your project, -By default our Sitemap includes: +By default the sitemap includes: * Each version of your documentation and when it was last updated, sorted by version number. @@ -31,8 +30,6 @@ Custom ``sitemap.xml`` If you need a custom sitemap, please let us know by contacting :doc:`/support`. -It's on our long term roadmap, -but hearing that users want it is a great way to have us prioritize it. .. _semantic versioning: https://semver.org/ .. _GitHub issue #5391: https://github.com/readthedocs/readthedocs.org/issues/5391 From 419dccf13802eb40cfcf141bbd4ff3cd2235ffdc Mon Sep 17 00:00:00 2001 From: Benjamin Bach Date: Tue, 14 Feb 2023 15:22:48 +0100 Subject: [PATCH 17/18] Lint --- docs/user/reference/robots.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user/reference/robots.rst b/docs/user/reference/robots.rst index 1c38d493657..43c667310c8 100644 --- a/docs/user/reference/robots.rst +++ b/docs/user/reference/robots.rst @@ -20,7 +20,7 @@ By default, the automatically created ``robots.txt``: but they aren't a guarantee that your pages will not be indexed. Search engines may choose to ignore your ``robots.txt`` file, and index your docs anyway. - + If you require *private* documentation, please see :doc:`/commercial/sharing`. How it works From 2435b302ddddf4522b31df95cbe1c357ede6fba8 Mon Sep 17 00:00:00 2001 From: Benjamin Bach Date: Tue, 14 Feb 2023 15:24:14 +0100 Subject: [PATCH 18/18] Fix typo --- docs/user/reference/features.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user/reference/features.rst b/docs/user/reference/features.rst index d69d4344180..75265978663 100644 --- a/docs/user/reference/features.rst +++ b/docs/user/reference/features.rst @@ -49,7 +49,7 @@ Hosting features but you can also customize it. -⏩️ :doc:`/commerical/sharing` +⏩️ :doc:`/commercial/sharing` It is possible to host private and password protected documentation on Read the Docs for Business. .. The TOC here will be refactored once we reorganize the files in docs/user/.