Technical docs SEO guide #6077

davidfischer · 2019-08-16T23:44:45Z

This PR adds a guide about SEO for technical docs with a focus on Sphinx. It includes the basics of SEO and how RTD and Sphinx help with that.
A lot of this is explanations with links to existing other docs like the sitemaps guide, canonical guide, redirects, GA guide, etc. I think that's a good thing!
One section that I would like to add but haven't yet is a section on how good technical writing should naturally lead to better SEO. That could almost be its own guide, but I would like to include stuff like:
- Using words and phrases that users use (and would search for) without keyword stuffing. This will help users search for stuff.
- Writing with simpler language will also help search engines understand content.
- Having different types of docs (how-to guides, tutorials, reference docs) and how that will cover different things people search for

humitos

I really like the Guide.

I left some comments about rst styling and opinions about its content structure.

humitos · 2019-08-19T11:01:16Z

docs/guides/technical-docs-seo-guide.rst

+Once a crawler or spider finds your site, it will follow links and redirects
+in an attempt to find any and all pages on your site.
+While there are a few ways to guide the search engine in its crawl
+for example by using a sitemap or a robots.txt file which we'll discuss shortly,


Not sure it worth, but we may want to link sitemap and robots.txt to each section.

humitos · 2019-08-19T11:02:59Z

docs/guides/technical-docs-seo-guide.rst

+
+Sphinx calls pages that don't have links to them "orphans"
+and will throw a warning while building documentation that contains an orphan
+unless the warning is silenced with the :ref:`orphan directive <sphinx:metadata>`:


For some reason, this :ref: is rendering as https://sphinx.readthedocs.io/en/latest/usage/restructuredtext/field-lists.html#metadata instead of https://sphinx-doc.org

This is due to this:

readthedocs.org/docs/conf.py

Line 59 in 9f92834

'sphinx': ('https://sphinx.readthedocs.io/en/latest/', None),

I can test out changing it.

humitos · 2019-08-19T11:04:45Z

docs/guides/technical-docs-seo-guide.rst

+and will throw a warning while building documentation that contains an orphan
+unless the warning is silenced with the :ref:`orphan directive <sphinx:metadata>`:
+
+::


Suggested change

::

.. prompt:: bash

Should disable selecting the $ and render the output accordingly.

I tried this but it doesn't come through cleanly:

Hrm... I think the $ in the commands (make) should be omitted. Although, I'm not sure how to tell what are commands and what's output.

We can leave as you wrote it for now, though.

humitos · 2019-08-19T11:06:29Z

docs/guides/technical-docs-seo-guide.rst

+.. sourcecode:: rst
+
+    .. image:: your-image.png
+        :align: right


I think we can remove the align option from here so it's cleaner, since that option is not related with what we are trying to show in the example.

docs/guides/technical-docs-seo-guide.rst

humitos · 2019-08-19T11:14:18Z

docs/guides/technical-docs-seo-guide.rst

+Use a robots.txt file
+~~~~~~~~~~~~~~~~~~~~~
+
+A ``robots.txt`` file is readable by crawlers


We don't have a full Guide explaining how robots.txt works, but we could link this FAQ somewhere in this section: https://docs.readthedocs.io/en/latest/faq.html#how-can-i-avoid-search-results-having-a-deprecated-version-of-my-docs

I considered that and I think you're right at least for now. I think it's probably better to create a guide at some point though.

Yes. robots.txt should have it's own guide, I agree.

When I wrote that FAQ I just wanted the minimal configuration required to live somewhere and I think it was a mistake to make it a FAQ, even with that small content should have been a Guide 😞

We are trying to either create a new guide or spruce up one of the existing guides at least every month. That could be the next one!

humitos · 2019-08-19T11:18:56Z

docs/guides/technical-docs-seo-guide.rst

+unsupported versions of your documentation while keeping those docs online in case people need them.
+
+By default, Read the Docs serves a ``robots.txt`` for you
+which points to a :ref:`sitemap <guides/technical-docs-seo-guide:Use a sitemap.xml file>`


I think I'd remove the mention to sitemap here, or I'd introduce the concept first.

When reading the guide, I felt confusing the mention to sitemap at this point without mentioning it previously or explaining anything about it.

Removing the first sentence al leaving only "To customize this file,..." works to me.

Also, the "By default," sentence can be merged into the following section (Use a sitemap.xml) where robots.txt was already presented and now we can mentioned that the default robots.txt will point to the auto-generated sitemap.

humitos · 2019-08-19T11:21:07Z

docs/guides/technical-docs-seo-guide.rst

+
+Read the Docs generates a sitemap for you that contains the last time
+your documentation was updated as well as links to active versions, subprojects, and translations your project has.
+We have a small separate guide on `sitemaps </guides/sitemaps>`_.


The link does not work since it starts with /. I recommend using :ref: here.

Suggested change

We have a small separate guide on `sitemaps </guides/sitemaps>`_.

We have a small separate guide on :ref:`sitemaps <guides/sitemaps>`_.

It should be a :doc: link, not a ref or the broken link I did.

humitos · 2019-08-19T11:27:08Z

docs/guides/technical-docs-seo-guide.rst

+    :figwidth: 80%
+    :target: ../_static/images/guides/google-search-engine-results.png
+
+    Google search engine results showing a customized meta description


This is not a good example 😃

The description meta tag of that page is:

<meta content="How to add additional CSS stylesheets or JavaScript files to your Sphinx documentation to override your Sphinx theme or add interactivity with JavaScript." lang="en" name="description" />

but Google shows something different for some reason.

This blog post meta description does appear as Google's result description https://blog.readthedocs.com/custom-css-and-js-in-sphinx/

You're right. I totally missed that. I wonder why it isn't!

It looks like Google chooses whether to use the meta description based on a few factors including the length and whether the keywords in the description match the keywords on the page. Sounds like dark arts.

In that case, we may want to mention something "this is black magic and it may not displayed if the engine decides to not show it" :)

Otherwise, I feel that people will come back to us saying: "Hey, meta description does not work on Read the Docs" 😛

If Google has a guide or the rules for this, we could link to that as well.

Otherwise, I feel that people will come back to us saying: "Hey, meta description does not work on Read the Docs" 😛

I added something to that effect yesterday 😛.

Cool! Just saw it. 👍

humitos · 2019-08-19T11:28:36Z

docs/guides/technical-docs-seo-guide.rst

+
+    .. meta::
+        :description lang=en:
+            Your meta description here


I think it's better if this example reflects what the image shows.

Following my next comment about the blog post, this rst code would look like:

.. meta:: :description lang=en: Customize your documentation by adding CSS stylesheets or JavaScript files to change your docs' look and feel

Co-Authored-By: Manuel Kaufmann <[email protected]>

ericholscher

This is 💯 content, and we should ship it ASAP :)

ericholscher · 2019-08-20T18:44:21Z

docs/conf.py

@@ -56,7 +56,7 @@ def get_version():
 intersphinx_mapping = {
    'python': ('https://python.readthedocs.io/en/latest/', None),
    'django': ('https://django.readthedocs.io/en/1.11.x/', None),


We should probably link to the canonical docs of these other two, also.

I'll open another PR for this. This is approved so I'm going to merge it.

ericholscher · 2019-08-20T18:47:04Z

We should also copy a lot of this content to the blog somehow :)

davidfischer · 2019-08-20T21:16:09Z

We should also copy a lot of this content to the blog somehow :)

Agreed. I was planning on a short blog post linking to this. We've had a lot of blog posts recently and a couple in the pipeline so I'm not sure on timing.

davidfischer · 2019-08-20T22:15:28Z

One section that I would like to add but haven't yet is a section on how good technical writing should naturally lead to better SEO.

I think a guide or maybe just a blog on technical writing (guides vs tutorials) as well as linking to resources about simpler language (good for SEO and translations) and other things is very good but probably not a fit in the SEO guide right now.

Technical docs SEO guide

6338d64

davidfischer requested a review from a team August 16, 2019 23:44

humitos approved these changes Aug 19, 2019

View reviewed changes

davidfischer and others added 2 commits August 19, 2019 08:54

Update docs/guides/technical-docs-seo-guide.rst

e8d3c26

Co-Authored-By: Manuel Kaufmann <[email protected]>

PR feedback updates

85b5a77

ericholscher approved these changes Aug 20, 2019

View reviewed changes

davidfischer merged commit 19cd521 into master Aug 20, 2019

davidfischer deleted the davidfischer/technical-docs-seo-guide branch August 20, 2019 22:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Technical docs SEO guide #6077

Technical docs SEO guide #6077

davidfischer commented Aug 16, 2019

humitos left a comment

humitos Aug 19, 2019

humitos Aug 19, 2019

davidfischer Aug 19, 2019

humitos Aug 19, 2019

davidfischer Aug 19, 2019

humitos Aug 20, 2019

humitos Aug 19, 2019

humitos Aug 19, 2019

davidfischer Aug 19, 2019

humitos Aug 20, 2019

davidfischer Aug 20, 2019

humitos Aug 19, 2019

humitos Aug 19, 2019

davidfischer Aug 19, 2019

humitos Aug 19, 2019

davidfischer Aug 19, 2019

davidfischer Aug 19, 2019

humitos Aug 20, 2019

humitos Aug 20, 2019

davidfischer Aug 20, 2019

humitos Aug 20, 2019

humitos Aug 19, 2019

ericholscher left a comment

ericholscher Aug 20, 2019

davidfischer Aug 20, 2019

davidfischer Aug 20, 2019

ericholscher commented Aug 20, 2019

davidfischer commented Aug 20, 2019

davidfischer commented Aug 20, 2019

	We have a small separate guide on `sitemaps </guides/sitemaps>`_.
	We have a small separate guide on :ref:`sitemaps <guides/sitemaps>`_.

Technical docs SEO guide #6077

Technical docs SEO guide #6077

Conversation

davidfischer commented Aug 16, 2019

humitos left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ericholscher left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ericholscher commented Aug 20, 2019

davidfischer commented Aug 20, 2019

davidfischer commented Aug 20, 2019