Skip to content

Update docs about build process #4515

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Aug 21, 2018
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 74 additions & 19 deletions docs/builds.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@ Our current build limits are:
We can increase build limits on a per-project basis,
if you provide a good reason your documentation needs more resources.

You can see the current Docker build images that we use in our `docker repository <https://github.com/rtfd/readthedocs-docker-images>`_. `Docker Hub <https://hub.docker.com/r/readthedocs/build/>`_ also shows the latest set of images that have been built.
You can see the current Docker build images that we use in our `docker repository <https://github.com/rtfd/readthedocs-docker-images>`_.
`Docker Hub <https://hub.docker.com/r/readthedocs/build/>`_ also shows the latest set of images that have been built.

Currently in production we're using the ``readthedocs/build:2.0`` docker image as our default image.

Expand Down Expand Up @@ -53,17 +54,23 @@ Read the Docs cannot create a PDF version of your documentation with the *Mkdocs
Understanding what's going on
-----------------------------

Understanding how Read the Docs builds your project will help you with debugging the problems you have with the site. It should also allow you to take advantage of certain things that happen during the build process.
Understanding how Read the Docs builds your project will help you with debugging the problems you have with the site.
It should also allow you to take advantage of certain things that happen during the build process.

The first step of the process is that we check out your code from the repository you have given us. If the code is already checked out, we update the copy to the branch that you have specified in your projects configuration.
The first step of the process is that we check out your code from the repository you have given us.
If the code is already checked out, we update the copy to the branch that you have specified in your projects configuration.

Then we build the proper backend code for the type of documentation you've selected.

If you have the *Install Project* option enabled, we will run ``setup.py install`` on your package, installing it into a virtual environment. You can also define additional packages to install with the *Requirements File* option.
If you have the *Install Project* option enabled, we will run ``setup.py install`` on your package, installing it into a virtual environment.
You can also define additional packages to install with the *Requirements File* option.

When we build your documentation, we run `sphinx-build -b html . _build/html`, where `html` would be replaced with the correct backend. We also create man pages and pdf's automatically based on your project.
When we build your documentation, we run `sphinx-build -b html . _build/html`,
where `html` would be replaced with the correct backend.
We also create pdf's and ePub's automatically based on your project.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rest of the lines are the same, they were just too long. Except here, here we have had man pages, but we don't build man pages anymore.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we need double for the command andhtml` here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah


Then these files are copied across to our application servers from the build server. Once on the application servers, they are served from nginx.
Then these files are copied across to our application servers from the build server.
Once on the application servers, they are served from nginx.

An example in code:

Expand All @@ -78,7 +85,7 @@ An example in code:
copy_files(artifact_dir)


Builder Responsibility
Builder responsibility
----------------------

Builders have a very specific job.
Expand All @@ -88,18 +95,66 @@ The artifacts should end up in ``self.version.project.artifact_path(version=self
Where ``type`` is the name of your builder.
All files that end up in the artifact directory should be in their final form.

Packages installed in the build environment
-------------------------------------------

The build server does have a select number of C libraries installed, because they are used across a wide array of python projects. We can't install every C library out there, but we try and support the major ones. We currently have the following libraries installed:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd keep a message similar to this one inside a .. note:: so, users are aware that we can't install everything inside the docker image.


* doxygen
* LaTeX (texlive-full)
* libevent (libevent-dev)
* dvipng
* graphviz
* libxslt1.1
* libxml2-dev
The build environment
---------------------

The build process is executed inside Docker containers,
by default the image used is ``readthedocs/build:2.0``,
but you can change that using a :doc:`configuration file <yaml-config>`.

Each image is described below.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd add a .. tip:: admonition here with indicating how to verify the version of each package installed in the docker image (linking to https://packages.ubuntu.com/ for each specific version and commenting how to do it)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea


2.0 (stable)
~~~~~~~~~~~~

:O.S: Ubuntu 16.04
:Conda: Miniconda 4.3.31
:Python:
* ``m2crypto``
* ``matplolib``
* ``numpy``
* ``scipy``
* ``pandas``
* ``pip``
:Other packages:
* ``libxml2-dev``
* ``libxslt1.1``
* ``libevent``
* ``textlive-full``
* ``graphviz``
* ``pandoc``
* ``doxygen``
* ``libjpeg``
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if all these packages are relevant, I keep the most that were in the previous description

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, they are relevant.


`More details <https://github.com/rtfd/readthedocs-docker-images/blob/releases/2.x/Dockerfile>`__

3.0 (latest)
~~~~~~~~~~~~

:O.S: Ubuntu 16.04
:Conda: Miniconda 4.4.10
:Python:
* ``matplolib``
* ``numpy``,
* ``scipy``
* ``pandas``
* ``pip``
:JavaScript:
* ``nodejs``
* ``npm``
* ``jsdoc``
:Other packages:
* ``libxml2-dev``
* ``libxslt1-dev``
* ``libevent-dev``
* ``textlive-full``
* ``libgraphviz-dev``
* ``pandoc``
* ``doxygen``
* ``libjpeg``
* ``plantuml``

`More details <https://github.com/rtfd/readthedocs-docker-images/blob/releases/3.x/Dockerfile>`__

Writing your own builder
------------------------
Expand Down