Skip to content

Promote mamba more in the documentation, hide CONDA_USES_MAMBA #8624

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 27, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 0 additions & 12 deletions docs/feature-flags.rst
Original file line number Diff line number Diff line change
Expand Up @@ -24,18 +24,6 @@ In case you prefer to use the latest ``conda`` version available, this is the fl
Makes Read the Docs to install all the requirements at once on ``conda create`` step.
This helps users to pin dependencies on conda and to improve build time.

``CONDA_USES_MAMBA``: :featureflags:`CONDA_USES_MAMBA`

``conda`` solver consumes 1Gb minimum when installing any package using ``conda-forge`` channel.
This seems to be `a known issue`_ due conda forge has so many packages on it, among others.
Using this feature flag allows you to use mamba_ instead of ``conda`` to create the environment
and install the dependencies.
``mamba`` is a drop-in replacement for conda that it's much faster and also
reduces considerably the amount of memory required to solve the dependencies.

.. _mamba: https://quantstack.net/mamba.html
.. _a known issue: https://www.anaconda.com/understanding-and-improving-condas-performance/

``DONT_OVERWRITE_SPHINX_CONTEXT``: :featureflags:`DONT_OVERWRITE_SPHINX_CONTEXT`

``DONT_SHALLOW_CLONE``: :featureflags:`DONT_SHALLOW_CLONE`
Expand Down
19 changes: 5 additions & 14 deletions docs/guides/build-using-too-many-resources.rst
Original file line number Diff line number Diff line change
Expand Up @@ -25,21 +25,12 @@ you can create a custom requirements file just for documentation.
This should speed up your documentation builds,
as well as reduce your memory footprint.

Use pip when possible
---------------------
Use mamba instead of conda
--------------------------

If you don't need ``conda`` to create your *documentation* environment,
consider using ``pip`` instead since ``conda`` could `require too much memory`_ to calculate the dependency tree
when using multiple channels.

.. _require too much memory: https://github.com/conda/conda/issues/5003


.. tip::

Even though your *project* environment is created with ``conda``, it may be not necessary for the *documentation* environment.
That is, to build the documentation is probably that you need fewer Python packages than to use your library itself.
So, in this case, you could use ``pip`` to install those fewer packages instead of creating a big environment with ``conda``.
If you need conda packages to build your documentation,
you can :ref:`use mamba as a drop-in replacement to conda <guides/conda:Making builds faster with mamba>`,
which requires less memory and is noticeably faster.

Document Python modules API statically
--------------------------------------
Expand Down
49 changes: 39 additions & 10 deletions docs/guides/conda.rst
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,39 @@ To maximize compatibility, we recommend putting ``conda-forge`` above ``defaults
.. _Anaconda Terms of Service: https://www.anaconda.com/terms-of-service
.. _the relevant conda docs: https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html?highlight=nodefaults#creating-an-environment-file-manually

Making builds faster with mamba
-------------------------------

One important thing to note is that, when enabling the ``conda-forge`` channel,
the conda dependency solver requires a large amount of RAM and long solve times.
This is `a known issue`_ due to the sheer amount of packages available in conda-forge.

As an alternative, you can instruct Read the Docs to use mamba_,
a drop-in replacement for conda that is much faster
and reduces the memory consumption of the dependency solving process.

To do that, add a ``.readthedocs.yaml`` :doc:`configuration file </config-file/v2>`
with these contents:

.. code-block:: yaml
:caption: .readthedocs.yaml

version: 2

build:
os: "ubuntu-20.04"
tools:
python: "mambaforge-4.10"

conda:
environment: environment.yml

You can read more about the :ref:`config-file/v2:build.tools.python` configuration
in our documentation.

.. _mamba: https://quantstack.net/mamba.html
.. _a known issue: https://www.anaconda.com/understanding-and-improving-condas-performance/

Mixing conda and pip packages
-----------------------------

Expand Down Expand Up @@ -168,23 +201,19 @@ For example, this conda environment contains the required dependencies to compil
Troubleshooting
---------------

As explained in our
:ref:`guide about build resources <guides/build-using-too-many-resources:Use pip when possible>`,
``conda`` is known to `require too much memory`_ when solving the satisfiability problem
for the specified dependencies. There are some conda-specific tips to consider to
minimize the running time or the memory usage:
If you have problems on the environment creation phase,
either because the build runs out of memory or time
or because some conflicts are found,
you can try some of these mitigations:

- Reduce the number of channels in ``environment.yml``, even leaving ``conda-forge`` only
and opting out of the defaults adding ``nodefaults``.
- Constrain the package versions as much as possible to reduce the solution space.
- Use mamba_, an alternative package manager fully compatible with conda packages,
by requesting the ``CONDA_USES_MAMBA`` :ref:`feature flag <feature-flags:Feature Flags>`.
- :ref:`Use mamba <guides/conda:Making builds faster with mamba>`,
an alternative package manager fully compatible with conda packages.
- And, if all else fails,
:ref:`request more resources <guides/build-using-too-many-resources:Requests more resources>`.

.. _require too much memory: https://github.com/conda/conda/issues/5003
.. _mamba: https://mamba.readthedocs.io/en/latest/

Custom Installs
---------------

Expand Down