Add support for pre-install step

[jaraco]

Details

• Read the Docs project URL: https://readthedocs.org/projects/setuptools
• Build URL (if applicable): https://readthedocs.org/projects/setuptools/builds/10445019/
• Read the Docs username (if applicable): jaraco

Expected Result

There would be a hook that setuptools could invoke to bootstrap the source tree prior to attempting to install it.

Actual Result

RTD attempts to install the source and fails because a bootstrap step is necessary.

---

In pypa/setuptools#2001, I've captured an emergent failure. After switching from pip_install = False to pip_install = True (in order to support extras and eventually support other features like Sphinx builds to include API docstrings), the builds no longer succeed.

Setuptools has a unique condition where it needs some of its functionality available (entry point definitions) before it's able to install itself as other packages do. To accomplish this task, setuptools provides its own ./bootstrap.py that must be invoked prior to invoking setup.py *anything* or pip install <source dir>.

I've looked into the custom build steps, but those seem to happen after any install step. What I desire is a hook where I can define behavior before any install.

I imagine I could build a custom RTD builder that also performs the install, and leave pip_install=False (or version 2 equivalent), but that feels hacky and brittle.

Would it be possible to add a hook pre_install that if set to python -m bootstrap would execute that prior to pip install? Do you have any other suggestions?

[jaraco]

If RTD had a hook that allowed overriding the entire install/build process, setuptools has a tox -e docs routine that builds the docs reliably. Could a plugin be written to invoke that behavior instead, providing a generalizable way for a client to define the larger process?

[humitos]

Hi @jaraco! Thanks for reporting this. We have a long discussion internally about features like this and we haven't made a decision yet. We are discussing topics like "allowing user to run just random commands" and "allow to upload a pre-built HTML" which are related in some way to what you are needing at this point.

I don't have a good answer and we don't know when we are going to implement these. We need to keep discussing and make a final decision. Although, your report will help us to have another use case where we need to make a decision and do something about this.

[NiklasRosenstein]

FWIW, I've hacked around this for my Pydoc-Markdown project (link to rtd). I created readthedocs-custom-steps which hooks into the MkDocs invocation on Read the Docs and instead routes it to a Python module that reads the contents of .readthedocs-custom-steps.yml.

Unfortunately .readthedocs.yml cannot contain additional keys, so it has to be in a separate file. This would be a lot more concise to set up if one could write the below instead of having to split it up into three files (RTD, requirements and custom steps).

version: 2
mkdocs: {}
python:
  version: 3.7
  install:
  - {method: pip, path: .}
  - requires: readthedocs-custom-steps
x-custom-steps:
- echo "This is a custom build step. Output folder is $SITE_DIR"
- ...

[NiklasRosenstein]

I've created a pull request to suggest a possible implementation for the above features: #7259

[ericholscher]

I'm +1 on just having a prebuild CLI step that runs. It's currently possible with hacky workarounds, but I think at this point feels like an obvious win.

[humitos]

This is another use case that could be supported by the proposed solution at #8190 by overriding just build.jobs.pre_install config.

[humitos]

Hi! In a few hours, we are deploying a new feature that will allow people to call arbitrary commands at different moments in the build process (see #9016). For example,

build:
  os: ubuntu-20.04
  jobs:
    pre_install:
      - python -m bootstrap

I think this new config key, build.jobs, will cover this case. Take a look at the documentation at https://docs.readthedocs.io/en/latest/config-file/v2.html#build-jobs and please let me know if this is useful for you.

[jaraco]

I no longer have the need that originally inspired this request, but it does appear the pre_create_environment setting of a build does exactly what would have fulfilled this need. Thanks for the work considering it and adding it. I'm confident it will prove useful in the future!