Projects depend on files in submodules for the config validation

[meteoswiss-mdr]

Details:

• Project URL: https://readthedocs.org/projects/pyrad-mch/
• Build URL(if applicable): https://readthedocs.org/projects/pyrad-mch/builds/8520948/
• Read the Docs username(if applicable): meteoswiss-mdr

Expected Result

First automatic generation of documentation.

Actual Result

Build fails without error shortly after cloning submodule. I am not sure the config file .readthedocs.yml is even used

Read the Docs build information
Build id: 8520948
Project: pyrad-mch
Version: latest
Commit: cb07f61a7f291a54df4cabb2f5c798930b7f3246
Date: 2019-02-06T08:48:37.714618Z
State: finished
Success: False

[rtd-command-info] start-time: 2019-02-06T08:48:38.165048Z, end-time: 2019-02-06T08:48:39.718576Z, duration: 1, exit-code: 0
git clone --no-single-branch --depth 50 https://github.com/meteoswiss-mdr/pyrad.git .
Cloning into '.'...

[rtd-command-info] start-time: 2019-02-06T08:48:39.996339Z, end-time: 2019-02-06T08:48:40.184542Z, duration: 0, exit-code: 0
git checkout --force origin/master
Note: checking out 'origin/master'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

git checkout -b

HEAD is now at cb07f61 testing read the docs configuration

[rtd-command-info] start-time: 2019-02-06T08:48:40.242882Z, end-time: 2019-02-06T08:48:40.253799Z, duration: 0, exit-code: 0
git clean -d -f -f

[rtd-command-info] start-time: 2019-02-06T08:48:40.541763Z, end-time: 2019-02-06T08:48:40.571979Z, duration: 0, exit-code: 0
git submodule sync

[rtd-command-info] start-time: 2019-02-06T08:48:40.633606Z, end-time: 2019-02-06T08:48:43.717137Z, duration: 3, exit-code: 0
git submodule update --init --force --recursive src/pyart
Submodule 'src/pyart' (https://github.com/meteoswiss-mdr/pyart.git) registered for path 'src/pyart'
Cloning into '/home/docs/checkouts/readthedocs.org/user_builds/pyrad-mch/checkouts/latest/src/pyart'...
Submodule path 'src/pyart': checked out '5bddde3a8ea2fa85931be2d7a084b885126cda26'

[humitos]

I think this is a known bug.

readthedocs.config.config.InvalidConfig: Invalid "python.install.1.path": path src/pyrad/ does not exist

The problem is that RTD is validating the YAML file before cloning submodules. So, it checks for src/pyrad/ and it does not exist yet.

[meteoswiss-mdr]

Thanks a lot for your fast answer.

Do you know if there is a work-around on this?

[stsewd]

Having the file in the project is the only solution, more background in #5017

[meteoswiss-mdr]

Sorry, I am new to readthedocs and I am not sure I understood what the problem is.
I have a superproject called "pyrad" with a submodule called "pyart" that needs to be compiled first for the superproject to work.

If I understood what you mean the problem is that in my readthedocs.yml file I have the following line:

Configuration of the Python environment to be used

python:
version: 3
install:
- method: setuptools
path: src/pyart/

That makes it crash because first the YAML file is validated (and cannot find the path) and AFTER it clones the repositories.

My questions:

• If that is the problem, could the validation send and error before even trying to clone anything? As it is it is quite confusing.

• Is it so strange to have submodules required for a superproject to compile properly? I am pretty sure I am not the only one. I think the proposed keyword "skip_validations" would be a wonderful addition. Otherwise I cannot see how to use readthedocs with my current setup. If somebody can propose a workable solution I would be very happy to hear it.

[stsewd]

Marking this as design decision, maybe having a mono repo is a valid case for this. In the meantime, I think you can use the git url to install from a requirements file using pip.

[stsewd]

Another solution is to remove the validations for files from the config module, and wait to blow up in the build process (or do the validation here)

[stsewd]

We decided in remove the validations for files in the config level and check for this before build. We already do some of these validations there.