Projects depend on files in submodules for the config validation

[neewy]

Details

• Read the Docs project URL: https://iroha.readthedocs.io/ja/latest/index.html
• Build URL (if applicable): https://readthedocs.org/projects/iroha-ja/builds/8299696/
• Read the Docs username (if applicable): neewy

Our multilanguage docs are stored at https://github.com/soramitsu/localized_docs repo. All .rst files are symbolic links that are pointing to original files in Hyperledger Iroha repo. Project is built when translation files are supplied to conf.py via yaml.

Expected Result

1. RTD platform clones https://github.com/soramitsu/localized_docs
2. RTD platform retrieves code in submodule with git submodule update --init --recursive;
3. RTD platform locates requirements.txt file and builds docs successfully 🌞

Actual Result

1. RTD platform clones https://github.com/soramitsu/localized_docs
2. It tries to find requirements.txt file, but since it is in the submodule build fails (although several month ago it used to build) ⛈

[stsewd]

Sorry about this, we did some changes to separate the submodules step (so users can disable submodules, or clone only the subdmodules that are needed), now we first validate the requirements file, and then we clone the submodules. Not sure how to solve this, but I'll take a look soon.

[neewy]

@stsewd can you just tell me what is the current pipeline for build so that I can modify repo a bit? Potentially we can store exact copies of requirements.txt until you guys come with modifications.

[stsewd]

Not sure what you mean, current solution is to add the requirements file on the main repo

[neewy]

@stsewd Could please tell me what step is right after pip install -r requirements.txt in docs build pipeline?

[stsewd]

After that we install the project if you have the install project option, and the we execute the sphinx-build command

[stsewd]

https://readthedocs.org/projects/iroha-de/builds/8300454/ this is the correct way, but I don't see the requirements file there https://github.com/soramitsu/localized_docs/tree/master/docs/de

[stsewd]

Sorry, now I see https://github.com/soramitsu/localized_docs/blob/master/docs/de/requirements.txt

[neewy]

@stsewd sorry for confusion

[stsewd]

And no again, if you check the commit, it was building from https://github.com/soramitsu/localized_docs/tree/36b37a4d0a91a793868582dac96812c2aa81b89a/docs/de, there isn't a requierements file there

[stsewd]

Could you please tell me the final value of requirements file?

[stsewd]

I'm not sure if this is a common workflow, having files to build docs in a submodule, I was thinking some possible solutions here:

• Add a skip_validations key to the config (it will only skip validations related to find a file)
• Split the validations, before and after we have the whole repo (submodules included). That way we have the submodules keys ready to use, and we can continue with the validation after that.
• Read the config from the provider (GitHub, BitButket, GitLab), and take the decisions about cloning, submodules, etc early.
• Or maybe this isn't a common pattern and just don't support it?

[neewy]

At least PhpMyAdmin uses the same approach. We kinda copied their schema with symbolic links here: https://github.com/phpmyadmin/localized_docs

[neewy]

But they don't host it with readthedocs... Anyway, it should not be that hard to maintain requirements.txt in both repos (original and translation) and for now it solves issues with submodules

[neewy]

we did some changes to separate the submodules step (so users can disable submodules, or clone only the subdmodules that are needed), now we first validate the requirements file, and then we clone the submodules

@stsewd please put this in documentation

[stsewd]

Hi all!, a new release was made today, we don't validate files before cloning submodules.