Can a translation dataflow be integrated into the build process?

[carukc]

I would lice to be able to 'tx push' automatically when building the original language and 'tx pull' automatically when building the translation.

Is there a way to extend the builder script or do I need to create my own builder (or perhaps another hook to trigger these processes on build)?

Thanks
Chris

[gregmuellegger]

So you would like to execute a command when the build of your documentation on readthedocs.org starts?

I see the problem with this is that the builds will become non deterministic. What this means is that the result of the built might differ for the very same version depending on whether the built happened before or after some new translations where pushed to transifex.

Maybe could you write a bit more about how your current workflow for changes to the english version vs changes to the translations is?

[carukc]

Hello and thanks for responding!

Our workflow is:

Gitlab (english) -> RTD (english) -> Transifex (french) -> (tx pull, git add and push) -> RTD (english+french)

Gitlab... We upload the docs (would be nice if we had drag and drop but not there yet) and add a bit of mark-up for headings, etc. Gitlab handles triggering a build using a webhook when something is committed. We actually use both GitLab and Redmine (with the Redmine git hosting plugin) so I may change this step so that we manage the content within Redmine but this makes no difference to the workflow itself.

RTD... Used to display, search, bookmark, etc

Transifex... to translate to other languages. I've written a python script that creates .tx/config based on the .rst files that exist in a particular project. After running the script a 'tx push' sends the files to Transifex.

RTD... back to RTD to view the translated content.

The bits that I still need to automate are the generation of the config file, pushing to Transifex and grabbing the translated docs from Transifex... The steps are pretty straight forward... pull from transifex, add/push to the git repo and then trigger a RTD rebuild

If we added config options for target translations (of just used the languages of the linked projects then we can easily automate getting everything to Transifex by running the script to generate the .tx/config and then pushing the docs. Perhaps using a new button in RTD to update/initiate Transifex translations by creating the cofig and pushing to Transifex.

Perhaps then we should add another button to retrieve translations from Transifex, that would do the 'tx pull', 'git add' and 'git push' to the source repo. The rebuild could then be handled by a webhook in GitLab (or triggered directly after the translation updates have been pushed)

I think that you are right re getting this in the right order... The key will be finding just the right point to initiate the rebuild based on something becoming available (perhaps post review) within Transifex. Perhaps the Github/Trtansifex hook can be modified for this to trigger an update through gitlab or RTD directly from Transifex.

[gregmuellegger]

Thanks for detailing the whole process.

I'm not convinced yet that readthedocs is the right place for it to happen. The purpose for readthedocs is to host your documentation, which includes the building of the docs as it is required to display them somehow. Also readthedocs can be used to host your translated documentation, however it doesn't impose any structure on you how you make the translation happen. Therefore I wouldn't like to see more automation for external services in the build process of readthedocs.

IMO a valid alternative would be a build server like what TravisCI is offering to implement this step. I think it's already possible to make TravisCI do exactly what you want. Probably the after_success documented here: http://docs.travis-ci.com/user/customizing-the-build/ would make a good point to start a script which handles your needs for tx pull and git push

[gregmuellegger]

@carukc What's your opinion on using Travis for this problem?

[carukc]

Hi, Thank you for responding and apologies for not replying sooner myself. TravisCI looks pretty nice. I'm giving it a try now to see if I can integrate with GitLab too. I still think that I need a way to manually initiate the pull/build process though (a button somewhere)... There will be lots of translation work being done and I would not want to initiate a rebuild every time someone translates some part of a document.

Transifex supports webhooks but the docs only say that it fires 'When the translations are done'. I'm not sure whether this means when a whole doc has been translated or when a paragraph has been translated.

I'll keep you posted as I test all this out.

[stsewd]

@carukc hi, is this still needed for you? Travis has a rebuild button (or another name, I don't remember)

[agjohnson]

I really would like to have transifex integration, see #3329 for more there, if we ever add this to our roadmap. Closing this due to lack of activity.