ENH: branch deployments of documentation

[afeld]

Is your feature request related to a problem?

When submitting large/complex pull requests around documentation, I've been (reasonably) asked to submit a screenshot of the change. These are tedious to capture, the links in screenshots aren't clickable, the text isn't searchable, etc.

Describe the solution you'd like

When changes to a branch are pushed up, the documentation is built and available at a stable URL, a la ReadTheDocs. I believe the first part happens already via the Web and docs job in GitHub Actions, so I think it would mostly be a matter of deploying (and later cleaning up?) the built docs. https://pandas.pydata.org/docs/dev/ shows the latest on master, so maybe the URL structure is something like:

https://pandas.pydata.org/docs/preview/<user>/<branch>/

Bonus: Adding a commit status that links to the generated docs for that branch, as Netlify does.

API breaking implications

n/a

Describe alternatives you've considered

In one pull request, I:

1. Generated the page
2. Opened it in my browser
3. Exported it as a PDF
4. Uploaded it to Google Drive
5. Changed the sharing settings
6. Pasted the link in the comment

This solved the clickability/searchability problems, but is even more work than taking a screenshot.

Additional context

none

[afeld]

Wrote something up about some other challenges to contributing to the pandas docs. Doing some brainstorming about how to make that workflow better overall.

[jreback]

Wrote something up about some other challenges to contributing to the pandas docs. Doing some brainstorming about how to make that workflow better overall.

thanks for the mention!

[TomAugspurger]

xref #17921

We own the webserver, if we wanted to use that. We could push the built docs to the webserver on each PR, but would need a cron job to clean them up.

Or we can see if the built docs can be uploaded as an artifact to Azure Pipelines. Then post that URL somewhere. And also host it as a static site?

[alimcmaster1]

xref #17921

We own the webserver, if we wanted to use that. We could push the built docs to the webserver on each PR, but would need a cron job to clean them up.

Or we can see if the built docs can be uploaded as an artifact to Azure Pipelines. Then post that URL somewhere. And also host it as a static site?

+1 for this.

Think Scikit-learn have a Github App that posts the artifact link directly in the PR. (This setup uses CircleCI)
scikit-learn/scikit-learn#14731

Azure approach would also work and we could host directly on azure? If we wanted to avoid upload to our webserver.
https://docs.microsoft.com/en-us/azure/devops/pipelines/release/deploy-pull-request-builds?view=azure-devops

[afeld]

FYI that while ReadTheDocs supports this (as well as versioning), they don't support custom setup commands. That and the max 15 minute build time would be blockers.

[jorisvandenbossche]

Also see the relevant discussion at #17921 (comment) for why it's probably not that easy (security-wise) to use the same server as we push to from the master branch.

Short term, it might be the easiest to move the doc building to an Azure pipeline, as mentioned above by @alimcmaster1 that they support hosting artifacts

[lithomas1]

We use CircleCI now(we got the 400,000 credits/month). I'll look into using scikit-learn's approach if our credit use allows for this.

[astrojuanlu]

I arrive late to this discussion, but Read the Docs would be happy to lift the 15 minute build time for pandas.

The real blocker would be that the docs build process is highly customized, and according to similar past experiences, it would be very hard to make it amenable for RTD. In addition, our story for redirects is not great, and they would have to be recreated using the API from the redirects.csv file.

Still, if folks ever want to have a chat about it, we'll be glad to help.

[humitos]

The real blocker would be that the docs build process is highly customized, and according to similar past experiences, it would be very hard to make it amenable for RTD

I haven't checked deeply this customization, but I wanted to comment here to note that Read the Docs now supports the ability to run custom commands at different steps of the build process by using the config file key build.jobs and that could be useful here. Check out the documentation at https://docs.readthedocs.io/en/stable/config-file/v2.html#build-jobs

[mroeschke]

This has been implemented recently with the /preview command so closing