Migrate docs to the main code repository

[uniphil]

Having documentation live with the code has some benefits:

• Discoverability: I spent a while digging through begriffs/postgrest looking for the docs to submit a docs correction, before I found this repo.
• Synchronization: Commits and pull-requests would atomically change source and documentation together.
• History: Finding old docs, or docs specific down to a specific commit in the source, would be the same as finding that version's source-code -- no cross-referencing or guesswork.
• Precedent: Many other projects do it this way. (I'm lazy but can get a list of examples if you want).

@begriffs summarized the original reasoning behind having this separate docs repo in begriffs/postgrest#805

1. Might want to store big assets like pictures in there (for instance all those pictures for the Heroku docs in 0.3) and don't want to slow people or CI down during the git clone.
2. Didn't want to litter the root of the project with docs configuration, but might not be a big deal.

In my opinion, the costs outweigh the benefit of those two points, but I guess it's somewhat subjective.

If a pull-request to move the docs to the main repo would be accepted, I'd be happy to start working on it :)

[ruslantalpa]

discoverability can be improved by simply having a link in the main repo to the docs repo
Synchronization there is nothing automatic about that :) and usually people are not too keen on writing the docs. more often then not a feature will be implemented and only after some time some docs will be added so i don't see a big gain here.
precedent - the projects that do this are usually small libs, anything that is over a certain size does not do this.
if the docs are in the same repo as code it means on each commit a whole chain of events is triggered (integration tests, docker builds ... stuff like that) even if the change is only a typo fix.

I think the only real problem here is the discoverability (links from both repos to one another) and that can be fixed with a few well placed links

[begriffs]

• Discoverability. You're right on this one, though there are small things to improve the situation. Could add a link to the docs from the main repo readme. Also the "Edit on Github" button on the pages on postgrest.com do all link back to this docs repo. Having docs and project in the same repo would of course be ideal, and as a bonus would change those "Edit on Github" to bring people to the main project.
• Synchronization. I agree that this would be improved by having docs in the main repo. When people open PRs we can more easily say "please add docs for this feature" before merging. In the two-repo approach the feature gets committed and then there's a good chance the contributor forgets or loses interest in submitting another PR. I'll also note that if combining the repos we would then not need two sets of Github issues, we can use a "Documentation" tag on issues in the main repo.
• History. This will be improved by properly using branches on Read The Docs. Notice at the bottom of the table of contents on postgrest.com it has a version selector dropdown:
  

Ruslan makes a point about the build system in the main repo. Docs changes would trigger the whole test suite. We've decided not to build Docker nightlies through CI so that part will be less of a problem. Just for context, it looks like CI builds take about four minutes these days. So docs changes would have that slight delay before being merge-able. Maybe not a big deal since it will usually take me much longer to get to reviewing the change anyway.

I'm starting to think you're right about putting the repositories together. To allay my fears about binaries accumulating over time we could put the images into git lfs. Otherwise remember that images never disappear from git history, even if they are deleted in later commits. All images from the beginning of time in all their versions hang around. I wonder if Read The Docs supports git lfs...

[begriffs]

I'm working on porting the 0.3 docs as a real version on read the docs before thinking about migration. In the meantime anyone want to research Git LFS integration with RTD?

[begriffs]

OK 0.3 docs are now live on our site. It also appears there is a way to use Git-LFS on read the docs (see readthedocs/readthedocs.org#1846).

Migrating the docs to the main repo requires a few steps. RTD uses tags or branches in the repo for docs versions. This might cause inconvenience in the main repo. Help me think this through, here are the steps I see to get the docs integrated with the main repo.

1. Open pull requests to the main repo incorporating changes from the docs repo.
   • docs v0.4 against main master
   • docs v0.3.2 against main v0.3.x
2. After PRs are merged, create proper branches on main repo for public RTD versions
   • v0.4 off master
   • v0.3.2 off v0.3.x (or maybe just rename v0.3.x to v0.3.2)
3. Point RTD at the main repo rather than the docs repo.
   • Mark v0.4 and v0.3.2 branches as active and trigger a docs rebuild
4. Adjust branching strategy in main repo
   • Here's where things might get awkward
   • At what point do we want the docs to update on the version marked publicly as "v0.4"?
   • Changes on the master branch will not update the public v0.4 version on postgrest.com.
   • RTD has a version called "latest" pointing at master, do we make latest the default version when someone visits postgrest.com? Or do we cherry-pick all subsequent commits from master to the v0.4 branch?

[phanimahesh]

Here are a few suggestions:

• Do not create one branch per point release. Instead use one branch per release line: This won't contain major changes. Docs within 0.3.x should not see major changes, so let the branch be called 0.3.x
• A branch is created when a release line changes. So 0.4.x branch will be created when 0.4 is officially released. Every point release update should have a corresponding update to the release branches. This can be either a bunch of cherry picks or straight up merge/fast forward depending on the changes included and your preference.
• Point RTD latest to either master or last released version. master would be my preference.

Maintenance effort will include marking a new release branch active in RTD when a major update happens. You can have a checklist in the repo to make sure it is not missed, since this will be an infrequent operation.

[begriffs]

(This is still on my radar, currently prioritizing reviewing open the open PRs and fixing environment vars in Docker which has been blocking people)

[begriffs]

Revisiting this again. You know, I do acknowledge that there are some benefits of merging the docs and main repo but it doesn't feel worth the effort right now. I'm going to put in some effort over the next few days to take care of a lot of other docs issues and let's just leave the repo as it is for now.