Add feature flagging

[agjohnson]

I started poking at django-waffle yesterday, in order to find a way to make me feel less anxious about making changes to critical parts of the web ui or build infra. However, we might need to roll our own form of this.

What we need

We need a list of flags that are modeled against the Project model. The API would return these flags for the builder to have access to these as well.

For build processes

• Version of the context data format that we pass to sphinx
• Project uses beta build image
• Allow per-user override of build limitations -- ie, mkdocs theme access to users that have been warned that everything breaks

For the UI

• Remove old pieces of the UI that we are moving to config
• Add beta feature display per user

How do we do it?

Per-project flags make more sense than user flags for our use case. Perhaps we can just fork the modeling around waffle, or extend this in some way. The Project model probably just needs a mapping for ProjectFlags instead.

We need several modes for feature flagging on projects:

1. User self selects project into feature
2. Project is assigned flag via admin
3. Project is assigned flag via creation date
4. Project is assigned flag via random sample

For #1, features that fall in here are good candidates for inclusion in our yaml config. For example, user wants to opt into a beta build feature, we can document it and it will be hands off testing for us.

Feature flags that fall into the #2 category are perhaps overrides per-project for build limitations. For example, projects that have been warned about feature limitation and want restrictions lifted on mkdocs themes or some of our other overrides. These would be unpublished and require and admin to flag the project manually.

For feature flags like #3, we would want to change how a feature works/displays based on a new feature release date. In this case, we might run a migration that adds a flag for all existing projects alongside code that adds the feature for newly created projects. We can also continue on to #4 for the flagged projects.

Feature flags like #4 are for features flags that we want disabled/enabled on a strictly random basis. Useful for slow roll out or deprecation of a feature.

It seems this can be boiled down to:

• Project flags from readthedocs.yml
• Project flags from database/API

Plan

Implement ProjectFlag model, and methods on the Project model or model manager for returning a list of project flags. Project flags would be designed as constants, in a projects/flags.py. The API would return a list of strings of flags with full project API return.