Subscriptions: use djstripe for products/features (#10238)

Instead of relying on our subscription models, we are now relying on the djstripe models, but the features of each subscription are still attached to our subscription models. In the new modeling, we have features attached to stripe products.

I'm using some data classes to make the representation instead of using plain dictionaries, they are more verbose than a plain dictionary, but are easier to manipulate in the code.

After this change has been deployed, we won't be needing our subscription models anymore. The available products are defined in .com, since in .org we don't have subscriptions. If we feel like we can have this in the DB, we can also switch the data classes to be Django models.

Closes #9313

Author: stsewd

docs/dev/index.rst

@@ -27,5 +27,6 @@ or taking the open source Read the Docs codebase for your own custom installatio
    i18n
    server-side-search
    search-integration
+   subscriptions
    settings
    tests

docs/dev/subscriptions.rst

@@ -0,0 +1,91 @@
+Subscriptions
+=============
+
+Subscriptions are available on |com_brand|,
+we make use of Stripe to handle the payments and subscriptions.
+We use dj-stripe to handle the integration with Stripe.
+
+Local testing
+-------------
+
+To test subscriptions locally, you need to have access to the Stripe account,
+and define the following settings with the keys from Stripe test mode:
+
+- ``STRIPE_SECRET``: https://dashboard.stripe.com/test/apikeys
+- ``STRIPE_TEST_SECRET_KEY``: https://dashboard.stripe.com/test/apikeys
+- ``DJSTRIPE_WEBHOOK_SECRET``: https://dashboard.stripe.com/test/webhooks
+
+To test the webhook locally, you need to run your local instance with ngrok, for example:
+
+.. code-block:: bash
+
+   ngrok http 80
+   inv docker.up --http-domain xxx.ngrok.io
+
+If this is your first time setting up subscriptions, you will to re-sync djstripe with Stripe:
+
+.. code-block:: bash
+
+   inv docker.manage djstripe_sync_models
+
+The subscription settings (``RTD_PRODUCTS``) already mapped to match the Stripe prices from the test mode.
+To subscribe to any plan, you can use any `test card from Stripe <https://stripe.com/docs/testing>`__,
+for example: ``4242 4242 4242 4242`` (use any future date and any value for the other fields).
+
+Modeling
+--------
+
+Subscriptions are attached to an organization (customer),
+and can have multiple products attached to it.
+A product can have multiple prices, usually monthly and yearly.
+
+When a user subscribes to a plan (product), they are subscribing to a price of a product,
+for example, the monthly price of the "Basic plan" product.
+
+A subscription has a "main" product (``RTDProduct(extra=False)``),
+and can have several "extra" products (``RTDProduct(extra=True)``).
+For example, an organization can have a subscription with a "Basic Plan" product, and an "Extra builder" product.
+
+Each product is mapped to a set of features (``RTD_PRODUCTS``) that the user will have access to
+(different prices of the same product have the same features).
+If a subscription has multiple products, the features are multiplied by the quantity and added together.
+For example, if a subscription has a "Basic Plan" product with a two concurrent builders,
+and an "Extra builder" product with quantity three, the total number of concurrent builders the
+organization has will be five.
+
+Life cycle of a subscription
+----------------------------
+
+When a new organization is created, a stripe customer is created for that organization,
+and this customer is subscribed to the trial product (``RTD_ORG_DEFAULT_STRIPE_SUBSCRIPTION_PRICE``).
+
+After the trial period is over, the subscription is canceled,
+and their organization is disabled.
+
+During or after the trial a user can upgrade their subscription to a paid plan
+(``RTDProduct(listed=True)``).
+
+Custom products
+---------------
+
+We provide 3 paid plans that users can subscribe to: Basic, Advanced and Pro.
+Additionally, we provide an Enterprise plan, this plan is customized for each customer,
+and it's manually created by the RTD core team.
+
+To create a custom plan, you need to create a new product in Stripe,
+and add the product id to the ``RTD_PRODUCTS`` setting mapped to the features that the plan will provide.
+After that, you can create a subscription for the organization with the custom product,
+our appliction will automatically relate this new product to the organization.
+
+Extra products
+--------------
+
+We have one extra product: Extra builder.
+
+To create a new extra product, you need to create a new product in Stripe,
+and add the product id to the ``RTD_PRODUCTS`` setting mapped to the features that the
+extra product will provide, this product should have the ``extra`` attribute set to ``True``.
+
+To subscribe an organization to an extra product,
+you just need to add the product to its subscription with the desired quantity,
+our appliction will automatically relate this new product to the organization.

readthedocs/api/v3/permissions.py

@@ -2,7 +2,7 @@
 
 from readthedocs.core.utils.extend import SettingsOverrideObject
 from readthedocs.subscriptions.constants import TYPE_EMBED_API
-from readthedocs.subscriptions.models import PlanFeature
+from readthedocs.subscriptions.products import get_feature
 
 
 class HasEmbedAPIAccess(BasePermission):
@@ -24,7 +24,7 @@ class HasEmbedAPIAccess(BasePermission):
     def has_permission(self, request, view):
         project = view._get_project()
         # The project is None when the is requesting a section from an external site.
-        if project and not PlanFeature.objects.has_feature(project, TYPE_EMBED_API):
+        if project and not get_feature(project, feature_type=TYPE_EMBED_API):
             return False
         return True
 

readthedocs/api/v3/tests/test_builds.py

@@ -4,16 +4,17 @@
 from django.urls import reverse
 
 from readthedocs.subscriptions.constants import TYPE_CONCURRENT_BUILDS
+from readthedocs.subscriptions.products import RTDProductFeature
 
 from .mixins import APIEndpointMixin
 
 
 @override_settings(
     RTD_ALLOW_ORGANIZATIONS=False,
     ALLOW_PRIVATE_REPOS=False,
-    RTD_DEFAULT_FEATURES={
-        TYPE_CONCURRENT_BUILDS: 4,
-    },
+    RTD_DEFAULT_FEATURES=dict(
+        [RTDProductFeature(TYPE_CONCURRENT_BUILDS, value=4).to_item()]
+    ),
 )
 @mock.patch('readthedocs.projects.tasks.builds.update_docs_task', mock.MagicMock())
 class BuildsEndpointTests(APIEndpointMixin):

readthedocs/builds/tests/test_build_queryset.py

@@ -5,15 +5,16 @@
 from readthedocs.organizations.models import Organization
 from readthedocs.projects.models import Project
 from readthedocs.subscriptions.constants import TYPE_CONCURRENT_BUILDS
+from readthedocs.subscriptions.products import RTDProductFeature
 
 
 @pytest.mark.django_db
 class TestBuildQuerySet:
     @pytest.fixture(autouse=True)
     def setup_method(self, settings):
-        settings.RTD_DEFAULT_FEATURES = {
-            TYPE_CONCURRENT_BUILDS: 4,
-        }
+        settings.RTD_DEFAULT_FEATURES = dict(
+            [RTDProductFeature(type=TYPE_CONCURRENT_BUILDS, value=4).to_item()]
+        )
 
     def test_concurrent_builds(self):
         project = fixture.get(

readthedocs/core/resolver.py

@@ -8,7 +8,7 @@
 from readthedocs.core.utils.extend import SettingsOverrideObject
 from readthedocs.core.utils.url import unsafe_join_url_path
 from readthedocs.subscriptions.constants import TYPE_CNAME
-from readthedocs.subscriptions.models import PlanFeature
+from readthedocs.subscriptions.products import get_feature
 
 log = structlog.get_logger(__name__)
 
@@ -416,7 +416,7 @@ def _use_subdomain(self):
 
     def _use_cname(self, project):
         """Test if to allow direct serving for project on CNAME."""
-        return PlanFeature.objects.has_feature(project, type=TYPE_CNAME)
+        return bool(get_feature(project, feature_type=TYPE_CNAME))
 
 
 class Resolver(SettingsOverrideObject):

readthedocs/embed/tests/test_api.py

@@ -13,6 +13,7 @@
 from readthedocs.projects.constants import MKDOCS, PUBLIC
 from readthedocs.projects.models import Project
 from readthedocs.subscriptions.constants import TYPE_EMBED_API
+from readthedocs.subscriptions.products import RTDProductFeature
 
 data_path = Path(__file__).parent.resolve() / 'data'
 
@@ -34,10 +35,10 @@ def setup_method(self, settings):
         self.version.save()
 
         settings.USE_SUBDOMAIN = True
-        settings.PUBLIC_DOMAIN = 'readthedocs.io'
-        settings.RTD_DEFAULT_FEATURES = {
-            TYPE_EMBED_API: 1,
-        }
+        settings.PUBLIC_DOMAIN = "readthedocs.io"
+        settings.RTD_DEFAULT_FEATURES = dict(
+            [RTDProductFeature(TYPE_EMBED_API).to_item()]
+        )
 
     def get(self, client, *args, **kwargs):
         """Wrapper around ``client.get`` to be overridden in the proxied api tests."""

readthedocs/embed/v3/tests/test_access.py

@@ -12,15 +12,14 @@
 from readthedocs.projects.constants import PRIVATE, PUBLIC
 from readthedocs.projects.models import Project
 from readthedocs.subscriptions.constants import TYPE_EMBED_API
+from readthedocs.subscriptions.products import RTDProductFeature
 
 
 @override_settings(
     USE_SUBDOMAIN=True,
     PUBLIC_DOMAIN="readthedocs.io",
     RTD_ALLOW_ORGAZATIONS=False,
-    RTD_DEFAULT_FEATURES={
-        TYPE_EMBED_API: 1,
-    },
+    RTD_DEFAULT_FEATURES=dict([RTDProductFeature(TYPE_EMBED_API).to_item()]),
 )
 @mock.patch("readthedocs.embed.v3.views.build_media_storage")
 class TestEmbedAPIV3Access(TestCase):

readthedocs/embed/v3/tests/test_internal_pages.py

@@ -11,6 +11,7 @@
 
 from readthedocs.projects.models import Project
 from readthedocs.subscriptions.constants import TYPE_EMBED_API
+from readthedocs.subscriptions.products import RTDProductFeature
 
 from .utils import srcdir
 
@@ -24,9 +25,9 @@ def setup_method(self, settings):
         settings.USE_SUBDOMAIN = True
         settings.PUBLIC_DOMAIN = 'readthedocs.io'
         settings.RTD_EMBED_API_EXTERNAL_DOMAINS = []
-        settings.RTD_DEFAULT_FEATURES = {
-            TYPE_EMBED_API: 1,
-        }
+        settings.RTD_DEFAULT_FEATURES = dict(
+            [RTDProductFeature(TYPE_EMBED_API).to_item()]
+        )
 
         self.api_url = reverse('embed_api_v3')
 

readthedocs/organizations/models.py

@@ -136,15 +136,17 @@ def get_or_create_stripe_subscription(self):
             # This only happens during development.
             log.warning("No default subscription created.")
             return None
+        return self.get_stripe_subscription()
 
+    def get_stripe_subscription(self):
         # Active subscriptions take precedence over non-active subscriptions,
         # otherwise we return the must recently created subscription.
         active_subscription = self.stripe_customer.subscriptions.filter(
             status=SubscriptionStatus.active
         ).first()
         if active_subscription:
             return active_subscription
-        return self.stripe_customer.subscriptions.latest()
+        return self.stripe_customer.subscriptions.order_by("created").last()
 
     def get_absolute_url(self):
         return reverse('organization_detail', args=(self.slug,))
@@ -157,7 +159,7 @@ def users(self):
     def members(self):
         return AdminPermission.members(self)
 
-    def save(self, *args, **kwargs):  # pylint: disable=signature-differs
+    def save(self, *args, **kwargs):
         if not self.slug:
             self.slug = slugify(self.name)
 
@@ -173,7 +175,6 @@ def get_stripe_metadata(self):
             "org:slug": self.slug,
         }
 
-    # pylint: disable=no-self-use
     def add_member(self, user, team):
         """
         Add member to organization team.
@@ -278,7 +279,7 @@ def get_absolute_url(self):
     def __str__(self):
         return self.name
 
-    def save(self, *args, **kwargs):  # pylint: disable=signature-differs
+    def save(self, *args, **kwargs):
         if not self.slug:
             self.slug = slugify(self.name)
         super().save(*args, **kwargs)
@@ -319,7 +320,7 @@ def __str__(self):
             team=self.team,
         )
 
-    def save(self, *args, **kwargs):  # pylint: disable=signature-differs
+    def save(self, *args, **kwargs):
         hash_ = salted_hmac(
             # HMAC key per applications
             '.'.join([self.__module__, self.__class__.__name__]),
@@ -345,12 +346,12 @@ def migrate(self):
         content_type = ContentType.objects.get_for_model(self.team)
         invitation, created = Invitation.objects.get_or_create(
             token=self.hash,
-            defaults=dict(
-                from_user=owner,
-                to_email=self.email,
-                content_type=content_type,
-                object_id=self.team.pk,
-            ),
+            defaults={
+                "from_user": owner,
+                "to_email": self.email,
+                "content_type": content_type,
+                "object_id": self.team.pk,
+            },
         )
         self.teammember_set.all().delete()
         return invitation, created

readthedocs/organizations/tests/test_views.py

@@ -17,6 +17,7 @@
 from readthedocs.projects.models import Project
 from readthedocs.rtd_tests.base import RequestFactoryTestMixin
 from readthedocs.subscriptions.constants import TYPE_AUDIT_LOGS
+from readthedocs.subscriptions.products import RTDProductFeature
 
 
 @override_settings(RTD_ALLOW_ORGANIZATIONS=True)
@@ -150,9 +151,9 @@ def test_add_owner(self):
 
 @override_settings(
     RTD_ALLOW_ORGANIZATIONS=True,
-    RTD_DEFAULT_FEATURES={
-        TYPE_AUDIT_LOGS: 90,
-    },
+    RTD_DEFAULT_FEATURES=dict(
+        [RTDProductFeature(type=TYPE_AUDIT_LOGS, value=90).to_item()]
+    ),
 )
 class OrganizationSecurityLogTests(TestCase):
 
@@ -380,9 +381,7 @@ def test_organization_signup(self):
 
 @override_settings(
     RTD_ALLOW_ORGANIZATIONS=True,
-    RTD_DEFAULT_FEATURES={
-        TYPE_AUDIT_LOGS: 90,
-    },
+    RTD_DEFAULT_FEATURES=dict([RTDProductFeature(TYPE_AUDIT_LOGS, value=90).to_item()]),
 )
 class OrganizationUnspecifiedChooser(TestCase):
     def setUp(self):
@@ -428,9 +427,7 @@ def test_choose_organization_edit(self):
 
 @override_settings(
     RTD_ALLOW_ORGANIZATIONS=True,
-    RTD_DEFAULT_FEATURES={
-        TYPE_AUDIT_LOGS: 90,
-    },
+    RTD_DEFAULT_FEATURES=dict([RTDProductFeature(TYPE_AUDIT_LOGS, value=90).to_item()]),
 )
 class OrganizationUnspecifiedSingleOrganizationRedirect(TestCase):
     def setUp(self):
@@ -456,9 +453,7 @@ def test_unspecified_slug_redirects_to_organization_edit(self):
 
 @override_settings(
     RTD_ALLOW_ORGANIZATIONS=True,
-    RTD_DEFAULT_FEATURES={
-        TYPE_AUDIT_LOGS: 90,
-    },
+    RTD_DEFAULT_FEATURES=dict([RTDProductFeature(TYPE_AUDIT_LOGS, value=90).to_item()]),
 )
 class OrganizationUnspecifiedNoOrganizationRedirect(TestCase):
     def setUp(self):