From 08b7941f59ce447e5a5b19f766c6ac35d212c19b Mon Sep 17 00:00:00 2001 From: Santos Gallegos Date: Thu, 18 Jul 2019 14:53:37 -0500 Subject: [PATCH 01/12] First draft for organizations design document --- docs/development/design/organizations.rst | 74 +++++++++++++++++++++++ 1 file changed, 74 insertions(+) create mode 100644 docs/development/design/organizations.rst diff --git a/docs/development/design/organizations.rst b/docs/development/design/organizations.rst new file mode 100644 index 00000000000..fb8e5b6a28d --- /dev/null +++ b/docs/development/design/organizations.rst @@ -0,0 +1,74 @@ +Organizations +============= + +Currently we don't support organizations in the community site +(a way to group different projects), +we only support individual accounts. + +Several integrations that we support have organizations, +where users group their repositories. + +Why support organizations in the community site? +------------------------------------------------ + +We support organizations in the commercial site, +having no organizations in the community site makes the code maintenance difficult. +Having organizations in the community site will make the differences between both more easy to manage. + +Users from the community site can have organizations in external sites from where we import their projects +(like GitHub, Gitlab). +Currently users have all projects from different organizations in their account. +Having not a clear way to group/separate those. + +How are we going to support organizations? +------------------------------------------ + +Currently users can have several projects. +With organizations we have two paths: + +#. Support users and organizations to own projects +#. Support only organizations to own projects + +With 1, the migration process would be straightforward for the community site. +With 2, the migration process would be straightforward for the commercial site. + +How to migrate current projects +------------------------------- + +If we choose to go with the second option from :ref:`development/design/organizations:How are we going to support organizations?` +we'll need to define a migration plan, +like creating an organization for all current projects. + +But, for whatever option we choose, +we'll need to move the models defined in the corporate site +to the community site. + +We should start by removing unused features and dead code from the organizations in the corporate site. +After that, it should be more easy to move the organizations *app* (or part of it) +to the community site (and not changes in table names would be required). + +Additionally, we are going to need to edit the current querysets, modify/add UI elements, +and add new endpoints to the API (v3 only). + +What features of organizations are we going to support? +------------------------------------------------------- + +We have this features present in the commercial site: + +- Owners +- Teams +- Permissions +- Subscriptions + +Owners should be included to represent owners of the current organization. + +Teams, this is also handy to manage access to different projects under the same organization. + +Permissions, +currently we have two type of permissions for teams: admin and read only. +Read only permissions doesn't make sense in the community site since we only support public projects/versions +(we do support private versions now, but we are planning to remove those). +So, we should only support admin permissions for teams. + +Subscriptions, this is only valid for the corporate site, +since we don't charge for use in the community site. From fa3533836497fc14fc1e5a7dd053445bb4480f6a Mon Sep 17 00:00:00 2001 From: Santos Gallegos Date: Thu, 18 Jul 2019 15:03:08 -0500 Subject: [PATCH 02/12] Namespace --- docs/development/design/organizations.rst | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/docs/development/design/organizations.rst b/docs/development/design/organizations.rst index fb8e5b6a28d..db9a2934235 100644 --- a/docs/development/design/organizations.rst +++ b/docs/development/design/organizations.rst @@ -72,3 +72,17 @@ So, we should only support admin permissions for teams. Subscriptions, this is only valid for the corporate site, since we don't charge for use in the community site. + +Namespace +--------- + +Currently we use the project's slug as namespace, +in the commercial site we use the combination of `organization.slug` + `project.slug` as namespace. + +For the community site probably this approach isn't the best, +since we alway serve docs publicly from `slug.readthedocs.io`. +And most of the users don't have a custom domain. + +We could keep the current behavior for the community site and use `organization.slug` + `project.slug` for the corporate site, +since in the corporate site we don't care so much about a unique namespace between all users, but a unique namespace per organization. +We can refactor the way we get the namespace to be more easy to manage in both sites. From 073af09e6aaab5b8a44a5b313d74ac55f61abf58 Mon Sep 17 00:00:00 2001 From: Santos Gallegos Date: Thu, 18 Jul 2019 15:06:34 -0500 Subject: [PATCH 03/12] More about migration --- docs/development/design/organizations.rst | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/development/design/organizations.rst b/docs/development/design/organizations.rst index db9a2934235..67af6e9f807 100644 --- a/docs/development/design/organizations.rst +++ b/docs/development/design/organizations.rst @@ -43,7 +43,11 @@ But, for whatever option we choose, we'll need to move the models defined in the corporate site to the community site. -We should start by removing unused features and dead code from the organizations in the corporate site. +From the technical site +~~~~~~~~~~~~~~~~~~~~~~~ + +We should start by removing unused features and dead code from the organizations in the corporate site, +and isolate/separate the models to be moved from the ones that aren't going to be moved. After that, it should be more easy to move the organizations *app* (or part of it) to the community site (and not changes in table names would be required). From 92821db1055ed14400aa3e5ac5c5d3c5a567c45e Mon Sep 17 00:00:00 2001 From: Santos Gallegos Date: Tue, 27 Aug 2019 14:43:03 -0500 Subject: [PATCH 04/12] More details on moving organizations --- docs/development/design/organizations.rst | 113 ++++++++++++++++------ 1 file changed, 83 insertions(+), 30 deletions(-) diff --git a/docs/development/design/organizations.rst b/docs/development/design/organizations.rst index 67af6e9f807..7f49a6394ae 100644 --- a/docs/development/design/organizations.rst +++ b/docs/development/design/organizations.rst @@ -8,8 +8,8 @@ we only support individual accounts. Several integrations that we support have organizations, where users group their repositories. -Why support organizations in the community site? ------------------------------------------------- +Why move organizations in the community site? +--------------------------------------------- We support organizations in the commercial site, having no organizations in the community site makes the code maintenance difficult. @@ -20,39 +20,22 @@ Users from the community site can have organizations in external sites from wher Currently users have all projects from different organizations in their account. Having not a clear way to group/separate those. +There is also a product decision. +Supporting organizations only on code, +but enable the feature only on the commercial site. + How are we going to support organizations? ------------------------------------------ -Currently users can have several projects. -With organizations we have two paths: - -#. Support users and organizations to own projects -#. Support only organizations to own projects - -With 1, the migration process would be straightforward for the community site. -With 2, the migration process would be straightforward for the commercial site. - -How to migrate current projects -------------------------------- - -If we choose to go with the second option from :ref:`development/design/organizations:How are we going to support organizations?` -we'll need to define a migration plan, -like creating an organization for all current projects. +Currently only users can own projects. +With organizations this is going to change to: +Users and organizations can own projects. -But, for whatever option we choose, -we'll need to move the models defined in the corporate site -to the community site. +With this, the migration process would be straightforward for the community site. -From the technical site -~~~~~~~~~~~~~~~~~~~~~~~ - -We should start by removing unused features and dead code from the organizations in the corporate site, -and isolate/separate the models to be moved from the ones that aren't going to be moved. -After that, it should be more easy to move the organizations *app* (or part of it) -to the community site (and not changes in table names would be required). - -Additionally, we are going to need to edit the current querysets, modify/add UI elements, -and add new endpoints to the API (v3 only). +For the commercial site we still have a decision. +Are we going to support users and organizations to own projects? +Or just organizations? What features of organizations are we going to support? ------------------------------------------------------- @@ -77,6 +60,68 @@ So, we should only support admin permissions for teams. Subscriptions, this is only valid for the corporate site, since we don't charge for use in the community site. +How to migrate current projects +------------------------------- + +Since we are not replacing the current implementation, +we don't need to migrate current projects from the community site nor from the corporate site. + +How to migrate the organizations app +------------------------------------ + +The migration can be split in: + +#. Remove/simplify code from the migrations app on the corporate site. +#. Isolate/separate models and code that isn't going to be moved. +#. Start by moving the models, managers, and figure out how to handle migrations. +#. Move the rest of the code as needed. +#. Activate organizations app on the community site. +#. Integrate the code from the community site to the new code. + +We should start by removing unused features and dead code from the organizations in the corporate site, +and simplify existing code if possible (some of this was already done). + +Isolate/separate the models to be moved from the ones that aren't going to be moved. +We should move the models that aren't going to me moved to another app. + +- Plan +- PlanFeature +- Subscription + +This app can be named *subscriptions*. +We can get around the table names and migrations by setting the explicitly the table name to `organizations_`, +and doing a fake migration. +Following suggestions in https://stackoverflow.com/questions/48860227/moving-multiple-models-from-one-django-app-to-another. +Code related to subscriptions should be moved out from the organizations app. + +After that, it should be more easy to move the organizations *app* (or part of it) +to the community site (and not changes in table names would be required). + +We start by moving the models. + +- Organization +- OrganizationOwner +- Team +- TeamInvite +- TeamMember + +Migrations aren't moved, since all current migrations depend on other models that aren't +going to be moved. +In the community site we run an initial migration, +for the corporate site we run a fake migration. + +For managers and querysets that depend on subscriptions, +we can use our pattern to make overridable classes (inheriting from ``SettingsOverrideObject``). + +Templates, urls, views, forms, notifications, signals, tasks can be moved later +(we just need to make use from the models from the `readthedocs.organizations` module). + +If we decide to integrate organizations in the community site, +we can enable the app. + +After the app is moved, +we can move more code that depends on organizations to the community site. + Namespace --------- @@ -90,3 +135,11 @@ And most of the users don't have a custom domain. We could keep the current behavior for the community site and use `organization.slug` + `project.slug` for the corporate site, since in the corporate site we don't care so much about a unique namespace between all users, but a unique namespace per organization. We can refactor the way we get the namespace to be more easy to manage in both sites. + +Future Changes +-------------- + +Changes that aren't needed immediately after the migration, +but that should be done: + +Edit the current querysets, modify/add UI elements, and add new endpoints to the API (v3 only). From d6cd4548198a3d1abdfa0d212a4e66bbf0791e60 Mon Sep 17 00:00:00 2001 From: Santos Gallegos Date: Mon, 7 Oct 2019 12:30:39 -0500 Subject: [PATCH 05/12] Typos Co-Authored-By: David Fischer --- docs/development/design/organizations.rst | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/development/design/organizations.rst b/docs/development/design/organizations.rst index 7f49a6394ae..58d491116a0 100644 --- a/docs/development/design/organizations.rst +++ b/docs/development/design/organizations.rst @@ -5,14 +5,14 @@ Currently we don't support organizations in the community site (a way to group different projects), we only support individual accounts. -Several integrations that we support have organizations, -where users group their repositories. +Several integrations that we support like GitHub and Bitbucket have organizations, +where users group their repositories and manage them in groups rather than individually. Why move organizations in the community site? --------------------------------------------- We support organizations in the commercial site, -having no organizations in the community site makes the code maintenance difficult. +having no organizations in the community site makes the code maintenance difficult for Read the Docs developers. Having organizations in the community site will make the differences between both more easy to manage. Users from the community site can have organizations in external sites from where we import their projects @@ -40,7 +40,7 @@ Or just organizations? What features of organizations are we going to support? ------------------------------------------------------- -We have this features present in the commercial site: +We have the following features in the commercial site that we don't have on the community site: - Owners - Teams @@ -94,8 +94,8 @@ and doing a fake migration. Following suggestions in https://stackoverflow.com/questions/48860227/moving-multiple-models-from-one-django-app-to-another. Code related to subscriptions should be moved out from the organizations app. -After that, it should be more easy to move the organizations *app* (or part of it) -to the community site (and not changes in table names would be required). +After that, it should be easier to move the organizations *app* (or part of it) +to the community site (and no changes to table names would be required). We start by moving the models. @@ -129,7 +129,7 @@ Currently we use the project's slug as namespace, in the commercial site we use the combination of `organization.slug` + `project.slug` as namespace. For the community site probably this approach isn't the best, -since we alway serve docs publicly from `slug.readthedocs.io`. +since we always serve docs publicly from `slug.readthedocs.io`. And most of the users don't have a custom domain. We could keep the current behavior for the community site and use `organization.slug` + `project.slug` for the corporate site, From c15f9036a7796bf0526158f6324673b599d1a539 Mon Sep 17 00:00:00 2001 From: Santos Gallegos Date: Mon, 7 Oct 2019 12:44:05 -0500 Subject: [PATCH 06/12] Feedback --- docs/development/design/organizations.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/development/design/organizations.rst b/docs/development/design/organizations.rst index 58d491116a0..669f4c5c8b4 100644 --- a/docs/development/design/organizations.rst +++ b/docs/development/design/organizations.rst @@ -71,7 +71,7 @@ How to migrate the organizations app The migration can be split in: -#. Remove/simplify code from the migrations app on the corporate site. +#. Remove/simplify code from the organizations app on the corporate site. #. Isolate/separate models and code that isn't going to be moved. #. Start by moving the models, managers, and figure out how to handle migrations. #. Move the rest of the code as needed. @@ -126,13 +126,13 @@ Namespace --------- Currently we use the project's slug as namespace, -in the commercial site we use the combination of `organization.slug` + `project.slug` as namespace. +in the commercial site we use the combination of ``organization.slug`` + ``project.slug`` as namespace. For the community site probably this approach isn't the best, -since we always serve docs publicly from `slug.readthedocs.io`. +since we always serve docs publicly from ``slug.readthedocs.io``. And most of the users don't have a custom domain. -We could keep the current behavior for the community site and use `organization.slug` + `project.slug` for the corporate site, +We could keep the current behavior for the community site and use ``organization.slug`` + ``project.slug`` for the corporate site, since in the corporate site we don't care so much about a unique namespace between all users, but a unique namespace per organization. We can refactor the way we get the namespace to be more easy to manage in both sites. From 35f997eaca5869d0c072b6ff3318066b98f52c6c Mon Sep 17 00:00:00 2001 From: Santos Gallegos Date: Tue, 8 Oct 2019 12:22:18 -0500 Subject: [PATCH 07/12] Typo --- docs/development/design/organizations.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/development/design/organizations.rst b/docs/development/design/organizations.rst index 669f4c5c8b4..983d022743b 100644 --- a/docs/development/design/organizations.rst +++ b/docs/development/design/organizations.rst @@ -114,7 +114,7 @@ For managers and querysets that depend on subscriptions, we can use our pattern to make overridable classes (inheriting from ``SettingsOverrideObject``). Templates, urls, views, forms, notifications, signals, tasks can be moved later -(we just need to make use from the models from the `readthedocs.organizations` module). +(we just need to make use of the models from the `readthedocs.organizations` module). If we decide to integrate organizations in the community site, we can enable the app. From 13ff1bd5aa0b32e0f5da6a956d868c3e0dc63147 Mon Sep 17 00:00:00 2001 From: Santos Gallegos Date: Thu, 10 Oct 2019 18:35:24 -0500 Subject: [PATCH 08/12] Little improvements --- docs/development/design/organizations.rst | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/development/design/organizations.rst b/docs/development/design/organizations.rst index 983d022743b..ebb37271d39 100644 --- a/docs/development/design/organizations.rst +++ b/docs/development/design/organizations.rst @@ -80,6 +80,9 @@ The migration can be split in: We should start by removing unused features and dead code from the organizations in the corporate site, and simplify existing code if possible (some of this was already done). +Another simplification that can be done is: +make the relationship between the models ``Organization`` and ``Project`` one to many +(currently many to many). Isolate/separate the models to be moved from the ones that aren't going to be moved. We should move the models that aren't going to me moved to another app. @@ -89,7 +92,7 @@ We should move the models that aren't going to me moved to another app. - Subscription This app can be named *subscriptions*. -We can get around the table names and migrations by setting the explicitly the table name to `organizations_`, +We can get around the table names and migrations by setting the explicitly the table name to ``organizations_``, and doing a fake migration. Following suggestions in https://stackoverflow.com/questions/48860227/moving-multiple-models-from-one-django-app-to-another. Code related to subscriptions should be moved out from the organizations app. @@ -114,7 +117,7 @@ For managers and querysets that depend on subscriptions, we can use our pattern to make overridable classes (inheriting from ``SettingsOverrideObject``). Templates, urls, views, forms, notifications, signals, tasks can be moved later -(we just need to make use of the models from the `readthedocs.organizations` module). +(we just need to make use of the models from the ``readthedocs.organizations`` module). If we decide to integrate organizations in the community site, we can enable the app. From 03144541d9ec59480db946dc78f0123227b30243 Mon Sep 17 00:00:00 2001 From: Santos Gallegos Date: Mon, 14 Oct 2019 12:14:45 -0500 Subject: [PATCH 09/12] Add user interface to the design doc --- docs/development/design/organizations.rst | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-) diff --git a/docs/development/design/organizations.rst b/docs/development/design/organizations.rst index ebb37271d39..cd90380cdd0 100644 --- a/docs/development/design/organizations.rst +++ b/docs/development/design/organizations.rst @@ -120,11 +120,24 @@ Templates, urls, views, forms, notifications, signals, tasks can be moved later (we just need to make use of the models from the ``readthedocs.organizations`` module). If we decide to integrate organizations in the community site, -we can enable the app. +we can add/move the UI elements and enable the app. After the app is moved, we can move more code that depends on organizations to the community site. +User interface +-------------- + +Import page +~~~~~~~~~~~ + +Project's list +~~~~~~~~~~~~~~ + +Organization's list +~~~~~~~~~~~~~~~~~~~ + + Namespace --------- @@ -145,4 +158,4 @@ Future Changes Changes that aren't needed immediately after the migration, but that should be done: -Edit the current querysets, modify/add UI elements, and add new endpoints to the API (v3 only). +Add new endpoints to the API (v3 only). From 2bb089f45359791b4300c2836949c36a74f4907e Mon Sep 17 00:00:00 2001 From: Santos Gallegos Date: Mon, 14 Oct 2019 15:42:38 -0500 Subject: [PATCH 10/12] Update doc --- docs/development/design/organizations.rst | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/docs/development/design/organizations.rst b/docs/development/design/organizations.rst index cd90380cdd0..7d4a05cd0ab 100644 --- a/docs/development/design/organizations.rst +++ b/docs/development/design/organizations.rst @@ -80,9 +80,6 @@ The migration can be split in: We should start by removing unused features and dead code from the organizations in the corporate site, and simplify existing code if possible (some of this was already done). -Another simplification that can be done is: -make the relationship between the models ``Organization`` and ``Project`` one to many -(currently many to many). Isolate/separate the models to be moved from the ones that aren't going to be moved. We should move the models that aren't going to me moved to another app. @@ -158,4 +155,6 @@ Future Changes Changes that aren't needed immediately after the migration, but that should be done: -Add new endpoints to the API (v3 only). +- Add new endpoints to the API (v3 only). +- Make the relationship between the models ``Organization`` and ``Project`` one to many + (currently many to many). From f2ffd1475b1b07c1efec534db35e9893d2569c9b Mon Sep 17 00:00:00 2001 From: Santos Gallegos Date: Wed, 16 Oct 2019 17:21:35 -0500 Subject: [PATCH 11/12] Clarification on design decisions --- docs/development/design/organizations.rst | 43 ++++++++++------------- 1 file changed, 18 insertions(+), 25 deletions(-) diff --git a/docs/development/design/organizations.rst b/docs/development/design/organizations.rst index 7d4a05cd0ab..e272fc220f2 100644 --- a/docs/development/design/organizations.rst +++ b/docs/development/design/organizations.rst @@ -20,22 +20,20 @@ Users from the community site can have organizations in external sites from wher Currently users have all projects from different organizations in their account. Having not a clear way to group/separate those. -There is also a product decision. -Supporting organizations only on code, -but enable the feature only on the commercial site. +We are going to first move the code, +and after that enable the feature on the community site. How are we going to support organizations? ------------------------------------------ -Currently only users can own projects. +Currently only users can own projects in the community site. With organizations this is going to change to: Users and organizations can own projects. With this, the migration process would be straightforward for the community site. -For the commercial site we still have a decision. -Are we going to support users and organizations to own projects? -Or just organizations? +For the commercial site we are only to allow organizations to own projects for now +(since the we have only subscriptions per organizations). What features of organizations are we going to support? ------------------------------------------------------- @@ -77,6 +75,7 @@ The migration can be split in: #. Move the rest of the code as needed. #. Activate organizations app on the community site. #. Integrate the code from the community site to the new code. +#. UI changes We should start by removing unused features and dead code from the organizations in the corporate site, and simplify existing code if possible (some of this was already done). @@ -91,7 +90,9 @@ We should move the models that aren't going to me moved to another app. This app can be named *subscriptions*. We can get around the table names and migrations by setting the explicitly the table name to ``organizations_``, and doing a fake migration. -Following suggestions in https://stackoverflow.com/questions/48860227/moving-multiple-models-from-one-django-app-to-another. +Following suggestions in https://stackoverflow.com/questions/48860227/moving-multiple-models-from-one-django-app-to-another, +that way we avoid having downtime during the migration and any inconvenient caused from renaming the tables manually. + Code related to subscriptions should be moved out from the organizations app. After that, it should be easier to move the organizations *app* (or part of it) @@ -109,6 +110,7 @@ Migrations aren't moved, since all current migrations depend on other models tha going to be moved. In the community site we run an initial migration, for the corporate site we run a fake migration. +The migrations left from the commercial site can be removed after that. For managers and querysets that depend on subscriptions, we can use our pattern to make overridable classes (inheriting from ``SettingsOverrideObject``). @@ -122,32 +124,22 @@ we can add/move the UI elements and enable the app. After the app is moved, we can move more code that depends on organizations to the community site. -User interface --------------- - -Import page -~~~~~~~~~~~ - -Project's list -~~~~~~~~~~~~~~ - -Organization's list -~~~~~~~~~~~~~~~~~~~ - - Namespace --------- Currently we use the project's slug as namespace, -in the commercial site we use the combination of ``organization.slug`` + ``project.slug`` as namespace. +in the commercial site we use the combination of ``organization.slug`` + ``project.slug`` as namespace, +since in the corporate site we don't care so much about a unique namespace between all users, +but a unique namespace per organization. For the community site probably this approach isn't the best, since we always serve docs publicly from ``slug.readthedocs.io``. And most of the users don't have a custom domain. -We could keep the current behavior for the community site and use ``organization.slug`` + ``project.slug`` for the corporate site, -since in the corporate site we don't care so much about a unique namespace between all users, but a unique namespace per organization. -We can refactor the way we get the namespace to be more easy to manage in both sites. +The corporate site will use ``organization.slug`` + ``project.slug`` as slug, +And the community site will always use ``project.slug`` as slug, even if the project belongs to an organization. + +We need to refactor the way we get the namespace to be more easy to manage in both sites. Future Changes -------------- @@ -155,6 +147,7 @@ Future Changes Changes that aren't needed immediately after the migration, but that should be done: +- UI for organizations in the community site. - Add new endpoints to the API (v3 only). - Make the relationship between the models ``Organization`` and ``Project`` one to many (currently many to many). From 45e6c573f34adf13fec87c9dcf26c81117c31427 Mon Sep 17 00:00:00 2001 From: Santos Gallegos Date: Wed, 16 Oct 2019 17:22:20 -0500 Subject: [PATCH 12/12] Typo --- docs/development/design/organizations.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/development/design/organizations.rst b/docs/development/design/organizations.rst index e272fc220f2..bfe8d47ce54 100644 --- a/docs/development/design/organizations.rst +++ b/docs/development/design/organizations.rst @@ -91,7 +91,7 @@ This app can be named *subscriptions*. We can get around the table names and migrations by setting the explicitly the table name to ``organizations_``, and doing a fake migration. Following suggestions in https://stackoverflow.com/questions/48860227/moving-multiple-models-from-one-django-app-to-another, -that way we avoid having downtime during the migration and any inconvenient caused from renaming the tables manually. +that way we avoid having any downtime during the migration and any inconvenient caused from renaming the tables manually. Code related to subscriptions should be moved out from the organizations app.