Add GitHub App service

docs/dev/github-app.rst

@@ -0,0 +1,41 @@
+GitHub App
+==========
+
+Our GitHub App integration consists of a GitHub App (one for each platform, readthedocs.org and readthedocs.com),
+which can be installed on a user's account or organization.
+
+After installing the GitHub App, users can grant acccess to all repositories or select specific repositories,
+this allows Read the Docs to access the repositories and perform actions on them, such as reporting build statuses,
+and subscribe to events like push and pull request events.
+
+Unlike our other Git integrations, this doesn't rely on user's OAuth2 tokens to interact with the repositories, instead it makes use of the installation.
+As long as the installation is active and has access to the repositories, Read the Docs can perform actions on them.
+
+You can read more about GitHub Apps in `GitHub's documentation <https://docs.github.com/en/apps/overview>`__.
+
+Modeling
+--------
+
+Since we no longer make use of user's OAuth2 tokens, we need to keep track of all the installations of our GitHub App in the database (``GitHubAppInstallation`` model).
+Each remote repository has a foreign key to the installation that has access to it,
+so we can use that installation to perform actions on the repository.
+
+Keeping data in sync
+--------------------
+
+Once a GitHub app is installed, GitHub will send webhook events to our application (``readthedocs.oauth.views``) related to the installation
+(even when the installation is uninstalled or revoked), repositories, and organizations.
+This allows us to always have the ``RemoteRepository`` and ``RemoteOrganization`` models and its permissions in sync with GitHub.
+
+The only use case for using the user's OAuth2 token is to get all the installations the user has access to,
+this helps us to keep permissions in sync with GitHub.
+
+Security
+--------
+
+- All webhooks are signed with a secret, which we verify before processing each event.
+- Since we make use of the installation to perform actions on the repositories instead of the user's OAuth2 token,
+  we make sure that only users with admin permissions on the repository can link the repository to a Read the Docs project.
+- Once we lose access to a repository (e.g. the installation is uninstalled or revoked, or the project was deselected from the installation),
+  we remove the remote repository from the database, as we don't want to keep the relation bettween the project and the repository.
+  This is to prevent connecting the repository to the project again without the user's consent if they grant access to the repository again.

docs/dev/index.rst

@@ -29,5 +29,6 @@ or taking the open source Read the Docs codebase for your own custom installatio
    server-side-search
    search-integration
    subscriptions
+   github-app
    settings
    tests

docs/dev/install.rst

@@ -270,17 +270,19 @@ Configuring GitHub App
 ~~~~~~~~~~~~~~~~~~~~~~
 
 - Create a new GitHub app from https://github.com/settings/apps/new.
-- Callback URL should be ``http://dev.readthedocs.org/accounts/githubapp/login/callback/``.
+- Callback URL should be ``http://devthedocs.org/accounts/githubapp/login/callback/``.
 - Keep marked "Expire user authorization tokens"
 - Activate the webhook, and set the URL to one provided by a service like `Webhook.site <https://docs.webhook.site/cli.html>`__ to forward all incoming webhooks to your local development instance.
-  You should forward all events to ``http://dev.readthedocs.org/webhook/githubapp/``.
+  You should forward all events to ``http://devthedocs.org/webhook/githubapp/``.
 - In permissions, select the following:
+
   - Repository permissions: Commit statuses (read and write, so we can create commit statuses),
     Contents (read only, so we can clone repos with a token),
     Metadata (read only, so we read the repo collaborators),
     Pull requests (read and write, so we can post a comment on PRs in the future).
   - Organization permissions: Members (read only so we can read the organization members).
   - Account permissions: Email addresses (read only, so allauth can fetch all verified emails).
+
 - Subscribe to the following events: Installation target, Member, Organization, Membership, Pull request, Push, and Repository.
 - Copy the "Client ID" and "Client Secret" and set them as :ref:`environment variables <settings:Allauth secrets>`.
 - Generate a webhook secret and a private key from the GitHub App settings,

docs/dev/settings.rst

@@ -174,7 +174,7 @@ You can use the following environment variables to set the settings used by the
 
 .. envvar:: RTD_GITHUB_APP_ID
 .. envvar:: RTD_GITHUB_APP_NAME
-.. envvar:: RTD_GITHUB_PRIVATE_KEY
+.. envvar:: RTD_GITHUB_APP_PRIVATE_KEY
 .. envvar:: RTD_GITHUB_APP_WEBHOOK_SECRET
 
 Stripe secrets

readthedocs/oauth/models.py

@@ -108,8 +108,10 @@ def delete_repositories(self, repository_ids: list[int] | None = None):
         :param repository_ids: List of repository ids (remote ID) to delete.
          If None, all repositories will be considered for deletion.
         """
+        # repository_ids is optionaal (None, which means all repositories),
+        # but if it's an empty list, we don't want to delete anything.
         if repository_ids is not None and not repository_ids:
-            log.info("No repositories to delete")
+            log.info("No remote repositories to delete")
             return
 
         remote_organizations = RemoteOrganization.objects.filter(
@@ -128,7 +130,7 @@ def delete_repositories(self, repository_ids: list[int] | None = None):
 
         count, deleted = remote_repositories.delete()
         log.info(
-            "Deleted repositories projects",
+            "Deleted remote repositories",
             count=count,
             deleted=deleted,
             installation_id=self.installation_id,
@@ -139,7 +141,7 @@ def delete_repositories(self, repository_ids: list[int] | None = None):
             repositories=None,
         ).delete()
         log.info(
-            "Deleted orphaned organizations",
+            "Deleted orphaned remote organizations",
             count=count,
             deleted=deleted,
             installation_id=self.installation_id,
@@ -152,7 +154,7 @@ def delete_organization(self, organization_id: int):
             vcs_provider=GITHUB_APP,
         ).delete()
         log.info(
-            "Deleted organization",
+            "Deleted remote organization",
             count=count,
             deleted=deleted,
             organization_id=organization_id,

readthedocs/oauth/services/githubapp.py

@@ -37,26 +37,26 @@ def __init__(self, installation: GitHubAppInstallation):
         self.installation = installation
 
     @cached_property
-    def gha_client(self):
+    def gh_app_client(self):
         return get_gh_app_client()
 
     @cached_property
     def app_installation(self) -> GHInstallation:
         """
         Return the installation object from the GitHub API.
 
-        Usefull to interact with installation related endpoints.
+        Useful to interact with installation related endpoints.
 
         If the installation is no longer accessible, this will raise a GithubException.
         """
-        return self.gha_client.get_app_installation(
+        return self.gh_app_client.get_app_installation(
             self.installation.installation_id,
         )
 
     @cached_property
     def installation_client(self) -> Github:
         """Return a client authenticated as the GitHub installation to interact with the GH API."""
-        return self.gha_client.get_github_for_installation(self.installation.installation_id)
+        return self.gh_app_client.get_github_for_installation(self.installation.installation_id)
 
     @classmethod
     def for_project(cls, project):
@@ -96,7 +96,7 @@ def for_user(cls, user):
 
            User access tokens expire after 8 hours, but our OAuth2 client should handle refreshing the token.
            But, the refresh token expires after 6 months, in order to refresh that token,
-           the user needs to sign in using GitHub again (just a normal sing-in, not a re-authorization or sign-up).
+           the user needs to sign in using GitHub again (just a normal sign in, not a re-authorization or sign-up).
         """
         social_accounts = SocialAccount.objects.filter(
             user=user,
@@ -229,6 +229,7 @@ def sync(self):
 
     def update_or_create_repositories(self, repository_ids: list[int]):
         """Update or create repositories from the given list of repository IDs."""
+        repositories_to_delete = []
         for repository_id in repository_ids:
             try:
                 # NOTE: we save the repository ID as a string in our database,
@@ -245,10 +246,13 @@ def update_or_create_repositories(self, repository_ids: list[int]):
                 # we remove the repository from the database,
                 # and clean up the collaborators and relations.
                 if e.status in [404, 403]:
-                    self.installation.delete_repositories([repository_id])
+                    repositories_to_delete.append(repository_id)
                 continue
             self._create_or_update_repository_from_gh(repo)
 
+        if repositories_to_delete:
+            self.installation.delete_repositories(repositories_to_delete)
+
     def _create_or_update_repository_from_gh(
         self, gh_repo: GHRepository
     ) -> RemoteRepository | None:
@@ -349,6 +353,8 @@ def _resync_collaborators(self, gh_repo: GHRepository, remote_repo: RemoteReposi
         Sync collaborators of a repository with the database.
 
         This method will remove collaborators that are no longer in the list.
+
+        See https://docs.github.com/en/rest/collaborators/collaborators?apiVersion=2022-11-28#list-repository-collaborators.
         """
         collaborators = {
             collaborator.id: collaborator for collaborator in gh_repo.get_collaborators()
@@ -453,7 +459,7 @@ def get_clone_token(self, project):
         # We can also pass a specific permissions object to get a token with specific permissions
         # if we want to scope this token even more.
         try:
-            access_token = self.gha_client.get_access_token(self.installation.installation_id)
+            access_token = self.gh_app_client.get_access_token(self.installation.installation_id)
             return f"x-access-token:{access_token.token}"
         except GithubException:
             log.info(

readthedocs/oauth/views.py

@@ -39,7 +39,7 @@ class GitHubAppWebhookView(APIView):
     authentication_classes = []
 
     @cached_property
-    def gha_client(self):
+    def gh_app_client(self):
         return get_gh_app_client()
 
     def post(self, request):
@@ -318,6 +318,7 @@ def _handle_repository_event(self):
         # - deleted: If the repository was linked to an installation,
         #   GH will be trigger an installation_repositories event.
         # - archived/unarchived: We don't do anything with archived repositories.
+        return
 
     def _handle_push_event(self):
         """
@@ -429,6 +430,7 @@ def _handle_organization_event(self):
 
         # Ignore other actions:
         # - member_invited: We don't do anything with invited members.
+        return
 
     def _handle_member_event(self):
         """
@@ -518,7 +520,7 @@ def _get_or_create_installation(self, sync_repositories_on_create: bool = True):
         # so we can create the installation object if needed.
         if not target_id or not target_type:
             log.debug("Incomplete installation object, fetching from the API")
-            gh_installation = self.gha_client.get_app_installation(installation_id)
+            gh_installation = self.gh_app_client.get_app_installation(installation_id)
             target_id = gh_installation.target_id
             target_type = gh_installation.target_type
             data = data.copy()