Skip to content

GitHub OAuth: use bigger pages to make fewer requests #9020

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Mar 17, 2022
Merged

GitHub OAuth: use bigger pages to make fewer requests #9020

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
12f70cf
GitHub OAuth: use bigger pages to make fewer requests
humitos Mar 16, 2022
9f42b54
OAuth services: use `params=` instead of `data=` on `request.get`
humitos Mar 16, 2022
3df6089
Lint
humitos Mar 17, 2022
4f54de2
GitLab OAuth: attribute not required
humitos Mar 17, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 7 additions & 10 deletions readthedocs/oauth/services/github.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
"""OAuth utility functions."""

import json
import structlog
import re

import structlog
from allauth.socialaccount.models import SocialToken
from allauth.socialaccount.providers.github.views import GitHubOAuth2Adapter
from django.conf import settings
Expand All @@ -12,10 +12,7 @@

from readthedocs.api.v2.client import api
from readthedocs.builds import utils as build_utils
from readthedocs.builds.constants import (
BUILD_STATUS_SUCCESS,
SELECT_BUILD_STATUS,
)
from readthedocs.builds.constants import BUILD_STATUS_SUCCESS, SELECT_BUILD_STATUS
from readthedocs.core.permissions import AdminPermission
from readthedocs.integrations.models import Integration

Expand All @@ -40,7 +37,7 @@ def sync_repositories(self):
remote_repositories = []

try:
repos = self.paginate('https://api.github.com/user/repos?per_page=100')
repos = self.paginate("https://api.github.com/user/repos", per_page=100)
for repo in repos:
remote_repository = self.create_repository(repo)
remote_repositories.append(remote_repository)
Expand All @@ -58,14 +55,14 @@ def sync_organizations(self):
remote_repositories = []

try:
orgs = self.paginate('https://api.github.com/user/orgs')
orgs = self.paginate("https://api.github.com/user/orgs", per_page=100)
for org in orgs:
org_details = self.get_session().get(org['url']).json()
remote_organization = self.create_organization(org_details)
# Add repos
# TODO ?per_page=100
org_url = org["url"]
org_repos = self.paginate(
'{org_url}/repos'.format(org_url=org['url']),
f"{org_url}/repos",
per_page=100,
)

remote_organizations.append(remote_organization)
Expand Down