-
-
Notifications
You must be signed in to change notification settings - Fork 3.6k
GitHub OAuth: use bigger pages to make fewer requests #9020
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Currently, we are using the default pagination that GitHub offers, which is 30. This commit increases the page size to 100 when fetching organization for a user and repositories for each organizations. Note we are already using 100 page size when fetching repositories for a user. Related to #8979
25b742f
to
12f70cf
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should test if
resp = self.get_session().get(url, data=kwargs) |
params
https://docs.python-requests.org/en/latest/user/quickstart/#passing-parameters-in-urls 🤔
A quick test shows that data is passed as the body, which get requests shouldn't have... we are only using the kwargs for gitlab, maybe it's just a happy coincidence that they handled the params inside the body as query args.
>>> r = requests.get('https://example.com', params={'foo': 'bar'})
>>> r.url
'https://example.com/?foo=bar'
>>> r.request.body
>>> r = requests.get('https://example.com', data={'foo': 'bar'})
>>> r.url
'https://example.com/'
>>> r.request.body
'foo=bar'
Maybe it's just safer to change data to params.
Good catch! This is definitely not working:
We need to use |
To pass GET attributes we need to use `params=`. https://docs.python-requests.org/en/latest/user/quickstart/#passing-parameters-in-urls
@stsewd I did a final QA locally with all the oauth services supported and all of them worked fine with the |
Currently, we are using the default pagination that GitHub offers, which is 30.
This commit increases the page size to 100 when fetching organization for a user
and repositories for each organizations.
Note we are already using 100 page size when fetching repositories for a user.
Related to #8979