-
-
Notifications
You must be signed in to change notification settings - Fork 46.8k
Get user tweets #5593
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Get user tweets #5593
Changes from all commits
010c4b8
8d3cbdc
6d7546c
184709e
71f2008
bccb8f2
f515e7a
bb67328
8b13026
e0c4d44
18dd999
1863372
e71c24a
c5810f1
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -14,5 +14,6 @@ sklearn | |
statsmodels | ||
sympy | ||
tensorflow | ||
tweepy | ||
types-requests | ||
xgboost |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,60 @@ | ||
import csv | ||
|
||
import tweepy | ||
|
||
# Twitter API credentials | ||
consumer_key = "" | ||
consumer_secret = "" | ||
access_key = "" | ||
access_secret = "" | ||
|
||
|
||
def get_all_tweets(screen_name: str) -> None: | ||
FirePing32 marked this conversation as resolved.
Show resolved
Hide resolved
FirePing32 marked this conversation as resolved.
Show resolved
Hide resolved
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. As there is no test file in this pull request nor any test function or class in the file There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is doctest necessary for these type of algos ? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We love doctests if you can figure out how to do them but it is difficult in web_programming. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @cclauss Let me try. Haven't tested CSV files with doctests before. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @cclauss Setting up doctest for this algorithm is becoming too complex. Better this PR be merged without it.
FirePing32 marked this conversation as resolved.
Show resolved
Hide resolved
FirePing32 marked this conversation as resolved.
Show resolved
Hide resolved
FirePing32 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
# authorize twitter, initialize tweepy | ||
auth = tweepy.OAuthHandler(consumer_key, consumer_secret) | ||
auth.set_access_token(access_key, access_secret) | ||
api = tweepy.API(auth) | ||
|
||
# initialize a list to hold all the tweepy Tweets | ||
alltweets = [] | ||
|
||
# make initial request for most recent tweets (200 is the maximum allowed count) | ||
new_tweets = api.user_timeline(screen_name=screen_name, count=200) | ||
|
||
# save most recent tweets | ||
alltweets.extend(new_tweets) | ||
|
||
# save the id of the oldest tweet less one | ||
oldest = alltweets[-1].id - 1 | ||
|
||
# keep grabbing tweets until there are no tweets left to grab | ||
while len(new_tweets) > 0: | ||
print(f"getting tweets before {oldest}") | ||
|
||
# all subsiquent requests use the max_id param to prevent duplicates | ||
new_tweets = api.user_timeline( | ||
screen_name=screen_name, count=200, max_id=oldest | ||
) | ||
|
||
# save most recent tweets | ||
alltweets.extend(new_tweets) | ||
|
||
# update the id of the oldest tweet less one | ||
oldest = alltweets[-1].id - 1 | ||
|
||
print(f"...{len(alltweets)} tweets downloaded so far") | ||
|
||
# transform the tweepy tweets into a 2D array that will populate the csv | ||
outtweets = [[tweet.id_str, tweet.created_at, tweet.text] for tweet in alltweets] | ||
|
||
# write the csv | ||
with open(f"new_{screen_name}_tweets.csv", "w") as f: | ||
writer = csv.writer(f) | ||
writer.writerow(["id", "created_at", "text"]) | ||
writer.writerows(outtweets) | ||
|
||
|
||
if __name__ == "__main__": | ||
# pass in the username of the account you want to download | ||
get_all_tweets("FirePing32") |
Uh oh!
There was an error while loading. Please reload this page.