Skip to content

Get user tweets #5593

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Oct 26, 2021
Merged
10 changes: 9 additions & 1 deletion DIRECTORY.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,11 @@
* [Newton Raphson](https://github.com/TheAlgorithms/Python/blob/master/arithmetic_analysis/newton_raphson.py)
* [Secant Method](https://github.com/TheAlgorithms/Python/blob/master/arithmetic_analysis/secant_method.py)

## Audio Filters
* [Butterworth Filter](https://github.com/TheAlgorithms/Python/blob/master/audio_filters/butterworth_filter.py)
* [Iir Filter](https://github.com/TheAlgorithms/Python/blob/master/audio_filters/iir_filter.py)
* [Show Response](https://github.com/TheAlgorithms/Python/blob/master/audio_filters/show_response.py)

## Backtracking
* [All Combinations](https://github.com/TheAlgorithms/Python/blob/master/backtracking/all_combinations.py)
* [All Permutations](https://github.com/TheAlgorithms/Python/blob/master/backtracking/all_permutations.py)
Expand Down Expand Up @@ -60,6 +65,7 @@
* [Base64 Encoding](https://github.com/TheAlgorithms/Python/blob/master/ciphers/base64_encoding.py)
* [Base85](https://github.com/TheAlgorithms/Python/blob/master/ciphers/base85.py)
* [Beaufort Cipher](https://github.com/TheAlgorithms/Python/blob/master/ciphers/beaufort_cipher.py)
* [Bifid](https://github.com/TheAlgorithms/Python/blob/master/ciphers/bifid.py)
* [Brute Force Caesar Cipher](https://github.com/TheAlgorithms/Python/blob/master/ciphers/brute_force_caesar_cipher.py)
* [Caesar Cipher](https://github.com/TheAlgorithms/Python/blob/master/ciphers/caesar_cipher.py)
* [Cryptomath Module](https://github.com/TheAlgorithms/Python/blob/master/ciphers/cryptomath_module.py)
Expand Down Expand Up @@ -106,8 +112,8 @@

## Conversions
* [Binary To Decimal](https://github.com/TheAlgorithms/Python/blob/master/conversions/binary_to_decimal.py)
* [Binary To Octal](https://github.com/TheAlgorithms/Python/blob/master/conversions/binary_to_octal.py)
* [Binary To Hexadecimal](https://github.com/TheAlgorithms/Python/blob/master/conversions/binary_to_hexadecimal.py)
* [Binary To Octal](https://github.com/TheAlgorithms/Python/blob/master/conversions/binary_to_octal.py)
* [Decimal To Any](https://github.com/TheAlgorithms/Python/blob/master/conversions/decimal_to_any.py)
* [Decimal To Binary](https://github.com/TheAlgorithms/Python/blob/master/conversions/decimal_to_binary.py)
* [Decimal To Binary Recursion](https://github.com/TheAlgorithms/Python/blob/master/conversions/decimal_to_binary_recursion.py)
Expand Down Expand Up @@ -352,6 +358,7 @@
* [Multi Heuristic Astar](https://github.com/TheAlgorithms/Python/blob/master/graphs/multi_heuristic_astar.py)
* [Page Rank](https://github.com/TheAlgorithms/Python/blob/master/graphs/page_rank.py)
* [Prim](https://github.com/TheAlgorithms/Python/blob/master/graphs/prim.py)
* [Random Graph Generator](https://github.com/TheAlgorithms/Python/blob/master/graphs/random_graph_generator.py)
* [Scc Kosaraju](https://github.com/TheAlgorithms/Python/blob/master/graphs/scc_kosaraju.py)
* [Strongly Connected Components](https://github.com/TheAlgorithms/Python/blob/master/graphs/strongly_connected_components.py)
* [Tarjans Scc](https://github.com/TheAlgorithms/Python/blob/master/graphs/tarjans_scc.py)
Expand Down Expand Up @@ -978,6 +985,7 @@
* [Fetch Jobs](https://github.com/TheAlgorithms/Python/blob/master/web_programming/fetch_jobs.py)
* [Get Imdb Top 250 Movies Csv](https://github.com/TheAlgorithms/Python/blob/master/web_programming/get_imdb_top_250_movies_csv.py)
* [Get Imdbtop](https://github.com/TheAlgorithms/Python/blob/master/web_programming/get_imdbtop.py)
* [Get User Tweets](https://github.com/TheAlgorithms/Python/blob/master/web_programming/get_user_tweets.py)
* [Giphy](https://github.com/TheAlgorithms/Python/blob/master/web_programming/giphy.py)
* [Instagram Crawler](https://github.com/TheAlgorithms/Python/blob/master/web_programming/instagram_crawler.py)
* [Instagram Pic](https://github.com/TheAlgorithms/Python/blob/master/web_programming/instagram_pic.py)
Expand Down
1 change: 1 addition & 0 deletions requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,6 @@ sklearn
statsmodels
sympy
tensorflow
tweepy
types-requests
xgboost
61 changes: 61 additions & 0 deletions web_programming/get_user_tweets.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
import csv

import tweepy

# Twitter API credentials
consumer_key = ""
consumer_secret = ""
access_key = ""
access_secret = ""


def get_all_tweets(screen_name: str) -> None:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As there is no test file in this pull request nor any test function or class in the file web_programming/get_user_tweets.py, please provide doctest for the function get_all_tweets

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is doctest necessary for these type of algos ?
I saw a lot of algos in web_programming which don't contain doctests.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We love doctests if you can figure out how to do them but it is difficult in web_programming.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cclauss Let me try. Haven't tested CSV files with doctests before.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cclauss Setting up doctest for this algorithm is becoming too complex. Better this PR be merged without it.


# authorize twitter, initialize tweepy
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_key, access_secret)
api = tweepy.API(auth)

# initialize a list to hold all the tweepy Tweets
alltweets = []

# make initial request for most recent tweets (200 is the maximum allowed count)
new_tweets = api.user_timeline(screen_name=screen_name, count=200)

# save most recent tweets
alltweets.extend(new_tweets)

# save the id of the oldest tweet less one
oldest = alltweets[-1].id - 1

# keep grabbing tweets until there are no tweets left to grab
while len(new_tweets) > 0:
print(f"getting tweets before {oldest}")

# all subsiquent requests use the max_id param to prevent duplicates
new_tweets = api.user_timeline(
screen_name=screen_name, count=200, max_id=oldest
)

# save most recent tweets
alltweets.extend(new_tweets)

# update the id of the oldest tweet less one
oldest = alltweets[-1].id - 1

print(f"...{len(alltweets)} tweets downloaded so far")

# transform the tweepy tweets into a 2D array that will populate the csv
outtweets = [[tweet.id_str, tweet.created_at, tweet.text] for tweet in alltweets]

# write the csv
with open(f"new_{screen_name}_tweets.csv", "w") as f:
writer = csv.writer(f)
writer.writerow(["id", "created_at", "text"])
writer.writerows(outtweets)



if __name__ == "__main__":
# pass in the username of the account you want to download
get_all_tweets("FirePing32")