LibGuides: Scraping Twitter: Scraping and Parsing Tools

Scraping Tools

A list of tools for scraping Twitter data, with and without using the API.

Tweepy
Twitter for Python!
Twython
Actively maintained, pure Python wrapper for the Twitter API. Supports both normal and streaming Twitter APIs.
Twitter Scrape
Download streaming tweets that match specific keywords, and dump the results to a file. Requires API keys.
Twitter Scraper
Twitter makes it hard to get all of a user's tweets (assuming they have more than 3200). This is a way to get around that using Python, Selenium, and Tweepy.
Tweets Analyzer
Metadata scraper & activity analyzer for tweets. Requires API keys.

Below is a list of other tools that might be useful as you scrape, parse, and store Twitter data.

SQLite
SQLite is a self-contained, high-reliability, embedded, full-featured, public-domain, SQL database engine. SQLite is the most used database engine in the world.
BeautifulSoup
A tool for parsing data from websites.
Natural Language Toolkit
NLTK is a leading platform for building Python programs to work with human language data.
Selenium
Selenium is a suite of tools to automate web browsers across many platforms.