LibGuides: Text Data Mining: Social Media & Internet Resources

Social Media & Internet Resources

Internet Archive
How to download files from archive.org in an automated way using wget.
reddit APIs
Access data from posts, threads, comments, users and more from reddit and subreddits.
Historical Reddit data has been collected at http://files.pushshift.io/reddit/ as monthly CSV downloads.
Stanford Large Network Dataset Collection
The SNAP library collects data on large social and information networks since 2004.
Twitter Streaming APIs
Public streams provide access to public data flowing through Twitter. Suitable for following specific users or topics, and data mining. You can also access single-user streams, containing roughly all of the data corresponding with a single user’s view of Twitter.
Wikimedia Data Dumps
Monthly database backups of all Wikimedia wikis in various formats.
Yelp Open Data Set
The Yelp dataset is a subset of businesses, reviews, and user data for use in connection with academic research. Available as JSON files, use it to teach students about databases, to learn NLP, or for sample production data while you learn how to make mobile apps.