LibGuides: Scraping X (Twitter): Example Projects and Tutorial Scripts

Studies Involving X/Twitter Scraping

A brief sampling of peer-reviewed projects involving Twitter scraping. Note the data as the API became more restrictive in 2023.

Hagemann, L., & Abramova, O. (2023). Sentiment, we-talk and engagement on social media: insights from Twitter data mining on the US presidential elections 2020. Internet Research, 33(6), 2058–2085. https://search.lib.utexas.edu/permalink/01UTAU_INST/1jebi5l/cdi_crossref_primary_10_1108_INTR_12_2021_0885

Natalia Sánchez-Querubín. (2017). Case Study: Webs and Streams – Mapping Issue Networks Using Hyperlinks,

Hashtags and (Potentially) Embedded Content. In The Datafied Society (pp. 95–106). Amsterdam University Press. https://search.lib.utexas.edu/permalink/01UTAU_INST/1jebi5l/cdi_walterdegruyter_books_10_1515_9789048531011_009

Sharma, K., & Singh, K. K. (2023). Psychological Study of Cyber-Bullying Against Adolescent Girls in India Using Twitter. International Journal of Cyber Behavior, Psychology, and Learning, 13(1), 1–22. https://search.lib.utexas.edu/permalink/01UTAU_INST/1jebi5l/cdi_proquest_journals_2851060078

Examples and Tutorials using Scraping Tools

Quotesbot
An example of webscraping with Scrapy, extracting quotations and their authors from http://quotes.toscrape.com/
Web Scraping with Request
A tutorial on using the Requests tool to scrape the web in Python from Medium.

Bluesky

Bluesky is an increasingly popular alternative to X that allows scraping using its open API. For documentation and more information about scraping Bluesky data, please consult its official documentation.