Corpus of African Digital News from 600 Websites Formatted for Text Mining / Computational Text Analysis

This dataset includes a corpus 200,000+ news articles published by 600 African news organizations between December 4, 2020 and January 3, 2021. The texts have been pre-processed (punctuation and English stopwords have been removed, features have been lowercased, lemmatized and POS-tagged) and stored in commonly used formats for text mining/computational text analysis. Users are advised to read the documentation for an explanation of the data collection process.

This dataset includes the following items:

• 31 tables (one per day) of lowercased and lemmatized tokens with the following additional variables: POS tags, document id, sentence id, token id and publication date (stored as a tibble).
• A single document-feature matrix (DFM) with raw counts of feature frequencies in each news article (stored as a quanteda dfm object). The DFM comes with the following metadata for each document: date of publication and source URL.
• A metadata table with the following fields: document id, publication date, source url, news source and country of the news source.
• A list of sources included in the course grouped by country name.

All items are stored in formats readable in R. The documentation provides instructions on how to load the RDS files to R.

If you decide to use the data for your own project, please do cite it using the information above. If you identify errors or missing sources, please contact us so that these can be addressed. R, 4.2