novah-ngo/nlp-qualitative-research

This project develops NLP tools to help NOVAH automate the analysis of focus group discussions (FGD) and in-depth interviews (IDI) from their educational interventions in East Africa. # NOVAH Text Mining Project Automated thematic and sentiment analysis for Intimate Partner Violence (IPV) prevention transcripts. ## Project Overview This project develops NLP tools to help NOVAH automate the analysis of focus group discussions (FGD) and in-depth interviews (IDI) from their educational interventions in East Africa. **Key Objectives:** - Thematic analysis: Identify recurring themes and concepts across transcripts - Sentiment analysis: Detect emotions and attitudes associated with themes - Automation: Reduce manual analysis time while maintaining quality insights ## Project Structure ``` novah/ ├── data/ │ ├── raw/ # Original transcript files (.docx) │ └── processed/ # Cleaned and processed data (.csv) ├── src/ │ ├── data_loader.py # Load and parse transcripts │ ├── preprocessor.py # Text cleaning and normalization │ ├── thematic_analysis.py # Topic modeling (to be created) │ └── sentiment_analysis.py # Emotion detection (to be created) ├── notebooks/ │ ├── 01_data_exploration.ipynb │ ├── 02_thematic_analysis.ipynb (to be created) │ └── 03_sentiment_analysis.ipynb (to be created) ├── models/ # Saved models ├── outputs/ │ ├── visualizations/ # Charts and graphs │ └── reports/ # Analysis reports └── requirements.txt # Python dependencies ``` ## Setup ### 1. Install Dependencies ```powershell pip install -r requirements.txt ``` ### 2. Download NLTK Data ```python import nltk nltk.download('punkt') nltk.download('stopwords') nltk.download('punkt_tab') ``` ### 3. Download spaCy Model (Optional) ```powershell python -m spacy download en_core_web_sm ``` ## Usage ### Load and Preprocess Transcripts ```python from src.data_loader import TranscriptLoader from src.preprocessor import TranscriptPreprocessor # Load transcripts loader = TranscriptLoader('data/raw') df = loader.load_all_transcripts() # Preprocess preprocessor …