MontaLabidi/Text-Threader

Text Threader is a web application made with Django and Angular 7 to detect the language and sentiment of a given text. It mainly detects Arabic or Tunisian dialect and a Positive or a Negative sentiment and supports testing multiple text documents. # Text Threader **Text Threader** is a web application made with Django 2 and Angular 7 to detect the language and sentiment of a given text. It mainly detects _Arabic_ or _Tunisian dialect_ and a _Positive_ or a _Negative_ sentiment and supports testing multiple text documents. ### Features * Detects the language of a text written in any character encoding (_Arabic_ / _Tunisian_/ _Other_) * Analyse the Sentiment of a text written in any character encoding (_Negative_ / _Positive_/ _Other_) * Supports streaming multiple files with texts to classify and analyse # Getting Started ## Pre-requisites For building and running the application you need: * Backend: - Python 3.6 - Django 2.1 * Frontend: - Node.js - npm (comes with Node.js) ## Installation ### Classification Model Setup This step is optional if you are just looking to use the application since it is already set up with the needed models, but if you want to tweak on the classification models used then install Jupyter notebook and open the following notebooks: * **Language identification** These steps give an overview on the language identification pipeline of the `Lang-classifier.ipynb` Jupyter notebook: * Text Cleaning * Construct the training and test dataframes using our labaled data * Convert the training documents into numeric feature vectors using the _BOW-tfidf_ method with _character ngrams_ * Create a language classifier using Naive Bayes method (tfidf version) * Evaluate performance of this classifier based on the test corpus: calculate classification accuracy, precision, recall, F1, and confusion matrix * **Sentiment analysis** These steps give an overview on the sentiment analysis pipeline of the `Sentiment-analysis.ipynb` Jupyter notebook: * Text Cleaning * Normalization & tokenization * Remove stop words * Stemming * Extract the vocabulary set from the corpus and calculate IDF values of each word in this set * tune the BOW configuration parameters (min_df, max_df, etc.) * …