Levis0045/SCIA-CRF_XGBOOST_POS

Augmentation based on position applied to CRF and XGBOOST algorithms applied to african languages for POS task # Sangkak session septembre 2023 Contextualize CRF/XGBOOST and Linguistic based feature for POS detection in Ghomala - CRF - XGBOOST - Position to position augmentation algorithm # Workshop presentation Check my presentation here: elvis_workshop_session_presentation.pdf # Get Masakhane POS dataset for project - if not exist, create data source folder for POS in this repository ```sh mkdir data_source cd data_source ``` - clone Masakhane POS dataset in `data_source` folder ```sh git clone github.com ``` - if you want to experiment with main dataset produce by NTeALan teams, use this folder instead: `lacuna_ntealan_data` (you will need to split the dataset) # Install Python dependencies - Reproduice poetry environnement ```sh poetry install --only dev && poetry shell or pip install -r requirements.txt ``` # Run all my mlflow experimentations - Reproduice all my mlflow experimentations ```sh ./run_experiments.sh ``` # Run CRF mlflow experimentations - CRF usage ``` python3 experimentations/mlflow_crf.py --help usage: mlflow_crf.py [-h] [--lang LANG] [--augment] [--description DESCRIPTION] [--re-organised-data] [--shuffle] [--iter ITER] [--c2 C2] [--algo ALGO] [--c1 C1] sklearn CRF Sangkak options: -h, --help show this help message and exit --lang LANG language of training data --augment Augment training data --description DESCRIPTION description of experiment --re-organised-data re-organised data of training --shuffle shuffle training data --iter ITER number of iterations --c2 C2 c2 regularization value for algorithm (default: 0.3) --algo ALGO crf algorithm (default: lbfgs) --c1 C1 c1 regularization value for algorithm (default: 0.0920512484757745) ``` - CRF run sample ```sh python3 experimentations/mlflow_crf.py --lang bbj --description "crf: iter 100" --iter 100 ``` # Run XGBOOST mlflow experi …