kamperh/bucktsong_eskmeans

Unsupervised segmentation and clustering of the Buckeye English and NCHLT Xitsonga datasets using the ES-KMeans algorithm. Embedded Segmental K-Means Applied to Buckeye English and NCHLT Xitsonga ======================================================================== Overview -------- Unsupervised acoustic word segmentation and clustering of Buckeye English and NCHLT Xitsonga data using the embedded segmental K-means (ES-KMeans) algorithm. The experiments are described in: > H. Kamper, K. Livescu, and S. J. Goldwater, "An embedded segmental K-means > model for unsupervised segmentation and clustering of speech," in *Proc. > ASRU*, 2017. [arXiv] Please cite this paper if you use the code. This recipe relies on the separate ES-KMeans package, which performs the actual unsupervised segmentation and clustering. Download datasets ----------------- The Buckeye English and portions of the NCHLT Xitsonga corpora are used: - Buckeye corpus: buckeyecorpus.osu.edu - NCHLT Xitsonga portion: zerospeech.com From the complete Buckeye corpus we split off several subsets. The most important are the sets labelled as `devpart1` and `zs`. These sets respectively correspond to `English1` and `English2` in (Kamper et al., 2016). Install dependencies -------------------- Dependencies can be installed in a conda environment: conda env create -f environment.yml conda activate eskmeans Install the ES-KMeans package: mkdir ../src/ git clone github.com ../src/eskmeans/ Extract speech features ----------------------- Extract MFCCs in `features/` as follows: cd features/ ./extract_features_buckeye.py ./extract_features_xitsonga.py More details on the feature file formats are given in features/readme.md. Unsupervised syllable boundary detection ---------------------------------------- As a preprocessing step, we constrain the allowed word boundary positions to boundaries detected by an unsupervised syllable boundary detection algorithm. We specifically use the algorithm described in: > O. J. Räsänen, G. Doyle, and M. C. Frank, "Pre-li …