miria00/CLD

Convex Language Detection for Low Resource Languages (lab repo at pilancilab/CLD) CLD Convex Low-resource Accent-Robust Language Detection in Speech Recognition A lightweight language-detection module for multilingual ASR, optimized via ADMM in JAX. --- This repository provides the official implementation of **CLD**, a lightweight language-detection module for multilingual ASR. This codebase contains our pip-installable Python package (`cld/`) including our training/benchmark scripts implemented in JAX and optimized via ADMM for high performance in low-resource settings. Simply, the package attaches a small language detection head (Convex NN / small NN / linear SVM) to ASR encoder representations, and use it to select the language token (Whisper) or adapter (MMS) before decoding. The paper PDF is available in `paper/` and the pip-installable package is published at pypi.org. ## Highlights - High Accuracy: Excels in binary and multiclass language detection (Table 3). - Low-Resource Robustness: Effective with limited data (Figures 1 & 2). - Efficient: 13x training speedup from traditional NNs due to ADMM optimization and JAX. ## Requirements The package is published on PyPI as `jaxcld`. For inference usage, install it directly: ```bash pip install jaxcld ``` If you've cloned this repo, you can instead install from source: - **Package-only install** (inference usage): ```bash pip install -e . ``` - **Full training/benchmark environment** (recommended if you run the scripts in this repo): ```bash pip install -e ".[train]" ``` If you prefer installing from the pinned dependency list instead: ```bash pip install -r requirements.txt ``` ## Using the package ### Minimal inference example (Whisper) ```python import numpy as np from cld import ASRModel, CVXNNLangDetectHead, NNLangDetectHead, SVMLangDetectHead # 1) Load the base ASR model languages = ["en", "hi", "id", "ms", "zh"] asr = ASRModel.from_pretrained("openai/whisper-small", config={"languages": languages}) # 2) Load a language detection hea …