feiyixiao/language-identification-fleurs
Domain:
natural language processingRecord type:
project
Creator:
fei
Host:
Spoken language identification (LID) on 8 typologically diverse languages (Mandarin, Japanese, English, German, Spanish, Italian, Arabic, Swahili) using the FLEURS dataset, with MLP baseline and typological distance analysis via lang2vec.
# Language Identification on FLEURS
Course project for Intro to Deep Learning (Stuttgart, WS 2025). The task is to identify which of 8 languages a spoken audio clip is in, using the FLEURS dataset.
Two approaches are compared: a simple MFCC feature baseline and fine-tuning wav2vec2-base.
## Languages
| Code | Language |
|------|----------|
| `cmn_hans_cn` | Mandarin Chinese |
| `ja_jp` | Japanese |
| `en_us` | English |
| `de_de` | German |
| `es_419` | Spanish |
| `it_it` | Italian |
| `ar_eg` | Arabic |
| `sw_ke` | Swahili |
## Approach
**Baseline** (`nn_baseline.py`, `svc.py`): extract 13 MFCC coefficients from each audio clip, then train either an MLP or a LinearSVC on top.
**Main model** (`train_wav2vec2.py`): fine-tune `facebook/wav2vec2-base` directly on raw audio. The CNN feature encoder is frozen; only the transformer layers and classification head are trained.
## Results
Evaluated on the official FLEURS test split.
| Method | Accuracy | Macro-F1 |
|--------|----------|----------|
| MFCC + LinearSVC | — | — |
| MFCC + MLP | 16.85% | 15.71% |
| wav2vec2 fine-tuned | — | — |
*The MLP baseline overfits to training speakers — it gets 91% on an internal split but drops to ~17% on unseen speakers. wav2vec2 results coming after training finishes.*
## How to run
```bash
pip install -r requirements.txt
```
MFCC baseline:
```bash
cd src
python nn_baseline.py # MLP
python svc.py # LinearSVC
```
Fine-tune wav2vec2 (~2 hours on MPS/GPU):
```bash
cd src
python train_wav2vec2.py
```
Inference:
```python
from transformers import Wav2Vec2FeatureExtractor, Wav2Vec2ForSequenceClassification
import torch
model_id = "feiyixiao/wav2vec2-fleurs-lid"
extractor = Wav2Vec2FeatureExtractor.from_pretrained(model_id)
model = Wav2Vec2ForSequenceClassification.from_pretrained(model_id)
model.eval()
inputs = extractor(audio_array, sampling_rate=16000, return_tensors="pt")
with torch.no_grad():
pred = model(**inputs).logits.argmax().item()
print(model.config.i …