soufianeboulahcen/Multimodal-Moroccan-SLG
Domain:
natural language processingRecord type:
model
Creator:
sou
Host:
multimodal moroccan sign language generation
# Multimodal Moroccan Sign Language Generation
Strict paper-faithful PyTorch reimplementation of **SignLLM**
(Fang et al., 2024) trained on the
**Moroccan Sign Language (MoSL) video dataset**
(Ben Zaid et al., 2026).
> **Headline finding.** On the MoSL isolated-word dataset, our strict
> paper-faithful SignLLM-Base reimplementation fails to outperform
> deterministic retrieval baselines. All three loss configurations from the
> paper's Table 5 ablation (MSE / RL / RL+PLC) underperform Nearest-Neighbor
> (test DTW 0.78), Mean-Pose (0.87), and Random-Clip (0.96), with our best
> trained model at 1.04. See `docs/RESULTS.md` for the
> full result table, ablation analysis, and the regression-to-the-mean
> diagnostic that explains the gap.
---
## Repository layout
```
.
├── mosl/ Python package (model, data, text, pose, train)
├── scripts/ CLI entry points (training, evaluation, figures)
├── patches/ Our patches to upstream third-party code
├── data/ Dataset metadata (CSVs) and tokenizer vocab
├── docker/ Container build (NGC PyTorch 26.04 base)
├── docs/ Methodology + decisions + results + walkthroughs
├── predictions/ Sample model outputs (NPZ + animated GIF)
└── runs/ Tracked: evaluation.json + baselines.json
(training logs and checkpoints not redistributed)
```
Each `docs/*.md` file is the source of truth for one aspect of the project:
| File | Contents |
|---|---|
| `docs/STATS.md` | Dataset statistics: counts, long-tail, FPS, splits |
| `docs/PIPELINE.md` | End-to-end preprocessing pipeline |
| `docs/POSE_EXTRACTION.md` | Phase 2 completion summary |
| `docs/PROMPT2SIGN.md` | What we adapted from upstream |
| `docs/MODEL.md` | Architecture spec + open questions |
| `docs/DECISIONS.md` | Every non-trivial decision, dated |
| `docs/RESULTS.md` | Final results, baseline comparison, analysis |
| `docs/ …