Speech Encoder Size and Projector Capacity Trade-offs in SLAM-ASR for High- and Low-Resource Languages

Large language models (LLMs) have demonstrated potential in handling spoken inputs for high-resource languages, reaching state-of-the-art performance in various tasks. However, their applicability is still less explored in low-resource settings. This work investigates the use of Speech LLMs for low-resource Automatic Speech Recognition using the SLAM-ASR framework, where a trainable lightweight projector connects a speech encoder and a LLM. Firstly, we assess training data volume requirements to match Whisper-only performance, re-emphasizing the challenges of limited data. Secondly, we show th Research goal: How does the trade-off between speech encoder size (e.g., 100M vs 500M parameters) and trainable projector capacity impact the WER in SLAM-ASR when using a fixed dataset size for both high-resource and low-resource languages? Autonomous synthesis report generated by Assignee Research. Tribunal consensus score: 7.5/10. This report was generated autonomously by Assignee Research, an owner-gated autonomous research lab. The content synthesizes findings from peer-reviewed papers. Tribunal score: 7.5/10.