csikasote/bigc

This repository contains the data resources for the LacunaFund supported project, Multimodal datasets for the Bemba Language of Zambia. # BIG-C: a Multimodal Multi-Purpose Dataset for Bemba --------------------------------------------- #### Introduction This repository contains the data resources for the project `multimodal datasets for the Bemba language`. The dataset consists ofa parallel corpus of speech and transcriptions of image-grounded dialogues between Bemba speakers and their corresponding English translations. It contains 92, 117 spoken utterances of both complete and incomplete dialogues, amounting to 187 hours of speech data grounded on 16, 229 unique images. The dataset will enable the development of `speech recognition`, `speech and text translation` systems for Bemba, as well as facilitate research in language grounding and multimodal model development. This work has been accepted to the ACL 2023 conference (main). The pre-print is available HERE! #### File Structure ----------------- big-c dataset . ├── Datasheet_for_Dataset_DRAFT ├── README.md └── data └── bem ├── datasheet.csv ├── audio/*.wav ├── image/*.jpg └── splits/ ├── conversations/ │   ├── conversations.jsonl │   ├── duplicate.zip │   ├── test.jsonl │   ├── train.jsonl │   ├── unaligned.jsonl │   └── valid.jsonl ├── *.jsonl └── *.tsv * `/data/datasheet.csv` the main data file * `/data/audio/*.wav` contains the audio files * `/data/image/*.jpg` contains the image files * `/data/splits/*` contains the dataset splits aligned to original `Flickr30K` splits. #### Citation Cite as follows if you use this work in research: @inproceedings{sikasote-etal-2023-big, title = "{BIG}-{C}: a Multimodal Multi-Purpose Dataset for {B}emba", author = "Sikasote, Claytone and Mukonde, Eunice and Alam, Md Mahfuz Ibn and Anastasopoulos, Antonios", editor = "Rogers, Anna and Boyd-Graber, Jordan and Okazaki, Naoaki", booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)", month = jul, year = "2023", address = "Toronto, Canada", publisher = "Association for Computat …