potatobot-rwanda/datasets

# Datasets ## Getting Started ### Reading List LLM Prompting * Read first section (Introduction and all sub-chapters under Introduction) * Read Few shot prompting TimeML standard (especially the TimeX3 standard for temporal expressions) * TimeML Website * TimeX3 specifications ## Data Formats We use a custom data format for our training data: ``` four weeks ago , I planted Ndamira potateos in my field in Nyamata . ``` This example contains annotations for all entity types LOCATION, POTATO and TIMEX3. ## Repository Overview ``` ├── README.md # This help file ├── data # data directory │   └── english_examples # english data │   ├── eng_dataset.csv # data format to send to the AI models team │   └── eng_dataset.xml # data generation format ├── requirements.txt └── src └── convert_xml_to_bio.py # convert from the xml data format to the csv dat format ``` ## Forkflow ### Workflow overview 1. Create NER data * Prompt engineering to create prompts that generate high quality data for all three entity classes (location, potato and temporal expressions) * Generate data for all three entitiy classes, individual and joint with LLMs 2. Annotate NER data * Develop NER data annotation scheme * Manually annotate smaller sample (e.g., 100 samples) * Analyse data quality 3. Develop data for location linking * Develop data generation scheme * Develop data annotation schem * Analyse data quality ### Generate NER data I used this prompt to develop my prompt: ``` Generate 5 conversations between a chatbot and a Rwandan farmer. The farmer calls the chatbot because he wants to know when to spray his potatoes. The chatbot asks four questions to the user: * When did you last spray your potatoes? * When did you plant your potatoes? * Where is your farm located? * Which potato variety do you plant? Here are some more instructions: * Assume that today is 28.6.2025 * The chatbot always starts the conversation * You do not need t …