yassir28/Darija-LLMs

Evaluation of Darija linguistics on LLMs (mainly gpt models) ## what can be proven? translateration > arabic letters ? semantics evaluation? syntatics evaluation? gpt4 > gpt 3.5 ? ## constructions, idioms Since I did A year of intensive studies in Morocco I collected and translated Moroccan cultural expressions with a great deal of help from my instructor. Proverbs, expressions and idioms in dialect are great for demonstrating knowledge of the culture. If your target audience, doesn't understand Darija this is no problem. You can take a chunk out of a word count pretty quickly explaining some of these expressions. # what could be next? see what's lacking in the ouput in the model's output. # work on idioms: # what to do recheck papers for the idioms how they went all the way for a very specific culture or equivalence. # How to evaluate a dialect on an LLM like gpt? i's basically getting influenced but in a way that you treat all cases. # how to make the paper/work systematic: - understand the api - we need to classify the evaluation: when do we say okay, when do we say not okay, when do we say perhaps. - Use with not just not the roles user and system roles but other roles and dictionary (few shot) = gloassary? - scrape the 14 idioms and play with them (qualitative descreptive work). - I should make sure that rather than getting influenced by the empirical tries(bias), to come up with a list of all things a prompt should have. # Evaluation: - synsets for words - # fewshot The upgrade: 1 - Multi-turn dialogue and single turn translation - Fake CoT: revision 2: - What is the purpose of the translation? - Who is the intended audience for the target sentence? - What is the locale of the target language? - What is the register? - Is a style guide available? + - Whether there will be a designated glossary? An actual chain of tought COT Please fulfill the following conditions when translating."+ "The translation must contain onl …