VerbalAid/Synthetic_vs_Translated_models_ABSA

Multilingual ABSA comparison fine-tuning mT5 on hotel reviews across four Spanish training conditions — real English, machine-translated, LLM-synthetic, and combined — to evaluate data source quality for low-resource sentiment analysis. # ABSA Triplet Extraction: English & Spanish Hotel Reviews **Darragh Kerins and Ander Peña** · UPV/EHU · 2025 --- ## What is ABSA? Aspect-Based Sentiment Analysis (ABSA) is a fine-grained form of sentiment analysis that goes beyond asking "is this review positive or negative?" Instead, it identifies *what* is being talked about and *how* the reviewer feels about each specific thing. For a sentence like: > *"The room was spotless but the check-in took forever."* A standard sentiment model might call this mixed or neutral. ABSA extracts the structure underneath: | Aspect term | Category | Sentiment | |---|---|---| | room | `ROOM#CLEANLINESS` | positive | | check-in | `HOTEL#SERVICE` | negative | Each of these three-part extractions is called a **triplet**: *(aspect term, aspect category, sentiment polarity)*. The aspect term is the exact span from the review text; the category is drawn from a fixed label schema; and the polarity is positive, negative, or neutral. This project trains a multilingual seq2seq model to extract all triplets from a hotel review in a single pass, and investigates whether LLM-generated synthetic training data can replace human-annotated or machine-translated data for Spanish. --- ## Results at a Glance | Model | Training Data | Test Set | Precision | Recall | F1 | |-------|--------------|----------|-----------|--------|----| | A – English baseline | M-ABSA EN (real) | EN | 60.71% | 46.21% | **52.48%** | | B – Spanish translated | M-ABSA ES (machine-translated) | ES | 50.76% | 37.60% | 43.20% | | C – Spanish synthetic | LLM-generated (Claude) | ES | 17.83% | 15.06% | 16.33% | | D – Spanish combined | Translated + synthetic | ES | 51.07% | 43.26% | **46.84%** | | Mistral-7B zero-shot | — | ES | 7.05% | 1.81% | 2.88% | **Key finding**: Synthetic data alone (Model C) substantially underperforms machine-translated data (Model B), but combining both (Model D) produces the best Spanish result — outperforming the translation-only baseline …