IsiXhosa-click/xh_lexicographic_taggers_poc
Domain:
natural language processingRecord type:
software
Creator:
Isi
Host:
# xh_lexicographic_taggers
Lexicographic POS tagging experiment for isiXhosa.
Data is from here:
"Linguistically enriched corpora for conjunctively written South African languages" by Martin Puttkammer and Tanja
Gaustad.
## Results
Best model (ours) is bolded:
|Model |Accuracy (including noun classes)|Accuracy (not including noun classes)|
|---------------------------------|---------------------------------|-------------------------------------|
|du Toit & Puttkammer | 74.3590% | 75.6410% |
|Ours | **82.0513%** | **87.1795%** |
|NLAPOST21 shared task winner, crf, comp sum, bigram | (does not attempt to tag noun classes) | 64.1026% |
|NLAPOST21 shared task winner, lstm, comp sum, bigram| (does not attempt to tag noun classes) | 60.2564% |
**NB:** It is very important to note that this is on words as they would appear in a dictionary, e.g. on IsiXhosa.click. This means it is an 'unfair' comparison for these taggers, which are trained on a sentence level. These results do not mean that our tagger is better for _any_ usecase _other_ than dictionary entries formatted like ours are on IsiXhosa.click.
## State of the codebase
I had to hack this together a bit as a proof-of-concept so the code is in a not-so-great
state. If you want to actually _use_ this, please contact me and I'll gladly clean it up for you.
This is mostly here for posterity + transparency and is released as-is.
## Licensing
All code is licensed under Apache 2.0.
Data retains its original licensing, and we merely redistribute it for convenience.
## References (baselines)
- du Toit, J. S., & Puttkammer, M. J. (2021). Developing Core Technologies for Resource-Scarce Nguni Languages. _Information, 12_(12), 520. doi.org …