ctextdev/ctextcore

An open-source Python package for existing NCHLT core technologies for ten South African languages. ## About The Project This project is an open-source Python package for existing NCHLT core technologies for ten South African languages (Afrikaans, isiNdebele, isiXhosa, isiZulu, Sesotho sa Leboa, Sesotho, Setswana, Siswati, Tshivenḓa, Xitsonga). The technologies include the following: Tokenisers, Sentence Separators, Part of Speech Taggers, Named Entity Recognisers, Phrase Chunkers, Optical Character Recognisers, Universal Part of Speech Taggers, Lemmatisers, and a Language Identifier. The package also includes a Morphological Analyser for isiNdebele, isiZulu, isiXhosa, and Siswati, totalling 85 technologies. ## Getting Started To get a local copy installed and running, follow these steps. ### Prerequisites * Python 3.8+ (python.org) * Java OpenJDK 17+ (openjdk.org) * Requests 2.32.3 (pypi.org) ### Installation ### pip ```sh pip install ctextcore ``` ### GitHub ``` # Download the source code from GitHub git clone github.com # Install from source cd ctextcore py -m pip install . # Install from source in Development Mode cd ctextcore py -m pip install -e . ``` ## Usage ### Importing the CTexT Core library ```Python from ctextcore.core import CCore as core server = core() ``` The core method accepts the following configuration arguments: ```Python port: 8079 # Set the port the server should use timeout: 60000 # Set the timeout of HTTP requests threads: 5 # Set the total number of threads to use memory: "4G" # Set the maximum memory allowed to be used by the server be_quiet: False # Set the logging output from the server max_char_length: 10000 # Set the maximum character length server = core(port=8081,memory="16G",...) ``` ### Language codes * Afrikaans -> af * isiNdebele -> nr * isiXhosa -> xh * isiZulu -> zu * Sesotho sa Leboa -> nso * Sesotho -> st * Setswana -> tn * Siswati -> ss * Tshivenḓa -> ve …