MikCil/inspicio

Open-vocabulary Word-Sense Disambiguation retrieval pipeline for historical and low-resource languages. # Inspicio Inspicio is a configurable, open-vocabulary Word-Sense Disambiguation retrieval pipeline for historical and low-resource languages. It retrieves a set of candidate synsets from the Open English WordNet for toekns in any language, based on their context. ## Inputs The CSV runner expects these columns by default: - `ID` - `language` - `TOKEN` - `LEMMA` - `SENTENCE` - `PoS` with one of `verb`, `noun`, `adjective`, `adverb` - `VERB SEMANTICS` for optional gold synset labels Gold labels may be blank. Rows without gold are still tagged and written to JSONL, but they are excluded from aggregate Recall/MRR. Column names are configurable under `columns:` in each YAML file. ## Install ```bash pip install -r requirements.txt ``` For local embedding models: ```bash pip install ".[local]" ``` Set the API keys needed by your config, for example: ```bash set OPENAI_API_KEY=... set DEEPSEEK_API_KEY=... set MISTRAL_API_KEY=... set OPENROUTER_API_KEY=... ``` On macOS/Linux, use `export` instead of `set`. ## Build Indexes Build all four PoS indexes: ```bash python scripts/build_index.py --config configs/deepseek_openai.yaml --pos all ``` Build one PoS index: ```bash python scripts/build_index.py --config configs/deepseek_openai.yaml --pos noun ``` The default paths are: - `./oewn_verb_embeddings` / `oewn_verbs` - `./oewn_noun_embeddings` / `oewn_nouns` - `./oewn_adjective_embeddings` / `oewn_adjectives` - `./oewn_adverb_embeddings` / `oewn_adverbs` Change `index.chroma_path_template` and `index.collection_template` to use different models, for example Cohere or Jina. ## Run Full Pipeline ```bash python scripts/run_pipeline.py --config configs/deepseek_openai.yaml ``` Provider examples: - DeepSeek: `configs/deepseek_openai.yaml` - Mistral: `configs/mistral_openai.yaml` - OpenRouter for Kimi/Qwen/GLM: `configs/openrouter_openai.yaml` - Any OpenAI-compatible LLM endpoint with a custom base URL: `configs/custom_llm_openai_compatible.yaml` For a custo …