arnoldkigozi0/tokenroute

Token-efficient hybrid routing agent - AMD Develope Hackathon: ACT II, Track 1. Team Inquiline (Uganda). # tokenroute **A token-efficient hybrid routing agent** — answers with a free local model whenever it can, and spends remote tokens only when it must. Built solo by team **Inquiline** (Uganda) for the **AMD Developer Hackathon: ACT II — Track 1: Hybrid Token-Efficient Routing Agent**. ## The idea Track 1 scores two things: **output accuracy** and **token count**, where *local tokens count as zero*. That makes the winning strategy routing intelligence, not raw compute: 1. **Cache first.** A repeated (or trivially re-worded) task is answered from the cache: zero tokens, zero latency (`--use-cache`). 2. **Pre-screen** each task with a zero-cost complexity heuristic. Tasks that clearly exceed the local model skip straight to the remote endpoint (saves latency, not tokens — a doomed local attempt is free but slow). 3. **Local first.** Everything else is drafted by the local model. Local tokens are free. 4. **Verify the draft** — zero-cost sanity checks (empty output, refusals, degenerate repetition), plus optionally the local model critiquing its own draft (`--self-check`): accuracy bought entirely with free tokens. 5. **Escalate only on failure.** Remote (Fireworks AI) is called only when the draft fails verification — those are the only tokens that ever hit the bill. This mirrors the router at the heart of Inquiline, a local-first agent built to run on zero-budget infrastructure — the same constraint this track rewards. ## Layout ``` tokenroute/ backends.py # OllamaBackend (local, free), FireworksBackend (remote, billed), StaticBackend (tests) router.py # cache → complexity pre-screen → local draft → verify/self-check → escalate cache.py # normalized-prompt answer cache, JSON-persistent (Inquiline memory pattern) brains.py # Ollama model lifecycle: one brain at a time on tight VRAM (Inquiline pattern) agent.py # CLI: single task or JSONL batch, emits JSON per task eval/ run_eval.py # accuracy + billable-token report over a task set …