ALI-OUALA/dz-bench

Independent Arabic-French and Algerian BAC document intelligence benchmark # DZ-Bench DZ-Bench is an independent, model-neutral benchmark for Arabic–French document intelligence. It exchanges artifacts with document engines through public JSON contracts; core scoring never imports DzDoc internals or any OCR framework. Phase A plus the synthetic BAC slice provides: - packaged draft 2020-12 schemas for manifests, assets, ground truth, predictions, and reports; - Pydantic v2 validation with checksums, page-pixel coordinates, blocks, lines, spans, provenance, confidence, warnings, and alternatives; - conservative NFC normalization, CER, WER, normalized edit similarity, digit exact accuracy, reading-order sequence score, block/layout matching, equation text similarity, and table structure similarity; - failure-aware scoring that counts missing, crashed, timed-out, invalid, and ground-truth-missing pages; - an original deterministic Arabic/French/mixed BAC-like corpus with mathematics, physics, equations, tables, diagrams, reading-order, and clean/compressed/skewed/photographed tags; - per-sample runtime and peak-memory fields with report summaries when a system measures them; - confidence calibration, hallucination, diagram, structured-field, financial-value, structured-coordinate, and structured-hallucination metrics; - an original four-quality Algerian invoice pack with fictional NIF/NIS/RC values, line-item tables, HT/TVA/TTC arithmetic, source coordinates, and validations; - a public JSON adapter boundary for DzDoc predictions; - a shell-free black-box runner that turns crashes, timeouts, missing, and invalid artifacts into explicit per-page failures instead of dropping them; - a stateless multipart scoring API and OCI image for private or CI evaluation; - a provenance-only BAC manifest with no protected documents or fabricated BAC ground truth. ## Install and verify ```powershell uv sync --extra dev uv run pytest uv run ruff check . uv run ruff format --check . uv run pyright ``` Install the optional deterministic PNG renderer: ```p …