kibmuikia/ke-katiba-digest
Domain:
natural language processingRecord type:
software
Creator:
kib
Host:
A Python processing pipeline for extracting, structuring, and serialising legal text from the Constitution of Kenya (2010) and associated statutory booklet PDFs into structured JSON and SQLite formats.
ke-katiba-digest
A modern, high-performance extraction and structuring pipeline for the Constitution of Kenya (2010).
Architecture •
Getting Started •
Usage •
Standards •
Schema
---
## Overview
`ke-katiba-digest` converts raw, multi-column statutory PDF documents into clean, structured AST representations. It parses legal hierarchies (**Chapter → Part → Article → Clause**) and serializes them into production-ready **JSON** and **SQLite** databases for downstream legal tech applications, search engines, and LLM indexing.
---
## Architecture & Data Flow
```mermaid
flowchart TD
A[PDF Sources: Constitution / Laws of Kenya] --> B[PDF Extraction Layer]
subgraph Step 1: Raw Extraction & Layout
B -->|extractText.py| C1[Raw Text .txt]
B -->|extractTextWithLayout.py| C2[Layout JSON .json]
B -->|extractTables.py| C3[Extracted Tables]
B -->|convertToMarkdown.py| C4[Cleaned Markdown .md]
end
C4 --> D[build_katiba_digest.py]
subgraph Step 2: Parsing & Structuring
D -->|Regex Parser| E[Hierarchical AST: Chapter -> Part -> Article -> Clause]
D -->|test_parser.py| F[Smoke Test & Regex Validation]
end
E -->|generate_output.py| G1[(SQLite Database: constitution_kenya_2010.db)]
E -->|generate_output.py| G2[Structured JSON: constitution_kenya_2010.json]
```
---
## Directory Structure
```
ke-katiba-digest/
├── build_katiba_digest.py # Core parser: AST builder & text cleanup pipeline
├── convertToMarkdown.py # PDF -> Markdown conversion CLI
├── extractTables.py # PDF tabular data extraction module
├── extractText.py # Baseline text extraction CLI
├── extractTextWithLayout.py # Layout-aware PDF text and bounding box extractor
├── generate_output.py # Exporter to SQLite (.db) and structured JSON (.json)
├── pdf_utils.py # Shared PDF processing & PyMuPDF/pdfplumber utilities
├── renderPageToImage.py # Page rendering engine for inspection/visual audit
├── test_parser.py # CLI test suite f …