Gossip Handshake: A Routing-Based Alternative to LoRA Weight-Space Merging for Decentralised Knowledge Sharing

Sharing specialised knowledge across decentralised communities of fine-tuned language models is a fundamental challenge. The dominant paradigm—weight-space merging of Low-Rank Adaptation (LoRA) adapters via methods such as TIES-Merging—promises a single unified model, but its effectiveness on heterogeneous, low-overlap domains remains poorly understood. We introduce the Gossip Handshake Protocol, a lightweight alternative that preserves individual domain adapters and routes incoming queries to the appropriate specialist at inference time.

In controlled experiments with $K=5$ disjoint African agricultural domains and two model scales (494M-parameter Qwen2.5-0.5B-Instruct and 1.54B-parameter Qwen2.5-1.5B-Instruct), we demonstrate that:

(1) TIES-Merging produces models with near-zero keyword recall at both scales (< 5.6% at 0.5B; < 19.9% at 1.5B);

(2) the Gossip Handshake Protocol retains 88-100% of specialist performance at 0.5B (75.2% overall) and consistently outperforms merging by 1.9x at 1.5B (37.9%), with zero additional training;

(3) a keyword-based router achieves 100% routing accuracy at both scales, establishing routing as a viable—and dramatically superior—alternative to weight-space merging for decentralised knowledge sharing.

Code, data, and adapters available at: https://github.com/tflux201…

Keywords: LoRA, adapter merging, decentralised AI, knowledge sharing, TIES-Merging, routing, gossip protocol, African agriculture, Large Language Models