Dialloni/Google-Deepmind-TRI-AI-Donga

# Donga — Agricultural Extension RAG (Kaggle) Retrieval for Agricultural Extension RAG: Smart Retrieval for Farmers. Given a smallholder farmer's question, rank the 5 most relevant extension documents from a 695-doc corpus. Metric: **nDCG@5**. ## Status Rank 2, public LB **0.92734** (leader 0.94076, TF-IDF baseline 0.551). ## Leaderboard history | Submission | Public LB | |---|---| | bge-base dense retrieval | 0.82640 | | 4-encoder blend + LightGBM LambdaRank | 0.86954 | | + bge-base fine-tuned on the qrels (Colab) | 0.91880 | | **+ bge-large fine-tuned, hard negatives, 2 seeds** | **0.92734** | | same, candidate list deepened to 350 | 0.92686 | The fine-tunes are measured honestly in Colab on a topic-grouped holdout, where the validation topics are excluded from training: | Model | Holdout nDCG@5 | |---|---| | bge-base, off the shelf | 0.8191 | | bge-base, fine-tuned, random negatives | 0.8893 | | bge-large, fine-tuned, mined hard negatives | 0.9225 | ## Findings - **Fine-tuning on the qrels is the whole game.** Every leaderboard jump came from a better fine-tuned encoder; feature work on top of them has been worth roughly nothing. - **The local CV is inflated and cannot referee tuning decisions.** Both fine-tuned encoders trained on all 308 train queries, so in cross-validation the held-out queries were already seen by the encoder. Deepening the candidate list from 50 to 350 gained 0.019 in CV and *lost* 0.0005 on the leaderboard, because a deeper list makes the ranker lean harder on exactly those in-sample features. Trustworthy CV would need out-of-fold fine-tunes, one per fold. - Not pooling bias: at K=350 the ranker's top-5 are *less* likely to be judged documents (0.856) than the plain blend's (0.870), so it is not learning to recognise the training pool. - **gte-large diverges to NaN on Colab.** Its Hub weights are fp16 and sentence-transformers honours that dtype, so Adam without loss scaling overflows. Force `torch_dtype=torch.float32`. - **Lexica …