Gatimu01/TinyModels

Improving the efficiency of small language models to approach the performance of larger Transformer models in low-resource settings. # TinyModels Improving the efficiency of small language models to approach the performance of larger Transformer models in low-resource settings. ## Goal Study how small models can achieve competitive performance under limited compute. ## Research Questions - How does model size affect performance? - Does positional encoding improve small model learning? - What are the compute vs performance tradeoffs? ## Approach - Train small transformers - Run controlled experiments - Analyze efficiency vs accuracy