uknowae58/bambara-lora-inference

Code to test Bambara LoRA adapter on Qwen2.5-0.5B without local GPU # Bambara LoRA Inference Run inference with your Bambara LoRA adapter on Qwen2.5-0.5B without local GPU! ## Option 1: HuggingFace Spaces (FREE GPU) ### Steps: 1. Go to HuggingFace Spaces 2. Create new Space → Choose **Gradio** template 3. Upload `app.py` and `requirements.txt` 4. Done! You'll get a free URL to test your model ### Use your LoRA: Update the code to load your LoRA from HuggingFace: ```python from peft import PeftModel model = PeftModel.from_pretrained(model, "uknowae58/bambara-qwen2.5-0.5b-v4") ``` ## Option 2: Google Colab (FREE GPU) ```python # Install !pip install unsloth transformers peft accelerate # Load model with LoRA from unsloth import FastLanguageModel model, tokenizer = FastLanguageModel.from_pretrained( model_name="Qwen/Qwen2.5-0.5B-Instruct", load_in_4bit=True, ) from peft import PeftModel model = PeftModel.from_pretrained(model, "uknowae58/bambara-qwen2.5-0.5b-v4") # Test prompt = "Traduis en bambara.: Hello world" inputs = tokenizer(prompt, return_tensors="pt").cuda() outputs = model.generate(**inputs, max_new_tokens=100) print(tokenizer.decode(outputs[0])) ``` ## Option 3: Run locally (needs GPU) ```bash pip install -r requirements.txt python app.py -k "Hello world" ``` ## Quick Demo ```python from transformers import pipeline # Use pipeline for easy inference pipe = pipeline("text-generation", model="uknowae58/bambara-qwen2.5-0.5b-v4") result = pipe("Traduis en bambara.: Hello") print(result) ``` ## Files | File | Description | |------|-------------| | `app.py` | Main inference script | | `gradio_app.py` | Gradio web interface | | `requirements.txt` | Dependencies | | `colab_demo.ipynb` | Google Colab notebook |