ChahineDerbali/RL-Tunisian-XO
Record type:
model
Creator:
Cha
Host:
# Tunisian XO — Reinforcement Learning Project
This project implements a **Reinforcement Learning (RL)** agent that learns to play the **Tunisian version of XO (Tic-Tac-Toe)**.
Unlike the classic game, each player:
- Places **exactly 3 pieces** during an initial placement phase
- Then **moves** pieces according to specific movement rules until a winner is found
The goal of the project is educational: to model the game as an MDP and apply **tabular Q-learning** with self-play.
---
## Game Rules (Tunisian XO)
### Board
- 3×3 grid
### Phase 1 — Placement
- X and O alternate placements
- X places 3 pieces, O places 3 pieces
- Order: X → O → X → O → X → O
- If a player wins during placement, the game ends
### Phase 2 — Movement
- X always starts moving first
- Players alternate moving **one** of their pieces
- Allowed moves:
- Horizontal and vertical adjacency
- Diagonal moves **only** between center and corners
- After each move, a win is checked
### End Conditions
- One player forms 3 in a row → win
- Maximum number of moves reached → draw
---
## Reinforcement Learning Design
### State
- Board configuration (9 cells)
- Number of pieces placed by X and O
- Current player
### Actions
- Placement: `("P", cell_index)`
- Movement: `("M", from_cell, to_cell)`
### Learning Algorithm
- **Tabular Q-learning**
- Two separate Q-tables:
- `Q_place` for placement actions
- `Q_move` for movement actions
### Training Strategy
- **Self-play with frozen snapshots**
- The agent trains against a periodically frozen version of itself
- Exploration via ε-greedy policy with slow decay
---
## Project Structure
```
tunisian_xo_rl/
│
├── env.py # Game environment (rules + transitions)
├── train.py # Self-play training loop
├── play.py # Human vs trained agent
├── main.py # CLI entry point
│
├── agents/
│ ├── q_agent.py # Q-learning agent (placement + movement)
│ ├── random_agent.py # Random baseline
│ └── human_agent. …