prajjwal1/fluence

A deep learning library based on Pytorch focussed on low resource language research and robustness Winner of Pytorch Global Hackathon 2020. Fluence is a Pytorch based deep learning library focussed on providing computationally efficient, low resource methods and algorithms for NLP. Although the main focus is to provide support with transformers for NLP tasks, it can be extended with other domains and architectures as well. Currently in pre-alpha stage. List of implemented papers #### Adaptive Methods - Adaptive Attention Span in Transformers (ACL 2019) - Adaptively Sparse Transformers (EMNLP 2019) - Reducing Transformer Depth on Demand with Structured Dropout (ICLR 2020) #### Debiasing - Learning Robust Representations by Projecting Superficial Statistics Out (ICLR 2019) ------------------------------------------------------------------------------- - Installation - Overview ## Why Fluence ? Fluence is targeted towards two main goals: 1. **Compute efficiency**: Low resource research: 2. **Robustness**: Algorithms that either enhance our understanding of current methods or show where SoTA methods fail. It is as straightforward to use as HF Transformers, and fully integrates with Pytorch. Please note that the current modules (meta-trainer, siamese-trainer) which rely on inherited `Trainer` works with `transformers==3.0`. Newer version comes with a modified `Trainer`. ## Installing For stable version: ```bash pip3 install --user fluence ``` For development version (recommended): ```bash git clone github.com cd fluence python3 setup.py install --user ``` ## Overview The library contains implementation for the following approaches (many more to come): | Module | Method with documentation | -------------------------------------------------------------------------------------- | ---------------------------- | `fluence.adaptive` | Adaptive Methods | | `fluence.datasets` | Datasets | | `fluence.optim` | Optimizers | | `fluence.sampling` | Importance Sampling | | `f …