Social media platforms allow users to freely share their sentiments and opinions about is- sues and events or anything they feel like; however, they also make it easier to spread hate and abusive content. This paper introduces the HERDPhobia dataset, a manually annotated Twitter hate speech dataset against Fulani herders in Nigeria. The paper describes the data collection method, the annotation process, and the baseline experiment. We present a benchmark experiment using pre-trained languages models to classify the tweets as either hate speech or not hate speech, and the results were promising. We will release the dataset for further research in this direction.