In this work, we build a Question classification (QC) dataset from a social media platform, namely the Telegram public channel called @AskAnythingEthiopia. The platform allows asking questions that belong to various domains, like Politics, Music, Technology, Religion and so on. Questions are posted in Amharic, English, or Amharic in Latin script. Since the questions are posed in a mixed-code, we apply different strategies to pre-process the dataset. As part of the pre-processing tools, we build a Latin-to-Ethiopic-Script transliteration tool. We collect 8k Amharic and 24K Amharic but written in Latin script questions and develop deep learning-based question answering classifiers that attain an F-score of 57.79 in 20 different question categories. The datasets and pre-processing scripts are open-sourced to facilitate further research on the Amharic community-based question answering.