Droughts are a major disaster in Africa, threatening livelihoods through their influence on crop yields but also by impacting and weakening ecosystems. Modeling the vegetation state can help anticipate and reduce the impact of droughts by predicting the vegetation response over time. Forecasting the state of vegetation is challenging: it depends on complex interactions between the plants and different environmental drivers, which can result in both instantaneous and time-lagged responses, as well as spatial effects. Furthermore, modeling these interactions at the fine resolution of landscape scale can only rely on remote sensing observations, as in-situ measurements are not global and weather models have a coarse grid. With the increasing availability of remote sensing data, deep learning methods are a promising avenue for these spatiotemporal tasks. Here, we introduce both a dataset and a baseline deep neural network, modeling the vegetation response to climate at landscape scale in Africa.EarthNet2021 [1] introduced leveraging self-supervised learning for satellite imagery forecasting based on coarse-scale weather in Europe. Here, we introduce EarthNet2023 with a more narrow focus on drought impacts in Africa. It contains over 45,000 Spatio-temporal minicubes (each 1.28x1.28km) at representative locations over the whole African continent. Alongside Sentinel-2 reflectance, ERA5 weather, and topography, it also contains Sentinel-1 backscatter, soil properties, and a long-term Normalized Difference Vegetation Index (NDVI) climatology based on Landsat. The latter allows evaluating models on vegetation anomalies, thereby including modeling of drought impacts. EarthNet2023 is intended as an open benchmark challenge, allowing multiple research groups to develop their approaches to drought impact modeling in Africa. As a baseline for EarthNet2023, we train a  Convolutional Long Short-Term Memory (ConvLSTM) deep learning model. Previous work has shown it is suitable for spatiotemporal satellite imagery forecasting [2, 3, 4]. The ConvLSTM baseline captures the seasonal evolution of NDVI over a wide range of vegetation types. General spatial patterns are well-captured as well as a first indication of skill during weather extremes is seen, although the accuracy of the predictions is inconsistent, and the confidence in the model is therefore too low. This suggests, with further development, deep learning approaches are promising for modeling vegetation evolution in Africa, potentially even up to the degree to support anticipatory action with drought impact modeling. [1] Requena-Mesa, C., Benson, V., Reichstein, M., Runge, J., & Denzler, J. (2021). EarthNet2021: A large-scale dataset and challenge for Earth surface forecasting as a guided video prediction task. In CVPR 2021 (pp. 1132-1142).[2] Diaconu, C. A., Saha, S., Günnemann, S., & Zhu, X. X. (2022). Understanding the Role of Weather Data for Earth Surface Forecasting Using a ConvLSTM-Based Model. In CVPR 2022 (pp. 1362-1371).[3] Kladny, K. R. W., Milanta, M., Mraz, O., Hufkens, K., & Stocker, B. D. (2022). Deep learning for satellite image forecasting of vegetation greenness. bioRxiv.[4] Robin, C., Requena-Mesa, C., Benson, V., Alonso, L., Poehls, J., Carvalhais, N., & Reichstein, M. (2022). Learning to forecast vegetation greenness at fine resolution over Africa with ConvLSTMs. In Tackling Climate Change with Machine Learning: workshop at NeurIPS 2022.