Cryo-EM is a powerful tool for understanding macromolecular structures, yet current methods for structure reconstruction are slow and computationally demanding. To accelerate research on pose estimation, we present CESPED, a data set specifically designed for supervised pose estimation in cryo-EM. Alongside CESPED, we provide a PyTorch package to simplify cryo-EM data handling and model evaluation. We evaluate the performance of a baseline model, Image2Sphere, on CESPED, which shows promising results but also highlights the need for further improvements. Additionally, we illustrate the potential of deep learning-based pose estimators to generalize across different samples, suggesting a promising path toward more efficient processing strategies.