This thesis explores the potential of machine learning methods for improving weather forecasts. Since weather is considered a spatiotemporal process that evolves over space through time, the thesis first investigates the design choices required for machine learning models to simulate synthetic spatiotemporal processes, such as the two-dimensional wave equation. It then develops a method for analyzing machine learning models that enables the extraction of unknown process-relevant context that parameterizes an observed simulated spatiotemporal process of interest. Relating these extracted factors to physical properties leads the thesis to physics-aware machine learning, where it explores how to fuse process knowledge from physics with the learning ability of artificial neural networks. Given the insights from those investigations, a competitive deep learning weather prediction model is designed to understand which design choices support data-driven algorithms to learn a meaningful function that predicts realistic and stable states of the atmosphere over hundreds of hours, days, and weeks into the future.