Visual cognitive processes have traditionally been examined with simplified stimuli, but generalization of these processes to the real-world is not always straightforward. Using images, computer-generated images, and virtual environments, researchers have examined processing of visual information in the real-world. Although referred to as scene perception, this research field encompasses many aspects of scene processing. Beyond the perception of visual features, scene processing is fundamentally influenced and constrained by semantic information as well as spatial layout and spatial associations with objects. In this review, we will present recent advances in how scene processing occurs within a few seconds of exposure, how scene information is retained in the long-term, and how different tasks affect attention in scene processing. By considering the characteristics of real-world scenes, as well as different time windows of processing, we can develop a fuller appreciation for the research that falls under the wider umbrella of scene processing.