Representation Learning in Earth Science
"In this project, we will perform large scale machine learning on historical observational data of the atmosphere to infer a description of the dynamics derived directly from real-world measurements."
The AtmoRep project asks if one can train one neural network that represents and describes all atmospheric dynamics. AtmoRep's ambition is hence to demonstrate that the concept of large-scale representation learning, whose principle feasibility and potential was established by large language models such as the GPT line by OpenAI and Google's PaLM model, is also applicable to scientific data and in particular to atmospheric dynamics. The project is enabled by the large amounts of atmospheric observations that have been made in the past as well as advances on neural network architectures and self-supervised learning that allow for effective training on petabytes of data. We aim to train on all of the ERA5 reanalysis and, furthermore, fine tune on observational data such as satellite measurements to move beyond the limits of reanalyses.