Representation Learning in Earth System Science

"AtmoRep uses large-scale representation learning from artificial intelligence to determine a general description of the highly complex, stochastic dynamics of the atmosphere."

The atmosphere affects humans in a multitude of ways, from loss of life due to adverse weather effects to long-term social and economic impacts on societies. Computer simulations of atmospheric dynamics are, therefore, of great importance for the well-being of our and future generations. Here, we propose AtmoRep, a novel, task-independent stochastic computer model of atmospheric dynamics that can provide skillful results for a wide range of applications. This is enabled by a novel self-supervised learning objective and a unique ensemble that samples from the stochastic model with a variability informed by the one in the historical record. Our work establishes that large-scale neural networks can provide skillful, task-independent models of atmospheric dynamics. With this, they provide a novel means to make the large record of atmospheric observations accessible for applications and for scientific inquiry, complementing existing simulations based on first principles.

Short term weather forecasting

The short-term forecasting skills of the pre-trained AtmoRep model are being tested against the numerical prediction models from ECMWF (IFS), as well as the main machine learning based forecasting algorithms now available.

Downscaling

Downscaling, also known as super-resolution in computer vision, aims to increase the resolution of a coarse spatial input. In particular, we are using the COSMO REA6 Reanalysis dataset, to obtain a downscaled version of AtmoRep.

Bias corrections

The presence of biases in models can cause shifts and trends in reanalyses that can lead to sub-optimal forecasts. In AtmoRep we correct for biases in the total precipitations rates by fine tuning the pre-trained model on radar data from the RADKLIM dataset.

Counterfactuals

counterfactuals are a methodology to answer "what if" questions, i.e. for the attribution of human impacts on extreme weather events. In AtmoRep, we tweaked the additional informations given to the network to generate counterfactual scenarios.

Attention maps

Attention maps have been used before in natural language processing and computer vision to understand what a transformer neural network has learned and to visualize and exemplify their generalization abilities. Link to AtmoRep attention maps.

Temporal interpolation

Temporal interpolation refers to the task of (re-)creating atmospheric state data with a higher temporal resolution than the input. The AtmoRep model shows substantially better skill compared to linear interpolation for 3 hours time windows.

History

Presentations

Journal publications:

Talks: Posters:

Who we are

AtmoRep is a multi disciplinary collaboration among Computer Scientists from ECMWF, Earth Scientists from the Jülich Supercomputing Center and physicists from CERN.

Christian Lessig

Christian Lessig is a machine learning expert at ECMWF, the Eropean Center for Medium Weather Forecast. His background is in computer science but he also works today in scientific computing and numerical analysis. In the last years, his research moved towards addressing climate change, in particular by developing hybrid weather and climate simulation models that combine classical discretizations of the governing partial differential equations with neural networks that account for phenomena that are too expensive to simulate or whose physics is not well understood.

Ilaria Luise

Ilaria Luise is a Senior Research Fellow at CERN, the European Center for Nuclear Research in Geneva. She works as a physicists within the Innovation Division of the CERN IT-Department. Her background is in high energy physics and big data management. She is Co-PI of the EMP2 project at CERN, which is part of the CERN Innovation Programme on Environmental Applications (CIPEA). The EMP2 project aims at implementing the AtmoRep model into a digital twin engine. This is performed in collaboration with the EU funded InterTwin project and the Digital Twin initiative at CERN.

Martin Schultz

Martin Schultz is the group leader of the Earth System Data Exploration research group at the Jülich Supercomputing Center. He has more than 30 years of experience in working with atmospheric data and numerical modeling of atmospheric composition and climate. He has authored and co-authored more than 130 publications and has been listed as a highly cited researcher in the field of environmental sciences in 2017 and 2020. He is an ERC Advanced Grant holder (IntelliAQ) where he explores the potential of machine learning for the analysis of air quality data.

Bing Gong

Bing Gong is a postdoctoral researcher at the Jülich Supercomputing Center since 2019. Her current duties in the group are developing state-of-art scalable deep learning neural networks with a focus on time series prediction and video frame prediction in weather and air quality applications. She obtained her Ph.D. in the field of artificial intelligence in the application of environmental science and energy from the Technical University of Madrid, Spain, in July 2017.

Michael Langguth

Michael Langguth holds a Master degree in Physics of the Earth and Atmosphere from the Rheinische Friedrich-Wilhelms-University of Bonn. During his PhD he implemented a hybrid parametrization scheme for deep convection in the ICOsahedral Non‐hydrostatic (ICON) model developed by the DWD and the MPI-M. His current research interests focus on machine learning for atmospheric Earth system, combined with expertise from numerical modelling.

Scarlet Stadtler

Scarlet Stadtler is a postdoctoral associate at the Jülich Supercomputing Centre (JSC). Her research focuses on explainable machine learning and uncertainty quantification. She is a trained meteorologist and atmospheric chemist, she applies data-driven techniques in air quality research. As PI of the KISTE project, AI strategy for Earth System data, she leads the construction of an Earth-AI software platform and Earth-AI e-learning platform.