The company Graphcore has used the MAELSTROM dataset on the emulation of gravity wave drag parametrisation for performance tests with their Intelligence Processing Units (IPUs). See Graphcore's blog here.
What data scientist do can be pretty abstract to non-datanerds. We felt we would like to give everybody some feeling for how important it is to understand and predict weather and climate better, but also how difficult it is. Finally, however, what awesome possibilities a combination of high performance computing and machine learning put in our hands! Have a look and tell us what you think!
Hello machine learning and weather & climate scientists! Learn about the first versions of our machine learning tools and solutions, including architectures and loss functions. Get the results of a survey of customized machine learning solutions here.
We're excited to have reached a big milestone end of August: a series of datasets went public.
Dataset for energy production forecast
Dataset for 2m temperature downscaling
Dataset for ensemble predictions
Datasets for 2m temp. and precipitation short-range forecasts
Dataset to emulate radiation
Feedback and comments will be much appreciated.
Why is project MAELSTROM urgently needed, and how will it impact weather & climate prediction, machine learning and supercomputing? Read in ECMWF's science blog how MAELSTROM will radically change W&C predition with new ML tools, HPC-ready development platforms for data scientists and high-performance compute system designs: ECMWF Science Blog
A paper on physics emulation using machine learning has been accepted for publication in the Journal of Advances in Modelling of the Earth System (JAMES). This work creates a MAELSTROM dataset to emulate a piece of ECMWF’s weather forecast model and will feature in JAMES as part of a new special collection on using machine learning for earth system modelling. The work emulates the parametrisation of non-orographic gravity wave drag (NOGWD), a key physical process in the accurate modelling of stratospheric winds and temperatures. Using neural networks we accurately emulate this physical process even over long simulation periods and by using a high-fidelity reference scheme, we are able to improve prediction quality when compared to the standard scheme. Creating and publishing a dataset will enable other researchers to test machine learning methods and seek to produce even better emulators. More information
At ISC 2021 - one of the largest virtual community get togethers for high performance computing (HPC), machine learning (ML) and high performance data analytics (HPDA) - MAELSTROM member Peter Dueben has chaired a focus session on machine learning for weather forecasting. Here, Jason Hickey from Google Research provided an overview on the latest developments of machine learning applications for now-casting, MAELSTROM member Martin Schultz from Forschungszentrum Juelich talked about whether bigger is always better for deep learning applications in air quality, and Duncan Watson-Parris from the University of Oxford showed us results on the emulation of climate models. More information
The MAELSTROM project has been kicked-off via a virtual meeting. The meeting started with a short talk by our Project Officer Daniel Opalka and continued with presentations and discussions of the different work packages. The public had the opportunity to learn more about the project during a public plenary presentation. The recording of the presentation can be found here.
Great news! In January 2020, we have submitted our application - and now we got notice that the European Commission has signed our grant agreement. We're excited to get the chance to turn the page and write a new chapter of weather and climate forecasting, but equally a new chapter of HPC-based machine learning software environments and hardware configuration.