Empowering weather and climate forecast


MAELSTROM ML-model live on yr.no

2024-03-26, Jan Mirus

One of the goals of MAELSTROM is to produce production-ready machine learning applications for weather and climate. At the end of February this year, MET Norway launched a deep-learning model developed by MAELSTROM on their public weather app Yr . This model—a U-Net with 1.3 million parameters—predicts 2 meter temperature for the Nordics for up to 58 hours into the future. The model combines information from a high-resolution numerical weather prediction model and temperature measurements from Netatmo’s network of citizen weather stations to create locally accurate predictions.
Thanks to MAELSTROM, the 6 million unique Yr users we have in Norway, Sweden, Denmark, Finland, Estonia, Latvia, and Lithuania can enjoy more accurate temperature forecasts to better plan their day.

Helping make the most out of what AI has to offer

2024-02-10, Jan Mirus

Weather forecasting is undergoing a transformation by integrating artificial intelligence (AI) into traditional physics-based models. Mariana Clare, a mathematician from the University of Oxford, specialized in numerical methods and fluid modeling, intertwining mathematics with real-world applications. Her journey led to a PhD in simulating coastal erosion and waves, during which she explored machine learning for weather prediction. Joining the European Centre for Medium-Range Weather Forecasts (ECMWF) in 2022, Mariana focused on enhancing forecast uncertainty information for the Destination Earth initiative. She pioneered the development of the AI-based Integrated Forecasting System (AIFS), which derives weather forecasts from past developments rather than physical laws. By introducing noise into machine-learning models, Mariana aims to create a reliable ensemble forecasting system. Despite its computational efficiency and promising performance, the AIFS requires further development to match the breadth and resolution of traditional models. Mariana envisions a future where physical and machine-learning models complement each other in providing high-resolution, accurate forecasts.

ESA-ECMWF workshop: abstract submission & registration now open

2024-01-26, Jan Mirus

ECMWF will hold a workshop together with ESA's European Space Research Institute:
Machine Learning for Earth System Observation and Prediction
7-10 May 2024 | ESA/ESRIN, Frascati (IT) | Free to attend

The workshop aims to explore the integration of machine learning (ML) and deep learning (DL) techniques with traditional Earth System Observation and Prediction (ESOP) methods. It highlights the significant advancements and potential synergies between ML/DL and conventional tools in satellite observations, weather and climate models, and post-processed model outputs. By leveraging ML/DL, researchers aim to enhance understanding and prediction capabilities of Earth's dynamics, contributing to goals such as the European Commission's Green Deal and Digital Strategy. The workshop spans four days and includes oral and poster presentations, along with parallel working groups aimed at addressing challenges, proposing solutions, and determining future directions in ML/DL for ESOP. The workshop seeks to generate a comprehensive report to advance the fusion of ML/DL with traditional ESOP techniques.
See workshop details | Register here

Survey on our apps and datasets now open

2023-11-07, Jan Mirus

How are we doing? We're collecting feedback. How useful are our apps and datasets? How easy is it to access them and work with them? Please take the time and complete our survey.

2023 workshop and bootcamp taking place next week – with audience from 59 countries!

2023-10-31, Jan Mirus

We will host our second workshop and second bootcamp next week at the premises of ECMWF in Reading, UK. We're thrilled to have gotten 242 registrations from 59 (!) countries on 6 continents!

Registration for our 2023 Dissemination Workshop closing tomorrow

2023-10-05, Jan Mirus

Hurry if you want to participate in our dissemination workshop on November 7th in Reading, UK or online.
Register here by tomorrow, 6th October.
The agenda is now online.

Introducing FZJ's AtmoRep

2023-08-29, Jan Mirus

This could become very influential:
Classical numerical models of the atmosphere exhibit biases due to incomplete process descriptions and they are computationally highly demanding. Recent AI-based weather forecasting models reduce the computational costs but lack the versatility of conventional models and do not provide probabilistic predictions.
The AtmoRep project asks if one can train one neural network that represents and describes all atmospheric dynamics. The FZJ team around Martin Schulz recently published a paper describing how this novel, task-independent stochastic computer model of atmospheric dynamics provides superior results for a wide range of applications: nowcasting, temporal interpolation, model correction, and counterfactuals.
Congrats to our MAELSTROM team members Bing Gong, Michael Langguth, Martin Schulz, and their FJZ colleagues!

Read the full paper here.
AtmoRep project page

Registration open!

2023-03-21, Jan Mirus

We're approaching two major events before MAELSTROM turns into the home stretch! The registration for both our second Dissemination Workshop and our second Boot Camp are open. Be sure to reserve your place.

4 open machine learning positions @ ECMWF!

2023-03-13, Jan Mirus

MAELSTROM partner and coordinator ECMWF has four (!) machine learning positions available. If you are a machine learning and/or HPC expert and want to walk the last mile towards operational use of ML in numerical weather predictions and climate services, get excited and check here!

MAELSTROM Hackathon II and Workshop II moved to November

2023-03-03, Jan Mirus

Our second hackathon and our second dissemination workshop, which were both originally planned for spring 2023, will be held in the week of November 6th, 2023. Please adjust your calendars and save the date. More information will follow soon!

MAELSTROM's take on women in science

2022-11-17, Jan Mirus

"We need more women in science." "We support women in science." - that's said easily. But when we took a closer look, we discovered lots of open questions and hidden conflicts. We conducted a range of interviews, checked existing initiatives and sources of information - but still have questions. Take a look and tell us what you think!

Thorsten Höfler awarded Sidney Fernbach Memorial Award

2022-10-04, Jan Mirus

Congratulations to Thorsten Höfler, who was honored for his pioneering contributions to large-​scale parallel processing systems and supercomputers. We are excited to have such a high-caliber contributor to science among the ranks of our team. Read more here.

Boot Camp registration now closed

2022-08-15, Jan Mirus

Registration is now closed!
43 hand-selected high caliber participants will join us for our first boot camp at Jülich Supercomputing Centre Sept 27 – Sept 30, 2022.
Hands-on work will allow them to learn more about the use of machine learning for weather and climate prediction in general, work with cutting-edge ML applications, test new workflow tools, and use one of Europe's fastest supercomputer!

Registration now open for our Boot Camp

2022-06-15, Jan Mirus

If you want to try out Germany's fastest supercomputer, hang with ECMWF's and JSC' scientific staff and other high-caliber people from across Europe, or be one of the first to try methods that may usher in a new era of weather & climate prediction:
You may be a candiate for our Boot Camp taking place Sept 27 through Sept 30 at Forschungszentrum Jülich!
Eligible for participation are Master and PhD students. You should have a basic background either in meteorology or machine learning or both. Some limited knowledge of Python and standard ML frameworks (e.g. TensorFlow or PyTorch) is preferred. Ideally, you are also familiar with Jupyter Notebooks and git, though this is not required.
This event will be held in English.
There will be no registration fee, but travel and accommodations must be paid by your home institution (or yourself).
Register here until 15th Aug 2022

We need more women in HPC, as it is changing the world

2022-04-24, Jan Mirus

We at MAELSTROM are no poster child either, no. Our team is a great bunch of people from all over the world, young and old, with all different kinds of backgrounds – but also mainly male.
HPC is about to change science and technology, and experience tells, that whatever changes science and technology, will sooner or later change our societies, our culture. Our hopes are that it will change the world for the better. But for this aim, women's perspectives, skills and opinions must be a full part of HPC.
A related event worth putting in your diaries is the Women in HPC — ISC’22 Workshop. Y'all probably have the ISC 2022 on your radars anyway, so make sure to allow time on Thu 02, Jun, 2022 9:00 am - 1:00 pm.

Workshop presentations, results & recordings ready for download

2022-03-29, Jan Mirus

60 participants, five MAELSTROM speakers, three speakers from our EuroHPC partner projects, and three speakers from the wider science and technology community, interactive polling and subsequent discussions made our first dissemination workshop a high-profile forum on the present and the future of ML-based weather & climate forecasting in a HPC context. To download the presentations that were shown, and watch the recordings of our talks, visit the ECMWF page.

First MAELSTROM workshop approaching!

2022-03-14, Jan Mirus

150 registrations for our first workshop! Yikes.
We want to make this a truly interactive event, and we're honored to welcome speakers of some of our EuroHPC sister projects: TimeX, Red-Sea and Deep-Sea.
Also, we were able to win more high-caliber speakers:
Jussi Leinonen (MeteoSwiss) will be talking about "Time-Consistent Downscaling of Atmospheric Fields with Generative Adversarial Networks";
Ryan Abernathey (Columbia) will introduce "Pangeo: An Open Source Ecosystem for Data-Intensive Science";
Thorsten Kurth (from our partner NVIDIA) will feature "Deep Learning for Earth Sciences in the HPC Context".
Luckily, this event is digital, so the size of our venue is flexible. You can still be part of it:
Visit the ECMWF page to register.

Happy women in science day!

2022-02-11, Peter Dueben

Let's celebrate all the great work of women in science. See, for example, a couple of fantastic female individuals working at ECMWF.

Machine Learning Crash Course for Earth System Science

2022-01-28, Peter Dueben

There has been been a 4-hour Crash Course in machine learning for Earth System Science at the "Center for Earth System Observation and Computational Analysis" (CESOC) by MAELSTROM Coordinator Peter Dueben that included both lectures and hands-on exercises.

Graphcore test their IPUs using MAELSTROM dataset

2021-11-30, Peter Dueben

The company Graphcore has used the MAELSTROM dataset on the emulation of gravity wave drag parametrisation for performance tests with their Intelligence Processing Units (IPUs). See Graphcore's blog here.

Imagine the dimensions

2021-11-09, Jan Mirus

What data scientist do can be pretty abstract to non-datanerds. We felt we would like to give everybody some feeling for how important it is to understand and predict weather and climate better, but also how difficult it is. Finally, however, what awesome possibilities a combination of high performance computing and machine learning put in our hands! Have a look and tell us what you think!

Another step towards new ml tools and solutions

2021-09-30, Jan Mirus

Hello machine learning and weather & climate scientists! Learn about the first versions of our machine learning tools and solutions, including architectures and loss functions. Get the results of a survey of customized machine learning solutions here.

Five datasets ready for download and public use

2021-09-01, Jan Mirus

We're excited to have reached a big milestone end of August: a series of datasets went public. Dataset for energy production forecast
Dataset for 2m temperature downscaling
Dataset for ensemble predictions
Datasets for 2m temp. and precipitation short-range forecasts
Dataset to emulate radiation
Feedback and comments will be much appreciated.

How MAELSTROM will change weather & climate prediction

2021-07-08, Peter Dueben

Why is project MAELSTROM urgently needed, and how will it impact weather & climate prediction, machine learning and supercomputing? Read in ECMWF's science blog how MAELSTROM will radically change W&C predition with new ML tools, HPC-ready development platforms for data scientists and high-performance compute system designs: ECMWF Science Blog

New paper to show that machine learning can improve the representation of gravity waves within weather models

2021-06-17, Peter Dueben

A paper on physics emulation using machine learning has been accepted for publication in the Journal of Advances in Modelling of the Earth System (JAMES). This work creates a MAELSTROM dataset to emulate a piece of ECMWF’s weather forecast model and will feature in JAMES as part of a new special collection on using machine learning for earth system modelling. The work emulates the parametrisation of non-orographic gravity wave drag (NOGWD), a key physical process in the accurate modelling of stratospheric winds and temperatures. Using neural networks we accurately emulate this physical process even over long simulation periods and by using a high-fidelity reference scheme, we are able to improve prediction quality when compared to the standard scheme. Creating and publishing a dataset will enable other researchers to test machine learning methods and seek to produce even better emulators. More information

Learn about machine learning for weather forecasting at ISC2021

2021-06-08, Peter Dueben

At ISC 2021 - one of the largest virtual community get togethers for high performance computing (HPC), machine learning (ML) and high performance data analytics (HPDA) - MAELSTROM member Peter Dueben has chaired a focus session on machine learning for weather forecasting. Here, Jason Hickey from Google Research provided an overview on the latest developments of machine learning applications for now-casting, MAELSTROM member Martin Schultz from Forschungszentrum Juelich talked about whether bigger is always better for deep learning applications in air quality, and Duncan Watson-Parris from the University of Oxford showed us results on the emulation of climate models. More information

MAELSTROM was successfully kicked-off (virtual)

2021-04-14, Jan Mirus

The MAELSTROM project has been kicked-off via a virtual meeting. The meeting started with a short talk by our Project Officer Daniel Opalka and continued with presentations and discussions of the different work packages. The public had the opportunity to learn more about the project during a public plenary presentation. The recording of the presentation can be found here.

We did it!

2020-12-02, Jan Mirus

Great news! In January 2020, we have submitted our application - and now we got notice that the European Commission has signed our grant agreement. We're excited to get the chance to turn the page and write a new chapter of weather and climate forecasting, but equally a new chapter of HPC-based machine learning software environments and hardware configuration.