Empowering weather and climate forecast
If you are interested in workflow tools for machine learning as they are available now, and as they will be developed by MAELSTROM, please have a look at this report. We present the results of a survey of requirements for software tools to support the MAELSTROM applications and outline how MAELSTROM will combine existing tools with customised products to fill the gaps towards the optimal software framework for machine learning developments in Earth system applications.
An exciting step! We are looking forward to disclosing our first version of MAELSTROM workflow tools for download and use.
The tools provide a framework to run and compare different machine learning libraries regarding performance and usability which allows to make fair performance benchmarks of different software solutions on various hardware frameworks. It is possible to store the developments using the MAELSTROM protocol. Once ready, access our GitHub page for detailed documentation and software download.
For individuals, it is almost impossible to make a fair comparison between the performance of different machine learning solutions based on different machine learning libraries when running on different hardware configurations. But we'll shed some light into the dark via a fair comparison of the MAELSTROM machine learning applications running on several hardware designs and using the MAELSTROM workflow tools which allow to compare the performance of different machine learning libraries. This should also help you to judge which software and hardware solutions may work best for your application.
Are you struggling to optimise the movement of data and I/O in your machine learning project? Complementing the MAELSTROM workflow, this deliverable (add link) is addressing the organization and distribution of data with respect to the MAELSTROM machine learning problems and an optimal solution to load and use data will be provided. Different strategies for the data loading pipeline will be devised and implemented, including aspects of overlapping bespoke weather data pre-processing with training.
The final version of the MAELSTROM workflow tools have been published and are available for download and use. Please have a look at this deliverable for further details.
This deliverable will provide a final update on the performance of different machine learning solutions, software products and hardware designs as developed in MAELSTROM.