Today in most modern enterprises, people are tackling the same question: how can we use AI to perform an integral function in less time and yield more accurate or useful results? Every industry is playing with this conundrum. Nowhere is this more the case than with weather forecasting.
Traditional methods are time consuming and rely on subject matter expertise of physics to model the atmosphere and oceans, with many advances relying on increased compute power and satellite imagery. But a more data-driven approach - using machine learning to learn from historical weather pattern data - is being adopted by more and more actors, both in big tech and the research community.
For energy enterprises where weather conditions dictate how the business functions, there are a myriad of potential applications and benefits of AI-enabled weather forecasting, both in the short and long term. With increased energy transition to renewable sources and the advent of microgeneration, improvements in weather forecasting will have a great impact on the energy system as a whole.
Traditional weather forecasting methods use Numerical Weather Prediction (NWP), which relies on physics-based models to simulate atmospheric and ocean behaviour. These models use thermodynamic principles, weather station data, and satellite imagery to create forecasts.
Systems like ECMWF’s Integrated Forecasting System (IFS) and NCEPSs Global Forecast System (GFS) - common weather models used by companies across all industries - are codified models of these physics equations, requiring significant computational resources to run and meteorological expertise to build.
However, improvements in NWP are generally slow and steady, with most gains coming from breakthroughs in physics and increased computational power. With the slowing of Moore’s law, so too will the improvements due to computational gains. Much of the existing NWP code is tied to traditional CPU based hardware, making it difficult to optimise for newer technologies, limiting further progress and opening the door for AI-driven approaches to accelerate innovation.
Cost: AI-enabled weather forecasting uses data-driven techniques to predict weather patterns, bypassing the need to directly model atmospheric physics. By learning from historical weather data, predictions can be generated in minutes on consumer-grade hardware, compared to hours on supercomputers for traditional NWP models. This has obvious cost benefits.
Accuracy: Some AI models – such as ECMWF’s AIFS, the group’s first advanced, AI-based forecasting system – have proven more accurate in predicting extreme weather events, like Hurricane Milton. Where AIFS predicted landfall within seven miles, compared to errors of up to 100 miles by traditional methods.
Expertise: AI forecasting also reduces the expertise needed to create and run models, making weather prediction more accessible. Machine learning skills are more widely available than deep meteorological knowledge, and AI models can run on consumer hardware. Organisations such as RWE are already investing in teams focusing on AI weather forecasting. However, AI models still require vast amounts of historical data (~100s TB) to train, and also still rely on NWP data for initialization before AI models take over forecasting time steps.
Technical benefits aside, all of this allows businesses to behave differently. Currently, NWP models are generally run 2-4 times a day, due to the availability of observation data and the time taken for data assimilation and running of the model. With the advent of AI weather forecasting models, this paradigm can change.
While most mainstream AI models still use NWP operational data as the initial state, research is ongoing into models that can run using individual observations, rather than assimilated data. This can be game changing for large organisations who already have networks of sensors or access to satellite data, which is now more accessible and affordable than ever before.
Similarly, models can be fine tuned for specific regions, increasing the accuracy in those regions but also improving spatial or temporal granularity - and businesses have options. ECMWF are now offering tools to aid users in running their own models with tools such as anemoi. Products like Jua AI and Salient are emerging to offer AI-driven weather forecasting as a service, capitalising on these advantages.
At a macro level, the economic loss caused by extreme weather events over the last 40 years totalled over $4 trillion. But the day to day operation and planning of businesses across the energy system is impacted by varying weather conditions. Balancing the power network requires knowledge of weather conditions to understand supply from renewable energy sources, or demand due to hot or cold weather. More advanced forecasting can bring efficiencies across multiple use cases.
For example, commodities traders attempting to predict crop yields or power demand will have no edge on the market using traditional NWP forecast data alone. AI can provide more accurate results, with custom models trained on specific areas or weather event paths giving users an extra advantage.
Knowing when a drought or storm might hit before others can secure lower prices and help inform decisions at every level. The figure below shows a comparison between DeepMind’s GraphCast model and ECMWFs NWP HRES model. On the left, the median cyclone track error is shown up to five days lead time. The GraphCast model has similar accuracy at one day lead time, however five days ahead it is ~10% more accurate than the NWP model.
Predicting energy demand and balancing this against supply is a key facet for energy generators, distributors, transmitters and retailers. Forecasting every six hours may not be sufficient to effectively balance the network - not to mention there are other models that come into play such as power flow analysis, which my colleague Byron Allen writes about here.
With modelling the input and output of energy with the addition of microgeneration and half hourly energy usage data, these more regular and more accurate weather forecasts can help network operators handle the fluctuations in supply and demand with less disruption to customers. In the long term, AI weather models that are fine tuned to different geographies could further improve usage predictions and improve with time.
Despite these widespread uses and benefits, ML-based weather forecasting is unlikely to replace NWP anytime soon. New models are using a combination of both ML and NWP, and certain parts of the NWP chain can be replaced by ML models in place giving the best of both worlds.
The field is moving quickly, and so there remain challenges around explainability and possessing the right skill set. In order to stay ahead of the curve, the energy and utilities industry needs to increase capability within the quantitative meteorology space - this includes capability in machine learning and AI. If companies are able to run, fine tune, or even build their own models, they will be able to benefit from increased efficiencies and gain an edge on competitors.
.png)
.png)
