AI and Future of Weather Forecasting for Pilots

Looking back for the past couple of years, artificial intelligence has been one of the hottest topics in the news and on social media. If you had any idea that AI was just another fad like the hula hoop, well, it’s not anymore.

Whether you like it or not, in the decades to come it will only continue to dominate our daily conversation and politics. It has the potential to change how we communicate, create, and even govern.

Coupled with the rapidly emerging technology of quantum computing, advancements in weather prediction will greatly benefit from AI over the next decade. Even so, it’s still in its infancy, with the road map still being defined, and there are many challenges before it will significantly help GA pilots make routine operational decisions.

But we’re closer than you might think.    

For nearly a century, weather prediction has been regarded as a complex physics problem. That is, scientists define a set of governing equations that describe or model how we understand the physical environment.

It’s through this understanding of the physical world that makes this largely a theory-driven approach. This approach creates a huge spatiotemporal dataset. In other words, there’s a ton of data generated that spreads over the entire world and from a time perspective, many days, weeks and even months into the future.    

Numerical weather prediction (NWP) has challenged scientists and computing professionals for the past century or so. Pilots benefit greatly from the weather predictions made by these number-crunching computer programs running on some of the fastest supercomputers on the planet. 

With AI and quantum computing emerging at the same time, there’s now an opportunity to provide forecasts to pilots based on new AI models at a fraction of the time it takes to generate traditional forecasts.  

Every dynamical NWP model must ingest a huge amount of weather data. Most of this is in the form of observations measured by many different sensors. Pilots are familiar with some of this observational data to include the routine and special surface observations (METARs and SPECIs) taken at many airports throughout the world. But these models ingest a variety of additional data sources that include radiosonde observations (from weather balloons), satellite data, aircraft reports, and ground-based radar data (e.g., NEXRAD) just to name a few. 

This data isn’t perfect by any stretch of the imagination and doesn’t adequately cover every square inch of the three-dimensional atmosphere we want to model. In fact, it’s sparse, especially over the vast oceans. NWP models assimilate all this data to produce a set of initial conditions called an analysis. It is this analysis that serves as the springboard for how these conditions will evolve over time based on those physical models scientists have developed going as far back as Lewis Fry Richardson, who in 1922 tried to manually predict the weather through the NWP approach. It wasn’t until 1950 when the first computer-aided weather forecasts were produced. 

NWP has been a quiet revolution, nonetheless. Although impressive, progress in this area has been incremental over the last 70 years with a focus of increasing the spatiotemporal resolution of the forecasts with each new version of the various models. Essentially a higher spatial resolution used in NWP models results in a higher realism of the produced weather forecasts that have a practical application to pilots.

For example, we can now provide a short-range forecast radar depiction at five-minute intervals that mimics our current ubiquitous NEXRAD ground-based radar mosaic. Given that pilots are good at interpreting ground-based radar imagery, it stands to reason they would also be good at interpreting these forecast radar images, which not only provides quantitative information (e.g., intensity, timing, and speed of movement) but also qualitative information (e.g., bowing line of convection implying strong straight-line winds or supercellular structure implying severe thunderstorms).    

When looking at the chaotic nature of the atmosphere, even small differences in these initial conditions will have a huge impact on the resulting prediction, especially for the accuracy of extended and long-range forecasts. There’s the infamous “butterfly effect” documented by meteorologist Edward Lorenz, who proposed that small errors in initial conditions will grow rapidly in the forecast.

In other words, the imperfect measurement of those initial conditions that are the input to the model, data assimilation phase, and incomplete understanding of the atmospheric physical processes inevitably introduce errors. This means that as the difference between the initialization time and the predicted time increases, the accuracy of the prediction will decrease. 

Back in 1966, meteorologist Jule Charney stated that this aspect of deterministic modeling would ultimately limit skillful weather forecasts to approximately two weeks. This makes perfect sense, and he was generous with the two week estimation. The further you are away in time from when those initial conditions were gathered, the more likely the forecast will stray from reality. In other words, meteorologists will have more confidence in a forecast  valid 12 hours from those initial conditions than one valid 72 hours from that initial point.   

“AI tools are statistical models: They recognize patterns in training datasets composed of decades of observational weather records and information gleaned from physical forecasting,” a Scientific American article stated. “Thus, these models may notice that the weather setup of a certain day resembles similar events in the past and make a forecast based on that pattern.” Furthermore, AI is not only generating more accurate weather predictions but also requires a lot less computational power. Making 10-day forecasts with AI takes less than a minute compared to the hours needed when using a conventional deterministic model approach that also employs a supercomputer with hundreds of processors.

It is an understatement to say that the history of AI or machine learning (ML)—these terms will be used interchangeably in this article—and that of NWP differ substantially. In fact, it wasn’t all that long ago that the topic of neural networks was front and center in the world of weather forecasting.

The term “neural network” evolved from how information is passed across neurons in a brain. That is, your brain observes some information such as seeing a grizzly bear in the distance, which is then processed by neurons and passed along a series of connections to numerous other neurons resulting in a thought or action such as, “That’s a grizzly bear, RUN!”

Nevertheless, neural networks as a method for weather forecasts soon ran into a dead end, given the computing resources at the time were not sufficient to solve these difficult problems. 

Moreover, solving such a problem using a traditional ML approach has not been optimal either. 

Now, deep learning (DL), as a particular form of ML and AI, has achieved promising results in fields such as speech recognition, computer vision, and other time-series problems. In recent years, DL has been applied to study these time series problems. Essentially, there’s a clear correlation between features that are obvious but hard to identify. The goal of a DL method is to extract the necessary temporal and spatial features from these large datasets. Weather forecasting is essentially a highly sophisticated time-series problem with a rather large geospatial dataset. 

DL revolutionizes weather forecasting by taking an entirely different approach to solving the weather forecasting problem. AI-generated weather forecasts rely on data rather than physical equations to predict the weather. Therefore, DL boils down to being a data-driven technique. It is essentially designed to discover and learn key relationships between the data received and the output data desired. This is accomplished by training it (i.e., learning) on reference input datasets and corresponding labeled output data.

So, is it possible to completely replace the NWP model described earlier with a pure data-driven AI-based model that has absolutely no knowledge of the physical world? 

As someone who spent most of my career studying and understanding the physical world of weather, I am reluctant to say the answer is…sigh…yes. However, one of the major bottlenecks before a pure AI-based solution is possible is the availability of sufficient training data. Although a pure AI-based model is likely possible, the best practical results in the short term are most likely to be from a hybrid of NWP and AI techniques. This will also facilitate getting all those geeky research meteorologists doing the sophisticated physical modeling a bit more interested in using AI—such as yours truly. That also will imply a new generation of scientists that will need to be trained to combine the AI and meteorology disciplines. 

AI Impact on Weather Forecasting

From a weather perspective, what can AI do for aviation? Most of the significant and practical advancements in weather forecasting with AI have been the extended-range forecasts.

Pilots are most interested in what might happen in six hours and not as much as what might happen in six days. In the next few years, you are not likely to see any groundbreaking improvement in turbulence, icing, or cloud cover and ceiling forecasts that are purely generated by AI. Turbulence, for example, happens on the scale of the size of the aircraft that has little or no training dataset for AI to learn. 

But there is hope. AI excels at processing vast amounts of real-time data from various sources (satellites, radar, ground stations, aircraft sensors) and identifies patterns and predicts immediate, short-term changes. This is crucial for “nowcasting”—forecasts for the next few minutes to a few hours, which is highly valuable for strategic and possible tactical aviation decisions. Certainly, it is possible that this AI form of nowcasting will include forecasts for adverse weather such as turbulence, ceiling heights, and visibility. Moreover, by predicting potential hazards with higher accuracy, AI empowers pilots to take proactive risk mitigation measures. This could involve suggesting alternative routes, recommending changes in altitude, or advising on appropriate departure/arrival times to avoid adverse conditions, ultimately leading to improved safety, especially in rapidly changing weather conditions.

A traditional weather briefing can be overwhelming with acronyms, raw data, and various charts. Through intelligent data synthesis and prioritization, AI can act as an assistant, sifting through NOTAMs, METARs, TAFs, and other information to synthesize it into a concise, easy-to-understand briefing. It can highlight critical information relevant to the specific flight (e.g., potential airframe icing at a certain altitude, a rapidly developing thunderstorm along the route), reducing the pilot’s workload and the chance of missing vital details. 

Also, through personalized and adaptive briefings, AI can learn a pilot’s preferences, experience level, and acceptable personal weather minimums. This allows for customized briefings that focus on the information most relevant to that pilot and their flight plan. For example, it could automatically flag conditions that exceed the pilot’s risk tolerance or warn of anomalies based on their preset criteria (e.g., no flying over mountains at night).

Though meteorologists generate TAFs, AI can enhance this with precise, frequent updates on airport conditions crucial for departures and arrivals. Services such as AirportWeather.com use AI to deliver ongoing, high-resolution AW-TAF (AI-powered TAFs) even at airports not serviced by a TAF.

A significant point noted in responses to recent advancements in weather forecasting is that this scientific practice has historically been considered a public good. The latest advancements in AI tools for weather forecasting have received mixed reactions, however. Many experts acknowledge their effectiveness while emphasizing that these tools will complement, rather than replace, traditional forecasting methods. 

The potential for misinformation from AI-generated weather models is an ongoing issue. It is important to involve expert oversight to validate the forecasts and address any errors. In the end, AI is still not a substitute for human ingenuity and traditional weather forecasting methods developed over many decades and rigorously tested in many real-world contexts will still exist for many years to come. 

Lastly, there is an increasing concern regarding the potential transition of weather forecasting ownership from the public to the private sector.

As a former NWS research meteorologist, this is likely the hardest part to swallow. There’s no doubt that the writing is on the wall. It’s already the case that AI has become, in many respects, too good to ignore. While AI promises significant improvements, it’s crucial to remember that it is a tool to assist the pilot, not replace their judgment.

It can be thought of as a copilot rather than an autopilot. Pilots will still need to understand the underlying weather phenomena and maintain ultimate responsibility for their flight decisions. 

This feature first appeared in the October Issue 963 of the FLYING print edition.