GenCast predicts weather and the risks of extreme conditions with state-of-the-art accuracy - Related to a, advances, forecasting, faster, computing

2023: A Year of Groundbreaking Advances in AI and Computing

organization 2023: A Year of Groundbreaking Advances in AI and Computing Share.

This has been a year of incredible progress in the field of Artificial Intelligence (AI) research and its practical applications. As ongoing research pushes AI even farther, we look back to our perspective , titled “Why we focus on AI (and to what end),” where we noted: We are committed to leading and setting the standard in developing and shipping useful and beneficial applications, applying ethical principles grounded in human values, and evolving our approaches as we learn from research, experience, users, and the wider community. We also believe that getting AI right — which to us involves innovating and delivering widely accessible benefits to people and society, while mitigating its risks — must be a collective effort involving us and others, including researchers, developers, users (individuals, businesses, and other organizations), governments, regulators, and citizens. We are convinced that the AI-enabled innovations we are focused on developing and delivering boldly and responsibly are useful, compelling, and have the potential to assist and improve lives of people everywhere — this is what compels us. In this Year-in-Review post we’ll go over some of Google Research’s and Google DeepMind’s efforts putting these paragraphs into practice safely throughout 2023. Advances in Products & Technologies This was the year generative AI captured the world’s attention, creating imagery, music, stories, and engaging conversation about everything imaginable, at a level of creativity and a speed almost implausible a few years ago. In February, we first launched Bard, a tool that you can use to explore creative ideas and explain things simply. It can generate text, translate languages, write different kinds of creative content and more.

Later in the year, we released Imagen 2, which improved outputs via a specialized image aesthetics model based on human preferences for qualities such as good lighting, framing, exposure, and sharpness. In October, we launched a feature that helps people practice speaking and improve their language skills. The key technology that enabled this functionality was a novel deep learning model developed in collaboration with the Google Translate team, called Deep Aligner. This single new model has led to dramatic improvements in alignment quality across all tested language pairs, reducing average alignment error rate from 25% to 5% compared to alignment approaches based on Hidden Markov models (HMMs). In November, in partnership with YouTube, we showcased Lyria, our most advanced AI music generation model to date. We released two experiments designed to open a new playground for creativity, DreamTrack and music AI tools, in concert with YouTube’s Principles for partnering with the music industry on AI technology. Then in December, we launched Gemini, our most capable and general AI model. Gemini was built to be multimodal from the ground up across text, audio, image and videos.

Our initial family of Gemini models comes in three different sizes, Nano, Pro, and Ultra. Nano models are our smallest and most efficient models for powering on-device experiences in products like Pixel. The Pro model is highly-capable and best for scaling across a wide range of tasks. The Ultra model is our largest and most capable model for highly complex tasks.

By providing algorithmic prompts, we can teach a model the rules of arithmetic via in-context learning.

In the domain of visual question answering, in a collaboration with UC Berkeley researchers, we showed how we could advanced answer complex visual questions (“Is the carriage to the right of the horse?”) by combining a visual model with a language model trained to answer visual questions by synthesizing a program to perform multi-step reasoning. We are now using a general model that understands many aspects of the software development life cycle to automatically generate code review comments, respond to code review comments, make performance-improving suggestions for pieces of code (by learning from past such changes in other contexts), fix code in response to compilation errors, and more. In a multi-year research collaboration with the Google Maps team, we were able to scale inverse reinforcement learning and apply it to the world-scale problem of improving route suggestions for over 1 billion consumers. Our work culminated in a 16–24% relative improvement in global route match rate, helping to ensure that routes are advanced aligned with user preferences. We also continue to work on techniques to improve the inference performance of machine learning models. In work on computationally-friendly approaches to pruning connections in neural networks, we were able to devise an approximation algorithm to the computationally intractable best-subset selection problem that is able to prune 70% of the edges from an image classification model and still retain almost all of the accuracy of the original. In work on accelerating on-device diffusion models, we were also able to apply a variety of optimizations to attention mechanisms, convolutional kernels, and fusion of operations to make it practical to run high quality image generation models on-device; for example, enabling “a photorealistic and high-resolution image of a cute puppy with surrounding flowers” to be generated in just 12 seconds on a smartphone.

Advances in capable language and multimodal models have also benefited our robotics research efforts. We combined separately trained language, vision, and robotic control models into PaLM-E, an embodied multi-modal model for robotics, and Robotic Transformer 2 (RT-2), a novel vision-language-action (VLA) model that learns from both web and robotics data, and translates this knowledge into generalized instructions for robotic control.

RT-2 architecture and training: We co-fine-tune a pre-trained vision-language model on robotics and web data. The resulting model takes in robot camera images and directly predicts actions for a robot to perform.

The TPUGraphs dataset has 44 million graphs for ML program optimization.

We developed a new load balancing algorithm for distributing queries to a server, called Prequal, which minimizes a combination of requests-in-flight and estimates the latency. Deployments across several systems have saved CPU, latency, and RAM significantly. We also designed a new analysis framework for the classical caching problem with capacity reservations.

Heatmaps of normalized CPU usage transitioning to Prequal at 08:00.

Pause video Play video Contrails detected over the United States using AI and GOES-16 satellite imagery.

Watch A selection of GraphCast’s predictions rolling across 10 days showing specific humidity at 700 hectopascals (about 3 km above surface), surface temperature, and surface wind speed.

Health and the Life Sciences The potential of AI to dramatically improve processes in healthcare is significant. Our initial Med-PaLM model was the first model capable of achieving a passing score on the [website] medical licensing exam. Our more recent Med-PaLM 2 model improved by a further 19%, achieving an expert-level accuracy of [website] These Med-PaLM models are language-based, enable clinicians to ask questions and have a dialogue about complex medical conditions, and are available to healthcare organizations as part of MedLM through Google Cloud. In the same way our general language models are evolving to handle multiple modalities, we have in the recent past shown research on a multimodal version of Med-PaLM capable of interpreting medical images, textual data, and other modalities, describing a path for how we can realize the exciting potential of AI models to help advance real-world clinical care.

Med-PaLM M is a large multimodal generative model that flexibly encodes and interprets biomedical data including clinical language, imaging, and genomics with the same model weights.

Med-PaLM M is a large multimodal generative model that flexibly encodes and interprets biomedical data including clinical language, imaging, and genomics with the same model weights.

Examples of AlphaMissense predictions overlaid on AlphaFold predicted structures (red – predicted as pathogenic; blue – predicted as benign; grey – uncertain). Red dots represent known pathogenic missense variants, blue dots represent known benign variants. Left: HBB protein. Variants in this protein can cause sickle cell anaemia. Right: CFTR protein. Variants in this protein can cause cystic fibrosis.

Monk Skin Tone (MST) Scale. See more at [website].

SynthID generates an imperceptible digital watermark for AI-generated images.

One of the most used attributes is “Explain error” — whenever the user encounters an execution error in Colab, the code assistance model provides an explanation along with a potential fix.

During the telecommunication boom, Claude Shannon, in his seminal 1948 paper¹, posed a question that would revoluti...

Impact Targeting early-onset Parkinson’s with AI Share.

AlphaFold predictions are paving the way towards new treatments that can im...

Research A catalogue of genetic mutations to help pinpoint the cause of diseases Share.

New AI tool classifies the effects of 71 mi...

GraphCast: AI model for faster and more accurate global weather forecasting

Research GraphCast: AI model for faster and more accurate global weather forecasting Share.

Our state-of-the-art model delivers 10-day weather predictions at unprecedented accuracy in under one minute The weather affects us all, in ways big and small. It can dictate how we dress in the morning, provide us with green energy and, in the worst cases, create storms that can devastate communities. In a world of increasingly extreme weather, fast and accurate forecasts have never been more essential. In a paper , we introduce GraphCast, a state-of-the-art AI model able to make medium-range weather forecasts with unprecedented accuracy. GraphCast predicts weather conditions up to 10 days in advance more accurately and much faster than the industry gold-standard weather simulation system – the High Resolution Forecast (HRES), produced by the European Centre for Medium-Range Weather Forecasts (ECMWF). GraphCast can also offer earlier warnings of extreme weather events. It can predict the tracks of cyclones with great accuracy further into the future, identifies atmospheric rivers associated with flood risk, and predicts the onset of extreme temperatures. This ability has the potential to save lives through greater preparedness. GraphCast takes a significant step forward in AI for weather prediction, offering more accurate and efficient forecasts, and opening paths to support decision-making critical to the needs of our industries and societies. And, by open sourcing the model code for GraphCast, we are enabling scientists and forecasters around the world to benefit billions of people in their everyday lives. GraphCast is already being used by weather agencies, including ECMWF, which is running a live experiment of our model’s forecasts on its website.

Watch A selection of GraphCast’s predictions rolling across 10 days showing specific humidity at 700 hectopascals (about 3 km above surface), surface temperature, and surface wind speed.

The challenge of global weather forecasting Weather prediction is one of the oldest and most challenging–scientific endeavours. Medium range predictions are crucial to support key decision-making across sectors, from renewable energy to event logistics, but are difficult to do accurately and efficiently. Forecasts typically rely on Numerical Weather Prediction (NWP), which begins with carefully defined physics equations, which are then translated into computer algorithms run on supercomputers. While this traditional approach has been a triumph of science and engineering, designing the equations and algorithms is time-consuming and requires deep expertise, as well as costly compute resources to make accurate predictions. Deep learning offers a different approach: using data instead of physical equations to create a weather forecast system. GraphCast is trained on decades of historical weather data to learn a model of the cause and effect relationships that govern how Earth’s weather evolves, from the present into the future. Crucially, GraphCast and traditional approaches go hand-in-hand: we trained GraphCast on four decades of weather reanalysis data, from the ECMWF’s ERA5 dataset. This trove is based on historical weather observations such as satellite images, radar, and weather stations using a traditional NWP to ‘fill in the blanks’ where the observations are incomplete, to reconstruct a rich record of global historical weather. GraphCast: An AI model for weather prediction GraphCast is a weather forecasting system based on machine learning and Graph Neural Networks (GNNs), which are a particularly useful architecture for processing spatially structured data. GraphCast makes forecasts at the high resolution of [website] degrees longitude/latitude (28km x 28km at the equator). That’s more than a million grid points covering the entire Earth’s surface. At each grid point the model predicts five Earth-surface variables – including temperature, wind speed and direction, and mean sea-level pressure – and six atmospheric variables at each of 37 levels of altitude, including specific humidity, wind speed and direction, and temperature. While GraphCast’s training was computationally intensive, the resulting forecasting model is highly efficient. Making 10-day forecasts with GraphCast takes less than a minute on a single Google TPU v4 machine. For comparison, a 10-day forecast using a conventional approach, such as HRES, can take hours of computation in a supercomputer with hundreds of machines. In a comprehensive performance evaluation against the gold-standard deterministic system, HRES, GraphCast provided more accurate predictions on more than 90% of 1380 test variables and forecast lead times (see our Science paper for details). When we limited the evaluation to the troposphere, the 6-20 kilometer high region of the atmosphere nearest to Earth’s surface where accurate forecasting is most crucial, our model outperformed HRES on [website] of the test variables for future weather.

For inputs, GraphCast requires just two sets of data: the state of the weather 6 hours ago, and the current state of the weather. The model then predicts the weather 6 hours in the future. This process can then be rolled forward in 6-hour increments to provide state-of-the-art forecasts up to 10 days in advance.

superior warnings for extreme weather events Our analyses revealed that GraphCast can also identify severe weather events earlier than traditional forecasting models, despite not having been trained to look for them. This is a prime example of how GraphCast could help with preparedness to save lives and reduce the impact of storms and extreme weather on communities. By applying a simple cyclone tracker directly onto GraphCast forecasts, we could predict cyclone movement more accurately than the HRES model. In September, a live version of our publicly available GraphCast model, deployed on the ECMWF website, accurately predicted about nine days in advance that Hurricane Lee would make landfall in Nova Scotia. By contrast, traditional forecasts had greater variability in where and when landfall would occur, and only locked in on Nova Scotia about six days in advance. GraphCast can also characterize atmospheric rivers – narrow regions of the atmosphere that transfer most of the water vapour outside of the tropics. The intensity of an atmospheric river can indicate whether it will bring beneficial rain or a flood-inducing deluge. GraphCast forecasts can help characterize atmospheric rivers, which could help planning emergency responses together with AI models to forecast floods. Finally, predicting extreme temperatures is of growing importance in our warming world. GraphCast can characterize when the heat is set to rise above the historical top temperatures for any given location on Earth. This is particularly useful in anticipating heat waves, disruptive and dangerous events that are becoming increasingly common.

Severe-event prediction - how GraphCast and HRES compare. Left: Cyclone tracking performances. As the lead time for predicting cyclone movements grows, GraphCast maintains greater accuracy than HRES. Right: Atmospheric river prediction. GraphCast’s prediction errors are markedly lower than HRES’s for the entirety of their 10-day predictions.

The future of AI for weather GraphCast is now the most accurate 10-day global weather forecasting system in the world, and can predict extreme weather events further into the future than was previously possible. As the weather patterns evolve in a changing climate, GraphCast will evolve and improve as higher quality data becomes available. To make AI-powered weather forecasting more accessible, we’ve open sourced our model’s code. ECMWF is already experimenting with GraphCast’s 10-day forecasts and we’re excited to see the possibilities it unlocks for researchers – from tailoring the model for particular weather phenomena to optimizing it for different parts of the world. GraphCast joins other state-of-the-art weather prediction systems from Google DeepMind and Google Research, including a regional Nowcasting model that produces forecasts up to 90 minutes ahead, and MetNet-3, a regional weather forecasting model already in operation across the US and Europe that produces more accurate 24-hour forecasts than any other system. Pioneering the use of AI in weather forecasting will benefit billions of people in their everyday lives. But our wider research is not just about anticipating weather – it’s about understanding the broader patterns of our climate. By developing new tools and accelerating research, we hope AI can empower the global community to tackle our greatest environmental challenges.

Impact The race to cure a billion people from a deadly parasitic disease Share.

Researchers accelerate their search of life-saving ...

enterprise 2023: A Year of Groundbreaking Advances in AI and Computing Share.

This has been a year of incredible progress in the field...

Impact Google Cloud: Driving digital transformation Share.

Google Cloud empowers organizations to digitally transform themselves in...

GenCast predicts weather and the risks of extreme conditions with state-of-the-art accuracy

Technologies GenCast predicts weather and the risks of extreme conditions with state-of-the-art accuracy Share.

New AI model advances the prediction of weather uncertainties and risks, delivering faster, more accurate forecasts up to 15 days ahead Weather impacts all of us — shaping our decisions, our safety, and our way of life. As climate change drives more extreme weather events, accurate and trustworthy forecasts are more essential than ever. Yet, weather cannot be predicted perfectly, and forecasts are especially uncertain beyond a few days. Because a perfect weather forecast is not possible, scientists and weather agencies use probabilistic ensemble forecasts, where the model predicts a range of likely weather scenarios. Such ensemble forecasts are more useful than relying on a single forecast, as they provide decision makers with a fuller picture of possible weather conditions in the coming days and weeks and how likely each scenario is. Today, in a paper , we present GenCast, our new high resolution ([website]°) AI ensemble model. GenCast provides superior forecasts of both day-to-day weather and extreme events than the top operational system, the European Centre for Medium-Range Weather Forecasts’ (ECMWF) ENS, up to 15 days in advance. We’ll be releasing our model’s code, weights, and forecasts, to support the wider weather forecasting community.

The evolution of AI weather models GenCast marks a critical advance in AI-based weather prediction that builds on our previous weather model, which was deterministic, and provided a single, best estimate of future weather. By contrast, a GenCast forecast comprises an ensemble of 50 or more predictions, each representing a possible weather trajectory. GenCast is a diffusion model, the type of generative AI model that underpins the recent, rapid advances in image, video and music generation. However, GenCast differs from these, in that it’s adapted to the spherical geometry of the Earth, and learns to accurately generate the complex probability distribution of future weather scenarios when given the most recent state of the weather as input. To train GenCast, we provided it with four decades of historical weather data from ECMWF’s ERA5 archive. This data includes variables such as temperature, wind speed, and pressure at various altitudes. The model learned global weather patterns, at [website]° resolution, directly from this processed weather data.

Setting a new standard for weather forecasting To rigorously evaluate GenCast's performance, we trained it on historical weather data up to 2018, and tested it on data from 2019. GenCast showed advanced forecasting skill than ECMWF’s ENS, the top operational ensemble forecasting system that many national and local decisions depend upon every day. We comprehensively tested both systems, looking at forecasts of different variables at different lead times — 1320 combinations in total. GenCast was more accurate than ENS on [website] of these targets, and on [website] at lead times greater than 36 hours.

improved forecasts of extreme weather, such as heat waves or strong winds, enable timely and cost-effective preventative actions. GenCast offers greater value than ENS when making decisions about preparations for extreme weather, across a wide range of decision-making scenarios.

An ensemble forecast expresses uncertainty by making multiple predictions that represent different possible scenarios. If most predictions show a cyclone hitting the same area, uncertainty is low. But if they predict different locations, uncertainty is higher. GenCast strikes the right balance, avoiding both overstating or understating its confidence in its forecasts. It takes a single Google Cloud TPU v5 just 8 minutes to produce one 15-day forecast in GenCast’s ensemble, and every forecast in the ensemble can be generated simultaneously, in parallel. Traditional physics-based ensemble forecasts such as those produced by ENS, at [website]° or [website]° resolution, take hours on a supercomputer with tens of thousands of processors.

Advanced forecasts for extreme weather events More accurate forecasts of risks of extreme weather can help officials safeguard more lives, avert damage, and save money. When we tested GenCast’s ability to predict extreme heat and cold, and high wind speeds, GenCast consistently outperformed ENS. Now consider tropical cyclones, also known as hurricanes and typhoons. Getting advanced and more advanced warnings of where they’ll strike land is invaluable. GenCast delivers superior predictions of the tracks of these deadly storms.

GenCast’s ensemble forecast reveals a wide range of possible paths for Typhoon Hagibis seven days in advance, but the spread of predicted paths tightens over several days into a high-confidence, accurate cluster as the devastating cyclone approaches the coast of Japan.

more effective forecasts could also play a key role in other aspects of society, such as renewable energy planning. For example, improvements in wind-power forecasting directly increase the reliability of wind-power as a source of sustainable energy, and will potentially accelerate its adoption. In a proof-of-principle experiment that analyzed predictions of the total wind power generated by groupings of wind farms all over the world, GenCast was more accurate than ENS.

Next generation forecasting and climate understanding at Google GenCast is part of Google’s growing suite of next-generation AI-based weather models, including Google DeepMind’s AI-based deterministic medium-range forecasts, and Google Research’s NeuralGCM, SEEDS, and floods models. These models are starting to power user experiences on Google Search and Maps, and improving the forecasting of precipitation, wildfires, flooding and extreme heat. We deeply value our partnerships with weather agencies, and will continue working with them to develop AI-based methods that enhance their forecasting. Meanwhile, traditional models remain essential for this work. For one thing, they supply the training data and initial weather conditions required by models such as GenCast. This cooperation between AI and traditional meteorology highlights the power of a combined approach to improve forecasts and enhanced serve society. To foster wider collaboration and help accelerate research and development in the weather and climate community, we’ve made GenCast an open model and released its code and weights, as we did for our deterministic medium-range global weather forecasting model. We'll soon be releasing real-time and historical forecasts from GenCast, and previous models, which will enable anyone to integrate these weather inputs into their own models and research workflows. We are eager to engage with the wider weather community, including academic researchers, meteorologists, data scientists, renewable energy companies, and organizations focused on food security and disaster response. Such partnerships offer deep insights and constructive feedback, as well as invaluable opportunities for commercial and non-commercial impact, all of which are critical to our mission to apply our models to benefit humanity.

Since founding Towards Data Science in 2016, we’ve built the largest publication on Medium with a dedicated community of readers and contributors focu...

Research AI achieves silver-medal standard solving International Mathematical Olympiad problems Share.

Impact AlphaFold unlocks one of the greatest puzzles in biology Share.

AI system helps researchers piece together one of the larges...