### README (README.md)
# WeatherNext
## WeatherNext 2
This repo contains the code for WeatherNext 2 (WN2), the global, medium-range
atmospheric and cyclone forecasting model developed by Google DeepMind and
Google Research.
It also contains code for prior generation models
[GraphCast](https://deepmind.google/blog/graphcast-ai-model-for-faster-and-more-accurate-global-weather-forecasting/)
and
[GenCast](https://deepmind.google/blog/gencast-predicts-weather-and-the-risks-of-extreme-conditions-with-sota-accuracy/).
### Accessing Forecast Data Feeds
If you are interested in directly accessing daily data feeds of WN2 model
outputs rather than running the model yourself, we provide them across multiple
platforms:
* [Google Cloud](https://developers.google.com/weathernext/guides/access-forecast)
(including Earth Engine, BigQuery, and Vertex AI).
* [WeatherLab](https://deepmind.google.com/science/weatherlab) (including cyclone tracks).
* [OpenMeteo](https://open-meteo.com/en/docs/google-weathernext-api) (including an API and interactive builder).
### Learn More
* **Model Guide & Documentation:** [Google Developers WeatherNext Guide](https://developers.google.com/weathernext/guides/models)
* **WeatherNext Cyclones Paper:** [Operational tropical cyclone forecasting
with AI](https://www.nature.com/articles/s41586-026-10953-2)
* **FGN/WN2 Technical Report:** [Skillful joint probabilistic weather
forecasting from marginals
(arXiv:2506.10772)](https://arxiv.org/abs/2506.10772)
* **WeatherNext 2 Blog Post:** [WeatherNext 2: Our most advanced weather forecasting model](https://blog.google/innovation-and-ai/models-and-research/google-deepmind/weathernext-2/)
* **WeatherNext Cyclones Blog Post:** [WeatherNext: AI model achieves breakthrough in forecasting cyclones](https://deepmind.google/blog/weathernext-ai-model-achieves-breakthrough-in-forecasting-cyclones/)
### Older Models
This repository serves as the primary home for the WeatherNext family models.
Alongside WN2, this repository also hosts the code and documentation for our
legacy and specialized models:
* [WeatherNext Graph](docs/weathernext1_graph/README.md): Deterministic
medium-range weather forecasting using graph neural networks. Published as
GraphCast.
* [WeatherNext Gen](docs/weathernext1_gen/README.md): Diffusion-based ensemble
forecasting for medium-range weather. Published as GenCast.
## Provided Pretrained Models
This repository provides code to run the different versions of WeatherNext 2 and
WeatherNext Cyclones. The only difference between them is that WN2 can also
predict 100m wind. In particular, WN2 also forecasts cyclones with the exact
same algorithm as WN Cyclones. Their weights are different due to independent
training runs.
### WeatherNext 2
1. **WeatherNext2_<2025** (Used Operationally): 0.25° resolution (~30km).
Fine-tuned on ECMWF HRES data and designed to be initialized directly from
operational HRES initial conditions rather than ERA5 reanalysis. Trained on
data through 2024. Corresponding weights files:
`WeatherNext2_<2025_model{1,2,3,4}.npz`.
### WeatherNext Cyclones - models which reproduce the results in paper
1. **WeatherNextCyclones_<2025** (Used Operationally): 0.25° resolution. The
model that ran live during the 2025 Atlantic hurricane season, publicly
referred to as FNV3 (NHC's postprocessed version was called GDMI). Trained
on data through 2024. The paper appendix contains a partial evaluation of
2025 in NHC basins for this model checkpoint, and how the tracker
improvement in September 2025 improved results. Corresponding weights files:
`WeatherNextCyclones_<2025_model{1,2,3,4}.npz`.
2. **WeatherNextCyclones_<2024**: 0.25° resolution. Reproduces results from the
paper on 2024. Trained on data through 2023. Corresponding weights files:
`WeatherNextCyclones_<2024_model{1,2,3,4}.npz`.
3. **WeatherNextCyclones_<2023**: 0.25° resolution. Reproduces results from the
paper on 2023. Trained on data through 2022. Corresponding weights files:
`WeatherNextCyclones_<2023_model{1,2,3,4}.npz`.
### WeatherNext Cyclones Mini
1. **WeatherNextCyclones_Mini_<2024**: 1° resolution. A lightweight version
suitable for lower memory and compute constraints (e.g., local testing or
single TPUs or GPUs). Not expected to match the performance of the larger
versions. Forecasts the same things as WeatherNext2_<2025, including
cyclones. Trained on data through 2023. Corresponding weights file:
`WeatherNextCyclones_Mini_<2024.npz`.
2. **WeatherNextCyclones_Mini_<2023**: As above, but only trained on data through
2022. Corresponding weights file: `WeatherNextCyclones_Mini_<2023.npz`.
Evaluation results for WeatherNextCyclones_Mini can be found in the appendix
of the
[WeatherNext Cyclones Paper](https://www.nature.com/articles/s41586-026-10953-2).
## Quick Start Guide
The easiest way to get started with WeatherNext 2 is by running our interactive
[Colab Notebook](docs/weathernext2/wn2_demo.ipynb), which can be opened from
[Colaboratory](https://colab.research.google.com/github/google-deepmind/weathernext/blob/master/docs/weathernext2/wn2_demo.ipynb).
This notebook defaults to WeatherNext Cyclones Mini, which we recommend running
using the `v5e-1` runtime, available for free as a Colab runtime. However, the
notebook can also be used to run the other models enumerated above (but these
will require a `v5p` accelerator).
In general, we recommend running WeatherNext 2 on TPU where possible, since its
implementation has been optimised for it. However, if choosing to run on GPU,
the attention implementation must be switched, as shown in the demo notebook.
The non-Mini models require H100 for sufficient VRAM. The Mini models should
manage inference on a P100.
Pre-trained weights and sample data are available on our [Google Cloud
Bucket](https://console.cloud.google.com/storage/browser/dm_graphcast).
**Inside the notebook, you will learn how to:**
1. Automatically load the required model weights from our storage bucket.
2. Load initial state weather data (e.g., HRES initial conditions).
3. Initialize the WN2 (FGN) architecture.
4. Run auto-regressive rollout steps to generate a forecast prediction.
5. Visualize the outputs (e.g., temperature, wind speed, geopotential height).
6. Run the direct tracker on model outputs to obtain track data for cyclones.
7. Compute the training loss on model predictions and targets, and take a
gradient step.
## Setup
### Installation
> [!NOTE]
> This is research code provided as-is for the purpose of running and
> experimenting with the published models. There are no guarantees of API
> stability and future updates may introduce breaking changes without notice. We
> recommend pinning to a specific release.
E.g.:
```bash
pip install git+https://github.com/google-deepmind/weathernext.git@v0.3.0
```
### Model Weights
To run WeatherNext 2 or WeatherNext Cyclones, you will need to download the
pre-trained model weights. You can access the weights on [Google Cloud
Bucket](https://console.cloud.google.com/storage/browser/dm_graphcast).
### Shared Utilities
The `utils/` directory contains shared libraries used by multiple WeatherNext
models, providing common infrastructure for autoregressive rollouts, input
normalization, graph building blocks, loss computation, and JAX-compatible
xarray utilities. See the per-model READMEs for model-specific code.
### Training Data
Full model training requires downloading the
[ERA5](https://www.ecmwf.int/en/forecasts/datasets/reanalysis-datasets/era5)
dataset from [ECMWF](https://www.ecmwf.int/), best accessed as Zarr via
[WeatherBench2](https://weatherbench2.readthedocs.io/en/latest/data-guide.html#era5).
Operational fine-tuning data is available via [WeatherBench2's HRES
data](https://weatherbench2.readthedocs.io/en/latest/data-guide.html#ifs-hres-t-0-analysis).
These datasets may be governed by separate terms and conditions. Check that you
can comply with any applicable restrictions before use.
## License
Copyright 2026 Google LLC.
The Colab notebooks and the associated code are licensed under the Apache
License, Version 2.0 (Apache 2.0); you may not use these materials except in
compliance with the Apache 2.0 license. You may obtain a copy of the License at:
https://www.apache.org/licenses/LICENSE-2.0.
All other materials are licensed under the Creative Commons Attribution 4.0
International (CC BY 4.0). You may obtain a copy of the License at:
[https://creativecommons.org/licenses/by/4.0/](https://creativecommons.org/licenses/by/4.0/).
Unless required by applicable law or agreed to in writing, all software and
materials distributed here under the Apache 2.0 or CC-BY 4.0 licenses are
distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
either express or implied. See the licenses for the specific language governing
permissions and limitations under those licenses.
## Disclaimers
This is not an officially supported Google product.
The WeatherNext models are part of an experimental research project. You are
solely responsible for determining the appropriateness of using or distributing
these models or any outputs they generate, and you assume all risks associated
with such use or distribution and your exercise of rights and permissions
granted by Google under the relevant license. Use discretion before relying on,
publishing, downloading, or otherwise using these models or any of their
outputs.
The WeatherNext models have not been produced in collaboration with nor endorsed
by any government meteorological agency or department, and in no way replaces
official alerts, warnings or notices published by such agencies.
## Citations
If you use any of the WeatherNext family of models in your research, please cite
our papers:
**WeatherNext 2 or WeatherNext Cyclones**
```latex
@article{Alet2026,
title={Operational Tropical Cyclone Forecasting with AI},
author={Alet, Ferran and Andersson, Tom R. and Price, Ilan and Markou, Stratis and El-Kadi, Andrew and Masters, Dominic and Li, Amy and Merchant, Samier and Williams, Natalie and Thornton, Gregory and MacKay, Ken and Graham, Olivia and Uddin, Akib and Gaiarin, Ben and Shah, Devaja and Kruse, Elinor and Hogsett, Wallace and Zelinsky, David and Cangialosi, John and Martinez, Jonathan and Franklin, James and DeMaria, Mark and Musgrave, Kate and Bain, Caroline L. and Titley, Helen and Stott, Jacklynn and Lam, Remi and Bell, Aaron and Komarek, Paul and Willson, Matthew and Sanchez-Gonzalez, Alvaro and Battaglia, Peter},
journal={Nature},
year={2026},
issn={1476-4687},
doi={10.1038/s41586-026-10953-2},
url={https://doi.org/10.1038/s41586-026-10953-2}
}
```
```latex
@article{alet2025skillful,
title={Skillful joint probabilistic weather forecasting from marginals},
author={Alet, Ferran and Price, Ilan and El-Kadi, Andrew and Masters, Dominic and Markou, Stratis and Andersson, Tom R and Stott, Jacklynn and Lam, Remi and Willson, Matthew and Sanchez-Gonzalez, Alvaro and Battaglia, Peter},
journal={arXiv preprint arXiv:2506.10772},
year={2025}
}
```
**WeatherNext 1 Gen (GenCast)**
```latex
@article{price2024gencast,
title={Probabilistic weather forecasting with machine learning},
author={Price, Ilan and Sanchez-Gonzalez, Alvaro and Alet, Ferran and Andersson, Tom R and El-Kadi, Andrew and Masters, Dominic and Ewalds, Timo and Stott, Jacklynn and Mohamed, Shakir and Battaglia, Peter and Lam, Remi and Willson, Matthew},
journal={Nature},
volume={637},
number={8044},
year={2024},
publisher={Nature Publishing Group},
doi={10.1038/s41586-024-08252-9}
}
```
**WeatherNext 1 Graph (GraphCast)**
```latex
@article{lam2023learning,
title={Learning skillful medium-range global weather forecasting},
author={Lam, Remi and Sanchez-Gonzalez, Alvaro and Willson, Matthew and Wirnsberger, Peter and Fortunato, Meire and Alet, Ferran and Ravuri, Suman and Ewalds, Timo and Eaton-Rosen, Zach and Hu, Weihua and others},
journal={Science},
volume={382},
number={6677},
pages={1416--1421},
year={2023},
publisher={American Association for the Advancement of Science}
}
```
## Acknowledgements
The WeatherNext models communicate with the following separate libraries and
packages:.
* Data and products of the European Centre for Medium-range Weather Forecasts
(ECMWF), as modified by Google.
* Modified Copernicus Climate Change Service information 2023\.
* NOAA's International Best Track Archive for Climate Stewardship (IBTrACS)
data, first accessed on 1 Dec 2022\.
Additionally, the colab notebooks include a few examples of ECMWF’s ERA5 and
HRES data that can be used as input to the models.
Neither the European Commission nor ECMWF is responsible for any use that may be
made of the Copernicus information or data it contains. ECMWF HRES datasets
Copyright statement: Copyright "© 2023 European Centre for Medium-Range Weather
Forecasts (ECMWF)". Source: [www.ecmwf.int](http://www.ecmwf.int/) License
Statement: ECMWF open data is published under a Creative Commons Attribution 4.0
International (CC BY 4.0).
[https://creativecommons.org/licenses/by/4.0/](https://creativecommons.org/licenses/by/4.0/)
Disclaimer: ECMWF does not accept any liability whatsoever for any error or
omission in the data, their availability, or for any loss or damage arising from
their use.
Use of the third-party materials referred to above may be governed by separate
terms and conditions or license provisions. Your use of the third-party
materials is subject to any such terms and you should check that you can comply
with any applicable restrictions or terms and conditions before use.
## Contact
For feedback and questions regarding the codebase or models, contact us at
`weathernext@google.com`.
Any information collected via email will be used in accordance with [Google's
privacy policy](http://policies.google.com/privacy).
---
### CONTRIBUTING (CONTRIBUTING.md)
# How to Contribute
## Contributor License Agreement
Contributions to this project must be accompanied by a Contributor License
Agreement. You (or your employer) retain the copyright to your contribution,
this simply gives us permission to use and redistribute your contributions as
part of the project. Head over to to see
your current agreements on file or to sign a new one.
You generally only need to submit a CLA once, so if you've already submitted one
(even if it was for a different project), you probably don't need to do it
again.
## Code reviews
All submissions, including submissions by project members, require review. We
use GitHub pull requests for this purpose. Consult
[GitHub Help](https://help.github.com/articles/about-pull-requests/) for more
information on using pull requests.
## Community Guidelines
This project follows [Google's Open Source Community
Guidelines](https://opensource.google/conduct/).
---
### Weathernext1 Gen/README (docs/weathernext1_gen/README.md)
## WeatherNext 1 Gen (GenCast)
This model was published as [GenCast: Diffusion-based ensemble forecasting for
medium-range weather](https://arxiv.org/abs/2312.15796). File and checkpoint
names retain the original "GenCast" naming for backward compatibility.
This package provides four pretrained models:
1. `GenCast 0p25deg <2019`, GenCast model at 0.25deg resolution with 13
pressure levels and a 6 times refined icosahedral mesh. This model is
trained on ERA5 data from 1979 to 2018 (inclusive), and can be causally
evaluated on 2019 and later years. This model was described in the paper
`GenCast: Diffusion-based ensemble forecasting for medium-range weather`
(https://arxiv.org/abs/2312.15796)
2. `GenCast 0p25deg Operational <2022`, GenCast model at 0.25deg resolution,
with 13 pressure levels and a 6 times refined icosahedral mesh. This model
is trained on ERA5 data from 1979 to 2018, and fine-tuned on HRES-fc0 data
from 2016 to 2021 and can be causally evaluated on 2022 and later years.
This model can make predictions in an operational setting (i.e., initialised
from HRES-fc0)
3. `GenCast 1p0deg <2019`, GenCast model at 1deg resolution, with 13 pressure
levels and a 5 times refined icosahedral mesh. This model is trained on ERA5
data from 1979 to 2018, and can be causally evaluated on 2019 and later
years. This model has a smaller memory footprint than the 0.25deg models
4. `GenCast 1p0deg Mini <2019`, GenCast model at 1deg resolution, with 13
pressure levels and a 4 times refined icosahedral mesh. This model is
trained on ERA5 data from 1979 to 2018, and can be causally evaluated on
2019 and later years. This model has the smallest memory footprint of those
provided and has been provided to enable low cost demonstrations (for
example, it is runnable in a free Colab notebook). While its performance is
reasonable, it is not representative of the performance of the GenCast
models (1-3) above. For reference, a scorecard comparing its performance to
ENS can be found in
[docs/](GenCast_1p0deg_Mini_ENS_scorecard.png).
Note that in this scorecard, GenCast Mini only uses 8 member ensembles (vs.
ENS' 50) so we use the fair (unbiased) CRPS to allow for fair comparison.
The best starting point is to open `gencast_mini_demo.ipynb` in
[Colaboratory](https://colab.research.google.com/github/google-deepmind/weathernext/blob/master/docs/weathernext1_gen/gencast_mini_demo.ipynb),
which gives an example of loading data, generating random weights or loading a
`GenCast 1p0deg Mini <2019` snapshot, generating predictions, computing the loss
and computing gradients. The one-step implementation of GenCast architecture is
provided in `gencast.py` and the relevant data, weights and statistics are in
the `gencast/` subdir of the Google Cloud Bucket.
### Instructions for running GenCast on Google Cloud compute
[cloud_vm_setup.md](cloud_vm_setup.md) contains
detailed instructions on launching a Google Cloud TPU VM. This provides a means
of running models (1-3) in the separate `gencast_demo_cloud_vm.ipynb` through
[Colaboratory](https://colab.research.google.com/github/google-deepmind/weathernext/blob/master/docs/weathernext1_gen/gencast_demo_cloud_vm.ipynb).
The document also provides
[instructions](cloud_vm_setup.md#running-inference-on-gpu)
for running GenCast on a GPU. This requires using a different attention
implementation.
### Brief description of relevant library files
In addition to the model-specific files below, shared library files
(`autoregressive.py`, `normalization.py`, `rollout.py`, etc.) are in the
`utils/` directory.
* `denoiser.py`: The GenCast denoiser for one step predictions.
* `denoisers_base.py`: Defines the interface of the denoiser.
* `dpm_solver_plus_plus_2s.py`: Sampler using DPM-Solver++ 2S from [1].
* `gencast.py`: Combines the GenCast model architecture, wrapped as a
denoiser, with a sampler to generate predictions.
* `nan_cleaning.py`: Wraps a predictor to allow it to work with data cleaned
of NaNs. Used to remove NaNs from sea surface temperature.
* `samplers_base.py`: Defines the interface of the sampler.
* `samplers_utils.py`: Utility methods for the sampler.
* `sparse_transformer.py`: General purpose sparse transformer that operates on
`TypedGraph`'s where both inputs and outputs are flat vectors of features
for each of the nodes and edges. `predictor.py` uses one of these for the
mesh GNN.
* `sparse_transformer_utils.py`: Utility methods for the sparse transformer.
* `transformer.py`: Wraps the mesh transformer, swapping the leading two axes
of the nodes in the input graph.
[1] DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic
Models, https://arxiv.org/abs/2211.01095
## License and Disclaimers
Copyright 2024-2026 Google LLC.
The Colab notebooks and the associated code are licensed under the Apache
License, Version 2.0 (Apache 2.0); you may not use these materials except in
compliance with the Apache 2.0 license. You may obtain a copy of the License at:
[https://www.apache.org/licenses/LICENSE-2.0](https://www.apache.org/licenses/LICENSE-2.0).
All other materials are licensed under the terms of the Creative Commons
Attribution 4.0 International (CC BY 4.0). You may obtain a copy of the License
at:
[https://creativecommons.org/licenses/by/4.0/](https://creativecommons.org/licenses/by/4.0/).
***August 6, 2026 UPDATE:** The license for the model weights in this repository
has been updated to permit commercial use. The previous license is now replaced
by the new terms outlined in this README. By continuing to access, download, or
use the weights (including those previously sourced from this repository), you
agree to the updated terms.*
This is not an officially supported Google product.
Unless required by applicable law or agreed to in writing, all software and
materials distributed here under the Apache 2.0 or CC-BY 4.0 licenses are
distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
either express or implied. See the licenses for the specific language governing
permissions and limitations under those licenses.
GenCast is part of an experimental research project. You are solely responsible
for determining the appropriateness of using or distributing GenCast or any
outputs generated and assume all risks associated with your use or distribution
of GenCast and outputs and your exercise of rights and permissions granted by
Google to you under the relevant License. Use discretion before relying on,
publishing, downloading or otherwise using GenCast or any outputs generated.
\[GenCast outputs have not been produced in collaboration with nor endorsed by
any government meteorological agency or department, and in no way replaces
official alerts, warnings or notices published by such agencies.\]
## Citations
If you use this work, consider citing our papers ([blog
post](https://deepmind.google/blog/gencast-predicts-weather-and-the-risks-of-extreme-conditions-with-sota-accuracy/),
[Nature](https://www.nature.com/articles/s41586-024-08252-9)):
```latex
@article{price2024gencast,
title={Probabilistic weather forecasting with machine learning},
author={Price, Ilan and Sanchez-Gonzalez, Alvaro and Alet, Ferran and Andersson, Tom R and El-Kadi, Andrew and Masters, Dominic and Ewalds, Timo and Stott, Jacklynn and Mohamed, Shakir and Battaglia, Peter and Lam, Remi and Willson, Matthew},
journal={Nature},
volume={637},
number={8044},
year={2024},
publisher={Nature Publishing Group},
doi={10.1038/s41586-024-08252-9}
}
```
## Acknowledgements
GenCast communicates with the following separate libraries and packages:.
* Data and products of the European Centre for Medium-range Weather Forecasts
(ECMWF), as modified by Google.
* Modified Copernicus Climate Change Service information 2023\.
Additionally, the colab notebooks include a few examples of ECMWF’s ERA5 and
HRES data that can be used as input to the models.
Neither the European Commission nor ECMWF is responsible for any use that may be
made of the Copernicus information or data it contains. ECMWF HRES datasets
Copyright statement: Copyright "© 2023 European Centre for Medium-Range Weather
Forecasts (ECMWF)". Source: [www.ecmwf.int](http://www.ecmwf.int/) License
Statement: ECMWF open data is published under a Creative Commons Attribution 4.0
International (CC BY 4.0).
[https://creativecommons.org/licenses/by/4.0/](https://creativecommons.org/licenses/by/4.0/)
Disclaimer: ECMWF does not accept any liability whatsoever for any error or
omission in the data, their availability, or for any loss or damage arising from
their use.
Use of the third-party materials referred to above may be governed by separate
terms and conditions or license provisions. Your use of the third-party
materials is subject to any such terms and you should check that you can comply
with any applicable restrictions or terms and conditions before use.
---
### Weathernext1 Gen/Cloud Vm Setup (docs/weathernext1_gen/cloud_vm_setup.md)
This document describes how to run `gencast_demo_cloud_vm.ipynb` through [Colaboratory](https://colab.research.google.com/github/deepmind/graphcast/blob/master/gencast_demo_cloud_vm.ipynb) using Google Cloud compute.
## TPU Cost and Availability
- There are 2 ways to access a Cloud TPU VM:
- "Spot"
- As per https://cloud.google.com/tpu/docs/spot, "Spot VMs
make unused capacity available at highly-discounted rates. Spot VMs can be
preempted (shut down) at any time… You can't restart TPU Spot VMs, and you
must recreate them after preemption."
- "On Demand"
- As per https://cloud.google.com/tpu/docs/quota#tpu_quota,
"On-demand resources won't be preempted, but on-demand quota does not
guarantee there will be enough available Cloud TPU resources to satisfy
your request."
- N.B. you may come across references to "preemptible" TPUs, these have been
[deprecated](https://cloud.google.com/tpu/docs/preemptible)
- As per https://cloud.google.com/compute/docs/instances/spot#pricing,
"Spot prices give you 60-91% discounts compared to the standard price for most
machine types and GPUs."
- As per https://cloud.google.com/tpu/pricing, at the time of
writing, "On Demand" prices (price pre-discount applied above) are:
- v4p: ~$3.20 per chip hour
- v5e: ~$1.20 per chip hour
- v5p: ~$5.20 per chip hour
- Reference pricing, estimating ~10 minutes for TPU set up/data transfer
- v5e: 4 sample, 30 step rollout of 1deg GenCast takes ~5 minutes, including
tracing and compilation, this will cost ~ 15 (minutes) * 1.20 ($/chip hour) *
1/60 (minutes to hour) * 4 (chips) * 0.09-0.4 (discount) = $0.11 - $0.48
- Once the TPU is set up and compilation/tracing is done, the same rollout
takes ~3 minutes = $0.07 - $0.29
- v5p: 8 sample, 30 step rollout of 0.25deg GenCast takes ~30 minutes,
including tracing and compilation, this will cost ~ 40 (minutes) * 5.20
($/chip hour) * 1/60 (minutes to hour) * 8 (chips) * 0.09-0.4 (discount) =
$2.50 - $11.10
- Once the TPU is set up and compilation/tracing is done, the same rollout
takes ~8 minutes = $0.50 - $2.22
## GenCast Memory Requirements
### Running inference on TPU
- This requires:
- 0.25deg GenCast: ~250GB of Host RAM (System Memory) and ~32GB of HBM (vRAM)
- 1deg GenCast: ~21GB of Host RAM (System Memory) and ~8GB of HBM (vRAM)
- Host RAM/System Memory is required for compilation. One way of accessing
sufficient host memory is to request a multi-device host, such as a 2x2x1
configuration of TPUs
- HBM/vRAM is required for running inference. Each TPU in a 2x2x1 TPUv4p+
configuration has enough HBM to run GenCast inference. As such, we make
efficient use of all devices and parallelise generating an ensemble across
them
- Note that compilation cost does not increase with the number of devices, it is
a fixed-cost one time operation whose result can be reused across any number
of devices
### Running Inference on GPU
- Inference can also be run on GPU, however, splash attention is currently not
supported for GPU in JAX
- Instead, inference must be run with a model whose `SparseTransformerConfig` has
```
attention_type = "triblockdiag_mha"
mask_type = "full"
```
set.
- To do this with one of the pre-trained checkpoints provided, load the model
and override the config
```
with ... as f:
ckpt = checkpoint.load(f, gencast.CheckPoint)
...
denoiser_architecture_config = ckpt.denoiser_architecture_config
denoiser_architecture_config.sparse_transformer_config.attention_type = "triblockdiag_mha"
denoiser_architecture_config.sparse_transformer_config.mask_type = "full
```
**Skill comparison vs. TPU**
- We tried running the model on a H100 using the `triblockdiag_mha` attention implementation and found that, while the performance is comparable, there is a small degradation (on average ~0.3% on unbiased Ensemble Mean RMSE and ~0.4% on unbiased CRPS). For reference, a scorecard comparing GenCast forecasts produced on a TPUv4 with `splash attention` vs. on a H100 with `triblockdiag_mha` can be found in [docs/](https://github.com/google-deepmind/graphcast/blob/main/docs/GenCast_0p25deg_accelerator_scorecard.png). Note that this scorecard differs from those found in the GenCast paper in a number of ways:
- 8 member ensembles (vs. 50 in the paper)
- 30 hour initialisation strides starting from 01-01-2019T00, i.e. a comparison
of 292 initialisations (vs. 730 in the paper)
- Colorbar limits of +-3% (vs. +-20% in the paper)
- There are two possible sources of this discrepancy. The first is the fact that the `splash` and `triblockdiag_mha` attention implementations are not exactly numerically equivalent (despite being algebraically equivalent). We have tested the isolated impact of these numerical differences by comparing performance with each attention implementation, both running on TPU. This comparison (scorecard [here](https://github.com/google-deepmind/graphcast/blob/main/docs/GenCast_0p25deg_attention_implementation_scorecard.png)) shows that there is very little difference caused by numerical differences between attention implementations. This implies that the minor degradation is caused primarily by running on GPU instead of TPU, and our initial investigations suggest that the root cause is the difference in the default precision of matmul operations on GPU compared to TPU.
**Memory requirement comparison vs. TPU**
- `triblockdiag_mha` also requires more memory, as such running inference on GPU
requires:
- 0.25deg GenCast: ~300GB of System Memory and ~60GB of vRAM
- 1deg GenCast: ~24GB of System Memory and ~16GB vRAM
**Inference time comparison vs. TPU**
- We have observed that running inference on H100 is slower than expected. Specifically we saw that a 30-step rollout of 0.25deg GenCast takes ~8min on TPUv5 with `splash_attention` (once compiled) whereas it takes ~25min on GPU with `triblockdiag_mha` attention.
- Part of this runtime discrepancy is caused by the fact that using `triblockdiag_mha` attention makes inference ~2x slower, such that running on TPU with `triblockdiag_mha` takes about ~15min, compared to the ~8min using `splash_attention`. However, there remains a discrepancy between the ~15min on a TPU and ~25min on GPU when using `triblockdiag_mha`.
## Prerequisites
### Create a Google Cloud account
- This can be done via https://cloud.google.com/
- Creating an account will prompt you to enter credit card information
- While this is necessary, please note that Google Cloud provides $300 worth
of free credits to start with, for 90 days. As per Cloud: "You won't be
auto-charged once your trial ends. **However, your resources are marked for
deletion and may be lost immediately**. Activate your full account anytime
during your trial to avoid losing your resources. After activating your full
account, you'll pay only for what you use."
- This will create a project by default
### Set up `gcloud` on your personal device
- Install `python` https://www.python.org/downloads
- Install `gcloud` CLI https://cloud.google.com/sdk/docs/install
and follow instructions to initialise
- Don't forget to restart your terminal once you have completed this to enable
`gcloud`
## Provisioning a Cloud VM TPU
- In what follows, set fields:
- Name: as preferred, we will assume they are the same and refer to them as
`` from here on
- Zone: at the time of writing, for each TPU type, availability can be found
in
- v4p: None ☹️
- v5e: `us-south1-a` or `asia-southeast1-b`
- v5p: `us-east5-a` or `europe-west4-b`
- Note that availability is variable and subject to change - we will refer to this
as `` from here on
- TPU type: based on the GenCast model being run, number of chips N desired and quota limits.
- The "GenCast Memory Requirements" section above details the accelerator recommendation for each model, where, in the TPU type dropdown:
- v5e → `v5litepod-N`
- v5p → `v5p-N`
- e.g.
- The number of chips `N` will correspond to the number of forecasts samples that can be generated in parallel
- Some zones have limits on number of chips that can be requested, e.g. at the time of writing `us-south1-a` limits to `v5litepod-4`
- Some TPU types have restrictions on the minimum number of chips that can be request, e.g. at the time of writing v5p appears to come in a minimum 8 chip configuration: `v5p-8`
- TPU software version: based on the TPU type
- v5e: `v2-tpuv5-litepod`
- v5p: `v2-alpha-tpuv5`
- Enabling "queueing" puts the creation request into a queue until resources become available. If you do not select this then if capacity is not instantaneously available at the time of the creation request, it will result in a `Stockout` error
- After creating the instance, you will find your request in the Console on "Compute Engine>TPUs>Queued Resources" tab
- Its status should change to "provisioning" and eventually "active"
**NOTE**: if you see a failed job in Queued Resources, you may have to delete it before launching again. This is because it is considered as quota and may cause you to hit the quota ceiling. Fortunately, this will not consume credits.
Summary:
| Model | Recommended Accelerator | Cells | Software Version | |
|:----------------------:|:-----------------------:|:---------------------------------:|:----------------:|:-:|
| GenCast 0p25deg (Oper) | v5p-N | us-east5-a or europe-west4-b | v2-alpha-tpuv5 | |
| GenCast 1deg (Mini) | v5litepod-N | us-south1-a or asia-southeast1-b | v2-tpuv5-litepod | |
**OPTION 1: Via the command line (required if requesting "Spot" TPUs)**:
- As per https://cloud.google.com/tpu/docs/spot#create-tpu-spot-vms
- If enabling queueing, set fields as above, and set
- `QUEUED_RESOURCE_ID` = ``
- `node_id` = ``
- `accelerator-type` = desired TPU type e.g. `v5litepod-4`
- `runtime-version` = desired TPU software version e.g. `v2-tpuv5-litepod`
- If not enabling queuing, set fields as above, and set
- `TPU_NAME` = ``
- `accelerator-type` = desired TPU type e.g. `v5litepod-4`
- `version` = desired TPU software version e.g. `v2-tpuv5-litepod`
- To request a spot device, append `--spot` to the command
- E.g.
```
gcloud compute tpus queued-resources create node-1 --node-id=node-1 --zone=us-south1-a --accelerator-type=v5litepod-4 --runtime-version=v2-tpuv5-litepod --spot
```
**OPTION 2: Via the Cloud Console**:
- In the Cloud console sidebar, navigate to "Create a VM>Compute Engine" and then to "TPUs" in the menu on the left
- On the first visit this will prompt you to "Enable" Compute Engine API"
- Click "Create TPU"
- Set fields as above
- N.B. The UI presents the option "Management>Preemptibility>Turn on pre-emptibility for this TPU node"
- This is actually a deprecated field and will result in a `Preemptibility is not available for the selected zone and type` error
- To request "Spot" TPUs see the command line method above
- Enable queuing by toggling the provided "Enable queuing" button
- E.g.
## Preparing an "active" Cloud VM TPU
- SSH into the TPU VM and port forward to your personal device
```
gcloud compute tpus tpu-vm ssh --zone --project -- -L 8081:localhost:8081
```
- Where `` can be found in the Cloud Console, e.g.
- Transfer relevant files onto the VM (this is required because as per https://github.com/googlecolab/colabtools/issues/2533, mounting Google Drive or Cloud Buckets is not possible in local runtimes), e.g. to run a 30 step rollout of the 1 degree model
```
gcloud storage cp gs://dm_graphcast/gencast/dataset/source-era5_date-2019-03-29_res-1.0_levels-13_steps-30.nc .
gcloud storage cp "gs://dm_graphcast/gencast/params/GenCast 1p0deg <2019.npz" .
gcloud storage cp --recursive gs://dm_graphcast/gencast/stats/ .
```
- Install
- JAX, to ensure the proper runtime libraries are installed to access the TPUs
```
pip install -U "jax[tpu]" -f https://storage.googleapis.com/jax-releases/libtpu_releases.html
```
- Jupyter, to launch a local runtime server
```
pip install jupyter
```
### (Optional) Set up a bucket to store predictions
- Now that the zone of the TPU is known, create a bucket, where
- `` is as preferred
- `` is the zone used for the TPU, using the names listed as per https://cloud.google.com/storage/docs/locations#available-locations,
- E,g, `us-south1-a` → `US-SOUTH-1`
```
gcloud storage buckets create gs:// --location
```
- Create a Cloud TPU service account
```
gcloud beta services identity create --service tpu.googleapis.com --project
```
- The above command returns a Cloud TPU Service Account with following format (we refer to this as ``):
```
service-@cloud-tpu.iam.gserviceaccount.com
```
- Grant writing permissions
```
gcloud storage buckets add-iam-policy-binding gs:// --member=serviceAccount: --role=roles/storage.objectCreator
```
- You can view the bucket in the Console UI by visiting Storage>Buckets in the sidebar
## Connecting Notebook to a prepared Cloud VM TPU
- While still SSHd to the TPU VM, start the Jupyter server
```
python3 -m notebook --NotebookApp.allow_origin='https://colab.research.google.com' --port=8081 --NotebookApp.port_retries=0 --no-browser
```
- This will generate a http://localhost… URL, e.g. http://localhost:8081/tree?token=bdb95985daa075099fa8103688120cca0a477326df24164f
- Connect to the local runtime in gencast_demo_cloud_vm.ipynb with this URL
- Happy forecasting! (Don't forget to delete the TPU VM once done)
## (Optional) Transferring data off of the TPU
- Run the "Save the predictions" cell in the notebook to save the predictions to the TPU filesystem
```
# @title (Optional) Save the predictions.
predictions.to_zarr("predictions.zarr")
```
- In the command line copy these predictions to the bucket created as per the instructions above
```
gcloud storage cp --recursive predictions.zarr gs:///
```
---
### Weathernext1 Graph/README (docs/weathernext1_graph/README.md)
## WeatherNext 1 Graph (GraphCast)
This model was published as [GraphCast: Learning skillful medium-range global
weather forecasting](https://www.science.org/doi/10.1126/science.adi2336). File
and checkpoint names retain the original "GraphCast" naming for backward
compatibility.
This package provides three pretrained models:
1. `GraphCast`, the high-resolution model used in the GraphCast paper (0.25
degree resolution, 37 pressure levels), trained on ERA5 data from 1979 to
2017,
2. `GraphCast_small`, a smaller, low-resolution version of GraphCast (1 degree
resolution, 13 pressure levels, and a smaller mesh), trained on ERA5 data
from 1979 to 2015, useful to run a model with lower memory and compute
constraints,
3. `GraphCast_operational`, a high-resolution model (0.25 degree resolution, 13
pressure levels) pre-trained on ERA5 data from 1979 to 2017 and fine-tuned
on HRES data from 2016 to 2021. This model can be initialized from HRES data
(does not require precipitation inputs).
The best starting point is to open `graphcast_demo.ipynb` in
[Colaboratory](https://colab.research.google.com/github/google-deepmind/weathernext/blob/master/docs/weathernext1_graph/graphcast_demo.ipynb),
which gives an example of loading data, generating random weights or load a
pre-trained snapshot, generating predictions, computing the loss and computing
gradients. The one-step implementation of GraphCast architecture, is provided in
`graphcast.py` and the relevant data, weights and statistics are in the
`graphcast/` subdir of the Google Cloud Bucket.
WARNING: For backwards compatibility, we have also left GraphCast data in the
top level of the bucket. These will eventually be deleted in favour of the
`graphcast/` subdir.
### Brief description of relevant library files:
In addition to the model-specific files below, shared library files
(`autoregressive.py`, `normalization.py`, `rollout.py`, etc.) are in the
`utils/` directory.
* `casting.py`: Wrapper used around GraphCast to make it work using BFloat16
precision.
* `graphcast.py`: The main GraphCast model architecture for one-step of
predictions.
* `solar_radiation.py`: Computes Top-Of-the-Atmosphere (TOA) incident solar
radiation compatible with ERA5. This is used as a forcing variable and thus
needs to be computed for target lead times in an operational setting.
## License and Disclaimers
Copyright 2024-2026 Google LLC.
The Colab notebooks and the associated code are licensed under the Apache
License, Version 2.0 (Apache 2.0); you may not use these materials except in
compliance with the Apache 2.0 license. You may obtain a copy of the License at:
[https://www.apache.org/licenses/LICENSE-2.0](https://www.apache.org/licenses/LICENSE-2.0).
All other materials are licensed under the Creative Commons Attribution 4.0
International (CC BY 4.0). You may obtain a copy of the License at:
[https://creativecommons.org/licenses/by/4.0/](https://creativecommons.org/licenses/by/4.0/).
***August 6, 2026 UPDATE:** The license for the model weights in this repository
has been updated to permit commercial use. The previous license is now replaced
by the new terms outlined in this README By continuing to access, download, or
use the weights (including those previously sourced from this repository), you
agree to the updated terms.*
This is not an officially supported Google product.
Unless required by applicable law or agreed to in writing, all software and
materials distributed here under the Apache 2.0 or CC-BY 4.0 licenses are
distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
either express or implied. See the licenses for the specific language governing
permissions and limitations under those licenses.
GraphCast is part of an experimental research project. You are solely
responsible for determining the appropriateness of using or distributing
GraphCast or any outputs generated and assume all risks associated with your use
or distribution of GraphCast and outputs and your exercise of rights and
permissions granted by Google to you under the relevant License. Use discretion
before relying on, publishing, downloading or otherwise using GraphCast or any
outputs generated. \[GraphCast outputs have not been produced in collaboration
with nor endorsed by any government meteorological agency or department, and in
no way replaces official alerts, warnings or notices published by such
agencies.\]
## Citations
If you use this work, consider citing our papers ([blog
post](https://deepmind.google/discover/blog/graphcast-ai-model-for-faster-and-more-accurate-global-weather-forecasting/),
[Science](https://www.science.org/doi/10.1126/science.adi2336)):
```latex
@article{lam2023learning,
title={Learning skillful medium-range global weather forecasting},
author={Lam, Remi and Sanchez-Gonzalez, Alvaro and Willson, Matthew and Wirnsberger, Peter and Fortunato, Meire and Alet, Ferran and Ravuri, Suman and Ewalds, Timo and Eaton-Rosen, Zach and Hu, Weihua and others},
journal={Science},
volume={382},
number={6677},
pages={1416--1421},
year={2023},
publisher={American Association for the Advancement of Science}
}
```
## Acknowledgements
GraphCast communicates with the following separate libraries and packages:.
* Data and products of the European Centre for Medium-range Weather Forecasts
(ECMWF), as modified by Google.
* Modified Copernicus Climate Change Service information 2023\.
Additionally, the colab notebooks include a few examples of ECMWF’s ERA5 and
HRES data that can be used as input to the models.
Neither the European Commission nor ECMWF is responsible for any use that may be
made of the Copernicus information or data it contains. ECMWF HRES datasets
Copyright statement: Copyright "© 2023 European Centre for Medium-Range Weather
Forecasts (ECMWF)". Source: [www.ecmwf.int](http://www.ecmwf.int/) License
Statement: ECMWF open data is published under a Creative Commons Attribution 4.0
International (CC BY 4.0).
[https://creativecommons.org/licenses/by/4.0/](https://creativecommons.org/licenses/by/4.0/)
Disclaimer: ECMWF does not accept any liability whatsoever for any error or
omission in the data, their availability, or for any loss or damage arising from
their use.
Use of the third-party materials referred to above may be governed by separate
terms and conditions or license provisions. Your use of the third-party
materials is subject to any such terms and you should check that you can comply
with any applicable restrictions or terms and conditions before use.
---