pytorch_geometric Documentation

File: README.md Copy Chapter

PyG (PyTorch Geometric) is a library built upon PyTorch to easily write and train Graph Neural Networks (GNNs) for a wide range of applications related to structured data.

It consists of various methods for deep learning on graphs and other irregular structures, also known as geometric deep learning, from a variety of published papers.
In addition, it consists of easy-to-use mini-batch loaders for operating on many small and single giant graphs, multi GPU-support, torch.compile support, DataPipe support, a large number of common benchmark datasets (based on simple interfaces to create your own), and helpful transforms, both for learning on arbitrary graphs as well as on 3D meshes or point clouds.

Click here to join our Slack community!

• Library Highlights
• Quick Tour for New Users
• Architecture Overview
• Implemented GNN Models
• Installation

Library Highlights Copy Chapter

Whether you are a machine learning researcher or first-time user of machine learning toolkits, here are some reasons to try out PyG for machine learning on graph-structured data.

• Easy-to-use and unified API:
  All it takes is 10-20 lines of code to get started with training a GNN model (see the next section for a quick tour).
  PyG is PyTorch-on-the-rocks: It utilizes a tensor-centric API and keeps design principles close to vanilla PyTorch.
  If you are already familiar with PyTorch, utilizing PyG is straightforward.
• Comprehensive and well-maintained GNN models:
  Most of the state-of-the-art Graph Neural Network architectures have been implemented by library developers or authors of research papers and are ready to be applied.
• Great flexibility:
  Existing PyG models can easily be extended for conducting your own research with GNNs.
  Making modifications to existing models or creating new architectures is simple, thanks to its easy-to-use message passing API, and a variety of operators and utility functions.
• Large-scale real-world GNN models:
  We focus on the need of GNN applications in challenging real-world scenarios, and support learning on diverse types of graphs, including but not limited to: scalable GNNs for graphs with millions of nodes; dynamic GNNs for node predictions over time; heterogeneous GNNs with multiple node types and edge types.

Quick Tour for New Users Copy Chapter

In this quick tour, we highlight the ease of creating and training a GNN model with only a few lines of code.

Train your own GNN model Copy Section

In the first glimpse of PyG, we implement the training of a GNN for classifying papers in a citation graph.
For this, we load the Cora dataset, and create a simple 2-layer GCN model using the pre-defined GCNConv:

import torch
from torch import Tensor
from torch_geometric.nn import GCNConv
from torch_geometric.datasets import Planetoid

dataset = Planetoid(root='.', name='Cora')

class GCN(torch.nn.Module):
    def __init__(self, in_channels, hidden_channels, out_channels):
        super().__init__()
        self.conv1 = GCNConv(in_channels, hidden_channels)
        self.conv2 = GCNConv(hidden_channels, out_channels)

    def forward(self, x: Tensor, edge_index: Tensor) -> Tensor:
        # x: Node feature matrix of shape [num_nodes, in_channels]
        # edge_index: Graph connectivity matrix of shape [2, num_edges]
        x = self.conv1(x, edge_index).relu()
        x = self.conv2(x, edge_index)
        return x

model = GCN(dataset.num_features, 16, dataset.num_classes)

We can now optimize the model in a training loop, similar to the standard PyTorch training procedure.

import torch.nn.functional as F

data = dataset[0]
optimizer = torch.optim.Adam(model.parameters(), lr=0.01)

for epoch in range(200):
    pred = model(data.x, data.edge_index)
    loss = F.cross_entropy(pred[data.train_mask], data.y[data.train_mask])

    # Backpropagation
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

More information about evaluating final model performance can be found in the corresponding example.

Create your own GNN layer Copy Section

In addition to the easy application of existing GNNs, PyG makes it simple to implement custom Graph Neural Networks (see here for the accompanying tutorial).
For example, this is all it takes to implement the edge convolutional layer from Wang et al.:

$$x_i^{\prime} ~ = ~ \max_{j \in \mathcal{N}(i)} ~ \textrm{MLP}_{\theta} \left( [ ~ x_i, ~ x_j - x_i ~ ] \right)$$

import torch
from torch import Tensor
from torch.nn import Sequential, Linear, ReLU
from torch_geometric.nn import MessagePassing

class EdgeConv(MessagePassing):
    def __init__(self, in_channels, out_channels):
        super().__init__(aggr="max")  # "Max" aggregation.
        self.mlp = Sequential(
            Linear(2 * in_channels, out_channels),
            ReLU(),
            Linear(out_channels, out_channels),
        )

    def forward(self, x: Tensor, edge_index: Tensor) -> Tensor:
        # x: Node feature matrix of shape [num_nodes, in_channels]
        # edge_index: Graph connectivity matrix of shape [2, num_edges]
        return self.propagate(edge_index, x=x)  # shape [num_nodes, out_channels]

    def message(self, x_j: Tensor, x_i: Tensor) -> Tensor:
        # x_j: Source node features of shape [num_edges, in_channels]
        # x_i: Target node features of shape [num_edges, in_channels]
        edge_features = torch.cat([x_i, x_j - x_i], dim=-1)
        return self.mlp(edge_features)  # shape [num_edges, out_channels]

Architecture Overview Copy Chapter

PyG provides a multi-layer framework that enables users to build Graph Neural Network solutions on both low and high levels.
It comprises of the following components:

• The PyG engine utilizes the powerful PyTorch deep learning framework with full torch.compile and TorchScript support, as well as additions of efficient CPU/CUDA libraries for operating on sparse data, e.g., pyg-lib.
• The PyG storage handles data processing, transformation and loading pipelines. It is capable of handling and processing large-scale graph datasets, and provides effective solutions for heterogeneous graphs. It further provides a variety of sampling solutions, which enable training of GNNs on large-scale graphs.
• The PyG operators bundle essential functionalities for implementing Graph Neural Networks. PyG supports important GNN building blocks that can be combined and applied to various parts of a GNN model, ensuring rich flexibility of GNN design.
• Finally, PyG provides an abundant set of GNN models, and examples that showcase GNN models on standard graph benchmarks. Thanks to its flexibility, users can easily build and modify custom GNN models to fit their specific needs.

Implemented GNN Models Copy Chapter

We list currently supported PyG models, layers and operators according to category:

GNN layers:
All Graph Neural Network layers are implemented via the nn.MessagePassing interface.
A GNN layer specifies how to perform message passing, i.e. by designing different message, aggregation and update functions as defined here.
These GNN layers can be stacked together to create Graph Neural Network models.

• GCNConv from Kipf and Welling: Semi-Supervised Classification with Graph Convolutional Networks (ICLR 2017) [Example]
• ChebConv from Defferrard et al.: Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering (NIPS 2016) [Example]
• GATConv from Veličković et al.: Graph Attention Networks (ICLR 2018) [Example]

Expand to see all implemented GNN layers...

• GCN2Conv from Chen et al.: Simple and Deep Graph Convolutional Networks (ICML 2020) [Example1, Example2]
• SplineConv from Fey et al.: SplineCNN: Fast Geometric Deep Learning with Continuous B-Spline Kernels (CVPR 2018) [Example1, Example2]
• NNConv from Gilmer et al.: Neural Message Passing for Quantum Chemistry (ICML 2017) [Example1, Example2]
• CGConv from Xie and Grossman: Crystal Graph Convolutional Neural Networks for an Accurate and Interpretable Prediction of Material Properties (Physical Review Letters 120, 2018)
• ECConv from Simonovsky and Komodakis: Edge-Conditioned Convolution on Graphs (CVPR 2017)
• EGConv from Tailor et al.: Adaptive Filters and Aggregator Fusion for Efficient Graph Convolutions (GNNSys 2021) [Example]
• GATv2Conv from Brody et al.: How Attentive are Graph Attention Networks? (ICLR 2022)
• TransformerConv from Shi et al.: Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification (CoRR 2020) [Example]
• SAGEConv from Hamilton et al.: Inductive Representation Learning on Large Graphs (NIPS 2017) [Example1, Example2, Example3, Example4]
• GraphConv from, e.g., Morris et al.: Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks (AAAI 2019)
• GatedGraphConv from Li et al.: Gated Graph Sequence Neural Networks (ICLR 2016)
• ResGatedGraphConv from Bresson and Laurent: Residual Gated Graph ConvNets (CoRR 2017)
• GINConv from Xu et al.: How Powerful are Graph Neural Networks? (ICLR 2019) [Example]
• GINEConv from Hu et al.: Strategies for Pre-training Graph Neural Networks (ICLR 2020)
• ARMAConv from Bianchi et al.: Graph Neural Networks with Convolutional ARMA Filters (CoRR 2019) [Example]
• SGConv from Wu et al.: Simplifying Graph Convolutional Networks (CoRR 2019) [Example]
• APPNP from Klicpera et al.: Predict then Propagate: Graph Neural Networks meet Personalized PageRank (ICLR 2019) [Example]
• MFConv from Duvenaud et al.: Convolutional Networks on Graphs for Learning Molecular Fingerprints (NIPS 2015)
• AGNNConv from Thekumparampil et al.: Attention-based Graph Neural Network for Semi-Supervised Learning (CoRR 2017) [Example]
• TAGConv from Du et al.: Topology Adaptive Graph Convolutional Networks (CoRR 2017) [Example]
• PNAConv from Corso et al.: Principal Neighbourhood Aggregation for Graph Nets (CoRR 2020) [Example]
• FAConv from Bo et al.: Beyond Low-Frequency Information in Graph Convolutional Networks (AAAI 2021)
• PDNConv from Rozemberczki et al.: Pathfinder Discovery Networks for Neural Message Passing (WWW 2021)
• RGCNConv from Schlichtkrull et al.: Modeling Relational Data with Graph Convolutional Networks (ESWC 2018) [Example1, Example2]
• RGATConv from Busbridge et al.: Relational Graph Attention Networks (CoRR 2019) [Example]
• FiLMConv from Brockschmidt: GNN-FiLM: Graph Neural Networks with Feature-wise Linear Modulation (ICML 2020) [Example]
• SignedConv from Derr et al.: Signed Graph Convolutional Network (ICDM 2018) [Example]
• DNAConv from Fey: Just Jump: Dynamic Neighborhood Aggregation in Graph Neural Networks (ICLR-W 2019) [Example]
• PANConv from Ma et al.: Path Integral Based Convolution and Pooling for Graph Neural Networks (NeurIPS 2020)
• PointNetConv (including Iterative Farthest Point Sampling, dynamic graph generation based on nearest neighbor or maximum distance, and k-NN interpolation for upsampling) from Qi et al.: PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation (CVPR 2017) and PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space (NIPS 2017) [Example1, Example2]
• EdgeConv from Wang et al.: Dynamic Graph CNN for Learning on Point Clouds (CoRR, 2018) [Example1, Example2]
• XConv from Li et al.: PointCNN: Convolution On X-Transformed Points (NeurIPS 2018) [Example]
• PPFConv from Deng et al.: PPFNet: Global Context Aware Local Features for Robust 3D Point Matching (CVPR 2018)
• GMMConv from Monti et al.: Geometric Deep Learning on Graphs and Manifolds using Mixture Model CNNs (CVPR 2017)
• FeaStConv from Verma et al.: FeaStNet: Feature-Steered Graph Convolutions for 3D Shape Analysis (CVPR 2018)
• PointTransformerConv from Zhao et al.: Point Transformer (2020)
• HypergraphConv from Bai et al.: Hypergraph Convolution and Hypergraph Attention (CoRR 2019)
• GravNetConv from Qasim et al.: Learning Representations of Irregular Particle-detector Geometry with Distance-weighted Graph Networks (European Physics Journal C, 2019)
• SuperGAT from Kim and Oh: How To Find Your Friendly Neighborhood: Graph Attention Design With Self-Supervision (ICLR 2021) [Example]
• HGTConv from Hu et al.: Heterogeneous Graph Transformer (WWW 2020) [Example]
• HEATConv from Mo et al.: Heterogeneous Edge-Enhanced Graph Attention Network For Multi-Agent Trajectory Prediction (CoRR 2021)
• SSGConv from Zhu et al.: Simple Spectral Graph Convolution (ICLR 2021)
• FusedGATConv from Zhang et al.: Understanding GNN Computational Graph: A Coordinated Computation, IO, and Memory Perspective (MLSys 2022)
• GPSConv from Rampášek et al.: Recipe for a General, Powerful, Scalable Graph Transformer (NeurIPS 2022) [Example]

Pooling layers:
Graph pooling layers combine the vectorial representations of a set of nodes in a graph (or a subgraph) into a single vector representation that summarizes its properties of nodes.
It is commonly applied to graph-level tasks, which require combining node features into a single graph representation.

• Top-K Pooling from Gao and Ji: Graph U-Nets (ICML 2019), Cangea et al.: Towards Sparse Hierarchical Graph Classifiers (NeurIPS-W 2018) and Knyazev et al.: Understanding Attention and Generalization in Graph Neural Networks (ICLR-W 2019) [Example]
• DiffPool from Ying et al.: Hierarchical Graph Representation Learning with Differentiable Pooling (NeurIPS 2018) [Example]

Expand to see all implemented pooling layers...

• Attentional Aggregation from Li et al.: Graph Matching Networks for Learning the Similarity of Graph Structured Objects (ICML 2019) [Example]
• Set2Set from Vinyals et al.: Order Matters: Sequence to Sequence for Sets (ICLR 2016) [Example]
• Sort Aggregation from Zhang et al.: An End-to-End Deep Learning Architecture for Graph Classification (AAAI 2018) [Example]
• MinCut Pooling from Bianchi et al.: Spectral Clustering with Graph Neural Networks for Graph Pooling (ICML 2020) [Example]
• DMoN Pooling from Tsitsulin et al.: Graph Clustering with Graph Neural Networks (CoRR 2020) [Example]
• Graclus Pooling from Dhillon et al.: Weighted Graph Cuts without Eigenvectors: A Multilevel Approach (PAMI 2007) [Example]
• Voxel Grid Pooling from, e.g., Simonovsky and Komodakis: Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs (CVPR 2017) [Example]
• SAG Pooling from Lee et al.: Self-Attention Graph Pooling (ICML 2019) and Knyazev et al.: Understanding Attention and Generalization in Graph Neural Networks (ICLR-W 2019) [Example]
• Edge Pooling from Diehl et al.: Towards Graph Pooling by Edge Contraction (ICML-W 2019) and Diehl: Edge Contraction Pooling for Graph Neural Networks (CoRR 2019) [Example]
• ASAPooling from Ranjan et al.: ASAP: Adaptive Structure Aware Pooling for Learning Hierarchical Graph Representations (AAAI 2020) [Example]
• PANPooling from Ma et al.: Path Integral Based Convolution and Pooling for Graph Neural Networks (NeurIPS 2020)
• MemPooling from Khasahmadi et al.: Memory-Based Graph Networks (ICLR 2020) [Example]
• Graph Multiset Transformer from Baek et al.: Accurate Learning of Graph Representations with Graph Multiset Pooling (ICLR 2021) [Example]
• Equilibrium Aggregation from Bartunov et al.: (UAI 2022) [Example]

GNN models:
Our supported GNN models incorporate multiple message passing layers, and users can directly use these pre-defined models to make predictions on graphs.
Unlike simple stacking of GNN layers, these models could involve pre-processing, additional learnable parameters, skip connections, graph coarsening, etc.

• SchNet from Schütt et al.: SchNet: A Continuous-filter Convolutional Neural Network for Modeling Quantum Interactions (NIPS 2017) [Example]
• DimeNet and DimeNetPlusPlus from Klicpera et al.: Directional Message Passing for Molecular Graphs (ICLR 2020) and Fast and Uncertainty-Aware Directional Message Passing for Non-Equilibrium Molecules (NeurIPS-W 2020) [Example]
• Node2Vec from Grover and Leskovec: node2vec: Scalable Feature Learning for Networks (KDD 2016) [Example]
• Deep Graph Infomax from Veličković et al.: Deep Graph Infomax (ICLR 2019) [Example1, Example2]
• Deep Multiplex Graph Infomax from Park et al.: Unsupervised Attributed Multiplex Network Embedding (AAAI 2020) [Example]
• Masked Label Prediction from Shi et al.: Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification (CoRR 2020) [Example]
• PMLP from Yang et al.: Graph Neural Networks are Inherently Good Generalizers: Insights by Bridging GNNs and MLPs (ICLR 2023)

Expand to see all implemented GNN models...

• Jumping Knowledge from Xu et al.: Representation Learning on Graphs with Jumping Knowledge Networks (ICML 2018) [Example]
• A MetaLayer for building any kind of graph network similar to the TensorFlow Graph Nets library from Battaglia et al.: Relational Inductive Biases, Deep Learning, and Graph Networks (CoRR 2018)
• MetaPath2Vec from Dong et al.: metapath2vec: Scalable Representation Learning for Heterogeneous Networks (KDD 2017) [Example]
• All variants of Graph Autoencoders and Variational Autoencoders from:
  • Variational Graph Auto-Encoders from Kipf and Welling (NIPS-W 2016) [Example]
  • Adversarially Regularized Graph Autoencoder for Graph Embedding from Pan et al. (IJCAI 2018) [Example]
  • Simple and Effective Graph Autoencoders with One-Hop Linear Models from Salha et al. (ECML 2020) [Example]
• SEAL from Zhang and Chen: Link Prediction Based on Graph Neural Networks (NeurIPS 2018) [Example]
• RENet from Jin et al.: Recurrent Event Network for Reasoning over Temporal Knowledge Graphs (ICLR-W 2019) [Example]
• GraphUNet from Gao and Ji: Graph U-Nets (ICML 2019) [Example]
• AttentiveFP from Xiong et al.: Pushing the Boundaries of Molecular Representation for Drug Discovery with the Graph Attention Mechanism (J. Med. Chem. 2020) [Example]
• DeepGCN and the GENConv from Li et al.: DeepGCNs: Can GCNs Go as Deep as CNNs? (ICCV 2019) and DeeperGCN: All You Need to Train Deeper GCNs (CoRR 2020) [Example]
• RECT from Wang et al.: Network Embedding with Completely-imbalanced Labels (TKDE 2020) [Example]
• GNNExplainer from Ying et al.: GNNExplainer: Generating Explanations for Graph Neural Networks (NeurIPS 2019) [Example1, Example2, Example3]
• Graph-less Neural Networks from Zhang et al.: Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillation (CoRR 2021) [Example]
• LINKX from Lim et al.: Large Scale Learning on Non-Homophilous Graphs:
  New Benchmarks and Strong Simple Methods (NeurIPS 2021) [Example]
• RevGNN from Li et al.: Training Graph Neural with 1000 Layers (ICML 2021) [Example]
• TransE from Bordes et al.: Translating Embeddings for Modeling Multi-Relational Data (NIPS 2013) [Example]
• ComplEx from Trouillon et al.: Complex Embeddings for Simple Link Prediction (ICML 2016) [Example]
• DistMult from Yang et al.: Embedding Entities and Relations for Learning and Inference in Knowledge Bases (ICLR 2015) [Example]
• RotatE from Sun et al.: RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space (ICLR 2019) [Example]

GNN operators and utilities:
PyG comes with a rich set of neural network operators that are commonly used in many GNN models.
They follow an extensible design: It is easy to apply these operators and graph utilities to existing GNN layers and models to further enhance model performance.

• DropEdge from Rong et al.: DropEdge: Towards Deep Graph Convolutional Networks on Node Classification (ICLR 2020)
• DropNode, MaskFeature and AddRandomEdge from You et al.: Graph Contrastive Learning with Augmentations (NeurIPS 2020)
• DropPath from Li et al.: MaskGAE: Masked Graph Modeling Meets Graph Autoencoders (arXiv 2022)
• ShuffleNode from Veličković et al.: Deep Graph Infomax (ICLR 2019)
• GraphNorm from Cai et al.: GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training (ICML 2021)
• GDC from Klicpera et al.: Diffusion Improves Graph Learning (NeurIPS 2019) [Example]

Expand to see all implemented GNN operators and utilities...

• GraphSizeNorm from Dwivedi et al.: Benchmarking Graph Neural Networks (CoRR 2020)
• PairNorm from Zhao and Akoglu: PairNorm: Tackling Oversmoothing in GNNs (ICLR 2020)
• MeanSubtractionNorm from Yang et al.: Revisiting "Over-smoothing" in Deep GCNs (CoRR 2020)
• DiffGroupNorm from Zhou et al.: Towards Deeper Graph Neural Networks with Differentiable Group Normalization (NeurIPS 2020)
• Tree Decomposition from Jin et al.: Junction Tree Variational Autoencoder for Molecular Graph Generation (ICML 2018)
• TGN from Rossi et al.: Temporal Graph Networks for Deep Learning on Dynamic Graphs (GRL+ 2020) [Example]
• Weisfeiler Lehman Operator from Weisfeiler and Lehman: A Reduction of a Graph to a Canonical Form and an Algebra Arising During this Reduction (Nauchno-Technicheskaya Informatsia 1968) [Example]
• Continuous Weisfeiler Lehman Operator from Togninalli et al.: Wasserstein Weisfeiler-Lehman Graph Kernels (NeurIPS 2019)
• Label Propagation from Zhu and Ghahramani: Learning from Labeled and Unlabeled Data with Label Propagation (CMU-CALD 2002) [Example]
• Local Degree Profile from Cai and Wang: A Simple yet Effective Baseline for Non-attribute Graph Classification (CoRR 2018)
• CorrectAndSmooth from Huang et al.: Combining Label Propagation And Simple Models Out-performs Graph Neural Networks (CoRR 2020) [Example]
• Gini and BRO regularization from Henderson et al.: Improving Molecular Graph Neural Network Explainability with Orthonormalization and Induced Sparsity (ICML 2021)
• RootedEgoNets and RootedRWSubgraph from Zhao et al.: From Stars to Subgraphs: Uplifting Any GNN with Local Structure Awareness (ICLR 2022)
• FeaturePropagation from Rossi et al.: On the Unreasonable Effectiveness of Feature Propagation in Learning on Graphs with Missing Node Features (CoRR 2021)

Scalable GNNs:
PyG supports the implementation of Graph Neural Networks that can scale to large-scale graphs.
Such application is challenging since the entire graph, its associated features and the GNN parameters cannot fit into GPU memory.
Many state-of-the-art scalability approaches tackle this challenge by sampling neighborhoods for mini-batch training, graph clustering and partitioning, or by using simplified GNN models.
These approaches have been implemented in PyG, and can benefit from the above GNN layers, operators and models.

• NeighborLoader from Hamilton et al.: Inductive Representation Learning on Large Graphs (NIPS 2017) [Example1, Example2, Example3]
• ClusterGCN from Chiang et al.: Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks (KDD 2019) [Example1, Example2]
• GraphSAINT from Zeng et al.: GraphSAINT: Graph Sampling Based Inductive Learning Method (ICLR 2020) [Example]

Expand to see all implemented scalable GNNs...

• ShaDow from Zeng et al.: Decoupling the Depth and Scope of Graph Neural Networks (NeurIPS 2021) [Example]
• SIGN from Rossi et al.: SIGN: Scalable Inception Graph Neural Networks (CoRR 2020) [Example]
• HGTLoader from Hu et al.: Heterogeneous Graph Transformer (WWW 2020) [Example]

Installation Copy Chapter

PyG is available for Python 3.10 to Python 3.14.

From PyG 2.3 onwards, you can install and use PyG without any external library required except for PyTorch.
For this, simply run

pip install torch_geometric

Additional Libraries Copy Section

If you want to utilize the full set of features from PyG, there exists several additional libraries you may want to install:

• pyg-lib: Heterogeneous GNN operators, graph sampling routines, graph clustering routines, and SplineConv support
• torch-scatter: Accelerated and efficient sparse reductions
• torch-sparse: SparseTensor support

These packages come with their own CPU and GPU kernel implementations based on the PyTorch C++/CUDA/hip(ROCm) extension interface.
For a basic usage of PyG, these dependencies are fully optional.
We recommend to start with a minimal installation, and install additional dependencies once you start to actually need them.

For ease of installation of these extensions, we provide pip wheels for all major OS/PyTorch/CUDA combinations, see here.

PyTorch 2.12

To install the binaries for PyTorch 2.12, simply run

pip install pyg_lib torch_scatter torch_sparse -f https://data.pyg.org/whl/torch-2.12.0+${CUDA}.html

where ${CUDA} should be replaced by either cpu, cu126, cu130, or cu132 depending on your PyTorch installation.

PyTorch 2.11

To install the binaries for PyTorch 2.11, simply run

pip install pyg_lib torch_scatter torch_sparse -f https://data.pyg.org/whl/torch-2.11.0+${CUDA}.html

where ${CUDA} should be replaced by either cpu, cu126, cu128, or cu130 depending on your PyTorch installation.

PyTorch 2.10

To install the binaries for PyTorch 2.10, simply run

pip install pyg_lib torch_scatter torch_sparse -f https://data.pyg.org/whl/torch-2.10.0+${CUDA}.html

where ${CUDA} should be replaced by either cpu, cu126, cu128, or cu130 depending on your PyTorch installation.

PyTorch 2.9

To install the binaries for PyTorch 2.9, simply run

pip install pyg_lib torch_scatter torch_sparse -f https://data.pyg.org/whl/torch-2.9.0+${CUDA}.html

where ${CUDA} should be replaced by either cpu, cu126, cu128, or cu130 depending on your PyTorch installation.

Note: Binaries of older versions are also provided for PyTorch 1.4.0, PyTorch 1.5.0, PyTorch 1.6.0, PyTorch 1.7.0/1.7.1, PyTorch 1.8.0/1.8.1, PyTorch 1.9.0, PyTorch 1.10.0/1.10.1/1.10.2, PyTorch 1.11.0, PyTorch 1.12.0/1.12.1, PyTorch 1.13.0/1.13.1, PyTorch 2.0.0/2.0.1, PyTorch 2.1.0/2.1.1/2.1.2, PyTorch 2.2.0/2.2.1/2.2.2, PyTorch 2.3.0/2.3.1, PyTorch 2.4.0/2.4.1, PyTorch 2.5.0/2.5.1, PyTorch 2.6.0, PyTorch 2.7.0/2.7.1, and PyTorch 2.8.0 (following the same procedure).
For older versions, you might need to explicitly specify the latest supported version number or install via pip install --no-index in order to prevent a manual installation from source.
You can look up the latest supported version number here.

NVIDIA PyG Container Copy Section

NVIDIA provides a PyG docker container for effortlessly training and deploying GPU accelerated GNNs with PyG, see here.

Nightly and Master Copy Section

In case you want to experiment with the latest PyG features which are not fully released yet, either install the nightly version of PyG via

pip install pyg-nightly

or install PyG from master via

pip install git+https://github.com/pyg-team/pytorch_geometric.git

ROCm Wheels Copy Section

The external pyg-rocm-build repository provides wheels and detailed instructions on how to install PyG for ROCm.
If you have any questions about it, please open an issue here.

Cite Copy Chapter

Please cite our PyG 1.0 and PyG 2.0 papers if you use this code in your own work:

@inproceedings{Fey/Lenssen/2019,
  title={Fast Graph Representation Learning with {PyTorch Geometric}},
  author={Fey, Matthias and Lenssen, Jan E.},
  booktitle={ICLR Workshop on Representation Learning on Graphs and Manifolds},
  year={2019},
}

@inproceedings{Fey/etal/2025,
  title={{PyG} 2.0: Scalable Learning on Real World Graphs},
  author={Fey, Matthias and Sunil, Jinu and Nitta, Akihiro and Puri, Rishi and Shah, Manan, and Stojanovi{\v{c}, Bla{\v{z} and Bendias, Ramona and Alexandria, Barghi and Kocijan, Vid and Zhang, Zecheng and He, Xinwei and Lenssen, Jan E. and Leskovec, Jure},
  booktitle={Temporal Graph Learning Workshop @ KDD},
  year={2025},
}

Feel free to email us if you wish your work to be listed in the external resources.
If you notice anything unexpected, please open an issue and let us know.
If you have any questions or are missing a specific feature, feel free to discuss them with us.
We are motivated to constantly make PyG even better.

File: benchmark/points/README.md Copy Chapter

Point Cloud classification

Evaluation scripts for various methods on the ModelNet10 dataset:

• MPNN: python mpnn.py
• PointNet++: python point_net.py
• EdgeCNN: python edge_cnn.py
• SplineCNN: python spline_cnn.py
• PointCNN: python point_cnn.py

File: benchmark/multi_gpu/training/README.md Copy Chapter

Training Benchmark

Running benchmark on CUDA GPU Copy Chapter

Run benchmark, e.g. assuming you have n NVIDIA GPUs:

python training_benchmark_cuda.py --dataset ogbn-products --model edge_cnn --num-epochs 3 --n_gpus <n>

Running benchmark on Intel GPU Copy Chapter

Environment setup Copy Section

Prerequisites Copy Section

• Intel Data Center GPU Max Series. You could try it through Intel DevCloud.
• Verify the Intel GPU Driver is installed, refer to the guide.

docker setup Copy Section

If you want to run your scripts inside a docker image, you could refer to the dockerfile and the corresponding guide.

bare-metal setup Copy Section

If you prefer to run your scripts directly on the bare-metal server. We recommend the installation guidance provided by Intel® Extension for PyTorch. The following are some key steps:

• Install Intel® oneAPI Base Toolkit, indluding Intel® oneAPI DPC++ Compiler, Intel® oneAPI Math Kernel Library (oneMKL), Intel® oneAPI Collective Communications Library (oneCCL), and Intel® oneCCL Bindings for PyTorch.

# Install oneCCL package on Ubuntu
sudo apt install -y intel-oneapi-dpcpp-cpp-2024.1=2024.1.0-963 intel-oneapi-mkl-devel=2024.1.0-691 intel-oneapi-ccl-devel=2021.12.0-309
# Install oneccl_bindings_for_pytorch
pip install oneccl_bind_pt==2.1.300+xpu --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/
# Runtime Dynamic Linking
source /opt/intel/oneapi/setvars.sh

• Install Intel® Extension for PyTorch and the corresponding version of PyTorch

pip install torch==2.1.0.post2 intel-extension-for-pytorch==2.1.30+xpu --extra-index-url https://pytorch-extension.intel.com/release-whl/stable/xpu/us/

Running benchmark Copy Section

This guide is helpful for you to launch DDP training on intel GPU.

To Run benchmark, e.g. assuming you have n XPUs:

mpirun -np <n> python training_benchmark_xpu.py --dataset ogbn-products --model edge_cnn --num-epochs 3

File: benchmark/kernel/README.md Copy Chapter

Graph Classification

Evaluation script for various methods on common benchmark datasets via 10-fold cross validation, where a training fold is randomly sampled to serve as a validation set.
Hyperparameter selection is performed for the number of hidden units and the number of layers with respect to the validation set:

• GCN
• GraphSAGE
• GIN
• Graclus
• Top-K Pooling
• SAG Pooling
• DiffPool
• EdgePool
• GlobalAttention
• Set2Set
• SortPool
• ASAPool

Run (or modify) the whole test suite via

$ python main.py

For more comprehensive time-measurement and memory usage information, you may use

$ python main_performance.py

File: benchmark/inference/README.md Copy Chapter

Inference Benchmark

Environment setup Copy Chapter

1. Confirm that PyG is properly installed.
2. Install dataset package:
   bash Copy

   pip install ogb

3. Install autoconf required for jemalloc setup:
   bash Copy

   sudo apt-get install autoconf

4. Install jemalloc for performance benchmark:
   bash Copy

   cd ${workspace}
   git clone https://github.com/jemalloc/jemalloc.git
   cd jemalloc
   git checkout 5.2.1
   ./autogen.sh
   ./configure --prefix=${workspace}/jemalloc-bin
   make
   make install

Running benchmark Copy Chapter

1. Set environment variables:
   bash Copy

   source activate env_name
   export DNNL_PRIMITIVE_CACHE_CAPACITY=1024
   export KMP_BLOCKTIME=1
   export KMP_AFFINITY=granularity=fine,compact,1,0
   
   jemalloc_lib=${workspace}/jemalloc-bin/lib/libjemalloc.so
   export LD_PRELOAD="$jemalloc_lib"
   export MALLOC_CONF="oversize_threshold:1,background_thread:true,metadata_thp:auto,dirty_decay_ms:9000000000,muzzy_decay_ms:9000000000"

2. Core binding, e.g., single socket / single core / 4 cores per instance:
   bash Copy

   OMP_NUM_THREADS=${CORES} numactl -C 0-${LAST_CORE} -m 0 CMD......

3. Execute benchmarks, e.g.:
   bash Copy

   python -u inference_benchmark.py --datasets=Reddit --models=gcn --eval-batch-sizes=512 --num-layers=2 --num-hidden-channels=64
   python -u inference_benchmark.py --datasets=Reddit --models=gcn --eval-batch-sizes=512 --num-layers=2 --num-hidden-channels=64 --use-sparse-tensor
   python -u inference_benchmark.py --datasets=ogbn-products --models=sage --eval-batch-sizes=512 --num-layers=2 --num-hidden-channels=64
   python -u inference_benchmark.py --datasets=ogbn-products --models=sage --eval-batch-sizes=512 --num-layers=2 --num-hidden-channels=64 --use-sparse-tensor

File: benchmark/citation/README.md Copy Chapter

Semi-supervised Node Classification

Evaluation scripts for various methods on the Cora, CiteSeer and PubMed citation networks.
Each experiment is repeated 100 times on either a fixed train/val/test split or on multiple random splits:

• GCN: python gcn.py
• GAT: python gat.py
• Cheby: python cheb.py
• SGC: python sgc.py
• ARMA: python arma.py
• APPNP: python appnp.py

Run the whole test suite via

$ ./run.sh

File: benchmark/runtime/README.md Copy Chapter

Runtimes

Run the test suite for PyG via

python main.py

Install dgl and run the test suite for DGL via

cd dgl
python main.py

File: benchmark/training/README.md Copy Chapter

Training Benchmark

Environment setup Copy Chapter

1. Confirm that PyG is properly installed.
2. Install dataset package:
   bash Copy

   pip install ogb

3. Install jemalloc for performance benchmark:
   bash Copy

   cd ${workspace}
   git clone https://github.com/jemalloc/jemalloc.git
   cd jemalloc
   git checkout 5.2.1
   ./autogen.sh
   ./configure --prefix=${workspace}/jemalloc-bin
   make
   make install

Running benchmark Copy Chapter

1. Set environment variables:
   bash Copy

   source activate env_name
   export DNNL_PRIMITIVE_CACHE_CAPACITY=1024
   export KMP_BLOCKTIME=1
   export KMP_AFFINITY=granularity=fine,compact,1,0
   
   jemalloc_lib=${workspace}/jemalloc-bin/lib/libjemalloc.so
   export LD_PRELOAD="$jemalloc_lib"
   export MALLOC_CONF="oversize_threshold:1,background_thread:true,metadata_thp:auto,dirty_decay_ms:9000000000,muzzy_decay_ms:9000000000"

2. Core binding, e.g., single socket / single core / 4 cores per instance:
   bash Copy

   OMP_NUM_THREADS=${CORES} numactl -C 0-${LAST_CORE} -m 0 CMD......

3. Execute benchmarks, e.g.:
   bash Copy

   python training_benchmark.py --models=gcn --datasets=Reddit --num-workers=0 --batch-sizes=512 --num-layers=2 --num-hidden-channels=64 --num-steps=50
   python training_benchmark.py --models=gcn --datasets=Reddit --num-workers=0 --batch-sizes=512 --num-layers=2 --num-hidden-channels=64 --num-steps=50 --use-sparse-tensor
   python training_benchmark.py --models=sage --datasets=ogbn-products --num-workers=0 --batch-sizes=512 --num-layers=2 --num-hidden-channels=64 --num-steps=50
   python training_benchmark.py --models=sage --datasets=ogbn-products --num-workers=0 --batch-sizes=512 --num-layers=2 --num-hidden-channels=64 --num-steps=50 --use-sparse-tensor

File: benchmark/README.md Copy Chapter

PyG Benchmark Suite

This benchmark suite provides evaluation scripts for semi-supervised node classification, graph classification, and point cloud classification and runtimes in order to compare various methods in homogeneous evaluation scenarios.
In particular, we take care to avoid to perform hyperparameter and model selection on the test set and instead use an additional validation set.

Installation Copy Chapter

$ pip install -e .

File: docker/README.md Copy Chapter

Docker on NVIDIA GPU

The recommended way to use Docker for NVIDIA hardware is described here.

You can also run PyG with CUDA 12.1 inside a docker image. This method is deprecated and we highly recommend the above mentioned official NVIDIA docker containers instead.

The creation of our dockerfile refers to the dockerfiles provided by NVIDIA and PyTorch.

1. Download the dockerfile to your host server.
2. $ docker build -t "custom image name"
3. $ docker run --rm -it --init --runtime=nvidia --ipc=host --network=host --volume=$PWD:/app -e NVIDIA_VISIBLE_DEVICES=0 "custom image name" /bin/bash

If you encounter any problems, please feel free to create a GitHub issue.

Docker on Intel GPU

You can also run PyG with Intel GPU inside a docker image.
The creation of our dockerfile refers to the dockerfiles provided by Intel and the installation guidance provided by Intel® Extension for PyTorch.

1. Download the dockerfile to your host server.
2. $ docker build -f docker/Dockerfile.xpu -t "custom image name"
3. $ docker run --rm -it --ipc=host -v /dev/dri:/dev/dri --volume=$PWD:/app "custom image name" /bin/bash

Singularity

You can run PyG inside a singularity image. An example singularity file can be found in this folder.

You might have to modify the script; depending on your needs, modify the following:

• cuda version 10.1: If you need another version, change From: nvidia/cuda:10.1-cudnn7-devel-ubuntu16.04 to the corresponding tag from https://hub.docker.com/r/nvidia/cuda. Same if you want to use anything but Ubuntu 16.04. Your host has to have at least this cuda version!
• python version 3.7.2: If you need another version, change pyenv install 3.7.2 and the following lines to the corresponding version.
• pytorch version 1.3.0: If you need another version, change pip install torch==1.3.0.
• pytorch_geometric versions: This uses specific versions for each of the pytorch_geometric requirements (scatter: 1.4.0, sparse: 0.4.3, cluster: 1.4.5, geometric 1.3.2). To change these, change the corresponding git checkout lines near the bottom.
• cuda compute capability 5.0 and 6.1: If you run it on multiply systems (likely, with singularity), ensure that all compute capabilities are listed here. If you have the cuda samples installed, check it with (for example) /usr/local/cuda-10.1/extras/demo_suite/deviceQuery | grep 'CUDA Capability'; if not, check here.

Note: If your harddisk runs full after multiple builds, this is known and apparently working as intended; delete the /tmp/sbuild-XXXXXXXXX files.

Building and Using the Container Copy Chapter

To build the container, run

sudo singularity build geometric.sif singularity

then wait. Once finished, you can run the GAT example in the folder you built the image in by calling

wget https://raw.githubusercontent.com/pyg-team/pytorch_geometric/master/examples/gat.py

(to download the sample),

then

singularity exec geometric.sif python3 gat.py

to run on the CPU, or

singularity exec --nv geometric.sif python3 gat.py

to run on the GPU.