{"owner":"pytorch","repo":"cpp","hasSkills":true,"totalSkillsCount":3,"totalTokensCount":14425,"categories":["root-instruction","cursor-rule"],"hasMcp":false,"mcpConfig":null,"found":["llms.txt","llms-full.txt",".cursorrules (cpp.mdc)"],"skills":{"llms.txt":"# PyTorch\n\n> PyTorch documentation.\n\n## Docs\n\n- [Tensor Accessors](api/aten/accessors.md)\n- [Tensor Creation](api/aten/creation.md)\n- [ATen: Tensor Library](api/aten/index.md)\n- [Tensor Indexing](api/aten/indexing.md)\n- [Tensor Class](api/aten/tensor.md)\n- [Custom Autograd Functions](api/autograd/custom_functions.md)\n- [Gradient Computation](api/autograd/gradient.md)\n- [Autograd: Automatic Differentiation](api/autograd/index.md)\n- [Gradient Modes](api/autograd/modes.md)\n- [Device and DeviceType](api/c10/device.md)\n- [Device Guards](api/c10/guards.md)\n- [C10: Core Utilities](api/c10/index.md)\n- [Streams](api/c10/streams.md)\n- [Core Types](api/c10/types.md)\n- [Utilities](api/c10/utilities.md)\n- [CUDA Guards](api/cuda/guards.md)\n- [CUDA Support](api/cuda/index.md)\n- [CUDA Streams](api/cuda/streams.md)\n- [CUDA Utility Functions](api/cuda/utilities.md)\n- [DataLoader](api/data/dataloader.md)\n- [Datasets](api/data/datasets.md)\n- [Data Loading (torch::data)](api/data/index.md)\n- [Samplers](api/data/samplers.md)\n- [Transforms](api/data/transforms.md)\n- [C++ API Reference](api/index.md)\n- [Custom Classes](api/library/custom_classes.md)\n- [Torch Library API](api/library/index.md)\n- [Operator Registration](api/library/registration.md)\n- [Library Versioning](api/library/versioning.md)\n- [Activation Functions](api/nn/activation.md)\n- [Containers](api/nn/containers.md)\n- [Convolution Layers](api/nn/convolution.md)\n- [Dropout Layers](api/nn/dropout.md)\n- [Embedding Layers](api/nn/embedding.md)\n- [Functional API](api/nn/functional.md)\n- [Neural Network Modules (torch::nn)](api/nn/index.md)\n- [Linear Layers](api/nn/linear.md)\n- [Loss Functions](api/nn/loss.md)\n- [Normalization Layers](api/nn/normalization.md)\n- [Pooling Layers](api/nn/pooling.md)\n- [Recurrent Layers](api/nn/recurrent.md)\n- [Transformer Layers](api/nn/transformer.md)\n- [Utilities](api/nn/utilities.md)\n- [Adaptive Learning Rate Optimizers](api/optim/adaptive.md)\n- [Gradient Descent Optimizers](api/optim/gradient_descent.md)\n- [Optimizers (torch::optim)](api/optim/index.md)\n- [Learning Rate Schedulers](api/optim/schedulers.md)\n- [Second-Order Optimizers](api/optim/second_order.md)\n- [Archives](api/serialize/archives.md)\n- [Checkpoints](api/serialize/checkpoints.md)\n- [Serialization (torch::serialize)](api/serialize/index.md)\n- [Saving and Loading](api/serialize/save_load.md)\n- [Torch Stable API](api/stable/index.md)\n- [Stable Operators](api/stable/operators.md)\n- [Library Registration Macros](api/stable/registration.md)\n- [Utilities](api/stable/utilities.md)\n- [XPU Support](api/xpu/index.md)\n- [XPU Streams](api/xpu/streams.md)\n- [XPU Utility Functions](api/xpu/utilities.md)\n- [FAQ](faq.md)\n- [The C++ Frontend](frontend.md)\n- [PyTorch C++ API](index.md)\n- [Installing C++ Distributions of PyTorch](installing.md)","llms-full.txt":"# PyTorch\n\n> PyTorch documentation.\n\n\n---\n\n# Tensor Accessors\n\nFor element-wise operations in custom kernels, use *accessors* to avoid\ndynamic dispatch overhead.\n\n## CPU Accessors\n\n```\ntorch::Tensor foo = torch::rand({12, 12});\n\n// Create accessor - validates type and dimensions once\nauto foo_a = foo.accessor<float, 2>();\n\nfloat trace = 0;\nfor (int i = 0; i < foo_a.size(0); i++) {\n trace += foo_a[i][i];\n}\n```\n\n## CUDA Packed Accessors\n\nFor CUDA kernels, use *packed accessors* which copy metadata instead of\npointing to it:\n\n```\n__global__ void kernel(torch::PackedTensorAccessor64<float, 2> foo, float* trace) {\n int i = threadIdx.x;\n gpuAtomicAdd(trace, foo[i][i]);\n}\n\ntorch::Tensor foo = torch::rand({12, 12}).cuda();\nauto foo_a = foo.packed_accessor64<float, 2>();\n\nfloat trace = 0;\nkernel<<<1, 12>>>(foo_a, &trace);\n```\n\nTip\n\nUse `PackedTensorAccessor32` and `packed_accessor32` for 32-bit indexing,\nwhich is faster on CUDA but may overflow for large tensors.\n\n---\n\n# Tensor Creation\n\nFactory functions create new tensors with different initialization patterns.\nAll factory functions follow a general schema:\n\n```\ntorch::<function-name>(<function-specific-options>, <sizes>, <tensor-options>)\n```\n\n## Available Factory Functions\n\n- `torch::zeros` - Tensor filled with zeros\n- `torch::ones` - Tensor filled with ones\n- `torch::empty` - Uninitialized tensor\n- `torch::full` - Tensor filled with a single value\n- `torch::rand` - Uniform random values on [0, 1)\n- `torch::randn` - Standard normal distribution\n- `torch::randint` - Random integers in a range\n- `torch::arange` - Sequence of integers\n- `torch::linspace` - Linearly spaced values\n- `torch::logspace` - Logarithmically spaced values\n- `torch::eye` - Identity matrix\n- `torch::randperm` - Random permutation of integers\n\n## Specifying a Size\n\nFunctions that do not require specific arguments can be invoked with just a\nsize. For example, the following line creates a vector with 5 components:\n\n```\ntorch::Tensor tensor = torch::ones(5);\n```\n\nAn `IntArrayRef` is constructed by specifying the size along each dimension in\ncurly braces. For example, `{2, 3}` for a matrix with two rows and three\ncolumns, `{3, 4, 5}` for a three-dimensional tensor:\n\n```\ntorch::Tensor tensor = torch::randn({3, 4, 5});\nassert(tensor.sizes() == std::vector<int64_t>{3, 4, 5});\n```\n\nYou can also pass an `std::vector<int64_t>` instead of curly braces.\nUse `tensor.size(i)` to access a single dimension.\n\n## Passing Function-Specific Parameters\n\nSome factory functions accept additional parameters. For example, `randint`\ntakes an upper bound on the value for the integers it generates:\n\n```\ntorch::Tensor tensor = torch::randint(/*high=*/10, {5, 5});\n\n// With a lower bound\ntorch::Tensor tensor = torch::randint(/*low=*/3, /*high=*/10, {5, 5});\n```\n\nTip\n\nThe size always follows the function-specific arguments.\n\nAttention\n\nSome functions like `arange` do not need a size at all, since it is fully\ndetermined by the function-specific arguments (the range bounds).\n\n## Configuring Properties with TensorOptions\n\n`TensorOptions` configures the data type, layout, device, and\n`requires_grad` of a new tensor. The construction axes are:\n\n- `dtype`: the data type of the elements (e.g. `kFloat32`, `kInt64`)\n- `layout`: either `kStrided` (dense) or `kSparse`\n- `device`: a compute device (e.g. `kCPU`, `kCUDA`)\n- `requires_grad`: whether to track gradients\n\nAllowed values:\n\n- `dtype`: `kUInt8`, `kInt8`, `kInt16`, `kInt32`, `kInt64`,\n`kFloat32`, `kFloat64`\n- `layout`: `kStrided`, `kSparse`\n- `device`: `kCPU`, or `kCUDA` (with an optional device index)\n- `requires_grad`: `true` or `false`\n\nTip\n\nRust-style shorthands exist for dtypes, like `kF32` instead of\n`kFloat32`. See\n[torch/types.h](https://github.com/pytorch/pytorch/blob/main/torch/csrc/api/include/torch/types.h)\nfor the full list.\n\nHere is an example of creating a `TensorOptions` object:\n\n```\nauto options =\n torch::TensorOptions()\n .dtype(torch::kFloat64)\n .layout(torch::kStrided)\n .device(torch::kCUDA, 1)\n .requires_grad(true);\n\ntorch::Tensor tensor = torch::full({3, 4}, /*value=*/123, options);\n\nassert(tensor.dtype() == torch::kFloat64);\nassert(tensor.layout() == torch::kStrided);\nassert(tensor.device().type() == torch::kCUDA);\nassert(tensor.device().index() == 1);\nassert(tensor.requires_grad());\n```\n\n**Defaults:** Any axis you omit takes its default value: `kFloat32` for dtype,\n`kStrided` for layout, `kCPU` for device, and `false` for\n`requires_grad`. This means you can omit `TensorOptions` entirely:\n\n```\n// A 32-bit float, strided, CPU tensor that does not require a gradient.\ntorch::Tensor tensor = torch::randn({3, 4});\n```\n\n**Shorthand syntax:** For each axis there is a free function in the `torch::`\nnamespace (`torch::dtype()`, `torch::device()`, `torch::layout()`,\n`torch::requires_grad()`). Each returns a `TensorOptions` object that can\nbe further refined with builder methods:\n\n```\n// These are equivalent:\ntorch::ones(10, torch::TensorOptions().dtype(torch::kFloat32))\ntorch::ones(10, torch::dtype(torch::kFloat32))\n\n// Chaining:\ntorch::ones(10, torch::dtype(torch::kFloat32).layout(torch::kStrided))\n```\n\n**Implicit construction:** `TensorOptions` is implicitly constructible from\nindividual values, so when only one axis differs from the default you can\nwrite:\n\n```\ntorch::ones(10, torch::kFloat32)\n```\n\nPutting it all together, a C++ tensor creation call mirrors the Python\nequivalent closely:\n\n```\n# Python\ntorch.randn(3, 4, dtype=torch.float32, device=torch.device('cuda', 1), requires_grad=True)\n```\n\n```\n// C++\ntorch::randn({3, 4}, torch::dtype(torch::kFloat32).device(torch::kCUDA, 1).requires_grad(true))\n```\n\n## Using Externally Created Data\n\nIf you already have tensor data allocated in memory (CPU or CUDA), use\n`from_blob` to view that memory as a `Tensor`:\n\n```\nfloat data[] = {1, 2, 3, 4, 5, 6};\ntorch::Tensor tensor = torch::from_blob(data, {2, 3});\n```\n\nNote\n\nTensors created with `from_blob` cannot be resized because ATen does not\nown the memory.\n\n## Tensor Conversion\n\nUse `to()` to convert tensors between dtypes and devices. The conversion\ncreates a new tensor and does not occur in-place:\n\n```\ntorch::Tensor source = torch::randn({2, 3}, torch::kInt64);\n\n// Convert dtype\ntorch::Tensor float_tensor = source.to(torch::kFloat32);\n\n// Move to GPU (default CUDA device)\ntorch::Tensor gpu_tensor = float_tensor.to(torch::kCUDA);\n\n// Specific GPU device\ntorch::Tensor gpu1_tensor = float_tensor.to(torch::Device(torch::kCUDA, 1));\n\n// Async copy\ntorch::Tensor async_tensor = gpu_tensor.to(torch::kCPU, /*non_blocking=*/true);\n```\n\nAttention\n\nThe result of the conversion is a new tensor pointing to new memory,\nunrelated to the source tensor.\n\n## Scalars and Zero-Dimensional Tensors\n\n`Scalar` represents a single dynamically-typed number. Like a `Tensor`,\n`Scalar` is dynamically typed and can hold any of ATen's number types.\nScalars can be implicitly constructed from C++ number types:\n\n```\nnamespace torch {\nTensor addmm(Scalar beta, const Tensor & self,\n Scalar alpha, const Tensor & mat1,\n const Tensor & mat2);\nScalar sum(const Tensor & self);\n} // namespace torch\n\n// Usage\ntorch::Tensor a = ...;\ntorch::Tensor b = ...;\ntorch::Tensor c = ...;\ntorch::Tensor r = torch::addmm(1.0, a, .5, b, c);\n```\n\nZero-dimensional tensors hold a single value and can reference elements in\nlarger tensors:\n\n```\ntorch::Tensor matrix = torch::rand({10, 20});\nmatrix[1][2] = 4; // matrix[1][2] is a zero-dimensional tensor\n```\n\n---\n\n# ATen: Tensor Library\n\nATen (A Tensor Library) is the foundational tensor and mathematical operation\nlibrary on which all of PyTorch is built. It provides the core `Tensor` class\nand hundreds of mathematical operations that work on tensors.\n\n**When to use ATen directly:**\n\n- When writing low-level tensor operations or custom kernels\n- When you need direct access to tensor data and metadata\n- When working with the PyTorch internals or extending PyTorch\n\n**Basic usage:**\n\n```\n#include <ATen/ATen.h>\n\n// Create tensors\nat::Tensor a = at::ones({2, 3});\nat::Tensor b = at::randn({2, 3});\n\n// Perform operations\nat::Tensor c = a + b;\nat::Tensor d = at::matmul(a.t(), b);\n\n// Move to GPU\nif (at::cuda::is_available()) {\n at::Tensor gpu_tensor = c.to(at::kCUDA);\n}\n```\n\nFor most applications, prefer using the higher-level `torch::` namespace\n(see [Neural Network Modules (torch::nn)](../nn/index.html), [Optimizers (torch::optim)](../optim/index.html)) which provides a more user-friendly API.\n\n## Header Files\n\nThe following headers are part of the ATen public API:\n\n- `ATen/ATen.h` - Main ATen header\n- `ATen/Backend.h` - Backend enumeration\n- `ATen/core/Tensor.h` - Tensor class\n- `ATen/core/ivalue.h` - IValue type (see [C10: Core Utilities](../c10/index.html))\n- `ATen/core/ScalarType.h` - Data type definitions\n- `ATen/TensorOptions.h` - Tensor creation options\n- `ATen/Scalar.h` - Scalar type\n- `ATen/Layout.h` - Tensor layout\n- `ATen/DeviceGuard.h` - Device context management\n- `ATen/native/TensorShape.h` - Tensor shape operations\n- `ATen/cuda/CUDAContext.h` - CUDA context (see [CUDA Support](../cuda/index.html))\n- `ATen/cudnn/Descriptors.h` - cuDNN descriptors\n- `ATen/mkl/Descriptors.h` - MKL descriptors\n\nNote\n\nThe core `at::Tensor` class is defined in a generated header file\n(`TensorBody.h`) that only exists after building PyTorch. The documentation\nbelow describes the API manually.\n\n## ATen Categories\n\n- [Tensor Class](tensor.html)\n- [Tensor Creation](creation.html)\n- [Tensor Indexing](indexing.html)\n- [Tensor Accessors](accessors.html)\n\n---\n\n# Tensor Indexing\n\nThe PyTorch C++ API provides tensor indexing similar to Python. Use\n`torch::indexing` namespace for index types:\n\n```\nusing namespace torch::indexing;\n```\n\nThe main difference from Python is that instead of using the `[]` operator,\nthe C++ API uses the `index` and `index_put_` methods:\n\n- `torch::Tensor::index` -- read elements\n- `torch::Tensor::index_put_` -- write elements\n\n## Index Types\n\nThe `TensorIndex` class accepts six types of indices via implicit constructors:\n\n| Type | C++ | Python equivalent |\n| --- | --- | --- |\n| None (unsqueeze) | `None` | `None` |\n| Ellipsis | `Ellipsis` or `\"...\"` | `...` |\n| Integer | `0`, `1`, `-1` | `0`, `1`, `-1` |\n| Boolean | `true`, `false` | `True`, `False` |\n| Slice | `Slice(start, stop, step)` | `start:stop:step` |\n| Tensor | `torch::tensor({0, 2})` | `torch.tensor([0, 2])` |\n\n## Getter Operations\n\n| Python | C++ |\n| --- | --- |\n| `tensor[0]` | `tensor.index({0})` |\n| `tensor[-1]` | `tensor.index({-1})` |\n| `tensor[1, 2]` | `tensor.index({1, 2})` |\n| `tensor[1, :, 3]` | `tensor.index({1, Slice(), 3})` |\n| `tensor[None]` | `tensor.index({None})` |\n| `tensor[:, None]` | `tensor.index({Slice(), None})` |\n| `tensor[...]` | `tensor.index({Ellipsis})` or `tensor.index({\"...\"})` |\n| `tensor[..., 0]` | `tensor.index({Ellipsis, 0})` |\n| `tensor[1::2]` | `tensor.index({Slice(1, None, 2)})` |\n| `tensor[True]` | `tensor.index({true})` |\n| `tensor[torch.tensor([1, 2])]` | `tensor.index({torch::tensor({1, 2})})` |\n| `tensor[bool_mask]` | `tensor.index({bool_mask})` |\n| `tensor[:, torch.tensor([[0,1],[4,3]])]` | `tensor.index({Slice(), torch::tensor({{0,1},{4,3}})})` |\n| `tensor[cond > 0]` | `tensor.index({cond > 0})` |\n\n## Setter Operations\n\n| Python | C++ |\n| --- | --- |\n| `tensor[0] = 1` | `tensor.index_put_({0}, 1)` |\n| `tensor[1, 2] = 1` | `tensor.index_put_({1, 2}, 1)` |\n| `tensor[1] = torch.arange(5)` | `tensor.index_put_({1}, torch::arange(5))` |\n| `tensor[1::2] = 1` | `tensor.index_put_({Slice(1, None, 2)}, 1)` |\n| `tensor[0, 1::2] = torch.tensor([3., 4.])` | `tensor.index_put_({0, Slice(1, None, 2)}, torch::tensor({3., 4.}))` |\n| `tensor[...] = 0` | `tensor.index_put_({Ellipsis}, 0)` |\n| `tensor[None] = value` | `tensor.index_put_({None}, value)` |\n| `tensor[bool_mask] = 0` | `tensor.index_put_({bool_mask}, 0)` |\n| `tensor[torch.tensor([0, 2])] = value` | `tensor.index_put_({torch::tensor({0, 2})}, value)` |\n| `tensor[1:2, torch.tensor([1,2])] = 0` | `tensor.index_put_({Slice(1, 2), torch::tensor({1, 2})}, 0)` |\n\nThe `index_put_` method also accepts an optional `accumulate` parameter.\nWhen `true`, values are added to existing values instead of replacing them:\n\n```\ntensor.index_put_({mask}, values, /*accumulate=*/true);\n```\n\n## Slice Syntax\n\nThe `Slice` constructor signature is:\n\n```\nSlice(\n std::optional<c10::SymInt> start = std::nullopt,\n std::optional<c10::SymInt> stop = std::nullopt,\n std::optional<c10::SymInt> step = std::nullopt);\n```\n\nPass `None` for open-ended bounds:\n\n| Python | C++ |\n| --- | --- |\n| `:` or `::` | `Slice()` or `Slice(None, None)` |\n| `1:` | `Slice(1, None)` |\n| `:3` | `Slice(None, 3)` |\n| `1:3` | `Slice(1, 3)` |\n| `1:3:2` | `Slice(1, 3, 2)` |\n| `::2` | `Slice(None, None, 2)` |\n\n## Full Example\n\n```\n#include <torch/torch.h>\n\nusing namespace torch::indexing;\n\nauto tensor = torch::arange(2 * 3 * 4).reshape({2, 3, 4});\n\n// Basic indexing\nauto row = tensor.index({0}); // tensor[0]\nauto elem = tensor.index({1, 2, 3}); // tensor[1, 2, 3]\n\n// Slicing\nauto sliced = tensor.index({Slice(), Slice(0, 2)}); // tensor[:, 0:2]\n\n// None (unsqueeze) and Ellipsis\nauto unsqueezed = tensor.index({None}); // tensor[None]\nauto last_dim = tensor.index({Ellipsis, -1}); // tensor[..., -1]\n\n// Boolean mask indexing\nauto mask = tensor > 10;\nauto selected = tensor.index({mask}); // tensor[tensor > 10]\n\n// Integer tensor (fancy) indexing\nauto idx = torch::tensor({0, 2});\nauto gathered = tensor.index({Slice(), idx}); // tensor[:, [0, 2]]\n\n// Setting values\ntensor.index_put_({0, Slice(), 0}, 99); // tensor[0, :, 0] = 99\ntensor.index_put_({mask}, 0); // tensor[tensor > 10] = 0\n```\n\n---\n\n# Tensor Class\n\nThe `at::Tensor` class is the primary tensor class in ATen, representing\na multi-dimensional array with a specific data type and device.\n\n## Tensor\n\nclass at::Tensor\n\nThe primary tensor class in ATen. Represents a multi-dimensional array\nwith a specific data type and device.\n\nTensor()\n\nDefault constructor. Creates an undefined tensor.\n\nint64_t dim() const\n\nReturns the number of dimensions of the tensor.\n\nint64_t size(int64_t dim) const\n\nReturns the size of the tensor at the given dimension.\n\nIntArrayRef sizes() const\n\nReturns the sizes of all dimensions.\n\nIntArrayRef strides() const\n\nReturns the strides of all dimensions.\n\nScalarType scalar_type() const\n\nReturns the data type of the tensor.\n\nDevice device() const\n\nReturns the device where the tensor is stored.\n\nbool is_cuda() const\n\nReturns true if the tensor is on a CUDA device.\n\nbool is_cpu() const\n\nReturns true if the tensor is on CPU.\n\nbool requires_grad() const\n\nReturns true if gradients need to be computed for this tensor.\n\nTensor &requires_grad_(bool requires_grad = true)\n\nSets whether gradients should be computed for this tensor.\n\nTensor to(Device device) const\n\nReturns a tensor on the specified device.\n\nTensor to(ScalarType dtype) const\n\nReturns a tensor with the specified data type.\n\nTensor contiguous() const\n\nReturns a contiguous tensor with the same data.\n\nvoid *data_ptr() const\n\nReturns a pointer to the underlying data.\n\n**Example:**\n\n```\n#include <ATen/ATen.h>\n\nat::Tensor a = at::ones({2, 2}, at::kInt);\nat::Tensor b = at::randn({2, 2});\nauto c = a + b.to(at::kInt);\n```\n\n## TensorOptions\n\nclass at::TensorOptions\n\nA class to specify options for tensor creation, including dtype, device,\nlayout, and requires_grad.\n\nTensorOptions()\n\nDefault constructor.\n\nTensorOptions dtype(ScalarType dtype) const\n\nReturns options with the specified data type.\n\nTensorOptions device(Device device) const\n\nReturns options with the specified device.\n\nTensorOptions layout(Layout layout) const\n\nReturns options with the specified layout.\n\nTensorOptions requires_grad(bool requires_grad) const\n\nReturns options with the specified requires_grad setting.\n\n**Example:**\n\n```\nauto options = at::TensorOptions()\n .dtype(at::kFloat)\n .device(at::kCUDA, 0)\n .requires_grad(true);\n\nat::Tensor tensor = at::zeros({3, 4}, options);\n```\n\n## Scalar\n\nclass at::Scalar\n\nRepresents a scalar value that can be converted to various numeric types.\n\nScalar(int64_t v)\n\nConstruct from an integer.\n\nScalar(double v)\n\nConstruct from a double.\n\ntemplate<typename T>\nT to() const\n\nConvert to the specified type.\n\nbool isIntegral(bool includeBool = false) const\n\nReturns true if the scalar is an integral type.\n\nbool isFloatingPoint() const\n\nReturns true if the scalar is a floating point type.\n\n## ScalarType\n\nenum class at::ScalarType\n\nEnumeration of data types supported by tensors.\n\nenumerator Byte\n\n8-bit unsigned integer (uint8_t)\n\nenumerator Char\n\n8-bit signed integer (int8_t)\n\nenumerator Short\n\n16-bit signed integer (int16_t)\n\nenumerator Int\n\n32-bit signed integer (int32_t)\n\nenumerator Long\n\n64-bit signed integer (int64_t)\n\nenumerator Half\n\n16-bit floating point (float16)\n\nenumerator Float\n\n32-bit floating point (float)\n\nenumerator Double\n\n64-bit floating point (double)\n\nenumerator Bool\n\nBoolean type\n\nenumerator BFloat16\n\nBrain floating point (bfloat16)\n\nConvenience constants:\n\n- `at::kByte`, `at::kChar`, `at::kShort`, `at::kInt`, `at::kLong`\n- `at::kHalf`, `at::kFloat`, `at::kDouble`, `at::kBFloat16`\n- `at::kBool`\n\n## DeviceGuard\n\nclass DeviceGuard\n\nRAII guard that sets a certain default device in its constructor, and changes it back to the device that was originally active upon destruction.\n\nThe device is always reset to the one that was active at the time of construction of the guard. Even if you `set_device` after construction, the destructor will still reset the device to the one that was active at construction time.\n\nThis device guard does NOT have an uninitialized state; it is guaranteed to reset a device on exit. If you are in a situation where you *might* want to setup a guard (i.e., are looking for the moral equivalent of std::optional<DeviceGuard>), see [OptionalDeviceGuard](../c10/guards.html#PyTorchclassc10_1_1_optional_device_guard).\n\nPublic Functions\n\nexplicit DeviceGuard() = delete\n\nNo default constructor; see Note [Omitted default constructor from RAII].\n\ninline explicit DeviceGuard([Device](../c10/device.html#_CPPv4N3c106DeviceE) device)\n\nSet the current device to the passed [Device](../c10/device.html#PyTorchstructc10_1_1_device).\n\ninline explicit DeviceGuard([Device](../c10/device.html#_CPPv4N3c106DeviceE) device, const impl::DeviceGuardImplInterface *impl)\n\nThis constructor is for testing only.\n\n~DeviceGuard() = default\n\nDeviceGuard(const DeviceGuard&) = delete\n\nCopy is disallowed.\n\nDeviceGuard &operator=(const DeviceGuard&) = delete\n\nDeviceGuard(DeviceGuard &&other) = delete\n\nMove is disallowed, as [DeviceGuard](../c10/guards.html#PyTorchclassc10_1_1_device_guard) does not have an uninitialized state, which is required for moves on types with nontrivial destructors.\n\nDeviceGuard &operator=(DeviceGuard &&other) = delete\n\ninline void reset_device(at::Device device)\n\nSets the device to the given one.\n\nThe specified device must be consistent with the device type originally specified during guard construction.\n\nTODO: The consistency check here is inconsistent with StreamGuard's behavior with set_stream, where a stream on a different device than the original one isn't an error; we just reset the stream and then switch devices.\n\ninline void reset_device(at::Device device, const impl::DeviceGuardImplInterface *impl)\n\nThis method is for testing only.\n\ninline void set_index(DeviceIndex index)\n\nSets the device index to the given one.\n\nThe device type is inferred from the original device type the guard was constructed with.\n\ninline [Device](../c10/device.html#_CPPv4N3c106DeviceE) original_device() const\n\nReturns the device that was set at the time the guard was constructed.\n\ninline [Device](../c10/device.html#_CPPv4N3c106DeviceE) current_device() const\n\nReturns the most recent device that was set using this device guard, either from construction, or via set_device.\n\n**Example:**\n\n```\n{\n c10::DeviceGuard guard(at::Device(at::kCUDA, 1));\n // Operations here run on CUDA device 1\n auto tensor = at::zeros({2, 2});\n}\n// Previous device is restored\n```\n\n## Layout\n\nenum class at::Layout\n\nEnumeration of tensor memory layouts.\n\nenumerator Strided\n\nDense tensor with strides.\n\nenumerator Sparse\n\nSparse tensor (COO format).\n\nenumerator SparseCsr\n\nSparse tensor in CSR format.\n\nenumerator SparseCsc\n\nSparse tensor in CSC format.\n\n---\n\n# Custom Autograd Functions\n\nPyTorch allows you to define custom autograd functions with custom forward\nand backward implementations.\n\n## Function Base Class\n\ntemplate<class T>\nstruct Function\n\nTo use custom autograd operations, implement a Function subclass with static forward and backward functions:\n\n`forward` can take as many arguments as you want and should return either a variable list or a Variable. Use of any direct Variable arguments will be registered in the graph but no vectors/sets or any other data structures will be traversed. You can use std::optional<Tensor> as one of the arguments and it will be registered as a variable in the graph if the argument has a value. It should take a pointer to `torch::autograd::AutogradContext` as the first argument. Variables can be saved in the `ctx` using `ctx->save_for_backward` (see `torch::autograd::AutogradContext::save_for_backward`) and other data can be saved in the `ctx->saved_data` map (see `torch::autograd::AutogradContext::saved_data`) in the form of `<std::string, at::IValue>` pairs.\n\n`backward` should take a pointer to `torch::autograd::AutogradContext` and a variable list containing as many Variables as there were outputs from `forward` as arguments. It should return as many Variables as there were inputs with each of them containing the gradient w.r.t. its corresponding input. Variables saved in `forward` can be accessed with `ctx->get_saved_variables` (see `torch::autograd::AutogradContext::get_saved_variables`) and other saved data can be accessed from `ctx->saved_data`. To enable compiled autograd support (torch.compile for backward) for your custom autograd operation, you can set MyFunction::is_traceable (see Function::istraceable notes below).\n\nFor example:\n\n```\nclass MyFunction : public Function<MyFunction> {\n public:\n static constexpr bool is_traceable = true;\n\n static variable_list forward(AutogradContext *ctx, int n, Variable var) {\n // Save data for backward in context\n ctx->saved_data[\"n\"] = n;\n var.mul_(n);\n // Mark var as modified by inplace operation\n ctx->mark_dirty({var});\n return {var};\n }\n\n static variable_list backward(AutogradContext *ctx, variable_list\n grad_output) {\n // Use data saved in forward\n auto n = ctx->saved_data[\"n\"].toInt();\n return {grad_output[0]*n};\n }\n};\n```\n\nTo use `MyFunction`:\n\n```\nVariable x;\nauto y = MyFunction::apply(6, x);\n// Example backward call\ny[0].sum().backward();\n```\n\nPublic Static Functions\n\ntemplate<typename X = T, typename ...Args>\nstatic auto apply(Args&&... args) -> std::enable_if_t<std::is_same_v<X, T>, forward_t<X, Args...>>\n\nPublic Static Attributes\n\nstatic constexpr bool is_traceable = false\n\n## AutogradContext\n\nstruct AutogradContext\n\nContext to save information during `forward` that can be accessed in `backward` in custom autograd operations (see `torch::autograd::Function` for details).\n\nPublic Functions\n\nAutogradContext() = default\n\nAutogradContext(const AutogradContext &other) = delete\n\nAutogradContext &operator=(const AutogradContext &other) = delete\n\nAutogradContext(AutogradContext &&other) = delete\n\nAutogradContext &operator=(AutogradContext &&other) = delete\n\n~AutogradContext() = default\n\nAutogradContext(PackedArgs &packed_args)\n\nvoid save_for_backward(variable_list to_save)\n\nSaves the list of variables for a future call to `backward`.\n\nThis should be called at most once from inside of `forward`.\n\nvoid mark_dirty(const variable_list &inputs)\n\nMarks variables in the list as modified in an in-place operation.\n\nThis should be called at most once from inside of `forward` and all arguments should be inputs.\n\nvoid mark_non_differentiable(const variable_list &outputs)\n\nMarks outputs in the list as not requiring gradients.\n\nThis should be called at most once from inside of `forward` and all arguments should be outputs.\n\nvoid set_materialize_grads(bool value)\n\nvariable_list get_saved_variables() const\n\nGet the list of variables that were saved in `forward` using `save_for_backward()`.\n\nBefore returning them to the user, a check is made to ensure that they were not modified by any in-place operations.\n\nconst std::unordered_set<at::TensorImpl*> &get_and_bump_dirty() const\n\nconst std::unordered_set<at::TensorImpl*> &get_non_differentiable() const\n\nbool needs_input_grad(size_t output_edge_index) const\n\nExpose the Node's `task_should_compute_output` method to the cpp custom autograd Function as `needs_input_grad`.\n\nbool needs_input_grad(std::initializer_list<IndexRange> idxs) const\n\nPublic Members\n\nska::flat_hash_map<std::string, at::IValue> saved_data\n\nCan be used to save non-variable data for `backward`.\n\n## Creating Custom Functions\n\nTo create a custom autograd function, inherit from `torch::autograd::Function`\nand implement the static `forward` and `backward` methods:\n\n**Example:**\n\n```\nclass MyReLU : public torch::autograd::Function<MyReLU> {\n public:\n static torch::Tensor forward(\n torch::autograd::AutogradContext* ctx,\n torch::Tensor input) {\n ctx->save_for_backward({input});\n return input.clamp_min(0);\n }\n\n static torch::autograd::variable_list backward(\n torch::autograd::AutogradContext* ctx,\n torch::autograd::variable_list grad_outputs) {\n auto saved = ctx->get_saved_variables();\n auto input = saved[0];\n auto grad_output = grad_outputs[0];\n auto grad_input = grad_output * (input > 0).to(grad_output.dtype());\n return {grad_input};\n }\n};\n\n// Usage\nauto output = MyReLU::apply(input);\n```\n\n## Custom Kernels and AutoDispatchBelowADInplaceOrView\n\nFor users implementing custom kernels who want to redispatch below `Autograd` dispatch\nkeys, use `at::AutoDispatchBelowADInplaceOrView` instead of `InferenceMode`:\n\n```\nclass ROIAlignFunction : public torch::autograd::Function<ROIAlignFunction> {\n public:\n static torch::autograd::variable_list forward(\n torch::autograd::AutogradContext* ctx,\n const torch::autograd::Variable& input,\n const torch::autograd::Variable& rois,\n double spatial_scale,\n int64_t pooled_height,\n int64_t pooled_width,\n int64_t sampling_ratio,\n bool aligned) {\n ctx->saved_data[\"spatial_scale\"] = spatial_scale;\n ctx->saved_data[\"pooled_height\"] = pooled_height;\n ctx->saved_data[\"pooled_width\"] = pooled_width;\n ctx->saved_data[\"sampling_ratio\"] = sampling_ratio;\n ctx->saved_data[\"aligned\"] = aligned;\n ctx->saved_data[\"input_shape\"] = input.sizes();\n ctx->save_for_backward({rois});\n\n at::AutoDispatchBelowADInplaceOrView guard;\n auto result = roi_align(\n input, rois, spatial_scale, pooled_height,\n pooled_width, sampling_ratio, aligned);\n return {result};\n }\n};\n```\n\nFor customized inplace and view kernels, see the\n[custom kernel tutorial](https://pytorch.org/tutorials/advanced/cpp_extension.html#backward-pass)\nfor more details.\n\n---\n\n# Gradient Computation\n\nPyTorch provides functions for computing gradients of tensors with respect\nto graph leaves.\n\n## Gradient Functions\n\nvoid torch::autograd::backward(const variable_list &tensors, const variable_list &grad_tensors = {}, std::optional<bool> retain_graph = std::nullopt, bool create_graph = false, const variable_list &inputs = {})\n\nComputes the sum of gradients of given tensors with respect to graph leaves.\n\nThe graph is differentiated using the chain rule. If any of `tensors`\nare non-scalar (i.e. their data has more than one element) and require\ngradient, then the Jacobian-vector product would be computed, in this case\nthe function additionally requires specifying `grad_tensors`. It should be a\nsequence of matching length, that contains the \"vector\" in the\nJacobian-vector product, usually the gradient of the differentiated function\nw.r.t. corresponding tensors (`torch::Tensor()` is an acceptable value for\nall tensors that don't need gradient tensors).\n\nThis function accumulates gradients in the leaves -- you might need to zero\nthem before calling it.\n\n:param tensors: Tensors of which the derivative will be computed.\n:param grad_tensors: The \"vector\" in the Jacobian-vector product, usually\ngradients w.r.t. each element of corresponding tensors.\n`torch::Tensor()` values can be specified for scalar Tensors or ones\nthat don't require grad. If a `torch::Tensor()` value would be\nacceptable for all grad_tensors, then this argument is optional.\n:param retain_graph: If `false`, the graph used to compute the grad will\nbe freed. Note that in nearly all cases setting this option to `true`\nis not needed and often can be worked around in a much more efficient\nway. Defaults to the value of `create_graph`.\n:param create_graph: If `true`, graph of the derivative will be\nconstructed, allowing to compute higher order derivative products.\nDefaults to `false`.\n:param inputs: Inputs w.r.t. which the gradient will be accumulated into\n`at::Tensor::grad`. All other Tensors will be ignored. If not\nprovided, the gradient is accumulated into all the leaf Tensors that\nwere used to compute `tensors`.\n\nvariable_list torch::autograd::grad(const variable_list &outputs, const variable_list &inputs, const variable_list &grad_outputs = {}, std::optional<bool> retain_graph = std::nullopt, bool create_graph = false, bool allow_unused = false)\n\nComputes and returns the sum of gradients of outputs with respect to the inputs.\n\n`grad_outputs` should be a sequence of length matching `output` containing the \"vector\" in Jacobian-vector product, usually the pre-computed gradients w.r.t. each of the outputs. If an output doesn't require_grad, then the gradient can be `torch::Tensor()`).\n\nParameters:\n\n- **outputs** - outputs of the differentiated function.\n- **inputs** - Inputs w.r.t. which the gradient will be returned (and not accumulated into `at::Tensor::grad`).\n- **grad_outputs** - The \"vector\" in the Jacobian-vector product. Usually gradients w.r.t. each output. `torch::Tensor()` values can be specified for scalar Tensors or ones that don't require grad. If a `torch::Tensor()` value would be acceptable for all grad_tensors, then this argument is optional. Default: `{}`.\n- **retain_graph** - If `false`, the graph used to compute the grad will be freed. Note that in nearly all cases setting this option to `true` is not needed and often can be worked around in a much more efficient way. Defaults to the value of `create_graph`.\n- **create_graph** - If `true`, graph of the derivative will be constructed, allowing to compute higher order derivative products. Default: `false`.\n- **allow_unused** - If `false`, specifying inputs that were not used when computing outputs (and therefore their grad is always zero) is an error. Defaults to `false`.\n\n**Example:**\n\n```\n#include <torch/torch.h>\n\nauto x = torch::randn({2, 2}, torch::requires_grad());\nauto y = x * x;\nauto z = y.sum();\n\n// Compute gradients\nz.backward();\nstd::cout << x.grad() << std::endl;\n\n// Or use grad() for specific outputs\nauto grads = torch::autograd::grad({z}, {x});\n```\n\n## Tensor Gradient Methods\n\nTensors have built-in methods for gradient computation:\n\n```\n// Enable gradient tracking\nauto x = torch::randn({2, 2}).requires_grad_(true);\n\n// Check if gradient is required\nbool needs_grad = x.requires_grad();\n\n// Access the gradient after backward\nauto grad = x.grad();\n\n// Detach from computation graph\nauto x_detached = x.detach();\n```\n\n---\n\n# Autograd: Automatic Differentiation\n\nPyTorch's autograd system provides automatic differentiation for all operations\non tensors. It records operations on tensors to build a computational graph,\nthen computes gradients automatically via backpropagation.\n\n**When to use Autograd:**\n\n- When training neural networks (gradients are computed automatically)\n- When implementing custom backward passes for specialized operations\n- When you need fine-grained control over gradient computation\n\n**Basic usage:**\n\n```\n#include <torch/torch.h>\n\n// Create tensor with gradient tracking\nauto x = torch::randn({2, 2}, torch::requires_grad());\nauto y = x * x;\nauto z = y.sum();\n\n// Compute gradients via backpropagation\nz.backward();\nstd::cout << x.grad() << std::endl; // dz/dx = 2x\n\n// Disable gradient tracking for inference\n{\n torch::NoGradGuard no_grad;\n auto result = model->forward(input); // No gradients computed\n}\n```\n\n## Header Files\n\n- `torch/csrc/autograd/autograd.h` - High-level autograd API\n- `torch/csrc/autograd/function.h` - Custom autograd functions\n- `torch/csrc/autograd/grad_mode.h` - Gradient computation modes\n- `torch/csrc/api/include/torch/autograd.h` - C++ Frontend autograd\n\n## Autograd Categories\n\n- [Gradient Computation](gradient.html)\n- [Custom Autograd Functions](custom_functions.html)\n- [Gradient Modes](modes.html)\n\n---\n\n# Gradient Modes\n\nPyTorch provides RAII guards to control gradient computation behavior.\n\n## NoGradGuard\n\nclass torch::NoGradGuard\n\nRAII guard that disables gradient computation within its scope.\n\nNoGradGuard()\n\nDisables gradient computation.\n\n~NoGradGuard()\n\nRestores previous gradient mode.\n\n**Example:**\n\n```\n{\n torch::NoGradGuard no_grad;\n // No gradients computed in this scope\n auto result = model->forward(input);\n}\n```\n\n## InferenceMode\n\n`c10::InferenceMode` is a RAII guard analogous to `NoGradMode` designed for use\nwhen you are certain your operations will have no interactions with autograd\n(e.g., model inference). Compared to `NoGradMode`, code run under this mode gets\nbetter performance by disabling autograd-related work like view tracking and version\ncounter bumps. However, tensors created inside `InferenceMode` have more limitations\nwhen interacting with the autograd system.\n\nclass c10::InferenceMode\n\nRAII guard that enables inference mode for optimized inference.\nThis is more efficient than NoGradGuard for inference-only workloads.\n\nexplicit InferenceMode(bool enabled = true)\n\nEnables or disables inference mode.\n\n**Inference Tensors:**\n\n`InferenceMode` can be enabled for a given block of code. Inside `InferenceMode`,\nall newly allocated (non-view) tensors are marked as inference tensors. Inference tensors:\n\n- Do not have a version counter, so an error will be raised if you try to read their version\n(e.g., because you saved this tensor for backward).\n- Are immutable outside `InferenceMode`. An error will be raised if you try to:\n\n- Mutate their data outside InferenceMode.\n- Mutate them to `requires_grad=True` outside InferenceMode.\n- To work around this, make a clone outside `InferenceMode` to get a normal tensor before mutating.\n\nA non-view tensor is an inference tensor if and only if it was allocated inside `InferenceMode`.\nA view tensor is an inference tensor if and only if it is a view of an inference tensor.\n\n**Performance Guarantees:**\n\nInside an `InferenceMode` block:\n\n- Like `NoGradMode`, all operations do not record `grad_fn` even if their inputs have\n`requires_grad=True`. This applies to both inference tensors and normal tensors.\n- View operations on inference tensors do not perform view tracking. View and non-view\ninference tensors are indistinguishable.\n- Inplace operations on inference tensors are guaranteed not to do a version bump.\n\nFor more implementation details, see the [RFC-0011-InferenceMode](https://github.com/pytorch/rfcs/pull/17).\n\n**Basic Example:**\n\n```\n{\n c10::InferenceMode guard;\n // Optimized inference without gradient tracking\n auto result = model->forward(input);\n}\n```\n\n**Inference Workload Example:**\n\n```\nc10::InferenceMode guard;\nmodel.load_jit(saved_model);\nauto inputs = preprocess_tensors(data);\nauto out = model.forward(inputs);\nauto outputs = postprocess_tensors(out);\n```\n\n**Nested InferenceMode:**\n\nUnlike some other guards, `InferenceMode` can be nested with different enabled/disabled states:\n\n```\n{\n c10::InferenceMode guard(true);\n // InferenceMode is on\n {\n c10::InferenceMode guard(false);\n // InferenceMode is off\n }\n // InferenceMode is on\n}\n// InferenceMode is off\n```\n\n## InferenceMode vs NoGradMode\n\n`InferenceMode` is preferred over `NoGradMode` for pure inference workloads because\nit provides better performance. Key differences:\n\n- Both guards affect tensor execution to skip work not related to inference, but\n`InferenceMode` also affects tensor creation while `NoGradMode` doesn't.\n- Tensors created inside `InferenceMode` are marked as inference tensors with\ncertain limitations that apply after exiting `InferenceMode`.\n- `InferenceMode` can be nested with enabled/disabled states.\n\n## Migrating from AutoNonVariableTypeMode\n\nThe legacy `AutoNonVariableTypeMode` guard (now renamed to\n`AutoDispatchBelowADInplaceOrView`) was commonly used for inference workloads\nbut is unsafe -- it can silently bypass safety checks and produce wrong results.\n\n- **For inference-only workloads** (e.g. loading a pretrained JIT model and\nrunning inference in C++ runtime), use `c10::InferenceMode` as a drop-in\nreplacement. It preserves the performance characteristics while providing\ncorrectness guarantees.\n- **For custom autograd kernels** that need to redispatch below the Autograd\ndispatch key, use `AutoDispatchBelowADInplaceOrView` instead:\n\n```\nclass ROIAlignFunction : public torch::autograd::Function<ROIAlignFunction> {\n public:\n static torch::autograd::variable_list forward(\n torch::autograd::AutogradContext* ctx,\n const torch::autograd::Variable& input,\n const torch::autograd::Variable& rois,\n double spatial_scale, int64_t pooled_height,\n int64_t pooled_width, int64_t sampling_ratio, bool aligned) {\n ctx->saved_data[\"spatial_scale\"] = spatial_scale;\n ctx->save_for_backward({rois});\n at::AutoDispatchBelowADInplaceOrView guard;\n auto result = roi_align(input, rois, spatial_scale,\n pooled_height, pooled_width, sampling_ratio, aligned);\n return {result};\n }\n};\n```\n\n---\n\n# Device and DeviceType\n\nPyTorch provides device abstractions for writing code that works across\nCPU, CUDA, and other backends.\n\n## Device\n\nstruct Device\n\nRepresents a compute device on which a tensor is located.\n\nA device is uniquely identified by a type, which specifies the type of machine it is (e.g. CPU or CUDA GPU), and a device index or ordinal, which identifies the specific compute device when there is more than one of a certain type. The device index is optional, and in its defaulted state represents (abstractly) \"the current device\". Further, there are two constraints on the value of the device index, if one is explicitly stored:\n\n1. A negative index represents the current device, a non-negative index represents a specific, concrete device,\n2. When the device type is CPU, the device index must be zero.\n\nPublic Types\n\nusing Type = DeviceType\n\nPublic Functions\n\ninline Device(DeviceType type, DeviceIndex index = -1)\n\nConstructs a new `Device` from a `DeviceType` and an optional device index.\n\nDevice(const std::string &device_string)\n\nConstructs a `Device` from a string description, for convenience.\n\nThe string supplied must follow the following schema: `(cpu|cuda)[:<device-index>]` where `cpu` or `cuda` specifies the device type, and `:<device-index>` optionally specifies a device index.\n\ninline bool operator==(const Device &other) const noexcept\n\nReturns true if the type and index of this `Device` matches that of `other`.\n\ninline bool operator!=(const Device &other) const noexcept\n\nReturns true if the type or index of this `Device` differs from that of `other`.\n\ninline void set_index(DeviceIndex index)\n\nSets the device index.\n\ninline DeviceType type() const noexcept\n\nReturns the type of device this is.\n\ninline DeviceIndex index() const noexcept\n\nReturns the optional index.\n\ninline bool has_index() const noexcept\n\nReturns true if the device has a non-default index.\n\ninline bool is_cuda() const noexcept\n\nReturn true if the device is of CUDA type.\n\ninline bool is_privateuseone() const noexcept\n\nReturn true if the device is of PrivateUse1 type.\n\ninline bool is_mps() const noexcept\n\nReturn true if the device is of MPS type.\n\ninline bool is_hip() const noexcept\n\nReturn true if the device is of HIP type.\n\ninline bool is_ve() const noexcept\n\nReturn true if the device is of VE type.\n\ninline bool is_xpu() const noexcept\n\nReturn true if the device is of XPU type.\n\ninline bool is_ipu() const noexcept\n\nReturn true if the device is of IPU type.\n\ninline bool is_xla() const noexcept\n\nReturn true if the device is of XLA type.\n\ninline bool is_mtia() const noexcept\n\nReturn true if the device is of MTIA type.\n\ninline bool is_hpu() const noexcept\n\nReturn true if the device is of HPU type.\n\ninline bool is_lazy() const noexcept\n\nReturn true if the device is of Lazy type.\n\ninline bool is_vulkan() const noexcept\n\nReturn true if the device is of Vulkan type.\n\ninline bool is_metal() const noexcept\n\nReturn true if the device is of Metal type.\n\ninline bool is_maia() const noexcept\n\nReturn true if the device is of MAIA type.\n\ninline bool is_meta() const noexcept\n\nReturn true if the device is of META type.\n\ninline bool is_cpu() const noexcept\n\nReturn true if the device is of CPU type.\n\ninline bool supports_as_strided() const noexcept\n\nReturn true if the device supports arbitrary strides.\n\nstd::string str() const\n\nSame string as returned from operator<<.\n\n**Example:**\n\n```\nc10::Device cpu_device(c10::kCPU);\nc10::Device cuda_device(c10::kCUDA, 0); // CUDA device 0\n\nif (cuda_device.is_cuda()) {\n std::cout << \"Using CUDA device \" << cuda_device.index() << std::endl;\n}\n```\n\n## DeviceType\n\nenum class c10::DeviceType\n\nEnumeration of supported device types.\n\nenumerator CPU = 0\n\nCPU device.\n\nenumerator CUDA = 1\n\nNVIDIA CUDA GPU.\n\nenumerator HIP = 6\n\nAMD HIP GPU.\n\nenumerator XLA = 9\n\nXLA / TPU.\n\nenumerator Vulkan = 10\n\nVulkan GPU.\n\nenumerator Metal = 11\n\nApple Metal GPU.\n\nenumerator XPU = 12\n\nIntel XPU GPU.\n\nenumerator MPS = 13\n\nApple Metal Performance Shaders.\n\nenumerator Meta = 14\n\nMeta tensors (shape only, no data).\n\nenumerator HPU = 15\n\nHabana HPU.\n\nenumerator Lazy = 17\n\nLazy tensors.\n\nenumerator IPU = 18\n\nGraphcore IPU.\n\nenumerator MTIA = 19\n\nMeta training and inference accelerator.\n\nenumerator PrivateUse1 = 20\n\nCustom backend registered via `c10::register_privateuse1_backend()`.\n\nConvenience constants:\n\n- `c10::kCPU`, `c10::kCUDA`, `c10::kHIP`\n- `c10::kXLA`, `c10::kVulkan`, `c10::kMetal`\n- `c10::kXPU`, `c10::kMPS`, `c10::kMeta`\n- `c10::kHPU`, `c10::kLazy`, `c10::kIPU`, `c10::kMTIA`\n- `c10::kPrivateUse1`\n\n---\n\n# Device Guards\n\nC10 provides device-agnostic RAII guards for managing the current device\ncontext. These guards work across all backends (CUDA, XPU, etc.) and\nautomatically restore the previous device when they go out of scope.\n\nFor backend-specific guards, see [CUDA Guards](../cuda/guards.html) and [XPU Support](../xpu/index.html).\n\n## DeviceGuard\n\nclass DeviceGuard\n\nRAII guard that sets a certain default device in its constructor, and changes it back to the device that was originally active upon destruction.\n\nThe device is always reset to the one that was active at the time of construction of the guard. Even if you `set_device` after construction, the destructor will still reset the device to the one that was active at construction time.\n\nThis device guard does NOT have an uninitialized state; it is guaranteed to reset a device on exit. If you are in a situation where you *might* want to setup a guard (i.e., are looking for the moral equivalent of std::optional<DeviceGuard>), see OptionalDeviceGuard.\n\nPublic Functions\n\nexplicit DeviceGuard() = delete\n\nNo default constructor; see Note [Omitted default constructor from RAII].\n\ninline explicit DeviceGuard([Device](device.html#_CPPv4N3c106DeviceE) device)\n\nSet the current device to the passed [Device](device.html#PyTorchstructc10_1_1_device).\n\ninline explicit DeviceGuard([Device](device.html#_CPPv4N3c106DeviceE) device, const [impl](../aten/tensor.html#_CPPv4N3c1011DeviceGuard11DeviceGuardE6DevicePKN4impl24DeviceGuardImplInterfaceE)::DeviceGuardImplInterface *impl)\n\nThis constructor is for testing only.\n\n~DeviceGuard() = default\n\nDeviceGuard(const [DeviceGuard](../aten/tensor.html#_CPPv4N3c1011DeviceGuard11DeviceGuardERK11DeviceGuard)&) = delete\n\nCopy is disallowed.\n\n[DeviceGuard](../aten/tensor.html#_CPPv4N3c1011DeviceGuardE) &operator=(const [DeviceGuard](../aten/tensor.html#_CPPv4N3c1011DeviceGuardE)&) = delete\n\nDeviceGuard([DeviceGuard](../aten/tensor.html#_CPPv4N3c1011DeviceGuard11DeviceGuardERR11DeviceGuard) &&other) = delete\n\nMove is disallowed, as DeviceGuard does not have an uninitialized state, which is required for moves on types with nontrivial destructors.\n\n[DeviceGuard](../aten/tensor.html#_CPPv4N3c1011DeviceGuardE) &operator=([DeviceGuard](../aten/tensor.html#_CPPv4N3c1011DeviceGuardE) &&other) = delete\n\ninline void reset_device(at::Device device)\n\nSets the device to the given one.\n\nThe specified device must be consistent with the device type originally specified during guard construction.\n\nTODO: The consistency check here is inconsistent with StreamGuard's behavior with set_stream, where a stream on a different device than the original one isn't an error; we just reset the stream and then switch devices.\n\ninline void reset_device(at::Device device, const [impl](../aten/tensor.html#_CPPv4N3c1011DeviceGuard12reset_deviceEN2at6DeviceEPKN4impl24DeviceGuardImplInterfaceE)::DeviceGuardImplInterface *impl)\n\nThis method is for testing only.\n\ninline void set_index(DeviceIndex index)\n\nSets the device index to the given one.\n\nThe device type is inferred from the original device type the guard was constructed with.\n\ninline [Device](device.html#_CPPv4N3c106DeviceE) original_device() const\n\nReturns the device that was set at the time the guard was constructed.\n\ninline [Device](device.html#_CPPv4N3c106DeviceE) current_device() const\n\nReturns the most recent device that was set using this device guard, either from construction, or via set_device.\n\n**Example:**\n\n```\n#include <c10/core/DeviceGuard.h>\n\n{\n c10::DeviceGuard guard(c10::Device(c10::kCUDA, 1));\n // All operations here run on CUDA device 1\n}\n// Previous device is restored\n```\n\n## OptionalDeviceGuard\n\nclass OptionalDeviceGuard\n\nA OptionalDeviceGuard is an RAII class that sets a device to some value on initialization, and resets the device to its original value on destruction.\n\nMorally, a OptionalDeviceGuard is equivalent to std::optional<DeviceGuard>, but with extra constructors and methods as appropriate.\n\nBesides its obvious use (optionally applying a DeviceGuard), OptionalDeviceGuard is often also used for the following idiom:\n\nOptionalDeviceGuard g; for (const auto& t : tensors) { g.set_device(t.device()); do_something_with(t); }\n\nThis usage is marginally more efficient than constructing a DeviceGuard every iteration of the for loop, as it avoids an unnecessary device reset.\n\nUnlike DeviceGuard, a OptionalDeviceGuard may be uninitialized. This occurs when you use the nullary constructor, or pass a nullopt to the constructor. Uninitialized OptionalDeviceGuards do *nothing*; they do not know what the original device was and they do not reset on destruction. This is why original_device() and current_device() return std::optional<Device> rather than [Device](device.html#PyTorchstructc10_1_1_device) (as they do in DeviceGuard), and also is why we didn't just provide OptionalDeviceGuard by default and hide DeviceGuard from users.\n\nThe semantics of an OptionalDeviceGuard are exactly explained by thinking of it as an std::optional<DeviceGuard>. In particular, an initialized OptionalDeviceGuard doesn't restore device to its value at construction; it restores device to its value *at initialization*. So if you have the program:\n\n```\nsetDevice(1);\nOptionalDeviceGuard g;\nsetDevice(2);\ng.reset_device(Device(DeviceType::CUDA, 3)); // initializes!\n```\n\n On destruction, g will reset device to 2, rather than 1.\n\nAn uninitialized OptionalDeviceGuard is distinct from a (initialized) DeviceGuard whose original_device_ and current_device_ match, since the DeviceGuard will still reset the device to original_device_.\n\nPublic Functions\n\nexplicit OptionalDeviceGuard() = default\n\nCreate an uninitialized guard. Set the guard later using reset_device.\n\ninline explicit OptionalDeviceGuard([Device](device.html#_CPPv4N3c106DeviceE) device)\n\nInitialize the guard, setting the current device to the passed [Device](device.html#PyTorchstructc10_1_1_device).\n\ninline explicit OptionalDeviceGuard(std::optional<[Device](device.html#_CPPv4N3c106DeviceE)> device)\n\nInitialize the guard if a [Device](device.html#PyTorchstructc10_1_1_device) is passed; otherwise leave the guard uninitialized.\n\ninline explicit OptionalDeviceGuard([Device](device.html#_CPPv4N3c106DeviceE) device, const impl::DeviceGuardImplInterface *impl)\n\nConstructor for testing only.\n\n~OptionalDeviceGuard() = default\n\nOptionalDeviceGuard(const OptionalDeviceGuard&) = delete\n\nCopy is disallowed.\n\nOptionalDeviceGuard &operator=(const OptionalDeviceGuard&) = delete\n\nOptionalDeviceGuard(OptionalDeviceGuard &&other) = delete\n\nMove is disallowed See Note [Explicit initialization of optional fields] and // Note [Move construction for RAII guards is tricky] for rationale.\n\nOptionalDeviceGuard &operator=(OptionalDeviceGuard &&other) = delete\n\ninline void reset_device(at::Device device)\n\nSets the device to the given one.\n\nThe specified device must be consistent with the device type originally specified during guard construction.\n\ninline void reset_device(at::Device device, const impl::DeviceGuardImplInterface *impl)\n\nFor testing only.\n\ninline std::optional<[Device](device.html#_CPPv4N3c106DeviceE)> original_device() const\n\nReturns the device that was set at the time the guard was constructed.\n\ninline std::optional<[Device](device.html#_CPPv4N3c106DeviceE)> current_device() const\n\nReturns the most recent device that was set using this device guard, either from construction, or via reset_device.\n\n**Example:**\n\n```\n#include <c10/core/DeviceGuard.h>\n\nc10::OptionalDeviceGuard guard;\nif (use_gpu) {\n guard.reset_device(c10::Device(c10::kCUDA, 0));\n}\n// Guard only restores device",".cursorrules (cpp.mdc)":"---\ndescription: Guide Cursor to write modern C++ and CMake code with clear structure, RAII, const-correctness, and safe error handling.\nglobs: **/*.c,**/*.cpp,**/*.h,**/*.hpp,**/*.cxx,CMakeLists.txt,*.cmake,conanfile.txt,Makefile,**/*.cc\nalwaysApply: false\n---\n# C++ Programming Guidelines\n\n## Basic Principles\n\n- Use English for all code and documentation.\n- Always declare the type of each variable and function (parameters and return value).\n- Create necessary types and classes.\n- Use Doxygen style comments to document public classes and methods.\n- Don't leave blank lines within a function.\n- Follow the one-definition rule (ODR).\n\n## Nomenclature\n\n- Use PascalCase for classes and structures.\n- Use camelCase for variables, functions, and methods.\n- Use ALL_CAPS for constants and macros.\n- Use snake_case for file and directory names.\n- Use UPPERCASE for environment variables.\n- Avoid magic numbers and define constants.\n- Start each function with a verb.\n- Use verbs for boolean variables. Example: isLoading, hasError, canDelete, etc.\n- Use complete words instead of abbreviations and ensure correct spelling.\n  - Except for standard abbreviations like API, URL, etc.\n  - Except for well-known abbreviations:\n    - i, j, k for loops\n    - err for errors\n    - ctx for contexts\n    - req, res for request/response parameters\n\n## Functions\n\n- Write short functions with a single purpose. Less than 20 instructions.\n- Name functions with a verb and something else.\n- If it returns a boolean, use isX or hasX, canX, etc.\n- If it doesn't return anything (void), use executeX or saveX, etc.\n- Avoid nesting blocks by:\n  - Early checks and returns.\n  - Extraction to utility functions.\n- Use standard library algorithms (std::for_each, std::transform, std::find, etc.) to avoid function nesting.\n- Use lambda functions for simple operations.\n- Use named functions for non-simple operations.\n- Use default parameter values instead of checking for null or nullptr.\n- Reduce function parameters using structs or classes\n  - Use an object to pass multiple parameters.\n  - Use an object to return multiple results.\n  - Declare necessary types for input arguments and output.\n- Use a single level of abstraction.\n\n## Data\n\n- Don't abuse primitive types and encapsulate data in composite types.\n- Avoid data validations in functions and use classes with internal validation.\n- Prefer immutability for data.\n- Use const for data that doesn't change.\n- Use constexpr for compile-time constants.\n- Use std::optional for possibly null values.\n\n## Classes\n\n- Follow SOLID principles.\n- Prefer composition over inheritance.\n- Declare interfaces as abstract classes or concepts.\n- Write small classes with a single purpose.\n  - Less than 200 instructions.\n  - Less than 10 public methods.\n  - Less than 10 properties.\n- Use the Rule of Five (or Rule of Zero) for resource management.\n- Make member variables private and provide getters/setters where necessary.\n- Use const-correctness for member functions.\n\n## Exceptions\n\n- Use exceptions to handle errors you don't expect.\n- If you catch an exception, it should be to:\n  - Fix an expected problem.\n  - Add context.\n  - Otherwise, use a global handler.\n- Use std::optional, std::expected, or error codes for expected failures.\n\n## Memory Management\n\n- Prefer smart pointers (std::unique_ptr, std::shared_ptr) over raw pointers.\n- Use RAII (Resource Acquisition Is Initialization) principles.\n- Avoid memory leaks by proper resource management.\n- Use std::vector and other standard containers instead of C-style arrays.\n\n## Testing\n\n- Follow the Arrange-Act-Assert convention for tests.\n- Name test variables clearly.\n- Follow the convention: inputX, mockX, actualX, expectedX, etc.\n- Write unit tests for each public function.\n- Use test doubles to simulate dependencies.\n  - Except for third-party dependencies that are not expensive to execute.\n- Write integration tests for each module.\n- Follow the Given-When-Then convention.\n\n## Project Structure\n\n- Use modular architecture\n- Organize code into logical directories:\n  - include/ for header files\n  - src/ for source files\n  - test/ for test files\n  - lib/ for libraries\n  - doc/ for documentation\n- Use CMake or similar build system.\n- Separate interface (.h) from implementation (.cpp).\n- Use namespaces to organize code logically.\n- Create a core namespace for foundational components.\n- Create a utils namespace for utility functions.\n\n## Standard Library\n\n- Use the C++ Standard Library whenever possible.\n- Prefer std::string over C-style strings.\n- Use std::vector, std::map, std::unordered_map, etc. for collections.\n- Use std::optional, std::variant, std::any for modern type safety.\n- Use std::filesystem for file operations.\n- Use std::chrono for time-related operations.\n\n## Concurrency\n\n- Use std::thread, std::mutex, std::lock_guard for thread safety.\n- Prefer task-based parallelism over thread-based parallelism.\n- Use std::atomic for atomic operations.\n- Avoid data races by proper synchronization.\n- Use thread-safe data structures when necessary.\n"},"files":{"llms.txt":"# PyTorch\n\n> PyTorch documentation.\n\n## Docs\n\n- [Tensor Accessors](api/aten/accessors.md)\n- [Tensor Creation](api/aten/creation.md)\n- [ATen: Tensor Library](api/aten/index.md)\n- [Tensor Indexing](api/aten/indexing.md)\n- [Tensor Class](api/aten/tensor.md)\n- [Custom Autograd Functions](api/autograd/custom_functions.md)\n- [Gradient Computation](api/autograd/gradient.md)\n- [Autograd: Automatic Differentiation](api/autograd/index.md)\n- [Gradient Modes](api/autograd/modes.md)\n- [Device and DeviceType](api/c10/device.md)\n- [Device Guards](api/c10/guards.md)\n- [C10: Core Utilities](api/c10/index.md)\n- [Streams](api/c10/streams.md)\n- [Core Types](api/c10/types.md)\n- [Utilities](api/c10/utilities.md)\n- [CUDA Guards](api/cuda/guards.md)\n- [CUDA Support](api/cuda/index.md)\n- [CUDA Streams](api/cuda/streams.md)\n- [CUDA Utility Functions](api/cuda/utilities.md)\n- [DataLoader](api/data/dataloader.md)\n- [Datasets](api/data/datasets.md)\n- [Data Loading (torch::data)](api/data/index.md)\n- [Samplers](api/data/samplers.md)\n- [Transforms](api/data/transforms.md)\n- [C++ API Reference](api/index.md)\n- [Custom Classes](api/library/custom_classes.md)\n- [Torch Library API](api/library/index.md)\n- [Operator Registration](api/library/registration.md)\n- [Library Versioning](api/library/versioning.md)\n- [Activation Functions](api/nn/activation.md)\n- [Containers](api/nn/containers.md)\n- [Convolution Layers](api/nn/convolution.md)\n- [Dropout Layers](api/nn/dropout.md)\n- [Embedding Layers](api/nn/embedding.md)\n- [Functional API](api/nn/functional.md)\n- [Neural Network Modules (torch::nn)](api/nn/index.md)\n- [Linear Layers](api/nn/linear.md)\n- [Loss Functions](api/nn/loss.md)\n- [Normalization Layers](api/nn/normalization.md)\n- [Pooling Layers](api/nn/pooling.md)\n- [Recurrent Layers](api/nn/recurrent.md)\n- [Transformer Layers](api/nn/transformer.md)\n- [Utilities](api/nn/utilities.md)\n- [Adaptive Learning Rate Optimizers](api/optim/adaptive.md)\n- [Gradient Descent Optimizers](api/optim/gradient_descent.md)\n- [Optimizers (torch::optim)](api/optim/index.md)\n- [Learning Rate Schedulers](api/optim/schedulers.md)\n- [Second-Order Optimizers](api/optim/second_order.md)\n- [Archives](api/serialize/archives.md)\n- [Checkpoints](api/serialize/checkpoints.md)\n- [Serialization (torch::serialize)](api/serialize/index.md)\n- [Saving and Loading](api/serialize/save_load.md)\n- [Torch Stable API](api/stable/index.md)\n- [Stable Operators](api/stable/operators.md)\n- [Library Registration Macros](api/stable/registration.md)\n- [Utilities](api/stable/utilities.md)\n- [XPU Support](api/xpu/index.md)\n- [XPU Streams](api/xpu/streams.md)\n- [XPU Utility Functions](api/xpu/utilities.md)\n- [FAQ](faq.md)\n- [The C++ Frontend](frontend.md)\n- [PyTorch C++ API](index.md)\n- [Installing C++ Distributions of PyTorch](installing.md)","llms-full.txt":"# PyTorch\n\n> PyTorch documentation.\n\n\n---\n\n# Tensor Accessors\n\nFor element-wise operations in custom kernels, use *accessors* to avoid\ndynamic dispatch overhead.\n\n## CPU Accessors\n\n```\ntorch::Tensor foo = torch::rand({12, 12});\n\n// Create accessor - validates type and dimensions once\nauto foo_a = foo.accessor<float, 2>();\n\nfloat trace = 0;\nfor (int i = 0; i < foo_a.size(0); i++) {\n trace += foo_a[i][i];\n}\n```\n\n## CUDA Packed Accessors\n\nFor CUDA kernels, use *packed accessors* which copy metadata instead of\npointing to it:\n\n```\n__global__ void kernel(torch::PackedTensorAccessor64<float, 2> foo, float* trace) {\n int i = threadIdx.x;\n gpuAtomicAdd(trace, foo[i][i]);\n}\n\ntorch::Tensor foo = torch::rand({12, 12}).cuda();\nauto foo_a = foo.packed_accessor64<float, 2>();\n\nfloat trace = 0;\nkernel<<<1, 12>>>(foo_a, &trace);\n```\n\nTip\n\nUse `PackedTensorAccessor32` and `packed_accessor32` for 32-bit indexing,\nwhich is faster on CUDA but may overflow for large tensors.\n\n---\n\n# Tensor Creation\n\nFactory functions create new tensors with different initialization patterns.\nAll factory functions follow a general schema:\n\n```\ntorch::<function-name>(<function-specific-options>, <sizes>, <tensor-options>)\n```\n\n## Available Factory Functions\n\n- `torch::zeros` - Tensor filled with zeros\n- `torch::ones` - Tensor filled with ones\n- `torch::empty` - Uninitialized tensor\n- `torch::full` - Tensor filled with a single value\n- `torch::rand` - Uniform random values on [0, 1)\n- `torch::randn` - Standard normal distribution\n- `torch::randint` - Random integers in a range\n- `torch::arange` - Sequence of integers\n- `torch::linspace` - Linearly spaced values\n- `torch::logspace` - Logarithmically spaced values\n- `torch::eye` - Identity matrix\n- `torch::randperm` - Random permutation of integers\n\n## Specifying a Size\n\nFunctions that do not require specific arguments can be invoked with just a\nsize. For example, the following line creates a vector with 5 components:\n\n```\ntorch::Tensor tensor = torch::ones(5);\n```\n\nAn `IntArrayRef` is constructed by specifying the size along each dimension in\ncurly braces. For example, `{2, 3}` for a matrix with two rows and three\ncolumns, `{3, 4, 5}` for a three-dimensional tensor:\n\n```\ntorch::Tensor tensor = torch::randn({3, 4, 5});\nassert(tensor.sizes() == std::vector<int64_t>{3, 4, 5});\n```\n\nYou can also pass an `std::vector<int64_t>` instead of curly braces.\nUse `tensor.size(i)` to access a single dimension.\n\n## Passing Function-Specific Parameters\n\nSome factory functions accept additional parameters. For example, `randint`\ntakes an upper bound on the value for the integers it generates:\n\n```\ntorch::Tensor tensor = torch::randint(/*high=*/10, {5, 5});\n\n// With a lower bound\ntorch::Tensor tensor = torch::randint(/*low=*/3, /*high=*/10, {5, 5});\n```\n\nTip\n\nThe size always follows the function-specific arguments.\n\nAttention\n\nSome functions like `arange` do not need a size at all, since it is fully\ndetermined by the function-specific arguments (the range bounds).\n\n## Configuring Properties with TensorOptions\n\n`TensorOptions` configures the data type, layout, device, and\n`requires_grad` of a new tensor. The construction axes are:\n\n- `dtype`: the data type of the elements (e.g. `kFloat32`, `kInt64`)\n- `layout`: either `kStrided` (dense) or `kSparse`\n- `device`: a compute device (e.g. `kCPU`, `kCUDA`)\n- `requires_grad`: whether to track gradients\n\nAllowed values:\n\n- `dtype`: `kUInt8`, `kInt8`, `kInt16`, `kInt32`, `kInt64`,\n`kFloat32`, `kFloat64`\n- `layout`: `kStrided`, `kSparse`\n- `device`: `kCPU`, or `kCUDA` (with an optional device index)\n- `requires_grad`: `true` or `false`\n\nTip\n\nRust-style shorthands exist for dtypes, like `kF32` instead of\n`kFloat32`. See\n[torch/types.h](https://github.com/pytorch/pytorch/blob/main/torch/csrc/api/include/torch/types.h)\nfor the full list.\n\nHere is an example of creating a `TensorOptions` object:\n\n```\nauto options =\n torch::TensorOptions()\n .dtype(torch::kFloat64)\n .layout(torch::kStrided)\n .device(torch::kCUDA, 1)\n .requires_grad(true);\n\ntorch::Tensor tensor = torch::full({3, 4}, /*value=*/123, options);\n\nassert(tensor.dtype() == torch::kFloat64);\nassert(tensor.layout() == torch::kStrided);\nassert(tensor.device().type() == torch::kCUDA);\nassert(tensor.device().index() == 1);\nassert(tensor.requires_grad());\n```\n\n**Defaults:** Any axis you omit takes its default value: `kFloat32` for dtype,\n`kStrided` for layout, `kCPU` for device, and `false` for\n`requires_grad`. This means you can omit `TensorOptions` entirely:\n\n```\n// A 32-bit float, strided, CPU tensor that does not require a gradient.\ntorch::Tensor tensor = torch::randn({3, 4});\n```\n\n**Shorthand syntax:** For each axis there is a free function in the `torch::`\nnamespace (`torch::dtype()`, `torch::device()`, `torch::layout()`,\n`torch::requires_grad()`). Each returns a `TensorOptions` object that can\nbe further refined with builder methods:\n\n```\n// These are equivalent:\ntorch::ones(10, torch::TensorOptions().dtype(torch::kFloat32))\ntorch::ones(10, torch::dtype(torch::kFloat32))\n\n// Chaining:\ntorch::ones(10, torch::dtype(torch::kFloat32).layout(torch::kStrided))\n```\n\n**Implicit construction:** `TensorOptions` is implicitly constructible from\nindividual values, so when only one axis differs from the default you can\nwrite:\n\n```\ntorch::ones(10, torch::kFloat32)\n```\n\nPutting it all together, a C++ tensor creation call mirrors the Python\nequivalent closely:\n\n```\n# Python\ntorch.randn(3, 4, dtype=torch.float32, device=torch.device('cuda', 1), requires_grad=True)\n```\n\n```\n// C++\ntorch::randn({3, 4}, torch::dtype(torch::kFloat32).device(torch::kCUDA, 1).requires_grad(true))\n```\n\n## Using Externally Created Data\n\nIf you already have tensor data allocated in memory (CPU or CUDA), use\n`from_blob` to view that memory as a `Tensor`:\n\n```\nfloat data[] = {1, 2, 3, 4, 5, 6};\ntorch::Tensor tensor = torch::from_blob(data, {2, 3});\n```\n\nNote\n\nTensors created with `from_blob` cannot be resized because ATen does not\nown the memory.\n\n## Tensor Conversion\n\nUse `to()` to convert tensors between dtypes and devices. The conversion\ncreates a new tensor and does not occur in-place:\n\n```\ntorch::Tensor source = torch::randn({2, 3}, torch::kInt64);\n\n// Convert dtype\ntorch::Tensor float_tensor = source.to(torch::kFloat32);\n\n// Move to GPU (default CUDA device)\ntorch::Tensor gpu_tensor = float_tensor.to(torch::kCUDA);\n\n// Specific GPU device\ntorch::Tensor gpu1_tensor = float_tensor.to(torch::Device(torch::kCUDA, 1));\n\n// Async copy\ntorch::Tensor async_tensor = gpu_tensor.to(torch::kCPU, /*non_blocking=*/true);\n```\n\nAttention\n\nThe result of the conversion is a new tensor pointing to new memory,\nunrelated to the source tensor.\n\n## Scalars and Zero-Dimensional Tensors\n\n`Scalar` represents a single dynamically-typed number. Like a `Tensor`,\n`Scalar` is dynamically typed and can hold any of ATen's number types.\nScalars can be implicitly constructed from C++ number types:\n\n```\nnamespace torch {\nTensor addmm(Scalar beta, const Tensor & self,\n Scalar alpha, const Tensor & mat1,\n const Tensor & mat2);\nScalar sum(const Tensor & self);\n} // namespace torch\n\n// Usage\ntorch::Tensor a = ...;\ntorch::Tensor b = ...;\ntorch::Tensor c = ...;\ntorch::Tensor r = torch::addmm(1.0, a, .5, b, c);\n```\n\nZero-dimensional tensors hold a single value and can reference elements in\nlarger tensors:\n\n```\ntorch::Tensor matrix = torch::rand({10, 20});\nmatrix[1][2] = 4; // matrix[1][2] is a zero-dimensional tensor\n```\n\n---\n\n# ATen: Tensor Library\n\nATen (A Tensor Library) is the foundational tensor and mathematical operation\nlibrary on which all of PyTorch is built. It provides the core `Tensor` class\nand hundreds of mathematical operations that work on tensors.\n\n**When to use ATen directly:**\n\n- When writing low-level tensor operations or custom kernels\n- When you need direct access to tensor data and metadata\n- When working with the PyTorch internals or extending PyTorch\n\n**Basic usage:**\n\n```\n#include <ATen/ATen.h>\n\n// Create tensors\nat::Tensor a = at::ones({2, 3});\nat::Tensor b = at::randn({2, 3});\n\n// Perform operations\nat::Tensor c = a + b;\nat::Tensor d = at::matmul(a.t(), b);\n\n// Move to GPU\nif (at::cuda::is_available()) {\n at::Tensor gpu_tensor = c.to(at::kCUDA);\n}\n```\n\nFor most applications, prefer using the higher-level `torch::` namespace\n(see [Neural Network Modules (torch::nn)](../nn/index.html), [Optimizers (torch::optim)](../optim/index.html)) which provides a more user-friendly API.\n\n## Header Files\n\nThe following headers are part of the ATen public API:\n\n- `ATen/ATen.h` - Main ATen header\n- `ATen/Backend.h` - Backend enumeration\n- `ATen/core/Tensor.h` - Tensor class\n- `ATen/core/ivalue.h` - IValue type (see [C10: Core Utilities](../c10/index.html))\n- `ATen/core/ScalarType.h` - Data type definitions\n- `ATen/TensorOptions.h` - Tensor creation options\n- `ATen/Scalar.h` - Scalar type\n- `ATen/Layout.h` - Tensor layout\n- `ATen/DeviceGuard.h` - Device context management\n- `ATen/native/TensorShape.h` - Tensor shape operations\n- `ATen/cuda/CUDAContext.h` - CUDA context (see [CUDA Support](../cuda/index.html))\n- `ATen/cudnn/Descriptors.h` - cuDNN descriptors\n- `ATen/mkl/Descriptors.h` - MKL descriptors\n\nNote\n\nThe core `at::Tensor` class is defined in a generated header file\n(`TensorBody.h`) that only exists after building PyTorch. The documentation\nbelow describes the API manually.\n\n## ATen Categories\n\n- [Tensor Class](tensor.html)\n- [Tensor Creation](creation.html)\n- [Tensor Indexing](indexing.html)\n- [Tensor Accessors](accessors.html)\n\n---\n\n# Tensor Indexing\n\nThe PyTorch C++ API provides tensor indexing similar to Python. Use\n`torch::indexing` namespace for index types:\n\n```\nusing namespace torch::indexing;\n```\n\nThe main difference from Python is that instead of using the `[]` operator,\nthe C++ API uses the `index` and `index_put_` methods:\n\n- `torch::Tensor::index` -- read elements\n- `torch::Tensor::index_put_` -- write elements\n\n## Index Types\n\nThe `TensorIndex` class accepts six types of indices via implicit constructors:\n\n| Type | C++ | Python equivalent |\n| --- | --- | --- |\n| None (unsqueeze) | `None` | `None` |\n| Ellipsis | `Ellipsis` or `\"...\"` | `...` |\n| Integer | `0`, `1`, `-1` | `0`, `1`, `-1` |\n| Boolean | `true`, `false` | `True`, `False` |\n| Slice | `Slice(start, stop, step)` | `start:stop:step` |\n| Tensor | `torch::tensor({0, 2})` | `torch.tensor([0, 2])` |\n\n## Getter Operations\n\n| Python | C++ |\n| --- | --- |\n| `tensor[0]` | `tensor.index({0})` |\n| `tensor[-1]` | `tensor.index({-1})` |\n| `tensor[1, 2]` | `tensor.index({1, 2})` |\n| `tensor[1, :, 3]` | `tensor.index({1, Slice(), 3})` |\n| `tensor[None]` | `tensor.index({None})` |\n| `tensor[:, None]` | `tensor.index({Slice(), None})` |\n| `tensor[...]` | `tensor.index({Ellipsis})` or `tensor.index({\"...\"})` |\n| `tensor[..., 0]` | `tensor.index({Ellipsis, 0})` |\n| `tensor[1::2]` | `tensor.index({Slice(1, None, 2)})` |\n| `tensor[True]` | `tensor.index({true})` |\n| `tensor[torch.tensor([1, 2])]` | `tensor.index({torch::tensor({1, 2})})` |\n| `tensor[bool_mask]` | `tensor.index({bool_mask})` |\n| `tensor[:, torch.tensor([[0,1],[4,3]])]` | `tensor.index({Slice(), torch::tensor({{0,1},{4,3}})})` |\n| `tensor[cond > 0]` | `tensor.index({cond > 0})` |\n\n## Setter Operations\n\n| Python | C++ |\n| --- | --- |\n| `tensor[0] = 1` | `tensor.index_put_({0}, 1)` |\n| `tensor[1, 2] = 1` | `tensor.index_put_({1, 2}, 1)` |\n| `tensor[1] = torch.arange(5)` | `tensor.index_put_({1}, torch::arange(5))` |\n| `tensor[1::2] = 1` | `tensor.index_put_({Slice(1, None, 2)}, 1)` |\n| `tensor[0, 1::2] = torch.tensor([3., 4.])` | `tensor.index_put_({0, Slice(1, None, 2)}, torch::tensor({3., 4.}))` |\n| `tensor[...] = 0` | `tensor.index_put_({Ellipsis}, 0)` |\n| `tensor[None] = value` | `tensor.index_put_({None}, value)` |\n| `tensor[bool_mask] = 0` | `tensor.index_put_({bool_mask}, 0)` |\n| `tensor[torch.tensor([0, 2])] = value` | `tensor.index_put_({torch::tensor({0, 2})}, value)` |\n| `tensor[1:2, torch.tensor([1,2])] = 0` | `tensor.index_put_({Slice(1, 2), torch::tensor({1, 2})}, 0)` |\n\nThe `index_put_` method also accepts an optional `accumulate` parameter.\nWhen `true`, values are added to existing values instead of replacing them:\n\n```\ntensor.index_put_({mask}, values, /*accumulate=*/true);\n```\n\n## Slice Syntax\n\nThe `Slice` constructor signature is:\n\n```\nSlice(\n std::optional<c10::SymInt> start = std::nullopt,\n std::optional<c10::SymInt> stop = std::nullopt,\n std::optional<c10::SymInt> step = std::nullopt);\n```\n\nPass `None` for open-ended bounds:\n\n| Python | C++ |\n| --- | --- |\n| `:` or `::` | `Slice()` or `Slice(None, None)` |\n| `1:` | `Slice(1, None)` |\n| `:3` | `Slice(None, 3)` |\n| `1:3` | `Slice(1, 3)` |\n| `1:3:2` | `Slice(1, 3, 2)` |\n| `::2` | `Slice(None, None, 2)` |\n\n## Full Example\n\n```\n#include <torch/torch.h>\n\nusing namespace torch::indexing;\n\nauto tensor = torch::arange(2 * 3 * 4).reshape({2, 3, 4});\n\n// Basic indexing\nauto row = tensor.index({0}); // tensor[0]\nauto elem = tensor.index({1, 2, 3}); // tensor[1, 2, 3]\n\n// Slicing\nauto sliced = tensor.index({Slice(), Slice(0, 2)}); // tensor[:, 0:2]\n\n// None (unsqueeze) and Ellipsis\nauto unsqueezed = tensor.index({None}); // tensor[None]\nauto last_dim = tensor.index({Ellipsis, -1}); // tensor[..., -1]\n\n// Boolean mask indexing\nauto mask = tensor > 10;\nauto selected = tensor.index({mask}); // tensor[tensor > 10]\n\n// Integer tensor (fancy) indexing\nauto idx = torch::tensor({0, 2});\nauto gathered = tensor.index({Slice(), idx}); // tensor[:, [0, 2]]\n\n// Setting values\ntensor.index_put_({0, Slice(), 0}, 99); // tensor[0, :, 0] = 99\ntensor.index_put_({mask}, 0); // tensor[tensor > 10] = 0\n```\n\n---\n\n# Tensor Class\n\nThe `at::Tensor` class is the primary tensor class in ATen, representing\na multi-dimensional array with a specific data type and device.\n\n## Tensor\n\nclass at::Tensor\n\nThe primary tensor class in ATen. Represents a multi-dimensional array\nwith a specific data type and device.\n\nTensor()\n\nDefault constructor. Creates an undefined tensor.\n\nint64_t dim() const\n\nReturns the number of dimensions of the tensor.\n\nint64_t size(int64_t dim) const\n\nReturns the size of the tensor at the given dimension.\n\nIntArrayRef sizes() const\n\nReturns the sizes of all dimensions.\n\nIntArrayRef strides() const\n\nReturns the strides of all dimensions.\n\nScalarType scalar_type() const\n\nReturns the data type of the tensor.\n\nDevice device() const\n\nReturns the device where the tensor is stored.\n\nbool is_cuda() const\n\nReturns true if the tensor is on a CUDA device.\n\nbool is_cpu() const\n\nReturns true if the tensor is on CPU.\n\nbool requires_grad() const\n\nReturns true if gradients need to be computed for this tensor.\n\nTensor &requires_grad_(bool requires_grad = true)\n\nSets whether gradients should be computed for this tensor.\n\nTensor to(Device device) const\n\nReturns a tensor on the specified device.\n\nTensor to(ScalarType dtype) const\n\nReturns a tensor with the specified data type.\n\nTensor contiguous() const\n\nReturns a contiguous tensor with the same data.\n\nvoid *data_ptr() const\n\nReturns a pointer to the underlying data.\n\n**Example:**\n\n```\n#include <ATen/ATen.h>\n\nat::Tensor a = at::ones({2, 2}, at::kInt);\nat::Tensor b = at::randn({2, 2});\nauto c = a + b.to(at::kInt);\n```\n\n## TensorOptions\n\nclass at::TensorOptions\n\nA class to specify options for tensor creation, including dtype, device,\nlayout, and requires_grad.\n\nTensorOptions()\n\nDefault constructor.\n\nTensorOptions dtype(ScalarType dtype) const\n\nReturns options with the specified data type.\n\nTensorOptions device(Device device) const\n\nReturns options with the specified device.\n\nTensorOptions layout(Layout layout) const\n\nReturns options with the specified layout.\n\nTensorOptions requires_grad(bool requires_grad) const\n\nReturns options with the specified requires_grad setting.\n\n**Example:**\n\n```\nauto options = at::TensorOptions()\n .dtype(at::kFloat)\n .device(at::kCUDA, 0)\n .requires_grad(true);\n\nat::Tensor tensor = at::zeros({3, 4}, options);\n```\n\n## Scalar\n\nclass at::Scalar\n\nRepresents a scalar value that can be converted to various numeric types.\n\nScalar(int64_t v)\n\nConstruct from an integer.\n\nScalar(double v)\n\nConstruct from a double.\n\ntemplate<typename T>\nT to() const\n\nConvert to the specified type.\n\nbool isIntegral(bool includeBool = false) const\n\nReturns true if the scalar is an integral type.\n\nbool isFloatingPoint() const\n\nReturns true if the scalar is a floating point type.\n\n## ScalarType\n\nenum class at::ScalarType\n\nEnumeration of data types supported by tensors.\n\nenumerator Byte\n\n8-bit unsigned integer (uint8_t)\n\nenumerator Char\n\n8-bit signed integer (int8_t)\n\nenumerator Short\n\n16-bit signed integer (int16_t)\n\nenumerator Int\n\n32-bit signed integer (int32_t)\n\nenumerator Long\n\n64-bit signed integer (int64_t)\n\nenumerator Half\n\n16-bit floating point (float16)\n\nenumerator Float\n\n32-bit floating point (float)\n\nenumerator Double\n\n64-bit floating point (double)\n\nenumerator Bool\n\nBoolean type\n\nenumerator BFloat16\n\nBrain floating point (bfloat16)\n\nConvenience constants:\n\n- `at::kByte`, `at::kChar`, `at::kShort`, `at::kInt`, `at::kLong`\n- `at::kHalf`, `at::kFloat`, `at::kDouble`, `at::kBFloat16`\n- `at::kBool`\n\n## DeviceGuard\n\nclass DeviceGuard\n\nRAII guard that sets a certain default device in its constructor, and changes it back to the device that was originally active upon destruction.\n\nThe device is always reset to the one that was active at the time of construction of the guard. Even if you `set_device` after construction, the destructor will still reset the device to the one that was active at construction time.\n\nThis device guard does NOT have an uninitialized state; it is guaranteed to reset a device on exit. If you are in a situation where you *might* want to setup a guard (i.e., are looking for the moral equivalent of std::optional<DeviceGuard>), see [OptionalDeviceGuard](../c10/guards.html#PyTorchclassc10_1_1_optional_device_guard).\n\nPublic Functions\n\nexplicit DeviceGuard() = delete\n\nNo default constructor; see Note [Omitted default constructor from RAII].\n\ninline explicit DeviceGuard([Device](../c10/device.html#_CPPv4N3c106DeviceE) device)\n\nSet the current device to the passed [Device](../c10/device.html#PyTorchstructc10_1_1_device).\n\ninline explicit DeviceGuard([Device](../c10/device.html#_CPPv4N3c106DeviceE) device, const impl::DeviceGuardImplInterface *impl)\n\nThis constructor is for testing only.\n\n~DeviceGuard() = default\n\nDeviceGuard(const DeviceGuard&) = delete\n\nCopy is disallowed.\n\nDeviceGuard &operator=(const DeviceGuard&) = delete\n\nDeviceGuard(DeviceGuard &&other) = delete\n\nMove is disallowed, as [DeviceGuard](../c10/guards.html#PyTorchclassc10_1_1_device_guard) does not have an uninitialized state, which is required for moves on types with nontrivial destructors.\n\nDeviceGuard &operator=(DeviceGuard &&other) = delete\n\ninline void reset_device(at::Device device)\n\nSets the device to the given one.\n\nThe specified device must be consistent with the device type originally specified during guard construction.\n\nTODO: The consistency check here is inconsistent with StreamGuard's behavior with set_stream, where a stream on a different device than the original one isn't an error; we just reset the stream and then switch devices.\n\ninline void reset_device(at::Device device, const impl::DeviceGuardImplInterface *impl)\n\nThis method is for testing only.\n\ninline void set_index(DeviceIndex index)\n\nSets the device index to the given one.\n\nThe device type is inferred from the original device type the guard was constructed with.\n\ninline [Device](../c10/device.html#_CPPv4N3c106DeviceE) original_device() const\n\nReturns the device that was set at the time the guard was constructed.\n\ninline [Device](../c10/device.html#_CPPv4N3c106DeviceE) current_device() const\n\nReturns the most recent device that was set using this device guard, either from construction, or via set_device.\n\n**Example:**\n\n```\n{\n c10::DeviceGuard guard(at::Device(at::kCUDA, 1));\n // Operations here run on CUDA device 1\n auto tensor = at::zeros({2, 2});\n}\n// Previous device is restored\n```\n\n## Layout\n\nenum class at::Layout\n\nEnumeration of tensor memory layouts.\n\nenumerator Strided\n\nDense tensor with strides.\n\nenumerator Sparse\n\nSparse tensor (COO format).\n\nenumerator SparseCsr\n\nSparse tensor in CSR format.\n\nenumerator SparseCsc\n\nSparse tensor in CSC format.\n\n---\n\n# Custom Autograd Functions\n\nPyTorch allows you to define custom autograd functions with custom forward\nand backward implementations.\n\n## Function Base Class\n\ntemplate<class T>\nstruct Function\n\nTo use custom autograd operations, implement a Function subclass with static forward and backward functions:\n\n`forward` can take as many arguments as you want and should return either a variable list or a Variable. Use of any direct Variable arguments will be registered in the graph but no vectors/sets or any other data structures will be traversed. You can use std::optional<Tensor> as one of the arguments and it will be registered as a variable in the graph if the argument has a value. It should take a pointer to `torch::autograd::AutogradContext` as the first argument. Variables can be saved in the `ctx` using `ctx->save_for_backward` (see `torch::autograd::AutogradContext::save_for_backward`) and other data can be saved in the `ctx->saved_data` map (see `torch::autograd::AutogradContext::saved_data`) in the form of `<std::string, at::IValue>` pairs.\n\n`backward` should take a pointer to `torch::autograd::AutogradContext` and a variable list containing as many Variables as there were outputs from `forward` as arguments. It should return as many Variables as there were inputs with each of them containing the gradient w.r.t. its corresponding input. Variables saved in `forward` can be accessed with `ctx->get_saved_variables` (see `torch::autograd::AutogradContext::get_saved_variables`) and other saved data can be accessed from `ctx->saved_data`. To enable compiled autograd support (torch.compile for backward) for your custom autograd operation, you can set MyFunction::is_traceable (see Function::istraceable notes below).\n\nFor example:\n\n```\nclass MyFunction : public Function<MyFunction> {\n public:\n static constexpr bool is_traceable = true;\n\n static variable_list forward(AutogradContext *ctx, int n, Variable var) {\n // Save data for backward in context\n ctx->saved_data[\"n\"] = n;\n var.mul_(n);\n // Mark var as modified by inplace operation\n ctx->mark_dirty({var});\n return {var};\n }\n\n static variable_list backward(AutogradContext *ctx, variable_list\n grad_output) {\n // Use data saved in forward\n auto n = ctx->saved_data[\"n\"].toInt();\n return {grad_output[0]*n};\n }\n};\n```\n\nTo use `MyFunction`:\n\n```\nVariable x;\nauto y = MyFunction::apply(6, x);\n// Example backward call\ny[0].sum().backward();\n```\n\nPublic Static Functions\n\ntemplate<typename X = T, typename ...Args>\nstatic auto apply(Args&&... args) -> std::enable_if_t<std::is_same_v<X, T>, forward_t<X, Args...>>\n\nPublic Static Attributes\n\nstatic constexpr bool is_traceable = false\n\n## AutogradContext\n\nstruct AutogradContext\n\nContext to save information during `forward` that can be accessed in `backward` in custom autograd operations (see `torch::autograd::Function` for details).\n\nPublic Functions\n\nAutogradContext() = default\n\nAutogradContext(const AutogradContext &other) = delete\n\nAutogradContext &operator=(const AutogradContext &other) = delete\n\nAutogradContext(AutogradContext &&other) = delete\n\nAutogradContext &operator=(AutogradContext &&other) = delete\n\n~AutogradContext() = default\n\nAutogradContext(PackedArgs &packed_args)\n\nvoid save_for_backward(variable_list to_save)\n\nSaves the list of variables for a future call to `backward`.\n\nThis should be called at most once from inside of `forward`.\n\nvoid mark_dirty(const variable_list &inputs)\n\nMarks variables in the list as modified in an in-place operation.\n\nThis should be called at most once from inside of `forward` and all arguments should be inputs.\n\nvoid mark_non_differentiable(const variable_list &outputs)\n\nMarks outputs in the list as not requiring gradients.\n\nThis should be called at most once from inside of `forward` and all arguments should be outputs.\n\nvoid set_materialize_grads(bool value)\n\nvariable_list get_saved_variables() const\n\nGet the list of variables that were saved in `forward` using `save_for_backward()`.\n\nBefore returning them to the user, a check is made to ensure that they were not modified by any in-place operations.\n\nconst std::unordered_set<at::TensorImpl*> &get_and_bump_dirty() const\n\nconst std::unordered_set<at::TensorImpl*> &get_non_differentiable() const\n\nbool needs_input_grad(size_t output_edge_index) const\n\nExpose the Node's `task_should_compute_output` method to the cpp custom autograd Function as `needs_input_grad`.\n\nbool needs_input_grad(std::initializer_list<IndexRange> idxs) const\n\nPublic Members\n\nska::flat_hash_map<std::string, at::IValue> saved_data\n\nCan be used to save non-variable data for `backward`.\n\n## Creating Custom Functions\n\nTo create a custom autograd function, inherit from `torch::autograd::Function`\nand implement the static `forward` and `backward` methods:\n\n**Example:**\n\n```\nclass MyReLU : public torch::autograd::Function<MyReLU> {\n public:\n static torch::Tensor forward(\n torch::autograd::AutogradContext* ctx,\n torch::Tensor input) {\n ctx->save_for_backward({input});\n return input.clamp_min(0);\n }\n\n static torch::autograd::variable_list backward(\n torch::autograd::AutogradContext* ctx,\n torch::autograd::variable_list grad_outputs) {\n auto saved = ctx->get_saved_variables();\n auto input = saved[0];\n auto grad_output = grad_outputs[0];\n auto grad_input = grad_output * (input > 0).to(grad_output.dtype());\n return {grad_input};\n }\n};\n\n// Usage\nauto output = MyReLU::apply(input);\n```\n\n## Custom Kernels and AutoDispatchBelowADInplaceOrView\n\nFor users implementing custom kernels who want to redispatch below `Autograd` dispatch\nkeys, use `at::AutoDispatchBelowADInplaceOrView` instead of `InferenceMode`:\n\n```\nclass ROIAlignFunction : public torch::autograd::Function<ROIAlignFunction> {\n public:\n static torch::autograd::variable_list forward(\n torch::autograd::AutogradContext* ctx,\n const torch::autograd::Variable& input,\n const torch::autograd::Variable& rois,\n double spatial_scale,\n int64_t pooled_height,\n int64_t pooled_width,\n int64_t sampling_ratio,\n bool aligned) {\n ctx->saved_data[\"spatial_scale\"] = spatial_scale;\n ctx->saved_data[\"pooled_height\"] = pooled_height;\n ctx->saved_data[\"pooled_width\"] = pooled_width;\n ctx->saved_data[\"sampling_ratio\"] = sampling_ratio;\n ctx->saved_data[\"aligned\"] = aligned;\n ctx->saved_data[\"input_shape\"] = input.sizes();\n ctx->save_for_backward({rois});\n\n at::AutoDispatchBelowADInplaceOrView guard;\n auto result = roi_align(\n input, rois, spatial_scale, pooled_height,\n pooled_width, sampling_ratio, aligned);\n return {result};\n }\n};\n```\n\nFor customized inplace and view kernels, see the\n[custom kernel tutorial](https://pytorch.org/tutorials/advanced/cpp_extension.html#backward-pass)\nfor more details.\n\n---\n\n# Gradient Computation\n\nPyTorch provides functions for computing gradients of tensors with respect\nto graph leaves.\n\n## Gradient Functions\n\nvoid torch::autograd::backward(const variable_list &tensors, const variable_list &grad_tensors = {}, std::optional<bool> retain_graph = std::nullopt, bool create_graph = false, const variable_list &inputs = {})\n\nComputes the sum of gradients of given tensors with respect to graph leaves.\n\nThe graph is differentiated using the chain rule. If any of `tensors`\nare non-scalar (i.e. their data has more than one element) and require\ngradient, then the Jacobian-vector product would be computed, in this case\nthe function additionally requires specifying `grad_tensors`. It should be a\nsequence of matching length, that contains the \"vector\" in the\nJacobian-vector product, usually the gradient of the differentiated function\nw.r.t. corresponding tensors (`torch::Tensor()` is an acceptable value for\nall tensors that don't need gradient tensors).\n\nThis function accumulates gradients in the leaves -- you might need to zero\nthem before calling it.\n\n:param tensors: Tensors of which the derivative will be computed.\n:param grad_tensors: The \"vector\" in the Jacobian-vector product, usually\ngradients w.r.t. each element of corresponding tensors.\n`torch::Tensor()` values can be specified for scalar Tensors or ones\nthat don't require grad. If a `torch::Tensor()` value would be\nacceptable for all grad_tensors, then this argument is optional.\n:param retain_graph: If `false`, the graph used to compute the grad will\nbe freed. Note that in nearly all cases setting this option to `true`\nis not needed and often can be worked around in a much more efficient\nway. Defaults to the value of `create_graph`.\n:param create_graph: If `true`, graph of the derivative will be\nconstructed, allowing to compute higher order derivative products.\nDefaults to `false`.\n:param inputs: Inputs w.r.t. which the gradient will be accumulated into\n`at::Tensor::grad`. All other Tensors will be ignored. If not\nprovided, the gradient is accumulated into all the leaf Tensors that\nwere used to compute `tensors`.\n\nvariable_list torch::autograd::grad(const variable_list &outputs, const variable_list &inputs, const variable_list &grad_outputs = {}, std::optional<bool> retain_graph = std::nullopt, bool create_graph = false, bool allow_unused = false)\n\nComputes and returns the sum of gradients of outputs with respect to the inputs.\n\n`grad_outputs` should be a sequence of length matching `output` containing the \"vector\" in Jacobian-vector product, usually the pre-computed gradients w.r.t. each of the outputs. If an output doesn't require_grad, then the gradient can be `torch::Tensor()`).\n\nParameters:\n\n- **outputs** - outputs of the differentiated function.\n- **inputs** - Inputs w.r.t. which the gradient will be returned (and not accumulated into `at::Tensor::grad`).\n- **grad_outputs** - The \"vector\" in the Jacobian-vector product. Usually gradients w.r.t. each output. `torch::Tensor()` values can be specified for scalar Tensors or ones that don't require grad. If a `torch::Tensor()` value would be acceptable for all grad_tensors, then this argument is optional. Default: `{}`.\n- **retain_graph** - If `false`, the graph used to compute the grad will be freed. Note that in nearly all cases setting this option to `true` is not needed and often can be worked around in a much more efficient way. Defaults to the value of `create_graph`.\n- **create_graph** - If `true`, graph of the derivative will be constructed, allowing to compute higher order derivative products. Default: `false`.\n- **allow_unused** - If `false`, specifying inputs that were not used when computing outputs (and therefore their grad is always zero) is an error. Defaults to `false`.\n\n**Example:**\n\n```\n#include <torch/torch.h>\n\nauto x = torch::randn({2, 2}, torch::requires_grad());\nauto y = x * x;\nauto z = y.sum();\n\n// Compute gradients\nz.backward();\nstd::cout << x.grad() << std::endl;\n\n// Or use grad() for specific outputs\nauto grads = torch::autograd::grad({z}, {x});\n```\n\n## Tensor Gradient Methods\n\nTensors have built-in methods for gradient computation:\n\n```\n// Enable gradient tracking\nauto x = torch::randn({2, 2}).requires_grad_(true);\n\n// Check if gradient is required\nbool needs_grad = x.requires_grad();\n\n// Access the gradient after backward\nauto grad = x.grad();\n\n// Detach from computation graph\nauto x_detached = x.detach();\n```\n\n---\n\n# Autograd: Automatic Differentiation\n\nPyTorch's autograd system provides automatic differentiation for all operations\non tensors. It records operations on tensors to build a computational graph,\nthen computes gradients automatically via backpropagation.\n\n**When to use Autograd:**\n\n- When training neural networks (gradients are computed automatically)\n- When implementing custom backward passes for specialized operations\n- When you need fine-grained control over gradient computation\n\n**Basic usage:**\n\n```\n#include <torch/torch.h>\n\n// Create tensor with gradient tracking\nauto x = torch::randn({2, 2}, torch::requires_grad());\nauto y = x * x;\nauto z = y.sum();\n\n// Compute gradients via backpropagation\nz.backward();\nstd::cout << x.grad() << std::endl; // dz/dx = 2x\n\n// Disable gradient tracking for inference\n{\n torch::NoGradGuard no_grad;\n auto result = model->forward(input); // No gradients computed\n}\n```\n\n## Header Files\n\n- `torch/csrc/autograd/autograd.h` - High-level autograd API\n- `torch/csrc/autograd/function.h` - Custom autograd functions\n- `torch/csrc/autograd/grad_mode.h` - Gradient computation modes\n- `torch/csrc/api/include/torch/autograd.h` - C++ Frontend autograd\n\n## Autograd Categories\n\n- [Gradient Computation](gradient.html)\n- [Custom Autograd Functions](custom_functions.html)\n- [Gradient Modes](modes.html)\n\n---\n\n# Gradient Modes\n\nPyTorch provides RAII guards to control gradient computation behavior.\n\n## NoGradGuard\n\nclass torch::NoGradGuard\n\nRAII guard that disables gradient computation within its scope.\n\nNoGradGuard()\n\nDisables gradient computation.\n\n~NoGradGuard()\n\nRestores previous gradient mode.\n\n**Example:**\n\n```\n{\n torch::NoGradGuard no_grad;\n // No gradients computed in this scope\n auto result = model->forward(input);\n}\n```\n\n## InferenceMode\n\n`c10::InferenceMode` is a RAII guard analogous to `NoGradMode` designed for use\nwhen you are certain your operations will have no interactions with autograd\n(e.g., model inference). Compared to `NoGradMode`, code run under this mode gets\nbetter performance by disabling autograd-related work like view tracking and version\ncounter bumps. However, tensors created inside `InferenceMode` have more limitations\nwhen interacting with the autograd system.\n\nclass c10::InferenceMode\n\nRAII guard that enables inference mode for optimized inference.\nThis is more efficient than NoGradGuard for inference-only workloads.\n\nexplicit InferenceMode(bool enabled = true)\n\nEnables or disables inference mode.\n\n**Inference Tensors:**\n\n`InferenceMode` can be enabled for a given block of code. Inside `InferenceMode`,\nall newly allocated (non-view) tensors are marked as inference tensors. Inference tensors:\n\n- Do not have a version counter, so an error will be raised if you try to read their version\n(e.g., because you saved this tensor for backward).\n- Are immutable outside `InferenceMode`. An error will be raised if you try to:\n\n- Mutate their data outside InferenceMode.\n- Mutate them to `requires_grad=True` outside InferenceMode.\n- To work around this, make a clone outside `InferenceMode` to get a normal tensor before mutating.\n\nA non-view tensor is an inference tensor if and only if it was allocated inside `InferenceMode`.\nA view tensor is an inference tensor if and only if it is a view of an inference tensor.\n\n**Performance Guarantees:**\n\nInside an `InferenceMode` block:\n\n- Like `NoGradMode`, all operations do not record `grad_fn` even if their inputs have\n`requires_grad=True`. This applies to both inference tensors and normal tensors.\n- View operations on inference tensors do not perform view tracking. View and non-view\ninference tensors are indistinguishable.\n- Inplace operations on inference tensors are guaranteed not to do a version bump.\n\nFor more implementation details, see the [RFC-0011-InferenceMode](https://github.com/pytorch/rfcs/pull/17).\n\n**Basic Example:**\n\n```\n{\n c10::InferenceMode guard;\n // Optimized inference without gradient tracking\n auto result = model->forward(input);\n}\n```\n\n**Inference Workload Example:**\n\n```\nc10::InferenceMode guard;\nmodel.load_jit(saved_model);\nauto inputs = preprocess_tensors(data);\nauto out = model.forward(inputs);\nauto outputs = postprocess_tensors(out);\n```\n\n**Nested InferenceMode:**\n\nUnlike some other guards, `InferenceMode` can be nested with different enabled/disabled states:\n\n```\n{\n c10::InferenceMode guard(true);\n // InferenceMode is on\n {\n c10::InferenceMode guard(false);\n // InferenceMode is off\n }\n // InferenceMode is on\n}\n// InferenceMode is off\n```\n\n## InferenceMode vs NoGradMode\n\n`InferenceMode` is preferred over `NoGradMode` for pure inference workloads because\nit provides better performance. Key differences:\n\n- Both guards affect tensor execution to skip work not related to inference, but\n`InferenceMode` also affects tensor creation while `NoGradMode` doesn't.\n- Tensors created inside `InferenceMode` are marked as inference tensors with\ncertain limitations that apply after exiting `InferenceMode`.\n- `InferenceMode` can be nested with enabled/disabled states.\n\n## Migrating from AutoNonVariableTypeMode\n\nThe legacy `AutoNonVariableTypeMode` guard (now renamed to\n`AutoDispatchBelowADInplaceOrView`) was commonly used for inference workloads\nbut is unsafe -- it can silently bypass safety checks and produce wrong results.\n\n- **For inference-only workloads** (e.g. loading a pretrained JIT model and\nrunning inference in C++ runtime), use `c10::InferenceMode` as a drop-in\nreplacement. It preserves the performance characteristics while providing\ncorrectness guarantees.\n- **For custom autograd kernels** that need to redispatch below the Autograd\ndispatch key, use `AutoDispatchBelowADInplaceOrView` instead:\n\n```\nclass ROIAlignFunction : public torch::autograd::Function<ROIAlignFunction> {\n public:\n static torch::autograd::variable_list forward(\n torch::autograd::AutogradContext* ctx,\n const torch::autograd::Variable& input,\n const torch::autograd::Variable& rois,\n double spatial_scale, int64_t pooled_height,\n int64_t pooled_width, int64_t sampling_ratio, bool aligned) {\n ctx->saved_data[\"spatial_scale\"] = spatial_scale;\n ctx->save_for_backward({rois});\n at::AutoDispatchBelowADInplaceOrView guard;\n auto result = roi_align(input, rois, spatial_scale,\n pooled_height, pooled_width, sampling_ratio, aligned);\n return {result};\n }\n};\n```\n\n---\n\n# Device and DeviceType\n\nPyTorch provides device abstractions for writing code that works across\nCPU, CUDA, and other backends.\n\n## Device\n\nstruct Device\n\nRepresents a compute device on which a tensor is located.\n\nA device is uniquely identified by a type, which specifies the type of machine it is (e.g. CPU or CUDA GPU), and a device index or ordinal, which identifies the specific compute device when there is more than one of a certain type. The device index is optional, and in its defaulted state represents (abstractly) \"the current device\". Further, there are two constraints on the value of the device index, if one is explicitly stored:\n\n1. A negative index represents the current device, a non-negative index represents a specific, concrete device,\n2. When the device type is CPU, the device index must be zero.\n\nPublic Types\n\nusing Type = DeviceType\n\nPublic Functions\n\ninline Device(DeviceType type, DeviceIndex index = -1)\n\nConstructs a new `Device` from a `DeviceType` and an optional device index.\n\nDevice(const std::string &device_string)\n\nConstructs a `Device` from a string description, for convenience.\n\nThe string supplied must follow the following schema: `(cpu|cuda)[:<device-index>]` where `cpu` or `cuda` specifies the device type, and `:<device-index>` optionally specifies a device index.\n\ninline bool operator==(const Device &other) const noexcept\n\nReturns true if the type and index of this `Device` matches that of `other`.\n\ninline bool operator!=(const Device &other) const noexcept\n\nReturns true if the type or index of this `Device` differs from that of `other`.\n\ninline void set_index(DeviceIndex index)\n\nSets the device index.\n\ninline DeviceType type() const noexcept\n\nReturns the type of device this is.\n\ninline DeviceIndex index() const noexcept\n\nReturns the optional index.\n\ninline bool has_index() const noexcept\n\nReturns true if the device has a non-default index.\n\ninline bool is_cuda() const noexcept\n\nReturn true if the device is of CUDA type.\n\ninline bool is_privateuseone() const noexcept\n\nReturn true if the device is of PrivateUse1 type.\n\ninline bool is_mps() const noexcept\n\nReturn true if the device is of MPS type.\n\ninline bool is_hip() const noexcept\n\nReturn true if the device is of HIP type.\n\ninline bool is_ve() const noexcept\n\nReturn true if the device is of VE type.\n\ninline bool is_xpu() const noexcept\n\nReturn true if the device is of XPU type.\n\ninline bool is_ipu() const noexcept\n\nReturn true if the device is of IPU type.\n\ninline bool is_xla() const noexcept\n\nReturn true if the device is of XLA type.\n\ninline bool is_mtia() const noexcept\n\nReturn true if the device is of MTIA type.\n\ninline bool is_hpu() const noexcept\n\nReturn true if the device is of HPU type.\n\ninline bool is_lazy() const noexcept\n\nReturn true if the device is of Lazy type.\n\ninline bool is_vulkan() const noexcept\n\nReturn true if the device is of Vulkan type.\n\ninline bool is_metal() const noexcept\n\nReturn true if the device is of Metal type.\n\ninline bool is_maia() const noexcept\n\nReturn true if the device is of MAIA type.\n\ninline bool is_meta() const noexcept\n\nReturn true if the device is of META type.\n\ninline bool is_cpu() const noexcept\n\nReturn true if the device is of CPU type.\n\ninline bool supports_as_strided() const noexcept\n\nReturn true if the device supports arbitrary strides.\n\nstd::string str() const\n\nSame string as returned from operator<<.\n\n**Example:**\n\n```\nc10::Device cpu_device(c10::kCPU);\nc10::Device cuda_device(c10::kCUDA, 0); // CUDA device 0\n\nif (cuda_device.is_cuda()) {\n std::cout << \"Using CUDA device \" << cuda_device.index() << std::endl;\n}\n```\n\n## DeviceType\n\nenum class c10::DeviceType\n\nEnumeration of supported device types.\n\nenumerator CPU = 0\n\nCPU device.\n\nenumerator CUDA = 1\n\nNVIDIA CUDA GPU.\n\nenumerator HIP = 6\n\nAMD HIP GPU.\n\nenumerator XLA = 9\n\nXLA / TPU.\n\nenumerator Vulkan = 10\n\nVulkan GPU.\n\nenumerator Metal = 11\n\nApple Metal GPU.\n\nenumerator XPU = 12\n\nIntel XPU GPU.\n\nenumerator MPS = 13\n\nApple Metal Performance Shaders.\n\nenumerator Meta = 14\n\nMeta tensors (shape only, no data).\n\nenumerator HPU = 15\n\nHabana HPU.\n\nenumerator Lazy = 17\n\nLazy tensors.\n\nenumerator IPU = 18\n\nGraphcore IPU.\n\nenumerator MTIA = 19\n\nMeta training and inference accelerator.\n\nenumerator PrivateUse1 = 20\n\nCustom backend registered via `c10::register_privateuse1_backend()`.\n\nConvenience constants:\n\n- `c10::kCPU`, `c10::kCUDA`, `c10::kHIP`\n- `c10::kXLA`, `c10::kVulkan`, `c10::kMetal`\n- `c10::kXPU`, `c10::kMPS`, `c10::kMeta`\n- `c10::kHPU`, `c10::kLazy`, `c10::kIPU`, `c10::kMTIA`\n- `c10::kPrivateUse1`\n\n---\n\n# Device Guards\n\nC10 provides device-agnostic RAII guards for managing the current device\ncontext. These guards work across all backends (CUDA, XPU, etc.) and\nautomatically restore the previous device when they go out of scope.\n\nFor backend-specific guards, see [CUDA Guards](../cuda/guards.html) and [XPU Support](../xpu/index.html).\n\n## DeviceGuard\n\nclass DeviceGuard\n\nRAII guard that sets a certain default device in its constructor, and changes it back to the device that was originally active upon destruction.\n\nThe device is always reset to the one that was active at the time of construction of the guard. Even if you `set_device` after construction, the destructor will still reset the device to the one that was active at construction time.\n\nThis device guard does NOT have an uninitialized state; it is guaranteed to reset a device on exit. If you are in a situation where you *might* want to setup a guard (i.e., are looking for the moral equivalent of std::optional<DeviceGuard>), see OptionalDeviceGuard.\n\nPublic Functions\n\nexplicit DeviceGuard() = delete\n\nNo default constructor; see Note [Omitted default constructor from RAII].\n\ninline explicit DeviceGuard([Device](device.html#_CPPv4N3c106DeviceE) device)\n\nSet the current device to the passed [Device](device.html#PyTorchstructc10_1_1_device).\n\ninline explicit DeviceGuard([Device](device.html#_CPPv4N3c106DeviceE) device, const [impl](../aten/tensor.html#_CPPv4N3c1011DeviceGuard11DeviceGuardE6DevicePKN4impl24DeviceGuardImplInterfaceE)::DeviceGuardImplInterface *impl)\n\nThis constructor is for testing only.\n\n~DeviceGuard() = default\n\nDeviceGuard(const [DeviceGuard](../aten/tensor.html#_CPPv4N3c1011DeviceGuard11DeviceGuardERK11DeviceGuard)&) = delete\n\nCopy is disallowed.\n\n[DeviceGuard](../aten/tensor.html#_CPPv4N3c1011DeviceGuardE) &operator=(const [DeviceGuard](../aten/tensor.html#_CPPv4N3c1011DeviceGuardE)&) = delete\n\nDeviceGuard([DeviceGuard](../aten/tensor.html#_CPPv4N3c1011DeviceGuard11DeviceGuardERR11DeviceGuard) &&other) = delete\n\nMove is disallowed, as DeviceGuard does not have an uninitialized state, which is required for moves on types with nontrivial destructors.\n\n[DeviceGuard](../aten/tensor.html#_CPPv4N3c1011DeviceGuardE) &operator=([DeviceGuard](../aten/tensor.html#_CPPv4N3c1011DeviceGuardE) &&other) = delete\n\ninline void reset_device(at::Device device)\n\nSets the device to the given one.\n\nThe specified device must be consistent with the device type originally specified during guard construction.\n\nTODO: The consistency check here is inconsistent with StreamGuard's behavior with set_stream, where a stream on a different device than the original one isn't an error; we just reset the stream and then switch devices.\n\ninline void reset_device(at::Device device, const [impl](../aten/tensor.html#_CPPv4N3c1011DeviceGuard12reset_deviceEN2at6DeviceEPKN4impl24DeviceGuardImplInterfaceE)::DeviceGuardImplInterface *impl)\n\nThis method is for testing only.\n\ninline void set_index(DeviceIndex index)\n\nSets the device index to the given one.\n\nThe device type is inferred from the original device type the guard was constructed with.\n\ninline [Device](device.html#_CPPv4N3c106DeviceE) original_device() const\n\nReturns the device that was set at the time the guard was constructed.\n\ninline [Device](device.html#_CPPv4N3c106DeviceE) current_device() const\n\nReturns the most recent device that was set using this device guard, either from construction, or via set_device.\n\n**Example:**\n\n```\n#include <c10/core/DeviceGuard.h>\n\n{\n c10::DeviceGuard guard(c10::Device(c10::kCUDA, 1));\n // All operations here run on CUDA device 1\n}\n// Previous device is restored\n```\n\n## OptionalDeviceGuard\n\nclass OptionalDeviceGuard\n\nA OptionalDeviceGuard is an RAII class that sets a device to some value on initialization, and resets the device to its original value on destruction.\n\nMorally, a OptionalDeviceGuard is equivalent to std::optional<DeviceGuard>, but with extra constructors and methods as appropriate.\n\nBesides its obvious use (optionally applying a DeviceGuard), OptionalDeviceGuard is often also used for the following idiom:\n\nOptionalDeviceGuard g; for (const auto& t : tensors) { g.set_device(t.device()); do_something_with(t); }\n\nThis usage is marginally more efficient than constructing a DeviceGuard every iteration of the for loop, as it avoids an unnecessary device reset.\n\nUnlike DeviceGuard, a OptionalDeviceGuard may be uninitialized. This occurs when you use the nullary constructor, or pass a nullopt to the constructor. Uninitialized OptionalDeviceGuards do *nothing*; they do not know what the original device was and they do not reset on destruction. This is why original_device() and current_device() return std::optional<Device> rather than [Device](device.html#PyTorchstructc10_1_1_device) (as they do in DeviceGuard), and also is why we didn't just provide OptionalDeviceGuard by default and hide DeviceGuard from users.\n\nThe semantics of an OptionalDeviceGuard are exactly explained by thinking of it as an std::optional<DeviceGuard>. In particular, an initialized OptionalDeviceGuard doesn't restore device to its value at construction; it restores device to its value *at initialization*. So if you have the program:\n\n```\nsetDevice(1);\nOptionalDeviceGuard g;\nsetDevice(2);\ng.reset_device(Device(DeviceType::CUDA, 3)); // initializes!\n```\n\n On destruction, g will reset device to 2, rather than 1.\n\nAn uninitialized OptionalDeviceGuard is distinct from a (initialized) DeviceGuard whose original_device_ and current_device_ match, since the DeviceGuard will still reset the device to original_device_.\n\nPublic Functions\n\nexplicit OptionalDeviceGuard() = default\n\nCreate an uninitialized guard. Set the guard later using reset_device.\n\ninline explicit OptionalDeviceGuard([Device](device.html#_CPPv4N3c106DeviceE) device)\n\nInitialize the guard, setting the current device to the passed [Device](device.html#PyTorchstructc10_1_1_device).\n\ninline explicit OptionalDeviceGuard(std::optional<[Device](device.html#_CPPv4N3c106DeviceE)> device)\n\nInitialize the guard if a [Device](device.html#PyTorchstructc10_1_1_device) is passed; otherwise leave the guard uninitialized.\n\ninline explicit OptionalDeviceGuard([Device](device.html#_CPPv4N3c106DeviceE) device, const impl::DeviceGuardImplInterface *impl)\n\nConstructor for testing only.\n\n~OptionalDeviceGuard() = default\n\nOptionalDeviceGuard(const OptionalDeviceGuard&) = delete\n\nCopy is disallowed.\n\nOptionalDeviceGuard &operator=(const OptionalDeviceGuard&) = delete\n\nOptionalDeviceGuard(OptionalDeviceGuard &&other) = delete\n\nMove is disallowed See Note [Explicit initialization of optional fields] and // Note [Move construction for RAII guards is tricky] for rationale.\n\nOptionalDeviceGuard &operator=(OptionalDeviceGuard &&other) = delete\n\ninline void reset_device(at::Device device)\n\nSets the device to the given one.\n\nThe specified device must be consistent with the device type originally specified during guard construction.\n\ninline void reset_device(at::Device device, const impl::DeviceGuardImplInterface *impl)\n\nFor testing only.\n\ninline std::optional<[Device](device.html#_CPPv4N3c106DeviceE)> original_device() const\n\nReturns the device that was set at the time the guard was constructed.\n\ninline std::optional<[Device](device.html#_CPPv4N3c106DeviceE)> current_device() const\n\nReturns the most recent device that was set using this device guard, either from construction, or via reset_device.\n\n**Example:**\n\n```\n#include <c10/core/DeviceGuard.h>\n\nc10::OptionalDeviceGuard guard;\nif (use_gpu) {\n guard.reset_device(c10::Device(c10::kCUDA, 0));\n}\n// Guard only restores device",".cursorrules (cpp.mdc)":"---\ndescription: Guide Cursor to write modern C++ and CMake code with clear structure, RAII, const-correctness, and safe error handling.\nglobs: **/*.c,**/*.cpp,**/*.h,**/*.hpp,**/*.cxx,CMakeLists.txt,*.cmake,conanfile.txt,Makefile,**/*.cc\nalwaysApply: false\n---\n# C++ Programming Guidelines\n\n## Basic Principles\n\n- Use English for all code and documentation.\n- Always declare the type of each variable and function (parameters and return value).\n- Create necessary types and classes.\n- Use Doxygen style comments to document public classes and methods.\n- Don't leave blank lines within a function.\n- Follow the one-definition rule (ODR).\n\n## Nomenclature\n\n- Use PascalCase for classes and structures.\n- Use camelCase for variables, functions, and methods.\n- Use ALL_CAPS for constants and macros.\n- Use snake_case for file and directory names.\n- Use UPPERCASE for environment variables.\n- Avoid magic numbers and define constants.\n- Start each function with a verb.\n- Use verbs for boolean variables. Example: isLoading, hasError, canDelete, etc.\n- Use complete words instead of abbreviations and ensure correct spelling.\n  - Except for standard abbreviations like API, URL, etc.\n  - Except for well-known abbreviations:\n    - i, j, k for loops\n    - err for errors\n    - ctx for contexts\n    - req, res for request/response parameters\n\n## Functions\n\n- Write short functions with a single purpose. Less than 20 instructions.\n- Name functions with a verb and something else.\n- If it returns a boolean, use isX or hasX, canX, etc.\n- If it doesn't return anything (void), use executeX or saveX, etc.\n- Avoid nesting blocks by:\n  - Early checks and returns.\n  - Extraction to utility functions.\n- Use standard library algorithms (std::for_each, std::transform, std::find, etc.) to avoid function nesting.\n- Use lambda functions for simple operations.\n- Use named functions for non-simple operations.\n- Use default parameter values instead of checking for null or nullptr.\n- Reduce function parameters using structs or classes\n  - Use an object to pass multiple parameters.\n  - Use an object to return multiple results.\n  - Declare necessary types for input arguments and output.\n- Use a single level of abstraction.\n\n## Data\n\n- Don't abuse primitive types and encapsulate data in composite types.\n- Avoid data validations in functions and use classes with internal validation.\n- Prefer immutability for data.\n- Use const for data that doesn't change.\n- Use constexpr for compile-time constants.\n- Use std::optional for possibly null values.\n\n## Classes\n\n- Follow SOLID principles.\n- Prefer composition over inheritance.\n- Declare interfaces as abstract classes or concepts.\n- Write small classes with a single purpose.\n  - Less than 200 instructions.\n  - Less than 10 public methods.\n  - Less than 10 properties.\n- Use the Rule of Five (or Rule of Zero) for resource management.\n- Make member variables private and provide getters/setters where necessary.\n- Use const-correctness for member functions.\n\n## Exceptions\n\n- Use exceptions to handle errors you don't expect.\n- If you catch an exception, it should be to:\n  - Fix an expected problem.\n  - Add context.\n  - Otherwise, use a global handler.\n- Use std::optional, std::expected, or error codes for expected failures.\n\n## Memory Management\n\n- Prefer smart pointers (std::unique_ptr, std::shared_ptr) over raw pointers.\n- Use RAII (Resource Acquisition Is Initialization) principles.\n- Avoid memory leaks by proper resource management.\n- Use std::vector and other standard containers instead of C-style arrays.\n\n## Testing\n\n- Follow the Arrange-Act-Assert convention for tests.\n- Name test variables clearly.\n- Follow the convention: inputX, mockX, actualX, expectedX, etc.\n- Write unit tests for each public function.\n- Use test doubles to simulate dependencies.\n  - Except for third-party dependencies that are not expensive to execute.\n- Write integration tests for each module.\n- Follow the Given-When-Then convention.\n\n## Project Structure\n\n- Use modular architecture\n- Organize code into logical directories:\n  - include/ for header files\n  - src/ for source files\n  - test/ for test files\n  - lib/ for libraries\n  - doc/ for documentation\n- Use CMake or similar build system.\n- Separate interface (.h) from implementation (.cpp).\n- Use namespaces to organize code logically.\n- Create a core namespace for foundational components.\n- Create a utils namespace for utility functions.\n\n## Standard Library\n\n- Use the C++ Standard Library whenever possible.\n- Prefer std::string over C-style strings.\n- Use std::vector, std::map, std::unordered_map, etc. for collections.\n- Use std::optional, std::variant, std::any for modern type safety.\n- Use std::filesystem for file operations.\n- Use std::chrono for time-related operations.\n\n## Concurrency\n\n- Use std::thread, std::mutex, std::lock_guard for thread safety.\n- Prefer task-based parallelism over thread-based parallelism.\n- Use std::atomic for atomic operations.\n- Avoid data races by proper synchronization.\n- Use thread-safe data structures when necessary.\n"},"items":[{"name":"llms-full.txt","path":"llms-full.txt","rawUrl":"https://raw.githubusercontent.com/pytorch/cpp/HEAD/llms-full.txt","title":"LLM Full Documentation Context","category":"root-instruction","format":"text","content":"# PyTorch\n\n> PyTorch documentation.\n\n\n---\n\n# Tensor Accessors\n\nFor element-wise operations in custom kernels, use *accessors* to avoid\ndynamic dispatch overhead.\n\n## CPU Accessors\n\n```\ntorch::Tensor foo = torch::rand({12, 12});\n\n// Create accessor - validates type and dimensions once\nauto foo_a = foo.accessor<float, 2>();\n\nfloat trace = 0;\nfor (int i = 0; i < foo_a.size(0); i++) {\n trace += foo_a[i][i];\n}\n```\n\n## CUDA Packed Accessors\n\nFor CUDA kernels, use *packed accessors* which copy metadata instead of\npointing to it:\n\n```\n__global__ void kernel(torch::PackedTensorAccessor64<float, 2> foo, float* trace) {\n int i = threadIdx.x;\n gpuAtomicAdd(trace, foo[i][i]);\n}\n\ntorch::Tensor foo = torch::rand({12, 12}).cuda();\nauto foo_a = foo.packed_accessor64<float, 2>();\n\nfloat trace = 0;\nkernel<<<1, 12>>>(foo_a, &trace);\n```\n\nTip\n\nUse `PackedTensorAccessor32` and `packed_accessor32` for 32-bit indexing,\nwhich is faster on CUDA but may overflow for large tensors.\n\n---\n\n# Tensor Creation\n\nFactory functions create new tensors with different initialization patterns.\nAll factory functions follow a general schema:\n\n```\ntorch::<function-name>(<function-specific-options>, <sizes>, <tensor-options>)\n```\n\n## Available Factory Functions\n\n- `torch::zeros` - Tensor filled with zeros\n- `torch::ones` - Tensor filled with ones\n- `torch::empty` - Uninitialized tensor\n- `torch::full` - Tensor filled with a single value\n- `torch::rand` - Uniform random values on [0, 1)\n- `torch::randn` - Standard normal distribution\n- `torch::randint` - Random integers in a range\n- `torch::arange` - Sequence of integers\n- `torch::linspace` - Linearly spaced values\n- `torch::logspace` - Logarithmically spaced values\n- `torch::eye` - Identity matrix\n- `torch::randperm` - Random permutation of integers\n\n## Specifying a Size\n\nFunctions that do not require specific arguments can be invoked with just a\nsize. For example, the following line creates a vector with 5 components:\n\n```\ntorch::Tensor tensor = torch::ones(5);\n```\n\nAn `IntArrayRef` is constructed by specifying the size along each dimension in\ncurly braces. For example, `{2, 3}` for a matrix with two rows and three\ncolumns, `{3, 4, 5}` for a three-dimensional tensor:\n\n```\ntorch::Tensor tensor = torch::randn({3, 4, 5});\nassert(tensor.sizes() == std::vector<int64_t>{3, 4, 5});\n```\n\nYou can also pass an `std::vector<int64_t>` instead of curly braces.\nUse `tensor.size(i)` to access a single dimension.\n\n## Passing Function-Specific Parameters\n\nSome factory functions accept additional parameters. For example, `randint`\ntakes an upper bound on the value for the integers it generates:\n\n```\ntorch::Tensor tensor = torch::randint(/*high=*/10, {5, 5});\n\n// With a lower bound\ntorch::Tensor tensor = torch::randint(/*low=*/3, /*high=*/10, {5, 5});\n```\n\nTip\n\nThe size always follows the function-specific arguments.\n\nAttention\n\nSome functions like `arange` do not need a size at all, since it is fully\ndetermined by the function-specific arguments (the range bounds).\n\n## Configuring Properties with TensorOptions\n\n`TensorOptions` configures the data type, layout, device, and\n`requires_grad` of a new tensor. The construction axes are:\n\n- `dtype`: the data type of the elements (e.g. `kFloat32`, `kInt64`)\n- `layout`: either `kStrided` (dense) or `kSparse`\n- `device`: a compute device (e.g. `kCPU`, `kCUDA`)\n- `requires_grad`: whether to track gradients\n\nAllowed values:\n\n- `dtype`: `kUInt8`, `kInt8`, `kInt16`, `kInt32`, `kInt64`,\n`kFloat32`, `kFloat64`\n- `layout`: `kStrided`, `kSparse`\n- `device`: `kCPU`, or `kCUDA` (with an optional device index)\n- `requires_grad`: `true` or `false`\n\nTip\n\nRust-style shorthands exist for dtypes, like `kF32` instead of\n`kFloat32`. See\n[torch/types.h](https://github.com/pytorch/pytorch/blob/main/torch/csrc/api/include/torch/types.h)\nfor the full list.\n\nHere is an example of creating a `TensorOptions` object:\n\n```\nauto options =\n torch::TensorOptions()\n .dtype(torch::kFloat64)\n .layout(torch::kStrided)\n .device(torch::kCUDA, 1)\n .requires_grad(true);\n\ntorch::Tensor tensor = torch::full({3, 4}, /*value=*/123, options);\n\nassert(tensor.dtype() == torch::kFloat64);\nassert(tensor.layout() == torch::kStrided);\nassert(tensor.device().type() == torch::kCUDA);\nassert(tensor.device().index() == 1);\nassert(tensor.requires_grad());\n```\n\n**Defaults:** Any axis you omit takes its default value: `kFloat32` for dtype,\n`kStrided` for layout, `kCPU` for device, and `false` for\n`requires_grad`. This means you can omit `TensorOptions` entirely:\n\n```\n// A 32-bit float, strided, CPU tensor that does not require a gradient.\ntorch::Tensor tensor = torch::randn({3, 4});\n```\n\n**Shorthand syntax:** For each axis there is a free function in the `torch::`\nnamespace (`torch::dtype()`, `torch::device()`, `torch::layout()`,\n`torch::requires_grad()`). Each returns a `TensorOptions` object that can\nbe further refined with builder methods:\n\n```\n// These are equivalent:\ntorch::ones(10, torch::TensorOptions().dtype(torch::kFloat32))\ntorch::ones(10, torch::dtype(torch::kFloat32))\n\n// Chaining:\ntorch::ones(10, torch::dtype(torch::kFloat32).layout(torch::kStrided))\n```\n\n**Implicit construction:** `TensorOptions` is implicitly constructible from\nindividual values, so when only one axis differs from the default you can\nwrite:\n\n```\ntorch::ones(10, torch::kFloat32)\n```\n\nPutting it all together, a C++ tensor creation call mirrors the Python\nequivalent closely:\n\n```\n# Python\ntorch.randn(3, 4, dtype=torch.float32, device=torch.device('cuda', 1), requires_grad=True)\n```\n\n```\n// C++\ntorch::randn({3, 4}, torch::dtype(torch::kFloat32).device(torch::kCUDA, 1).requires_grad(true))\n```\n\n## Using Externally Created Data\n\nIf you already have tensor data allocated in memory (CPU or CUDA), use\n`from_blob` to view that memory as a `Tensor`:\n\n```\nfloat data[] = {1, 2, 3, 4, 5, 6};\ntorch::Tensor tensor = torch::from_blob(data, {2, 3});\n```\n\nNote\n\nTensors created with `from_blob` cannot be resized because ATen does not\nown the memory.\n\n## Tensor Conversion\n\nUse `to()` to convert tensors between dtypes and devices. The conversion\ncreates a new tensor and does not occur in-place:\n\n```\ntorch::Tensor source = torch::randn({2, 3}, torch::kInt64);\n\n// Convert dtype\ntorch::Tensor float_tensor = source.to(torch::kFloat32);\n\n// Move to GPU (default CUDA device)\ntorch::Tensor gpu_tensor = float_tensor.to(torch::kCUDA);\n\n// Specific GPU device\ntorch::Tensor gpu1_tensor = float_tensor.to(torch::Device(torch::kCUDA, 1));\n\n// Async copy\ntorch::Tensor async_tensor = gpu_tensor.to(torch::kCPU, /*non_blocking=*/true);\n```\n\nAttention\n\nThe result of the conversion is a new tensor pointing to new memory,\nunrelated to the source tensor.\n\n## Scalars and Zero-Dimensional Tensors\n\n`Scalar` represents a single dynamically-typed number. Like a `Tensor`,\n`Scalar` is dynamically typed and can hold any of ATen's number types.\nScalars can be implicitly constructed from C++ number types:\n\n```\nnamespace torch {\nTensor addmm(Scalar beta, const Tensor & self,\n Scalar alpha, const Tensor & mat1,\n const Tensor & mat2);\nScalar sum(const Tensor & self);\n} // namespace torch\n\n// Usage\ntorch::Tensor a = ...;\ntorch::Tensor b = ...;\ntorch::Tensor c = ...;\ntorch::Tensor r = torch::addmm(1.0, a, .5, b, c);\n```\n\nZero-dimensional tensors hold a single value and can reference elements in\nlarger tensors:\n\n```\ntorch::Tensor matrix = torch::rand({10, 20});\nmatrix[1][2] = 4; // matrix[1][2] is a zero-dimensional tensor\n```\n\n---\n\n# ATen: Tensor Library\n\nATen (A Tensor Library) is the foundational tensor and mathematical operation\nlibrary on which all of PyTorch is built. It provides the core `Tensor` class\nand hundreds of mathematical operations that work on tensors.\n\n**When to use ATen directly:**\n\n- When writing low-level tensor operations or custom kernels\n- When you need direct access to tensor data and metadata\n- When working with the PyTorch internals or extending PyTorch\n\n**Basic usage:**\n\n```\n#include <ATen/ATen.h>\n\n// Create tensors\nat::Tensor a = at::ones({2, 3});\nat::Tensor b = at::randn({2, 3});\n\n// Perform operations\nat::Tensor c = a + b;\nat::Tensor d = at::matmul(a.t(), b);\n\n// Move to GPU\nif (at::cuda::is_available()) {\n at::Tensor gpu_tensor = c.to(at::kCUDA);\n}\n```\n\nFor most applications, prefer using the higher-level `torch::` namespace\n(see [Neural Network Modules (torch::nn)](../nn/index.html), [Optimizers (torch::optim)](../optim/index.html)) which provides a more user-friendly API.\n\n## Header Files\n\nThe following headers are part of the ATen public API:\n\n- `ATen/ATen.h` - Main ATen header\n- `ATen/Backend.h` - Backend enumeration\n- `ATen/core/Tensor.h` - Tensor class\n- `ATen/core/ivalue.h` - IValue type (see [C10: Core Utilities](../c10/index.html))\n- `ATen/core/ScalarType.h` - Data type definitions\n- `ATen/TensorOptions.h` - Tensor creation options\n- `ATen/Scalar.h` - Scalar type\n- `ATen/Layout.h` - Tensor layout\n- `ATen/DeviceGuard.h` - Device context management\n- `ATen/native/TensorShape.h` - Tensor shape operations\n- `ATen/cuda/CUDAContext.h` - CUDA context (see [CUDA Support](../cuda/index.html))\n- `ATen/cudnn/Descriptors.h` - cuDNN descriptors\n- `ATen/mkl/Descriptors.h` - MKL descriptors\n\nNote\n\nThe core `at::Tensor` class is defined in a generated header file\n(`TensorBody.h`) that only exists after building PyTorch. The documentation\nbelow describes the API manually.\n\n## ATen Categories\n\n- [Tensor Class](tensor.html)\n- [Tensor Creation](creation.html)\n- [Tensor Indexing](indexing.html)\n- [Tensor Accessors](accessors.html)\n\n---\n\n# Tensor Indexing\n\nThe PyTorch C++ API provides tensor indexing similar to Python. Use\n`torch::indexing` namespace for index types:\n\n```\nusing namespace torch::indexing;\n```\n\nThe main difference from Python is that instead of using the `[]` operator,\nthe C++ API uses the `index` and `index_put_` methods:\n\n- `torch::Tensor::index` -- read elements\n- `torch::Tensor::index_put_` -- write elements\n\n## Index Types\n\nThe `TensorIndex` class accepts six types of indices via implicit constructors:\n\n| Type | C++ | Python equivalent |\n| --- | --- | --- |\n| None (unsqueeze) | `None` | `None` |\n| Ellipsis | `Ellipsis` or `\"...\"` | `...` |\n| Integer | `0`, `1`, `-1` | `0`, `1`, `-1` |\n| Boolean | `true`, `false` | `True`, `False` |\n| Slice | `Slice(start, stop, step)` | `start:stop:step` |\n| Tensor | `torch::tensor({0, 2})` | `torch.tensor([0, 2])` |\n\n## Getter Operations\n\n| Python | C++ |\n| --- | --- |\n| `tensor[0]` | `tensor.index({0})` |\n| `tensor[-1]` | `tensor.index({-1})` |\n| `tensor[1, 2]` | `tensor.index({1, 2})` |\n| `tensor[1, :, 3]` | `tensor.index({1, Slice(), 3})` |\n| `tensor[None]` | `tensor.index({None})` |\n| `tensor[:, None]` | `tensor.index({Slice(), None})` |\n| `tensor[...]` | `tensor.index({Ellipsis})` or `tensor.index({\"...\"})` |\n| `tensor[..., 0]` | `tensor.index({Ellipsis, 0})` |\n| `tensor[1::2]` | `tensor.index({Slice(1, None, 2)})` |\n| `tensor[True]` | `tensor.index({true})` |\n| `tensor[torch.tensor([1, 2])]` | `tensor.index({torch::tensor({1, 2})})` |\n| `tensor[bool_mask]` | `tensor.index({bool_mask})` |\n| `tensor[:, torch.tensor([[0,1],[4,3]])]` | `tensor.index({Slice(), torch::tensor({{0,1},{4,3}})})` |\n| `tensor[cond > 0]` | `tensor.index({cond > 0})` |\n\n## Setter Operations\n\n| Python | C++ |\n| --- | --- |\n| `tensor[0] = 1` | `tensor.index_put_({0}, 1)` |\n| `tensor[1, 2] = 1` | `tensor.index_put_({1, 2}, 1)` |\n| `tensor[1] = torch.arange(5)` | `tensor.index_put_({1}, torch::arange(5))` |\n| `tensor[1::2] = 1` | `tensor.index_put_({Slice(1, None, 2)}, 1)` |\n| `tensor[0, 1::2] = torch.tensor([3., 4.])` | `tensor.index_put_({0, Slice(1, None, 2)}, torch::tensor({3., 4.}))` |\n| `tensor[...] = 0` | `tensor.index_put_({Ellipsis}, 0)` |\n| `tensor[None] = value` | `tensor.index_put_({None}, value)` |\n| `tensor[bool_mask] = 0` | `tensor.index_put_({bool_mask}, 0)` |\n| `tensor[torch.tensor([0, 2])] = value` | `tensor.index_put_({torch::tensor({0, 2})}, value)` |\n| `tensor[1:2, torch.tensor([1,2])] = 0` | `tensor.index_put_({Slice(1, 2), torch::tensor({1, 2})}, 0)` |\n\nThe `index_put_` method also accepts an optional `accumulate` parameter.\nWhen `true`, values are added to existing values instead of replacing them:\n\n```\ntensor.index_put_({mask}, values, /*accumulate=*/true);\n```\n\n## Slice Syntax\n\nThe `Slice` constructor signature is:\n\n```\nSlice(\n std::optional<c10::SymInt> start = std::nullopt,\n std::optional<c10::SymInt> stop = std::nullopt,\n std::optional<c10::SymInt> step = std::nullopt);\n```\n\nPass `None` for open-ended bounds:\n\n| Python | C++ |\n| --- | --- |\n| `:` or `::` | `Slice()` or `Slice(None, None)` |\n| `1:` | `Slice(1, None)` |\n| `:3` | `Slice(None, 3)` |\n| `1:3` | `Slice(1, 3)` |\n| `1:3:2` | `Slice(1, 3, 2)` |\n| `::2` | `Slice(None, None, 2)` |\n\n## Full Example\n\n```\n#include <torch/torch.h>\n\nusing namespace torch::indexing;\n\nauto tensor = torch::arange(2 * 3 * 4).reshape({2, 3, 4});\n\n// Basic indexing\nauto row = tensor.index({0}); // tensor[0]\nauto elem = tensor.index({1, 2, 3}); // tensor[1, 2, 3]\n\n// Slicing\nauto sliced = tensor.index({Slice(), Slice(0, 2)}); // tensor[:, 0:2]\n\n// None (unsqueeze) and Ellipsis\nauto unsqueezed = tensor.index({None}); // tensor[None]\nauto last_dim = tensor.index({Ellipsis, -1}); // tensor[..., -1]\n\n// Boolean mask indexing\nauto mask = tensor > 10;\nauto selected = tensor.index({mask}); // tensor[tensor > 10]\n\n// Integer tensor (fancy) indexing\nauto idx = torch::tensor({0, 2});\nauto gathered = tensor.index({Slice(), idx}); // tensor[:, [0, 2]]\n\n// Setting values\ntensor.index_put_({0, Slice(), 0}, 99); // tensor[0, :, 0] = 99\ntensor.index_put_({mask}, 0); // tensor[tensor > 10] = 0\n```\n\n---\n\n# Tensor Class\n\nThe `at::Tensor` class is the primary tensor class in ATen, representing\na multi-dimensional array with a specific data type and device.\n\n## Tensor\n\nclass at::Tensor\n\nThe primary tensor class in ATen. Represents a multi-dimensional array\nwith a specific data type and device.\n\nTensor()\n\nDefault constructor. Creates an undefined tensor.\n\nint64_t dim() const\n\nReturns the number of dimensions of the tensor.\n\nint64_t size(int64_t dim) const\n\nReturns the size of the tensor at the given dimension.\n\nIntArrayRef sizes() const\n\nReturns the sizes of all dimensions.\n\nIntArrayRef strides() const\n\nReturns the strides of all dimensions.\n\nScalarType scalar_type() const\n\nReturns the data type of the tensor.\n\nDevice device() const\n\nReturns the device where the tensor is stored.\n\nbool is_cuda() const\n\nReturns true if the tensor is on a CUDA device.\n\nbool is_cpu() const\n\nReturns true if the tensor is on CPU.\n\nbool requires_grad() const\n\nReturns true if gradients need to be computed for this tensor.\n\nTensor &requires_grad_(bool requires_grad = true)\n\nSets whether gradients should be computed for this tensor.\n\nTensor to(Device device) const\n\nReturns a tensor on the specified device.\n\nTensor to(ScalarType dtype) const\n\nReturns a tensor with the specified data type.\n\nTensor contiguous() const\n\nReturns a contiguous tensor with the same data.\n\nvoid *data_ptr() const\n\nReturns a pointer to the underlying data.\n\n**Example:**\n\n```\n#include <ATen/ATen.h>\n\nat::Tensor a = at::ones({2, 2}, at::kInt);\nat::Tensor b = at::randn({2, 2});\nauto c = a + b.to(at::kInt);\n```\n\n## TensorOptions\n\nclass at::TensorOptions\n\nA class to specify options for tensor creation, including dtype, device,\nlayout, and requires_grad.\n\nTensorOptions()\n\nDefault constructor.\n\nTensorOptions dtype(ScalarType dtype) const\n\nReturns options with the specified data type.\n\nTensorOptions device(Device device) const\n\nReturns options with the specified device.\n\nTensorOptions layout(Layout layout) const\n\nReturns options with the specified layout.\n\nTensorOptions requires_grad(bool requires_grad) const\n\nReturns options with the specified requires_grad setting.\n\n**Example:**\n\n```\nauto options = at::TensorOptions()\n .dtype(at::kFloat)\n .device(at::kCUDA, 0)\n .requires_grad(true);\n\nat::Tensor tensor = at::zeros({3, 4}, options);\n```\n\n## Scalar\n\nclass at::Scalar\n\nRepresents a scalar value that can be converted to various numeric types.\n\nScalar(int64_t v)\n\nConstruct from an integer.\n\nScalar(double v)\n\nConstruct from a double.\n\ntemplate<typename T>\nT to() const\n\nConvert to the specified type.\n\nbool isIntegral(bool includeBool = false) const\n\nReturns true if the scalar is an integral type.\n\nbool isFloatingPoint() const\n\nReturns true if the scalar is a floating point type.\n\n## ScalarType\n\nenum class at::ScalarType\n\nEnumeration of data types supported by tensors.\n\nenumerator Byte\n\n8-bit unsigned integer (uint8_t)\n\nenumerator Char\n\n8-bit signed integer (int8_t)\n\nenumerator Short\n\n16-bit signed integer (int16_t)\n\nenumerator Int\n\n32-bit signed integer (int32_t)\n\nenumerator Long\n\n64-bit signed integer (int64_t)\n\nenumerator Half\n\n16-bit floating point (float16)\n\nenumerator Float\n\n32-bit floating point (float)\n\nenumerator Double\n\n64-bit floating point (double)\n\nenumerator Bool\n\nBoolean type\n\nenumerator BFloat16\n\nBrain floating point (bfloat16)\n\nConvenience constants:\n\n- `at::kByte`, `at::kChar`, `at::kShort`, `at::kInt`, `at::kLong`\n- `at::kHalf`, `at::kFloat`, `at::kDouble`, `at::kBFloat16`\n- `at::kBool`\n\n## DeviceGuard\n\nclass DeviceGuard\n\nRAII guard that sets a certain default device in its constructor, and changes it back to the device that was originally active upon destruction.\n\nThe device is always reset to the one that was active at the time of construction of the guard. Even if you `set_device` after construction, the destructor will still reset the device to the one that was active at construction time.\n\nThis device guard does NOT have an uninitialized state; it is guaranteed to reset a device on exit. If you are in a situation where you *might* want to setup a guard (i.e., are looking for the moral equivalent of std::optional<DeviceGuard>), see [OptionalDeviceGuard](../c10/guards.html#PyTorchclassc10_1_1_optional_device_guard).\n\nPublic Functions\n\nexplicit DeviceGuard() = delete\n\nNo default constructor; see Note [Omitted default constructor from RAII].\n\ninline explicit DeviceGuard([Device](../c10/device.html#_CPPv4N3c106DeviceE) device)\n\nSet the current device to the passed [Device](../c10/device.html#PyTorchstructc10_1_1_device).\n\ninline explicit DeviceGuard([Device](../c10/device.html#_CPPv4N3c106DeviceE) device, const impl::DeviceGuardImplInterface *impl)\n\nThis constructor is for testing only.\n\n~DeviceGuard() = default\n\nDeviceGuard(const DeviceGuard&) = delete\n\nCopy is disallowed.\n\nDeviceGuard &operator=(const DeviceGuard&) = delete\n\nDeviceGuard(DeviceGuard &&other) = delete\n\nMove is disallowed, as [DeviceGuard](../c10/guards.html#PyTorchclassc10_1_1_device_guard) does not have an uninitialized state, which is required for moves on types with nontrivial destructors.\n\nDeviceGuard &operator=(DeviceGuard &&other) = delete\n\ninline void reset_device(at::Device device)\n\nSets the device to the given one.\n\nThe specified device must be consistent with the device type originally specified during guard construction.\n\nTODO: The consistency check here is inconsistent with StreamGuard's behavior with set_stream, where a stream on a different device than the original one isn't an error; we just reset the stream and then switch devices.\n\ninline void reset_device(at::Device device, const impl::DeviceGuardImplInterface *impl)\n\nThis method is for testing only.\n\ninline void set_index(DeviceIndex index)\n\nSets the device index to the given one.\n\nThe device type is inferred from the original device type the guard was constructed with.\n\ninline [Device](../c10/device.html#_CPPv4N3c106DeviceE) original_device() const\n\nReturns the device that was set at the time the guard was constructed.\n\ninline [Device](../c10/device.html#_CPPv4N3c106DeviceE) current_device() const\n\nReturns the most recent device that was set using this device guard, either from construction, or via set_device.\n\n**Example:**\n\n```\n{\n c10::DeviceGuard guard(at::Device(at::kCUDA, 1));\n // Operations here run on CUDA device 1\n auto tensor = at::zeros({2, 2});\n}\n// Previous device is restored\n```\n\n## Layout\n\nenum class at::Layout\n\nEnumeration of tensor memory layouts.\n\nenumerator Strided\n\nDense tensor with strides.\n\nenumerator Sparse\n\nSparse tensor (COO format).\n\nenumerator SparseCsr\n\nSparse tensor in CSR format.\n\nenumerator SparseCsc\n\nSparse tensor in CSC format.\n\n---\n\n# Custom Autograd Functions\n\nPyTorch allows you to define custom autograd functions with custom forward\nand backward implementations.\n\n## Function Base Class\n\ntemplate<class T>\nstruct Function\n\nTo use custom autograd operations, implement a Function subclass with static forward and backward functions:\n\n`forward` can take as many arguments as you want and should return either a variable list or a Variable. Use of any direct Variable arguments will be registered in the graph but no vectors/sets or any other data structures will be traversed. You can use std::optional<Tensor> as one of the arguments and it will be registered as a variable in the graph if the argument has a value. It should take a pointer to `torch::autograd::AutogradContext` as the first argument. Variables can be saved in the `ctx` using `ctx->save_for_backward` (see `torch::autograd::AutogradContext::save_for_backward`) and other data can be saved in the `ctx->saved_data` map (see `torch::autograd::AutogradContext::saved_data`) in the form of `<std::string, at::IValue>` pairs.\n\n`backward` should take a pointer to `torch::autograd::AutogradContext` and a variable list containing as many Variables as there were outputs from `forward` as arguments. It should return as many Variables as there were inputs with each of them containing the gradient w.r.t. its corresponding input. Variables saved in `forward` can be accessed with `ctx->get_saved_variables` (see `torch::autograd::AutogradContext::get_saved_variables`) and other saved data can be accessed from `ctx->saved_data`. To enable compiled autograd support (torch.compile for backward) for your custom autograd operation, you can set MyFunction::is_traceable (see Function::istraceable notes below).\n\nFor example:\n\n```\nclass MyFunction : public Function<MyFunction> {\n public:\n static constexpr bool is_traceable = true;\n\n static variable_list forward(AutogradContext *ctx, int n, Variable var) {\n // Save data for backward in context\n ctx->saved_data[\"n\"] = n;\n var.mul_(n);\n // Mark var as modified by inplace operation\n ctx->mark_dirty({var});\n return {var};\n }\n\n static variable_list backward(AutogradContext *ctx, variable_list\n grad_output) {\n // Use data saved in forward\n auto n = ctx->saved_data[\"n\"].toInt();\n return {grad_output[0]*n};\n }\n};\n```\n\nTo use `MyFunction`:\n\n```\nVariable x;\nauto y = MyFunction::apply(6, x);\n// Example backward call\ny[0].sum().backward();\n```\n\nPublic Static Functions\n\ntemplate<typename X = T, typename ...Args>\nstatic auto apply(Args&&... args) -> std::enable_if_t<std::is_same_v<X, T>, forward_t<X, Args...>>\n\nPublic Static Attributes\n\nstatic constexpr bool is_traceable = false\n\n## AutogradContext\n\nstruct AutogradContext\n\nContext to save information during `forward` that can be accessed in `backward` in custom autograd operations (see `torch::autograd::Function` for details).\n\nPublic Functions\n\nAutogradContext() = default\n\nAutogradContext(const AutogradContext &other) = delete\n\nAutogradContext &operator=(const AutogradContext &other) = delete\n\nAutogradContext(AutogradContext &&other) = delete\n\nAutogradContext &operator=(AutogradContext &&other) = delete\n\n~AutogradContext() = default\n\nAutogradContext(PackedArgs &packed_args)\n\nvoid save_for_backward(variable_list to_save)\n\nSaves the list of variables for a future call to `backward`.\n\nThis should be called at most once from inside of `forward`.\n\nvoid mark_dirty(const variable_list &inputs)\n\nMarks variables in the list as modified in an in-place operation.\n\nThis should be called at most once from inside of `forward` and all arguments should be inputs.\n\nvoid mark_non_differentiable(const variable_list &outputs)\n\nMarks outputs in the list as not requiring gradients.\n\nThis should be called at most once from inside of `forward` and all arguments should be outputs.\n\nvoid set_materialize_grads(bool value)\n\nvariable_list get_saved_variables() const\n\nGet the list of variables that were saved in `forward` using `save_for_backward()`.\n\nBefore returning them to the user, a check is made to ensure that they were not modified by any in-place operations.\n\nconst std::unordered_set<at::TensorImpl*> &get_and_bump_dirty() const\n\nconst std::unordered_set<at::TensorImpl*> &get_non_differentiable() const\n\nbool needs_input_grad(size_t output_edge_index) const\n\nExpose the Node's `task_should_compute_output` method to the cpp custom autograd Function as `needs_input_grad`.\n\nbool needs_input_grad(std::initializer_list<IndexRange> idxs) const\n\nPublic Members\n\nska::flat_hash_map<std::string, at::IValue> saved_data\n\nCan be used to save non-variable data for `backward`.\n\n## Creating Custom Functions\n\nTo create a custom autograd function, inherit from `torch::autograd::Function`\nand implement the static `forward` and `backward` methods:\n\n**Example:**\n\n```\nclass MyReLU : public torch::autograd::Function<MyReLU> {\n public:\n static torch::Tensor forward(\n torch::autograd::AutogradContext* ctx,\n torch::Tensor input) {\n ctx->save_for_backward({input});\n return input.clamp_min(0);\n }\n\n static torch::autograd::variable_list backward(\n torch::autograd::AutogradContext* ctx,\n torch::autograd::variable_list grad_outputs) {\n auto saved = ctx->get_saved_variables();\n auto input = saved[0];\n auto grad_output = grad_outputs[0];\n auto grad_input = grad_output * (input > 0).to(grad_output.dtype());\n return {grad_input};\n }\n};\n\n// Usage\nauto output = MyReLU::apply(input);\n```\n\n## Custom Kernels and AutoDispatchBelowADInplaceOrView\n\nFor users implementing custom kernels who want to redispatch below `Autograd` dispatch\nkeys, use `at::AutoDispatchBelowADInplaceOrView` instead of `InferenceMode`:\n\n```\nclass ROIAlignFunction : public torch::autograd::Function<ROIAlignFunction> {\n public:\n static torch::autograd::variable_list forward(\n torch::autograd::AutogradContext* ctx,\n const torch::autograd::Variable& input,\n const torch::autograd::Variable& rois,\n double spatial_scale,\n int64_t pooled_height,\n int64_t pooled_width,\n int64_t sampling_ratio,\n bool aligned) {\n ctx->saved_data[\"spatial_scale\"] = spatial_scale;\n ctx->saved_data[\"pooled_height\"] = pooled_height;\n ctx->saved_data[\"pooled_width\"] = pooled_width;\n ctx->saved_data[\"sampling_ratio\"] = sampling_ratio;\n ctx->saved_data[\"aligned\"] = aligned;\n ctx->saved_data[\"input_shape\"] = input.sizes();\n ctx->save_for_backward({rois});\n\n at::AutoDispatchBelowADInplaceOrView guard;\n auto result = roi_align(\n input, rois, spatial_scale, pooled_height,\n pooled_width, sampling_ratio, aligned);\n return {result};\n }\n};\n```\n\nFor customized inplace and view kernels, see the\n[custom kernel tutorial](https://pytorch.org/tutorials/advanced/cpp_extension.html#backward-pass)\nfor more details.\n\n---\n\n# Gradient Computation\n\nPyTorch provides functions for computing gradients of tensors with respect\nto graph leaves.\n\n## Gradient Functions\n\nvoid torch::autograd::backward(const variable_list &tensors, const variable_list &grad_tensors = {}, std::optional<bool> retain_graph = std::nullopt, bool create_graph = false, const variable_list &inputs = {})\n\nComputes the sum of gradients of given tensors with respect to graph leaves.\n\nThe graph is differentiated using the chain rule. If any of `tensors`\nare non-scalar (i.e. their data has more than one element) and require\ngradient, then the Jacobian-vector product would be computed, in this case\nthe function additionally requires specifying `grad_tensors`. It should be a\nsequence of matching length, that contains the \"vector\" in the\nJacobian-vector product, usually the gradient of the differentiated function\nw.r.t. corresponding tensors (`torch::Tensor()` is an acceptable value for\nall tensors that don't need gradient tensors).\n\nThis function accumulates gradients in the leaves -- you might need to zero\nthem before calling it.\n\n:param tensors: Tensors of which the derivative will be computed.\n:param grad_tensors: The \"vector\" in the Jacobian-vector product, usually\ngradients w.r.t. each element of corresponding tensors.\n`torch::Tensor()` values can be specified for scalar Tensors or ones\nthat don't require grad. If a `torch::Tensor()` value would be\nacceptable for all grad_tensors, then this argument is optional.\n:param retain_graph: If `false`, the graph used to compute the grad will\nbe freed. Note that in nearly all cases setting this option to `true`\nis not needed and often can be worked around in a much more efficient\nway. Defaults to the value of `create_graph`.\n:param create_graph: If `true`, graph of the derivative will be\nconstructed, allowing to compute higher order derivative products.\nDefaults to `false`.\n:param inputs: Inputs w.r.t. which the gradient will be accumulated into\n`at::Tensor::grad`. All other Tensors will be ignored. If not\nprovided, the gradient is accumulated into all the leaf Tensors that\nwere used to compute `tensors`.\n\nvariable_list torch::autograd::grad(const variable_list &outputs, const variable_list &inputs, const variable_list &grad_outputs = {}, std::optional<bool> retain_graph = std::nullopt, bool create_graph = false, bool allow_unused = false)\n\nComputes and returns the sum of gradients of outputs with respect to the inputs.\n\n`grad_outputs` should be a sequence of length matching `output` containing the \"vector\" in Jacobian-vector product, usually the pre-computed gradients w.r.t. each of the outputs. If an output doesn't require_grad, then the gradient can be `torch::Tensor()`).\n\nParameters:\n\n- **outputs** - outputs of the differentiated function.\n- **inputs** - Inputs w.r.t. which the gradient will be returned (and not accumulated into `at::Tensor::grad`).\n- **grad_outputs** - The \"vector\" in the Jacobian-vector product. Usually gradients w.r.t. each output. `torch::Tensor()` values can be specified for scalar Tensors or ones that don't require grad. If a `torch::Tensor()` value would be acceptable for all grad_tensors, then this argument is optional. Default: `{}`.\n- **retain_graph** - If `false`, the graph used to compute the grad will be freed. Note that in nearly all cases setting this option to `true` is not needed and often can be worked around in a much more efficient way. Defaults to the value of `create_graph`.\n- **create_graph** - If `true`, graph of the derivative will be constructed, allowing to compute higher order derivative products. Default: `false`.\n- **allow_unused** - If `false`, specifying inputs that were not used when computing outputs (and therefore their grad is always zero) is an error. Defaults to `false`.\n\n**Example:**\n\n```\n#include <torch/torch.h>\n\nauto x = torch::randn({2, 2}, torch::requires_grad());\nauto y = x * x;\nauto z = y.sum();\n\n// Compute gradients\nz.backward();\nstd::cout << x.grad() << std::endl;\n\n// Or use grad() for specific outputs\nauto grads = torch::autograd::grad({z}, {x});\n```\n\n## Tensor Gradient Methods\n\nTensors have built-in methods for gradient computation:\n\n```\n// Enable gradient tracking\nauto x = torch::randn({2, 2}).requires_grad_(true);\n\n// Check if gradient is required\nbool needs_grad = x.requires_grad();\n\n// Access the gradient after backward\nauto grad = x.grad();\n\n// Detach from computation graph\nauto x_detached = x.detach();\n```\n\n---\n\n# Autograd: Automatic Differentiation\n\nPyTorch's autograd system provides automatic differentiation for all operations\non tensors. It records operations on tensors to build a computational graph,\nthen computes gradients automatically via backpropagation.\n\n**When to use Autograd:**\n\n- When training neural networks (gradients are computed automatically)\n- When implementing custom backward passes for specialized operations\n- When you need fine-grained control over gradient computation\n\n**Basic usage:**\n\n```\n#include <torch/torch.h>\n\n// Create tensor with gradient tracking\nauto x = torch::randn({2, 2}, torch::requires_grad());\nauto y = x * x;\nauto z = y.sum();\n\n// Compute gradients via backpropagation\nz.backward();\nstd::cout << x.grad() << std::endl; // dz/dx = 2x\n\n// Disable gradient tracking for inference\n{\n torch::NoGradGuard no_grad;\n auto result = model->forward(input); // No gradients computed\n}\n```\n\n## Header Files\n\n- `torch/csrc/autograd/autograd.h` - High-level autograd API\n- `torch/csrc/autograd/function.h` - Custom autograd functions\n- `torch/csrc/autograd/grad_mode.h` - Gradient computation modes\n- `torch/csrc/api/include/torch/autograd.h` - C++ Frontend autograd\n\n## Autograd Categories\n\n- [Gradient Computation](gradient.html)\n- [Custom Autograd Functions](custom_functions.html)\n- [Gradient Modes](modes.html)\n\n---\n\n# Gradient Modes\n\nPyTorch provides RAII guards to control gradient computation behavior.\n\n## NoGradGuard\n\nclass torch::NoGradGuard\n\nRAII guard that disables gradient computation within its scope.\n\nNoGradGuard()\n\nDisables gradient computation.\n\n~NoGradGuard()\n\nRestores previous gradient mode.\n\n**Example:**\n\n```\n{\n torch::NoGradGuard no_grad;\n // No gradients computed in this scope\n auto result = model->forward(input);\n}\n```\n\n## InferenceMode\n\n`c10::InferenceMode` is a RAII guard analogous to `NoGradMode` designed for use\nwhen you are certain your operations will have no interactions with autograd\n(e.g., model inference). Compared to `NoGradMode`, code run under this mode gets\nbetter performance by disabling autograd-related work like view tracking and version\ncounter bumps. However, tensors created inside `InferenceMode` have more limitations\nwhen interacting with the autograd system.\n\nclass c10::InferenceMode\n\nRAII guard that enables inference mode for optimized inference.\nThis is more efficient than NoGradGuard for inference-only workloads.\n\nexplicit InferenceMode(bool enabled = true)\n\nEnables or disables inference mode.\n\n**Inference Tensors:**\n\n`InferenceMode` can be enabled for a given block of code. Inside `InferenceMode`,\nall newly allocated (non-view) tensors are marked as inference tensors. Inference tensors:\n\n- Do not have a version counter, so an error will be raised if you try to read their version\n(e.g., because you saved this tensor for backward).\n- Are immutable outside `InferenceMode`. An error will be raised if you try to:\n\n- Mutate their data outside InferenceMode.\n- Mutate them to `requires_grad=True` outside InferenceMode.\n- To work around this, make a clone outside `InferenceMode` to get a normal tensor before mutating.\n\nA non-view tensor is an inference tensor if and only if it was allocated inside `InferenceMode`.\nA view tensor is an inference tensor if and only if it is a view of an inference tensor.\n\n**Performance Guarantees:**\n\nInside an `InferenceMode` block:\n\n- Like `NoGradMode`, all operations do not record `grad_fn` even if their inputs have\n`requires_grad=True`. This applies to both inference tensors and normal tensors.\n- View operations on inference tensors do not perform view tracking. View and non-view\ninference tensors are indistinguishable.\n- Inplace operations on inference tensors are guaranteed not to do a version bump.\n\nFor more implementation details, see the [RFC-0011-InferenceMode](https://github.com/pytorch/rfcs/pull/17).\n\n**Basic Example:**\n\n```\n{\n c10::InferenceMode guard;\n // Optimized inference without gradient tracking\n auto result = model->forward(input);\n}\n```\n\n**Inference Workload Example:**\n\n```\nc10::InferenceMode guard;\nmodel.load_jit(saved_model);\nauto inputs = preprocess_tensors(data);\nauto out = model.forward(inputs);\nauto outputs = postprocess_tensors(out);\n```\n\n**Nested InferenceMode:**\n\nUnlike some other guards, `InferenceMode` can be nested with different enabled/disabled states:\n\n```\n{\n c10::InferenceMode guard(true);\n // InferenceMode is on\n {\n c10::InferenceMode guard(false);\n // InferenceMode is off\n }\n // InferenceMode is on\n}\n// InferenceMode is off\n```\n\n## InferenceMode vs NoGradMode\n\n`InferenceMode` is preferred over `NoGradMode` for pure inference workloads because\nit provides better performance. Key differences:\n\n- Both guards affect tensor execution to skip work not related to inference, but\n`InferenceMode` also affects tensor creation while `NoGradMode` doesn't.\n- Tensors created inside `InferenceMode` are marked as inference tensors with\ncertain limitations that apply after exiting `InferenceMode`.\n- `InferenceMode` can be nested with enabled/disabled states.\n\n## Migrating from AutoNonVariableTypeMode\n\nThe legacy `AutoNonVariableTypeMode` guard (now renamed to\n`AutoDispatchBelowADInplaceOrView`) was commonly used for inference workloads\nbut is unsafe -- it can silently bypass safety checks and produce wrong results.\n\n- **For inference-only workloads** (e.g. loading a pretrained JIT model and\nrunning inference in C++ runtime), use `c10::InferenceMode` as a drop-in\nreplacement. It preserves the performance characteristics while providing\ncorrectness guarantees.\n- **For custom autograd kernels** that need to redispatch below the Autograd\ndispatch key, use `AutoDispatchBelowADInplaceOrView` instead:\n\n```\nclass ROIAlignFunction : public torch::autograd::Function<ROIAlignFunction> {\n public:\n static torch::autograd::variable_list forward(\n torch::autograd::AutogradContext* ctx,\n const torch::autograd::Variable& input,\n const torch::autograd::Variable& rois,\n double spatial_scale, int64_t pooled_height,\n int64_t pooled_width, int64_t sampling_ratio, bool aligned) {\n ctx->saved_data[\"spatial_scale\"] = spatial_scale;\n ctx->save_for_backward({rois});\n at::AutoDispatchBelowADInplaceOrView guard;\n auto result = roi_align(input, rois, spatial_scale,\n pooled_height, pooled_width, sampling_ratio, aligned);\n return {result};\n }\n};\n```\n\n---\n\n# Device and DeviceType\n\nPyTorch provides device abstractions for writing code that works across\nCPU, CUDA, and other backends.\n\n## Device\n\nstruct Device\n\nRepresents a compute device on which a tensor is located.\n\nA device is uniquely identified by a type, which specifies the type of machine it is (e.g. CPU or CUDA GPU), and a device index or ordinal, which identifies the specific compute device when there is more than one of a certain type. The device index is optional, and in its defaulted state represents (abstractly) \"the current device\". Further, there are two constraints on the value of the device index, if one is explicitly stored:\n\n1. A negative index represents the current device, a non-negative index represents a specific, concrete device,\n2. When the device type is CPU, the device index must be zero.\n\nPublic Types\n\nusing Type = DeviceType\n\nPublic Functions\n\ninline Device(DeviceType type, DeviceIndex index = -1)\n\nConstructs a new `Device` from a `DeviceType` and an optional device index.\n\nDevice(const std::string &device_string)\n\nConstructs a `Device` from a string description, for convenience.\n\nThe string supplied must follow the following schema: `(cpu|cuda)[:<device-index>]` where `cpu` or `cuda` specifies the device type, and `:<device-index>` optionally specifies a device index.\n\ninline bool operator==(const Device &other) const noexcept\n\nReturns true if the type and index of this `Device` matches that of `other`.\n\ninline bool operator!=(const Device &other) const noexcept\n\nReturns true if the type or index of this `Device` differs from that of `other`.\n\ninline void set_index(DeviceIndex index)\n\nSets the device index.\n\ninline DeviceType type() const noexcept\n\nReturns the type of device this is.\n\ninline DeviceIndex index() const noexcept\n\nReturns the optional index.\n\ninline bool has_index() const noexcept\n\nReturns true if the device has a non-default index.\n\ninline bool is_cuda() const noexcept\n\nReturn true if the device is of CUDA type.\n\ninline bool is_privateuseone() const noexcept\n\nReturn true if the device is of PrivateUse1 type.\n\ninline bool is_mps() const noexcept\n\nReturn true if the device is of MPS type.\n\ninline bool is_hip() const noexcept\n\nReturn true if the device is of HIP type.\n\ninline bool is_ve() const noexcept\n\nReturn true if the device is of VE type.\n\ninline bool is_xpu() const noexcept\n\nReturn true if the device is of XPU type.\n\ninline bool is_ipu() const noexcept\n\nReturn true if the device is of IPU type.\n\ninline bool is_xla() const noexcept\n\nReturn true if the device is of XLA type.\n\ninline bool is_mtia() const noexcept\n\nReturn true if the device is of MTIA type.\n\ninline bool is_hpu() const noexcept\n\nReturn true if the device is of HPU type.\n\ninline bool is_lazy() const noexcept\n\nReturn true if the device is of Lazy type.\n\ninline bool is_vulkan() const noexcept\n\nReturn true if the device is of Vulkan type.\n\ninline bool is_metal() const noexcept\n\nReturn true if the device is of Metal type.\n\ninline bool is_maia() const noexcept\n\nReturn true if the device is of MAIA type.\n\ninline bool is_meta() const noexcept\n\nReturn true if the device is of META type.\n\ninline bool is_cpu() const noexcept\n\nReturn true if the device is of CPU type.\n\ninline bool supports_as_strided() const noexcept\n\nReturn true if the device supports arbitrary strides.\n\nstd::string str() const\n\nSame string as returned from operator<<.\n\n**Example:**\n\n```\nc10::Device cpu_device(c10::kCPU);\nc10::Device cuda_device(c10::kCUDA, 0); // CUDA device 0\n\nif (cuda_device.is_cuda()) {\n std::cout << \"Using CUDA device \" << cuda_device.index() << std::endl;\n}\n```\n\n## DeviceType\n\nenum class c10::DeviceType\n\nEnumeration of supported device types.\n\nenumerator CPU = 0\n\nCPU device.\n\nenumerator CUDA = 1\n\nNVIDIA CUDA GPU.\n\nenumerator HIP = 6\n\nAMD HIP GPU.\n\nenumerator XLA = 9\n\nXLA / TPU.\n\nenumerator Vulkan = 10\n\nVulkan GPU.\n\nenumerator Metal = 11\n\nApple Metal GPU.\n\nenumerator XPU = 12\n\nIntel XPU GPU.\n\nenumerator MPS = 13\n\nApple Metal Performance Shaders.\n\nenumerator Meta = 14\n\nMeta tensors (shape only, no data).\n\nenumerator HPU = 15\n\nHabana HPU.\n\nenumerator Lazy = 17\n\nLazy tensors.\n\nenumerator IPU = 18\n\nGraphcore IPU.\n\nenumerator MTIA = 19\n\nMeta training and inference accelerator.\n\nenumerator PrivateUse1 = 20\n\nCustom backend registered via `c10::register_privateuse1_backend()`.\n\nConvenience constants:\n\n- `c10::kCPU`, `c10::kCUDA`, `c10::kHIP`\n- `c10::kXLA`, `c10::kVulkan`, `c10::kMetal`\n- `c10::kXPU`, `c10::kMPS`, `c10::kMeta`\n- `c10::kHPU`, `c10::kLazy`, `c10::kIPU`, `c10::kMTIA`\n- `c10::kPrivateUse1`\n\n---\n\n# Device Guards\n\nC10 provides device-agnostic RAII guards for managing the current device\ncontext. These guards work across all backends (CUDA, XPU, etc.) and\nautomatically restore the previous device when they go out of scope.\n\nFor backend-specific guards, see [CUDA Guards](../cuda/guards.html) and [XPU Support](../xpu/index.html).\n\n## DeviceGuard\n\nclass DeviceGuard\n\nRAII guard that sets a certain default device in its constructor, and changes it back to the device that was originally active upon destruction.\n\nThe device is always reset to the one that was active at the time of construction of the guard. Even if you `set_device` after construction, the destructor will still reset the device to the one that was active at construction time.\n\nThis device guard does NOT have an uninitialized state; it is guaranteed to reset a device on exit. If you are in a situation where you *might* want to setup a guard (i.e., are looking for the moral equivalent of std::optional<DeviceGuard>), see OptionalDeviceGuard.\n\nPublic Functions\n\nexplicit DeviceGuard() = delete\n\nNo default constructor; see Note [Omitted default constructor from RAII].\n\ninline explicit DeviceGuard([Device](device.html#_CPPv4N3c106DeviceE) device)\n\nSet the current device to the passed [Device](device.html#PyTorchstructc10_1_1_device).\n\ninline explicit DeviceGuard([Device](device.html#_CPPv4N3c106DeviceE) device, const [impl](../aten/tensor.html#_CPPv4N3c1011DeviceGuard11DeviceGuardE6DevicePKN4impl24DeviceGuardImplInterfaceE)::DeviceGuardImplInterface *impl)\n\nThis constructor is for testing only.\n\n~DeviceGuard() = default\n\nDeviceGuard(const [DeviceGuard](../aten/tensor.html#_CPPv4N3c1011DeviceGuard11DeviceGuardERK11DeviceGuard)&) = delete\n\nCopy is disallowed.\n\n[DeviceGuard](../aten/tensor.html#_CPPv4N3c1011DeviceGuardE) &operator=(const [DeviceGuard](../aten/tensor.html#_CPPv4N3c1011DeviceGuardE)&) = delete\n\nDeviceGuard([DeviceGuard](../aten/tensor.html#_CPPv4N3c1011DeviceGuard11DeviceGuardERR11DeviceGuard) &&other) = delete\n\nMove is disallowed, as DeviceGuard does not have an uninitialized state, which is required for moves on types with nontrivial destructors.\n\n[DeviceGuard](../aten/tensor.html#_CPPv4N3c1011DeviceGuardE) &operator=([DeviceGuard](../aten/tensor.html#_CPPv4N3c1011DeviceGuardE) &&other) = delete\n\ninline void reset_device(at::Device device)\n\nSets the device to the given one.\n\nThe specified device must be consistent with the device type originally specified during guard construction.\n\nTODO: The consistency check here is inconsistent with StreamGuard's behavior with set_stream, where a stream on a different device than the original one isn't an error; we just reset the stream and then switch devices.\n\ninline void reset_device(at::Device device, const [impl](../aten/tensor.html#_CPPv4N3c1011DeviceGuard12reset_deviceEN2at6DeviceEPKN4impl24DeviceGuardImplInterfaceE)::DeviceGuardImplInterface *impl)\n\nThis method is for testing only.\n\ninline void set_index(DeviceIndex index)\n\nSets the device index to the given one.\n\nThe device type is inferred from the original device type the guard was constructed with.\n\ninline [Device](device.html#_CPPv4N3c106DeviceE) original_device() const\n\nReturns the device that was set at the time the guard was constructed.\n\ninline [Device](device.html#_CPPv4N3c106DeviceE) current_device() const\n\nReturns the most recent device that was set using this device guard, either from construction, or via set_device.\n\n**Example:**\n\n```\n#include <c10/core/DeviceGuard.h>\n\n{\n c10::DeviceGuard guard(c10::Device(c10::kCUDA, 1));\n // All operations here run on CUDA device 1\n}\n// Previous device is restored\n```\n\n## OptionalDeviceGuard\n\nclass OptionalDeviceGuard\n\nA OptionalDeviceGuard is an RAII class that sets a device to some value on initialization, and resets the device to its original value on destruction.\n\nMorally, a OptionalDeviceGuard is equivalent to std::optional<DeviceGuard>, but with extra constructors and methods as appropriate.\n\nBesides its obvious use (optionally applying a DeviceGuard), OptionalDeviceGuard is often also used for the following idiom:\n\nOptionalDeviceGuard g; for (const auto& t : tensors) { g.set_device(t.device()); do_something_with(t); }\n\nThis usage is marginally more efficient than constructing a DeviceGuard every iteration of the for loop, as it avoids an unnecessary device reset.\n\nUnlike DeviceGuard, a OptionalDeviceGuard may be uninitialized. This occurs when you use the nullary constructor, or pass a nullopt to the constructor. Uninitialized OptionalDeviceGuards do *nothing*; they do not know what the original device was and they do not reset on destruction. This is why original_device() and current_device() return std::optional<Device> rather than [Device](device.html#PyTorchstructc10_1_1_device) (as they do in DeviceGuard), and also is why we didn't just provide OptionalDeviceGuard by default and hide DeviceGuard from users.\n\nThe semantics of an OptionalDeviceGuard are exactly explained by thinking of it as an std::optional<DeviceGuard>. In particular, an initialized OptionalDeviceGuard doesn't restore device to its value at construction; it restores device to its value *at initialization*. So if you have the program:\n\n```\nsetDevice(1);\nOptionalDeviceGuard g;\nsetDevice(2);\ng.reset_device(Device(DeviceType::CUDA, 3)); // initializes!\n```\n\n On destruction, g will reset device to 2, rather than 1.\n\nAn uninitialized OptionalDeviceGuard is distinct from a (initialized) DeviceGuard whose original_device_ and current_device_ match, since the DeviceGuard will still reset the device to original_device_.\n\nPublic Functions\n\nexplicit OptionalDeviceGuard() = default\n\nCreate an uninitialized guard. Set the guard later using reset_device.\n\ninline explicit OptionalDeviceGuard([Device](device.html#_CPPv4N3c106DeviceE) device)\n\nInitialize the guard, setting the current device to the passed [Device](device.html#PyTorchstructc10_1_1_device).\n\ninline explicit OptionalDeviceGuard(std::optional<[Device](device.html#_CPPv4N3c106DeviceE)> device)\n\nInitialize the guard if a [Device](device.html#PyTorchstructc10_1_1_device) is passed; otherwise leave the guard uninitialized.\n\ninline explicit OptionalDeviceGuard([Device](device.html#_CPPv4N3c106DeviceE) device, const impl::DeviceGuardImplInterface *impl)\n\nConstructor for testing only.\n\n~OptionalDeviceGuard() = default\n\nOptionalDeviceGuard(const OptionalDeviceGuard&) = delete\n\nCopy is disallowed.\n\nOptionalDeviceGuard &operator=(const OptionalDeviceGuard&) = delete\n\nOptionalDeviceGuard(OptionalDeviceGuard &&other) = delete\n\nMove is disallowed See Note [Explicit initialization of optional fields] and // Note [Move construction for RAII guards is tricky] for rationale.\n\nOptionalDeviceGuard &operator=(OptionalDeviceGuard &&other) = delete\n\ninline void reset_device(at::Device device)\n\nSets the device to the given one.\n\nThe specified device must be consistent with the device type originally specified during guard construction.\n\ninline void reset_device(at::Device device, const impl::DeviceGuardImplInterface *impl)\n\nFor testing only.\n\ninline std::optional<[Device](device.html#_CPPv4N3c106DeviceE)> original_device() const\n\nReturns the device that was set at the time the guard was constructed.\n\ninline std::optional<[Device](device.html#_CPPv4N3c106DeviceE)> current_device() const\n\nReturns the most recent device that was set using this device guard, either from construction, or via reset_device.\n\n**Example:**\n\n```\n#include <c10/core/DeviceGuard.h>\n\nc10::OptionalDeviceGuard guard;\nif (use_gpu) {\n guard.reset_device(c10::Device(c10::kCUDA, 0));\n}\n// Guard only restores device","isInternal":false,"tokens":12489,"sizeBytes":50000},{"name":"llms.txt","path":"llms.txt","rawUrl":"https://raw.githubusercontent.com/pytorch/cpp/HEAD/llms.txt","title":"LLM Index & Context Digest","category":"root-instruction","format":"text","content":"# PyTorch\n\n> PyTorch documentation.\n\n## Docs\n\n- [Tensor Accessors](api/aten/accessors.md)\n- [Tensor Creation](api/aten/creation.md)\n- [ATen: Tensor Library](api/aten/index.md)\n- [Tensor Indexing](api/aten/indexing.md)\n- [Tensor Class](api/aten/tensor.md)\n- [Custom Autograd Functions](api/autograd/custom_functions.md)\n- [Gradient Computation](api/autograd/gradient.md)\n- [Autograd: Automatic Differentiation](api/autograd/index.md)\n- [Gradient Modes](api/autograd/modes.md)\n- [Device and DeviceType](api/c10/device.md)\n- [Device Guards](api/c10/guards.md)\n- [C10: Core Utilities](api/c10/index.md)\n- [Streams](api/c10/streams.md)\n- [Core Types](api/c10/types.md)\n- [Utilities](api/c10/utilities.md)\n- [CUDA Guards](api/cuda/guards.md)\n- [CUDA Support](api/cuda/index.md)\n- [CUDA Streams](api/cuda/streams.md)\n- [CUDA Utility Functions](api/cuda/utilities.md)\n- [DataLoader](api/data/dataloader.md)\n- [Datasets](api/data/datasets.md)\n- [Data Loading (torch::data)](api/data/index.md)\n- [Samplers](api/data/samplers.md)\n- [Transforms](api/data/transforms.md)\n- [C++ API Reference](api/index.md)\n- [Custom Classes](api/library/custom_classes.md)\n- [Torch Library API](api/library/index.md)\n- [Operator Registration](api/library/registration.md)\n- [Library Versioning](api/library/versioning.md)\n- [Activation Functions](api/nn/activation.md)\n- [Containers](api/nn/containers.md)\n- [Convolution Layers](api/nn/convolution.md)\n- [Dropout Layers](api/nn/dropout.md)\n- [Embedding Layers](api/nn/embedding.md)\n- [Functional API](api/nn/functional.md)\n- [Neural Network Modules (torch::nn)](api/nn/index.md)\n- [Linear Layers](api/nn/linear.md)\n- [Loss Functions](api/nn/loss.md)\n- [Normalization Layers](api/nn/normalization.md)\n- [Pooling Layers](api/nn/pooling.md)\n- [Recurrent Layers](api/nn/recurrent.md)\n- [Transformer Layers](api/nn/transformer.md)\n- [Utilities](api/nn/utilities.md)\n- [Adaptive Learning Rate Optimizers](api/optim/adaptive.md)\n- [Gradient Descent Optimizers](api/optim/gradient_descent.md)\n- [Optimizers (torch::optim)](api/optim/index.md)\n- [Learning Rate Schedulers](api/optim/schedulers.md)\n- [Second-Order Optimizers](api/optim/second_order.md)\n- [Archives](api/serialize/archives.md)\n- [Checkpoints](api/serialize/checkpoints.md)\n- [Serialization (torch::serialize)](api/serialize/index.md)\n- [Saving and Loading](api/serialize/save_load.md)\n- [Torch Stable API](api/stable/index.md)\n- [Stable Operators](api/stable/operators.md)\n- [Library Registration Macros](api/stable/registration.md)\n- [Utilities](api/stable/utilities.md)\n- [XPU Support](api/xpu/index.md)\n- [XPU Streams](api/xpu/streams.md)\n- [XPU Utility Functions](api/xpu/utilities.md)\n- [FAQ](faq.md)\n- [The C++ Frontend](frontend.md)\n- [PyTorch C++ API](index.md)\n- [Installing C++ Distributions of PyTorch](installing.md)","isInternal":false,"tokens":820,"sizeBytes":2806},{"name":".cursorrules","path":".cursorrules (cpp.mdc)","rawUrl":"https://raw.githubusercontent.com/PatrickJS/awesome-cursorrules/main/rules/cpp.mdc","title":"Awesome Cursorrules: cpp","category":"cursor-rule","format":"markdown","content":"---\ndescription: Guide Cursor to write modern C++ and CMake code with clear structure, RAII, const-correctness, and safe error handling.\nglobs: **/*.c,**/*.cpp,**/*.h,**/*.hpp,**/*.cxx,CMakeLists.txt,*.cmake,conanfile.txt,Makefile,**/*.cc\nalwaysApply: false\n---\n# C++ Programming Guidelines\n\n## Basic Principles\n\n- Use English for all code and documentation.\n- Always declare the type of each variable and function (parameters and return value).\n- Create necessary types and classes.\n- Use Doxygen style comments to document public classes and methods.\n- Don't leave blank lines within a function.\n- Follow the one-definition rule (ODR).\n\n## Nomenclature\n\n- Use PascalCase for classes and structures.\n- Use camelCase for variables, functions, and methods.\n- Use ALL_CAPS for constants and macros.\n- Use snake_case for file and directory names.\n- Use UPPERCASE for environment variables.\n- Avoid magic numbers and define constants.\n- Start each function with a verb.\n- Use verbs for boolean variables. Example: isLoading, hasError, canDelete, etc.\n- Use complete words instead of abbreviations and ensure correct spelling.\n  - Except for standard abbreviations like API, URL, etc.\n  - Except for well-known abbreviations:\n    - i, j, k for loops\n    - err for errors\n    - ctx for contexts\n    - req, res for request/response parameters\n\n## Functions\n\n- Write short functions with a single purpose. Less than 20 instructions.\n- Name functions with a verb and something else.\n- If it returns a boolean, use isX or hasX, canX, etc.\n- If it doesn't return anything (void), use executeX or saveX, etc.\n- Avoid nesting blocks by:\n  - Early checks and returns.\n  - Extraction to utility functions.\n- Use standard library algorithms (std::for_each, std::transform, std::find, etc.) to avoid function nesting.\n- Use lambda functions for simple operations.\n- Use named functions for non-simple operations.\n- Use default parameter values instead of checking for null or nullptr.\n- Reduce function parameters using structs or classes\n  - Use an object to pass multiple parameters.\n  - Use an object to return multiple results.\n  - Declare necessary types for input arguments and output.\n- Use a single level of abstraction.\n\n## Data\n\n- Don't abuse primitive types and encapsulate data in composite types.\n- Avoid data validations in functions and use classes with internal validation.\n- Prefer immutability for data.\n- Use const for data that doesn't change.\n- Use constexpr for compile-time constants.\n- Use std::optional for possibly null values.\n\n## Classes\n\n- Follow SOLID principles.\n- Prefer composition over inheritance.\n- Declare interfaces as abstract classes or concepts.\n- Write small classes with a single purpose.\n  - Less than 200 instructions.\n  - Less than 10 public methods.\n  - Less than 10 properties.\n- Use the Rule of Five (or Rule of Zero) for resource management.\n- Make member variables private and provide getters/setters where necessary.\n- Use const-correctness for member functions.\n\n## Exceptions\n\n- Use exceptions to handle errors you don't expect.\n- If you catch an exception, it should be to:\n  - Fix an expected problem.\n  - Add context.\n  - Otherwise, use a global handler.\n- Use std::optional, std::expected, or error codes for expected failures.\n\n## Memory Management\n\n- Prefer smart pointers (std::unique_ptr, std::shared_ptr) over raw pointers.\n- Use RAII (Resource Acquisition Is Initialization) principles.\n- Avoid memory leaks by proper resource management.\n- Use std::vector and other standard containers instead of C-style arrays.\n\n## Testing\n\n- Follow the Arrange-Act-Assert convention for tests.\n- Name test variables clearly.\n- Follow the convention: inputX, mockX, actualX, expectedX, etc.\n- Write unit tests for each public function.\n- Use test doubles to simulate dependencies.\n  - Except for third-party dependencies that are not expensive to execute.\n- Write integration tests for each module.\n- Follow the Given-When-Then convention.\n\n## Project Structure\n\n- Use modular architecture\n- Organize code into logical directories:\n  - include/ for header files\n  - src/ for source files\n  - test/ for test files\n  - lib/ for libraries\n  - doc/ for documentation\n- Use CMake or similar build system.\n- Separate interface (.h) from implementation (.cpp).\n- Use namespaces to organize code logically.\n- Create a core namespace for foundational components.\n- Create a utils namespace for utility functions.\n\n## Standard Library\n\n- Use the C++ Standard Library whenever possible.\n- Prefer std::string over C-style strings.\n- Use std::vector, std::map, std::unordered_map, etc. for collections.\n- Use std::optional, std::variant, std::any for modern type safety.\n- Use std::filesystem for file operations.\n- Use std::chrono for time-related operations.\n\n## Concurrency\n\n- Use std::thread, std::mutex, std::lock_guard for thread safety.\n- Prefer task-based parallelism over thread-based parallelism.\n- Use std::atomic for atomic operations.\n- Avoid data races by proper synchronization.\n- Use thread-safe data structures when necessary.\n","tokens":1116,"sizeBytes":5064}],"systemPromptSnippet":"<yakaai_skills repo=\"pytorch/cpp\">\n<!-- File: llms-full.txt (Tokens: ~12489 | Category: root-instruction) -->\n# PyTorch\n\n> PyTorch documentation.\n\n\n---\n\n# Tensor Accessors\n\nFor element-wise operations in custom kernels, use *accessors* to avoid\ndynamic dispatch overhead.\n\n## CPU Accessors\n\n```\ntorch::Tensor foo = torch::rand({12, 12});\n\n// Create accessor - validates type and dimensions once\nauto foo_a = foo.accessor<float, 2>();\n\nfloat trace = 0;\nfor (int i = 0; i < foo_a.size(0); i++) {\n trace += foo_a[i][i];\n}\n```\n\n## CUDA Packed Accessors\n\nFor CUDA kernels, use *packed accessors* which copy metadata instead of\npointing to it:\n\n```\n__global__ void kernel(torch::PackedTensorAccessor64<float, 2> foo, float* trace) {\n int i = threadIdx.x;\n gpuAtomicAdd(trace, foo[i][i]);\n}\n\ntorch::Tensor foo = torch::rand({12, 12}).cuda();\nauto foo_a = foo.packed_accessor64<float, 2>();\n\nfloat trace = 0;\nkernel<<<1, 12>>>(foo_a, &trace);\n```\n\nTip\n\nUse `PackedTensorAccessor32` and `packed_accessor32` for 32-bit indexing,\nwhich is faster on CUDA but may overflow for large tensors.\n\n---\n\n# Tensor Creation\n\nFactory functions create new tensors with different initialization patterns.\nAll factory functions follow a general schema:\n\n```\ntorch::<function-name>(<function-specific-options>, <sizes>, <tensor-options>)\n```\n\n## Available Factory Functions\n\n- `torch::zeros` - Tensor filled with zeros\n- `torch::ones` - Tensor filled with ones\n- `torch::empty` - Uninitialized tensor\n- `torch::full` - Tensor filled with a single value\n- `torch::rand` - Uniform random values on [0, 1)\n- `torch::randn` - Standard normal distribution\n- `torch::randint` - Random integers in a range\n- `torch::arange` - Sequence of integers\n- `torch::linspace` - Linearly spaced values\n- `torch::logspace` - Logarithmically spaced values\n- `torch::eye` - Identity matrix\n- `torch::randperm` - Random permutation of integers\n\n## Specifying a Size\n\nFunctions that do not require specific arguments can be invoked with just a\nsize. For example, the following line creates a vector with 5 components:\n\n```\ntorch::Tensor tensor = torch::ones(5);\n```\n\nAn `IntArrayRef` is constructed by specifying the size along each dimension in\ncurly braces. For example, `{2, 3}` for a matrix with two rows and three\ncolumns, `{3, 4, 5}` for a three-dimensional tensor:\n\n```\ntorch::Tensor tensor = torch::randn({3, 4, 5});\nassert(tensor.sizes() == std::vector<int64_t>{3, 4, 5});\n```\n\nYou can also pass an `std::vector<int64_t>` instead of curly braces.\nUse `tensor.size(i)` to access a single dimension.\n\n## Passing Function-Specific Parameters\n\nSome factory functions accept additional parameters. For example, `randint`\ntakes an upper bound on the value for the integers it generates:\n\n```\ntorch::Tensor tensor = torch::randint(/*high=*/10, {5, 5});\n\n// With a lower bound\ntorch::Tensor tensor = torch::randint(/*low=*/3, /*high=*/10, {5, 5});\n```\n\nTip\n\nThe size always follows the function-specific arguments.\n\nAttention\n\nSome functions like `arange` do not need a size at all, since it is fully\ndetermined by the function-specific arguments (the range bounds).\n\n## Configuring Properties with TensorOptions\n\n`TensorOptions` configures the data type, layout, device, and\n`requires_grad` of a new tensor. The construction axes are:\n\n- `dtype`: the data type of the elements (e.g. `kFloat32`, `kInt64`)\n- `layout`: either `kStrided` (dense) or `kSparse`\n- `device`: a compute device (e.g. `kCPU`, `kCUDA`)\n- `requires_grad`: whether to track gradients\n\nAllowed values:\n\n- `dtype`: `kUInt8`, `kInt8`, `kInt16`, `kInt32`, `kInt64`,\n`kFloat32`, `kFloat64`\n- `layout`: `kStrided`, `kSparse`\n- `device`: `kCPU`, or `kCUDA` (with an optional device index)\n- `requires_grad`: `true` or `false`\n\nTip\n\nRust-style shorthands exist for dtypes, like `kF32` instead of\n`kFloat32`. See\n[torch/types.h](https://github.com/pytorch/pytorch/blob/main/torch/csrc/api/include/torch/types.h)\nfor the full list.\n\nHere is an example of creating a `TensorOptions` object:\n\n```\nauto options =\n torch::TensorOptions()\n .dtype(torch::kFloat64)\n .layout(torch::kStrided)\n .device(torch::kCUDA, 1)\n .requires_grad(true);\n\ntorch::Tensor tensor = torch::full({3, 4}, /*value=*/123, options);\n\nassert(tensor.dtype() == torch::kFloat64);\nassert(tensor.layout() == torch::kStrided);\nassert(tensor.device().type() == torch::kCUDA);\nassert(tensor.device().index() == 1);\nassert(tensor.requires_grad());\n```\n\n**Defaults:** Any axis you omit takes its default value: `kFloat32` for dtype,\n`kStrided` for layout, `kCPU` for device, and `false` for\n`requires_grad`. This means you can omit `TensorOptions` entirely:\n\n```\n// A 32-bit float, strided, CPU tensor that does not require a gradient.\ntorch::Tensor tensor = torch::randn({3, 4});\n```\n\n**Shorthand syntax:** For each axis there is a free function in the `torch::`\nnamespace (`torch::dtype()`, `torch::device()`, `torch::layout()`,\n`torch::requires_grad()`). Each returns a `TensorOptions` object that can\nbe further refined with builder methods:\n\n```\n// These are equivalent:\ntorch::ones(10, torch::TensorOptions().dtype(torch::kFloat32))\ntorch::ones(10, torch::dtype(torch::kFloat32))\n\n// Chaining:\ntorch::ones(10, torch::dtype(torch::kFloat32).layout(torch::kStrided))\n```\n\n**Implicit construction:** `TensorOptions` is implicitly constructible from\nindividual values, so when only one axis differs from the default you can\nwrite:\n\n```\ntorch::ones(10, torch::kFloat32)\n```\n\nPutting it all together, a C++ tensor creation call mirrors the Python\nequivalent closely:\n\n```\n# Python\ntorch.randn(3, 4, dtype=torch.float32, device=torch.device('cuda', 1), requires_grad=True)\n```\n\n```\n// C++\ntorch::randn({3, 4}, torch::dtype(torch::kFloat32).device(torch::kCUDA, 1).requires_grad(true))\n```\n\n## Using Externally Created Data\n\nIf you already have tensor data allocated in memory (CPU or CUDA), use\n`from_blob` to view that memory as a `Tensor`:\n\n```\nfloat data[] = {1, 2, 3, 4, 5, 6};\ntorch::Tensor tensor = torch::from_blob(data, {2, 3});\n```\n\nNote\n\nTensors created with `from_blob` cannot be resized because ATen does not\nown the memory.\n\n## Tensor Conversion\n\nUse `to()` to convert tensors between dtypes and devices. The conversion\ncreates a new tensor and does not occur in-place:\n\n```\ntorch::Tensor source = torch::randn({2, 3}, torch::kInt64);\n\n// Convert dtype\ntorch::Tensor float_tensor = source.to(torch::kFloat32);\n\n// Move to GPU (default CUDA device)\ntorch::Tensor gpu_tensor = float_tensor.to(torch::kCUDA);\n\n// Specific GPU device\ntorch::Tensor gpu1_tensor = float_tensor.to(torch::Device(torch::kCUDA, 1));\n\n// Async copy\ntorch::Tensor async_tensor = gpu_tensor.to(torch::kCPU, /*non_blocking=*/true);\n```\n\nAttention\n\nThe result of the conversion is a new tensor pointing to new memory,\nunrelated to the source tensor.\n\n## Scalars and Zero-Dimensional Tensors\n\n`Scalar` represents a single dynamically-typed number. Like a `Tensor`,\n`Scalar` is dynamically typed and can hold any of ATen's number types.\nScalars can be implicitly constructed from C++ number types:\n\n```\nnamespace torch {\nTensor addmm(Scalar beta, const Tensor & self,\n Scalar alpha, const Tensor & mat1,\n const Tensor & mat2);\nScalar sum(const Tensor & self);\n} // namespace torch\n\n// Usage\ntorch::Tensor a = ...;\ntorch::Tensor b = ...;\ntorch::Tensor c = ...;\ntorch::Tensor r = torch::addmm(1.0, a, .5, b, c);\n```\n\nZero-dimensional tensors hold a single value and can reference elements in\nlarger tensors:\n\n```\ntorch::Tensor matrix = torch::rand({10, 20});\nmatrix[1][2] = 4; // matrix[1][2] is a zero-dimensional tensor\n```\n\n---\n\n# ATen: Tensor Library\n\nATen (A Tensor Library) is the foundational tensor and mathematical operation\nlibrary on which all of PyTorch is built. It provides the core `Tensor` class\nand hundreds of mathematical operations that work on tensors.\n\n**When to use ATen directly:**\n\n- When writing low-level tensor operations or custom kernels\n- When you need direct access to tensor data and metadata\n- When working with the PyTorch internals or extending PyTorch\n\n**Basic usage:**\n\n```\n#include <ATen/ATen.h>\n\n// Create tensors\nat::Tensor a = at::ones({2, 3});\nat::Tensor b = at::randn({2, 3});\n\n// Perform operations\nat::Tensor c = a + b;\nat::Tensor d = at::matmul(a.t(), b);\n\n// Move to GPU\nif (at::cuda::is_available()) {\n at::Tensor gpu_tensor = c.to(at::kCUDA);\n}\n```\n\nFor most applications, prefer using the higher-level `torch::` namespace\n(see [Neural Network Modules (torch::nn)](../nn/index.html), [Optimizers (torch::optim)](../optim/index.html)) which provides a more user-friendly API.\n\n## Header Files\n\nThe following headers are part of the ATen public API:\n\n- `ATen/ATen.h` - Main ATen header\n- `ATen/Backend.h` - Backend enumeration\n- `ATen/core/Tensor.h` - Tensor class\n- `ATen/core/ivalue.h` - IValue type (see [C10: Core Utilities](../c10/index.html))\n- `ATen/core/ScalarType.h` - Data type definitions\n- `ATen/TensorOptions.h` - Tensor creation options\n- `ATen/Scalar.h` - Scalar type\n- `ATen/Layout.h` - Tensor layout\n- `ATen/DeviceGuard.h` - Device context management\n- `ATen/native/TensorShape.h` - Tensor shape operations\n- `ATen/cuda/CUDAContext.h` - CUDA context (see [CUDA Support](../cuda/index.html))\n- `ATen/cudnn/Descriptors.h` - cuDNN descriptors\n- `ATen/mkl/Descriptors.h` - MKL descriptors\n\nNote\n\nThe core `at::Tensor` class is defined in a generated header file\n(`TensorBody.h`) that only exists after building PyTorch. The documentation\nbelow describes the API manually.\n\n## ATen Categories\n\n- [Tensor Class](tensor.html)\n- [Tensor Creation](creation.html)\n- [Tensor Indexing](indexing.html)\n- [Tensor Accessors](accessors.html)\n\n---\n\n# Tensor Indexing\n\nThe PyTorch C++ API provides tensor indexing similar to Python. Use\n`torch::indexing` namespace for index types:\n\n```\nusing namespace torch::indexing;\n```\n\nThe main difference from Python is that instead of using the `[]` operator,\nthe C++ API uses the `index` and `index_put_` methods:\n\n- `torch::Tensor::index` -- read elements\n- `torch::Tensor::index_put_` -- write elements\n\n## Index Types\n\nThe `TensorIndex` class accepts six types of indices via implicit constructors:\n\n| Type | C++ | Python equivalent |\n| --- | --- | --- |\n| None (unsqueeze) | `None` | `None` |\n| Ellipsis | `Ellipsis` or `\"...\"` | `...` |\n| Integer | `0`, `1`, `-1` | `0`, `1`, `-1` |\n| Boolean | `true`, `false` | `True`, `False` |\n| Slice | `Slice(start, stop, step)` | `start:stop:step` |\n| Tensor | `torch::tensor({0, 2})` | `torch.tensor([0, 2])` |\n\n## Getter Operations\n\n| Python | C++ |\n| --- | --- |\n| `tensor[0]` | `tensor.index({0})` |\n| `tensor[-1]` | `tensor.index({-1})` |\n| `tensor[1, 2]` | `tensor.index({1, 2})` |\n| `tensor[1, :, 3]` | `tensor.index({1, Slice(), 3})` |\n| `tensor[None]` | `tensor.index({None})` |\n| `tensor[:, None]` | `tensor.index({Slice(), None})` |\n| `tensor[...]` | `tensor.index({Ellipsis})` or `tensor.index({\"...\"})` |\n| `tensor[..., 0]` | `tensor.index({Ellipsis, 0})` |\n| `tensor[1::2]` | `tensor.index({Slice(1, None, 2)})` |\n| `tensor[True]` | `tensor.index({true})` |\n| `tensor[torch.tensor([1, 2])]` | `tensor.index({torch::tensor({1, 2})})` |\n| `tensor[bool_mask]` | `tensor.index({bool_mask})` |\n| `tensor[:, torch.tensor([[0,1],[4,3]])]` | `tensor.index({Slice(), torch::tensor({{0,1},{4,3}})})` |\n| `tensor[cond > 0]` | `tensor.index({cond > 0})` |\n\n## Setter Operations\n\n| Python | C++ |\n| --- | --- |\n| `tensor[0] = 1` | `tensor.index_put_({0}, 1)` |\n| `tensor[1, 2] = 1` | `tensor.index_put_({1, 2}, 1)` |\n| `tensor[1] = torch.arange(5)` | `tensor.index_put_({1}, torch::arange(5))` |\n| `tensor[1::2] = 1` | `tensor.index_put_({Slice(1, None, 2)}, 1)` |\n| `tensor[0, 1::2] = torch.tensor([3., 4.])` | `tensor.index_put_({0, Slice(1, None, 2)}, torch::tensor({3., 4.}))` |\n| `tensor[...] = 0` | `tensor.index_put_({Ellipsis}, 0)` |\n| `tensor[None] = value` | `tensor.index_put_({None}, value)` |\n| `tensor[bool_mask] = 0` | `tensor.index_put_({bool_mask}, 0)` |\n| `tensor[torch.tensor([0, 2])] = value` | `tensor.index_put_({torch::tensor({0, 2})}, value)` |\n| `tensor[1:2, torch.tensor([1,2])] = 0` | `tensor.index_put_({Slice(1, 2), torch::tensor({1, 2})}, 0)` |\n\nThe `index_put_` method also accepts an optional `accumulate` parameter.\nWhen `true`, values are added to existing values instead of replacing them:\n\n```\ntensor.index_put_({mask}, values, /*accumulate=*/true);\n```\n\n## Slice Syntax\n\nThe `Slice` constructor signature is:\n\n```\nSlice(\n std::optional<c10::SymInt> start = std::nullopt,\n std::optional<c10::SymInt> stop = std::nullopt,\n std::optional<c10::SymInt> step = std::nullopt);\n```\n\nPass `None` for open-ended bounds:\n\n| Python | C++ |\n| --- | --- |\n| `:` or `::` | `Slice()` or `Slice(None, None)` |\n| `1:` | `Slice(1, None)` |\n| `:3` | `Slice(None, 3)` |\n| `1:3` | `Slice(1, 3)` |\n| `1:3:2` | `Slice(1, 3, 2)` |\n| `::2` | `Slice(None, None, 2)` |\n\n## Full Example\n\n```\n#include <torch/torch.h>\n\nusing namespace torch::indexing;\n\nauto tensor = torch::arange(2 * 3 * 4).reshape({2, 3, 4});\n\n// Basic indexing\nauto row = tensor.index({0}); // tensor[0]\nauto elem = tensor.index({1, 2, 3}); // tensor[1, 2, 3]\n\n// Slicing\nauto sliced = tensor.index({Slice(), Slice(0, 2)}); // tensor[:, 0:2]\n\n// None (unsqueeze) and Ellipsis\nauto unsqueezed = tensor.index({None}); // tensor[None]\nauto last_dim = tensor.index({Ellipsis, -1}); // tensor[..., -1]\n\n// Boolean mask indexing\nauto mask = tensor > 10;\nauto selected = tensor.index({mask}); // tensor[tensor > 10]\n\n// Integer tensor (fancy) indexing\nauto idx = torch::tensor({0, 2});\nauto gathered = tensor.index({Slice(), idx}); // tensor[:, [0, 2]]\n\n// Setting values\ntensor.index_put_({0, Slice(), 0}, 99); // tensor[0, :, 0] = 99\ntensor.index_put_({mask}, 0); // tensor[tensor > 10] = 0\n```\n\n---\n\n# Tensor Class\n\nThe `at::Tensor` class is the primary tensor class in ATen, representing\na multi-dimensional array with a specific data type and device.\n\n## Tensor\n\nclass at::Tensor\n\nThe primary tensor class in ATen. Represents a multi-dimensional array\nwith a specific data type and device.\n\nTensor()\n\nDefault constructor. Creates an undefined tensor.\n\nint64_t dim() const\n\nReturns the number of dimensions of the tensor.\n\nint64_t size(int64_t dim) const\n\nReturns the size of the tensor at the given dimension.\n\nIntArrayRef sizes() const\n\nReturns the sizes of all dimensions.\n\nIntArrayRef strides() const\n\nReturns the strides of all dimensions.\n\nScalarType scalar_type() const\n\nReturns the data type of the tensor.\n\nDevice device() const\n\nReturns the device where the tensor is stored.\n\nbool is_cuda() const\n\nReturns true if the tensor is on a CUDA device.\n\nbool is_cpu() const\n\nReturns true if the tensor is on CPU.\n\nbool requires_grad() const\n\nReturns true if gradients need to be computed for this tensor.\n\nTensor &requires_grad_(bool requires_grad = true)\n\nSets whether gradients should be computed for this tensor.\n\nTensor to(Device device) const\n\nReturns a tensor on the specified device.\n\nTensor to(ScalarType dtype) const\n\nReturns a tensor with the specified data type.\n\nTensor contiguous() const\n\nReturns a contiguous tensor with the same data.\n\nvoid *data_ptr() const\n\nReturns a pointer to the underlying data.\n\n**Example:**\n\n```\n#include <ATen/ATen.h>\n\nat::Tensor a = at::ones({2, 2}, at::kInt);\nat::Tensor b = at::randn({2, 2});\nauto c = a + b.to(at::kInt);\n```\n\n## TensorOptions\n\nclass at::TensorOptions\n\nA class to specify options for tensor creation, including dtype, device,\nlayout, and requires_grad.\n\nTensorOptions()\n\nDefault constructor.\n\nTensorOptions dtype(ScalarType dtype) const\n\nReturns options with the specified data type.\n\nTensorOptions device(Device device) const\n\nReturns options with the specified device.\n\nTensorOptions layout(Layout layout) const\n\nReturns options with the specified layout.\n\nTensorOptions requires_grad(bool requires_grad) const\n\nReturns options with the specified requires_grad setting.\n\n**Example:**\n\n```\nauto options = at::TensorOptions()\n .dtype(at::kFloat)\n .device(at::kCUDA, 0)\n .requires_grad(true);\n\nat::Tensor tensor = at::zeros({3, 4}, options);\n```\n\n## Scalar\n\nclass at::Scalar\n\nRepresents a scalar value that can be converted to various numeric types.\n\nScalar(int64_t v)\n\nConstruct from an integer.\n\nScalar(double v)\n\nConstruct from a double.\n\ntemplate<typename T>\nT to() const\n\nConvert to the specified type.\n\nbool isIntegral(bool includeBool = false) const\n\nReturns true if the scalar is an integral type.\n\nbool isFloatingPoint() const\n\nReturns true if the scalar is a floating point type.\n\n## ScalarType\n\nenum class at::ScalarType\n\nEnumeration of data types supported by tensors.\n\nenumerator Byte\n\n8-bit unsigned integer (uint8_t)\n\nenumerator Char\n\n8-bit signed integer (int8_t)\n\nenumerator Short\n\n16-bit signed integer (int16_t)\n\nenumerator Int\n\n32-bit signed integer (int32_t)\n\nenumerator Long\n\n64-bit signed integer (int64_t)\n\nenumerator Half\n\n16-bit floating point (float16)\n\nenumerator Float\n\n32-bit floating point (float)\n\nenumerator Double\n\n64-bit floating point (double)\n\nenumerator Bool\n\nBoolean type\n\nenumerator BFloat16\n\nBrain floating point (bfloat16)\n\nConvenience constants:\n\n- `at::kByte`, `at::kChar`, `at::kShort`, `at::kInt`, `at::kLong`\n- `at::kHalf`, `at::kFloat`, `at::kDouble`, `at::kBFloat16`\n- `at::kBool`\n\n## DeviceGuard\n\nclass DeviceGuard\n\nRAII guard that sets a certain default device in its constructor, and changes it back to the device that was originally active upon destruction.\n\nThe device is always reset to the one that was active at the time of construction of the guard. Even if you `set_device` after construction, the destructor will still reset the device to the one that was active at construction time.\n\nThis device guard does NOT have an uninitialized state; it is guaranteed to reset a device on exit. If you are in a situation where you *might* want to setup a guard (i.e., are looking for the moral equivalent of std::optional<DeviceGuard>), see [OptionalDeviceGuard](../c10/guards.html#PyTorchclassc10_1_1_optional_device_guard).\n\nPublic Functions\n\nexplicit DeviceGuard() = delete\n\nNo default constructor; see Note [Omitted default constructor from RAII].\n\ninline explicit DeviceGuard([Device](../c10/device.html#_CPPv4N3c106DeviceE) device)\n\nSet the current device to the passed [Device](../c10/device.html#PyTorchstructc10_1_1_device).\n\ninline explicit DeviceGuard([Device](../c10/device.html#_CPPv4N3c106DeviceE) device, const impl::DeviceGuardImplInterface *impl)\n\nThis constructor is for testing only.\n\n~DeviceGuard() = default\n\nDeviceGuard(const DeviceGuard&) = delete\n\nCopy is disallowed.\n\nDeviceGuard &operator=(const DeviceGuard&) = delete\n\nDeviceGuard(DeviceGuard &&other) = delete\n\nMove is disallowed, as [DeviceGuard](../c10/guards.html#PyTorchclassc10_1_1_device_guard) does not have an uninitialized state, which is required for moves on types with nontrivial destructors.\n\nDeviceGuard &operator=(DeviceGuard &&other) = delete\n\ninline void reset_device(at::Device device)\n\nSets the device to the given one.\n\nThe specified device must be consistent with the device type originally specified during guard construction.\n\nTODO: The consistency check here is inconsistent with StreamGuard's behavior with set_stream, where a stream on a different device than the original one isn't an error; we just reset the stream and then switch devices.\n\ninline void reset_device(at::Device device, const impl::DeviceGuardImplInterface *impl)\n\nThis method is for testing only.\n\ninline void set_index(DeviceIndex index)\n\nSets the device index to the given one.\n\nThe device type is inferred from the original device type the guard was constructed with.\n\ninline [Device](../c10/device.html#_CPPv4N3c106DeviceE) original_device() const\n\nReturns the device that was set at the time the guard was constructed.\n\ninline [Device](../c10/device.html#_CPPv4N3c106DeviceE) current_device() const\n\nReturns the most recent device that was set using this device guard, either from construction, or via set_device.\n\n**Example:**\n\n```\n{\n c10::DeviceGuard guard(at::Device(at::kCUDA, 1));\n // Operations here run on CUDA device 1\n auto tensor = at::zeros({2, 2});\n}\n// Previous device is restored\n```\n\n## Layout\n\nenum class at::Layout\n\nEnumeration of tensor memory layouts.\n\nenumerator Strided\n\nDense tensor with strides.\n\nenumerator Sparse\n\nSparse tensor (COO format).\n\nenumerator SparseCsr\n\nSparse tensor in CSR format.\n\nenumerator SparseCsc\n\nSparse tensor in CSC format.\n\n---\n\n# Custom Autograd Functions\n\nPyTorch allows you to define custom autograd functions with custom forward\nand backward implementations.\n\n## Function Base Class\n\ntemplate<class T>\nstruct Function\n\nTo use custom autograd operations, implement a Function subclass with static forward and backward functions:\n\n`forward` can take as many arguments as you want and should return either a variable list or a Variable. Use of any direct Variable arguments will be registered in the graph but no vectors/sets or any other data structures will be traversed. You can use std::optional<Tensor> as one of the arguments and it will be registered as a variable in the graph if the argument has a value. It should take a pointer to `torch::autograd::AutogradContext` as the first argument. Variables can be saved in the `ctx` using `ctx->save_for_backward` (see `torch::autograd::AutogradContext::save_for_backward`) and other data can be saved in the `ctx->saved_data` map (see `torch::autograd::AutogradContext::saved_data`) in the form of `<std::string, at::IValue>` pairs.\n\n`backward` should take a pointer to `torch::autograd::AutogradContext` and a variable list containing as many Variables as there were outputs from `forward` as arguments. It should return as many Variables as there were inputs with each of them containing the gradient w.r.t. its corresponding input. Variables saved in `forward` can be accessed with `ctx->get_saved_variables` (see `torch::autograd::AutogradContext::get_saved_variables`) and other saved data can be accessed from `ctx->saved_data`. To enable compiled autograd support (torch.compile for backward) for your custom autograd operation, you can set MyFunction::is_traceable (see Function::istraceable notes below).\n\nFor example:\n\n```\nclass MyFunction : public Function<MyFunction> {\n public:\n static constexpr bool is_traceable = true;\n\n static variable_list forward(AutogradContext *ctx, int n, Variable var) {\n // Save data for backward in context\n ctx->saved_data[\"n\"] = n;\n var.mul_(n);\n // Mark var as modified by inplace operation\n ctx->mark_dirty({var});\n return {var};\n }\n\n static variable_list backward(AutogradContext *ctx, variable_list\n grad_output) {\n // Use data saved in forward\n auto n = ctx->saved_data[\"n\"].toInt();\n return {grad_output[0]*n};\n }\n};\n```\n\nTo use `MyFunction`:\n\n```\nVariable x;\nauto y = MyFunction::apply(6, x);\n// Example backward call\ny[0].sum().backward();\n```\n\nPublic Static Functions\n\ntemplate<typename X = T, typename ...Args>\nstatic auto apply(Args&&... args) -> std::enable_if_t<std::is_same_v<X, T>, forward_t<X, Args...>>\n\nPublic Static Attributes\n\nstatic constexpr bool is_traceable = false\n\n## AutogradContext\n\nstruct AutogradContext\n\nContext to save information during `forward` that can be accessed in `backward` in custom autograd operations (see `torch::autograd::Function` for details).\n\nPublic Functions\n\nAutogradContext() = default\n\nAutogradContext(const AutogradContext &other) = delete\n\nAutogradContext &operator=(const AutogradContext &other) = delete\n\nAutogradContext(AutogradContext &&other) = delete\n\nAutogradContext &operator=(AutogradContext &&other) = delete\n\n~AutogradContext() = default\n\nAutogradContext(PackedArgs &packed_args)\n\nvoid save_for_backward(variable_list to_save)\n\nSaves the list of variables for a future call to `backward`.\n\nThis should be called at most once from inside of `forward`.\n\nvoid mark_dirty(const variable_list &inputs)\n\nMarks variables in the list as modified in an in-place operation.\n\nThis should be called at most once from inside of `forward` and all arguments should be inputs.\n\nvoid mark_non_differentiable(const variable_list &outputs)\n\nMarks outputs in the list as not requiring gradients.\n\nThis should be called at most once from inside of `forward` and all arguments should be outputs.\n\nvoid set_materialize_grads(bool value)\n\nvariable_list get_saved_variables() const\n\nGet the list of variables that were saved in `forward` using `save_for_backward()`.\n\nBefore returning them to the user, a check is made to ensure that they were not modified by any in-place operations.\n\nconst std::unordered_set<at::TensorImpl*> &get_and_bump_dirty() const\n\nconst std::unordered_set<at::TensorImpl*> &get_non_differentiable() const\n\nbool needs_input_grad(size_t output_edge_index) const\n\nExpose the Node's `task_should_compute_output` method to the cpp custom autograd Function as `needs_input_grad`.\n\nbool needs_input_grad(std::initializer_list<IndexRange> idxs) const\n\nPublic Members\n\nska::flat_hash_map<std::string, at::IValue> saved_data\n\nCan be used to save non-variable data for `backward`.\n\n## Creating Custom Functions\n\nTo create a custom autograd function, inherit from `torch::autograd::Function`\nand implement the static `forward` and `backward` methods:\n\n**Example:**\n\n```\nclass MyReLU : public torch::autograd::Function<MyReLU> {\n public:\n static torch::Tensor forward(\n torch::autograd::AutogradContext* ctx,\n torch::Tensor input) {\n ctx->save_for_backward({input});\n return input.clamp_min(0);\n }\n\n static torch::autograd::variable_list backward(\n torch::autograd::AutogradContext* ctx,\n torch::autograd::variable_list grad_outputs) {\n auto saved = ctx->get_saved_variables();\n auto input = saved[0];\n auto grad_output = grad_outputs[0];\n auto grad_input = grad_output * (input > 0).to(grad_output.dtype());\n return {grad_input};\n }\n};\n\n// Usage\nauto output = MyReLU::apply(input);\n```\n\n## Custom Kernels and AutoDispatchBelowADInplaceOrView\n\nFor users implementing custom kernels who want to redispatch below `Autograd` dispatch\nkeys, use `at::AutoDispatchBelowADInplaceOrView` instead of `InferenceMode`:\n\n```\nclass ROIAlignFunction : public torch::autograd::Function<ROIAlignFunction> {\n public:\n static torch::autograd::variable_list forward(\n torch::autograd::AutogradContext* ctx,\n const torch::autograd::Variable& input,\n const torch::autograd::Variable& rois,\n double spatial_scale,\n int64_t pooled_height,\n int64_t pooled_width,\n int64_t sampling_ratio,\n bool aligned) {\n ctx->saved_data[\"spatial_scale\"] = spatial_scale;\n ctx->saved_data[\"pooled_height\"] = pooled_height;\n ctx->saved_data[\"pooled_width\"] = pooled_width;\n ctx->saved_data[\"sampling_ratio\"] = sampling_ratio;\n ctx->saved_data[\"aligned\"] = aligned;\n ctx->saved_data[\"input_shape\"] = input.sizes();\n ctx->save_for_backward({rois});\n\n at::AutoDispatchBelowADInplaceOrView guard;\n auto result = roi_align(\n input, rois, spatial_scale, pooled_height,\n pooled_width, sampling_ratio, aligned);\n return {result};\n }\n};\n```\n\nFor customized inplace and view kernels, see the\n[custom kernel tutorial](https://pytorch.org/tutorials/advanced/cpp_extension.html#backward-pass)\nfor more details.\n\n---\n\n# Gradient Computation\n\nPyTorch provides functions for computing gradients of tensors with respect\nto graph leaves.\n\n## Gradient Functions\n\nvoid torch::autograd::backward(const variable_list &tensors, const variable_list &grad_tensors = {}, std::optional<bool> retain_graph = std::nullopt, bool create_graph = false, const variable_list &inputs = {})\n\nComputes the sum of gradients of given tensors with respect to graph leaves.\n\nThe graph is differentiated using the chain rule. If any of `tensors`\nare non-scalar (i.e. their data has more than one element) and require\ngradient, then the Jacobian-vector product would be computed, in this case\nthe function additionally requires specifying `grad_tensors`. It should be a\nsequence of matching length, that contains the \"vector\" in the\nJacobian-vector product, usually the gradient of the differentiated function\nw.r.t. corresponding tensors (`torch::Tensor()` is an acceptable value for\nall tensors that don't need gradient tensors).\n\nThis function accumulates gradients in the leaves -- you might need to zero\nthem before calling it.\n\n:param tensors: Tensors of which the derivative will be computed.\n:param grad_tensors: The \"vector\" in the Jacobian-vector product, usually\ngradients w.r.t. each element of corresponding tensors.\n`torch::Tensor()` values can be specified for scalar Tensors or ones\nthat don't require grad. If a `torch::Tensor()` value would be\nacceptable for all grad_tensors, then this argument is optional.\n:param retain_graph: If `false`, the graph used to compute the grad will\nbe freed. Note that in nearly all cases setting this option to `true`\nis not needed and often can be worked around in a much more efficient\nway. Defaults to the value of `create_graph`.\n:param create_graph: If `true`, graph of the derivative will be\nconstructed, allowing to compute higher order derivative products.\nDefaults to `false`.\n:param inputs: Inputs w.r.t. which the gradient will be accumulated into\n`at::Tensor::grad`. All other Tensors will be ignored. If not\nprovided, the gradient is accumulated into all the leaf Tensors that\nwere used to compute `tensors`.\n\nvariable_list torch::autograd::grad(const variable_list &outputs, const variable_list &inputs, const variable_list &grad_outputs = {}, std::optional<bool> retain_graph = std::nullopt, bool create_graph = false, bool allow_unused = false)\n\nComputes and returns the sum of gradients of outputs with respect to the inputs.\n\n`grad_outputs` should be a sequence of length matching `output` containing the \"vector\" in Jacobian-vector product, usually the pre-computed gradients w.r.t. each of the outputs. If an output doesn't require_grad, then the gradient can be `torch::Tensor()`).\n\nParameters:\n\n- **outputs** - outputs of the differentiated function.\n- **inputs** - Inputs w.r.t. which the gradient will be returned (and not accumulated into `at::Tensor::grad`).\n- **grad_outputs** - The \"vector\" in the Jacobian-vector product. Usually gradients w.r.t. each output. `torch::Tensor()` values can be specified for scalar Tensors or ones that don't require grad. If a `torch::Tensor()` value would be acceptable for all grad_tensors, then this argument is optional. Default: `{}`.\n- **retain_graph** - If `false`, the graph used to compute the grad will be freed. Note that in nearly all cases setting this option to `true` is not needed and often can be worked around in a much more efficient way. Defaults to the value of `create_graph`.\n- **create_graph** - If `true`, graph of the derivative will be constructed, allowing to compute higher order derivative products. Default: `false`.\n- **allow_unused** - If `false`, specifying inputs that were not used when computing outputs (and therefore their grad is always zero) is an error. Defaults to `false`.\n\n**Example:**\n\n```\n#include <torch/torch.h>\n\nauto x = torch::randn({2, 2}, torch::requires_grad());\nauto y = x * x;\nauto z = y.sum();\n\n// Compute gradients\nz.backward();\nstd::cout << x.grad() << std::endl;\n\n// Or use grad() for specific outputs\nauto grads = torch::autograd::grad({z}, {x});\n```\n\n## Tensor Gradient Methods\n\nTensors have built-in methods for gradient computation:\n\n```\n// Enable gradient tracking\nauto x = torch::randn({2, 2}).requires_grad_(true);\n\n// Check if gradient is required\nbool needs_grad = x.requires_grad();\n\n// Access the gradient after backward\nauto grad = x.grad();\n\n// Detach from computation graph\nauto x_detached = x.detach();\n```\n\n---\n\n# Autograd: Automatic Differentiation\n\nPyTorch's autograd system provides automatic differentiation for all operations\non tensors. It records operations on tensors to build a computational graph,\nthen computes gradients automatically via backpropagation.\n\n**When to use Autograd:**\n\n- When training neural networks (gradients are computed automatically)\n- When implementing custom backward passes for specialized operations\n- When you need fine-grained control over gradient computation\n\n**Basic usage:**\n\n```\n#include <torch/torch.h>\n\n// Create tensor with gradient tracking\nauto x = torch::randn({2, 2}, torch::requires_grad());\nauto y = x * x;\nauto z = y.sum();\n\n// Compute gradients via backpropagation\nz.backward();\nstd::cout << x.grad() << std::endl; // dz/dx = 2x\n\n// Disable gradient tracking for inference\n{\n torch::NoGradGuard no_grad;\n auto result = model->forward(input); // No gradients computed\n}\n```\n\n## Header Files\n\n- `torch/csrc/autograd/autograd.h` - High-level autograd API\n- `torch/csrc/autograd/function.h` - Custom autograd functions\n- `torch/csrc/autograd/grad_mode.h` - Gradient computation modes\n- `torch/csrc/api/include/torch/autograd.h` - C++ Frontend autograd\n\n## Autograd Categories\n\n- [Gradient Computation](gradient.html)\n- [Custom Autograd Functions](custom_functions.html)\n- [Gradient Modes](modes.html)\n\n---\n\n# Gradient Modes\n\nPyTorch provides RAII guards to control gradient computation behavior.\n\n## NoGradGuard\n\nclass torch::NoGradGuard\n\nRAII guard that disables gradient computation within its scope.\n\nNoGradGuard()\n\nDisables gradient computation.\n\n~NoGradGuard()\n\nRestores previous gradient mode.\n\n**Example:**\n\n```\n{\n torch::NoGradGuard no_grad;\n // No gradients computed in this scope\n auto result = model->forward(input);\n}\n```\n\n## InferenceMode\n\n`c10::InferenceMode` is a RAII guard analogous to `NoGradMode` designed for use\nwhen you are certain your operations will have no interactions with autograd\n(e.g., model inference). Compared to `NoGradMode`, code run under this mode gets\nbetter performance by disabling autograd-related work like view tracking and version\ncounter bumps. However, tensors created inside `InferenceMode` have more limitations\nwhen interacting with the autograd system.\n\nclass c10::InferenceMode\n\nRAII guard that enables inference mode for optimized inference.\nThis is more efficient than NoGradGuard for inference-only workloads.\n\nexplicit InferenceMode(bool enabled = true)\n\nEnables or disables inference mode.\n\n**Inference Tensors:**\n\n`InferenceMode` can be enabled for a given block of code. Inside `InferenceMode`,\nall newly allocated (non-view) tensors are marked as inference tensors. Inference tensors:\n\n- Do not have a version counter, so an error will be raised if you try to read their version\n(e.g., because you saved this tensor for backward).\n- Are immutable outside `InferenceMode`. An error will be raised if you try to:\n\n- Mutate their data outside InferenceMode.\n- Mutate them to `requires_grad=True` outside InferenceMode.\n- To work around this, make a clone outside `InferenceMode` to get a normal tensor before mutating.\n\nA non-view tensor is an inference tensor if and only if it was allocated inside `InferenceMode`.\nA view tensor is an inference tensor if and only if it is a view of an inference tensor.\n\n**Performance Guarantees:**\n\nInside an `InferenceMode` block:\n\n- Like `NoGradMode`, all operations do not record `grad_fn` even if their inputs have\n`requires_grad=True`. This applies to both inference tensors and normal tensors.\n- View operations on inference tensors do not perform view tracking. View and non-view\ninference tensors are indistinguishable.\n- Inplace operations on inference tensors are guaranteed not to do a version bump.\n\nFor more implementation details, see the [RFC-0011-InferenceMode](https://github.com/pytorch/rfcs/pull/17).\n\n**Basic Example:**\n\n```\n{\n c10::InferenceMode guard;\n // Optimized inference without gradient tracking\n auto result = model->forward(input);\n}\n```\n\n**Inference Workload Example:**\n\n```\nc10::InferenceMode guard;\nmodel.load_jit(saved_model);\nauto inputs = preprocess_tensors(data);\nauto out = model.forward(inputs);\nauto outputs = postprocess_tensors(out);\n```\n\n**Nested InferenceMode:**\n\nUnlike some other guards, `InferenceMode` can be nested with different enabled/disabled states:\n\n```\n{\n c10::InferenceMode guard(true);\n // InferenceMode is on\n {\n c10::InferenceMode guard(false);\n // InferenceMode is off\n }\n // InferenceMode is on\n}\n// InferenceMode is off\n```\n\n## InferenceMode vs NoGradMode\n\n`InferenceMode` is preferred over `NoGradMode` for pure inference workloads because\nit provides better performance. Key differences:\n\n- Both guards affect tensor execution to skip work not related to inference, but\n`InferenceMode` also affects tensor creation while `NoGradMode` doesn't.\n- Tensors created inside `InferenceMode` are marked as inference tensors with\ncertain limitations that apply after exiting `InferenceMode`.\n- `InferenceMode` can be nested with enabled/disabled states.\n\n## Migrating from AutoNonVariableTypeMode\n\nThe legacy `AutoNonVariableTypeMode` guard (now renamed to\n`AutoDispatchBelowADInplaceOrView`) was commonly used for inference workloads\nbut is unsafe -- it can silently bypass safety checks and produce wrong results.\n\n- **For inference-only workloads** (e.g. loading a pretrained JIT model and\nrunning inference in C++ runtime), use `c10::InferenceMode` as a drop-in\nreplacement. It preserves the performance characteristics while providing\ncorrectness guarantees.\n- **For custom autograd kernels** that need to redispatch below the Autograd\ndispatch key, use `AutoDispatchBelowADInplaceOrView` instead:\n\n```\nclass ROIAlignFunction : public torch::autograd::Function<ROIAlignFunction> {\n public:\n static torch::autograd::variable_list forward(\n torch::autograd::AutogradContext* ctx,\n const torch::autograd::Variable& input,\n const torch::autograd::Variable& rois,\n double spatial_scale, int64_t pooled_height,\n int64_t pooled_width, int64_t sampling_ratio, bool aligned) {\n ctx->saved_data[\"spatial_scale\"] = spatial_scale;\n ctx->save_for_backward({rois});\n at::AutoDispatchBelowADInplaceOrView guard;\n auto result = roi_align(input, rois, spatial_scale,\n pooled_height, pooled_width, sampling_ratio, aligned);\n return {result};\n }\n};\n```\n\n---\n\n# Device and DeviceType\n\nPyTorch provides device abstractions for writing code that works across\nCPU, CUDA, and other backends.\n\n## Device\n\nstruct Device\n\nRepresents a compute device on which a tensor is located.\n\nA device is uniquely identified by a type, which specifies the type of machine it is (e.g. CPU or CUDA GPU), and a device index or ordinal, which identifies the specific compute device when there is more than one of a certain type. The device index is optional, and in its defaulted state represents (abstractly) \"the current device\". Further, there are two constraints on the value of the device index, if one is explicitly stored:\n\n1. A negative index represents the current device, a non-negative index represents a specific, concrete device,\n2. When the device type is CPU, the device index must be zero.\n\nPublic Types\n\nusing Type = DeviceType\n\nPublic Functions\n\ninline Device(DeviceType type, DeviceIndex index = -1)\n\nConstructs a new `Device` from a `DeviceType` and an optional device index.\n\nDevice(const std::string &device_string)\n\nConstructs a `Device` from a string description, for convenience.\n\nThe string supplied must follow the following schema: `(cpu|cuda)[:<device-index>]` where `cpu` or `cuda` specifies the device type, and `:<device-index>` optionally specifies a device index.\n\ninline bool operator==(const Device &other) const noexcept\n\nReturns true if the type and index of this `Device` matches that of `other`.\n\ninline bool operator!=(const Device &other) const noexcept\n\nReturns true if the type or index of this `Device` differs from that of `other`.\n\ninline void set_index(DeviceIndex index)\n\nSets the device index.\n\ninline DeviceType type() const noexcept\n\nReturns the type of device this is.\n\ninline DeviceIndex index() const noexcept\n\nReturns the optional index.\n\ninline bool has_index() const noexcept\n\nReturns true if the device has a non-default index.\n\ninline bool is_cuda() const noexcept\n\nReturn true if the device is of CUDA type.\n\ninline bool is_privateuseone() const noexcept\n\nReturn true if the device is of PrivateUse1 type.\n\ninline bool is_mps() const noexcept\n\nReturn true if the device is of MPS type.\n\ninline bool is_hip() const noexcept\n\nReturn true if the device is of HIP type.\n\ninline bool is_ve() const noexcept\n\nReturn true if the device is of VE type.\n\ninline bool is_xpu() const noexcept\n\nReturn true if the device is of XPU type.\n\ninline bool is_ipu() const noexcept\n\nReturn true if the device is of IPU type.\n\ninline bool is_xla() const noexcept\n\nReturn true if the device is of XLA type.\n\ninline bool is_mtia() const noexcept\n\nReturn true if the device is of MTIA type.\n\ninline bool is_hpu() const noexcept\n\nReturn true if the device is of HPU type.\n\ninline bool is_lazy() const noexcept\n\nReturn true if the device is of Lazy type.\n\ninline bool is_vulkan() const noexcept\n\nReturn true if the device is of Vulkan type.\n\ninline bool is_metal() const noexcept\n\nReturn true if the device is of Metal type.\n\ninline bool is_maia() const noexcept\n\nReturn true if the device is of MAIA type.\n\ninline bool is_meta() const noexcept\n\nReturn true if the device is of META type.\n\ninline bool is_cpu() const noexcept\n\nReturn true if the device is of CPU type.\n\ninline bool supports_as_strided() const noexcept\n\nReturn true if the device supports arbitrary strides.\n\nstd::string str() const\n\nSame string as returned from operator<<.\n\n**Example:**\n\n```\nc10::Device cpu_device(c10::kCPU);\nc10::Device cuda_device(c10::kCUDA, 0); // CUDA device 0\n\nif (cuda_device.is_cuda()) {\n std::cout << \"Using CUDA device \" << cuda_device.index() << std::endl;\n}\n```\n\n## DeviceType\n\nenum class c10::DeviceType\n\nEnumeration of supported device types.\n\nenumerator CPU = 0\n\nCPU device.\n\nenumerator CUDA = 1\n\nNVIDIA CUDA GPU.\n\nenumerator HIP = 6\n\nAMD HIP GPU.\n\nenumerator XLA = 9\n\nXLA / TPU.\n\nenumerator Vulkan = 10\n\nVulkan GPU.\n\nenumerator Metal = 11\n\nApple Metal GPU.\n\nenumerator XPU = 12\n\nIntel XPU GPU.\n\nenumerator MPS = 13\n\nApple Metal Performance Shaders.\n\nenumerator Meta = 14\n\nMeta tensors (shape only, no data).\n\nenumerator HPU = 15\n\nHabana HPU.\n\nenumerator Lazy = 17\n\nLazy tensors.\n\nenumerator IPU = 18\n\nGraphcore IPU.\n\nenumerator MTIA = 19\n\nMeta training and inference accelerator.\n\nenumerator PrivateUse1 = 20\n\nCustom backend registered via `c10::register_privateuse1_backend()`.\n\nConvenience constants:\n\n- `c10::kCPU`, `c10::kCUDA`, `c10::kHIP`\n- `c10::kXLA`, `c10::kVulkan`, `c10::kMetal`\n- `c10::kXPU`, `c10::kMPS`, `c10::kMeta`\n- `c10::kHPU`, `c10::kLazy`, `c10::kIPU`, `c10::kMTIA`\n- `c10::kPrivateUse1`\n\n---\n\n# Device Guards\n\nC10 provides device-agnostic RAII guards for managing the current device\ncontext. These guards work across all backends (CUDA, XPU, etc.) and\nautomatically restore the previous device when they go out of scope.\n\nFor backend-specific guards, see [CUDA Guards](../cuda/guards.html) and [XPU Support](../xpu/index.html).\n\n## DeviceGuard\n\nclass DeviceGuard\n\nRAII guard that sets a certain default device in its constructor, and changes it back to the device that was originally active upon destruction.\n\nThe device is always reset to the one that was active at the time of construction of the guard. Even if you `set_device` after construction, the destructor will still reset the device to the one that was active at construction time.\n\nThis device guard does NOT have an uninitialized state; it is guaranteed to reset a device on exit. If you are in a situation where you *might* want to setup a guard (i.e., are looking for the moral equivalent of std::optional<DeviceGuard>), see OptionalDeviceGuard.\n\nPublic Functions\n\nexplicit DeviceGuard() = delete\n\nNo default constructor; see Note [Omitted default constructor from RAII].\n\ninline explicit DeviceGuard([Device](device.html#_CPPv4N3c106DeviceE) device)\n\nSet the current device to the passed [Device](device.html#PyTorchstructc10_1_1_device).\n\ninline explicit DeviceGuard([Device](device.html#_CPPv4N3c106DeviceE) device, const [impl](../aten/tensor.html#_CPPv4N3c1011DeviceGuard11DeviceGuardE6DevicePKN4impl24DeviceGuardImplInterfaceE)::DeviceGuardImplInterface *impl)\n\nThis constructor is for testing only.\n\n~DeviceGuard() = default\n\nDeviceGuard(const [DeviceGuard](../aten/tensor.html#_CPPv4N3c1011DeviceGuard11DeviceGuardERK11DeviceGuard)&) = delete\n\nCopy is disallowed.\n\n[DeviceGuard](../aten/tensor.html#_CPPv4N3c1011DeviceGuardE) &operator=(const [DeviceGuard](../aten/tensor.html#_CPPv4N3c1011DeviceGuardE)&) = delete\n\nDeviceGuard([DeviceGuard](../aten/tensor.html#_CPPv4N3c1011DeviceGuard11DeviceGuardERR11DeviceGuard) &&other) = delete\n\nMove is disallowed, as DeviceGuard does not have an uninitialized state, which is required for moves on types with nontrivial destructors.\n\n[DeviceGuard](../aten/tensor.html#_CPPv4N3c1011DeviceGuardE) &operator=([DeviceGuard](../aten/tensor.html#_CPPv4N3c1011DeviceGuardE) &&other) = delete\n\ninline void reset_device(at::Device device)\n\nSets the device to the given one.\n\nThe specified device must be consistent with the device type originally specified during guard construction.\n\nTODO: The consistency check here is inconsistent with StreamGuard's behavior with set_stream, where a stream on a different device than the original one isn't an error; we just reset the stream and then switch devices.\n\ninline void reset_device(at::Device device, const [impl](../aten/tensor.html#_CPPv4N3c1011DeviceGuard12reset_deviceEN2at6DeviceEPKN4impl24DeviceGuardImplInterfaceE)::DeviceGuardImplInterface *impl)\n\nThis method is for testing only.\n\ninline void set_index(DeviceIndex index)\n\nSets the device index to the given one.\n\nThe device type is inferred from the original device type the guard was constructed with.\n\ninline [Device](device.html#_CPPv4N3c106DeviceE) original_device() const\n\nReturns the device that was set at the time the guard was constructed.\n\ninline [Device](device.html#_CPPv4N3c106DeviceE) current_device() const\n\nReturns the most recent device that was set using this device guard, either from construction, or via set_device.\n\n**Example:**\n\n```\n#include <c10/core/DeviceGuard.h>\n\n{\n c10::DeviceGuard guard(c10::Device(c10::kCUDA, 1));\n // All operations here run on CUDA device 1\n}\n// Previous device is restored\n```\n\n## OptionalDeviceGuard\n\nclass OptionalDeviceGuard\n\nA OptionalDeviceGuard is an RAII class that sets a device to some value on initialization, and resets the device to its original value on destruction.\n\nMorally, a OptionalDeviceGuard is equivalent to std::optional<DeviceGuard>, but with extra constructors and methods as appropriate.\n\nBesides its obvious use (optionally applying a DeviceGuard), OptionalDeviceGuard is often also used for the following idiom:\n\nOptionalDeviceGuard g; for (const auto& t : tensors) { g.set_device(t.device()); do_something_with(t); }\n\nThis usage is marginally more efficient than constructing a DeviceGuard every iteration of the for loop, as it avoids an unnecessary device reset.\n\nUnlike DeviceGuard, a OptionalDeviceGuard may be uninitialized. This occurs when you use the nullary constructor, or pass a nullopt to the constructor. Uninitialized OptionalDeviceGuards do *nothing*; they do not know what the original device was and they do not reset on destruction. This is why original_device() and current_device() return std::optional<Device> rather than [Device](device.html#PyTorchstructc10_1_1_device) (as they do in DeviceGuard), and also is why we didn't just provide OptionalDeviceGuard by default and hide DeviceGuard from users.\n\nThe semantics of an OptionalDeviceGuard are exactly explained by thinking of it as an std::optional<DeviceGuard>. In particular, an initialized OptionalDeviceGuard doesn't restore device to its value at construction; it restores device to its value *at initialization*. So if you have the program:\n\n```\nsetDevice(1);\nOptionalDeviceGuard g;\nsetDevice(2);\ng.reset_device(Device(DeviceType::CUDA, 3)); // initializes!\n```\n\n On destruction, g will reset device to 2, rather than 1.\n\nAn uninitialized OptionalDeviceGuard is distinct from a (initialized) DeviceGuard whose original_device_ and current_device_ match, since the DeviceGuard will still reset the device to original_device_.\n\nPublic Functions\n\nexplicit OptionalDeviceGuard() = default\n\nCreate an uninitialized guard. Set the guard later using reset_device.\n\ninline explicit OptionalDeviceGuard([Device](device.html#_CPPv4N3c106DeviceE) device)\n\nInitialize the guard, setting the current device to the passed [Device](device.html#PyTorchstructc10_1_1_device).\n\ninline explicit OptionalDeviceGuard(std::optional<[Device](device.html#_CPPv4N3c106DeviceE)> device)\n\nInitialize the guard if a [Device](device.html#PyTorchstructc10_1_1_device) is passed; otherwise leave the guard uninitialized.\n\ninline explicit OptionalDeviceGuard([Device](device.html#_CPPv4N3c106DeviceE) device, const impl::DeviceGuardImplInterface *impl)\n\nConstructor for testing only.\n\n~OptionalDeviceGuard() = default\n\nOptionalDeviceGuard(const OptionalDeviceGuard&) = delete\n\nCopy is disallowed.\n\nOptionalDeviceGuard &operator=(const OptionalDeviceGuard&) = delete\n\nOptionalDeviceGuard(OptionalDeviceGuard &&other) = delete\n\nMove is disallowed See Note [Explicit initialization of optional fields] and // Note [Move construction for RAII guards is tricky] for rationale.\n\nOptionalDeviceGuard &operator=(OptionalDeviceGuard &&other) = delete\n\ninline void reset_device(at::Device device)\n\nSets the device to the given one.\n\nThe specified device must be consistent with the device type originally specified during guard construction.\n\ninline void reset_device(at::Device device, const impl::DeviceGuardImplInterface *impl)\n\nFor testing only.\n\ninline std::optional<[Device](device.html#_CPPv4N3c106DeviceE)> original_device() const\n\nReturns the device that was set at the time the guard was constructed.\n\ninline std::optional<[Device](device.html#_CPPv4N3c106DeviceE)> current_device() const\n\nReturns the most recent device that was set using this device guard, either from construction, or via reset_device.\n\n**Example:**\n\n```\n#include <c10/core/DeviceGuard.h>\n\nc10::OptionalDeviceGuard guard;\nif (use_gpu) {\n guard.reset_device(c10::Device(c10::kCUDA, 0));\n}\n// Guard only restores device\n\n<!-- File: llms.txt (Tokens: ~820 | Category: root-instruction) -->\n# PyTorch\n\n> PyTorch documentation.\n\n## Docs\n\n- [Tensor Accessors](api/aten/accessors.md)\n- [Tensor Creation](api/aten/creation.md)\n- [ATen: Tensor Library](api/aten/index.md)\n- [Tensor Indexing](api/aten/indexing.md)\n- [Tensor Class](api/aten/tensor.md)\n- [Custom Autograd Functions](api/autograd/custom_functions.md)\n- [Gradient Computation](api/autograd/gradient.md)\n- [Autograd: Automatic Differentiation](api/autograd/index.md)\n- [Gradient Modes](api/autograd/modes.md)\n- [Device and DeviceType](api/c10/device.md)\n- [Device Guards](api/c10/guards.md)\n- [C10: Core Utilities](api/c10/index.md)\n- [Streams](api/c10/streams.md)\n- [Core Types](api/c10/types.md)\n- [Utilities](api/c10/utilities.md)\n- [CUDA Guards](api/cuda/guards.md)\n- [CUDA Support](api/cuda/index.md)\n- [CUDA Streams](api/cuda/streams.md)\n- [CUDA Utility Functions](api/cuda/utilities.md)\n- [DataLoader](api/data/dataloader.md)\n- [Datasets](api/data/datasets.md)\n- [Data Loading (torch::data)](api/data/index.md)\n- [Samplers](api/data/samplers.md)\n- [Transforms](api/data/transforms.md)\n- [C++ API Reference](api/index.md)\n- [Custom Classes](api/library/custom_classes.md)\n- [Torch Library API](api/library/index.md)\n- [Operator Registration](api/library/registration.md)\n- [Library Versioning](api/library/versioning.md)\n- [Activation Functions](api/nn/activation.md)\n- [Containers](api/nn/containers.md)\n- [Convolution Layers](api/nn/convolution.md)\n- [Dropout Layers](api/nn/dropout.md)\n- [Embedding Layers](api/nn/embedding.md)\n- [Functional API](api/nn/functional.md)\n- [Neural Network Modules (torch::nn)](api/nn/index.md)\n- [Linear Layers](api/nn/linear.md)\n- [Loss Functions](api/nn/loss.md)\n- [Normalization Layers](api/nn/normalization.md)\n- [Pooling Layers](api/nn/pooling.md)\n- [Recurrent Layers](api/nn/recurrent.md)\n- [Transformer Layers](api/nn/transformer.md)\n- [Utilities](api/nn/utilities.md)\n- [Adaptive Learning Rate Optimizers](api/optim/adaptive.md)\n- [Gradient Descent Optimizers](api/optim/gradient_descent.md)\n- [Optimizers (torch::optim)](api/optim/index.md)\n- [Learning Rate Schedulers](api/optim/schedulers.md)\n- [Second-Order Optimizers](api/optim/second_order.md)\n- [Archives](api/serialize/archives.md)\n- [Checkpoints](api/serialize/checkpoints.md)\n- [Serialization (torch::serialize)](api/serialize/index.md)\n- [Saving and Loading](api/serialize/save_load.md)\n- [Torch Stable API](api/stable/index.md)\n- [Stable Operators](api/stable/operators.md)\n- [Library Registration Macros](api/stable/registration.md)\n- [Utilities](api/stable/utilities.md)\n- [XPU Support](api/xpu/index.md)\n- [XPU Streams](api/xpu/streams.md)\n- [XPU Utility Functions](api/xpu/utilities.md)\n- [FAQ](faq.md)\n- [The C++ Frontend](frontend.md)\n- [PyTorch C++ API](index.md)\n- [Installing C++ Distributions of PyTorch](installing.md)\n\n<!-- File: .cursorrules (cpp.mdc) (Tokens: ~1116 | Category: cursor-rule) -->\n---\ndescription: Guide Cursor to write modern C++ and CMake code with clear structure, RAII, const-correctness, and safe error handling.\nglobs: **/*.c,**/*.cpp,**/*.h,**/*.hpp,**/*.cxx,CMakeLists.txt,*.cmake,conanfile.txt,Makefile,**/*.cc\nalwaysApply: false\n---\n# C++ Programming Guidelines\n\n## Basic Principles\n\n- Use English for all code and documentation.\n- Always declare the type of each variable and function (parameters and return value).\n- Create necessary types and classes.\n- Use Doxygen style comments to document public classes and methods.\n- Don't leave blank lines within a function.\n- Follow the one-definition rule (ODR).\n\n## Nomenclature\n\n- Use PascalCase for classes and structures.\n- Use camelCase for variables, functions, and methods.\n- Use ALL_CAPS for constants and macros.\n- Use snake_case for file and directory names.\n- Use UPPERCASE for environment variables.\n- Avoid magic numbers and define constants.\n- Start each function with a verb.\n- Use verbs for boolean variables. Example: isLoading, hasError, canDelete, etc.\n- Use complete words instead of abbreviations and ensure correct spelling.\n  - Except for standard abbreviations like API, URL, etc.\n  - Except for well-known abbreviations:\n    - i, j, k for loops\n    - err for errors\n    - ctx for contexts\n    - req, res for request/response parameters\n\n## Functions\n\n- Write short functions with a single purpose. Less than 20 instructions.\n- Name functions with a verb and something else.\n- If it returns a boolean, use isX or hasX, canX, etc.\n- If it doesn't return anything (void), use executeX or saveX, etc.\n- Avoid nesting blocks by:\n  - Early checks and returns.\n  - Extraction to utility functions.\n- Use standard library algorithms (std::for_each, std::transform, std::find, etc.) to avoid function nesting.\n- Use lambda functions for simple operations.\n- Use named functions for non-simple operations.\n- Use default parameter values instead of checking for null or nullptr.\n- Reduce function parameters using structs or classes\n  - Use an object to pass multiple parameters.\n  - Use an object to return multiple results.\n  - Declare necessary types for input arguments and output.\n- Use a single level of abstraction.\n\n## Data\n\n- Don't abuse primitive types and encapsulate data in composite types.\n- Avoid data validations in functions and use classes with internal validation.\n- Prefer immutability for data.\n- Use const for data that doesn't change.\n- Use constexpr for compile-time constants.\n- Use std::optional for possibly null values.\n\n## Classes\n\n- Follow SOLID principles.\n- Prefer composition over inheritance.\n- Declare interfaces as abstract classes or concepts.\n- Write small classes with a single purpose.\n  - Less than 200 instructions.\n  - Less than 10 public methods.\n  - Less than 10 properties.\n- Use the Rule of Five (or Rule of Zero) for resource management.\n- Make member variables private and provide getters/setters where necessary.\n- Use const-correctness for member functions.\n\n## Exceptions\n\n- Use exceptions to handle errors you don't expect.\n- If you catch an exception, it should be to:\n  - Fix an expected problem.\n  - Add context.\n  - Otherwise, use a global handler.\n- Use std::optional, std::expected, or error codes for expected failures.\n\n## Memory Management\n\n- Prefer smart pointers (std::unique_ptr, std::shared_ptr) over raw pointers.\n- Use RAII (Resource Acquisition Is Initialization) principles.\n- Avoid memory leaks by proper resource management.\n- Use std::vector and other standard containers instead of C-style arrays.\n\n## Testing\n\n- Follow the Arrange-Act-Assert convention for tests.\n- Name test variables clearly.\n- Follow the convention: inputX, mockX, actualX, expectedX, etc.\n- Write unit tests for each public function.\n- Use test doubles to simulate dependencies.\n  - Except for third-party dependencies that are not expensive to execute.\n- Write integration tests for each module.\n- Follow the Given-When-Then convention.\n\n## Project Structure\n\n- Use modular architecture\n- Organize code into logical directories:\n  - include/ for header files\n  - src/ for source files\n  - test/ for test files\n  - lib/ for libraries\n  - doc/ for documentation\n- Use CMake or similar build system.\n- Separate interface (.h) from implementation (.cpp).\n- Use namespaces to organize code logically.\n- Create a core namespace for foundational components.\n- Create a utils namespace for utility functions.\n\n## Standard Library\n\n- Use the C++ Standard Library whenever possible.\n- Prefer std::string over C-style strings.\n- Use std::vector, std::map, std::unordered_map, etc. for collections.\n- Use std::optional, std::variant, std::any for modern type safety.\n- Use std::filesystem for file operations.\n- Use std::chrono for time-related operations.\n\n## Concurrency\n\n- Use std::thread, std::mutex, std::lock_guard for thread safety.\n- Prefer task-based parallelism over thread-based parallelism.\n- Use std::atomic for atomic operations.\n- Avoid data races by proper synchronization.\n- Use thread-safe data structures when necessary.\n</yakaai_skills>"}