lxd Documentation

Technical Documentation: canonical/lxd

ℹ️ Provenance: Multi-Tier Ingestion · CodeWiki Reference

Overview (`README.md`) Copy Section

LXD

LXD is a modern, secure and powerful system container and virtual machine manager.

It provides a unified experience for running and managing full Linux systems inside containers or virtual machines. LXD supports images for a large number of Linux distributions (official Ubuntu images and images provided by the community) and is built around a very powerful, yet pretty simple, REST API. LXD scales from one instance on a single machine to a cluster in a full data center rack, making it suitable for running workloads both for development and in production.

LXD allows you to easily set up a system that feels like a small private cloud. You can run any type of workload in an efficient way while keeping your resources optimized.

You should consider using LXD if you want to containerize different environments or run virtual machines, or in general run and manage your infrastructure in a cost-effective way.

Get started Copy Chapter

See Getting started in the LXD documentation for installation instructions and first steps.

• Release announcements: https://discourse.ubuntu.com/tags/c/project/lxd/news/143/release
• Release tarballs: https://github.com/canonical/lxd/releases/
• Documentation: https://canonical.com/lxd/docs/latest/

Status Copy Chapter

Installing LXD from packages Copy Chapter

The LXD daemon only works on Linux but the client tool (lxc) is available on most platforms.

The LXD snap packaging files are available in the snap directory.

For more instructions on installing LXD for a wide variety of Linux distributions and operating systems, and to install LXD from source, see How to install LXD in the documentation.

Client SDK packages Copy Chapter

The LXD project provides SDK client packages for interacting with LXD servers from your own software.

These SDKs are licensed as Apache-2.0.

For more information on using the LXD API, see REST API in the documentation.

Tools for managing LXD Copy Chapter

If you are looking for tools (beyond lxc CLI) to manage LXD at scale (from single server to wide clusters), the following projects can be useful:

Security Copy Chapter

Consider the following aspects to ensure that your LXD installation is secure:

• Keep your operating system up-to-date and install all available security patches.

• Use only supported LXD versions (LTS releases or the latest feature release).

• Restrict access to the LXD daemon and the remote API.

• Configure your network interfaces to be secure.

• Do not use privileged containers unless required. If you use privileged containers, put appropriate security measures in place.

  See Container security for more information.

See Security for detailed information.

IMPORTANT:

Local access to LXD through the Unix socket always grants full access to LXD.
This includes the ability to attach file system paths or devices to any instance as well as tweak the security features on any instance.

Therefore, you should only give such access to users who you'd trust with root access to your system.

Community support Copy Chapter

You can seek support from the LXD developers as well as the wider community through the following channels.

Forum Copy Section

Ask questions or engage in discussions: https://discourse.ubuntu.com/c/project/lxd/

Documentation Copy Section

Access the official documentation: https://canonical.com/lxd/docs/latest/

Bug reports and feature requests Copy Section

To file a new bug or feature request, submit an issue on GitHub.

Other community resources Copy Section

You can find additional resources on the LXD website, on YouTube, and the community-created tutorials.

Commercial support Copy Chapter

LTS releases of LXD receive standard support for five years, which means they receive continuous updates. Commercial support for LXD is provided as part of Ubuntu Pro (both Infra-only and full Ubuntu Pro), including for attached LXD instances running Ubuntu. See the full service description for details.

Managed solutions and firefighting support are also available for LXD deployments. See: Managed services.

Contributing Copy Chapter

Fixes and new features are greatly appreciated. Make sure to read our contributing guidelines first!

Overview (`doc/README.md`) Copy Section

LXD documentation

The LXD documentation is available at: https://canonical.com/lxd/docs/latest/

GitHub provides a basic rendering of the documentation as well, but important features like includes and clickable links are missing. Therefore, we recommend reading the published documentation.

How it works Copy Chapter

Documentation framework Copy Section

LXD's documentation is built with Sphinx and hosted on Read the Docs.

It is written in Markdown with MyST extensions.
For syntax help and guidelines, see the MyST syntax guide in the Sphinx Stack documentation.

The documentation structure follows the Diátaxis framework.

Build the documentation Copy Section

To build the documentation, run make doc from the root directory of the repository.
This command installs the required tools and renders the output to the doc/_build/ directory.
To update the documentation for changed files only (without re-installing the tools), run make doc-incremental.

Before opening a pull request, make sure that the documentation builds without any warnings (warnings are treated as errors).
To preview the documentation locally, run make doc-serve and go to http://localhost:8000 to view the rendered documentation.

Overview (`lxd-qemu-snap/README.md`) Copy Section

External QEMU snap for LXD snap

The idea behind this feature is to allow LXD users to build their own snap
containing QEMU and then plug it into the LXD snap, so that LXD uses this
custom QEMU instead of the built-in one.

To make it easier to test this feature and simplify the user experience,
we provide a reference snapcraft.yaml file. It is intended solely as
a starting point for building and customizing your own QEMU snap.

Please note that we do not provide regular or security updates for this
reference example, as it is meant for documentation and educational purposes only.

How to build and use (example) Copy Chapter

cd lxd-qemu-snap

# clean up previous builds
rm -f qemu-for-lxd_*.snap

# build
snapcraft pack

# install
sudo snap install qemu-for-lxd_*.snap --devmode

# connect snaps
sudo snap connect lxd:gpu-2604 mesa-2604:gpu-2604
sudo snap connect lxd:qemu-external qemu-for-lxd:qemu-external

How to use with virgl (only specific to this example of snapcraft.yaml) Copy Chapter

As part of this external QEMU snap example, we build QEMU with virglrenderer
library support. Of course, this is not required, and in your specific use
case you may choose to omit it. The following instructions show how to verify
that the external QEMU snap with virgl support is working correctly.

lxc init images:ubuntu/noble/desktop desktop -c limits.memory=8GiB --vm

# modify instance configuration:
lxc config edit desktop

# choose an appropriate renderer:
ls -la /dev/dri/by-path/*-render

# for example, on my system it is /dev/dri/by-path/pci-0000:67:00.0-render
# lspci | grep -E "(3D|VGA)" shows:
# 67:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Rembrandt (rev d8)

# add the following lines:
  raw.apparmor: |-
    /snap/lxd/*/gpu-2604/** mr,
    /dev/dri/ r,
    /dev/dri/card[0-9]* rw,
    /dev/dri/renderD[0-9]* rw,
    /run/udev/data/c226:[0-9]* r,  # 226 drm
    /sys/devices/** r,
    /sys/bus/** r,
  raw.qemu: -display egl-headless,rendernode=/dev/dri/by-path/pci-0000:67:00.0-render
  raw.qemu.conf: |-
    [device "qemu_gpu"]
    driver = "virtio-vga-gl"

# try it
lxc start desktop --console=vga

# you can check output from:
dmesg | grep -i drm
# if you see:
# [drm] features: +virgl ...
# it's a good sign

glxinfo | grep -i vir
# it should show something like:
# Device: virgl ...

References Copy Chapter

https://github.com/snapcore/snapd/blob/5c8d8431baa425464b279ff26b8c44eecb9aab22/interfaces/builtin/opengl.go#L41

https://gitlab.gnome.org/GNOME/gnome-boxes/-/issues/586

Overview (`snap/README.md`) Copy Section

LXD

LXD is a modern, secure and powerful system container and virtual machine manager.

This directory contains the snap packaging used to build the LXD snap. Additional snap resources are located in the local directory.

Build the LXD snap locally Copy Chapter

Local builds require the LXD snap to be installed as snapcraft creates a container to use as build environment. Here's how to do a local build for the native architecture:

snapcraft pack

Build the LXD snap on Launchpad Copy Chapter

To build the snap for multiple architectures, Launchpad builders can be used.

They are available for various architectures (amd64, armhf, arm64, ppc64el, riscv64 and s390x) and you can ask for multiple to be built in parallel. Here's how to build for both amd64 and arm64:

snapcraft remote-build --launchpad-accept-public-upload --build-for amd64,arm64

GitHub Actions Copy Chapter

`lp-snap-build` action Copy Section

The .github/actions/lp-snap-build composite action sets up SSH access to Launchpad and clones the snap packaging repository in preparation for pushing a build trigger commit.

It also installs the lxd-snapcraft helper tool, which can read and update version and source-commit fields in snapcraft.yaml.

Inputs

Required environment variables

Usage from another repo Copy Section

/* Detailed source-code truncated for AI context efficiency. */

Overview (`test/README.md`) Copy Section

LXD tests

For a description of where and how LXD is tested, please refer to the overview document.

How to run Copy Chapter

To run all tests, including the Go tests, run from repository root:

sudo -E make check

To run only the integration tests, run from the test directory:

sudo -E ./main.sh

Snap integration tests Copy Chapter

Snap integration test scripts are in test/snap/ and can be run directly
from the repository root:

sudo -E ./test/snap/cgroup latest/edge

The snap channel is optional and defaults to latest/edge. The runner installs
the selected store snap by default. To remove the installed LXD snap before a
test run, which is useful for repeated local testing, set PURGE_LXD:

sudo -E env PURGE_LXD=1 ./test/snap/interception latest/edge

To reuse an installed LXD snap without installing or refreshing it, set
KEEP_LXD:

sudo -E env KEEP_LXD=1 ./test/snap/interception latest/edge

Test a source build Copy Section

To test binaries built from the current LXD source tree, first run make, then
enable sideloading:

make
sudo -E env LXD_SNAP_SIDELOAD=1 ./test/snap/interception latest/edge

Set LXD_SNAP_BINARY_DIR to use binaries from a directory other than
$(go env GOPATH)/bin:

sudo -E env LXD_SNAP_SIDELOAD=1 LXD_SNAP_BINARY_DIR=/path/to/bin ./test/snap/interception latest/edge

To sideload individual binaries while testing an installed store snap, set the
corresponding path:

sudo -E env LXD_SIDELOAD_PATH=/path/to/lxd ./test/snap/interception latest/edge
sudo -E env LXC_SIDELOAD_PATH=/path/to/lxc ./test/snap/interception latest/edge
sudo -E env LXD_AGENT_SIDELOAD_PATH=/path/to/lxd-agent ./test/snap/vm latest/edge

Test a local snap Copy Section

To test a locally built snap, set LXD_SNAP_PATH:

sudo -E env LXD_SNAP_PATH=/path/to/lxd_0+git_amd64.snap ./test/snap/interception latest/edge

To use the system's ZFS tools instead of the tools bundled in the LXD snap, set
LXD_ZFS_EXTERNAL:

sudo -E env LXD_ZFS_EXTERNAL=1 ./test/snap/interception latest/edge

Test OVN Copy Section

The network-ovn suite can use the host's OVN packages or a chosen MicroOVN
snap channel:

sudo -E env OVN_SOURCE=deb PURGE_LXD=1 ./test/snap/network-ovn latest/edge
sudo -E env OVN_SOURCE=22.03/edge PURGE_LXD=1 ./test/snap/network-ovn latest/edge
sudo -E env OVN_SOURCE=24.03/edge PURGE_LXD=1 ./test/snap/network-ovn latest/edge
sudo -E env PURGE_LXD=1 ./test/snap/network-ovn latest/edge

The test host must have snapd installed and the command must run as root.

Environment variables Copy Chapter

Recommendations Copy Chapter

`sub_test` usage Copy Section

Use sub_test to label meaningful phases within a test and make logs easier to scan.
Prefer a small number of focused sub-tests over excessive nesting.
Use sub_test before a logical group of commands that verifies a specific expected behavior for a bug fix or feature.
Comments within the sub-test block are appropriate to explain why specific commands are used, any setup or initial configuration, and other intent that isn't obvious from the commands.

Good:

sub_test "Verify intended behavior X"
...
sub_test "Verify intended behavior Y"
...

`echo` context Copy Section

Prefer sub_test labels and concise comments for context instead of adding echo statements.
Use echo only when you need to debug flaky behavior.

Expected failure Copy Section

If a command is expected to fail, special care needs to be used in testing.

Bad:

set -e
...

! cmd_should_fail

some_other_command

Good:

set -e

! cmd_should_fail || false

some_other_command

Best:

set -e

if cmd_should_fail; then
  echo "ERROR: cmd_should_fail unexpectedly succeeded, aborting" >&2
  exit 1
fi

some_other_command

In the "bad" example, if the command unexpectedly succeeds, the script won't
abort because bash ignores set -e for compounded commands (! cmd_should_fail).

The "good" example works around the problem of compound commands by falling
back to executing false in case of unexpected success of the command.

The "best" example also works around the problem of compound commands but in a
very intuitive and readable form, albeit longer.

This odd behavior of `set -e` with compound commands does not apply inside `[]`.

```sh
set -e
# Does the right thing of failing if the file unexpectedly exist
[ ! -e "should/not/exist" ]
```

However, note that in the above example, if the `!` is moved outside of the `[]`, it would also warrant a ` || false` fallback.

For error message assertions, prefer single-quoted strings so error text with " does not require escaping and the comparisons stay readable.

`jq` usage Copy Section

Always use jq --exit-status (or the shorthand -e) when checking for the presence of fields or specific values. This makes test failures explicit:

Bad:

jq '.field' output.json > /dev/null  # Silent fail if field missing

Good:

jq -e '.field' output.json          # Explicit exit code if field missing
jq -e '.field == "expected"' output.json

Avoid `grep -c` for presence or absence checks Copy Section

Avoid using grep -c to test for presence or absence — it makes tests brittle and harder to read.
grep -c for actual counting (where the count matters) is fine.

Bad:

# Using -c when you only care about presence, not count
[ "$(lxc list | grep -c c1)" = "1" ]

Good:

# Just check presence
lxc list | grep -wF c1

Best:

# Filter output directly and compare exactly
[ "$(lxc list -f csv -c n)" = "c1" ]

OK (counting genuinely matters):

[ "$(grep -c "pattern" logfile)" = "3" ]

Overview (`lxd/lxd-metadata/README.md`) Copy Section

lxd-metadata

A small CLI to parse comments in a Golang codebase meant to be used for a documentation tool (like Sphinx for example).

It parses the comments from the AST and extracts their documentation.

Usage

$ lxd-metadata -h
Usage of lxd-metadata:
-e value
    Path that will be excluded from the process

Formatting

A comment is formatted this way:

/* Detailed source-code truncated for AI context efficiency. */

The go-swagger spec from source generator can only handles swagger:meta (global file/package level documentation), swagger:route (API endpoints), swagger:params (function parameters), swagger:operation (method documentation), swagger:response (API response content documentation), swagger:model (struct documentation) generation. In our use case, we would want a config variable spec generator that can bundle any key-value data pairs alongside metadata to build a sense of hierarchy and identity (we want to associate a unique key to each lxddoc comment group that will also be displayed in the generated documentation)

In a swagger fashion, lxd-metadata can associate metadata key-value pairs (here for example, group and key) to data key-value pairs. As a result, it can generate a YAML tree out of the code documentation and also a Markdown document.

Output Copy Chapter

Here is the JSON output of the example shown above:

/* Detailed source-code truncated for AI context efficiency. */

Here is the .txt output of the example shown above:

\`\`\`{config:option} user.* cluster
:type: string
:liveupdate: `yes`
:shortdesc: Free form user key/value storage (can be used in search).
:condition: container
:default: -

This is the real long desc.

With two paragraphs.

And a list:

- Item
- Item
- Item

example of a table:

Key                                 | Type      | Scope     | Default                                          | Description
:--                                 | :---      | :----     | :------                                          | :----------
`acme.agree_tos`                    | bool      | global    | `false`                                          | Agree to ACME terms of service
`acme.ca_url`                       | string    | global    | `https://acme-v02.api.letsencrypt.org/directory` | URL to the directory resource of the ACME service
`acme.domain`                       | string    | global    | -                                                | Domain for which the certificate is issued
`acme.email`                        | string    | global    | -                                                | Email address used for the account registration


\`\`\`

\`\`\`{config:option} scheduler.instance cluster
:liveupdate: `yes`
:shortdesc: Possible values are all, manual and group. See Automatic placement of instances for more information.
:condition: container
:default: `all`
:type: integer


\`\`\`

Overview (`shared/usbid/README.md`) Copy Section

This Go package is a variant of usbid from https://github.com/google/gousb.
It was written by the gousb maintainers and contributors and then adapted for use by LXD.

Main changes:

• Doesn't load on import (reduced memory footprint).
• Uses system-local USB database (to always match lsusb).
• Doesn't import anything outside of built-in Go packages.
• Doesn't use or indirectly rely on CGO as this code is often used in cross-built static binaries.

Most users will want to stick to the upstream google/gousb version instead.
This fork is really meant for LXD's special use case and may be further trimmed down in the future.

Overview (`test/mini-oidc/README.md`) Copy Section

mini-oidc is an extremely basic OIDC provider which can be used with the lxc command line.
It doesn't use web authentication and instead just automatically approves any authentication request.

By default, it will authenticate everyone as unknown, but this can be overridden by writing the username to be returned in the user.data file.
This effectively allows scripting a variety of users without having to deal with actual login.

The storage sub-package is a copy of https://github.com/zitadel/oidc/tree/main/example/server/storage with the exception of the added LXDDeviceClient.

Overview (`.github/workflows/testflinger/README.md`) Copy Section

Testflinger scripts

This directory contains the scripts used for GPU testing (NVIDIA and AMD) via GitHub Actions and Testflinger.
The tests run on devices within Canonical's test farm.

The NVIDIA and NVIDIA MIG templates clone the LXD revision that triggered the
workflow and run the relevant script under test/snap/ directly. The Ubuntu
Core and AMD CDI templates test the selected store snap directly.

Run locally Copy Chapter

Running the tests locally is only possible if your machine has access to the Testflinger server.

Tested distros:

• core24-latest
• jammy
• noble

Current queue/workflow combinations:

• NVIDIA (CDI and legacy runtime): JOB_QUEUE=lxd-nvidia, TFWORKFLOW=nvidia-gpu-job
• NVIDIA MIG (Multi-Instance GPU): JOB_QUEUE=lxd-mig, TFWORKFLOW=nvidia-mig-job
• Ubuntu Core + NVIDIA CDI: JOB_QUEUE=lxd-nvidia, TFWORKFLOW=uc-nvidia-cdi-job
• AMD CDI: JOB_QUEUE=lxd-amd, TFWORKFLOW=amd-cdi-job

Ensure testflinger is installed:

sudo snap install testflinger-cli

Set the input variables and execute the script from the same directory:

JOB_QUEUE=lxd-nvidia SNAP_CHANNEL=latest/edge DISTRO=core24-latest TFWORKFLOW=uc-nvidia-cdi-job ./run.sh

The above replaces the inputs in the scripts and submits the Testflinger job.
To prepare the scripts only but not submit the job, set the --dryrun flag.

Examples Copy Chapter

Notice, that some Testflinger workflows are only compatible with core24-latest, while others only
with classic Ubuntu images.

To test Ubuntu Core + LXD + GPU passthrough in CDI mode:

JOB_QUEUE=lxd-nvidia SNAP_CHANNEL=latest/edge DISTRO=core24-latest TFWORKFLOW=uc-nvidia-cdi-job ./run.sh

To test Ubuntu Noble + LXD + GPU passthrough (CDI and legacy nvidia runtime):

JOB_QUEUE=lxd-nvidia SNAP_CHANNEL=latest/edge DISTRO=noble TFWORKFLOW=nvidia-gpu-job ./run.sh

To test Ubuntu Noble + LXD + AMD GPU passthrough in CDI mode:

JOB_QUEUE=lxd-amd SNAP_CHANNEL=latest/edge DISTRO=noble TFWORKFLOW=amd-cdi-job ./run.sh

To test Ubuntu Noble + LXD + NVIDIA MIG (Multi-Instance GPU) passthrough:

JOB_QUEUE=lxd-mig SNAP_CHANNEL=latest/edge DISTRO=noble TFWORKFLOW=nvidia-mig-job ./run.sh

Overview (`lxd/db/dbgen/README.md`) Copy Section

`dbgen`

dbgen inspects Go struct definitions for expected comments and tags.
If it finds any, it generates functions for the type such that it implements the interfaces defined in generic.go.
The type can then be used with the generic functions therein.

Examples Copy Chapter

Simple table Copy Section

For a simple table, just add a db:model comment above the type and add a tag for each field representing the column name.

CREATE TABLE auth_groups (
    id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
    name TEXT NOT NULL,
    description TEXT NOT NULL,
    UNIQUE (name)
);

package main

// AuthGroupsRow represents a single row of the auth_groups table.
// db:model auth_groups
type AuthGroupsRow struct {
	ID          int64  `db:"id"`
	Name        string `db:"name"`
	Description string `db:"description"`
}

In this example dbgen assumes that the id column is the primary key.

Composite primary keys Copy Section

Some tables don't have an auto-incrementing primary key. These are defined WITHOUT ROWID.
For these tables we can define the primary key as some unique column combination.
To specify the primary keys, add a // db:primary comment above those fields.

CREATE TABLE networks_load_balancer_pools_instances (
	network_load_balancer_pool_id INTEGER NOT NULL,
	instance_id INTEGER NOT NULL,
	target_port INTEGER NOT NULL,
	PRIMARY KEY (network_load_balancer_pool_id, instance_id),
	FOREIGN KEY (network_load_balancer_pool_id) REFERENCES networks_load_balancer_pools (id) ON DELETE CASCADE,
	FOREIGN KEY (instance_id) REFERENCES instances (id) ON DELETE CASCADE
) WITHOUT ROWID;

package main

// NetworksLoadBalancerPoolsInstancesRow represents a row of the networks_load_balancer_pools_instances table.
// db:model networks_load_balancer_pools_instances
type NetworksLoadBalancerPoolsInstancesRow struct {
	// db:primary
	NetworkLoadBalancerPoolID int64 `db:"network_load_balancer_pool_id"`
	// db:primary
	InstanceID int64 `db:"instance_id"`
	TargetPort int64 `db:"target_port"`
}

Joins Copy Section

If a Go type is modelled via dbgen to represent a single row of a single table, then the interface implementations that
dbgen generates can be used to perform CRUD operations on that table. Structs like this can be referenced from other structs
to perform more complex joins, but these secondary structs are read-only.

To reference another struct, add it to the wrapper struct as the first field, with the field name Row.
To perform a join, add a // db:join comment above the field that requires a join.

CREATE TABLE "projects" (
    id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
    name TEXT NOT NULL,
    description TEXT NOT NULL,
    UNIQUE (name)
);
CREATE TABLE placement_groups (
    id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
    name TEXT NOT NULL,
    description TEXT NOT NULL,
    project_id INTEGER NOT NULL,
    FOREIGN KEY (project_id) REFERENCES projects (id) ON DELETE CASCADE,
    UNIQUE (project_id, name)
);

package main

// PlacementGroupsRow represents a single row of the placement_groups table.
// db:model placement_groups
type PlacementGroupsRow struct {
	ID          int64  `db:"id"`
	Name        string `db:"name"`
	Description string `db:"description"`
	ProjectID   int64  `db:"project_id"`
}

// PlacementGroup contains [PlacementGroupsRow] with additional joins.
// db:model placement_groups
type PlacementGroup struct {
	Row PlacementGroupsRow

	// db:join JOIN projects ON placement_groups.project_id = projects.id
	ProjectName string `db:"projects.name"`
}

Overview (`lxd/db/generate/README.md`) Copy Section

`lxd-generate`

Introduction Copy Chapter

lxd-generate is a database statement and associated go function generator
for LXD and related projects. lxd-generate utilizes go's code generation
directives (//go:generate ...) alongside go's ast
package for parsing the syntax tree for go structs and variables. We use
lxd-generate for the majority of our SQL statements and database interactions
on the go side for consistency and predictability.

Usage Copy Chapter

Initialization Copy Section

Generally the first thing we will want to do for any newly generated file is to
establish the command, the target file, and ensure the file has been cleared of
content:

//go:generate -command mapper lxd-generate db mapper -t instances.mapper.go
//go:generate mapper reset -i -b "//go:build linux && cgo && !agent"

This will initiate a call to lxd-generate db mapper -t instances.mapper.go on
each call to the go generation directive mapper.

Generation Directive Arguments Copy Section

• //go:generate mapper stmt -e instance objects table=table_name

The table key can be used to override the generated table name for a specified one.

• //go:generate mapper method -e instance Create references=Config,Device

For some tables (defined below under Additional Information as EntityTable, the references=<ReferenceEntity> key can be provided with the name of
a ReferenceTable or MapTable struct. This directive would produce CreateInstance in addition to CreateInstanceConfig and CreateInstanceDevices

• //go:generate mapper method -i -e instance_profile Create struct=Instance
• //go:generate mapper method -i -e instance_profile Create struct=Profile

For some tables (defined below under Additional Information as AssociationTable, method declarations must
include a struct=<Entity> to indicate the directionality of the function. An invocation can be called for each direction.
This would produce CreateInstanceProfiles and CreateProfileInstances respectively.

SQL Statement Generation Copy Section

SQL generation supports the following SQL statement types:

Examples

//go:generate mapper stmt -e instance objects
//go:generate mapper stmt -e instance objects-by-Name-and-Project
//go:generate mapper stmt -e instance create
//go:generate mapper stmt -e instance update
//go:generate mapper stmt -e instance delete-by-Name-and-Project

Statement Related Go Tags

There are several tags that can be added to fields of a struct that will be parsed by the ast package.

Go Function Generation Copy Section

Go function generation supports the following types:

//go:generate mapper method -i -e instance GetMany
//go:generate mapper method -i -e instance GetOne
//go:generate mapper method -i -e instance ID
//go:generate mapper method -i -e instance Exist
//go:generate mapper method -i -e instance Create
//go:generate mapper method -i -e instance Update
//go:generate mapper method -i -e instance DeleteOne-by-Project-and-Name
//go:generate mapper method -i -e instance DeleteMany-by-Name

Additional Information Copy Section

All structs should have an ID field, as well as an additional Filter struct prefixed with the original struct name.
This should include any fields that should be considered for filtering in WHERE clauses.
These fields should be pointers to facilitate omission and inclusion without setting default values.

Example:

type Instance struct {
  ID int
  Name string
  Project string
}

type InstanceFilter struct {
  Name *string
  Project *string
}

lxd-generate will handle parsing of structs differently based on the composition of the struct in four different ways.

Non-EntityType structs will only support GetMany, Create, Update, and Delete functions.

EntityTable Copy Section

Most structs will get treated this way, and represent a normal table.

• If a table has an associated table for which a ReferenceTable or MapTable as defined below is applicable, functions specific to this entity can be generated by
  including a comma separated list to references=<OtherEntity> in the code generation directive for GetMany, Create, or Update directives.

• The Create method directive for EntityTable will expect on the ID and Exist method directives to be present.

• All CREATE, UPDATE, and DELETE statements that include a joined table will expect a var <entity>ID = RegisterStmt('SQL String') to exist for the joining table.

ReferenceTable Copy Section

A struct that contains a field named ReferenceID will be parsed this way.
lxd-generate will use this struct to generate more abstract SQL statements and functions of the form _.

Real world invocation of these statements and functions should be done through an EntityTable method call with the tag references=. This EntityTable will replace the `` above.

Example:

//go:generate mapper stmt -e device create
//go:generate mapper method -e device Create

type Device struct {
  ID int
  ReferenceID int
  Name string
  Type string
}

//...
//go:generate mapper method -e instance Create references=Device
// This will produce a function called `CreateInstanceDevices`.

MapTable Copy Section

This is a special type of ReferenceTable with fields named Key and Value.
On the SQL side, this is treated exactly like a ReferenceTable, but on the go side, the return values will be a map.

Example:

//go:generate mapper stmt -e config create
//go:generate mapper method -e config Create

type Config struct {
  ID int
  ReferenceID int
  Key string
  Value string
}

//...
//go:generate mapper method -e instance Create references=Config
// This will produce a function called `CreateInstanceConfig`, which will return a `map[string]string`.

AssociationTable Copy Section

This is a special type of table that contains two fields of the form <Entity>ID, where <Entity> corresponds to two other structs present in the same package.
This will generate code for compound tables of the form <entity1>_<entity2> that are generally used to associate two tables together by their IDs.

method generation declarations for these statements should include a struct=<Entity> to indicate the directionality of the function.
An invocation can be called for each direction.

Example:

//go:generate mapper method -i -e instance_profile Create struct=Instance
//go:generate mapper method -i -e instance_profile Create struct=Profile

type InstanceProfile struct {
  InstanceID int
  ProfileID int
}

Overview (`lxd/instance/drivers/libkrun/README.md`) Copy Section

libkrun (reduced)

Minimal Go bindings for libkrun required by LXD.

Runtime Loading Copy Chapter

The package resolves libkrun at runtime with dlopen(3)/dlsym(3).

• If LIBKRUN_PATH is set, it is used first.
• Otherwise it falls back to libkrun.so then libkrun.so.0.

Doc Explanation Index (`doc/explanation/index.md`) Copy Section

myst: html_meta: Copy Chapter

(explanation)=

Explanation

The explanatory guides in this section discuss the concepts used in LXD and help you understand how things fit together.

(explanation-concepts)=

Important concepts Copy Chapter

LXD's core concepts include its relationship with LXC and the instance types it supports: system containers and virtual machines.

:titlesonly:

/explanation/lxd_lxc
/explanation/instances

(explanation-entities)=

Entities in LXD Copy Chapter

LXD uses several distinct entity types, including images, storage pools, networks, and projects. To learn how to use them, refer to the {ref}howtos.

:titlesonly:

/image-handling
/explanation/storage
/explanation/networks
/database
/explanation/lxc_show_info

(explanation-iam)=

Access management Copy Chapter

LXD supports multiple methods for authenticating remote API clients and provides fine-grained authorization controls. Projects can also be used to scope and restrict access.

:titlesonly:

/authentication
/explanation/authorization
/explanation/projects

(explanation-production)=

Production setup Copy Chapter

For scalable, reliable, and secure LXD deployments, these guides help you understand the key concepts around clustering, performance tuning, and security.

:titlesonly:

/explanation/clusters
/explanation/disaster_recovery
/explanation/performance_tuning
/explanation/security
/explanation/bpf

(explanation-csi)=

The LXD CSI driver Copy Chapter

The LXD CSI driver is an open source implementation of the Container Storage Interface (CSI) that integrates LXD storage backends with Kubernetes.

:titlesonly:

/explanation/csi

Doc Howto Index (`doc/howto/index.md`) Copy Section

myst: html_meta: Copy Chapter

(howtos)=

How-to guides

These how-to guides cover key operations and processes in LXD.

Set up the LXD server and initial access Copy Chapter

LXD can be installed and initialized in multiple ways. Afterward, the server can be configured for network access through the CLI or UI client.

:titlesonly:
:maxdepth: 1

/getting_started
/operation

Work with LXD Copy Chapter

Instances are created from images and can be either {ref}system containers or virtual machines <containers-and-vms>. Projects are useful for grouping related instances and managing user access.

:titlesonly:
:maxdepth: 1

/instances
/images
/projects
/storage
/networks

Get ready for production Copy Chapter

For production deployments, clusters of LXD servers help support higher loads. The production setup guides also cover performance, monitoring, backup, and disaster recovery.

:titlesonly:
:maxdepth: 1

/clustering
/production-setup
/howto/disaster_recovery

Perform server administration Copy Chapter

:titlesonly:
:maxdepth: 1
Manage the snap </howto/snap>
Harden security </howto/security_harden>
/howto/troubleshoot

Authenticate to the APIs Copy Chapter

Bearer tokens can be used to authenticate to the LXD API; refer to {ref}authentication for other methods. The DevLXD API is used for communication between instances and their host.

:titlesonly:
:maxdepth: 1

Authenticate to the LXD API using bearer tokens </howto/auth_bearer>
Authenticate to the DevLXD API </howto/devlxd_authenticate>

Decommission LXD Copy Chapter

:titlesonly:
:maxdepth: 1

Decommission LXD </howto/decommission>

Engage with us Copy Chapter

:titlesonly:
:maxdepth: 1

Get support </support>
Contribute to LXD </contributing>

Doc Reference Index (`doc/reference/index.md`) Copy Section

myst: html_meta: Copy Chapter

(reference)=

Reference

The reference material in this section provides technical descriptions of LXD.

(reference-general)=

General information Copy Chapter

These guides include compatibility information for operating systems running in instances, and man pages for the lxc CLI.

:titlesonly:
:maxdepth: 1

/requirements
/architectures
/guest-os-compatibility
Container environment </container-environment>
/reference/manpages

Releases Copy Chapter

Release notes, deprecation notices, and details about the LXD release cadence and its snap.

:titlesonly:
:maxdepth: 1

/reference/release-notes/index
/reference/deprecation_notices
/reference/releases-snap

Images Copy Chapter

Reference information for remote image servers and the LXD image format.

:titlesonly:
:maxdepth: 1

/reference/remote_image_servers
/reference/image_format

(reference-config)=

Configuration options Copy Chapter

LXD is highly configurable, with options available for major entities as well as permissions for access control.

:titlesonly:
:maxdepth: 1
:includehidden:

Configuration option index </config-options>
/server
/explanation/instance_config
/reference/preseed_yaml_fields
/reference/projects
/reference/storage_drivers
/reference/networks
/reference/image_registries
/reference/placement_groups
/reference/clusters
/reference/replicator_config
/reference/permissions

(reference-production)=

Production setup Copy Chapter

The LXD server can be optimized for production workloads and can monitor server metrics.

:titlesonly:
:maxdepth: 1

Production server settings </reference/server_settings>
/reference/provided_metrics

(reference-api)=

API and integrations Copy Chapter

LXD exposes a REST API for managing all resources. The LXD CSI driver integrates LXD storage backends with Kubernetes.

:titlesonly:
:maxdepth: 1

/restapi_landing
/reference/driver_csi

(reference-internal)=

Internal implementation details Copy Chapter

These guides are primarily of interest to advanced users, contributors, and developers.

:titlesonly:
:maxdepth: 1

/internals

:hidden:

Project repository <https://github.com/canonical/lxd>
Image server <https://images.lxd.canonical.com>

Doc Reference Release Notes Index (`doc/reference/release-notes/index.md`) Copy Section

myst: html_meta: Copy Chapter

(ref-release-notes)=

Release notes

This page lists release notes for LXD, which highlight new features, bug fixes, and other important information for each release.

Release policy and schedule Copy Chapter

For details about the release policy and schedule, along with information about the LXD snap channels, see: {ref}ref-releases-snap.

Upgrade instructions Copy Chapter

For full instructions on updating or upgrading LXD via its snap package, see {ref}howto-snap:

• Feature releases are published on the {ref}current feature track <ref-snap-track-feature>. If you are already following this track and you want to manually update to the most recent feature release, see the {ref}howto-snap-updates-manual section.
• To move from one LTS track to a higher LTS track or the feature track, see the {ref}howto-snap-change section.

(ref-release-notes-series)=

Series Copy Chapter

:titlesonly:
:maxdepth: 1

LXD 6 LTS <6/index>
LXD 5.21 LTS <5.21/index>
LXD 5.0 LTS <5.0/index>
LXD 4.0 LTS <4.0/index>

Doc Reference Release Notes 4.0 Index (`doc/reference/release-notes/4.0/index.md`) Copy Section

myst: html_meta: Copy Chapter

(ref-release-notes-4.0)=

Release notes

This page lists recent release notes for LXD 4.0 LTS.

(ref-release-notes-releases-4.0)=

Releases Copy Chapter

:titlesonly:
:maxdepth: 1

LXD 4.0.14 LTS <release-notes-4.0.14>
LXD 4.0.13 LTS <release-notes-4.0.13>
LXD 4.0.12 LTS <release-notes-4.0.12>
LXD 4.0.11 LTS <release-notes-4.0.11>

For older release notes, see our Discourse forum.

Doc Reference Release Notes 5.0 Index (`doc/reference/release-notes/5.0/index.md`) Copy Section

myst: html_meta: Copy Chapter

(ref-release-notes-5.0)=

Release notes

This page lists recent release notes for LXD 5.0 LTS.

(ref-release-notes-releases-5.0)=

Releases Copy Chapter

:titlesonly:
:maxdepth: 1

LXD 5.0.10 LTS <release-notes-5.0.10>
LXD 5.0.9 LTS <release-notes-5.0.9>
LXD 5.0.8 LTS <release-notes-5.0.8>
LXD 5.0.7 LTS <release-notes-5.0.7>

For older release notes, see our Discourse forum.

Doc Reference Release Notes 5.21 Index (`doc/reference/release-notes/5.21/index.md`) Copy Section

myst: html_meta: Copy Chapter

(ref-release-notes-5.21)=

Release notes

This page lists recent release notes for LXD 5.21 LTS.

(ref-release-notes-releases-5.21)=

Releases Copy Chapter

:titlesonly:
:maxdepth: 1

LXD 5.21.8 LTS <release-notes-5.21.8>
LXD 5.21.7 LTS <release-notes-5.21.7>
LXD 5.21.6 LTS <release-notes-5.21.6>
LXD 5.21.5 LTS <release-notes-5.21.5>

For older release notes, see our Discourse forum.

Doc Reference Release Notes 6 Index (`doc/reference/release-notes/6/index.md`) Copy Section

myst: html_meta: Copy Chapter

(ref-release-notes-6)=

Release notes

This page lists recent release notes for LXD 6.
Releases 6.10.z are LTS releases in the 6 series.
Earlier releases in the 6 series are feature releases.
See {ref}ref-releases-snap for more information.

(ref-release-notes-releases-6)=

Releases Copy Chapter

:titlesonly:
:maxdepth: 1

LXD 6.10.0 LTS <release-notes-6.10.0>
LXD 6.9 <release-notes-6.9>
LXD 6.8 <release-notes-6.8>
LXD 6.7 <release-notes-6.7>
LXD 6.6 <release-notes-6.6>

For older release notes, see our Discourse forum.

Doc .Custom Wordlist (`doc/.custom_wordlist.txt`) Copy Section

AAAA
ABI
ACL
ACLs
ACL's
AGPL
AIO
allocator
AMD
AppArmor
ARMv
ARP
ASN
attacher
Auth
authentik
AXFR
backend
backends
backport
backported
backporting
backticks
balancers
BARs
benchmarking
BGP
BitLocker
bitmask
BLK
bool
bootable
bpffs
BPFFS
BPF
BMC
Btrfs
bugfix
bugfixes
CA
CAs
CCLA
CDI
CentOS
Ceph
CephFS
Ceph's
CFS
cgroup
cgroupfs
cgroups
checksum
checksums
Chocolatey
CIDR
CLIs
COPR
CPUs
CRIU
CRD
CRDs
CRL
cron
cryptographic
cryptographically
CSI
CSM
CSV
CUDA
DaemonSet
dataset
dGPU
DCO
deduplicated
dereferenced
DevLXD
DHCP
DHCPv
Diffie
discoverability
DNS
DNSSEC
DoS
Dqlite
DRM
EB
Ebit
eBPF
ECDHE
ECDSA
EiB
Eibit
endian
Entra
Enablement
EOL
ES
ESA
ESM
ETag
failback
failover
filesystem
firmware
frontend
FQDNs
gapped
GARP
GbE
Gbit
Geneve
GiB
Gibit
GID
GIDs
Golang
goroutines
GPUs
GPU's
HAProxy
Hellman
Homebrew
hotplug
hotplugged
hotplugging
HTTPS
HWE
ICMP
idmap
idmapped
idmaps
IdP
iGPU
iGPUs
incrementing
intra
InfiniBand
init
inode
IOPS
IOV
IPAM
IPs
IPv
IPVLAN
IPVS
JIT
JWT
journaling
jq
kB
kbit
Keepalived
Keycloak
keymaps
KiB
kibi
Kibit
Kubelet
Kubelets
KVM
libbpf
Libera
liveness
LogCLI
lookups
LoongArch
loopback
LRU
LV
LVM
LXC
LXCFS
LXC's
LXD
LXD's
macOS
macvlan
manpages
Mbit
MDM
MiB
Mibit
MicroCeph
MicroCloud
MicroOVN
MII
MITM
MMIO
MNIST
MTU
Mullvad
multicast
multipath
namespaced
NATed
natively
NDP
netmask
NBD
nbdcopy
nbdinfo
NFS
NIC
NICs
NIC's
NUMA
numpad
NVMe
NVML
NVRAM
NVIDIA
OAuth
OCI
OData
OIDC
onboarding
OpenFGA
OpenID
OpenMetrics
OpenSSL
OpenSUSE
OpenVSwitch
OptiPNG
Ory
OSD
overcommitting
OverlayFS
OVMF
OVN
OVS
pprof
Pbit
PCI
PCIe
PDU
passthrough
passthroughs
peerings
performant
PersistentVolume
PersistentVolumes
PersistentVolumeClaim
PersistentVolumeClaims
PFs
PiB
Pibit
PID
PKI
PNG
Pongo
POSIX
PowerFlex
preconfigured
prepended
preseed
provision
provisioner
provisioning
proxied
proxying
PTS
PV
PVs
PVC
PVCs
qdisc
qdiscs
QEMU
QFQ
QMP
qgroup
qgroups
RADOS
radosgw
RBAC
RBD
RDP
README
reconfiguring
requestor
resizer
RESTful
RGW
RHEL
rollout
rootfs
RCG
RPC
RPCs
RSA
ROCm
runtime
SATA
scalable
scriptlet
SDC
SDK
SDN
SDNs
SDS
SDT
SDTs
SeaBIOS
Seccomp
SELinux
SEV
SFTP
SHA
SIGHUP
SIGTERM
simplestreams
SKBPRIO
SLAAC
SLES
SMTP
Snapcraft
snapshotter
SoC
Solaris
SPAs
SPL
SquashFS
SQLite
SSDs
SSL
SSO
Starlark
stateful
StatefulSet
stderr
stdin
stdout
StorageClass
StorageClasses
STP
struct
structs
subcommand
subcommands
subnet
subnets
subpage
subpages
submenu
subtrees
subvolume
subvolumes
superset
SVG
swtpm
symlink
symlinks
syscall
syscalls
sysfs
syslog
SystemAdmin
Tbit
TCP
TensorRT
Tegra
TiB
Tibit
TinyPNG
TLS
tmpfs
toleration
tolerations
toolchain
tooltips
topologies
TPM
TSIG
TTL
UDP
UEFI
uefivars
UFW
UI's
UID
UIDs
unconfigured
unevictable
unicast
unixgram
unmanaged
unmount
unmounting
unsets
untrusted
uplink
uplinks
uptime
URI
URIs
userspace
UUIDs
USG
vCPU
vCPUs
vGPU
vGPUs
VDPA
VFS
VFs
VirtIO
virtiofs
virtiofsd
virtualize
virtualized
VLAN
VLANs
VMs
VM's
VolumeAttachment
VolumeAttachments
VolumeSnapshot
VolumeSnapshotClass
VolumeSnapshotContent
VPD
VPN
VRAM
Vsock
vSwitch
vTree
VXLAN
WebSocket
WebSockets
XFS
XHR
YAML's
Zettabyte
ZFS
zpool
zpools
HPE
Alletra
WSAPI
CPG
vLUN
vLUNs
VRRP
Everpure
FlashArray
FlashArrays

Doc .Readthedocs.Yaml (`doc/.readthedocs.yaml`) Copy Section

.readthedocs.yaml

Read the Docs configuration file

See https://docs.readthedocs.io/en/stable/config-file/v2.html for details

Required

version: 2

Set the version of Python and other tools you might need

build:
os: ubuntu-24.04
tools:
golang: "1.25"
python: "3.12"
jobs:
post_checkout:
- git fetch --unshallow || true
# Cancel building pull requests when there are no changes in the doc directory.
# If there are no changes (git diff exits with 0), we force the command to return with 183.
# This is a special exit code on Read the Docs that will cancel the build immediately.
# https://docs.readthedocs.io/en/stable/build-customization.html#cancel-build-based-on-a-condition
- |
if [ "$READTHEDOCS_VERSION_TYPE" = "external" ] && git diff --quiet origin/main -- 'doc/' '.readthedocs.yaml';
then
exit 183;
fi
pre_build:
- go build -ldflags "-s -w" -o trimpath -o lxc.bin ./lxc

Build documentation in the docs/ directory with Sphinx

sphinx:
builder: dirhtml
configuration: doc/conf.py
fail_on_warning: true

If using Sphinx, optionally build your docs in additional formats such as PDF

formats:

• pdf

Optionally declare the Python requirements required to build your docs

python:
install:

• requirements: doc/requirements.txt

Doc Api Extensions (`doc/api-extensions.md`) Copy Section

API extensions

The changes below were introduced to the LXD API after the 1.0 API was finalized.

They are all backward compatible and can be detected by client tools by
looking at the api_extensions field in GET /1.0.

(extension-storage-zfs-remove-snapshots)=

`storage_zfs_remove_snapshots` Copy Chapter

A {config:option}storage-zfs-volume-conf:zfs.remove_snapshots daemon configuration key was introduced.

It's a Boolean that defaults to false and that when set to true instructs LXD
to remove any needed snapshot when attempting to restore another.

This is needed as ZFS will only let you restore the latest snapshot.

(extension-container-host-shutdown-timeout)=

`container_host_shutdown_timeout` Copy Chapter

A {config:option}instance-boot:boot.host_shutdown_timeout container configuration key was introduced.

It's an integer which indicates how long LXD should wait for the container
to stop before killing it.

Its value is only used on clean LXD daemon shutdown. It defaults to 30s.

(extension-container-stop-priority)=

`container_stop_priority` Copy Chapter

A {config:option}instance-boot:boot.stop.priority container configuration key was introduced.

It's an integer which indicates the priority of a container during shutdown.

Containers will shutdown starting with the highest priority level.

Containers with the same priority will shutdown in parallel. It defaults to 0.

(extension-container-syscall-filtering)=

`container_syscall_filtering` Copy Chapter

A number of new syscalls related container configuration keys were introduced.

• {config:option}instance-security:security.syscalls.deny_default
• {config:option}instance-security:security.syscalls.deny_compat
• {config:option}instance-security:security.syscalls.deny
• {config:option}instance-security:security.syscalls.allow

See Instance configuration for how to use them.

Initially, those configuration keys were (accidentally) introduced with
offensive names. They have since been renamed
(`container_syscall_filtering_allow_deny_syntax`), and the old names are no
longer accepted.

(extension-auth-pki)=

`auth_pki` Copy Chapter

This indicates support for PKI authentication mode.

In this mode, the client and server both must use certificates issued by the same PKI.

See {ref}security for details.

(extension-container-last-used-at)=

`container_last_used_at` Copy Chapter

A last_used_at field was added to the GET /1.0/instances/<name> endpoint.

It is a timestamp of the last time the instance was started.

If an instance has been created but not started yet, last_used_at field
will be 1970-01-01T00:00:00Z

(extension-etag)=

`etag` Copy Chapter

Add support for the ETag header on all relevant endpoints.

This adds the following HTTP header on answers to GET:

• ETag (SHA-256 of user modifiable content)

And adds support for the following HTTP header on PUT requests:

• If-Match (ETag value retrieved through previous GET)

This makes it possible to GET a LXD object, modify it and PUT it without
risking to hit a race condition where LXD or another client modified the
object in the meantime.

(extension-patch)=

`patch` Copy Chapter

Add support for the HTTP PATCH method.

PATCH allows for partial update of an object in place of PUT.

(extension-usb-devices)=

`usb_devices` Copy Chapter

Add support for USB hotplug.

(extension-https-allowed-credentials)=

`https_allowed_credentials` Copy Chapter

To use LXD API with all Web Browsers (via SPAs) you must send credentials
(certificate) with each XHR (in order for this to happen, you should set
withCredentials=true
flag to each XHR Request).

Some browsers like Firefox and Safari can't accept server response without
Access-Control-Allow-Credentials: true header. To ensure that the server will
return a response with that header, set {config:option}server-core:core.https_allowed_credentials to true.

(extension-image-compression-algorithm)=

`image_compression_algorithm` Copy Chapter

This adds support for a compression_algorithm property when creating an image (POST /1.0/images).

Setting this property overrides the server default value ({config:option}server-images:images.compression_algorithm).

(extension-directory-manipulation)=

`directory_manipulation` Copy Chapter

This allows for creating and listing directories via the LXD API, and exports
the file type via the X-LXD-type header, which can be either file or
directory right now.

(extension-container-cpu-time)=

`container_cpu_time` Copy Chapter

This adds support for retrieving CPU time for a running container.

(extension-storage-zfs-use-refquota)=

`storage_zfs_use_refquota` Copy Chapter

Introduces a new server property {config:option}storage-zfs-volume-conf:zfs.use_refquota which instructs LXD
to set the refquota property instead of quota when setting a size limit
on a container. LXD will also then use usedbydataset in place of used
when being queried about disk utilization.

This effectively controls whether disk usage by snapshots should be
considered as part of the container's disk space usage.

(extension-storage-lvm-mount-options)=

`storage_lvm_mount_options` Copy Chapter

Adds a new storage.lvm_mount_options daemon configuration option
which defaults to discard and allows the user to set additional mount
options for the file system used by the LVM LV.

(extension-network)=

`network` Copy Chapter

Network management API for LXD.

This includes:

• Addition of the managed property on /1.0/networks entries
• All the network configuration options (see Network configuration for details)
• POST /1.0/networks (see RESTful API for details)
• PUT /1.0/networks/<entry> (see RESTful API for details)
• PATCH /1.0/networks/<entry> (see RESTful API for details)
• DELETE /1.0/networks/<entry> (see RESTful API for details)
• ipv4.address property on nic type devices (when nictype is bridged)
• ipv6.address property on nic type devices (when nictype is bridged)
• security.mac_filtering property on nic type devices (when nictype is bridged)

(extension-profile-usedby)=

`profile_usedby` Copy Chapter

Adds a new used_by field to profile entries listing the instances that are using it.

(extension-container-push)=

`container_push` Copy Chapter

When an instance is created in push mode, the client serves as a proxy between
the source and target server. This is useful in cases where the target server
is behind a NAT or firewall and cannot directly communicate with the source
server and operate in pull mode.

(extension-container-exec-recording)=

`container_exec_recording` Copy Chapter

Introduces a new Boolean record-output, parameter to
/1.0/instances/<name>/exec which when set to true and combined with
with wait-for-websocket set to false, will record stdout and stderr to
disk and make them available through the logs interface.

The URL to the recorded output is included in the operation metadata
once the command is done running.

That output will expire similarly to other log files, typically after 48 hours.

(extension-certificate-update)=

`certificate_update` Copy Chapter

Adds the following to the REST API:

• ETag header on GET of a certificate
• PUT of certificate entries
• PATCH of certificate entries

(extension-container-exec-signal-handling)=

`container_exec_signal_handling` Copy Chapter

Adds support /1.0/instances/<name>/exec for forwarding signals sent to the
client to the processes executing in the instance. Currently SIGTERM and
SIGHUP are forwarded. Further signals that can be forwarded might be added
later.

(extension-gpu-devices)=

`gpu_devices` Copy Chapter

Enables adding GPUs to a container.

(extension-container-image-properties)=

`container_image_properties` Copy Chapter

Introduces a new image configuration key space. Read-only, includes the properties of the parent image.

(extension-migration-progress)=

`migration_progress` Copy Chapter

Transfer progress is now exported as part of the operation, on both sending and receiving ends.
This shows up as a fs_progress attribute in the operation metadata.

(extension-id-map)=

`id_map` Copy Chapter

Enables setting the {config:option}instance-security:security.idmap.isolated,
{config:option}instance-security:security.idmap.size, and {config:option}instance-raw:raw.idmap fields.

(extension-network-firewall-filtering)=

`network_firewall_filtering` Copy Chapter

Add two new keys, {config:option}network-bridge-network-conf:ipv4.firewall and {config:option}network-bridge-network-conf:ipv6.firewall which if set to
false will turn off the generation of iptables FORWARDING rules. NAT
rules will still be added so long as the matching {config:option}network-bridge-network-conf:ipv4.nat or
{config:option}network-bridge-network-conf:ipv6.nat key is set to true.

Rules necessary for dnsmasq to work (DHCP/DNS) will always be applied if
dnsmasq is enabled on the bridge.

(extension-network-routes)=

`network_routes` Copy Chapter

Introduces {config:option}network-bridge-network-conf:ipv4.routes and {config:option}network-bridge-network-conf:ipv6.routes which allow routing additional subnets to a LXD bridge.

(extension-storage)=

`storage` Copy Chapter

Storage management API for LXD.

This includes:

• GET /1.0/storage-pools

• POST /1.0/storage-pools (see RESTful API for details)

• GET /1.0/storage-pools/<name> (see RESTful API for details)

• POST /1.0/storage-pools/<name> (see RESTful API for details)

• PUT /1.0/storage-pools/<name> (see RESTful API for details)

• PATCH /1.0/storage-pools/<name> (see RESTful API for details)

• DELETE /1.0/storage-pools/<name> (see RESTful API for details)

• GET /1.0/storage-pools/<name>/volumes (see RESTful API for details)

• GET /1.0/storage-pools/<name>/volumes/<volume_type> (see RESTful API for details)

• POST /1.0/storage-pools/<name>/volumes/<volume_type> (see RESTful API for details)

• GET /1.0/storage-pools//volumes/<volume_type>/<name> (see RESTful API for details)

• POST /1.0/storage-pools//volumes/<volume_type>/<name> (see RESTful API for details)

• PUT /1.0/storage-pools//volumes/<volume_type>/<name> (see RESTful API for details)

• PATCH /1.0/storage-pools//volumes/<volume_type>/<name> (see RESTful API for details)

• DELETE /1.0/storage-pools//volumes/<volume_type>/<name> (see RESTful API for details)

• All storage configuration options (see Storage configuration for details)

(extension-file-delete)=

`file_delete` Copy Chapter

Implements DELETE in /1.0/instances/<name>/files

(extension-file-append)=

`file_append` Copy Chapter

Implements the X-LXD-write header which can be one of overwrite or append.

(extension-network-dhcp-expiry)=

`network_dhcp_expiry` Copy Chapter

Introduces {config:option}network-bridge-network-conf:ipv4.dhcp.expiry and {config:option}network-bridge-network-conf:ipv6.dhcp.expiry allowing to set the DHCP lease expiry time.

(extension-storage-lvm-vg-rename)=

`storage_lvm_vg_rename` Copy Chapter

Introduces the ability to rename a volume group by setting {config:option}storage-lvm-pool-conf:lvm.vg_name.

(extension-storage-lvm-thinpool-rename)=

`storage_lvm_thinpool_rename` Copy Chapter

Introduces the ability to rename a thin pool name by setting {config:option}storage-lvm-pool-conf:lvm.thinpool_name.

(extension-network-vlan)=

`network_vlan` Copy Chapter

This adds a new {config:option}device-nic-macvlan-device-conf:vlan property to macvlan network devices.

When set, this will instruct LXD to attach to the specified VLAN. LXD
will look for an existing interface for that VLAN on the host. If one
can't be found it will create one itself and then use that as the
macvlan parent.

(extension-image-create-aliases)=

`image_create_aliases` Copy Chapter

Adds a new aliases field to POST /1.0/images allowing for aliases to
be set at image creation/import time.

(extension-container-stateless-copy)=

`container_stateless_copy` Copy Chapter

This introduces a new live attribute in POST /1.0/instances/<name>.
Setting it to false tells LXD not to attempt running state transfer.

(extension-container-only-migration)=

`container_only_migration` Copy Chapter

Introduces a new Boolean container_only attribute. When set to true only the
instance will be copied or moved.

(extension-storage-zfs-clone-copy)=

`storage_zfs_clone_copy` Copy Chapter

Introduces a new Boolean {config:option}storage-zfs-pool-conf:zfs.clone_copy property for ZFS storage
pools. When set to false copying an instance will be done through zfs send and
receive. This will make the target instance independent of its source
instance thus avoiding the need to keep dependent snapshots in the ZFS pool
around. However, this also entails less efficient storage usage for the
affected pool.
The default value for this property is true, i.e. space-efficient snapshots
will be used unless explicitly set to false.

(extension-unix-device-rename)=

`unix_device_rename` Copy Chapter

Introduces the ability to rename the unix-block/unix-char device inside container by setting path,
and the source attribute is added to specify the device on host.
If source is set without a path, we should assume that path will be the same as source.
If path is set without source and major/minor isn't set,
we should assume that source will be the same as path.
So at least one of them must be set.

(extension-storage-lvm-use-thinpool)=

`storage_lvm_use_thinpool` Copy Chapter

Adds the {config:option}storage-lvm-pool-conf:lvm.use_thinpool configuration key, a Boolean that defaults to true.
When set to false, LXD creates plain logical volumes for storage entities instead of using an LVM thin pool.

(extension-storage-rsync-bwlimit)=

`storage_rsync_bwlimit` Copy Chapter

When rsync has to be invoked to transfer storage entities setting rsync.bwlimit
places an upper limit on the amount of socket I/O allowed.

(extension-network-vxlan-interface)=

`network_vxlan_interface` Copy Chapter

This introduces a new {config:option}network-bridge-network-conf:tunnel.NAME.interface option for networks.

This key control what host network interface is used for a VXLAN tunnel.

(extension-storage-btrfs-mount-options)=

`storage_btrfs_mount_options` Copy Chapter

This introduces the {config:option}storage-btrfs-pool-conf:btrfs.mount_options property for Btrfs storage pools.

This key controls what mount options will be used for the Btrfs storage pool.

(extension-entity-description)=

`entity_description` Copy Chapter

This adds descriptions to entities like instances, snapshots, networks, storage pools and volumes.

(extension-image-force-refresh)=

`image_force_refresh` Copy Chapter

This allows forcing a refresh for an existing image.

(extension-storage-lvm-lv-resizing)=

`storage_lvm_lv_resizing` Copy Chapter

This introduces the ability to resize logical volumes by setting the size
property in the instance root disk device.

(extension-id-map-base)=

`id_map_base` Copy Chapter

This introduces a new {config:option}instance-security:security.idmap.base allowing the user to skip the
map auto-selection process for isolated containers and specify what host
UID/GID to use as the base.

(extension-file-symlinks)=

`file_symlinks` Copy Chapter

This adds support for transferring symlinks through the file API.
X-LXD-type can now be symlink with the request content being the target path.

(extension-container-push-target)=

`container_push_target` Copy Chapter

This adds the target field to POST /1.0/instances/<name> which can be
used to have the source LXD host connect to the target during migration.

(extension-network-vlan-physical)=

`network_vlan_physical` Copy Chapter

Allows use of {config:option}network-physical-network-conf:vlan property with physical network devices.

When set, this will instruct LXD to attach to the specified VLAN on the parent interface.
LXD will look for an existing interface for that parent and VLAN on the host.
If one can't be found it will create one itself.
Then, LXD will directly attach this interface to the container.

(extension-storage-images-delete)=

`storage_images_delete` Copy Chapter

This enabled the storage API to delete storage volumes for images from
a specific storage pool.

(extension-container-edit-metadata)=

`container_edit_metadata` Copy Chapter

This adds support for editing an instance metadata.yaml and related templates
via API, by accessing URLs under /1.0/instances/<name>/metadata. It can be used
to edit an instance before publishing an image from it.

(extension-container-snapshot-stateful-migration)=

`container_snapshot_stateful_migration` Copy Chapter

This enables migrating stateful container snapshots to new containers.

(extension-storage-driver-ceph)=

`storage_driver_ceph` Copy Chapter

This adds a Ceph storage driver.

(extension-storage-ceph-user-name)=

`storage_ceph_user_name` Copy Chapter

This adds the ability to specify the Ceph user.

(extension-resource-limits)=

`resource_limits` Copy Chapter

This adds the instance_type field to the container creation request.
Its value is expanded to LXD resource limits.

(extension-storage-volatile-initial-source)=

`storage_volatile_initial_source` Copy Chapter

This records the actual source passed to LXD during storage pool creation.

(extension-storage-ceph-force-osd-reuse)=

`storage_ceph_force_osd_reuse` Copy Chapter

This introduces the ceph.osd.force_reuse property for the Ceph storage
driver. When set to true LXD will reuse an OSD storage pool that is already in
use by another LXD instance.

(extension-storage-block-filesystem-btrfs)=

`storage_block_filesystem_btrfs` Copy Chapter

This adds support for Btrfs as a storage volume file system, in addition to ext4
and xfs.

(extension-resources)=

`resources` Copy Chapter

This adds support for querying a LXD daemon for the system resources it has
available.

(extension-kernel-limits)=

`kernel_limits` Copy Chapter

This adds support for setting process limits such as maximum number of open
files for the container via nofile. The format is limits.kernel.[limit name].

(extension-storage-api-volume-rename)=

`storage_api_volume_rename` Copy Chapter

This adds support for renaming custom storage volumes.

(extension-network-sriov)=

`network_sriov` Copy Chapter

This adds support for SR-IOV enabled network devices.

(extension-console)=

`console` Copy Chapter

This adds support to interact with the container console device and console log.

(extension-restrict-devlxd)=

`restrict_devlxd` Copy Chapter

A new {config:option}instance-security:security.devlxd container configuration key was introduced.
The key controls whether the /dev/lxd interface is made available to the instance.
If set to false, this effectively prevents the container from interacting with the LXD daemon.

(extension-migration-pre-copy)=

`migration_pre_copy` Copy Chapter

This adds support for optimized memory transfer during live migration.

(extension-infiniband)=

`infiniband` Copy Chapter

This adds support to use InfiniBand network devices.

(extension-devlxd-events)=

`devlxd_events` Copy Chapter

This adds a WebSocket API to the devlxd socket.

When connecting to /1.0/events over the devlxd socket, you will now be
getting a stream of events over WebSocket.

(extension-proxy)=

`proxy` Copy Chapter

This adds a new proxy device type to containers, allowing forwarding
of connections between the host and container.

(extension-network-dhcp-gateway)=

`network_dhcp_gateway` Copy Chapter

Introduces a new {config:option}network-bridge-network-conf:ipv4.dhcp.gateway network configuration key to set an alternate gateway.

(extension-file-get-symlink)=

`file_get_symlink` Copy Chapter

This makes it possible to retrieve symlinks using the file API.

(extension-network-leases)=

`network_leases` Copy Chapter

Adds a new /1.0/networks/NAME/leases API endpoint to query the lease database on
bridges which run a LXD-managed DHCP server.

(extension-unix-device-hotplug)=

`unix_device_hotplug` Copy Chapter

This adds support for the {config:option}device-unix-hotplug-device-conf:required property for Unix devices.

(extension-storage-api-local-volume-handling)=

`storage_api_local_volume_handling` Copy Chapter

This add the ability to copy and move custom storage volumes locally in the
same and between storage pools.

(extension-operation-description)=

`operation_description` Copy Chapter

Adds a description field to all operations.

(extension-clustering)=

`clustering` Copy Chapter

Clustering API for LXD.

This includes the following new endpoints (see RESTful API for details):

• GET /1.0/cluster

• UPDATE /1.0/cluster

• GET /1.0/cluster/members

• GET /1.0/cluster/members/<name>

• POST /1.0/cluster/members/<name>

• DELETE /1.0/cluster/members/<name>

The following existing endpoints have been modified:

• POST /1.0/instances accepts a new target query parameter
• POST /1.0/storage-pools accepts a new target query parameter
• GET /1.0/storage-pool/<name> accepts a new target query parameter
• POST /1.0/storage-pool//volumes/<type> accepts a new target query parameter
• GET /1.0/storage-pool//volumes/<type>/<name> accepts a new target query parameter
• POST /1.0/storage-pool//volumes/<type>/<name> accepts a new target query parameter
• PUT /1.0/storage-pool//volumes/<type>/<name> accepts a new target query parameter
• PATCH /1.0/storage-pool//volumes/<type>/<name> accepts a new target query parameter
• DELETE /1.0/storage-pool//volumes/<type>/<name> accepts a new target query parameter
• POST /1.0/networks accepts a new target query parameter
• GET /1.0/networks/<name> accepts a new target query parameter

(extension-event-lifecycle)=

`event_lifecycle` Copy Chapter

This adds a new lifecycle message type to the events API.

(extension-storage-api-remote-volume-handling)=

`storage_api_remote_volume_handling` Copy Chapter

This adds the ability to copy and move custom storage volumes between remote.

(extension-container-mount-propagation)=

`container_mount_propagation` Copy Chapter

This adds a new {config:option}device-disk-device-conf:propagation option to the disk device type, allowing
the configuration of kernel mount propagation.

(extension-container-backup)=

`container_backup` Copy Chapter

Add container backup support.

This includes the following new endpoints (see RESTful API for details):

• GET /1.0/instances/<name>/backups

• POST /1.0/instances/<name>/backups

• GET /1.0/instances/<name>/backups/<name>

• POST /1.0/instances/<name>/backups/<name>

• DELETE /1.0/instances/<name>/backups/<name>

• GET /1.0/instances/<name>/backups/<name>/export

The following existing endpoint has been modified:

• POST /1.0/instances accepts the new source type backup

(extension-devlxd-images)=

`devlxd_images` Copy Chapter

Adds a {config:option}instance-security:security.devlxd.images configuration option for containers which
controls the availability of a /1.0/images/FINGERPRINT/export API over
devlxd. This can be used by a container running nested LXD to access raw
images from the host.

(extension-container-local-cross-pool-handling)=

`container_local_cross_pool_handling` Copy Chapter

This enables copying or moving containers between storage pools on the same LXD
instance.

(extension-proxy-unix)=

`proxy_unix` Copy Chapter

Add support for both Unix sockets and abstract Unix sockets in proxy devices.
They can be used by specifying the address as unix:/path/to/unix.sock (normal
socket) or unix:@/tmp/unix.sock (abstract socket).

Supported connections are now:

• TCP <-> TCP
• UNIX <-> UNIX
• TCP <-> UNIX
• UNIX <-> TCP

(extension-proxy-udp)=

`proxy_udp` Copy Chapter

Add support for UDP in proxy devices.

Supported connections are now:

• TCP <-> TCP
• UNIX <-> UNIX
• TCP <-> UNIX
• UNIX <-> TCP
• UDP <-> UDP
• TCP <-> UDP
• UNIX <-> UDP

(extension-clustering-join)=

`clustering_join` Copy Chapter

This makes GET /1.0/cluster return information about which storage pools and
networks are required to be created by joining nodes and which node-specific
configuration keys they are required to use when creating them. Likewise the PUT /1.0/cluster endpoint now accepts the same format to pass information about
storage pools and networks to be automatically created before attempting to join
a cluster.

(extension-proxy-tcp-udp-multi-port-handling)=

`proxy_tcp_udp_multi_port_handling` Copy Chapter

Adds support for forwarding traffic for multiple ports. Forwarding is allowed
between a range of ports if the port range is equal for source and target
(for example 1.2.3.4 0-1000 -> 5.6.7.8 1000-2000) and between a range of source
ports and a single target port (for example 1.2.3.4 0-1000 -> 5.6.7.8 1000).

(extension-network-state)=

`network_state` Copy Chapter

Adds support for retrieving a network's state.

This adds the following new endpoint (see RESTful API for details):

• GET /1.0/networks/<name>/state

(extension-proxy-unix-dac-properties)=

`proxy_unix_dac_properties` Copy Chapter

This adds support for GID, UID, and mode properties for non-abstract Unix
sockets.

(extension-container-protection-delete)=

`container_protection_delete` Copy Chapter

Enables setting the {config:option}instance-security:security.protection.delete field which prevents containers
from being deleted if set to true. Snapshots are not affected by this setting.

(extension-unix-priv-drop)=

`unix_priv_drop` Copy Chapter

Adds {config:option}device-proxy-device-conf:security.uid and {config:option}device-proxy-device-conf:security.gid for the proxy devices, allowing
privilege dropping and effectively changing the UID/GID used for
connections to Unix sockets too.

(extension-pprof-http)=

`pprof_http` Copy Chapter

This adds a new {config:option}server-core:core.debug_address configuration option to start a debugging HTTP server.

That server currently includes a pprof API and replaces the old
cpu-profile, memory-profile and print-goroutines debug options.

(extension-proxy-haproxy-protocol)=

`proxy_haproxy_protocol` Copy Chapter

Adds a {config:option}device-proxy-device-conf:proxy_protocol key to the proxy device which controls the use of the HAProxy PROXY protocol header.

(extension-network-hwaddr)=

`network_hwaddr` Copy Chapter

Adds a {config:option}network-bridge-network-conf:bridge.hwaddr key to control the MAC address of the bridge.

(extension-proxy-nat)=

`proxy_nat` Copy Chapter

This adds optimized UDP/TCP proxying. If the configuration allows, proxying
will be done via iptables instead of proxy devices.

(extension-network-nat-order)=

`network_nat_order` Copy Chapter

This introduces the {config:option}network-bridge-network-conf:ipv4.nat.order and {config:option}network-bridge-network-conf:ipv6.nat.order configuration keys for LXD bridges.
Those keys control whether to put the LXD rules before or after any pre-existing rules in the chain.

(extension-container-full)=

`container_full` Copy Chapter

This introduces a new recursion=2 mode for GET /1.0/instances which allows for the retrieval of
all instance structs, including the state, snapshots and backup structs.

This effectively allows for lxc list to get all it needs in one query.

(extension-backup-compression)=

`backup_compression` Copy Chapter

This introduces a new {config:option}server-miscellaneous:backups.compression_algorithm configuration key which
allows configuration of backup compression.

(extension-storage-api-volume-snapshots)=

`storage_api_volume_snapshots` Copy Chapter

Add support for storage volume snapshots. They work like container snapshots,
only for volumes.

This adds the following new endpoint (see RESTful API for details):

• GET /1.0/storage-pools//volumes/<type>/<name>/snapshots

• POST /1.0/storage-pools//volumes/<type>/<name>/snapshots

• GET /1.0/storage-pools//volumes/<type>/<volume>/snapshots/<name>

• PUT /1.0/storage-pools//volumes/<type>/<volume>/snapshots/<name>

• POST /1.0/storage-pools//volumes/<type>/<volume>/snapshots/<name>

• DELETE /1.0/storage-pools//volumes/<type>/<volume>/snapshots/<name>

(extension-storage-unmapped)=

`storage_unmapped` Copy Chapter

Introduces a new security.unmapped Boolean on storage volumes.

Setting it to true will flush the current map on the volume and prevent
any further idmap tracking and remapping on the volume.

This can be used to share data between isolated containers after
attaching it to the container which requires write access.

(extension-projects)=

`projects` Copy Chapter

Add a new project API, supporting creation, update and deletion of projects.

Projects can hold containers, profiles or images at this point and let
you get a separate view of your LXD resources by switching to it.

(extension-network-vxlan-ttl)=

`network_vxlan_ttl` Copy Chapter

This adds a new {config:option}network-bridge-network-conf:tunnel.NAME.ttl network configuration option which
makes it possible to raise the TTL on VXLAN tunnels.

(extension-container-incremental-copy)=

`container_incremental_copy` Copy Chapter

This adds support for incremental container copy. When copying a container
using the --refresh flag, only the missing or outdated files will be
copied over. Should the target container not exist yet, a normal copy operation
is performed.

(extension-usb-optional-vendorid)=

`usb_optional_vendorid` Copy Chapter

As the name implies, the {config:option}device-unix-usb-device-conf:vendorid field on USB devices attached to
containers has now been made optional, allowing for all USB devices to
be passed to a container (similar to what's done for GPUs).

(extension-snapshot-scheduling)=

`snapshot_scheduling` Copy Chapter

This adds support for snapshot scheduling. It introduces three new
configuration keys: snapshots.schedule, snapshots.schedule.stopped, and
snapshots.pattern. Snapshots can be created automatically up to every minute.

(extension-snapshot-schedule-aliases)=

`snapshot_schedule_aliases` Copy Chapter

Snapshot schedule can be configured by a comma-separated list of schedule aliases.
Available aliases are <@hourly> <@daily> <@midnight> <@weekly> <@monthly> <@annually> <@yearly> <@startup> for instances,
and <@hourly> <@daily> <@midnight> <@weekly> <@monthly> <@annually> <@yearly> for storage volumes.

(extension-container-copy-project)=

`container_copy_project` Copy Chapter

Introduces a project field to the container source JSON object, allowing for
copy/move of containers between projects.

(extension-clustering-server-address)=

`clustering_server_address` Copy Chapter

This adds support for configuring a server network address which differs from
the REST API client network address. When bootstrapping a new cluster, clients
can set the new {config:option}server-cluster:cluster.https_address configuration key to specify the address of
the initial server. When joining a new server, clients can set the
{config:option}server-core:core.https_address configuration key of the joining server to the REST API
address the joining server should listen at, and set the server_address
key in the PUT /1.0/cluster API to the address the joining server should
use for clustering traffic (the value of server_address will be
automatically copied to the cluster.https_address configuration key of the
joining server).

(extension-clustering-image-replication)=

`clustering_image_replication` Copy Chapter

Enable image replication across the nodes in the cluster.
A new {config:option}server-cluster:cluster.images_minimal_replica configuration key was introduced can be used
to specify to the minimal numbers of nodes for image replication.

(extension-container-protection-shift)=

`container_protection_shift` Copy Chapter

Enables setting the {config:option}instance-security:security.protection.shift option which prevents containers
from having their file system shifted.

(extension-snapshot-expiry)=

`snapshot_expiry` Copy Chapter

This adds support for snapshot expiration. The task is run minutely. The configuration
option {config:option}instance-snapshots:snapshots.expiry takes an expression in the form of 1M 2H 3d 4w 5m 6y (1 minute, 2 hours, 3 days, 4 weeks, 5 months, 6 years), however not all
parts have to be used.

Snapshots which are then created will be given an expiry date based on the
expression. This expiry date, defined by expires_at, can be manually edited
using the API or lxc config edit. Snapshots with a valid expiry date will be
removed when the task in run. Expiry can be disabled by setting expires_at to
an empty string or 0001-01-01T00:00:00Z (zero time). This is the default if
snapshots.expiry is not set.

This adds the following new endpoint (see RESTful API for details):

• PUT /1.0/instances/<name>/snapshots/<name>

(extension-container-backup-override-pool)=

`container_backup_override_pool` Copy Chapter

Adds the PoolName field to InstanceBackupArgs, sent as the X-LXD-pool header on POST /1.0/instances,
allowing the storage pool that an instance is restored into from a backup to be overridden instead of using
the pool recorded in the backup.

(extension-snapshot-expiry-creation)=

`snapshot_expiry_creation` Copy Chapter

Adds expires_at to container creation, allowing for override of a
snapshot's expiry at creation time.

(extension-network-leases-location)=

`network_leases_location` Copy Chapter

Introduces a Location field in the leases list.
This is used when querying a cluster to show what node a particular
lease was found on.

(extension-resources-cpu-socket)=

`resources_cpu_socket` Copy Chapter

Add Socket field to CPU resources in case we get out of order socket information.

(extension-resources-gpu)=

`resources_gpu` Copy Chapter

Add a new GPU struct to the server resources, listing all usable GPUs on the system.

(extension-resources-numa)=

`resources_numa` Copy Chapter

Shows the NUMA node for all CPUs and GPUs.

(extension-kernel-features)=

`kernel_features` Copy Chapter

Exposes the state of optional kernel features through the server environment.

(extension-id-map-current)=

`id_map_current` Copy Chapter

This introduces a new internal {config:option}instance-volatile:volatile.idmap.current key which is
used to track the current mapping for the container.

This effectively gives us:

• {config:option}instance-volatile:volatile.last_state.idmap => On-disk idmap
• {config:option}instance-volatile:volatile.idmap.current => Current kernel map
• {config:option}instance-volatile:volatile.idmap.next => Next on-disk idmap

This is required to implement environments where the on-disk map isn't
changed but the kernel map is (e.g. idmapped mounts).

(extension-event-location)=

`event_location` Copy Chapter

Expose the location of the generation of API events.

(extension-storage-api-remote-volume-snapshots)=

`storage_api_remote_volume_snapshots` Copy Chapter

This allows migrating storage volumes including their snapshots.

(extension-network-nat-address)=

`network_nat_address` Copy Chapter

This introduces the {config:option}network-bridge-network-conf:ipv4.nat.address and {config:option}network-bridge-network-conf:ipv6.nat.address configuration keys for LXD bridges.
Those keys control the source address used for outbound traffic from the bridge.

(extension-container-nic-routes)=

`container_nic_routes` Copy Chapter

This introduces the {config:option}device-nic-bridged-device-conf:ipv4.routes and {config:option}device-nic-bridged-device-conf:ipv6.routes properties on nic type devices.
This allows adding static routes on host to container's NIC.

(extension-cluster-internal-copy)=

`cluster_internal_copy` Copy Chapter

This makes it possible to do a normal POST /1.0/instances to copy an
instance between cluster nodes with LXD internally detecting whether a
migration is required.

(extension-seccomp-notify)=

`seccomp_notify` Copy Chapter

If the kernel supports seccomp-based syscall interception LXD can be notified
by a container that a registered syscall has been performed. LXD can then
decide to trigger various actions.

(extension-lxc-features)=

`lxc_features` Copy Chapter

This introduces the lxc_features section output from the lxc info command
via the GET /1.0 route. It outputs the result of checks for key features being present in the
underlying LXC library.

(extension-container-nic-ipvlan)=

`container_nic_ipvlan` Copy Chapter

This introduces the ipvlan nic device type.

(extension-network-vlan-sriov)=

`network_vlan_sriov` Copy Chapter

This introduces VLAN ({config:option}device-nic-sriov-device-conf:vlan) and MAC filtering ({config:option}device-nic-sriov-device-conf:security.mac_filtering) support for SR-IOV devices.

(extension-storage-cephfs)=

`storage_cephfs` Copy Chapter

Add support for CephFS as a storage pool driver. This can only be used
for custom volumes, images and containers should be on Ceph (RBD)
instead.

(extension-container-nic-ipfilter)=

`container_nic_ipfilter` Copy Chapter

This introduces container IP filtering ({config:option}device-nic-bridged-device-conf:security.ipv4_filtering and {config:option}device-nic-bridged-device-conf:security.ipv6_filtering) support for bridged NIC devices.

(extension-resources-v2)=

`resources_v2` Copy Chapter

Rework the resources API at /1.0/resources, especially:

• CPU
  • Fix reporting to track sockets, cores and threads
  • Track NUMA node per core
  • Track base and turbo frequency per socket
  • Track current frequency per core
  • Add CPU cache information
  • Export the CPU architecture
  • Show online/offline status of threads
• Memory
  • Add huge-pages tracking
  • Track memory consumption per NUMA node too
• GPU
  • Split DRM information to separate struct
  • Export device names and nodes in DRM struct
  • Export device name and node in NVIDIA struct
  • Add SR-IOV VF tracking

(extension-container-exec-user-group-cwd)=

`container_exec_user_group_cwd` Copy Chapter

Adds support for specifying User, Group and Cwd during POST /1.0/instances/NAME/exec.

(extension-container-syscall-intercept)=

`container_syscall_intercept` Copy Chapter

Adds the security.syscalls.intercept.* configuration keys to control
what system calls will be intercepted by LXD and processed with
elevated permissions.

(extension-container-disk-shift)=

`container_disk_shift` Copy Chapter

Adds the {config:option}device-disk-device-conf:shift property on disk devices which controls the use of the idmapped mounts overlay.

(extension-storage-shifted)=

`storage_shifted` Copy Chapter

Introduces a new security.shifted Boolean on storage volumes.

Setting it to true will allow multiple isolated containers to attach the
same storage volume while keeping the file system writable from all of
them.

This makes use of idmapped mounts as an overlay file system.

(extension-resources-infiniband)=

`resources_infiniband` Copy Chapter

Export InfiniBand character device information (issm, umad, uverb) as part of the resources API.

(extension-daemon-storage)=

`daemon_storage` Copy Chapter

This introduces two new configuration keys {config:option}server-miscellaneous:storage.images_volume and
{config:option}server-miscellaneous:storage.backups_volume to allow for a storage volume on an existing
pool be used for storing the daemon-wide images and backups artifacts.

(extension-instances)=

`instances` Copy Chapter

This introduces the concept of instances, of which currently the only type is container.

(extension-image-types)=

`image_types` Copy Chapter

This introduces support for a new Type field on images, indicating what type of images they are.

(extension-resources-disk-sata)=

`resources_disk_sata` Copy Chapter

Extends the disk resource API struct to include:

• Proper detection of SATA devices (type)
• Device path
• Drive RPM
• Block size
• Firmware version
• Serial number

(extension-clustering-roles)=

`clustering_roles` Copy Chapter

This adds a new roles attribute to cluster entries, exposing a list of
roles that the member serves in the cluster.

(extension-images-expiry)=

`images_expiry` Copy Chapter

This allows for editing of the expiry date on images.

(extension-resources-network-firmware)=

`resources_network_firmware` Copy Chapter

Adds a FirmwareVersion field to network card entries.

(extension-backup-compression-algorithm)=

`backup_compression_algorithm` Copy Chapter

This adds support for a compression_algorithm property when creating a backup (POST /1.0/instances/<name>/backups).

Setting this property overrides the server default value ({config:option}server-miscellaneous:backups.compression_algorithm).

(extension-ceph-data-pool-name)=

`ceph_data_pool_name` Copy Chapter

This adds support for an optional argument ({config:option}storage-ceph-pool-conf:ceph.osd.data_pool_name) when creating
storage pools using Ceph RBD, when this argument is used the pool will store it's
actual data in the pool specified with data_pool_name while keeping the metadata
in the pool specified by pool_name.

(extension-container-syscall-intercept-mount)=

`container_syscall_intercept_mount` Copy Chapter

Adds the {config:option}instance-security:security.syscalls.intercept.mount,
{config:option}instance-security:security.syscalls.intercept.mount.allowed, and
{config:option}instance-security:security.syscalls.intercept.mount.shift configuration keys to control whether
and how the mount system call will be intercepted by LXD and processed with
elevated permissions.

(extension-compression-squashfs)=

`compression_squashfs` Copy Chapter

Adds support for importing/exporting of images/backups using SquashFS file system format.

(extension-container-raw-mount)=

`container_raw_mount` Copy Chapter

This adds support for passing in raw mount options for disk devices.

(extension-container-nic-routed)=

`container_nic_routed` Copy Chapter

This introduces the routed nic device type.

(extension-container-syscall-intercept-mount-fuse)=

`container_syscall_intercept_mount_fuse` Copy Chapter

Adds the {config:option}instance-security:security.syscalls.intercept.mount.fuse key. It can be used to
redirect file-system mounts to their fuse implementation. To this end, set e.g.
security.syscalls.intercept.mount.fuse=ext4=fuse2fs.

(extension-container-disk-ceph)=

`container_disk_ceph` Copy Chapter

This allows for existing a Ceph RBD or CephFS to be directly connected to a LXD container.

(extension-virtual-machines)=

`virtual-machines` Copy Chapter

Add virtual machine support.

(extension-image-profiles)=

`image_profiles` Copy Chapter

Allows a list of profiles to be applied to an image when launching a new container.

(extension-clustering-architecture)=

`clustering_architecture` Copy Chapter

This adds a new architecture attribute to cluster members which indicates a cluster
member's architecture.

(extension-resources-disk-id)=

`resources_disk_id` Copy Chapter

Add a new device_id field in the disk entries on the resources API.

(extension-storage-lvm-stripes)=

`storage_lvm_stripes` Copy Chapter

This adds the ability to use LVM stripes on normal volumes and thin pool volumes.

(extension-vm-boot-priority)=

`vm_boot_priority` Copy Chapter

Adds a boot.priority property on NIC and disk devices to control the boot order.

(extension-unix-hotplug-devices)=

`unix_hotplug_devices` Copy Chapter

Adds support for Unix char and block device hotplugging.

(extension-api-filtering)=

`api_filtering` Copy Chapter

Adds support for filtering the result of a GET request for instances and images.

(extension-instance-nic-network)=

`instance_nic_network` Copy Chapter

Adds support for the network property on a NIC device to allow a NIC to be linked to a managed network.
This allows it to inherit some of the network's settings and allows better validation of IP settings.

(extension-clustering-sizing)=

`clustering_sizing` Copy Chapter

Support specifying a custom values for database voters and standbys.
The new {config:option}server-cluster:cluster.max_voters and {config:option}server-cluster:cluster.max_standby configuration keys were introduced
to specify to the ideal number of database voter and standbys.

(extension-firewall-driver)=

`firewall_driver` Copy Chapter

Adds the Firewall property to the ServerEnvironment struct indicating the firewall driver being used.

(extension-projects-limits)=

`projects_limits` Copy Chapter

Adds support for project-level resource limits, through the {config:option}project-limits:limits.cpu,
{config:option}project-limits:limits.memory, {config:option}project-limits:limits.processes,
{config:option}project-limits:limits.containers, and {config:option}project-limits:limits.virtual-machines
project configuration keys.

`storage_lvm_vg_force_reuse` Copy Chapter

Introduces the ability to create a storage pool from an existing non-empty volume group.
This option should be used with care, as LXD can then not guarantee that volume name conflicts won't occur
with non-LXD created volumes in the same volume group.
This could also potentially lead to LXD deleting a non-LXD volume should name conflicts occur.

(extension-container-syscall-intercept-hugetlbfs)=

`container_syscall_intercept_hugetlbfs` Copy Chapter

When mount syscall interception is enabled and hugetlbfs is specified as an
allowed file system type LXD will mount a separate hugetlbfs instance for the
container with the UID and GID mount options set to the container's root UID
and GID. This ensures that processes in the container can use huge pages.

(extension-limits-hugepages)=

`limits_hugepages` Copy Chapter

This allows to limit the number of huge pages a container can use through the
hugetlb cgroup. This means the hugetlb cgroup needs to be available. Note, that
limiting huge pages is recommended when intercepting the mount syscall for the
hugetlbfs file system to avoid allowing the container to exhaust the host's
huge pages resources.

(extension-container-nic-routed-gateway)=

`container_nic_routed_gateway` Copy Chapter

This introduces the {config:option}device-nic-routed-device-conf:ipv4.gateway and {config:option}device-nic-routed-device-conf:ipv6.gateway NIC configuration keys that can take a value of either auto or
none. The default value for the key if unspecified is auto. This will cause the current behavior of a default
gateway being added inside the container and the same gateway address being added to the host-side interface.
If the value is set to none then no default gateway nor will the address be added to the host-side interface.
This allows multiple routed NIC devices to be added to a container.

(extension-projects-restrictions)=

`projects_restrictions` Copy Chapter

This introduces support for the {config:option}project-restricted:restricted configuration key on project, which
can prevent the use of security-sensitive features in a project.

(extension-custom-volume-snapshot-expiry)=

`custom_volume_snapshot_expiry` Copy Chapter

This allows custom volume snapshots to expiry.
Expiry dates can be set individually, or by setting the snapshots.expiry configuration key on the parent custom volume which then automatically applies to all created snapshots.

(extension-volume-snapshot-scheduling)=

`volume_snapshot_scheduling` Copy Chapter

This adds support for custom volume snapshot scheduling. It introduces two new
configuration keys: snapshots.schedule and
snapshots.pattern. Snapshots can be created automatically up to every minute.

(extension-trust-ca-certificates)=

`trust_ca_certificates` Copy Chapter

This allows for checking client certificates trusted by the provided CA (server.ca).
It can be enabled by setting {config:option}server-core:core.trust_ca_certificates to true.
If enabled, it will perform the check, and bypass the trusted password if true.
An exception will be made if the connecting client certificate is in the provided CRL (ca.crl).
In this case, it will ask for the password.

(extension-snapshot-disk-usage)=

`snapshot_disk_usage` Copy Chapter

This adds a new size field to the output of /1.0/instances/<name>/snapshots/<snapshot> which represents the disk usage of the snapshot.

(extension-clustering-edit-roles)=

`clustering_edit_roles` Copy Chapter

This adds a writable endpoint for cluster members, allowing the editing of their roles.

(extension-container-nic-routed-host-address)=

`container_nic_routed_host_address` Copy Chapter

This introduces the {config:option}device-nic-routed-device-conf:ipv4.host_address and {config:option}device-nic-routed-device-conf:ipv6.host_address NIC configuration keys that can be used to control the
host-side veth interface's IP addresses. This can be useful when using multiple routed NICs at the same time and
needing a predictable next-hop address to use.

This also alters the behavior of {config:option}device-nic-routed-device-conf:ipv4.gateway and {config:option}device-nic-routed-device-conf:ipv6.gateway NIC configuration keys. When they are set to auto
the container will have its default gateway set to the value of ipv4.host_address or ipv6.host_address respectively.

The default values are:

ipv4.host_address: 169.254.0.1
ipv6.host_address: fe80::1

This is backward compatible with the previous default behavior.

(extension-container-nic-ipvlan-gateway)=

`container_nic_ipvlan_gateway` Copy Chapter

This introduces the {config:option}device-nic-ipvlan-device-conf:ipv4.gateway and {config:option}device-nic-ipvlan-device-conf:ipv6.gateway NIC configuration keys that can take a value of either auto or
none. The default value for the key if unspecified is auto. This will cause the current behavior of a default
gateway being added inside the container and the same gateway address being added to the host-side interface.
If the value is set to none then no default gateway nor will the address be added to the host-side interface.
This allows multiple IPVLAN NIC devices to be added to a container.

(extension-resources-usb-pci)=

`resources_usb_pci` Copy Chapter

This adds USB and PCI devices to the output of /1.0/resources.

(extension-resources-cpu-threads-numa)=

`resources_cpu_threads_numa` Copy Chapter

This indicates that the numa_node field is now recorded per-thread
rather than per core as some hardware apparently puts threads in
different NUMA domains.

(extension-resources-cpu-core-die)=

`resources_cpu_core_die` Copy Chapter

Exposes the die_id information on each core.

(extension-api-os)=

`api_os` Copy Chapter

This introduces two new fields in /1.0, os and os_version.

Those are taken from the OS-release data on the system.

(extension-container-nic-routed-host-table)=

`container_nic_routed_host_table` Copy Chapter

This introduces the {config:option}device-nic-routed-device-conf:ipv4.host_table and {config:option}device-nic-routed-device-conf:ipv6.host_table NIC configuration keys that can be used to add static routes
for the instance's IPs to a custom policy routing table by ID.

(extension-container-nic-ipvlan-host-table)=

`container_nic_ipvlan_host_table` Copy Chapter

This introduces the {config:option}device-nic-ipvlan-device-conf:ipv4.host_table and {config:option}device-nic-ipvlan-device-conf:ipv6.host_table NIC configuration keys that can be used to add static routes
for the instance's IPs to a custom policy routing table by ID.

(extension-container-nic-ipvlan-mode)=

`container_nic_ipvlan_mode` Copy Chapter

This introduces the {config:option}device-nic-ipvlan-device-conf:mode NIC configuration key that can be used to switch the ipvlan mode into either l2 or l3s.
If not specified, the default value is l3s (which is the old behavior).

In l2 mode the {config:option}device-nic-ipvlan-device-conf:ipv4.address and {config:option}device-nic-ipvlan-device-conf:ipv6.address keys will accept addresses in either CIDR or singular formats.
If singular format is used, the default subnet size is taken to be /24 and /64 for IPv4 and IPv6 respectively.

In l2 mode the {config:option}device-nic-ipvlan-device-conf:ipv4.gateway and {config:option}device-nic-ipvlan-device-conf:ipv6.gateway keys accept only a singular IP address.

(extension-resources-system)=

`resources_system` Copy Chapter

This adds system information to the output of /1.0/resources.

(extension-images-push-relay)=

`images_push_relay` Copy Chapter

This adds the push and relay modes to image copy.
It also introduces the following new endpoint:

• POST 1.0/images/<fingerprint>/export

(extension-network-dns-search)=

`network_dns_search` Copy Chapter

This introduces the dns.search configuration option on networks.

(extension-container-nic-routed-limits)=

`container_nic_routed_limits` Copy Chapter

This introduces {config:option}device-nic-routed-device-conf:limits.ingress, {config:option}device-nic-routed-device-conf:limits.egress and {config:option}device-nic-routed-device-conf:limits.max for routed NICs.

(extension-instance-nic-bridged-vlan)=

`instance_nic_bridged_vlan` Copy Chapter

This introduces the {config:option}device-nic-bridged-device-conf:vlan and {config:option}device-nic-bridged-device-conf:vlan.tagged settings for bridged NICs.

vlan specifies the non-tagged VLAN to join, and vlan.tagged is a comma-delimited list of tagged VLANs to join.

(extension-network-state-bond-bridge)=

`network_state_bond_bridge` Copy Chapter

This adds a bridge and bond section to the /1.0/networks/NAME/state API.

Those contain additional state information relevant to those particular types.

Bond:

• Mode
• Transmit hash
• Up delay
• Down delay
• MII frequency
• MII state
• Lower devices

Bridge:

• ID
• Forward delay
• STP mode
• Default VLAN
• VLAN filtering
• Upper devices

`resources_cpu_isolated` Copy Chapter

Add an Isolated property on CPU threads to indicate if the thread is
physically Online but is configured not to accept tasks.

(extension-usedby-consistency)=

`usedby_consistency` Copy Chapter

This extension indicates that UsedBy should now be consistent with
suitable ?project= and ?target= when appropriate.

The 5 entities that have UsedBy are:

• Profiles
• Projects
• Networks
• Storage pools
• Storage volumes

(extension-custom-block-volumes)=

`custom_block_volumes` Copy Chapter

This adds support for creating and attaching custom block volumes to instances.
It introduces the new --type flag when creating custom storage volumes, and accepts the values fs and block.

(extension-clustering-failure-domains)=

`clustering_failure_domains` Copy Chapter

This extension adds a new failure_domain field to the PUT /1.0/cluster/<node> API,
which can be used to set the failure domain of a node.

(extension-container-syscall-filtering-allow-deny-syntax)=

`container_syscall_filtering_allow_deny_syntax` Copy Chapter

A number of new syscalls related container configuration keys were updated.

• {config:option}instance-security:security.syscalls.deny_default
• {config:option}instance-security:security.syscalls.deny_compat
• {config:option}instance-security:security.syscalls.deny
• {config:option}instance-security:security.syscalls.allow

Support for the offensively named variants was removed.

(extension-resources-gpu-mdev)=

`resources_gpu_mdev` Copy Chapter

Expose available mediated device profiles and devices in /1.0/resources.

(extension-console-vga-type)=

`console_vga_type` Copy Chapter

This extends the /1.0/console endpoint to take a ?type= argument, which can
be set to console (default) or vga (the new type added by this extension).

When doing a POST to /1.0//console?type=vga the data WebSocket
returned by the operation in the metadata field will be a bidirectional proxy
attached to a SPICE Unix socket of the target virtual machine.

(extension-projects-limits-disk)=

`projects_limits_disk` Copy Chapter

Add {config:option}project-limits:limits.disk to the available project configuration keys. If set, it limits
the total amount of disk space that instances volumes, custom volumes and images
volumes can use in the project.

(extension-network-type-macvlan)=

`network_type_macvlan` Copy Chapter

Adds support for additional network type macvlan and adds {config:option}network-macvlan-network-conf:parent configuration key for this network type to
specify which parent interface should be used for creating NIC device interfaces on top of.

Also adds {config:option}device-nic-macvlan-device-conf:network configuration key support for macvlan NICs to allow them to specify the associated network of
the same type that they should use as the basis for the NIC device.

(extension-network-type-sriov)=

`network_type_sriov` Copy Chapter

Adds support for additional network type sriov and adds {config:option}network-sriov-network-conf:parent configuration key for this network type to
specify which parent interface should be used for creating NIC device interfaces on top of.

Also adds {config:option}device-nic-sriov-device-conf:network configuration key support for sriov NICs to allow them to specify the associated network of
the same type that they should use as the basis for the NIC device.

(extension-container-syscall-intercept-bpf-devices)=

`container_syscall_intercept_bpf_devices` Copy Chapter

This adds support to intercept the bpf syscall in containers. Specifically, it allows to manage device cgroup bpf programs.

(extension-network-type-ovn)=

`network_type_ovn` Copy Chapter

Adds support for additional network type ovn with the ability to specify a bridge type network as the parent.

Introduces a new NIC device type of ovn which allows the network configuration key to specify which ovn
type network they should connect to.

Also introduces two new global configuration keys that apply to all ovn networks and NIC devices:

• {config:option}server-miscellaneous:network.ovn.integration_bridge - the OVS integration bridge to use.
• {config:option}server-miscellaneous:network.ovn.northbound_connection - the OVN northbound database connection string.

(extension-projects-networks)=

`projects_networks` Copy Chapter

Adds the {config:option}project-features:features.networks configuration key to projects and the ability for a project to hold networks.

(extension-projects-networks-restricted-uplinks)=

`projects_networks_restricted_uplinks` Copy Chapter

Adds the {config:option}project-restricted:restricted.networks.uplinks project configuration key to indicate (as a comma-delimited list) which networks
the networks created inside the project can use as their uplink network.

(extension-custom-volume-backup)=

`custom_volume_backup` Copy Chapter

Add custom volume backup support.

This includes the following new endpoints (see RESTful API for details):

• GET /1.0/storage-pools//<type>/<volume>/backups

• POST /1.0/storage-pools//<type>/<volume>/backups

• GET /1.0/storage-pools//<type>/<volume>/backups/<name>

• POST /1.0/storage-pools//<type>/<volume>/backups/<name>

• DELETE /1.0/storage-pools//<type>/<volume>/backups/<name>

• GET /1.0/storage-pools//<type>/<volume>/backups/<name>/export

The following existing endpoint has been modified:

• POST /1.0/storage-pools//<type>/<volume> accepts the new source type backup

(extension-backup-override-name)=

`backup_override_name` Copy Chapter

Adds Name field to InstanceBackupArgs to allow specifying a different instance name when restoring a backup.

Adds Name and PoolName fields to StoragePoolVolumeBackupArgs to allow specifying a different volume name
when restoring a custom volume backup.

(extension-storage-rsync-compression)=

`storage_rsync_compression` Copy Chapter

Adds rsync.compression configuration key to storage pools. This key can be used
to disable compression in rsync while migrating storage pools.

(extension-network-type-physical)=

`network_type_physical` Copy Chapter

Adds support for additional network type physical that can be used as an uplink for ovn networks.

The interface specified by {config:option}network-physical-network-conf:parent on the physical network will be connected to the ovn network's gateway.

(extension-network-ovn-external-subnets)=

`network_ovn_external_subnets` Copy Chapter

Adds support for ovn networks to use external subnets from uplink networks.

Introduces the {config:option}network-physical-network-conf:ipv4.routes and {config:option}network-physical-network-conf:ipv6.routes setting on physical networks that defines the external routes
allowed to be used in child OVN networks in their {config:option}device-nic-ovn-device-conf:ipv4.routes.external and {config:option}device-nic-ovn-device-conf:ipv6.routes.external settings.

Introduces the {config:option}project-restricted:restricted.networks.subnets project setting that specifies which external subnets are allowed to
be used by OVN networks inside the project (if not set then all routes defined on the uplink network are allowed).

(extension-network-ovn-nat)=

`network_ovn_nat` Copy Chapter

Adds support for {config:option}network-ovn-network-conf:ipv4.nat and {config:option}network-ovn-network-conf:ipv6.nat settings on ovn networks.

When creating the network if these settings are unspecified, and an equivalent IP address is being generated for
the subnet, then the appropriate NAT setting will added set to true.

If the setting is missing then the value is taken as false.

(extension-network-ovn-external-routes-remove)=

`network_ovn_external_routes_remove` Copy Chapter

Removes the settings ipv4.routes.external and ipv6.routes.external from ovn networks.

The equivalent settings on the ovn NIC type can be used instead for this, rather than having to specify them
both at the network and NIC level.

(extension-tpm-device-type)=

`tpm_device_type` Copy Chapter

This introduces the tpm device type.

(extension-storage-zfs-clone-copy-rebase)=

`storage_zfs_clone_copy_rebase` Copy Chapter

This introduces rebase as a value for {config:option}storage-zfs-pool-conf:zfs.clone_copy causing LXD to
track down any image dataset in the ancestry line and then perform
send/receive on top of that.

(extension-gpu-mdev)=

`gpu_mdev` Copy Chapter

This adds support for virtual GPUs (vGPUs). It introduces the {config:option}device-gpu-mdev-device-conf:mdev configuration key for GPU devices which takes
a supported mdev type, e.g. i915-GVTg_V5_4.

(extension-resources-pci-iommu)=

`resources_pci_iommu` Copy Chapter

This adds the IOMMUGroup field for PCI entries in the resources API.

(extension-resources-network-usb)=

`resources_network_usb` Copy Chapter

Adds the usb_address field to the network card entries in the resources API.

(extension-resources-disk-address)=

`resources_disk_address` Copy Chapter

Adds the usb_address and pci_address fields to the disk entries in the resources API.

(extension-network-physical-ovn-ingress-mode)=

`network_physical_ovn_ingress_mode` Copy Chapter

Adds {config:option}network-physical-network-conf:ovn.ingress_mode setting for physical networks.

Sets the method that OVN NIC external IPs will be advertised on uplink network.

Either l2proxy (proxy ARP/NDP) or routed.

(extension-network-ovn-dhcp)=

`network_ovn_dhcp` Copy Chapter

Adds {config:option}network-ovn-network-conf:ipv4.dhcp and {config:option}network-ovn-network-conf:ipv6.dhcp settings for ovn networks.

Allows DHCP (and RA for IPv6) to be disabled. Defaults to on.

(extension-network-physical-routes-anycast)=

`network_physical_routes_anycast` Copy Chapter

Adds {config:option}network-physical-network-conf:ipv4.routes.anycast and {config:option}network-physical-network-conf:ipv6.routes.anycast Boolean settings for physical networks. Defaults to false.

Allows OVN networks using physical network as uplink to relax external subnet/route overlap detection when used
with {config:option}network-physical-network-conf:ovn.ingress_mode set to routed.

(extension-projects-limits-instances)=

`projects_limits_instances` Copy Chapter

Adds {config:option}project-limits:limits.instances to the available project configuration keys. If set, it
limits the total number of instances (VMs and containers) that can be used in the project.

(extension-network-state-vlan)=

`network_state_vlan` Copy Chapter

This adds a vlan section to the /1.0/networks/NAME/state API.

Those contain additional state information relevant to VLAN interfaces:

• lower_device
• vid

(extension-instance-nic-bridged-port-isolation)=

`instance_nic_bridged_port_isolation` Copy Chapter

This adds the {config:option}device-nic-bridged-device-conf:security.port_isolation field for bridged NIC instances.

(extension-instance-bulk-state-change)=

`instance_bulk_state_change` Copy Chapter

Adds the following endpoint for bulk state change (see RESTful API for details):

• PUT /1.0/instances

(extension-network-gvrp)=

`network_gvrp` Copy Chapter

This adds an optional gvrp property to macvlan and physical networks,
and to ipvlan, macvlan, routed and physical NIC devices.

When set, this specifies whether the VLAN should be registered using GARP VLAN
Registration Protocol. Defaults to false.

(extension-instance-pool-move)=

`instance_pool_move` Copy Chapter

This adds a pool field to the POST /1.0/instances/NAME API,
allowing for easy move of an instance root disk between pools.

(extension-gpu-sriov)=

`gpu_sriov` Copy Chapter

This adds support for SR-IOV enabled GPUs.
It introduces the sriov GPU type property.

(extension-pci-device-type)=

`pci_device_type` Copy Chapter

This introduces the pci device type.

(extension-storage-volume-state)=

`storage_volume_state` Copy Chapter

Add new /1.0/storage-pools/POOL/volumes/VOLUME/state API endpoint to get usage data on a volume.

(extension-network-acl)=

`network_acl` Copy Chapter

This adds the concept of network ACLs to API under the API endpoint prefix /1.0/network-acls.

(extension-migration-stateful)=

`migration_stateful` Copy Chapter

Add a new {config:option}instance-migration:migration.stateful configuration key.

(extension-disk-state-quota)=

`disk_state_quota` Copy Chapter

This introduces the {config:option}device-disk-device-conf:size.state device configuration key on disk devices.

(extension-storage-ceph-features)=

`storage_ceph_features` Copy Chapter

Adds a new {config:option}storage-ceph-pool-conf:ceph.rbd.features configuration key on storage pools to control the RBD features used for new volumes.

(extension-projects-compression)=

`projects_compression` Copy Chapter

Adds new {config:option}project-specific:backups.compression_algorithm and {config:option}project-specific:images.compression_algorithm configuration keys which
allows configuration of backup and image compression per-project.

(extension-projects-images-remote-cache-expiry)=

`projects_images_remote_cache_expiry` Copy Chapter

Add new {config:option}server-images:images.remote_cache_expiry configuration key to projects,
allowing for set number of days after which an unused cached remote image will be flushed.

(extension-certificate-project)=

`certificate_project` Copy Chapter

Adds a new restricted property to certificates in the API as well as
projects holding a list of project names that the certificate has
access to.

(extension-network-ovn-acl)=

`network_ovn_acl` Copy Chapter

Adds a new security.acls property to OVN networks and OVN NICs, allowing Network ACLs to be applied.

(extension-projects-images-auto-update)=

`projects_images_auto_update` Copy Chapter

Adds new {config:option}server-images:images.auto_update_cached and {config:option}server-images:images.auto_update_interval configuration keys which
allows configuration of images auto update in projects

(extension-projects-restricted-cluster-target)=

`projects_restricted_cluster_target` Copy Chapter

Adds new {config:option}project-restricted:restricted.cluster.target configuration key to project which prevent the user from using --target
to specify what cluster member to place a workload on or the ability to move a workload between members.

(extension-images-default-architecture)=

`images_default_architecture` Copy Chapter

Adds new {config:option}server-images:images.default_architecture global configuration key and matching per-project key which lets user tell LXD
what architecture to go with when no specific one is specified as part of the image request.

(extension-network-ovn-acl-defaults)=

`network_ovn_acl_defaults` Copy Chapter

Adds new security.acls.default.{in,e}gress.action and security.acls.default.{in,e}gress.logged configuration keys for
OVN networks and NICs. This replaces the removed ACL default.action and default.logged keys.

(extension-gpu-mig)=

`gpu_mig` Copy Chapter

This adds support for NVIDIA MIG. It introduces the mig GPU type and associated configuration keys.

(extension-project-usage)=

`project_usage` Copy Chapter

Adds an API endpoint to get current resource allocations in a project.
Accessible at API GET /1.0/projects/<name>/state.

(extension-network-bridge-acl)=

`network_bridge_acl` Copy Chapter

Adds a new {config:option}network-bridge-network-conf:security.acls configuration key to bridge networks, allowing Network ACLs to be applied.

Also adds security.acls.default.{in,e}gress.action and security.acls.default.{in,e}gress.logged configuration keys for
specifying the default behavior for unmatched traffic.

(extension-warnings)=

`warnings` Copy Chapter

Warning API for LXD.

This includes the following endpoints (see Restful API for details):

• GET /1.0/warnings

• GET /1.0/warnings/

• PUT /1.0/warnings/

• DELETE /1.0/warnings/

(extension-projects-restricted-backups-and-snapshots)=

`projects_restricted_backups_and_snapshots` Copy Chapter

Adds new {config:option}project-restricted:restricted.backups and {config:option}project-restricted:restricted.snapshots configuration keys to project which
prevents the user from creation of backups and snapshots.

(extension-clustering-join-token)=

`clustering_join_token` Copy Chapter

Adds POST /1.0/cluster/members API endpoint for requesting a join token used when adding new cluster members
without using the trust password.

(extension-clustering-description)=

`clustering_description` Copy Chapter

Adds an editable description to the cluster members.

(extension-server-trusted-proxy)=

`server_trusted_proxy` Copy Chapter

This introduces support for {config:option}server-core:core.https_trusted_proxy which has LXD
parse a HAProxy style connection header on such connections and if
present, will rewrite the request's source address to that provided by
the proxy server.

(extension-clustering-update-cert)=

`clustering_update_cert` Copy Chapter

Adds PUT /1.0/cluster/certificate endpoint for updating the cluster
certificate across the whole cluster

(extension-storage-api-project)=

`storage_api_project` Copy Chapter

This adds support for copy/move custom storage volumes between projects.

(extension-server-instance-driver-operational)=

`server_instance_driver_operational` Copy Chapter

This modifies the driver output for the /1.0 endpoint to only include drivers which are actually supported and
operational on the server (as opposed to being included in LXD but not operational on the server).

(extension-server-supported-storage-drivers)=

`server_supported_storage_drivers` Copy Chapter

This adds supported storage driver info to server environment info.

(extension-event-lifecycle-requestor-address)=

`event_lifecycle_requestor_address` Copy Chapter

Adds a new address field to lifecycle requestor.

(extension-resources-gpu-usb)=

`resources_gpu_usb` Copy Chapter

Add a new USBAddress (usb_address) field to ResourcesGPUCard (GPU entries) in the resources API.

(extension-clustering-evacuation)=

`clustering_evacuation` Copy Chapter

Adds POST /1.0/cluster/members/<name>/state endpoint for evacuating and restoring cluster members.
It also adds the configuration keys {config:option}instance-miscellaneous:cluster.evacuate and {config:option}instance-volatile:volatile.evacuate.origin for setting the evacuation method (auto, stop or migrate) and the origin of any migrated instance respectively.

(extension-network-ovn-nat-address)=

`network_ovn_nat_address` Copy Chapter

This introduces the {config:option}network-ovn-network-conf:ipv4.nat.address and {config:option}network-ovn-network-conf:ipv6.nat.address configuration keys for LXD ovn networks.
Those keys control the source address used for outbound traffic from the OVN virtual network.
These keys can only be specified when the OVN network's uplink network has {config:option}network-physical-network-conf:ovn.ingress_mode set to routed.

(extension-network-bgp)=

`network_bgp` Copy Chapter

This introduces support for LXD acting as a BGP router to advertise
routes to bridge and ovn networks.

This comes with the addition to global configuration of:

• {config:option}server-core:core.bgp_address
• {config:option}server-core:core.bgp_asn
• {config:option}server-core:core.bgp_routerid

The following network configurations keys (bridge and physical):

• bgp.peers.<name>.address
• bgp.peers.<name>.asn
• bgp.peers.<name>.password

The nexthop configuration keys (bridge):

• {config:option}network-bridge-network-conf:bgp.ipv4.nexthop
• {config:option}network-bridge-network-conf:bgp.ipv6.nexthop

And the following NIC-specific configuration keys (bridged NIC type):

• {config:option}device-nic-bridged-device-conf:ipv4.routes.external
• {config:option}device-nic-bridged-device-conf:ipv6.routes.external

(extension-network-forward)=

`network_forward` Copy Chapter

This introduces the networking address forward functionality. Allowing for bridge and ovn networks to define
external IP addresses that can be forwarded to internal IP(s) inside their respective networks.

(extension-custom-volume-refresh)=

`custom_volume_refresh` Copy Chapter

Adds support for refresh during volume migration.

(extension-network-counters-errors-dropped)=

`network_counters_errors_dropped` Copy Chapter

This adds the received and sent errors as well as inbound and outbound dropped packets to the network counters.

(extension-metrics)=

`metrics` Copy Chapter

This adds metrics to LXD. It returns metrics of running instances using the OpenMetrics format.

This includes the following endpoints:

• GET /1.0/metrics

(extension-image-source-project)=

`image_source_project` Copy Chapter

Adds a new project field to POST /1.0/images allowing for the source project
to be set at image copy time.

(extension-clustering-config)=

`clustering_config` Copy Chapter

Adds new config property to cluster members with configurable key/value pairs.

(extension-network-peer)=

`network_peer` Copy Chapter

This adds network peering to allow traffic to flow between OVN networks without leaving the OVN subsystem.

(extension-linux-sysctl)=

`linux_sysctl` Copy Chapter

Adds new linux.sysctl.* configuration keys allowing users to modify certain kernel parameters
within containers.

(extension-network-dns)=

`network_dns` Copy Chapter

Introduces a built-in DNS server and zones API to provide DNS records for LXD instances.

This introduces the following server configuration key:

• {config:option}server-core:core.dns_address

The following network configuration key:

• dns.zone.forward
• dns.zone.reverse.ipv4
• dns.zone.reverse.ipv6

And the following project configuration key:

• {config:option}project-restricted:restricted.networks.zones

A new REST API is also introduced to manage DNS zones:

• /1.0/network-zones (GET, POST)
• /1.0/network-zones/<name> (GET, PUT, PATCH, DELETE)

(extension-ovn-nic-acceleration)=

`ovn_nic_acceleration` Copy Chapter

Adds new {config:option}device-nic-ovn-device-conf:acceleration configuration key to OVN NICs which can be used for enabling hardware acceleration.
It takes the values none or sriov.

(extension-certificate-self-renewal)=

`certificate_self_renewal` Copy Chapter

This adds support for renewing a client's own trust certificate.

(extension-instance-project-move)=

`instance_project_move` Copy Chapter

This adds a project field to the POST /1.0/instances/NAME API,
allowing for easy move of an instance between projects.

(extension-storage-volume-project-move)=

`storage_volume_project_move` Copy Chapter

This adds support for moving storage volume between projects.

(extension-cloud-init)=

`cloud_init` Copy Chapter

This adds a new cloud-init configuration key namespace which contains the following keys:

• {config:option}instance-cloud-init:cloud-init.vendor-data
• {config:option}instance-cloud-init:cloud-init.user-data
• {config:option}instance-cloud-init:cloud-init.network-config

It also adds a new endpoint /1.0/devices to devlxd which shows an instance's devices.

(extension-network-dns-nat)=

`network_dns_nat` Copy Chapter

This introduces network.nat as a configuration option on network zones (DNS).

It defaults to the current behavior of generating records for all
instances NICs but if set to false, it will instruct LXD to only
generate records for externally reachable addresses.

(extension-database-leader)=

`database_leader` Copy Chapter

Adds new database-leader role which is assigned to cluster leader.

(extension-instance-all-projects)=

`instance_all_projects` Copy Chapter

This adds support for displaying instances from all projects.

(extension-clustering-groups)=

`clustering_groups` Copy Chapter

Add support for grouping cluster members.

This introduces the following new endpoints:

• /1.0/cluster/groups (GET, POST)
• /1.0/cluster/groups/<name> (GET, POST, PUT, PATCH, DELETE)

The following project restriction is added:

• {config:option}project-restricted:restricted.cluster.groups

(extension-ceph-rbd-du)=

`ceph_rbd_du` Copy Chapter

Adds a new {config:option}storage-ceph-pool-conf:ceph.rbd.du Boolean on Ceph storage pools which allows
disabling the use of the potentially slow rbd du calls.

(extension-instance-get-full)=

`instance_get_full` Copy Chapter

This introduces a new recursion=1 mode for GET /1.0/instances/{name} which allows for the retrieval of
all instance structs, including the state, snapshots and backup structs.

(extension-qemu-metrics)=

`qemu_metrics` Copy Chapter

This adds a new {config:option}instance-security:security.agent.metrics Boolean which defaults to true.
When set to false, it doesn't connect to the lxd-agent for metrics and other state information, but relies on stats from QEMU.

(extension-gpu-mig-uuid)=

`gpu_mig_uuid` Copy Chapter

Adds support for the new MIG UUID format used by NVIDIA 470+ drivers (for example, MIG-74c6a31a-fde5-5c61-973b-70e12346c202),
the MIG- prefix can be omitted

This extension supersedes old mig.gi and mig.ci parameters which are kept for compatibility with old drivers and
cannot be set together.

(extension-event-project)=

`event_project` Copy Chapter

Expose the project an API event belongs to.

(extension-clustering-evacuation-live)=

`clustering_evacuation_live` Copy Chapter

This adds live-migrate as a configuration option to {config:option}instance-miscellaneous:cluster.evacuate, which forces live-migration
of instances during cluster evacuation.

(extension-instance-allow-inconsistent-copy)=

`instance_allow_inconsistent_copy` Copy Chapter

Adds allow_inconsistent field to instance source on POST /1.0/instances. If true, rsync will ignore the
Partial transfer due to vanished source files (code 24) error when creating an instance from a copy.

(extension-network-state-ovn)=

`network_state_ovn` Copy Chapter

This adds an ovn section to the /1.0/networks/NAME/state API which contains additional state information relevant to
OVN networks:

• chassis

(extension-storage-volume-api-filtering)=

`storage_volume_api_filtering` Copy Chapter

Adds support for filtering the result of a GET request for storage volumes.

(extension-image-restrictions)=

`image_restrictions` Copy Chapter

This extension adds on to the image properties to include image restrictions/host requirements. These requirements
help determine the compatibility between an instance and the host system.

(extension-storage-zfs-export)=

`storage_zfs_export` Copy Chapter

Introduces the ability to disable zpool export when unmounting pool by setting {config:option}storage-zfs-pool-conf:zfs.export.

(extension-network-dns-records)=

`network_dns_records` Copy Chapter

This extends the network zones (DNS) API to add the ability to create and manage custom records.

This adds:

• GET /1.0/network-zones/ZONE/records
• POST /1.0/network-zones/ZONE/records
• GET /1.0/network-zones/ZONE/records/RECORD
• PUT /1.0/network-zones/ZONE/records/RECORD
• PATCH /1.0/network-zones/ZONE/records/RECORD
• DELETE /1.0/network-zones/ZONE/records/RECORD

(extension-storage-zfs-reserve-space)=

`storage_zfs_reserve_space` Copy Chapter

Adds ability to set the reservation/refreservation ZFS property along with quota/refquota.

(extension-network-acl-log)=

`network_acl_log` Copy Chapter

Adds a new GET /1.0/networks-acls/NAME/log API to retrieve ACL firewall logs.

(extension-storage-zfs-blocksize)=

`storage_zfs_blocksize` Copy Chapter

Introduces a new {config:option}storage-zfs-volume-conf:zfs.blocksize property for ZFS storage volumes which allows to set volume block size.

(extension-metrics-cpu-seconds)=

`metrics_cpu_seconds` Copy Chapter

This is used to detect whether LXD was fixed to output used CPU time in seconds rather than as milliseconds.

(extension-instance-snapshot-never)=

`instance_snapshot_never` Copy Chapter

Adds a @never option to snapshots.schedule which allows disabling inheritance.

(extension-certificate-token)=

`certificate_token` Copy Chapter

This adds token-based certificate addition to the trust store as a safer alternative to a trust password.

It adds the token field to POST /1.0/certificates.

(extension-instance-nic-routed-neighbor-probe)=

`instance_nic_routed_neighbor_probe` Copy Chapter

This adds the ability to disable the routed NIC IP neighbor probing for availability on the parent network.

Adds the {config:option}device-nic-routed-device-conf:ipv4.neighbor_probe and {config:option}device-nic-routed-device-conf:ipv6.neighbor_probe NIC settings. Defaulting to true if not specified.

(extension-event-hub)=

`event_hub` Copy Chapter

This adds support for event-hub cluster member role and the ServerEventMode environment field.

(extension-agent-nic-config)=

`agent_nic_config` Copy Chapter

If set to true, on VM start-up the lxd-agent will apply NIC configuration to change the names and MTU of the instance NIC
devices.

(extension-projects-restricted-intercept)=

`projects_restricted_intercept` Copy Chapter

Adds new {config:option}project-restricted:restricted.containers.interception configuration key to allow usually safe system call interception options.

(extension-metrics-authentication)=

`metrics_authentication` Copy Chapter

Introduces a new {config:option}server-core:core.metrics_authentication server configuration option to
allow for the /1.0/metrics endpoint to be generally available without
client authentication.

(extension-images-target-project)=

`images_target_project` Copy Chapter

Adds ability to copy image to a project different from the source.

(extension-cluster-migration-inconsistent-copy)=

`cluster_migration_inconsistent_copy` Copy Chapter

Adds allow_inconsistent field to POST /1.0/instances/<name>. Set to true to allow inconsistent copying between cluster
members.

(extension-cluster-ovn-chassis)=

`cluster_ovn_chassis` Copy Chapter

Introduces a new ovn-chassis cluster role which allows for specifying what cluster member should act as an OVN chassis.

(extension-container-syscall-intercept-sched-setscheduler)=

`container_syscall_intercept_sched_setscheduler` Copy Chapter

Adds the {config:option}instance-security:security.syscalls.intercept.sched_setscheduler to allow advanced process priority management in containers.

(extension-storage-lvm-thinpool-metadata-size)=

`storage_lvm_thinpool_metadata_size` Copy Chapter

Introduces the ability to specify the thin pool metadata volume size via {config:option}storage-lvm-pool-conf:lvm.thinpool_metadata_size.

If this is not specified then the default is to let LVM pick an appropriate thin pool metadata volume size.

(extension-storage-volume-state-total)=

`storage_volume_state_total` Copy Chapter

This adds total field to the GET /1.0/storage-pools/{name}/volumes/{type}/{volume}/state API.

(extension-instance-file-head)=

`instance_file_head` Copy Chapter

Implements HEAD on /1.0/instances/NAME/file.

(extension-instances-nic-host-name)=

`instances_nic_host_name` Copy Chapter

This introduces the {config:option}server-miscellaneous:instances.nic.host_name server configuration key that can take a value of either random or
mac. The default value for the key if unspecified is random. If it is set to random then use the random host interface names.
If it's set to mac, then generate a name in the form lxd1122334455.

(extension-image-copy-profile)=

`image_copy_profile` Copy Chapter

Adds ability to modify the set of profiles when image is copied.

(extension-container-syscall-intercept-sysinfo)=

`container_syscall_intercept_sysinfo` Copy Chapter

Adds the {config:option}instance-security:security.syscalls.intercept.sysinfo to allow the sysinfo syscall to be populated with cgroup-based resource usage information.

(extension-clustering-evacuation-mode)=

`clustering_evacuation_mode` Copy Chapter

This introduces a mode field to the evacuation request which allows
for overriding the evacuation mode traditionally set through
{config:option}instance-miscellaneous:cluster.evacuate.

(extension-resources-pci-vpd)=

`resources_pci_vpd` Copy Chapter

Adds a new VPD struct to the PCI resource entries.
This struct extracts vendor provided data including the full product name and additional key/value configuration pairs.

(extension-qemu-raw-conf)=

`qemu_raw_conf` Copy Chapter

Introduces a {config:option}instance-raw:raw.qemu.conf configuration key to override select sections of the generated qemu.conf.

(extension-storage-cephfs-fscache)=

`storage_cephfs_fscache` Copy Chapter

Add support for fscache/cachefilesd on CephFS pools through a new {config:option}storage-cephfs-pool-conf:cephfs.fscache configuration option.

(extension-network-load-balancer)=

`network_load_balancer` Copy Chapter

This introduces the networking load balancer functionality. Allowing ovn networks to define port(s) on external
IP addresses that can be forwarded to one or more internal IP(s) inside their respective networks.

(extension-vsock-api)=

`vsock_api` Copy Chapter

This introduces a bidirectional vsock interface which allows the lxd-agent and the LXD server to communicate better.

(extension-instance-ready-state)=

`instance_ready_state` Copy Chapter

This introduces a new Ready state for instances which can be set using devlxd.

(extension-network-bgp-holdtime)=

`network_bgp_holdtime` Copy Chapter

This introduces a new bgp.peers.<name>.holdtime configuration key to control the BGP hold time for a particular peer.

(extension-storage-volumes-all-projects)=

`storage_volumes_all_projects` Copy Chapter

This introduces the ability to list storage volumes from all projects.

(extension-metrics-memory-oom-total)=

`metrics_memory_oom_total` Copy Chapter

This introduces a new lxd_memory_OOM_kills_total metric to the /1.0/metrics API.
It reports the number of times the out of memory killer (OOM) has been triggered.

(extension-storage-buckets)=

`storage_buckets` Copy Chapter

This introduces the storage bucket API. It allows the management of S3 object storage buckets for storage pools.

(extension-storage-buckets-create-credentials)=

`storage_buckets_create_credentials` Copy Chapter

This updates the storage bucket API to return initial admin credentials at bucket creation time.

(extension-metrics-cpu-effective-total)=

`metrics_cpu_effective_total` Copy Chapter

This introduces a new lxd_cpu_effective_total metric to the /1.0/metrics API.
It reports the total number of effective CPUs.

(extension-projects-networks-restricted-access)=

`projects_networks_restricted_access` Copy Chapter

Adds the {config:option}project-restricted:restricted.networks.access project configuration key to indicate (as a comma-delimited list) which networks can be accessed inside the project.
If not specified, all networks are accessible (assuming it is also allowed by the {config:option}project-restricted:restricted.devices.nic setting, described below).

This also introduces a change whereby network access is controlled by the project's {config:option}project-restricted:restricted.devices.nic setting:

• If restricted.devices.nic is set to managed (the default if not specified), only managed networks are accessible.
• If restricted.devices.nic is set to allow, all networks are accessible (dependent on the restricted.networks.access setting).
• If restricted.devices.nic is set to block, no networks are accessible.

(extension-loki)=

`loki` Copy Chapter

This adds support for sending events to a Loki server.

It adds the following global configuration keys:

• {config:option}server-loki:loki.api.ca_cert: CA certificate which can be used when sending events to the Loki server
• {config:option}server-loki:loki.api.url: URL to the Loki server (protocol, name or IP and port)
• {config:option}server-loki:loki.auth.username and {config:option}server-loki:loki.auth.password: Used if Loki is behind a reverse proxy with basic authentication enabled
• {config:option}server-loki:loki.labels: Comma-separated list of values which are to be used as labels for Loki events.
• {config:option}server-loki:loki.loglevel: Minimum log level for events sent to the Loki server.
• {config:option}server-loki:loki.types: Types of events which are to be sent to the Loki server (any combination of lifecycle, logging, ovn, and security).

(extension-acme)=

`acme` Copy Chapter

This adds ACME support, which allows Let's Encrypt or other ACME services to issue certificates.

It adds the following global configuration keys:

• {config:option}server-acme:acme.domain: The domain for which the certificate should be issued.
• {config:option}server-acme:acme.email: The email address used for the account of the ACME service.
• {config:option}server-acme:acme.ca_url: The directory URL of the ACME service, defaults to https://acme-v02.api.letsencrypt.org/directory.

It also adds the following endpoint, which is required for the HTTP-01 challenge:

• /.well-known/acme-challenge/<token>

(extension-internal-metrics)=

`internal_metrics` Copy Chapter

This adds internal metrics to the list of metrics.
These include:

• Total running operations
• Total active warnings
• Daemon uptime in seconds
• Go memory stats
• Number of goroutines

(extension-cluster-join-token-expiry)=

`cluster_join_token_expiry` Copy Chapter

This adds an expiry to cluster join tokens which defaults to 3 hours, but can be changed by setting the {config:option}server-cluster:cluster.join_token_expiry configuration key.

(extension-remote-token-expiry)=

`remote_token_expiry` Copy Chapter

This adds an expiry to remote add join tokens.
It can be set in the {config:option}server-core:core.remote_token_expiry configuration key, and defaults to 15 days.

(extension-init-preseed)=

`init_preseed` Copy Chapter

Adds the InitPreseed type and related exported types (such as InitLocalPreseed, InitClusterPreseed, and
InitNetworksProjectPost) to shared/api, formalizing the YAML structure accepted by lxd init --preseed so
that it can be generated and parsed by other tools.

(extension-storage-volumes-created-at)=

`storage_volumes_created_at` Copy Chapter

This change adds support for storing the creation date and time of storage volumes and their snapshots.

This adds the CreatedAt field to the StorageVolume and StorageVolumeSnapshot API types.

(extension-cpu-hotplug)=

`cpu_hotplug` Copy Chapter

This adds CPU hotplugging for VMs.
Hotplugging is disabled when using CPU pinning, because this would require hotplugging NUMA devices as well, which is not possible.

(extension-projects-networks-zones)=

`projects_networks_zones` Copy Chapter

This adds support for the {config:option}project-features:features.networks.zones project feature, which changes which project network zones are
associated with when they are created. Previously network zones were tied to the value of {config:option}project-features:features.networks,
meaning they were created in the same project as networks were.

Now this has been decoupled from {config:option}project-features:features.networks to allow projects that share a network in the default project
(i.e those with features.networks=false) to have their own project level DNS zones that give a project oriented
"view" of the addresses on that shared network (which only includes addresses from instances in their project).

This also introduces a change to the network dns.zone.forward setting, which now accepts a comma-separated of
DNS zone names (a maximum of one per project) in order to associate a shared network with multiple zones.

No change to the dns.zone.reverse.* settings have been made, they still only allow a single DNS zone to be set.
However the resulting zone content that is generated now includes PTR records covering addresses from all
projects that are referencing that network via one of their forward zones.

Existing projects that have features.networks=true will have features.networks.zones=true set automatically,
but new projects will need to specify this explicitly.

(extension-network-txqueuelen)=

`network_txqueuelen` Copy Chapter

Adds a txqueuelen key to control the txqueuelen parameter of the NIC device.

(extension-cluster-member-state)=

`cluster_member_state` Copy Chapter

Adds GET /1.0/cluster/members/<member>/state API endpoint and associated ClusterMemberState API response type.

(extension-storage-pool-source-wipe)=

`storage_pool_source_wipe` Copy Chapter

Adds support for a source.wipe Boolean on the storage pool, indicating
that LXD should wipe partition headers off the requested disk rather
than potentially fail due to pre-existing file systems.

(extension-zfs-block-mode)=

`zfs_block_mode` Copy Chapter

This adds support for using ZFS block {spellexception}filesystem volumes allowing the use of different file systems on top of ZFS.

This adds the following new configuration options for ZFS storage pools:

• volume.zfs.block_mode
• volume.block.mount_options
• volume.block.filesystem

(extension-instance-generation-id)=

`instance_generation_id` Copy Chapter

Adds support for instance generation ID. The VM or container generation ID will change whenever the instance's place in time moves backwards. As of now, the generation ID is only exposed through to VM type instances. This allows for the VM guest OS to reinitialize any state it needs to avoid duplicating potential state that has already occurred:

• {config:option}instance-volatile:volatile.uuid.generation

(extension-disk-io-cache)=

`disk_io_cache` Copy Chapter

This introduces a new {config:option}device-disk-device-conf:io.cache property to disk devices which can be used to override the VM caching behavior.

(extension-amd-sev)=

`amd_sev` Copy Chapter

Adds support for AMD SEV (Secure Encrypted Virtualization) that can be used to encrypt the memory of a guest VM.

This adds the following new configuration options for SEV encryption:

• {config:option}instance-security:security.sev : (bool) is SEV enabled for this VM
• {config:option}instance-security:security.sev.policy.es : (bool) is SEV-ES enabled for this VM
• {config:option}instance-security:security.sev.session.dh : (string) guest owner's base64-encoded Diffie-Hellman key
• {config:option}instance-security:security.sev.session.data : (string) guest owner's base64-encoded session blob

(extension-storage-pool-loop-resize)=

`storage_pool_loop_resize` Copy Chapter

This allows growing loop file backed storage pools by changing the size setting of the pool.

(extension-migration-vm-live)=

`migration_vm_live` Copy Chapter

This adds support for performing VM QEMU to QEMU live migration for both shared storage (clustered Ceph) and
non-shared storage pools.

This also adds the CRIUType_VM_QEMU value of 3 for the migration CRIUType protobuf field.

(extension-ovn-nic-nesting)=

`ovn_nic_nesting` Copy Chapter

This adds support for nesting an ovn NIC inside another ovn NIC on the same instance.
This allows for an OVN logical switch port to be tunneled inside another OVN NIC using VLAN tagging.

This feature is configured by specifying the parent NIC name using the {config:option}device-nic-ovn-device-conf:nested property and the VLAN ID to use for tunneling with the {config:option}device-nic-ovn-device-conf:vlan property.

(extension-oidc)=

`oidc` Copy Chapter

This adds support for OpenID Connect (OIDC) authentication.

This adds the following new configuration keys:

• {config:option}server-oidc:oidc.issuer
• {config:option}server-oidc:oidc.client.id
• {config:option}server-oidc:oidc.audience

(extension-network-ovn-l3only)=

`network_ovn_l3only` Copy Chapter

This adds the ability to set an ovn network into "layer 3 only" mode.
This mode can be enabled at IPv4 or IPv6 level using {config:option}network-ovn-network-conf:ipv4.l3only and {config:option}network-ovn-network-conf:ipv6.l3only configuration options respectively.

With this mode enabled the following changes are made to the network:

• The virtual router's internal port address will be configured with a single host netmask (e.g. /32 for IPv4 or /128 for IPv6).
• Static routes for active instance NIC addresses will be added to the virtual router.
• A discard route for the entire internal subnet will be added to the virtual router to prevent packets destined for inactive addresses from escaping to the uplink network.
• The DHCPv4 server will be configured to indicate that a netmask of 255.255.255.255 be used for instance configuration.

(extension-ovn-nic-acceleration-vdpa)=

`ovn_nic_acceleration_vdpa` Copy Chapter

This updates the ovn_nic_acceleration API extension. The {config:option}device-nic-ovn-device-conf:acceleration configuration key for OVN NICs can now takes the value vdpa to support Virtual Data Path Acceleration (VDPA).

(extension-instances-state-total)=

`instances_state_total` Copy Chapter

This extension adds a new total field to InstanceStateDisk and InstanceStateMemory, both part of the instance's state API.

(extension-auth-user)=

`auth_user` Copy Chapter

Add current user details to the main API endpoint.

This introduces:

• auth_user_name
• auth_user_method

(extension-security-csm)=

`security_csm` Copy Chapter

Introduce a new instance-security:security.csm configuration key to control the use of
CSM (Compatibility Support Module) to allow legacy operating systems to
be run in LXD VMs.

The `security.csm` key has been replaced by `boot.mode`. See {ref}`extension-instance-boot-mode`.

(extension-instances-rebuild)=

`instances_rebuild` Copy Chapter

This extension adds the ability to rebuild an instance with the same origin image, alternate image or as empty. A new POST /1.0/instances/<name>/rebuild?project= API endpoint has been added as well as a new CLI command lxc rebuild.

(extension-numa-cpu-placement)=

`numa_cpu_placement` Copy Chapter

This adds the possibility to place a set of CPUs in a desired set of NUMA nodes.

This adds the following new configuration key:

• {config:option}instance-resource-limits:limits.cpu.nodes : (string) comma-separated list of NUMA node IDs or NUMA node ID ranges to place the CPUs (chosen with a dynamic value of {config:option}instance-resource-limits:limits.cpu) in.

(extension-custom-volume-iso)=

`custom_volume_iso` Copy Chapter

This adds the possibility to import ISO images as custom storage volumes.

This adds the --type flag to lxc storage volume import.

(extension-network-allocations)=

`network_allocations` Copy Chapter

This adds the possibility to list a LXD deployment's network allocations.

Through the lxc network list-allocations command and the --project | --all-projects flags,
you can list all the used IP addresses, hardware addresses (for instances), resource URIs and whether it uses NAT for
each instance, network, network forward and network load-balancer.

(extension-storage-api-remote-volume-snapshot-copy)=

`storage_api_remote_volume_snapshot_copy` Copy Chapter

This allows copying storage volume snapshots to and from remotes.

(extension-zfs-delegate)=

`zfs_delegate` Copy Chapter

This implements a new {config:option}storage-zfs-volume-conf:zfs.delegate volume Boolean for volumes on a ZFS storage driver.
When enabled and a suitable system is in use (requires ZFS 2.2 or higher), the ZFS dataset will be delegated to the container, allowing for its use through the zfs command line tool.

(extension-operations-get-query-all-projects)=

`operations_get_query_all_projects` Copy Chapter

This introduces support for the all-projects query parameter for the GET API calls to both /1.0/operations and /1.0/operations?recursion=1.
This parameter allows bypassing the project name filter.

(extension-metadata-configuration)=

`metadata_configuration` Copy Chapter

Adds the GET /1.0/metadata/configuration API endpoint to retrieve the generated metadata configuration in a JSON format. The JSON structure adopts the structure "configs" > `ENTITY` > `ENTITY_SECTION` > "keys" > [<CONFIG_OPTION_0>, <CONFIG_OPTION_1>, ...].
Check the list of {doc}configuration options </config-options> to see which configuration options are included.

(extension-syslog-socket)=

`syslog_socket` Copy Chapter

This introduces a syslog socket that can receive syslog formatted log messages. These can be viewed in the events API and lxc monitor, and can be forwarded to Loki. To enable this feature, set {config:option}server-core:core.syslog_socket to true.

(extension-event-lifecycle-name-and-project)=

`event_lifecycle_name_and_project` Copy Chapter

This adds the fields Name and Project to lifecycle events.

(extension-instances-nic-limits-priority)=

`instances_nic_limits_priority` Copy Chapter

This introduces a new per-NIC limits.priority option that works with both cgroup1 and cgroup2 unlike the deprecated limits.network.priority instance setting, which only worked with cgroup1.

(extension-disk-initial-volume-configuration)=

`disk_initial_volume_configuration` Copy Chapter

This API extension provides the capability to set initial volume configurations for instance root devices.
Initial volume configurations are prefixed with initial. and can be specified either through profiles or directly
during instance initialization using the --device flag.

Note that these configuration are applied only at the time of instance creation and subsequent modifications have
no effect on existing devices.

(extension-operation-wait)=

`operation_wait` Copy Chapter

This API extension indicates that the /1.0/operations/{id}/wait endpoint exists on the server. This indicates to the client
that the endpoint can be used to wait for an operation to complete rather than waiting for an operation event via the
/1.0/events endpoint.

(extension-cluster-internal-custom-volume-copy)=

`cluster_internal_custom_volume_copy` Copy Chapter

This extension adds support for copying and moving custom storage volumes within a cluster with a single API call.
Calling POST /1.0/storage-pools//custom?target=<target> will copy the custom volume specified in the source part of the request.
Calling POST /1.0/storage-pools//custom/<volume>?target=<target> will move the custom volume from the source, specified in the source part of the request, to the target.

(extension-disk-io-bus)=

`disk_io_bus` Copy Chapter

This introduces a new {config:option}device-disk-device-conf:io.bus property to disk devices which can be used to override the bus the disk is attached to.

(extension-storage-cephfs-create-missing)=

`storage_cephfs_create_missing` Copy Chapter

This introduces the configuration keys {config:option}storage-cephfs-pool-conf:cephfs.create_missing, {config:option}storage-cephfs-pool-conf:cephfs.osd_pg_num, {config:option}storage-cephfs-pool-conf:cephfs.meta_pool and {config:option}storage-cephfs-pool-conf:cephfs.data_pool to be used when adding a cephfs storage pool to instruct LXD to create the necessary entities for the storage pool, if they do not exist.

(extension-instance-move-config)=

`instance_move_config` Copy Chapter

This API extension provides the ability to use flags --profile, --no-profile, --device, and --config
when moving an instance between projects and/or storage pools.

(extension-ovn-ssl-config)=

`ovn_ssl_config` Copy Chapter

This introduces new server configuration keys to provide the SSL CA and client key pair to access the OVN databases.
The new configuration keys are {config:option}server-miscellaneous:network.ovn.ca_cert, {config:option}server-miscellaneous:network.ovn.client_cert and {config:option}server-miscellaneous:network.ovn.client_key.

(extension-init-preseed-storage-volumes)=

`init_preseed_storage_volumes` Copy Chapter

This API extension provides the ability to configure storage volumes in preseed init.

(extension-metrics-instances-count)=

`metrics_instances_count` Copy Chapter

This extends the metrics to include the containers and virtual machines counts. Instances are counted irrespective of their state.

(extension-server-instance-type-info)=

`server_instance_type_info` Copy Chapter

This API extension enables querying a server's supported instance types.
When querying the /1.0 endpoint, a new field named instance_types is added to the retrieved data.
This field indicates which instance types are supported by the server.

(extension-resources-disk-mounted)=

`resources_disk_mounted` Copy Chapter

Adds a mounted field to disk resources that LXD discovers on the system, reporting whether that disk or partition is
mounted.

(extension-server-version-lts)=

`server_version_lts` Copy Chapter

The API extension adds indication whether the LXD version is an LTS release.
This is indicated when command lxc version is executed or when /1.0 endpoint is queried.

(extension-oidc-groups-claim)=

`oidc_groups_claim` Copy Chapter

This API extension enables setting an {config:option}server-oidc:oidc.groups.claim configuration key.
If OIDC authentication is configured and this claim is set, LXD will request this claim in the scope of OIDC flow.
The value of the claim will be extracted and might be used to make authorization decisions.

(extension-loki-config-instance)=

`loki_config_instance` Copy Chapter

Adds a new {config:option}server-loki:loki.instance server configuration key to customize the instance field in Loki events.
This can be used to expose the name of the cluster rather than the individual system name sending
the event as that's usually already covered by the location field.

(extension-storage-volatile-uuid)=

`storage_volatile_uuid` Copy Chapter

Adds a new volatile.uuid configuration key to all storage volumes, snapshots and buckets.
This information can be used by storage drivers as a separate identifier besides the name
when working with volumes.

(extension-import-instance-devices)=

`import_instance_devices` Copy Chapter

This API extension provides the ability to use flags --device when importing an instance to override instance's devices.

(extension-instances-uefi-vars)=

`instances_uefi_vars` Copy Chapter

This API extension indicates that the /1.0/instances/{name}/uefi-vars endpoint is supported on the server. This endpoint allows to get the full list of UEFI variables (HTTP method GET) or replace the entire set of UEFI variables (HTTP method PUT).

(extension-instances-migration-stateful)=

`instances_migration_stateful` Copy Chapter

This API extension allows newly created VMs to have their {config:option}instance-migration:migration.stateful configuration key automatically set
through the new server-level configuration key {config:option}server-miscellaneous:instances.migration.stateful. If migration.stateful is already set at the profile or instance level then instances.migration.stateful is not applied.

(extension-access-management)=

`access_management` Copy Chapter

Adds new APIs under /1.0/auth for viewing and managing identities, groups, and permissions.
Adds an embedded OpenFGA authorization driver for enforcing fine-grained permissions.

Prior to the addition of this extension, all OIDC clients were given full access to LXD (equivalent to Unix socket access).
This extension revokes access to all OIDC clients.
To regain access, a user must:
1. Make a call to the OIDC enabled LXD remote (e.g. `lxc info`) to ensure that their OIDC identity is added to the LXD database.
2. Create a group: `lxc auth group create <group_name>`
3. Grant the group a suitable permission.
   As all OIDC clients prior to this extension have had full access to LXD, the corresponding permission is `admin` on `server`.
   To grant this permission to your group, run: `lxc auth group permission add <group_name> server admin`
4. Add themselves to the group. To do this, run: `lxc auth identity group add oidc/<email_address> <group_name>`

Steps 2 to 4 above cannot be performed via OIDC authentication (access has been revoked).
They must be performed by a sufficiently privileged user, either via Unix socket or unrestricted TLS client certificate.

For more information on access control for OIDC clients, see {ref}`fine-grained-authorization`.

(extension-vm-disk-io-limits)=

`vm_disk_io_limits` Copy Chapter

Adds the ability to limit disk I/O for virtual machines.

(extension-storage-volumes-all)=

`storage_volumes_all` Copy Chapter

This API extension adds support for listing storage volumes from all storage pools via /1.0/storage-volumes or /1.0/storage-volumes/{type} to filter by volume type. Also adds a pool field to storage volumes.

(extension-instances-files-modify-permissions)=

`instances_files_modify_permissions` Copy Chapter

Adds the ability for POST /1.0/instances/{name}/files to modify the permissions of files that already exist via the X-LXD-modify-perm header.

X-LXD-modify-perm should be a comma-separated list of 0 or more of mode, uid, and gid.

(extension-image-restriction-nesting)=

`image_restriction_nesting` Copy Chapter

This extension adds a new image restriction, requirements.nesting which when true indicates that an image cannot be run without nesting.

(extension-container-syscall-intercept-finit-module)=

`container_syscall_intercept_finit_module` Copy Chapter

Adds the {config:option}instance-miscellaneous:linux.kernel_modules.load container configuration option. If the option is set to ondemand, the finit_modules() syscall is intercepted and a privileged user in the container's user namespace can load the Linux kernel modules specified in the
allow list {config:option}instance-miscellaneous:linux.kernel_modules.

(extension-device-usb-serial)=

`device_usb_serial` Copy Chapter

This adds new configuration keys {config:option}device-unix-usb-device-conf:serial, {config:option}device-unix-usb-device-conf:busnum and {config:option}device-unix-usb-device-conf:devnum for device type usb.
The feature has been added to make it possible to distinguish between devices with identical {config:option}device-unix-usb-device-conf:vendorid and {config:option}device-unix-usb-device-conf:productid.

(extension-network-allocate-external-ips)=

`network_allocate_external_ips` Copy Chapter

Adds the ability to use an unspecified IPv4 (0.0.0.0) or IPv6 (::) address in the listen_address field of the request body for POST /1.0/networks/{networkName}/load-balancers and POST /1.0/networks/{networkName}/forwards.
If an unspecified IP address is used, supported drivers will allocate an available listen address automatically.
Allocation of external IP addresses is currently supported by the OVN network driver.
The OVN driver will allocate IP addresses from the subnets specified in the uplink network's ipv4.routes and ipv6.routes configuration options.

(extension-explicit-trust-token)=

`explicit_trust_token` Copy Chapter

Adds the ability to explicitly specify a trust token when creating a certificate
and joining an existing cluster.

(extension-shared-custom-block-volumes)=

`shared_custom_block_volumes` Copy Chapter

This adds a configuration key security.shared to custom block volumes.
If unset or false, the custom block volume cannot be attached to multiple instances.
This feature was added to prevent data loss which can happen when custom block volumes are attached to multiple instances at once.

(extension-instance-import-conversion)=

`instance_import_conversion` Copy Chapter

Adds the ability to convert images from different formats (e.g. VMDK or QCow2) into RAW image format and import them as LXD instances.

(extension-instance-create-start)=

`instance_create_start` Copy Chapter

Adds a start field to the POST /1.0/instances API which when set
to true will have the instance automatically start upon creation.

In this scenario, the creation and startup is part of a single background operation.

(extension-instance-protection-start)=

`instance_protection_start` Copy Chapter

Enables setting the {config:option}instance-security:security.protection.start field which prevents instances
from being started if set to true.

(extension-devlxd-images-vm)=

`devlxd_images_vm` Copy Chapter

Enables the {config:option}instance-security:security.devlxd.images configuration option for virtual machines.
This controls the availability of a /1.0/images/FINGERPRINT/export API over devlxd.
This can be used by a virtual machine running LXD to access raw images from the host.

(extension-disk-io-bus-virtio-blk)=

`disk_io_bus_virtio_blk` Copy Chapter

Adds a new virtio-blk value for io.bus on disk devices which allows
for the attached disk to be connected to the virtio-blk bus.

(extension-metrics-api-requests)=

`metrics_api_requests` Copy Chapter

Adds the following internal metrics:

• Total completed requests
• Number of ongoing requests

(extension-projects-limits-disk-pool)=

`projects_limits_disk_pool` Copy Chapter

This introduces per-pool project disk limits, introducing a limits.disk.pool.NAME
configuration option to the project limits. When limits.disk.pool.POOLNAME: 0
for a project, the pool is excluded from lxc storage list in that project.

(extension-ubuntu-pro-guest-attach)=

`ubuntu_pro_guest_attach` Copy Chapter

Adds a new {config:option}instance-miscellaneous:ubuntu_pro.guest_attach configuration option for instances.
When set to on, if the host has guest attachment enabled, the guest can request a guest token for Ubuntu Pro via devlxd.

For more information, see: {ref}instances-ubuntu-pro-attach.

(extension-metadata-configuration-entity-types)=

`metadata_configuration_entity_types` Copy Chapter

This adds entity type metadata to GET /1.0/metadata/configuration.
The entity type metadata is a JSON object under the entities key.

(extension-access-management-tls)=

`access_management_tls` Copy Chapter

Expands APIs under /1.0/auth to include:

1. Creation of fine-grained TLS identities, whose permissions are managed via group membership.
   This is performed via POST /1.0/auth/identities/tls.
   If the request body contains {"token": true}, a token will be returned that may be used by a non-authenticated caller to gain trust with the LXD server (the caller must send their certificate during the TLS handshake).
   If the request body contains {"certificate": ""}", the identity will be created directly.
   The request body may also specify an array of group names.
   The caller must have can_create_identities on server.
2. Deletion of OIDC and fine-grained TLS identities.
   This is performed via DELETE /1.0/auth/identities/tls/{nameOrFingerprint} or DELETE /1.0/auth/identities/oidc/{nameOrEmailAddress}.
   The caller must have can_delete on the identity. All identities may delete their own identity.
   For OIDC identities this revokes all access but does not revoke trust (authentication is performed by the identity provider).
   For fine-grained TLS identities, this revokes all access and revokes trust.
3. Functionality to update the certificate of a fine-grained TLS identity.
   This is performed via PUT /1.0/auth/identities/tls/{nameOrFingerprint} or PATCH /1.0/auth/identities/tls/{nameOrFingerprint}.
   The caller must provide a base64 encoded x509 certificate in the certificate field of the request body.
   Fine-grained TLS identities may update their own certificate.
   To update the certificate of another identity, the caller must have can_edit on the identity.

(extension-network-allocations-ovn-uplink)=

`network_allocations_ovn_uplink` Copy Chapter

Includes OVN virtual routers external IPs to /1.0/network-allocations responses with the type uplink.
Introduces the network field on each allocation, indicating to which network each allocated address belongs.
And lastly, adds a project field on leases, leases can be retrieved via /1.0/networks/<network>/leases.

(extension-network-ovn-uplink-vlan)=

`network_ovn_uplink_vlan` Copy Chapter

Adds support for using a bridge network with a specified VLAN ID as an OVN uplink.

(extension-state-logical-cpus)=

`state_logical_cpus` Copy Chapter

Adds logical_cpus field to GET /1.0/cluster/members/{name}/state which
contains the total available logical CPUs available when LXD started.

(extension-vm-limits-cpu-pin-strategy)=

`vm_limits_cpu_pin_strategy` Copy Chapter

Adds a new {config:option}instance-resource-limits:limits.cpu.pin_strategy configuration option for virtual machines. This option controls the CPU pinning strategy. When set to none, CPU auto pinning is disabled. When set to auto, CPU auto pinning is enabled.

(extension-gpu-cdi)=

`gpu_cdi` Copy Chapter

Adds support for using the Container Device Interface (CDI) specification to configure GPU passthrough in LXD containers. The id field of GPU devices now accepts CDI identifiers (for example, {VENDOR_DOMAIN_NAME}/gpu=gpu{INDEX}) for containers, in addition to DRM card IDs. This enables GPU passthrough for devices that don't use PCI addressing (like NVIDIA Tegra iGPUs) and provides a more flexible way to identify and configure GPU devices.

(extension-images-all-projects)=

`images_all_projects` Copy Chapter

This adds support for listing images across all projects using the all-projects parameter in GET /1.0/images requests.

(extension-metadata-configuration-scope)=

`metadata_configuration_scope` Copy Chapter

This adds scope metadata to GET /1.0/metadata/configuration. Options marked with a global scope are applied to all cluster members. Options with a local scope must be set on a per-member basis.

(extension-unix-device-hotplug-ownership-inherit)=

`unix_device_hotplug_ownership_inherit` Copy Chapter

Adds a new {config:option}device-unix-hotplug-device-conf:ownership.inherit configuration option for unix-hotplug devices. This option controls whether the device inherits ownership (GID and/or UID) from the host. When set to true and GID and/or UID are unset, host ownership is inherited. When set to false, host ownership is not inherited and ownership can be configured by setting {config:option}device-unix-hotplug-device-conf:gid and {config:option}device-unix-hotplug-device-conf:uid.

(extension-unix-device-hotplug-subsystem-device-option)=

`unix_device_hotplug_subsystem_device_option` Copy Chapter

Adds a new {config:option}device-unix-hotplug-device-conf:subsystem configuration option for unix-hotplug devices. This adds support for detecting unix-hotplug devices by subsystem, and can be used in conjunction with {config:option}device-unix-hotplug-device-conf:productid and {config:option}device-unix-hotplug-device-conf:vendorid.

(extension-storage-ceph-osd-pool-size)=

`storage_ceph_osd_pool_size` Copy Chapter

This introduces the configuration keys {config:option}storage-ceph-pool-conf:ceph.osd.pool_size, and {config:option}storage-cephfs-pool-conf:cephfs.osd_pool_size to be used when adding or updating a ceph or cephfs storage pool to instruct LXD to create set the replication size for the underlying OSD pools.

(extension-network-get-target)=

`network_get_target` Copy Chapter

Adds optional target parameter to GET /1.0/network. When target is set, forward the request to the specified cluster member and return the non-managed interfaces from that member.

(extension-network-zones-all-projects)=

`network_zones_all_projects` Copy Chapter

This adds support for listing network zones across all projects using the all-projects parameter in GET /1.0/network-zones requests.

(extension-vm-root-volume-attachment)=

`vm_root_volume_attachment` Copy Chapter

Adds support for virtual-machine root volumes and snapshots to be attached to other instances as disk devices. Introduces the source.type and source.snapshot keys for disk devices.

(extension-projects-limits-uplink-ips)=

`projects_limits_uplink_ips` Copy Chapter

Introduces per-project uplink IP limits for each available uplink network, adding limits.networks.uplink_ips.ipv4.NETWORK_NAME and limits.networks.uplink_ips.ipv6.NETWORK_NAME configuration keys for projects with features.networks enabled.
These keys define the maximum value of IPs made available on a network named NETWORK_NAME to be assigned as uplink IPs for entities inside a certain project. These entities can be other networks, network forwards or load balancers.

(extension-entities-with-entitlements)=

`entities_with_entitlements` Copy Chapter

Adds fine_grained field to GET /1.0/auth/identities/current to indicate if the current identity
interacting with the LXD API is fine-grained (that is, associated permissions are managed via group membership).
Allows LXD entities to be returned with an access_entitlements field if the current identity is fine-grained and the
GET request to fetch the LXD entities has the with-access-entitlements=<comma_separated_list_of_candidate_entitlements> query parameter.

(extension-profiles-all-projects)=

`profiles_all_projects` Copy Chapter

This adds support for listing profiles across all projects using the all-projects parameter in GET /1.0/profiles requests.

(extension-storage-driver-powerflex)=

`storage_driver_powerflex` Copy Chapter

Adds a new powerflex storage driver which allows the consumption of storage volumes from a Dell PowerFlex storage array using NVMe/TCP and SDC.
The following new pool level configuration keys have been added:

1. {config:option}storage-powerflex-pool-conf:powerflex.snapshot_copy
2. {config:option}storage-powerflex-pool-conf:powerflex.domain
3. {config:option}storage-powerflex-pool-conf:powerflex.gateway
4. {config:option}storage-powerflex-pool-conf:powerflex.gateway.verify
5. {config:option}storage-powerflex-pool-conf:powerflex.mode
6. {config:option}storage-powerflex-pool-conf:powerflex.pool
7. {config:option}storage-powerflex-pool-conf:powerflex.sdt
8. {config:option}storage-powerflex-pool-conf:powerflex.user.name
9. {config:option}storage-powerflex-pool-conf:powerflex.user.password

The following configuration keys have been added for volumes backed by PowerFlex:

1. {config:option}storage-powerflex-volume-conf:block.type

(extension-storage-driver-pure)=

`storage_driver_pure` Copy Chapter

Adds a new pure storage driver which allows the consumption of storage volumes from an Everpure (formerly known as Pure Storage) FlashArray using either iSCSI or NVMe/TCP.

The following pool level configuration keys have been added:

1. {config:option}storage-pure-pool-conf:pure.gateway
2. {config:option}storage-pure-pool-conf:pure.gateway.verify
3. {config:option}storage-pure-pool-conf:pure.api.token
4. {config:option}storage-pure-pool-conf:pure.mode
5. {config:option}storage-pure-pool-conf:pure.target

(extension-cloud-init-ssh-keys)=

`cloud_init_ssh_keys` Copy Chapter

Adds support for injecting additional SSH public keys into instances through {ref}cloud-init without conflicting with any configuration present on {config:option}instance-cloud-init:cloud-init.vendor-data or {config:option}instance-cloud-init:cloud-init.user-data.

To achieve this, the cloud-init.ssh-keys.KEYNAME configuration key is added for both instances and profiles. This key is used to define a public key to be injected. KEYNAME can be any arbitrary name for the injected key.

The value for cloud-init.ssh-keys.KEYNAME should be :<key>, where `` is the name of the user for whom to inject the key. For <key>, provide either the public key or a cloud-init import ID for a key hosted elsewhere. Example valid values for cloud-init.ssh-keys.KEYNAME are root:gh:githubUser or myUser:ssh-keyAlg base64PublicKey.

(extension-oidc-scopes)=

`oidc_scopes` Copy Chapter

This API extension enables setting an {config:option}server-oidc:oidc.scopes configuration key, which accepts a space-separated list of OIDC scopes to request from the identity provider.
This configuration option can be used to request additional scopes that might be required for retrieving {ref}identity provider groups from the identity provider.
Additionally, the optional scopes profile and offline_access can be unset via this setting.
Note that the openid and email scopes are always required.

(extension-project-default-network-and-storage)=

`project_default_network_and_storage` Copy Chapter

Adds flags --network and --storage. The --network flag adds a network device connected to the specified network to the default profile. The --storage flag adds a root disk device using the specified storage pool to the default profile.

(extension-client-cert-presence)=

`client_cert_presence` Copy Chapter

Adds the field client_certificate to GET /1.0 to indicate if the current request has a client certificate in it. This is for informational purposes only and does not affect the behavior of the API.

(extension-clustering-groups-used-by)=

`clustering_groups_used_by` Copy Chapter

This API extension adds a used_by field to the API response for a {ref}cluster group <cluster-groups>.
Deletion of a cluster group is disallowed if the cluster group is referenced by project configuration (see {config:option}project-restricted:restricted.cluster.groups).

(extension-container-bpf-delegation)=

`container_bpf_delegation` Copy Chapter

Adds new {config:option}instance-security:security.delegate_bpf.* group of options in order to support eBPF delegation using BPF Token mechanism. See {ref}bpf-delegation-token for more information.

(extension-override-snapshot-profiles-on-copy)=

`override_snapshot_profiles_on_copy` Copy Chapter

This adds a request option to set snapshot's target profile on instance copy to be inherited from target instance.

(extension-resources-device-fs-uuid)=

`resources_device_fs_uuid` Copy Chapter

Adds the field device_fs_uuid including the respective UUID to each disk and partition indicating whether or not a filesystem is located on the device.

(extension-backup-metadataversion)=
(extension-backup-metadata-version)=

`backup_metadata_version` Copy Chapter

Adds the field version when exporting instances and custom storage volumes to define the backup file format.
In case the field is omitted, the server's default version is used.
This maintains backwards compatibility with older clients.

When exporting an instance, the specific version can be provided using the --export-version flag:

lxc export v1 --export-version 2

The same applies when exporting a custom storage volume:

lxc storage volume export pool1 vol1 --export-version 2

(extension-storage-buckets-all-projects)=

`storage_buckets_all_projects` Copy Chapter

This adds support for listing storage buckets across all projects using the all-projects parameter in GET /1.0/storage-pools/POOL/buckets requests.

(extension-network-acls-all-projects)=

`network_acls_all_projects` Copy Chapter

This adds support for listing network ACLs across all projects using the all-projects parameter in GET /1.0/network-acls requests.

(extension-networks-all-projects)=

`networks_all_projects` Copy Chapter

This adds support for listing networks across all projects using the all-projects parameter in GET /1.0/networks requests.

(extension-clustering-restore-skip-mode)=

`clustering_restore_skip_mode` Copy Chapter

Adds a skip mode to the restore request. This mode restores a cluster member's status to ONLINE without restarting any of its stopped local instances or migrating back instances that were evacuated to other cluster members.

(extension-disk-io-threads-virtiofsd)=

`disk_io_threads_virtiofsd` Copy Chapter

Adds the {config:option}device-disk-device-conf:io.threads option on disk devices which is used to control the virtiofsd thread pool size when sharing file systems into VMs. This can help improve I/O performance.

(extension-oidc-client-secret)=

`oidc_client_secret` Copy Chapter

This adds support for the {config:option}server-oidc:oidc.client.secret configuration key.
If set, the LXD server will use this value in the OpenID Connect (OIDC) authorization code flow, which is used by LXD UI.
This configuration value is not shared with other LXD clients (such as the LXD CLI).

(extension-pci-hotplug)=

`pci_hotplug` Copy Chapter

This adds PCI device hotplugging for VMs.

(extension-device-patch-removal)=

`device_patch_removal` Copy Chapter

The PATCH /1.0/instances/{name} endpoint allows removing an instance device by setting its value to null in the devices map.

(extension-daemon-storage-per-project)=

`daemon_storage_per_project` Copy Chapter

This introduces two new configuration keys {config:option}server-miscellaneous:storage.project.{name}.images_volume and
{config:option}server-miscellaneous:storage.project.{name}.backups_volume per each project to allow for a storage volume on an existing
pool be used for storing the project-specific images and backups artifacts.

(extension-ovn-internal-load-balancer)=

`ovn_internal_load_balancer` Copy Chapter

This introduces support for internal OVN load balancers and network forwards. This approach allows ovn networks to define ports on internal IP addresses that can be forwarded to other internal IPs inside their respective networks.
This change removes the previous limitation on ovn networks that load balancers and network forwards could only use external IP addresses to forward to internal IPs.

(extension-auth-bearer-devlxd)=

`auth_bearer_devlxd` Copy Chapter

Adds a new bearer authentication method and enables authentication to the DevLXD API.
See {ref}DevLXD bearer tokens <devlxd-authentication-bearer>.

(extension-devlxd-volume-management)=

`devlxd_volume_management` Copy Chapter

Enables additional DevLXD endpoints for managing custom storage volumes and instance devices when security.devlxd.management.volumes is set to true.
These endpoints are primarily intended for use by the LXD CSI driver.

Management is limited to custom storage volumes.
Additionally, the client is required to authenticate using DevLXD identity, which is used to track storage volume and instance device owner.
Volumes and instance devices created through the DevLXD are marked as owned by the authenticated identity and can later be modified or removed through DevLXD only by the same identity.

New DevLXD endpoints:

• GET /1.0/storage-pools/{pool} — Retrieve a storage pool.
• GET /1.0/storage-pools/{pool}/volumes — List owned custom volumes in a pool.
• GET /1.0/storage-pools/{pool}/volumes/{volume} — Retrieve an owned custom volume.
• POST /1.0/storage-pools/{pool}/volumes — Create a new owned custom volume in a pool.
• PUT /1.0/storage-pools/{pool}/volumes/{volume} — Update an owned custom volume.
• DELETE /1.0/storage-pools/{pool}/volumes/{volume} — Delete an owned custom volume.
• GET /1.0/instances/{inst} — Retrieve an instance and its owned devices.
• PATCH /1.0/instances/{inst} — Add new instance devices or modify owned devices.

Changes to existing DevLXD endpoints:

• GET /1.0 — Adds a supported_storage_drivers field to the response, which is populated when security.devlxd.management.volumes is enabled.

(extension-storage-driver-alletra)=

`storage_driver_alletra` Copy Chapter

Adds a new alletra storage driver for the consumption of storage volumes from an HPE Alletra storage array.

The following pool-level configuration keys have been added:

1. {config:option}storage-alletra-pool-conf:alletra.wsapi
2. {config:option}storage-alletra-pool-conf:alletra.wsapi.verify
3. {config:option}storage-alletra-pool-conf:alletra.user.name
4. {config:option}storage-alletra-pool-conf:alletra.user.password
5. {config:option}storage-alletra-pool-conf:alletra.cpg
6. {config:option}storage-alletra-pool-conf:alletra.target
7. {config:option}storage-alletra-pool-conf:alletra.mode

(extension-resources-disk-used-by)=

`resources_disk_used_by` Copy Chapter

Adds the field used_by to potential storage disk returned by the resources end point to indicate its use by any virtual parent device, e.g. bcache.

(extension-ovn-dhcp-ranges)=

`ovn_dhcp_ranges` Copy Chapter

This introduces support for the {config:option}network-ovn-network-conf:ipv4.dhcp.ranges configuration key for ovn networks. This key allows specifying a list of IPv4 ranges
reserved for dynamic allocation using DHCP.

(extension-operation-requestor)=

`operation_requestor` Copy Chapter

This adds a new requestor field to operations, which contains information about the caller that initiated the operation.

(extension-import-custom-volume-tar)=

`import_custom_volume_tar` Copy Chapter

This adds new option tar for parameter --type in POST /1.0/storage-pools/{poolName}/volumes/{type} API call.

(extension-projects-force-delete)=

`projects_force_delete` Copy Chapter

Adds support for force deleting projects and their entities (instances, profiles, images, networks, network ACLs, network zones, storage volumes, and storage buckets) by setting the force query parameter on DELETE /1.0/projects/{name} requests.

(extension-auth-oidc-sessions)=

`auth_oidc_sessions` Copy Chapter

Adds session support for OIDC authentication. This enables compatibility with identity providers that issue opaque access tokens.

When a session expires, LXD will re-verify the login with the identity provider.
The duration of OIDC sessions defaults to one week and can be configured via the {config:option}server-oidc:oidc.session.expiry configuration key.

Verification of an OIDC session depends on a new, cluster-wide core secret.
The OIDC session expiry can never be greater than the expiry of the core secret.
The {config:option}server-core:core.auth_secret_expiry configuration option can set to define how long a given secret can be used for before it expires.

(extension-instance-snapshots-multi-volume)=

`instance_snapshots_multi_volume` Copy Chapter

Enables the creation of a multi-volume snapshot and the restoration of an instance together with its attached volumes, while ensuring crash consistency across volumes.

Adds DiskVolumesMode to POST /1.0/instances/{name}/snapshots and RestoreDiskVolumesMode to PUT /1.0/instances/{name} to select which attached volumes are included in snapshots/restores: "root" includes only the instance's root disk; "all-exclusive" includes the root disk and any exclusively attached (non-shared) volumes. Defaults to "root".

This extension also introduces a new volatile configuration key, {config:option}instance-volatile:volatile.attached_volumes, in the configuration of supported storage drivers for instance snapshots. This key contains a JSON-serialized map of attached volume UUIDs to the UUIDs of their corresponding snapshots. Example value: {<volume1-uuid>: <snapshot1-uuid>,<volume2-uuid>: <snapshot2-uuid>}.

(extension-vm-persistent-bus)=

`vm_persistent_bus` Copy Chapter

Adds support for persistently recording VM PCIe bus allocations in volatile configuration keys.

This introduces two new volatile VM configuration keys:

• {config:option}instance-volatile:volatile.<name>.bus - records the bus number for the device.
• {config:option}instance-volatile:volatile.bus.mode - records whether "persistent" mode is being used for a VM.

(extension-instance-placement-groups)=

`instance_placement_groups` Copy Chapter

Placement groups define how instances are scheduled across cluster members according to a configurable placement policy (spread or compact) and rigor (strict or permissive). They can be referenced in instance and profile configuration through the {config:option}instance-placement:placement.group key to control placement behavior during instance creation and migration.

New API endpoints:

1. GET /1.0/placement-groups — List all placement groups
2. POST /1.0/placement-groups — Create a new placement group
3. GET /1.0/placement-groups/{name} — Retrieve details for a specific placement group
4. PUT /1.0/placement-groups/{name} — Replace a placement group definition
5. PATCH /1.0/placement-groups/{name} — Update selected properties of a placement group
6. DELETE /1.0/placement-groups/{name} — Delete a placement group

(extension-ovn-nic-acceleration-parent)=

`ovn_nic_acceleration_parent` Copy Chapter

Adds support for specifying the OVN NIC acceleration physical function interfaces to allocate virtual functions from.

This avoids the need for adding physical function interfaces to the OVN integration bridge.

This introduces a new ovn network and ovn NIC configuration key:

• {config:option}device-nic-ovn-device-conf:acceleration.parent - Comma separated list of physical function (PF) interfaces to allocate virtual functions (VFs) from for hardware acceleration when {config:option}device-nic-ovn-device-conf:acceleration is enabled.

(extension-storage-and-profile-operations)=

`storage_and_profile_operations` Copy Chapter

Certain storage and profile endpoints that were previously synchronous now return an operation and behave asynchronously.

Newer LXD Go clients detect the presence of this API extension. When it is available, the client receives an operation object directly from the LXD server.
If the extension is not present, the server response is wrapped in a completed operation, allowing the caller to handle it as an operation while lacking a retrievable operation ID.

Older LXD Go clients are incompatible with servers that include this extension.
Instead of returning a successful response, they will receive an operation response.

Endpoints converted to asynchronous behavior:

• POST /storage-pools/{pool}/volumes/{type} - Create storage volume
• PUT /storage-pools/{pool}/volumes/{type}/{vol} - Update storage volume
• PATCH /storage-pools/{pool}/volumes/{type}/{vol} - Patch storage volume
• POST /storage-pools/{pool}/volumes/{type}/{vol} - Rename storage volume
• DELETE /storage-pools/{pool}/volumes/{type}/{vol} - Delete storage volume
• PUT /storage-pools/{pool}/volumes/{type}/{vol}/snapshots/{snap} - Update storage volume snapshot
• PATCH /storage-pools/{pool}/volumes/{type}/{vol}/snapshots/{snap} - Patch storage volume snapshot
• PUT /1.0/profiles/{name} - Update profile
• PATCH /1.0/profiles/{name} - Patch profile

(extension-storage-source-recover)=

`storage_source_recover` Copy Chapter

As part of the recovery process it might be necessary to recover existing storage pools previously created by LXD.

Before this was only partially possible for some of the drivers (e.g. by using lvm.vg.force_reuse), but not directly supported.
The new pool source.recover configuration key can be set per cluster member to allow reuse of an existing pool source.
What it does not allow is reusing the same source for multiple storage pools.
The LVM storage driver has the specific lvm.vg.force_reuse configuration key for this purpose.

(extension-instance-force-delete)=

`instance_force_delete` Copy Chapter

This adds support for a force query parameter to the DELETE /1.0/instances/{name} endpoint. When set, running instances will be forcibly stopped before deletion.

(extension-operation-metadata-entity-name)=
(extension-operation-metadata-entity-url)=

`operation_metadata_entity_url` Copy Chapter

Each {ref}operation event <ref-events-operation> has a resources field that contains URLs of LXD entities that the operation depends on.

When an instance, instance backup, or storage volume backup is created, it is not strictly required for the caller to provide the name of the new resource.
In this case, the URL of the expected resource is added to the resources map for clients to inspect and use.
The resources field then contains both a dependency of the operation, and the newly created resource (which may not exist yet).

To improve consistency, this API extension adds an entity_url field to operation metadata.
The field contains the expected URL of the created entity.
The field is only included when a resource is being created asynchronously (operation response), and where it is not required for the entity name to be specified by the client.
For synchronous resource creation, clients should inspect the Location header.

The resources field should no longer be relied upon for this information.

(extension-instance-boot-mode)=

`instance_boot_mode` Copy Chapter

Introduces the new {config:option}instance-boot:boot.mode configuration key to control the VM boot firmware mode.
This replaces the removed security.csm and security.secureboot settings.

The new setting accepts:

• uefi-secureboot (default) - Use UEFI firmware with secure boot enabled
• uefi-nosecureboot - Use UEFI firmware with secure boot disabled
• bios - Use legacy BIOS firmware (SeaBIOS), x86_64 (amd64) only

(extension-auth-bearer-lxd)=
(extension-auth-bearer)=

`auth_bearer` Copy Chapter

Adds new identity type bearer that allows authentication with the LXD API using bearer tokens.
See {ref}LXD bearer tokens <authentication-bearer>.

If applicable, the endpoint /1.0/auth/identities/current now also exposes the credential expiration time.
The expires_at field is set when the current identity is trusted and the authentication method is either bearer or tls.
In these cases, it reports the expiration time of the bearer token or the TLS certificate, respectively.

(extension-vm-limits-max-bus-ports)=

`vm_limits_max_bus_ports` Copy Chapter

This introduces support for the {config:option}instance-resource-limits:limits.max_bus_ports configuration key for virtual machines. This option controls the maximum allowed number of user configurable devices requiring a dedicated PCI/PCIe port for a virtual machine.
This number includes both the devices attached before the instance start and the devices hotplugged at runtime.

(extension-instances-state-selective-recursion)=

`instances_state_selective_recursion` Copy Chapter

Adds support for selective recursion when querying instances.

The API now supports selective state field fetching using semicolon-separated syntax in the recursion parameter:

• recursion=2;fields=state.disk - Fetch only disk information
• recursion=2;fields=state.network - Fetch only network information
• recursion=2;fields=state.disk,state.network - Fetch both disk and network
• recursion=2;fields= - Fetch no expensive state fields (disk and network skipped)
• recursion=2 - Fetch all fields (default behavior)

The semicolon and equals signs must be URL-encoded when used in HTTP requests (%3B for ; and %3D for =).

The lxc list command automatically optimizes queries based on requested columns.

(extension-project-delete-operation)=

`project_delete_operation` Copy Chapter

The {ref}forced project deletion <extension-projects-force-delete> API extension added support for forcibly deleting a project and all of its contents.
This can take a long time, but the DELETE /1.0/projects/{name} endpoint still returned a synchronous response.

This extension converts this endpoint to an asynchronous operation response.
As with the {ref}storage and profile operation extension <extension-storage-and-profile-operations>, this extension is forward compatible only.

(extension-gpu-cdi-amd)=

`gpu_cdi_amd` Copy Chapter

Adds support for using the Container Device Interface (CDI) specification to configure AMD GPU passthrough in LXD containers. The id field of GPU devices now accepts CDI identifiers (for example, amd.com/gpu=gpu{INDEX}) for containers, in addition to DRM card IDs.

(extension-instance-refresh-config)=

`instance_refresh_config` Copy Chapter

Ensures that instance copy --refresh operations apply target config/profile/device updates server-side.

This applies to both direct copies and migration-based refresh operations.

During refresh, the server applies the target instance configuration from the request (including config, devices, and profiles) before the data transfer completes.
The request payload is treated as the full desired writable target configuration for the refresh update.
The server does not merge or preserve omitted target keys automatically.
Clients must include any target values that should remain on the destination after refresh.

(extension-clustering-control-plane)=

`clustering_control_plane` Copy Chapter

Adds a new control-plane cluster member role that can be manually assigned to designate which members participate in Raft consensus. Control plane mode is inactive by default until the number of members assigned the control-plane role reaches 3 or more. During this inactive period, all cluster members are eligible for automatic promotion to database roles. Once control plane mode activates, only members with the control-plane role are eligible to become voters, standbys, or the database leader. Members without the control-plane role are automatically assigned the RAFT_SPARE role and are excluded from automatic promotion to database roles, enabling safe scaling of cluster members without affecting quorum.

If fewer than 3 members are assigned the control-plane role, all members remain eligible for automatic promotion to database roles, maintaining backwards compatibility with existing cluster behavior.

The control-plane role is displayed alongside the database role (database-leader, database-voter, database-standby) for members participating in Raft.

The role also controls internal event routing:

• When control plane mode is active, members with the control-plane role act as event hubs.
• When control plane mode is inactive, the cluster uses full-mesh event connectivity.

These behaviors are applied asynchronously on heartbeat. Role changes are detected on the next heartbeat cycle, and control-plane mode activation or deactivation triggers automatic role rebalancing.

The event-hub role is deprecated in favor of the functionally equivalent control-plane role.

For more information, see {ref}dqlite-internals-lxd-cluster-roles and {ref}cluster-manage-control-plane.

(extension-storage-remote-drop-source)=

`storage_remote_drop_source` Copy Chapter

Starting with this extension, support for the source configuration key in both the Ceph RBD ceph and CephFS cephfs drivers is dropped.
This ensures the storage pool configuration is done consistently across all remote drivers.

For Ceph RBD use the ceph.osd.pool_name configuration key.
For CephFS use the cephfs.path configuration key.

(extension-storage-ceph-use-rbd-defaults)=

`storage_ceph_use_rbd_defaults` Copy Chapter

Starting with this extension, new volumes (and clones) in Ceph RBD (ceph) pools are no longer created with only --image-feature layering.
Instead the default RBD features of the respective Ceph cluster are used implicitly.
This also applies to new volumes in already existing Ceph RBD storage pools.

In case the ceph.rbd.features configuration key is already set the pool, new volumes continue using this list of features and won't
use the defaults set in the Ceph cluster.

(extension-bulk-operations)=

`bulk_operations` Copy Chapter

Adds a recursion=2 mode to GET /1.0/operations, enabling retrieval of parent-child relationships between operations. The parent-child operations are now also returned by the GET /1.0/operations/{id} endpoint when recursion=1 is specified.

(extension-ovn-dynamic-northbound-connection)=

`ovn_dynamic_northbound_connection` Copy Chapter

Starting with this extension, if the {config:option}server-miscellaneous:network.ovn.northbound_connection server configuration is not specified, LXD dynamically determines the OVN Northbound database connection string based on the environment.
If the MicroOVN snap is used, LXD reads the configuration from the MicroOVN ovn.env file. Otherwise, it defaults to using the unix:/var/run/ovn/ovnnb_db.sock socket.

(extension-storage-zfs-promote)=

`storage_zfs_promote` Copy Chapter

A {config:option}storage-zfs-volume-conf:zfs.promote configuration key was introduced.

It's a Boolean that defaults to false and that when set to true instructs LXD to ZFS promote the volume being created (or re-created) from a clone.

This is primarily useful when combined with the initial. disk device configuration options, as it allows controlling ZFS promotion when creating instances from other instances.

(extension-storage-and-network-operations)=

`storage_and_network_operations` Copy Chapter

Storage pool and network endpoints that were previously synchronous now return background operations. This affects create, update, delete, and rename actions on storage pools, networks, network ACLs, network zones, network zone records, network forwards, network load balancers, network peers, and storage buckets (including bucket keys).

Clients should check for this extension and handle the asynchronous response by waiting on the returned operation. Operation metadata may include additional data, such as storage bucket admin credentials on bucket creation.

(extension-gpu-cdi-hotplug)=

`gpu_cdi_hotplug` Copy Chapter

Adds support for hotplugging GPU devices into containers when using {ref}gpu-physical-cdi.

(extension-image-extended-metadata)=

`image_extended_metadata` Copy Chapter

Adds release_codename and release_title fields to the api.Image struct. These fields are optional and are populated from the SimpleStreams index when available.

Also updates the generated image description for SimpleStreams images to include variant if available. The image creation date and architecture are no longer used for image description.

(extension-cluster-links)=

`cluster_links` Copy Chapter

Cluster links enable secure, authenticated communication between separate LXD clusters using mutual TLS certificates.
See {ref}exp-cluster-links for more information.

This introduces the cluster links API and identity type.

This includes the following new endpoints (see {ref}rest-api for details):

• GET /1.0/cluster/links/<name>
• GET /1.0/cluster/links
• GET /1.0/cluster/links/<name>/state
• PUT /1.0/cluster/links/<name>
• PATCH /1.0/cluster/links/<name>
• POST /1.0/cluster/links
• POST /1.0/cluster/links/<name>
• DELETE /1.0/cluster/links/<name>

(extension-replicators)=

`replicators` Copy Chapter

This introduces the replicators API. Replicators are used to replicate project instances across clusters.

This includes the following new endpoints (see {ref}rest-api for details):

• GET /1.0/replicators
• GET /1.0/replicators/<name>
• POST /1.0/replicators
• PUT /1.0/replicators/<name>
• PATCH /1.0/replicators/<name>
• POST /1.0/replicators/<name>
• DELETE /1.0/replicators/<name>
• GET /1.0/replicators/<name>/state

(extension-event-security)=

`event_security` Copy Chapter

Adds a new security event type conforming to OWASP security event logging guidelines.
Security events are accessible via GET /1.0/events?type=security and can be routed to Grafana Loki by adding security to the loki.types server configuration.

(extension-storage-driver-powerstore)=

`storage_driver_powerstore` Copy Chapter

Adds a new powerstore storage driver which allows the consumption of storage volumes from a PowerStore storage array using iSCSI or Fibre Channel (FC).

The following pool level configuration keys have been added:

1. {config:option}storage-powerstore-pool-conf:powerstore.gateway
2. {config:option}storage-powerstore-pool-conf:powerstore.gateway.verify
3. {config:option}storage-powerstore-pool-conf:powerstore.user.name
4. {config:option}storage-powerstore-pool-conf:powerstore.user.password
5. {config:option}storage-powerstore-pool-conf:powerstore.mode
6. {config:option}storage-powerstore-pool-conf:powerstore.target

(extension-oidc-device-client-id)=

`oidc_device_client_id` Copy Chapter

The {ref}OIDC client secret configuration extension <extension-oidc-client-secret> is used to configure LXD to send a
secret to the identity provider for authentication. It is not secure to make this secret public for consumption by the
LXD CLI (lxc). For this reason, if the client configured for LXD in the identity provider requires a secret, that
client cannot be used by CLI users.

This extension adds a new {config:option}server-oidc:oidc.device.client.id configuration key for the CLI to use. An
administrator can create two clients in the identity provider, one for LXD UI requiring a secret (and {abbr}PKCE ),
and one for the CLI that enables the device authorization grant and does not require a secret. The device client ID will
be public to the CLI, falling back to {config:option}server-oidc:oidc.client.id. This configuration option cannot be
set unless {config:option}server-oidc:oidc.client.id is set.

(extension-storage-nvme-tcp)=

`storage_nvme_tcp` Copy Chapter

Renames storage pool NVMe/TCP mode from nvme to nvme/tcp.

(extension-project-replica-mode)=

`project_replica_mode` Copy Chapter

Adds a replica_mode field to projects and a new PUT /1.0/projects/<name>/state endpoint for promoting and demoting projects between leader and standby modes for replication.

(extension-cluster-links-used-by)=

`cluster_links_used_by` Copy Chapter

Adds a used_by field to ClusterLink resources, returned by GET /1.0/cluster/links (with recursion=1) and GET /1.0/cluster/links/{name}. The field lists URLs of entities that reference the cluster link, filtered by the caller's view permissions.

(extension-network-load-balancer-pool)=

`network_load_balancer_pool` Copy Chapter

This introduces the load balancer pools for OVN networks.
Pools are used to group instances to which a load balancer forwards traffic.

This includes the following new endpoints (see {ref}rest-api for details):

• GET /1.0/networks/{networkName}/load-balancer-pools
• POST /1.0/networks/{networkName}/load-balancer-pools
• GET /1.0/networks/{networkName}/load-balancer-pools/{poolName}
• PUT /1.0/networks/{networkName}/load-balancer-pools/{poolName}
• DELETE /1.0/networks/{networkName}/load-balancer-pools/{poolName}

(extension-clustering-evacuation-force)=

`clustering_evacuation_force` Copy Chapter

Adds a force field to POST /1.0/cluster/members/{name}/state when performing an evacuate action.

When force is set to true, LXD skips the quorum safety check that otherwise would not allow you to evacuate an online raft voter when the remaining online voters would fall below the required majority.

(extension-gpu-mig-cdi)=

`gpu_mig_cdi` Copy Chapter

Updates gputype=mig GPU devices to use the Container Device Interface (CDI) for device passthrough instead of the legacy nvidia.runtime approach.

The following changes are included:

• mig.uuid: The UUID is now resolved to a CDI identifier (nvidia.com/mig=) internally. The MIG- prefix remains optional. Because a MIG UUID uniquely identifies the device, no parent GPU selector (pci, vendorid, productid, or id) is required alongside mig.uuid. If a parent GPU selector is provided, it is used to validate that the matching GPU exists and is an NVIDIA card, preserving backward compatibility.
• mig.gi and mig.ci: The GPU instance ID and compute instance ID pair is now resolved to a MIG UUID via NVML, which is then used as a CDI identifier.
• id: Now accepts either a CDI identifier for gputype=mig devices or a DRM card ID selector for the parent GPU. Supported CDI formats are nvidia.com/mig= and nvidia.com/mig=<dev_idx>:<mig_idx>. When a CDI identifier is provided, it is used directly and is mutually exclusive with mig.uuid, mig.gi, and mig.ci.

(extension-cluster-links-unidirectional)=

`cluster_links_unidirectional` Copy Chapter

This extends the {ref}cluster links <exp-cluster-links> API with support for unidirectional cluster links. The local cluster consumes a trust token issued by the remote cluster to establish the link and pin the remote's certificate. The remote cluster creates a dedicated identity for the local cluster and can authenticate its incoming requests, but does not store addresses for the local cluster and cannot initiate outbound requests to it.

(extension-network-load-balancer-pool-health-checks)=

`network_load_balancer_pool_health_checks` Copy Chapter

This introduces health checks for OVN load balancer pools.
New configuration options on the load balancer pool are added to further customize (or disable) the health check:

• {config:option}network-load-balancer-pool-properties:healthcheck
• {config:option}network-load-balancer-pool-properties:healthcheck.interval
• {config:option}network-load-balancer-pool-properties:healthcheck.timeout
• {config:option}network-load-balancer-pool-properties:healthcheck.success_count
• {config:option}network-load-balancer-pool-properties:healthcheck.failure_count

In addition a new endpoint GET /1.0/networks/{networkName}/load-balancer-pools/{poolName}/state is added which returns the health check status for all instances in the pool.

(extension-operation-child-count)=

`operation_child_count` Copy Chapter

Adds a child_count field to the Operation struct, indicating the number of child operations. This allows clients to determine whether an operation has children without requiring recursion=2.

(extension-storage-driver-powerstore-nvme)=

`storage_driver_powerstore_nvme` Copy Chapter

Adds NVMe/TCP and NVMe/FC support to the Dell PowerStore storage driver.
NVMe/TCP is now the default PowerStore mode when powerstore.mode is not set.

(extension-access-management-expiry)=

`access_management_expiry` Copy Chapter

Adds an expires_at field to the Identity struct, so that the expiry of an identity's credential is reported when identities are listed or individually retrieved, and not only for the current identity.

For identities whose authentication method is tls, this is the expiry of the identity's certificate.
For identities whose authentication method is bearer, this is the expiry of the issued token.
For bearer tokens the expiry is recorded when a token is issued and cleared when a token is revoked.
The field is omitted for identities whose credential has no expiry, that have no credential yet (pending identities), or whose token has been revoked.

Note that bearer identities created prior to this extension will have an omitted expires_at field until a new token is issued.

(extension-access-management-identity-effective-groups)=

`access_management_identity_effective_groups` Copy Chapter

Adds recursion=2 support to GET /1.0/auth/identities and a new GET /1.0/auth/identities/{method}/{identifier}/state endpoint.

The state endpoint reports an effective_groups field for the identity.
When identities are listed with recursion=2, the same effective_groups field is returned for each identity.
The effective_groups field reports the deduplicated union of:

• Direct authorization groups from identities_auth_groups
• Authorization groups mapped from the identity's OIDC identity_provider_groups metadata through identity provider group mappings

For non-OIDC identities, effective_groups is equal to direct group membership.

(extension-cluster-links-public)=

`cluster_links_public` Copy Chapter

This extends the {ref}cluster links <exp-cluster-links> API with support for public cluster links. The initiating cluster (Cluster A) connects to the remote cluster (Cluster B) without presenting a client certificate, relying solely on TLS certificate pinning for server authentication. Cluster B has no record of the link, and neither cluster creates an identity for the other. This is useful when Cluster B exposes resources publicly or when anonymous read access to Cluster B is sufficient.

Creating a public cluster link is a two-phase process:

1. Send a POST request to Cluster A with a name and remote_address (no fingerprint) to create a pending link. Cluster A fetches Cluster B's TLS certificate, confirms Cluster B is serving the LXD API, creates the pending link, and returns the certificate's fingerprint for user verification. The certificate is held in volatile.pending_certificate and the verified address in volatile.pending_address. Neither is pinned yet, so the link remains inert (unreachable) until confirmed. Repeating this request for a link that is still pending refreshes it instead of returning a conflict, so a caller can retry an interrupted creation.
2. To confirm the certificate, send a second POST request to Cluster A with the same name and the fingerprint returned by the first request. Cluster A rejects this second request if the fingerprint does not match or if the request sets remote_address. If the fingerprint matches, Cluster A pins the certificate already held in volatile.pending_certificate, and pins the canonical address (with an explicit port) already recorded in volatile.pending_address into volatile.addresses. This ensures that Cluster A always pins the original certificate fetched from Cluster B and the verified address, rather than a certificate or address supplied in the confirm request.

A new ClusterLinkCertificate response type is introduced, containing the certificate fingerprint returned by the first request, and a matching fingerprint field is added to ClusterLinksPost for echoing it back. Both are used only by public cluster links.

Replication requires the remote cluster to authenticate the connection, so {ref}replicators <exp-replicators> and a project's replica.cluster accept only cluster link types that present a client certificate (currently bidirectional and unidirectional). Both reject a public cluster link at configuration time.

(extension-durable-operations)=

`durable_operations` Copy Chapter

Introduces new operation class for durable operations.
Durable operations are restarted on the DQLite raft leader if the member that is running the operation fails to respond to heartbeats.
If the leader was running the operation and goes offline, the operation is restarted on the newly elected leader.

(extension-access-management-bearer-pending)=

`access_management_bearer_pending` Copy Chapter

Adds three identity types, Client token bearer (pending), DevLXD token bearer (pending) and Initial UI token bearer (pending), that represent bearer identities for which no token is currently issued.

A bearer identity is created in the pending state because no token has been issued for it yet. Issuing a token promotes the identity to its active type of Client token bearer, DevLXD token bearer or Initial UI token bearer, and revoking the token demotes it back to the pending type. A token that has expired but has not been revoked leaves the identity active.

(extension-replicator-custom-volumes)=

`replicator_custom_volumes` Copy Chapter

Adds disk_volumes_mode to the instance migration request (POST /1.0/instances/<name> with migration: true)
and to the migration source of POST /1.0/instances, with the values of the snapshot API: root (default) and
all-exclusive.

With all-exclusive the source transfers the custom storage volumes that are attached only to the migrated
instance, with their snapshots, in the same operation, and lists them in the migration index header. A volume is
transferred when no other instance references it on the source, whether it is attached directly or through a
profile. A target in the same mode creates the instance before those volumes exist, checks when the index header
arrives that it lists every volume the target is still waiting for, that every listed volume is referenced by
the instance's effective devices on the target, and that no listed volume is already attached to another
instance there, and refuses the migration before any data is sent otherwise. Each listed
volume is created in the pool of the same name, or refreshed when it already exists, and the instance's disk
devices are restored afterwards. A refresh only sends the snapshots the target is missing, and instance_only
drops the volume snapshots along with the instance's. The default mode keeps the existing behavior, so
lxc copy and lxc move are unchanged. Projects that do not own their custom volumes, live migrations and
moves between cluster members reject the mode.

All-exclusive snapshots also skip volumes attached to more than one instance, so a snapshot of an instance
whose volumes are all shared records the root volume alone. This applies to every caller of the snapshot API,
not only to replicators.

Replicators use all-exclusive, so an instance is replicated together with the custom volumes attached only to
it, including their snapshots. The pre-replication snapshot of the instance captures those volumes at the same
moment as its root disk. Volumes attached to more than one instance or to no instance are not replicated and
must exist on the standby before the instances using them can be replicated. A volume attached through a profile
is replicated when it is exclusive to one instance; create it on the standby and add the device to the standby's
profile before the first run, which then refreshes it.
lxc replicator info lists the volumes that will travel.

(extension-metrics-replicators)=

`metrics_replicators` Copy Chapter

Adds observability for {ref}replicators <exp-replicators>.

The Replicator API type gains two read-only fields, last_success_at and last_success_oldest_snapshot_at, recording when the replicator last completed a run successfully and the creation time of the oldest snapshot that run replicated. The latter is the effective recovery point objective (RPO) of the replicated project.

Four gauges are added to the {ref}metrics API <metrics>: lxd_replicators, lxd_replicator_last_run_status, lxd_replicator_last_success_timestamp and lxd_replicator_last_success_oldest_snapshot_timestamp.

The replicator-run lifecycle event now fires when a run completes rather than when a manual run starts, so it covers scheduled runs as well, and carries the run outcome in its context.
(extension-storage-driver-pure-scsifc)=

`storage_driver_pure_scsifc` Copy Chapter

Adds SCSI/FC support to the Everpure storage driver.
Fibre Channel targets are discovered through the host bus adapter, so pure.target has no effect when pure.mode is set to scsi/fc.

(extension-operation-wait-status-code)=

`operation_wait_status_code` Copy Chapter

The LXD and DevLXD operation wait endpoints (GET /1.0/operations/{id}/wait) now return HTTP status 200 when the wait itself succeeds, even if the operation it waited on failed.
When the server supports this extension, the DevLXD operation also has a new err_code field that carries the HTTP status code associated with the failed operation.

(extension-vm-volatile-maxcpus)=

`vm_volatile_maxcpus` Copy Chapter

Adds a new volatile VM configuration key {config:option}instance-volatile:volatile.cpu.maxcpus that records the vCPU hotplug limit (SMP maxcpus) used when the VM booted. The value is reused on stateful start (stateful resume or live migration target) so that the QEMU SMP topology matches on both ends of a live migration regardless of the CPU count on each host.

(extension-storage-driver-pure-nvmefc)=

`storage_driver_pure_nvmefc` Copy Chapter

Adds NVMe/FC support to the Everpure storage driver.
Like SCSI/FC, Fibre Channel targets are discovered through the host bus adapter, so pure.target has no effect when pure.mode is set to nvme/fc.

(extension-storage-ceph-replicator)=

`storage_ceph_replicator` Copy Chapter

This introduces the ceph.replicator. configuration key on ceph storage pools.
It records the peer Ceph site to which a project's volumes on the pool are mirrored.
If a project with a replicator holds volumes on a Ceph pool configured with this key, then the replicator enrolls those volumes in RBD mirroring, triggers a mirror snapshot of each volume on every replication run, and completes the run only after the peer site confirms that it has received every snapshot.

(extension-project-replica-mode-optional)=

`project_replica_mode_optional` Copy Chapter

The replica_mode field is now omitted from project responses when the project takes no part in replication, instead of being returned as an empty string.
The field is still returned for projects in leader or standby mode.

(extension-image-registries)=

`image_registries` Copy Chapter

This introduces the image registries API.

An image registry is a cluster-wide entity that points to a source of images, such as a Simple Streams server or another LXD server.
Because image registries are managed on the server, all clients of a LXD server (and all members of a cluster) share the same set of image sources.
For more information, see {ref}howto-image-registries and {ref}ref-image-registries.