# Technical Documentation: canonical/lxd > ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/canonical/lxd) ### Overview (`README.md`) # LXD LXD is a modern, secure and powerful system container and virtual machine manager. It provides a unified experience for running and managing full Linux systems inside containers or virtual machines. LXD supports images for a large number of Linux distributions (official Ubuntu images and images provided by the community) and is built around a very powerful, yet pretty simple, REST API. LXD scales from one instance on a single machine to a cluster in a full data center rack, making it suitable for running workloads both for development and in production. LXD allows you to easily set up a system that feels like a small private cloud. You can run any type of workload in an efficient way while keeping your resources optimized. You should consider using LXD if you want to containerize different environments or run virtual machines, or in general run and manage your infrastructure in a cost-effective way. ## Get started See [Getting started](https://canonical.com/lxd/docs/latest/getting_started/) in the LXD documentation for installation instructions and first steps. - Release announcements: [`https://discourse.ubuntu.com/tags/c/project/lxd/news/143/release`](https://discourse.ubuntu.com/tags/c/project/lxd/news/143/release) - Release tarballs: [`https://github.com/canonical/lxd/releases/`](https://github.com/canonical/lxd/releases/) - Documentation: [`https://canonical.com/lxd/docs/latest/`](https://canonical.com/lxd/docs/latest/) ## Status Type | Service | Status --- | --- | --- Tests | GitHub | [](https://github.com/canonical/lxd/actions/workflows/tests.yml) Go documentation | Godoc | [](https://godoc.org/github.com/canonical/lxd/client) Static analysis | GoReport | [](https://goreportcard.com/report/github.com/canonical/lxd) ## Installing LXD from packages The LXD daemon only works on Linux but the client tool (`lxc`) is available on most platforms. OS | Format | Command --- | --- | --- Linux | [Snap](https://snapcraft.io/lxd) | `snap install lxd` Windows | [Chocolatey](https://chocolatey.org/packages/lxc) | `choco install lxc` macOS | [Homebrew](https://formulae.brew.sh/formula/lxc) | `brew install lxc` The LXD snap packaging files are available in the [snap](snap) directory. For more instructions on installing LXD for a wide variety of Linux distributions and operating systems, and to install LXD from source, see [How to install LXD](https://canonical.com/lxd/docs/latest/installing/) in the documentation. ## Client SDK packages The LXD project provides SDK client packages for interacting with LXD servers from your own software. These SDKs are licensed as Apache-2.0. Language | URL --- | --- Go | https://pkg.go.dev/github.com/canonical/lxd/client Python | https://github.com/canonical/pylxd For more information on using the LXD API, see [REST API](https://canonical.com/lxd/docs/latest/restapi_landing/) in the documentation. ## Tools for managing LXD If you are looking for tools (beyond `lxc` CLI) to manage LXD at scale (from single server to wide clusters), the following projects can be useful: Tool | Link --- | --- Ansible - connection plugin | https://galaxy.ansible.com/ui/repo/published/community/general/content/connection/lxd/ Ansible - inventory plugin | https://galaxy.ansible.com/ui/repo/published/community/general/content/inventory/lxd/ Bolt - LXD transport | https://www.puppet.com/docs/bolt/latest/bolt_transports_reference.html#lxd MicroCloud | https://canonical.com/microcloud Packer - LXD builder | https://developer.hashicorp.com/packer/integrations/hashicorp/lxd/latest/components/builder/lxd Terraform provider | https://registry.terraform.io/providers/terraform-lxd/lxd ## Security Consider the following aspects to ensure that your LXD installation is secure: - Keep your operating system up-to-date and install all available security patches. - Use only supported LXD versions (LTS releases or the latest feature release). - Restrict access to the LXD daemon and the remote API. - Configure your network interfaces to be secure. - Do not use privileged containers unless required. If you use privileged containers, put appropriate security measures in place. See [Container security](https://canonical.com/lxd/docs/latest/explanation/security/#container-security) for more information. See [Security](https://canonical.com/lxd/docs/latest/explanation/security/) for detailed information. **IMPORTANT:** Local access to LXD through the Unix socket always grants full access to LXD. This includes the ability to attach file system paths or devices to any instance as well as tweak the security features on any instance. Therefore, you should only give such access to users who you'd trust with root access to your system. ## Community support You can seek support from the LXD developers as well as the wider community through the following channels. ### Forum Ask questions or engage in discussions: [`https://discourse.ubuntu.com/c/project/lxd/`](https://discourse.ubuntu.com/c/project/lxd/126) ### Documentation Access the official documentation: [`https://canonical.com/lxd/docs/latest/`](https://canonical.com/lxd/docs/latest/) ### Bug reports and feature requests To file a new bug or feature request, [submit an issue on GitHub](https://github.com/canonical/lxd/issues/new). ### Other community resources You can find additional resources on the [LXD website](https://canonical.com/lxd), on [YouTube](https://www.youtube.com/channel/UCuP6xPt0WTeZu32CkQPpbvA), and the [community-created tutorials](https://discourse.ubuntu.com/c/project/lxd/tutorials/146). ## Commercial support LTS releases of LXD receive standard support for five years, which means they receive continuous updates. Commercial support for LXD is provided as part of [Ubuntu Pro](https://ubuntu.com/pro) (both Infra-only and full Ubuntu Pro), including for [attached LXD instances running Ubuntu](https://canonical.com/lxd/docs/latest/howto/instances_ubuntu_pro_attach/). See the [full service description](https://ubuntu.com/legal/ubuntu-pro-description) for details. Managed solutions and firefighting support are also available for LXD deployments. See: [Managed services](https://ubuntu.com/managed). ## Contributing Fixes and new features are greatly appreciated. Make sure to read our [contributing guidelines](CONTRIBUTING.md) first! --- ### Overview (`doc/README.md`) # LXD documentation The LXD documentation is available at: GitHub provides a basic rendering of the documentation as well, but important features like includes and clickable links are missing. Therefore, we recommend reading the [published documentation](https://canonical.com/lxd/docs/latest/). ## How it works ### Documentation framework LXD's documentation is built with [Sphinx](https://www.sphinx-doc.org) and hosted on [Read the Docs](https://about.readthedocs.com/). It is written in [Markdown](https://commonmark.org/) with [MyST](https://myst-parser.readthedocs.io/) extensions. For syntax help and guidelines, see the [MyST syntax guide](https://documentation.ubuntu.com/sphinx-stack/latest/reference/myst-syntax/) in the [Sphinx Stack documentation](https://documentation.ubuntu.com/sphinx-stack/latest/). The documentation structure follows the [Diátaxis](https://diataxis.fr/) framework. ### Build the documentation To build the documentation, run `make doc` from the root directory of the repository. This command installs the required tools and renders the output to the `doc/_build/` directory. To update the documentation for changed files only (without re-installing the tools), run `make doc-incremental`. Before opening a pull request, make sure that the documentation builds without any warnings (warnings are treated as errors). To preview the documentation locally, run `make doc-serve` and go to [`http://localhost:8000`](http://localhost:8000) to view the rendered documentation. --- ### Overview (`lxd-qemu-snap/README.md`) # External QEMU snap for LXD snap The idea behind this feature is to allow LXD users to build their own snap containing QEMU and then plug it into the LXD snap, so that LXD uses this custom QEMU instead of the built-in one. To make it easier to test this feature and simplify the user experience, we provide a reference snapcraft.yaml file. It is intended solely as a starting point for building and customizing your own QEMU snap. Please note that we do not provide regular or security updates for this reference example, as it is meant for documentation and educational purposes only. ## How to build and use (example) ```sh cd lxd-qemu-snap # clean up previous builds rm -f qemu-for-lxd_*.snap # build snapcraft pack # install sudo snap install qemu-for-lxd_*.snap --devmode # connect snaps sudo snap connect lxd:gpu-2604 mesa-2604:gpu-2604 sudo snap connect lxd:qemu-external qemu-for-lxd:qemu-external ``` ## How to use with virgl (only specific to this example of snapcraft.yaml) As part of this external QEMU snap example, we build QEMU with virglrenderer library support. Of course, this is not required, and in your specific use case you may choose to omit it. The following instructions show how to verify that the external QEMU snap with virgl support is working correctly. ```sh lxc init images:ubuntu/noble/desktop desktop -c limits.memory=8GiB --vm # modify instance configuration: lxc config edit desktop # choose an appropriate renderer: ls -la /dev/dri/by-path/*-render # for example, on my system it is /dev/dri/by-path/pci-0000:67:00.0-render # lspci | grep -E "(3D|VGA)" shows: # 67:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Rembrandt (rev d8) # add the following lines: raw.apparmor: |- /snap/lxd/*/gpu-2604/** mr, /dev/dri/ r, /dev/dri/card[0-9]* rw, /dev/dri/renderD[0-9]* rw, /run/udev/data/c226:[0-9]* r, # 226 drm /sys/devices/** r, /sys/bus/** r, raw.qemu: -display egl-headless,rendernode=/dev/dri/by-path/pci-0000:67:00.0-render raw.qemu.conf: |- [device "qemu_gpu"] driver = "virtio-vga-gl" # try it lxc start desktop --console=vga # you can check output from: dmesg | grep -i drm # if you see: # [drm] features: +virgl ... # it's a good sign glxinfo | grep -i vir # it should show something like: # Device: virgl ... ``` ## References --- ### Overview (`snap/README.md`) # LXD LXD is a modern, secure and powerful system container and virtual machine manager. This directory contains the snap packaging used to build the [LXD snap](https://snapcraft.io/lxd). Additional snap resources are located in the [local](local) directory. ## Build the LXD snap locally Local builds require the LXD snap to be installed as `snapcraft` creates a container to use as build environment. Here's how to do a local build for the native architecture: ```sh snapcraft pack ``` ## Build the LXD snap on Launchpad To build the snap for multiple architectures, Launchpad builders can be used. They are available for various architectures (`amd64`, `armhf`, `arm64`, `ppc64el`, `riscv64` and `s390x`) and you can ask for multiple to be built in parallel. Here's how to build for both `amd64` and `arm64`: ```sh snapcraft remote-build --launchpad-accept-public-upload --build-for amd64,arm64 ``` ## GitHub Actions ### `lp-snap-build` action The `.github/actions/lp-snap-build` composite action sets up SSH access to Launchpad and clones the snap packaging repository in preparation for pushing a build trigger commit. It also installs the `lxd-snapcraft` helper tool, which can read and update `version` and `source-commit` fields in `snapcraft.yaml`. #### Inputs | Input | Required | Description | | :--- | :--- | :--- | | `ssh-key` | yes | Private ed25519 SSH key for Launchpad access and commit signing | #### Required environment variables | Variable | Description | | :--- | :--- | | `SSH_AUTH_SOCK` | Path to the SSH agent socket | | `REPO` | URL of the Launchpad snap packaging repository | | `BRANCH` | Branch to clone from the Launchpad repository | | `PACKAGE` | Name of the snap package (used for the clone directory `~/${PACKAGE}-pkg-snap-lp`) | ### Usage from another repo ``` /* Detailed source-code truncated for AI context efficiency. */ ``` --- ### Overview (`test/README.md`) # LXD tests For a description of where and how LXD is tested, please refer to the [overview](./overview.md) document. ## How to run To run all tests, including the Go tests, run from repository root: ```sh sudo -E make check ``` To run only the integration tests, run from the test directory: ```sh sudo -E ./main.sh ``` ## Snap integration tests Snap integration test scripts are in `test/snap/` and can be run directly from the repository root: ```sh sudo -E ./test/snap/cgroup latest/edge ``` The snap channel is optional and defaults to `latest/edge`. The runner installs the selected store snap by default. To remove the installed LXD snap before a test run, which is useful for repeated local testing, set `PURGE_LXD`: ```sh sudo -E env PURGE_LXD=1 ./test/snap/interception latest/edge ``` To reuse an installed LXD snap without installing or refreshing it, set `KEEP_LXD`: ```sh sudo -E env KEEP_LXD=1 ./test/snap/interception latest/edge ``` ### Test a source build To test binaries built from the current LXD source tree, first run `make`, then enable sideloading: ```sh make sudo -E env LXD_SNAP_SIDELOAD=1 ./test/snap/interception latest/edge ``` Set `LXD_SNAP_BINARY_DIR` to use binaries from a directory other than `$(go env GOPATH)/bin`: ```sh sudo -E env LXD_SNAP_SIDELOAD=1 LXD_SNAP_BINARY_DIR=/path/to/bin ./test/snap/interception latest/edge ``` To sideload individual binaries while testing an installed store snap, set the corresponding path: ```sh sudo -E env LXD_SIDELOAD_PATH=/path/to/lxd ./test/snap/interception latest/edge sudo -E env LXC_SIDELOAD_PATH=/path/to/lxc ./test/snap/interception latest/edge sudo -E env LXD_AGENT_SIDELOAD_PATH=/path/to/lxd-agent ./test/snap/vm latest/edge ``` ### Test a local snap To test a locally built snap, set `LXD_SNAP_PATH`: ```sh sudo -E env LXD_SNAP_PATH=/path/to/lxd_0+git_amd64.snap ./test/snap/interception latest/edge ``` To use the system's ZFS tools instead of the tools bundled in the LXD snap, set `LXD_ZFS_EXTERNAL`: ```sh sudo -E env LXD_ZFS_EXTERNAL=1 ./test/snap/interception latest/edge ``` ### Test OVN The `network-ovn` suite can use the host's OVN packages or a chosen MicroOVN snap channel: ```sh sudo -E env OVN_SOURCE=deb PURGE_LXD=1 ./test/snap/network-ovn latest/edge sudo -E env OVN_SOURCE=22.03/edge PURGE_LXD=1 ./test/snap/network-ovn latest/edge sudo -E env OVN_SOURCE=24.03/edge PURGE_LXD=1 ./test/snap/network-ovn latest/edge sudo -E env PURGE_LXD=1 ./test/snap/network-ovn latest/edge ``` The test host must have snapd installed and the command must run as root. ## Environment variables Name | Default | Description :-- | :--- | :---------- `LXD_BACKEND` | dir | What backend to test against (btrfs, ceph, dir, lvm, zfs, or random) `LXD_BACKENDS` | `LXD_BACKEND` | Space-delimited list of backends to run test against. Accepts `fast` (`btrfs` or `dir`), `fasts` (`btrfs` and `dir`) and `all` (see the list in `LXD_BACKEND`) `LXD_CEPH_CLUSTER` | ceph | The name of the ceph cluster to create osd pools in `LXD_CEPH_CEPHFS` | cephfs | Enables the CephFS tests using the specified cephfs filesystem for `cephfs` pools `LXD_CEPH_CEPHOBJECT_RADOSGW` | "" | Enables the Ceph Object tests using the specified radosgw HTTP endpoint for `cephobject` pools `LXD_VERBOSE` | "" | Run lxd, lxc and the shell in verbose mode (used in CI; less verbose than `LXD_DEBUG`) `LXD_DEBUG` | "" | Run lxd, lxc and the shell in debug mode (very verbose) `LXD_INSPECT` | 0 | Set to 1 to start an inspection shell in the test environment on failure `LXD_LOGS` | "" | Path to a directory to copy all the LXD logs to `LXD_REPEAT_TESTS` | 1 | Number of times to repeat test(s) `LXD_RANDOMIZE_TESTS` | 0 | Randomize the order of tests in a group `LXD_SKIP_TESTS` | "" | Space-delimited list of test names to skip `LXD_TEST_IMAGE` | "" (busybox test image) | Path to an image tarball to use instead of the default busybox image `LXD_TEST_LIVE_MIGRATION_ON_THE_SAME_HOST` | "" | When set to true, LXD ignores EBUSY errors during same-host live migration when unmounting a source volume that is already mounted on the destination instance. `LXD_TMPFS` | 0 | Sets up a tmpfs for the whole testsuite to run on (fast but needs memory) `LXD_NIC_SRIOV_PARENT` | "" | Enables SR-IOV NIC tests using the specified parent device `LXD_IB_PHYSICAL_PARENT` | "" | Enables Infiniband physical tests using the specified parent device `LXD_IB_SRIOV_PARENT` | "" | Enables Infiniband SR-IOV tests using the specified parent device `LXD_NIC_BRIDGED_DRIVER` | "" | Specifies bridged NIC driver for tests (either native or openvswitch, defaults to native) `LXD_REQUIRED_TESTS` | "" | Space-delimited list of test names that must not be skipped if their prerequisites are not met `LXD_VM_TESTS` | 1 | Enables tests using VMs and the on-demand installation of the needed tools ## Recommendations ### `sub_test` usage Use `sub_test` to label meaningful phases within a test and make logs easier to scan. Prefer a small number of focused sub-tests over excessive nesting. Use `sub_test` before a logical group of commands that verifies a specific expected behavior for a bug fix or feature. Comments within the sub-test block are appropriate to explain why specific commands are used, any setup or initial configuration, and other intent that isn't obvious from the commands. Good: ```sh sub_test "Verify intended behavior X" ... sub_test "Verify intended behavior Y" ... ``` ### `echo` context Prefer `sub_test` labels and concise comments for context instead of adding `echo` statements. Use `echo` only when you need to debug flaky behavior. ### Expected failure If a command is expected to fail, special care needs to be used in testing. Bad: ```sh set -e ... ! cmd_should_fail some_other_command ``` Good: ```sh set -e ! cmd_should_fail || false some_other_command ``` Best: ```sh set -e if cmd_should_fail; then echo "ERROR: cmd_should_fail unexpectedly succeeded, aborting" >&2 exit 1 fi some_other_command ``` In the "bad" example, if the command unexpectedly succeeds, the script won't abort because `bash` ignores `set -e` for compounded commands (`! cmd_should_fail`). The "good" example works around the problem of compound commands by falling back to executing `false` in case of unexpected success of the command. The "best" example also works around the problem of compound commands but in a very intuitive and readable form, albeit longer. ````{note} This odd behavior of `set -e` with compound commands does not apply inside `[]`. ```sh set -e # Does the right thing of failing if the file unexpectedly exist [ ! -e "should/not/exist" ] ``` However, note that in the above example, if the `!` is moved outside of the `[]`, it would also warrant a ` || false` fallback. ```` For error message assertions, prefer single-quoted strings so error text with `"` does not require escaping and the comparisons stay readable. ### `jq` usage Always use `jq --exit-status` (or the shorthand `-e`) when checking for the presence of fields or specific values. This makes test failures explicit: Bad: ```sh jq '.field' output.json > /dev/null # Silent fail if field missing ``` Good: ```sh jq -e '.field' output.json # Explicit exit code if field missing jq -e '.field == "expected"' output.json ``` ### Avoid `grep -c` for presence or absence checks Avoid using `grep -c` to test for presence or absence — it makes tests brittle and harder to read. `grep -c` for actual counting (where the count matters) is fine. Bad: ```sh # Using -c when you only care about presence, not count [ "$(lxc list | grep -c c1)" = "1" ] ``` Good: ```sh # Just check presence lxc list | grep -wF c1 ``` Best: ```sh # Filter output directly and compare exactly [ "$(lxc list -f csv -c n)" = "c1" ] ``` OK (counting genuinely matters): ```sh [ "$(grep -c "pattern" logfile)" = "3" ] ``` --- ### Overview (`lxd/lxd-metadata/README.md`) # lxd-metadata A small CLI to parse comments in a Golang codebase meant to be used for a documentation tool (like Sphinx for example). It parses the comments from the AST and extracts their documentation. # Usage $ lxd-metadata -h Usage of lxd-metadata: -e value Path that will be excluded from the process # Formatting A comment is formatted this way: ``` /* Detailed source-code truncated for AI context efficiency. */ ``` The go-swagger spec from source generator can only handles `swagger:meta` (global file/package level documentation), `swagger:route` (API endpoints), `swagger:params` (function parameters), `swagger:operation` (method documentation), `swagger:response` (API response content documentation), `swagger:model` (struct documentation) generation. In our use case, we would want a config variable spec generator that can bundle any key-value data pairs alongside metadata to build a sense of hierarchy and identity (we want to associate a unique key to each lxddoc comment group that will also be displayed in the generated documentation) In a swagger fashion, `lxd-metadata` can associate metadata key-value pairs (here for example, `group` and `key`) to data key-value pairs. As a result, it can generate a YAML tree out of the code documentation and also a Markdown document. ## Output Here is the JSON output of the example shown above: ``` /* Detailed source-code truncated for AI context efficiency. */ ``` Here is the `.txt` output of the example shown above: ``` \`\`\`{config:option} user.* cluster :type: string :liveupdate: `yes` :shortdesc: Free form user key/value storage (can be used in search). :condition: container :default: - This is the real long desc. With two paragraphs. And a list: - Item - Item - Item example of a table: Key | Type | Scope | Default | Description :-- | :--- | :---- | :------ | :---------- `acme.agree_tos` | bool | global | `false` | Agree to ACME terms of service `acme.ca_url` | string | global | `https://acme-v02.api.letsencrypt.org/directory` | URL to the directory resource of the ACME service `acme.domain` | string | global | - | Domain for which the certificate is issued `acme.email` | string | global | - | Email address used for the account registration \`\`\` \`\`\`{config:option} scheduler.instance cluster :liveupdate: `yes` :shortdesc: Possible values are all, manual and group. See Automatic placement of instances for more information. :condition: container :default: `all` :type: integer \`\`\` ``` --- ### Overview (`shared/usbid/README.md`) This Go package is a variant of `usbid` from https://github.com/google/gousb. It was written by the gousb maintainers and contributors and then adapted for use by LXD. Main changes: - Doesn't load on import (reduced memory footprint). - Uses system-local USB database (to always match `lsusb`). - Doesn't import anything outside of built-in Go packages. - Doesn't use or indirectly rely on CGO as this code is often used in cross-built static binaries. Most users will want to stick to the upstream google/gousb version instead. This fork is really meant for LXD's special use case and may be further trimmed down in the future. --- ### Overview (`test/mini-oidc/README.md`) `mini-oidc` is an extremely basic OIDC provider which can be used with the `lxc` command line. It doesn't use web authentication and instead just automatically approves any authentication request. By default, it will authenticate everyone as `unknown`, but this can be overridden by writing the username to be returned in the `user.data` file. This effectively allows scripting a variety of users without having to deal with actual login. The `storage` sub-package is a copy of https://github.com/zitadel/oidc/tree/main/example/server/storage with the exception of the added LXDDeviceClient. --- ### Overview (`.github/workflows/testflinger/README.md`) # Testflinger scripts This directory contains the scripts used for GPU testing (NVIDIA and AMD) via GitHub Actions and Testflinger. The tests run on devices within Canonical's test farm. The NVIDIA and NVIDIA MIG templates clone the LXD revision that triggered the workflow and run the relevant script under `test/snap/` directly. The Ubuntu Core and AMD CDI templates test the selected store snap directly. ## Run locally Running the tests locally is only possible if your machine has access to the Testflinger server. Tested distros: - `core24-latest` - `jammy` - `noble` Current queue/workflow combinations: - NVIDIA (CDI and legacy runtime): `JOB_QUEUE=lxd-nvidia`, `TFWORKFLOW=nvidia-gpu-job` - NVIDIA MIG (Multi-Instance GPU): `JOB_QUEUE=lxd-mig`, `TFWORKFLOW=nvidia-mig-job` - Ubuntu Core + NVIDIA CDI: `JOB_QUEUE=lxd-nvidia`, `TFWORKFLOW=uc-nvidia-cdi-job` - AMD CDI: `JOB_QUEUE=lxd-amd`, `TFWORKFLOW=amd-cdi-job` Ensure `testflinger` is installed: ``` sudo snap install testflinger-cli ``` Set the input variables and execute the script from the same directory: ```bash JOB_QUEUE=lxd-nvidia SNAP_CHANNEL=latest/edge DISTRO=core24-latest TFWORKFLOW=uc-nvidia-cdi-job ./run.sh ``` The above replaces the inputs in the scripts and submits the Testflinger job. To prepare the scripts only but not submit the job, set the `--dryrun` flag. ## Examples Notice, that some Testflinger workflows are only compatible with `core24-latest`, while others only with classic Ubuntu images. To test Ubuntu Core + LXD + GPU passthrough in CDI mode: ```bash JOB_QUEUE=lxd-nvidia SNAP_CHANNEL=latest/edge DISTRO=core24-latest TFWORKFLOW=uc-nvidia-cdi-job ./run.sh ``` To test Ubuntu Noble + LXD + GPU passthrough (CDI and legacy nvidia runtime): ```bash JOB_QUEUE=lxd-nvidia SNAP_CHANNEL=latest/edge DISTRO=noble TFWORKFLOW=nvidia-gpu-job ./run.sh ``` To test Ubuntu Noble + LXD + AMD GPU passthrough in CDI mode: ```bash JOB_QUEUE=lxd-amd SNAP_CHANNEL=latest/edge DISTRO=noble TFWORKFLOW=amd-cdi-job ./run.sh ``` To test Ubuntu Noble + LXD + NVIDIA MIG (Multi-Instance GPU) passthrough: ```bash JOB_QUEUE=lxd-mig SNAP_CHANNEL=latest/edge DISTRO=noble TFWORKFLOW=nvidia-mig-job ./run.sh ``` --- ### Overview (`lxd/db/dbgen/README.md`) # `dbgen` `dbgen` inspects Go struct definitions for expected comments and tags. If it finds any, it generates functions for the type such that it implements the interfaces defined in [`generic.go`](../query/generic.go). The type can then be used with the generic functions therein. ## Examples ### Simple table For a simple table, just add a `db:model ` comment above the type and add a tag for each field representing the column name. ```sqlite CREATE TABLE auth_groups ( id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, name TEXT NOT NULL, description TEXT NOT NULL, UNIQUE (name) ); ``` ```go package main // AuthGroupsRow represents a single row of the auth_groups table. // db:model auth_groups type AuthGroupsRow struct { ID int64 `db:"id"` Name string `db:"name"` Description string `db:"description"` } ``` In this example `dbgen` assumes that the `id` column is the primary key. ### Composite primary keys Some tables don't have an auto-incrementing primary key. These are defined `WITHOUT ROWID`. For these tables we can define the primary key as some unique column combination. To specify the primary keys, add a `// db:primary` comment above those fields. ```sqlite CREATE TABLE networks_load_balancer_pools_instances ( network_load_balancer_pool_id INTEGER NOT NULL, instance_id INTEGER NOT NULL, target_port INTEGER NOT NULL, PRIMARY KEY (network_load_balancer_pool_id, instance_id), FOREIGN KEY (network_load_balancer_pool_id) REFERENCES networks_load_balancer_pools (id) ON DELETE CASCADE, FOREIGN KEY (instance_id) REFERENCES instances (id) ON DELETE CASCADE ) WITHOUT ROWID; ``` ```go package main // NetworksLoadBalancerPoolsInstancesRow represents a row of the networks_load_balancer_pools_instances table. // db:model networks_load_balancer_pools_instances type NetworksLoadBalancerPoolsInstancesRow struct { // db:primary NetworkLoadBalancerPoolID int64 `db:"network_load_balancer_pool_id"` // db:primary InstanceID int64 `db:"instance_id"` TargetPort int64 `db:"target_port"` } ``` ### Joins If a Go type is modelled via `dbgen` to represent a single row of a single table, then the interface implementations that `dbgen` generates can be used to perform CRUD operations on that table. Structs like this can be referenced from other structs to perform more complex joins, but these secondary structs are read-only. To reference another struct, add it to the wrapper struct as the first field, with the field name `Row`. To perform a join, add a `// db:join` comment above the field that requires a join. ```sqlite CREATE TABLE "projects" ( id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, name TEXT NOT NULL, description TEXT NOT NULL, UNIQUE (name) ); CREATE TABLE placement_groups ( id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, name TEXT NOT NULL, description TEXT NOT NULL, project_id INTEGER NOT NULL, FOREIGN KEY (project_id) REFERENCES projects (id) ON DELETE CASCADE, UNIQUE (project_id, name) ); ``` ```go package main // PlacementGroupsRow represents a single row of the placement_groups table. // db:model placement_groups type PlacementGroupsRow struct { ID int64 `db:"id"` Name string `db:"name"` Description string `db:"description"` ProjectID int64 `db:"project_id"` } // PlacementGroup contains [PlacementGroupsRow] with additional joins. // db:model placement_groups type PlacementGroup struct { Row PlacementGroupsRow // db:join JOIN projects ON placement_groups.project_id = projects.id ProjectName string `db:"projects.name"` } ``` --- ### Overview (`lxd/db/generate/README.md`) # `lxd-generate` ## Introduction `lxd-generate` is a database statement and associated `go` function generator for LXD and related projects. `lxd-generate` utilizes `go`'s code generation directives (`//go:generate ...`) alongside go's [ast](https://pkg.go.dev/go/ast) package for parsing the syntax tree for go structs and variables. We use `lxd-generate` for the majority of our SQL statements and database interactions on the `go` side for consistency and predictability. ## Usage ### Initialization Generally the first thing we will want to do for any newly generated file is to establish the command, the target file, and ensure the file has been cleared of content: ```go //go:generate -command mapper lxd-generate db mapper -t instances.mapper.go //go:generate mapper reset -i -b "//go:build linux && cgo && !agent" ``` This will initiate a call to `lxd-generate db mapper -t instances.mapper.go` on each call to the go generation directive `mapper`. ### Generation Directive Arguments * `//go:generate mapper stmt -e instance objects table=table_name` The `table` key can be used to override the generated table name for a specified one. * `//go:generate mapper method -e instance Create references=Config,Device` For some tables (defined below under [Additional Information](#additional-information) as [EntityTable](#entitytable), the `references=` key can be provided with the name of a [ReferenceTable](#referencetable) or [MapTable](#maptable) struct. This directive would produce `CreateInstance` in addition to `CreateInstanceConfig` and `CreateInstanceDevices` * //go:generate mapper method -i -e instance_profile Create struct=Instance * //go:generate mapper method -i -e instance_profile Create struct=Profile For some tables (defined below under [Additional Information](#additional-information) as [AssociationTable](#associationtable), `method` declarations must include a `struct=` to indicate the directionality of the function. An invocation can be called for each direction. This would produce `CreateInstanceProfiles` and `CreateProfileInstances` respectively. ### SQL Statement Generation SQL generation supports the following SQL statement types: Type | Description :--- | :---- `objects` | Creates a basic SELECT statement of the form `SELECT FROM
ORDER BY `. `objects-by--and-...` | Parses a pre-existing SELECT statement variable declaration of the form produced by`objects`, and appends a `WHERE` clause with the given fields located in the associated struct. Specifically looks for a variable declaration of the form 'var Objects = RegisterStmt(`SQL String`)' `create` | Creates a basic INSERT statement of the form `INSERT INTO
VALUES`. `create-or-replace` | Creates a basic INSERT statement of the form `INSERT OR REPLACE INTO
VALUES`. `delete-by--and-...` | Creates a DELETE statement of the form `DELETE FROM
WHERE ` where the constraint is based on the given fields of the associated struct. `id` | Creates a basic SELECT statement that returns just the internal ID of the table. `rename` | Creates an UPDATE statement that updates the primary key of a table: `UPDATE
SET WHERE `. `update` | Creates an UPDATE statement of the form `UPDATE
SET WHERE `. #### Examples ```go //go:generate mapper stmt -e instance objects //go:generate mapper stmt -e instance objects-by-Name-and-Project //go:generate mapper stmt -e instance create //go:generate mapper stmt -e instance update //go:generate mapper stmt -e instance delete-by-Name-and-Project ``` #### Statement Related Go Tags There are several tags that can be added to fields of a struct that will be parsed by the `ast` package. Tag | Description :-- | :---- `sql=
.` | Supply an explicit table and column name to use for this struct field. `coalesce=` | Generates a SQL coalesce function with the given value `coalesce(, value)`. `order=yes` | Override the default `ORDER BY` columns with all fields specifying this tag. `join=` | Applies a `JOIN` of the form `JOIN ON
. = `. `leftjoin=` | Applies a `LEFT JOIN` of the same form as a `JOIN`. `joinon=
.` | Overrides the default `JOIN ON` clause with the given table and column, replacing `
.` above. `primary=yes` | Assigns column associated with the field to be sufficient for returning a row from the table. Will default to `Name` if unspecified. Fields with this key will be included in the default 'ORDER BY' clause. `omit=` | Omits a given field from consideration for the comma separated list of statement types (`create`, `objects-by-Name`, `update`). `ignore=yes` | Outright ignore the struct field as though it does not exist. `marshal=yes` | Marshal/Unmarshal data into the field. The column must be a TEXT column and the type must implement both `query.Marshal` and `query.Unmarshal`. This works for entity tables only, and not for association or mapping tables. ### Go Function Generation Go function generation supports the following types: Type | Description :--- | :---- `GetMany` | Return a slice of structs for all rows in a table matching the filter. `GetOne` | Return a single struct corresponding to a row with the given primary keys. Depends on `GetMany`. `ID` | Return the ID column from the table corresponding to the given primary keys. `Exists` | Returns whether there is an row in the table with the given primary keys. Depends on `ID.` `Create` | Insert a row from the given struct into the table if not already present. Depends on `Exists` `CreateOrReplace` | Insert a row from the given struct into the table, regardless of if an entry already exists. `Rename` | Update the primary key for a table row. `Update` | Update the columns at a given row, specified by primary key. `DeleteOne` | Delete exactly one row from the table. `DeleteMany` | Delete one or more rows from the table. ```go //go:generate mapper method -i -e instance GetMany //go:generate mapper method -i -e instance GetOne //go:generate mapper method -i -e instance ID //go:generate mapper method -i -e instance Exist //go:generate mapper method -i -e instance Create //go:generate mapper method -i -e instance Update //go:generate mapper method -i -e instance DeleteOne-by-Project-and-Name //go:generate mapper method -i -e instance DeleteMany-by-Name ``` ### Additional Information All structs should have an `ID` field, as well as an additional `Filter` struct prefixed with the original struct name. This should include any fields that should be considered for filtering in `WHERE` clauses. These fields should be pointers to facilitate omission and inclusion without setting default values. Example: ```go type Instance struct { ID int Name string Project string } type InstanceFilter struct { Name *string Project *string } ``` `lxd-generate` will handle parsing of structs differently based on the composition of the struct in four different ways. Non-`EntityType` structs will only support `GetMany`, `Create`, `Update`, and `Delete` functions. ### EntityTable Most structs will get treated this way, and represent a normal table. * If a table has an associated table for which a `ReferenceTable` or `MapTable` as defined below is applicable, functions specific to this entity can be generated by including a comma separated list to `references=` in the code generation directive for `GetMany`, `Create`, or `Update` directives. * The `Create` method directive for `EntityTable` will expect on the `ID` and `Exist` method directives to be present. * All `CREATE`, `UPDATE`, and `DELETE` statements that include a joined table will expect a `var ID = RegisterStmt('SQL String')` to exist for the joining table. ### ReferenceTable A struct that contains a field named `ReferenceID` will be parsed this way. `lxd-generate` will use this struct to generate more abstract SQL statements and functions of the form `_`. Real world invocation of these statements and functions should be done through an `EntityTable` `method` call with the tag `references=`. This `EntityTable` will replace the `` above. Example: ```go //go:generate mapper stmt -e device create //go:generate mapper method -e device Create type Device struct { ID int ReferenceID int Name string Type string } //... //go:generate mapper method -e instance Create references=Device // This will produce a function called `CreateInstanceDevices`. ``` ### MapTable This is a special type of `ReferenceTable` with fields named `Key` and `Value`. On the SQL side, this is treated exactly like a `ReferenceTable`, but on the `go` side, the return values will be a map. Example: ```go //go:generate mapper stmt -e config create //go:generate mapper method -e config Create type Config struct { ID int ReferenceID int Key string Value string } //... //go:generate mapper method -e instance Create references=Config // This will produce a function called `CreateInstanceConfig`, which will return a `map[string]string`. ``` ### AssociationTable This is a special type of table that contains two fields of the form `ID`, where `` corresponds to two other structs present in the same package. This will generate code for compound tables of the form `_` that are generally used to associate two tables together by their IDs. `method` generation declarations for these statements should include a `struct=` to indicate the directionality of the function. An invocation can be called for each direction. Example: ```go //go:generate mapper method -i -e instance_profile Create struct=Instance //go:generate mapper method -i -e instance_profile Create struct=Profile type InstanceProfile struct { InstanceID int ProfileID int } ``` --- ### Overview (`lxd/instance/drivers/libkrun/README.md`) # libkrun (reduced) Minimal Go bindings for libkrun required by LXD. ## Runtime Loading The package resolves libkrun at runtime with `dlopen(3)`/`dlsym(3)`. - If `LIBKRUN_PATH` is set, it is used first. - Otherwise it falls back to `libkrun.so` then `libkrun.so.0`. --- ### Doc Explanation Index (`doc/explanation/index.md`) --- myst: html_meta: description: An index of explanatory guides for LXD, covering instances, images, storage, networking, access management, clustering, production setup, and the CSI driver. --- (explanation)= # Explanation The explanatory guides in this section discuss the concepts used in LXD and help you understand how things fit together. (explanation-concepts)= ## Important concepts LXD's core concepts include its relationship with LXC and the instance types it supports: system containers and virtual machines. ```{toctree} :titlesonly: /explanation/lxd_lxc /explanation/instances ``` (explanation-entities)= ## Entities in LXD LXD uses several distinct entity types, including images, storage pools, networks, and projects. To learn how to use them, refer to the {ref}`howtos`. ```{toctree} :titlesonly: /image-handling /explanation/storage /explanation/networks /database /explanation/lxc_show_info ``` (explanation-iam)= ## Access management LXD supports multiple methods for authenticating remote API clients and provides fine-grained authorization controls. Projects can also be used to scope and restrict access. ```{toctree} :titlesonly: /authentication /explanation/authorization /explanation/projects ``` (explanation-production)= ## Production setup For scalable, reliable, and secure LXD deployments, these guides help you understand the key concepts around clustering, performance tuning, and security. ```{toctree} :titlesonly: /explanation/clusters /explanation/disaster_recovery /explanation/performance_tuning /explanation/security /explanation/bpf ``` (explanation-csi)= ## The LXD CSI driver The LXD CSI driver is an open source implementation of the Container Storage Interface (CSI) that integrates LXD storage backends with Kubernetes. ```{toctree} :titlesonly: /explanation/csi ``` --- ### Doc Howto Index (`doc/howto/index.md`) --- myst: html_meta: description: An index of how-to guides for installing, configuring, and managing LXD, including instances, storage, networks, clustering, and production deployments. --- (howtos)= # How-to guides These how-to guides cover key operations and processes in LXD. ## Set up the LXD server and initial access LXD can be installed and initialized in multiple ways. Afterward, the server can be configured for network access through the CLI or UI client. ```{toctree} :titlesonly: :maxdepth: 1 /getting_started /operation ``` ## Work with LXD Instances are created from images and can be either {ref}`system containers or virtual machines `. Projects are useful for grouping related instances and managing user access. ```{toctree} :titlesonly: :maxdepth: 1 /instances /images /projects /storage /networks ``` ## Get ready for production For production deployments, clusters of LXD servers help support higher loads. The production setup guides also cover performance, monitoring, backup, and disaster recovery. ```{toctree} :titlesonly: :maxdepth: 1 /clustering /production-setup /howto/disaster_recovery ``` ## Perform server administration ```{toctree} :titlesonly: :maxdepth: 1 Manage the snap Harden security /howto/troubleshoot ``` ## Authenticate to the APIs Bearer tokens can be used to authenticate to the LXD API; refer to {ref}`authentication` for other methods. The DevLXD API is used for communication between instances and their host. ```{toctree} :titlesonly: :maxdepth: 1 Authenticate to the LXD API using bearer tokens Authenticate to the DevLXD API ``` ## Decommission LXD ```{toctree} :titlesonly: :maxdepth: 1 Decommission LXD ``` ## Engage with us ```{toctree} :titlesonly: :maxdepth: 1 Get support Contribute to LXD ``` --- ### Doc Reference Index (`doc/reference/index.md`) --- myst: html_meta: description: An index of reference guides for LXD, including configuration settings, REST API documentation, production setup, permissions, and internal details. --- (reference)= # Reference The reference material in this section provides technical descriptions of LXD. (reference-general)= ## General information These guides include compatibility information for operating systems running in instances, and man pages for the `lxc` CLI. ```{toctree} :titlesonly: :maxdepth: 1 /requirements /architectures /guest-os-compatibility Container environment /reference/manpages ``` ## Releases Release notes, deprecation notices, and details about the LXD release cadence and its snap. ```{toctree} :titlesonly: :maxdepth: 1 /reference/release-notes/index /reference/deprecation_notices /reference/releases-snap ``` ## Images Reference information for remote image servers and the LXD image format. ```{toctree} :titlesonly: :maxdepth: 1 /reference/remote_image_servers /reference/image_format ``` (reference-config)= ## Configuration options LXD is highly configurable, with options available for major entities as well as permissions for access control. ```{toctree} :titlesonly: :maxdepth: 1 :includehidden: Configuration option index /server /explanation/instance_config /reference/preseed_yaml_fields /reference/projects /reference/storage_drivers /reference/networks /reference/image_registries /reference/placement_groups /reference/clusters /reference/replicator_config /reference/permissions ``` (reference-production)= ## Production setup The LXD server can be optimized for production workloads and can monitor server metrics. ```{toctree} :titlesonly: :maxdepth: 1 Production server settings /reference/provided_metrics ``` (reference-api)= ## API and integrations LXD exposes a REST API for managing all resources. The LXD CSI driver integrates LXD storage backends with Kubernetes. ```{toctree} :titlesonly: :maxdepth: 1 /restapi_landing /reference/driver_csi ``` (reference-internal)= ## Internal implementation details These guides are primarily of interest to advanced users, contributors, and developers. ```{toctree} :titlesonly: :maxdepth: 1 /internals ``` ```{toctree} :hidden: Project repository Image server ``` --- ### Doc Reference Release Notes Index (`doc/reference/release-notes/index.md`) --- myst: html_meta: description: This page lists recent release notes for LXD, which highlight new features, bug fixes, and other important information for each release. --- (ref-release-notes)= # Release notes This page lists release notes for LXD, which highlight new features, bug fixes, and other important information for each release. ## Release policy and schedule For details about the release policy and schedule, along with information about the LXD snap channels, see: {ref}`ref-releases-snap`. ## Upgrade instructions For full instructions on updating or upgrading LXD via its [snap package](https://snapcraft.io/lxd), see {ref}`howto-snap`: - Feature releases are published on the {ref}`current feature track `. If you are already following this track and you want to manually update to the most recent feature release, see the {ref}`howto-snap-updates-manual` section. - To move from one LTS track to a higher LTS track or the feature track, see the {ref}`howto-snap-change` section. (ref-release-notes-series)= ## Series ```{toctree} :titlesonly: :maxdepth: 1 LXD 6 LTS <6/index> LXD 5.21 LTS <5.21/index> LXD 5.0 LTS <5.0/index> LXD 4.0 LTS <4.0/index> ``` --- ### Doc Reference Release Notes 4.0 Index (`doc/reference/release-notes/4.0/index.md`) --- myst: html_meta: description: This page lists recent release notes for LXD 4.0 LTS. --- (ref-release-notes-4.0)= # Release notes This page lists recent release notes for LXD 4.0 LTS. (ref-release-notes-releases-4.0)= ## Releases ```{toctree} :titlesonly: :maxdepth: 1 LXD 4.0.14 LTS LXD 4.0.13 LTS LXD 4.0.12 LTS LXD 4.0.11 LTS ``` For older release notes, see [our Discourse forum](https://discourse.ubuntu.com/tags/c/lxd/126/release). --- ### Doc Reference Release Notes 5.0 Index (`doc/reference/release-notes/5.0/index.md`) --- myst: html_meta: description: This page lists recent release notes for LXD 5.0 LTS. --- (ref-release-notes-5.0)= # Release notes This page lists recent release notes for LXD 5.0 LTS. (ref-release-notes-releases-5.0)= ## Releases ```{toctree} :titlesonly: :maxdepth: 1 LXD 5.0.10 LTS LXD 5.0.9 LTS LXD 5.0.8 LTS LXD 5.0.7 LTS ``` For older release notes, see [our Discourse forum](https://discourse.ubuntu.com/tags/c/lxd/126/release). --- ### Doc Reference Release Notes 5.21 Index (`doc/reference/release-notes/5.21/index.md`) --- myst: html_meta: description: This page lists recent release notes for LXD 5.21 LTS. --- (ref-release-notes-5.21)= # Release notes This page lists recent release notes for LXD 5.21 LTS. (ref-release-notes-releases-5.21)= ## Releases ```{toctree} :titlesonly: :maxdepth: 1 LXD 5.21.8 LTS LXD 5.21.7 LTS LXD 5.21.6 LTS LXD 5.21.5 LTS ``` For older release notes, see [our Discourse forum](https://discourse.ubuntu.com/tags/c/lxd/126/release). --- ### Doc Reference Release Notes 6 Index (`doc/reference/release-notes/6/index.md`) --- myst: html_meta: description: This page lists recent release notes for LXD 6. --- (ref-release-notes-6)= # Release notes This page lists recent release notes for LXD 6. Releases 6.10._z_ are LTS releases in the 6 series. Earlier releases in the 6 series are feature releases. See {ref}`ref-releases-snap` for more information. (ref-release-notes-releases-6)= ## Releases ```{toctree} :titlesonly: :maxdepth: 1 LXD 6.10.0 LTS LXD 6.9 LXD 6.8 LXD 6.7 LXD 6.6 ``` For older release notes, see [our Discourse forum](https://discourse.ubuntu.com/tags/c/lxd/126/release). --- ### Doc .Custom Wordlist (`doc/.custom_wordlist.txt`) AAAA ABI ACL ACLs ACL's AGPL AIO allocator AMD AppArmor ARMv ARP ASN attacher Auth authentik AXFR backend backends backport backported backporting backticks balancers BARs benchmarking BGP BitLocker bitmask BLK bool bootable bpffs BPFFS BPF BMC Btrfs bugfix bugfixes CA CAs CCLA CDI CentOS Ceph CephFS Ceph's CFS cgroup cgroupfs cgroups checksum checksums Chocolatey CIDR CLIs COPR CPUs CRIU CRD CRDs CRL cron cryptographic cryptographically CSI CSM CSV CUDA DaemonSet dataset dGPU DCO deduplicated dereferenced DevLXD DHCP DHCPv Diffie discoverability DNS DNSSEC DoS Dqlite DRM EB Ebit eBPF ECDHE ECDSA EiB Eibit endian Entra Enablement EOL ES ESA ESM ETag failback failover filesystem firmware frontend FQDNs gapped GARP GbE Gbit Geneve GiB Gibit GID GIDs Golang goroutines GPUs GPU's HAProxy Hellman Homebrew hotplug hotplugged hotplugging HTTPS HWE ICMP idmap idmapped idmaps IdP iGPU iGPUs incrementing intra InfiniBand init inode IOPS IOV IPAM IPs IPv IPVLAN IPVS JIT JWT journaling jq kB kbit Keepalived Keycloak keymaps KiB kibi Kibit Kubelet Kubelets KVM libbpf Libera liveness LogCLI lookups LoongArch loopback LRU LV LVM LXC LXCFS LXC's LXD LXD's macOS macvlan manpages Mbit MDM MiB Mibit MicroCeph MicroCloud MicroOVN MII MITM MMIO MNIST MTU Mullvad multicast multipath namespaced NATed natively NDP netmask NBD nbdcopy nbdinfo NFS NIC NICs NIC's NUMA numpad NVMe NVML NVRAM NVIDIA OAuth OCI OData OIDC onboarding OpenFGA OpenID OpenMetrics OpenSSL OpenSUSE OpenVSwitch OptiPNG Ory OSD overcommitting OverlayFS OVMF OVN OVS pprof Pbit PCI PCIe PDU passthrough passthroughs peerings performant PersistentVolume PersistentVolumes PersistentVolumeClaim PersistentVolumeClaims PFs PiB Pibit PID PKI PNG Pongo POSIX PowerFlex preconfigured prepended preseed provision provisioner provisioning proxied proxying PTS PV PVs PVC PVCs qdisc qdiscs QEMU QFQ QMP qgroup qgroups RADOS radosgw RBAC RBD RDP README reconfiguring requestor resizer RESTful RGW RHEL rollout rootfs RCG RPC RPCs RSA ROCm runtime SATA scalable scriptlet SDC SDK SDN SDNs SDS SDT SDTs SeaBIOS Seccomp SELinux SEV SFTP SHA SIGHUP SIGTERM simplestreams SKBPRIO SLAAC SLES SMTP Snapcraft snapshotter SoC Solaris SPAs SPL SquashFS SQLite SSDs SSL SSO Starlark stateful StatefulSet stderr stdin stdout StorageClass StorageClasses STP struct structs subcommand subcommands subnet subnets subpage subpages submenu subtrees subvolume subvolumes superset SVG swtpm symlink symlinks syscall syscalls sysfs syslog SystemAdmin Tbit TCP TensorRT Tegra TiB Tibit TinyPNG TLS tmpfs toleration tolerations toolchain tooltips topologies TPM TSIG TTL UDP UEFI uefivars UFW UI's UID UIDs unconfigured unevictable unicast unixgram unmanaged unmount unmounting unsets untrusted uplink uplinks uptime URI URIs userspace UUIDs USG vCPU vCPUs vGPU vGPUs VDPA VFS VFs VirtIO virtiofs virtiofsd virtualize virtualized VLAN VLANs VMs VM's VolumeAttachment VolumeAttachments VolumeSnapshot VolumeSnapshotClass VolumeSnapshotContent VPD VPN VRAM Vsock vSwitch vTree VXLAN WebSocket WebSockets XFS XHR YAML's Zettabyte ZFS zpool zpools HPE Alletra WSAPI CPG vLUN vLUNs VRRP Everpure FlashArray FlashArrays --- ### Doc .Readthedocs.Yaml (`doc/.readthedocs.yaml`) # .readthedocs.yaml # Read the Docs configuration file # See https://docs.readthedocs.io/en/stable/config-file/v2.html for details # Required version: 2 # Set the version of Python and other tools you might need build: os: ubuntu-24.04 tools: golang: "1.25" python: "3.12" jobs: post_checkout: - git fetch --unshallow || true # Cancel building pull requests when there are no changes in the doc directory. # If there are no changes (git diff exits with 0), we force the command to return with 183. # This is a special exit code on Read the Docs that will cancel the build immediately. # https://docs.readthedocs.io/en/stable/build-customization.html#cancel-build-based-on-a-condition - | if [ "$READTHEDOCS_VERSION_TYPE" = "external" ] && git diff --quiet origin/main -- 'doc/' '.readthedocs.yaml'; then exit 183; fi pre_build: - go build -ldflags "-s -w" -o trimpath -o lxc.bin ./lxc # Build documentation in the docs/ directory with Sphinx sphinx: builder: dirhtml configuration: doc/conf.py fail_on_warning: true # If using Sphinx, optionally build your docs in additional formats such as PDF formats: - pdf # Optionally declare the Python requirements required to build your docs python: install: - requirements: doc/requirements.txt --- ### Doc Api Extensions (`doc/api-extensions.md`) # API extensions The changes below were introduced to the LXD API after the 1.0 API was finalized. They are all backward compatible and can be detected by client tools by looking at the `api_extensions` field in `GET /1.0`. (extension-storage-zfs-remove-snapshots)= ## `storage_zfs_remove_snapshots` A {config:option}`storage-zfs-volume-conf:zfs.remove_snapshots` daemon configuration key was introduced. It's a Boolean that defaults to `false` and that when set to `true` instructs LXD to remove any needed snapshot when attempting to restore another. This is needed as ZFS will only let you restore the latest snapshot. (extension-container-host-shutdown-timeout)= ## `container_host_shutdown_timeout` A {config:option}`instance-boot:boot.host_shutdown_timeout` container configuration key was introduced. It's an integer which indicates how long LXD should wait for the container to stop before killing it. Its value is only used on clean LXD daemon shutdown. It defaults to 30s. (extension-container-stop-priority)= ## `container_stop_priority` A {config:option}`instance-boot:boot.stop.priority` container configuration key was introduced. It's an integer which indicates the priority of a container during shutdown. Containers will shutdown starting with the highest priority level. Containers with the same priority will shutdown in parallel. It defaults to 0. (extension-container-syscall-filtering)= ## `container_syscall_filtering` A number of new syscalls related container configuration keys were introduced. * {config:option}`instance-security:security.syscalls.deny_default` * {config:option}`instance-security:security.syscalls.deny_compat` * {config:option}`instance-security:security.syscalls.deny` * {config:option}`instance-security:security.syscalls.allow` See [Instance configuration](instance-config) for how to use them. ```{note} Initially, those configuration keys were (accidentally) introduced with offensive names. They have since been renamed (`container_syscall_filtering_allow_deny_syntax`), and the old names are no longer accepted. ``` (extension-auth-pki)= ## `auth_pki` This indicates support for PKI authentication mode. In this mode, the client and server both must use certificates issued by the same PKI. See {ref}`security` for details. (extension-container-last-used-at)= ## `container_last_used_at` A `last_used_at` field was added to the `GET /1.0/instances/` endpoint. It is a timestamp of the last time the instance was started. If an instance has been created but not started yet, `last_used_at` field will be `1970-01-01T00:00:00Z` (extension-etag)= ## `etag` Add support for the ETag header on all relevant endpoints. This adds the following HTTP header on answers to GET: * ETag (SHA-256 of user modifiable content) And adds support for the following HTTP header on PUT requests: * If-Match (ETag value retrieved through previous GET) This makes it possible to GET a LXD object, modify it and PUT it without risking to hit a race condition where LXD or another client modified the object in the meantime. (extension-patch)= ## `patch` Add support for the HTTP PATCH method. PATCH allows for partial update of an object in place of PUT. (extension-usb-devices)= ## `usb_devices` Add support for USB hotplug. (extension-https-allowed-credentials)= ## `https_allowed_credentials` To use LXD API with all Web Browsers (via SPAs) you must send credentials (certificate) with each XHR (in order for this to happen, you should set [`withCredentials=true`](https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest/withCredentials) flag to each XHR Request). Some browsers like Firefox and Safari can't accept server response without `Access-Control-Allow-Credentials: true` header. To ensure that the server will return a response with that header, set {config:option}`server-core:core.https_allowed_credentials` to `true`. (extension-image-compression-algorithm)= ## `image_compression_algorithm` This adds support for a `compression_algorithm` property when creating an image (`POST /1.0/images`). Setting this property overrides the server default value ({config:option}`server-images:images.compression_algorithm`). (extension-directory-manipulation)= ## `directory_manipulation` This allows for creating and listing directories via the LXD API, and exports the file type via the X-LXD-type header, which can be either `file` or `directory` right now. (extension-container-cpu-time)= ## `container_cpu_time` This adds support for retrieving CPU time for a running container. (extension-storage-zfs-use-refquota)= ## `storage_zfs_use_refquota` Introduces a new server property {config:option}`storage-zfs-volume-conf:zfs.use_refquota` which instructs LXD to set the `refquota` property instead of `quota` when setting a size limit on a container. LXD will also then use `usedbydataset` in place of `used` when being queried about disk utilization. This effectively controls whether disk usage by snapshots should be considered as part of the container's disk space usage. (extension-storage-lvm-mount-options)= ## `storage_lvm_mount_options` Adds a new `storage.lvm_mount_options` daemon configuration option which defaults to `discard` and allows the user to set additional mount options for the file system used by the LVM LV. (extension-network)= ## `network` Network management API for LXD. This includes: * Addition of the `managed` property on `/1.0/networks` entries * All the network configuration options (see [Network configuration](networks.md) for details) * `POST /1.0/networks` (see [RESTful API](rest-api.md) for details) * `PUT /1.0/networks/` (see [RESTful API](rest-api.md) for details) * `PATCH /1.0/networks/` (see [RESTful API](rest-api.md) for details) * `DELETE /1.0/networks/` (see [RESTful API](rest-api.md) for details) * `ipv4.address` property on `nic` type devices (when `nictype` is `bridged`) * `ipv6.address` property on `nic` type devices (when `nictype` is `bridged`) * `security.mac_filtering` property on `nic` type devices (when `nictype` is `bridged`) (extension-profile-usedby)= ## `profile_usedby` Adds a new `used_by` field to profile entries listing the instances that are using it. (extension-container-push)= ## `container_push` When an instance is created in push mode, the client serves as a proxy between the source and target server. This is useful in cases where the target server is behind a NAT or firewall and cannot directly communicate with the source server and operate in pull mode. (extension-container-exec-recording)= ## `container_exec_recording` Introduces a new Boolean `record-output`, parameter to `/1.0/instances//exec` which when set to `true` and combined with with `wait-for-websocket` set to `false`, will record stdout and stderr to disk and make them available through the logs interface. The URL to the recorded output is included in the operation metadata once the command is done running. That output will expire similarly to other log files, typically after 48 hours. (extension-certificate-update)= ## `certificate_update` Adds the following to the REST API: * ETag header on GET of a certificate * PUT of certificate entries * PATCH of certificate entries (extension-container-exec-signal-handling)= ## `container_exec_signal_handling` Adds support `/1.0/instances//exec` for forwarding signals sent to the client to the processes executing in the instance. Currently SIGTERM and SIGHUP are forwarded. Further signals that can be forwarded might be added later. (extension-gpu-devices)= ## `gpu_devices` Enables adding GPUs to a container. (extension-container-image-properties)= ## `container_image_properties` Introduces a new `image` configuration key space. Read-only, includes the properties of the parent image. (extension-migration-progress)= ## `migration_progress` Transfer progress is now exported as part of the operation, on both sending and receiving ends. This shows up as a `fs_progress` attribute in the operation metadata. (extension-id-map)= ## `id_map` Enables setting the {config:option}`instance-security:security.idmap.isolated`, {config:option}`instance-security:security.idmap.size`, and {config:option}`instance-raw:raw.idmap` fields. (extension-network-firewall-filtering)= ## `network_firewall_filtering` Add two new keys, {config:option}`network-bridge-network-conf:ipv4.firewall` and {config:option}`network-bridge-network-conf:ipv6.firewall` which if set to `false` will turn off the generation of `iptables` FORWARDING rules. NAT rules will still be added so long as the matching {config:option}`network-bridge-network-conf:ipv4.nat` or {config:option}`network-bridge-network-conf:ipv6.nat` key is set to `true`. Rules necessary for `dnsmasq` to work (DHCP/DNS) will always be applied if `dnsmasq` is enabled on the bridge. (extension-network-routes)= ## `network_routes` Introduces {config:option}`network-bridge-network-conf:ipv4.routes` and {config:option}`network-bridge-network-conf:ipv6.routes` which allow routing additional subnets to a LXD bridge. (extension-storage)= ## `storage` Storage management API for LXD. This includes: * `GET /1.0/storage-pools` * `POST /1.0/storage-pools` (see [RESTful API](rest-api.md) for details) * `GET /1.0/storage-pools/` (see [RESTful API](rest-api.md) for details) * `POST /1.0/storage-pools/` (see [RESTful API](rest-api.md) for details) * `PUT /1.0/storage-pools/` (see [RESTful API](rest-api.md) for details) * `PATCH /1.0/storage-pools/` (see [RESTful API](rest-api.md) for details) * `DELETE /1.0/storage-pools/` (see [RESTful API](rest-api.md) for details) * `GET /1.0/storage-pools//volumes` (see [RESTful API](rest-api.md) for details) * `GET /1.0/storage-pools//volumes/` (see [RESTful API](rest-api.md) for details) * `POST /1.0/storage-pools//volumes/` (see [RESTful API](rest-api.md) for details) * `GET /1.0/storage-pools//volumes//` (see [RESTful API](rest-api.md) for details) * `POST /1.0/storage-pools//volumes//` (see [RESTful API](rest-api.md) for details) * `PUT /1.0/storage-pools//volumes//` (see [RESTful API](rest-api.md) for details) * `PATCH /1.0/storage-pools//volumes//` (see [RESTful API](rest-api.md) for details) * `DELETE /1.0/storage-pools//volumes//` (see [RESTful API](rest-api.md) for details) * All storage configuration options (see [Storage configuration](storage.md) for details) (extension-file-delete)= ## `file_delete` Implements `DELETE` in `/1.0/instances//files` (extension-file-append)= ## `file_append` Implements the `X-LXD-write` header which can be one of `overwrite` or `append`. (extension-network-dhcp-expiry)= ## `network_dhcp_expiry` Introduces {config:option}`network-bridge-network-conf:ipv4.dhcp.expiry` and {config:option}`network-bridge-network-conf:ipv6.dhcp.expiry` allowing to set the DHCP lease expiry time. (extension-storage-lvm-vg-rename)= ## `storage_lvm_vg_rename` Introduces the ability to rename a volume group by setting {config:option}`storage-lvm-pool-conf:lvm.vg_name`. (extension-storage-lvm-thinpool-rename)= ## `storage_lvm_thinpool_rename` Introduces the ability to rename a thin pool name by setting {config:option}`storage-lvm-pool-conf:lvm.thinpool_name`. (extension-network-vlan)= ## `network_vlan` This adds a new {config:option}`device-nic-macvlan-device-conf:vlan` property to `macvlan` network devices. When set, this will instruct LXD to attach to the specified VLAN. LXD will look for an existing interface for that VLAN on the host. If one can't be found it will create one itself and then use that as the macvlan parent. (extension-image-create-aliases)= ## `image_create_aliases` Adds a new `aliases` field to `POST /1.0/images` allowing for aliases to be set at image creation/import time. (extension-container-stateless-copy)= ## `container_stateless_copy` This introduces a new `live` attribute in `POST /1.0/instances/`. Setting it to `false` tells LXD not to attempt running state transfer. (extension-container-only-migration)= ## `container_only_migration` Introduces a new Boolean `container_only` attribute. When set to `true` only the instance will be copied or moved. (extension-storage-zfs-clone-copy)= ## `storage_zfs_clone_copy` Introduces a new Boolean {config:option}`storage-zfs-pool-conf:zfs.clone_copy` property for ZFS storage pools. When set to `false` copying an instance will be done through `zfs send` and receive. This will make the target instance independent of its source instance thus avoiding the need to keep dependent snapshots in the ZFS pool around. However, this also entails less efficient storage usage for the affected pool. The default value for this property is `true`, i.e. space-efficient snapshots will be used unless explicitly set to `false`. (extension-unix-device-rename)= ## `unix_device_rename` Introduces the ability to rename the `unix-block`/`unix-char` device inside container by setting `path`, and the `source` attribute is added to specify the device on host. If `source` is set without a `path`, we should assume that `path` will be the same as `source`. If `path` is set without `source` and `major`/`minor` isn't set, we should assume that `source` will be the same as `path`. So at least one of them must be set. (extension-storage-lvm-use-thinpool)= ## `storage_lvm_use_thinpool` Adds the {config:option}`storage-lvm-pool-conf:lvm.use_thinpool` configuration key, a Boolean that defaults to `true`. When set to `false`, LXD creates plain logical volumes for storage entities instead of using an LVM thin pool. (extension-storage-rsync-bwlimit)= ## `storage_rsync_bwlimit` When `rsync` has to be invoked to transfer storage entities setting `rsync.bwlimit` places an upper limit on the amount of socket I/O allowed. (extension-network-vxlan-interface)= ## `network_vxlan_interface` This introduces a new {config:option}`network-bridge-network-conf:tunnel.NAME.interface` option for networks. This key control what host network interface is used for a VXLAN tunnel. (extension-storage-btrfs-mount-options)= ## `storage_btrfs_mount_options` This introduces the {config:option}`storage-btrfs-pool-conf:btrfs.mount_options` property for Btrfs storage pools. This key controls what mount options will be used for the Btrfs storage pool. (extension-entity-description)= ## `entity_description` This adds descriptions to entities like instances, snapshots, networks, storage pools and volumes. (extension-image-force-refresh)= ## `image_force_refresh` This allows forcing a refresh for an existing image. (extension-storage-lvm-lv-resizing)= ## `storage_lvm_lv_resizing` This introduces the ability to resize logical volumes by setting the `size` property in the instance root disk device. (extension-id-map-base)= ## `id_map_base` This introduces a new {config:option}`instance-security:security.idmap.base` allowing the user to skip the map auto-selection process for isolated containers and specify what host UID/GID to use as the base. (extension-file-symlinks)= ## `file_symlinks` This adds support for transferring symlinks through the file API. X-LXD-type can now be `symlink` with the request content being the target path. (extension-container-push-target)= ## `container_push_target` This adds the `target` field to `POST /1.0/instances/` which can be used to have the source LXD host connect to the target during migration. (extension-network-vlan-physical)= ## `network_vlan_physical` Allows use of {config:option}`network-physical-network-conf:vlan` property with `physical` network devices. When set, this will instruct LXD to attach to the specified VLAN on the `parent` interface. LXD will look for an existing interface for that `parent` and VLAN on the host. If one can't be found it will create one itself. Then, LXD will directly attach this interface to the container. (extension-storage-images-delete)= ## `storage_images_delete` This enabled the storage API to delete storage volumes for images from a specific storage pool. (extension-container-edit-metadata)= ## `container_edit_metadata` This adds support for editing an instance `metadata.yaml` and related templates via API, by accessing URLs under `/1.0/instances//metadata`. It can be used to edit an instance before publishing an image from it. (extension-container-snapshot-stateful-migration)= ## `container_snapshot_stateful_migration` This enables migrating stateful container snapshots to new containers. (extension-storage-driver-ceph)= ## `storage_driver_ceph` This adds a Ceph storage driver. (extension-storage-ceph-user-name)= ## `storage_ceph_user_name` This adds the ability to specify the Ceph user. (extension-resource-limits)= ## `resource_limits` This adds the `instance_type` field to the container creation request. Its value is expanded to LXD resource limits. (extension-storage-volatile-initial-source)= ## `storage_volatile_initial_source` This records the actual source passed to LXD during storage pool creation. (extension-storage-ceph-force-osd-reuse)= ## `storage_ceph_force_osd_reuse` This introduces the `ceph.osd.force_reuse` property for the Ceph storage driver. When set to `true` LXD will reuse an OSD storage pool that is already in use by another LXD instance. (extension-storage-block-filesystem-btrfs)= ## `storage_block_filesystem_btrfs` This adds support for Btrfs as a storage volume file system, in addition to `ext4` and `xfs`. (extension-resources)= ## `resources` This adds support for querying a LXD daemon for the system resources it has available. (extension-kernel-limits)= ## `kernel_limits` This adds support for setting process limits such as maximum number of open files for the container via `nofile`. The format is `limits.kernel.[limit name]`. (extension-storage-api-volume-rename)= ## `storage_api_volume_rename` This adds support for renaming custom storage volumes. (extension-network-sriov)= ## `network_sriov` This adds support for SR-IOV enabled network devices. (extension-console)= ## `console` This adds support to interact with the container console device and console log. (extension-restrict-devlxd)= ## `restrict_devlxd` A new {config:option}`instance-security:security.devlxd` container configuration key was introduced. The key controls whether the `/dev/lxd` interface is made available to the instance. If set to `false`, this effectively prevents the container from interacting with the LXD daemon. (extension-migration-pre-copy)= ## `migration_pre_copy` This adds support for optimized memory transfer during live migration. (extension-infiniband)= ## `infiniband` This adds support to use InfiniBand network devices. (extension-devlxd-events)= ## `devlxd_events` This adds a WebSocket API to the `devlxd` socket. When connecting to `/1.0/events` over the `devlxd` socket, you will now be getting a stream of events over WebSocket. (extension-proxy)= ## `proxy` This adds a new `proxy` device type to containers, allowing forwarding of connections between the host and container. (extension-network-dhcp-gateway)= ## `network_dhcp_gateway` Introduces a new {config:option}`network-bridge-network-conf:ipv4.dhcp.gateway` network configuration key to set an alternate gateway. (extension-file-get-symlink)= ## `file_get_symlink` This makes it possible to retrieve symlinks using the file API. (extension-network-leases)= ## `network_leases` Adds a new `/1.0/networks/NAME/leases` API endpoint to query the lease database on bridges which run a LXD-managed DHCP server. (extension-unix-device-hotplug)= ## `unix_device_hotplug` This adds support for the {config:option}`device-unix-hotplug-device-conf:required` property for Unix devices. (extension-storage-api-local-volume-handling)= ## `storage_api_local_volume_handling` This add the ability to copy and move custom storage volumes locally in the same and between storage pools. (extension-operation-description)= ## `operation_description` Adds a `description` field to all operations. (extension-clustering)= ## `clustering` Clustering API for LXD. This includes the following new endpoints (see [RESTful API](rest-api.md) for details): * `GET /1.0/cluster` * `UPDATE /1.0/cluster` * `GET /1.0/cluster/members` * `GET /1.0/cluster/members/` * `POST /1.0/cluster/members/` * `DELETE /1.0/cluster/members/` The following existing endpoints have been modified: * `POST /1.0/instances` accepts a new `target` query parameter * `POST /1.0/storage-pools` accepts a new `target` query parameter * `GET /1.0/storage-pool/` accepts a new `target` query parameter * `POST /1.0/storage-pool//volumes/` accepts a new `target` query parameter * `GET /1.0/storage-pool//volumes//` accepts a new `target` query parameter * `POST /1.0/storage-pool//volumes//` accepts a new `target` query parameter * `PUT /1.0/storage-pool//volumes//` accepts a new `target` query parameter * `PATCH /1.0/storage-pool//volumes//` accepts a new `target` query parameter * `DELETE /1.0/storage-pool//volumes//` accepts a new `target` query parameter * `POST /1.0/networks` accepts a new `target` query parameter * `GET /1.0/networks/` accepts a new `target` query parameter (extension-event-lifecycle)= ## `event_lifecycle` This adds a new `lifecycle` message type to the events API. (extension-storage-api-remote-volume-handling)= ## `storage_api_remote_volume_handling` This adds the ability to copy and move custom storage volumes between remote. (extension-container-mount-propagation)= ## `container_mount_propagation` This adds a new {config:option}`device-disk-device-conf:propagation` option to the disk device type, allowing the configuration of kernel mount propagation. (extension-container-backup)= ## `container_backup` Add container backup support. This includes the following new endpoints (see [RESTful API](rest-api.md) for details): * `GET /1.0/instances//backups` * `POST /1.0/instances//backups` * `GET /1.0/instances//backups/` * `POST /1.0/instances//backups/` * `DELETE /1.0/instances//backups/` * `GET /1.0/instances//backups//export` The following existing endpoint has been modified: * `POST /1.0/instances` accepts the new source type `backup` (extension-devlxd-images)= ## `devlxd_images` Adds a {config:option}`instance-security:security.devlxd.images` configuration option for containers which controls the availability of a `/1.0/images/FINGERPRINT/export` API over `devlxd`. This can be used by a container running nested LXD to access raw images from the host. (extension-container-local-cross-pool-handling)= ## `container_local_cross_pool_handling` This enables copying or moving containers between storage pools on the same LXD instance. (extension-proxy-unix)= ## `proxy_unix` Add support for both Unix sockets and abstract Unix sockets in proxy devices. They can be used by specifying the address as `unix:/path/to/unix.sock` (normal socket) or `unix:@/tmp/unix.sock` (abstract socket). Supported connections are now: * `TCP <-> TCP` * `UNIX <-> UNIX` * `TCP <-> UNIX` * `UNIX <-> TCP` (extension-proxy-udp)= ## `proxy_udp` Add support for UDP in proxy devices. Supported connections are now: * `TCP <-> TCP` * `UNIX <-> UNIX` * `TCP <-> UNIX` * `UNIX <-> TCP` * `UDP <-> UDP` * `TCP <-> UDP` * `UNIX <-> UDP` (extension-clustering-join)= ## `clustering_join` This makes `GET /1.0/cluster` return information about which storage pools and networks are required to be created by joining nodes and which node-specific configuration keys they are required to use when creating them. Likewise the `PUT /1.0/cluster` endpoint now accepts the same format to pass information about storage pools and networks to be automatically created before attempting to join a cluster. (extension-proxy-tcp-udp-multi-port-handling)= ## `proxy_tcp_udp_multi_port_handling` Adds support for forwarding traffic for multiple ports. Forwarding is allowed between a range of ports if the port range is equal for source and target (for example `1.2.3.4 0-1000 -> 5.6.7.8 1000-2000`) and between a range of source ports and a single target port (for example `1.2.3.4 0-1000 -> 5.6.7.8 1000`). (extension-network-state)= ## `network_state` Adds support for retrieving a network's state. This adds the following new endpoint (see [RESTful API](rest-api.md) for details): * `GET /1.0/networks//state` (extension-proxy-unix-dac-properties)= ## `proxy_unix_dac_properties` This adds support for GID, UID, and mode properties for non-abstract Unix sockets. (extension-container-protection-delete)= ## `container_protection_delete` Enables setting the {config:option}`instance-security:security.protection.delete` field which prevents containers from being deleted if set to `true`. Snapshots are not affected by this setting. (extension-unix-priv-drop)= ## `unix_priv_drop` Adds {config:option}`device-proxy-device-conf:security.uid` and {config:option}`device-proxy-device-conf:security.gid` for the proxy devices, allowing privilege dropping and effectively changing the UID/GID used for connections to Unix sockets too. (extension-pprof-http)= ## `pprof_http` This adds a new {config:option}`server-core:core.debug_address` configuration option to start a debugging HTTP server. That server currently includes a `pprof` API and replaces the old `cpu-profile`, `memory-profile` and `print-goroutines` debug options. (extension-proxy-haproxy-protocol)= ## `proxy_haproxy_protocol` Adds a {config:option}`device-proxy-device-conf:proxy_protocol` key to the proxy device which controls the use of the HAProxy PROXY protocol header. (extension-network-hwaddr)= ## `network_hwaddr` Adds a {config:option}`network-bridge-network-conf:bridge.hwaddr` key to control the MAC address of the bridge. (extension-proxy-nat)= ## `proxy_nat` This adds optimized UDP/TCP proxying. If the configuration allows, proxying will be done via `iptables` instead of proxy devices. (extension-network-nat-order)= ## `network_nat_order` This introduces the {config:option}`network-bridge-network-conf:ipv4.nat.order` and {config:option}`network-bridge-network-conf:ipv6.nat.order` configuration keys for LXD bridges. Those keys control whether to put the LXD rules before or after any pre-existing rules in the chain. (extension-container-full)= ## `container_full` This introduces a new `recursion=2` mode for `GET /1.0/instances` which allows for the retrieval of all instance structs, including the state, snapshots and backup structs. This effectively allows for [`lxc list`](lxc_list.md) to get all it needs in one query. (extension-backup-compression)= ## `backup_compression` This introduces a new {config:option}`server-miscellaneous:backups.compression_algorithm` configuration key which allows configuration of backup compression. (extension-storage-api-volume-snapshots)= ## `storage_api_volume_snapshots` Add support for storage volume snapshots. They work like container snapshots, only for volumes. This adds the following new endpoint (see [RESTful API](rest-api.md) for details): * `GET /1.0/storage-pools//volumes///snapshots` * `POST /1.0/storage-pools//volumes///snapshots` * `GET /1.0/storage-pools//volumes///snapshots/` * `PUT /1.0/storage-pools//volumes///snapshots/` * `POST /1.0/storage-pools//volumes///snapshots/` * `DELETE /1.0/storage-pools//volumes///snapshots/` (extension-storage-unmapped)= ## `storage_unmapped` Introduces a new `security.unmapped` Boolean on storage volumes. Setting it to `true` will flush the current map on the volume and prevent any further idmap tracking and remapping on the volume. This can be used to share data between isolated containers after attaching it to the container which requires write access. (extension-projects)= ## `projects` Add a new project API, supporting creation, update and deletion of projects. Projects can hold containers, profiles or images at this point and let you get a separate view of your LXD resources by switching to it. (extension-network-vxlan-ttl)= ## `network_vxlan_ttl` This adds a new {config:option}`network-bridge-network-conf:tunnel.NAME.ttl` network configuration option which makes it possible to raise the TTL on VXLAN tunnels. (extension-container-incremental-copy)= ## `container_incremental_copy` This adds support for incremental container copy. When copying a container using the `--refresh` flag, only the missing or outdated files will be copied over. Should the target container not exist yet, a normal copy operation is performed. (extension-usb-optional-vendorid)= ## `usb_optional_vendorid` As the name implies, the {config:option}`device-unix-usb-device-conf:vendorid` field on USB devices attached to containers has now been made optional, allowing for all USB devices to be passed to a container (similar to what's done for GPUs). (extension-snapshot-scheduling)= ## `snapshot_scheduling` This adds support for snapshot scheduling. It introduces three new configuration keys: `snapshots.schedule`, `snapshots.schedule.stopped`, and `snapshots.pattern`. Snapshots can be created automatically up to every minute. (extension-snapshot-schedule-aliases)= ## `snapshot_schedule_aliases` Snapshot schedule can be configured by a comma-separated list of schedule aliases. Available aliases are `<@hourly> <@daily> <@midnight> <@weekly> <@monthly> <@annually> <@yearly> <@startup>` for instances, and `<@hourly> <@daily> <@midnight> <@weekly> <@monthly> <@annually> <@yearly>` for storage volumes. (extension-container-copy-project)= ## `container_copy_project` Introduces a `project` field to the container source JSON object, allowing for copy/move of containers between projects. (extension-clustering-server-address)= ## `clustering_server_address` This adds support for configuring a server network address which differs from the REST API client network address. When bootstrapping a new cluster, clients can set the new {config:option}`server-cluster:cluster.https_address` configuration key to specify the address of the initial server. When joining a new server, clients can set the {config:option}`server-core:core.https_address` configuration key of the joining server to the REST API address the joining server should listen at, and set the `server_address` key in the `PUT /1.0/cluster` API to the address the joining server should use for clustering traffic (the value of `server_address` will be automatically copied to the `cluster.https_address` configuration key of the joining server). (extension-clustering-image-replication)= ## `clustering_image_replication` Enable image replication across the nodes in the cluster. A new {config:option}`server-cluster:cluster.images_minimal_replica` configuration key was introduced can be used to specify to the minimal numbers of nodes for image replication. (extension-container-protection-shift)= ## `container_protection_shift` Enables setting the {config:option}`instance-security:security.protection.shift` option which prevents containers from having their file system shifted. (extension-snapshot-expiry)= ## `snapshot_expiry` This adds support for snapshot expiration. The task is run minutely. The configuration option {config:option}`instance-snapshots:snapshots.expiry` takes an expression in the form of `1M 2H 3d 4w 5m 6y` (1 minute, 2 hours, 3 days, 4 weeks, 5 months, 6 years), however not all parts have to be used. Snapshots which are then created will be given an expiry date based on the expression. This expiry date, defined by `expires_at`, can be manually edited using the API or [`lxc config edit`](lxc_config_edit.md). Snapshots with a valid expiry date will be removed when the task in run. Expiry can be disabled by setting `expires_at` to an empty string or `0001-01-01T00:00:00Z` (zero time). This is the default if `snapshots.expiry` is not set. This adds the following new endpoint (see [RESTful API](rest-api.md) for details): * `PUT /1.0/instances//snapshots/` (extension-container-backup-override-pool)= ## `container_backup_override_pool` Adds the `PoolName` field to `InstanceBackupArgs`, sent as the `X-LXD-pool` header on `POST /1.0/instances`, allowing the storage pool that an instance is restored into from a backup to be overridden instead of using the pool recorded in the backup. (extension-snapshot-expiry-creation)= ## `snapshot_expiry_creation` Adds `expires_at` to container creation, allowing for override of a snapshot's expiry at creation time. (extension-network-leases-location)= ## `network_leases_location` Introduces a `Location` field in the leases list. This is used when querying a cluster to show what node a particular lease was found on. (extension-resources-cpu-socket)= ## `resources_cpu_socket` Add Socket field to CPU resources in case we get out of order socket information. (extension-resources-gpu)= ## `resources_gpu` Add a new GPU struct to the server resources, listing all usable GPUs on the system. (extension-resources-numa)= ## `resources_numa` Shows the NUMA node for all CPUs and GPUs. (extension-kernel-features)= ## `kernel_features` Exposes the state of optional kernel features through the server environment. (extension-id-map-current)= ## `id_map_current` This introduces a new internal {config:option}`instance-volatile:volatile.idmap.current` key which is used to track the current mapping for the container. This effectively gives us: * {config:option}`instance-volatile:volatile.last_state.idmap` => On-disk idmap * {config:option}`instance-volatile:volatile.idmap.current` => Current kernel map * {config:option}`instance-volatile:volatile.idmap.next` => Next on-disk idmap This is required to implement environments where the on-disk map isn't changed but the kernel map is (e.g. `idmapped mounts`). (extension-event-location)= ## `event_location` Expose the location of the generation of API events. (extension-storage-api-remote-volume-snapshots)= ## `storage_api_remote_volume_snapshots` This allows migrating storage volumes including their snapshots. (extension-network-nat-address)= ## `network_nat_address` This introduces the {config:option}`network-bridge-network-conf:ipv4.nat.address` and {config:option}`network-bridge-network-conf:ipv6.nat.address` configuration keys for LXD bridges. Those keys control the source address used for outbound traffic from the bridge. (extension-container-nic-routes)= ## `container_nic_routes` This introduces the {config:option}`device-nic-bridged-device-conf:ipv4.routes` and {config:option}`device-nic-bridged-device-conf:ipv6.routes` properties on `nic` type devices. This allows adding static routes on host to container's NIC. (extension-cluster-internal-copy)= ## `cluster_internal_copy` This makes it possible to do a normal `POST /1.0/instances` to copy an instance between cluster nodes with LXD internally detecting whether a migration is required. (extension-seccomp-notify)= ## `seccomp_notify` If the kernel supports `seccomp`-based syscall interception LXD can be notified by a container that a registered syscall has been performed. LXD can then decide to trigger various actions. (extension-lxc-features)= ## `lxc_features` This introduces the `lxc_features` section output from the [`lxc info`](lxc_info.md) command via the `GET /1.0` route. It outputs the result of checks for key features being present in the underlying LXC library. (extension-container-nic-ipvlan)= ## `container_nic_ipvlan` This introduces the `ipvlan` `nic` device type. (extension-network-vlan-sriov)= ## `network_vlan_sriov` This introduces VLAN ({config:option}`device-nic-sriov-device-conf:vlan`) and MAC filtering ({config:option}`device-nic-sriov-device-conf:security.mac_filtering`) support for SR-IOV devices. (extension-storage-cephfs)= ## `storage_cephfs` Add support for CephFS as a storage pool driver. This can only be used for custom volumes, images and containers should be on Ceph (RBD) instead. (extension-container-nic-ipfilter)= ## `container_nic_ipfilter` This introduces container IP filtering ({config:option}`device-nic-bridged-device-conf:security.ipv4_filtering` and {config:option}`device-nic-bridged-device-conf:security.ipv6_filtering`) support for `bridged` NIC devices. (extension-resources-v2)= ## `resources_v2` Rework the resources API at `/1.0/resources`, especially: * CPU * Fix reporting to track sockets, cores and threads * Track NUMA node per core * Track base and turbo frequency per socket * Track current frequency per core * Add CPU cache information * Export the CPU architecture * Show online/offline status of threads * Memory * Add huge-pages tracking * Track memory consumption per NUMA node too * GPU * Split DRM information to separate struct * Export device names and nodes in DRM struct * Export device name and node in NVIDIA struct * Add SR-IOV VF tracking (extension-container-exec-user-group-cwd)= ## `container_exec_user_group_cwd` Adds support for specifying `User`, `Group` and `Cwd` during `POST /1.0/instances/NAME/exec`. (extension-container-syscall-intercept)= ## `container_syscall_intercept` Adds the `security.syscalls.intercept.*` configuration keys to control what system calls will be intercepted by LXD and processed with elevated permissions. (extension-container-disk-shift)= ## `container_disk_shift` Adds the {config:option}`device-disk-device-conf:shift` property on `disk` devices which controls the use of the `idmapped mounts` overlay. (extension-storage-shifted)= ## `storage_shifted` Introduces a new `security.shifted` Boolean on storage volumes. Setting it to `true` will allow multiple isolated containers to attach the same storage volume while keeping the file system writable from all of them. This makes use of `idmapped mounts` as an overlay file system. (extension-resources-infiniband)= ## `resources_infiniband` Export InfiniBand character device information (`issm`, `umad`, `uverb`) as part of the resources API. (extension-daemon-storage)= ## `daemon_storage` This introduces two new configuration keys {config:option}`server-miscellaneous:storage.images_volume` and {config:option}`server-miscellaneous:storage.backups_volume` to allow for a storage volume on an existing pool be used for storing the daemon-wide images and backups artifacts. (extension-instances)= ## `instances` This introduces the concept of instances, of which currently the only type is `container`. (extension-image-types)= ## `image_types` This introduces support for a new Type field on images, indicating what type of images they are. (extension-resources-disk-sata)= ## `resources_disk_sata` Extends the disk resource API struct to include: * Proper detection of SATA devices (type) * Device path * Drive RPM * Block size * Firmware version * Serial number (extension-clustering-roles)= ## `clustering_roles` This adds a new `roles` attribute to cluster entries, exposing a list of roles that the member serves in the cluster. (extension-images-expiry)= ## `images_expiry` This allows for editing of the expiry date on images. (extension-resources-network-firmware)= ## `resources_network_firmware` Adds a `FirmwareVersion` field to network card entries. (extension-backup-compression-algorithm)= ## `backup_compression_algorithm` This adds support for a `compression_algorithm` property when creating a backup (`POST /1.0/instances//backups`). Setting this property overrides the server default value ({config:option}`server-miscellaneous:backups.compression_algorithm`). (extension-ceph-data-pool-name)= ## `ceph_data_pool_name` This adds support for an optional argument ({config:option}`storage-ceph-pool-conf:ceph.osd.data_pool_name`) when creating storage pools using Ceph RBD, when this argument is used the pool will store it's actual data in the pool specified with `data_pool_name` while keeping the metadata in the pool specified by `pool_name`. (extension-container-syscall-intercept-mount)= ## `container_syscall_intercept_mount` Adds the {config:option}`instance-security:security.syscalls.intercept.mount`, {config:option}`instance-security:security.syscalls.intercept.mount.allowed`, and {config:option}`instance-security:security.syscalls.intercept.mount.shift` configuration keys to control whether and how the `mount` system call will be intercepted by LXD and processed with elevated permissions. (extension-compression-squashfs)= ## `compression_squashfs` Adds support for importing/exporting of images/backups using SquashFS file system format. (extension-container-raw-mount)= ## `container_raw_mount` This adds support for passing in raw mount options for disk devices. (extension-container-nic-routed)= ## `container_nic_routed` This introduces the `routed` `nic` device type. (extension-container-syscall-intercept-mount-fuse)= ## `container_syscall_intercept_mount_fuse` Adds the {config:option}`instance-security:security.syscalls.intercept.mount.fuse` key. It can be used to redirect file-system mounts to their fuse implementation. To this end, set e.g. `security.syscalls.intercept.mount.fuse=ext4=fuse2fs`. (extension-container-disk-ceph)= ## `container_disk_ceph` This allows for existing a Ceph RBD or CephFS to be directly connected to a LXD container. (extension-virtual-machines)= ## `virtual-machines` Add virtual machine support. (extension-image-profiles)= ## `image_profiles` Allows a list of profiles to be applied to an image when launching a new container. (extension-clustering-architecture)= ## `clustering_architecture` This adds a new `architecture` attribute to cluster members which indicates a cluster member's architecture. (extension-resources-disk-id)= ## `resources_disk_id` Add a new `device_id` field in the disk entries on the resources API. (extension-storage-lvm-stripes)= ## `storage_lvm_stripes` This adds the ability to use LVM stripes on normal volumes and thin pool volumes. (extension-vm-boot-priority)= ## `vm_boot_priority` Adds a `boot.priority` property on NIC and disk devices to control the boot order. (extension-unix-hotplug-devices)= ## `unix_hotplug_devices` Adds support for Unix char and block device hotplugging. (extension-api-filtering)= ## `api_filtering` Adds support for filtering the result of a GET request for instances and images. (extension-instance-nic-network)= ## `instance_nic_network` Adds support for the `network` property on a NIC device to allow a NIC to be linked to a managed network. This allows it to inherit some of the network's settings and allows better validation of IP settings. (extension-clustering-sizing)= ## `clustering_sizing` Support specifying a custom values for database voters and standbys. The new {config:option}`server-cluster:cluster.max_voters` and {config:option}`server-cluster:cluster.max_standby` configuration keys were introduced to specify to the ideal number of database voter and standbys. (extension-firewall-driver)= ## `firewall_driver` Adds the `Firewall` property to the `ServerEnvironment` struct indicating the firewall driver being used. (extension-projects-limits)= ## `projects_limits` Adds support for project-level resource limits, through the {config:option}`project-limits:limits.cpu`, {config:option}`project-limits:limits.memory`, {config:option}`project-limits:limits.processes`, {config:option}`project-limits:limits.containers`, and {config:option}`project-limits:limits.virtual-machines` project configuration keys. ## `storage_lvm_vg_force_reuse` Introduces the ability to create a storage pool from an existing non-empty volume group. This option should be used with care, as LXD can then not guarantee that volume name conflicts won't occur with non-LXD created volumes in the same volume group. This could also potentially lead to LXD deleting a non-LXD volume should name conflicts occur. (extension-container-syscall-intercept-hugetlbfs)= ## `container_syscall_intercept_hugetlbfs` When mount syscall interception is enabled and `hugetlbfs` is specified as an allowed file system type LXD will mount a separate `hugetlbfs` instance for the container with the UID and GID mount options set to the container's root UID and GID. This ensures that processes in the container can use huge pages. (extension-limits-hugepages)= ## `limits_hugepages` This allows to limit the number of huge pages a container can use through the `hugetlb` cgroup. This means the `hugetlb` cgroup needs to be available. Note, that limiting huge pages is recommended when intercepting the mount syscall for the `hugetlbfs` file system to avoid allowing the container to exhaust the host's huge pages resources. (extension-container-nic-routed-gateway)= ## `container_nic_routed_gateway` This introduces the {config:option}`device-nic-routed-device-conf:ipv4.gateway` and {config:option}`device-nic-routed-device-conf:ipv6.gateway` NIC configuration keys that can take a value of either `auto` or `none`. The default value for the key if unspecified is `auto`. This will cause the current behavior of a default gateway being added inside the container and the same gateway address being added to the host-side interface. If the value is set to `none` then no default gateway nor will the address be added to the host-side interface. This allows multiple routed NIC devices to be added to a container. (extension-projects-restrictions)= ## `projects_restrictions` This introduces support for the {config:option}`project-restricted:restricted` configuration key on project, which can prevent the use of security-sensitive features in a project. (extension-custom-volume-snapshot-expiry)= ## `custom_volume_snapshot_expiry` This allows custom volume snapshots to expiry. Expiry dates can be set individually, or by setting the `snapshots.expiry` configuration key on the parent custom volume which then automatically applies to all created snapshots. (extension-volume-snapshot-scheduling)= ## `volume_snapshot_scheduling` This adds support for custom volume snapshot scheduling. It introduces two new configuration keys: `snapshots.schedule` and `snapshots.pattern`. Snapshots can be created automatically up to every minute. (extension-trust-ca-certificates)= ## `trust_ca_certificates` This allows for checking client certificates trusted by the provided CA (`server.ca`). It can be enabled by setting {config:option}`server-core:core.trust_ca_certificates` to `true`. If enabled, it will perform the check, and bypass the trusted password if `true`. An exception will be made if the connecting client certificate is in the provided CRL (`ca.crl`). In this case, it will ask for the password. (extension-snapshot-disk-usage)= ## `snapshot_disk_usage` This adds a new `size` field to the output of `/1.0/instances//snapshots/` which represents the disk usage of the snapshot. (extension-clustering-edit-roles)= ## `clustering_edit_roles` This adds a writable endpoint for cluster members, allowing the editing of their roles. (extension-container-nic-routed-host-address)= ## `container_nic_routed_host_address` This introduces the {config:option}`device-nic-routed-device-conf:ipv4.host_address` and {config:option}`device-nic-routed-device-conf:ipv6.host_address` NIC configuration keys that can be used to control the host-side `veth` interface's IP addresses. This can be useful when using multiple routed NICs at the same time and needing a predictable next-hop address to use. This also alters the behavior of {config:option}`device-nic-routed-device-conf:ipv4.gateway` and {config:option}`device-nic-routed-device-conf:ipv6.gateway` NIC configuration keys. When they are set to `auto` the container will have its default gateway set to the value of `ipv4.host_address` or `ipv6.host_address` respectively. The default values are: `ipv4.host_address`: `169.254.0.1` `ipv6.host_address`: `fe80::1` This is backward compatible with the previous default behavior. (extension-container-nic-ipvlan-gateway)= ## `container_nic_ipvlan_gateway` This introduces the {config:option}`device-nic-ipvlan-device-conf:ipv4.gateway` and {config:option}`device-nic-ipvlan-device-conf:ipv6.gateway` NIC configuration keys that can take a value of either `auto` or `none`. The default value for the key if unspecified is `auto`. This will cause the current behavior of a default gateway being added inside the container and the same gateway address being added to the host-side interface. If the value is set to `none` then no default gateway nor will the address be added to the host-side interface. This allows multiple IPVLAN NIC devices to be added to a container. (extension-resources-usb-pci)= ## `resources_usb_pci` This adds USB and PCI devices to the output of `/1.0/resources`. (extension-resources-cpu-threads-numa)= ## `resources_cpu_threads_numa` This indicates that the `numa_node` field is now recorded per-thread rather than per core as some hardware apparently puts threads in different NUMA domains. (extension-resources-cpu-core-die)= ## `resources_cpu_core_die` Exposes the `die_id` information on each core. (extension-api-os)= ## `api_os` This introduces two new fields in `/1.0`, `os` and `os_version`. Those are taken from the OS-release data on the system. (extension-container-nic-routed-host-table)= ## `container_nic_routed_host_table` This introduces the {config:option}`device-nic-routed-device-conf:ipv4.host_table` and {config:option}`device-nic-routed-device-conf:ipv6.host_table` NIC configuration keys that can be used to add static routes for the instance's IPs to a custom policy routing table by ID. (extension-container-nic-ipvlan-host-table)= ## `container_nic_ipvlan_host_table` This introduces the {config:option}`device-nic-ipvlan-device-conf:ipv4.host_table` and {config:option}`device-nic-ipvlan-device-conf:ipv6.host_table` NIC configuration keys that can be used to add static routes for the instance's IPs to a custom policy routing table by ID. (extension-container-nic-ipvlan-mode)= ## `container_nic_ipvlan_mode` This introduces the {config:option}`device-nic-ipvlan-device-conf:mode` NIC configuration key that can be used to switch the `ipvlan` mode into either `l2` or `l3s`. If not specified, the default value is `l3s` (which is the old behavior). In `l2` mode the {config:option}`device-nic-ipvlan-device-conf:ipv4.address` and {config:option}`device-nic-ipvlan-device-conf:ipv6.address` keys will accept addresses in either CIDR or singular formats. If singular format is used, the default subnet size is taken to be /24 and /64 for IPv4 and IPv6 respectively. In `l2` mode the {config:option}`device-nic-ipvlan-device-conf:ipv4.gateway` and {config:option}`device-nic-ipvlan-device-conf:ipv6.gateway` keys accept only a singular IP address. (extension-resources-system)= ## `resources_system` This adds system information to the output of `/1.0/resources`. (extension-images-push-relay)= ## `images_push_relay` This adds the push and relay modes to image copy. It also introduces the following new endpoint: * `POST 1.0/images//export` (extension-network-dns-search)= ## `network_dns_search` This introduces the `dns.search` configuration option on networks. (extension-container-nic-routed-limits)= ## `container_nic_routed_limits` This introduces {config:option}`device-nic-routed-device-conf:limits.ingress`, {config:option}`device-nic-routed-device-conf:limits.egress` and {config:option}`device-nic-routed-device-conf:limits.max` for routed NICs. (extension-instance-nic-bridged-vlan)= ## `instance_nic_bridged_vlan` This introduces the {config:option}`device-nic-bridged-device-conf:vlan` and {config:option}`device-nic-bridged-device-conf:vlan.tagged` settings for `bridged` NICs. `vlan` specifies the non-tagged VLAN to join, and `vlan.tagged` is a comma-delimited list of tagged VLANs to join. (extension-network-state-bond-bridge)= ## `network_state_bond_bridge` This adds a `bridge` and `bond` section to the `/1.0/networks/NAME/state` API. Those contain additional state information relevant to those particular types. Bond: * Mode * Transmit hash * Up delay * Down delay * MII frequency * MII state * Lower devices Bridge: * ID * Forward delay * STP mode * Default VLAN * VLAN filtering * Upper devices ## `resources_cpu_isolated` Add an `Isolated` property on CPU threads to indicate if the thread is physically `Online` but is configured not to accept tasks. (extension-usedby-consistency)= ## `usedby_consistency` This extension indicates that `UsedBy` should now be consistent with suitable `?project=` and `?target=` when appropriate. The 5 entities that have `UsedBy` are: * Profiles * Projects * Networks * Storage pools * Storage volumes (extension-custom-block-volumes)= ## `custom_block_volumes` This adds support for creating and attaching custom block volumes to instances. It introduces the new `--type` flag when creating custom storage volumes, and accepts the values `fs` and `block`. (extension-clustering-failure-domains)= ## `clustering_failure_domains` This extension adds a new `failure_domain` field to the `PUT /1.0/cluster/` API, which can be used to set the failure domain of a node. (extension-container-syscall-filtering-allow-deny-syntax)= ## `container_syscall_filtering_allow_deny_syntax` A number of new syscalls related container configuration keys were updated. * {config:option}`instance-security:security.syscalls.deny_default` * {config:option}`instance-security:security.syscalls.deny_compat` * {config:option}`instance-security:security.syscalls.deny` * {config:option}`instance-security:security.syscalls.allow` Support for the offensively named variants was removed. (extension-resources-gpu-mdev)= ## `resources_gpu_mdev` Expose available mediated device profiles and devices in `/1.0/resources`. (extension-console-vga-type)= ## `console_vga_type` This extends the `/1.0/console` endpoint to take a `?type=` argument, which can be set to `console` (default) or `vga` (the new type added by this extension). When doing a `POST` to `/1.0//console?type=vga` the data WebSocket returned by the operation in the metadata field will be a bidirectional proxy attached to a SPICE Unix socket of the target virtual machine. (extension-projects-limits-disk)= ## `projects_limits_disk` Add {config:option}`project-limits:limits.disk` to the available project configuration keys. If set, it limits the total amount of disk space that instances volumes, custom volumes and images volumes can use in the project. (extension-network-type-macvlan)= ## `network_type_macvlan` Adds support for additional network type `macvlan` and adds {config:option}`network-macvlan-network-conf:parent` configuration key for this network type to specify which parent interface should be used for creating NIC device interfaces on top of. Also adds {config:option}`device-nic-macvlan-device-conf:network` configuration key support for `macvlan` NICs to allow them to specify the associated network of the same type that they should use as the basis for the NIC device. (extension-network-type-sriov)= ## `network_type_sriov` Adds support for additional network type `sriov` and adds {config:option}`network-sriov-network-conf:parent` configuration key for this network type to specify which parent interface should be used for creating NIC device interfaces on top of. Also adds {config:option}`device-nic-sriov-device-conf:network` configuration key support for `sriov` NICs to allow them to specify the associated network of the same type that they should use as the basis for the NIC device. (extension-container-syscall-intercept-bpf-devices)= ## `container_syscall_intercept_bpf_devices` This adds support to intercept the `bpf` syscall in containers. Specifically, it allows to manage device cgroup `bpf` programs. (extension-network-type-ovn)= ## `network_type_ovn` Adds support for additional network type `ovn` with the ability to specify a `bridge` type network as the `parent`. Introduces a new NIC device type of `ovn` which allows the `network` configuration key to specify which `ovn` type network they should connect to. Also introduces two new global configuration keys that apply to all `ovn` networks and NIC devices: * {config:option}`server-miscellaneous:network.ovn.integration_bridge` - the OVS integration bridge to use. * {config:option}`server-miscellaneous:network.ovn.northbound_connection` - the OVN northbound database connection string. (extension-projects-networks)= ## `projects_networks` Adds the {config:option}`project-features:features.networks` configuration key to projects and the ability for a project to hold networks. (extension-projects-networks-restricted-uplinks)= ## `projects_networks_restricted_uplinks` Adds the {config:option}`project-restricted:restricted.networks.uplinks` project configuration key to indicate (as a comma-delimited list) which networks the networks created inside the project can use as their uplink network. (extension-custom-volume-backup)= ## `custom_volume_backup` Add custom volume backup support. This includes the following new endpoints (see [RESTful API](rest-api.md) for details): * `GET /1.0/storage-pools////backups` * `POST /1.0/storage-pools////backups` * `GET /1.0/storage-pools////backups/` * `POST /1.0/storage-pools////backups/` * `DELETE /1.0/storage-pools////backups/` * `GET /1.0/storage-pools////backups//export` The following existing endpoint has been modified: * `POST /1.0/storage-pools///` accepts the new source type `backup` (extension-backup-override-name)= ## `backup_override_name` Adds `Name` field to `InstanceBackupArgs` to allow specifying a different instance name when restoring a backup. Adds `Name` and `PoolName` fields to `StoragePoolVolumeBackupArgs` to allow specifying a different volume name when restoring a custom volume backup. (extension-storage-rsync-compression)= ## `storage_rsync_compression` Adds `rsync.compression` configuration key to storage pools. This key can be used to disable compression in `rsync` while migrating storage pools. (extension-network-type-physical)= ## `network_type_physical` Adds support for additional network type `physical` that can be used as an uplink for `ovn` networks. The interface specified by {config:option}`network-physical-network-conf:parent` on the `physical` network will be connected to the `ovn` network's gateway. (extension-network-ovn-external-subnets)= ## `network_ovn_external_subnets` Adds support for `ovn` networks to use external subnets from uplink networks. Introduces the {config:option}`network-physical-network-conf:ipv4.routes` and {config:option}`network-physical-network-conf:ipv6.routes` setting on `physical` networks that defines the external routes allowed to be used in child OVN networks in their {config:option}`device-nic-ovn-device-conf:ipv4.routes.external` and {config:option}`device-nic-ovn-device-conf:ipv6.routes.external` settings. Introduces the {config:option}`project-restricted:restricted.networks.subnets` project setting that specifies which external subnets are allowed to be used by OVN networks inside the project (if not set then all routes defined on the uplink network are allowed). (extension-network-ovn-nat)= ## `network_ovn_nat` Adds support for {config:option}`network-ovn-network-conf:ipv4.nat` and {config:option}`network-ovn-network-conf:ipv6.nat` settings on `ovn` networks. When creating the network if these settings are unspecified, and an equivalent IP address is being generated for the subnet, then the appropriate NAT setting will added set to `true`. If the setting is missing then the value is taken as `false`. (extension-network-ovn-external-routes-remove)= ## `network_ovn_external_routes_remove` Removes the settings `ipv4.routes.external` and `ipv6.routes.external` from `ovn` networks. The equivalent settings on the `ovn` NIC type can be used instead for this, rather than having to specify them both at the network and NIC level. (extension-tpm-device-type)= ## `tpm_device_type` This introduces the `tpm` device type. (extension-storage-zfs-clone-copy-rebase)= ## `storage_zfs_clone_copy_rebase` This introduces `rebase` as a value for {config:option}`storage-zfs-pool-conf:zfs.clone_copy` causing LXD to track down any `image` dataset in the ancestry line and then perform send/receive on top of that. (extension-gpu-mdev)= ## `gpu_mdev` This adds support for virtual GPUs (vGPUs). It introduces the {config:option}`device-gpu-mdev-device-conf:mdev` configuration key for GPU devices which takes a supported `mdev` type, e.g. `i915-GVTg_V5_4`. (extension-resources-pci-iommu)= ## `resources_pci_iommu` This adds the `IOMMUGroup` field for PCI entries in the resources API. (extension-resources-network-usb)= ## `resources_network_usb` Adds the `usb_address` field to the network card entries in the resources API. (extension-resources-disk-address)= ## `resources_disk_address` Adds the `usb_address` and `pci_address` fields to the disk entries in the resources API. (extension-network-physical-ovn-ingress-mode)= ## `network_physical_ovn_ingress_mode` Adds {config:option}`network-physical-network-conf:ovn.ingress_mode` setting for `physical` networks. Sets the method that OVN NIC external IPs will be advertised on uplink network. Either `l2proxy` (proxy ARP/NDP) or `routed`. (extension-network-ovn-dhcp)= ## `network_ovn_dhcp` Adds {config:option}`network-ovn-network-conf:ipv4.dhcp` and {config:option}`network-ovn-network-conf:ipv6.dhcp` settings for `ovn` networks. Allows DHCP (and RA for IPv6) to be disabled. Defaults to on. (extension-network-physical-routes-anycast)= ## `network_physical_routes_anycast` Adds {config:option}`network-physical-network-conf:ipv4.routes.anycast` and {config:option}`network-physical-network-conf:ipv6.routes.anycast` Boolean settings for `physical` networks. Defaults to `false`. Allows OVN networks using physical network as uplink to relax external subnet/route overlap detection when used with {config:option}`network-physical-network-conf:ovn.ingress_mode` set to `routed`. (extension-projects-limits-instances)= ## `projects_limits_instances` Adds {config:option}`project-limits:limits.instances` to the available project configuration keys. If set, it limits the total number of instances (VMs and containers) that can be used in the project. (extension-network-state-vlan)= ## `network_state_vlan` This adds a `vlan` section to the `/1.0/networks/NAME/state` API. Those contain additional state information relevant to VLAN interfaces: * `lower_device` * `vid` (extension-instance-nic-bridged-port-isolation)= ## `instance_nic_bridged_port_isolation` This adds the {config:option}`device-nic-bridged-device-conf:security.port_isolation` field for bridged NIC instances. (extension-instance-bulk-state-change)= ## `instance_bulk_state_change` Adds the following endpoint for bulk state change (see [RESTful API](rest-api.md) for details): * `PUT /1.0/instances` (extension-network-gvrp)= ## `network_gvrp` This adds an optional `gvrp` property to `macvlan` and `physical` networks, and to `ipvlan`, `macvlan`, `routed` and `physical` NIC devices. When set, this specifies whether the VLAN should be registered using GARP VLAN Registration Protocol. Defaults to `false`. (extension-instance-pool-move)= ## `instance_pool_move` This adds a `pool` field to the `POST /1.0/instances/NAME` API, allowing for easy move of an instance root disk between pools. (extension-gpu-sriov)= ## `gpu_sriov` This adds support for SR-IOV enabled GPUs. It introduces the `sriov` GPU type property. (extension-pci-device-type)= ## `pci_device_type` This introduces the `pci` device type. (extension-storage-volume-state)= ## `storage_volume_state` Add new `/1.0/storage-pools/POOL/volumes/VOLUME/state` API endpoint to get usage data on a volume. (extension-network-acl)= ## `network_acl` This adds the concept of network ACLs to API under the API endpoint prefix `/1.0/network-acls`. (extension-migration-stateful)= ## `migration_stateful` Add a new {config:option}`instance-migration:migration.stateful` configuration key. (extension-disk-state-quota)= ## `disk_state_quota` This introduces the {config:option}`device-disk-device-conf:size.state` device configuration key on `disk` devices. (extension-storage-ceph-features)= ## `storage_ceph_features` Adds a new {config:option}`storage-ceph-pool-conf:ceph.rbd.features` configuration key on storage pools to control the RBD features used for new volumes. (extension-projects-compression)= ## `projects_compression` Adds new {config:option}`project-specific:backups.compression_algorithm` and {config:option}`project-specific:images.compression_algorithm` configuration keys which allows configuration of backup and image compression per-project. (extension-projects-images-remote-cache-expiry)= ## `projects_images_remote_cache_expiry` Add new {config:option}`server-images:images.remote_cache_expiry` configuration key to projects, allowing for set number of days after which an unused cached remote image will be flushed. (extension-certificate-project)= ## `certificate_project` Adds a new `restricted` property to certificates in the API as well as `projects` holding a list of project names that the certificate has access to. (extension-network-ovn-acl)= ## `network_ovn_acl` Adds a new `security.acls` property to OVN networks and OVN NICs, allowing Network ACLs to be applied. (extension-projects-images-auto-update)= ## `projects_images_auto_update` Adds new {config:option}`server-images:images.auto_update_cached` and {config:option}`server-images:images.auto_update_interval` configuration keys which allows configuration of images auto update in projects (extension-projects-restricted-cluster-target)= ## `projects_restricted_cluster_target` Adds new {config:option}`project-restricted:restricted.cluster.target` configuration key to project which prevent the user from using --target to specify what cluster member to place a workload on or the ability to move a workload between members. (extension-images-default-architecture)= ## `images_default_architecture` Adds new {config:option}`server-images:images.default_architecture` global configuration key and matching per-project key which lets user tell LXD what architecture to go with when no specific one is specified as part of the image request. (extension-network-ovn-acl-defaults)= ## `network_ovn_acl_defaults` Adds new `security.acls.default.{in,e}gress.action` and `security.acls.default.{in,e}gress.logged` configuration keys for OVN networks and NICs. This replaces the removed ACL `default.action` and `default.logged` keys. (extension-gpu-mig)= ## `gpu_mig` This adds support for NVIDIA MIG. It introduces the `mig` GPU type and associated configuration keys. (extension-project-usage)= ## `project_usage` Adds an API endpoint to get current resource allocations in a project. Accessible at API `GET /1.0/projects//state`. (extension-network-bridge-acl)= ## `network_bridge_acl` Adds a new {config:option}`network-bridge-network-conf:security.acls` configuration key to `bridge` networks, allowing Network ACLs to be applied. Also adds `security.acls.default.{in,e}gress.action` and `security.acls.default.{in,e}gress.logged` configuration keys for specifying the default behavior for unmatched traffic. (extension-warnings)= ## `warnings` Warning API for LXD. This includes the following endpoints (see [Restful API](rest-api.md) for details): * `GET /1.0/warnings` * `GET /1.0/warnings/` * `PUT /1.0/warnings/` * `DELETE /1.0/warnings/` (extension-projects-restricted-backups-and-snapshots)= ## `projects_restricted_backups_and_snapshots` Adds new {config:option}`project-restricted:restricted.backups` and {config:option}`project-restricted:restricted.snapshots` configuration keys to project which prevents the user from creation of backups and snapshots. (extension-clustering-join-token)= ## `clustering_join_token` Adds `POST /1.0/cluster/members` API endpoint for requesting a join token used when adding new cluster members without using the trust password. (extension-clustering-description)= ## `clustering_description` Adds an editable description to the cluster members. (extension-server-trusted-proxy)= ## `server_trusted_proxy` This introduces support for {config:option}`server-core:core.https_trusted_proxy` which has LXD parse a HAProxy style connection header on such connections and if present, will rewrite the request's source address to that provided by the proxy server. (extension-clustering-update-cert)= ## `clustering_update_cert` Adds `PUT /1.0/cluster/certificate` endpoint for updating the cluster certificate across the whole cluster (extension-storage-api-project)= ## `storage_api_project` This adds support for copy/move custom storage volumes between projects. (extension-server-instance-driver-operational)= ## `server_instance_driver_operational` This modifies the `driver` output for the `/1.0` endpoint to only include drivers which are actually supported and operational on the server (as opposed to being included in LXD but not operational on the server). (extension-server-supported-storage-drivers)= ## `server_supported_storage_drivers` This adds supported storage driver info to server environment info. (extension-event-lifecycle-requestor-address)= ## `event_lifecycle_requestor_address` Adds a new address field to `lifecycle` requestor. (extension-resources-gpu-usb)= ## `resources_gpu_usb` Add a new `USBAddress` (`usb_address`) field to `ResourcesGPUCard` (GPU entries) in the resources API. (extension-clustering-evacuation)= ## `clustering_evacuation` Adds `POST /1.0/cluster/members//state` endpoint for evacuating and restoring cluster members. It also adds the configuration keys {config:option}`instance-miscellaneous:cluster.evacuate` and {config:option}`instance-volatile:volatile.evacuate.origin` for setting the evacuation method (`auto`, `stop` or `migrate`) and the origin of any migrated instance respectively. (extension-network-ovn-nat-address)= ## `network_ovn_nat_address` This introduces the {config:option}`network-ovn-network-conf:ipv4.nat.address` and {config:option}`network-ovn-network-conf:ipv6.nat.address` configuration keys for LXD `ovn` networks. Those keys control the source address used for outbound traffic from the OVN virtual network. These keys can only be specified when the OVN network's uplink network has {config:option}`network-physical-network-conf:ovn.ingress_mode` set to `routed`. (extension-network-bgp)= ## `network_bgp` This introduces support for LXD acting as a BGP router to advertise routes to `bridge` and `ovn` networks. This comes with the addition to global configuration of: * {config:option}`server-core:core.bgp_address` * {config:option}`server-core:core.bgp_asn` * {config:option}`server-core:core.bgp_routerid` The following network configurations keys (`bridge` and `physical`): * `bgp.peers..address` * `bgp.peers..asn` * `bgp.peers..password` The `nexthop` configuration keys (`bridge`): * {config:option}`network-bridge-network-conf:bgp.ipv4.nexthop` * {config:option}`network-bridge-network-conf:bgp.ipv6.nexthop` And the following NIC-specific configuration keys (`bridged` NIC type): * {config:option}`device-nic-bridged-device-conf:ipv4.routes.external` * {config:option}`device-nic-bridged-device-conf:ipv6.routes.external` (extension-network-forward)= ## `network_forward` This introduces the networking address forward functionality. Allowing for `bridge` and `ovn` networks to define external IP addresses that can be forwarded to internal IP(s) inside their respective networks. (extension-custom-volume-refresh)= ## `custom_volume_refresh` Adds support for refresh during volume migration. (extension-network-counters-errors-dropped)= ## `network_counters_errors_dropped` This adds the received and sent errors as well as inbound and outbound dropped packets to the network counters. (extension-metrics)= ## `metrics` This adds metrics to LXD. It returns metrics of running instances using the OpenMetrics format. This includes the following endpoints: * `GET /1.0/metrics` (extension-image-source-project)= ## `image_source_project` Adds a new `project` field to `POST /1.0/images` allowing for the source project to be set at image copy time. (extension-clustering-config)= ## `clustering_config` Adds new `config` property to cluster members with configurable key/value pairs. (extension-network-peer)= ## `network_peer` This adds network peering to allow traffic to flow between OVN networks without leaving the OVN subsystem. (extension-linux-sysctl)= ## `linux_sysctl` Adds new `linux.sysctl.*` configuration keys allowing users to modify certain kernel parameters within containers. (extension-network-dns)= ## `network_dns` Introduces a built-in DNS server and zones API to provide DNS records for LXD instances. This introduces the following server configuration key: * {config:option}`server-core:core.dns_address` The following network configuration key: * `dns.zone.forward` * `dns.zone.reverse.ipv4` * `dns.zone.reverse.ipv6` And the following project configuration key: * {config:option}`project-restricted:restricted.networks.zones` A new REST API is also introduced to manage DNS zones: * `/1.0/network-zones` (GET, POST) * `/1.0/network-zones/` (GET, PUT, PATCH, DELETE) (extension-ovn-nic-acceleration)= ## `ovn_nic_acceleration` Adds new {config:option}`device-nic-ovn-device-conf:acceleration` configuration key to OVN NICs which can be used for enabling hardware acceleration. It takes the values `none` or `sriov`. (extension-certificate-self-renewal)= ## `certificate_self_renewal` This adds support for renewing a client's own trust certificate. (extension-instance-project-move)= ## `instance_project_move` This adds a `project` field to the `POST /1.0/instances/NAME` API, allowing for easy move of an instance between projects. (extension-storage-volume-project-move)= ## `storage_volume_project_move` This adds support for moving storage volume between projects. (extension-cloud-init)= ## `cloud_init` This adds a new `cloud-init` configuration key namespace which contains the following keys: * {config:option}`instance-cloud-init:cloud-init.vendor-data` * {config:option}`instance-cloud-init:cloud-init.user-data` * {config:option}`instance-cloud-init:cloud-init.network-config` It also adds a new endpoint `/1.0/devices` to `devlxd` which shows an instance's devices. (extension-network-dns-nat)= ## `network_dns_nat` This introduces `network.nat` as a configuration option on network zones (DNS). It defaults to the current behavior of generating records for all instances NICs but if set to `false`, it will instruct LXD to only generate records for externally reachable addresses. (extension-database-leader)= ## `database_leader` Adds new `database-leader` role which is assigned to cluster leader. (extension-instance-all-projects)= ## `instance_all_projects` This adds support for displaying instances from all projects. (extension-clustering-groups)= ## `clustering_groups` Add support for grouping cluster members. This introduces the following new endpoints: * `/1.0/cluster/groups` (GET, POST) * `/1.0/cluster/groups/` (GET, POST, PUT, PATCH, DELETE) The following project restriction is added: * {config:option}`project-restricted:restricted.cluster.groups` (extension-ceph-rbd-du)= ## `ceph_rbd_du` Adds a new {config:option}`storage-ceph-pool-conf:ceph.rbd.du` Boolean on Ceph storage pools which allows disabling the use of the potentially slow `rbd du` calls. (extension-instance-get-full)= ## `instance_get_full` This introduces a new `recursion=1` mode for `GET /1.0/instances/{name}` which allows for the retrieval of all instance structs, including the state, snapshots and backup structs. (extension-qemu-metrics)= ## `qemu_metrics` This adds a new {config:option}`instance-security:security.agent.metrics` Boolean which defaults to `true`. When set to `false`, it doesn't connect to the `lxd-agent` for metrics and other state information, but relies on stats from QEMU. (extension-gpu-mig-uuid)= ## `gpu_mig_uuid` Adds support for the new MIG UUID format used by NVIDIA `470+` drivers (for example, `MIG-74c6a31a-fde5-5c61-973b-70e12346c202`), the `MIG-` prefix can be omitted This extension supersedes old `mig.gi` and `mig.ci` parameters which are kept for compatibility with old drivers and cannot be set together. (extension-event-project)= ## `event_project` Expose the project an API event belongs to. (extension-clustering-evacuation-live)= ## `clustering_evacuation_live` This adds `live-migrate` as a configuration option to {config:option}`instance-miscellaneous:cluster.evacuate`, which forces live-migration of instances during cluster evacuation. (extension-instance-allow-inconsistent-copy)= ## `instance_allow_inconsistent_copy` Adds `allow_inconsistent` field to instance source on `POST /1.0/instances`. If `true`, `rsync` will ignore the `Partial transfer due to vanished source files` (code 24) error when creating an instance from a copy. (extension-network-state-ovn)= ## `network_state_ovn` This adds an `ovn` section to the `/1.0/networks/NAME/state` API which contains additional state information relevant to OVN networks: * chassis (extension-storage-volume-api-filtering)= ## `storage_volume_api_filtering` Adds support for filtering the result of a GET request for storage volumes. (extension-image-restrictions)= ## `image_restrictions` This extension adds on to the image properties to include image restrictions/host requirements. These requirements help determine the compatibility between an instance and the host system. (extension-storage-zfs-export)= ## `storage_zfs_export` Introduces the ability to disable zpool export when unmounting pool by setting {config:option}`storage-zfs-pool-conf:zfs.export`. (extension-network-dns-records)= ## `network_dns_records` This extends the network zones (DNS) API to add the ability to create and manage custom records. This adds: * `GET /1.0/network-zones/ZONE/records` * `POST /1.0/network-zones/ZONE/records` * `GET /1.0/network-zones/ZONE/records/RECORD` * `PUT /1.0/network-zones/ZONE/records/RECORD` * `PATCH /1.0/network-zones/ZONE/records/RECORD` * `DELETE /1.0/network-zones/ZONE/records/RECORD` (extension-storage-zfs-reserve-space)= ## `storage_zfs_reserve_space` Adds ability to set the `reservation`/`refreservation` ZFS property along with `quota`/`refquota`. (extension-network-acl-log)= ## `network_acl_log` Adds a new `GET /1.0/networks-acls/NAME/log` API to retrieve ACL firewall logs. (extension-storage-zfs-blocksize)= ## `storage_zfs_blocksize` Introduces a new {config:option}`storage-zfs-volume-conf:zfs.blocksize` property for ZFS storage volumes which allows to set volume block size. (extension-metrics-cpu-seconds)= ## `metrics_cpu_seconds` This is used to detect whether LXD was fixed to output used CPU time in seconds rather than as milliseconds. (extension-instance-snapshot-never)= ## `instance_snapshot_never` Adds a `@never` option to `snapshots.schedule` which allows disabling inheritance. (extension-certificate-token)= ## `certificate_token` This adds token-based certificate addition to the trust store as a safer alternative to a trust password. It adds the `token` field to `POST /1.0/certificates`. (extension-instance-nic-routed-neighbor-probe)= ## `instance_nic_routed_neighbor_probe` This adds the ability to disable the `routed` NIC IP neighbor probing for availability on the parent network. Adds the {config:option}`device-nic-routed-device-conf:ipv4.neighbor_probe` and {config:option}`device-nic-routed-device-conf:ipv6.neighbor_probe` NIC settings. Defaulting to `true` if not specified. (extension-event-hub)= ## `event_hub` This adds support for `event-hub` cluster member role and the `ServerEventMode` environment field. (extension-agent-nic-config)= ## `agent_nic_config` If set to `true`, on VM start-up the `lxd-agent` will apply NIC configuration to change the names and MTU of the instance NIC devices. (extension-projects-restricted-intercept)= ## `projects_restricted_intercept` Adds new {config:option}`project-restricted:restricted.containers.interception` configuration key to allow usually safe system call interception options. (extension-metrics-authentication)= ## `metrics_authentication` Introduces a new {config:option}`server-core:core.metrics_authentication` server configuration option to allow for the `/1.0/metrics` endpoint to be generally available without client authentication. (extension-images-target-project)= ## `images_target_project` Adds ability to copy image to a project different from the source. (extension-cluster-migration-inconsistent-copy)= ## `cluster_migration_inconsistent_copy` Adds `allow_inconsistent` field to `POST /1.0/instances/`. Set to `true` to allow inconsistent copying between cluster members. (extension-cluster-ovn-chassis)= ## `cluster_ovn_chassis` Introduces a new `ovn-chassis` cluster role which allows for specifying what cluster member should act as an OVN chassis. (extension-container-syscall-intercept-sched-setscheduler)= ## `container_syscall_intercept_sched_setscheduler` Adds the {config:option}`instance-security:security.syscalls.intercept.sched_setscheduler` to allow advanced process priority management in containers. (extension-storage-lvm-thinpool-metadata-size)= ## `storage_lvm_thinpool_metadata_size` Introduces the ability to specify the thin pool metadata volume size via {config:option}`storage-lvm-pool-conf:lvm.thinpool_metadata_size`. If this is not specified then the default is to let LVM pick an appropriate thin pool metadata volume size. (extension-storage-volume-state-total)= ## `storage_volume_state_total` This adds `total` field to the `GET /1.0/storage-pools/{name}/volumes/{type}/{volume}/state` API. (extension-instance-file-head)= ## `instance_file_head` Implements HEAD on `/1.0/instances/NAME/file`. (extension-instances-nic-host-name)= ## `instances_nic_host_name` This introduces the {config:option}`server-miscellaneous:instances.nic.host_name` server configuration key that can take a value of either `random` or `mac`. The default value for the key if unspecified is `random`. If it is set to random then use the random host interface names. If it's set to `mac`, then generate a name in the form `lxd1122334455`. (extension-image-copy-profile)= ## `image_copy_profile` Adds ability to modify the set of profiles when image is copied. (extension-container-syscall-intercept-sysinfo)= ## `container_syscall_intercept_sysinfo` Adds the {config:option}`instance-security:security.syscalls.intercept.sysinfo` to allow the `sysinfo` syscall to be populated with cgroup-based resource usage information. (extension-clustering-evacuation-mode)= ## `clustering_evacuation_mode` This introduces a `mode` field to the evacuation request which allows for overriding the evacuation mode traditionally set through {config:option}`instance-miscellaneous:cluster.evacuate`. (extension-resources-pci-vpd)= ## `resources_pci_vpd` Adds a new VPD struct to the PCI resource entries. This struct extracts vendor provided data including the full product name and additional key/value configuration pairs. (extension-qemu-raw-conf)= ## `qemu_raw_conf` Introduces a {config:option}`instance-raw:raw.qemu.conf` configuration key to override select sections of the generated `qemu.conf`. (extension-storage-cephfs-fscache)= ## `storage_cephfs_fscache` Add support for `fscache`/`cachefilesd` on CephFS pools through a new {config:option}`storage-cephfs-pool-conf:cephfs.fscache` configuration option. (extension-network-load-balancer)= ## `network_load_balancer` This introduces the networking load balancer functionality. Allowing `ovn` networks to define port(s) on external IP addresses that can be forwarded to one or more internal IP(s) inside their respective networks. (extension-vsock-api)= ## `vsock_api` This introduces a bidirectional `vsock` interface which allows the `lxd-agent` and the LXD server to communicate better. (extension-instance-ready-state)= ## `instance_ready_state` This introduces a new `Ready` state for instances which can be set using `devlxd`. (extension-network-bgp-holdtime)= ## `network_bgp_holdtime` This introduces a new `bgp.peers..holdtime` configuration key to control the BGP hold time for a particular peer. (extension-storage-volumes-all-projects)= ## `storage_volumes_all_projects` This introduces the ability to list storage volumes from all projects. (extension-metrics-memory-oom-total)= ## `metrics_memory_oom_total` This introduces a new `lxd_memory_OOM_kills_total` metric to the `/1.0/metrics` API. It reports the number of times the out of memory killer (`OOM`) has been triggered. (extension-storage-buckets)= ## `storage_buckets` This introduces the storage bucket API. It allows the management of S3 object storage buckets for storage pools. (extension-storage-buckets-create-credentials)= ## `storage_buckets_create_credentials` This updates the storage bucket API to return initial admin credentials at bucket creation time. (extension-metrics-cpu-effective-total)= ## `metrics_cpu_effective_total` This introduces a new `lxd_cpu_effective_total` metric to the `/1.0/metrics` API. It reports the total number of effective CPUs. (extension-projects-networks-restricted-access)= ## `projects_networks_restricted_access` Adds the {config:option}`project-restricted:restricted.networks.access` project configuration key to indicate (as a comma-delimited list) which networks can be accessed inside the project. If not specified, all networks are accessible (assuming it is also allowed by the {config:option}`project-restricted:restricted.devices.nic` setting, described below). This also introduces a change whereby network access is controlled by the project's {config:option}`project-restricted:restricted.devices.nic` setting: * If `restricted.devices.nic` is set to `managed` (the default if not specified), only managed networks are accessible. * If `restricted.devices.nic` is set to `allow`, all networks are accessible (dependent on the `restricted.networks.access` setting). * If `restricted.devices.nic` is set to `block`, no networks are accessible. (extension-loki)= ## `loki` This adds support for sending events to a Loki server. It adds the following global configuration keys: * {config:option}`server-loki:loki.api.ca_cert`: CA certificate which can be used when sending events to the Loki server * {config:option}`server-loki:loki.api.url`: URL to the Loki server (protocol, name or IP and port) * {config:option}`server-loki:loki.auth.username` and {config:option}`server-loki:loki.auth.password`: Used if Loki is behind a reverse proxy with basic authentication enabled * {config:option}`server-loki:loki.labels`: Comma-separated list of values which are to be used as labels for Loki events. * {config:option}`server-loki:loki.loglevel`: Minimum log level for events sent to the Loki server. * {config:option}`server-loki:loki.types`: Types of events which are to be sent to the Loki server (any combination of `lifecycle`, `logging`, `ovn`, and `security`). (extension-acme)= ## `acme` This adds ACME support, which allows [Let's Encrypt](https://letsencrypt.org/) or other ACME services to issue certificates. It adds the following global configuration keys: * {config:option}`server-acme:acme.domain`: The domain for which the certificate should be issued. * {config:option}`server-acme:acme.email`: The email address used for the account of the ACME service. * {config:option}`server-acme:acme.ca_url`: The directory URL of the ACME service, defaults to `https://acme-v02.api.letsencrypt.org/directory`. It also adds the following endpoint, which is required for the HTTP-01 challenge: * `/.well-known/acme-challenge/` (extension-internal-metrics)= ## `internal_metrics` This adds internal metrics to the list of metrics. These include: * Total running operations * Total active warnings * Daemon uptime in seconds * Go memory stats * Number of goroutines (extension-cluster-join-token-expiry)= ## `cluster_join_token_expiry` This adds an expiry to cluster join tokens which defaults to 3 hours, but can be changed by setting the {config:option}`server-cluster:cluster.join_token_expiry` configuration key. (extension-remote-token-expiry)= ## `remote_token_expiry` This adds an expiry to remote add join tokens. It can be set in the {config:option}`server-core:core.remote_token_expiry` configuration key, and defaults to 15 days. (extension-init-preseed)= ## `init_preseed` Adds the `InitPreseed` type and related exported types (such as `InitLocalPreseed`, `InitClusterPreseed`, and `InitNetworksProjectPost`) to `shared/api`, formalizing the YAML structure accepted by `lxd init --preseed` so that it can be generated and parsed by other tools. (extension-storage-volumes-created-at)= ## `storage_volumes_created_at` This change adds support for storing the creation date and time of storage volumes and their snapshots. This adds the `CreatedAt` field to the `StorageVolume` and `StorageVolumeSnapshot` API types. (extension-cpu-hotplug)= ## `cpu_hotplug` This adds CPU hotplugging for VMs. Hotplugging is disabled when using CPU pinning, because this would require hotplugging NUMA devices as well, which is not possible. (extension-projects-networks-zones)= ## `projects_networks_zones` This adds support for the {config:option}`project-features:features.networks.zones` project feature, which changes which project network zones are associated with when they are created. Previously network zones were tied to the value of {config:option}`project-features:features.networks`, meaning they were created in the same project as networks were. Now this has been decoupled from {config:option}`project-features:features.networks` to allow projects that share a network in the default project (i.e those with `features.networks=false`) to have their own project level DNS zones that give a project oriented "view" of the addresses on that shared network (which only includes addresses from instances in their project). This also introduces a change to the network `dns.zone.forward` setting, which now accepts a comma-separated of DNS zone names (a maximum of one per project) in order to associate a shared network with multiple zones. No change to the `dns.zone.reverse.*` settings have been made, they still only allow a single DNS zone to be set. However the resulting zone content that is generated now includes `PTR` records covering addresses from all projects that are referencing that network via one of their forward zones. Existing projects that have `features.networks=true` will have `features.networks.zones=true` set automatically, but new projects will need to specify this explicitly. (extension-network-txqueuelen)= ## `network_txqueuelen` Adds a `txqueuelen` key to control the `txqueuelen` parameter of the NIC device. (extension-cluster-member-state)= ## `cluster_member_state` Adds `GET /1.0/cluster/members//state` API endpoint and associated `ClusterMemberState` API response type. (extension-storage-pool-source-wipe)= ## `storage_pool_source_wipe` Adds support for a `source.wipe` Boolean on the storage pool, indicating that LXD should wipe partition headers off the requested disk rather than potentially fail due to pre-existing file systems. (extension-zfs-block-mode)= ## `zfs_block_mode` This adds support for using ZFS block {spellexception}`filesystem` volumes allowing the use of different file systems on top of ZFS. This adds the following new configuration options for ZFS storage pools: * `volume.zfs.block_mode` * `volume.block.mount_options` * `volume.block.filesystem` (extension-instance-generation-id)= ## `instance_generation_id` Adds support for instance generation ID. The VM or container generation ID will change whenever the instance's place in time moves backwards. As of now, the generation ID is only exposed through to VM type instances. This allows for the VM guest OS to reinitialize any state it needs to avoid duplicating potential state that has already occurred: * {config:option}`instance-volatile:volatile.uuid.generation` (extension-disk-io-cache)= ## `disk_io_cache` This introduces a new {config:option}`device-disk-device-conf:io.cache` property to disk devices which can be used to override the VM caching behavior. (extension-amd-sev)= ## `amd_sev` Adds support for AMD SEV (Secure Encrypted Virtualization) that can be used to encrypt the memory of a guest VM. This adds the following new configuration options for SEV encryption: * {config:option}`instance-security:security.sev` : (bool) is SEV enabled for this VM * {config:option}`instance-security:security.sev.policy.es` : (bool) is SEV-ES enabled for this VM * {config:option}`instance-security:security.sev.session.dh` : (string) guest owner's `base64`-encoded Diffie-Hellman key * {config:option}`instance-security:security.sev.session.data` : (string) guest owner's `base64`-encoded session blob (extension-storage-pool-loop-resize)= ## `storage_pool_loop_resize` This allows growing loop file backed storage pools by changing the `size` setting of the pool. (extension-migration-vm-live)= ## `migration_vm_live` This adds support for performing VM QEMU to QEMU live migration for both shared storage (clustered Ceph) and non-shared storage pools. This also adds the `CRIUType_VM_QEMU` value of `3` for the migration `CRIUType` `protobuf` field. (extension-ovn-nic-nesting)= ## `ovn_nic_nesting` This adds support for nesting an `ovn` NIC inside another `ovn` NIC on the same instance. This allows for an OVN logical switch port to be tunneled inside another OVN NIC using VLAN tagging. This feature is configured by specifying the parent NIC name using the {config:option}`device-nic-ovn-device-conf:nested` property and the VLAN ID to use for tunneling with the {config:option}`device-nic-ovn-device-conf:vlan` property. (extension-oidc)= ## `oidc` This adds support for OpenID Connect (OIDC) authentication. This adds the following new configuration keys: * {config:option}`server-oidc:oidc.issuer` * {config:option}`server-oidc:oidc.client.id` * {config:option}`server-oidc:oidc.audience` (extension-network-ovn-l3only)= ## `network_ovn_l3only` This adds the ability to set an `ovn` network into "layer 3 only" mode. This mode can be enabled at IPv4 or IPv6 level using {config:option}`network-ovn-network-conf:ipv4.l3only` and {config:option}`network-ovn-network-conf:ipv6.l3only` configuration options respectively. With this mode enabled the following changes are made to the network: * The virtual router's internal port address will be configured with a single host netmask (e.g. /32 for IPv4 or /128 for IPv6). * Static routes for active instance NIC addresses will be added to the virtual router. * A discard route for the entire internal subnet will be added to the virtual router to prevent packets destined for inactive addresses from escaping to the uplink network. * The DHCPv4 server will be configured to indicate that a netmask of 255.255.255.255 be used for instance configuration. (extension-ovn-nic-acceleration-vdpa)= ## `ovn_nic_acceleration_vdpa` This updates the `ovn_nic_acceleration` API extension. The {config:option}`device-nic-ovn-device-conf:acceleration` configuration key for OVN NICs can now takes the value `vdpa` to support Virtual Data Path Acceleration (VDPA). (extension-instances-state-total)= ## `instances_state_total` This extension adds a new `total` field to `InstanceStateDisk` and `InstanceStateMemory`, both part of the instance's state API. (extension-auth-user)= ## `auth_user` Add current user details to the main API endpoint. This introduces: * `auth_user_name` * `auth_user_method` (extension-security-csm)= ## `security_csm` Introduce a new `instance-security:security.csm` configuration key to control the use of `CSM` (Compatibility Support Module) to allow legacy operating systems to be run in LXD VMs. ```{note} The `security.csm` key has been replaced by `boot.mode`. See {ref}`extension-instance-boot-mode`. ``` (extension-instances-rebuild)= ## `instances_rebuild` This extension adds the ability to rebuild an instance with the same origin image, alternate image or as empty. A new `POST /1.0/instances//rebuild?project=` API endpoint has been added as well as a new CLI command [`lxc rebuild`](lxc_rebuild.md). (extension-numa-cpu-placement)= ## `numa_cpu_placement` This adds the possibility to place a set of CPUs in a desired set of NUMA nodes. This adds the following new configuration key: * {config:option}`instance-resource-limits:limits.cpu.nodes` : (string) comma-separated list of NUMA node IDs or NUMA node ID ranges to place the CPUs (chosen with a dynamic value of {config:option}`instance-resource-limits:limits.cpu`) in. (extension-custom-volume-iso)= ## `custom_volume_iso` This adds the possibility to import ISO images as custom storage volumes. This adds the `--type` flag to [`lxc storage volume import`](lxc_storage_volume_import.md). (extension-network-allocations)= ## `network_allocations` This adds the possibility to list a LXD deployment's network allocations. Through the [`lxc network list-allocations`](lxc_network_list-allocations.md) command and the `--project | --all-projects` flags, you can list all the used IP addresses, hardware addresses (for instances), resource URIs and whether it uses NAT for each `instance`, `network`, `network forward` and `network load-balancer`. (extension-storage-api-remote-volume-snapshot-copy)= ## `storage_api_remote_volume_snapshot_copy` This allows copying storage volume snapshots to and from remotes. (extension-zfs-delegate)= ## `zfs_delegate` This implements a new {config:option}`storage-zfs-volume-conf:zfs.delegate` volume Boolean for volumes on a ZFS storage driver. When enabled and a suitable system is in use (requires ZFS 2.2 or higher), the ZFS dataset will be delegated to the container, allowing for its use through the `zfs` command line tool. (extension-operations-get-query-all-projects)= ## `operations_get_query_all_projects` This introduces support for the `all-projects` query parameter for the GET API calls to both `/1.0/operations` and `/1.0/operations?recursion=1`. This parameter allows bypassing the project name filter. (extension-metadata-configuration)= ## `metadata_configuration` Adds the `GET /1.0/metadata/configuration` API endpoint to retrieve the generated metadata configuration in a JSON format. The JSON structure adopts the structure ```"configs" > `ENTITY` > `ENTITY_SECTION` > "keys" > [, , ...]```. Check the list of {doc}`configuration options ` to see which configuration options are included. (extension-syslog-socket)= ## `syslog_socket` This introduces a syslog socket that can receive syslog formatted log messages. These can be viewed in the events API and `lxc monitor`, and can be forwarded to Loki. To enable this feature, set {config:option}`server-core:core.syslog_socket` to `true`. (extension-event-lifecycle-name-and-project)= ## `event_lifecycle_name_and_project` This adds the fields `Name` and `Project` to `lifecycle` events. (extension-instances-nic-limits-priority)= ## `instances_nic_limits_priority` This introduces a new per-NIC `limits.priority` option that works with both cgroup1 and cgroup2 unlike the deprecated `limits.network.priority` instance setting, which only worked with cgroup1. (extension-disk-initial-volume-configuration)= ## `disk_initial_volume_configuration` This API extension provides the capability to set initial volume configurations for instance root devices. Initial volume configurations are prefixed with `initial.` and can be specified either through profiles or directly during instance initialization using the `--device` flag. Note that these configuration are applied only at the time of instance creation and subsequent modifications have no effect on existing devices. (extension-operation-wait)= ## `operation_wait` This API extension indicates that the `/1.0/operations/{id}/wait` endpoint exists on the server. This indicates to the client that the endpoint can be used to wait for an operation to complete rather than waiting for an operation event via the `/1.0/events` endpoint. (extension-cluster-internal-custom-volume-copy)= ## `cluster_internal_custom_volume_copy` This extension adds support for copying and moving custom storage volumes within a cluster with a single API call. Calling `POST /1.0/storage-pools//custom?target=` will copy the custom volume specified in the `source` part of the request. Calling `POST /1.0/storage-pools//custom/?target=` will move the custom volume from the source, specified in the `source` part of the request, to the target. (extension-disk-io-bus)= ## `disk_io_bus` This introduces a new {config:option}`device-disk-device-conf:io.bus` property to disk devices which can be used to override the bus the disk is attached to. (extension-storage-cephfs-create-missing)= ## `storage_cephfs_create_missing` This introduces the configuration keys {config:option}`storage-cephfs-pool-conf:cephfs.create_missing`, {config:option}`storage-cephfs-pool-conf:cephfs.osd_pg_num`, {config:option}`storage-cephfs-pool-conf:cephfs.meta_pool` and {config:option}`storage-cephfs-pool-conf:cephfs.data_pool` to be used when adding a `cephfs` storage pool to instruct LXD to create the necessary entities for the storage pool, if they do not exist. (extension-instance-move-config)= ## `instance_move_config` This API extension provides the ability to use flags `--profile`, `--no-profile`, `--device`, and `--config` when moving an instance between projects and/or storage pools. (extension-ovn-ssl-config)= ## `ovn_ssl_config` This introduces new server configuration keys to provide the SSL CA and client key pair to access the OVN databases. The new configuration keys are {config:option}`server-miscellaneous:network.ovn.ca_cert`, {config:option}`server-miscellaneous:network.ovn.client_cert` and {config:option}`server-miscellaneous:network.ovn.client_key`. (extension-init-preseed-storage-volumes)= ## `init_preseed_storage_volumes` This API extension provides the ability to configure storage volumes in preseed init. (extension-metrics-instances-count)= ## `metrics_instances_count` This extends the metrics to include the containers and virtual machines counts. Instances are counted irrespective of their state. (extension-server-instance-type-info)= ## `server_instance_type_info` This API extension enables querying a server's supported instance types. When querying the `/1.0` endpoint, a new field named `instance_types` is added to the retrieved data. This field indicates which instance types are supported by the server. (extension-resources-disk-mounted)= ## `resources_disk_mounted` Adds a `mounted` field to disk resources that LXD discovers on the system, reporting whether that disk or partition is mounted. (extension-server-version-lts)= ## `server_version_lts` The API extension adds indication whether the LXD version is an LTS release. This is indicated when command `lxc version` is executed or when `/1.0` endpoint is queried. (extension-oidc-groups-claim)= ## `oidc_groups_claim` This API extension enables setting an {config:option}`server-oidc:oidc.groups.claim` configuration key. If OIDC authentication is configured and this claim is set, LXD will request this claim in the scope of OIDC flow. The value of the claim will be extracted and might be used to make authorization decisions. (extension-loki-config-instance)= ## `loki_config_instance` Adds a new {config:option}`server-loki:loki.instance` server configuration key to customize the `instance` field in Loki events. This can be used to expose the name of the cluster rather than the individual system name sending the event as that's usually already covered by the `location` field. (extension-storage-volatile-uuid)= ## `storage_volatile_uuid` Adds a new `volatile.uuid` configuration key to all storage volumes, snapshots and buckets. This information can be used by storage drivers as a separate identifier besides the name when working with volumes. (extension-import-instance-devices)= ## `import_instance_devices` This API extension provides the ability to use flags `--device` when importing an instance to override instance's devices. (extension-instances-uefi-vars)= ## `instances_uefi_vars` This API extension indicates that the `/1.0/instances/{name}/uefi-vars` endpoint is supported on the server. This endpoint allows to get the full list of UEFI variables (HTTP method GET) or replace the entire set of UEFI variables (HTTP method PUT). (extension-instances-migration-stateful)= ## `instances_migration_stateful` This API extension allows newly created VMs to have their {config:option}`instance-migration:migration.stateful` configuration key automatically set through the new server-level configuration key {config:option}`server-miscellaneous:instances.migration.stateful`. If `migration.stateful` is already set at the profile or instance level then `instances.migration.stateful` is not applied. (extension-access-management)= ## `access_management` Adds new APIs under `/1.0/auth` for viewing and managing identities, groups, and permissions. Adds an embedded OpenFGA authorization driver for enforcing fine-grained permissions. ```{important} Prior to the addition of this extension, all OIDC clients were given full access to LXD (equivalent to Unix socket access). This extension revokes access to all OIDC clients. To regain access, a user must: 1. Make a call to the OIDC enabled LXD remote (e.g. `lxc info`) to ensure that their OIDC identity is added to the LXD database. 2. Create a group: `lxc auth group create ` 3. Grant the group a suitable permission. As all OIDC clients prior to this extension have had full access to LXD, the corresponding permission is `admin` on `server`. To grant this permission to your group, run: `lxc auth group permission add server admin` 4. Add themselves to the group. To do this, run: `lxc auth identity group add oidc/ ` Steps 2 to 4 above cannot be performed via OIDC authentication (access has been revoked). They must be performed by a sufficiently privileged user, either via Unix socket or unrestricted TLS client certificate. For more information on access control for OIDC clients, see {ref}`fine-grained-authorization`. ``` (extension-vm-disk-io-limits)= ## `vm_disk_io_limits` Adds the ability to limit disk I/O for virtual machines. (extension-storage-volumes-all)= ## `storage_volumes_all` This API extension adds support for listing storage volumes from all storage pools via `/1.0/storage-volumes` or `/1.0/storage-volumes/{type}` to filter by volume type. Also adds a `pool` field to storage volumes. (extension-instances-files-modify-permissions)= ## `instances_files_modify_permissions` Adds the ability for `POST /1.0/instances/{name}/files` to modify the permissions of files that already exist via the `X-LXD-modify-perm` header. `X-LXD-modify-perm` should be a comma-separated list of 0 or more of `mode`, `uid`, and `gid`. (extension-image-restriction-nesting)= ## `image_restriction_nesting` This extension adds a new image restriction, `requirements.nesting` which when `true` indicates that an image cannot be run without nesting. (extension-container-syscall-intercept-finit-module)= ## `container_syscall_intercept_finit_module` Adds the {config:option}`instance-miscellaneous:linux.kernel_modules.load` container configuration option. If the option is set to `ondemand`, the `finit_modules()` syscall is intercepted and a privileged user in the container's user namespace can load the Linux kernel modules specified in the allow list {config:option}`instance-miscellaneous:linux.kernel_modules`. (extension-device-usb-serial)= ## `device_usb_serial` This adds new configuration keys {config:option}`device-unix-usb-device-conf:serial`, {config:option}`device-unix-usb-device-conf:busnum` and {config:option}`device-unix-usb-device-conf:devnum` for [device type `usb`](devices-usb). The feature has been added to make it possible to distinguish between devices with identical {config:option}`device-unix-usb-device-conf:vendorid` and {config:option}`device-unix-usb-device-conf:productid`. (extension-network-allocate-external-ips)= ## `network_allocate_external_ips` Adds the ability to use an unspecified IPv4 (`0.0.0.0`) or IPv6 (`::`) address in the `listen_address` field of the request body for `POST /1.0/networks/{networkName}/load-balancers` and `POST /1.0/networks/{networkName}/forwards`. If an unspecified IP address is used, supported drivers will allocate an available listen address automatically. Allocation of external IP addresses is currently supported by the OVN network driver. The OVN driver will allocate IP addresses from the subnets specified in the uplink network's `ipv4.routes` and `ipv6.routes` configuration options. (extension-explicit-trust-token)= ## `explicit_trust_token` Adds the ability to explicitly specify a trust token when creating a certificate and joining an existing cluster. (extension-shared-custom-block-volumes)= ## `shared_custom_block_volumes` This adds a configuration key `security.shared` to custom block volumes. If unset or `false`, the custom block volume cannot be attached to multiple instances. This feature was added to prevent data loss which can happen when custom block volumes are attached to multiple instances at once. (extension-instance-import-conversion)= ## `instance_import_conversion` Adds the ability to convert images from different formats (e.g. VMDK or QCow2) into RAW image format and import them as LXD instances. (extension-instance-create-start)= ## `instance_create_start` Adds a `start` field to the `POST /1.0/instances` API which when set to `true` will have the instance automatically start upon creation. In this scenario, the creation and startup is part of a single background operation. (extension-instance-protection-start)= ## `instance_protection_start` Enables setting the {config:option}`instance-security:security.protection.start` field which prevents instances from being started if set to `true`. (extension-devlxd-images-vm)= ## `devlxd_images_vm` Enables the {config:option}`instance-security:security.devlxd.images` configuration option for virtual machines. This controls the availability of a `/1.0/images/FINGERPRINT/export` API over `devlxd`. This can be used by a virtual machine running LXD to access raw images from the host. (extension-disk-io-bus-virtio-blk)= ## `disk_io_bus_virtio_blk` Adds a new `virtio-blk` value for `io.bus` on `disk` devices which allows for the attached disk to be connected to the `virtio-blk` bus. (extension-metrics-api-requests)= ## `metrics_api_requests` Adds the following internal metrics: * Total completed requests * Number of ongoing requests (extension-projects-limits-disk-pool)= ## `projects_limits_disk_pool` This introduces per-pool project disk limits, introducing a `limits.disk.pool.NAME` configuration option to the project limits. When `limits.disk.pool.POOLNAME: 0` for a project, the pool is excluded from `lxc storage list` in that project. (extension-ubuntu-pro-guest-attach)= ## `ubuntu_pro_guest_attach` Adds a new {config:option}`instance-miscellaneous:ubuntu_pro.guest_attach` configuration option for instances. When set to `on`, if the host has guest attachment enabled, the guest can request a guest token for Ubuntu Pro via `devlxd`. For more information, see: {ref}`instances-ubuntu-pro-attach`. (extension-metadata-configuration-entity-types)= ## `metadata_configuration_entity_types` This adds entity type metadata to `GET /1.0/metadata/configuration`. The entity type metadata is a JSON object under the `entities` key. (extension-access-management-tls)= ## `access_management_tls` Expands APIs under `/1.0/auth` to include: 1. Creation of fine-grained TLS identities, whose permissions are managed via group membership. This is performed via `POST /1.0/auth/identities/tls`. If the request body contains `{"token": true}`, a token will be returned that may be used by a non-authenticated caller to gain trust with the LXD server (the caller must send their certificate during the TLS handshake). If the request body contains `{"certificate": ""}"`, the identity will be created directly. The request body may also specify an array of group names. The caller must have `can_create_identities` on `server`. 1. Deletion of OIDC and fine-grained TLS identities. This is performed via `DELETE /1.0/auth/identities/tls/{nameOrFingerprint}` or `DELETE /1.0/auth/identities/oidc/{nameOrEmailAddress}`. The caller must have `can_delete` on the identity. All identities may delete their own identity. For OIDC identities this revokes all access but does not revoke trust (authentication is performed by the identity provider). For fine-grained TLS identities, this revokes all access and revokes trust. 1. Functionality to update the certificate of a fine-grained TLS identity. This is performed via `PUT /1.0/auth/identities/tls/{nameOrFingerprint}` or `PATCH /1.0/auth/identities/tls/{nameOrFingerprint}`. The caller must provide a base64 encoded x509 certificate in the `certificate` field of the request body. Fine-grained TLS identities may update their own certificate. To update the certificate of another identity, the caller must have `can_edit` on the identity. (extension-network-allocations-ovn-uplink)= ## `network_allocations_ovn_uplink` Includes OVN virtual routers external IPs to `/1.0/network-allocations` responses with the type `uplink`. Introduces the `network` field on each allocation, indicating to which network each allocated address belongs. And lastly, adds a `project` field on leases, leases can be retrieved via `/1.0/networks//leases`. (extension-network-ovn-uplink-vlan)= ## `network_ovn_uplink_vlan` Adds support for using a bridge network with a specified VLAN ID as an OVN uplink. (extension-state-logical-cpus)= ## `state_logical_cpus` Adds `logical_cpus` field to `GET /1.0/cluster/members/{name}/state` which contains the total available logical CPUs available when LXD started. (extension-vm-limits-cpu-pin-strategy)= ## `vm_limits_cpu_pin_strategy` Adds a new {config:option}`instance-resource-limits:limits.cpu.pin_strategy` configuration option for virtual machines. This option controls the CPU pinning strategy. When set to `none`, CPU auto pinning is disabled. When set to `auto`, CPU auto pinning is enabled. (extension-gpu-cdi)= ## `gpu_cdi` Adds support for using the Container Device Interface (CDI) specification to configure GPU passthrough in LXD containers. The `id` field of GPU devices now accepts CDI identifiers (for example, `{VENDOR_DOMAIN_NAME}/gpu=gpu{INDEX}`) for containers, in addition to DRM card IDs. This enables GPU passthrough for devices that don't use PCI addressing (like NVIDIA Tegra iGPUs) and provides a more flexible way to identify and configure GPU devices. (extension-images-all-projects)= ## `images_all_projects` This adds support for listing images across all projects using the `all-projects` parameter in `GET /1.0/images` requests. (extension-metadata-configuration-scope)= ## `metadata_configuration_scope` This adds scope metadata to `GET /1.0/metadata/configuration`. Options marked with a `global` scope are applied to all cluster members. Options with a `local` scope must be set on a per-member basis. (extension-unix-device-hotplug-ownership-inherit)= ## `unix_device_hotplug_ownership_inherit` Adds a new {config:option}`device-unix-hotplug-device-conf:ownership.inherit` configuration option for `unix-hotplug` devices. This option controls whether the device inherits ownership (GID and/or UID) from the host. When set to `true` and GID and/or UID are unset, host ownership is inherited. When set to `false`, host ownership is not inherited and ownership can be configured by setting {config:option}`device-unix-hotplug-device-conf:gid` and {config:option}`device-unix-hotplug-device-conf:uid`. (extension-unix-device-hotplug-subsystem-device-option)= ## `unix_device_hotplug_subsystem_device_option` Adds a new {config:option}`device-unix-hotplug-device-conf:subsystem` configuration option for `unix-hotplug` devices. This adds support for detecting `unix-hotplug` devices by subsystem, and can be used in conjunction with {config:option}`device-unix-hotplug-device-conf:productid` and {config:option}`device-unix-hotplug-device-conf:vendorid`. (extension-storage-ceph-osd-pool-size)= ## `storage_ceph_osd_pool_size` This introduces the configuration keys {config:option}`storage-ceph-pool-conf:ceph.osd.pool_size`, and {config:option}`storage-cephfs-pool-conf:cephfs.osd_pool_size` to be used when adding or updating a `ceph` or `cephfs` storage pool to instruct LXD to create set the replication size for the underlying OSD pools. (extension-network-get-target)= ## `network_get_target` Adds optional `target` parameter to `GET /1.0/network`. When target is set, forward the request to the specified cluster member and return the non-managed interfaces from that member. (extension-network-zones-all-projects)= ## `network_zones_all_projects` This adds support for listing network zones across all projects using the `all-projects` parameter in `GET /1.0/network-zones` requests. (extension-vm-root-volume-attachment)= ## `vm_root_volume_attachment` Adds support for virtual-machine root volumes and snapshots to be attached to other instances as disk devices. Introduces the `source.type` and `source.snapshot` keys for disk devices. (extension-projects-limits-uplink-ips)= ## `projects_limits_uplink_ips` Introduces per-project uplink IP limits for each available uplink network, adding `limits.networks.uplink_ips.ipv4.NETWORK_NAME` and `limits.networks.uplink_ips.ipv6.NETWORK_NAME` configuration keys for projects with `features.networks` enabled. These keys define the maximum value of IPs made available on a network named NETWORK_NAME to be assigned as uplink IPs for entities inside a certain project. These entities can be other networks, network forwards or load balancers. (extension-entities-with-entitlements)= ## `entities_with_entitlements` Adds `fine_grained` field to `GET /1.0/auth/identities/current` to indicate if the current identity interacting with the LXD API is fine-grained (that is, associated permissions are managed via group membership). Allows LXD entities to be returned with an `access_entitlements` field if the current identity is fine-grained and the GET request to fetch the LXD entities has the `with-access-entitlements=` query parameter. (extension-profiles-all-projects)= ## `profiles_all_projects` This adds support for listing profiles across all projects using the `all-projects` parameter in `GET /1.0/profiles` requests. (extension-storage-driver-powerflex)= ## `storage_driver_powerflex` Adds a new `powerflex` storage driver which allows the consumption of storage volumes from a Dell PowerFlex storage array using NVMe/TCP and SDC. The following new pool level configuration keys have been added: 1. {config:option}`storage-powerflex-pool-conf:powerflex.snapshot_copy` 1. {config:option}`storage-powerflex-pool-conf:powerflex.domain` 1. {config:option}`storage-powerflex-pool-conf:powerflex.gateway` 1. {config:option}`storage-powerflex-pool-conf:powerflex.gateway.verify` 1. {config:option}`storage-powerflex-pool-conf:powerflex.mode` 1. {config:option}`storage-powerflex-pool-conf:powerflex.pool` 1. {config:option}`storage-powerflex-pool-conf:powerflex.sdt` 1. {config:option}`storage-powerflex-pool-conf:powerflex.user.name` 1. {config:option}`storage-powerflex-pool-conf:powerflex.user.password` The following configuration keys have been added for volumes backed by PowerFlex: 1. {config:option}`storage-powerflex-volume-conf:block.type` (extension-storage-driver-pure)= ## `storage_driver_pure` Adds a new `pure` storage driver which allows the consumption of storage volumes from an Everpure (formerly known as Pure Storage) FlashArray using either iSCSI or NVMe/TCP. The following pool level configuration keys have been added: 1. {config:option}`storage-pure-pool-conf:pure.gateway` 1. {config:option}`storage-pure-pool-conf:pure.gateway.verify` 1. {config:option}`storage-pure-pool-conf:pure.api.token` 1. {config:option}`storage-pure-pool-conf:pure.mode` 1. {config:option}`storage-pure-pool-conf:pure.target` (extension-cloud-init-ssh-keys)= ## `cloud_init_ssh_keys` Adds support for injecting additional SSH public keys into instances through {ref}`cloud-init ` without conflicting with any configuration present on {config:option}`instance-cloud-init:cloud-init.vendor-data` or {config:option}`instance-cloud-init:cloud-init.user-data`. To achieve this, the `cloud-init.ssh-keys.KEYNAME` configuration key is added for both instances and profiles. This key is used to define a public key to be injected. `KEYNAME` can be any arbitrary name for the injected key. The value for `cloud-init.ssh-keys.KEYNAME` should be `:`, where `` is the name of the user for whom to inject the key. For ``, provide either the public key or a `cloud-init` import ID for a key hosted elsewhere. Example valid values for `cloud-init.ssh-keys.KEYNAME` are `root:gh:githubUser` or `myUser:ssh-keyAlg base64PublicKey`. (extension-oidc-scopes)= ## `oidc_scopes` This API extension enables setting an {config:option}`server-oidc:oidc.scopes` configuration key, which accepts a space-separated list of OIDC scopes to request from the identity provider. This configuration option can be used to request additional scopes that might be required for retrieving {ref}`identity provider groups ` from the identity provider. Additionally, the optional scopes `profile` and `offline_access` can be unset via this setting. Note that the `openid` and `email` scopes are always required. (extension-project-default-network-and-storage)= ## `project_default_network_and_storage` Adds flags --network and --storage. The --network flag adds a network device connected to the specified network to the default profile. The --storage flag adds a root disk device using the specified storage pool to the default profile. (extension-client-cert-presence)= ## `client_cert_presence` Adds the field `client_certificate` to `GET /1.0` to indicate if the current request has a client certificate in it. This is for informational purposes only and does not affect the behavior of the API. (extension-clustering-groups-used-by)= ## `clustering_groups_used_by` This API extension adds a `used_by` field to the API response for a {ref}`cluster group `. Deletion of a cluster group is disallowed if the cluster group is referenced by project configuration (see {config:option}`project-restricted:restricted.cluster.groups`). (extension-container-bpf-delegation)= ## `container_bpf_delegation` Adds new {config:option}`instance-security:security.delegate_bpf`.* group of options in order to support eBPF delegation using BPF Token mechanism. See {ref}`bpf-delegation-token` for more information. (extension-override-snapshot-profiles-on-copy)= ## `override_snapshot_profiles_on_copy` This adds a request option to set snapshot's target profile on instance copy to be inherited from target instance. (extension-resources-device-fs-uuid)= ## `resources_device_fs_uuid` Adds the field `device_fs_uuid` including the respective UUID to each disk and partition indicating whether or not a filesystem is located on the device. (extension-backup-metadataversion)= (extension-backup-metadata-version)= ## `backup_metadata_version` Adds the field `version` when exporting instances and custom storage volumes to define the backup file format. In case the field is omitted, the server's default version is used. This maintains backwards compatibility with older clients. When exporting an instance, the specific version can be provided using the `--export-version` flag: `lxc export v1 --export-version 2` The same applies when exporting a custom storage volume: `lxc storage volume export pool1 vol1 --export-version 2` (extension-storage-buckets-all-projects)= ## `storage_buckets_all_projects` This adds support for listing storage buckets across all projects using the `all-projects` parameter in `GET /1.0/storage-pools/POOL/buckets` requests. (extension-network-acls-all-projects)= ## `network_acls_all_projects` This adds support for listing network ACLs across all projects using the `all-projects` parameter in `GET /1.0/network-acls` requests. (extension-networks-all-projects)= ## `networks_all_projects` This adds support for listing networks across all projects using the `all-projects` parameter in `GET /1.0/networks` requests. (extension-clustering-restore-skip-mode)= ## `clustering_restore_skip_mode` Adds a `skip` mode to the restore request. This mode restores a cluster member's status to `ONLINE` without restarting any of its stopped local instances or migrating back instances that were evacuated to other cluster members. (extension-disk-io-threads-virtiofsd)= ## `disk_io_threads_virtiofsd` Adds the {config:option}`device-disk-device-conf:io.threads` option on `disk` devices which is used to control the `virtiofsd` thread pool size when sharing file systems into VMs. This can help improve I/O performance. (extension-oidc-client-secret)= ## `oidc_client_secret` This adds support for the {config:option}`server-oidc:oidc.client.secret` configuration key. If set, the LXD server will use this value in the OpenID Connect (OIDC) authorization code flow, which is used by LXD UI. This configuration value is not shared with other LXD clients (such as the LXD CLI). (extension-pci-hotplug)= ## `pci_hotplug` This adds PCI device hotplugging for VMs. (extension-device-patch-removal)= ## `device_patch_removal` The `PATCH /1.0/instances/{name}` endpoint allows removing an instance device by setting its value to `null` in the devices map. (extension-daemon-storage-per-project)= ## `daemon_storage_per_project` This introduces two new configuration keys {config:option}`server-miscellaneous:storage.project.{name}.images_volume` and {config:option}`server-miscellaneous:storage.project.{name}.backups_volume` per each project to allow for a storage volume on an existing pool be used for storing the project-specific images and backups artifacts. (extension-ovn-internal-load-balancer)= ## `ovn_internal_load_balancer` This introduces support for internal OVN load balancers and network forwards. This approach allows `ovn` networks to define ports on internal IP addresses that can be forwarded to other internal IPs inside their respective networks. This change removes the previous limitation on `ovn` networks that load balancers and network forwards could only use external IP addresses to forward to internal IPs. (extension-auth-bearer-devlxd)= ## `auth_bearer_devlxd` Adds a new `bearer` authentication method and enables authentication to the DevLXD API. See {ref}`DevLXD bearer tokens `. (extension-devlxd-volume-management)= ## `devlxd_volume_management` Enables additional DevLXD endpoints for managing custom storage volumes and instance devices when `security.devlxd.management.volumes` is set to true. These endpoints are primarily intended for use by the LXD CSI driver. Management is limited to custom storage volumes. Additionally, the client is required to authenticate using DevLXD identity, which is used to track storage volume and instance device owner. Volumes and instance devices created through the DevLXD are marked as owned by the authenticated identity and can later be modified or removed through DevLXD only by the same identity. New DevLXD endpoints: * `GET /1.0/storage-pools/{pool}` — Retrieve a storage pool. * `GET /1.0/storage-pools/{pool}/volumes` — List owned custom volumes in a pool. * `GET /1.0/storage-pools/{pool}/volumes/{volume}` — Retrieve an owned custom volume. * `POST /1.0/storage-pools/{pool}/volumes` — Create a new owned custom volume in a pool. * `PUT /1.0/storage-pools/{pool}/volumes/{volume}` — Update an owned custom volume. * `DELETE /1.0/storage-pools/{pool}/volumes/{volume}` — Delete an owned custom volume. * `GET /1.0/instances/{inst}` — Retrieve an instance and its owned devices. * `PATCH /1.0/instances/{inst}` — Add new instance devices or modify owned devices. Changes to existing DevLXD endpoints: * `GET /1.0` — Adds a `supported_storage_drivers` field to the response, which is populated when `security.devlxd.management.volumes` is enabled. (extension-storage-driver-alletra)= ## `storage_driver_alletra` Adds a new `alletra` storage driver for the consumption of storage volumes from an HPE Alletra storage array. The following pool-level configuration keys have been added: 1. {config:option}`storage-alletra-pool-conf:alletra.wsapi` 1. {config:option}`storage-alletra-pool-conf:alletra.wsapi.verify` 1. {config:option}`storage-alletra-pool-conf:alletra.user.name` 1. {config:option}`storage-alletra-pool-conf:alletra.user.password` 1. {config:option}`storage-alletra-pool-conf:alletra.cpg` 1. {config:option}`storage-alletra-pool-conf:alletra.target` 1. {config:option}`storage-alletra-pool-conf:alletra.mode` (extension-resources-disk-used-by)= ## `resources_disk_used_by` Adds the field `used_by` to potential storage disk returned by the resources end point to indicate its use by any virtual parent device, e.g. `bcache`. (extension-ovn-dhcp-ranges)= ## `ovn_dhcp_ranges` This introduces support for the {config:option}`network-ovn-network-conf:ipv4.dhcp.ranges` configuration key for `ovn` networks. This key allows specifying a list of IPv4 ranges reserved for dynamic allocation using DHCP. (extension-operation-requestor)= ## `operation_requestor` This adds a new `requestor` field to operations, which contains information about the caller that initiated the operation. (extension-import-custom-volume-tar)= ## `import_custom_volume_tar` This adds new option `tar` for parameter `--type` in `POST /1.0/storage-pools/{poolName}/volumes/{type}` API call. (extension-projects-force-delete)= ## `projects_force_delete` Adds support for force deleting projects and their entities (instances, profiles, images, networks, network ACLs, network zones, storage volumes, and storage buckets) by setting the `force` query parameter on `DELETE /1.0/projects/{name}` requests. (extension-auth-oidc-sessions)= ## `auth_oidc_sessions` Adds session support for OIDC authentication. This enables compatibility with identity providers that issue opaque access tokens. When a session expires, LXD will re-verify the login with the identity provider. The duration of OIDC sessions defaults to one week and can be configured via the {config:option}`server-oidc:oidc.session.expiry` configuration key. Verification of an OIDC session depends on a new, cluster-wide core secret. The OIDC session expiry can never be greater than the expiry of the core secret. The {config:option}`server-core:core.auth_secret_expiry` configuration option can set to define how long a given secret can be used for before it expires. (extension-instance-snapshots-multi-volume)= ## `instance_snapshots_multi_volume` Enables the creation of a multi-volume snapshot and the restoration of an instance together with its attached volumes, while ensuring crash consistency across volumes. Adds `DiskVolumesMode` to `POST /1.0/instances/{name}/snapshots` and `RestoreDiskVolumesMode` to `PUT /1.0/instances/{name}` to select which attached volumes are included in snapshots/restores: "root" includes only the instance's root disk; "all-exclusive" includes the root disk and any exclusively attached (non-shared) volumes. Defaults to "root". This extension also introduces a new volatile configuration key, {config:option}`instance-volatile:volatile.attached_volumes`, in the configuration of supported storage drivers for instance snapshots. This key contains a JSON-serialized map of attached volume UUIDs to the UUIDs of their corresponding snapshots. Example value: `{: ,: }`. (extension-vm-persistent-bus)= ## `vm_persistent_bus` Adds support for persistently recording VM PCIe bus allocations in volatile configuration keys. This introduces two new volatile VM configuration keys: * {config:option}`instance-volatile:volatile..bus` - records the bus number for the device. * {config:option}`instance-volatile:volatile.bus.mode` - records whether "persistent" mode is being used for a VM. (extension-instance-placement-groups)= ## `instance_placement_groups` Placement groups define how instances are scheduled across cluster members according to a configurable placement policy (`spread` or `compact`) and rigor (`strict` or `permissive`). They can be referenced in instance and profile configuration through the {config:option}`instance-placement:placement.group` key to control placement behavior during instance creation and migration. New API endpoints: 1. `GET /1.0/placement-groups` — List all placement groups 1. `POST /1.0/placement-groups` — Create a new placement group 1. `GET /1.0/placement-groups/{name}` — Retrieve details for a specific placement group 1. `PUT /1.0/placement-groups/{name}` — Replace a placement group definition 1. `PATCH /1.0/placement-groups/{name}` — Update selected properties of a placement group 1. `DELETE /1.0/placement-groups/{name}` — Delete a placement group (extension-ovn-nic-acceleration-parent)= ## `ovn_nic_acceleration_parent` Adds support for specifying the OVN NIC acceleration physical function interfaces to allocate virtual functions from. This avoids the need for adding physical function interfaces to the OVN integration bridge. This introduces a new `ovn` network and `ovn` NIC configuration key: * {config:option}`device-nic-ovn-device-conf:acceleration.parent` - Comma separated list of physical function (PF) interfaces to allocate virtual functions (VFs) from for hardware acceleration when {config:option}`device-nic-ovn-device-conf:acceleration` is enabled. (extension-storage-and-profile-operations)= ## `storage_and_profile_operations` Certain storage and profile endpoints that were previously synchronous now return an operation and behave asynchronously. Newer LXD Go clients detect the presence of this API extension. When it is available, the client receives an operation object directly from the LXD server. If the extension is not present, the server response is wrapped in a completed operation, allowing the caller to handle it as an operation while lacking a retrievable operation ID. Older LXD Go clients are incompatible with servers that include this extension. Instead of returning a successful response, they will receive an operation response. Endpoints converted to asynchronous behavior: * `POST /storage-pools/{pool}/volumes/{type}` - Create storage volume * `PUT /storage-pools/{pool}/volumes/{type}/{vol}` - Update storage volume * `PATCH /storage-pools/{pool}/volumes/{type}/{vol}` - Patch storage volume * `POST /storage-pools/{pool}/volumes/{type}/{vol}` - Rename storage volume * `DELETE /storage-pools/{pool}/volumes/{type}/{vol}` - Delete storage volume * `PUT /storage-pools/{pool}/volumes/{type}/{vol}/snapshots/{snap}` - Update storage volume snapshot * `PATCH /storage-pools/{pool}/volumes/{type}/{vol}/snapshots/{snap}` - Patch storage volume snapshot * `PUT /1.0/profiles/{name}` - Update profile * `PATCH /1.0/profiles/{name}` - Patch profile (extension-storage-source-recover)= ## `storage_source_recover` As part of the recovery process it might be necessary to recover existing storage pools previously created by LXD. Before this was only partially possible for some of the drivers (e.g. by using `lvm.vg.force_reuse`), but not directly supported. The new pool `source.recover` configuration key can be set per cluster member to allow reuse of an existing pool `source`. What it does not allow is reusing the same source for multiple storage pools. The LVM storage driver has the specific `lvm.vg.force_reuse` configuration key for this purpose. (extension-instance-force-delete)= ## `instance_force_delete` This adds support for a `force` query parameter to the `DELETE /1.0/instances/{name}` endpoint. When set, running instances will be forcibly stopped before deletion. (extension-operation-metadata-entity-name)= (extension-operation-metadata-entity-url)= ## `operation_metadata_entity_url` Each {ref}`operation event ` has a `resources` field that contains URLs of LXD entities that the operation depends on. When an instance, instance backup, or storage volume backup is created, it is not strictly required for the caller to provide the name of the new resource. In this case, the URL of the expected resource is added to the resources map for clients to inspect and use. The `resources` field then contains both a dependency of the operation, and the newly created resource (which may not exist yet). To improve consistency, this API extension adds an `entity_url` field to operation metadata. The field contains the expected URL of the created entity. The field is only included when a resource is being created asynchronously (operation response), and where it is not required for the entity name to be specified by the client. For synchronous resource creation, clients should inspect the `Location` header. The `resources` field should no longer be relied upon for this information. (extension-instance-boot-mode)= ## `instance_boot_mode` Introduces the new {config:option}`instance-boot:boot.mode` configuration key to control the VM boot firmware mode. This replaces the removed `security.csm` and `security.secureboot` settings. The new setting accepts: * `uefi-secureboot` (default) - Use UEFI firmware with secure boot enabled * `uefi-nosecureboot` - Use UEFI firmware with secure boot disabled * `bios` - Use legacy BIOS firmware (SeaBIOS), `x86_64` (`amd64`) only (extension-auth-bearer-lxd)= (extension-auth-bearer)= ## `auth_bearer` Adds new identity type `bearer` that allows authentication with the LXD API using bearer tokens. See {ref}`LXD bearer tokens `. If applicable, the endpoint `/1.0/auth/identities/current` now also exposes the credential expiration time. The `expires_at` field is set when the current identity is trusted and the authentication method is either `bearer` or `tls`. In these cases, it reports the expiration time of the bearer token or the TLS certificate, respectively. (extension-vm-limits-max-bus-ports)= ## `vm_limits_max_bus_ports` This introduces support for the {config:option}`instance-resource-limits:limits.max_bus_ports` configuration key for virtual machines. This option controls the maximum allowed number of user configurable devices requiring a dedicated PCI/PCIe port for a virtual machine. This number includes both the devices attached before the instance start and the devices hotplugged at runtime. (extension-instances-state-selective-recursion)= ## `instances_state_selective_recursion` Adds support for selective recursion when querying instances. The API now supports selective state field fetching using semicolon-separated syntax in the `recursion` parameter: * `recursion=2;fields=state.disk` - Fetch only disk information * `recursion=2;fields=state.network` - Fetch only network information * `recursion=2;fields=state.disk,state.network` - Fetch both disk and network * `recursion=2;fields=` - Fetch no expensive state fields (disk and network skipped) * `recursion=2` - Fetch all fields (default behavior) The semicolon and equals signs must be URL-encoded when used in HTTP requests (`%3B` for `;` and `%3D` for `=`). The `lxc list` command automatically optimizes queries based on requested columns. (extension-project-delete-operation)= ## `project_delete_operation` The {ref}`forced project deletion ` API extension added support for forcibly deleting a project and all of its contents. This can take a long time, but the `DELETE /1.0/projects/{name}` endpoint still returned a synchronous response. This extension converts this endpoint to an asynchronous operation response. As with the {ref}`storage and profile operation extension `, this extension is forward compatible only. (extension-gpu-cdi-amd)= ## `gpu_cdi_amd` Adds support for using the Container Device Interface (CDI) specification to configure AMD GPU passthrough in LXD containers. The `id` field of GPU devices now accepts CDI identifiers (for example, `amd.com/gpu=gpu{INDEX}`) for containers, in addition to DRM card IDs. (extension-instance-refresh-config)= ## `instance_refresh_config` Ensures that instance `copy --refresh` operations apply target config/profile/device updates server-side. This applies to both direct copies and migration-based refresh operations. During refresh, the server applies the target instance configuration from the request (including config, devices, and profiles) before the data transfer completes. The request payload is treated as the full desired writable target configuration for the refresh update. The server does not merge or preserve omitted target keys automatically. Clients must include any target values that should remain on the destination after refresh. (extension-clustering-control-plane)= ## `clustering_control_plane` Adds a new `control-plane` cluster member role that can be manually assigned to designate which members participate in Raft consensus. Control plane mode is inactive by default until the number of members assigned the `control-plane` role reaches 3 or more. During this inactive period, all cluster members are eligible for automatic promotion to database roles. Once control plane mode activates, only members with the `control-plane` role are eligible to become voters, standbys, or the database leader. Members without the `control-plane` role are automatically assigned the `RAFT_SPARE` role and are excluded from automatic promotion to database roles, enabling safe scaling of cluster members without affecting quorum. If fewer than 3 members are assigned the `control-plane` role, all members remain eligible for automatic promotion to database roles, maintaining backwards compatibility with existing cluster behavior. The `control-plane` role is displayed alongside the database role (`database-leader`, `database-voter`, `database-standby`) for members participating in Raft. The role also controls internal event routing: - When control plane mode is active, members with the `control-plane` role act as event hubs. - When control plane mode is inactive, the cluster uses full-mesh event connectivity. These behaviors are applied asynchronously on heartbeat. Role changes are detected on the next heartbeat cycle, and control-plane mode activation or deactivation triggers automatic role rebalancing. The `event-hub` role is deprecated in favor of the functionally equivalent `control-plane` role. For more information, see {ref}`dqlite-internals-lxd-cluster-roles` and {ref}`cluster-manage-control-plane`. (extension-storage-remote-drop-source)= ## `storage_remote_drop_source` Starting with this extension, support for the `source` configuration key in both the Ceph RBD `ceph` and CephFS `cephfs` drivers is dropped. This ensures the storage pool configuration is done consistently across all remote drivers. For Ceph RBD use the `ceph.osd.pool_name` configuration key. For CephFS use the `cephfs.path` configuration key. (extension-storage-ceph-use-rbd-defaults)= ## `storage_ceph_use_rbd_defaults` Starting with this extension, new volumes (and clones) in Ceph RBD (`ceph`) pools are no longer created with only `--image-feature layering`. Instead the default RBD features of the respective Ceph cluster are used implicitly. This also applies to new volumes in already existing Ceph RBD storage pools. In case the `ceph.rbd.features` configuration key is already set the pool, new volumes continue using this list of features and won't use the defaults set in the Ceph cluster. (extension-bulk-operations)= ## `bulk_operations` Adds a `recursion=2` mode to `GET /1.0/operations`, enabling retrieval of parent-child relationships between operations. The parent-child operations are now also returned by the `GET /1.0/operations/{id}` endpoint when `recursion=1` is specified. (extension-ovn-dynamic-northbound-connection)= ## `ovn_dynamic_northbound_connection` Starting with this extension, if the {config:option}`server-miscellaneous:network.ovn.northbound_connection` server configuration is not specified, LXD dynamically determines the OVN Northbound database connection string based on the environment. If the MicroOVN snap is used, LXD reads the configuration from the MicroOVN `ovn.env` file. Otherwise, it defaults to using the `unix:/var/run/ovn/ovnnb_db.sock` socket. (extension-storage-zfs-promote)= ## `storage_zfs_promote` A {config:option}`storage-zfs-volume-conf:zfs.promote` configuration key was introduced. It's a Boolean that defaults to `false` and that when set to `true` instructs LXD to ZFS promote the volume being created (or re-created) from a clone. This is primarily useful when combined with the `initial.` `disk` device configuration options, as it allows controlling ZFS promotion when creating instances from other instances. (extension-storage-and-network-operations)= ## `storage_and_network_operations` Storage pool and network endpoints that were previously synchronous now return background operations. This affects create, update, delete, and rename actions on storage pools, networks, network ACLs, network zones, network zone records, network forwards, network load balancers, network peers, and storage buckets (including bucket keys). Clients should check for this extension and handle the asynchronous response by waiting on the returned operation. Operation metadata may include additional data, such as storage bucket admin credentials on bucket creation. (extension-gpu-cdi-hotplug)= ## `gpu_cdi_hotplug` Adds support for hotplugging GPU devices into containers when using {ref}`gpu-physical-cdi`. (extension-image-extended-metadata)= ## `image_extended_metadata` Adds `release_codename` and `release_title` fields to the `api.Image` struct. These fields are optional and are populated from the SimpleStreams index when available. Also updates the generated image description for SimpleStreams images to include variant if available. The image creation date and architecture are no longer used for image description. (extension-cluster-links)= ## `cluster_links` Cluster links enable secure, authenticated communication between separate LXD clusters using mutual TLS certificates. See {ref}`exp-cluster-links` for more information. This introduces the cluster links API and identity type. This includes the following new endpoints (see {ref}`rest-api` for details): * [`GET /1.0/cluster/links/`](swagger:/cluster-links/cluster_link_get) * [`GET /1.0/cluster/links`](swagger:/cluster-links/cluster_links_get) * [`GET /1.0/cluster/links//state`](swagger:/cluster-links/cluster_link_state_get) * [`PUT /1.0/cluster/links/`](swagger:/cluster-links/cluster_link_put) * [`PATCH /1.0/cluster/links/`](swagger:/cluster-links/cluster_link_patch) * [`POST /1.0/cluster/links`](swagger:/cluster-links/cluster_links_post) * [`POST /1.0/cluster/links/`](swagger:/cluster-links/cluster_link_post) * [`DELETE /1.0/cluster/links/`](swagger:/cluster-links/cluster_link_delete) (extension-replicators)= ## `replicators` This introduces the replicators API. Replicators are used to replicate project instances across clusters. This includes the following new endpoints (see {ref}`rest-api` for details): * [`GET /1.0/replicators`](swagger:/replicators/replicators_get) * [`GET /1.0/replicators/`](swagger:/replicators/replicator_get) * [`POST /1.0/replicators`](swagger:/replicators/replicators_post) * [`PUT /1.0/replicators/`](swagger:/replicators/replicator_put) * [`PATCH /1.0/replicators/`](swagger:/replicators/replicator_patch) * [`POST /1.0/replicators/`](swagger:/replicators/replicator_post) * [`DELETE /1.0/replicators/`](swagger:/replicators/replicator_delete) * [`GET /1.0/replicators//state`](swagger:/replicators/replicator_state_get) (extension-event-security)= ## `event_security` Adds a new `security` event type conforming to OWASP security event logging guidelines. Security events are accessible via `GET /1.0/events?type=security` and can be routed to Grafana Loki by adding `security` to the `loki.types` server configuration. (extension-storage-driver-powerstore)= ## `storage_driver_powerstore` Adds a new `powerstore` storage driver which allows the consumption of storage volumes from a PowerStore storage array using iSCSI or Fibre Channel (FC). The following pool level configuration keys have been added: 1. {config:option}`storage-powerstore-pool-conf:powerstore.gateway` 1. {config:option}`storage-powerstore-pool-conf:powerstore.gateway.verify` 1. {config:option}`storage-powerstore-pool-conf:powerstore.user.name` 1. {config:option}`storage-powerstore-pool-conf:powerstore.user.password` 1. {config:option}`storage-powerstore-pool-conf:powerstore.mode` 1. {config:option}`storage-powerstore-pool-conf:powerstore.target` (extension-oidc-device-client-id)= ## `oidc_device_client_id` The {ref}`OIDC client secret configuration extension ` is used to configure LXD to send a secret to the identity provider for authentication. It is not secure to make this secret public for consumption by the LXD CLI (`lxc`). For this reason, if the client configured for LXD in the identity provider requires a secret, that client cannot be used by CLI users. This extension adds a new {config:option}`server-oidc:oidc.device.client.id` configuration key for the CLI to use. An administrator can create two clients in the identity provider, one for LXD UI requiring a secret (and {abbr}`PKCE `), and one for the CLI that enables the device authorization grant and does not require a secret. The device client ID will be public to the CLI, falling back to {config:option}`server-oidc:oidc.client.id`. This configuration option cannot be set unless {config:option}`server-oidc:oidc.client.id` is set. (extension-storage-nvme-tcp)= ## `storage_nvme_tcp` Renames storage pool NVMe/TCP mode from `nvme` to `nvme/tcp`. (extension-project-replica-mode)= ## `project_replica_mode` Adds a `replica_mode` field to projects and a new `PUT /1.0/projects//state` endpoint for promoting and demoting projects between leader and standby modes for replication. (extension-cluster-links-used-by)= ## `cluster_links_used_by` Adds a `used_by` field to `ClusterLink` resources, returned by `GET /1.0/cluster/links` (with `recursion=1`) and `GET /1.0/cluster/links/{name}`. The field lists URLs of entities that reference the cluster link, filtered by the caller's view permissions. (extension-network-load-balancer-pool)= ## `network_load_balancer_pool` This introduces the load balancer pools for OVN networks. Pools are used to group instances to which a load balancer forwards traffic. This includes the following new endpoints (see {ref}`rest-api` for details): * [`GET /1.0/networks/{networkName}/load-balancer-pools`](swagger:/network-load-balancer-pools/network_load_balancer_pools_get) * [`POST /1.0/networks/{networkName}/load-balancer-pools`](swagger:/network-load-balancer-pools/network_load_balancer_pools_post) * [`GET /1.0/networks/{networkName}/load-balancer-pools/{poolName}`](swagger:/network-load-balancer-pools/network_load_balancer_pool_get) * [`PUT /1.0/networks/{networkName}/load-balancer-pools/{poolName}`](swagger:/network-load-balancer-pools/network_load_balancer_pool_put) * [`DELETE /1.0/networks/{networkName}/load-balancer-pools/{poolName}`](swagger:/network-load-balancer-pools/network_load_balancer_pool_delete) (extension-clustering-evacuation-force)= ## `clustering_evacuation_force` Adds a `force` field to `POST /1.0/cluster/members/{name}/state` when performing an `evacuate` action. When `force` is set to `true`, LXD skips the quorum safety check that otherwise would not allow you to evacuate an online raft voter when the remaining online voters would fall below the required majority. (extension-gpu-mig-cdi)= ## `gpu_mig_cdi` Updates `gputype=mig` GPU devices to use the Container Device Interface (CDI) for device passthrough instead of the legacy `nvidia.runtime` approach. The following changes are included: - `mig.uuid`: The UUID is now resolved to a CDI identifier (`nvidia.com/mig=`) internally. The `MIG-` prefix remains optional. Because a MIG UUID uniquely identifies the device, no parent GPU selector (`pci`, `vendorid`, `productid`, or `id`) is required alongside `mig.uuid`. If a parent GPU selector is provided, it is used to validate that the matching GPU exists and is an NVIDIA card, preserving backward compatibility. - `mig.gi` and `mig.ci`: The GPU instance ID and compute instance ID pair is now resolved to a MIG UUID via NVML, which is then used as a CDI identifier. - `id`: Now accepts either a CDI identifier for `gputype=mig` devices or a DRM card ID selector for the parent GPU. Supported CDI formats are `nvidia.com/mig=` and `nvidia.com/mig=:`. When a CDI identifier is provided, it is used directly and is mutually exclusive with `mig.uuid`, `mig.gi`, and `mig.ci`. (extension-cluster-links-unidirectional)= ## `cluster_links_unidirectional` This extends the {ref}`cluster links ` API with support for unidirectional cluster links. The local cluster consumes a trust token issued by the remote cluster to establish the link and pin the remote's certificate. The remote cluster creates a dedicated identity for the local cluster and can authenticate its incoming requests, but does not store addresses for the local cluster and cannot initiate outbound requests to it. (extension-network-load-balancer-pool-health-checks)= ## `network_load_balancer_pool_health_checks` This introduces health checks for OVN load balancer pools. New configuration options on the load balancer pool are added to further customize (or disable) the health check: * {config:option}`network-load-balancer-pool-properties:healthcheck` * {config:option}`network-load-balancer-pool-properties:healthcheck.interval` * {config:option}`network-load-balancer-pool-properties:healthcheck.timeout` * {config:option}`network-load-balancer-pool-properties:healthcheck.success_count` * {config:option}`network-load-balancer-pool-properties:healthcheck.failure_count` In addition a new endpoint [`GET /1.0/networks/{networkName}/load-balancer-pools/{poolName}/state`](swagger:/network-load-balancer-pools/network_load_balancer_pool_state_get) is added which returns the health check status for all instances in the pool. (extension-operation-child-count)= ## `operation_child_count` Adds a `child_count` field to the `Operation` struct, indicating the number of child operations. This allows clients to determine whether an operation has children without requiring `recursion=2`. (extension-storage-driver-powerstore-nvme)= ## `storage_driver_powerstore_nvme` Adds NVMe/TCP and NVMe/FC support to the Dell PowerStore storage driver. NVMe/TCP is now the default PowerStore mode when `powerstore.mode` is not set. (extension-access-management-expiry)= ## `access_management_expiry` Adds an `expires_at` field to the `Identity` struct, so that the expiry of an identity's credential is reported when identities are listed or individually retrieved, and not only for the current identity. For identities whose authentication method is `tls`, this is the expiry of the identity's certificate. For identities whose authentication method is `bearer`, this is the expiry of the issued token. For bearer tokens the expiry is recorded when a token is issued and cleared when a token is revoked. The field is omitted for identities whose credential has no expiry, that have no credential yet (pending identities), or whose token has been revoked. Note that bearer identities created prior to this extension will have an omitted `expires_at` field until a new token is issued. (extension-access-management-identity-effective-groups)= ## `access_management_identity_effective_groups` Adds `recursion=2` support to `GET /1.0/auth/identities` and a new `GET /1.0/auth/identities/{method}/{identifier}/state` endpoint. The state endpoint reports an `effective_groups` field for the identity. When identities are listed with `recursion=2`, the same `effective_groups` field is returned for each identity. The `effective_groups` field reports the deduplicated union of: - Direct authorization groups from `identities_auth_groups` - Authorization groups mapped from the identity's OIDC `identity_provider_groups` metadata through identity provider group mappings For non-OIDC identities, `effective_groups` is equal to direct group membership. (extension-cluster-links-public)= ## `cluster_links_public` This extends the {ref}`cluster links ` API with support for public cluster links. The initiating cluster (Cluster A) connects to the remote cluster (Cluster B) without presenting a client certificate, relying solely on TLS certificate pinning for server authentication. Cluster B has no record of the link, and neither cluster creates an identity for the other. This is useful when Cluster B exposes resources publicly or when anonymous read access to Cluster B is sufficient. Creating a public cluster link is a two-phase process: 1. Send a `POST` request to Cluster A with a name and `remote_address` (no `fingerprint`) to create a pending link. Cluster A fetches Cluster B's TLS certificate, confirms Cluster B is serving the LXD API, creates the pending link, and returns the certificate's fingerprint for user verification. The certificate is held in `volatile.pending_certificate` and the verified address in `volatile.pending_address`. Neither is pinned yet, so the link remains inert (unreachable) until confirmed. Repeating this request for a link that is still pending refreshes it instead of returning a conflict, so a caller can retry an interrupted creation. 2. To confirm the certificate, send a second `POST` request to Cluster A with the same name and the `fingerprint` returned by the first request. Cluster A rejects this second request if the fingerprint does not match or if the request sets `remote_address`. If the fingerprint matches, Cluster A pins the certificate already held in `volatile.pending_certificate`, and pins the canonical address (with an explicit port) already recorded in `volatile.pending_address` into `volatile.addresses`. This ensures that Cluster A always pins the original certificate fetched from Cluster B and the verified address, rather than a certificate or address supplied in the confirm request. A new `ClusterLinkCertificate` response type is introduced, containing the certificate fingerprint returned by the first request, and a matching `fingerprint` field is added to `ClusterLinksPost` for echoing it back. Both are used only by public cluster links. Replication requires the remote cluster to authenticate the connection, so {ref}`replicators ` and a project's `replica.cluster` accept only cluster link types that present a client certificate (currently `bidirectional` and `unidirectional`). Both reject a public cluster link at configuration time. (extension-durable-operations)= ## `durable_operations` Introduces new operation class for durable operations. Durable operations are restarted on the DQLite raft leader if the member that is running the operation fails to respond to heartbeats. If the leader was running the operation and goes offline, the operation is restarted on the newly elected leader. (extension-access-management-bearer-pending)= ## `access_management_bearer_pending` Adds three identity types, `Client token bearer (pending)`, `DevLXD token bearer (pending)` and `Initial UI token bearer (pending)`, that represent bearer identities for which no token is currently issued. A bearer identity is created in the pending state because no token has been issued for it yet. Issuing a token promotes the identity to its active type of `Client token bearer`, `DevLXD token bearer` or `Initial UI token bearer`, and revoking the token demotes it back to the pending type. A token that has expired but has not been revoked leaves the identity active. (extension-replicator-custom-volumes)= ## `replicator_custom_volumes` Adds `disk_volumes_mode` to the instance migration request (`POST /1.0/instances/` with `migration: true`) and to the migration source of `POST /1.0/instances`, with the values of the snapshot API: `root` (default) and `all-exclusive`. With `all-exclusive` the source transfers the custom storage volumes that are attached only to the migrated instance, with their snapshots, in the same operation, and lists them in the migration index header. A volume is transferred when no other instance references it on the source, whether it is attached directly or through a profile. A target in the same mode creates the instance before those volumes exist, checks when the index header arrives that it lists every volume the target is still waiting for, that every listed volume is referenced by the instance's effective devices on the target, and that no listed volume is already attached to another instance there, and refuses the migration before any data is sent otherwise. Each listed volume is created in the pool of the same name, or refreshed when it already exists, and the instance's disk devices are restored afterwards. A refresh only sends the snapshots the target is missing, and `instance_only` drops the volume snapshots along with the instance's. The default mode keeps the existing behavior, so `lxc copy` and `lxc move` are unchanged. Projects that do not own their custom volumes, live migrations and moves between cluster members reject the mode. All-exclusive snapshots also skip volumes attached to more than one instance, so a snapshot of an instance whose volumes are all shared records the root volume alone. This applies to every caller of the snapshot API, not only to replicators. Replicators use `all-exclusive`, so an instance is replicated together with the custom volumes attached only to it, including their snapshots. The pre-replication snapshot of the instance captures those volumes at the same moment as its root disk. Volumes attached to more than one instance or to no instance are not replicated and must exist on the standby before the instances using them can be replicated. A volume attached through a profile is replicated when it is exclusive to one instance; create it on the standby and add the device to the standby's profile before the first run, which then refreshes it. `lxc replicator info` lists the volumes that will travel. (extension-metrics-replicators)= ## `metrics_replicators` Adds observability for {ref}`replicators `. The `Replicator` API type gains two read-only fields, `last_success_at` and `last_success_oldest_snapshot_at`, recording when the replicator last completed a run successfully and the creation time of the oldest snapshot that run replicated. The latter is the effective recovery point objective (RPO) of the replicated project. Four gauges are added to the {ref}`metrics API `: `lxd_replicators`, `lxd_replicator_last_run_status`, `lxd_replicator_last_success_timestamp` and `lxd_replicator_last_success_oldest_snapshot_timestamp`. The `replicator-run` lifecycle event now fires when a run completes rather than when a manual run starts, so it covers scheduled runs as well, and carries the run outcome in its context. (extension-storage-driver-pure-scsifc)= ## `storage_driver_pure_scsifc` Adds SCSI/FC support to the Everpure storage driver. Fibre Channel targets are discovered through the host bus adapter, so `pure.target` has no effect when `pure.mode` is set to `scsi/fc`. (extension-operation-wait-status-code)= ## `operation_wait_status_code` The LXD and DevLXD operation wait endpoints (`GET /1.0/operations/{id}/wait`) now return HTTP status `200` when the wait itself succeeds, even if the operation it waited on failed. When the server supports this extension, the DevLXD operation also has a new `err_code` field that carries the HTTP status code associated with the failed operation. (extension-vm-volatile-maxcpus)= ## `vm_volatile_maxcpus` Adds a new volatile VM configuration key {config:option}`instance-volatile:volatile.cpu.maxcpus` that records the vCPU hotplug limit (SMP `maxcpus`) used when the VM booted. The value is reused on stateful start (stateful resume or live migration target) so that the QEMU SMP topology matches on both ends of a live migration regardless of the CPU count on each host. (extension-storage-driver-pure-nvmefc)= ## `storage_driver_pure_nvmefc` Adds NVMe/FC support to the Everpure storage driver. Like SCSI/FC, Fibre Channel targets are discovered through the host bus adapter, so `pure.target` has no effect when `pure.mode` is set to `nvme/fc`. (extension-storage-ceph-replicator)= ## `storage_ceph_replicator` This introduces the `ceph.replicator.` configuration key on `ceph` storage pools. It records the peer Ceph site to which a project's volumes on the pool are mirrored. If a project with a replicator holds volumes on a Ceph pool configured with this key, then the replicator enrolls those volumes in RBD mirroring, triggers a mirror snapshot of each volume on every replication run, and completes the run only after the peer site confirms that it has received every snapshot. (extension-project-replica-mode-optional)= ## `project_replica_mode_optional` The `replica_mode` field is now omitted from project responses when the project takes no part in replication, instead of being returned as an empty string. The field is still returned for projects in `leader` or `standby` mode. (extension-image-registries)= ## `image_registries` This introduces the image registries API. An image registry is a cluster-wide entity that points to a source of images, such as a Simple Streams server or another LXD server. Because image registries are managed on the server, all clients of a LXD server (and all members of a cluster) share the same set of image sources. For more information, see {ref}`howto-image-registries` and {ref}`ref-image-registries`. It includes the following new endpoints (see {ref}`rest-api` for details): * [`GET /1.0/image-registries`](swagger:/image-registries/image_registries_get) * [`GET /1.0/image-registries/`](swagger:/image-registries/image_registry_get) * [`GET /1.0/image-registries//images`](swagger:/image-registries/image_registry_images_get) * [`POST /1.0/image-registries`](swagger:/image-registries/image_registries_post) * [`POST /1.0/image-registries/`](swagger:/image-registries/image_registry_post) * [`PUT /1.0/image-registries/`](swagger:/image-registries/image_registry_put) * [`PATCH /1.0/image-registries/`](swagger:/image-registries/image_registry_patch) * [`DELETE /1.0/image-registries/`](swagger:/image-registries/image_registry_delete) This also adds a new project-level configuration key: * {config:option}`project-restricted:restricted.registries` - Comma separated list of allowed image registries for use in a project. (extension-loki-config-api-check-ready)= ## `loki_config_api_check_ready` Adds a new {config:option}`server-loki:loki.api.check_ready` server configuration key to configure whether or not LXD should probe the Loki API's `/ready` endpoint. By default it is set to `true`. Disabling the check allows using an OTLP forwarder which only implements the `/loki/api/v1/push` endpoint. (extension-server-state)= ## `server_state` This adds support for the `GET /1.0/state` endpoint, which reports the current runtime state of the LXD server. The response contains the server uptime, load averages, total and free memory and swap, number of processes, and number of logical CPUs. This endpoint is not available on clustered servers where the cluster member api already exposes similar information. (extension-storage-volume-block-tracking)= ## `storage_volume_block_tracking` ```{warning} **Do not** enable this extension in production environments. It is a {ref}`feature preview `. ``` Feature gate: `changed_block_tracking` (see {ref}`howto-snap-configure-feature-previews`) Adds changed block tracking for the block volumes of virtual machines, implemented as QEMU dirty bitmaps that are kept across a stop and a start of the instance, and adds the ability to export snapshot data over NBD tunneled through the LXD API. The snapshot request `POST /1.0/instances/{name}/snapshots` accepts a new `bitmap` field. When it is set, the bitmaps of every block volume of the snapshot are copied into the snapshot and a new bitmap named after the snapshot starts recording on each of those volumes at the instant of the snapshot. With `disk_volumes_mode` set to `all-exclusive` this covers the exclusively attached custom block volumes as well as the root disk. A snapshot whose name exists is rejected with `409`. Deleting a snapshot removes the bitmap of its name, and renaming one removes the bitmaps of its old and new names. The following endpoints list bitmaps. The UUID of a bitmap is the `volatile.uuid` of the root volume snapshot of the instance snapshot it was created with: * `GET /1.0/instances/{name}/snapshots/{snapshot}/bitmaps` lists the bitmaps of an instance snapshot, grouped by name with one entry per volume. * `GET /1.0/instances/{name}/snapshots/{snapshot}/bitmaps/{bitmap}` shows one bitmap of an instance snapshot. The following endpoints export volume data over NBD. Each is reached with the `Upgrade: nbd` header and returns `101 Switching Protocols`, after which the connection carries the NBD protocol. LXD sends the first NBD handshake message right after the response headers. An HTTP library can read part of that message into its buffer together with the headers, and the NBD client must read those buffered bytes before it reads from the connection. * `GET /1.0/instances/{name}/snapshots/{snapshot}/nbd` exports the block volume snapshots of an instance snapshot read-only, each under an NBD export named after its disk device, together with the bitmaps of the snapshot as `qemu:dirty-bitmap:` metadata contexts. The `device` query parameter, repeated once per device, selects a subset of the devices. The `previous_snapshot_uuid` query parameter limits the bitmaps to the ones created with the instance snapshot of that UUID. * `POST /1.0/storage-pools/{pool}/volumes/{type}/{volume}/nbd` exports one block volume read-write, for writing a backup back. The volume must be detached or attached to a stopped instance, and its bitmaps are deleted before the export. A new `can_connect_nbd` entitlement on instances and storage volumes governs access to the NBD endpoints. New `lxc` commands: * `lxc snapshot --bitmap` creates a snapshot with a bitmap. * `lxc bitmap list` and `lxc bitmap show` list the bitmaps of an instance snapshot. * `lxc nbd` exports an instance snapshot over NBD, with `--previous-snapshot-uuid` to limit the bitmaps. * `lxc storage volume nbd --writable` serves a storage volume read-write over NBD. (extension-cluster-members-state-recursion)= ## `cluster_members_state_recursion` Adds support for `recursion=2` on `GET /1.0/cluster/members`. In addition to the cluster member information returned with `recursion=1`, each entry contains a `state` field with the member state (as returned by `GET /1.0/cluster/members/{name}/state`). The state of all members is fetched in parallel by the member handling the request. The `state` field is `null` for offline members and for members whose state could not be retrieved. --- ### Doc Api (`doc/api.md`) (api-specification)= # Main API specification
--- ### Doc Architectures (`doc/architectures.md`) (architectures)= # Architectures LXD can run on just about any architecture that is supported by the Linux kernel and by Go. Some entities in LXD are tied to an architecture, for example, the instances, instance snapshots and images. The following table lists all supported architectures including their unique identifier and the name used to refer to them. The architecture names are typically aligned with the Linux kernel architecture names. ID | Kernel name | Description | Personalities :--- | :--- | :---- | :------------ 1 | `i686` | 32bit Intel x86 | 2 | `x86_64` | 64bit Intel x86 | `x86` 3 | `armv7l` | 32bit ARMv7 little-endian | 4 | `aarch64` | 64bit ARMv8 little-endian | `armv7l` (optional) 5 | `ppc` | 32bit PowerPC big-endian | 6 | `ppc64` | 64bit PowerPC big-endian | `powerpc` 7 | `ppc64le` | 64bit PowerPC little-endian | 8 | `s390x` | 64bit ESA/390 big-endian | 9 | `mips` | 32bit MIPS | 10 | `mips64` | 64bit MIPS | `mips` 11 | `riscv32` | 32bit RISC-V little-endian | 12 | `riscv64` | 64bit RISC-V little-endian | 13 | `armv6l` | 32bit ARMv6 little-endian | 14 | `armv8l` | 32bit ARMv8 little-endian | 15 | `loongarch64` | 64bit LoongArch | ```{note} LXD cares only about the kernel architecture, not the particular userspace flavor as determined by the toolchain. That means that LXD considers ARMv7 hard-float to be the same as ARMv7 soft-float and refers to both as `armv7l`. If useful to the user, the exact userspace ABI may be set as an image and container property, allowing easy query. ``` ## Virtual machine support LXD only supports running virtual machines on the following host architectures: - `x86_64` - `aarch64` - `ppc64le` - `s390x` The virtual machine guest architecture can usually be the 32bit personality of the host architecture, so long as the virtual machine firmware is capable of booting it. --- ### Doc Authentication (`doc/authentication.md`) --- discourse: lxc:[Token based remote connection](13114),lxc:[ACME support for server certificate](15142) relatedlinks: "[LXD for multi-user systems - YouTube](https://www.youtube.com/watch?v=6O0q3rSWr8A)" myst: html_meta: description: Explanation of LXD remote API authentication, including TLS client certificates, OpenID Connect, bearer tokens, and TLS server certificates. --- (authentication)= # Remote API authentication Remote communications with the LXD daemon happen using JSON over HTTPS. This requires the LXD API to be exposed over the network; see {ref}`server-expose` for instructions. To be able to access the remote API, clients must authenticate with the LXD server. The following authentication methods are supported: - {ref}`authentication-tls-certs` - {ref}`authentication-openid` - {ref}`authentication-bearer` (authentication-tls-certs)= ## TLS client certificates ```{youtube} https://www.youtube.com/watch?v=4iNpiL-lrXU :title: LXD token based remote authentication ``` When using {abbr}`TLS (Transport Layer Security)` client certificates for authentication, both the client and the server will generate a key pair the first time they're launched. The server will use that key pair for all HTTPS connections to the LXD socket. The client will use its certificate as a client certificate for any client-server communication. To cause certificates to be regenerated, simply remove the old ones. On the next connection, a new certificate is generated. ### Communication protocol The supported protocol must be TLS 1.3 or better. LXD uses the [Go TLS stack](https://pkg.go.dev/crypto/tls) as the TLS implementation. All communications must use perfect forward secrecy, and ciphers must be limited to strong elliptic curve ones (such as ECDHE-RSA or ECDHE-ECDSA). Any generated key should be at least 4096 bit RSA, preferably 384 bit ECDSA. When using signatures, only SHA-2 signatures should be trusted. Since we control both client and server, there is no reason to support any backward compatibility to broken protocol or ciphers. (authentication-trusted-clients)= ### Trusted TLS clients The workflow to authenticate with the server is similar to that of SSH, where an initial connection to an unknown server triggers a prompt: 1. When the user adds a server with [`lxc remote add`](lxc_remote_add.md), the server is contacted over HTTPS, its certificate is downloaded and the fingerprint is shown to the user. 1. The user is asked to confirm that this is indeed the server's fingerprint, which they can manually check by connecting to the server or by asking someone with access to the server to run the info command and compare the fingerprints. 1. The server attempts to authenticate the client: - If the client certificate is in the server's trust store, the connection is granted. - If the client certificate is not in the server's trust store, the server prompts the user for a token. If the provided token matches, the client certificate is added to the server's trust store and the connection is granted. Otherwise, the connection is rejected. See {ref}`server-expose` and {ref}`server-authenticate` for instructions on how to configure TLS authentication and add trusted clients. (authentication-pki)= ### Using a PKI system In a {abbr}`PKI (Public key infrastructure)` setup, a system administrator manages a central PKI that issues client certificates for all the LXD clients and server certificates for all the LXD daemons. In PKI mode, TLS authentication requires that client certificates are signed be the {abbr}`CA (Certificate authority)`. This requirement does not apply to clients that authenticate via [OIDC](authentication-openid). The steps for enabling PKI mode differ slightly depending on whether you use an ACME provider in addition (see {ref}`authentication-server-certificate`). `````{tabs} ````{group-tab} Only PKI If you use a PKI system, both the server and client certificates are issued by intermediate CA(s). The `client.ca` file contains the certificate used by the client to verify the server certificate it receives when making a connection to a remote. The `server.ca` file contains the certificate used by the server to verify the client certificate associated with an incoming connection. Both files contain trust anchors used to evaluate if the received leaf certificate from the other end of the connection is to be trusted or not. If the leaf certificate's chain of trust leads to one of the trusted anchors it will be trusted (unless revoked). 1. Add the CA certificate to all machines: - Place the `client.ca` file in the clients' configuration directories (`~/.config/lxc` or `~/snap/lxd/common/config` for snap users). - Place the `server.ca` file in the server's configuration directory (`/var/lib/lxd` or `/var/snap/lxd/common/lxd` for snap users). ```{note} In a cluster setup, the CA certificate must be named `cluster.ca`, and the same file must be added to all cluster members. ``` 1. Place the certificates issued by the CA in the clients' configuration directories, replacing the automatically generated `client.crt` and `client.key` files. 1. If you want clients to automatically trust the server, place the certificates issued by the CA in the server's configuration directory, replacing the automatically generated `server.crt` and `server.key` files. ```{note} In a cluster setup, the certificate files must be named `cluster.crt` and `cluster.key`. They must be identical on all cluster members. ``` When a client adds a PKI-enabled server or cluster as a remote, it checks the server certificate and prompts the user to trust the server certificate only if the certificate has not been signed by the CA. 1. Restart the LXD daemon. ```` ````{group-tab} PKI and ACME If you use a PKI system alongside an ACME provider, the server certificates are issued by the ACME provider, and the client certificates are issued by a secondary CA. 1. Place the CA certificate for the server (`server.ca`) in the server's configuration directory (`/var/lib/lxd` or `/var/snap/lxd/common/lxd` for snap users), so that the server can authenticate the clients. ```{note} In a cluster setup, the CA certificate must be named `cluster.ca`, and the same file must be added to all cluster members. ``` 1. Place the certificates issued by the CA in the clients' configuration directories, replacing the automatically generated `client.crt` and `client.key` files. 1. Restart the LXD daemon. ```` ````` #### Trusting certificates CA-signed client certificates are not automatically trusted. You must still add them to the server in one of the ways described in {ref}`authentication-trusted-clients`. To automatically trust CA-signed client certificates, set the {config:option}`server-core:core.trust_ca_certificates` server configuration to true. When `core.trust_ca_certificates` is enabled, any new clients with a CA-signed certificate will have full access to LXD. (authentication-revoke-certificates)= #### Revoking certificates To revoke certificates via the PKI, place a certificate revocation list in the server's configuration directory as `ca.crl` and restart the LXD daemon. A client with a CA-signed certificate that has been revoked, and is present in `ca.crl`, will not be able to authenticate with LXD, nor add LXD as a remote via [mutual TLS](authentication-trusted-clients). (authentication-openid)= ## OpenID Connect authentication LXD supports using [OpenID Connect](https://openid.net/developers/how-connect-works/) to authenticate users through an {abbr}`OIDC (OpenID Connect)` Identity Provider. To configure LXD to use OIDC authentication, set the [`oidc.*`](server-options-oidc) server configuration options. See the {ref}`how-to guides ` for more information. Once configured, the LXD UI will display a {guilabel}`Log in with SSO` button which will redirect you to the Identity Provider to log in. To use OIDC authentication in the LXD CLI, run [`lxc remote add `](lxc_remote_add.md). This defaults to OIDC authentication if configured on the remote server. You are then prompted to authenticate through your web browser, where you must confirm that the device code displayed in the browser matches the device code that is displayed in the terminal window. The LXD client then retrieves and stores an access token, which it provides to LXD for all interactions. The identity provider might also provide a refresh token. In this case, the LXD client uses this refresh token to attempt to retrieve another access token when the current access token has expired. When an OIDC client initially authenticates with LXD, it does not have access to the majority of the LXD API. OIDC clients must be granted access by an administrator, see {ref}`fine-grained-authorization`. (authentication-server-certificate)= ## TLS server certificate LXD supports issuing server certificates using {abbr}`ACME (Automatic Certificate Management Environment)` services, for example, [Let's Encrypt](https://letsencrypt.org/). To enable this feature, set the following server configuration: - {config:option}`server-acme:acme.domain`: The domain for which the certificate should be issued. - {config:option}`server-acme:acme.email`: The email address used for the account of the ACME service. - {config:option}`server-acme:acme.agree_tos`: Must be set to `true` to agree to the ACME service's terms of service. - {config:option}`server-acme:acme.ca_url`: The directory URL of the ACME service. By default, LXD uses "Let's Encrypt". LXD currently only supports the [`HTTP-01 challenge`](https://letsencrypt.org/docs/challenge-types/#http-01-challenge), which requires handling incoming HTTP requests on port 80. This can be achieved by using a reverse proxy such as [HAProxy](https://www.haproxy.org/). The HAProxy configuration example below uses `lxd.example.net` as the domain. After the certificate has been issued, LXD will be reachable from `https://lxd.example.net/`. It applies filtering to minimize the amount of undesired traffic coming from the internet reaching the protected LXD cluster. ``` /* Detailed source-code truncated for AI context efficiency. */ ``` (authentication-bearer)= ## Bearer token authentication LXD supports authenticating to the LXD API using bearer tokens. Bearer tokens provide a secure way to authenticate API requests without requiring {ref}`client certificates ` or {ref}`OpenID Connect configuration `. Bearer tokens can be issued for identities of type `bearer`. The permissions associated with a token are derived from the identity it belongs to and are enforced through {ref}`fine-grained-authorization`. A bearer identity is created in a pending state, with the type `Client token bearer (pending)` or `DevLXD token bearer (pending)`. Issuing a bearer token activates the identity and its type changes to `Client token bearer` or `DevLXD token bearer`, respectively. Revoking the token removes it and returns the identity to the pending state. An expired token remains associated with the identity. Therefore, the identity remains active, even though its token can no longer be used for authentication. To authenticate an API request using a bearer token, include it in the `Authorization` header as `Authorization: Bearer `, where `` represents an actual token value. By default, bearer tokens expire after 10 years, unless they are manually revoked. The expiration time can be customized when issuing the token. See {ref}`howto-auth-bearer` to learn how to issue and use bearer token in LXD. ## Failure scenarios In the following scenarios, authentication is expected to fail. ### Server certificate changed The server certificate might change in the following cases: - The server was fully reinstalled and therefore got a new certificate. - The connection is being intercepted ({abbr}`MITM (Machine in the middle)`). In such cases, the client will refuse to connect to the server because the certificate fingerprint does not match the fingerprint in the configuration for this remote. It is then up to the user to contact the server administrator to check if the certificate did in fact change. If it did, the certificate can be replaced by the new one, or the remote can be removed altogether and re-added. ### Server trust relationship revoked The server trust relationship is revoked for a client if another trusted client or the local server administrator removes the trust entry for the client on the server. In this case, the server still uses the same certificate, but all API calls return a 403 code with an error indicating that the client isn't trusted. ## Related topics How-to guides: - {ref}`howto-security-harden` - {ref}`server-expose` {{security_exp}} --- ### Doc Backup (`doc/backup.md`) (backups)= # How to back up a LXD server ```{youtube} https://www.youtube.com/watch?v=IFOZpAxckPo :title: LXD backup and disaster recovery ``` In a production setup, you should always back up the contents of your LXD server. The LXD server contains a variety of different entities, and when choosing your backup strategy, you must decide which of these entities you want to back up and how frequently you want to save them. ## What to back up The various contents of your LXD server are located on your file system and, in addition, recorded in the {ref}`LXD database `. Therefore, only backing up the database or only backing up the files on disk does not give you a full functional backup. Your LXD server contains the following entities: - Instances (database records and file systems) - Images (database records, image files, and file systems) - Networks (database records and state files) - Profiles (database records) - Storage volumes (database records and file systems) Consider which of these you need to back up. For example, if you don't use custom images, you don't need to back up your images since they are available on the image server. If you use only the `default` profile, or only the standard `lxdbr0` network bridge, you might not need to worry about backing them up, because they can easily be re-created. ## Full backup To create a full backup of all contents of your LXD server, back up the `/var/snap/lxd/common/lxd` (for snap users) or `/var/lib/lxd` (otherwise) directory. This directory contains your local storage, the LXD database, and your configuration. It does not contain separate storage devices, however. That means that whether the directory also contains the data of your instances depends on the storage drivers that you use. ```{important} If your LXD server uses any external storage (for example, LVM volume groups, ZFS zpools, or any other resource that isn't directly self-contained to LXD), you must back this up separately. See {ref}`howto-storage-backup-volume` for instructions. ``` To back up your data, create a tarball of `/var/snap/lxd/common/lxd` (for snap users) or `/var/lib/lxd` (otherwise). If you are not using the snap package and your source system has a `/etc/subuid` and `/etc/subgid` file, you should also back up these files. Restoring them avoids needless shifting of instance file systems. To restore your data, complete the following steps: 1. Stop LXD on your server (for example, with `sudo snap stop lxd`). 1. Delete the directory (`/var/snap/lxd/common/lxd` for snap users or `/var/lib/lxd` otherwise). 1. Restore the directory from the backup. 1. Delete and restore any external storage devices. 1. If you are not using the snap, restore the `/etc/subuid` and `/etc/subgid` files. 1. Restart LXD (for example, with `sudo snap start lxd` or by restarting your machine). ### Export a snapshot If you are using the LXD snap, you can also create a full backup by exporting a snapshot of the snap: 1. Create a snapshot: sudo snap save lxd Note down the ID of the snapshot (shown in the `Set` column). 1. Export the snapshot to a file: sudo snap export-snapshot See {ref}`snap:how-to-guides-manage-snaps-create-data-snapshots` in the Snap documentation for details. ## Partial backup If you decide to only back up specific entities, you have different options for how to do this. You should consider doing some of these partial backups even if you are doing full backups in addition. It can be easier and safer to, for example, restore a single instance or reconfigure a profile than to restore the full LXD server. ### Back up instances and volumes Instances and storage volumes are backed up in a very similar way (because when backing up an instance, you basically back up its instance volume, see {ref}`storage-volume-types`). See {ref}`instances-backup` and {ref}`howto-storage-backup-volume` for detailed information. The following sections give a brief summary of the options you have for backing up instances and volumes. (secondary-backup-server)= #### Secondary backup LXD server LXD supports copying and moving instances and storage volumes between two hosts. See {ref}`howto-instances-migrate` and {ref}`howto-storage-move-volume` for instructions. So if you have a spare server, you can regularly copy your instances and storage volumes to that secondary server to back them up. Use the `--refresh` flag to update the copies (see {ref}`storage-optimized-volume-transfer` for the benefits). If needed, you can either switch over to the secondary server or copy your instances or storage volumes back from it. If you use the secondary server as a pure storage server, it doesn't need to be as powerful as your main LXD server. #### Export tarballs You can use the `export` command to export instances and volumes to a backup tarball. By default, those tarballs include all snapshots. You can use an optimized export option, which is usually quicker and results in a smaller size of the tarball. However, you must then use the same storage driver when restoring the backup tarball. See {ref}`instances-backup-export` and {ref}`storage-backup-export` for instructions. #### Snapshots Snapshots save the state of an instance or volume at a specific point in time. However, they are stored in the same storage pool and are therefore likely to be lost if the original data is deleted or lost. This means that while snapshots are very quick and easy to create and restore, they don't constitute a secure backup. See {ref}`instances-snapshots` and {ref}`storage-backup-snapshots` for more information. (backup-database)= ### Back up the database While there is no trivial method to restore the contents of the {ref}`LXD database `, it can still be very convenient to keep a backup of its content. Such a backup can make it much easier to re-create, for example, networks or profiles if the need arises. Use the following command to dump the content of the local database to a file: lxd sql local .dump > Use the following command to dump the content of the global database to a file: lxd sql global .dump > You should include these two commands in your regular LXD backup. --- ### Doc Cloud Init (`doc/cloud-init.md`) --- discourse: lxc:[First class cloud-init support](12559) relatedlinks: "[Cloud-init documentation](https://cloudinit.readthedocs.org/)" --- (cloud-init)= # How to use `cloud-init` ```{youtube} https://www.youtube.com/watch?v=8OCG15TAldI :title: LXD instance configuration with cloud-init ``` [`cloud-init`](https://cloud-init.io/) is a tool for automatically initializing and customizing an instance of a Linux distribution. By adding `cloud-init` configuration to your instance, you can instruct `cloud-init` to execute specific actions at the first start of an instance. Possible actions include, for example: * Updating and installing packages * Applying certain configurations * Adding users * Enabling services * Running commands or scripts * Automatically growing the file system of a VM to the size (quota) of the disk See the {ref}`cloud-init:index` for detailed information. ```{note} The `cloud-init` actions are run only once on the first start of the instance. Rebooting the instance does not re-trigger the actions. ``` (cloud-init-support)= ## `cloud-init` support in images To use `cloud-init`, you must base your instance on an image that has `cloud-init` installed: * All images from the `ubuntu` and `ubuntu-daily` {ref}`image registries ` have `cloud-init` support. However, images for Ubuntu releases prior to 20.04 LTS require special handling to integrate properly with `cloud-init`, so that `lxc exec` works correctly with virtual machines that use those images. Refer to [VM `cloud-init`](vm-cloud-init-config). * Images from the [`images` registry](https://images.lxd.canonical.com/) have `cloud-init`-enabled variants, which are usually bigger in size than the default variant. The cloud variants use the `/cloud` suffix, for example, `images:alpine/edge/cloud`. ## Configuration options LXD supports two different sets of configuration options for configuring `cloud-init`: `cloud-init.*` and `user.*`. Which of these sets you must use depends on the `cloud-init` support in the image that you use. As a rule of thumb, newer images support the `cloud-init.*` configuration options, while older images support `user.*`. However, there might be exceptions to that rule. The following configuration options are supported: * `cloud-init.vendor-data` or `user.vendor-data` (see {ref}`cloud-init:vendor-data`) * `cloud-init.user-data` or `user.user-data` (see {ref}`cloud-init:user_data_formats`) * `cloud-init.network-config` or `user.network-config` (see {ref}`cloud-init:network_config`) For more information about the configuration options, see the [`cloud-init` instance options](instance-options-cloud-init), and the documentation for the {ref}`LXD data source ` in the `cloud-init` documentation. ```{note} Ubuntu 20.04 and earlier have recent versions of the `cloud-init` package but support for the modern `cloud-init.*` configuration options is disabled in those series. As such, when using such old instances, remember to use the `user.*` configuration options instead. ``` ### Vendor data and user data Both `vendor-data` and `user-data` are used to provide {ref}`cloud configuration data ` to `cloud-init`. The main idea is that `vendor-data` is used for the general default configuration, while `user-data` is used for instance-specific configuration. This means that you should specify `vendor-data` in a profile and `user-data` in the instance configuration. LXD does not enforce this method, but allows using both `vendor-data` and `user-data` in profiles and in the instance configuration. If both `vendor-data` and `user-data` are supplied for an instance, `cloud-init` merges the two configurations. However, if you use the same keys in both configurations, merging might not be possible. In this case, configure how `cloud-init` should merge the provided data. See {ref}`cloud-init:merging_user_data` for instructions. ## How to configure `cloud-init` To configure `cloud-init` for an instance, add the corresponding configuration options to a {ref}`profile ` that the instance uses or directly to the {ref}`instance configuration `. When configuring `cloud-init` directly for an instance, keep in mind that `cloud-init` runs only on instance start. This means any changes to `cloud-init` configuration only take effect after the next instance start. To ensure `cloud-init` configurations are applied on every boot, LXD changes the instance ID whenever relevant `cloud-init` configuration keys are modified. This triggers `cloud-init` to fetch and apply the updated data from LXD as if it were the instance's first boot. For more information, see the `cloud-init` docs regarding {ref}`cloud-init:first_boot_determination`. To add your configuration: ````{tabs} ```{group-tab} CLI Write the configuration to a file and pass that file to the `lxc config` command. For example, create `cloud-init.yml` with the following content: #cloud-config package_upgrade: true packages: - package1 - package2 Then run the following command: lxc config set cloud-init.user-data - < cloud-init.yml ``` ```{group-tab} API Provide the `cloud-init` configuration as a string with escaped newline characters. For example: lxc query --request PATCH /1.0/instances/ --data '{ "config": { "cloud-init.user-data": "#cloud-config\npackage_upgrade: true\npackages:\n - package1\n - package2" } }' Alternatively, to avoid mistakes, write the configuration to a file and include that in your request. For example, create `cloud-init.txt` with the following content: #cloud-config package_upgrade: true packages: - package1 - package2 Then send the following request: lxc query --request PATCH /1.0/instances/ --data '{ "config": { "cloud-init.user-data": "'"$(awk -v ORS='\\n' '1' cloud-init.txt)"'" } }' ``` ```{group-tab} UI Go to the {guilabel}`Configuration` tab of the instance detail page and select {guilabel}`Advanced > Cloud init`. Then click {guilabel}`Edit instance` and override the configuration for one or more of the `cloud-init` configuration options. ``` ```` ### YAML format for `cloud-init` configuration The `cloud-init` options require YAML's [literal style format](https://yaml.org/spec/1.2.2/#812-literal-style). You use a pipe symbol (`|`) to indicate that all indented text after the pipe should be passed to `cloud-init` as a single string, with new lines and indentation preserved. The `vendor-data` and `user-data` options usually start with `#cloud-config`. But `cloud-init` has an array of {ref}`configuration types ` available. For example: ```yaml config: cloud-init.user-data: | #cloud-config package_upgrade: true packages: - package1 - package2 ``` ```yaml config: cloud-init.user-data: | #!/usr/bin/bash echo hello | tee -a /tmp/example.txt ``` ```{tip} See {ref}`How to validate user data ` for information on how to check whether the syntax is correct. ``` ## How to check the `cloud-init` status `cloud-init` runs automatically on the first start of an instance. Depending on the configured actions, it might take a while until it finishes. To check the `cloud-init` status, log on to the instance and enter the following command: cloud-init status If the result is `status: running`, `cloud-init` is still working. If the result is `status: done`, it has finished. Alternatively, use the `--wait` flag to be notified only when `cloud-init` is finished: ```{terminal} :user: root :host: instance cloud-init status --wait ..................................... status: done ``` ## How to specify user or vendor data The `user-data` and `vendor-data` configuration can be used to, for example, upgrade or install packages, add users, or run commands. The provided values must have a first line that indicates what type of {ref}`user data format ` is being passed to `cloud-init`. For activities like upgrading packages or setting up a user, `#cloud-config` is the data format to use. The configuration data is stored in the following files in the instance's root file system: * `/var/lib/cloud/instance/cloud-config.txt` * `/var/lib/cloud/instance/user-data.txt` ### Examples See the following sections for the user data (or vendor data) configuration for different example use cases. You can find more advanced {ref}`examples ` in the `cloud-init` documentation. #### Upgrade packages To trigger a package upgrade from the repositories for the instance right after the instance is created, use the `package_upgrade` key: ```yaml config: cloud-init.user-data: | #cloud-config package_upgrade: true ``` #### Install packages To install specific packages when the instance is set up, use the `packages` key and specify the package names as a list: ```yaml config: cloud-init.user-data: | #cloud-config packages: - git - openssh-server ``` #### Set the time zone To set the time zone for the instance on instance creation, use the `timezone` key: ```yaml config: cloud-init.user-data: | #cloud-config timezone: Europe/Rome ``` #### Run commands To run a command (such as writing a marker file), use the `runcmd` key and specify the commands as a list: ```yaml config: cloud-init.user-data: | #cloud-config runcmd: - [touch, /run/cloud.init.ran] ``` #### Add a user account To add a user account, use the `users` key. See the {ref}`cloud-init:reference/examples:including users and groups` example in the `cloud-init` documentation for details about default users and which keys are supported. ```yaml config: cloud-init.user-data: | #cloud-config users: - name: documentation_example ``` ## How to specify network configuration data By default, `cloud-init` configures a DHCP client on an instance's `eth0` interface. You can define your own network configuration using the `network-config` option to override the default configuration (this is due to how the template is structured). `cloud-init` then renders the relevant network configuration on the system using either `ifupdown` or `netplan`, depending on the Ubuntu release. The configuration data is stored in the following files in the instance's root file system: * `/var/lib/cloud/seed/nocloud-net/network-config` * `/etc/network/interfaces.d/50-cloud-init.cfg` (if using `ifupdown`) * `/etc/netplan/50-cloud-init.yaml` (if using `netplan`) ### Example To configure a specific network interface with a static IPv4 address and also use a custom name server, use the following configuration: ```yaml config: cloud-init.network-config: | version: 2 ethernets: eth1: addresses: - 10.10.101.20/24 gateway4: 10.10.101.1 nameservers: addresses: - 10.10.10.254 ``` ## How to inject SSH keys into instances To inject SSH keys into LXD instances for an arbitrary user, use the configuration key `cloud-init.ssh-keys.`. Use the format `:` for its value, where `` is a Linux username and `` can be either a pure SSH public key or an import ID for a key hosted elsewhere. For example, `root:gh:githubUser` and `myUser:ssh-keyAlg publicKeyHash` are valid values. To prevent a particular SSH key from being inherited from a profile by an instance, edit the instance configuration by setting the `cloud-init.ssh-keys.` key that references the target SSH key to `none`, and the key will not be injected. The contents of the `cloud-init.ssh-keys.` keys are merged into both {config:option}`instance-cloud-init:cloud-init.vendor-data` and {config:option}`instance-cloud-init:cloud-init.user-data` before being passed to the guest, following the `cloud-config` specification. (See the {doc}`cloud-init reference ` for details.) Therefore, keys defined via `cloud-init.ssh-keys.` cannot be applied if LXD cannot parse the existing `cloud-init.vendor-data` and `cloud-init.user-data` for that instance. This might occur if those keys are not in YAML format or contain invalid YAML. Other configuration formats are not yet supported. You can define SSH keys via `cloud-init.vendor-data` or `cloud-init.user-data` directly. Keys defined using `cloud-init.ssh-keys.` do not conflict with those defined in either of those settings. For details on defining SSH keys with `cloud-config`, see {ref}`the cloud-init documentation for SSH configuration `. Changing a `cloud-init.*` key does not remove previously applied keys. Since `cloud-init` only runs on instance start, updates to `cloud-init.*` keys on a running instance only take effect after restart. ### Examples The following command injects `someuser`'s key from Launchpad into the newly created `container`: ```bash lxc launch ubuntu:24.04 container -c cloud-init.ssh-keys.mykey=root:lp:someuser ``` The example profile configuration below defines a key to be injected on an instance. The injected key enables the owner of the private key to SSH into the instance as a user named `user`: ```yaml config: cloud-init.vendor-data: | users: - name: user ssh_authorized_keys: ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJFDWcYmMrCZdk9JI29bAiHKD90oEUr8tqK5VvoO8Vcj ``` --- ### Doc Clustering (`doc/clustering.md`) --- discourse: lxc:[LXD cluster on Raspberry Pi 4](9076) relatedlinks: "[MicroCloud](https://canonical.com/microcloud)" myst: html_meta: description: An index of how-to guides for LXD clusters, covering forming and managing clusters, configuring cluster networking and storage, and more. --- (clustering)= # Clustering These how-to guides cover common operations related to clustering in LXD. ## Create and configure clusters LXD servers can join together as members of a cluster, then be configured for features such as control plane mode and cluster healing. ```{toctree} :titlesonly: Form a cluster Manage a cluster Configure networks Configure storage ``` ## Manage instances and cluster groups Instances on cluster members can be accessed from and migrated to other members. Cluster groups and placement groups can be used to control how instances are distributed across cluster members. ```{toctree} :titlesonly: Manage instances Set up cluster groups Use placement groups ``` ## Recover clusters or cluster volumes Quorum loss from member failures and orphaned volume entries from interrupted migrations can both be recovered. ```{toctree} :titlesonly: Recover a cluster Recover orphaned volume entries ``` ## Set up a virtual IP A highly available virtual IP provides a stable entry point for client connections even if individual cluster members go offline. ```{toctree} :titlesonly: Set up a highly available virtual IP ``` ## Link clusters Clusters can be linked together for authenticated communication, enabling features such as replicators. ```{toctree} :titlesonly: Create cluster links Manage cluster links ``` ## Related topics {{clustering_exp}} {{clustering_ref}} --- ### Doc Config Options Cheat Sheet (`doc/config_options_cheat_sheet.md`) --- orphan: true nosearch: true --- # Configuration options ```{important} This page shows how to output configuration option documentation. The content in this page is for demonstration purposes only. ``` Some instance options: ```{config:option} agent.nic_config instance :shortdesc: Set the name and MTU to be the same as the instance devices :default: "`false`" :type: bool :liveupdate: "`no`" :condition: Virtual machine Controls whether to set the name and MTU of the default network interfaces to be the same as the instance devices (this happens automatically for containers) ``` ```{config:option} migration.incremental.memory.iterations instance :shortdesc: Maximum number of transfer operations :condition: container :default: 10 :type: integer :liveupdate: "yes" Maximum number of transfer operations to go through before stopping the instance ``` ```{config:option} cluster.evacuate instance :shortdesc: What to do when evacuating the instance :default: "`auto`" :type: string :liveupdate: "no" Controls what to do when evacuating the instance (`auto`, `migrate`, `live-migrate`, or `stop`) ``` These need the `instance` scope to be specified as second argument. The default scope is `server`, so this argument isn't required. Some server options: ```{config:option} backups.compression_algorithm server :shortdesc: Compression algorithm for images :type: string :scope: global :default: "`gzip`" Compression algorithm to use for new images (`bzip2`, `gzip`, `lzma`, `xz` or `none`) ``` ```{config:option} instances.nic.host_name :shortdesc: How to generate a host name :type: string :scope: global :default: "`random`" If set to `random`, use the random host interface name as the host name; if set to `mac`, generate a host name in the form `lxd` (MAC without leading two digits) ``` Any other scope is also possible. This scope shows that you can use formatting, mainly in the short description and the description, and the available options. ```{config:option} test1 something :shortdesc: testing Testing. ``` ```{config:option} test2 something :shortdesc: Hello! **bold** and `code` This is the real text. With two paragraphs. And a list: - Item - Item - Item And a table: Key | Type | Scope | Default | Description :-- | :--- | :---- | :------ | :---------- `acme.agree_tos` | bool | global | `false` | Agree to ACME terms of service `acme.ca_url` | string | global | `https://acme-v02.api.letsencrypt.org/directory` | URL to the directory resource of the ACME service `acme.domain` | string | global | - | Domain for which the certificate is issued `acme.email` | string | global | - | Email address used for the account registration ``` ```{config:option} test3 something :shortdesc: testing :default: "`false`" :type: Type :liveupdate: Python parses the options, so "no" is converted to "False" - to prevent this, put quotes around the text ("no" or "`no`") :condition: "yes" :readonly: "`maybe` - also add quotes if the option starts with code" :resource: Resource, :managed: Managed :required: Required :scope: (this is something like "global" or "local", **not** the scope of the option (`server`, `instance`, ...) Content ``` To reference an option, use `{config:option}`. It is not possible to override the link text. Except for server options (default), you must specify the scope. {config:option}`instance:migration.incremental.memory.iterations` {config:option}`something:test1` The index is here: {ref}`config-options` --- ### Doc Container Environment (`doc/container-environment.md`) (container-runtime-environment)= # Container runtime environment LXD attempts to present a consistent environment to all containers it runs. The exact environment will differ slightly based on kernel features and user configuration, but otherwise, it is identical for all containers. ## File system LXD assumes that any image it uses to create a new container comes with at least the following root-level directories: - `/dev` (empty) - `/proc` (empty) - `/sbin/init` (executable) - `/sys` (empty) ## Devices LXD containers have a minimal and ephemeral `/dev` based on a `tmpfs` file system. Since this is a `tmpfs` and not a `devtmpfs` file system, device nodes appear only if manually created. The following standard set of device nodes is set up automatically: - `/dev/console` - `/dev/fd` - `/dev/full` - `/dev/log` - `/dev/null` - `/dev/ptmx` - `/dev/random` - `/dev/stdin` - `/dev/stderr` - `/dev/stdout` - `/dev/tty` - `/dev/urandom` - `/dev/zero` In addition to the standard set of devices, the following devices are also set up for convenience: - `/dev/fuse` - `/dev/net/tun` - `/dev/mqueue` ### Network LXD containers may have any number of network devices attached to them. The naming for those (unless overridden by the user) is `ethX`, where `X` is an incrementing number. ### Container-to-host communication LXD sets up a socket at `/dev/lxd/sock` that the root user in the container can use to communicate with LXD on the host. See {doc}`dev-lxd` for the API documentation. ## Mounts The following mounts are set up by default: - `/proc` ({spellexception}`proc`) - `/sys` (`sysfs`) - `/sys/fs/cgroup/*` (`cgroupfs`) (only on kernels that lack cgroup namespace support) If they are present on the host, the following paths will also automatically be mounted: - `/proc/sys/fs/binfmt_misc` (only on kernels that lack `binfmt_misc` namespace support) - `/sys/firmware/efi/efivars` - `/sys/fs/fuse/connections` - `/sys/fs/pstore` - `/sys/kernel/debug` - `/sys/kernel/security` The reason for passing all of those paths is that legacy init systems require them to be mounted, or be mountable, inside the container. The majority of those paths will not be writable (or even readable) from inside an unprivileged container. In privileged containers, they will be blocked by the AppArmor policy. ### LXCFS If LXCFS is present on the host, it is automatically set up for the container. This normally results in a number of `/proc` files being overridden through bind-mounts. On older kernels, a virtual version of `/sys/fs/cgroup` might also be set up by LXCFS. (container-runtime-environment-binfmt-misc)= ### `binfmt_misc` The `binfmt_misc` kernel feature allows the system to run binaries for foreign architectures by registering interpreters for different binary types. See the [kernel documentation](https://docs.kernel.org/admin-guide/binfmt-misc.html) for more details. On kernels that lack `binfmt_misc` namespace support, LXD automatically bind-mounts `/proc/sys/fs/binfmt_misc` from the host into the container. This allows containers to use the host's registered binary types. On kernels with `binfmt_misc` namespace support, unprivileged containers can mount their own isolated `binfmt_misc`. To enable `binfmt_misc` in an unprivileged container, run: ```bash mount binfmt_misc -t binfmt_misc /proc/sys/fs/binfmt_misc ``` The container can now register its own binary types without affecting the host or other containers. This is useful for running foreign architecture binaries within the container. ## PID1 LXD spawns whatever is located at `/sbin/init` as the initial process of the container (PID 1). This binary should act as a proper init system, including handling re-parented processes. LXD's communication with PID1 in the container is limited to two signals: - `SIGINT` to trigger a reboot of the container - `SIGPWR` (or alternatively `SIGRTMIN`+3) to trigger a clean shutdown of the container The initial environment of PID1 is blank except for `container=lxc`, which can be used by the init system to detect the runtime. All file descriptors above the default three are closed prior to PID1 being spawned. ## Related topics {{instances_how}} {{instances_exp}} --- ### Doc Contributing (`doc/contributing.md`) --- relatedlinks: "[LXD's development process - YouTube](https://www.youtube.com/watch?v=pqV0Z1qwbkg)" --- (howto-contribute)= # How to contribute to LXD % Include content from [../CONTRIBUTING.md](../CONTRIBUTING.md) ```{include} ../CONTRIBUTING.md :start-after: :end-before: ``` ## Contribute to the code Follow the steps below to set up your development environment and start working on new LXD features. ### Install LXD from source To build the dependencies, follow the instructions in {ref}`installing_from_source`. ### Add your fork as a remote After setting up your build environment, add your GitHub fork as a remote and fetch the latest updates: git remote add myfork git@github.com:/lxd.git git remote update Then switch to the main branch of your fork: git switch myfork/main ### Build LXD Now you can build your fork of the project by running: make Before making changes, create a new branch on your fork: ```bash git switch -c ``` Set up tracking for the new branch to make future pushes easier: ```bash git push -u myfork ``` ### Important notes for new LXD contributors - Persistent data is stored in the `LXD_DIR` directory, which is created by running `lxd init`. - By default, `LXD_DIR` is located at `/var/lib/lxd` (for non-snap installations) or `/var/snap/lxd/common/lxd` (for snap users). - To prevent version conflicts, consider setting a separate `LXD_DIR` for your development fork. - Binaries compiled from your source are placed in `$(go env GOPATH)/bin` by default. - When testing, explicitly invoke these binaries instead of the global `lxd` you might have installed. - For convenience, you can create an alias in your `~/.bashrc` to call these binaries with the appropriate flags. - If you have a `systemd` service running LXD from a previous installation, consider disabling it to prevent version conflicts with your development build. ## Contribute to the documentation We strive to make LXD as easy and straightforward to use as possible. To achieve this, our documentation aims to provide the information users need, cover all common use cases, and answer typical questions. You can contribute to the documentation in several ways. We appreciate your help! ### Ways to contribute Document new features or improvements you contribute to the code. : - Submit documentation updates in pull requests alongside your code changes. We will review and merge them together with the code. Clarify concepts or common questions based on your own experience. : - Submit a pull request with your documentation improvements. Report documentation issues by opening an issue on [GitHub](https://github.com/canonical/lxd/issues). : - We will evaluate and update the documentation as needed. Ask questions or suggest improvements in the [LXD forum](https://discourse.ubuntu.com/c/project/lxd/126). : - We monitor discussions and update the documentation when necessary. If you contribute images to `doc/images`: - Use **SVG** or **PNG** formats. - Optimize PNG images for smaller file size using a tool like [TinyPNG](https://tinypng.com/) (web-based), [OptiPNG](https://optipng.sourceforge.net/) (CLI-based), or similar. % Include content from [README.md](README.md) ```{include} README.md :start-after: ``` When you open a pull request, a preview of the documentation hosted by Read the Docs is built automatically. To see this, view the details for the `docs/readthedocs.com:canonical-lxd` check on the pull request. Others can also use this preview to validate your changes. ### Automatic documentation checks GitHub runs automatic checks on the documentation to verify the spelling, the validity of links, correct formatting of the Markdown files, and the use of inclusive language. You can (and should!) run these tests locally before pushing your changes: - Check the spelling: `make doc-spellcheck` (or `make doc-spelling` to first build the documentation and then check it) - Check the validity of links: `make doc-linkcheck` - Check the Markdown formatting: `make doc-lint` - Check for inclusive language: `make doc-woke` ### Document instructions (how-to guides) LXD can be used with different clients, primarily the command-line interface (CLI), API, and UI. The documentation contains instructions for all of these, so when adding or updating how-to guides, remember to update the documentation for all clients. #### Using tabs for client-specific information When instructions differ between clients, use tabs to organize them: ````` ````{tabs} ```{group-tab} CLI [...] ``` ```{group-tab} API [...] ``` ```{group-tab} UI [...] ``` ```` ````` ```{tip} You might need to increase the number of backticks (`) if there are code blocks or other directives in the tab content. ``` #### Guidelines for writing instructions CLI instructions : - Link to the relevant `lxc` command reference. Example: ``[`lxc init`](lxc_init.md)`` - You don't need to document all available command flags, but mention any that are especially relevant. - Examples are very helpful, so add a few if it makes sense. API instructions : - When possible, use [`lxc query`](lxc_query.md) to demonstrate API calls. For complex calls, use `curl` or other widely available tools. - In the request data, include all required fields but keep it minimal—there's no need to list every possible field. - Link to the API call reference. Example: ``[`POST /1.0/instances`](swagger:/instances/instances_post)`` UI instructions : - Use screenshots sparingly—they are difficult to keep up to date. - When referring to labels in the UI, use the `{guilabel}` role. Example: ``To create an instance, go to the {guilabel}`Instances` section and click {guilabel}`Create instance`.`` ### Document configuration options Configuration options are documented by comments in the Go code. These comments are extracted automatically. #### Adding or modifying configuration options - Look for comments that start with `lxdmeta:generate` in the code. - When adding or modifying a configuration option, include the corresponding documentation comment. - Refer to the [`lxd-metadata` README file](https://github.com/canonical/lxd/blob/main/lxd/lxd-metadata/README.md) for formatting guidelines. - When you add or modify configuration options, you must re-generate `doc/metadata.txt` and `lxd/metadata/configuration.json`. See the [](#configuration-options-updates) section for instructions. #### Including configuration options in documentation The documentation pulls sections from `doc/metadata.txt` to display a group of configuration options. For example, to include the core server options, use: ```` % Include content from [metadata.txt](metadata.txt) ```{include} metadata.txt :start-after: :end-before: ``` ```` #### When to update documentation files - If you add a new option to an existing group, no changes to the documentation files are needed, aside from [re-generating `metadata.txt`](#configuration-options-updates). The option will be included automatically. - If you define a new group, to add it to the documentation, you must add an `{include}` directive to the appropriate Markdown file in `doc/`. --- ### Doc Daemon Behavior (`doc/daemon-behavior.md`) (daemon-behavior)= # Daemon behavior This specification covers some of the {ref}`lxd-daemon`'s behavior. ## Startup On every start, LXD checks that its directory structure exists. If it doesn't, it creates the required directories, generates a key pair and initializes the database. Once the daemon is ready for work, LXD scans the instances table for any instance for which the stored power state differs from the current one. If an instance's power state was recorded as running and the instance isn't running, LXD starts it. ## Signal handling ### `SIGINT`, `SIGQUIT`, `SIGTERM` For those signals, LXD assumes that it's being temporarily stopped and will be restarted at a later time to continue handling the instances. The instances will keep running and LXD will close all connections and exit cleanly. ### `SIGPWR` Indicates to LXD that the host is going down. LXD will attempt a clean shutdown of all the instances. After 30 seconds, it kills any remaining instance. The instance `power_state` in the instances table is kept as it was so that LXD can restore the instances as they were after the host is done rebooting. ### `SIGUSR1` Write a memory profile dump to the file specified with `--memprofile`. --- ### Doc Database (`doc/database.md`) --- relatedlinks: "[Canonical Dqlite](https://canonical.com/dqlite), [Dqlite GitHub](https://github.com/canonical/dqlite)" myst: html_meta: description: Explanation of the LXD Dqlite database, which stores server configuration and state for both single machines and clusters. --- (database)= # The LXD Dqlite database LXD uses a distributed database to store the server configuration and state, which allows for quicker queries than if the configuration was stored inside each instance's directory (as it is done by LXC, for example). To understand the advantages, consider a query against the configuration of all instances, like "what instances are using `br0`?". To answer that question without a database, you would have to iterate through every single instance, load and parse its configuration, and then check which network devices are defined in there. With a database, you can run a simple query on the database to retrieve this information. ## Dqlite In a LXD cluster, all members of the cluster must share the same database state. Therefore, LXD uses [Dqlite](https://canonical.com/dqlite), a distributed version of SQLite. Dqlite provides replication, fault-tolerance, and automatic failover without the need of external database processes. When using LXD as a single machine and not as a cluster, the Dqlite database effectively behaves like a regular SQLite database. For more information, see {ref}`dqlite-internals`. (database-location)= ## File location The database files are stored in the `database` sub-directory of your LXD data directory (thus `/var/snap/lxd/common/lxd/database/` if you use the snap, or `/var/lib/lxd/database/` otherwise). Upgrading LXD to a newer version might require updating the database schema. In this case, LXD automatically stores a backup of the database and then runs the update. See {ref}`ref-snap-database` for more information. ## Backup See {ref}`backup-database` for instructions on how to back up the contents of the LXD database. --- ### Doc Debugging (`doc/debugging.md`) # How to debug LXD For information on debugging instance issues, see {ref}`instances-troubleshoot`. ## Debugging `lxc` and `lxd` Here are different ways to help troubleshooting `lxc` and `lxd` code. ### `lxc --debug` Adding `--debug` flag to any client command will give extra information about internals. If there is no useful info, it can be added with the logging call: logger.Debugf("Hello: %s", "Debug") ### `lxc monitor` This command will monitor messages as they appear on remote server. ## REST API through local socket On server side the most easy way is to communicate with LXD through local socket. This command accesses `GET /1.0` and formats JSON into human readable form using [jq](https://stedolan.github.io/jq/tutorial/) utility: ```bash curl --unix-socket /var/lib/lxd/unix.socket lxd/1.0 | jq . ``` or for snap users: ```bash curl --unix-socket /var/snap/lxd/common/lxd/unix.socket lxd/1.0 | jq . ``` See the [RESTful API](rest-api.md) for available API. ## REST API through HTTPS {ref}`HTTPS connection to LXD ` requires valid client certificate that is generated on first [`lxc remote add`](lxc_remote_add.md). This certificate should be passed to connection tools for authentication and encryption. If desired, `openssl` can be used to examine the certificate (`~/.config/lxc/client.crt` or `~/snap/lxd/common/config/client.crt` for snap users): ```bash openssl x509 -text -noout -in client.crt ``` Among the lines you should see: Certificate purposes: SSL client : Yes ### With command line tools ```bash wget --no-check-certificate --certificate=$HOME/.config/lxc/client.crt --private-key=$HOME/.config/lxc/client.key -qO - https://127.0.0.1:8443/1.0 # or for snap users wget --no-check-certificate --certificate=$HOME/snap/lxd/common/config/client.crt --private-key=$HOME/snap/lxd/common/config/client.key -qO - https://127.0.0.1:8443/1.0 ``` ### With browser Some browser plugins provide convenient interface to create, modify and replay web requests. To authenticate against LXD server, convert `lxc` client certificate into importable format and import it into browser. For example this produces `client.pfx` in Windows-compatible format: ```bash openssl pkcs12 -clcerts -inkey client.key -in client.crt -export -out client.pfx ``` After that, opening [`https://127.0.0.1:8443/1.0`](https://127.0.0.1:8443/1.0) should work as expected. ## Debug LXD using `pprof` LXD provides a Go [`pprof`](https://pkg.go.dev/net/http/pprof) server when the {config:option}`server-core:core.debug_address` is set. The debug server should not be exposed to an externally accessible address for production use cases. Use the following command to enable the server on the loopback interface: lxc config set core.debug_address=localhost:8080 If the LXD server is running on your workstation, you can view a summary of available information by navigating to [`http://localhost:8080/debug/pprof/`](http://localhost:8080/debug/pprof/). ## Debug the LXD database The files of the global {ref}`database ` are stored under the `./database/global` sub-directory of your LXD data directory (e.g. `/var/lib/lxd/database/global` or `/var/snap/lxd/common/lxd/database/global` for snap users). Since each member of the cluster also needs to keep some data which is specific to that member, LXD also uses a plain SQLite database (the "local" database), which you can find in `./database/local.db`. Backups of the global database directory and of the local database file are made before upgrades, and are tagged with the `.bak` suffix. You can use those if you need to revert the state as it was before the upgrade. ### Dumping the database content or schema If you want to get a SQL text dump of the content or the schema of the databases, use the `lxd sql [.dump|.schema]` command, which produces the equivalent output of the `.dump` or `.schema` directives of the `sqlite3` command line tool. ### Running custom queries from the console If you need to perform SQL queries (e.g. `SELECT`, `INSERT`, `UPDATE`) against the local or global database, you can use the `lxd sql` command (run `lxd sql --help` for details). You should only need to do that in order to recover from broken updates or bugs. Please consult the LXD team first (creating a [GitHub issue](https://github.com/canonical/lxd/issues/new) or [forum](https://discourse.ubuntu.com/c/project/lxd/126) post). ### Running custom queries at LXD daemon startup In case the LXD daemon fails to start after an upgrade because of SQL data migration bugs or similar problems, it's possible to recover the situation by creating `.sql` files containing queries that repair the broken update. To perform repairs against the local database, write a `./database/patch.local.sql` file containing the relevant queries, and similarly a `./database/patch.global.sql` for global database repairs. Those files will be loaded very early in the daemon startup sequence and deleted if the queries were successful (if they fail, no state will change as they are run in a SQL transaction). As above, please consult the LXD team first. ### Syncing the cluster database to disk If you want to flush the content of the cluster database to disk, use the `lxd sql global .sync` command, that will write a plain SQLite database file into `./database/global/db.bin`, which you can then inspect with the `sqlite3` command line tool. ## Inspect a core dump file In our continuous integration tests, we have configured the `core_pattern` as follows: echo '|/bin/sh -c $@ -- eval exec gzip --fast > /var/crash/%e.%p.gz' | sudo tee /proc/sys/kernel/core_pattern Additionally, we have set the `GOTRACEBACK` environment variable to `crash`. Together, these ensure that when LXD crashes a core dump is compressed with `gzip` and placed in `/var/crash`. To inspect a core dump file, you will need the LXD binary that was running at the time of the crash. The binary must include symbols; you can check this with the `file` utility. You will also need any C libraries that are used by LXD which must also include symbols. You can inspect a core dump using [Delve](https://github.com/go-delve/delve) (see the [Go Wiki](https://go.dev/wiki/CoreDumpDebugging) for more information), but this does not support any dynamically linked C libraries. Instead, you can use [GDB](https://sourceware.org/gdb/) which can inspect linked libraries and allows sourcing a file to load Golang support. To do this, run: gdb Then in the GDB REPL, run: (gdb) source /src/runtime/runtime-gdb.py Substituting in the actual path to your `$GOROOT`. This will add Golang runtime support. Finally, set the search path for C libraries using: (gdb) set solib-search-path You can now use the GDB REPL to inspect the core dump. Some useful commands are: - `backtrace` (print stack trace). - `info goroutines` (show goroutines). - `info threads` (show threads). - `thread ` (change thread). --- ### Doc Dev Lxd (`doc/dev-lxd.md`) (dev-lxd)= # Communication between instance and host ```{youtube} https://www.youtube.com/watch?v=xZSnqqWykmo :title: The LXD instance API ``` The DevLXD API allows for limited communication between guest instances and the host. The API is available inside each LXD guest as a Unix socket at `/dev/lxd/sock`, using JSON over plain HTTP. Multiple concurrent connections are allowed. ```{note} {config:option}`instance-security:security.devlxd` must be set to `true` (which is the default) for an instance to allow access to the socket. Additionally, for virtual machines, the LXD agent must be present and running for the socket to be available. ``` (dev-lxd-implementation)= ## Implementation details (dev-lxd-implementation-containers)= ### Containers LXD on the host binds `/var/lib/lxd/devlxd/sock` and listens for connections. This single socket is exposed into every container started by LXD at `/dev/lxd/sock`. ```{note} The alternative to using a single socket is to create a socket for every container. This approach was discarded to avoid issues with file descriptor limits for hosts with thousands of containers. ``` (dev-lxd-implementation-vms)= ### Virtual machines LXD on the host starts a HTTPS {abbr}`Vsock (Virtual Socket)` server. The LXD agent on the virtual machine communicates securely with the Vsock server using a certificate mounted in the VM's configuration drive. The LXD agent creates the socket at `/dev/lxd/sock` and proxies requests to the Vsock server. (devlxd-authentication)= ## Authentication Queries on `/dev/lxd/sock` only return information related to the requesting instance. For containers, LXD inspects user credentials associated with the connection and matches them with a running instance. For virtual machines, LXD extracts the virtual socket ID from the remote address of the caller (the LXD agent), matches it with a virtual machine, and validates the virtual machine's certificate to establish mTLS. (devlxd-authentication-bearer)= ### Bearer tokens Processes within guest instances can authenticate over the DevLXD socket using a bearer token. To do this, set an `Authorization: Bearer {token}` header on requests to the socket. Bearer tokens can be obtained by creating a `DevLXD token bearer` identity in the identities API and issuing a token for it. For more information, see {ref}`devlxd-authenticate`. (devlxd-api-spec)= ## REST-API
--- ### Doc Devlxd Api.Yaml (`doc/devlxd-api.yaml`) swagger: "2.0" info: contact: email: lxd@lists.canonical.com name: LXD upstream url: https://github.com/canonical/lxd description: |- The DevLXD API is exposed to guest instances. For virtual-machines, this requires the LXD agent to be running. The API is JSON over a unix+http socket and is available at `/dev/lxd/sock` within a guest instance. license: name: AGPL-3.0-only url: https://www.gnu.org/licenses/agpl-3.0.en.html title: DevLXD API version: "1.0" paths: /: get: operationId: api_get summary: Get the supported API endpoints description: |- Returns a list of supported API versions (URLs). produces: - application/json responses: "200": description: List of endpoints schema: type: array items: type: string example: - /1.0 /1.0: get: operationId: state_get summary: Get the instance state description: Shows the instance state and API details. produces: - application/json - text/plain responses: "200": $ref: '#/definitions/DevLXDGet' "500": description: "Internal error" schema: type: string example: "Internal server error occurred" patch: operationId: state_patch summary: Set instance readiness. description: |- Updates the instance state. This allows the guest to update the instance state from "Started" to "Ready". This can be used to indicate that the guest has successfully started any required services e.g. via Cloud Init. consumes: - application/json produces: - text/plain parameters: - description: Instance state in: body name: state required: true schema: $ref: '#/definitions/DevLXDPut' responses: "200": description: "Updated successfully" "400": description: "Bad request" schema: type: string example: "Invalid state" "500": description: "Internal error" schema: type: string example: "Internal server error occurred" /1.0/config: get: operationId: config_keys_get summary: List instance configuration key API endpoints description: |- Produces a list of API endpoints of configuration keys. The configuration keys are filtered. Only the `user.*` and `cloud-init.*` namespaces are returned. produces: - application/json - text/plain responses: "200": description: List of endpoints schema: type: array items: type: string example: - /1.0/user.foo "500": description: "Internal error" schema: type: string example: "Internal server error occurred" /1.0/config/{key}: get: operationId: config_key_get summary: Get the value corresponding to a configuration key description: |- Returns the value corresponding to a configuration key. Only configuration keys listed by `GET /1.0/config` can be queried. produces: - text/plain responses: "200": description: "Configration value" schema: type: string example: "foo" "400": description: "Bad request" schema: type: string example: "Bad request" "403": description: "Forbidden" schema: type: string example: "Forbidden" "404": description: "Not found" schema: type: string example: "Key not found" "500": description: "Internal error" schema: type: string example: "Internal server error occurred" /1.0/devices: get: operationId: devices_get summary: Get a map of instance devices description: |- Returns the devices that are configured on the instance. produces: - application/json - text/plain responses: "200": description: The instance devices schema: type: object example: {"eth0":{"name": "eth0","network": "lxdbr0","type": "nic"},"root": {"path": "/","pool": "default","type": "disk"}} "500": description: "Internal error" schema: type: string example: "Internal server error occurred" /1.0/events: get: operationId: events_get summary: Get instance events description: |- Listen for events that concern the instance. This includes updates to `user.*` configuration keys, and changes to devices. Requests are upgraded to a WebSocket, which will only close if the client disconnects. parameters: - description: Event type(s), comma separated (valid types are config, device) example: config,device in: query name: type type: string produces: - application/json - text/plain responses: "200": description: Websocket message (JSON) schema: $ref: '#/definitions/Event' "500": description: "Internal error" schema: type: string example: "Internal server error occurred" /1.0/images/{fingerprint}/export: get: operationId: image_export_get summary: Get the raw image file(s) description: |- Download the raw image file(s) from the host. If the image is in split format, a multipart http transfer occurs. This is useful when running LXD in a guest instance. The full image fingerprint must be provided (unlike the main API, where a fingerprint prefix can be used). This endpoint requires `security.devlxd.images` to be set to `true` for the instance (it is `false` by default). Due to an historical oversight, error responses from this endpoint have the same schema as the main LXD API, which is different to those from the DevLXD API. produces: - application/octet-stream - multipart/form-data responses: "200": description: Raw image data "403": $ref: '#/responses/MainAPIForbidden' "404": $ref: '#/responses/MainAPINotFound' "500": $ref: '#/responses/MainAPIInternalServerError' /1.0/ubuntu-pro: get: operationId: ubuntu_pro_get summary: Get Ubuntu Pro guest attachment settings description: |- Returns settings which determine Ubuntu Pro auto attachment behavior. produces: - application/json - text/plain responses: "200": description: The Pro attachment settings schema: $ref: '#/definitions/DevLXDUbuntuProSettings' "500": description: "Internal error" schema: type: string example: "Internal server error occurred" /1.0/ubuntu-pro/token: post: operationId: ubuntu_pro_token_post summary: Get an Ubuntu Pro token description: |- Returns an Ubuntu Pro guest attachment token. produces: - application/json - text/plain responses: "200": description: The Pro attachment settings schema: $ref: '#/definitions/DevLXDUbuntuProGuestTokenResponse' "500": description: "Internal error" schema: type: string example: "Internal server error occurred" "403": description: "Forbidden" schema: type: string example: "Forbidden" responses: MainAPIForbidden: description: Forbidden schema: properties: error: example: not authorized type: string error_code: example: 403 format: int64 type: integer type: example: error type: string type: object MainAPIInternalServerError: description: Internal Server Error schema: properties: error: example: internal server error type: string error_code: example: 500 format: int64 type: integer type: example: error type: string type: object MainAPINotFound: description: Not found schema: properties: error: example: not found type: string error_code: example: 404 format: int64 type: integer type: example: error type: string type: object definitions: DevLXDGet: title: The instance state and properties. type: object properties: api_version: description: The DevLXD API version number example: "1.0" type: string instance_type: description: The type of instance example: container type: string location: description: The cluster member where this instance is located example: member01 type: string state: description: The instance state example: Started type: string auth: description: Whether the client is trusted or untrusted example: trusted type: string DevLXDPut: title: The instance state type: object properties: state: description: The instance state example: Started type: string Event: description: Event represents an event entry (over websocket) properties: metadata: description: JSON encoded metadata example: action: added config: {"type":"unix-char","path":"/dev/kvm"} name: kvm type: object timestamp: description: Time at which the event was sent example: "2021-02-24T19:00:45.452649098-05:00" format: date-time type: string type: description: Event type (one of config, device) example: device type: string type: object DevLXDUbuntuProSettings: title: Settings for Ubuntu Pro guest attachment. type: object properties: guest_attach: description: Guest attach settings for the instance ("on", "off", or "available"). example: "off" type: string DevLXDUbuntuProGuestTokenResponse: title: An Ubuntu Pro guest token response type: object properties: expires: description: The expiry date of the token example: "2025-03-23T20:00:00-04:00" type: string guest_token: description: The token example: "" type: string id: description: A unique identifier for the token example: "340bac0d-f5b2-48d5-bcf3-100c573ea51a" type: string --- ### Doc Environment (`doc/environment.md`) # Environment variables The LXD client and daemon respect some environment variables to adapt to the user's environment and to turn some advanced features on and off. ```{note} These environment variables are not available if you use the LXD snap. ``` ## Common Name | Description :--- | :---- `LXD_DIR` | The LXD data directory `PATH` | List of paths to look into when resolving binaries `http_proxy` | Proxy server URL for HTTP `https_proxy` | Proxy server URL for HTTPS `no_proxy` | List of domains, IP addresses or CIDR ranges that don't require the use of a proxy ## Client environment variable Name | Description :--- | :---- `EDITOR` | What text editor to use `VISUAL` | What text editor to use (if `EDITOR` isn't set) `LXD_CONF` | Path to the LXC configuration directory `LXD_GLOBAL_CONF` | Path to the global LXC configuration directory `LXC_REMOTE` | Name of the remote to use (overrides configured default remote) ## Server environment variable Name | Description :--- | :---- `LXD_EXEC_PATH` | Full path to the LXD binary (used when forking subcommands) `LXD_LXC_TEMPLATE_CONFIG` | Path to the LXC template configuration directory `LXD_SECURITY_APPARMOR` | If set to `false`, forces AppArmor off `LXD_UNPRIVILEGED_ONLY` | If set to `true`, enforces that only unprivileged containers can be created. Note that any privileged containers that have been created before setting LXD_UNPRIVILEGED_ONLY will continue to be privileged. To use this option effectively it should be set when the LXD daemon is first set up. `LXD_QEMU_FW_PATH` | Path (or `:` separated list of paths) to firmware (OVMF, SeaBIOS) to be used by QEMU `LXD_IDMAPPED_MOUNTS_DISABLE` | Disable idmapped mounts support (useful when testing traditional UID shifting) `LXD_DEVMONITOR_DIR` | Path to be monitored by the device monitor. This is primarily for testing. `LXD_FSMONITOR_DRIVER` | Driver to be used for file system monitoring. This is primarily for testing. --- ### Doc Events (`doc/events.md`) --- myst: html_meta: description: LXD events API reference covering logging, operation, lifecycle, ovn, and security events. Learn event types, structures, and how to access events via monitor or WebSocket. --- (events)= # Events ## Introduction Events are messages about actions that have occurred over LXD. Using the API endpoint `/1.0/events` directly or via [`lxc monitor`](lxc_monitor.md) will connect to a WebSocket through which events of the selected types will be streamed. ## Event types LXD currently supports five event types. - `logging`: Shows all logging messages regardless of the server logging level. - `operation`: Shows all ongoing operations from creation to completion (including updates to their state and progress metadata). - `lifecycle`: Shows an audit trail for specific actions occurring over LXD. - `ovn`: Shows network-related events from OVN (Open Virtual Network). - `security`: Shows security-related events including authentication attempts, authorization decisions, and administrative changes. Requires appropriate permissions to view. ## Event structure ### Example ```yaml location: cluster_name metadata: action: network-updated requestor: protocol: unix username: root source: /1.0/networks/lxdbr0 timestamp: "2021-03-14T00:00:00Z" type: lifecycle ``` - `location`: The cluster member name (if clustered). - `timestamp`: Time that the event occurred in RFC3339 format. - `type`: Type of event (one of `logging`, `operation`, `lifecycle`, `ovn`, or `security`). - `metadata`: Information about the specific event type. ### Logging event structure - `message`: The log message. - `level`: The log-level of the log. - `context`: Additional information included in the event. (ref-events-operation)= ### Operation event structure - `id`: The UUID of the operation. - `class`: The type of operation (`task`, `token`, or `websocket`). - `description`: A description of the operation. - `created_at`: The operation's creation date. - `updated_at`: The operation's date of last change. - `status`: The current state of the operation. - `status_code`: The operation status code. - `resources`: Resources affected by this operation. - `metadata`: Operation specific metadata. - `may_cancel`: Whether the operation may be canceled. - `err`: Error message of the operation. - `location`: The cluster member name (if clustered). ### Life-cycle event structure - `action`: The life-cycle action that occurred. - `requestor`: Information about who is making the request (if applicable). - `source`: Path to what is being acted upon. - `context`: Additional information included in the event. ## Supported life-cycle events | Name | Description | Additional Information | | :------------------------------------- | :-------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------- | | `certificate-created` | A new certificate has been added to the server trust store. | | | `certificate-deleted` | The certificate has been deleted from the trust store. | | | `certificate-updated` | The certificate's configuration has been updated. | | | `cluster-certificate-updated` | The certificate for the whole cluster has changed. | | | `cluster-disabled` | Clustering has been disabled for this machine. | | | `cluster-enabled` | Clustering has been enabled for this machine. | | | `cluster-group-created` | A new cluster group has been created. | | | `cluster-group-deleted` | A cluster group has been deleted. | | | `cluster-group-renamed` | A cluster group has been renamed. | | | `cluster-group-updated` | A cluster group has been updated. | | | `cluster-member-added` | A new machine has joined the cluster. | | | `cluster-member-removed` | The cluster member has been removed from the cluster. | | | `cluster-member-renamed` | The cluster member has been renamed. | `old_name`: the previous name. | | `cluster-member-updated` | The cluster member's configuration been edited. | | | `cluster-token-created` | A join token for adding a cluster member has been created. | | | `config-updated` | The server configuration has changed. | | | `image-alias-created` | An alias has been created for an existing image. | `target`: the original instance. | | `image-alias-deleted` | An alias has been deleted for an existing image. | `target`: the original instance. | | `image-alias-renamed` | The alias for an existing image has been renamed. | `old_name`: the previous name. | | `image-alias-updated` | The configuration for an image alias has changed. | `target`: the original instance. | | `image-created` | A new image has been added to the image store. | `type`: `container` or `vm`. | | `image-deleted` | The image has been deleted from the image store. | | | `image-refreshed` | The local image copy has updated to the current source image version. | | | `image-retrieved` | The raw image file has been downloaded from the server. | `target`: destination server. | | `image-secret-created` | A one-time key to fetch this image has been created. | | | `image-updated` | The image's configuration has changed. | | | `instance-backup-created` | A backup of the instance has been created. | | | `instance-backup-deleted` | The instance backup has been deleted. | | | `instance-backup-renamed` | The instance backup has been renamed. | `old_name`: the previous name. | | `instance-backup-retrieved` | The raw instance backup file has been downloaded. | | | `instance-console` | Connected to the console of the instance. | `type`: `console` or `vga`. | | `instance-console-reset` | The console buffer has been reset. | | | `instance-console-retrieved` | The console log has been downloaded. | | | `instance-created` | A new instance has been created. | | | `instance-deleted` | The instance has been deleted. | | | `instance-exec` | A command has been executed on the instance. | `command`: the command to be executed. | | `instance-file-deleted` | A file on the instance has been deleted. | `file`: path to the file. | | `instance-file-pushed` | The file has been pushed to the instance. | `file-source`: local file path. `file-destination`: destination file path. `info`: file information. | | `instance-file-retrieved` | The file has been downloaded from the instance. | `file-source`: instance file path. `file-destination`: destination file path. | | `instance-log-deleted` | The instance's specified log file has been deleted. | | | `instance-log-retrieved` | The instance's specified log file has been downloaded. | | | `instance-metadata-retrieved` | The instance's image metadata has been downloaded. | | | `instance-metadata-template-created` | A new image template file for the instance has been created. | `path`: relative file path. | | `instance-metadata-template-deleted` | The image template file for the instance has been deleted. | `path`: relative file path. | | `instance-metadata-template-retrieved` | The image template file for the instance has been downloaded. | `path`: relative file path. | | `instance-metadata-updated` | The instance's image metadata has changed. | | | `instance-paused` | The instance has been put in a paused state. | | | `instance-ready` | The instance is ready. | | | `instance-renamed` | The instance has been renamed. | `old_name`: the previous name. | | `instance-restarted` | The instance has restarted. | | | `instance-restored` | The instance has been restored from a snapshot. | `snapshot`: name of the snapshot being restored. | | `instance-resumed` | The instance has resumed after being paused. | | | `instance-shutdown` | The instance has shut down. | | | `instance-snapshot-created` | A snapshot of the instance has been created. | | | `instance-snapshot-deleted` | The instance snapshot has been deleted. | | | `instance-snapshot-renamed` | The instance snapshot has been renamed. | `old_name`: the previous name. | | `instance-snapshot-updated` | The instance snapshot's configuration has changed. | | | `instance-started` | The instance has started. | | | `instance-stopped` | The instance has stopped. | | | `instance-updated` | The instance's configuration has changed. | | | `network-acl-created` | A new network ACL has been created. | | | `network-acl-deleted` | The network ACL has been deleted. | | | `network-acl-renamed` | The network ACL has been renamed. | `old_name`: the previous name. | | `network-acl-updated` | The network ACL configuration has changed. | | | `network-created` | A network device has been created. | | | `network-deleted` | The network device has been deleted. | | | `network-forward-created` | A new network forward has been created. | | | `network-forward-deleted` | The network forward has been deleted. | | | `network-forward-updated` | The network forward has been updated. | | | `network-peer-created` | A new network peer has been created. | | | `network-peer-deleted` | The network peer has been deleted. | | | `network-peer-updated` | The network peer has been updated. | | | `network-renamed` | The network device has been renamed. | `old_name`: the previous name. | | `network-updated` | The network device's configuration has changed. | | | `network-zone-created` | A new network zone has been created. | | | `network-zone-deleted` | The network zone has been deleted. | | | `network-zone-record-created` | A new network zone record has been created. | | | `network-zone-record-deleted` | The network zone record has been deleted. | | | `network-zone-record-updated` | The network zone record has been updated. | | | `network-zone-updated` | The network zone has been updated. | | | `operation-cancelled` | The operation has been canceled. | | | `profile-created` | A new profile has been created. | | | `profile-deleted` | The profile has been deleted. | | | `profile-renamed` | The profile has been renamed . | `old_name`: the previous name. | | `profile-updated` | The profile's configuration has changed. | | | `project-created` | A new project has been created. | | | `project-deleted` | The project has been deleted. | | | `project-renamed` | The project has been renamed. | `old_name`: the previous name. | | `project-updated` | The project's configuration has changed. | | | `storage-pool-created` | A new storage pool has been created. | `target`: cluster member name. | | `storage-pool-deleted` | The storage pool has been deleted. | | | `storage-pool-updated` | The storage pool's configuration has changed. | `target`: cluster member name. | | `storage-volume-backup-created` | A new backup for the storage volume has been created. | `type`: `container`, `virtual-machine`, `image`, or `custom`. | | `storage-volume-backup-deleted` | The storage volume's backup has been deleted. | | | `storage-volume-backup-renamed` | The storage volume's backup has been renamed. | `old_name`: the previous name. | | `storage-volume-backup-retrieved` | The storage volume's backup has been downloaded. | | | `storage-volume-created` | A new storage volume has been created. | `type`: `container`, `virtual-machine`, `image`, or `custom`. | | `storage-volume-deleted` | The storage volume has been deleted. | | | `storage-volume-renamed` | The storage volume has been renamed. | `old_name`: the previous name. | | `storage-volume-restored` | The storage volume has been restored from a snapshot. | `snapshot`: name of the snapshot being restored. | | `storage-volume-snapshot-created` | A new storage volume snapshot has been created. | `type`: `container`, `virtual-machine`, `image`, or `custom`. | | `storage-volume-snapshot-deleted` | The storage volume's snapshot has been deleted. | | | `storage-volume-snapshot-renamed` | The storage volume's snapshot has been renamed. | `old_name`: the previous name. | | `storage-volume-snapshot-updated` | The configuration for the storage volume's snapshot has changed. | | | `storage-volume-updated` | The storage volume's configuration has changed. | | | `warning-acknowledged` | The warning's status has been set to "acknowledged". | | | `warning-deleted` | The warning has been deleted. | | | `warning-reset` | The warning's status has been set to "new". | | (events-security)= ## Security events ### Security event structure - `name`: The security event identifier (e.g., `authn_login_fail:tls`, `authz_fail:can_edit:/1.0/projects/foo`). - `level`: The severity level (`info`, `warning`). - `description`: A human-readable description of the event. - `requestor`: Who triggered the event (username, protocol, address, user agent). Omitted for daemon-level events. - `project`: The project the request targeted. Omitted for daemon-level events. - `request_path`: The REST API endpoint path. Omitted for daemon-level events. - `request_method`: The HTTP method used. Omitted for daemon-level events. ### Security event types LXD emits events across four categories. **Authentication events** | Event | Description | |-------|-------------| | `authn_login_fail:tls` | Failed authentication attempt when an untrusted TLS client certificate is presented to a protected endpoint. | | `authn_token_created:` | A new bearer token was issued for an identity. The identity UUID is included in the event identifier. | | `authn_token_revoked:` | A bearer token was revoked for an identity. The identity UUID is included in the event identifier. | | `authn_token_reuse` | A bearer token was presented in an invalid, expired, or otherwise disallowed way, indicating possible token reuse, tampering, or other misuse. | | `authn_certificate_change:` | A TLS client certificate was replaced. The old certificate fingerprint is included in the event identifier. | **Authorization events** | Event | Description | |-------|-------------| | `authz_fail::` | An action was denied due to insufficient permissions. Includes the required entitlement and the entity path that was accessed. | | `authz_admin:group_create:` | A new authentication group was created. | | `authz_admin:group_edit:` | An authentication group was modified. | | `authz_admin:group_delete:` | An authentication group was deleted. | | `authz_admin:idp_group_create:` | A new identity provider group was created. | | `authz_admin:idp_group_edit:` | An identity provider group was modified. | | `authz_admin:idp_group_delete:` | An identity provider group was deleted. | | `authz_admin:identity_create:/` | A new identity was created (TLS certificates or bearer tokens only). OIDC identities are not created via API actions. | | `authz_admin:identity_edit:/` | An identity was modified. | | `authz_admin:identity_delete:/` | An identity was deleted. | **Daemon lifecycle events** | Event | Description | |-------|-------------| | `sys_startup` | The LXD daemon has started. Emitted once the event system is fully available. | | `sys_shutdown` | The LXD daemon is shutting down. | | `sys_monitor_disabled` | Security event monitoring (Loki) was disabled via a configuration change. This is a `warning`-level event. | **User lifecycle events** | Event | Description | |-------|-------------| | `user_created` | A new identity has been created (TLS, bearer, OIDC, or cluster-link methods). For OIDC, this fires on first login. | | `user_updated` | An identity has been modified. For OIDC, this fires when user metadata changes on subsequent logins. | | `user_deleted` | An identity has been deleted. | (events-security-loki-fields)= ### Security event fields in Loki When security events are forwarded to Loki, they are stored in [OWASP (Open Worldwide Application Security Project)](https://owasp.org/) audit log format with the following key fields: | Field | Description | |-------|-------------| | `name` | The security event type identifier. | | `event_source` | The cluster member name where the event occurred. | | `cluster_identifier` | Unique identifier for the LXD cluster. | | `cluster_member_name` | Name of the cluster member. | | `project` | The project targeted by the request. Empty or omitted for daemon-level events. | | `request_method` | The HTTP method used. | | `request_uri` | The API endpoint path. | | `user_id` | The requestor identity in format `/`. | | `source_ip` | The source IP address of the request. | | `useragent` | The HTTP user agent string. | | `level` | The event severity (`info`, `warning`). | | `description` | Human-readable event description. | For how to monitor and query security events, see {ref}`howto-security-events`. --- ### Doc Faq (`doc/faq.md`) --- discourse: lxc:[Cgroup2 related issue](14705) --- # Frequently asked questions The following sections give answers to frequently asked questions. They explain how to resolve common issues and point you to more detailed information. ## Why do my instances not have network access? Most likely, your firewall blocks network access for your instances. See {ref}`network-bridge-firewall` for more information about the problem and how to fix it. Another frequent reason for connectivity issues is running LXD and Docker on the same host. See {ref}`network-lxd-docker` for instructions on how to fix such issues. ## How to enable the LXD server for remote access? By default, the LXD server is not accessible from the network, because it only listens on a local Unix socket. You can enable it for remote access by following the instructions in {ref}`server-expose`. ## When I do a `lxc remote add`, it asks for a token? To be able to access the remote API, clients must authenticate with the LXD server. See {ref}`server-authenticate` for instructions on how to authenticate using a trust token. ## Why should I not run privileged containers? A privileged container can do things that affect the entire host - for example, it can use things in `/sys` to reset the network card, which will reset it for the entire host, causing network blips. See {ref}`container-security` for more information. Almost everything can be run in an unprivileged container, or - in cases of things that require unusual privileges, like wanting to mount NFS file systems inside the container - you might need to use bind mounts. ## Can I bind-mount my home directory in a container? Yes, you can do this by using a {ref}`disk device `: lxc config device add container-name home disk source=/home/${USER} path=/home/ubuntu For unprivileged containers, you need to make sure that the user in the container has working read/write permissions. Otherwise, all files will show up as the overflow UID/GID (`65536:65536`) and access to anything that's not world-readable will fail. Use either of the following methods to grant the required permissions: - Pass `shift=true` to the [`lxc config device add`](lxc_config_device_add.md) call. This depends on the kernel and file system supporting either idmapped mounts (see [`lxc info`](lxc_info.md)). - Add a `raw.idmap` entry (see [Idmaps for user namespace](userns-idmap.md)). - Place recursive POSIX ACLs on your home directory. Privileged containers do not have this issue because all UID/GID in the container are the same as outside. But that's also the cause of most of the security issues with such privileged containers. ## How can I run Docker inside a LXD container? ```{youtube} https://www.youtube.com/watch?v=_fCSSEyiGro :title: Running Docker inside of a LXD container ``` To run Docker inside a LXD container, set the {config:option}`instance-security:security.nesting` option of the container to `true`: lxc config set security.nesting true If you plan to use the OverlayFS storage driver in Docker, you should also set the {config:option}`instance-security:security.syscalls.intercept.mknod` and {config:option}`instance-security:security.syscalls.intercept.setxattr` options to `true`. See [`mknod` / `mknodat`](syscall-mknod) and [`setxattr`](syscall-setxattr) for more information. Note that LXD containers cannot load kernel modules, so depending on your Docker configuration, you might need to have extra kernel modules loaded by the host. You can do so by setting a comma-separated list of kernel modules that your container needs: lxc config set linux.kernel_modules In addition, creating a `/.dockerenv` file in your container can help Docker ignore some errors it's getting due to running in a nested environment. ## Where does the LXD client (`lxc`) store its configuration? The [`lxc`](lxc.md) command stores its configuration under `~/.config/lxc`, or in `~/snap/lxd/common/config` for snap users. Various configuration files are stored in that directory, for example: - `client.crt`: client certificate (generated on demand) - `client.key`: client key (generated on demand) - `config.yml`: configuration file (info about `remotes`, `aliases`, etc.) - `servercerts/`: directory with server certificates belonging to `remotes` ## Why can I not ping my LXD instance from another host? Many switches do not allow MAC address changes, and will either drop traffic with an incorrect MAC or disable the port totally. If you can ping a LXD instance from the host, but are not able to ping it from a different host, this could be the cause. The way to diagnose this problem is to run a `tcpdump` on the uplink and you will see either ``ARP Who has `xx.xx.xx.xx` tell `yy.yy.yy.yy` ``, with you sending responses but them not getting acknowledged, or ICMP packets going in and out successfully, but never being received by the other host. (faq-monitor)= ## How can I monitor what LXD is doing? To see detailed information about what LXD is doing and what processes it is running, use the [`lxc monitor`](lxc_monitor.md) command. For example, to show a human-readable output of all types of messages, enter the following command: lxc monitor --pretty See [`lxc monitor --help`](lxc_monitor.md) for all options, and {doc}`debugging` for more information. ## Why does LXD stall when creating an instance? Check if your storage pool is out of space (by running [`lxc storage info `](lxc_storage_info.md)). In that case, LXD cannot finish unpacking the image, and the instance that you're trying to create shows up as stopped. To get more insight into what is happening, run [`lxc monitor`](lxc_monitor.md) (see {ref}`faq-monitor`), and check `sudo dmesg` for any I/O errors. ## Why does starting containers suddenly fail? If starting containers suddenly fails with a cgroup-related error message (`Failed to mount "/sys/fs/cgroup"`), this might be due to running a VPN client on the host. This is a known issue for both [Mullvad VPN](https://github.com/mullvad/mullvadvpn-app/issues/3651) and [Private Internet Access VPN](https://github.com/pia-foss/desktop/issues/50), but might occur for other VPN clients as well. The problem is that the VPN client mounts the `net_cls` cgroup1 over cgroup2 (which LXD uses). The easiest fix for this problem is to stop the VPN client and unmount the `net_cls` cgroup1 with the following command: umount /sys/fs/cgroup/net_cls If you need to keep the VPN client running, mount the `net_cls` cgroup1 in another location and reconfigure your VPN client accordingly. See [this Discourse post](https://discuss.linuxcontainers.org/t/help-help-help-cgroup2-related-issue-on-ubuntu-jammy-with-mullvad-and-privateinternetaccess-vpn/14705/18) for instructions for Mullvad VPN. ## Why does LXD not start on Ubuntu 22.04 LTS or earlier? If you are running LXD on Ubuntu 22.04 LTS or earlier, you might be missing support for ZFS 2.2 in the kernel (see the {ref}`requirements `). If LXD fails to start, check the `/var/snap/lxd/common/lxd/logs/lxd.log` log file for the following error to see if the reason is missing ZFS support: Error: Required tool ‘zpool’ is missing If you are on Ubuntu 22.04 LTS, you can resolve the issue by installing the HWE kernel and rebooting the nodes to provide the required kernel drivers for ZFS 2.2: sudo apt-get update sudo apt-get install linux-generic-hwe-22.04 If you are on earlier versions of Ubuntu, you should use a compatible LTS release of LXD. (faq-gpu-passthrough-stop)= ## Why does my VM stop responding when I try to pass through a GPU? If you try to pass through a GPU with a large amount of VRAM, the VM might stop responding during boot or fail to start. This is often caused by the default {abbr}`MMIO (Memory-Mapped Input/Output)` window size in QEMU being too small to map the GPU's memory. To resolve this, stop the instance, then increase the available 64-bit PCI MMIO address space by setting the following values in {config:option}`instance-raw:raw.qemu`: ```bash lxc config set raw.qemu=' -global q35-pcihost.pci-hole64-size=2048G -fw_cfg name=opt/ovmf/X-PciMmio64Mb,string=65536 ' ``` These settings reserve sufficient 64-bit MMIO space in both the QEMU host and the guest firmware ({abbr}`OVMF (Open Virtual Machine Firmware)`), which is required for GPUs with large {abbr}`BARs (Base Address Registers)`. --- ### Doc Getting Started (`doc/getting_started.md`) --- relatedlinks: "[Running LXD in production - YouTube](https://www.youtube.com/watch?v=QyXOOE_4cm0)" myst: html_meta: description: An index of how-to guides for getting started with LXD, including installation, initialization, and accessing the UI and offline documentation. --- (getting-started)= # Getting started The following how-to guides cover the initial steps for setting up and accessing LXD. ## Perform initial setup LXD is most commonly installed using its snap, but other installation methods are possible. Afterward, LXD can be initialized using an interactive CLI or a preseed file. ```{toctree} :maxdepth: 1 Install LXD Initialize LXD ``` ## Access the UI and offline documentation The LXD UI client provides a graphical, browser-based alternative to the CLI for interacting with the LXD server. The offline documentation is especially useful for air-gapped deployments. ```{toctree} :maxdepth: 1 Access the UI Access documentation locally ``` ## Watch videos The following clip gives a quick and easy introduction for standard use cases:
A series of demos and tutorials is also available on [YouTube](https://www.youtube.com/c/LXDvideos): ## Related topics {{getting_started_tut}} {{getting_started_exp}} {{getting_started_ref}} --- ### Doc Guest Os Compatibility (`doc/guest-os-compatibility.md`) --- myst: html_meta: description: Compatibility matrix for guest operating systems in LXD virtual machines and containers. --- (guest-os-compatibility)= # Guest OS compatibility ## Virtual machines The following operating systems (OS) were tested as virtual machine guest running on top of on LXD `5.21/stable`. Each OS was tested by doing a manual installation using the official ISO as provided by the vendor. OS vendor | OS version | OS support | [LXD agent](#lxd-agent) | VirtIO-SCSI | VirtIO-BLK | NVMe | CSM (BIOS) | UEFI | Secure Boot :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- CentOS | CentOS 6.10 [^1] | EOL | ❌ [^2] | ✅ | ❌ [^6] | 🟢 | ✅ | ❌ | ❌ CentOS | CentOS 7.9 | EOL | ❌ [^2] | ✅ | 🟢 | 🟢 | 🟢 | ✅ | ✅ CentOS | CentOS 8.5 | EOL | ✅ | ✅ | 🟢 | 🟢 | 🟢 | ✅ | ✅ CentOS | CentOS 8-Stream | EOL | ✅ | ✅ | 🟢 | 🟢 | 🟢 | ✅ | ✅ CentOS | CentOS 9-Stream | Supported | ✅ | ✅ | 🟢 | 🟢 | 🟢 | ✅ | ✅ Red Hat | RHEL 7.9 | EOL | ❌ [^2] | ✅ | 🟢 | 🟢 | 🟢 | ✅ | ✅ Red Hat | RHEL 8.10 | Supported | ✅ | ✅ | 🟢 | 🟢 | 🟢 | ✅ | ✅ Red Hat | RHEL 9.4 | Supported | ✅ | ✅ | 🟢 | 🟢 | 🟢 | ✅ | ✅ SUSE | SLES 12 SP5 | Supported | ✅ | ✅ | 🟢 | 🟢 | 🟢 | ✅ | ✅ SUSE | SLES 15 SP6 | Supported | ✅ | ✅ | 🟢 | 🟢 | 🟢 | ✅ | ✅ Ubuntu | 14.04.6 LTS | EOL | ❌ [^7] | ✅ | 🟢 | 🟢 | 🟢 | ✅ | ✅ Ubuntu | 16.04.7 LTS | ESM | ✅ [^8][^9] | ✅ | 🟢 | 🟢 | 🟢 | ✅ | ✅ Ubuntu | 18.04.6 LTS | ESM | ✅ [^9] | ✅ | 🟢 | 🟢 | 🟢 | ✅ | ✅ Ubuntu | 20.04.6 LTS | Supported | ✅ | ✅ | 🟢 | 🟢 | 🟢 | ✅ | ✅ Ubuntu | 22.04.4 LTS | Supported | ✅ | ✅ | 🟢 | 🟢 | 🟢 | ✅ | ✅ Ubuntu | 24.04.1 LTS | Supported | ✅ | ✅ | 🟢 | 🟢 | 🟢 | ✅ | ✅ Windows | Server 2012 | Supported | ➖ | ✅ | 🟢 | ❌ | 🟢 | ✅ | ✅ Windows | Server 2016 | Supported | ➖ | ✅ | 🟢 | 🟢 [^3] | ❌ [^5] | ✅ | ✅ Windows | Server 2019 | Supported | ➖ | ✅ | 🟢 | 🟢 | ❌ [^5] | ✅ | ✅ Windows | Server 2022 | Supported | ➖ | ✅ | 🟢 | 🟢 | ❌ [^5] | ✅ | ✅ Windows | 10 22H2 | Supported | ➖ | ✅ | 🟢 | 🟢 | ❌ [^5] | ✅ | ✅ Windows | 11 23H2 [^4] | Supported | ➖ | ✅ | 🟢 | 🟢 | ❌ | ✅ | ✅ [^1]: No network support despite having VirtIO-NET module. [^2]: Support for 9P or `virtiofs` not available. Note: CentOS 7 has a `kernel-plus` kernel with 9P support allowing LXD agent to work (with `selinux=0`). [^3]: NVMe disks are visible but the installer lists all 255 namespaces slowing down the initialization. [^4]: A virtual TPM is required. [^5]: The OS installer stalls when booting in CSM/BIOS mode. [^6]: The OS installer stalls when booting with VirtIO-BLK despite having VirtIO-BLK supported by the kernel. [^7]: This Linux version does not use `systemd` which the LXD agent requires. [^8]: Requires the HWE kernel (`4.15`) for proper `vsock` support which is required by the LXD agent. [^9]: The `lxd-agent-installer` package is not available so `lxd-agent` has to be manually setup (see {ref}`lxd-agent-manual-install`) or through `cloud-init` (see {ref}`vm-cloud-init-config`). Legend | Icon :--- | :--- recommended | ✅ supported | 🟢 not applicable | ➖ not supported | ❌ ## Notes ### LXD agent The LXD agent provides the ability to execute commands inside of the virtual machine guest without relying on traditional access solution like secure shell (SSH) or Remote Desktop Protocol (RDP). This agent is only supported on Linux guests using `systemd`. For how to manually setup the agent, see {ref}`lxd-agent-manual-install`. ### BIOS boot ```bash lxc config set v1 boot.mode=bios ``` ### Virtual TPM ```bash lxc config device add v1 vtpm tpm path=/dev/tpm0 ``` ### VirtIO-BLK or NVMe ```bash lxc config device override v1 root io.bus=virtio-blk # or lxc config device override v1 root io.bus=nvme ``` ### Disconnect the ISO ```bash lxc config device remove v1 iso ``` ## Containers Unlike virtual machines, container guests rely on the host's kernel for execution. Since each Linux distribution ships with a unique set of features supported by their official kernels, the possibilities are almost endless. As such, the following compatibility table focuses on hosts running Ubuntu LTS releases with LXD `5.21/stable` and Ubuntu releases as container guests. The main compatibility factor is the `cgroup` version required by the container and supported by the host. Host OS / Guest OS | Ubuntu 16.04 LTS | Ubuntu 18.04 LTS | Ubuntu 20.04 LTS | Ubuntu 22.04 LTS | Ubuntu 24.04 LTS | Ubuntu 24.10 :--- | :--- | :--- | :--- | :--- | :--- | :--- Ubuntu 20.04 LTS 5.4.0 [^10] | 🟢 | 🟢 | 🟢 | 🟢 | 🟢 | ❌ [^11] Ubuntu 20.04 LTS 5.15.0 (HWE) | ✅ | ✅ | ✅ | ✅ | ✅ | 🟢 [^12] Ubuntu 22.04 LTS 5.15.0 | 🟢 [^13] | ✅ | ✅ | ✅ | ✅ | ✅ Ubuntu 22.04 LTS 6.8.0 (HWE) | 🟢 [^13] | ✅ | ✅ | ✅ | ✅ | ✅ Ubuntu 24.04 LTS 6.8.0 | 🟢 [^13] | ✅ | ✅ | ✅ | ✅ | ✅ [^10]: The 5.4.0 kernel is below the minimum required version (see {ref}`requirements`) [^11]: Ubuntu 24.10 and later require `cgroupv2` which is not supported by Ubuntu 20.04 LTS regular kernel. [^12]: Requires enabling `cgroupv2` support by booting with `systemd.unified_cgroup_hierarchy=1` [^13]: Requires enabling `cgroupv1` support by booting with `systemd.unified_cgroup_hierarchy=0` Legend | Icon :--- | :--- recommended | ✅ supported | 🟢 not applicable | ➖ not supported | ❌ --- ### Doc Image Handling (`doc/image-handling.md`) --- myst: html_meta: description: Explanation of how LXD uses local and remote images, including image registries, caching, auto-update, and special image properties. --- (about-images)= # Local and remote images ```{youtube} https://www.youtube.com/watch?v=wT7IDjo0Wgg :title: Image servers and image handling in LXD ``` LXD uses an image-based workflow. Each instance is based on an image, which contains a basic operating system (for example, a Linux distribution) and some LXD-related information. To download images, LXD uses {ref}`image registries `. An image registry is a server-side entity that points to a source of images, such as a [simple streams](https://git.launchpad.net/simplestreams/tree/) server or another LXD server. LXD comes pre-configured with a set of {ref}`built-in image registries ` for the most common image sources, and you can {ref}`add your own `. LXD uses the [Go TLS stack](https://pkg.go.dev/crypto/tls) to connect to the image sources. You can also create your own images, either based on an existing instance or a rootfs image. Because image registries are managed on the server, all clients of a LXD server (and all members of a cluster) share the same set of image sources. ```{note} In earlier versions of LXD, images were downloaded through client-side *remotes*. Downloading images now goes through image registries instead. In the LXD command-line client, remotes are still used to connect to other LXD servers, but no longer to source images. For backwards compatibility, older clients that still reference a remote image source keep working, as long as a matching image registry exists on the server (LXD auto-creates one for a small set of well-known public image sources). ``` Once an image is downloaded, it is stored in the local image store. You can copy local images to other LXD servers and use a local image to create a remote instance. Each image is identified by a fingerprint (SHA256). To make it easier to manage images, LXD allows defining one or more aliases for each image. (image-registries-diagram)= ```{figure} /images/security/image-registries.svg :width: 100% :alt: Illustration of how image registries point to image sources for image downloads. Image registries ``` ## Caching When you create an instance using an image from a registry, LXD downloads the image and caches it locally. It is stored in the local image store with the cached flag set. The image is kept locally as a private image until either: - The image has not been used to create a new instance for the number of days set in {config:option}`server-images:images.remote_cache_expiry`. - The image's expiry date (one of the image properties; see {ref}`images-manage-edit` for information on how to change it) is reached. LXD keeps track of the image usage by updating the `last_used_at` image property every time a new instance is spawned from the image. ## Auto-update LXD can automatically keep images that come from an image registry up to date. ```{note} Only images that are requested through an alias can be updated. If you request an image through a fingerprint, you request an exact image version. ``` Whether auto-update is enabled for an image depends on how the image was downloaded: - If the image was downloaded and cached when creating an instance, it is automatically updated if {config:option}`server-images:images.auto_update_cached` was set to `true` (the default) at download time. - If the image was copied from another server using the [`lxc image copy`](lxc_image_copy.md) command, it is automatically updated only if the `--auto-update` flag was specified. You can change this behavior for an image by [editing the `auto_update` property](images-manage-edit). On startup and after every {config:option}`server-images:images.auto_update_interval` (by default, every six hours), the LXD daemon checks for more recent versions of all the images in the store that are marked to be auto-updated and have a recorded source registry. When a new version of an image is found, it is downloaded into the image store. Then any aliases pointing to the old image are moved to the new one, and the old image is removed from the store. To not delay instance creation, LXD does not check if a new version is available when creating an instance from a cached image. This means that the instance might use an older version of an image for the new instance until the image is updated at the next update interval. ## Special image properties Image properties that begin with the prefix `requirements` (for example, `requirements.XYZ`) are used by LXD to determine the compatibility of the host system and the instance that is created based on the image. If these are incompatible, LXD does not start the instance. The following requirements are supported: Key | Type | Default | Description :-- | :--- | :------ | :---------- `requirements.secureboot` | string | - | If set to `false`, indicates that the image cannot boot under secure boot. `requirements.cgroup` | string | - | If set to `v1`, indicates that the image requires the host to run cgroup v1. `requirements.nesting` | bool | - | If set to `true`, indicates that the image cannot work without nesting enabled. ## Related topics {{images_how}} {{images_ref}} --- ### Doc Images (`doc/images.md`) --- myst: html_meta: description: An index of how-to guides for LXD images, including using remote images, managing local images, and copying, importing, or creating images. --- (images)= # Images These how-to guides cover common operations related to LXD images. ## Work with existing images Images are used to create instances. Pre-configured images can be downloaded from remote servers and managed locally, including configuring profiles to use specific images. ```{toctree} :titlesonly: Manage image registries Use remote images Manage images Associate profiles ``` ## Import and create images Images can be copied or imported from other servers or files. They can also be created from scratch, or from existing instances or snapshots. ```{toctree} :titlesonly: Copy and import images Create images ``` ## Related topics {{images_exp}} {{images_ref}} --- ### Doc Index (`doc/index.rst`) .. Turn spell check off temporarily until this issue is resolved: https://github.com/canonical/documentation-style-guide/issues/207 .. vale off :relatedlinks: [Run system containers with LXD](https://canonical.com/lxd), [Open source for beginners: setting up your dev environment with LXD](https://ubuntu.com/blog/open-source-for-beginners-dev-environment-with-lxd) .. meta:: :description: LXD home page .. _lxd-homepage: LXD === LXD (|lxd-listen-anchor-open|\ ``[lɛks'di:]``\ |lxd-listen-anchor-close|) is a modern, secure and powerful system container and virtual machine manager. .. raw:: html .. |lxd-listen-anchor-open| raw:: html .. |lxd-listen-anchor-close| raw:: html 🔈 .. include:: ../README.md :parser: myst_parser.docutils_ :start-after: :end-before: ---- .. toctree:: :hidden: :titlesonly: self Tutorial howto/index explanation/index reference/index .. only:: html In this documentation --------------------- Start here ~~~~~~~~~~ Follow the tutorial for a guided introduction to LXD, and refer to these guides for installation and initialization instructions. .. domain:: .. slice:: Get started :doc:`Tutorial ` :doc:`Installation ` :doc:`Initialization ` :doc:`Preseed file fields for non-interactive configuration ` System containers and virtual machines ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ LXD runs workloads on system containers or virtual machines. These instances are created using images and can be grouped under projects. .. domain:: .. slice:: Instances :doc:`About containers and virtual machines ` slice :doc:`Create ` slice :doc:`Configure ` slice :doc:`Manage ` slice :doc:`Back up ` slice :doc:`Migrate ` slice :doc:`Import ` slice :doc:`Guest OS compatibility matrix ` :doc:`Configuration options ` :doc:`Store configuration options in profiles ` :doc:`Automate configuration with cloud-init ` .. slice:: Images :doc:`About local and remote images ` :doc:`Built-in image registries ` :doc:`Manage image registries ` :doc:`Manage images ` :doc:`Image format ` .. slice:: Projects :doc:`Overview ` slice :doc:`Create and configure ` slice :doc:`Confine users to projects ` :doc:`Configuration options ` slice Clusters ~~~~~~~~ LXD servers can be joined together into clusters, and clusters can be connected through cluster links. .. domain:: .. slice:: Clusters and cluster members :doc:`Overview ` slice :doc:`Form a cluster ` :doc:`Use placement groups to distribute instances across a cluster ` :doc:`Recover a cluster ` :doc:`Set up a highly available virtual IP ` :doc:`Cluster member configuration options ` .. slice:: Multiple clusters :doc:`Create cluster links ` :doc:`Manage cluster links ` :doc:`Cluster link configuration options ` Storage and networks ~~~~~~~~~~~~~~~~~~~~ Each LXD server is configured with storage and network options. These guides will help you understand and work with these resources. .. domain:: .. slice:: Storage :doc:`Overview ` slice :doc:`Driver types and configuration options ` :doc:`Manage pools ` :doc:`Manage volumes ` :doc:`Manage buckets ` :doc:`Create or move instances in a pool ` :doc:`Back up volumes ` :doc:`Move or copy volumes ` .. slice:: Networking :doc:`Networking setups ` :doc:`Network types and configuration options ` :doc:`Create ` slice :doc:`Configure networks ` :doc:`Configure LXD as a BGP server ` :doc:`Configure ACLs ` :doc:`Configure forwards ` :doc:`Configure network zones ` :doc:`Set up OVN ` :doc:`Configure load balancers ` :doc:`Configure peer routing ` Access LXD ~~~~~~~~~~ These guides help you to access and communicate with LXD. .. domain:: .. slice:: Server :doc:`Expose the server to the network ` :doc:`Server configuration options ` :doc:`Production server settings ` :doc:`Supported server architectures ` .. slice:: Authentication :doc:`Overview ` slice :doc:`Access the graphical UI ` :doc:`Use single sign-on with OIDC ` :doc:`Use bearer tokens ` :doc:`Add remote servers ` :doc:`Authenticate to the DevLXD API ` .. slice:: Authorization :doc:`Overview ` slice :doc:`Permissions reference ` .. slice:: Client-server communication :doc:`REST API reference ` :doc:`lxc CLI man pages ` :doc:`About the lxd-lxc CLIs ` :doc:`DevLXD API reference ` Quality ~~~~~~~ Follow these guides to secure your LXD deployment and optimize its performance. .. domain:: .. slice:: Security :doc:`Overview ` slice :doc:`Harden security ` :doc:`Monitor security events ` :doc:`Decommission LXD ` .. slice:: Performance :doc:`Benchmark performance ` :doc:`Performance tuning ` :doc:`Increase network bandwidth ` .. slice:: Monitoring :doc:`Monitor metrics ` :doc:`Send logs to Loki ` :doc:`Visualize metrics and logs with Grafana ` :doc:`Metrics reference ` :doc:`Events reference ` Lifecycle and administration ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ These guides cover lifecycle and ongoing administration concerns. .. domain:: .. slice:: Update and upgrade :doc:`Snap updates and upgrades ` :doc:`Releases and snap reference ` :doc:`Release notes ` :doc:`Deprecation notices ` .. slice:: Troubleshooting :doc:`Configure a firewall ` :doc:`Troubleshoot instances ` :doc:`Troubleshoot networks ` :doc:`Troubleshoot Dqlite ` :doc:`Debug LXD ` :doc:`Track a bugfix in the LXD snap ` :doc:`Frequently asked questions ` .. slice:: Disaster recovery :doc:`Overview ` slice :doc:`Set up replicators ` :doc:`Manage replicators ` :doc:`Perform disaster recovery with replicators ` :doc:`Set up storage replication ` :doc:`Perform disaster recovery with storage replication ` :doc:`Back up a server ` :doc:`Recover LXD database records ` :doc:`Replicator configuration options ` How this documentation is organized ----------------------------------- This documentation uses the Diátaxis documentation structure. - The :ref:`Tutorial ` takes you step-by-step through installing and initializing LXD, and learning how to use basic features such as launching instances. - The :ref:`howtos` assume you have basic familiarity with LXD. They walk you through specific tasks, such as creating storage pools and managing clusters. - The :ref:`reference` guides include configuration options, API references, and other technical details. - The :ref:`explanation` section includes topic overviews and detailed explanations of key concepts, such as the difference between system containers and virtual machines. .. vale on Project and community --------------------- LXD is a member of the `Canonical `_ family. It’s an open source project that warmly welcomes community contributions, suggestions, fixes, and constructive feedback. Get involved ~~~~~~~~~~~~ * :ref:`Support ` * `Discussion forum `_ * :ref:`Contribute ` * `YouTube channel `_ Releases ~~~~~~~~ * :ref:`ref-release-notes` * `Release tarballs `_ Governance and policies ~~~~~~~~~~~~~~~~~~~~~~~ * `Code of conduct `_ Commercial support ~~~~~~~~~~~~~~~~~~ Thinking about using LXD for your next project? `Get in touch `_! --- ### Doc Installing (`doc/installing.md`) --- discourse: "[Discourse: Building custom LXD binaries for side loading into an existing snap installation](37327)" myst: html_meta: description: Install the LXD server and client on Linux using snap or other package managers, build from source, or install the client only on macOS and Windows. --- (installing)= # How to install LXD ````{only} integrated ```{admonition} For MicroCloud users :class: note The MicroCloud setup process installs LXD on cluster members. Thus, you do not need to follow the steps on this page. ``` ```` There are multiple approaches to installing LXD, depending on your Linux distribution, operating system, and use case. (installing-snap-package)= ## Install the LXD snap package The recommended way to install LXD is its [snap package](https://snapcraft.io/lxd), available for many Linux distributions. For alternative methods, see: {ref}`installing-other`, {ref}`installing-other-os`, or {ref}`installing-from-source`. ### Requirements - The LXD snap must be [available for your Linux distribution](https://snapcraft.io/lxd#distros). - The `snapd` daemon must be installed. See {ref}`snap:tutorials-install-the-daemon-index` in the Snap documentation for details. ### Install Use this command to install LXD from the recommended {ref}`default snap track ` (currently {{current_lts_track}}): ```bash sudo snap install lxd ``` If you are installing LXD on a {ref}`cluster member `, add the `--cohort="+"` flag to {ref}`keep cluster members synchronized ` to the same snap version: ```bash sudo snap install lxd --cohort="+" ``` Next, follow the {ref}`installing-snap-post` steps below. (installing-snap-channel)= #### Optionally specify a channel Channels correspond to different {ref}`LXD releases `. When unspecified, the LXD snap defaults to the most recent `stable` LTS, which is recommended for most use cases. To specify a different channel, add the `--channel` flag at installation: ```bash sudo snap install lxd --channel= [--cohort="+"] ``` For example, to use the `6/stable` channel, run: ```bash sudo snap install lxd --channel=6/stable ``` For details about LXD snap channels, see: {ref}`ref-snap-channels`. (installing-snap-post)= ### Post-installation Follow these steps after installing the LXD snap. (installing-snap-user)= #### Add the current user To allow the current user to interact with the LXD daemon, update the `lxd` group: ```bash getent group lxd | grep -qwF "$USER" || sudo usermod -aG lxd "$USER" ``` Afterward, apply the change to your current shell session by running: ```bash newgrp lxd ``` This only applies to the current shell. You will need to log out and log back in again for the change to appear in a new terminal. For more information, see the {ref}`installing-manage-access` section below. (installing-snap-hold-updates)= #### Hold or schedule updates When a new release is published to a snap channel, installed snaps following that channel update automatically by default. For {ref}`LXD clusters `, or on any machine where you want control over updates, you should override this default behavior by either holding or scheduling updates. See: {ref}`howto-snap-updates`. (installing-other)= ## Other Linux installation options Some Linux installations can use package managers other than Snap to install LXD. These managers all install the latest {ref}`feature release `. `````{tabs} ````{group-tab} Alpine Linux Run: ```bash apk add lxd ``` ```` ````{group-tab} Arch Linux Run: ```bash pacman -S lxd ``` ```` ````{group-tab} Fedora Fedora RPM packages for LXC/LXD are available in the [COPR repository](https://copr.fedorainfracloud.org/coprs/ganto/lxc4/). These are unofficial and minimally tested; use at your own risk. View the [installation guide](https://github.com/ganto/copr-lxc4/wiki) for details. ```` ````{group-tab} Gentoo Run: ```bash emerge --ask lxd ``` ```` ````` Following installation, make sure to {ref}`manage access to LXD `. (installing-other-os)= ## Other operating systems Builds of the [`lxc`](lxc.md) client are available for non-Linux operating systems to enable interaction with remote LXD servers. For more information, see: [About `lxd` and `lxc`](lxd-lxc). `````{tabs} ````{group-tab} macOS The [Homebrew](https://brew.sh) package manager must be installed on your system. To install the client from the latest {ref}`feature release ` of LXD, run: ```bash brew install lxc ``` ```` ````{group-tab} Windows The [Chocolatey](https://chocolatey.org) package manager must be installed on your system. To install the client from the latest {ref}`feature release ` of LXD, run: ```bash choco install lxc ``` ```` ````` (installing-native)= ## Native builds of the client You can find native builds of the [`lxc`](lxc.md) client on [GitHub](https://github.com/canonical/lxd): - Linux: [`bin.linux.lxc.aarch64`](https://github.com/canonical/lxd/releases/latest/download/bin.linux.lxc.aarch64), [`bin.linux.lxc.x86_64`](https://github.com/canonical/lxd/releases/latest/download/bin.linux.lxc.x86_64) - Windows: [`bin.windows.lxc.aarch64.exe`](https://github.com/canonical/lxd/releases/latest/download/bin.windows.lxc.aarch64.exe), [`bin.windows.lxc.x86_64.exe`](https://github.com/canonical/lxd/releases/latest/download/bin.windows.lxc.x86_64.exe) - macOS: [`bin.macos.lxc.aarch64`](https://github.com/canonical/lxd/releases/latest/download/bin.macos.lxc.aarch64), [`bin.macos.lxc.x86_64`](https://github.com/canonical/lxd/releases/latest/download/bin.macos.lxc.x86_64) To download a specific build: 1. Make sure that you are logged into your GitHub account. 1. Filter for the branch or tag that you are interested in (for example, the latest release tag or `main`). 1. Select the latest build and download the suitable artifact. These builds are for the [`lxc`](lxc.md) client only, not the LXD daemon. For an explanation of the differences, see: [About `lxd` and `lxc`](lxd-lxc). (installing-from-source)= (installing_from_source)= ## Install LXD from source These instructions for building and installing from source are suitable for developers who want to build the latest version of LXD, or to build a specific release of LXD which may not be offered by their Linux distribution. Source builds for integration into Linux distributions are not covered. This guide is written for Ubuntu 24.04 LTS on x86_64. We recommend having the latest versions of `liblxc` (see {ref}`LXC requirements `) available for LXD development. For convenience, `make deps` will pull the appropriate versions of `liblxc` and `dqlite` from their corresponding upstream Git repository. Additionally, LXD requires a modern Golang (see {ref}`requirements-go`) version to work. On Ubuntu, you can install these with: ```bash sudo apt update sudo apt install \ autoconf \ automake \ build-essential \ git \ libacl1-dev \ libapparmor-dev \ libcap-dev \ liblz4-dev \ libseccomp-dev \ libsqlite3-dev \ libtool \ libudev-dev \ libuv1-dev \ xfslibs-dev \ make \ meson \ ninja-build \ pkg-config \ python3-venv command -v snap >/dev/null || sudo apt-get install snapd sudo snap install --classic go ``` ```{note} If you use the `liblxc-dev` package and get compile time errors when building the `go-lxc` module, ensure that the value for `LXC_DEVEL` is `0` for your `liblxc` build. To check this, look at `/usr/include/lxc/version.h`. If the `LXC_DEVEL` value is `1`, replace it with `0` to work around the problem. It's a packaging bug that is now fixed, see [LP: #2039873](https://bugs.launchpad.net/ubuntu/+source/lxc/+bug/2039873). ``` For your local build of LXD to support virtual machines, you must install QEMU with: ```bash sudo apt install --no-install-recommends \ qemu-system-x86 \ qemu-block-extra \ qemu-utils \ qemu-system-modules-spice \ virtiofsd \ ovmf \ swtpm ``` There are a few storage drivers for LXD besides the default `dir` driver. Installing these tools adds a bit to `initramfs` and may slow down your host boot, but are needed if you'd like to use a particular driver: ```bash sudo apt install lvm2 thin-provisioning-tools sudo apt install btrfs-progs ``` At runtime, LXD might need the following packages to be installed on the host system: ```bash sudo apt update sudo apt install \ attr \ iproute2 \ nftables \ rsync \ squashfs-tools \ squashfs-tools-ng \ tar \ xz-utils # `nftables` can be replaced by `iptables` on older systems ``` To run the test suite or test related `make` targets, you'll also need: ```bash sudo apt update sudo apt install \ acl \ bind9-dnsutils \ btrfs-progs \ busybox-static \ curl \ dnsmasq-base \ dosfstools \ e2fsprogs \ expect \ iputils-ping \ jq \ netcat-openbsd \ s3cmd \ shellcheck \ socat \ sqlite3 \ swtpm \ xdelta3 \ xfsprogs \ yq ``` ### From source: Build the latest version These instructions for building from source are suitable for individual developers who want to build the latest version of LXD, or build a specific release of LXD which may not be offered by their Linux distribution. Source builds for integration into Linux distributions are not covered here and may be covered in detail in a separate document in the future. ```bash git clone https://github.com/canonical/lxd cd lxd ``` This will download the current development tree of LXD and place you in the source tree. Then proceed to the instructions below to actually build and install LXD. ### From source: Build a release The LXD release tarballs bundle a complete dependency tree as well as a local copy `libdqlite` for LXD's database setup. ```bash tar zxvf lxd-4.18.tar.gz cd lxd-4.18 ``` This will unpack the release tarball and place you inside of the source tree. Then proceed to the instructions below to actually build and install LXD. ### Start the build The actual building is done by two separate invocations of the {file}`Makefile`: `make deps` -- which builds libraries required by LXD -- and `make`, which builds LXD itself. At the end of `make deps`, a message will be displayed which will specify environment variables that should be set prior to invoking `make`. As new versions of LXD are released, these environment variable settings may change, so be sure to use the ones displayed at the end of the `make deps` process, as the ones below (shown for example purposes) may not exactly match what your version of LXD requires: We recommend having at least 2GiB of RAM to allow the build to complete. ```{terminal} make deps ... make[1]: Leaving directory '/root/go/deps/dqlite' # environment Please set the following in your environment (possibly ~/.bashrc) # export CGO_CFLAGS="${CGO_CFLAGS} -I$(go env GOPATH)/deps/dqlite/include/" # export CGO_LDFLAGS="${CGO_LDFLAGS} -L$(go env GOPATH)/deps/dqlite/.libs/" # export LD_LIBRARY_PATH="$(go env GOPATH)/deps/dqlite/.libs/${LD_LIBRARY_PATH}" # export CGO_LDFLAGS_ALLOW="(-Wl,-wrap,pthread_create)|(-Wl,-z,now)" ``` ```{terminal} make ``` ### From source: Install Once the build completes, you simply keep the source tree, add the directory referenced by `$(go env GOPATH)/bin` to your shell path, and set the `LD_LIBRARY_PATH` variable printed by `make deps` to your environment. This might look something like this for a `~/.bashrc` file: ```bash export PATH="${PATH}:$(go env GOPATH)/bin" export LD_LIBRARY_PATH="$(go env GOPATH)/deps/dqlite/.libs/:${LD_LIBRARY_PATH}" ``` Now, the `lxd` and `lxc` binaries will be available to you and can be used to set up LXD. The binaries will automatically find and use the dependencies built in `$(go env GOPATH)/deps` thanks to the `LD_LIBRARY_PATH` environment variable. ### Machine setup You'll need sub{u,g}ids for root, so that LXD can create the unprivileged containers: ```bash echo "root:1000000:1000000000" | sudo tee -a /etc/subuid /etc/subgid ``` By default, only users added to the `lxd` group can interact with the LXD daemon. Installing from source doesn't guarantee that the `lxd` group exists in the system. If you want the current user (or any other user) to be able to interact with the LXD daemon, create the group and add the user to it: ```bash getent group lxd >/dev/null || sudo groupadd --system lxd # create the group if needed getent group lxd | grep -qwF "$USER" || sudo usermod -aG lxd "$USER" ``` ```{include} installing.md :start-after: :end-before: ``` Now you can run the daemon (the `--group lxd` bit allows everyone in the `lxd` group to talk to LXD): ```bash sudo PATH=${PATH} LD_LIBRARY_PATH=${LD_LIBRARY_PATH} $(go env GOPATH)/bin/lxd --group lxd ``` ```{note} If `newuidmap/newgidmap` tools are present on your system and `/etc/subuid`, `/etc/subgid` exist, they must be configured to allow the root user a contiguous range of at least 65536 UIDs/GIDs. ``` ### Shell completions Shell completion profiles can be generated with `lxc completion ` (e.g. `lxc completion bash`). Supported shells are `bash`, `zsh`, `fish`, and `powershell`. ```bash lxc completion bash > /etc/bash_completion.d/lxc # generating completions for bash as an example . /etc/bash_completion.d/lxc ``` (installing-manage-access)= ## Manage access to LXD Access control for LXD is based on group membership. The root user and all members of the `lxd` group can interact with the local daemon. On Ubuntu images, the `lxd` group already exists and the root user is automatically added to it. The group is also created during installation if you {ref}`installed LXD from the snap `. To check if the `lxd` group exists, run: ```bash getent group lxd ``` If this command returns no result, the `lxd` group is missing from your system. This might be the case if you {ref}`installed LXD from source `. To create the group and restart the LXD daemon, run: ```bash getent group lxd >/dev/null || sudo groupadd --system lxd ``` Afterward, add trusted users to the group so they can use LXD. The following command adds the current user: ```bash getent group lxd | grep -qwF "$USER" || sudo usermod -aG lxd "$USER" ``` ```{include} installing.md :start-after: :end-before: ``` ````{admonition} Important security notice :class: important % Include content from [../README.md](../README.md) ```{include} ../README.md :start-after: :end-before: ``` For more information, see {ref}`security-daemon-access`. ```` (installing-upgrade)= ## Updates and upgrades For information on updates and upgrades, see the relevant sections in the following guides: How-to guide: - {ref}`howto-snap` Reference: - {ref}`ref-releases-snap` --- ### Doc Instance Exec (`doc/instance-exec.md`) (run-commands)= # How to run commands in an instance LXD allows to run commands inside an instance using the LXD client or the API, without needing to access the instance through the network. For containers, this always works and is handled directly by LXD. For virtual machines, the `lxd-agent` process must be running inside of the virtual machine for this to work. ```{note} The UI does not currently support sending commands to an instance. However, it provides a terminal that gives you {ref}`shell access to your instance `. ``` ## Run commands inside your instance ````{tabs} ```{group-tab} CLI To run a single command from the terminal of the host machine, use the [`lxc exec`](lxc_exec.md) command: lxc exec -- For example, enter the following command to update the package list on your container: lxc exec my-instance -- apt-get update ``` ```{group-tab} API Send a POST request to the instance's `exec` endpoint to run a single command from the terminal of the host machine: lxc query --request POST /1.0/instances//exec --data '{ "command": [ "" ] }' For example, enter the following command to update the package list on your container: lxc query --request POST /1.0/instances/my-instance/exec --data '{ "command": [ "apt-get", "update" ] }' See [`POST /1.0/instances/{name}/exec`](swagger:/instances/instance_exec_post) for more information. ``` ```` ### Execution mode LXD can execute commands either interactively or non-interactively. ````{tabs} ```{group-tab} CLI In interactive mode, a pseudo-terminal device (PTS) is used to handle input (stdin) and output (stdout, stderr). This mode is automatically selected by the CLI if connected to a terminal emulator (and not run from a script). To force interactive mode, add either `--force-interactive` or `--mode interactive` to the command. In non-interactive mode, pipes are allocated instead (one for each of stdin, stdout and stderr). This method allows running a command and properly getting separate stdin, stdout and stderr as required by many scripts. To force non-interactive mode, add either `--force-noninteractive` or `--mode non-interactive` to the command. ``` ```{group-tab} API In both modes, the operation creates a control socket that can be used for out-of-band communication with LXD. You can send signals and window sizing information through this socket. Interactive mode : In interactive mode, the operation creates an additional single bi-directional WebSocket. To force interactive mode, add `"interactive": true` and `"wait-for-websocket": true` to the request data. For example: lxc query --request POST /1.0/instances/my-instance/exec --data '{ "command": [ "/bin/bash" ], "interactive": true, "wait-for-websocket": true }' Non-interactive mode : In non-interactive mode, the operation creates three additional WebSockets: one each for stdin, stdout, and stderr. To force non-interactive mode, add `"interactive": false` to the request data. When running a command in non-interactive mode, you can instruct LXD to record the output of the command. To do so, add `"record-output": true` to the request data. You can then send a request to the `exec-output` endpoint to retrieve the list of files that contain command output: lxc query --request GET /1.0/instances//logs/exec-output To display the output of one of the files, send a request to one of the files: lxc query --request GET /1.0/instances//logs/exec-output/ When you don't need the command output anymore, you can delete it: lxc query --request DELETE /1.0/instances//logs/exec-output/ See [`GET /1.0/instances/{name}/logs/exec-output`](swagger:/instances/instance_exec-output_get), [`GET /1.0/instances/{name}/logs/exec-output/{filename}`](swagger:/instances/instance_exec-output_get), and [`DELETE /1.0/instances/{name}/logs/exec-output/{filename}`](swagger:/instances/instance_exec-output_delete) for more information. ``` ```` ### User, groups and working directory LXD has a policy not to read data from within the instances or trust anything that can be found in the instance. Therefore, LXD does not parse files like `/etc/passwd`, `/etc/group` or `/etc/nsswitch.conf` to handle user and group resolution. As a result, LXD doesn't know the home directory for the user or the supplementary groups the user is in. By default, LXD runs commands as `root` (UID 0) with the default group (GID 0) and the working directory set to `/root`. You can override the user, group and working directory by specifying absolute values. ````{tabs} ```{group-tab} CLI You can override the default settings by adding the following flags to the [`lxc exec`](lxc_exec.md) command: - `--user` - the user ID for running the command - `--group` - the group ID for running the command - `--cwd` - the directory in which the command should run ``` ```{group-tab} API You can override the default settings by adding the following fields to the request data: - `"user": ` - the user ID for running the command - `"group": ` - the group ID for running the command - `"cwd": ""` - the directory in which the command should run ``` ```` ### Environment You can pass environment variables to an exec session in the following two ways: Set environment variables as instance options : ````{tabs} ```{group-tab} CLI To set the `` environment variable to `` in the instance, set the `environment.` instance option (see {config:option}`instance-miscellaneous:environment.*`): lxc config set environment.= ``` ```{group-tab} API To set the `` environment variable to `` in the instance, set the `environment.` instance option (see {config:option}`instance-miscellaneous:environment.*`): lxc query --request PATCH /1.0/instances/ --data '{ "config": { "environment.": "" } }' ``` ```{group-tab} UI To set the `` environment variable to `` in the instance, go to the instance detail page, switch to the {guilabel}`Configuration` tab and select {guilabel}`YAML configuration`. Then click {guilabel}`Edit instance`. Add the `environment.` configuration under the `config` section. For example: config: environment.: "" Click {guilabel}`Save changes`. ``` ```` Pass environment variables to the exec command : ````{tabs} ```{group-tab} CLI To pass an environment variable to the exec command, use the `--env` flag. For example: lxc exec --env = -- ``` ```{group-tab} API To pass an environment variable to the exec command, add an `environment` field to the request data. For example: lxc query --request POST /1.0/instances//exec --data '{ "command": [ "" ], "environment": { "": "" } }' ``` ```` In addition, LXD sets the following default values (unless they are passed in one of the ways described above): ```{list-table} :header-rows: 1 * - Variable name - Condition - Value * - `PATH` - \- - Concatenation of: - `/usr/local/sbin` - `/usr/local/bin` - `/usr/sbin` - `/usr/bin` - `/sbin` - `/bin` - `/snap` (if applicable) - `/etc/NIXOS` (if applicable) * - `LANG` - \- - `C.UTF-8` * - `HOME` - running as root (UID 0) - `/root` * - `USER` - running as root (UID 0) - `root` ``` (run-commands-shell)= ## Get shell access to your instance If you want to run commands directly in your instance, run a shell command inside it. ````{tabs} ```{group-tab} CLI Enter the following command (assuming that the `/bin/bash` command exists in your instance): lxc exec -- /bin/bash ``` ```{group-tab} API Enter the following command (assuming that the `/bin/bash` command exists in your instance): lxc query --request POST /1.0/instances//exec --data '{ "command": [ "/bin/bash" ] }' ``` ```{group-tab} UI Navigate to the instance detail page and switch to the {guilabel}`Terminal` tab to access the shell. ``` ```` By default, you are logged in as the `root` user. If you want to log in as a different user, enter the following command: ````{tabs} ```{group-tab} CLI lxc exec -- su --login To exit the instance shell, enter `exit` or press {kbd}`Ctrl`+{kbd}`d`. ``` ```{group-tab} API lxc query --request POST /1.0/instances//exec --data '{ "command": [ "su", "--login", "" ] }' ``` ```{group-tab} UI su --login To exit the user shell and go back to the root shell, enter `exit` or press {kbd}`Ctrl`+{kbd}`d`. ``` ```` ```{note} Depending on the operating system that you run in your instance, you might need to create a user first. ``` --- ### Doc Instances (`doc/instances.md`) --- relatedlinks: "[How to install a Windows 11 VM using LXD](https://ubuntu.com/tutorials/how-to-install-a-windows-11-vm-using-lxd)" myst: html_meta: description: An index of how-to guides for LXD instances, including creating, configuring, and managing instances, backup, migration, and GPU passthrough. --- (instances)= # Instances These how-to guides cover common operations related to LXD instances. ## Create and manage instances LXD supports both system containers and virtual machines, configured using direct settings or reusable profiles. ```{toctree} :titlesonly: Create instances Configure instances Manage instances Use profiles Troubleshoot errors ``` ## Attach instances to Ubuntu Pro A LXD server can automatically attach guest instances to its Ubuntu Pro subscription. ```{toctree} :titlesonly: Auto attach Ubuntu Pro ``` ## Work with instances Instance files can be accessed from the host, and the instance console can be attached to for log output and debugging. Commands can also be run inside instances through the `lxc` CLI or by opening a shell. ```{toctree} :titlesonly: Access files Access the console Run commands Use cloud-init Add a routed NIC to a VM ``` ## Back up, import, and migrate instances Instances can be backed up using snapshots, export files, or copies. Physical machines, as well as virtual machines and containers created using a different technology, can be imported as LXD instances. Instances can also be migrated between LXD servers, including live migration for VMs. ```{toctree} :titlesonly: Back up instances Import existing machines Migrate instances ``` ## Pass through an NVIDIA GPU An NVIDIA GPU can be passed through to a container running a Docker workload. ```{toctree} :titlesonly: Pass NVIDIA GPUs ``` ## Related topics {{instances_exp}} {{instances_ref}} --- ### Doc Internals (`doc/internals.md`) --- myst: html_meta: description: An index of reference information for LXD internals, including daemon behavior, syscall interception, namespaces, OVN implementation, live migration, and Dqlite. --- # Internals These reference guides document the internal workings of LXD and are primarily intended for contributors and developers. ## Runtime behavior The LXD client and daemon can use environment variables for paths, proxies, and advanced features. Daemon startup, shutdown, and signal handling are also documented here. ```{toctree} :titlesonly: /environment /daemon-behavior /reference/uefi_variables ``` ## Security and isolation Specific system calls from containers can be intercepted and handled safely by the LXD daemon. User namespaces use UID/GID idmaps to isolate containers from the host. ```{toctree} :titlesonly: /syscall-interception User namespace setup ``` ## Subsystem internals ```{toctree} :titlesonly: OVN implementation VM live migration implementation Dqlite database for cluster state ZFS storage driver ``` ## Related topics How-to guides: - {ref}`troubleshoot` - {ref}`snap-track-bugfix` --- ### Doc Lxd Test.Yaml (`doc/lxd-test.yaml`) name: lxd-test description: "Profile to build and test LXD in VM" config: cloud-init.user-data: |- #cloud-config apt: # Speed things up by not pulling from backports/security and avoid restricted/multiverse pockets. # In general, backported packages or those from restricted/multiverse shouldn't be relied on because # they don't come with the same level of support as those from main for example. # The security repo is not necessary when packages are coming from the primary archive (maintained by Canonical). disable_suites: - backports - security conf: | # Faster downloads Acquire::Languages "none"; APT::Get::Show-Versions "true"; # Install just what's in the packages list below APT::Install-Recommends "false"; # Bypass apt phasing in new package versions Update-Manager::Always-Include-Phased-Updates true; APT::Get::Always-Include-Phased-Updates true; write_files: # Faster dpkg installs - content: "force-unsafe-io\n" path: /etc/dpkg/dpkg.cfg append: true # wait for cloud-init to be done - content: "#!/bin/sh\nexec cloud-init status --wait --long\n" path: /usr/local/bin/ci-wait permissions: "0755" runcmd: # Speeds up testing and avoid problem with virtiofs - echo "LXD_TMPFS=1" >> /etc/environment # Disable lxd-installer - chmod -x /usr/sbin/lxc /usr/sbin/lxd # Disable needrestart - rm -f /etc/apt/apt.conf.d/99needrestart /etc/dpkg/dpkg.cfg.d/needrestart # Remove sources of noise - systemctl disable --now networkd-dispatcher.service - systemctl disable --now unattended-upgrades.service - cd /etc/systemd/system/timers.target.wants/ && systemctl disable --now *.timer # Prepare LXD build - printf '[safe]\n directory = *\n' >> /root/.gitconfig - chmod o+x /root - echo "root:1000000:1000000000" | tee /etc/subuid /etc/subgid package_update: true package_upgrade: true packages: # dev comfort - bash-completion - nano - vim # build deps - autoconf - automake - build-essential - git - libacl1-dev - libapparmor-dev - libcap-dev - liblz4-dev - libseccomp-dev - libsqlite3-dev - libtool - libudev-dev - libuv1-dev - xfslibs-dev - make - meson - ninja-build - pkg-config - python3-venv - shellcheck # run deps - attr - iproute2 - nftables - rsync - squashfs-tools - squashfs-tools-ng - tar - xz-utils # test deps - acl - bcache-tools - bind9-dnsutils - btrfs-progs - busybox-static - curl - dnsmasq-base - dosfstools - e2fsprogs - easy-rsa - expect - iptables - iputils-ping - jq - netcat-openbsd - s3cmd - shellcheck - socat - sqlite3 - swtpm - xdelta3 - xfsprogs - yq snap: commands: - snap install go --classic limits.cpu: "4" limits.memory: 16GiB security.devlxd.images: "true" devices: eth0: name: eth0 network: lxdbr0 type: nic lxd-git: path: /root/lxd source: @@PATH_TO_LXD_GIT@@ type: disk root: path: / pool: default size: 12GiB type: disk used_by: [] project: default --- ### Doc Metadata (`doc/metadata.txt`) // Code generated by lxd-metadata; DO NOT EDIT. ```{config:option} scheduler.instance cluster-cluster :defaultdesc: "`all`" :shortdesc: "Controls how instances are scheduled to run on this member" :type: "string" Possible values are `all`, `manual`, and `group`. See {ref}`clustering-instance-placement` for more information. ``` ```{config:option} user.* cluster-cluster :shortdesc: "Free form user key/value storage" :type: "string" User keys can be used in search. ``` ```{config:option} user.* cluster-link-conf :shortdesc: "Free form user key/value storage" :type: "string" User keys can be used in search. ``` ```{config:option} volatile.addresses cluster-link-volatile-conf :scope: "global" :shortdesc: "Cluster link member addresses." :type: "string" A comma-separated list of cluster link member addresses. ``` ```{config:option} volatile.pending_address cluster-link-volatile-conf :scope: "global" :shortdesc: "Pending public cluster link address." :type: "string" The address that was contacted and verified for a pending public cluster link, awaiting confirmation. ``` ```{config:option} volatile.pending_certificate cluster-link-volatile-conf :scope: "global" :shortdesc: "Pending public cluster link certificate." :type: "string" The PEM-encoded certificate fetched for a pending public cluster link, awaiting confirmation. ``` ```{config:option} volatile.uuid cluster-link-volatile-conf :scope: "global" :shortdesc: "Linked cluster UUID." :type: "string" The UUID of the linked cluster. ``` ```{config:option} config cluster-link-properties :required: "no" :shortdesc: "Cluster link configuration map" :type: "string set" ``` ```{config:option} description cluster-link-properties :required: "no" :shortdesc: "Description of the cluster link" :type: "string" ``` ```{config:option} name cluster-link-properties :required: "yes" :shortdesc: "Name of the cluster link" :type: "string" ``` ```{config:option} type cluster-link-properties :required: "yes" :shortdesc: "Type of the cluster link" :type: "string" ``` ```{config:option} boot.priority device-disk-device-conf :condition: "virtual machine" :required: "no" :shortdesc: "Boot priority for VMs" :type: "integer" A higher value indicates a higher boot precedence for the disk device. This is useful for prioritizing boot sources like ISO-backed disks. ``` ```{config:option} ceph.cluster_name device-disk-device-conf :defaultdesc: "`ceph`" :required: "for Ceph or CephFS sources" :shortdesc: "Cluster name of the Ceph cluster" :type: "string" ``` ```{config:option} ceph.user_name device-disk-device-conf :defaultdesc: "`admin`" :required: "for Ceph or CephFS sources" :shortdesc: "User name of the Ceph cluster" :type: "string" ``` ```{config:option} initial.* device-disk-device-conf :required: "no" :shortdesc: "Initial volume configuration" :type: "n/a" Initial volume configuration allows setting unique configurations independent of the default storage pool settings. See {ref}`devices-disk-initial-config` for more information. ``` ```{config:option} io.bus device-disk-device-conf :condition: "virtual machine" :defaultdesc: "`virtio-scsi`" :required: "no" :shortdesc: "Bus for the device" :type: "string" Possible values are `virtio-scsi`, `virtio-blk` or `nvme`. ``` ```{config:option} io.cache device-disk-device-conf :condition: "virtual machine" :defaultdesc: "`none`" :required: "no" :shortdesc: "Caching mode for the device" :type: "string" Possible values are `none`, `writeback`, or `unsafe`. ``` ```{config:option} io.threads device-disk-device-conf :condition: "virtual machine" :defaultdesc: "`0`" :required: "no" :shortdesc: "Thread pool for virtiofs file system shares" :type: "integer" This option controls the `virtiofsd` thread pool size, which can help improve I/O performance. Only applies to virtiofs file system shares. In {config:option}`project-restricted:restricted` projects, it can only be used when {config:option}`project-restricted:restricted.virtual-machines.lowlevel` is set to `allow`. ``` ```{config:option} limits.max device-disk-device-conf :required: "no" :shortdesc: "I/O limit in byte/s or IOPS for both read and write" :type: "string" This option is the same as setting both {config:option}`device-disk-device-conf:limits.read` and {config:option}`device-disk-device-conf:limits.write`. You can specify a value in byte/s (various suffixes supported, see {ref}`instances-limit-units`) or in IOPS (must be suffixed with `iops`). See also {ref}`storage-configure-io`. ``` ```{config:option} limits.read device-disk-device-conf :required: "no" :shortdesc: "Read I/O limit in byte/s or IOPS" :type: "string" You can specify a value in byte/s (various suffixes supported, see {ref}`instances-limit-units`) or in IOPS (must be suffixed with `iops`). See also {ref}`storage-configure-io`. ``` ```{config:option} limits.write device-disk-device-conf :required: "no" :shortdesc: "Write I/O limit in byte/s or IOPS" :type: "string" You can specify a value in byte/s (various suffixes supported, see {ref}`instances-limit-units`) or in IOPS (must be suffixed with `iops`). See also {ref}`storage-configure-io`. ``` ```{config:option} path device-disk-device-conf :condition: "container" :required: "yes" :shortdesc: "Mount path" :type: "string" This option specifies the path inside the container where the disk will be mounted. For containers, this option allows mounting filesystem disk devices, as well as specific paths and individual files within those devices. For VMs, this option allows mounting filesystem disk devices and paths within them. Mounting individual files is not supported. ``` ```{config:option} pool device-disk-device-conf :condition: "storage volumes managed by LXD" :required: "no" :shortdesc: "Storage pool to which the disk device belongs" :type: "string" ``` ```{config:option} propagation device-disk-device-conf :defaultdesc: "`private`" :required: "no" :shortdesc: "How a bind-mount is shared between the instance and the host" :type: "string" Possible values are `private` (the default), `shared`, `slave`, `unbindable`, `rshared`, `rslave`, `runbindable`, `rprivate`. See the Linux Kernel [shared subtree](https://www.kernel.org/doc/Documentation/filesystems/sharedsubtree.txt) documentation for a full explanation. ``` ```{config:option} raw.mount.options device-disk-device-conf :required: "no" :shortdesc: "File system specific mount options" :type: "string" ``` ```{config:option} readonly device-disk-device-conf :defaultdesc: "`false`" :required: "no" :shortdesc: "Whether to make the mount read-only" :type: "bool" ``` ```{config:option} recursive device-disk-device-conf :defaultdesc: "`false`" :required: "no" :shortdesc: "Whether to recursively mount the source path" :type: "bool" ``` ```{config:option} required device-disk-device-conf :defaultdesc: "`true`" :required: "no" :shortdesc: "Whether to fail instance start if the source doesn’t exist" :type: "bool" ``` ```{config:option} shift device-disk-device-conf :defaultdesc: "`false`" :required: "no" :shortdesc: "Whether to set up a UID/GID shifting overlay" :type: "bool" For containers, if enabled, this option sets up a shifting overlay to translate the source UID/GID to match the instance. For virtual machines, the source UID/GID is passed through unchanged, even if the instance `raw.idmap` is set. ``` ```{config:option} size device-disk-device-conf :required: "no" :shortdesc: "Disk size" :type: "string" This option is supported only for the rootfs (`/`). Specify a value in bytes (various suffixes supported, see {ref}`instances-limit-units`). ``` ```{config:option} size.state device-disk-device-conf :condition: "virtual machine" :required: "no" :shortdesc: "Size of the file-system volume used for saving runtime state" :type: "string" This option is similar to {config:option}`device-disk-device-conf:size`, but applies to the file-system volume used for saving the runtime state in VMs. ``` ```{config:option} source device-disk-device-conf :required: "yes" :shortdesc: "Source of a file system or block device" :type: "string" See {ref}`devices-disk-types` for details. ``` ```{config:option} source.snapshot device-disk-device-conf :required: "no" :shortdesc: "`source` snapshot name" :type: "string" Snapshot of the volume given by `source`. ``` ```{config:option} source.type device-disk-device-conf :defaultdesc: "`custom`" :required: "no" :shortdesc: "Type of the backing storage volume" :type: "string" Possible values are `custom` (the default) or `virtual-machine`. This key is only valid when `source` is the name of a storage volume. ``` ```{config:option} user.* device-disk-device-conf :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} id device-gpu-mdev-device-conf :shortdesc: "DRM card ID of the GPU device" :type: "string" ``` ```{config:option} mdev device-gpu-mdev-device-conf :defaultdesc: "`0`" :required: "yes" :shortdesc: "The `mdev` profile to use" :type: "string" For example: `i915-GVTg_V5_4` ``` ```{config:option} pci device-gpu-mdev-device-conf :shortdesc: "PCI address of the GPU device" :type: "string" ``` ```{config:option} productid device-gpu-mdev-device-conf :shortdesc: "Product ID of the GPU device" :type: "string" ``` ```{config:option} user.* device-gpu-mdev-device-conf :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} vendorid device-gpu-mdev-device-conf :shortdesc: "Vendor ID of the GPU device" :type: "string" ``` ```{config:option} id device-gpu-mig-device-conf :shortdesc: "DRM card ID of the parent GPU, or CDI identifier of the MIG GPU device" :type: "string" The ID can either be the DRM card ID of the parent GPU device, or a fully-qualified Container Device Interface (CDI) name for the MIG device. Here are some examples of fully-qualified CDI names: - `nvidia.com/mig=0:2`: Instructs LXD to pass through the MIG device at index 2 on GPU 0. You can find the index using `nvidia-smi -L` on your host. - `nvidia.com/mig=MIG-74c6a31a-fde5-5c61-973b-70e12346c202`: Instructs LXD to pass through a MIG device with a given unique identifier. This identifier should also appear with `nvidia-smi -L`. ``` ```{config:option} mig.ci device-gpu-mig-device-conf :shortdesc: "Existing MIG compute instance ID" :type: "integer" Use this together with {config:option}`device-gpu-mig-device-conf:mig.gi` to identify a MIG device by its GPU instance ID and compute instance ID. LXD resolves the pair to a MIG device UUID using NVML and then uses CDI to pass the device through to the container. ``` ```{config:option} mig.gi device-gpu-mig-device-conf :shortdesc: "Existing MIG GPU instance ID" :type: "integer" Use this together with {config:option}`device-gpu-mig-device-conf:mig.ci` to identify a MIG device by its GPU instance ID and compute instance ID. LXD resolves the pair to a MIG device UUID using NVML and then uses CDI to pass the device through to the container. ``` ```{config:option} mig.uuid device-gpu-mig-device-conf :shortdesc: "Existing MIG device UUID" :type: "string" You can omit the `MIG-` prefix when specifying this option. LXD uses CDI to pass the MIG device through to the container. ``` ```{config:option} pci device-gpu-mig-device-conf :shortdesc: "PCI address of the GPU device" :type: "string" ``` ```{config:option} productid device-gpu-mig-device-conf :shortdesc: "Product ID of the GPU device" :type: "string" ``` ```{config:option} user.* device-gpu-mig-device-conf :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} vendorid device-gpu-mig-device-conf :shortdesc: "Vendor ID of the GPU device" :type: "string" ``` ```{config:option} gid device-gpu-physical-device-conf :condition: "container" :defaultdesc: "`0`" :shortdesc: "GID of the device owner in the container" :type: "integer" ``` ```{config:option} id device-gpu-physical-device-conf :shortdesc: "ID of the GPU device" :type: "string" The ID can either be the DRM card ID of the GPU device (container or VM) or a fully-qualified Container Device Interface (CDI) name (container only). Here are some examples of fully-qualified CDI names: - `nvidia.com/gpu=0`: Instructs LXD to operate a discrete GPU (dGPU) pass-through of brand NVIDIA with the first discovered GPU on your system. You can use the `nvidia-smi` tool on your host to find out which identifier to use. - `nvidia.com/gpu=1833c8b5-9aa0-5382-b784-68b7e77eb185`: Instructs LXD to operate a discrete GPU (dGPU) pass-through of brand NVIDIA with a given GPU unique identifier. This identifier should also appear with `nvidia-smi -L`. - `nvidia.com/igpu=all`: Instructs LXD to pass all the host integrated GPUs (iGPU) of brand NVIDIA. The concept of an index does not currently map to iGPUs. It is possible to list them with the `nvidia-smi -L` command. A special `nvgpu` mention should appear in the generated list to indicate a device to be an iGPU. - `nvidia.com/gpu=all`: Instructs LXD to pass all the host GPUs of brand NVIDIA through to the container. ``` ```{config:option} mode device-gpu-physical-device-conf :condition: "container" :defaultdesc: "`0660`" :shortdesc: "Mode of the device in the container" :type: "integer" ``` ```{config:option} pci device-gpu-physical-device-conf :shortdesc: "PCI address of the GPU device" :type: "string" ``` ```{config:option} productid device-gpu-physical-device-conf :shortdesc: "Product ID of the GPU device" :type: "string" ``` ```{config:option} uid device-gpu-physical-device-conf :condition: "container" :defaultdesc: "`0`" :shortdesc: "UID of the device owner in the container" :type: "integer" ``` ```{config:option} user.* device-gpu-physical-device-conf :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} vendorid device-gpu-physical-device-conf :shortdesc: "Vendor ID of the GPU device" :type: "string" ``` ```{config:option} id device-gpu-sriov-device-conf :shortdesc: "DRM card ID of the parent GPU device" :type: "string" ``` ```{config:option} pci device-gpu-sriov-device-conf :shortdesc: "PCI address of the parent GPU device" :type: "string" ``` ```{config:option} productid device-gpu-sriov-device-conf :shortdesc: "Product ID of the parent GPU device" :type: "string" ``` ```{config:option} user.* device-gpu-sriov-device-conf :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} vendorid device-gpu-sriov-device-conf :shortdesc: "Vendor ID of the parent GPU device" :type: "string" ``` ```{config:option} hwaddr device-infiniband-device-conf :defaultdesc: "randomly assigned" :required: "no" :shortdesc: "MAC address of the new interface" :type: "string" You can specify either the full 20-byte variant or the short 8-byte variant (which will modify only the last 8 bytes of the parent device). ``` ```{config:option} mtu device-infiniband-device-conf :defaultdesc: "parent MTU" :required: "no" :shortdesc: "MTU of the new interface" :type: "integer" ``` ```{config:option} name device-infiniband-device-conf :defaultdesc: "kernel assigned" :required: "no" :shortdesc: "Name of the interface inside the instance" :type: "string" ``` ```{config:option} nictype device-infiniband-device-conf :required: "yes" :shortdesc: "Device type" :type: "string" Possible values are `physical` and `sriov`. ``` ```{config:option} parent device-infiniband-device-conf :required: "yes" :shortdesc: "The name of the host device or bridge" :type: "string" ``` ```{config:option} user.* device-infiniband-device-conf :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} boot.priority device-nic-bridged-device-conf :managed: "no" :shortdesc: "Boot priority for VMs" :type: "integer" A higher value for this option means that the VM boots first. ``` ```{config:option} host_name device-nic-bridged-device-conf :defaultdesc: "randomly assigned" :managed: "no" :shortdesc: "Name of the interface inside the host" :type: "string" ``` ```{config:option} hwaddr device-nic-bridged-device-conf :defaultdesc: "randomly assigned" :managed: "no" :shortdesc: "MAC address of the new interface" :type: "string" ``` ```{config:option} ipv4.address device-nic-bridged-device-conf :managed: "no" :shortdesc: "IPv4 address to assign to the instance through DHCP" :type: "string" Set this option to `none` to restrict all IPv4 traffic when {config:option}`device-nic-bridged-device-conf:security.ipv4_filtering` is set. ``` ```{config:option} ipv4.routes device-nic-bridged-device-conf :managed: "no" :shortdesc: "IPv4 static routes for the NIC to add on the host" :type: "string" Specify a comma-delimited list of IPv4 static routes for this NIC to add on the host. ``` ```{config:option} ipv4.routes.external device-nic-bridged-device-conf :managed: "no" :shortdesc: "IPv4 static routes to route to NIC" :type: "string" Specify a comma-delimited list of IPv4 static routes to route to the NIC and publish on the uplink network (BGP). ``` ```{config:option} ipv6.address device-nic-bridged-device-conf :managed: "no" :shortdesc: "IPv6 address to assign to the instance through DHCP" :type: "string" Set this option to `none` to restrict all IPv6 traffic when {config:option}`device-nic-bridged-device-conf:security.ipv6_filtering` is set. ``` ```{config:option} ipv6.routes device-nic-bridged-device-conf :managed: "no" :shortdesc: "IPv6 static routes for the NIC to add on the host" :type: "string" Specify a comma-delimited list of IPv6 static routes for this NIC to add on the host. ``` ```{config:option} ipv6.routes.external device-nic-bridged-device-conf :managed: "no" :shortdesc: "IPv6 static routes to route to NIC" :type: "string" Specify a comma-delimited list of IPv6 static routes to route to the NIC and publish on the uplink network (BGP). ``` ```{config:option} limits.egress device-nic-bridged-device-conf :managed: "no" :shortdesc: "I/O limit for outgoing traffic" :type: "string" Specify the limit in bit/s. Various suffixes are supported (see {ref}`instances-limit-units`). ``` ```{config:option} limits.ingress device-nic-bridged-device-conf :managed: "no" :shortdesc: "I/O limit for incoming traffic" :type: "string" Specify the limit in bit/s. Various suffixes are supported (see {ref}`instances-limit-units`). ``` ```{config:option} limits.max device-nic-bridged-device-conf :managed: "no" :shortdesc: "I/O limit for both incoming and outgoing traffic" :type: "string" This option is the same as setting both {config:option}`device-nic-bridged-device-conf:limits.ingress` and {config:option}`device-nic-bridged-device-conf:limits.egress`. Specify the limit in bit/s. Various suffixes are supported (see {ref}`instances-limit-units`). ``` ```{config:option} limits.priority device-nic-bridged-device-conf :managed: "no" :shortdesc: "`skb->priority` value for outgoing traffic" :type: "integer" The `skb->priority` value for outgoing traffic is used by the kernel queuing discipline (qdisc) to prioritize network packets. Specify the value as a 32-bit unsigned integer. The effect of this value depends on the particular qdisc implementation, for example, `SKBPRIO` or `QFQ`. Consult the kernel qdisc documentation before setting this value. ``` ```{config:option} mtu device-nic-bridged-device-conf :defaultdesc: "parent MTU" :managed: "yes" :shortdesc: "MTU of the new interface" :type: "integer" ``` ```{config:option} name device-nic-bridged-device-conf :defaultdesc: "kernel assigned" :managed: "no" :shortdesc: "Name of the interface inside the instance" :type: "string" ``` ```{config:option} network device-nic-bridged-device-conf :managed: "no" :shortdesc: "Managed network to link the device to" :type: "string" You can specify this option instead of specifying the `nictype` directly. ``` ```{config:option} parent device-nic-bridged-device-conf :managed: "yes" :required: "if specifying the `nictype` directly" :shortdesc: "Name of the host device" :type: "string" ``` ```{config:option} queue.tx.length device-nic-bridged-device-conf :managed: "no" :shortdesc: "Transmit queue length for the NIC" :type: "integer" ``` ```{config:option} security.ipv4_filtering device-nic-bridged-device-conf :defaultdesc: "`false`" :managed: "no" :shortdesc: "Whether to prevent the instance from spoofing an IPv4 address" :type: "bool" Set this option to `true` to prevent the instance from spoofing another instance’s IPv4 address. This option enables {config:option}`device-nic-bridged-device-conf:security.mac_filtering`. ``` ```{config:option} security.ipv6_filtering device-nic-bridged-device-conf :defaultdesc: "`false`" :managed: "no" :shortdesc: "Whether to prevent the instance from spoofing an IPv6 address" :type: "bool" Set this option to `true` to prevent the instance from spoofing another instance’s IPv6 address. This option enables {config:option}`device-nic-bridged-device-conf:security.mac_filtering`. ``` ```{config:option} security.mac_filtering device-nic-bridged-device-conf :defaultdesc: "`false`" :managed: "no" :shortdesc: "Whether to prevent the instance from spoofing a MAC address" :type: "bool" Set this option to `true` to prevent the instance from spoofing another instance’s MAC address. ``` ```{config:option} security.port_isolation device-nic-bridged-device-conf :defaultdesc: "`false`" :managed: "no" :shortdesc: "Whether to respect port isolation" :type: "bool" Set this option to `true` to prevent the NIC from communicating with other NICs in the network that have port isolation enabled. ``` ```{config:option} user.* device-nic-bridged-device-conf :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} vlan device-nic-bridged-device-conf :managed: "no" :shortdesc: "VLAN ID to use for non-tagged traffic" :type: "integer" Set this option to `none` to remove the port from the default VLAN. ``` ```{config:option} vlan.tagged device-nic-bridged-device-conf :managed: "no" :shortdesc: "VLAN IDs or VLAN ranges to join for tagged traffic" :type: "integer" Specify the VLAN IDs or ranges as a comma-delimited list. ``` ```{config:option} gvrp device-nic-ipvlan-device-conf :defaultdesc: "`false`" :shortdesc: "Whether to use GARP VLAN Registration Protocol" :type: "bool" This option specifies whether to register the VLAN using the GARP VLAN Registration Protocol. ``` ```{config:option} hwaddr device-nic-ipvlan-device-conf :defaultdesc: "randomly assigned" :shortdesc: "MAC address of the new interface" :type: "string" ``` ```{config:option} ipv4.address device-nic-ipvlan-device-conf :shortdesc: "IPv4 static addresses to add to the instance" :type: "string" Specify a comma-delimited list of IPv4 static addresses to add to the instance. In `l2` mode, you can specify them as CIDR values or singular addresses using a subnet of `/24`. ``` ```{config:option} ipv4.gateway device-nic-ipvlan-device-conf :defaultdesc: "`auto` (`l3s`), `-` (`l2`)" :shortdesc: "IPv4 gateway" :type: "string" In `l3s` mode, the option specifies whether to add an automatic default IPv4 gateway. Possible values are `auto` and `none`. In `l2` mode, this option specifies the IPv4 address of the gateway. ``` ```{config:option} ipv4.host_table device-nic-ipvlan-device-conf :shortdesc: "Custom policy routing table ID to add IPv4 static routes to" :type: "integer" The custom policy routing table is in addition to the main routing table. ``` ```{config:option} ipv6.address device-nic-ipvlan-device-conf :shortdesc: "IPv6 static addresses to add to the instance" :type: "string" Specify a comma-delimited list of IPv6 static addresses to add to the instance. In `l2` mode, you can specify them as CIDR values or singular addresses using a subnet of `/64`. ``` ```{config:option} ipv6.gateway device-nic-ipvlan-device-conf :defaultdesc: "`auto` (`l3s`), `-` (`l2`)" :shortdesc: "IPv6 gateway" :type: "string" In `l3s` mode, the option specifies whether to add an automatic default IPv6 gateway. Possible values are `auto` and `none`. In `l2` mode, this option specifies the IPv6 address of the gateway. ``` ```{config:option} ipv6.host_table device-nic-ipvlan-device-conf :shortdesc: "Custom policy routing table ID to add IPv6 static routes to" :type: "integer" The custom policy routing table is in addition to the main routing table. ``` ```{config:option} mode device-nic-ipvlan-device-conf :defaultdesc: "`l3s`" :shortdesc: "IPVLAN mode" :type: "string" Possible values are `l2` and `l3s`. ``` ```{config:option} mtu device-nic-ipvlan-device-conf :defaultdesc: "parent MTU" :shortdesc: "The MTU of the new interface" :type: "integer" ``` ```{config:option} name device-nic-ipvlan-device-conf :defaultdesc: "kernel assigned" :shortdesc: "Name of the interface inside the instance" :type: "string" ``` ```{config:option} parent device-nic-ipvlan-device-conf :required: "yes" :shortdesc: "Name of the host device" :type: "string" ``` ```{config:option} user.* device-nic-ipvlan-device-conf :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} vlan device-nic-ipvlan-device-conf :shortdesc: "VLAN ID to attach to" :type: "integer" ``` ```{config:option} boot.priority device-nic-macvlan-device-conf :managed: "no" :shortdesc: "Boot priority for VMs" :type: "integer" A higher value for this option means that the VM boots first. ``` ```{config:option} gvrp device-nic-macvlan-device-conf :defaultdesc: "`false`" :managed: "no" :shortdesc: "Whether to use GARP VLAN Registration Protocol" :type: "bool" This option specifies whether to register the VLAN using the GARP VLAN Registration Protocol. ``` ```{config:option} hwaddr device-nic-macvlan-device-conf :defaultdesc: "randomly assigned" :managed: "no" :shortdesc: "MAC address of the new interface" :type: "string" ``` ```{config:option} mtu device-nic-macvlan-device-conf :defaultdesc: "parent MTU" :managed: "yes" :shortdesc: "MTU of the new interface" :type: "integer" ``` ```{config:option} name device-nic-macvlan-device-conf :defaultdesc: "kernel assigned" :managed: "no" :shortdesc: "Name of the interface inside the instance" :type: "string" ``` ```{config:option} network device-nic-macvlan-device-conf :managed: "no" :shortdesc: "Managed network to link the device to" :type: "string" You can specify this option instead of specifying the `nictype` directly. ``` ```{config:option} parent device-nic-macvlan-device-conf :managed: "yes" :required: "if specifying the `nictype` directly" :shortdesc: "Name of the host device" :type: "string" ``` ```{config:option} user.* device-nic-macvlan-device-conf :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} vlan device-nic-macvlan-device-conf :managed: "no" :shortdesc: "VLAN ID to attach to" :type: "integer" ``` ```{config:option} acceleration device-nic-ovn-device-conf :defaultdesc: "`none`" :managed: "no" :shortdesc: "Enable hardware acceleration" :type: "string" Possible values are `none`, `sriov`, or `vdpa`. See {ref}`devices-nic-hw-acceleration` for more information. ``` ```{config:option} acceleration.parent device-nic-ovn-device-conf :managed: "yes" :shortdesc: "Physical function interfaces to allocate virtual functions from for hardware acceleration" :type: "string" Comma separated list of physical function (PF) interfaces to allocate virtual functions (VFs) from for hardware acceleration when {config:option}`device-nic-ovn-device-conf:acceleration` is enabled. In {config:option}`project-restricted:restricted` projects, it can only be used when {config:option}`project-restricted:restricted.virtual-machines.lowlevel` or {config:option}`project-restricted:restricted.containers.lowlevel` is set to `allow`. If this is not specified, and {config:option}`device-nic-ovn-device-conf:acceleration` is enabled then all PFs connected to the OVS integration bridge are scanned for a free VF. See {ref}`devices-nic-hw-acceleration` for more information. ``` ```{config:option} boot.priority device-nic-ovn-device-conf :managed: "no" :shortdesc: "Boot priority for VMs" :type: "integer" A higher value for this option means that the VM boots first. ``` ```{config:option} host_name device-nic-ovn-device-conf :defaultdesc: "randomly assigned" :managed: "no" :shortdesc: "Name of the interface inside the host" :type: "string" ``` ```{config:option} hwaddr device-nic-ovn-device-conf :defaultdesc: "randomly assigned" :managed: "no" :shortdesc: "MAC address of the new interface" :type: "string" ``` ```{config:option} ipv4.address device-nic-ovn-device-conf :managed: "no" :shortdesc: "IPv4 address to assign to the instance through DHCP" :type: "string" ``` ```{config:option} ipv4.routes device-nic-ovn-device-conf :managed: "no" :shortdesc: "IPv4 static routes to route for the NIC" :type: "string" Specify a comma-delimited list of IPv4 static routes to route for this NIC. ``` ```{config:option} ipv4.routes.external device-nic-ovn-device-conf :managed: "no" :shortdesc: "IPv4 static routes to route to NIC" :type: "string" Specify a comma-delimited list of IPv4 static routes to route to the NIC and publish on the uplink network. ``` ```{config:option} ipv6.address device-nic-ovn-device-conf :managed: "no" :shortdesc: "IPv6 address to assign to the instance through DHCP" :type: "string" ``` ```{config:option} ipv6.routes device-nic-ovn-device-conf :managed: "no" :shortdesc: "IPv6 static routes to route to the NIC" :type: "string" Specify a comma-delimited list of IPv6 static routes to route to the NIC. ``` ```{config:option} ipv6.routes.external device-nic-ovn-device-conf :managed: "no" :shortdesc: "IPv6 static routes to route to NIC" :type: "string" Specify a comma-delimited list of IPv6 static routes to route to the NIC and publish on the uplink network. ``` ```{config:option} name device-nic-ovn-device-conf :defaultdesc: "kernel assigned" :managed: "no" :shortdesc: "Name of the interface inside the instance" :type: "string" ``` ```{config:option} nested device-nic-ovn-device-conf :managed: "no" :shortdesc: "Parent NIC name to nest this NIC under" :type: "string" See also {config:option}`device-nic-ovn-device-conf:vlan`. ``` ```{config:option} network device-nic-ovn-device-conf :managed: "yes" :required: "yes" :shortdesc: "Managed network to link the device to" :type: "string" ``` ```{config:option} security.acls device-nic-ovn-device-conf :managed: "no" :shortdesc: "Network ACLs to apply" :type: "string" Specify a comma-separated list ``` ```{config:option} security.acls.default.egress.action device-nic-ovn-device-conf :defaultdesc: "`reject`" :managed: "no" :shortdesc: "Default action to use for egress traffic" :type: "string" The specified action is used for all egress traffic that doesn’t match any ACL rule. ``` ```{config:option} security.acls.default.egress.logged device-nic-ovn-device-conf :defaultdesc: "`false`" :managed: "no" :shortdesc: "Whether to log egress traffic that doesn’t match any ACL rule" :type: "bool" ``` ```{config:option} security.acls.default.ingress.action device-nic-ovn-device-conf :defaultdesc: "`reject`" :managed: "no" :shortdesc: "Default action to use for ingress traffic" :type: "string" The specified action is used for all ingress traffic that doesn’t match any ACL rule. ``` ```{config:option} security.acls.default.ingress.logged device-nic-ovn-device-conf :defaultdesc: "`false`" :managed: "no" :shortdesc: "Whether to log ingress traffic that doesn’t match any ACL rule" :type: "bool" ``` ```{config:option} user.* device-nic-ovn-device-conf :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} vlan device-nic-ovn-device-conf :managed: "no" :shortdesc: "VLAN ID to use when nesting" :type: "integer" See also {config:option}`device-nic-ovn-device-conf:nested`. ``` ```{config:option} boot.priority device-nic-p2p-device-conf :shortdesc: "Boot priority for VMs" :type: "integer" A higher value for this option means that the VM boots first. ``` ```{config:option} host_name device-nic-p2p-device-conf :defaultdesc: "randomly assigned" :shortdesc: "Name of the interface inside the host" :type: "string" ``` ```{config:option} hwaddr device-nic-p2p-device-conf :defaultdesc: "randomly assigned" :shortdesc: "MAC address of the new interface" :type: "string" ``` ```{config:option} ipv4.routes device-nic-p2p-device-conf :shortdesc: "IPv4 static routes for the NIC to add on the host" :type: "string" Specify a comma-delimited list of IPv4 static routes for this NIC to add on the host. ``` ```{config:option} ipv6.routes device-nic-p2p-device-conf :shortdesc: "IPv6 static routes for the NIC to add on the host" :type: "string" Specify a comma-delimited list of IPv6 static routes for this NIC to add on the host. ``` ```{config:option} limits.egress device-nic-p2p-device-conf :shortdesc: "I/O limit for outgoing traffic" :type: "string" Specify the limit in bit/s. Various suffixes are supported (see {ref}`instances-limit-units`). ``` ```{config:option} limits.ingress device-nic-p2p-device-conf :shortdesc: "I/O limit for incoming traffic" :type: "string" Specify the limit in bit/s. Various suffixes are supported (see {ref}`instances-limit-units`). ``` ```{config:option} limits.max device-nic-p2p-device-conf :shortdesc: "I/O limit for both incoming and outgoing traffic" :type: "string" This option is the same as setting both {config:option}`device-nic-bridged-device-conf:limits.ingress` and {config:option}`device-nic-bridged-device-conf:limits.egress`. Specify the limit in bit/s. Various suffixes are supported (see {ref}`instances-limit-units`). ``` ```{config:option} limits.priority device-nic-p2p-device-conf :shortdesc: "`skb->priority` value for outgoing traffic" :type: "integer" The `skb->priority` value for outgoing traffic is used by the kernel queuing discipline (qdisc) to prioritize network packets. Specify the value as a 32-bit unsigned integer. The effect of this value depends on the particular qdisc implementation, for example, `SKBPRIO` or `QFQ`. Consult the kernel qdisc documentation before setting this value. ``` ```{config:option} mtu device-nic-p2p-device-conf :defaultdesc: "kernel assigned" :shortdesc: "MTU of the new interface" :type: "integer" ``` ```{config:option} name device-nic-p2p-device-conf :defaultdesc: "kernel assigned" :shortdesc: "Name of the interface inside the instance" :type: "string" ``` ```{config:option} queue.tx.length device-nic-p2p-device-conf :shortdesc: "Transmit queue length for the NIC" :type: "integer" ``` ```{config:option} user.* device-nic-p2p-device-conf :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} boot.priority device-nic-physical-device-conf :managed: "no" :shortdesc: "Boot priority for VMs" :type: "integer" A higher value for this option means that the VM boots first. ``` ```{config:option} gvrp device-nic-physical-device-conf :defaultdesc: "`false`" :managed: "no" :shortdesc: "Whether to use GARP VLAN Registration Protocol" :type: "bool" This option specifies whether to register the VLAN using the GARP VLAN Registration Protocol. ``` ```{config:option} hwaddr device-nic-physical-device-conf :condition: "container" :defaultdesc: "parent MAC address" :managed: "no" :shortdesc: "MAC address of the new interface" :type: "string" ``` ```{config:option} mtu device-nic-physical-device-conf :condition: "container" :defaultdesc: "parent MTU" :managed: "no" :shortdesc: "MTU of the new interface" :type: "integer" ``` ```{config:option} name device-nic-physical-device-conf :defaultdesc: "kernel assigned" :managed: "no" :shortdesc: "Name of the interface inside the instance" :type: "string" ``` ```{config:option} network device-nic-physical-device-conf :managed: "no" :shortdesc: "Managed network to link the device to" :type: "string" You can specify this option instead of specifying the `nictype` directly. ``` ```{config:option} parent device-nic-physical-device-conf :managed: "yes" :required: "if specifying the `nictype` directly" :shortdesc: "Name of the host device" :type: "string" ``` ```{config:option} user.* device-nic-physical-device-conf :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} vlan device-nic-physical-device-conf :condition: "container" :managed: "no" :shortdesc: "VLAN ID to attach to" :type: "integer" ``` ```{config:option} gvrp device-nic-routed-device-conf :defaultdesc: "`false`" :shortdesc: "Whether to use GARP VLAN Registration Protocol" :type: "bool" This option specifies whether to register the VLAN using the GARP VLAN Registration Protocol. ``` ```{config:option} host_name device-nic-routed-device-conf :defaultdesc: "randomly assigned" :shortdesc: "Name of the interface inside the host" :type: "string" ``` ```{config:option} hwaddr device-nic-routed-device-conf :defaultdesc: "randomly assigned" :shortdesc: "MAC address of the new interface" :type: "string" ``` ```{config:option} ipv4.address device-nic-routed-device-conf :shortdesc: "IPv4 static addresses to add to the instance" :type: "string" Specify a comma-delimited list of IPv4 static addresses to add to the instance. ``` ```{config:option} ipv4.gateway device-nic-routed-device-conf :defaultdesc: "`auto`" :shortdesc: "Whether to add an automatic default IPv4 gateway" :type: "string" Possible values are `auto` and `none`. ``` ```{config:option} ipv4.host_address device-nic-routed-device-conf :defaultdesc: "`169.254.0.1`" :shortdesc: "IPv4 address to add to the host-side `veth` interface" :type: "string" ``` ```{config:option} ipv4.host_table device-nic-routed-device-conf :shortdesc: "Custom policy routing table ID to add IPv4 static routes to" :type: "integer" The custom policy routing table is in addition to the main routing table. ``` ```{config:option} ipv4.neighbor_probe device-nic-routed-device-conf :defaultdesc: "`true`" :shortdesc: "Whether to probe the parent network for IPv4 address availability" :type: "bool" ``` ```{config:option} ipv4.routes device-nic-routed-device-conf :shortdesc: "IPv4 static routes for the NIC to add on the host" :type: "string" Specify a comma-delimited list of IPv4 static routes for this NIC to add on the host (without L2 ARP/NDP proxy). ``` ```{config:option} ipv6.address device-nic-routed-device-conf :shortdesc: "IPv6 static addresses to add to the instance" :type: "string" Specify a comma-delimited list of IPv6 static addresses to add to the instance. ``` ```{config:option} ipv6.gateway device-nic-routed-device-conf :defaultdesc: "`auto`" :shortdesc: "Whether to add an automatic default IPv6 gateway" :type: "string" Possible values are `auto` and `none`. ``` ```{config:option} ipv6.host_address device-nic-routed-device-conf :defaultdesc: "`fe80::1`" :shortdesc: "IPv6 address to add to the host-side `veth` interface" :type: "string" ``` ```{config:option} ipv6.host_table device-nic-routed-device-conf :shortdesc: "Custom policy routing table ID to add IPv6 static routes to" :type: "integer" The custom policy routing table is in addition to the main routing table. ``` ```{config:option} ipv6.neighbor_probe device-nic-routed-device-conf :defaultdesc: "`true`" :shortdesc: "Whether to probe the parent network for IPv6 address availability" :type: "bool" ``` ```{config:option} ipv6.routes device-nic-routed-device-conf :shortdesc: "IPv6 static routes for the NIC to add on the host" :type: "string" Specify a comma-delimited list of IPv6 static routes for this NIC to add on the host (without L2 ARP/NDP proxy). ``` ```{config:option} limits.egress device-nic-routed-device-conf :shortdesc: "I/O limit for outgoing traffic" :type: "string" Specify the limit in bit/s. Various suffixes are supported (see {ref}`instances-limit-units`). ``` ```{config:option} limits.ingress device-nic-routed-device-conf :shortdesc: "I/O limit for incoming traffic" :type: "string" Specify the limit in bit/s. Various suffixes are supported (see {ref}`instances-limit-units`). ``` ```{config:option} limits.max device-nic-routed-device-conf :shortdesc: "I/O limit for both incoming and outgoing traffic" :type: "string" This option is the same as setting both {config:option}`device-nic-bridged-device-conf:limits.ingress` and {config:option}`device-nic-bridged-device-conf:limits.egress`. Specify the limit in bit/s. Various suffixes are supported (see {ref}`instances-limit-units`). ``` ```{config:option} limits.priority device-nic-routed-device-conf :shortdesc: "`skb->priority` value for outgoing traffic" :type: "integer" The `skb->priority` value for outgoing traffic is used by the kernel queuing discipline (qdisc) to prioritize network packets. Specify the value as a 32-bit unsigned integer. The effect of this value depends on the particular qdisc implementation, for example, `SKBPRIO` or `QFQ`. Consult the kernel qdisc documentation before setting this value. ``` ```{config:option} mtu device-nic-routed-device-conf :defaultdesc: "parent MTU" :shortdesc: "The MTU of the new interface" :type: "integer" ``` ```{config:option} name device-nic-routed-device-conf :defaultdesc: "kernel assigned" :shortdesc: "Name of the interface inside the instance" :type: "string" ``` ```{config:option} parent device-nic-routed-device-conf :shortdesc: "Name of the host device to join the instance to" :type: "string" ``` ```{config:option} queue.tx.length device-nic-routed-device-conf :shortdesc: "Transmit queue length for the NIC" :type: "integer" ``` ```{config:option} user.* device-nic-routed-device-conf :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} vlan device-nic-routed-device-conf :shortdesc: "VLAN ID to attach to" :type: "integer" ``` ```{config:option} boot.priority device-nic-sriov-device-conf :managed: "no" :shortdesc: "Boot priority for VMs" :type: "integer" A higher value for this option means that the VM boots first. ``` ```{config:option} hwaddr device-nic-sriov-device-conf :defaultdesc: "randomly assigned" :managed: "no" :shortdesc: "MAC address of the new interface" :type: "string" ``` ```{config:option} mtu device-nic-sriov-device-conf :defaultdesc: "kernel assigned" :managed: "yes" :shortdesc: "MTU of the new interface" :type: "integer" ``` ```{config:option} name device-nic-sriov-device-conf :defaultdesc: "kernel assigned" :managed: "no" :shortdesc: "Name of the interface inside the instance" :type: "string" ``` ```{config:option} network device-nic-sriov-device-conf :managed: "no" :shortdesc: "Managed network to link the device to" :type: "string" You can specify this option instead of specifying the `nictype` directly. ``` ```{config:option} parent device-nic-sriov-device-conf :managed: "yes" :required: "if specifying the `nictype` directly" :shortdesc: "Name of the host device" :type: "string" ``` ```{config:option} security.mac_filtering device-nic-sriov-device-conf :defaultdesc: "`false`" :managed: "no" :shortdesc: "Whether to prevent the instance from spoofing a MAC address" :type: "bool" Set this option to `true` to prevent the instance from spoofing another instance’s MAC address. ``` ```{config:option} user.* device-nic-sriov-device-conf :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} vlan device-nic-sriov-device-conf :managed: "no" :shortdesc: "VLAN ID to attach to" :type: "integer" ``` ```{config:option} address device-pci-device-conf :required: "yes" :shortdesc: "PCI address of the device" :type: "string" ``` ```{config:option} user.* device-pci-device-conf :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} bind device-proxy-device-conf :defaultdesc: "`host`" :required: "no" :shortdesc: "Which side to bind on" :type: "string" Possible values are `host` and `instance`. ``` ```{config:option} connect device-proxy-device-conf :required: "yes" :shortdesc: "Address and port to connect to" :type: "string" Use the following format to specify the address and port: `::[-][,]` ``` ```{config:option} gid device-proxy-device-conf :defaultdesc: "`0`" :required: "no" :shortdesc: "GID of the owner of the listening Unix socket" :type: "integer" ``` ```{config:option} listen device-proxy-device-conf :required: "yes" :shortdesc: "Address and port to bind and listen" :type: "string" Use the following format to specify the address and port: `::[-][,]` ``` ```{config:option} mode device-proxy-device-conf :defaultdesc: "`0644`" :required: "no" :shortdesc: "Mode for the listening Unix socket" :type: "integer" ``` ```{config:option} nat device-proxy-device-conf :defaultdesc: "`false`" :required: "no" :shortdesc: "Whether to optimize proxying via NAT" :type: "bool" This option requires that the instance NIC has a static IP address. ``` ```{config:option} proxy_protocol device-proxy-device-conf :defaultdesc: "`false`" :required: "no" :shortdesc: "Whether to use the HAProxy PROXY protocol" :type: "bool" This option specifies whether to use the HAProxy PROXY protocol to transmit sender information. ``` ```{config:option} security.gid device-proxy-device-conf :defaultdesc: "`0`" :required: "no" :shortdesc: "What GID to drop privilege to" :type: "integer" ``` ```{config:option} security.uid device-proxy-device-conf :defaultdesc: "`0`" :required: "no" :shortdesc: "What UID to drop privilege to" :type: "integer" ``` ```{config:option} uid device-proxy-device-conf :defaultdesc: "`0`" :required: "no" :shortdesc: "UID of the owner of the listening Unix socket" :type: "integer" ``` ```{config:option} user.* device-proxy-device-conf :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} path device-tpm-device-conf :required: "for containers" :shortdesc: "Path inside the container" :type: "string" For example: `/dev/tpm0` ``` ```{config:option} pathrm device-tpm-device-conf :required: "for containers" :shortdesc: "Resource manager path inside the container" :type: "string" For example: `/dev/tpmrm0` ``` ```{config:option} user.* device-tpm-device-conf :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} gid device-unix-block-device-conf :defaultdesc: "`0`" :shortdesc: "GID of the device owner in the container" :type: "integer" ``` ```{config:option} major device-unix-block-device-conf :defaultdesc: "device on host" :shortdesc: "Device major number" :type: "integer" ``` ```{config:option} minor device-unix-block-device-conf :defaultdesc: "device on host" :shortdesc: "Device minor number" :type: "integer" ``` ```{config:option} mode device-unix-block-device-conf :defaultdesc: "`0660`" :shortdesc: "Mode of the device in the container" :type: "integer" ``` ```{config:option} path device-unix-block-device-conf :required: "either `source` or `path` must be set" :shortdesc: "Path inside the container" :type: "string" ``` ```{config:option} required device-unix-block-device-conf :defaultdesc: "`true`" :shortdesc: "Whether this device is required to start the container" :type: "bool" See {ref}`devices-unix-block-hotplugging` for more information. ``` ```{config:option} source device-unix-block-device-conf :required: "either `source` or `path` must be set" :shortdesc: "Path on the host" :type: "string" ``` ```{config:option} uid device-unix-block-device-conf :defaultdesc: "`0`" :shortdesc: "UID of the device owner in the container" :type: "integer" ``` ```{config:option} user.* device-unix-block-device-conf :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} gid device-unix-char-device-conf :defaultdesc: "`0`" :shortdesc: "GID of the device owner in the container" :type: "integer" ``` ```{config:option} major device-unix-char-device-conf :defaultdesc: "device on host" :shortdesc: "Device major number" :type: "integer" ``` ```{config:option} minor device-unix-char-device-conf :defaultdesc: "device on host" :shortdesc: "Device minor number" :type: "integer" ``` ```{config:option} mode device-unix-char-device-conf :defaultdesc: "`0660`" :shortdesc: "Mode of the device in the container" :type: "integer" ``` ```{config:option} path device-unix-char-device-conf :required: "either `source` or `path` must be set" :shortdesc: "Path inside the container" :type: "string" ``` ```{config:option} required device-unix-char-device-conf :defaultdesc: "`true`" :shortdesc: "Whether this device is required to start the container" :type: "bool" See {ref}`devices-unix-char-hotplugging` for more information. ``` ```{config:option} source device-unix-char-device-conf :required: "either `source` or `path` must be set" :shortdesc: "Path on the host" :type: "string" ``` ```{config:option} uid device-unix-char-device-conf :defaultdesc: "`0`" :shortdesc: "UID of the device owner in the container" :type: "integer" ``` ```{config:option} user.* device-unix-char-device-conf :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} gid device-unix-hotplug-device-conf :defaultdesc: "`0`" :shortdesc: "GID of the device owner in the container" :type: "integer" ``` ```{config:option} mode device-unix-hotplug-device-conf :defaultdesc: "`0660`" :shortdesc: "Mode of the device in the container" :type: "integer" ``` ```{config:option} ownership.inherit device-unix-hotplug-device-conf :defaultdesc: "`false`" :shortdesc: "Whether this device inherits ownership (GID and/or UID) from the host" :type: "bool" ``` ```{config:option} productid device-unix-hotplug-device-conf :shortdesc: "Product ID of the Unix device" :type: "string" ``` ```{config:option} required device-unix-hotplug-device-conf :defaultdesc: "`false`" :shortdesc: "Whether this device is required to start the container" :type: "bool" The default is `false`, which means that all devices can be hotplugged. ``` ```{config:option} subsystem device-unix-hotplug-device-conf :shortdesc: "Subsystem of the Unix device" :type: "string" ``` ```{config:option} uid device-unix-hotplug-device-conf :defaultdesc: "`0`" :shortdesc: "UID of the device owner in the container" :type: "integer" ``` ```{config:option} user.* device-unix-hotplug-device-conf :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} vendorid device-unix-hotplug-device-conf :shortdesc: "Vendor ID of the Unix device" :type: "string" ``` ```{config:option} busnum device-unix-usb-device-conf :shortdesc: "The bus number of which the USB device is attached" :type: "int" ``` ```{config:option} devnum device-unix-usb-device-conf :shortdesc: "The device number of the USB device" :type: "int" ``` ```{config:option} gid device-unix-usb-device-conf :condition: "container" :defaultdesc: "`0`" :shortdesc: "GID of the device owner in the instance" :type: "integer" ``` ```{config:option} mode device-unix-usb-device-conf :condition: "container" :defaultdesc: "`0660`" :shortdesc: "Mode of the device in the instance" :type: "integer" ``` ```{config:option} productid device-unix-usb-device-conf :shortdesc: "Product ID of the USB device" :type: "string" ``` ```{config:option} required device-unix-usb-device-conf :defaultdesc: "`false`" :shortdesc: "Whether this device is required to start the instance" :type: "bool" The default is `false`, which means that all devices can be hotplugged. ``` ```{config:option} serial device-unix-usb-device-conf :shortdesc: "The serial number of the USB device" :type: "string" ``` ```{config:option} uid device-unix-usb-device-conf :condition: "container" :defaultdesc: "`0`" :shortdesc: "UID of the device owner in the instance" :type: "integer" ``` ```{config:option} user.* device-unix-usb-device-conf :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} vendorid device-unix-usb-device-conf :shortdesc: "Vendor ID of the USB device" :type: "string" ``` ```{config:option} cluster image-registry-image-registry-conf :required: "no" :shortdesc: Cluster link name for image registry using \"LXD\" protocol :type: "string" ``` ```{config:option} source_project image-registry-image-registry-conf :required: "no" :shortdesc: Source project for image registry using \"LXD\" protocol :type: "string" ``` ```{config:option} url image-registry-image-registry-conf :required: "no" :shortdesc: Source URL for image registry using \"SimpleStreams\" protocol :type: "string" ``` ```{config:option} user.* image-registry-image-registry-conf :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} boot.autostart instance-boot :liveupdate: "no" :shortdesc: "Whether to always start the instance when LXD starts" :type: "bool" If set to `true`, the instance will always be auto-started, unless `security.protection.start` is also enabled. If set to `false`, the instance will not be started on LXD start up. If this option is not set, the instance will be restored to its last known state. ``` ```{config:option} boot.autostart.delay instance-boot :defaultdesc: "`0`" :liveupdate: "no" :shortdesc: "Delay after starting the instance" :type: "integer" The number of seconds to wait after the instance started before starting the next one. ``` ```{config:option} boot.autostart.priority instance-boot :defaultdesc: "`0`" :liveupdate: "no" :shortdesc: "What order to start the instances in" :type: "integer" The instance with the highest value is started first. ``` ```{config:option} boot.debug_edk2 instance-boot :shortdesc: "Enable debug version of the `edk2`" :type: "bool" The instance should use a debug version of the `edk2`. A log file can be found in `$LXD_DIR/logs//edk2.log`. ``` ```{config:option} boot.host_shutdown_timeout instance-boot :defaultdesc: "`30`" :liveupdate: "yes" :shortdesc: "How long to wait for the instance to shut down" :type: "integer" Number of seconds to wait for the instance to shut down before it is force-stopped. ``` ```{config:option} boot.mode instance-boot :condition: "virtual machine" :defaultdesc: "`uefi-secureboot`" :liveupdate: "no" :shortdesc: "Boot firmware mode for the VM (uefi-secureboot, uefi-nosecureboot or bios)" :type: "string" The `uefi-secureboot` mode uses UEFI firmware with secure boot enabled. The `uefi-nosecureboot` mode uses UEFI firmware with secure boot disabled. The `bios` mode is supported only on `x86_64` (`amd64`). ``` ```{config:option} boot.stop.priority instance-boot :defaultdesc: "`0`" :liveupdate: "no" :shortdesc: "What order to shut down the instances in" :type: "integer" The instance with the highest value is shut down first. ``` ```{config:option} cloud-init.network-config instance-cloud-init :condition: "If supported by image" :defaultdesc: "`DHCP on eth0`" :liveupdate: "no" :shortdesc: "Network configuration for `cloud-init`" :type: "string" The content is used as seed value for `cloud-init`. ``` ```{config:option} cloud-init.ssh-keys.KEYNAME instance-cloud-init :condition: "If supported by image" :liveupdate: "no" :shortdesc: "Additional SSH key to be injected on the instance by `cloud-init`" :type: "string" Represents an additional SSH public key to be merged into existing `cloud-init` seed data and injected into an instance. Has the format `{user}:{key}`, where {user} is a Linux username and {key} can be either a pure SSH public key or an import ID for a key hosted elsewhere. // For example: `root:gh:githubUser`, `myUser:ssh-keyAlg publicKeyHash` ``` ```{config:option} cloud-init.user-data instance-cloud-init :condition: "If supported by image" :defaultdesc: "`#cloud-config`" :liveupdate: "no" :shortdesc: "User data for `cloud-init`" :type: "string" The content is used as seed value for `cloud-init`. ``` ```{config:option} cloud-init.vendor-data instance-cloud-init :condition: "If supported by image" :defaultdesc: "`#cloud-config`" :liveupdate: "no" :shortdesc: "Vendor data for `cloud-init`" :type: "string" The content is used as seed value for `cloud-init`. ``` ```{config:option} user.network-config instance-cloud-init :condition: "If supported by image" :defaultdesc: "`DHCP on eth0`" :liveupdate: "no" :shortdesc: "Legacy version of `cloud-init.network-config`" :type: "string" ``` ```{config:option} user.user-data instance-cloud-init :condition: "If supported by image" :defaultdesc: "`#cloud-config`" :liveupdate: "no" :shortdesc: "Legacy version of `cloud-init.user-data`" :type: "string" ``` ```{config:option} user.vendor-data instance-cloud-init :condition: "If supported by image" :defaultdesc: "`#cloud-config`" :liveupdate: "no" :shortdesc: "Legacy version of `cloud-init.vendor-data`" :type: "string" ``` ```{config:option} migration.incremental.memory instance-migration :condition: "container" :defaultdesc: "`false`" :liveupdate: "yes" :shortdesc: "Whether to use incremental memory transfer" :type: "bool" Using incremental memory transfer of the instance's memory can reduce downtime. ``` ```{config:option} migration.incremental.memory.goal instance-migration :condition: "container" :defaultdesc: "`70`" :liveupdate: "yes" :shortdesc: "Percentage of memory to have in sync before stopping the instance" :type: "integer" ``` ```{config:option} migration.incremental.memory.iterations instance-migration :condition: "container" :defaultdesc: "`10`" :liveupdate: "yes" :shortdesc: "Maximum number of transfer operations to go through before stopping the instance" :type: "integer" ``` ```{config:option} migration.stateful instance-migration :condition: "virtual machine" :defaultdesc: "`false` or value from profiles or `instances.migration.stateful` (if set)" :liveupdate: "no" :shortdesc: "Whether to allow for stateful stop/start and snapshots" :type: "bool" Enabling this option prevents the use of some features that are incompatible with it. ``` ```{config:option} agent.nic_config instance-miscellaneous :condition: "virtual machine" :defaultdesc: "`false`" :liveupdate: "no" :shortdesc: "Whether to use the name and MTU of the default network interfaces" :type: "bool" When set to true, the name and MTU of the default network interfaces inside the virtual machine will match those of the instance devices. ``` ```{config:option} cluster.evacuate instance-miscellaneous :defaultdesc: "`auto`" :liveupdate: "no" :shortdesc: "What to do when evacuating the instance" :type: "string" The `cluster.evacuate` provides control over how instances are handled when a cluster member is being evacuated. Available Modes: - `auto` *(default)*: The system will automatically decide the best evacuation method based on the instance's type and configured devices: + If any device is not suitable for migration, the instance will not be migrated (only stopped). + Live migration will be used only for virtual machines with the `migration.stateful` setting enabled and for which all its devices can be migrated as well. - `live-migrate`: Eligible instances are live-migrated to another node. This means the instance remains running and operational during the migration process, ensuring minimal disruption. Note: Live migration is supported for virtual machines only. If no target member is available, an instance is skipped. If a live migration attempt fails, the evacuation operation fails. - `migrate`: In this mode, instances are migrated to another node in the cluster. The migration process will not be live, meaning there will be a brief downtime for the instance during the migration. - `stop`: Instances are not migrated. Instead, they are stopped on the current node. See {ref}`cluster-evacuate` for more information. ``` ```{config:option} environment.* instance-miscellaneous :liveupdate: "yes" :shortdesc: "Free-form environment key/value" :type: "string" Extra environment variables to set on boot (for containers) and during exec. ``` ```{config:option} image.* instance-miscellaneous :liveupdate: "yes" :shortdesc: "Property of the base image" :type: "string" A property of the image that the instance was created from (not set if the instance was not created from an image). ``` ```{config:option} linux.kernel_modules instance-miscellaneous :condition: "container" :liveupdate: "yes" :shortdesc: "Kernel modules to load or allow loading" :type: "string" Specify the kernel modules as a comma-separated list. The modules are loaded before the instance starts, or they can be loaded by a privileged user if {config:option}`instance-miscellaneous:linux.kernel_modules.load` is set to `ondemand`. ``` ```{config:option} linux.kernel_modules.load instance-miscellaneous :condition: "container" :defaultdesc: "`boot`" :liveupdate: "no" :shortdesc: "How to load kernel modules" :type: "string" This option specifies how to load the kernel modules that are specified in {config:option}`instance-miscellaneous:linux.kernel_modules`. Possible values are `boot` (load the modules when booting the container) and `ondemand` (intercept the `finit_modules()` syscall and allow a privileged user in the container's user namespace to load the modules). ``` ```{config:option} linux.sysctl.* instance-miscellaneous :condition: "container" :liveupdate: "no" :shortdesc: "Override for the corresponding `sysctl` setting in the container" :type: "string" ``` ```{config:option} ubuntu_pro.guest_attach instance-miscellaneous :liveupdate: "no" :shortdesc: "Whether to auto-attach Ubuntu Pro." :type: "string" Indicate whether the guest should auto-attach Ubuntu Pro at start up. See {ref}`instances-ubuntu-pro-attach` for more information. ``` ```{config:option} user.* instance-miscellaneous :liveupdate: "no" :shortdesc: "Free-form user key/value storage" :type: "string" User keys can be used in search. ``` ```{config:option} placement.group instance-placement :liveupdate: "yes" :shortdesc: "Placement group controlling instance scheduling" :type: "string" Specifies the placement group that determines where this instance is scheduled within the cluster. The placement group defines the placement policy (e.g. spread or compact) and rigor (e.g. strict or permissive) used to determine eligible cluster members during LXD scheduling events. ``` ```{config:option} raw.apparmor instance-raw :liveupdate: "yes" :shortdesc: "AppArmor profile entries" :type: "blob" The specified entries are appended to the generated profile. This is a low-level option and is not recommended for production use, as it allows for unsupported configurations that may cease to work in future versions. ``` ```{config:option} raw.idmap instance-raw :liveupdate: "no" :shortdesc: "Raw idmap configuration" :type: "blob" For example: `both 1000 1000` ``` ```{config:option} raw.lxc instance-raw :condition: "container" :liveupdate: "no" :shortdesc: "Raw LXC configuration to be appended to the generated one" :type: "blob" Additional LXC configuration is appended to the generated configuration. This is a low-level option and is not recommended for production use, as it allows for unsupported configurations that may cease to work in future versions. ``` ```{config:option} raw.qemu instance-raw :condition: "virtual machine" :liveupdate: "no" :shortdesc: "Raw QEMU configuration to be appended to the generated command line" :type: "blob" Additional QEMU command line configuration is appended to the generated command line. This is a low-level option and is not recommended for production use, as it allows for unsupported configurations that may cease to work in future versions. ``` ```{config:option} raw.qemu.conf instance-raw :condition: "virtual machine" :liveupdate: "no" :shortdesc: "Addition/override to the generated `qemu.conf` file" :type: "blob" This is a low-level option and is not recommended for production use, as it allows for unsupported configurations that may cease to work in future versions. See {ref}`instance-options-qemu` for more information. ``` ```{config:option} raw.seccomp instance-raw :condition: "container" :liveupdate: "no" :shortdesc: "Raw Seccomp configuration" :type: "blob" Additional Seccomp configuration is appended to the generated policy. This is a low-level option and is not recommended for production use, as it allows for unsupported configurations that may cease to work in future versions. ``` ```{config:option} limits.cpu instance-resource-limits :defaultdesc: "1 (VMs)" :liveupdate: "yes" :shortdesc: "Which CPUs to expose to the instance" :type: "string" A number or a specific range of CPUs to expose to the instance. See {ref}`instance-options-limits-cpu` for more information. ``` ```{config:option} limits.cpu.allowance instance-resource-limits :condition: "container" :defaultdesc: "100%" :liveupdate: "yes" :shortdesc: "How much of the CPU can be used" :type: "string" To control how much of the CPU can be used, specify either a percentage (`50%`) for a soft limit or a chunk of time (`25ms/100ms`) for a hard limit. See {ref}`instance-options-limits-cpu-container` for more information. ``` ```{config:option} limits.cpu.nodes instance-resource-limits :liveupdate: "yes" :shortdesc: "Which NUMA nodes to place the instance CPUs on" :type: "string" A comma-separated list of NUMA node IDs or ranges to place the instance CPUs on. See {ref}`instance-options-limits-cpu-container` for more information. ``` ```{config:option} limits.cpu.pin_strategy instance-resource-limits :condition: "virtual machine" :defaultdesc: "`none`" :liveupdate: "no" :shortdesc: "VM CPU auto pinning strategy" :type: "string" Specify the strategy for VM CPU auto pinning. Possible values: `none` (disables CPU auto pinning) and `auto` (enables CPU auto pinning). See {ref}`instance-options-limits-cpu-vm` for more information. ``` ```{config:option} limits.cpu.priority instance-resource-limits :condition: "container" :defaultdesc: "`10` (maximum)" :liveupdate: "yes" :shortdesc: "CPU scheduling priority compared to other instances" :type: "integer" When overcommitting resources, specify the CPU scheduling priority compared to other instances that share the same CPUs. Specify an integer between 0 and 10. See {ref}`instance-options-limits-cpu-container` for more information. ``` ```{config:option} limits.disk.priority instance-resource-limits :defaultdesc: "`5` (medium)" :liveupdate: "yes" :shortdesc: "Priority of the instance's I/O requests" :type: "integer" Controls how much priority to give to the instance's I/O requests when under load. Specify an integer between 0 and 10. ``` ```{config:option} limits.hugepages.1GB instance-resource-limits :condition: "container" :liveupdate: "yes" :shortdesc: "Limit for the number of 1 GB huge pages" :type: "string" Fixed value (in bytes) to limit the number of 1 GB huge pages. Various suffixes are supported (see {ref}`instances-limit-units`). See {ref}`instance-options-limits-hugepages` for more information. ``` ```{config:option} limits.hugepages.1MB instance-resource-limits :condition: "container" :liveupdate: "yes" :shortdesc: "Limit for the number of 1 MB huge pages" :type: "string" Fixed value (in bytes) to limit the number of 1 MB huge pages. Various suffixes are supported (see {ref}`instances-limit-units`). See {ref}`instance-options-limits-hugepages` for more information. ``` ```{config:option} limits.hugepages.2MB instance-resource-limits :condition: "container" :liveupdate: "yes" :shortdesc: "Limit for the number of 2 MB huge pages" :type: "string" Fixed value (in bytes) to limit the number of 2 MB huge pages. Various suffixes are supported (see {ref}`instances-limit-units`). See {ref}`instance-options-limits-hugepages` for more information. ``` ```{config:option} limits.hugepages.64KB instance-resource-limits :condition: "container" :liveupdate: "yes" :shortdesc: "Limit for the number of 64 KB huge pages" :type: "string" Fixed value (in bytes) to limit the number of 64 KB huge pages. Various suffixes are supported (see {ref}`instances-limit-units`). See {ref}`instance-options-limits-hugepages` for more information. ``` ```{config:option} limits.max_bus_ports instance-resource-limits :condition: "virtual machine" :defaultdesc: "`8`" :liveupdate: "no" :shortdesc: "Limit of allowed PCI/PCIe devices" :type: "integer" Total number of user configurable PCI/PCIe devices that can be attached to the VM. ``` ```{config:option} limits.memory instance-resource-limits :defaultdesc: "`1GiB` (VMs)" :liveupdate: "yes" :shortdesc: "Usage limit for the host's memory" :type: "string" Percentage of the host's memory or a fixed value in bytes. Various suffixes are supported. See {ref}`instances-limit-units` for details. ``` ```{config:option} limits.memory.enforce instance-resource-limits :condition: "container" :defaultdesc: "`hard`" :liveupdate: "yes" :shortdesc: "Whether the memory limit is `hard` or `soft`" :type: "string" If the instance's memory limit is `hard`, the instance cannot exceed its limit. If it is `soft`, the instance can exceed its memory limit when extra host memory is available. ``` ```{config:option} limits.memory.hugepages instance-resource-limits :condition: "virtual machine" :defaultdesc: "`false`" :liveupdate: "no" :shortdesc: "Whether to back the instance using huge pages" :type: "bool" If this option is set to `false`, regular system memory is used. ``` ```{config:option} limits.memory.swap instance-resource-limits :condition: "container" :defaultdesc: "`true`" :liveupdate: "yes" :shortdesc: "Whether to encourage/discourage swapping less used pages for this instance" :type: "bool" ``` ```{config:option} limits.memory.swap.priority instance-resource-limits :condition: "container" :defaultdesc: "`10` (maximum)" :liveupdate: "yes" :shortdesc: "Prevents the instance from being swapped to disk" :type: "integer" Specify an integer between 0 and 10. The higher the value, the less likely the instance is to be swapped to disk. ``` ```{config:option} limits.processes instance-resource-limits :condition: "container" :defaultdesc: "empty" :liveupdate: "yes" :shortdesc: "Maximum number of processes that can run in the instance" :type: "integer" If left empty, no limit is set. ``` ```{config:option} security.agent.metrics instance-security :condition: "virtual machine" :defaultdesc: "`true`" :liveupdate: "no" :shortdesc: "Whether the `lxd-agent` is queried for state information and metrics" :type: "bool" ``` ```{config:option} security.delegate_bpf instance-security :condition: "unprivileged container" :defaultdesc: "`false`" :liveupdate: "no" :shortdesc: "Whether to enable eBPF delegation using BPF Token mechanism" :type: "bool" This option enables BPF functionality delegation mechanism (using BPF Token). Note: `security.delegate_bpf.cmd_types`, `security.delegate_bpf.map_types`, `security.delegate_bpf.prog_types`, `security.delegate_bpf.attach_types` need to be configured depending on BPF workload in the container. See {ref}`bpf-delegation-token` for more information. ``` ```{config:option} security.delegate_bpf.attach_types instance-security :condition: "unprivileged container" :defaultdesc: "`false`" :liveupdate: "no" :shortdesc: "Which eBPF attach types to allow with delegation mechanism" :type: "bool" Which eBPF program attachment types to allow with delegation mechanism. Syntax follows a kernel one for `delegate_attachs` bpffs mount option. A number (bitmask) or `:`-separated list of attachment types to allow can be specified. For example, `cgroup_inet_ingress` allows `BPF_CGROUP_INET_INGRESS` attachment type. ``` ```{config:option} security.delegate_bpf.cmd_types instance-security :condition: "unprivileged container" :defaultdesc: "`false`" :liveupdate: "no" :shortdesc: "Which eBPF commands to allow with delegation mechanism" :type: "bool" Which eBPF commands to allow with delegation mechanism. Syntax follows a kernel one for `delegate_cmds` bpffs mount option. A number (bitmask) or `:`-separated list of commands to allow can be specified. For example, `prog_load:map_create` allows eBPF programs loading and eBPF maps creation. Notice: `security.delegate_bpf.prog_types` and `security.delegate_bpf.map_types` still need to be configured accordingly. ``` ```{config:option} security.delegate_bpf.map_types instance-security :condition: "unprivileged container" :defaultdesc: "`false`" :liveupdate: "no" :shortdesc: "Which eBPF maps to allow with delegation mechanism" :type: "bool" Which eBPF maps to allow with delegation mechanism. Syntax follows a kernel one for `delegate_maps` bpffs mount option. A number (bitmask) or `:`-separated list of map types to allow can be specified. For example, `ringbuf` allows `BPF_MAP_TYPE_RINGBUF` map. ``` ```{config:option} security.delegate_bpf.prog_types instance-security :condition: "unprivileged container" :defaultdesc: "`false`" :liveupdate: "no" :shortdesc: "Which eBPF program types to allow with delegation mechanism" :type: "bool" Which eBPF program types to allow with delegation mechanism. Syntax follows a kernel one for `delegate_progs` bpffs mount option. A number (bitmask) or `:`-separated list of program types to allow can be specified. For example, `socket_filter` allows `BPF_PROG_TYPE_SOCKET_FILTER` program type. ``` ```{config:option} security.devlxd instance-security :defaultdesc: "`true`" :liveupdate: "no" :shortdesc: "Whether `/dev/lxd` is present in the instance" :type: "bool" See {ref}`dev-lxd` for more information. ``` ```{config:option} security.devlxd.images instance-security :defaultdesc: "`false`" :liveupdate: "yes" :shortdesc: "Controls the availability of the `/1.0/images` API over `devlxd`" :type: "bool" ``` ```{config:option} security.devlxd.management.volumes instance-security :defaultdesc: "`false`" :liveupdate: "yes" :shortdesc: "Controls the availability of the volume management API over `devlxd`" :type: "bool" ``` ```{config:option} security.idmap.base instance-security :condition: "unprivileged container" :liveupdate: "no" :shortdesc: "The base host ID to use for the allocation" :type: "integer" Setting this option overrides auto-detection. ``` ```{config:option} security.idmap.isolated instance-security :condition: "unprivileged container" :defaultdesc: "`false`" :liveupdate: "no" :shortdesc: "Whether to use a unique idmap for this instance" :type: "bool" If specified, the idmap used for this instance is unique among instances that have this option set. ``` ```{config:option} security.idmap.size instance-security :condition: "unprivileged container" :liveupdate: "no" :shortdesc: "The size of the idmap to use" :type: "integer" ``` ```{config:option} security.nesting instance-security :condition: "container" :defaultdesc: "`false`" :liveupdate: "yes" :shortdesc: "Whether to support running LXD (nested) inside the instance" :type: "bool" ``` ```{config:option} security.privileged instance-security :condition: "container" :defaultdesc: "`false`" :liveupdate: "no" :shortdesc: "Whether to run the instance in privileged mode" :type: "bool" See {ref}`container-security` for more information. ``` ```{config:option} security.protection.delete instance-security :defaultdesc: "`false`" :liveupdate: "container" :shortdesc: "Whether to prevent the instance from being deleted" :type: "bool" ``` ```{config:option} security.protection.shift instance-security :condition: "container" :defaultdesc: "`false`" :liveupdate: "yes" :shortdesc: "Whether to protect the file system from being UID/GID shifted" :type: "bool" Set this option to `true` to prevent the instance's file system from being UID/GID shifted on startup. ``` ```{config:option} security.protection.start instance-security :defaultdesc: "`false`" :liveupdate: "container" :shortdesc: "Whether to prevent the instance from being started" :type: "bool" ``` ```{config:option} security.sev instance-security :condition: "virtual machine" :defaultdesc: "`false`" :liveupdate: "no" :shortdesc: "Whether AMD SEV (Secure Encrypted Virtualization) is enabled for this VM" :type: "bool" ``` ```{config:option} security.sev.policy.es instance-security :condition: "virtual machine" :defaultdesc: "`false`" :liveupdate: "no" :shortdesc: "Whether AMD SEV-ES (SEV Encrypted State) is enabled for this VM" :type: "bool" ``` ```{config:option} security.sev.session.data instance-security :condition: "virtual machine" :defaultdesc: "`true`" :liveupdate: "no" :shortdesc: "The guest owner's `base64`-encoded session blob" :type: "string" ``` ```{config:option} security.sev.session.dh instance-security :condition: "virtual machine" :defaultdesc: "`true`" :liveupdate: "no" :shortdesc: "The guest owner's `base64`-encoded Diffie-Hellman key" :type: "string" ``` ```{config:option} security.syscalls.allow instance-security :condition: "container" :liveupdate: "no" :shortdesc: "List of syscalls to allow" :type: "string" A `\n`-separated list of syscalls to allow. This list must be mutually exclusive with `security.syscalls.deny*`. ``` ```{config:option} security.syscalls.deny instance-security :condition: "container" :liveupdate: "no" :shortdesc: "List of syscalls to deny" :type: "string" A `\n`-separated list of syscalls to deny. This list must be mutually exclusive with `security.syscalls.allow`. ``` ```{config:option} security.syscalls.deny_compat instance-security :condition: "container" :defaultdesc: "`false`" :liveupdate: "no" :shortdesc: "Whether to block `compat_*` syscalls (`x86_64` only)" :type: "bool" On `x86_64`, this option controls whether to block `compat_*` syscalls. On other architectures, the option is ignored. ``` ```{config:option} security.syscalls.deny_default instance-security :condition: "container" :defaultdesc: "`true`" :liveupdate: "no" :shortdesc: "Whether to enable the default syscall deny" :type: "bool" ``` ```{config:option} security.syscalls.intercept.bpf instance-security :condition: "container" :defaultdesc: "`false`" :liveupdate: "no" :shortdesc: "Whether to handle the `bpf()` system call" :type: "bool" ``` ```{config:option} security.syscalls.intercept.bpf.devices instance-security :condition: "container" :defaultdesc: "`false`" :liveupdate: "no" :shortdesc: "Whether to allow BPF programs" :type: "bool" This option controls whether to allow BPF programs for the devices cgroup in the unified hierarchy to be loaded. ``` ```{config:option} security.syscalls.intercept.mknod instance-security :condition: "container" :defaultdesc: "`false`" :liveupdate: "no" :shortdesc: "Whether to handle the `mknod` and `mknodat` system calls" :type: "bool" These system calls allow creation of a limited subset of char/block devices. ``` ```{config:option} security.syscalls.intercept.mount instance-security :condition: "container" :defaultdesc: "`false`" :liveupdate: "no" :shortdesc: "Whether to handle the `mount` system call" :type: "bool" ``` ```{config:option} security.syscalls.intercept.mount.allowed instance-security :condition: "container" :liveupdate: "yes" :shortdesc: "File systems that can be mounted" :type: "string" Specify a comma-separated list of file systems that are safe to mount for processes inside the instance. ``` ```{config:option} security.syscalls.intercept.mount.fuse instance-security :condition: "container" :liveupdate: "yes" :shortdesc: "File system that should be redirected to FUSE implementation" :type: "string" Specify the mounts of a given file system that should be redirected to their FUSE implementation (for example, `ext4=fuse2fs`). ``` ```{config:option} security.syscalls.intercept.mount.shift instance-security :condition: "container" :defaultdesc: "`false`" :liveupdate: "yes" :shortdesc: "Whether to use idmapped mounts for syscall interception" :type: "bool" ``` ```{config:option} security.syscalls.intercept.sched_setscheduler instance-security :condition: "container" :defaultdesc: "`false`" :liveupdate: "no" :shortdesc: "Whether to handle the `sched_setscheduler` system call" :type: "bool" This system call allows increasing process priority. ``` ```{config:option} security.syscalls.intercept.setxattr instance-security :condition: "container" :defaultdesc: "`false`" :liveupdate: "no" :shortdesc: "Whether to handle the `setxattr` system call" :type: "bool" This system call allows setting a limited subset of restricted extended attributes. ``` ```{config:option} security.syscalls.intercept.sysinfo instance-security :condition: "container" :defaultdesc: "`false`" :liveupdate: "no" :shortdesc: "Whether to handle the `sysinfo` system call" :type: "bool" This system call can be used to get cgroup-based resource usage information. ``` ```{config:option} snapshots.expiry instance-snapshots :liveupdate: "no" :shortdesc: "Time until snapshots are deleted" :type: "string" Specify an expression like `1M 2H 3d 4w 5m 6y`. ``` ```{config:option} snapshots.pattern instance-snapshots :defaultdesc: "`snap%d`" :liveupdate: "no" :shortdesc: "Template for the snapshot name" :type: "string" Specify a Pongo2 template string that represents the snapshot name. This template is used for scheduled snapshots and for unnamed snapshots. See {ref}`instance-options-snapshots-names` for more information. ``` ```{config:option} snapshots.schedule instance-snapshots :defaultdesc: "empty" :liveupdate: "no" :shortdesc: "Schedule for automatic instance snapshots" :type: "string" Specify either a cron expression (` `), a comma-separated list of schedule aliases (`@hourly`, `@daily`, `@midnight`, `@weekly`, `@monthly`, `@annually`, `@yearly`), or leave empty to disable automatic snapshots. ``` ```{config:option} snapshots.schedule.stopped instance-snapshots :defaultdesc: "`false`" :liveupdate: "no" :shortdesc: "Whether to automatically snapshot stopped instances" :type: "bool" ``` ```{config:option} volatile..apply_quota instance-volatile :shortdesc: "Disk quota" :type: "string" The disk quota is applied the next time the instance starts. ``` ```{config:option} volatile..bus instance-volatile :shortdesc: "Persistent VM bus number" :type: "integer" Persistent VM bus number. ``` ```{config:option} volatile..ceph_rbd instance-volatile :shortdesc: "RBD device path for Ceph disk devices" :type: "string" RBD device path for Ceph disk devices. ``` ```{config:option} volatile..devlxd.owner instance-volatile :shortdesc: "DevLXD identity ID that owns the device." :type: "string" ID of the DevLXD identity that owns the device. It is used by DevLXD to restrict access of an identity to devices that were created by that identity. ``` ```{config:option} volatile..host_name instance-volatile :shortdesc: "Network device name on the host" :type: "string" Network device name on the host. ``` ```{config:option} volatile..hwaddr instance-volatile :shortdesc: "Network device MAC address" :type: "string" The network device MAC address is used when no `hwaddr` property is set on the device itself. ``` ```{config:option} volatile..last_state.created instance-volatile :shortdesc: "Whether the network device physical device was created" :type: "bool" Possible values are `true` or `false`. ``` ```{config:option} volatile..last_state.hwaddr instance-volatile :shortdesc: "Network device original MAC" :type: "string" The original MAC that was used when moving a physical device into an instance. ``` ```{config:option} volatile..last_state.mtu instance-volatile :shortdesc: "Network device original MTU" :type: "string" The original MTU that was used when moving a physical device into an instance. ``` ```{config:option} volatile..last_state.pci.driver instance-volatile :shortdesc: "Name of driver device was using on host" :type: "string" Name of driver device was using on host. ``` ```{config:option} volatile..last_state.usb.bus instance-volatile :shortdesc: "USB Bus Number" :type: "string" USB Bus Number. ``` ```{config:option} volatile..last_state.usb.device instance-volatile :shortdesc: "USB Device Number" :type: "string" USB Device Number. ``` ```{config:option} volatile..last_state.vdpa.name instance-volatile :shortdesc: "VDPA device name" :type: "string" The VDPA device name used when moving a VDPA device file descriptor into an instance. ``` ```{config:option} volatile..last_state.vf.hwaddr instance-volatile :shortdesc: "SR-IOV virtual function original MAC" :type: "string" The original MAC used when moving a VF into an instance. ``` ```{config:option} volatile..last_state.vf.id instance-volatile :shortdesc: "SR-IOV virtual function ID" :type: "string" The ID used when moving a VF into an instance. ``` ```{config:option} volatile..last_state.vf.spoofcheck instance-volatile :shortdesc: "SR-IOV virtual function original spoof check setting" :type: "string" The original spoof check setting used when moving a VF into an instance. ``` ```{config:option} volatile..last_state.vf.vlan instance-volatile :shortdesc: "SR-IOV virtual function original VLAN" :type: "string" The original VLAN used when moving a VF into an instance. ``` ```{config:option} volatile.apply_nvram instance-volatile :shortdesc: "Whether to regenerate VM NVRAM the next time the instance starts" :type: "bool" ``` ```{config:option} volatile.apply_template instance-volatile :shortdesc: "Template hook" :type: "string" The template with the given name is triggered upon next startup. ``` ```{config:option} volatile.attached_volumes instance-volatile :condition: "snapshot" :shortdesc: "JSON-serialized map of attached volume device names to the UUIDs of their corresponding snapshots." :type: "string" JSON-serialized map of attached volume device names to the UUIDs of their corresponding snapshots, created as part of a multi-volume snapshot. ``` ```{config:option} volatile.base_image instance-volatile :shortdesc: "Hash of the base image" :type: "string" The hash of the image that the instance was created from (empty if the instance was not created from an image). ``` ```{config:option} volatile.bus.mode instance-volatile :shortdesc: "Device bus allocation mode" :type: "string" Set to `persistent` when persistent bus allocation mode is enabled. ``` ```{config:option} volatile.cloud-init.instance-id instance-volatile :shortdesc: "`instance-id` (UUID) exposed to `cloud-init`" :type: "string" ``` ```{config:option} volatile.cluster.group instance-volatile :shortdesc: "The target cluster group" :type: "string" The target cluster group at instance creation or migration time. This is used during scheduling events such as evacuation to ensure the instance is placed correctly. ``` ```{config:option} volatile.cpu.maxcpus instance-volatile :shortdesc: "vCPU hotplug limit used as of last boot" :type: "integer" The vCPU hotplug limit (SMP `maxcpus`) recorded when the VM booted. It is reused on stateful start (stateful resume or live migration) so that the QEMU SMP topology matches on both ends of a live migration regardless of the CPU count on each host. ``` ```{config:option} volatile.evacuate.origin instance-volatile :shortdesc: "The origin of the evacuated instance" :type: "string" The cluster member that the instance lived on before evacuation. ``` ```{config:option} volatile.idmap.base instance-volatile :condition: "container" :shortdesc: "The first ID in the container's primary idmap range" :type: "integer" ``` ```{config:option} volatile.idmap.current instance-volatile :condition: "container" :shortdesc: "The idmap currently in use by the container" :type: "string" ``` ```{config:option} volatile.idmap.next instance-volatile :condition: "container" :shortdesc: "The idmap to use the next time the container starts" :type: "string" ``` ```{config:option} volatile.last_state.idmap instance-volatile :condition: "container" :shortdesc: "On-disk UID/GID map for the container's rootfs" :type: "string" The UID/GID map that has been applied to the container's underlying storage. This is usually set for containers created on older kernels that don't support idmapped mounts. ``` ```{config:option} volatile.last_state.power instance-volatile :shortdesc: "Instance state as of last host shutdown" :type: "string" ``` ```{config:option} volatile.uuid instance-volatile :shortdesc: "Instance UUID" :type: "string" The instance UUID is globally unique across all servers and projects. ``` ```{config:option} volatile.uuid.generation instance-volatile :shortdesc: "Instance generation UUID" :type: "string" The instance generation UUID changes whenever the instance's place in time moves backwards. It is globally unique across all servers and projects. ``` ```{config:option} volatile.vsock_id instance-volatile :shortdesc: "Instance `vsock ID` used as of last start" :type: "string" ``` ```{config:option} architecture instance-property-instance-conf :readonly: "no" :shortdesc: "Instance architecture" :type: "string" ``` ```{config:option} name instance-property-instance-conf :readonly: "yes" :shortdesc: "Instance name" :type: "string" See {ref}`instance-name-requirements`. ``` ```{config:option} config network-acl-acl-properties :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string set" The only supported keys are `user.*` custom keys. ``` ```{config:option} description network-acl-acl-properties :required: "no" :shortdesc: "Description of the network ACL" :type: "string" ``` ```{config:option} egress network-acl-acl-properties :required: "no" :shortdesc: "Egress traffic rules" :type: "rule list" ``` ```{config:option} ingress network-acl-acl-properties :required: "no" :shortdesc: "Ingress traffic rules" :type: "rule list" ``` ```{config:option} name network-acl-acl-properties :required: "yes" :shortdesc: "Unique name of the network ACL in the project" :type: "string" ``` ```{config:option} action network-acl-rule-properties :required: "yes" :shortdesc: "Action to take for matching traffic" :type: "string" Possible values are `allow`, `reject`, and `drop`. ``` ```{config:option} description network-acl-rule-properties :required: "no" :shortdesc: "Description of the rule" :type: "string" ``` ```{config:option} destination network-acl-rule-properties :required: "no" :shortdesc: "Comma-separated list of destinations" :type: "string" Destinations can be specified as CIDR or IP ranges, destination subject name selectors (for egress rules), or be left empty for any. ``` ```{config:option} destination_port network-acl-rule-properties :required: "no" :shortdesc: "Destination ports or port ranges" :type: "string" This option is valid only if the protocol is `udp` or `tcp`. Specify a comma-separated list of ports or port ranges (start-end inclusive), or leave the value empty for any. ``` ```{config:option} icmp_code network-acl-rule-properties :required: "no" :shortdesc: "ICMP message code" :type: "string" This option is valid only if the protocol is `icmp4` or `icmp6`. Specify the ICMP code number, or leave the value empty for any. ``` ```{config:option} icmp_type network-acl-rule-properties :required: "no" :shortdesc: "Type of ICMP message" :type: "string" This option is valid only if the protocol is `icmp4` or `icmp6`. Specify the ICMP type number, or leave the value empty for any. ``` ```{config:option} protocol network-acl-rule-properties :required: "no" :shortdesc: "Protocol to match" :type: "string" Possible values are `icmp4`, `icmp6`, `tcp`, and `udp`. Leave the value empty to match any protocol. ``` ```{config:option} source network-acl-rule-properties :required: "no" :shortdesc: "Comma-separated list of sources" :type: "string" Sources can be specified as CIDR or IP ranges, source subject name selectors (for ingress rules), or be left empty for any. ``` ```{config:option} source_port network-acl-rule-properties :required: "no" :shortdesc: "Source ports or port ranges" :type: "string" This option is valid only if the protocol is `udp` or `tcp`. Specify a comma-separated list of ports or port ranges (start-end inclusive), or leave the value empty for any. ``` ```{config:option} state network-acl-rule-properties :defaultdesc: "`enabled`" :required: "yes" :shortdesc: "State of the rule" :type: "string" Possible values are `enabled`, `disabled`, and `logged`. ``` ```{config:option} bgp.ipv4.nexthop network-bridge-network-conf :condition: "BGP server" :defaultdesc: "local address" :scope: "local" :shortdesc: "Override the IPv4 next-hop for advertised prefixes" :type: "string" ``` ```{config:option} bgp.ipv6.nexthop network-bridge-network-conf :condition: "BGP server" :defaultdesc: "local address" :scope: "local" :shortdesc: "Override the IPv6 next-hop for advertised prefixes" :type: "string" ``` ```{config:option} bgp.peers.NAME.address network-bridge-network-conf :condition: "BGP server" :scope: "global" :shortdesc: "Peer address (IPv4 or IPv6)" :type: "string" ``` ```{config:option} bgp.peers.NAME.asn network-bridge-network-conf :condition: "BGP server" :scope: "global" :shortdesc: "Peer AS number" :type: "integer" ``` ```{config:option} bgp.peers.NAME.holdtime network-bridge-network-conf :condition: "BGP server" :defaultdesc: "`180`" :required: "no" :scope: "global" :shortdesc: "Peer session hold time" :type: "integer" Specify the hold time in seconds. ``` ```{config:option} bgp.peers.NAME.password network-bridge-network-conf :condition: "BGP server" :defaultdesc: "(no password)" :required: "no" :scope: "global" :shortdesc: "Peer session password" :type: "string" ``` ```{config:option} bridge.driver network-bridge-network-conf :defaultdesc: "`native`" :scope: "global" :shortdesc: "Bridge driver" :type: "string" Possible values are `native` and `openvswitch`. ``` ```{config:option} bridge.external_interfaces network-bridge-network-conf :scope: "local" :shortdesc: "Unconfigured network interfaces to include in the bridge" :type: "string" Specify a comma-separated list of unconfigured network interfaces to include in the bridge. ``` ```{config:option} bridge.hwaddr network-bridge-network-conf :scope: "global" :shortdesc: "MAC address for the bridge" :type: "string" ``` ```{config:option} bridge.mode network-bridge-network-conf :defaultdesc: "`standard`" :scope: "global" :shortdesc: "Bridge operation mode" :type: "string" Possible values are `standard` and `fan`. ``` ```{config:option} bridge.mtu network-bridge-network-conf :defaultdesc: "`1400` when tunnels are configured, otherwise `1500` if `bridge.mode=standard` or `1450` if `bridge.mode=fan`" :scope: "global" :shortdesc: "Bridge MTU" :type: "integer" The default value varies depending on whether the bridge uses a tunnel or a fan setup. ``` ```{config:option} dns.domain network-bridge-network-conf :defaultdesc: "`lxd`" :scope: "global" :shortdesc: "Domain to advertise to DHCP clients and use for DNS resolution" :type: "string" ``` ```{config:option} dns.mode network-bridge-network-conf :defaultdesc: "`managed`" :scope: "global" :shortdesc: "DNS registration mode" :type: "string" Possible values are `none` for no DNS record, `managed` for LXD-generated static records, and `dynamic` for client-generated records. ``` ```{config:option} dns.search network-bridge-network-conf :defaultdesc: "`dns.domain` value" :scope: "global" :shortdesc: "Full domain search list" :type: "string" Specify a comma-separated list of domains. ``` ```{config:option} dns.zone.forward network-bridge-network-conf :scope: "global" :shortdesc: "DNS zone names for forward DNS records" :type: "string" Specify a comma-separated list of DNS zone names. ``` ```{config:option} dns.zone.reverse.ipv4 network-bridge-network-conf :scope: "global" :shortdesc: "DNS zone name for IPv4 reverse DNS records" :type: "string" ``` ```{config:option} dns.zone.reverse.ipv6 network-bridge-network-conf :scope: "global" :shortdesc: "DNS zone name for IPv6 reverse DNS records" :type: "string" ``` ```{config:option} fan.overlay_subnet network-bridge-network-conf :condition: "fan mode" :defaultdesc: "`240.0.0.0/8`" :scope: "global" :shortdesc: "Subnet to use as the overlay for the FAN" :type: "string" Use CIDR notation. ``` ```{config:option} fan.underlay_subnet network-bridge-network-conf :condition: "fan mode" :defaultdesc: "initial value on creation: `auto`" :scope: "global" :shortdesc: "Subnet to use as the underlay for the FAN" :type: "string" Use CIDR notation. You can set the option to `auto` to use the default gateway subnet. ``` ```{config:option} ipv4.address network-bridge-network-conf :condition: "standard mode" :defaultdesc: "initial value on creation: `auto`" :scope: "global" :shortdesc: "IPv4 address for the bridge" :type: "string" Use CIDR notation. You can set the option to `none` to turn off IPv4, or to `auto` to generate a new random unused subnet. ``` ```{config:option} ipv4.dhcp network-bridge-network-conf :condition: "IPv4 address" :defaultdesc: "`true`" :scope: "global" :shortdesc: "Whether to allocate IPv4 addresses using DHCP" :type: "bool" ``` ```{config:option} ipv4.dhcp.expiry network-bridge-network-conf :condition: "IPv4 DHCP" :defaultdesc: "`1h`" :scope: "global" :shortdesc: "When to expire DHCP leases" :type: "string" ``` ```{config:option} ipv4.dhcp.gateway network-bridge-network-conf :condition: "IPv4 DHCP" :defaultdesc: "IPv4 address" :scope: "global" :shortdesc: "Address of the gateway for the IPv4 subnet" :type: "string" ``` ```{config:option} ipv4.dhcp.ranges network-bridge-network-conf :condition: "IPv4 DHCP" :defaultdesc: "all addresses" :scope: "global" :shortdesc: "IPv4 ranges to use for DHCP" :type: "string" Specify a comma-separated list of IPv4 ranges in FIRST-LAST format. ``` ```{config:option} ipv4.firewall network-bridge-network-conf :condition: "IPv4 address" :defaultdesc: "`true`" :scope: "global" :shortdesc: "Whether to generate filtering firewall rules for this network" :type: "bool" ``` ```{config:option} ipv4.nat network-bridge-network-conf :condition: "IPv4 address" :defaultdesc: "`false` (initial value on creation if `ipv4.address` is set to `auto`: `true`)" :scope: "global" :shortdesc: "Whether to use NAT for IPv4" :type: "bool" ``` ```{config:option} ipv4.nat.address network-bridge-network-conf :condition: "IPv4 address" :scope: "global" :shortdesc: "Source address used for outbound traffic from the bridge" :type: "string" ``` ```{config:option} ipv4.nat.order network-bridge-network-conf :condition: "IPv4 address" :defaultdesc: "`before`" :scope: "global" :shortdesc: "Where to add the required NAT rules" :type: "string" Set this option to `before` to add the NAT rules before any pre-existing rules, or to `after` to add them after the pre-existing rules. ``` ```{config:option} ipv4.ovn.ranges network-bridge-network-conf :scope: "global" :shortdesc: "IPv4 ranges to use for child OVN network routers" :type: "string" Specify a comma-separated list of IPv4 ranges in FIRST-LAST format. ``` ```{config:option} ipv4.routes network-bridge-network-conf :condition: "IPv4 address" :scope: "global" :shortdesc: "Additional IPv4 CIDR subnets to route to the bridge" :type: "string" Specify a comma-separated list of IPv4 CIDR subnets. ``` ```{config:option} ipv4.routing network-bridge-network-conf :condition: "IPv4 address" :defaultdesc: "`true`" :scope: "global" :shortdesc: "Whether to route IPv4 traffic in and out of the bridge" :type: "bool" ``` ```{config:option} ipv6.address network-bridge-network-conf :condition: "standard mode" :defaultdesc: "initial value on creation: `auto`" :scope: "global" :shortdesc: "IPv6 address for the bridge" :type: "string" Use CIDR notation. You can set the option to `none` to turn off IPv6, or to `auto` to generate a new random unused subnet. ``` ```{config:option} ipv6.dhcp network-bridge-network-conf :condition: "IPv6 address" :defaultdesc: "`true`" :scope: "global" :shortdesc: "Whether to provide additional network configuration over DHCP" :type: "bool" ``` ```{config:option} ipv6.dhcp.expiry network-bridge-network-conf :condition: "IPv6 DHCP" :defaultdesc: "`1h`" :scope: "global" :shortdesc: "When to expire DHCP leases" :type: "string" ``` ```{config:option} ipv6.dhcp.ranges network-bridge-network-conf :condition: "IPv6 stateful DHCP" :defaultdesc: "all addresses" :scope: "global" :shortdesc: "IPv6 ranges to use for DHCP" :type: "string" Specify a comma-separated list of IPv6 ranges in FIRST-LAST format. ``` ```{config:option} ipv6.dhcp.stateful network-bridge-network-conf :condition: "IPv6 DHCP" :defaultdesc: "`false`" :scope: "global" :shortdesc: "Whether to allocate IPv6 addresses using DHCP" :type: "bool" ``` ```{config:option} ipv6.firewall network-bridge-network-conf :condition: "IPv6 DHCP" :defaultdesc: "`true`" :scope: "global" :shortdesc: "Whether to generate filtering firewall rules for this network" :type: "bool" ``` ```{config:option} ipv6.nat network-bridge-network-conf :condition: "IPv6 address" :defaultdesc: "`false` (initial value on creation if `ipv6.address` is set to `auto`: `true`)" :scope: "global" :shortdesc: "Whether to use NAT for IPv6" :type: "bool" ``` ```{config:option} ipv6.nat.address network-bridge-network-conf :condition: "IPv6 address" :scope: "global" :shortdesc: "Source address used for outbound traffic from the bridge" :type: "string" ``` ```{config:option} ipv6.nat.order network-bridge-network-conf :condition: "IPv6 address" :defaultdesc: "`before`" :scope: "global" :shortdesc: "Where to add the required NAT rules" :type: "string" Set this option to `before` to add the NAT rules before any pre-existing rules, or to `after` to add them after the pre-existing rules. ``` ```{config:option} ipv6.ovn.ranges network-bridge-network-conf :scope: "global" :shortdesc: "IPv6 ranges to use for child OVN network routers" :type: "string" Specify a comma-separated list of IPv6 ranges in FIRST-LAST format. ``` ```{config:option} ipv6.routes network-bridge-network-conf :condition: "IPv6 address" :scope: "global" :shortdesc: "Additional IPv6 CIDR subnets to route to the bridge" :type: "string" Specify a comma-separated list of IPv6 CIDR subnets. ``` ```{config:option} ipv6.routing network-bridge-network-conf :condition: "IPv6 address" :scope: "global" :shortdesc: "Whether to route IPv6 traffic in and out of the bridge" :type: "bool" ``` ```{config:option} raw.dnsmasq network-bridge-network-conf :scope: "global" :shortdesc: "Additional `dnsmasq` configuration to append to the configuration file" :type: "string" Additional `dnsmasq` configuration is appended to the generated configuration file. This is a low-level option and is not recommended for production use, as it allows for unsupported configurations that may cease to work in future versions. ``` ```{config:option} security.acls network-bridge-network-conf :scope: "global" :shortdesc: "Network ACLs to apply to NICs connected to this network" :type: "string" Specify a comma-separated list of network ACLs. Also see {ref}`network-acls-bridge-limitations`. ``` ```{config:option} security.acls.default.egress.action network-bridge-network-conf :condition: "`security.acls`" :scope: "global" :shortdesc: "Default action to use for egress traffic" :type: "string" The specified action is used for all egress traffic that doesn’t match any ACL rule. ``` ```{config:option} security.acls.default.egress.logged network-bridge-network-conf :condition: "`security.acls`" :scope: "global" :shortdesc: "Whether to log egress traffic that doesn’t match any ACL rule" :type: "bool" ``` ```{config:option} security.acls.default.ingress.action network-bridge-network-conf :condition: "`security.acls`" :scope: "global" :shortdesc: "Default action to use for ingress traffic" :type: "string" The specified action is used for all ingress traffic that doesn’t match any ACL rule. ``` ```{config:option} security.acls.default.ingress.logged network-bridge-network-conf :condition: "`security.acls`" :scope: "global" :shortdesc: "Whether to log ingress traffic that doesn’t match any ACL rule" :type: "bool" ``` ```{config:option} tunnel.NAME.group network-bridge-network-conf :condition: "`vxlan`" :shortdesc: "Multicast address for `vxlan`" :type: "string" This address is used if {config:option}`network-bridge-network-conf:tunnel.NAME.local` and {config:option}`network-bridge-network-conf:tunnel.NAME.remote` aren’t set. ``` ```{config:option} tunnel.NAME.id network-bridge-network-conf :condition: "`vxlan`" :shortdesc: "Specific tunnel ID to use for the `vxlan` tunnel" :type: "integer" ``` ```{config:option} tunnel.NAME.interface network-bridge-network-conf :condition: "`vxlan`" :shortdesc: "Specific host interface to use for the tunnel" :type: "string" ``` ```{config:option} tunnel.NAME.local network-bridge-network-conf :condition: "`gre` or `vxlan`" :required: "not required for multicast `vxlan`" :shortdesc: "Local address for the tunnel" :type: "string" ``` ```{config:option} tunnel.NAME.port network-bridge-network-conf :condition: "`vxlan`" :defaultdesc: "`0`" :shortdesc: "Specific port to use for the `vxlan` tunnel" :type: "integer" ``` ```{config:option} tunnel.NAME.protocol network-bridge-network-conf :condition: "standard mode" :shortdesc: "Tunneling protocol" :type: "string" Possible values are `vxlan` and `gre`. ``` ```{config:option} tunnel.NAME.remote network-bridge-network-conf :condition: "`gre` or `vxlan`" :required: "not required for multicast `vxlan`" :shortdesc: "Remote address for the tunnel" :type: "string" ``` ```{config:option} tunnel.NAME.ttl network-bridge-network-conf :condition: "`vxlan`" :defaultdesc: "`1`" :shortdesc: "Specific TTL to use for multicast routing topologies" :type: "string" ``` ```{config:option} user.* network-bridge-network-conf :scope: "global" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} config network-forward-forward-properties :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string set" The only supported keys are `target_address` and `user.*` custom keys. The `target_address` key is for the default target address of the network forward. It must be an IP address within the subnet of the network the forward belongs to. ``` ```{config:option} description network-forward-forward-properties :required: "yes" :shortdesc: "Description of the network forward" :type: "string" ``` ```{config:option} listen_address network-forward-forward-properties :required: "no" :shortdesc: "IP address to listen on" :type: "string" See {ref}`network-forwards-listen-addresses`. ``` ```{config:option} ports network-forward-forward-properties :required: "no" :shortdesc: "List of port specifications" :type: "port list" See {ref}`network-forwards-port-specifications`. ``` ```{config:option} description network-forward-port-properties :required: "no" :shortdesc: "Description of the port or ports" :type: "string" ``` ```{config:option} listen_port network-forward-port-properties :required: "yes" :shortdesc: "Listen port or ports" :type: "string" For example: `80,90-100` ``` ```{config:option} protocol network-forward-port-properties :required: "yes" :shortdesc: "Protocol for the port or ports" :type: "string" Possible values are `tcp` and `udp`. ``` ```{config:option} target_address network-forward-port-properties :required: "yes" :shortdesc: "IP address to forward to" :type: "string" This `target_address` must be within the subnet of the network the forward belongs to. Also, it must be different from the forward’s default target address. ``` ```{config:option} target_port network-forward-port-properties :defaultdesc: "same as `listen_port`" :required: "no" :shortdesc: "Target port or ports" :type: "string" For example: `70,80-90` or `90` ``` ```{config:option} description network-load-balancer-load-balancer-backend-properties :required: "no" :shortdesc: "Description of the backend" :type: "string" ``` ```{config:option} name network-load-balancer-load-balancer-backend-properties :required: "yes" :shortdesc: "Name of the backend" :type: "string" ``` ```{config:option} target_address network-load-balancer-load-balancer-backend-properties :required: "yes" :shortdesc: "IP address to forward to" :type: "string" ``` ```{config:option} target_port network-load-balancer-load-balancer-backend-properties :defaultdesc: "same as {config:option}`network-load-balancer-load-balancer-port-properties:listen_port`" :required: "no" :shortdesc: "Target port or ports" :type: "string" For example: `70,80-90` or `90` ``` ```{config:option} description network-load-balancer-load-balancer-port-properties :required: "no" :shortdesc: "Description of the port or ports" :type: "string" ``` ```{config:option} listen_port network-load-balancer-load-balancer-port-properties :required: "yes" :shortdesc: "Listen port or ports" :type: "string" For example: `80,90-100` ``` ```{config:option} protocol network-load-balancer-load-balancer-port-properties :required: "yes" :shortdesc: "Protocol for the port or ports" :type: "string" Possible values are `tcp` and `udp`. ``` ```{config:option} target_backend network-load-balancer-load-balancer-port-properties :required: "no" :shortdesc: "Backend name(s) to forward to" :type: "backend list" If you do not provide a list of backends, then you must provide a pool to `target_pool`. ``` ```{config:option} target_pool network-load-balancer-load-balancer-port-properties :required: "no" :shortdesc: "Pool of instances to forward to" :type: "string" If you do not provide a pool, then you must provide a list of backends to `target_backend`. ``` ```{config:option} backends network-load-balancer-load-balancer-properties :required: "no" :shortdesc: "List of backend specifications" :type: "backend list" See {ref}`network-load-balancers-backend-specifications`. ``` ```{config:option} config network-load-balancer-load-balancer-properties :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string set" The only supported keys are `user.*` custom keys. ``` ```{config:option} description network-load-balancer-load-balancer-properties :required: "no" :shortdesc: "Description of the network load balancer" :type: "string" ``` ```{config:option} listen_address network-load-balancer-load-balancer-properties :required: "no" :shortdesc: "IP address to listen on" :type: "string" ``` ```{config:option} ports network-load-balancer-load-balancer-properties :required: "no" :shortdesc: "List of port specifications" :type: "port list" See {ref}`network-load-balancers-port-specifications`. ``` ```{config:option} healthcheck network-load-balancer-pool-properties :defaultdesc: "`true`" :required: "no" :shortdesc: "Whether to enable or disable health checks" :type: "bool" ``` ```{config:option} healthcheck.failure_count network-load-balancer-pool-properties :defaultdesc: "`1`" :required: "no" :shortdesc: "Number of failed probe attempts after which an instance is considered unhealthy." :type: "integer" ``` ```{config:option} healthcheck.interval network-load-balancer-pool-properties :defaultdesc: "`5`" :required: "no" :shortdesc: "Interval in seconds between probes of the pool's instances." :type: "integer" ``` ```{config:option} healthcheck.success_count network-load-balancer-pool-properties :defaultdesc: "`1`" :required: "no" :shortdesc: "Number of successful probe attempts after which an instance is considered healthy." :type: "integer" ``` ```{config:option} healthcheck.timeout network-load-balancer-pool-properties :defaultdesc: "`3`" :required: "no" :shortdesc: "Timeout in seconds after a probe appears to be faulty." :type: "integer" ``` ```{config:option} protocol network-load-balancer-pool-properties :defaultdesc: "`tcp`" :required: "no" :shortdesc: "Protocol used for ingress pool traffic." :type: "string" Can be either `tcp` or `udp`. ``` ```{config:option} target_port network-load-balancer-pool-properties :required: "yes" :shortdesc: "Port used on instances for ingress pool traffic" :type: "string" ``` ```{config:option} user.* network-load-balancer-pool-properties :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} gvrp network-macvlan-network-conf :defaultdesc: "`false`" :scope: "global" :shortdesc: "Whether to use GARP VLAN Registration Protocol" :type: "bool" This option specifies whether to register the VLAN using the GARP VLAN Registration Protocol. ``` ```{config:option} mtu network-macvlan-network-conf :scope: "global" :shortdesc: "MTU of the new interface" :type: "integer" ``` ```{config:option} parent network-macvlan-network-conf :scope: "local" :shortdesc: "Parent interface to create `macvlan` NICs on" :type: "string" ``` ```{config:option} user.* network-macvlan-network-conf :scope: "global" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} vlan network-macvlan-network-conf :scope: "global" :shortdesc: "VLAN ID to attach to" :type: "integer" ``` ```{config:option} acceleration.parent network-ovn-network-conf :shortdesc: "Physical function interfaces to allocate virtual functions from for hardware acceleration" :type: "string" Comma separated list of physical function (PF) interfaces to allocate virtual functions (VFs) from for hardware acceleration when {config:option}`device-nic-ovn-device-conf:acceleration` is enabled. See {ref}`devices-nic-hw-acceleration` for more information. ``` ```{config:option} bridge.hwaddr network-ovn-network-conf :shortdesc: "MAC address for the bridge" :type: "string" ``` ```{config:option} bridge.mtu network-ovn-network-conf :defaultdesc: "`1442`" :shortdesc: "Bridge MTU" :type: "integer" The default value allows the host to host Geneve tunnels. ``` ```{config:option} dns.domain network-ovn-network-conf :defaultdesc: "`lxd`" :shortdesc: "Domain to advertise to DHCP clients and use for DNS resolution" :type: "string" ``` ```{config:option} dns.search network-ovn-network-conf :defaultdesc: "`dns.domain` value" :shortdesc: "Full domain search list" :type: "string" Specify a comma-separated list of domains. ``` ```{config:option} dns.zone.forward network-ovn-network-conf :shortdesc: "DNS zone names for forward DNS records" :type: "string" Specify a comma-separated list of DNS zone names. ``` ```{config:option} dns.zone.reverse.ipv4 network-ovn-network-conf :shortdesc: "DNS zone name for IPv4 reverse DNS records" :type: "string" ``` ```{config:option} dns.zone.reverse.ipv6 network-ovn-network-conf :shortdesc: "DNS zone name for IPv6 reverse DNS records" :type: "string" ``` ```{config:option} ipv4.address network-ovn-network-conf :condition: "standard mode" :defaultdesc: "initial value on creation: `auto`" :shortdesc: "IPv4 address for the OVN network" :type: "string" Use CIDR notation. You can set the option to `none` to turn off IPv4, or to `auto` to generate a new random unused subnet. ``` ```{config:option} ipv4.dhcp network-ovn-network-conf :condition: "IPv4 address" :defaultdesc: "`true`" :shortdesc: "Whether to allocate IPv4 addresses using DHCP" :type: "bool" ``` ```{config:option} ipv4.dhcp.ranges network-ovn-network-conf :condition: "IPv4 DHCP" :defaultdesc: "all addresses" :scope: "global" :shortdesc: "IPv4 ranges to use for DHCP" :type: "string" Specify a comma-separated list of IPv4 ranges in FIRST-LAST format. ``` ```{config:option} ipv4.l3only network-ovn-network-conf :condition: "IPv4 address" :defaultdesc: "`false`" :shortdesc: "Whether to enable layer 3 only mode for IPv4" :type: "bool" ``` ```{config:option} ipv4.nat network-ovn-network-conf :condition: "IPv4 address" :defaultdesc: "`false` (initial value on creation if `ipv4.address` is set to `auto`: `true`)" :shortdesc: "Whether to use NAT for IPv4" :type: "bool" ``` ```{config:option} ipv4.nat.address network-ovn-network-conf :condition: "IPv4 address; requires uplink `ovn.ingress_mode=routed`" :shortdesc: "Source address used for outbound traffic from the network" :type: "string" ``` ```{config:option} ipv6.address network-ovn-network-conf :condition: "standard mode" :defaultdesc: "initial value on creation: `auto`" :shortdesc: "IPv6 address for the OVN network" :type: "string" Use CIDR notation. You can set the option to `none` to turn off IPv6, or to `auto` to generate a new random unused subnet. ``` ```{config:option} ipv6.dhcp network-ovn-network-conf :condition: "IPv6 address" :defaultdesc: "`true`" :shortdesc: "Whether to provide additional network configuration over DHCP" :type: "bool" ``` ```{config:option} ipv6.dhcp.stateful network-ovn-network-conf :condition: "IPv6 DHCP" :defaultdesc: "`false`" :shortdesc: "Whether to allocate IPv6 addresses using DHCP" :type: "bool" ``` ```{config:option} ipv6.l3only network-ovn-network-conf :condition: "IPv6 DHCP stateful" :defaultdesc: "`false`" :shortdesc: "Whether to enable layer 3 only mode for IPv6" :type: "bool" ``` ```{config:option} ipv6.nat network-ovn-network-conf :condition: "IPv6 address" :defaultdesc: "`false` (initial value on creation if `ipv6.address` is set to `auto`: `true`)" :shortdesc: "Whether to use NAT for IPv6" :type: "bool" ``` ```{config:option} ipv6.nat.address network-ovn-network-conf :condition: "IPv6 address; requires uplink `ovn.ingress_mode=routed`" :shortdesc: "Source address used for outbound traffic from the network" :type: "string" ``` ```{config:option} network network-ovn-network-conf :shortdesc: "Uplink network to use for external network access" :type: "string" ``` ```{config:option} security.acls network-ovn-network-conf :shortdesc: "Network ACLs to apply to NICs connected to this network" :type: "string" Specify a comma-separated list of network ACLs. ``` ```{config:option} security.acls.default.egress.action network-ovn-network-conf :condition: "`security.acls`" :defaultdesc: "`reject`" :shortdesc: "Default action to use for egress traffic" :type: "string" The specified action is used for all egress traffic that doesn’t match any ACL rule. ``` ```{config:option} security.acls.default.egress.logged network-ovn-network-conf :condition: "`security.acls`" :defaultdesc: "`false`" :shortdesc: "Whether to log egress traffic that doesn’t match any ACL rule" :type: "bool" ``` ```{config:option} security.acls.default.ingress.action network-ovn-network-conf :condition: "`security.acls`" :defaultdesc: "`reject`" :shortdesc: "Default action to use for ingress traffic" :type: "string" The specified action is used for all ingress traffic that doesn’t match any ACL rule. ``` ```{config:option} security.acls.default.ingress.logged network-ovn-network-conf :condition: "`security.acls`" :defaultdesc: "`false`" :shortdesc: "Whether to log ingress traffic that doesn’t match any ACL rule" :type: "bool" ``` ```{config:option} user.* network-ovn-network-conf :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} volatile.network.ipv4.address network-ovn-network-conf :shortdesc: "IPv4 address for the OVN network router on the uplink network" :type: "string" If not set, LXD allocates an address from the `ipv4.ovn.ranges` of the uplink network. ``` ```{config:option} volatile.network.ipv6.address network-ovn-network-conf :shortdesc: "IPv6 address for the OVN network router on the uplink network" :type: "string" If not set, LXD allocates an address from the `ipv6.ovn.ranges` of the uplink network. If the uplink network has no `ipv6.ovn.ranges`, LXD derives the address from the MAC address of the OVN network router. ``` ```{config:option} config network-peering-peering-properties :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string set" The only supported keys are `user.*` custom keys. ``` ```{config:option} description network-peering-peering-properties :required: "no" :shortdesc: "Description of the network peering" :type: "string" ``` ```{config:option} name network-peering-peering-properties :required: "yes" :shortdesc: "Name of the network peering on the local network" :type: "string" ``` ```{config:option} status network-peering-peering-properties :required: "--" :shortdesc: "Status indicating if pending or created" :type: "string" Indicates if mutual peering exists with the target network. This property is read-only and cannot be updated. ``` ```{config:option} target_network network-peering-peering-properties :required: "yes" :shortdesc: "Which network to create a peering with" :type: "string" This option must be set at create time. ``` ```{config:option} target_project network-peering-peering-properties :required: "yes" :shortdesc: "Which project the target network exists in" :type: "string" This option must be set at create time. ``` ```{config:option} bgp.peers.NAME.address network-physical-network-conf :condition: "BGP server" :scope: "global" :shortdesc: "Peer address for use by `ovn` downstream networks" :type: "string" The address can be IPv4 or IPv6. ``` ```{config:option} bgp.peers.NAME.asn network-physical-network-conf :condition: "BGP server" :scope: "global" :shortdesc: "Peer AS number for use by `ovn` downstream networks" :type: "integer" ``` ```{config:option} bgp.peers.NAME.holdtime network-physical-network-conf :condition: "BGP server" :defaultdesc: "`180`" :required: "no" :scope: "global" :shortdesc: "Peer session hold time" :type: "integer" Specify the peer session hold time in seconds. ``` ```{config:option} bgp.peers.NAME.password network-physical-network-conf :condition: "BGP server" :defaultdesc: "(no password)" :required: "no" :scope: "global" :shortdesc: "Peer session password for use by `ovn` downstream networks" :type: "string" ``` ```{config:option} dns.nameservers network-physical-network-conf :condition: "standard mode" :scope: "global" :shortdesc: "DNS server IPs on physical network" :type: "string" Specify a list of DNS server IPs. ``` ```{config:option} gvrp network-physical-network-conf :defaultdesc: "`false`" :scope: "global" :shortdesc: "Whether to use GARP VLAN Registration Protocol" :type: "bool" This option specifies whether to register the VLAN using the GARP VLAN Registration Protocol. ``` ```{config:option} ipv4.gateway network-physical-network-conf :condition: "standard mode" :scope: "global" :shortdesc: "IPv4 address for the gateway and network" :type: "string" Use CIDR notation. ``` ```{config:option} ipv4.ovn.ranges network-physical-network-conf :scope: "global" :shortdesc: "IPv4 ranges to use for child OVN network routers" :type: "string" Specify a comma-separated list of IPv4 ranges in FIRST-LAST format. ``` ```{config:option} ipv4.routes network-physical-network-conf :condition: "IPv4 address" :scope: "global" :shortdesc: "Additional IPv4 CIDR subnets" :type: "string" Specify a comma-separated list of IPv4 CIDR subnets that can be used with child OVN network forwarders, load-balancers and {config:option}`device-nic-ovn-device-conf:ipv4.routes.external` setting. ``` ```{config:option} ipv4.routes.anycast network-physical-network-conf :condition: "IPv4 address" :defaultdesc: "`false`" :scope: "global" :shortdesc: "Whether to allow IPv4 routes on multiple networks/NICs" :type: "bool" If set to `true`, this option allows the overlapping routes to be used on multiple networks/NICs at the same time. ``` ```{config:option} ipv6.gateway network-physical-network-conf :condition: "standard mode" :scope: "global" :shortdesc: "IPv6 address for the gateway and network" :type: "string" Use CIDR notation. ``` ```{config:option} ipv6.ovn.ranges network-physical-network-conf :scope: "global" :shortdesc: "IPv6 ranges to use for child OVN network routers" :type: "string" Specify a comma-separated list of IPv6 ranges in FIRST-LAST format. ``` ```{config:option} ipv6.routes network-physical-network-conf :condition: "IPv6 address" :scope: "global" :shortdesc: "Additional IPv6 CIDR subnets" :type: "string" Specify a comma-separated list of IPv6 CIDR subnets that can be used with child OVN network forwarders, load-balancers and {config:option}`device-nic-ovn-device-conf:ipv6.routes.external` setting. ``` ```{config:option} ipv6.routes.anycast network-physical-network-conf :condition: "IPv6 address" :defaultdesc: "`false`" :scope: "global" :shortdesc: "Whether to allow IPv6 routes on multiple networks/NICs" :type: "bool" If set to `true`, this option allows the overlapping routes to be used on multiple networks/NICs at the same time. ``` ```{config:option} mtu network-physical-network-conf :scope: "global" :shortdesc: "MTU of the new interface" :type: "integer" ``` ```{config:option} ovn.ingress_mode network-physical-network-conf :condition: "standard mode" :defaultdesc: "`l2proxy`" :scope: "global" :shortdesc: "How OVN NIC external IPs are advertised on uplink network" :type: "string" Possible values are `l2proxy` (proxy ARP/NDP) and `routed`. ``` ```{config:option} parent network-physical-network-conf :scope: "local" :shortdesc: "Existing interface to use for network" :type: "string" ``` ```{config:option} user.* network-physical-network-conf :scope: "global" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} vlan network-physical-network-conf :scope: "global" :shortdesc: "VLAN ID to attach to" :type: "integer" ``` ```{config:option} mtu network-sriov-network-conf :scope: "global" :shortdesc: "MTU of the new interface" :type: "integer" ``` ```{config:option} parent network-sriov-network-conf :scope: "local" :shortdesc: "Parent interface to create `sriov` NICs on" :type: "string" ``` ```{config:option} user.* network-sriov-network-conf :scope: "global" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} vlan network-sriov-network-conf :scope: "global" :shortdesc: "VLAN ID to attach to" :type: "integer" ``` ```{config:option} dns.nameservers network-zone-config-options :required: "no" :shortdesc: "Comma-separated list of DNS server FQDNs (for NS records)" :type: "string set" ``` ```{config:option} network.nat network-zone-config-options :defaultdesc: "true" :required: "no" :shortdesc: "Whether to generate records for NAT-ed subnets" :type: "bool" ``` ```{config:option} peers.NAME.address network-zone-config-options :required: "no" :shortdesc: "IP address of a DNS server" :type: "string" ``` ```{config:option} peers.NAME.key network-zone-config-options :required: "no" :shortdesc: "TSIG key for the server" :type: "string" ``` ```{config:option} user.* network-zone-config-options :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} config network-zone-record-properties :required: "no" :shortdesc: "User-provided free-form key/value pairs" :type: "string set" The only supported keys are `user.*` custom keys. ``` ```{config:option} description network-zone-record-properties :required: "no" :shortdesc: "Description of the record" :type: "string" ``` ```{config:option} entries network-zone-record-properties :required: "no" :shortdesc: "List of DNS entries" :type: "entry list" ``` ```{config:option} name network-zone-record-properties :required: "yes" :shortdesc: "Unique name of the record" :type: "string" ``` ```{config:option} policy placement-group-placement-group :required: "yes" :shortdesc: "Instance placement policy" :type: "string" Determines whether instances are spread across cluster members or compacted onto the same cluster member(s). Possible values are `spread` and `compact`. See {ref}`clustering-instance-placement` for more information. ``` ```{config:option} rigor placement-group-placement-group :required: "yes" :shortdesc: "Enforcement level of the placement policy" :type: "string" Determines whether the policy is strictly enforced or allows fallback. Possible values are `strict` and `permissive`. See {ref}`clustering-instance-placement` for more information. ``` ```{config:option} user.* placement-group-placement-group :shortdesc: "Free form user key/value storage" :type: "string" User keys can be used in search. ``` ```{config:option} features.images project-features :defaultdesc: "`false`" :initialvaluedesc: "`true`" :shortdesc: "Whether to use a separate set of images for the project" :type: "bool" This setting applies to both images and image aliases. ``` ```{config:option} features.networks project-features :defaultdesc: "`false`" :initialvaluedesc: "`false`" :shortdesc: "Whether to use a separate set of networks for the project" :type: "bool" This feature requires `features.profiles` to be enabled. ``` ```{config:option} features.networks.zones project-features :defaultdesc: "`false`" :initialvaluedesc: "`false`" :shortdesc: "Whether to use a separate set of network zones for the project" :type: "bool" ``` ```{config:option} features.profiles project-features :defaultdesc: "`false`" :initialvaluedesc: "`true`" :shortdesc: "Whether to use a separate set of profiles for the project" :type: "bool" ``` ```{config:option} features.storage.buckets project-features :defaultdesc: "`false`" :initialvaluedesc: "`true`" :shortdesc: "Whether to use a separate set of storage buckets for the project" :type: "bool" ``` ```{config:option} features.storage.volumes project-features :defaultdesc: "`false`" :initialvaluedesc: "`true`" :shortdesc: "Whether to use a separate set of storage volumes for the project" :type: "bool" ``` ```{config:option} limits.containers project-limits :shortdesc: "Maximum number of containers that can be created in the project" :type: "integer" ``` ```{config:option} limits.cpu project-limits :shortdesc: "Maximum number of CPUs to use in the project" :type: "integer" This value is the maximum value for the sum of the individual {config:option}`instance-resource-limits:limits.cpu` configurations set on the instances of the project. ``` ```{config:option} limits.disk project-limits :shortdesc: "Maximum disk space used by the project" :type: "string" This value is the maximum value of the aggregate disk space used by all instance volumes, custom volumes, and images of the project. ``` ```{config:option} limits.disk.pool.POOL_NAME project-limits :shortdesc: "Maximum disk space used by the project on this pool" :type: "string" This value is the maximum value of the aggregate disk space used by all instance volumes, custom volumes, and images of the project on this specific storage pool. When set to 0, the pool is excluded from storage pool list for the project. ``` ```{config:option} limits.instances project-limits :shortdesc: "Maximum number of instances that can be created in the project" :type: "integer" ``` ```{config:option} limits.memory project-limits :shortdesc: "Usage limit for the host's memory for the project" :type: "string" The value is the maximum value for the sum of the individual {config:option}`instance-resource-limits:limits.memory` configurations set on the instances of the project. ``` ```{config:option} limits.networks project-limits :shortdesc: "Maximum number of networks that the project can have" :type: "integer" ``` ```{config:option} limits.networks.uplink_ips.ipv4.NETWORK_NAME project-limits :shortdesc: "Quota of IPv4 addresses from a specified uplink network that can be used by entities in this project" :type: "string" Maximum number of IPv4 addresses that this project can consume from the specified uplink network. This number of IPs can be consumed by networks, forwards and load balancers in this project. ``` ```{config:option} limits.networks.uplink_ips.ipv6.NETWORK_NAME project-limits :shortdesc: "Quota of IPv6 addresses from a specified uplink network that can be used by entities in this project" :type: "string" Maximum number of IPv6 addresses that this project can consume from the specified uplink network. This number of IPs can be consumed by networks, forwards and load balancers in this project. ``` ```{config:option} limits.processes project-limits :shortdesc: "Maximum number of processes within the project" :type: "integer" This value is the maximum value for the sum of the individual {config:option}`instance-resource-limits:limits.processes` configurations set on the instances of the project. ``` ```{config:option} limits.virtual-machines project-limits :shortdesc: "Maximum number of VMs that can be created in the project" :type: "integer" ``` ```{config:option} replica.cluster project-replica :shortdesc: "Cluster link this project replicates with" :type: "string" Projects containing {ref}`replicators ` must specify this configuration value. Its meaning depends on the project replica mode: - `leader`: Specifies the cluster link that is used for replication. - `standby`: Specifies the cluster link that is allowed to perform replication (since all other modifications are denied in standby mode). ``` ```{config:option} restricted project-restricted :defaultdesc: "`false`" :shortdesc: "Whether to block access to security-sensitive features" :type: "bool" This option must be enabled to allow the `restricted.*` keys to take effect. To temporarily remove the restrictions, you can disable this option instead of clearing the related keys. ``` ```{config:option} restricted.backups project-restricted :defaultdesc: "`block`" :shortdesc: "When set to `block`, creating instance or volume backups is prevented" :type: "string" Possible values are `allow` or `block`. ``` ```{config:option} restricted.cluster.groups project-restricted :shortdesc: "Cluster groups that can be targeted" :type: "string" If specified, this option prevents targeting cluster groups other than the provided ones. ``` ```{config:option} restricted.cluster.target project-restricted :defaultdesc: "`block`" :shortdesc: "When set to `block`, targeting of cluster members is prevented" :type: "string" Possible values are `allow` or `block`. When set to `allow`, this option allows targeting of cluster members (either directly or via a group) when creating or moving instances. ``` ```{config:option} restricted.containers.interception project-restricted :defaultdesc: "`block`" :shortdesc: "When set to `block`, using system call interception options is prevented" :type: "string" Possible values are `allow`, `block`, or `full`. When set to `allow`, interception options that are usually safe are allowed. File system mounting remains blocked. ``` ```{config:option} restricted.containers.lowlevel project-restricted :defaultdesc: "`block`" :shortdesc: "When set to `block`, using low-level container options is prevented" :type: "string" Possible values are `allow` or `block`. When set to `allow`, low-level container options like {config:option}`instance-raw:raw.lxc`, {config:option}`instance-raw:raw.idmap`, `volatile.*`, etc. can be used. ``` ```{config:option} restricted.containers.nesting project-restricted :defaultdesc: "`block`" :shortdesc: "When set to `block`, running nested LXD is prevented" :type: "string" Possible values are `allow` or `block`. When set to `allow`, {config:option}`instance-security:security.nesting` can be set to `true` for an instance. ``` ```{config:option} restricted.containers.privilege project-restricted :defaultdesc: "`unprivileged`" :shortdesc: "Which settings for privileged containers to prevent" :type: "string" Possible values are `unprivileged`, `isolated`, and `allow`. - When set to `unprivileged`, this option prevents setting {config:option}`instance-security:security.privileged` to `true`. - When set to `isolated`, this option prevents setting {config:option}`instance-security:security.privileged` to `true` and forces using a unique idmap per container using {config:option}`instance-security:security.idmap.isolated` set to `true`. - When set to `allow`, there is no restriction. ``` ````{config:option} restricted.devices.disk project-restricted :defaultdesc: "`managed`" :shortdesc: "Which disk devices can be used" :type: "string" Possible values are `allow`, `block`, or `managed`. - When set to `block`, this option prevents using all disk devices except the root one. - When set to `managed`, this option allows using disk devices only if `pool=` is set. - When set to `allow`, there is no restriction on which disk devices can be used. ```{important} When allowing all disk devices, make sure to set {config:option}`project-restricted:restricted.devices.disk.paths` to a list of path prefixes that you want to allow. If you do not restrict the allowed paths, users can attach any disk device, including shifted devices (`disk` devices with [`shift`](devices-disk-options) set to `true`), which can be used to gain root access to the system. ``` ```` ```{config:option} restricted.devices.disk.paths project-restricted :shortdesc: "Which `source` can be used for `disk` devices" :type: "string" If {config:option}`project-restricted:restricted.devices.disk` is set to `allow`, this option controls which `source` can be used for `disk` devices. Specify a comma-separated list of path prefixes that restrict the `source` setting. If this option is left empty, all paths are allowed. ``` ```{config:option} restricted.devices.gpu project-restricted :defaultdesc: "`block`" :shortdesc: "When set to `block`, using devices of type `gpu` is prevented" :type: "string" Possible values are `allow` or `block`. ``` ```{config:option} restricted.devices.infiniband project-restricted :defaultdesc: "`block`" :shortdesc: "When set to `block`, using devices of type `infiniband` is prevented" :type: "string" Possible values are `allow` or `block`. ``` ```{config:option} restricted.devices.nic project-restricted :defaultdesc: "`managed`" :shortdesc: "Which network devices can be used" :type: "string" Possible values are `allow`, `block`, or `managed`. - When set to `block`, this option prevents using all network devices. - When set to `managed`, this option allows using network devices only if `network=` is set. - When set to `allow`, there is no restriction on which network devices can be used. ``` ```{config:option} restricted.devices.pci project-restricted :defaultdesc: "`block`" :shortdesc: "When set to `block`, using devices of type `pci` is prevented" :type: "string" Possible values are `allow` or `block`. ``` ```{config:option} restricted.devices.proxy project-restricted :defaultdesc: "`block`" :shortdesc: "When set to `block`, using devices of type `proxy` is prevented" :type: "string" Possible values are `allow` or `block`. ``` ```{config:option} restricted.devices.unix-block project-restricted :defaultdesc: "`block`" :shortdesc: "When set to `block`, using devices of type `unix-block` is prevented" :type: "string" Possible values are `allow` or `block`. ``` ```{config:option} restricted.devices.unix-char project-restricted :defaultdesc: "`block`" :shortdesc: "When set to `block`, using devices of type `unix-char` is prevented" :type: "string" Possible values are `allow` or `block`. ``` ```{config:option} restricted.devices.unix-hotplug project-restricted :defaultdesc: "`block`" :shortdesc: "When set to `block`, using devices of type `unix-hotplug` is prevented" :type: "string" Possible values are `allow` or `block`. ``` ```{config:option} restricted.devices.usb project-restricted :defaultdesc: "`block`" :shortdesc: "When set to `block`, using devices of type `usb` is prevented" :type: "string" Possible values are `allow` or `block`. ``` ```{config:option} restricted.idmap.gid project-restricted :shortdesc: "Which host GID ranges are allowed in `raw.idmap`" :type: "string" This option specifies the host GID ranges that are allowed in the instance's {config:option}`instance-raw:raw.idmap` setting. ``` ```{config:option} restricted.idmap.uid project-restricted :shortdesc: "Which host UID ranges are allowed in `raw.idmap`" :type: "string" This option specifies the host UID ranges that are allowed in the instance's {config:option}`instance-raw:raw.idmap` setting. ``` ```{config:option} restricted.networks.access project-restricted :shortdesc: "Which network names are allowed for use in this project" :type: "string" Specify a comma-delimited list of network names that are allowed for use in this project. If this option is not set, all networks are accessible. Note that this setting depends on the {config:option}`project-restricted:restricted.devices.nic` setting. ``` ```{config:option} restricted.networks.subnets project-restricted :defaultdesc: "`block`" :shortdesc: "Which network subnets are allocated for use in this project" :type: "string" Specify a comma-delimited list of CIDR network routes from the uplink network's {config:option}`network-physical-network-conf:ipv4.routes` {config:option}`network-physical-network-conf:ipv6.routes` that are allowed for use in this project. Use the form `:`. Example value: `lxdbr0:192.0.168.0/24,lxdbr0:10.1.19.5/32` ``` ```{config:option} restricted.networks.uplinks project-restricted :defaultdesc: "`block`" :shortdesc: "Which network names can be used as uplink in this project" :type: "string" Specify a comma-delimited list of network names that can be used as uplink for networks in this project. ``` ```{config:option} restricted.networks.zones project-restricted :defaultdesc: "`block`" :shortdesc: "Which network zones can be used in this project" :type: "string" Specify a comma-delimited list of network zones that can be used (or something under them) in this project. ``` ```{config:option} restricted.registries project-restricted :defaultdesc: "`builtin`" :shortdesc: "Which image registries can be used in this project" :type: "string" Specify a comma-delimited list of image registry names that are allowed for use in this project. The value can also be one of the keywords `allow`, `block`, or `builtin`. - When set to `allow`, there is no restriction on which image registries can be used. - When set to `block`, this option prevents using all image registries, including the built-in ones. - When set to `builtin`, the built-in image registries are excluded from the restriction. This keyword can also be included in the comma-delimited list. ``` ```{config:option} restricted.snapshots project-restricted :defaultdesc: "`block`" :shortdesc: "When set to `block`, creating instance or volume snapshots is prevented" :type: "string" ``` ```{config:option} restricted.virtual-machines.lowlevel project-restricted :defaultdesc: "`block`" :shortdesc: "When set to `block`, using low-level VM options is prevented" :type: "string" Possible values are `allow` or `block`. When set to `allow`, low-level VM options like {config:option}`instance-raw:raw.qemu`, `volatile.*`, etc. can be used. ``` ```{config:option} backups.compression_algorithm project-specific :shortdesc: "Compression algorithm to use for backups" :type: "string" Specify which compression algorithm to use for backups in this project. Possible values are `bzip2`, `gzip`, `lzma`, `xz`, or `none`. ``` ```{config:option} images.auto_update_cached project-specific :shortdesc: "Whether to automatically update cached images in the project" :type: "bool" ``` ```{config:option} images.auto_update_interval project-specific :shortdesc: "Interval at which to look for updates to cached images" :type: "integer" Specify the interval in hours. To disable looking for updates to cached images, set this option to `0`. ``` ```{config:option} images.compression_algorithm project-specific :shortdesc: "Compression algorithm to use for new images in the project" :type: "string" Possible values are `bzip2`, `gzip`, `lzma`, `xz`, or `none`. ``` ```{config:option} images.default_architecture project-specific :shortdesc: "Default architecture to use in a mixed-architecture cluster" :type: "string" ``` ```{config:option} images.remote_cache_expiry project-specific :shortdesc: "When an unused cached remote image is flushed in the project" :type: "integer" Specify the number of days after which the unused cached image expires. ``` ```{config:option} user.* project-specific :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} cluster replicator-conf :scope: "global" :shortdesc: "Target cluster link name." :type: "string" Required when creating a replicator. ``` ```{config:option} schedule replicator-conf :scope: "global" :shortdesc: "Cron expression for the replication schedule." :type: "string" Specify a cron expression for the replication schedule. For example, `@daily` or `0 6 * * *`. ``` ```{config:option} user.* replicator-miscellaneous :shortdesc: "Free form user key/value storage" :type: "string" User keys can be used in search. ``` ```{config:option} acme.agree_tos server-acme :defaultdesc: "`false`" :scope: "global" :shortdesc: "Agree to ACME terms of service" :type: "bool" ``` ```{config:option} acme.ca_url server-acme :defaultdesc: "`https://acme-v02.api.letsencrypt.org/directory`" :scope: "global" :shortdesc: "URL to the directory resource of the ACME service" :type: "string" ``` ```{config:option} acme.domain server-acme :scope: "global" :shortdesc: "Domain for which the certificate is issued" :type: "string" ``` ```{config:option} acme.email server-acme :scope: "global" :shortdesc: "Email address used for the account registration" :type: "string" ``` ```{config:option} cluster.https_address server-cluster :scope: "local" :shortdesc: "Address to use for clustering traffic" :type: "string" See {ref}`cluster-https-address`. ``` ```{config:option} cluster.images_minimal_replica server-cluster :defaultdesc: "`3`" :scope: "global" :shortdesc: "Number of cluster members that replicate an image" :type: "integer" Specify the minimal number of cluster members that keep a copy of a particular image. Set this option to `1` for no replication, or to `-1` to replicate images on all members. ``` ```{config:option} cluster.join_token_expiry server-cluster :defaultdesc: "`3H`" :scope: "global" :shortdesc: "Time after which a cluster join token expires" :type: "string" ``` ```{config:option} cluster.max_standby server-cluster :defaultdesc: "`2`" :scope: "global" :shortdesc: "Number of database stand-by members" :type: "integer" Specify the maximum number of cluster members that are assigned the database stand-by role. This must be a number between `0` and `5`. ``` ```{config:option} cluster.max_voters server-cluster :defaultdesc: "`3`" :scope: "global" :shortdesc: "Number of database voter members" :type: "integer" Specify the maximum number of cluster members that are assigned the database voter role. This must be an odd number >= `3`. ``` ```{config:option} cluster.offline_threshold server-cluster :defaultdesc: "`20`" :scope: "global" :shortdesc: "Threshold when an unresponsive member is considered offline" :type: "integer" Specify the number of seconds after which an unresponsive member is considered offline. ``` ```{config:option} core.auth_secret_expiry server-core :defaultdesc: "`1m`" :scope: "global" :shortdesc: "How long to use a given cluster secret" :type: "string" The secret is used for various cryptographic purposes, such as cookie encryption. When a given secret is older than the configured expiry, a new secret is generated. This configuration option accepts multiple space-separated values of the form `[0-9]+(S|M|H|d|w|m|y)`, where `S` is seconds, `M` is minutes, `H` is hours, `d` is days, `w` is weeks, `m` is months, and `y` is years. For example, `1d 3H` is 1 day and 3 hours. The default value is `1m` (1 month). The minimum value is `1d` (1 day). ``` ```{config:option} core.bgp_address server-core :scope: "local" :shortdesc: "Address to bind the BGP server to" :type: "string" See {ref}`network-bgp`. ``` ```{config:option} core.bgp_asn server-core :defaultdesc: "`0`" :scope: "global" :shortdesc: "BGP Autonomous System Number for the local server" :type: "string" ``` ```{config:option} core.bgp_routerid server-core :scope: "local" :shortdesc: "A unique identifier for the BGP server" :type: "string" The identifier must be formatted as an IPv4 address. ``` ```{config:option} core.debug_address server-core :scope: "local" :shortdesc: "Address to bind the [`pprof`](https://pkg.go.dev/net/http/pprof) debug server to (HTTP)" :type: "string" ``` ```{config:option} core.dns_address server-core :scope: "local" :shortdesc: "Address to bind the authoritative DNS server to" :type: "string" See {ref}`network-dns-server`. ``` ```{config:option} core.https_address server-core :scope: "local" :shortdesc: "Address to bind for the remote API (HTTPS)" :type: "string" See {ref}`server-expose`. ``` ```{config:option} core.https_allowed_credentials server-core :defaultdesc: "`false`" :scope: "global" :shortdesc: "Whether to set `Access-Control-Allow-Credentials`" :type: "bool" If enabled, the `Access-Control-Allow-Credentials` HTTP header value is set to `true`. ``` ```{config:option} core.https_allowed_headers server-core :scope: "global" :shortdesc: "`Access-Control-Allow-Headers` HTTP header value" :type: "string" ``` ```{config:option} core.https_allowed_methods server-core :scope: "global" :shortdesc: "`Access-Control-Allow-Methods` HTTP header value" :type: "string" ``` ```{config:option} core.https_allowed_origin server-core :scope: "global" :shortdesc: "`Access-Control-Allow-Origin` HTTP header value" :type: "string" ``` ```{config:option} core.https_trusted_proxy server-core :scope: "global" :shortdesc: "Trusted servers to provide the client's address via the PROXY protocol" :type: "string" Specify a comma-separated list of IP addresses of trusted servers that provide the client's address through the PROXY protocol connection header. ``` ```{config:option} core.metrics_address server-core :scope: "local" :shortdesc: "Address to bind the metrics server to (HTTPS)" :type: "string" See {ref}`metrics`. ``` ```{config:option} core.metrics_authentication server-core :defaultdesc: "`true`" :scope: "global" :shortdesc: "Whether to enforce authentication on the metrics endpoint" :type: "bool" ``` ```{config:option} core.proxy_http server-core :scope: "global" :shortdesc: "HTTP proxy to use" :type: "string" If this option is not specified, LXD falls back to the `HTTP_PROXY` environment variable (if set). ``` ```{config:option} core.proxy_https server-core :scope: "global" :shortdesc: "HTTPS proxy to use" :type: "string" If this option is not specified, LXD falls back to the `HTTPS_PROXY` environment variable (if set). ``` ```{config:option} core.proxy_ignore_hosts server-core :scope: "global" :shortdesc: "Hosts that don't need the proxy" :type: "string" Specify this option in a similar format to `NO_PROXY` (for example, `1.2.3.4,1.2.3.5`) If this option is not specified, LXD falls back to the `NO_PROXY` environment variable (if set). ``` ```{config:option} core.remote_token_expiry server-core :defaultdesc: "`15d`" :scope: "global" :shortdesc: "Time after which a remote add token expires" :type: "string" ``` ```{config:option} core.shutdown_timeout server-core :defaultdesc: "`5`" :scope: "global" :shortdesc: "How long to wait before shutdown" :type: "integer" Specify the number of minutes to wait for running operations to complete before the LXD server shuts down. ``` ```{config:option} core.syslog_socket server-core :defaultdesc: "`false`" :scope: "local" :shortdesc: "Whether to enable the syslog unixgram socket listener" :type: "bool" Set this option to `true` to enable the syslog unixgram socket to receive log messages from external processes. ``` ```{config:option} core.trust_ca_certificates server-core :defaultdesc: "`false`" :scope: "global" :shortdesc: "Whether to automatically trust clients signed by the CA" :type: "bool" ``` ```{config:option} images.auto_update_cached server-images :defaultdesc: "`true`" :scope: "global" :shortdesc: "Whether to automatically update cached images" :type: "bool" ``` ```{config:option} images.auto_update_interval server-images :defaultdesc: "`6`" :scope: "global" :shortdesc: "Interval at which to look for updates to cached images" :type: "integer" Specify the interval in hours. To disable looking for updates to cached images, set this option to `0`. ``` ```{config:option} images.compression_algorithm server-images :defaultdesc: "`gzip`" :scope: "global" :shortdesc: "Compression algorithm to use for new images" :type: "string" Possible values are `bzip2`, `gzip`, `lzma`, `xz`, or `none`. ``` ```{config:option} images.default_architecture server-images :shortdesc: "Default architecture to use in a mixed-architecture cluster" :type: "string" ``` ```{config:option} images.remote_cache_expiry server-images :defaultdesc: "`10`" :scope: "global" :shortdesc: "When an unused cached remote image is flushed" :type: "integer" Specify the number of days after which the unused cached image expires. ``` ```{config:option} loki.api.ca_cert server-loki :scope: "global" :shortdesc: "CA certificate for the Loki server" :type: "string" ``` ```{config:option} loki.api.check_ready server-loki :defaultdesc: "`true`" :scope: "global" :shortdesc: "Loki server readiness check" :type: "bool" Specify whether to check if the Loki server is ready before sending logs. Disabling it may be necessary for endpoints that don't expose Loki readiness checks. ``` ```{config:option} loki.api.url server-loki :scope: "global" :shortdesc: "URL to the Loki server" :type: "string" Specify the protocol, name or IP and port. For example `https://loki.example.com:3100`. LXD will automatically add the `/loki/api/v1/push` suffix so there's no need to add it here. ``` ```{config:option} loki.auth.password server-loki :scope: "global" :shortdesc: "Password used for Loki authentication" :type: "string" ``` ```{config:option} loki.auth.username server-loki :scope: "global" :shortdesc: "User name used for Loki authentication" :type: "string" ``` ```{config:option} loki.instance server-loki :defaultdesc: "Local server host name or cluster member name" :scope: "global" :shortdesc: "Name to use as the instance field in Loki events." :type: "string" This allows replacing the default instance value (server host name) by a more relevant value like a cluster identifier. ``` ```{config:option} loki.labels server-loki :scope: "global" :shortdesc: "Labels for a Loki log entry" :type: "string" Specify a comma-separated list of values that should be used as labels for a Loki log entry. ``` ```{config:option} loki.loglevel server-loki :defaultdesc: "`info`" :scope: "global" :shortdesc: "Minimum log level to send to the Loki server" :type: "string" ``` ```{config:option} loki.types server-loki :defaultdesc: "`lifecycle,logging`" :scope: "global" :shortdesc: "Events to send to the Loki server" :type: "string" Specify a comma-separated list of events to send to the Loki server. The events can be any combination of `lifecycle`, `logging`, `ovn`, and `security`. ``` ```{config:option} backups.compression_algorithm server-miscellaneous :defaultdesc: "`gzip`" :scope: "global" :shortdesc: "Compression algorithm to use for backups" :type: "string" Possible values are `bzip2`, `gzip`, `lzma`, `xz`, or `none`. ``` ```{config:option} instances.migration.stateful server-miscellaneous :defaultdesc: "`false`" :scope: "global" :shortdesc: "Whether to set `migration.stateful` to `true` for the instances" :type: "bool" You can override this setting for relevant instances, either in the instance-specific configuration or through a profile. ``` ```{config:option} instances.nic.host_name server-miscellaneous :defaultdesc: "`random`" :scope: "global" :shortdesc: "How to set the host name for a NIC" :type: "string" Possible values are `random` and `mac`. If set to `random`, use the random host interface name as the host name. If set to `mac`, generate a host name in the form `lxd` (MAC without leading two digits). ``` ```{config:option} network.ovn.ca_cert server-miscellaneous :defaultdesc: "Content of `/etc/ovn/ovn-central.crt` if present" :scope: "global" :shortdesc: "OVN SSL certificate authority" :type: "string" ``` ```{config:option} network.ovn.client_cert server-miscellaneous :defaultdesc: "Content of `/etc/ovn/cert_host` if present" :scope: "global" :shortdesc: "OVN SSL client certificate" :type: "string" ``` ```{config:option} network.ovn.client_key server-miscellaneous :defaultdesc: "Content of `/etc/ovn/key_host` if present" :scope: "global" :shortdesc: "OVN SSL client key" :type: "string" ``` ```{config:option} network.ovn.integration_bridge server-miscellaneous :defaultdesc: "`br-int`" :scope: "global" :shortdesc: "OVS integration bridge to use for OVN networks" :type: "string" ``` ```{config:option} network.ovn.northbound_connection server-miscellaneous :defaultdesc: "`unix:/var/run/ovn/ovnnb_db.sock` or MicroOVN configuration" :scope: "global" :shortdesc: "OVN northbound database connection string (default determined by environment)" :type: "string" Specify a connection string for OVN Northbound database. If value is not specified, LXD will determine the connection string based on the environment. LXD snap will use the MicroOVN environment settings if connected to MicroOVN snap. Otherwise, LXD will use `unix:/var/run/ovn/ovnnb_db.sock` as the connection string. ``` ```{config:option} storage.backups_volume server-miscellaneous :scope: "local" :shortdesc: "Volume to use to store backup tarballs" :type: "string" Specify the volume using the syntax `POOL/VOLUME`. ``` ```{config:option} storage.images_volume server-miscellaneous :scope: "local" :shortdesc: "Volume to use to store the image tarballs" :type: "string" Specify the volume using the syntax `POOL/VOLUME`. ``` ```{config:option} storage.project.{name}.backups_volume server-miscellaneous :scope: "local" :shortdesc: "Volume to use to store project-specific backup tarballs" :type: "string" Specify the volume using the syntax `POOL/VOLUME`. ``` ```{config:option} storage.project.{name}.images_volume server-miscellaneous :scope: "local" :shortdesc: "Volume to use to store project-specific image tarballs" :type: "string" Specify the volume using the syntax `POOL/VOLUME`. ``` ```{config:option} user.* server-miscellaneous :scope: "global" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} user.instances.placement.scriptlet server-miscellaneous :scope: "global" :shortdesc: "Legacy storage for `instances.placement.scriptlet` (no effect)" :type: "string" Stores the migrated value from the deprecated `instances.placement.scriptlet` configuration key. LXD ignores this key; changing it has no effect. It exists only to preserve previously stored data and may be removed in a future release. ``` ```{config:option} volatile.uuid server-miscellaneous :scope: "global" :shortdesc: "A random v7 UUID" :type: "string" This UUID is used as a stable identifier for the cluster. It cannot be changed. ``` ```{config:option} oidc.audience server-oidc :scope: "global" :shortdesc: "Expected audience value for the application" :type: "string" This value is required by some providers. ``` ```{config:option} oidc.client.id server-oidc :scope: "global" :shortdesc: "OpenID Connect client ID" :type: "string" ``` ```{config:option} oidc.client.secret server-oidc :scope: "global" :shortdesc: "OpenID Connect client secret" :type: "string" ``` ```{config:option} oidc.device.client.id server-oidc :defaultdesc: "The value of `oidc.client.id` if set." :scope: "global" :shortdesc: "OpenID Connect client ID (CLI)" :type: "string" The OIDC client ID used by the LXD CLI. This configuration value overrides the client ID that is made public to the LXD CLI. ``` ```{config:option} oidc.groups.claim server-oidc :scope: "global" :shortdesc: "A claim used for mapping identity provider groups to LXD groups." :type: "string" Specify a custom token claim to denote groups defined at the identity provider. The contents of this claim can be mapped to LXD groups for managing access control. The value of the claim is expected to be a JSON string array. ``` ```{config:option} oidc.issuer server-oidc :scope: "global" :shortdesc: "OpenID Connect Discovery URL for the provider" :type: "string" ``` ```{config:option} oidc.scopes server-oidc :defaultdesc: "`openid email offline_access profile`" :scope: "global" :shortdesc: "Space-separated list of OpenID Connect scopes" :type: "space-delimited string" A list of OpenID Connect scopes to request from the identity provider. This must include the `openid` and `email` scopes. The remaining optional scopes are `offline_access` and `profile`. If you remove the `offline_access` scope, users might be required to log in more frequently. If you remove the `profile` scope, user information may not be displayed in LXD UI (or in `lxc auth identity` commands). You may add additional scopes if this is required by your identity provider, or if necessary for configuration of {ref}`identity provider groups `. ``` ```{config:option} oidc.session.expiry server-oidc :defaultdesc: "`1w`" :scope: "global" :shortdesc: "The duration of an OIDC session" :type: "string" The duration of an OIDC session. This configuration option accepts multiple space-separated values of the form `[0-9]+(S|M|H|d|w|m|y)`, where `S` is seconds, `M` is minutes, `H` is hours, `d` is days, `w` is weeks, `m` is months, and `y` is years. For example, `1d 3H` is 1 day and 3 hours. The default value is `1w` (1 week). The minimum value is `1H` (1 hour). ``` ```{config:option} alletra.cpg storage-alletra-pool-conf :shortdesc: "HPE Alletra Common Provisioning Group (CPG) name" :type: "string" ``` ```{config:option} alletra.mode storage-alletra-pool-conf :defaultdesc: "`nvme/tcp`" :shortdesc: "How volumes are mapped to the local server" :type: "string" The mode to use to map storage volumes to the local server. Supported values are `iscsi` and `nvme/tcp`. ``` ```{config:option} alletra.target storage-alletra-pool-conf :defaultdesc: "all available targets" :shortdesc: "List of target addresses." :type: "string" A comma-separated list of target addresses. If empty, LXD discovers and connects to all available targets. Otherwise, it only connects to the specified addresses. ``` ```{config:option} alletra.user.name storage-alletra-pool-conf :shortdesc: "HPE Alletra storage admin username" :type: "string" ``` ```{config:option} alletra.user.password storage-alletra-pool-conf :shortdesc: "HPE Alletra storage admin password" :type: "string" ``` ```{config:option} alletra.wsapi storage-alletra-pool-conf :shortdesc: "Address of the HPE Alletra Storage UI/WSAPI" :type: "string" ``` ```{config:option} alletra.wsapi.verify storage-alletra-pool-conf :defaultdesc: "`true`" :shortdesc: "Whether to verify the HPE Alletra Storage UI/WSAPI certificate" :type: "bool" ``` ```{config:option} rsync.bwlimit storage-alletra-pool-conf :defaultdesc: "`0` (no limit)" :scope: "global" :shortdesc: "Upper limit on the socket I/O for `rsync`" :type: "string" When `rsync` must be used to transfer storage entities, this option specifies the upper limit to be placed on the socket I/O. ``` ```{config:option} rsync.compression storage-alletra-pool-conf :defaultdesc: "`true`" :scope: "global" :shortdesc: "Whether to use compression while migrating storage pools" :type: "bool" ``` ```{config:option} source.recover storage-alletra-pool-conf :defaultdesc: "`false`" :scope: "local" :shortdesc: "Whether to recover an existing `source`" :type: "bool" Set this option to true to recover an existing source which was previously created by LXD. ``` ```{config:option} user.* storage-alletra-pool-conf :scope: "global" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} volume.size storage-alletra-pool-conf :defaultdesc: "`10GiB`" :shortdesc: "Size/quota of the storage volume" :type: "string" Default storage volume size rounded to 256MiB. The minimum size is 256MiB. ``` ```{config:option} block.filesystem storage-alletra-volume-conf :condition: "block-based volume with content type `filesystem`" :defaultdesc: "same as `volume.block.filesystem`" :shortdesc: "File system of the storage volume" :type: "string" Valid options: `btrfs`, `ext4`, `xfs` If not set, `ext4` is assumed. ``` ```{config:option} block.mount_options storage-alletra-volume-conf :condition: "block-based volume with content type `filesystem`" :defaultdesc: "same as `volume.block.mount_options`" :shortdesc: "Mount options for block-backed file system volumes" :type: "string" ``` ```{config:option} security.shared storage-alletra-volume-conf :condition: "virtual-machine or custom block volume" :defaultdesc: "same as `volume.security.shared` or `false`" :scope: "global" :shortdesc: "Enable volume sharing" :type: "bool" Enable this option to allow the volume to be shared across multiple instances despite the possibility of data loss. ``` ```{config:option} security.shifted storage-alletra-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.security.shifted` or `false`" :scope: "global" :shortdesc: "Enable ID shifting overlay" :type: "bool" Enable this option to allow the volume to be attached to multiple isolated instances. ``` ```{config:option} security.unmapped storage-alletra-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.security.unmapped` or `false`" :scope: "global" :shortdesc: "Disable ID mapping for the volume" :type: "bool" ``` ```{config:option} size storage-alletra-volume-conf :defaultdesc: "`10GiB`" :shortdesc: "Size/quota of the storage volume" :type: "string" Default storage volume size rounded to 256MiB. The minimum size is 256MiB. ``` ```{config:option} snapshots.expiry storage-alletra-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.snapshots.expiry`" :scope: "global" :shortdesc: "Time until snapshots are deleted" :type: "string" Specify an expression like `1M 2H 3d 4w 5m 6y`. ``` ```{config:option} snapshots.pattern storage-alletra-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.snapshots.pattern` or `snap%d`" :scope: "global" :shortdesc: "Template for the snapshot name" :type: "string" You can specify a naming template for scheduled snapshots and unnamed snapshots. The `snapshots.pattern` option takes a Pongo2 template string to format the snapshot name. To add a time stamp to the snapshot name, use the Pongo2 context variable `creation_date`. Make sure to format the date in your template string to avoid forbidden characters in the snapshot name. For example, set `snapshots.pattern` to `{{ creation_date|date:'2006-01-02_15-04-05' }}` to name the snapshots after their time of creation, down to the precision of a second. Another way to avoid name collisions is to use the placeholder `%d` in the pattern. If no matching snapshots exist, the placeholder is replaced with `0`. Otherwise, it is replaced with the next snapshot index, which is one higher than the highest existing matching snapshot index. ``` ```{config:option} snapshots.schedule storage-alletra-volume-conf :condition: "custom volume" :defaultdesc: "same as `snapshots.schedule`" :scope: "global" :shortdesc: "Schedule for automatic volume snapshots" :type: "string" Specify either a cron expression (` `), a comma-separated list of schedule aliases (`@hourly`, `@daily`, `@midnight`, `@weekly`, `@monthly`, `@annually`, `@yearly`), or leave empty to disable automatic snapshots (the default). ``` ```{config:option} user.* storage-alletra-volume-conf :scope: "global" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} volatile.devlxd.owner storage-alletra-volume-conf :defaultdesc: "DevLXD owner identity ID" :scope: "global" :shortdesc: "ID of the DevLXD identity that owns the volume" :type: "string" ``` ```{config:option} volatile.idmap.last storage-alletra-volume-conf :condition: "filesystem" :shortdesc: "JSON-serialized UID/GID map that has been applied to the volume" :type: "string" ``` ```{config:option} volatile.idmap.next storage-alletra-volume-conf :condition: "filesystem" :shortdesc: "JSON-serialized UID/GID map that has been applied to the volume" :type: "string" ``` ```{config:option} volatile.uuid storage-alletra-volume-conf :defaultdesc: "random UUID" :scope: "global" :shortdesc: "Volume UUID" :type: "string" ``` ```{config:option} btrfs.mount_options storage-btrfs-pool-conf :defaultdesc: "`user_subvol_rm_allowed`" :scope: "global" :shortdesc: "Mount options for block devices" :type: "string" ``` ```{config:option} rsync.bwlimit storage-btrfs-pool-conf :defaultdesc: "`0` (no limit)" :scope: "global" :shortdesc: "Upper limit on the socket I/O for `rsync`" :type: "string" When `rsync` must be used to transfer storage entities, this option specifies the upper limit to be placed on the socket I/O. ``` ```{config:option} rsync.compression storage-btrfs-pool-conf :defaultdesc: "`true`" :scope: "global" :shortdesc: "Whether to use compression while migrating storage pools" :type: "bool" ``` ```{config:option} size storage-btrfs-pool-conf :defaultdesc: "auto (20% of free disk space, >= 5 GiB and <= 30 GiB)" :scope: "local" :shortdesc: "Size of the storage pool (for loop-based pools)" :type: "string" When creating loop-based pools, specify the size in bytes ({ref}`suffixes ` are supported). You can increase the size to grow the storage pool. The default (`auto`) creates a storage pool that uses 20% of the free disk space, with a minimum of 5 GiB and a maximum of 30 GiB. ``` ```{config:option} source storage-btrfs-pool-conf :scope: "local" :shortdesc: "Path to an existing block device, loop file, or Btrfs subvolume" :type: "string" ``` ```{config:option} source.recover storage-btrfs-pool-conf :defaultdesc: "`false`" :scope: "local" :shortdesc: "Whether to recover an existing `source`" :type: "bool" Set this option to true to recover an existing source which was previously created by LXD. ``` ```{config:option} source.wipe storage-btrfs-pool-conf :defaultdesc: "`false`" :scope: "local" :shortdesc: "Whether to wipe the block device before creating the pool" :type: "bool" Set this option to `true` to wipe the block device specified in `source` prior to creating the storage pool. ``` ```{config:option} user.* storage-btrfs-pool-conf :scope: "global" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} security.shared storage-btrfs-volume-conf :condition: "virtual-machine or custom block volume" :defaultdesc: "same as `volume.security.shared` or `false`" :scope: "global" :shortdesc: "Enable volume sharing" :type: "bool" Enable this option to allow the volume to be shared across multiple instances despite the possibility of data loss. ``` ```{config:option} security.shifted storage-btrfs-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.security.shifted` or `false`" :scope: "global" :shortdesc: "Enable ID shifting overlay" :type: "bool" Enable this option to allow the volume to be attached to multiple isolated instances. ``` ```{config:option} security.unmapped storage-btrfs-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.security.unmapped` or `false`" :scope: "global" :shortdesc: "Disable ID mapping for the volume" :type: "bool" ``` ```{config:option} size storage-btrfs-volume-conf :condition: "appropriate driver" :defaultdesc: "same as `volume.size`" :scope: "global" :shortdesc: "Size/quota of the storage volume" :type: "string" ``` ```{config:option} snapshots.expiry storage-btrfs-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.snapshots.expiry`" :scope: "global" :shortdesc: "Time until snapshots are deleted" :type: "string" Specify an expression like `1M 2H 3d 4w 5m 6y`. ``` ```{config:option} snapshots.pattern storage-btrfs-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.snapshots.pattern` or `snap%d`" :scope: "global" :shortdesc: "Template for the snapshot name" :type: "string" You can specify a naming template for scheduled snapshots and unnamed snapshots. The `snapshots.pattern` option takes a Pongo2 template string to format the snapshot name. To add a time stamp to the snapshot name, use the Pongo2 context variable `creation_date`. Make sure to format the date in your template string to avoid forbidden characters in the snapshot name. For example, set `snapshots.pattern` to `{{ creation_date|date:'2006-01-02_15-04-05' }}` to name the snapshots after their time of creation, down to the precision of a second. Another way to avoid name collisions is to use the placeholder `%d` in the pattern. If no matching snapshots exist, the placeholder is replaced with `0`. Otherwise, it is replaced with the next snapshot index, which is one higher than the highest existing matching snapshot index. ``` ```{config:option} snapshots.schedule storage-btrfs-volume-conf :condition: "custom volume" :defaultdesc: "same as `snapshots.schedule`" :scope: "global" :shortdesc: "Schedule for automatic volume snapshots" :type: "string" Specify either a cron expression (` `), a comma-separated list of schedule aliases (`@hourly`, `@daily`, `@midnight`, `@weekly`, `@monthly`, `@annually`, `@yearly`), or leave empty to disable automatic snapshots (the default). ``` ```{config:option} user.* storage-btrfs-volume-conf :scope: "global" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} volatile.devlxd.owner storage-btrfs-volume-conf :defaultdesc: "DevLXD owner identity ID" :scope: "global" :shortdesc: "ID of the DevLXD identity that owns the volume" :type: "string" ``` ```{config:option} volatile.idmap.last storage-btrfs-volume-conf :condition: "filesystem" :shortdesc: "JSON-serialized UID/GID map that has been applied to the volume" :type: "string" ``` ```{config:option} volatile.idmap.next storage-btrfs-volume-conf :condition: "filesystem" :shortdesc: "JSON-serialized UID/GID map that has been applied to the volume" :type: "string" ``` ```{config:option} volatile.uuid storage-btrfs-volume-conf :defaultdesc: "random UUID" :scope: "global" :shortdesc: "Volume UUID" :type: "string" ``` ```{config:option} ceph.cluster_name storage-ceph-pool-conf :defaultdesc: "`ceph`" :scope: "global" :shortdesc: "Name of the Ceph cluster in which to create new storage pools" :type: "string" ``` ```{config:option} ceph.osd.data_pool_name storage-ceph-pool-conf :scope: "global" :shortdesc: "Name of the OSD data pool" :type: "string" ``` ```{config:option} ceph.osd.pg_num storage-ceph-pool-conf :defaultdesc: "`32`" :scope: "global" :shortdesc: "Number of placement groups for the OSD storage pool" :type: "string" ``` ```{config:option} ceph.osd.pool_name storage-ceph-pool-conf :defaultdesc: "name of the pool" :scope: "global" :shortdesc: "Name of the OSD storage pool" :type: "string" This option specifies the name of the OSD storage pool. The OSD storage pool gets created if missing. ``` ```{config:option} ceph.osd.pool_size storage-ceph-pool-conf :defaultdesc: "`3`" :shortdesc: "Number of RADOS object replicas. Set to 1 for no replication." :type: "string" This option specifies the name for the file metadata OSD pool that should be used when creating a file system automatically. ``` ```{config:option} ceph.rbd.clone_copy storage-ceph-pool-conf :defaultdesc: "`true`" :scope: "global" :shortdesc: "Whether to use RBD lightweight clones" :type: "bool" Enable this option to use RBD lightweight clones rather than full dataset copies. ``` ```{config:option} ceph.rbd.du storage-ceph-pool-conf :defaultdesc: "`true`" :scope: "global" :shortdesc: "Whether to use RBD `du`" :type: "bool" This option specifies whether to use RBD `du` to obtain disk usage data for stopped instances. ``` ```{config:option} ceph.rbd.features storage-ceph-pool-conf :defaultdesc: "Default features defined in Ceph cluster" :scope: "global" :shortdesc: "Comma-separated list of RBD features to enable on the volumes" :type: "string" ``` ```{config:option} ceph.replicator. storage-ceph-pool-conf :scope: "global" :shortdesc: "Name of the peer Ceph site to mirror a project to" :type: "string" This option specifies the peer site, as registered in Ceph, that the volumes of the given project are mirrored to. One OSD pool can back several projects, each replicating to a different peer. `ceph.rbd.clone_copy` must also be set to `false` before any container is created on the pool, because Ceph cannot mirror a volume that is a clone of its image. The project must exist before this option is set. Deleting the project with `--force` removes the option from the pool. ``` ```{config:option} ceph.user.name storage-ceph-pool-conf :defaultdesc: "`admin`" :scope: "global" :shortdesc: "The Ceph user to use when creating storage pools and volumes" :type: "string" ``` ```{config:option} rsync.bwlimit storage-ceph-pool-conf :defaultdesc: "`0` (no limit)" :scope: "global" :shortdesc: "Upper limit on the socket I/O for `rsync`" :type: "string" When `rsync` must be used to transfer storage entities, this option specifies the upper limit to be placed on the socket I/O. ``` ```{config:option} rsync.compression storage-ceph-pool-conf :defaultdesc: "`true`" :scope: "global" :shortdesc: "Whether to use compression while migrating storage pools" :type: "bool" ``` ```{config:option} source.recover storage-ceph-pool-conf :defaultdesc: "`false`" :scope: "local" :shortdesc: "Whether to recover an existing `source`" :type: "bool" Set this option to true to recover an existing source which was previously created by LXD. ``` ```{config:option} user.* storage-ceph-pool-conf :scope: "global" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} volatile.pool.pristine storage-ceph-pool-conf :defaultdesc: "`true`" :scope: "global" :shortdesc: "Whether the pool was empty on creation time" :type: "string" ``` ```{config:option} block.filesystem storage-ceph-volume-conf :condition: "block-based volume with content type `filesystem`" :defaultdesc: "same as `volume.block.filesystem`" :scope: "global" :shortdesc: "File system of the storage volume" :type: "string" Valid options: `btrfs`, `ext4`, `xfs` If not set, `ext4` is assumed. ``` ```{config:option} block.mount_options storage-ceph-volume-conf :condition: "block-based volume with content type `filesystem`" :defaultdesc: "same as `volume.block.mount_options`" :scope: "global" :shortdesc: "Mount options for block-backed file system volumes" :type: "string" ``` ```{config:option} security.shared storage-ceph-volume-conf :condition: "virtual-machine or custom block volume" :defaultdesc: "same as `volume.security.shared` or `false`" :scope: "global" :shortdesc: "Enable volume sharing" :type: "bool" Enable this option to allow the volume to be shared across multiple instances despite the possibility of data loss. ``` ```{config:option} security.shifted storage-ceph-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.security.shifted` or `false`" :scope: "global" :shortdesc: "Enable ID shifting overlay" :type: "bool" Enable this option to allow the volume to be attached to multiple isolated instances. ``` ```{config:option} security.unmapped storage-ceph-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.security.unmapped` or `false`" :scope: "global" :shortdesc: "Disable ID mapping for the volume" :type: "bool" ``` ```{config:option} size storage-ceph-volume-conf :condition: "appropriate driver" :defaultdesc: "same as `volume.size`" :scope: "global" :shortdesc: "Size/quota of the storage volume" :type: "string" ``` ```{config:option} snapshots.expiry storage-ceph-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.snapshots.expiry`" :scope: "global" :shortdesc: "Time until snapshots are deleted" :type: "string" Specify an expression like `1M 2H 3d 4w 5m 6y`. ``` ```{config:option} snapshots.pattern storage-ceph-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.snapshots.pattern` or `snap%d`" :scope: "global" :shortdesc: "Template for the snapshot name" :type: "string" You can specify a naming template for scheduled snapshots and unnamed snapshots. The `snapshots.pattern` option takes a Pongo2 template string to format the snapshot name. To add a time stamp to the snapshot name, use the Pongo2 context variable `creation_date`. Make sure to format the date in your template string to avoid forbidden characters in the snapshot name. For example, set `snapshots.pattern` to `{{ creation_date|date:'2006-01-02_15-04-05' }}` to name the snapshots after their time of creation, down to the precision of a second. Another way to avoid name collisions is to use the placeholder `%d` in the pattern. If no matching snapshots exist, the placeholder is replaced with `0`. Otherwise, it is replaced with the next snapshot index, which is one higher than the highest existing matching snapshot index. ``` ```{config:option} snapshots.schedule storage-ceph-volume-conf :condition: "custom volume" :defaultdesc: "same as `snapshots.schedule`" :scope: "global" :shortdesc: "Schedule for automatic volume snapshots" :type: "string" Specify either a cron expression (` `), a comma-separated list of schedule aliases (`@hourly`, `@daily`, `@midnight`, `@weekly`, `@monthly`, `@annually`, `@yearly`), or leave empty to disable automatic snapshots (the default). ``` ```{config:option} user.* storage-ceph-volume-conf :scope: "global" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} volatile.devlxd.owner storage-ceph-volume-conf :defaultdesc: "DevLXD owner identity ID" :scope: "global" :shortdesc: "ID of the DevLXD identity that owns the volume" :type: "string" ``` ```{config:option} volatile.idmap.last storage-ceph-volume-conf :condition: "filesystem" :shortdesc: "JSON-serialized UID/GID map that has been applied to the volume" :type: "string" ``` ```{config:option} volatile.idmap.next storage-ceph-volume-conf :condition: "filesystem" :shortdesc: "JSON-serialized UID/GID map that has been applied to the volume" :type: "string" ``` ```{config:option} volatile.uuid storage-ceph-volume-conf :defaultdesc: "random UUID" :scope: "global" :shortdesc: "Volume UUID" :type: "string" ``` ```{config:option} cephfs.cluster_name storage-cephfs-pool-conf :defaultdesc: "`ceph`" :scope: "global" :shortdesc: "Name of the Ceph cluster that contains the CephFS file system" :type: "string" ``` ```{config:option} cephfs.create_missing storage-cephfs-pool-conf :defaultdesc: "`false`" :scope: "global" :shortdesc: "Automatically create the CephFS file system" :type: "bool" Use this option if the CephFS file system does not exist yet. LXD will then automatically create the file system and the missing data and metadata OSD pools. ``` ```{config:option} cephfs.data_pool storage-cephfs-pool-conf :scope: "global" :shortdesc: "Data OSD pool name" :type: "string" This option specifies the name for the data OSD pool that should be used when creating a file system automatically. ``` ```{config:option} cephfs.fscache storage-cephfs-pool-conf :defaultdesc: "`false`" :scope: "global" :shortdesc: "Enable use of kernel `fscache` and `cachefilesd`" :type: "bool" ``` ```{config:option} cephfs.meta_pool storage-cephfs-pool-conf :scope: "global" :shortdesc: "Metadata OSD pool name" :type: "string" This option specifies the name for the file metadata OSD pool that should be used when creating a file system automatically. ``` ```{config:option} cephfs.osd_pg_num storage-cephfs-pool-conf :scope: "global" :shortdesc: "Number of placement groups when creating missing OSD pools" :type: "string" This option specifies the number of OSD pool placement groups (`pg_num`) to use when creating a missing OSD pool. ``` ```{config:option} cephfs.osd_pool_size storage-cephfs-pool-conf :defaultdesc: "`3`" :shortdesc: "Number of RADOS object replicas. Set to 1 for no replication." :type: "string" This option specifies the number of OSD pool replicas to use when creating an OSD pool. ``` ```{config:option} cephfs.path storage-cephfs-pool-conf :defaultdesc: "`/`" :scope: "global" :shortdesc: "The base path for the CephFS mount" :type: "string" This option specifies the base path for the CephFS mount. The path gets created if missing. ``` ```{config:option} cephfs.user.name storage-cephfs-pool-conf :defaultdesc: "`admin`" :scope: "global" :shortdesc: "The Ceph user to use" :type: "string" ``` ```{config:option} rsync.bwlimit storage-cephfs-pool-conf :defaultdesc: "`0` (no limit)" :scope: "global" :shortdesc: "Upper limit on the socket I/O for `rsync`" :type: "string" When `rsync` must be used to transfer storage entities, this option specifies the upper limit to be placed on the socket I/O. ``` ```{config:option} rsync.compression storage-cephfs-pool-conf :defaultdesc: "`true`" :scope: "global" :shortdesc: "Whether to use compression while migrating storage pools" :type: "bool" ``` ```{config:option} source.recover storage-cephfs-pool-conf :defaultdesc: "`false`" :scope: "local" :shortdesc: "Whether to recover an existing `source`" :type: "bool" Set this option to true to recover an existing source which was previously created by LXD. ``` ```{config:option} user.* storage-cephfs-pool-conf :scope: "global" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} security.shifted storage-cephfs-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.security.shifted` or `false`" :scope: "global" :shortdesc: "Enable ID shifting overlay" :type: "bool" Enable this option to allow the volume to be attached to multiple isolated instances. ``` ```{config:option} security.unmapped storage-cephfs-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.security.unmapped` or `false`" :scope: "global" :shortdesc: "Disable ID mapping for the volume" :type: "bool" ``` ```{config:option} size storage-cephfs-volume-conf :condition: "appropriate driver" :defaultdesc: "same as `volume.size`" :scope: "global" :shortdesc: "Size/quota of the storage volume" :type: "string" ``` ```{config:option} snapshots.expiry storage-cephfs-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.snapshots.expiry`" :scope: "global" :shortdesc: "Time until snapshots are deleted" :type: "string" Specify an expression like `1M 2H 3d 4w 5m 6y`. ``` ```{config:option} snapshots.pattern storage-cephfs-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.snapshots.pattern` or `snap%d`" :scope: "global" :shortdesc: "Template for the snapshot name" :type: "string" You can specify a naming template for scheduled snapshots and unnamed snapshots. The `snapshots.pattern` option takes a Pongo2 template string to format the snapshot name. To add a time stamp to the snapshot name, use the Pongo2 context variable `creation_date`. Make sure to format the date in your template string to avoid forbidden characters in the snapshot name. For example, set `snapshots.pattern` to `{{ creation_date|date:'2006-01-02_15-04-05' }}` to name the snapshots after their time of creation, down to the precision of a second. Another way to avoid name collisions is to use the placeholder `%d` in the pattern. If no matching snapshots exist, the placeholder is replaced with `0`. Otherwise, it is replaced with the next snapshot index, which is one higher than the highest existing matching snapshot index. ``` ```{config:option} snapshots.schedule storage-cephfs-volume-conf :condition: "custom volume" :defaultdesc: "same as `snapshots.schedule`" :scope: "global" :shortdesc: "Schedule for automatic volume snapshots" :type: "string" Specify either a cron expression (` `), a comma-separated list of schedule aliases (`@hourly`, `@daily`, `@midnight`, `@weekly`, `@monthly`, `@annually`, `@yearly`), or leave empty to disable automatic snapshots (the default). ``` ```{config:option} user.* storage-cephfs-volume-conf :scope: "global" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} volatile.devlxd.owner storage-cephfs-volume-conf :defaultdesc: "DevLXD owner identity ID" :scope: "global" :shortdesc: "ID of the DevLXD identity that owns the volume" :type: "string" ``` ```{config:option} volatile.idmap.last storage-cephfs-volume-conf :condition: "filesystem" :shortdesc: "JSON-serialized UID/GID map that has been applied to the volume" :type: "string" ``` ```{config:option} volatile.idmap.next storage-cephfs-volume-conf :condition: "filesystem" :shortdesc: "JSON-serialized UID/GID map that has been applied to the volume" :type: "string" ``` ```{config:option} volatile.uuid storage-cephfs-volume-conf :defaultdesc: "random UUID" :scope: "global" :shortdesc: "Volume UUID" :type: "string" ``` ```{config:option} size storage-cephobject-bucket-conf :scope: "local" :shortdesc: "Quota of the storage bucket" :type: "string" ``` ```{config:option} user.* storage-cephobject-bucket-conf :scope: "global" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} cephobject.bucket.name_prefix storage-cephobject-pool-conf :scope: "global" :shortdesc: "Prefix to add to bucket names in Ceph" :type: "string" ``` ```{config:option} cephobject.cluster_name storage-cephobject-pool-conf :scope: "global" :shortdesc: "The Ceph cluster to use" :type: "string" ``` ```{config:option} cephobject.radosgw.endpoint storage-cephobject-pool-conf :scope: "global" :shortdesc: "URL of the `radosgw` gateway process" :type: "string" ``` ```{config:option} cephobject.radosgw.endpoint_cert_file storage-cephobject-pool-conf :scope: "global" :shortdesc: "TLS client certificate to use for endpoint communication" :type: "string" Specify the path to the file that contains the TLS client certificate. ``` ```{config:option} cephobject.user.name storage-cephobject-pool-conf :defaultdesc: "`admin`" :scope: "global" :shortdesc: "The Ceph user to use" :type: "string" ``` ```{config:option} source.recover storage-cephobject-pool-conf :defaultdesc: "`false`" :scope: "local" :shortdesc: "Whether to recover an existing `source`" :type: "bool" Set this option to true to recover an existing source which was previously created by LXD. ``` ```{config:option} user.* storage-cephobject-pool-conf :scope: "global" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} volatile.pool.pristine storage-cephobject-pool-conf :defaultdesc: "`true`" :scope: "global" :shortdesc: "Whether the `radosgw` `lxd-admin` user existed at creation time" :type: "string" ``` ```{config:option} rsync.bwlimit storage-dir-pool-conf :defaultdesc: "`0` (no limit)" :scope: "global" :shortdesc: "Upper limit on the socket I/O for `rsync`" :type: "string" When `rsync` must be used to transfer storage entities, this option specifies the upper limit to be placed on the socket I/O. ``` ```{config:option} rsync.compression storage-dir-pool-conf :defaultdesc: "`true`" :scope: "global" :shortdesc: "Whether to use compression while migrating storage pools" :type: "bool" ``` ```{config:option} source storage-dir-pool-conf :scope: "local" :shortdesc: "Path to an existing directory" :type: "string" ``` ```{config:option} source.recover storage-dir-pool-conf :defaultdesc: "`false`" :scope: "local" :shortdesc: "Whether to recover an existing `source`" :type: "bool" Set this option to true to recover an existing source which was previously created by LXD. ``` ```{config:option} user.* storage-dir-pool-conf :scope: "global" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} security.shared storage-dir-volume-conf :condition: "virtual-machine or custom block volume" :defaultdesc: "same as `volume.security.shared` or `false`" :scope: "global" :shortdesc: "Enable volume sharing" :type: "bool" Enable this option to allow the volume to be shared across multiple instances despite the possibility of data loss. ``` ```{config:option} security.shifted storage-dir-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.security.shifted` or `false`" :scope: "global" :shortdesc: "Enable ID shifting overlay" :type: "bool" Enable this option to allow the volume to be attached to multiple isolated instances. ``` ```{config:option} security.unmapped storage-dir-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.security.unmapped` or `false`" :scope: "global" :shortdesc: "Disable ID mapping for the volume" :type: "bool" ``` ```{config:option} size storage-dir-volume-conf :condition: "appropriate driver" :defaultdesc: "same as `volume.size`" :scope: "global" :shortdesc: "Size/quota of the storage volume" :type: "string" ``` ```{config:option} snapshots.expiry storage-dir-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.snapshots.expiry`" :scope: "global" :shortdesc: "Time until snapshots are deleted" :type: "string" Specify an expression like `1M 2H 3d 4w 5m 6y`. ``` ```{config:option} snapshots.pattern storage-dir-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.snapshots.pattern` or `snap%d`" :scope: "global" :shortdesc: "Template for the snapshot name" :type: "string" You can specify a naming template for scheduled snapshots and unnamed snapshots. The `snapshots.pattern` option takes a Pongo2 template string to format the snapshot name. To add a time stamp to the snapshot name, use the Pongo2 context variable `creation_date`. Make sure to format the date in your template string to avoid forbidden characters in the snapshot name. For example, set `snapshots.pattern` to `{{ creation_date|date:'2006-01-02_15-04-05' }}` to name the snapshots after their time of creation, down to the precision of a second. Another way to avoid name collisions is to use the placeholder `%d` in the pattern. If no matching snapshots exist, the placeholder is replaced with `0`. Otherwise, it is replaced with the next snapshot index, which is one higher than the highest existing matching snapshot index. ``` ```{config:option} snapshots.schedule storage-dir-volume-conf :condition: "custom volume" :defaultdesc: "same as `snapshots.schedule`" :scope: "global" :shortdesc: "Schedule for automatic volume snapshots" :type: "string" Specify either a cron expression (` `), a comma-separated list of schedule aliases (`@hourly`, `@daily`, `@midnight`, `@weekly`, `@monthly`, `@annually`, `@yearly`), or leave empty to disable automatic snapshots (the default). ``` ```{config:option} user.* storage-dir-volume-conf :scope: "global" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} volatile.devlxd.owner storage-dir-volume-conf :defaultdesc: "DevLXD owner identity ID" :scope: "global" :shortdesc: "ID of the DevLXD identity that owns the volume" :type: "string" ``` ```{config:option} volatile.idmap.last storage-dir-volume-conf :condition: "filesystem" :shortdesc: "JSON-serialized UID/GID map that has been applied to the volume" :type: "string" ``` ```{config:option} volatile.idmap.next storage-dir-volume-conf :condition: "filesystem" :shortdesc: "JSON-serialized UID/GID map that has been applied to the volume" :type: "string" ``` ```{config:option} volatile.uuid storage-dir-volume-conf :defaultdesc: "random UUID" :scope: "global" :shortdesc: "Volume UUID" :type: "string" ``` ```{config:option} lvm.thinpool_metadata_size storage-lvm-pool-conf :defaultdesc: "`0` (auto)" :scope: "global" :shortdesc: "The size of the thin pool metadata volume" :type: "string" By default, LVM calculates an appropriate size. ``` ```{config:option} lvm.thinpool_name storage-lvm-pool-conf :defaultdesc: "`LXDThinPool`" :scope: "local" :shortdesc: "Thin pool where volumes are created" :type: "string" ``` ```{config:option} lvm.use_thinpool storage-lvm-pool-conf :defaultdesc: "`true`" :scope: "global" :shortdesc: "Whether the storage pool uses a thin pool for logical volumes" :type: "bool" ``` ```{config:option} lvm.vg.force_reuse storage-lvm-pool-conf :defaultdesc: "`false`" :scope: "global" :shortdesc: "Force using an existing non-empty volume group" :type: "bool" ``` ```{config:option} lvm.vg_name storage-lvm-pool-conf :defaultdesc: "name of the pool" :scope: "local" :shortdesc: "Name of the volume group to create" :type: "string" ``` ```{config:option} rsync.bwlimit storage-lvm-pool-conf :defaultdesc: "`0` (no limit)" :scope: "global" :shortdesc: "Upper limit on the socket I/O for `rsync`" :type: "string" When `rsync` must be used to transfer storage entities, this option specifies the upper limit to be placed on the socket I/O. ``` ```{config:option} rsync.compression storage-lvm-pool-conf :defaultdesc: "`true`" :scope: "global" :shortdesc: "Whether to use compression while migrating storage pools" :type: "bool" ``` ```{config:option} size storage-lvm-pool-conf :defaultdesc: "auto (20% of free disk space, >= 5 GiB and <= 30 GiB)" :scope: "local" :shortdesc: "Size of the storage pool (for loop-based pools)" :type: "string" When creating loop-based pools, specify the size in bytes ({ref}`suffixes ` are supported). You can increase the size to grow the storage pool. The default (`auto`) creates a storage pool that uses 20% of the free disk space, with a minimum of 5 GiB and a maximum of 30 GiB. ``` ```{config:option} source storage-lvm-pool-conf :scope: "local" :shortdesc: "Path to an existing block device, loop file, or LVM volume group" :type: "string" ``` ```{config:option} source.recover storage-lvm-pool-conf :defaultdesc: "`false`" :scope: "local" :shortdesc: "Whether to recover an existing `source`" :type: "bool" Set this option to true to recover an existing source which was previously created by LXD. ``` ```{config:option} source.wipe storage-lvm-pool-conf :defaultdesc: "`false`" :scope: "local" :shortdesc: "Whether to wipe the block device before creating the pool" :type: "bool" Set this option to `true` to wipe the block device specified in `source` prior to creating the storage pool. ``` ```{config:option} user.* storage-lvm-pool-conf :scope: "global" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} block.filesystem storage-lvm-volume-conf :condition: "block-based volume with content type `filesystem`" :defaultdesc: "same as `volume.block.filesystem`" :scope: "global" :shortdesc: "File system of the storage volume" :type: "string" Valid options: `btrfs`, `ext4`, `xfs` If not set, `ext4` is assumed. ``` ```{config:option} block.mount_options storage-lvm-volume-conf :condition: "block-based volume with content type `filesystem`" :defaultdesc: "same as `volume.block.mount_options`" :scope: "global" :shortdesc: "Mount options for block-backed file system volumes" :type: "string" ``` ```{config:option} lvm.stripes storage-lvm-volume-conf :defaultdesc: "same as `volume.lvm.stripes`" :scope: "global" :shortdesc: "Number of stripes to use for new volumes (or thin pool volume)" :type: "string" ``` ```{config:option} lvm.stripes.size storage-lvm-volume-conf :defaultdesc: "same as `volume.lvm.stripes.size`" :scope: "global" :shortdesc: "Size of stripes to use" :type: "string" The size must be at least 4096 bytes, and a multiple of 512 bytes. ``` ```{config:option} security.shared storage-lvm-volume-conf :condition: "virtual-machine or custom block volume" :defaultdesc: "same as `volume.security.shared` or `false`" :scope: "global" :shortdesc: "Enable volume sharing" :type: "bool" Enable this option to allow the volume to be shared across multiple instances despite the possibility of data loss. ``` ```{config:option} security.shifted storage-lvm-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.security.shifted` or `false`" :scope: "global" :shortdesc: "Enable ID shifting overlay" :type: "bool" Enable this option to allow the volume to be attached to multiple isolated instances. ``` ```{config:option} security.unmapped storage-lvm-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.security.unmapped` or `false`" :scope: "global" :shortdesc: "Disable ID mapping for the volume" :type: "bool" ``` ```{config:option} size storage-lvm-volume-conf :condition: "appropriate driver" :defaultdesc: "same as `volume.size`" :scope: "global" :shortdesc: "Size/quota of the storage volume" :type: "string" ``` ```{config:option} snapshots.expiry storage-lvm-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.snapshots.expiry`" :scope: "global" :shortdesc: "Time until snapshots are deleted" :type: "string" Specify an expression like `1M 2H 3d 4w 5m 6y`. ``` ```{config:option} snapshots.pattern storage-lvm-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.snapshots.pattern` or `snap%d`" :scope: "global" :shortdesc: "Template for the snapshot name" :type: "string" You can specify a naming template for scheduled snapshots and unnamed snapshots. The `snapshots.pattern` option takes a Pongo2 template string to format the snapshot name. To add a time stamp to the snapshot name, use the Pongo2 context variable `creation_date`. Make sure to format the date in your template string to avoid forbidden characters in the snapshot name. For example, set `snapshots.pattern` to `{{ creation_date|date:'2006-01-02_15-04-05' }}` to name the snapshots after their time of creation, down to the precision of a second. Another way to avoid name collisions is to use the placeholder `%d` in the pattern. If no matching snapshots exist, the placeholder is replaced with `0`. Otherwise, it is replaced with the next snapshot index, which is one higher than the highest existing matching snapshot index. ``` ```{config:option} snapshots.schedule storage-lvm-volume-conf :condition: "custom volume" :defaultdesc: "same as `snapshots.schedule`" :scope: "global" :shortdesc: "Schedule for automatic volume snapshots" :type: "string" Specify either a cron expression (` `), a comma-separated list of schedule aliases (`@hourly`, `@daily`, `@midnight`, `@weekly`, `@monthly`, `@annually`, `@yearly`), or leave empty to disable automatic snapshots (the default). ``` ```{config:option} user.* storage-lvm-volume-conf :scope: "global" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} volatile.devlxd.owner storage-lvm-volume-conf :defaultdesc: "DevLXD owner identity ID" :scope: "global" :shortdesc: "ID of the DevLXD identity that owns the volume" :type: "string" ``` ```{config:option} volatile.idmap.last storage-lvm-volume-conf :condition: "filesystem" :shortdesc: "JSON-serialized UID/GID map that has been applied to the volume" :type: "string" ``` ```{config:option} volatile.idmap.next storage-lvm-volume-conf :condition: "filesystem" :shortdesc: "JSON-serialized UID/GID map that has been applied to the volume" :type: "string" ``` ```{config:option} volatile.uuid storage-lvm-volume-conf :defaultdesc: "random UUID" :scope: "global" :shortdesc: "Volume UUID" :type: "string" ``` ```{config:option} powerflex.domain storage-powerflex-pool-conf :scope: "global" :shortdesc: "Name of the PowerFlex protection domain" :type: "string" This option is required only if {config:option}`storage-powerflex-pool-conf:powerflex.pool` is specified using its name. ``` ```{config:option} powerflex.gateway storage-powerflex-pool-conf :scope: "global" :shortdesc: "Address of the PowerFlex Gateway" :type: "string" ``` ```{config:option} powerflex.gateway.verify storage-powerflex-pool-conf :defaultdesc: "`true`" :scope: "global" :shortdesc: "Whether to verify the PowerFlex Gateway's certificate" :type: "bool" ``` ```{config:option} powerflex.mode storage-powerflex-pool-conf :defaultdesc: "`nvme/tcp`" :scope: "global" :shortdesc: "How volumes are mapped to the local server" :type: "string" The mode gets discovered automatically if the system provides the necessary kernel modules. This can be either `nvme/tcp` or `sdc`. ``` ```{config:option} powerflex.pool storage-powerflex-pool-conf :scope: "global" :shortdesc: "ID of the PowerFlex storage pool" :type: "string" If you want to specify the storage pool via its name, also set {config:option}`storage-powerflex-pool-conf:powerflex.domain`. ``` ```{config:option} powerflex.sdt storage-powerflex-pool-conf :scope: "global" :shortdesc: "Comma separated list of PowerFlex NVMe/TCP SDTs" :type: "string" ``` ```{config:option} powerflex.snapshot_copy storage-powerflex-pool-conf :defaultdesc: "`false`" :scope: "global" :shortdesc: "Whether to use sparse snapshots for copies" :type: "bool" If this option is set to `true`, PowerFlex makes a sparse snapshot when copying an instance or custom volume. See {ref}`storage-powerflex-limitations` for more information. ``` ```{config:option} powerflex.user.name storage-powerflex-pool-conf :defaultdesc: "`admin`" :scope: "global" :shortdesc: "User for PowerFlex Gateway authentication" :type: "string" Must have at least SystemAdmin role to give LXD full control over managed storage pools. ``` ```{config:option} powerflex.user.password storage-powerflex-pool-conf :scope: "global" :shortdesc: "Password for PowerFlex Gateway authentication" :type: "string" ``` ```{config:option} rsync.bwlimit storage-powerflex-pool-conf :defaultdesc: "`0` (no limit)" :scope: "global" :shortdesc: "Upper limit on the socket I/O for `rsync`" :type: "string" When `rsync` must be used to transfer storage entities, this option specifies the upper limit to be placed on the socket I/O. ``` ```{config:option} rsync.compression storage-powerflex-pool-conf :defaultdesc: "`true`" :scope: "global" :shortdesc: "Whether to use compression while migrating storage pools" :type: "bool" ``` ```{config:option} source.recover storage-powerflex-pool-conf :defaultdesc: "`false`" :scope: "local" :shortdesc: "Whether to recover an existing `source`" :type: "bool" Set this option to true to recover an existing source which was previously created by LXD. ``` ```{config:option} user.* storage-powerflex-pool-conf :scope: "global" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} volatile.powerflex.version storage-powerflex-pool-conf :defaultdesc: "Discovered version" :scope: "global" :shortdesc: "Software version of the PowerFlex array." :type: "string" This field is automatically populated after querying the PowerFlex version. It cannot be set by the user. ``` ```{config:option} volume.size storage-powerflex-pool-conf :scope: "global" :shortdesc: "Size/quota of the storage volume" :type: "string" The size must be in multiples of 8 GiB for PowerFlex 4. Starting with PowerFlex 5, the size can be in multiples of 1 GiB. See {ref}`storage-powerflex-limitations` for more information. ``` ```{config:option} block.filesystem storage-powerflex-volume-conf :condition: "block-based volume with content type `filesystem`" :defaultdesc: "same as `volume.block.filesystem`" :scope: "global" :shortdesc: "File system of the storage volume" :type: "string" Valid options: `btrfs`, `ext4`, `xfs` If not set, `ext4` is assumed. ``` ```{config:option} block.mount_options storage-powerflex-volume-conf :condition: "block-based volume with content type `filesystem`" :defaultdesc: "same as `volume.block.mount_options`" :scope: "global" :shortdesc: "Mount options for block-backed file system volumes" :type: "string" ``` ```{config:option} block.type storage-powerflex-volume-conf :defaultdesc: "same as `volume.block.type` or `thick`" :scope: "global" :shortdesc: "Whether to create a `thin` or `thick` provisioned volume" :type: "string" ``` ```{config:option} security.shared storage-powerflex-volume-conf :condition: "virtual-machine or custom block volume" :defaultdesc: "same as `volume.security.shared` or `false`" :scope: "global" :shortdesc: "Enable volume sharing" :type: "bool" Enable this option to allow the volume to be shared across multiple instances despite the possibility of data loss. ``` ```{config:option} security.shifted storage-powerflex-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.security.shifted` or `false`" :scope: "global" :shortdesc: "Enable ID shifting overlay" :type: "bool" Enable this option to allow the volume to be attached to multiple isolated instances. ``` ```{config:option} security.unmapped storage-powerflex-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.security.unmapped` or `false`" :scope: "global" :shortdesc: "Disable ID mapping for the volume" :type: "bool" ``` ```{config:option} size storage-powerflex-volume-conf :defaultdesc: "same as `volume.size`" :scope: "global" :shortdesc: "Size/quota of the storage volume" :type: "string" The size must be in multiples of 8 GiB for PowerFlex 4. Starting with PowerFlex 5, the size can be in multiples of 1 GiB. See {ref}`storage-powerflex-limitations` for more information. ``` ```{config:option} snapshots.expiry storage-powerflex-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.snapshots.expiry`" :scope: "global" :shortdesc: "Time until snapshots are deleted" :type: "string" Specify an expression like `1M 2H 3d 4w 5m 6y`. ``` ```{config:option} snapshots.pattern storage-powerflex-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.snapshots.pattern` or `snap%d`" :scope: "global" :shortdesc: "Template for the snapshot name" :type: "string" You can specify a naming template for scheduled snapshots and unnamed snapshots. The `snapshots.pattern` option takes a Pongo2 template string to format the snapshot name. To add a time stamp to the snapshot name, use the Pongo2 context variable `creation_date`. Make sure to format the date in your template string to avoid forbidden characters in the snapshot name. For example, set `snapshots.pattern` to `{{ creation_date|date:'2006-01-02_15-04-05' }}` to name the snapshots after their time of creation, down to the precision of a second. Another way to avoid name collisions is to use the placeholder `%d` in the pattern. If no matching snapshots exist, the placeholder is replaced with `0`. Otherwise, it is replaced with the next snapshot index, which is one higher than the highest existing matching snapshot index. ``` ```{config:option} snapshots.schedule storage-powerflex-volume-conf :condition: "custom volume" :defaultdesc: "same as `snapshots.schedule`" :scope: "global" :shortdesc: "Schedule for automatic volume snapshots" :type: "string" Specify either a cron expression (` `), a comma-separated list of schedule aliases (`@hourly`, `@daily`, `@midnight`, `@weekly`, `@monthly`, `@annually`, `@yearly`), or leave empty to disable automatic snapshots (the default). ``` ```{config:option} user.* storage-powerflex-volume-conf :scope: "global" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} volatile.devlxd.owner storage-powerflex-volume-conf :defaultdesc: "DevLXD owner identity ID" :scope: "global" :shortdesc: "ID of the DevLXD identity that owns the volume" :type: "string" ``` ```{config:option} volatile.idmap.last storage-powerflex-volume-conf :condition: "filesystem" :shortdesc: "JSON-serialized UID/GID map that has been applied to the volume" :type: "string" ``` ```{config:option} volatile.idmap.next storage-powerflex-volume-conf :condition: "filesystem" :shortdesc: "JSON-serialized UID/GID map that has been applied to the volume" :type: "string" ``` ```{config:option} volatile.uuid storage-powerflex-volume-conf :defaultdesc: "random UUID" :scope: "global" :shortdesc: "Volume UUID" :type: "string" ``` ```{config:option} powerstore.gateway storage-powerstore-pool-conf :scope: "global" :shortdesc: "Address of the PowerStore Gateway" :type: "string" ``` ```{config:option} powerstore.gateway.verify storage-powerstore-pool-conf :defaultdesc: "`true`" :scope: "global" :shortdesc: "Whether to verify the PowerStore Gateway's certificate" :type: "bool" ``` ```{config:option} powerstore.mode storage-powerstore-pool-conf :defaultdesc: "`nvme/tcp`" :scope: "global" :shortdesc: "How volumes are mapped to the local server" :type: "string" The mode to use to map PowerStore volumes to the local server. Supported values are `iscsi`, `scsi/fc`, `nvme/tcp`, and `nvme/fc`. ``` ```{config:option} powerstore.target storage-powerstore-pool-conf :defaultdesc: "all available targets" :shortdesc: "List of target addresses the LXD connects to." :type: "string" A comma-separated list of target addresses. If empty, LXD discovers and connects to all available targets. Otherwise, it only connects to the specified addresses. ``` ```{config:option} powerstore.user.name storage-powerstore-pool-conf :defaultdesc: "`admin`" :scope: "global" :shortdesc: "User for PowerStore Gateway authentication" :type: "string" Name of the PowerStore user with an admin role that gives LXD full control over managed storage pools. ``` ```{config:option} powerstore.user.password storage-powerstore-pool-conf :scope: "global" :shortdesc: "Password for PowerStore Gateway authentication" :type: "string" ``` ```{config:option} rsync.bwlimit storage-powerstore-pool-conf :defaultdesc: "`0` (no limit)" :scope: "global" :shortdesc: "Upper limit on the socket I/O for `rsync`" :type: "string" When `rsync` must be used to transfer storage entities, this option specifies the upper limit to be placed on the socket I/O. ``` ```{config:option} rsync.compression storage-powerstore-pool-conf :defaultdesc: "`true`" :scope: "global" :shortdesc: "Whether to use compression while migrating storage pools" :type: "bool" ``` ```{config:option} source.recover storage-powerstore-pool-conf :defaultdesc: "`false`" :scope: "local" :shortdesc: "Whether to recover an existing `source`" :type: "bool" Set this option to true to recover an existing source which was previously created by LXD. ``` ```{config:option} user.* storage-powerstore-pool-conf :scope: "global" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} volume.size storage-powerstore-pool-conf :defaultdesc: "`10GiB`" :scope: "global" :shortdesc: "Size/quota of the storage volume" :type: "string" The size must be in multiples of 1 MiB. The minimum size is 1 MiB and maximum is 256 TiB. ``` ```{config:option} block.filesystem storage-powerstore-volume-conf :condition: "block-based volume with content type `filesystem`" :defaultdesc: "same as `volume.block.filesystem`" :scope: "global" :shortdesc: "File system of the storage volume" :type: "string" Valid options: `btrfs`, `ext4`, `xfs` If not set, `ext4` is assumed. ``` ```{config:option} block.mount_options storage-powerstore-volume-conf :condition: "block-based volume with content type `filesystem`" :defaultdesc: "same as `volume.block.mount_options`" :scope: "global" :shortdesc: "Mount options for block-backed file system volumes" :type: "string" ``` ```{config:option} security.shared storage-powerstore-volume-conf :condition: "virtual-machine or custom block volume" :defaultdesc: "same as `volume.security.shared` or `false`" :scope: "global" :shortdesc: "Enable volume sharing" :type: "bool" Enable this option to allow the volume to be shared across multiple instances despite the possibility of data loss. ``` ```{config:option} security.shifted storage-powerstore-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.security.shifted` or `false`" :scope: "global" :shortdesc: "Enable ID shifting overlay" :type: "bool" Enable this option to allow the volume to be attached to multiple isolated instances. ``` ```{config:option} security.unmapped storage-powerstore-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.security.unmapped` or `false`" :scope: "global" :shortdesc: "Disable ID mapping for the volume" :type: "bool" ``` ```{config:option} size storage-powerstore-volume-conf :defaultdesc: "same as `volume.size`" :scope: "global" :shortdesc: "Size/quota of the storage volume" :type: "string" The size must be in multiples of 1 MiB. The minimum size is 1 MiB and maximum is 256 TiB. ``` ```{config:option} snapshots.expiry storage-powerstore-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.snapshots.expiry`" :scope: "global" :shortdesc: "Time until snapshots are deleted" :type: "string" Specify an expression like `1M 2H 3d 4w 5m 6y`. ``` ```{config:option} snapshots.pattern storage-powerstore-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.snapshots.pattern` or `snap%d`" :scope: "global" :shortdesc: "Template for the snapshot name" :type: "string" You can specify a naming template for scheduled snapshots and unnamed snapshots. The `snapshots.pattern` option takes a Pongo2 template string to format the snapshot name. To add a time stamp to the snapshot name, use the Pongo2 context variable `creation_date`. Make sure to format the date in your template string to avoid forbidden characters in the snapshot name. For example, set `snapshots.pattern` to `{{ creation_date|date:'2006-01-02_15-04-05' }}` to name the snapshots after their time of creation, down to the precision of a second. Another way to avoid name collisions is to use the placeholder `%d` in the pattern. If no matching snapshots exist, the placeholder is replaced with `0`. Otherwise, it is replaced with the next snapshot index, which is one higher than the highest existing matching snapshot index. ``` ```{config:option} snapshots.schedule storage-powerstore-volume-conf :condition: "custom volume" :defaultdesc: "same as `snapshots.schedule`" :scope: "global" :shortdesc: "Schedule for automatic volume snapshots" :type: "string" Specify either a cron expression (` `), a comma-separated list of schedule aliases (`@hourly`, `@daily`, `@midnight`, `@weekly`, `@monthly`, `@annually`, `@yearly`), or leave empty to disable automatic snapshots (the default). ``` ```{config:option} user.* storage-powerstore-volume-conf :scope: "global" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} volatile.devlxd.owner storage-powerstore-volume-conf :defaultdesc: "DevLXD owner identity ID" :scope: "global" :shortdesc: "ID of the DevLXD identity that owns the volume" :type: "string" ``` ```{config:option} volatile.idmap.last storage-powerstore-volume-conf :condition: "filesystem" :shortdesc: "JSON-serialized UID/GID map that has been applied to the volume" :type: "string" ``` ```{config:option} volatile.idmap.next storage-powerstore-volume-conf :condition: "filesystem" :shortdesc: "JSON-serialized UID/GID map that has been applied to the volume" :type: "string" ``` ```{config:option} volatile.uuid storage-powerstore-volume-conf :defaultdesc: "random UUID" :scope: "global" :shortdesc: "Volume UUID" :type: "string" ``` ```{config:option} pure.api.token storage-pure-pool-conf :shortdesc: "API authorization token for the Everpure FlashArray" :type: "string" API authorization token for the Everpure FlashArray. Must have array_admin role to give LXD full control over managed storage pools (Everpure pods). ``` ```{config:option} pure.gateway storage-pure-pool-conf :shortdesc: "Management VIP address of the Everpure FlashArray" :type: "string" ``` ```{config:option} pure.gateway.verify storage-pure-pool-conf :defaultdesc: "`true`" :shortdesc: "Whether to verify the Everpure FlashArray's certificate" :type: "bool" ``` ```{config:option} pure.mode storage-pure-pool-conf :defaultdesc: "`nvme/tcp`" :shortdesc: "How volumes are mapped to the local server" :type: "string" The mode to use to map Everpure volumes to the local server. Supported values are `iscsi`, `nvme/tcp`, `nvme/fc`, and `scsi/fc`. ``` ```{config:option} pure.target storage-pure-pool-conf :defaultdesc: "all available targets" :shortdesc: "List of target addresses." :type: "string" A comma-separated list of target addresses. If empty, LXD discovers and connects to all available targets. Otherwise, it only connects to the specified addresses. This option has no effect in the Fibre Channel modes (`scsi/fc` and `nvme/fc`), because their targets are addressed by World Wide Name rather than by network address. ``` ```{config:option} rsync.bwlimit storage-pure-pool-conf :defaultdesc: "`0` (no limit)" :scope: "global" :shortdesc: "Upper limit on the socket I/O for `rsync`" :type: "string" When `rsync` must be used to transfer storage entities, this option specifies the upper limit to be placed on the socket I/O. ``` ```{config:option} rsync.compression storage-pure-pool-conf :defaultdesc: "`true`" :scope: "global" :shortdesc: "Whether to use compression while migrating storage pools" :type: "bool" ``` ```{config:option} size storage-pure-pool-conf :scope: "local" :shortdesc: "Size of the storage pool" :type: "string" Size in bytes LXD sets as the quota of the Everpure pod. ``` ```{config:option} source.recover storage-pure-pool-conf :defaultdesc: "`false`" :scope: "local" :shortdesc: "Whether to recover an existing `source`" :type: "bool" Set this option to true to recover an existing source which was previously created by LXD. ``` ```{config:option} user.* storage-pure-pool-conf :scope: "global" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} volume.size storage-pure-pool-conf :defaultdesc: "`10GiB`" :shortdesc: "Size/quota of the storage volume" :type: "string" Default Everpure volume size rounded to 512B. The minimum size is 1MiB. ``` ```{config:option} block.filesystem storage-pure-volume-conf :condition: "block-based volume with content type `filesystem`" :defaultdesc: "same as `volume.block.filesystem`" :shortdesc: "File system of the storage volume" :type: "string" Valid options: `btrfs`, `ext4`, `xfs` If not set, `ext4` is assumed. ``` ```{config:option} block.mount_options storage-pure-volume-conf :condition: "block-based volume with content type `filesystem`" :defaultdesc: "same as `volume.block.mount_options`" :shortdesc: "Mount options for block-backed file system volumes" :type: "string" ``` ```{config:option} security.shared storage-pure-volume-conf :condition: "virtual-machine or custom block volume" :defaultdesc: "same as `volume.security.shared` or `false`" :scope: "global" :shortdesc: "Enable volume sharing" :type: "bool" Enable this option to allow the volume to be shared across multiple instances despite the possibility of data loss. ``` ```{config:option} security.shifted storage-pure-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.security.shifted` or `false`" :scope: "global" :shortdesc: "Enable ID shifting overlay" :type: "bool" Enable this option to allow the volume to be attached to multiple isolated instances. ``` ```{config:option} security.unmapped storage-pure-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.security.unmapped` or `false`" :scope: "global" :shortdesc: "Disable ID mapping for the volume" :type: "bool" ``` ```{config:option} size storage-pure-volume-conf :defaultdesc: "same as `volume.size`" :shortdesc: "Size/quota of the storage volume" :type: "string" Default Everpure volume size rounded to 512B. The minimum size is 1MiB. ``` ```{config:option} snapshots.expiry storage-pure-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.snapshots.expiry`" :scope: "global" :shortdesc: "Time until snapshots are deleted" :type: "string" Specify an expression like `1M 2H 3d 4w 5m 6y`. ``` ```{config:option} snapshots.pattern storage-pure-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.snapshots.pattern` or `snap%d`" :scope: "global" :shortdesc: "Template for the snapshot name" :type: "string" You can specify a naming template for scheduled snapshots and unnamed snapshots. The `snapshots.pattern` option takes a Pongo2 template string to format the snapshot name. To add a time stamp to the snapshot name, use the Pongo2 context variable `creation_date`. Make sure to format the date in your template string to avoid forbidden characters in the snapshot name. For example, set `snapshots.pattern` to `{{ creation_date|date:'2006-01-02_15-04-05' }}` to name the snapshots after their time of creation, down to the precision of a second. Another way to avoid name collisions is to use the placeholder `%d` in the pattern. If no matching snapshots exist, the placeholder is replaced with `0`. Otherwise, it is replaced with the next snapshot index, which is one higher than the highest existing matching snapshot index. ``` ```{config:option} snapshots.schedule storage-pure-volume-conf :condition: "custom volume" :defaultdesc: "same as `snapshots.schedule`" :scope: "global" :shortdesc: "Schedule for automatic volume snapshots" :type: "string" Specify either a cron expression (` `), a comma-separated list of schedule aliases (`@hourly`, `@daily`, `@midnight`, `@weekly`, `@monthly`, `@annually`, `@yearly`), or leave empty to disable automatic snapshots (the default). ``` ```{config:option} user.* storage-pure-volume-conf :scope: "global" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} volatile.devlxd.owner storage-pure-volume-conf :defaultdesc: "DevLXD owner identity ID" :scope: "global" :shortdesc: "ID of the DevLXD identity that owns the volume" :type: "string" ``` ```{config:option} volatile.idmap.last storage-pure-volume-conf :condition: "filesystem" :shortdesc: "JSON-serialized UID/GID map that has been applied to the volume" :type: "string" ``` ```{config:option} volatile.idmap.next storage-pure-volume-conf :condition: "filesystem" :shortdesc: "JSON-serialized UID/GID map that has been applied to the volume" :type: "string" ``` ```{config:option} volatile.uuid storage-pure-volume-conf :defaultdesc: "random UUID" :scope: "global" :shortdesc: "Volume UUID" :type: "string" ``` ```{config:option} rsync.bwlimit storage-zfs-pool-conf :defaultdesc: "`0` (no limit)" :scope: "global" :shortdesc: "Upper limit on the socket I/O for `rsync`" :type: "string" When `rsync` must be used to transfer storage entities, this option specifies the upper limit to be placed on the socket I/O. ``` ```{config:option} rsync.compression storage-zfs-pool-conf :defaultdesc: "`true`" :scope: "global" :shortdesc: "Whether to use compression while migrating storage pools" :type: "bool" ``` ```{config:option} size storage-zfs-pool-conf :defaultdesc: "auto (20% of free disk space, >= 5 GiB and <= 30 GiB)" :scope: "local" :shortdesc: "Size of the storage pool (for loop-based pools)" :type: "string" When creating loop-based pools, specify the size in bytes ({ref}`suffixes ` are supported). You can increase the size to grow the storage pool. The default (`auto`) creates a storage pool that uses 20% of the free disk space, with a minimum of 5 GiB and a maximum of 30 GiB. ``` ```{config:option} source storage-zfs-pool-conf :scope: "local" :shortdesc: "Path to an existing block device, loop file, or ZFS dataset/pool" :type: "string" ``` ```{config:option} source.recover storage-zfs-pool-conf :defaultdesc: "`false`" :scope: "local" :shortdesc: "Whether to recover an existing `source`" :type: "bool" Set this option to true to recover an existing source which was previously created by LXD. ``` ```{config:option} source.wipe storage-zfs-pool-conf :defaultdesc: "`false`" :scope: "local" :shortdesc: "Whether to wipe the block device before creating the pool" :type: "bool" Set this option to `true` to wipe the block device specified in `source` prior to creating the storage pool. ``` ```{config:option} user.* storage-zfs-pool-conf :scope: "global" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} zfs.clone_copy storage-zfs-pool-conf :defaultdesc: "`true`" :scope: "global" :shortdesc: "Whether to use ZFS lightweight clones" :type: "string" Set this option to `true` or `false` to enable or disable using ZFS lightweight clones rather than full dataset copies. Set the option to `rebase` to copy based on the initial image. ``` ```{config:option} zfs.export storage-zfs-pool-conf :defaultdesc: "`true`" :scope: "global" :shortdesc: "Whether to export the zpool when an unmount is being performed" :type: "bool" ``` ```{config:option} zfs.pool_name storage-zfs-pool-conf :defaultdesc: "name of the pool" :scope: "local" :shortdesc: "Name of the zpool" :type: "string" ``` ```{config:option} block.filesystem storage-zfs-volume-conf :condition: "block-based volume with content type `filesystem` (`zfs.block_mode` enabled)" :defaultdesc: "same as `volume.block.filesystem`" :scope: "global" :shortdesc: "File system of the storage volume" :type: "string" Valid options: `btrfs`, `ext4`, `xfs` If not set, `ext4` is assumed. ``` ```{config:option} block.mount_options storage-zfs-volume-conf :condition: "block-based volume with content type `filesystem` (`zfs.block_mode` enabled)" :defaultdesc: "same as `volume.block.mount_options`" :scope: "global" :shortdesc: "Mount options for block-backed file system volumes" :type: "string" ``` ```{config:option} security.shared storage-zfs-volume-conf :condition: "virtual-machine or custom block volume" :defaultdesc: "same as `volume.security.shared` or `false`" :scope: "global" :shortdesc: "Enable volume sharing" :type: "bool" Enable this option to allow the volume to be shared across multiple instances despite the possibility of data loss. ``` ```{config:option} security.shifted storage-zfs-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.security.shifted` or `false`" :scope: "global" :shortdesc: "Enable ID shifting overlay" :type: "bool" Enable this option to allow the volume to be attached to multiple isolated instances. ``` ```{config:option} security.unmapped storage-zfs-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.security.unmapped` or `false`" :scope: "global" :shortdesc: "Disable ID mapping for the volume" :type: "bool" ``` ```{config:option} size storage-zfs-volume-conf :condition: "appropriate driver" :defaultdesc: "same as `volume.size`" :scope: "global" :shortdesc: "Size/quota of the storage volume" :type: "string" ``` ```{config:option} snapshots.expiry storage-zfs-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.snapshots.expiry`" :scope: "global" :shortdesc: "Time until snapshots are deleted" :type: "string" Specify an expression like `1M 2H 3d 4w 5m 6y`. ``` ```{config:option} snapshots.pattern storage-zfs-volume-conf :condition: "custom volume" :defaultdesc: "same as `volume.snapshots.pattern` or `snap%d`" :scope: "global" :shortdesc: "Template for the snapshot name" :type: "string" You can specify a naming template for scheduled snapshots and unnamed snapshots. The `snapshots.pattern` option takes a Pongo2 template string to format the snapshot name. To add a time stamp to the snapshot name, use the Pongo2 context variable `creation_date`. Make sure to format the date in your template string to avoid forbidden characters in the snapshot name. For example, set `snapshots.pattern` to `{{ creation_date|date:'2006-01-02_15-04-05' }}` to name the snapshots after their time of creation, down to the precision of a second. Another way to avoid name collisions is to use the placeholder `%d` in the pattern. If no matching snapshots exist, the placeholder is replaced with `0`. Otherwise, it is replaced with the next snapshot index, which is one higher than the highest existing matching snapshot index. ``` ```{config:option} snapshots.schedule storage-zfs-volume-conf :condition: "custom volume" :defaultdesc: "same as `snapshots.schedule`" :scope: "global" :shortdesc: "Schedule for automatic volume snapshots" :type: "string" Specify either a cron expression (` `), a comma-separated list of schedule aliases (`@hourly`, `@daily`, `@midnight`, `@weekly`, `@monthly`, `@annually`, `@yearly`), or leave empty to disable automatic snapshots (the default). ``` ```{config:option} user.* storage-zfs-volume-conf :scope: "global" :shortdesc: "User-provided free-form key/value pairs" :type: "string" ``` ```{config:option} volatile.devlxd.owner storage-zfs-volume-conf :defaultdesc: "DevLXD owner identity ID" :scope: "global" :shortdesc: "ID of the DevLXD identity that owns the volume" :type: "string" ``` ```{config:option} volatile.idmap.last storage-zfs-volume-conf :condition: "filesystem" :shortdesc: "JSON-serialized UID/GID map that has been applied to the volume" :type: "string" ``` ```{config:option} volatile.idmap.next storage-zfs-volume-conf :condition: "filesystem" :shortdesc: "JSON-serialized UID/GID map that has been applied to the volume" :type: "string" ``` ```{config:option} volatile.uuid storage-zfs-volume-conf :defaultdesc: "random UUID" :scope: "global" :shortdesc: "Volume UUID" :type: "string" ``` ```{config:option} zfs.block_mode storage-zfs-volume-conf :defaultdesc: "same as `volume.zfs.block_mode`" :scope: "global" :shortdesc: "Whether to use a formatted `zvol` rather than a dataset" :type: "bool" `zfs.block_mode` can be set only for custom storage volumes. To enable ZFS block mode for all storage volumes in the pool, including instance volumes, use `volume.zfs.block_mode`. ``` ```{config:option} zfs.blocksize storage-zfs-volume-conf :defaultdesc: "same as `volume.zfs.blocksize`" :scope: "global" :shortdesc: "Size of the ZFS block" :type: "string" The size must be between 512 bytes and 16 MiB and must be a power of 2. For a block volume, a maximum value of 128 KiB will be used even if a higher value is set. Depending on the value of {config:option}`storage-zfs-volume-conf:zfs.block_mode`, the specified size is used to set either `volblocksize` or `recordsize` in ZFS. ``` ```{config:option} zfs.delegate storage-zfs-volume-conf :condition: "ZFS 2.2 or higher" :defaultdesc: "same as `volume.zfs.delegate`" :scope: "global" :shortdesc: "Whether to delegate the ZFS dataset" :type: "bool" This option controls whether to delegate the ZFS dataset and anything underneath it to the container or containers that use it. When used in conjunction with {config:option}`instance-security:security.nesting`, this allows using the `zfs` command in the container. ``` ```{config:option} zfs.promote storage-zfs-volume-conf :defaultdesc: "false" :scope: "global" :shortdesc: "Whether to promote the ZFS dataset" :type: "bool" This option controls whether to promote the ZFS dataset at volume create/refresh time. When enabled, if the source volume is a clone, the new volume will be promoted to be a parent. ``` ```{config:option} zfs.remove_snapshots storage-zfs-volume-conf :defaultdesc: "same as `volume.zfs.remove_snapshots` or `false`" :scope: "global" :shortdesc: "Remove snapshots as needed" :type: "bool" ``` ```{config:option} zfs.reserve_space storage-zfs-volume-conf :defaultdesc: "same as `volume.zfs.reserve_space` or `false`" :scope: "global" :shortdesc: "Use `reservation`/`refreservation` along with `quota`/`refquota`" :type: "bool" ``` ```{config:option} zfs.use_refquota storage-zfs-volume-conf :defaultdesc: "same as `volume.zfs.use_refquota` or `false`" :scope: "global" :shortdesc: "Use `refquota` instead of `quota` for space" :type: "bool" ``` `can_view` : Grants permission to view the certificate. `can_edit` : Grants permission to view and edit the certificate. `can_delete` : Grants permission to view and delete the certificate. `can_view` : Grants permission to view the cluster link. `can_edit` : Grants permission to view and edit the cluster link. `can_delete` : Grants permission to view and delete the cluster link. `can_view` : Grants permission to view the group. Identities can always view groups that they are a member of. `can_edit` : Grants permission to view and edit the group. `can_delete` : Grants permission to view and delete the group. `can_view` : Grants permission to view the identity. `can_edit` : Grants permission to view and edit the identity. To edit an identity, it is additionally required that the caller is able to view all groups that the identity is a member of. `can_delete` : Grants permission to view and delete the identity. `can_view` : Grants permission to view the identity provider group. `can_edit` : Grants permission to view and edit the identity provider group. `can_delete` : Grants permission to view and delete the identity provider group. `can_edit` : Grants permission to view and edit the image. `can_delete` : Grants permission to view and delete the image. `can_view` : Grants permission to view the image. `can_edit` : Grants permission to view and edit the image alias. `can_delete` : Grants permission to view and delete the image alias. `can_view` : Grants permission to view the image alias. `can_view` : Grants permission to view the image registry. `can_edit` : Grants permission to view and edit the image registry. `can_delete` : Grants permission to view and delete the image registry. `user` : Grants permission to view the instance, to access files, and to start a terminal or console session. `operator` : Grants permission to view the instance, to access files, start a terminal or console session, and to manage snapshots and backups. `can_edit` : Grants permission to view and edit the instance. `can_delete` : Grants permission to view and delete the instance. `can_view` : Grants permission to view the instance and any snapshots or backups it might have. `can_update_state` : Grants permission to change the instance state. `can_manage_snapshots` : Grants permission to create and delete snapshots of the instance. `can_manage_backups` : Grants permission to create and delete backups of the instance. `can_connect_sftp` : Grants permission to get an SFTP client for the instance. `can_connect_nbd` : Grants permission to connect to the instance's disks over NBD. `can_access_files` : Grants permission to push or pull files into or out of the instance. `can_access_console` : Grants permission to start a console session. `can_exec` : Grants permission to start a terminal session. `can_edit` : Grants permission to view and edit the network. `can_delete` : Grants permission to view and delete the network. `can_view` : Grants permission to view the network. `can_edit` : Grants permission to view and edit the network ACL. `can_delete` : Grants permission to view and delete the network ACL. `can_view` : Grants permission to view the network ACL. `can_edit` : Grants permission to view and edit the network zone. `can_delete` : Grants permission to view and delete the network zone. `can_view` : Grants permission to view the network zone. `can_edit` : Grants permission to view and edit the placement group. `can_delete` : Grants permission to view and delete the placement group. `can_view` : Grants permission to view the placement group. `can_edit` : Grants permission to view and edit the profile. `can_delete` : Grants permission to view and delete the profile. `can_view` : Grants permission to view the profile. `operator` : Grants permission to create, view, edit, and delete all resources belonging to the project, but does not grant permission to edit the project configuration itself. `viewer` : Grants permission to view all resources belonging to the project. `can_view` : Grants permission to view the project. `can_edit` : Grants permission to view and edit the project. `can_delete` : Grants permission to view and delete the project. `image_manager` : Grants permission to create, view, edit, and delete all images belonging to the project. `can_create_images` : Grants permission to create images. `can_view_images` : Grants permission to view images. `can_edit_images` : Grants permission to view and edit images. `can_delete_images` : Grants permission to view and delete images. `image_alias_manager` : Grants permission to create, view, edit, and delete all image aliases belonging to the project. `can_create_image_aliases` : Grants permission to create image aliases. `can_view_image_aliases` : Grants permission to view image aliases. `can_edit_image_aliases` : Grants permission to view and edit image aliases. `can_delete_image_aliases` : Grants permission to view and delete image aliases. `instance_manager` : Grants permission to create, view, edit, and delete all instances belonging to the project. `can_create_instances` : Grants permission to create instances. `can_view_instances` : Grants permission to view instances. `can_edit_instances` : Grants permission to view and edit instances. `can_delete_instances` : Grants permission to view and delete instances. `can_operate_instances` : Grants permission to view instances, manage their state, manage their snapshots and backups, start terminal or console sessions, and access their files. `network_manager` : Grants permission to create, view, edit, and delete all networks belonging to the project. `can_create_networks` : Grants permission to create networks. `can_view_networks` : Grants permission to view networks. `can_edit_networks` : Grants permission to view and edit networks. `can_delete_networks` : Grants permission to view and delete networks. `network_acl_manager` : Grants permission to create, view, edit, and delete all network ACLs belonging to the project. `can_create_network_acls` : Grants permission to create network ACLs. `can_view_network_acls` : Grants permission to view network ACLs. `can_edit_network_acls` : Grants permission to view and edit network ACLs. `can_delete_network_acls` : Grants permission to view and delete network ACLs. `network_zone_manager` : Grants permission to create, view, edit, and delete all network zones belonging to the project. `can_create_network_zones` : Grants permission to create network zones. `can_view_network_zones` : Grants permission to view network zones. `can_edit_network_zones` : Grants permission to view and edit network zones. `can_delete_network_zones` : Grants permission to view and delete network zones. `profile_manager` : Grants permission to create, view, edit, and delete all profiles belonging to the project. `can_create_profiles` : Grants permission to create profiles. `can_view_profiles` : Grants permission to view profiles. `can_edit_profiles` : Grants permission to view and edit profiles. `can_delete_profiles` : Grants permission to view and delete profiles. `storage_volume_manager` : Grants permission to create, view, edit, and delete all storage volumes belonging to the project. `can_create_storage_volumes` : Grants permission to create storage volumes. `can_view_storage_volumes` : Grants permission to view storage volumes. `can_edit_storage_volumes` : Grants permission to view and edit storage volumes. `can_delete_storage_volumes` : Grants permission to view and delete storage volumes. `storage_bucket_manager` : Grants permission to create, view, edit, and delete all storage buckets belonging to the project. `can_create_storage_buckets` : Grants permission to create storage buckets. `can_view_storage_buckets` : Grants permission to view storage buckets. `can_edit_storage_buckets` : Grants permission to view and edit storage buckets. `can_delete_storage_buckets` : Grants permission to view and delete storage buckets. `placement_group_manager` : Grants permission to create, view, edit, and delete all placement groups belonging to the project. `can_create_placement_groups` : Grants permission to create placement groups. `can_view_placement_groups` : Grants permission to view placement groups. `can_edit_placement_groups` : Grants permission to view and edit placement groups. `can_delete_placement_groups` : Grants permission to view and delete placement groups. `replicator_manager` : Grants permission to create, view, edit, and delete all replicators belonging to the project. `can_create_replicators` : Grants permission to create replicators. `can_view_replicators` : Grants permission to view replicators. `can_edit_replicators` : Grants permission to view and edit replicators. `can_delete_replicators` : Grants permission to view and delete replicators. `can_view_operations` : Grants permission to view operations relating to the project. `can_view_events` : Grants permission to view life cycle events relating to the project. `can_view_metrics` : Grants permission to view project level metrics. `can_edit` : Grants permission to view and edit the replicator. `can_delete` : Grants permission to view and delete the replicator. `can_view` : Grants permission to view the replicator. `admin` : Grants full access to LXD as if via Unix socket. `viewer` : Grants access to view all resources in the LXD server. `can_edit` : Grants permission to edit server configuration, to edit cluster member configuration, to update the state of a cluster member, to create, edit, and delete cluster groups, to create, edit, and delete cluster links, to create, edit, and delete image registries, to update cluster member certificates, and to edit or delete warnings. `permission_manager` : Grants permission to view permissions, to create, edit, and delete identities, to view, create, edit, and delete authorization groups, and to view, create, edit, and delete identity provider groups. Note that clients with this permission are able to elevate their own privileges. `can_view_permissions` : Grants permission to view permissions. `can_create_identities` : Grants permission to create identities. `can_view_identities` : Grants permission to view identities. `can_edit_identities` : Grants permission to view and edit identities. Note that clients with this permission are able to elevate their own privileges. `can_delete_identities` : Grants permission to view and delete identities. `can_create_groups` : Grants permission to create authorization groups. `can_view_groups` : Grants permission to view authorization groups. `can_edit_groups` : Grants permission to view and edit authorization groups. Note that clients with this permission are able to elevate their own privileges. `can_delete_groups` : Grants permission to view and delete authorization groups. `can_create_identity_provider_groups` : Grants permission to create identity provider groups. `can_view_identity_provider_groups` : Grants permission to view identity provider groups. `can_edit_identity_provider_groups` : Grants permission to view and edit identity provider groups. Note that clients with this permission are able to elevate their own privileges. `can_delete_identity_provider_groups` : Grants permission to view and delete identity provider groups. `storage_pool_manager` : Grants permission to create, edit, and delete storage pools. `can_create_storage_pools` : Grants permission to create storage pools. `can_edit_storage_pools` : Grants permission to edit storage pools. `can_delete_storage_pools` : Grants permission to delete storage pools. `project_manager` : Grants permission to create, view, edit, and delete projects, and to create, view, edit, and delete resources belonging to any project. `can_create_projects` : Grants permission to create projects. `can_view_projects` : Grants permission to view projects, and all resources within those projects. `can_edit_projects` : Grants permission to view and edit projects, and all resources within those projects. `can_delete_projects` : Grants permission to view and delete projects. `can_override_cluster_target_restriction` : If a project is configured with `restricted.cluster.target`, clients with this permission can override the restriction. `can_view_events` : Grants permission to view `logging` events, `ovn` events, and all `lifecycle` events that are not specific to a project. `can_view_operations` : Grants permission to view operations that are not specific to a project. `can_view_resources` : Grants permission to view server and storage pool resource usage information. `can_view_metrics` : Grants permission to view all server and project level metrics. `can_view_warnings` : Grants permission to view warnings. `can_view_unmanaged_networks` : Grants permission to view unmanaged networks on the LXD host machines. `can_create_cluster_links` : Grants permission to create cluster links. `can_view_cluster_links` : Grants permission to view cluster links. `can_edit_cluster_links` : Grants permission to view and edit cluster links. `can_delete_cluster_links` : Grants permission to view and delete cluster links. `can_create_image_registries` : Grants permission to create image registries. `can_view_image_registries` : Grants permission to view image registries. `can_edit_image_registries` : Grants permission to edit image registries. `can_delete_image_registries` : Grants permission to delete image registries. `can_edit` : Grants permission to view and edit the storage bucket. `can_delete` : Grants permission to view and delete the storage bucket. `can_view` : Grants permission to view the storage bucket. `can_edit` : Grants permission to edit the storage pool. `can_delete` : Grants permission to delete the storage pool. `can_edit` : Grants permission to view and edit the storage volume. `can_delete` : Grants permission to view and delete the storage volume. `can_view` : Grants permission to view the storage volume and any snapshots or backups it might have. `can_manage_snapshots` : Grants permission to create and delete snapshots of the storage volume. `can_manage_backups` : Grants permission to create and delete backups of the storage volume. `can_connect_nbd` : Grants permission to connect to the storage volume over NBD. --- --- METRICS --- - Files Extracted: 51 - Estimated Token Budget: ~166427 tokens - Canonical Reference: https://codewiki.google/github.com/canonical/lxd