### DOCKER
# Docker Instructions
There are two flavors of Dockerfiles available:
- `Dockerfile`: Used for production builds. Checks out the source code from
GitHub during build. The build argument `--build-arg checkout=v0.x.x-beta`
can be used to specify what git tag or commit to `git checkout`
before building.
- `dev.Dockerfile` Used for development or testing builds. Uses the local code
when building and allows local changes to be tested more easily.
## Development/testing
To build a standalone development image from the local source directory, use the
following command:
```shell
$ docker build --tag=myrepository/lnd-dev -f dev.Dockerfile .
```
There is also a `docker-compose` setup available for development or testing that
spins up a `btcd` backend alongside `lnd`. Check out the documentation at
[docker/README.md](../docker/README.md) to learn more about how to use that
setup to create a small local Lightning Network.
## Production (manual build)
To use Docker in a production environment, you can run `lnd` by creating a
Docker container, adding the appropriate command-line options as parameters.
You first need to build the `lnd` docker image:
```shell
$ docker build --tag=myrepository/lnd --build-arg checkout=v0.14.1-beta .
```
It is recommended that you check out the latest released tag.
You can continue by creating and running the container:
```shell
$ docker run myrepository/lnd [command-line options]
```
## Production (official images)
Starting with `lnd v0.12.0-beta`, there are official, automatically built docker
images of `lnd` available in the
[`lightninglabs/lnd` repository on Docker Hub](https://hub.docker.com/r/lightninglabs/lnd).
You can just pull those images by specifying a release tag:
```shell
$ docker pull lightninglabs/lnd:v0.14.1-beta
$ docker run lightninglabs/lnd [command-line options]
```
Note that **`daily-*` tags are unstable and not for production use**.
They are only suitable for development and pre-release testing.
### Verifying docker images
To verify the `lnd` and `lncli` binaries inside the docker images against the
signed, [reproducible release binaries](release.md), there is a verification
script in the image that can be called (before starting the container for
example):
```shell
$ docker run --rm --entrypoint="" lightninglabs/lnd:v0.14.1-beta /verify-install.sh v0.14.1-beta
$ OK=$?
$ if [ "$OK" -ne "0" ]; then echo "Verification failed!"; exit 1; done
$ docker run lightninglabs/lnd:v0.14.1-beta [command-line options]
```
## Volumes
A Docker volume will be created with your `.lnd` directory automatically, and will
persist through container restarts.
You can also optionally manually specify a local folder to be used as a volume:
```shell
$ docker create --name=mylndcontainer -v /media/lnd-docker/:/root/.lnd myrepository/lnd [command-line options]
```
## Example
Here is an example testnet `lnd` that uses Neutrino:
```shell
$ docker run --name lnd-testnet myrepository/lnd --bitcoin.active --bitcoin.testnet --bitcoin.node=neutrino --neutrino.connect=faucet.lightning.community
```
Create a wallet (and write down the seed):
```shell
$ docker exec -it lnd-testnet lncli create
```
Confirm `lnd` has begun to synchronize:
```shell
$ docker logs lnd-testnet
[snipped]
2018-05-01 02:28:01.201 [INF] RPCS: RPC server listening on 127.0.0.1:10009
2018-05-01 02:28:01.201 [INF] LTND: Waiting for chain backend to finish sync, start_height=2546
2018-05-01 02:28:01.201 [INF] RPCS: gRPC proxy started at 127.0.0.1:8080
2018-05-01 02:28:08.999 [INF] LNWL: Caught up to height 10000
2018-05-01 02:28:09.872 [INF] BTCN: Processed 10547 blocks in the last 10.23s (height 10547, 2012-05-28 05:02:32 +0000 UTC)
```
This is a simple example, it is possible to use any command-line options necessary
to expose RPC ports, use `btcd` or `bitcoind`, or add additional chains.
## LND Development and Testing
To test the Docker production image locally, run the following from the project root:
```shell
$ docker build . -t myrepository/lnd:master
```
To choose a specific [branch](https://github.com/lightningnetwork/lnd/branches) or [tag](https://hub.docker.com/r/lightninglabs/lnd/tags?page=1&ordering=last_updated) instead, use the `checkout` build-arg. For example, to build the latest tagged commit:
```shell
$ docker build . --build-arg checkout=v0.14.1-beta -t myrepository/lnd:v0.14.1-beta
```
To build the image using the most current tag:
```shell
$ docker build . --build-arg checkout=$(git describe --tags `git rev-list --tags --max-count=1`) -t myrepository/lnd:latest-tag
```
You can also specify a different repo than the default (`https://github.com/lightningnetwork/lnd`) using the `git_url` build-arg:
```shell
$ docker build . --build-arg git_url=https://git.example.com/lnd.git --build-arg checkout=mybranch -t myrepository/lnd:v0.13.0-beta
```
Once the image has been built and tagged locally, start the container:
```shell
$ docker run --name=lnd-testnet -it myrepository/lnd:latest-tag --bitcoin.active --bitcoin.testnet --bitcoin.node=neutrino --neutrino.connect=faucet.lightning.community
```
---
### INSTALL
# Table of Contents
- [Table of Contents](#table-of-contents)
- [Installation](#installation)
- [Installing a binary release](#installing-a-binary-release)
- [Building a tagged version with Docker](#building-a-tagged-version-with-docker)
- [Building a development version from source](#building-a-development-version-from-source)
- [Installing Go](#installing-go)
- [Go modules](#go-modules)
- [Installing lnd from source](#installing-lnd-from-source)
- [Available Backend Operating Modes](#available-backend-operating-modes)
- [btcd Options](#btcd-options)
- [Neutrino Options](#neutrino-options)
- [Bitcoind Options](#bitcoind-options)
- [Using btcd](#using-btcd)
- [Installing btcd](#installing-btcd)
- [Starting btcd](#starting-btcd)
- [Running lnd using the btcd backend](#running-lnd-using-the-btcd-backend)
- [Using Neutrino](#using-neutrino)
- [Using bitcoind ](#using-bitcoind)
- [Creating a wallet](#creating-a-wallet)
- [Macaroons](#macaroons)
- [Network Reachability](#network-reachability)
- [Simnet vs. Testnet Development](#simnet-vs-testnet-development)
- [Creating an lnd.conf (Optional)](#creating-an-lndconf-optional)
# Installation
There are multiple ways to install `lnd`. For most users the easiest way is to
[download and install an official release binary](#installing-a-binary-release).
Those release binaries are always built with production in mind and have all
RPC subservers enabled.
More advanced users that want to build `lnd` from source also have multiple
options. To build a tagged version, there is a docker build helper script that
allows users to
[build `lnd` from source without needing to install `golang`](#building-a-tagged-version-with-docker).
That is also the preferred way to build and verify the reproducible builds that
are released by the team. See
[release.md for more information about reproducible builds](release.md).
Finally, there is the option to build `lnd` fully manually. This requires more
tooling to be set up first but allows producing non-production (debug,
development) builds.
## Installing a binary release
Downloading and installing an official release binary is recommended for use on
mainnet.
[Visit the release page on GitHub](https://github.com/lightningnetwork/lnd/releases)
and select the latest version that does not have the "Pre-release" label set
(unless you explicitly want to help test a Release Candidate, RC).
Choose the package that best fits your operating system and system architecture.
It is recommended to choose 64bit versions over 32bit ones, if your operating
system supports both.
Extract the package and place the two binaries (`lnd` and `lncli` or `lnd.exe`
and `lncli.exe` on Windows) somewhere where the operating system can find them.
## Building a tagged version with Docker
To use the Docker build helper, you need to have the following software
installed and set up on your machine:
- Docker
- `make`
- `bash`
To build a specific git tag of `lnd`, simply run the following steps (assuming
`v0.x.y-beta` is the tagged version to build):
```shell
git clone https://github.com/lightningnetwork/lnd
cd lnd
git checkout v0.x.y-beta
make docker-release tag=v0.x.y-beta
```
This will create a directory called `lnd-v0.x.y-beta` that contains the release
binaries for all operating system and architecture pairs. A single pair can also
be selected by specifying the `sys=linux-amd64` flag for example. See
[release.md for more information on reproducible builds](release.md).
## Building a development version from source
Building and installing `lnd` from source is only recommended for advanced users
and/or developers. Running the latest commit from the `master` branch is not
recommended for mainnet. The `master` branch can at times be unstable and
running your node off of it can prevent it to go back to a previous, stable
version if there are database migrations present.
In order to work with [`lnd`](https://github.com/lightningnetwork/lnd), the
following build dependencies are required:
### Installing Go
`lnd` is written in Go, with a minimum version of `1.25.11` (or, in case this
document gets out of date, whatever the Go version in the main `go.mod` file
requires). To install, run one of the following commands for your OS:
Linux (x86-64)
```
wget https://dl.google.com/go/go1.25.11.linux-amd64.tar.gz
echo "34f14304e856893f4ba30c2cacfe93906e9de7915c5f6aaaf3a81cdccd7ba30b go1.25.11.linux-amd64.tar.gz" | sha256sum --check
```
The command above should output `go1.25.11.linux-amd64.tar.gz: OK`. If it
doesn't, then the target REPO HAS BEEN MODIFIED, and you shouldn't install
this version of Go. If it matches, then proceed to install Go:
```
sudo rm -rf /usr/local/go && sudo tar -C /usr/local -xzf go1.25.11.linux-amd64.tar.gz
export PATH=$PATH:/usr/local/go/bin
```
Linux (ARMv6)
```
wget https://dl.google.com/go/go1.25.11.linux-armv6l.tar.gz
echo "492d69badee59cae12e9a36282dfce94041bd4aac88fdddea575a7d99a2bd05d go1.25.11.linux-armv6l.tar.gz" | sha256sum --check
```
The command above should output `go1.25.11.linux-armv6l.tar.gz: OK`. If it
isn't, then the target REPO HAS BEEN MODIFIED, and you shouldn't install
this version of Go. If it matches, then proceed to install Go:
```
sudo rm -rf /usr/local/go && sudo tar -C /usr/local -xzf go1.25.11.linux-armv6l.tar.gz
export PATH=$PATH:/usr/local/go/bin
```
macOS
First, install [Homebrew](https://brew.sh) if you don't already have it.
Then
```
brew install go
```
FreeBSD
```
pkg install go
```
Alternatively, one can download the pre-compiled binaries hosted on the
[Golang download page](https://golang.org/dl/). If one seeks to install
from source, then more detailed installation instructions can be found
[here](https://golang.org/doc/install).
***Important***
At this point, you should set your `$GOPATH` environment variable, which
represents the path to your workspace. By default, `$GOPATH` is set to
`~/go`. You will also need to add `$GOPATH/bin` to your `PATH`. This ensures
that your shell will be able to detect the binaries you install.
```shell
export GOPATH=~/go
export PATH=$PATH:$GOPATH/bin
```
---
We recommend placing the above in your `.bashrc`, `.zshrc` or in a setup script
so that you can avoid typing this every time you open a new terminal window.
### Go modules
This project uses [Go modules](https://github.com/golang/go/wiki/Modules)
to manage dependencies as well as to provide *reproducible builds*.
Usage of Go modules (with Go 1.13) means that you no longer need to clone
`lnd` into your `$GOPATH` for development purposes. Instead, your `lnd`
repo can now live anywhere!
---
Note: For mobile development, having the source code in `$GOPATH` is still
required due to a current limitation in
[Go mobile](https://pkg.go.dev/golang.org/x/mobile). Take a look at the
documentation for [building mobile libraries](../mobile) to learn more.
---
### Installing lnd from source
With the preliminary steps completed, to install `lnd`, `lncli`, and all
related dependencies run the following commands:
```shell
git clone https://github.com/lightningnetwork/lnd
cd lnd
make install
```
The command above will install the current _master_ branch of `lnd`. If you
wish to install a tagged release of `lnd` (as the master branch can at times be
unstable), then [visit the release page to locate the latest
release](https://github.com/lightningnetwork/lnd/releases). Assuming the name
of the release is `v0.x.x`, then you can compile this release from source with
a small modification to the above command:
```shell
git clone https://github.com/lightningnetwork/lnd
cd lnd
git checkout v0.x.x
make install
```
**NOTE**: Our instructions still use the `$GOPATH` directory from prior
versions of Go, but with Go 1.13, it's now possible for `lnd` to live
_anywhere_ on your file system.
For Windows WSL users, make will need to be referenced directly via
/usr/bin/make/, or alternatively by wrapping quotation marks around make,
like so:
```shell
/usr/bin/make && /usr/bin/make install
"make" && "make" install
```
On FreeBSD, use gmake instead of make.
Alternatively, if one doesn't wish to use `make`, then the `go` commands can be
used directly:
```shell
go install -v ./...
```
**Tags**
Release binaries and installations from source using `make release-install`
will have the following tags:
- [autopilotrpc](/lnrpc/autopilotrpc/autopilot.proto)
- [signrpc](/lnrpc/signrpc/signer.proto)
- [walletrpc](/lnrpc/walletrpc/walletkit.proto)
- [chainrpc](/lnrpc/chainrpc/chainnotifier.proto)
- [invoicesrpc](/lnrpc/invoicesrpc/invoices.proto)
- [neutrinorpc](/lnrpc/neutrinorpc/neutrino.proto)
- [routerrpc](/lnrpc/routerrpc/router.proto)
- [watchtowerrpc](/lnrpc/watchtowerrpc/watchtower.proto)
- [monitoring](/monitoring) (for Prometheus integration)
- [peersrpc](/lnrpc/peersrpc/peers.proto)
- [kvdb_postrgres](/docs/postgres.md)
- [kvdb_sqlite](/docs/sqlite.md)
The `dev` tag is used for development builds, and is not included in the
release builds & installation.
You can specify a custom set of tags when installing from source using the `tags=""` parameter. For example:
```shell
make install tags="signrpc walletrpc routerrpc invoicesrpc"
```
**Updating**
To update your version of `lnd` to the latest version run the following
commands:
```shell
cd $GOPATH/src/github.com/lightningnetwork/lnd
git pull
make clean && make && make install
```
On FreeBSD, use gmake instead of make.
Alternatively, if one doesn't wish to use `make`, then the `go` commands can be
used directly:
```shell
cd $GOPATH/src/github.com/lightningnetwork/lnd
git pull
go install -v ./...
```
**Tests**
To check that `lnd` was installed properly run the following command:
```shell
make check
```
This command requires `bitcoind` (almost any version should do) to be available
in the system's `$PATH` variable. Otherwise, some tests will fail.
**Command-line completion for `lncli`**
_Bash_: See `contrib/lncli.bash-completion`
_Fish_: Run: `lncli fish-completion > $HOME/.config/fish/completions/lncli.fish`
# Available Backend Operating Modes
In order to run, `lnd` requires, that the user specify a chain backend. At the
time of writing of this document, there are three available chain backends:
`btcd`, `neutrino`, `bitcoind`. All including neutrino can run on mainnet with
an out of the box `lnd` instance. We don't require `--txindex` when running
with `bitcoind` or `btcd` but activating the `txindex` will generally make
`lnd` run faster. Note that since version 0.13 pruned nodes are supported,
although they cause performance penalty and higher network usage.
The set of arguments for each of the backend modes is as follows:
## btcd Options
```text
btcd:
--btcd.dir= The base directory that contains the node's data, logs, configuration file, etc. (default: /Users//Library/Application Support/Btcd)
--btcd.rpchost= The daemon's rpc listening address. If a port is omitted, then the default port for the selected chain parameters will be used. (default: localhost)
--btcd.rpcuser= Username for RPC connections
--btcd.rpcpass= Password for RPC connections
--btcd.rpccert= File containing the daemon's certificate file (default: /Users//Library/Application Support/Btcd/rpc.cert)
--btcd.rawrpccert= The raw bytes of the daemon's PEM-encoded certificate chain which will be used to authenticate the RPC connection.
```
## Neutrino Options
```text
neutrino:
-a, --neutrino.addpeer= Add a peer to connect with at startup
--neutrino.connect= Connect only to the specified peers at startup
--neutrino.maxpeers= Max number of inbound and outbound peers
--neutrino.banduration= How long to ban misbehaving peers. Valid time units are {s, m, h}. Minimum 1 second
--neutrino.banthreshold= Maximum allowed ban score before disconnecting and banning misbehaving peers.
--neutrino.useragentname= Used to help identify ourselves to other bitcoin peers.
--neutrino.useragentversion= Used to help identify ourselves to other bitcoin peers.
```
## Bitcoind Options
```text
bitcoind:
--bitcoind.dir= The base directory that contains the node's data, logs, configuration file, etc. (default: /Users//Library/Application Support/Bitcoin)
--bitcoind.rpchost= The daemon's rpc listening address. If a port is omitted, then the default port for the selected chain parameters will be used. (default: localhost)
--bitcoind.rpcuser= Username for RPC connections
--bitcoind.rpcpass= Password for RPC connections
--bitcoind.zmqpubrawblock= The address listening for ZMQ connections to deliver raw block notifications
--bitcoind.zmqpubrawtx= The address listening for ZMQ connections to deliver raw transaction notifications
--bitcoind.estimatemode= The fee estimate mode. Must be either "ECONOMICAL" or "CONSERVATIVE". (default: CONSERVATIVE)
```
## Using btcd
### Installing btcd
On FreeBSD, use gmake instead of make.
In order to be able to utilize the latest Taproot features, [`btcd` version
`v0.23.5`](https://github.com/btcsuite/btcd/releases/tag/v0.23.5) MUST be used.
To install btcd, run the following commands:
Install **btcd**:
```shell
make btcd
```
Alternatively, you can install [`btcd` directly from its
repo](https://github.com/btcsuite/btcd).
### Starting btcd
Running the following command will create `rpc.cert` and default `btcd.conf`.
```shell
btcd --testnet --rpcuser=REPLACEME --rpcpass=REPLACEME
```
If you want to use `lnd` on testnet, `btcd` needs to first fully sync the
testnet blockchain. Depending on your hardware, this may take up to a few
hours. Note that adding `--txindex` is optional, as it will take longer to sync
the node, but then `lnd` will generally operate faster as it can hit the index
directly, rather than scanning blocks or BIP 158 filters for relevant items.
(NOTE: It may take several minutes to find segwit-enabled peers.)
While `btcd` is syncing you can check on its progress using btcd's `getinfo`
RPC command:
```shell
btcctl --testnet --rpcuser=REPLACEME --rpcpass=REPLACEME getinfo
{
"version": 120000,
"protocolversion": 70002,
"blocks": 1114996,
"timeoffset": 0,
"connections": 7,
"proxy": "",
"difficulty": 422570.58270815,
"testnet": true,
"relayfee": 0.00001,
"errors": ""
}
```
Additionally, you can monitor btcd's logs to track its syncing progress in real
time.
You can test your `btcd` node's connectivity using the `getpeerinfo` command:
```shell
btcctl --testnet --rpcuser=REPLACEME --rpcpass=REPLACEME getpeerinfo | more
```
### Running lnd using the btcd backend
If you are on testnet, run this command after `btcd` has finished syncing.
Otherwise, replace `--bitcoin.testnet` with `--bitcoin.simnet`. If you are
installing `lnd` in preparation for the
[tutorial](https://dev.lightning.community/tutorial), you may skip this step.
```shell
lnd --bitcoin.active --bitcoin.testnet --debuglevel=debug \
--btcd.rpcuser=kek --btcd.rpcpass=kek --externalip=X.X.X.X
```
## Using Neutrino
In order to run `lnd` in its light client mode, you'll need to locate a
full-node which is capable of serving this new light client mode. `lnd` uses
[BIP 157](https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki) and [BIP
158](https://github.com/bitcoin/bips/blob/master/bip-0158.mediawiki) for its light client
mode. A public instance of such a node can be found at
`faucet.lightning.community`.
To run lnd in neutrino mode, run `lnd` with the following arguments, (swapping
in `--bitcoin.simnet` if needed), and also your own `btcd` node if available:
```shell
lnd --bitcoin.active --bitcoin.testnet --debuglevel=debug \
--bitcoin.node=neutrino --neutrino.connect=faucet.lightning.community
```
## Using bitcoind
Note that adding `-txindex` is optional, as it will take longer to sync the
node, but then `lnd` will generally operate faster as it can hit the index
directly, rather than scanning blocks or BIP 158 filters for relevant items.
To configure your bitcoind backend for use with lnd, first complete and verify
the following:
- Since `lnd` uses
[ZeroMQ](https://github.com/bitcoin/bitcoin/blob/master/doc/zmq.md) to
interface with `bitcoind`, *your `bitcoind` installation must be compiled with
ZMQ*. Note that if you installed `bitcoind` from source and ZMQ was not present,
then ZMQ support will be disabled, and `lnd` will quit on a `connection refused` error.
If you installed `bitcoind` via Homebrew in the past ZMQ may not be included
([this has now been fixed](https://github.com/Homebrew/homebrew-core/pull/23088)
in the latest Homebrew recipe for bitcoin)
- Configure the `bitcoind` instance for ZMQ with `-zmqpubrawblock` and
`-zmqpubrawtx`. These options must each use their own unique address in order
to provide a reliable delivery of notifications (e.g.
`-zmqpubrawblock=tcp://127.0.0.1:28332` and
`-zmqpubrawtx=tcp://127.0.0.1:28333`).
- Make sure the config setting `-rpcserialversion` in `bitcoind` is either set
to 1 or NOT used because bitcoind's default behaviour is already correct
(see [bitcoin/issues/28730](https://github.com/bitcoin/bitcoin/issues/28730)
for more info). Lightning depends on segwit transactions therefore we need
the witness data when querying the bitcoind backend for transaction details.
- Start `bitcoind` running against testnet, and let it complete a full sync with
the testnet chain (alternatively, use `--bitcoind.regtest` instead).
Here's a sample `bitcoin.conf` for use with lnd:
```text
testnet=1
server=1
daemon=1
zmqpubrawblock=tcp://127.0.0.1:28332
zmqpubrawtx=tcp://127.0.0.1:28333
```
Once all of the above is complete, and you've confirmed `bitcoind` is fully
updated with the latest blocks on testnet, run the command below to launch
`lnd` with `bitcoind` as your backend (as with `bitcoind`, you can create an
`lnd.conf` to save these options, more info on that is described further
below):
```shell
lnd --bitcoin.active --bitcoin.testnet --debuglevel=debug \
--bitcoin.node=bitcoind --bitcoind.rpcuser=REPLACEME \
--bitcoind.rpcpass=REPLACEME \
--bitcoind.zmqpubrawblock=tcp://127.0.0.1:28332 \
--bitcoind.zmqpubrawtx=tcp://127.0.0.1:28333 \
--externalip=X.X.X.X
```
*NOTE:*
- The auth parameters `rpcuser` and `rpcpass` parameters can typically be
determined by `lnd` for a `bitcoind` instance running under the same user,
including when using cookie auth. In this case, you can exclude them from the
`lnd` options entirely.
- If you DO choose to explicitly pass the auth parameters in your `lnd.conf` or
command line options for `lnd` (`bitcoind.rpcuser` and `bitcoind.rpcpass` as
shown in example command above), you must also specify the
`bitcoind.zmqpubrawblock` and `bitcoind.zmqpubrawtx` options. Otherwise, `lnd`
will attempt to get the configuration from your `bitcoin.conf`.
- You must ensure the same addresses are used for the `bitcoind.zmqpubrawblock`
and `bitcoind.zmqpubrawtx` options passed to `lnd` as for the `zmqpubrawblock`
and `zmqpubrawtx` passed in the `bitcoind` options respectively.
- When running lnd and bitcoind on the same Windows machine, ensure you use
127.0.0.1, not localhost, for all configuration options that require a TCP/IP
host address. If you use "localhost" as the host name, you may see extremely
slow inter-process-communication between lnd and the bitcoind backend. If lnd
is experiencing this issue, you'll see "Waiting for chain backend to finish
sync, start_height=XXXXXX" as the last entry in the console or log output, and
lnd will appear to hang. Normal lnd output will quickly show multiple
messages like this as lnd consumes blocks from bitcoind.
- Don't connect more than two or three instances of `lnd` to `bitcoind`. With
the default `bitcoind` settings, having more than one instance of `lnd`, or
`lnd` plus any application that consumes the RPC could cause `lnd` to miss
crucial updates from the backend.
- The default fee estimate mode in `bitcoind` is CONSERVATIVE. You can set
`bitcoind.estimatemode=ECONOMICAL` to change it into ECONOMICAL. Furthermore,
if you start `bitcoind` in `regtest`, this configuration won't take any effect.
# Creating a wallet
If `lnd` is being run for the first time, create a new wallet with:
```shell
lncli create
```
This will prompt for a wallet password, and optionally a cipher seed
passphrase.
`lnd` will then print a 24 word cipher seed mnemonic, which can be used to
recover the wallet in case of data loss. The user should write this down and
keep in a safe place.
More [information about managing wallets can be found in the wallet management
document](wallet.md).
# Macaroons
`lnd`'s authentication system is called **macaroons**, which are decentralized
bearer credentials allowing for delegation, attenuation, and other cool
features. You can learn more about them in Alex Akselrod's [writeup on
GitHub](https://github.com/lightningnetwork/lnd/issues/20).
Running `lncli create` to create a wallet, will by default generate
the `admin.macaroon`, `read_only.macaroon`, and `macaroons.db`
files that are used to authenticate
into `lnd`. They will be stored in the network directory (default:
`lnddir/data/chain/bitcoin/mainnet`) so that it's possible to use a distinct
password for mainnet, testnet, simnet, etc. Note that if you specified an
alternative data directory (via the `--datadir` argument), you will have to
additionally pass the updated location of the `admin.macaroon` file into `lncli`
using the `--macaroonpath` argument.
To disable macaroons for testing, pass the `--no-macaroons` flag into *both*
`lnd` and `lncli`.
# Network Reachability
If you'd like to signal to other nodes on the network that you'll accept
incoming channels (as peers need to connect inbound to initiate a channel
funding workflow), then the `--externalip` flag should be set to your publicly
reachable IP address.
# Simnet vs. Testnet Development
If you are doing local development, such as for the tutorial, you'll want to
start both `btcd` and `lnd` in the `simnet` mode. Simnet is similar to regtest
in that you'll be able to instantly mine blocks as needed to test `lnd`
locally. In order to start either daemon in the `simnet` mode use `simnet`
instead of `testnet`, adding the `--bitcoin.simnet` flag instead of the
`--bitcoin.testnet` flag.
Another relevant command line flag for local testing of new `lnd` developments
is the `--debughtlc` flag. When starting `lnd` with this flag, it'll be able to
automatically settle a special type of HTLC sent to it. This means that you
won't need to manually insert invoices in order to test payment connectivity.
To send this "special" HTLC type, include the `--debugsend` command at the end
of your `sendpayment` commands.
There are currently two primary ways to run `lnd`: one requires a local `btcd`
instance with the RPC service exposed, and the other uses a fully integrated
light client powered by [neutrino](https://github.com/lightninglabs/neutrino).
For testing scenarios like integration tests where wallet security is not
important, the `itest/lnd-itest` binary can be used which uses very weak
password stretching for the wallet encryption and therefore starts up faster
than a production/mainnet/release build. The binary can be built by running
`make build-itest`.
# Creating an lnd.conf (Optional)
Optionally, if you'd like to have a persistent configuration between `lnd`
launches, allowing you to simply type `lnd --bitcoin.testnet --bitcoin.active`
at the command line, you can create an `lnd.conf`.
**On macOS, located at:**
`/Users//Library/Application Support/Lnd/lnd.conf`
**On Linux, located at:**
`~/.lnd/lnd.conf`
Here's a sample `lnd.conf` for `btcd` to get you started:
```text
[Application Options]
debuglevel=trace
maxpendingchannels=10
[Bitcoin]
bitcoin.active=1
```
Notice the `[Bitcoin]` section. This section houses the parameters for the
Bitcoin chain. See a more detailed sample config file available
[here](https://github.com/lightningnetwork/lnd/blob/master/sample-lnd.conf)
and explore the other sections for node configuration, including `[Btcd]`,
`[Bitcoind]` and `[Neutrino]` depending on which chain and node type you're
using.
---
### MAKEFILE
Makefile
========
To build, verify, and install `lnd` from source, use the following
commands:
```shell
$ make
$ make check
$ make install
```
The command `make check` requires `bitcoind` (almost any version should do) to
be available in the system's `$PATH` variable. Otherwise, some tests will
fail.
Developers
==========
This document specifies all commands available from `lnd`'s `Makefile`.
The commands included handle:
- Installation of all go-related dependencies.
- Compilation and installation of `lnd` and `lncli`.
- Compilation and installation of `btcd` and `btcctl`.
- Running unit and integration suites.
- Testing, debugging, and flake hunting.
- Formatting and linting.
Commands
========
- [`all`](#scratch)
- [`btcd`](#btcd)
- [`build`](#build)
- [`check`](#check)
- [`clean`](#clean)
- [`default`](#default)
- [`dep`](#dep)
- [`flake-unit`](#flake-unit)
- [`flakehunter`](#flakehunter)
- [`fmt`](#fmt)
- [`install`](#install)
- [`itest`](#itest)
- [`lint`](#lint)
- [`list`](#list)
- [`rpc`](#rpc)
- [`scratch`](#scratch)
- [`travis`](#travis)
- [`unit`](#unit)
- [`unit-cover`](#unit-cover)
- [`unit-race`](#unit-race)
`all`
-----
Compiles, tests, and installs `lnd` and `lncli`. Equivalent to
[`scratch`](#scratch) [`check`](#check) [`install`](#install).
`btcd`
------
Ensures that the [`github.com/btcsuite/btcd`][btcd] repository is checked out
locally. Lastly, installs the version of
[`github.com/btcsuite/btcd`][btcd] specified in `Gopkg.toml`
`build`
-------
Compiles the current source and vendor trees, creating `./lnd` and
`./lncli`.
`check`
-------
Installs the version of [`github.com/btcsuite/btcd`][btcd] specified
in `Gopkg.toml`, then runs the unit tests followed by the integration
tests.
Related: [`unit`](#unit) [`itest`](#itest)
`clean`
-------
Removes compiled versions of both `./lnd` and `./lncli`, and removes the
`vendor` tree.
`default`
---------
Alias for [`scratch`](#scratch).
`flake-unit`
------------
Runs the unit test endlessly until a failure is detected.
Arguments:
- `pkg=`
- `case=`
- `timeout=`
Related: [`unit`](#unit)
`flakehunter`
-------------
Runs the integration test suite endlessly until a failure is detected.
Arguments:
- `icase=`
- `timeout=`
Related: [`itest`](#itest)
`fmt`
-----
Runs `go fmt` on the entire project.
`install`
---------
Copies the compiled `lnd` and `lncli` binaries into `$GOPATH/bin`.
`itest`
-------
Installs the version of [`github.com/btcsuite/btcd`][btcd] specified in
`Gopkg.toml`, builds the `./lnd` and `./lncli` binaries, then runs the
integration test suite.
Arguments:
- `icase=` (the snake_case version of the testcase name field in the testCases slice (i.e. sweep_coins), not the test func name)
- `timeout=`
`itest-parallel`
------
Does the same as `itest` but splits the total set of tests into
`NUM_ITEST_TRANCHES` tranches (currently set to 6 by default, can be overwritten
by setting `tranches=Y`) and runs them in parallel.
Arguments:
- `icase=`: The snake_case version of the testcase name field in the
testCases slice (i.e. `sweep_coins`, not the test func name) or any regular
expression describing a set of tests.
- `timeout=`
- `tranches=`: The number of parts/tranches to split the
total set of tests into.
- `parallel=`: The number of threads to run in parallel. Must
be greater or equal to `tranches`, otherwise undefined behavior is expected.
`flakehunter-parallel`
------
Runs the test specified by `icase` simultaneously `parallel` (default=6) times
until an error occurs. Useful for hunting flakes.
Example:
```shell
$ make flakehunter-parallel icase='(data_loss_protection|channel_backup)' backend=neutrino
```
`lint`
------
Ensures that [`gopkg.in/alecthomas/gometalinter.v1`][gometalinter] is
installed, then lints the project.
`list`
------
Lists all known make targets.
`rpc`
-----
Compiles the `lnrpc` proto files.
`sample-conf-check`
-------------------
Checks whether all required options of `lnd --help` are included in [sample-lnd.conf](github.com/lightningnetwork/lnd/blob/master/sample-lnd.conf) and that the default values of `lnd --help` are also mentioned correctly.
`scratch`
---------
Compiles all dependencies and builds the `./lnd` and `./lncli` binaries.
Equivalent to [`lint`](#lint) [`btcd`](#btcd)
[`unit-race`](#unit-race).
`unit`
------
Runs the unit test suite. By default, this will run all known unit tests.
Arguments:
- `pkg=`
- `case=`
- `timeout=`
- `log="stdlog[ ]"` prints logs to stdout
- `` can be `info` (default), `debug`, `trace`, `warn`, `error`, `critical`, or `off`
`unit-cover`
------------
Runs the unit test suite with test coverage, compiling the statistics in
`profile.cov`.
Arguments:
- `pkg=`
- `case=`
- `timeout=`
- `log="stdlog[ ]"` prints logs to stdout
- `` can be `info` (default), `debug`, `trace`, `warn`, `error`, `critical`, or `off`
Related: [`unit`](#unit)
`unit-race`
-----------
Runs the unit test suite with go's race detector.
Arguments:
- `pkg=`
- `case=`
- `timeout=`
- `log="stdlog[ ]"` prints logs to stdout
- `` can be `info` (default), `debug`, `trace`, `warn`, `error`, `critical`, or `off`
Related: [`unit`](#unit)
[btcd]: https://github.com/btcsuite/btcd (github.com/btcsuite/btcd")
[gometalinter]: https://gopkg.in/alecthomas/gometalinter.v1 (gopkg.in/alecthomas/gometalinter.v1)
---
### SphinxReplayDB
# Sphinx Onion Routing in Lightning Network
The Lightning Network uses a Sphinx-based onion message protocol to send
messages across the Lightning Network. These messages have the property that a
node which is part of such a message and for example forwards such an onion
message cannot learn about the destination of this whole packet. It only knows
the predecessor and the successor of the message. In other words, it only knows
where the message came from and where it needs to be forwarded. This makes the
message protocol of the Lightning Network very private. Only the sender
(creator of the whole onion packet) knows the whole route of the packet. Also,
the receiver has no idea from which node the message originated.
This diagram illustrates how Sphinx onion routing works in the Lightning
Network, showing the privacy properties at each hop:
```ascii
Alice (Sender)
|
| Knows Full Route: Alice → Bob → Carol → David → Eve
|
v
Bob (Hop 1)
|
| Knows: Alice → Carol
|
v
Carol (Hop 2)
|
| Knows: Bob → David
|
v
David (Hop 3)
|
| Knows: Carol → Eve
|
v
Eve (Receiver)
|
| Knows: From David
|
v
Privacy Properties:
- Each hop only knows its immediate neighbors
- Receiver doesn't know the original sender
- Only sender knows the complete route
- Each hop peels one layer of the onion
```
## Privacy Properties
1. **Hop Privacy**: Each intermediate node only knows:
- The previous hop (where the packet came from)
- The next hop (where to forward the packet)
- Cannot see the full route or final destination
2. **Sender Privacy**: Only the sender (Alice) knows:
- The complete route
- All intermediate nodes
- The final destination
3. **Receiver Privacy**: The receiver (Eve) only knows:
- The immediate previous hop (David)
- Cannot determine the original sender
4. **Onion Encryption**: Each hop peels one layer of encryption, revealing only
the next hop's information
The detailed mechanics are described in [BOLT 04](https://github.com/lightning/bolts/blob/master/04-onion-routing.md)
## Replaying Onion Packets
The [sphinx protocol](https://cypherpunks.ca/~iang/pubs/Sphinx_Oakland09.pdf)
clearly states that implementations of the protocol should guard against
replay attacks.
Replaying (resending) onion packets into the network can compromise the
privacy guarantees promised by the protocol. So it is crucial for node
participants to not forward replayed onion packets to guard the privacy of all
network participants. Compared to the original Sphinx protocol, the Lightning
Network has an improved replay protection in place especially when it comes to
forwarding HTLCs, which are different from Onion Messages introduced later on
because HTLCs lock a payment to the particular onion packet. Therefore, sending
HTLCs packets comes with a cost. Moreover, every HTLC has an expiry date, also
called CLTV (absolute locktime), which prevents the replay of packets that have
already expired. In addition to the CLTV expiry of a packet, every HTLC onion
packet commits to the payment hash in the HMAC of the message (to be precise,
the associated data), so this prevents an attacker from attaching an old onion
packet to a new payment hash, which now also comes with the risk for the
attacker that he does not only have to lock funds when replaying an onion
packet but he also risks that the next node settles the HTLC because it already
knows the preimage of the HTLC. Although the attack comes with a high cost,
Lightning implementations should prevent replayed onion packets from
propagating through the network to safeguard the privacy for every network
participant.
## Replay attack
An attacker could execute a re-injection attack by strategically positioning
well-connected nodes within the network, ensuring they participate in a high
volume of payment forwarding. In this scenario, the attacker's nodes act as
forwarding intermediaries. These malicious nodes would collect passing onion
packets and concurrently monitor network gossip. Upon observing a successful
payment forward, the attacker could re-inject the captured onion message. By
then monitoring channel updates or closures from the neighboring node, the
attacker aims to determine if that channel was part of the payment path.
Furthermore, if the neighboring node, lacking a suitable outgoing channel,
instead settles the payment (i.e., acts as the final recipient), it becomes
highly probable that it was the ultimate receiving node. It's important to note,
however, that such an attack carries significant costs.
### Replay Protection in LND
In LND there are two interchanging DB names which save information of
those onion packets to prevent replays from happening. They are called
`Sphinx-Replay-DB` or `Decayed-Log-DB`.
Currently LND only implements onion messages which are tied to payments
(i.e. HTLCs), and as explained above, HTLCs expire after an absolute lock time
and therefore can be garbage collected because they will not be forwarded by
nodes anyways as that would entail a risk of losing funds.
Sphinx replay protection storage for the different backends:
1. When running LND with the BBolt backend the db is called: `sphinxreplay.db`.
2. For Postgres the table is called: `decayedlogdb_kv`.
3. For SQLite the table is called `decayedlogdb_kv` and is part of the
`channel.sqlite` file.
LND employs an internal stores to prevent replay attacks. The internal store
maintains a record of each onion packet's shared secret alongside its absolute
timelock (CLTV). This enables LND to efficiently identify and discard replayed
packets that present an already-seen shared secret.
This store is subject to garbage collection, ensuring it does not impose a
sustained burden on memory resources. After the CLTV values expiry, the entries
can be removed as described earlier.
#### What happens if LND encounters a replayed onion HTLC packet?
When LND encounters a replay, it will reject the HTLC and it will not signal a
specific error that a replay occurred which would reveal that the node was
indeed part of the route in the onion packet. LND will reject the HTLC and
respond with the message `invalid_onion_version`. See
[BOLT 04](https://github.com/lightning/bolts/blob/master/04-onion-routing.md)
## Other attacks
In the Lightning Network, the Sphinx protocol effectively mitigates
`tagging attacks.` This is due to the fact that each onion packet incorporates
a cryptographic HMAC (Hash-based Message Authentication Code), which is derived
from the packet's encrypted contents. Consequently, any attempt to tamper with
the packet's data would render its HMAC invalid, causing the packet to be
discarded by honest nodes.
---
### Backport Workflow
# Automated Backport Workflow
This document describes the automated backport workflow for the LND project.
## Table of Contents
1. [Overview](#overview)
2. [How to Use](#how-to-use)
3. [Workflow Triggers](#workflow-triggers)
4. [Label Format](#label-format)
5. [Workflow Steps](#workflow-steps)
6. [Handling Conflicts](#handling-conflicts)
7. [Multiple Backports](#multiple-backports)
8. [Technical Details](#technical-details)
9. [Troubleshooting](#troubleshooting)
## Overview
The automated backport workflow simplifies the process of backporting merged PRs from the `master` branch to release branches (e.g., `v0.20.x-branch`, `v0.19.x-branch`).
Instead of manually creating branches, cherry-picking commits, and creating PRs, maintainers can simply add a label to the master PR, and the workflow handles the rest.
## How to Use
### Basic Usage
1. **Merge a PR to master** (or have it already merged)
2. **Add a backport label** in the format: `backport-v-branch`
- Example: `backport-v0.20.x-branch`
3. **The workflow automatically**:
- Validates the target branch exists
- Cherry-picks the commits
- Creates a new PR targeting the release branch
- Adds the `no-changelog` label (since release notes are in the master PR)
### Example Scenario
```
Day 1, 10:00 - PR #1234 "Fix critical bug" merged to master
Day 1, 10:30 - Add label: backport-v0.20.x-branch
Day 1, 10:31 - Workflow creates PR #1235 automatically
Title: "[v0.20.x-branch] Backport #1234: Fix critical bug"
Base: v0.20.x-branch
Labels: no-changelog
Day 1, 14:00 - Maintainer reviews and merges PR #1235
```
## Workflow Triggers
The backport workflow triggers in two scenarios:
### Scenario 1: Label Before Merge
```
1. Open PR #1234
2. Add label: backport-v0.20.x-branch
3. Review and approve PR
4. Merge PR #1234
5. → Workflow triggers on PR close event
6. → Backport PR #1235 created immediately
```
### Scenario 2: Label After Merge
```
1. Open PR #1234
2. Review, approve, and merge PR #1234
3. Later... decide it needs backporting
4. Add label: backport-v0.20.x-branch
5. → Workflow triggers on label event
6. → Backport PR #1235 created immediately
```
Both scenarios work identically.
## Label Format
### Valid Labels
Labels **must** start with `backport-v` to trigger the workflow:
- ✅ `backport-v0.20.x-branch` → backports to `v0.20.x-branch`
- ✅ `backport-v0.19.x-branch` → backports to `v0.19.x-branch`
- ✅ `backport-v0.18.x-beta-branch` → backports to `v0.18.x-beta-branch`
### Invalid Labels (Will NOT Trigger)
These labels are ignored by the workflow:
- ❌ `backport candidate` - discussion label only
- ❌ `backport-candidate` - doesn't start with `backport-v`
- ❌ `backport-needed` - doesn't start with `backport-v`
- ❌ `needs-backport` - wrong prefix
This allows you to use discussion labels without accidentally triggering backports.
### Label to Branch Mapping
The label format directly maps to the target branch:
```
Label: backport-v0.20.x-branch
↓ (removes "backport-" prefix)
Branch: v0.20.x-branch
```
## Workflow Steps
The workflow executes the following steps when triggered:
### Step 1: Checkout Repository
```yaml
- Fetches the full git history
- Checks out the base branch (usually master)
```
### Step 2: Validate Target Branches
```bash
For each backport label:
1. Extract branch name from label
backport-v0.20.x-branch → v0.20.x-branch
2. Check if branch exists in remote repository
git ls-remote --heads origin v0.20.x-branch
3. If branch doesn't exist:
- Log error message
- Add branch to missing_branches list
4. After checking all labels:
- If any branches are missing → FAIL workflow
- If all branches exist → Continue
```
**Example validation output:**
```
All labels: ["backport-v0.20.x-branch", "bug-fix", "backport-v0.19.x-branch"]
Found backport labels:
backport-v0.20.x-branch
backport-v0.19.x-branch
Checking if branch exists: v0.20.x-branch
✓ Branch 'v0.20.x-branch' exists
Checking if branch exists: v0.19.x-branch
✓ Branch 'v0.19.x-branch' exists
✓ All target branches validated successfully
```
### Step 3: Create Backport PRs
For each valid backport label, the workflow:
1. **Creates a new branch**
- Branch name: `backport--to-`
- Example: `backport-1234-to-v0.20.x-branch`
- Based on: the target release branch
2. **Cherry-picks commits**
- Uses `git cherry-pick` (not merge or rebase)
- Cherry-picks all commits from the original PR
- Preserves commit messages and authorship
- Skips merge commits
3. **Creates a new PR**
- Title: `[v0.20.x-branch] Backport #1234: `
- Base branch: `v0.20.x-branch`
- Head branch: `backport-1234-to-v0.20.x-branch`
- Labels: `no-changelog` (automatically added)
4. **PR Description**
```markdown
Backport of #1234
Original PR: https://github.com/lightningnetwork/lnd/pull/1234
---
[Original PR description here]
```
## Handling Conflicts
The workflow handles merge conflicts gracefully using the `draft_commit_conflicts` strategy.
### When Cherry-pick Succeeds
```
1. Cherry-pick completes cleanly
2. Creates regular PR (ready for review)
3. PR is NOT in draft mode
4. Maintainer can review and merge immediately
```
### When Cherry-pick Has Conflicts
```
1. Cherry-pick encounters conflicts
2. Workflow commits the conflict markers:
<<<<<<< HEAD
[code from release branch]
=======
[code from master PR]
>>>>>>> commit-hash
3. Creates DRAFT PR
4. PR description indicates there were conflicts
5. Manual resolution required:
a. git fetch origin
b. git checkout backport-1234-to-v0.20.x-branch
c. Resolve conflicts in affected files
d. git add
e. git commit -m "Resolve backport conflicts"
f. git push origin backport-1234-to-v0.20.x-branch
g. Mark PR as "Ready for review" in GitHub UI
6. Maintainer reviews and merges
```
### Conflict Resolution Best Practices
- **Review the original PR**: Understand what changed
- **Check the release branch**: Understand why conflicts occurred
- **Test after resolving**: Run tests locally before pushing
- **Update commit message**: Explain what conflicts were resolved and how
- **Request review**: Don't merge without review, even after resolving conflicts
## Multiple Backports
You can backport to multiple release branches simultaneously by adding multiple labels.
### Example: Backport to Two Branches
```
PR #1234 merged with labels:
- backport-v0.20.x-branch
- backport-v0.19.x-branch
Workflow creates TWO backport PRs:
PR #1235:
Title: [v0.20.x-branch] Backport #1234: Original title
Base: v0.20.x-branch
Labels: no-changelog
PR #1236:
Title: [v0.19.x-branch] Backport #1234: Original title
Base: v0.19.x-branch
Labels: no-changelog
```
### Independent Processing
Each backport is processed independently:
- One backport may succeed while another has conflicts
- One backport may fail validation while another succeeds
- Each backport creates a separate branch and PR
- Each backport PR is reviewed and merged independently
### Example with Mixed Results
```
PR #1234 with labels:
- backport-v0.20.x-branch → ✓ Clean cherry-pick, regular PR created
- backport-v0.19.x-branch → ✗ Conflicts, draft PR created
- backport-v0.99.x-branch → ✗ Branch doesn't exist, workflow fails
```
In this case:
1. PR #1235 to v0.20.x-branch is ready for review
2. PR #1236 to v0.19.x-branch needs conflict resolution
3. No PR created for v0.99.x-branch (validation failed)
4. Remove the incorrect label and add the correct one. The workflow will
re-trigger when you add the new label.
## Technical Details
### Workflow File
Location: `.github/workflows/backport.yml`
### Trigger Events
```yaml
on:
pull_request_target:
types: [closed, labeled]
```
- **closed**: Triggers when PR is closed (checks if merged)
- **labeled**: Triggers when any label is added
### Permissions Required
```yaml
permissions:
contents: write # Create branches and commits
pull-requests: write # Create and manage PRs
issues: read # Read PR metadata
```
### Workflow Condition
```yaml
if: |
github.event.pull_request.merged == true &&
contains(join(github.event.pull_request.labels.*.name, ','), 'backport-v')
```
Only runs when:
1. PR is actually merged (not just closed)
2. At least one label contains `backport-v`
### Label Pattern
```yaml
label_pattern: '^backport-(v.+)$'
```
Regex explanation:
- `^` - Start of string
- `backport-` - Literal text
- `(v.+)` - Capture group: "v" followed by one or more characters
- `$` - End of string
Examples:
- `backport-v0.20.x-branch` → matches, captures `v0.20.x-branch`
- `backport-v0.19.x-branch` → matches, captures `v0.19.x-branch`
- `backport-candidate` → doesn't match (no "v" after dash)
### Cherry-pick Strategy
```yaml
merge_commits: skip
```
- Uses `git cherry-pick` for clean history
- Skips merge commits (only cherry-picks actual changes)
- Preserves original commit messages and authorship
- Maintains PGP signatures where present
### Conflict Resolution Strategy
```yaml
experimental: |
conflict_resolution: draft_commit_conflicts
```
- Creates draft PR with conflict markers
- Allows manual resolution
- Preserves all context and metadata
## Troubleshooting
### Problem: Workflow Doesn't Trigger
**Symptoms:**
- Added `backport-v0.20.x-branch` label
- No workflow run appears in Actions tab
**Possible causes:**
1. **Label format is wrong**
- ❌ `backport-0.20.x-branch` (missing "v")
- ✅ `backport-v0.20.x-branch`
2. **PR is not merged**
- Workflow only runs on merged PRs
- Check PR status
3. **Workflow is disabled**
- Check `.github/workflows/backport.yml` exists
- Check workflow is enabled in Settings → Actions
### Problem: Workflow Fails with "Branch doesn't exist"
**Error message:**
```
Error: Target branch 'v0.21.x-branch' does not exist (from label 'backport-v0.21.x-branch')
Error: The following target branches do not exist: v0.21.x-branch
Error: Please ensure the branch exists before adding the backport label
```
**Solution:**
1. **Verify branch name:**
```bash
git ls-remote --heads origin | grep v0.21
```
2. **Check available release branches:**
```bash
git branch -r | grep origin/v0 | grep -v fork
```
3. **Fix the label:**
- Remove incorrect label
- Add correct label with existing branch name
### Problem: Cherry-pick Has Conflicts
**Symptoms:**
- Backport PR created as DRAFT
- PR description mentions conflicts
- Branch has files with conflict markers
**Solution:**
1. **Fetch and checkout the branch:**
```bash
git fetch origin
git checkout backport-1234-to-v0.20.x-branch
```
2. **Find conflicted files:**
```bash
grep -r "<<<<<<< HEAD" .
```
3. **Resolve each conflict:**
- Open the file in an editor
- Review both versions:
```
<<<<<<< HEAD
[Release branch version]
=======
[Master PR version]
>>>>>>> commit-hash
```
- Choose the correct code or merge both
- Remove conflict markers
4. **Commit the resolution:**
```bash
git add
git commit -m "Resolve backport conflicts for PR #1234
Conflicts occurred due to [explanation].
Resolution: [describe what you did]"
git push origin backport-1234-to-v0.20.x-branch
```
5. **Mark PR ready for review:**
- Go to PR on GitHub
- Click "Ready for review"
### Problem: Multiple Labels but Only One Backport Created
**Symptoms:**
- Added `backport-v0.20.x-branch` and `backport-v0.19.x-branch`
- Only one PR created
**Possible causes:**
1. **One label format is wrong**
- Check both labels start with `backport-v`
- Fix incorrect label, workflow will retry
2. **One branch doesn't exist**
- Check workflow logs for validation errors
- Verify both branches exist
3. **Workflow still running**
- Check Actions tab for in-progress runs
- Wait for workflow to complete
### Problem: Backport PR Missing `no-changelog` Label
**Symptoms:**
- Backport PR created successfully
- CI fails on changelog check
**Solution:**
1. **Manually add the label:**
- Add `no-changelog` label to the backport PR
2. **Check workflow configuration:**
- Verify `.github/workflows/backport.yml` has:
```yaml
add_labels: no-changelog
```
3. **Re-run the workflow:**
- Remove and re-add the backport label on original PR
- New backport PR will have correct label
### Getting Help
If you encounter issues not covered here:
1. **Check workflow logs:**
- Go to Actions tab
- Click on the failed workflow run
- Review step-by-step logs
2. **Check workflow file:**
- `.github/workflows/backport.yml`
- Verify configuration matches this documentation
3. **Manual backport:**
- If automated backport fails repeatedly, you can backport manually:
```bash
# Create branch from target release branch
git checkout v0.20.x-branch
git checkout -b manual-backport-123-to-v0.20.x
# Cherry-pick commits from the original PR
git cherry-pick
# Resolve any conflicts, then:
git add .
git commit
git push origin manual-backport-123-to-v0.20.x
# Create PR targeting v0.20.x-branch with no-changelog label
gh pr create --base v0.20.x-branch --label no-changelog
```
4. **Report issues:**
- If you find a bug in the workflow
- Open an issue with workflow logs and details
---
### Benchmark Perf Loop
# The Go Performance Optimization Loop: From Benchmarks to Zero Allocations
When optimizing Go code for performance, particularly in hot paths like
cryptographic operations or protocol handling, the journey from identifying
bottlenecks to achieving zero-allocation code follows a well-defined
methodology. This document walks through the complete optimization loop using
Go's built-in tooling, demonstrating how to systematically eliminate allocations
and improve performance.
## Understanding the Performance Baseline
The first step in any optimization effort is establishing a measurable baseline.
Go's benchmark framework provides the foundation for this measurement. When
writing benchmarks for allocation-sensitive code, always include a call to
`b.ReportAllocs()` before `b.ResetTimer()`. This ensures the benchmark reports
both timing and allocation statistics without including setup costs in the
measurements.
Consider a benchmark that exercises a cryptographic write path with the largest
possible message size to stress test allocations:
```go
func BenchmarkWriteMessage(b *testing.B) {
// Setup code here...
b.ReportAllocs() // Essential for tracking allocations
b.ResetTimer()
for i := 0; i < b.N; i++ {
// Hot path being measured
}
}
```
Running the benchmark with `go test -bench=BenchmarkWriteMessage -benchmem
-count=10` provides statistical confidence through multiple runs. The
`-benchmem` flag is redundant if you've called `b.ReportAllocs()`, but it
doesn't hurt to include it explicitly. The output reveals three critical
metrics: nanoseconds per operation, bytes allocated per operation, and the
number of distinct allocations per operation.
## Profiling Memory Allocations
Once you have baseline measurements showing undesirable allocations, the next
phase involves profiling to understand where these allocations originate.
Generate memory profiles during benchmark execution using:
```
go test -bench=BenchmarkWriteMessage -memprofile=mem.prof -cpuprofile=cpu.prof -count=1
```
The resulting profile can be analyzed through several lenses. To see which
functions allocate the most memory by total bytes, use:
`go tool pprof -alloc_space -top mem.prof`.
However, for understanding allocation frequency rather than size, `go tool pprof -alloc_objects -top mem.prof` often provides more actionable insights, especially when hunting small but frequent allocations.
Here's what the allocation object analysis might reveal:
```
$ go tool pprof -alloc_objects -top mem.prof | head -20
File: brontide.test
Type: alloc_objects
Time: Aug 30, 2024 at 2:07pm (WEST)
Showing nodes accounting for 39254, 100% of 39272 total
Dropped 32 nodes (cum <= 196)
flat flat% sum% cum cum%
32768 83.44% 83.44% 32768 83.44% github.com/lightningnetwork/lnd/brontide.(*cipherState).Encrypt
5461 13.91% 97.34% 5461 13.91% runtime.acquireSudog
1025 2.61% 100% 1025 2.61% runtime.allocm
```
This output immediately shows that `cipherState.Encrypt` is responsible for 83%
of allocations by count, focusing our investigation.
The most powerful profiling technique involves examining allocations at the
source line level. Running `go tool pprof -list 'FunctionName' mem.prof` shows
exactly which lines within a function trigger heap allocations:
```
$ go tool pprof -list 'cipherState.*Encrypt' mem.prof
Total: 8.73MB
ROUTINE ======================== github.com/lightningnetwork/lnd/brontide.(*cipherState).Encrypt
512.01kB 512.01kB (flat, cum) 5.73% of Total
. . 111:func (c *cipherState) Encrypt(associatedData, cipherText, plainText []byte) []byte {
. . 112: defer func() {
. . 113: c.nonce++
. . 114:
. . 115: if c.nonce == keyRotationInterval {
. . 116: c.rotateKey()
. . 117: }
. . 118: }()
. . 119:
512.01kB 512.01kB 120: var nonce [12]byte
. . 121: binary.LittleEndian.PutUint64(nonce[4:], c.nonce)
. . 122:
. . 123: return c.cipher.Seal(cipherText, nonce[:], plainText, associatedData)
```
This granular view reveals that line 120, a seemingly innocent stack array
declaration, is allocating 512KB total across all benchmark iterations.
## CPU Profiling for Hot Spots
While memory allocations often dominate optimization efforts, CPU profiling
reveals where computational time is spent. The CPU profile generated alongside
the memory profile provides complementary insights:
```
$ go tool pprof -top cpu.prof | head -15
File: brontide.test
Type: cpu
Time: Aug 30, 2024 at 2:07pm (WEST)
Duration: 1.8s, Total samples = 1.71s (94.40%)
Showing nodes accounting for 1.65s, 96.49% of 1.71s total
flat flat% sum% cum cum%
0.51s 29.82% 29.82% 0.51s 29.82% vendor/golang.org/x/crypto/chacha20poly1305.(*chacha20poly1305).sealGeneric
0.28s 16.37% 46.20% 0.28s 16.37% vendor/golang.org/x/crypto/internal/poly1305.updateGeneric
0.24s 14.04% 60.23% 0.24s 14.04% vendor/golang.org/x/crypto/chacha20.(*Cipher).XORKeyStream
0.19s 11.11% 71.35% 0.19s 11.11% runtime.memmove
0.12s 7.02% 78.36% 0.86s 50.29% github.com/lightningnetwork/lnd/brontide.(*cipherState).Encrypt
```
This profile shows that cryptographic operations dominate CPU usage, which is
expected. However, note the presence of `runtime.memmove` at 11% - this often
indicates unnecessary copying that could be eliminated through careful buffer
management.
For line-level CPU analysis of a specific function:
```
$ go tool pprof -list 'WriteMessage' cpu.prof
Total: 1.71s
ROUTINE ======================== github.com/lightningnetwork/lnd/brontide.(*Machine).WriteMessage
10ms 1.21s (flat, cum) 70.76% of Total
. . 734:func (b *Machine) WriteMessage(p []byte) error {
. . 735: if len(p) > math.MaxUint16 {
. . 736: return ErrMaxMessageLengthExceeded
. . 737: }
. . 738:
. 10ms 739: if len(b.nextHeaderSend) > 0 || len(b.nextBodySend) > 0 {
. . 740: return ErrMessageNotFlushed
. . 741: }
. . 742:
10ms 10ms 743: fullLength := uint16(len(p))
. . 744: var pktLen [2]byte
. 10ms 745: binary.BigEndian.PutUint16(pktLen[:], fullLength)
. . 746:
. 580ms 747: b.nextHeaderSend = b.sendCipher.Encrypt(nil, nil, pktLen[:])
. 600ms 748: b.nextBodySend = b.sendCipher.Encrypt(nil, nil, p)
```
This shows that the two `Encrypt` calls consume virtually all the CPU time in
`WriteMessage`, confirming that cryptographic operations are the bottleneck
rather than the message handling logic itself.
## Understanding Escape Analysis
When the profiler indicates that seemingly stack-local variables are being heap
allocated, escape analysis becomes your next investigative tool. The Go
compiler's escape analysis determines whether variables can remain on the stack
or must be moved to the heap. Variables escape to the heap when their lifetime
extends beyond the function that creates them or when the compiler cannot prove
they remain local.
To see the compiler's escape analysis decisions, build with verbose flags:
```
go build -gcflags="-m" ./...
```
For more detailed output including the reasons for escape, use `-m=2`. The
output reveals escape flows, showing exactly why variables move to the heap.
When investigating specific escapes, you can grep for the variable in question:
```
$ go build -gcflags="-m=2" ./... 2>&1 | grep -A2 -B2 "nonce escapes"
./noise.go:183:17: &errors.errorString{...} does not escape
./noise.go:183:17: new(chacha20poly1305.chacha20poly1305) escapes to heap
./noise.go:120:6: nonce escapes to heap:
./noise.go:120:6: flow: {heap} = &nonce:
./noise.go:120:6: from nonce (address-of) at ./noise.go:123:40
--
./noise.go:469:21: &keychain.PrivKeyECDH{...} escapes to heap
./noise.go:483:40: []byte{} escapes to heap
./noise.go:138:6: nonce escapes to heap:
./noise.go:138:6: flow: {heap} = &nonce:
./noise.go:138:6: from nonce (address-of) at ./noise.go:141:39
```
This output shows the exact flow analysis: the nonce array escapes because its
address is taken when creating a slice (`nonce[:]`) and passed to a function
that the compiler cannot fully analyze.
Common causes include passing pointers to interfaces, storing references in
heap-allocated structures, or passing slices of stack arrays to functions that
might retain them. A particularly instructive example is the seemingly innocent
pattern of passing a stack array to a function:
```go
var nonce [12]byte
binary.LittleEndian.PutUint64(nonce[4:], counter)
return cipher.Seal(ciphertext, nonce[:], plaintext, nil)
```
Here, `nonce[:]` creates a slice backed by the stack array, but if the compiler
cannot prove that `cipher.Seal` won't retain a reference to this slice, the
entire array escapes to the heap.
## The Optimization Strategy
Armed with profiling data and escape analysis insights, the optimization phase
begins. The general strategy for eliminating allocations follows a predictable
pattern: move temporary buffers from function scope to longer-lived structures,
typically as fields in the enclosing type. This transformation changes
allocation from per-operation to per-instance.
For the nonce example above, the optimization involves adding a buffer field to
the containing struct:
```go
type cipherState struct {
// ... other fields ...
nonceBuffer [12]byte // Reusable buffer to avoid allocations
}
func (c *cipherState) Encrypt(...) []byte {
binary.LittleEndian.PutUint64(c.nonceBuffer[4:], c.nonce)
return c.cipher.Seal(ciphertext, c.nonceBuffer[:], plaintext, nil)
}
```
This pattern extends to any temporary buffer. When dealing with variable-sized
data up to a known maximum, pre-allocate buffers at that maximum size and slice
into them as needed. The key insight is using the three-index slice notation to
control capacity separately from length:
```go
// Pre-allocated: var buffer [maxSize]byte
// Creating a zero-length slice with full capacity for append:
slice := buffer[:0] // length=0, capacity=maxSize
```
## Verification and Iteration
After implementing optimizations, the cycle returns to benchmarking. Run the
same benchmark to measure improvement, but don't stop at the aggregate numbers.
Generate new profiles to verify that specific allocations have been eliminated
and to identify any remaining allocation sites.
The benchstat tool provides statistical comparison between runs:
```
go test -bench=BenchmarkWriteMessage -count=10 > old.txt
# Make optimizations
go test -bench=BenchmarkWriteMessage -count=10 > new.txt
benchstat old.txt new.txt
```
This comparison reveals not just whether performance improved, but whether the
improvement is statistically significant. A typical benchstat output after
successful optimization looks like:
```
goos: darwin
goarch: arm64
pkg: github.com/lightningnetwork/lnd/brontide
cpu: Apple M4 Max
│ old.txt │ new.txt │
│ sec/op │ sec/op vs base │
WriteMessage-16 50.34µ ± 1% 46.48µ ± 0% -7.68% (p=0.000 n=10)
│ old.txt │ new.txt │
│ B/op │ B/op vs base │
WriteMessage-16 73788.000 ± 0% 2.000 ± 0% -100.00% (p=0.000 n=10)
│ old.txt │ new.txt │
│ allocs/op │ allocs/op vs base │
WriteMessage-16 5.000 ± 0% 0.000 ± 0% -100.00% (p=0.000 n=10)
```
The key metrics to examine are:
- The percentage change (vs base column) showing the magnitude of improvement
- The p-value (p=0.000) indicating statistical significance - values below 0.05
suggest real improvements rather than noise
- The variance (± percentages) showing consistency across runs
This output confirms both a 7.68% speed improvement and complete elimination of
allocations, with high statistical confidence.
If allocations remain, the cycle continues. Profile again, identify the source,
understand why the allocation occurs through escape analysis, and apply the
appropriate optimization pattern. Each iteration should show measurable progress
toward the goal of zero allocations in the hot path.
## Advanced Techniques
When standard profiling doesn't reveal the allocation source, more advanced
techniques come into play. Memory profiling with different granularities can
help. Instead of looking at total allocations, examine the profile with `go tool
pprof -sample_index=alloc_objects` to focus on allocation count rather than
size. This distinction matters when hunting for small, frequent allocations that
might not show up prominently in byte-focused views.
Additional pprof commands that prove invaluable during optimization:
```bash
# Interactive mode for exploring the profile
go tool pprof mem.prof
(pprof) top10 # Show top 10 memory consumers
(pprof) list regexp # List functions matching regexp
(pprof) web # Open visual graph in browser
# Generate a flame graph for visual analysis
go tool pprof -http=:8080 mem.prof
# Compare two profiles directly
go tool pprof -base=old.prof new.prof
# Show allocations only from specific packages
go tool pprof -focus=github.com/lightningnetwork/lnd/brontide mem.prof
# Check for specific small allocations
go tool pprof -alloc_space -inuse_space mem.prof
```
When dealing with elusive allocations, checking what might be escaping to heap
can be done more surgically:
```bash
# Check specific function or type for escapes
go build -gcflags="-m" 2>&1 | grep -E "(YourType|yourFunc)"
# See all heap allocations in a package
go build -gcflags="-m" 2>&1 | grep "moved to heap"
# Check which variables are confirmed to stay on the stack
go build -gcflags="-m=2" 2>&1 | grep "does not escape"
```
For particularly elusive allocations, instrumenting the code with runtime memory
statistics can provide real-time feedback:
```go
var m runtime.MemStats
runtime.ReadMemStats(&m)
before := m.Alloc
// Operation being measured
runtime.ReadMemStats(&m)
allocated := m.Alloc - before
```
While this approach adds overhead and shouldn't be used in production, it can
help isolate allocations to specific code sections during development.
## The Zero-Allocation Goal
Achieving zero allocations in hot paths represents more than just a performance
optimization. It provides predictable latency, reduces garbage collection
pressure, and improves overall system behavior under load. In systems handling
thousands of operations per second, the difference between five allocations per
operation and zero can mean the difference between smooth operation and periodic
latency spikes during garbage collection.
The journey from initial benchmark to zero-allocation code demonstrates the
power of Go's built-in tooling. By systematically applying the
benchmark-profile-optimize loop, even complex code paths can be transformed into
allocation-free implementations. The key lies not in guessing or premature
optimization, but in measuring, understanding, and methodically addressing each
allocation source.
It's best to focus optimization efforts on true hot paths identified through
production profiling or realistic load testing. The techniques described here
provide the tools to achieve zero-allocation code when it matters, but the
judgment of when to apply them remains a critical engineering decision.
---
### Code Contribution Guidelines
# Table of Contents
1. [Overview](#overview)
2. [Minimum Recommended Skillset](#minimum-recommended-skillset)
3. [Required Reading](#required-reading)
4. [Substantial contributions only](#substantial-contributions-only)
5. [New Contributors](#new-contributors)
6. [Development Practices](#development-practices)
1. [Share Early, Share Often](#share-early-share-often)
1. [Development Guidelines](#development-guidelines)
7. [Code Approval Process](#code-approval-process)
1. [Code Review](#code-review)
1. [Rework Code (if needed)](#rework-code-if-needed)
1. [Acceptance](#acceptance)
1. [Backporting Changes](#backporting-changes)
1. [Review Bot](#review-bot)
8. [Contribution Standards](#contribution-standards)
1. [Contribution Checklist](#contribution-checklist)
1. [Licensing of Contributions](#licensing-of-contributions)
# Overview
Developing cryptocurrencies is an exciting endeavor that touches a wide variety
of areas such as wire protocols, peer-to-peer networking, databases,
cryptography, language interpretation (transaction scripts), adversarial
threat-modeling, and RPC systems. They also represent a radical shift to the
current monetary system and as a result provide an opportunity to help reshape
the entire financial system. With the advent of the [Lightning Network
(LN)](https://lightning.network/), new layers are being constructed upon the
base blockchain layer which have the potential to alleviate many of the
limitations and constraints inherent in the design of blockchains. There are
few projects that offer this level of diversity and impact all in one code
base.
However, as exciting as it is, one must keep in mind that cryptocurrencies
represent real money and introducing bugs and security vulnerabilities can have
far more dire consequences than in typical projects where having a small bug is
minimal by comparison. In the world of cryptocurrencies, even the smallest bug
in the wrong area can cost people a significant amount of money. For this
reason, the Lightning Network Daemon (`lnd`) has a formalized and rigorous
development process (heavily inspired by
[btcsuite](https://github.com/btcsuite)) which is outlined on this page.
We highly encourage code contributions, however it is imperative that you adhere
to the guidelines established on this page.
# Minimum Recommended Skillset
The following list is a set of core competencies that we recommend you possess
before you really start attempting to contribute code to the project. These are
not hard requirements as we will gladly accept code contributions as long as
they follow the guidelines set forth on this page. That said, if you don't have
the following basic qualifications you will likely find it quite difficult to
contribute to the core layers of Lightning. However, there are still a number
of low-hanging fruit which can be tackled without having full competency in the
areas mentioned below.
- A reasonable understanding of bitcoin at a high level (see the
[Required Reading](#required-reading) section for the original white paper)
- A reasonable understanding of the Lightning Network at a high level
- Experience in some type of C-like language
- An understanding of data structures and their performance implications
- Familiarity with unit testing
- Debugging experience
- Ability to understand not only the area you are making a change in, but also
the code your change relies on, and the code which relies on your changed code
Building on top of those core competencies, the recommended skill set largely
depends on the specific areas you are looking to contribute to. For example,
if you wish to contribute to the cryptography code, you should have a good
understanding of the various aspects involved with cryptography such as the
security and performance implications.
# Required Reading
- [Effective Go](https://golang.org/doc/effective_go.html) - The entire `lnd`
project follows the guidelines in this document. For your code to be accepted,
it must follow the guidelines therein.
- [Original Satoshi Whitepaper](https://bitcoin.org/bitcoin.pdf) - This is the white paper that started it all. Having a solid
foundation to build on will make the code much more comprehensible.
- [Lightning Network Whitepaper](https://lightning.network/lightning-network-paper.pdf) - This is the white paper that kicked off the Layer 2 revolution. Having a good grasp of the concepts of Lightning will make the core logic within the daemon much more comprehensible: Bitcoin Script, off-chain blockchain protocols, payment channels, bidirectional payment channels, relative and absolute time-locks, commitment state revocations, and Segregated Witness.
- The original LN was written for a rather narrow audience, the paper may be a bit unapproachable to many. Thanks to the Bitcoin community, there exist many easily accessible supplemental resources which can help one see how all the pieces fit together from double-spend protection all the way up to commitment state transitions and Hash Time Locked Contracts (HTLCs):
- [Lightning Network Summary](https://lightning.network/lightning-network-summary.pdf)
- [Understanding the Lightning Network 3-Part series](https://bitcoinmagazine.com/technical/understanding-the-lightning-network-part-building-a-bidirectional-payment-channel-1464710791)
- [Deployable Lightning](https://github.com/ElementsProject/lightning/blob/master/doc/miscellaneous/deployable-lightning.pdf)
Note that the core design of the Lightning Network has shifted over time as
concrete implementation and design has expanded our knowledge beyond the
original white paper. Therefore, specific information outlined in the resources
above may be a bit out of date. Many implementers are currently working on an
initial [Lightning Network Specifications](https://github.com/lightningnetwork/lightning-rfc).
Once the specification is finalized, it will be the most up-to-date
comprehensive document explaining the Lightning Network. As a result, it will
be recommended for newcomers to read first in order to get up to speed.
# Substantial contributions only
Due to the prevalence of automated analysis and pull request authoring tools
and online competitions that incentivize creating commits in popular
repositories, the maintainers of this project are flooded with trivial pull
requests that only change some typos or other insubstantial content (e.g. the
year in the license file).
If you are an honest user that wants to contribute to this project, please
consider that every pull request takes precious time from the maintainers to
review and consider the impact of changes. Time that could be spent writing
features or fixing bugs.
If you really want to contribute, [consider reviewing and testing other users'
pull requests instead](review.md).
First-time reviewer friendly [pull requests can be found
here](https://github.com/lightningnetwork/lnd/pulls?q=is%3Aopen+is%3Apr+label%3A%22good+first+review%22).
Once you are familiar with the project's code style, testing and review
procedure, your own pull requests will likely require less guidance and fewer
maintainer review cycles, resulting in potentially faster merges.
Also, consider increasing the test coverage of the code by writing more unit
tests first, which is also a very valuable way to contribute and learn more
about the code base.
# New Contributors
Due to the ubiquity of LLM coding tools, pull requests
from new contributors are not prioritized for review. If you're a
new contributor with aspirations to contribute to LND, we recommend starting
with issue triage and PR reviews. These are a better avenue to demonstrate
your knowledge and desire to contribute to open source than new code in this
era of easy AI-assisted code generation. This should also help build a track
record that makes your future PRs easier to prioritize.
If you spot a glaring issue, we may still merge the fix or take it over
ourselves. And if you're a new developer who notices an issue with the code,
consider opening a detailed issue instead of a PR.
# Development Practices
Developers are expected to work in their own trees and submit pull requests when
they feel their feature or bug fix is ready for integration into the master
branch.
## Share Early, Share Often
We firmly believe in the share early, share often approach. The basic premise
of the approach is to announce your plans **before** you start work, and once
you have started working, craft your changes into a stream of small and easily
reviewable commits.
This approach has several benefits:
- Announcing your plans to work on a feature **before** you begin work avoids
duplicate work
- It permits discussions which can help you achieve your goals in a way that is
consistent with the existing architecture
- It minimizes the chances of you spending time and energy on a change that
might not fit with the consensus of the community or existing architecture and
potentially be rejected as a result
- The quicker your changes are merged to master, the less time you will need to
spend rebasing and otherwise trying to keep up with the main code base
## Development Guidelines
The `lnd` project emphasizes code readability and maintainability through
specific development guidelines. Key aspects include: thorough code
documentation with clear function comments and meaningful inline comments;
consistent code spacing to separate logical blocks; adherence to an 80-character
line limit with specific rules for wrapping function calls, definitions, and log
messages (including structured logging); comprehensive unit and integration
testing for all changes; well-structured Git commit messages with package
prefixes and atomic commits; signing Git commits; proper handling of Go module
dependencies and submodules; and appropriate use of log levels. Developers are
encouraged to configure their editors to align with these standards.
For the complete set of rules and examples, please refer to the detailed
[Development Guidelines](development_guidelines.md).
# Code Approval Process
This section describes the code approval process that is used for code
contributions. This is how to get your changes into `lnd`.
## Code Review
All code which is submitted will need to be reviewed before inclusion into the
master branch. This process is performed by the project maintainers and usually
other committers who are interested in the area you are working in as well.
### Code Review Timeframe
The timeframe for a code review will vary greatly depending on factors such as
the number of other pull requests which need to be reviewed, the size and
complexity of the contribution, how well you followed the guidelines presented
on this page, and how easy it is for the reviewers to digest your commits. For
example, if you make one monolithic commit that makes sweeping changes to things
in multiple subsystems, it will obviously take much longer to review. You will
also likely be asked to split the commit into several smaller, and hence more
manageable, commits.
Keeping the above in mind, most small changes will be reviewed within a few
days, while large or far-reaching changes may take weeks. This is a good reason
to stick with the [Share Early, Share Often](#share-early-share-often)
development practice outlined above.
### What is the review looking for?
The review is mainly ensuring the code follows the
[Development Practices](#development-practices) and
[Code Contribution Standards](#contribution-standards). However, there are a few
other checks which are generally performed as follows:
- The code is stable and has no stability or security concerns
- The code is properly using existing APIs and generally fits well into the
overall architecture
- The change is not something which is deemed inappropriate by community
consensus
## Rework Code (if needed)
After the code review, the change will be accepted immediately if no issues are
found. If there are any concerns or questions, you will be provided with
feedback along with the next steps needed to get your contribution merged with
master. In certain cases the code reviewer(s) or interested committers may help
you rework the code, but generally you will simply be given feedback for you to
make the necessary changes.
During the process of responding to review comments, we prefer that changes be
made with [fixup commits](https://robots.thoughtbot.com/autosquashing-git-commits).
The reason for this is twofold: it makes it easier for the reviewer to see
what changes have been made between versions (since Github doesn't easily show
prior versions like Critique) and it makes it easier on the PR author as they
can set it to auto squash the fix up commits on rebase.
This process will continue until the code is finally accepted.
## Acceptance
Before your code is accepted, the [release notes we keep in-tree for the next
upcoming milestone should be extended to describe the changes contained in your
PR](https://github.com/lightningnetwork/lnd/tree/master/docs/release-notes).
Unless otherwise mentioned by the reviewers of your PR, the description of your
changes should live in the document set for the _next_ major release.
Once your code is accepted, it will be integrated with the master branch. After
2+ (sometimes 1) LGTM's (approvals) are given on a PR, it's eligible to land in
master. At this final phase, it may be necessary to rebase the PR in order to
resolve any conflicts and also squash fix up commits. Ideally, the set of
[commits by new contributors are PGP signed](https://git-scm.com/book/en/v2/Git-Tools-Signing-Your-Work),
although this isn't a strong requirement (but we prefer it!). In order to keep
these signatures intact, we prefer using merge commits. PR proposers can use
`git rebase --signoff` to sign and rebase at the same time as a final step.
Rejoice as you will now be listed as a [contributor](https://github.com/lightningnetwork/lnd/graphs/contributors)!
## Backporting Changes
After a PR is merged to master, it may need to be backported to release branches
(e.g., `v0.20.x-branch`) to include the fix or feature in upcoming patch releases.
The project uses an **automated backport workflow** to simplify this process. Simply
add a label like `backport-v0.20.x-branch` to your merged PR, and a GitHub Action
will automatically create a backport PR for you.
For complete documentation on the automated backport workflow, including:
- How to use backport labels
- Handling merge conflicts
- Multiple backports
- Troubleshooting
See [backport-workflow.md](backport-workflow.md)
## Review Bot
In order to keep the review flow going, Lightning Labs uses a bot to remind
PR reviewers about their outstanding reviews or to remind authors to address
recent reviews. Here are some important things to know about the bot and some
controls for adjusting its behaviour:
####🤖 Expected Behaviour:
- The bot will not do anything if your PR is in draft mode.
- It will ping a pending reviewer if they have not reviewed or commented on the
PR in x days since the last update or the last time the bot pinged them.
(default x = 3)
- It will ping the author of the PR if they have not addressed a review on a PR
after x days since last review or the last time the bot pinged them. It will
also ping them to remind them to re-request review if needed. (default x = 3)
####🤖 Controls:
To control the bot, you need to add a comment on the PR starting with
`!lightninglabs-deploy` followed by the command. There are 2 control types:
mute/unmute & cadence. Only the latest comment for each control type will be
used. This also means you don't need to keep adding new control comments, just
edit the latest comment for that control type.
- `!lightninglabs-deploy mute` will mute the bot on the PR completely.
- `!lightninglabs-deploy mute 72h30m` will mute the bot for the given duration.
- `!lightninglabs-deploy mute 2022-Feb-02` will mute the bot until the given
date (must be in this format!).
- `!lightninglabs-deploy mute #4` will mute the bot until the given PR of the
same repo has been merged.
- `!lightninglabs-deploy unmute` will unmute the bot (or just delete the comment
that was muting it)
- `!lightninglabs-deploy cadence 60h` change the cadence of the bot from the
default of 3 days to the given duration.
- it will auto-mute if the PR is in Draft mode
# Contribution Standards
## Contribution Checklist
See [template](https://github.com/lightningnetwork/lnd/blob/master/.github/pull_request_template.md).
## Licensing of Contributions
****
All contributions must be licensed with the
[MIT license](https://github.com/lightningnetwork/lnd/blob/master/LICENSE). This is
the same license as all of the code found within lnd.
# Acknowledgements
This document was heavily inspired by a [similar document outlining the code
contribution](https://github.com/btcsuite/btcd/blob/master/docs/code_contribution_guidelines.md)
guidelines for btcd.
---
### Configuring Tor
# Table of Contents
1. [Overview](#overview)
2. [Getting Started](#getting-started)
3. [Tor Stream Isolation](#tor-stream-isolation)
4. [Authentication](#authentication)
5. [Listening for Inbound Connections](#listening-for-inbound-connections)
## Overview
`lnd` currently has complete support for using Lightning over
[Tor](https://www.torproject.org/). Usage of Lightning over Tor is valuable as
routing nodes no longer need to potentially expose their location via their
advertised IP address. Additionally, leaf nodes can also protect their location
by using Tor for anonymous networking to establish connections.
With widespread usage of Onion Services within the network, concerns about the
difficulty of proper NAT traversal are alleviated, as usage of onion services
allows nodes to accept inbound connections even if they're behind a NAT. `lnd`
supports v3 onion services only; legacy v2 onion service support has been
removed.
Before following the remainder of this documentation, you should ensure that
you already have Tor installed locally. **Make sure that you run at least
version 0.3.3.6 of Tor in order to use v3 Onion Services.**
Official instructions to install the latest release of Tor can be found
[here](https://www.torproject.org/docs/tor-doc-unix.html.en).
**NOTE**: This documentation covers how to ensure that `lnd`'s _Lightning
protocol traffic_ is tunneled over Tor. Users must ensure that when also running
a Bitcoin full-node, that it is also proxying all traffic over Tor. If using the
`neutrino` backend for `lnd`, then it will automatically also default to Tor
usage if active within `lnd`.
## Getting Started
First, you'll want to run `tor` locally before starting up `lnd`. Depending on
how you installed Tor, you'll find the configuration file at
`/usr/local/etc/tor/torrc`. Here's an example configuration file that we'll be
using for the remainder of the tutorial:
```text
SOCKSPort 9050
Log notice stdout
ControlPort 9051
CookieAuthentication 1
```
With the configuration file created, you'll then want to start the Tor daemon:
```shell
$ tor
Feb 05 17:02:06.501 [notice] Tor 0.3.1.8 (git-ad5027f7dc790624) running on Darwin with Libevent 2.1.8-stable, OpenSSL 1.0.2l, Zlib 1.2.8, Liblzma N/A, and Libzstd N/A.
Feb 05 17:02:06.502 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
Feb 05 17:02:06.502 [notice] Read configuration file "/usr/local/etc/tor/torrc".
Feb 05 17:02:06.506 [notice] Opening Socks listener on 127.0.0.1:9050
Feb 05 17:02:06.506 [notice] Opening Control listener on 127.0.0.1:9051
```
Once the `tor` daemon has started and it has finished bootstrapping, you'll see this in the logs:
```text
Feb 05 17:02:06.000 [notice] Bootstrapped 0%: Starting
Feb 05 17:02:07.000 [notice] Starting with guard context "default"
Feb 05 17:02:07.000 [notice] Bootstrapped 80%: Connecting to the Tor network
Feb 05 17:02:07.000 [notice] Bootstrapped 85%: Finishing handshake with first hop
Feb 05 17:02:08.000 [notice] Bootstrapped 90%: Establishing a Tor circuit
Feb 05 17:02:11.000 [notice] Tor has successfully opened a circuit. Looks like client functionality is working.
Feb 05 17:02:11.000 [notice] Bootstrapped 100%: Done
```
This indicates the daemon is fully bootstrapped and ready to proxy connections.
At this point, we can now start `lnd` with the relevant arguments:
```shell
$ ./lnd -h
Tor:
--tor.active Allow outbound and inbound connections to be routed through Tor
--tor.socks= The host:port that Tor's exposed SOCKS5 proxy is listening on (default: localhost:9050)
--tor.dns= The DNS server as host:port that Tor will use for SRV queries - NOTE must have TCP resolution enabled (default: soa.nodes.lightning.directory:53)
--tor.streamisolation Enable Tor stream isolation by randomizing user credentials for each connection.
--tor.control= The host:port that Tor is listening on for Tor control connections (default: localhost:9051)
--tor.targetipaddress= IP address that Tor should use as the target of the hidden service
--tor.password= The password used to arrive at the HashedControlPassword for the control port. If provided, the HASHEDPASSWORD authentication method will be used instead of the SAFECOOKIE one.
--tor.v3 Automatically set up a v3 onion service to listen for inbound connections
--tor.privatekeypath= The path to the private key of the onion service being created
```
There are a couple of things here, so let's dissect them. The `--tor.active`
flag allows `lnd` to route all outbound and inbound connections through Tor.
Outbound connections are possible with the use of the `--tor.socks` and
`--tor.dns` arguments. The `--tor.socks` argument should point to the interface
that the `Tor` daemon is listening on to proxy connections. The `--tor.dns` flag
is required in order to be able to properly automatically bootstrap a set of
peer connections. The `tor` daemon doesn't currently support proxying `SRV`
queries over Tor. So instead, we need to connect directly to the authoritative
DNS server over TCP, in order query for `SRV` records that we can use to
bootstrap our connections.
Inbound connections are possible due to `lnd` automatically creating an onion
service. A path to save the onion service's private key can be specified with
the `--tor.privatekeypath` flag.
Most of these arguments have defaults, so as long as they apply to you, routing
all outbound and inbound connections through Tor can simply be done with v3
onion services:
```shell
$ ./lnd --tor.active --tor.v3
```
See [Listening for Inbound Connections](#listening-for-inbound-connections) for
more info about allowing inbound connections via Tor.
Outbound support only can also be used with:
```shell
$ ./lnd --tor.active
```
This will allow you to make all outgoing connections over Tor. Listening is
disabled to prevent inadvertent leaks.
## Tor Stream Isolation
Our support for Tor also has an additional privacy enhancing modified: stream
isolation. Usage of this mode means that Tor will always use _new circuit_ for
each connection. This added features means that it's harder to correlate
connections. As otherwise, several applications using Tor might share the same
circuit.
Activating stream isolation is very straightforward, we only require the
specification of an additional argument:
```shell
$ ./lnd --tor.active --tor.streamisolation
```
## Authentication
In order for `lnd` to communicate with the Tor daemon securely, it must first
establish an authenticated connection. `lnd` supports the following Tor control
authentication methods (arguably, from most to least secure):
* `SAFECOOKIE`: This authentication method relies on a cookie created and
stored by the Tor daemon and is the default assuming the Tor daemon supports
it by specifying `CookieAuthentication 1` in its configuration file.
* `HASHEDPASSWORD`: This authentication method is stateless as it relies on a
password hash scheme and may be useful if the Tor daemon is operating under a
separate host from the `lnd` node. The password hash can be obtained through
the Tor daemon with `tor --hash-password PASSWORD`, which should then be
specified in Tor's configuration file with `HashedControlPassword
PASSWORD_HASH`. Finally, to use it within `lnd`, the `--tor.password` flag
should be provided with the corresponding password.
* `NULL`: To bypass any authentication at all, this scheme can be used instead.
It doesn't require any additional flags to `lnd` or configuration options to
the Tor daemon.
## Listening for Inbound Connections
In order to listen for inbound connections through Tor, an onion service must be
created. `lnd` supports v3 onion services, the latest generation of onion
services. To learn more about these, see
[Intro to Next Gen Onion Services](https://trac.torproject.org/projects/tor/wiki/doc/NextGenOnions).
v3 onion services are created and used automatically by `lnd` via the `tor.v3`
flag. To prevent unintentional leaking of identifying information, it is also
necessary to add the flag `listen=localhost`.
For example, v3 onion services can be used with the following flags:
```shell
$ ./lnd --tor.active --tor.v3 --listen=localhost
```
This will automatically create a hidden service for your node to use to listen
for inbound connections and advertise itself to the network. The onion service's
private key is saved to a file named `v3_onion_private_key` in `lnd`'s base
directory. This will allow `lnd` to recreate the same hidden service upon
restart. If you wish to generate a new onion service, you can simply delete this
file. The path to this private key file can also be modified with the
`--tor.privatekeypath` argument.
You can optionally encrypt the Tor private key by using the `--tor.encryptkey`
flag. This will still write to the same private key files. However instead of
writing the plaintext private key, `lnd` encrypts the private key using the
wallet's seed and writes the encrypted blob to the file.
---
### Db Migration Guide
# `LND Database Upgrade Guide`: A Two-Stage Migration for Node Operators
*Table of Contents*
- [Overview](#overview)
- [Operation Modes](#operation-modes)
- [Stage 1: Migration from bbolt to SQLite/Postgres (kvdb)](#stage-1-migration-from-bbolt-to-sqlitepostgres-kvdb)
- [Choosing Your Target Backend](#choosing-your-target-backend)
- [Postgres kvdb Migration](#postgres-kvdb-migration)
- [SQLite kvdb Migration](#sqlite-kvdb-migration)
- [Stage 2: Migration from kvdb to Relational Database](#stage-2-migration-from-kvdb-to-relational-database)
- [Subsystem Readiness](#subsystem-readiness)
- [Known Limitations and Edge Cases](#known-limitations-and-edge-cases)
- [Best Practices for Node Operators](#best-practices-for-node-operators)
- [Choosing the Right Path](#choosing-the-right-path)
- [Timing Your Migration](#timing-your-migration)
- [Validation](#validation)
- [Implementation Examples](#implementation-examples)
- [Migrating to SQLite kvdb](#migrating-to-sqlite-kvdb)
- [Migrating Invoices to Relational Backend](#migrating-invoices-to-relational-backend)
- [Future Improvements](#future-improvements)
- [Conclusion](#conclusion)
---
## Overview
The `LND Database Upgrade` process enables node operators to migrate from the legacy **bbolt** key-value store to modern **SQLite** or **PostgreSQL** backends—first in **kvdb mode**, then ultimately to a **relational database** format. This two-stage migration addresses long-standing performance bottlenecks, and lays the foundation for scalable, maintainable node operations.
This guide explains the migration path, backend trade-offs, subsystem readiness, and best practices for safely upgrading your LND database.
---
## Operation Modes
The LND database upgrade operates in two sequential stages, each with distinct tooling and implications:
```
/* Detailed source-code truncated for AI context efficiency. */
```
- **Stage 1**: Migrate from bbolt to a SQL-based **kvdb** backend using the [lndinit](https://github.com/lightninglabs/lndinit/blob/main/docs/data-migration.md) tool.
- **Stage 2**: Incrementally migrate subsystem data (invoices, graph, payments, etc.) from **kvdb** to SQL native **relational tables** as support becomes available.
---
## Stage 1: Migration from bbolt to SQLite/Postgres (kvdb)
LND cannot run with mixed backends, so all users must first leave bbolt behind. This stage uses the `lndinit` utility to perform an offline migration.
### Choosing Your Target Backend
| Backend | Performance (in kvdb mode) | Default in LND? | Long-Term Viability |
|-----------|--------------------------|------------------|----------------------|
| Postgres | Mediocre | No | ✅ |
| SQLite | Good | **Yes (future)** | ✅ |
> 💡 **Recommendation**: Unless you require Postgres for infrastructure reasons,
**migrate to SQLite kvdb** as your backend.
### Postgres kvdb Migration
A migration script is available via `lndinit`:
- [Postgres migration script](https://github.com/lightninglabs/lndinit/blob/main/docs/data-migration.md#using-postgres-as-the-destination-remote-database)
- **Caveat**: Users report degraded performance in kvdb mode. Only proceed if you plan to **immediately follow with Stage 2** to migrate the available data stores to relational DB.
This will mitigate the poor Postgres performance on kvdb.
### SQLite kvdb Migration
- [SQLite migration script](https://github.com/lightninglabs/lndinit/blob/main/docs/data-migration.md#using-sqlite-as-the-destination-remote-database)
- **Advantage**: Maintains good performance while waiting for full relational migration.
---
## Stage 2: Migration from kvdb to Relational Database
This stage unlocks true SQL performance by restructuring data into relational tables. Migration is **per-subsystem** and **incremental**.
The migration steps are automatically applied when LND is restarted after step 1 was successfully completed and the config value db.use-native-sql=true is set.
You will see log lines from the `SQLD` subsystem about the migration, such as `Starting migration of invoices from KV to SQL`.
### Subsystem Readiness
| Subsystem | Relational Backend | Migration Script | Status |
|---------------------|--------------------|------------------|--------|
| Invoices | ✅ Available | ✅ | Available with **v0.19** |
| Graph | ✅ Available | ✅ | Available with **v0.20** |
| Payments | ✅ Available | ✅ | Available with **v0.21** |
| Btcwallet | 🚧 In Progress | Planned | Targeted with **v0.22**|
| Channel State | 🚧 In Progress | Planned | Targeted with **v0.22**|
---
## Known Limitations and Edge Cases
- **Single database engine required**: LND requires a single consistent backend.
You cannot run invoices in relational mode while graph remain in kvdb *unless*
both are on the same SQL engine (e.g., SQLite).
- **Data loss risk**: Always **back up your `data/` directory** before migration.
- **Downtime required**: Stage 1 requires LND to be offline. Stage 2 is done at startup, requiring a LND restart.
- **Postgres kvdb performance**: Postgres performance on kvdb is sub-optimal. It is
recommended to make the stage 2 migration immediately to avoid performance bottlenecks. Certain RPCs like `listpayments` may not perform well on Postgres if the node has a lot of payments data. As of **v0.21**, the payments relational
backend is available, so payment-heavy nodes can migrate payments to relational
mode to restore good `listpayments` performance.
- **No migration path between SQL backend**: Once migrated to either Postgres or
SQLite, it is not possible to switch to the other, so choose your target backend carefully.
---
## Best Practices for Node Operators
### Choosing the Right Path
- **For most users**: Choose SQLite, then migrate. Later, adopt relational backends subsystem-by-subsystem.
- **Enterprise/Postgres users**: With the **payments relational backend** available
as of **v0.21**, you can now perform **Stage 1 + Stage 2 in quick succession**.
### Timing Your Migration
- Perform migrations during **low-activity periods**.
- Monitor LND release notes for relational DB support of different subsystems.
### Validation
1. Stop LND.
2. Run migration with `lndinit`.
3. Start LND with new backend flags, to execute stage 2 migrations.
4. Validate node health: channels, balance, invoice/payment history.
---
## Implementation Examples
### Migrating to SQLite kvdb
```bash
# Stop LND
lnd --shutdown
# Backup
cp -r ~/.lnd ~/lnd-backup-$(date +%Y%m%d)
# Run migration (e.g. sqlite)
lndinit --debuglevel info migrate-db \
--source.bolt.data-dir ~/.lnd/data \
--dest.backend sqlite \
--dest.sqlite.data-dir ~/.lnd/data --network mainnet
# Start LND with SQLite backend
lnd --db.backend=sqlite
```
> 📝 Add `db.backend=sqlite` to your `lnd.conf` to make it persistent.
### Migrating Invoices to Relational Backend
Once on LND v0.19+ with SQLite/Postgres:
```bash
# Ensure backend is set
echo "db.backend=sqlite" >> ~/.lnd/lnd.conf
# Start LND — invoice migration runs automatically
lnd
```
Check logs for:
```
Migrating invoices from kvdb to relational format...
Invoice migration completed successfully.
```
---
## Future Improvements
The LND team is actively working on:
- **Btcwallet relational backend** and migration tooling (Stage 2)
- **Channel state relational backend** and migration tooling (Stage 2)
- **Automatic detection** of migration readiness in `lnd`
Node operators should monitor:
- [LND GitHub Releases](https://github.com/lightningnetwork/lnd/releases)
- [lndinit repository](https://github.com/lightninglabs/lndinit)
---
## Conclusion
The LND database upgrade is a strategic two-stage process designed to eliminate bbolt performance limitations while ensuring data integrity and operational continuity.
By **first migrating to SQLite/Postgres kvdb** and **then adopting relational backends incrementally**, node operators can achieve significant performance gains—especially for payment-heavy workloads—without rushing into unstable configurations.
Choose your path wisely, back up rigorously, and stay informed. The future of LND is relational, and this guide ensures you get there safely.
---
### Debugging Lnd
# Table of Contents
1. [Overview](#overview)
1. [Debug Logging](#debug-logging)
1. [LND's built-in profiler](#built-in-profiler-in-lnd)
## Overview
`lnd` ships with a few useful features for debugging, such as a built-in
profiler and tunable logging levels. If you need to submit a bug report
for `lnd`, it may be helpful to capture debug logging and performance
data ahead of time.
## Debug Logging
LND supports different logging levels and you can also specify different logging
levels per subsystem. This makes it easy to focus on a particular subsystem
without clogging up the logs with a lot of noise. One can either set the logging
in the lnd.conf file or pass the flag `--debuglevel` with the specified level
when starting lnd.
LND supports the following logging levels (see [log.go](/build/log.go) and
[sample-lnd.conf](/sample-lnd.conf) for more info):
- `trace`
- `debug`
- `info`
- `warn`
- `error`
- `critical`
- `off`
LND is composed of many subsystems, those subsystems can be listed either by
setting the starting flag `--debuglevel` or by using the lncli program.
Show all subsystems:
```shell
$ lnd --debuglevel=show
$ lncli debuglevel --show
```
For more details see [log.go](/log.go).
You may also specify logging per-subsystem, like this:
```shell
$ lnd --debuglevel==,=,...
$ lncli debuglevel --level==,=,...
```
The default global logging level is `info`. So if one wants to change the
global logging level and in addition also set a more detailed logging for a
particular subsystem the command would look like this (using `HSWC`
(htlcswitch) as an example subsystem):
```shell
$ lnd --debuglevel=critical,HSWC=debug
$ lncli debuglevel --level=critical,HSWC=debug
```
The subsystem names are case-sensitive and must be all uppercase.
To identify the subsystems defined by an abbreviated name, you can search for
the abbreviation in the [log.go](/log.go) file. Each subsystem
declares a `btclog.Logger` instance locally which is then assigned via the
`UseLogger` function call in the `SetupLoggers` function.
Example HSWC:
For the `HSWC` subsystem a new sublogger is injected into the htlcswitch
package via the `UseLogger` function call in the `SetupLoggers` function. So
the HSWC subsystem handles the logging in the htlcswitch package.
```go
AddSubLogger(root, "HSWC", interceptor, htlcswitch.UseLogger)
```
Caution: Some logger subsystems are overwritten during the instanziation. An
example here is the `neutrino/query` package which instead of using the `BTCN`
prefix is overwritten by the `LNWL` subsystem.
Moreover when using the `lncli` command the return value will provide the
updated list of all subsystems and their associated logging levels. This makes
it easy to get an overview of the current logging level for the whole system.
Example:
```shell
$ lncli debuglevel --level=critical,HSWC=debug
{
"sub_systems": "ARPC=INF, ATPL=INF, BLPT=INF, BRAR=INF, BTCN=INF, BTWL=INF, CHAC=INF, CHBU=INF, CHCL=INF, CHDB=INF, CHFD=INF, CHFT=INF, CHNF=INF, CHRE=INF, CLUS=INF, CMGR=INF, CNCT=INF, CNFG=INF, CRTR=INF, DISC=INF, DRPC=INF, FNDG=INF, GRPH=INF, HLCK=INF, HSWC=DBG, INVC=INF, IRPC=INF, LNWL=INF, LTND=INF, NANN=INF, NRPC=INF, NTFN=INF, NTFR=INF, PEER=INF, PRNF=INF, PROM=INF, PRPC=INF, RPCP=INF, RPCS=INF, RPWL=INF, RRPC=INF, SGNR=INF, SPHX=INF, SRVR=INF, SWPR=INF, TORC=INF, UTXN=INF, VRPC=INF, WLKT=INF, WTCL=INF, WTWR=INF"
}
```
## Built-in profiler in LND
`LND` has a built-in feature which allows you to capture profiling data at
runtime using [pprof](https://golang.org/pkg/runtime/pprof/), a profiler for
Go. It is recommended to enable the profiling server so that an analyis can be
triggered during runtime. There is only little overhead in enabling this
feature, because profiling is only started when calling the server endpoints.
However LND also allows to specify a cpu profile file via the `cpuprofile` flag
which triggers a cpu profile when LND starts and stops it when LND shuts down.
This is only recommended for debugging purposes, because the overhead is much
higher. To enable the profile server, start `lnd` with the `--profile` option
using a free port. As soon as the server is up different profiles can be
fetched from the `debug/pprof` endpoint using either the web interface or for
example `curl`.
Example port `9736` is used for the profile server in the following examples.
```shell
$ lnd --profile=9736
```
NOTE: The `--profile` flag of the lncli program does not relate to profiling and
the profiling server. It has a different context and allows a node operator to
manage different LND daemons without providing all the cmd flags every time.
For more details see [lncli profile](/cmd/commands/profile.go).
### Different types of profiles
#### CPU profile
A cpu profile can be used to analyze the CPU usage of the program. When
obtaining it via the profile http endpoint you can specify the time duration as
a query parameter.
```shell
$ curl http://localhost:9736/debug/pprof/profile?seconds=10 > cpu.prof
```
#### Goroutine profile
The goroutine profile is very useful when analyzing deadlocks and lock
contention. It can be obtained via the web interface or the following endpoint:
```shell
$ curl http://localhost:9736/debug/pprof/goroutine?debug=2 > goroutine.prof
```
The query parameter `debug=2` is optional but recommended and referes to the
format of the output file. Only this format has the necessary information to
identify goroutines deadlocks. Otherwise `go tool pprof` needs to be used to
visualize the data and interpret the results.
#### Heap profile
The heap profile is useful to analyze memory allocations. It can be obtained
via the following endpoint:
```shell
$ curl http://localhost:9736/debug/pprof/heap > heap.prof
```
The documentation of the pprof package states that a gc can be triggered before
obtaining the heap profile. This can be done by setting the gc query parameter
(`gc=1`).
#### Other profiles
There are several other options available like a mutex profile or a block
profile which gives insights into contention and bottlenecks of your program.
The web interface lists all the available profiles/endpoints which can be
obtained.
However mutex and block profiling need to be enabled separately by setting the
sampling rate via the config values `BlockingProfile` and `MutexProfile`. They
are off by default (0). These values represent sampling rates meaning that a
value of `1` will record every event leading to a significant overhead whereas
a sample rate of `n` will only record 1 out of nth events decreasing the
aggressiveness of the profiler.
Fetching the block and mutex profile:
```shell
$ curl http://localhost:9736/debug/pprof/mutex?debug=2
$ curl http://localhost:9736/debug/pprof/block?debug=2
```
The full programm command can also be fetched which shows how LND was started
and which flags were provided to the program.
```shell
$ curl http://localhost:9736/debug/pprof/cmdline > cmdline.prof
```
There are also other endpoints available see the
[pprof documentation](https://golang.org/pkg/runtime/pprof/) for more details.
#### Visualizing the profile dumps
It can be hard to make sense of the profile dumps by just looking at them
therefore the Golang ecosystem provides tools to analyze those profile dumps
either via the terminal or by visualizing them. One of the tools is
`go tool pprof`.
Assuming the profile was fetched via `curl` as in the examples above a nice
svg visualization can be generated for the cpu profile like this:
```shell
$ go tool pprof -svg cpu.prof > cpu.svg
```
Details how to interpret these visualizations can be found in the
[pprof documentation](https://github.com/google/pprof/blob/main/doc/README.md#interpreting-the-callgraph).
---
### Development Guidelines
# Development Guidelines
1. [Code Documentation and Commenting](#code-documentation-and-commenting)
1. [Code Spacing and Formatting](#code-spacing-and-formatting)
1. [Additional Style Constraints](#additional-style-constraints)
1. [Recommended settings for your editor](#recommended-settings-for-your-editor)
1. [Testing](#testing)
1. [Model Git Commit Messages](#model-git-commit-messages)
1. [Ideal Git Commit Structure](#ideal-git-commit-structure)
1. [Sign Your Git Commits](#sign-your-git-commits)
1. [Pointing to Remote Dependent Branches in Go Modules](#pointing-to-remote-dependent-branches-in-go-modules)
1. [Use of Log Levels](#use-of-log-levels)
1. [Use of Golang submodules](#use-of-golang-submodules)
## Why this emphasis on formatting?
Code in general (and Open Source code specifically) is _read_ by developers many
more times during its lifecycle than it is modified. With this fact in mind, the
Golang language was designed for readability (among other goals).
While the enforced formatting of `go fmt` and some best practices already
eliminate many discussions, the resulting code can still look and feel very
differently among different developers.
We aim to enforce a few additional rules to unify the look and feel of all code
in `lnd` to help improve the overall readability.
## Code Documentation and Commenting
- At a minimum every function must be commented with its intended purpose and
any assumptions that it makes
- Function comments must always begin with the name of the function per
[Effective Go](https://golang.org/doc/effective_go.html)
- Function comments should be complete sentences since they allow a wide
variety of automated presentations such as [godoc.org](https://godoc.org)
- The general rule of thumb is to look at it as if you were completely
unfamiliar with the code and ask yourself, would this give me enough
information to understand what this function does and how I'd probably want to
use it?
- Exported functions should also include detailed information the caller of the
function will likely need to know and/or understand:
**WRONG**
```go
// generates a revocation key
func DeriveRevocationPubkey(commitPubKey *btcec.PublicKey,
revokePreimage []byte) *btcec.PublicKey {
```
**RIGHT**
```go
// DeriveRevocationPubkey derives the revocation public key given the
// counterparty's commitment key, and revocation preimage derived via a
// pseudo-random-function. In the event that we (for some reason) broadcast a
// revoked commitment transaction, then if the other party knows the revocation
// preimage, then they'll be able to derive the corresponding private key to
// this private key by exploiting the homomorphism in the elliptic curve group:
// * https://en.wikipedia.org/wiki/Group_homomorphism#Homomorphisms_of_abelian_groups
//
// The derivation is performed as follows:
//
// revokeKey := commitKey + revokePoint
// := G*k + G*h
// := G * (k+h)
//
// Therefore, once we divulge the revocation preimage, the remote peer is able to
// compute the proper private key for the revokeKey by computing:
// revokePriv := commitPriv + revokePreimge mod N
//
// Where N is the order of the sub-group.
func DeriveRevocationPubkey(commitPubKey *btcec.PublicKey,
revokePreimage []byte) *btcec.PublicKey {
```
- Comments in the body of the code are highly encouraged, but they should
explain the intention of the code as opposed to just calling out the
obvious
**WRONG**
```go
// return err if amt is less than 546
if amt < 546 {
return err
}
```
**RIGHT**
```go
// Treat transactions with amounts less than the amount which is considered dust
// as non-standard.
if amt < 546 {
return err
}
```
**NOTE:** The above should really use a constant as opposed to a magic number,
but it was left as a magic number to show how much of a difference a good
comment can make.
## Code Spacing and formatting
Code in general (and Open Source code specifically) is _read_ by developers many
more times during its lifecycle than it is modified. With this fact in mind, the
Golang language was designed for readability (among other goals).
While the enforced formatting of `go fmt` and some best practices already
eliminate many discussions, the resulting code can still look and feel very
differently among different developers.
We aim to enforce a few additional rules to unify the look and feel of all code
in `lnd` to help improve the overall readability.
Blocks of code within `lnd` should be segmented into logical stanzas of
operation. Such spacing makes the code easier to follow at a skim, and reduces
unnecessary line noise. Coupled with the commenting scheme specified in the
[contribution guide](#code-documentation-and-commenting),
proper spacing allows readers to quickly scan code, extracting semantics quickly.
Functions should _not_ just be laid out as a bare contiguous block of code.
**WRONG**
```go
witness := make([][]byte, 4)
witness[0] = nil
if bytes.Compare(pubA, pubB) == -1 {
witness[1] = sigB
witness[2] = sigA
} else {
witness[1] = sigA
witness[2] = sigB
}
witness[3] = witnessScript
return witness
```
**RIGHT**
```go
witness := make([][]byte, 4)
// When spending a p2wsh multi-sig script, rather than an OP_0, we add
// a nil stack element to eat the extra pop.
witness[0] = nil
// When initially generating the witnessScript, we sorted the serialized
// public keys in descending order. So we do a quick comparison in order
// to ensure the signatures appear on the Script Virtual Machine stack in
// the correct order.
if bytes.Compare(pubA, pubB) == -1 {
witness[1] = sigB
witness[2] = sigA
} else {
witness[1] = sigA
witness[2] = sigB
}
// Finally, add the preimage as the last witness element.
witness[3] = witnessScript
return witness
```
Additionally, we favor spacing between stanzas within syntax like: switch case
statements and select statements.
**WRONG**
```go
switch {
case a:
case b:
case c:
case d:
default:
}
```
**RIGHT**
```go
switch {
// Brief comment detailing instances of this case (repeat below).
case a:
case b:
case c:
case d:
default:
}
```
## Additional Style Constraints
Before a PR is submitted, the proposer should ensure that the file passes the
set of linting scripts run by `make lint`. These include `gofmt`. In addition
to `gofmt` we've opted to enforce the following style guidelines.
### 80 character line length
ALL columns (on a best effort basis) should be wrapped to 80 line columns.
Editors should be set to treat a **tab as 8 spaces**.
**WRONG**
```go
myKey := "0214cd678a565041d00e6cf8d62ef8add33b4af4786fb2beb87b366a2e151fcee7"
```
**RIGHT**
```go
myKey := "0214cd678a565041d00e6cf8d62ef8add33b4af4786fb2beb87b366a2e1" +
"51fcee7"
```
### Wrapping long function calls
When wrapping a line that contains a function call as the unwrapped line exceeds
the column limit, the close parenthesis should be placed on its own line.
Additionally, all arguments should begin in a new line after the open parenthesis.
**WRONG**
```go
value, err := bar(a,
a, b, c)
```
**RIGHT**
```go
value, err := bar(
a, a, b, c,
)
```
As long as the visual symmetry of the opening and closing parentheses (or curly
braces) is preserved, arguments that would otherwise introduce a new level of
indentation are allowed to be written in a more compact form.
Visual symmetry here means that when two or more opening parentheses or curly
braces are on the same line, then they must also be closed on the same line.
And the closing line needs to have the same indentation level as the opening
line.
Example with inline struct creation:
**ACCEPTABLE**
```go
response, err := node.AddInvoice(
ctx, &lnrpc.Invoice{
Memo: "invoice",
ValueMsat: int64(oneUnitMilliSat - 1),
},
)
```
**PREFERRED**
```go
response, err := node.AddInvoice(ctx, &lnrpc.Invoice{
Memo: "invoice",
ValueMsat: int64(oneUnitMilliSat - 1),
})
```
**WRONG**
```go
response, err := node.AddInvoice(ctx, &lnrpc.Invoice{
Memo: "invoice",
ValueMsat: int64(oneUnitMilliSat - 1)})
```
Example with nested function call:
**ACCEPTABLE**:
```go
payInvoiceWithSatoshi(
t.t, dave, invoiceResp2, withFailure(
lnrpc.Payment_FAILED, failureNoRoute,
),
)
```
**PREFERRED**:
```go
payInvoiceWithSatoshi(t.t, dave, invoiceResp2, withFailure(
lnrpc.Payment_FAILED, failureNoRoute,
))
```
#### Exception for log and error message formatting
**Note that the above guidelines don't apply to log or error messages.** For
log and error messages, committers should attempt to minimize the number of
lines utilized, while still adhering to the 80-character column limit. For
example:
**WRONG**
```go
return fmt.Errorf(
"this is a long error message with a couple (%d) place holders",
len(things),
)
log.Debugf(
"Something happened here that we need to log: %v",
longVariableNameHere,
)
```
**RIGHT**
```go
return fmt.Errorf("this is a long error message with a couple (%d) place "+
"holders", len(things))
log.Debugf("Something happened here that we need to log: %v",
longVariableNameHere)
```
This helps to visually distinguish those formatting statements (where nothing
of consequence happens except for formatting an error message or writing
to a log) from actual method or function calls. This compact formatting should
be used for calls to formatting functions like `fmt.Errorf`,
`log.(Trace|Debug|Info|Warn|Error)f` and `fmt.Printf`.
But not for statements that are important for the flow or logic of the code,
like `require.NoErrorf()`.
#### Exceptions and additional styling for structured logging
When making use of structured logging calls (there are any `btclog.Logger`
methods ending in `S`), a few different rules and exceptions apply.
1) **Static messages:** Structured log calls take a `context.Context` as a first
parameter and a _static_ string as the second parameter (the `msg` parameter).
Formatted strings should ideally not be used for the construction of the `msg`
parameter. Instead, key-value pairs (or `slog` attributes) should be used to
provide additional variables to the log line.
**WRONG**
```go
log.DebugS(ctx, fmt.Sprintf("User %d just spent %.8f to open a channel", userID, 0.0154))
```
**RIGHT**
```go
log.InfoS(ctx, "Channel open performed",
slog.Int("user_id", userID),
btclog.Fmt("amount", "%.8f", 0.00154))
```
2) **Key-value attributes**: The third parameter in any structured log method is
a variadic list of the `any` type but it is required that these are provided in
key-value pairs such that an associated `slog.Attr` variable can be created for
each key-value pair. The simplest way to specify this is to directly pass in the
key-value pairs as raw literals as follows:
```go
log.InfoS(ctx, "Channel open performed", "user_id", userID, "amount", 0.00154)
```
This does work, but it becomes easy to make a mistake and accidentally leave out
a value for each key provided leading to a nonsensical log line. To avoid this,
it is suggested to make use of the various `slog.Attr` helper functions as
follows:
```go
log.InfoS(ctx, "Channel open performed",
slog.Int("user_id", userID),
btclog.Fmt("amount", "%.8f", 0.00154))
```
3) **Line wrapping**: Structured log lines are an exception to the 80-character
line wrapping rule. This is so that the key-value pairs can be easily read and
reasoned about. If it is the case that there is only a single key-value pair
and the entire log line is still less than 80 characters, it is acceptable to
have the key-value pair on the same line as the log message. However, if there
are multiple key-value pairs, it is suggested to use the one line per key-value
pair format. Due to this suggestion, it is acceptable for any single key-value
pair line to exceed 80 characters for the sake of readability.
**WRONG**
```go
// Example 1.
log.InfoS(ctx, "User connected",
"user_id", userID)
// Example 2.
log.InfoS(ctx, "Channel open performed", "user_id", userID,
btclog.Fmt("amount", "%.8f", 0.00154), "channel_id", channelID)
// Example 3.
log.InfoS(ctx, "Bytes received",
"user_id", userID,
btclog.Hex("peer_id", peerID.SerializeCompressed()),
btclog.Hex("message", []bytes{
0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08,
})))
```
**RIGHT**
```go
// Example 1.
log.InfoS(ctx, "User connected", "user_id", userID)
// Example 2.
log.InfoS(ctx, "Channel open performed",
slog.Int("user_id", userID),
btclog.Fmt("amount", "%.8f", 0.00154),
slog.String("channel_id", channelID))
// Example 3.
log.InfoS(ctx, "Bytes received",
"user_id", userID,
btclog.Hex("peer_id", peerID.SerializeCompressed()),
btclog.Hex("message", []bytes{0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08})))
```
### Wrapping long function definitions
If one is forced to wrap lines of function arguments that exceed the
80-character limit, then indentation must be kept on the following lines. Also,
lines should not end with an open parenthesis if the function definition isn't
finished yet.
**WRONG**
```go
func foo(a, b, c,
) (d, error) {
func bar(a, b, c) (
d, error,
) {
func baz(a, b, c) (
d, error) {
```
**RIGHT**
```go
func foo(a, b,
c) (d, error) {
func baz(a, b, c) (d,
error) {
func longFunctionName(
a, b, c) (d, error) {
```
If a function declaration spans multiple lines the body should start with an
empty line to help visually distinguishing the two elements.
**WRONG**
```go
func foo(a, b, c,
d, e) error {
var a int
}
```
**RIGHT**
```go
func foo(a, b, c,
d, e) error {
var a int
}
```
### Inline slice definitions
In Go a list of slices can be initialized with values directly, using curly
braces. Whenever possible, the more verbose/indented style should be used for
better readability and easier git diff handling. Because that results in more
levels of code indentation, the more compact version is allowed in situations
where the remaining space would otherwise be too restricted, resulting in too
long lines (or excessive use of the `// nolint: ll` directive).
**ACCEPTABLE**
```go
testCases := []testCase{{
name: "spend exactly all",
coins: []wallet.Coin{{
TxOut: wire.TxOut{
PkScript: p2wkhScript,
Value: 1 * btcutil.SatoshiPerBitcoin,
},
}},
}, {
name: "spend more",
coins: []wallet.Coin{{
TxOut: wire.TxOut{
PkScript: p2wkhScript,
Value: 1 * btcutil.SatoshiPerBitcoin,
},
}},
}}
```
**PREFERRED**
```go
coin := btcutil.SatoshiPerBitcoin
testCases := []testCase{
{
name: "spend exactly all",
coins: []wallet.Coin{
{
TxOut: wire.TxOut{
PkScript: p2wkhScript,
Value: 1 * coin,
},
},
},
},
{
name: "spend more",
coins: []wallet.Coin{
{
TxOut: wire.TxOut{
PkScript: p2wkhScript,
Value: 1 * coin,
},
},
},
},
}
```
## Recommended settings for your editor
To make it easier to follow the rules outlined above, we recommend setting up
your editor with at least the following two settings:
1. Set your tabulator width (also called "tab size") to **8 spaces**.
2. Set a ruler or visual guide at 80 character.
Note that the two above settings are automatically applied in editors that
support the `EditorConfig` scheme (for example GoLand, GitHub, GitLab,
VisualStudio). In addition, specific settings for Visual Studio Code are checked
into the code base as well.
Other editors (for example Atom, Notepad++, Vim, Emacs and so on) might install
a plugin to understand the rules in the `.editorconfig` file.
In Vim, you might want to use `set colorcolumn=80`.
## Testing
One of the major design goals of all of `lnd`'s packages and the daemon itself is
to aim for a high degree of test coverage. This is financial software so bugs
and regressions in the core logic can cost people real money. For this reason
every effort must be taken to ensure the code is as accurate and bug-free as
possible. Thorough testing is a good way to help achieve that goal.
Unless a new feature you submit is completely trivial, it will probably be
rejected unless it is also accompanied by adequate test coverage for both
positive and negative conditions. That is to say, the tests must ensure your
code works correctly when it is fed correct data as well as incorrect data
(error paths).
Go provides an excellent test framework that makes writing test code and
checking coverage statistics straightforward. For more information about the
test coverage tools, see the [golang cover blog post](https://blog.golang.org/cover).
A quick summary of test practices follows:
- All new code should be accompanied by tests that ensure the code behaves
correctly when given expected values, and, perhaps even more importantly, that
it handles errors gracefully.
- When you fix a bug, it should be accompanied by tests which exercise the bug
to both prove it has been resolved and to prevent future regressions.
- Changes to publicly exported packages such as
[brontide](https://github.com/lightningnetwork/lnd/tree/master/brontide) should
be accompanied by unit tests exercising the new or changed behavior.
- Changes to behavior within the daemon's interaction with the P2P protocol,
or RPC's will need to be accompanied by integration tests which use the
[`networkHarness`framework](https://github.com/lightningnetwork/lnd/blob/master/lntest/harness.go)
contained within `lnd`. For example integration tests, see
[`lnd_test.go`](https://github.com/lightningnetwork/lnd/blob/master/itest/lnd_test.go).
- The itest log files are automatically scanned for `[ERR]` lines. There
shouldn't be any of those in the logs, see [Use of Log Levels](#use-of-log-levels).
Throughout the process of contributing to `lnd`, you'll likely also be
extensively using the commands within our `Makefile`. As a result, we recommend
[perusing the make file documentation](https://github.com/lightningnetwork/lnd/blob/master/docs/MAKEFILE.md).
Before committing the changes made, you should run unit tests to validate the
changes, and provide a new integration test when necessary. The unit tests
should pass at two levels,
- Run `make unit-debug log="stdlog trace" pkg=$pkg case=$case timeout=10s`,
where `pkg` is the package that's updated, and `case` is the newly added or
affected unit test. This command should run against all the newly added and
affected test cases. In addition, you should pay attention to the logs added
here, and make sure they are correctly formatted and no spammy. Also notice
the timeout - 10 seconds should be more than enough to run a single unit test
case, if it takes longer than that, consider break the test to make it more
"unit".
- Run `make unit pkg=$pkg timeout=5m` to make sure all existing unit tests still
pass.
- If there are newly added integration tests, or the changes may alter the
workflow of specific areas, run `make itest icase=$icase` to validate the
behavior, where `icase` is the affected test case.
## Model Git Commit Messages
This project prefers to keep a clean commit history with well-formed commit
messages. This section illustrates a model commit message and provides a bit
of background for it. This content was originally created by Tim Pope and made
available on his website, however that website is no longer active, so it is
being provided here.
Here’s a model Git commit message:
```text
Short (50 chars or less) summary of changes
More detailed explanatory text, if necessary. Wrap it to about 72
characters or so. In some contexts, the first line is treated as the
subject of an email and the rest of the text as the body. The blank
line separating the summary from the body is critical (unless you omit
the body entirely); tools like rebase can get confused if you run the
two together.
Write your commit message in the present tense: "Fix bug" and not "Fixed
bug." This convention matches up with commit messages generated by
commands like git merge and git revert.
Further paragraphs come after blank lines.
- Bullet points are okay, too
- Typically a hyphen or asterisk is used for the bullet, preceded by a
single space, with blank lines in between, but conventions vary here
- Use a hanging indent
```
Here are some of the reasons why wrapping your commit messages to 72 columns is
a good thing.
- git log doesn't do any special wrapping of the commit messages. With
the default pager of less -S, this means your paragraphs flow far off the edge
of the screen, making them difficult to read. On an 80 column terminal, if we
subtract 4 columns for the indent on the left and 4 more for symmetry on the
right, we’re left with 72 columns.
- git format-patch --stdout converts a series of commits to a series of emails,
using the messages for the message body. Good email netiquette dictates we
wrap our plain text emails such that there’s room for a few levels of nested
reply indicators without overflow in an 80 column terminal.
In addition to the Git commit message structure adhered to within the daemon
all short-[commit messages are to be prefixed according to the convention
outlined in the Go project](https://golang.org/doc/contribute.html#change). All
commits should begin with the subsystem or package primarily affected by the
change. In the case of a widespread change, the packages are to be delimited by
either a '+' or a ','. This prefix seems minor but can be extremely helpful in
determining the scope of a commit at a glance, or when bug hunting to find a
commit which introduced a bug or regression.
## Ideal Git Commit Structure
Within the project we prefer small, contained commits for a pull request over a
single giant commit that touches several files/packages. Ideal commits build on
their own, in order to facilitate easy usage of tools like `git bisect` to `git
cherry-pick`. It's preferred that commits contain an isolated change in a
single package. In this case, the commit header message should begin with the
prefix of the modified package. For example, if a commit was made to modify the
`lnwallet` package, it should start with `lnwallet: `.
In the case of changes that only build in tandem with changes made in other
packages, it is permitted for a single commit to be made which contains several
prefixes such as: `lnwallet+htlcswitch`. This prefix structure along with the
requirement for atomic contained commits (when possible) make things like
scanning the set of commits and debugging easier. In the case of changes that
touch several packages, and can only compile with the change across several
packages, a `multi: ` prefix should be used.
Examples of common patterns w.r.t commit structures within the project:
* It is common that during the work on a PR, existing bugs are found and
fixed. If they can be fixed in isolation, they should have their own
commit.
* File restructuring like moving a function to another file or changing order
of functions: with a separate commit because it is much easier to review
the real changes that go on top of the restructuring.
* Preparatory refactorings that are functionally equivalent: own commit.
* Project or package wide file renamings should be in their own commit.
* Ideally if a new package/struct/sub-system is added in a PR, there should
be a single commit which adds the new functionality, with follow up
individual commits that begin to integrate the functionality within the
codebase.
* If a PR only fixes a trivial issue, such as updating documentation on a
small scale, fix typos, or any changes that do not modify the code, the
commit message of the HEAD commit of the PR should end with `[skip ci]` to
skip the CI checks. When pushing to such an existing PR, the latest commit
being pushed should end with `[skip ci]` as to not inadvertently trigger the
CI checks.
## Sign your git commits
When contributing to `lnd` it is recommended to sign your git commits. This is
easy to do and will help in assuring the integrity of the tree. See [mailing
list entry](https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2014-May/005877.html)
for more information.
### How to sign your commits?
Provide the `-S` flag (or `--gpg-sign`) to git commit when you commit
your changes, for example
```shell
$ git commit -m "Commit message" -S
```
Optionally you can provide a key id after the `-S` option to sign with a
specific key.
To instruct `git` to auto-sign every commit, add the following lines to your
`~/.gitconfig` file:
```text
[commit]
gpgsign = true
```
### What if I forgot?
You can retroactively sign your previous commit using `--amend`, for example
```shell
$ git commit -S --amend
```
If you need to go further back, you can use the interactive rebase
command with 'edit'. Replace `HEAD~3` with the base commit from which
you want to start.
```shell
$ git rebase -i HEAD~3
```
Replace 'pick' by 'edit' for the commit that you want to sign and the
rebasing will stop after that commit. Then you can amend the commit as
above. Afterwards, do
```shell
$ git rebase --continue
```
As this will rewrite history, you cannot do this when your commit is
already merged. In that case, too bad, better luck next time.
If you rewrite history for another reason - for example when squashing
commits - make sure that you re-sign as the signatures will be lost.
Multiple commits can also be re-signed with `git rebase`. For example, signing
the last three commits can be done with:
```shell
$ git rebase --exec 'git commit --amend --no-edit -n -S' -i HEAD~3
```
### How to check if commits are signed?
Use `git log` with `--show-signature`,
```shell
$ git log --show-signature
```
You can also pass the `--show-signature` option to `git show` to check a single
commit.
## Pointing to Remote Dependent Branches in Go Modules
It's common that a developer may need to make a change in a dependent project
of `lnd` such as `btcd`, `neutrino`, `btcwallet`, etc. In order to test changes
without testing infrastructure, or simply make a PR into `lnd` that will build
without any further work, the `go.mod` and `go.sum` files will need to be
updated. Luckily, the `go mod` command has a handy tool to do this
automatically so developers don't need to manually edit the `go.mod` file:
```shell
$ go mod edit -replace=IMPORT-PATH-IN-LND@LND-VERSION=DEV-FORK-IMPORT-PATH@DEV-FORK-VERSION
```
Here's an example replacing the `lightning-onion` version checked into `lnd` with a version in roasbeef's fork:
```shell
$ go mod edit -replace=github.com/lightningnetwork/lightning-onion@v0.0.0-20180605012408-ac4d9da8f1d6=github.com/roasbeef/lightning-onion@2e5ae87696046298365ab43bcd1cf3a7a1d69695
```
## Use of Log Levels
There are six log levels available: `trace`, `debug`, `info`, `warn`, `error` and `critical`.
Only use `error` for internal errors that are never expected to happen during
normal operation. No event triggered by external sources (rpc, chain backend,
etc) should lead to an `error` log.
## Use of Golang submodules
Changes to packages that are their own submodules (e.g. they contain a `go.mod`
and `go.sum` file, for example `tor/go.mod`) require a specific process.
We want to avoid the use of local replace directives in the root `go.mod`,
therefore changes to a submodule are a bit involved.
The main process for updating and then using code in a submodule is as follows:
- Create a PR for the changes to the submodule itself (e.g. edit something in
the `tor` package)
- Wait for the PR to be merged and a new tag (for example `tor/v1.0.x`) to be
pushed.
- Create a second PR that bumps the updated submodule in the root `go.mod` and
uses the new functionality in the main module.
Of course the two PRs can be opened at the same time and be built on top of each
other. But the merge and tag push order should always be maintained.
---
### Estimate Route Fee
# `EstimateRouteFee`: A Guide for Wallet Developers
## Table of Contents
- [Overview](#overview)
- [Operation Modes](#operation-modes)
- [Graph-Based Estimation](#graph-based-estimation)
- [Probe-Based Estimation](#probe-based-estimation)
- [Private Channel and Hop Hint Handling](#private-channel-and-hop-hint-handling)
- [Hop Hint Processing](#hop-hint-processing)
- [Integration with Pathfinding](#integration-with-pathfinding)
- [LSP Detection and Special Handling](#lsp-detection-and-special-handling)
- [The LSP Detection Heuristic](#the-lsp-detection-heuristic)
- [How Probing Differs When an LSP is Detected](#how-probing-differs-when-an-lsp-is-detected)
- [Route Hint Transformation for LSP Probing](#route-hint-transformation-for-lsp-probing)
- [Relationship to Zero-Conf Channels](#relationship-to-zero-conf-channels)
- [Fee Assembly After LSP Probing](#fee-assembly-after-lsp-probing)
- [Known Limitations and Edge Cases](#known-limitations-and-edge-cases)
- [Best Practices for Wallet Integration](#best-practices-for-wallet-integration)
- [Choosing the Appropriate Mode](#choosing-the-appropriate-mode)
- [Handling Timeouts](#handling-timeouts)
- [Error Handling](#error-handling)
- [Fee Presentation](#fee-presentation)
- [Implementation Examples](#implementation-examples)
- [Basic Graph-Based Estimation](#basic-graph-based-estimation)
- [Invoice-Based Estimation with Timeout](#invoice-based-estimation-with-timeout)
- [Future Improvements](#future-improvements)
- [Conclusion](#conclusion)
## Overview
The `EstimateRouteFee` RPC call provides wallet applications with fee estimates
for Lightning Network payments. Understanding its operation modes and heuristics
is essential for building reliable payment experiences, particularly when
dealing with private channels and Lightning Service Providers (LSPs).
This document explains the behavioral characteristics, assumptions, and best
practices for integrating `EstimateRouteFee` into wallet applications, whether
you're building directly on LND or developing third-party wallet software.
## Operation Modes
`EstimateRouteFee` operates in two distinct modes, each optimized for different
use cases and accuracy requirements.
```mermaid
flowchart TD
Start([EstimateRouteFee Called]) --> Check{Input Type?}
Check -->|Destination + Amount| Graph[Graph-Based Estimation]
Check -->|Payment Request/Invoice| Invoice[Invoice Processing]
Graph --> LocalPath[Use Local Channel Graph]
LocalPath --> Mission[Apply Mission Control Data]
Mission --> CalcFee[Calculate Route & Fee]
CalcFee --> ReturnFast([Return Fee Estimate
~100ms])
Invoice --> HasHints{Has Route Hints?}
HasHints -->|No| StandardProbe[Standard Probe to Destination]
HasHints -->|Yes| CheckLSP{Check LSP Heuristic}
CheckLSP -->|Not LSP| StandardProbe
CheckLSP -->|Is LSP| LSPProbe[LSP-Aware Probe]
StandardProbe --> SendProbe1[Send Probe with Random Hash]
LSPProbe --> ModifyHints[Transform Route Hints]
ModifyHints --> SendProbe2[Probe to LSP Node]
SendProbe1 --> ProbeResult1[Wait for Probe Result]
SendProbe2 --> ProbeResult2[Calculate LSP Fees]
ProbeResult1 --> ReturnProbe([Return Fee Estimate
1-60s])
ProbeResult2 --> ReturnProbe
```
### Graph-Based Estimation
When provided with a destination public key and amount, `EstimateRouteFee`
performs local pathfinding using the in-memory channel graph. This mode executes
entirely locally without network interaction, making it fast but potentially
less accurate for complex routing scenarios.
This approach uses your node's view of the network topology and mission control
data (historical payment success rates) to calculate the most economical route.
A 1 BTC maximum fee limit prevents unreasonable calculations. The estimate
represents the difference between sent and received amounts.
Best for well-connected public nodes with sufficient routing information in the
public graph. Response times are typically sub-second.
### Probe-Based Estimation
When provided with an invoice, `EstimateRouteFee` sends probe payments through
the network using a random payment hash. This ensures the payment fails with
"incorrect payment details" at the destination, confirming route viability
without transferring funds.
More accurate than graph-based estimation as it tests actual network conditions
(liquidity, availability, current fees). May take seconds to minutes, especially
for private channels or LSPs.
## Private Channel and Hop Hint Handling
Private channels present unique challenges for fee estimation since they don't
exist in the public channel graph. `EstimateRouteFee` handles these through hop
hints provided in BOLT11 invoices.
### Hop Hint Processing
When an invoice contains route hints, `EstimateRouteFee` treats them as
additional routing information that extends the known network graph. Each hop
hint describes a private channel that can be used to reach the destination,
including the channel's routing policies such as fees and timelock requirements.
The system makes several important assumptions about hop hints:
- **Capacity assumptions**: Private channels are assumed to have sufficient
capacity for pathfinding calculations. This high capacity assumption
prevents the pathfinding algorithm from prematurely rejecting routes based
on amount constraints, as the receiver is expected to have adequate inbound
liquidity when they include a hop hint for that channel.
- **Policy trust**: Routing policies in hop hints (base fee, proportional
fee, CLTV delta) are trusted without verification, as there's no way to
independently validate private channel information.
### Integration with Pathfinding
Hop hints are treated as legitimate routing options alongside public channels,
enabling paths through both public and private channels. The pathfinding
algorithm works backward from destination to source, making hop hints critical
for reaching private destinations.
Private edges bypass normal validation and capacity checks. The system trusts
invoice creators to provide accurate routing information, as incorrect hints
prevent payment receipt.
## LSP Detection and Special Handling
Lightning Service Providers require special handling due to their role as
intermediaries for nodes without direct channel connectivity. `EstimateRouteFee`
implements sophisticated heuristics to detect LSP scenarios and fundamentally
modifies its probing behavior when an LSP configuration is identified.
### The LSP Detection Heuristic
`EstimateRouteFee` employs a pattern-matching algorithm to identify when a
payment destination is likely behind an LSP. This detection is crucial because
probing through an LSP requires different handling than standard payment
probing.
The heuristic examines the structure of route hints provided in the invoice to
identify characteristic LSP patterns. The detection operates on the principle
that LSPs typically maintain private channels to their users and appear as
public nodes in the network, while the final destination is private.
```mermaid
flowchart TD
Start([Route Hints Received]) --> Empty{Empty Hints?}
Empty -->|Yes| NotLSP([Not LSP])
Empty -->|No| CheckTarget{Invoice Target
in Graph?}
CheckTarget -->|Yes| NotLSP
CheckTarget -->|No| GetFirstDest[Get First Hint's
Destination Hop]
GetFirstDest --> CheckPub1{Destination Node
in Graph?}
CheckPub1 -->|Yes| IsLSP([Detected as LSP])
CheckPub1 -->|No| MoreHints{More Hints?}
MoreHints -->|No| NotLSP
MoreHints -->|Yes| NextHint[Check Next Hint]
NextHint --> GetNextDest[Get Destination Hop]
GetNextDest --> CheckPub2{Destination Node
in Graph?}
CheckPub2 -->|Yes| IsLSP
CheckPub2 -->|No| MoreHints
```
The detection follows three simple rules applied sequentially:
**Rule 1: Public Invoice Target → NOT an LSP**
- If the invoice target (destination) is a public node that exists in the
channel graph, the payment can be routed directly to it
- This means it's not an LSP setup, regardless of what route hints are provided
- Example: A well-connected merchant node with route hints for liquidity
signaling
**Rule 2: Public Destination Hop → IS an LSP**
- If at least one route hint has a destination hop (last hop in the route hint)
that is a public node in the graph, LSP detection is triggered
- This indicates the destination hop is an LSP serving a private client
- The private client is reached through the LSP's private channel
**Rule 3: All Private Destination Hops → NOT an LSP**
- If all destination hops in all route hints are private nodes (not in the
public graph), this is not treated as an LSP setup
- The payment will be routed directly to the invoice destination using the
route hints as additional path information
- This is the standard case for private channel payments
This pattern effectively distinguishes LSP configurations from other routing
scenarios. For instance, some Lightning implementations like CLN include route
hints even for public nodes to signal liquidity availability or preferred
routing paths. The heuristic correctly identifies these as non-LSP scenarios
by Rule 1 (detecting that the invoice target itself is public).
### How Probing Differs When an LSP is Detected
When the LSP detection heuristic identifies an LSP configuration,
`EstimateRouteFee` fundamentally changes its probing strategy. Understanding
these differences is crucial for wallet developers to correctly interpret fee
estimates.
#### Standard Probing Behavior
When no LSP is detected, the probe targets the invoice's actual destination
using complete route hints. The probe amount matches the invoice amount with
standard timelock requirements. Works well for directly reachable destinations.
#### LSP-Aware Probing Behavior
When an LSP configuration is detected, the probing strategy undergoes several
fundamental changes that reflect the unique characteristics of LSP-mediated
payments:
- **Probe destination changes to the LSP node** - Instead of targeting the
final payment recipient, the probe targets the LSP itself, recognizing that
the service provider handles the final hop independently
- **Route hints are modified via prepareLspRouteHints** - The final hop is
stripped from all route hints, removing the LSP-to-destination segment
while preserving intermediate hops that help reach the LSP
- **Probe amount increases by the LSP's maximum fee** - The system calculates
the worst-case fee across all route hints and adds it to the probe amount,
ensuring the estimate accounts for the LSP's forwarding charges
- **Timelock requirements switch to LSP's CLTV delta** - The probe uses the
LSP's timelock requirements instead of the invoice's, with the final
destination's CLTV added to the estimate after probing
- **Modified hints prevent traversal past the LSP** - By removing the final
hop, the probe cannot attempt to reach the actual destination, which would
likely fail due to the possible non-existence of LSP-to-user channels in
the public graph
These modifications ensure that fee estimation accurately reflects the two-stage
nature of LSP-mediated payments: first reaching the LSP through the public
network, then the LSP's own hop to the final destination.
### Route Hint Transformation for LSP Probing
When an LSP is detected, the system performs a sophisticated transformation of
the route hints to enable accurate fee estimation. This transformation, handled
by the prepareLspRouteHints function, serves three critical purposes.
```mermaid
flowchart LR
subgraph "Original Route Hints"
H1[Hop A → Hop B → LSP → Destination]
H2[Hop C → LSP → Destination]
H3[Hop D → Hop E → LSP → Destination]
end
Transform[prepareLspRouteHints
Transformation]
subgraph "Modified for Probing"
M1[Hop A → Hop B → LSP]
M2[Hop C → LSP]
M3[Hop D → Hop E → LSP]
end
LSPHint[Synthetic LSP Hint
Max Fees & CLTV]
H1 --> Transform
H2 --> Transform
H3 --> Transform
Transform --> M1
Transform --> M2
Transform --> M3
Transform --> LSPHint
style LSPHint fill:#f9f,stroke:#333,stroke-width:2px
```
The transformation serves three purposes:
1. **Creates a synthetic LSP hop hint** with worst-case fees and CLTV across
all route hints, ensuring conservative but reliable estimates
2. **Strips the final hop** from all route hints, removing the
LSP-to-destination segment that would cause probe failure
3. **Preserves intermediate hops** that help reach the LSP
This worst-case approach prioritizes reliability over optimism, particularly
important for mobile wallets where payment success matters more than minimal
fees.
### Relationship to Zero-Conf Channels
LSP detection has important interactions with zero-conf channels, which are
commonly used in LSP deployments for instant liquidity provision.
#### Why LSPs Use Zero-Conf Channels
LSPs use zero-conf channels for instant liquidity provision to new users. These
channels enable immediate routing using SCID aliases, remain private/unconfirmed
in the public graph, and rely on LSP-user trust relationships.
#### Impact on LSP Detection
Zero-conf channels align with LSP detection patterns since SCID aliases don't
appear in the public graph, always appearing as private channels. This alignment
reflects real-world LSP deployment patterns and strengthens the heuristic's
effectiveness.
#### Implications for Fee Estimation
Key considerations for zero-conf channels behind LSPs:
- Cannot verify channel existence (not in public graph)
- Capacity assumptions may be optimistic
- Route hint fees trusted without validation
- LSP handles liquidity management
### Fee Assembly After LSP Probing
After successful LSP probing, the system adds:
- LSP's worst-case fee for the final hop
- Invoice's final CLTV requirement
This two-stage calculation captures both the cost to reach the LSP and the
LSP's forwarding charges.
## Known Limitations and Edge Cases
### Probe Success Risk
Theoretical risk of probe completion if destination has bugs or non-standard
behavior. Funds would be lost with only a warning logged. Additionally, probes
may get stuck in the network, which is why setting appropriate timeouts is
crucial. Note that liquidity may shift between the probe and the actual payment
attempt. For larger payments, Multi-Path Payments (MPP) might find better routes
than the single-path probe, potentially resulting in lower actual fees.
### LSP Heuristic Accuracy
Can produce false positives (CLN nodes with liquidity hints) and false negatives
(magic routing hints from services like Boltz). Recent improvements check public
graph existence to reduce false positives.
### Route Hint Validation
Route hints trusted without validation. Assumes invoice creators have incentives
for accuracy (incorrect hints prevent payment receipt).
### Capacity and Liquidity Assumptions
Capacity assumptions may exceed actual availability. Payments may fail despite
successful estimation—handle failures gracefully.
## Best Practices for Wallet Integration
### Choosing the Appropriate Mode
**Graph-based**: Quick estimates for well-connected public nodes when you
have the destination key.
**Probe-based**: Accurate estimates for invoices, especially:
- Private or poorly connected destinations
- Large payments requiring fee accuracy
- LSP or complex routing scenarios
### Handling Timeouts
Set timeouts based on UX needs: 30s standard, 60s for important payments, 15s
for responsive UIs. Show progress indicators and provide cancel options.
Consider graph-based fallback for consistent timeouts.
### Error Handling
Common failures:
- **NO_ROUTE**: Destination offline, no liquidity, or unreachable
- **INSUFFICIENT_BALANCE**: Insufficient funds for payment + fees
- **TIMEOUT**: Probe exceeded timeout (congestion or poor connectivity)
Translate errors to user-friendly messages with suggested actions.
### Fee Presentation
Consider showing fee ranges or confidence levels since estimates may be
conservative (especially for LSPs using worst-case fees). Probe-based estimates
reflect current conditions but may change before actual payment.
## Implementation Examples
### Basic Graph-Based Estimation
For graph-based estimation, provide the destination public key and payment
amount. The response will include the routing fee in millisatoshis and an
estimated timelock delay. This mode is ideal for quick estimates where you have
the destination's node ID.
```shell
# Estimate fee for 100,000 satoshi payment to a specific node
lncli estimateroutefee --dest 0266a18ed969ef95c8a5aa314b443b2b3b8d91ed1d9f8e95476f5f4647efdec079 --amt 100000
```
The typical flow involves calling `EstimateRouteFee` with just the destination
and amount parameters. The response arrives quickly (usually under 100ms) since
it uses only local data. Convert the returned fee from millisatoshis to satoshis
for display, and consider showing the timelock delay to inform users about the
maximum time their funds might be locked.
### Invoice-Based Estimation with Timeout
For invoice-based estimation, provide the full payment request string and
optionally specify a timeout. This mode sends actual probe payments, so timeouts
are important to prevent long waits for poorly connected destinations.
```shell
# Estimate fee for an invoice with 60-second timeout
lncli estimateroutefee --pay_req lnbc100n1p3e... --timeout 60s
# Shorter timeout for responsive UIs
lncli estimateroutefee --pay_req lnbc100n1p3e... --timeout 15s
```
When calling `EstimateRouteFee` with a payment request, always set a reasonable
timeout (30-60 seconds is typical). The response includes not just the fee but
also a failure reason if the probe fails. Common failure reasons include
NO_ROUTE (destination unreachable), INSUFFICIENT_BALANCE (not enough funds), or
TIMEOUT (probe took too long). Handle these gracefully in your UI to guide users
appropriately.
## Future Improvements
The `EstimateRouteFee` implementation continues to evolve based on real-world
usage patterns. Ongoing discussions in the LND community focus on:
**Multi-Path Payment Support**: Extending fee estimation to support MPP
scenarios where payments split across multiple routes.
**Trampoline Routing Compatibility**: Adapting fee estimation for future
trampoline routing implementations where intermediate nodes handle pathfinding.
**Blinded Path Integration**: Ensuring fee estimation works correctly with
blinded paths as they become more prevalent in the network.
Wallet developers should monitor LND releases and participate in community
discussions to stay informed about improvements and changes to fee estimation
behavior.
## Conclusion
`EstimateRouteFee` provides essential functionality for wallet applications to
present accurate fee information to users. By understanding its dual-mode
operation, hop hint processing, and LSP detection heuristics, developers can
build robust payment experiences that handle both simple public node payments
and complex private channel scenarios.
The key to successful integration lies in choosing the appropriate estimation
mode for each use case, handling edge cases gracefully, and presenting fee
information in a way that helps users make informed payment decisions. As the
Lightning Network evolves, staying informed about `EstimateRouteFee`
improvements will ensure wallets continue to provide accurate and reliable fee
estimates.
---
### Forwarding History Privacy
# Forwarding History Privacy Management
## Introduction
The Lightning Network excels at providing fast, low-cost payments with strong
privacy properties. However, routing nodes and Lightning Service Providers
(LSPs) face a unique challenge: their operational databases accumulate
forwarding history that, if compromised or subpoenaed, could reveal sensitive
information about payment flows across the network. This document explores the
privacy implications of forwarding logs and introduces LND's solution for
implementing data retention policies without migrating to a new node instance.
## Understanding Forwarding History
When your LND node routes a payment between two other nodes, it records detailed
information about that forwarding event in its database. This serves several
important operational purposes, including fee accounting, channel performance
analysis, and troubleshooting. Each forwarding event captures the incoming and
outgoing channels, amounts transferred, fees earned, and precise timestamps.
Over months or years of operation, a busy routing node accumulates millions of
these records. While this historical data provides valuable insights into node
performance, it also creates a potential privacy liability. An attacker who
gains access to this database—whether through a security breach, or physical
seizure—could potentially reconstruct payment paths across the network by
correlating forwarding events across multiple compromised nodes.
## Privacy Implications for Operators
For individual routing node operators, the privacy risks of retaining unlimited
forwarding history are modest but real. If an adversary gains access to your
node's database, they could analyze your forwarding patterns to infer
information about the network topology you participate in and potentially
identify payment patterns involving your channels.
### The Traditional Dilemma
Prior to this feature, routing node operators faced an uncomfortable tradeoff.
To implement a data retention policy and purge old forwarding logs, the only
practical option was to shut down the node, reset the database, and restore
channels from backups—effectively migrating to a fresh node instance. This
process carries significant operational risks, including potential channel
closures, loss of channel state, and extended downtime. For LSPs serving
customers around the clock, such maintenance windows are highly disruptive.
## The DeleteForwardingHistory Solution
LND's `DeleteForwardingHistory` RPC addresses this challenge by providing a
safe, reversible-only-forward way to implement data retention policies. The
feature allows operators to specify a time threshold—either as a relative
duration or an absolute timestamp—and permanently delete all forwarding events
older than that threshold. The deletion operation executes in configurable
batches to avoid holding large database locks, and it returns statistics about
the deleted events, including the total fees earned during that period for
accounting purposes.
### How It Works
The deletion mechanism operates at the database layer, directly manipulating the
forwarding log bucket in LND's embedded bbolt database. The forwarding log
stores events using nanosecond-precision timestamps as keys, which enables
efficient time-based range queries. When you invoke a deletion, LND constructs a
cursor-based iteration that walks through events in chronological order,
collecting keys for events older than your specified cutoff time. It then
deletes these events in batches, with each batch executed within its own
database transaction.
```mermaid
sequenceDiagram
participant User
participant CLI
participant Router RPC
participant ForwardingLog
participant Database
User->>CLI: deletefwdhistory --age="-720h"
CLI->>Router RPC: DeleteForwardingHistory(duration: "-720h")
Router RPC->>Router RPC: Parse duration → absolute time
Router RPC->>Router RPC: Validate minimum age (1 hour)
loop For each batch (default: 10,000 events)
Router RPC->>ForwardingLog: DeleteForwardingEvents(endTime, batchSize)
ForwardingLog->>Database: Begin transaction
ForwardingLog->>Database: Iterate events <= endTime
ForwardingLog->>ForwardingLog: Calculate fees for batch
ForwardingLog->>Database: Delete batch of keys
ForwardingLog->>Database: Commit transaction
end
ForwardingLog->>Router RPC: Return stats (deleted count, total fees)
Router RPC->>CLI: DeleteForwardingHistoryResponse
CLI->>User: Display deletion results
```
This batched approach ensures that even nodes with millions of forwarding events
can safely purge old data without causing database performance issues. Each
batch completes within a separate transaction, limiting lock contention and
allowing other database operations to proceed between batches.
### Security Considerations
The implementation includes several safeguards to prevent accidental data loss.
First, the RPC enforces a minimum age requirement: you cannot delete events less
than one hour old. This prevents mishaps where an operator accidentally deletes
recent forwarding history due to a timestamp parsing error or misunderstanding
the time format. The CLI command additionally requires explicit confirmation
before proceeding with the deletion.
Second, the RPC requires the "offchain:write" macaroon permission, treating
forwarding history deletion as a sensitive write operation similar to payment
deletion. This ensures that only authorized users can purge forwarding data.
Third, the operation is logged extensively. LND writes detailed log messages
before and after each deletion operation, recording the time threshold, batch
size, number of events deleted, and total fees from the deleted period. These
audit trails help operators verify that deletions executed as intended.
### Fee Accounting
One critical requirement for LSPs implementing data retention policies is
maintaining accurate accounting records. Even after purging old forwarding
events for privacy reasons, operators need to know how much revenue their node
generated during those periods for tax reporting and business analytics.
The deletion operation addresses this by calculating and returning the sum of
all fees earned from the deleted events. For each event, LND computes the fee as
the difference between the incoming and outgoing amounts, then aggregates these
fees across all deleted events. The response includes this total in
millisatoshis, allowing operators to record their earnings before purging the
detailed records.
```mermaid
graph TD
A[Forwarding Event] --> B{Calculate Fee}
B --> C[Fee = AmtIn - AmtOut]
C --> D[Accumulate to TotalFees]
D --> E{More Events?}
E -->|Yes| A
E -->|No| F[Return Total to User]
F --> G[Operator Records
for Accounting]
G --> H[Delete Detailed Events]
```
This approach separates accounting data from operational surveillance data. You
can maintain aggregate financial records while minimizing the detailed
forwarding logs that pose privacy risks.
## Usage Guide
### Command Line Interface
The `lncli deletefwdhistory` command provides the primary interface for
operators. The command accepts time specifications in two formats: relative
durations for convenience, or absolute Unix timestamps for precision.
For most use cases, relative durations offer the most intuitive interface. To
implement a 90-day retention policy, you would periodically run:
```bash
lncli deletefwdhistory --age="-90d"
```
The supported time units cover a wide range of retention policies:
- Seconds (`s`) and minutes (`m`) for testing or very short-term retention
- Hours (`h`) and days (`d`) for common operational timeframes
- Weeks (`w`) for weekly cleanup schedules
- Months (`M`, averaged to 30.44 days) for typical retention policies
- Years (`y`, averaged to 365.25 days) for long-term archives
The minus sign prefix indicates you're specifying how far back in time to
delete. This convention matches the relative time syntax used elsewhere in LND
and makes the intent clear: "delete events from more than X time ago."
For precise control, you can specify an absolute Unix timestamp:
```bash
lncli deletefwdhistory --before=1704067200
```
This deletes all events before January 1, 2024 00:00:00 UTC. Absolute timestamps
are particularly useful when implementing policies tied to specific dates, such
as calendar year boundaries for accounting purposes or regulatory compliance
deadlines.
### Batch Size Tuning
The `--batch_size` flag controls how many events are deleted per database
transaction. The default value of 10,000 provides a good balance for most nodes,
but you may want to adjust this based on your node's characteristics.
For nodes with slower disk I/O or running on resource-constrained hardware,
reducing the batch size decreases the duration of each database lock, improving
responsiveness to concurrent operations:
```bash
lncli deletefwdhistory --age="-1M" --batch_size=5000
```
Conversely, for nodes with fast SSDs and low concurrent load, increasing the
batch size can speed up the overall deletion process:
```bash
lncli deletefwdhistory --age="-1M" --batch_size=25000
```
The implementation caps the maximum batch size at 50,000 to prevent excessively
large transactions from degrading database performance.
### Automation and Scheduling
Most operators will want to automate forwarding history cleanup rather than
running deletions manually. The command integrates naturally with cron jobs or
systemd timers. For a monthly cleanup maintaining a 90-day retention window:
```bash
# Run at 3 AM on the first day of each month
0 3 1 * * /usr/local/bin/lncli deletefwdhistory --age="-90d" --force >> /var/log/lnd/fwdhistory_cleanup.log 2>&1
```
The `--force` flag skips the interactive confirmation prompt, which is
required for unattended automation. In production you should implement
additional safeguards such as pre-deletion validation checks and alerting
on unexpected results.
For more sophisticated automation, consider implementing a script that:
1. Queries current forwarding history statistics
2. Calculates the appropriate deletion threshold based on database size and growth rate
3. Executes the deletion
4. Records the fees returned for accounting
5. Monitors the resulting database size and alerts if disk space isn't reclaimed as expected
### Database Compaction
Deleting forwarding events frees space within LND's bbolt database, but this
space isn't immediately returned to the operating system. bbolt uses a
copy-on-write structure where deleted data leaves "free pages" that can be
reused for future writes, but the overall file size doesn't shrink until you
compact the database.
LND supports automatic compaction via the configuration option:
```
db.bolt.auto-compact=true
```
With auto-compaction enabled, LND periodically performs compaction during normal
operation, typically triggered when the amount of free space exceeds a
threshold. However, after a large deletion operation, you may want to trigger
compaction immediately to reclaim disk space.
The recommended approach is to schedule compaction shortly after your regular
deletion operations:
1. Run `deletefwdhistory` to purge old events
2. Restart LND with `--db.bolt.auto-compact=true` if not already enabled
3. Monitor database file size to confirm space reclamation
Be aware that database compaction requires free disk space equal to the current
database size during the operation, as it creates a new, compacted copy of the
database before replacing the original.
## Integration with Existing Tools
### Forwarding History Analysis
The deletion operation doesn't interfere with LND's existing `forwardinghistory`
RPC, which allows you to query and analyze forwarding events. After a deletion,
queries for time ranges that have been purged will simply return no events for
those periods, while more recent events remain accessible.
This means you can continue using analytical tools and scripts that query
forwarding history, but you should design them to handle sparse historical data
gracefully. Tools should not assume that forwarding history extends back to the
node's inception date.
### Channel Analytics
Similarly, channel performance analysis tools that rely on forwarding history
will only have access to events within your retention window. When evaluating
channel performance metrics like forwarding frequency or fee revenue, be mindful
that historical data before your retention cutoff is no longer available.
For long-term performance tracking, consider aggregating statistics before
purging detailed events. You might maintain summary records showing weekly or
monthly aggregate forwarding counts and fees per channel, even after deleting
the individual event records.
## Privacy Best Practices
While the deletion feature provides operators with a mechanism to implement data
retention policies, it's important to understand what it does and doesn't
protect against.
### What Deletion Protects
Deleting old forwarding history reduces your node's exposure if the database is
compromised in the future. An attacker who gains access to your node after
you've implemented a 90-day retention policy can only observe the last 90 days
of forwarding activity, not the entire operational history. This limits the
window during which surveillance or correlation attacks could be performed using
your node's data.
### What Deletion Doesn't Protect
The revocation log for _active_ channels contains information that can be used
to reconstruct transaction flows. Once channels are closed, this data is
automatically deleted.
The normal logs of a node also contain information that can be used to correlate
transactions. Users can set up automated systems to manually purge logs, or
configure the logging directory to a purely in-memory file system.
### Defense in Depth
Forwarding history deletion should be one component of a comprehensive privacy
strategy, not your only defense. Other important measures include:
- Restricting physical and network access to the node
- Implementing strong authentication and access controls
- Regularly auditing who has access to the node and its backups
- Using channel aliases and avoiding personally identifiable information in
channel names
- Running your node over Tor to hide the network-level correlation between node
identity and IP address
The deletion feature gives you control over how long your node retains detailed
forwarding records, but it doesn't eliminate all privacy risks inherent in
operating a Lightning Network routing node.
## Troubleshooting
### Database Lock Timeouts
During deletion of very large numbers of events, you might encounter database
lock timeout errors if other operations are trying to access the database
concurrently. If this occurs:
1. Reduce the batch size to shorten each transaction
2. Schedule deletions during low-traffic periods
3. Temporarily pause other operations that query forwarding history frequently
### Insufficient Disk Space for Compaction
Database compaction requires temporary free space roughly equal to the size of
your database. If compaction fails due to insufficient disk space, you'll need
to free up space before the compaction can proceed:
1. Delete other unnecessary files from the disk
2. Move log files or other non-critical data to alternate storage
3. Consider whether you can safely delete older database backups
## Performance Considerations
Deletion performance scales linearly with the number of events being deleted.
Performance varies significantly depending on storage hardware and database
size; operators should benchmark on their own hardware before relying on
specific throughput estimates.
The operation's impact on node performance during deletion is minimal. Each
batch executes quickly, and the gaps between batches allow other database
operations to proceed. You can safely run deletions while the node is actively
routing payments, though you may want to avoid doing so during peak traffic
times on very busy nodes.
Database compaction has a more significant performance impact, as it requires
LND to copy the entire database. During compaction, expect elevated CPU and disk
I/O, and budget several minutes for the operation to complete depending on your
database size. LND remains operational during compaction, but you may observe
increased latency for database-heavy operations.
---
### Fuzz
# Fuzzing LND #
The following runs all fuzz tests on default settings:
```shell
$ make fuzz
```
The following runs all fuzz tests inside the lnwire package, each for a total of 1 minute, using 4 procs.
It is recommended that processes be set to the number of processor cores in the system:
```shell
$ make fuzz pkg=lnwire fuzztime=1m parallel=4
```
Alternatively, individual fuzz tests can be run manually by setting the working directory to the location of the .go file holding the fuzz tests.
The go test command can only test one fuzz test at a time:
```shell
$ cd lnwire
$ go test -fuzz=FuzzAcceptChannel -fuzztime=1m -parallel=4
```
The following can be used to show all fuzz tests in the working directory:
```shell
$ cd lnwire
$ go test -list=Fuzz.*
```
Fuzz tests can be run as normal tests, which only runs the seed corpus:
```shell
$ cd lnwire
$ go test -run=FuzzAcceptChannel -parallel=4
```
The generated corpus values can be found in the $(go env GOCACHE)/fuzz directory.
## Options ##
Several parameters can be appended to the end of the make commands to tune the build process or the way the fuzzer runs.
- `fuzztime` specifies how long each fuzz test runs for, corresponding to the `go test -fuzztime` option. The default is 30s.
- `parallel` specifies the number of parallel processes to use while running the harnesses, corresponding to the `go test -parallel` option.
- `pkg` specifies the `lnd` packages to build or fuzz. The default is to build and run all available packages (`brontide lnwire watchtower/wtwire zpay32`). This can be changed to build/run against individual packages.
## Corpus ##
Fuzzing generally works best with a corpus that is of minimal size while achieving the maximum coverage.
## Disclosure ##
If you find any crashers that affect LND security, please disclose with the information found [here](https://github.com/lightningnetwork/lnd/#security).
---
### Gossip Rate Limiting
# Gossip Rate Limiting Configuration Guide
When running a Lightning node, one of the most critical yet often overlooked
aspects is properly configuring the gossip rate limiting system. This guide will
help you understand how LND manages outbound gossip traffic and how to tune
these settings for your specific needs.
## Understanding Gossip Rate Limiting
At its core, LND uses a token bucket algorithm to control how much bandwidth it
dedicates to sending gossip messages to other nodes. Think of it as a bucket
that fills with tokens at a steady rate. Each time your node sends a gossip
message, it consumes tokens equal to the message size. If the bucket runs dry,
messages must wait until enough tokens accumulate.
This system serves an important purpose: it prevents any single peer, or group
of peers, from overwhelming your node's network resources. Without rate
limiting, a misbehaving peer could request your entire channel graph repeatedly,
consuming all your bandwidth and preventing normal operation.
## Core Configuration Options
The gossip rate limiting system has several configuration options that work
together to control your node's behavior.
### Setting the Sustained Rate: gossip.msg-rate-bytes
The most fundamental setting is `gossip.msg-rate-bytes`, which determines how
many bytes per second your node will allocate to outbound gossip messages. This
rate is shared across all connected peers, not per-peer.
The default value of 102,400 bytes per second (100 KB/s) works well for most
nodes, but you may need to adjust it based on your situation. Setting this value
too low can cause serious problems. When the rate limit is exhausted, peers
waiting to synchronize must queue up, potentially waiting minutes between
messages. Values below 50 KB/s can make initial synchronization fail entirely,
as peers timeout before receiving the data they need.
### Managing Burst Capacity: gossip.msg-burst-bytes
The burst capacity, configured via `gossip.msg-burst-bytes`, determines the
initial capacity of your token bucket. This value must be greater than
`gossip.msg-rate-bytes` for the rate limiter to function properly. The burst
capacity represents the maximum number of bytes that can be sent immediately
when the bucket is full.
The default of 204,800 bytes (200 KB) is set to be double the default rate
(100 KB/s), providing a good balance. This ensures that when the rate limiter
starts or after a period of inactivity, you can send up to 200 KB worth of
messages immediately before rate limiting kicks in. Any single message larger
than this value can never be sent, regardless of how long you wait.
### Controlling Concurrent Operations: gossip.filter-concurrency
When peers apply gossip filters to request specific channel updates, these
operations can consume significant resources. The `gossip.filter-concurrency`
setting limits how many of these operations can run simultaneously. The default
value of 5 provides a reasonable balance between resource usage and
responsiveness.
Large routing nodes handling many simultaneous peer connections might benefit
from increasing this value to 10 or 15, while resource-constrained nodes should
keep it at the default or even reduce it slightly.
### Preventing Spam: gossip.ban-threshold
To protect your node from spam and misbehaving peers, LND uses a ban score
system controlled by `gossip.ban-threshold`. Each time a peer sends a gossip
message that is considered invalid, its ban score is incremented. Once the score
reaches this threshold, the peer is banned for a default of 48 hours, and your
node will no longer process gossip messages from them.
A gossip message can be considered invalid for several reasons, including:
- Invalid signature on the announcement.
- Stale timestamp, older than what we already have.
- Too many channel updates for the same channel in a short period.
- Announcing a channel that is not found on-chain.
- Announcing a channel that has already been closed.
- Announcing a channel with an invalid proof.
The default value is 100. Setting this value to 0 disables banning completely,
which is not recommended for most operators.
### Understanding Connection Limits: num-restricted-slots
The `num-restricted-slots` configuration deserves special attention because it
directly affects your gossip bandwidth requirements. This setting limits inbound
connections, but not in the way you might expect.
LND maintains a three-tier system for peer connections. Peers you've ever had
channels with enjoy "protected" status and can always connect. Peers currently
opening channels with you have "temporary" status. Everyone else—new peers
without channels—must compete for the limited "restricted" slots.
When a new peer without channels connects inbound, they consume one restricted
slot. If all slots are full, additional peers are turned away. However, as soon
as a restricted peer begins opening a channel, they're upgraded to temporary
status, freeing their slot. This creates breathing room for large nodes to form
new channel relationships without constantly rejecting connections.
The relationship between restricted slots and rate limiting is straightforward:
more allowed connections mean more peers requesting data, requiring more
bandwidth. A reasonable rule of thumb is to allocate at least 1 KB/s of rate
limit per restricted slot.
## Calculating Appropriate Values
To set these values correctly, you need to understand your node's position in
the network and its typical workload. The fundamental question is: how much
gossip traffic does your node actually need to handle?
Start by considering how many peers typically connect to your node. A hobbyist
node might have 10-20 connections, while a well-connected routing node could
easily exceed 100. Each peer generates gossip traffic when syncing channel
updates, announcing new channels, or requesting historical data.
The calculation itself is straightforward. Take your average message size
(approximately 210 bytes for gossip messages), multiply by your peer count and
expected message frequency, then add a safety factor for traffic spikes. Since
each channel generates approximately 842 bytes of bandwidth (including both
channel announcements and updates), you can also calculate based on your
channel count. Here's the formula:
```
rate = avg_msg_size × peer_count × msgs_per_second × safety_factor
```
Let's walk through some real-world examples to make this concrete.
For a small node with 15 peers, you might see 10 messages per peer per second
during normal operation. With an average message size of 210 bytes and a safety
factor of 1.5, you'd need about 47 KB/s. Rounding up to 50 KB/s provides
comfortable headroom.
A medium-sized node with 75 peers faces different challenges. These nodes often
relay more traffic and handle more frequent updates. With 15 messages per peer
per second, the calculation yields about 237 KB/s. Setting the limit to 250 KB/s
ensures smooth operation without waste.
Large routing nodes require the most careful consideration. With 150 or more
peers and high message frequency, bandwidth requirements can exceed 1 MB/s.
These nodes form the backbone of the Lightning Network and need generous
allocations to serve their peers effectively.
Remember that the relationship between restricted slots and rate limiting is
direct: each additional slot potentially adds another peer requesting data. Plan
for at least 1 KB/s per restricted slot to maintain healthy synchronization.
## Network Size and Geography
The Lightning Network's growth directly impacts your gossip bandwidth needs.
With over 80,000 public channels at the time of writing, each generating
multiple updates daily, the volume of gossip traffic continues to increase. A
channel update occurs whenever a node adjusts its fees, changes its routing
policy, or goes offline temporarily. During volatile market conditions or fee
market adjustments, update frequency can spike dramatically.
Geographic distribution adds another layer of complexity. If your node connects
to peers across continents, the inherent network latency affects how quickly you
can exchange messages. However, this primarily impacts initial connection
establishment rather than ongoing rate limiting.
## Troubleshooting Common Issues
When rate limiting isn't configured properly, the symptoms are often subtle at
first but can cascade into serious problems.
The most common issue is slow initial synchronization. New peers attempting to
download your channel graph experience long delays between messages. You'll see
entries in your logs like "rate limiting gossip replies, responding in 30s" or
even longer delays. This happens because the rate limiter has exhausted its
tokens and must wait for refill. The solution is straightforward: increase your
msg-rate-bytes setting.
Peer disconnections present a more serious problem. When peers wait too long for
gossip responses, they may timeout and disconnect. This creates a vicious cycle
where peers repeatedly connect, attempt to sync, timeout, and reconnect. Look
for "peer timeout" errors in your logs. If you see these, you need to increase
your rate limit.
Sometimes you'll notice unusually high CPU usage from your LND process. This
often indicates that many goroutines are blocked waiting for rate limiter
tokens. The rate limiter must constantly calculate delays and manage waiting
threads. Increasing the rate limit reduces this contention and lowers CPU usage.
To debug these issues, focus on your LND logs rather than high-level commands.
Search for "rate limiting" messages to understand how often delays occur and how
long they last. Look for patterns in peer disconnections that might correlate
with rate limiting delays. The specific commands that matter are:
```bash
# View peer connections and sync state
lncli listpeers | grep -A5 "sync_type"
# Check recent rate limiting events
grep "rate limiting" ~/.lnd/logs/bitcoin/mainnet/lnd.log | tail -20
```
Pay attention to log entries showing "Timestamp range queue full" if you've
implemented the queue-based approach—this indicates your system is shedding load
due to overwhelming demand.
## Best Practices for Configuration
Experience has shown that starting with conservative (higher) rate limits and
reducing them if needed works better than starting too low and debugging
problems. It's much easier to notice excess bandwidth usage than to diagnose
subtle synchronization failures.
Monitor your node's actual bandwidth usage and sync times after making changes.
Most operating systems provide tools to track network usage per process. When
adjusting settings, make gradual changes of 25-50% rather than dramatic shifts.
This helps you understand the impact of each change and find the sweet spot for
your setup.
Keep your burst size at least double the largest message size you expect to
send. While the default 200 KB is usually sufficient, monitor your logs for any
"message too large" errors that would indicate a need to increase this value.
As your node grows and attracts more peers, revisit these settings periodically.
What works for 50 peers may cause problems with 150 peers. Regular review
prevents gradual degradation as conditions change.
## Configuration Examples
For most users running a personal node, conservative settings provide reliable
operation without excessive resource usage:
```
[Application Options]
gossip.msg-rate-bytes=204800
gossip.msg-burst-bytes=409600
gossip.filter-concurrency=5
num-restricted-slots=100
```
Well-connected nodes that route payments regularly need more generous
allocations:
```
[Application Options]
gossip.msg-rate-bytes=524288
gossip.msg-burst-bytes=1048576
gossip.filter-concurrency=10
num-restricted-slots=200
```
Large routing nodes at the heart of the network require the most resources:
```
[Application Options]
gossip.msg-rate-bytes=1048576
gossip.msg-burst-bytes=2097152
gossip.filter-concurrency=15
num-restricted-slots=300
```
## Critical Warning About Low Values
Setting `gossip.msg-rate-bytes` below 50 KB/s creates serious operational
problems that may not be immediately obvious. Initial synchronization, which
typically transfers 10-20 MB of channel graph data, can take hours or fail
entirely. Peers appear to connect but remain stuck in a synchronization loop,
never completing their initial download.
Your channel graph remains perpetually outdated, causing routing failures as you
attempt to use channels that have closed or changed their fee policies. The
gossip subsystem appears to work, but operates so slowly that it cannot keep
pace with network changes.
During normal operation, a well-connected node processes hundreds of channel
updates per minute. Each update is small, but they add up quickly. Factor in
occasional bursts during network-wide fee adjustments or major routing node
policy changes, and you need substantial headroom above the theoretical minimum.
The absolute minimum viable configuration requires at least enough bandwidth to
complete initial sync in under an hour and process ongoing updates without
falling behind. This translates to no less than 50 KB/s for even the smallest
nodes.
---
### Key Import
# Overview
This document serves as an introductory point for users interested in reducing
their hot-wallet risks, allowing them to maintain on-chain funds outside of
`lnd` but still be able to manage them within `lnd`. As of `v0.13.0-beta`, `lnd`
is able to import BIP-0049 and BIP-0084 extended public keys either at the
account path (`m/purpose'/coin_type'/account'`) or at the address index path
(`m/purpose'/coin_type'/account'/change/address_index`) as watch-only through
the `WalletKit` APIs.
Note that in order to follow the rest of this document and/or use the
`WalletKit` APIs, users will need to obtain an `lnd` build compiled with the
`walletrpc` tag. Our release builds already include this tag by default, so this
would only be necessary when compiling from source.
# `lnd`'s Default Wallet Accounts
Upon initializing `lnd`, a wallet is created with four default accounts:
* A custom BIP-0049 account (more on this later) to generate NP2WKH external
addresses.
* A BIP-0084 account to generate P2WKH external and change addresses.
* A catch-all BIP-0049 account where all imported BIP-0049 address keys (NP2WKH
addresses) exist within.
* A catch-all BIP-0084 account where all imported BIP-0049 address keys (P2WKH
addresses) exist within.
Prior to `v0.13.0-beta`, these accounts were abstracted away from users. As part
of the key import feature, they are now exposed through the new `WalletKit` RPCs
(`ListAccounts`, `ImportAccount`, `ImportPublicKey`) and the `lncli wallet
accounts` command.
```shell
$ lncli wallet accounts
NAME:
lncli wallet accounts - Interact with wallet accounts.
USAGE:
lncli wallet accounts command [command options] [arguments...]
COMMANDS:
list Retrieve information of existing on-chain wallet accounts.
import Import an on-chain account into the wallet through its extended public key.
import-pubkey Import a public key as watch-only into the wallet.
OPTIONS:
--help, -h show help
```
## Account Details
Before interacting with the new set of APIs, users will want to become familiar
with how wallet accounts are represented within `lnd`. The
`WalletKit.ListAccounts` RPC or `lncli wallet accounts list` command can be used
to retrieve the details of accounts.
```shell
$ lncli wallet accounts list
{
"accounts": [
{
"name": "default",
"address_type": "HYBRID_NESTED_WITNESS_PUBKEY_HASH",
"extended_public_key": "upub5EbJZz2tYCpPFgDAMDnXpTeLs5EMNJAfyzRKQuUiTugSaJDjnDdk9vNcENzpw1FnxkerNW7jLuBeoxmcGMtopGExmaWqrMB7wRgU8tExTMz",
"master_key_fingerprint": null,
"derivation_path": "m/49'/0'/0'",
"external_key_count": 0,
"internal_key_count": 0,
"watch_only": false
},
{
"name": "default",
"address_type": "WITNESS_PUBKEY_HASH",
"extended_public_key": "vpub5Z9beF6NYCrHeDmKC38tM3xXMDFFSARa9sdHRPChEMGqtxiELfZB8hm6FwBpBvfPpX2HGG8edYVV9Wupe43PEJJhhfnz1egtQNNaDXyYExn",
"master_key_fingerprint": null,
"derivation_path": "m/84'/0'/0'",
"external_key_count": 0,
"internal_key_count": 0,
"watch_only": false
}
]
}
```
There's a lot to unpack in the response above, so let's cover each account field
in detail. As mentioned above, four default accounts should exist, though only
two are shown in the output. The catch-all imported accounts are hidden by
default until a key has been imported into them.
* `name`: Each account has a name it can be identified by. `lnd`'s default
spendable accounts have the name "default". The default catch-all imported
accounts have the name "imported".
* `extended_public_key`: The BIP-0044 extended public key for the account. Any
addresses generated for the account are derived from this key. Each key has a
version prefix that identifies the chain and derivation scheme being used. At
the time of writing, `lnd` supports the following versions:
* `xpub/tpub`: The commonly used version prefix originally intended for
BIP-0032 mainnet/testnet extended keys. Since `lnd` does not support
BIP-0032 extended keys, this version serves as a catch-all for the other
versions.
* `ypub/upub`: The version prefix for BIP-0049 mainnet/testnet extended keys.
* `zpub/vpub`: The version prefix for BIP-0084 mainnet/testnet extended keys.
* `address_type`: The type of addresses the account can derive. There are three
supported address types:
* `WITNESS_PUBKEY_HASH`: The standard derivation scheme for BIP-0084 with
P2WKH for external and change addresses.
* `NESTED_WITNESS_PUBKEY_HASH`: The standard derivation scheme for BIP-0049
with P2WKH for external and change addresses.
* `HYBRID_NESTED_WITNESS_PUBKEY_HASH` A custom derivation scheme for BIP-0049
used by `lnd` where NP2WKH is used for external addresses and P2WKH for
change addresses.
* `master_key_fingerprint`: The 4 byte fingerprint of the master key
corresponding to the account. This is usually required by hardware
wallet/external signers to identify the proper signing key.
* `derivation_path`: The BIP-0044 derivation path used on the master key to
obtain the account key.
* `external_key_count`: The number of external addresses generated.
* `internal_key_count`: The number of change addresses generated.
* `watch_only`: Whether the wallet has private key information for the account.
`lnd`'s default wallet accounts always have private key information, so this
value is `false`.
# Key Import
An existing limitation to the key import APIs is that events (deposits/spends)
for imported keys, including those derived from an imported account, will only
be detected by lnd if they happen after the import. Rescans to detect past
events are currently not supported, but will come at a later time.
## Account Key Import
The `WalletKit.ImportAccount` RPC and `lncli wallet accounts import` command can
be used to import an account. At the time of writing, importing an account has
the following request parameters:
* `name` (required): A name to identify the imported account with.
* `extended_public_key` (required): A public key that corresponds to a wallet account
represented as an extended key. It must conform to a derivation path of the
form `m/purpose'/coin_type'/account'`.
* `master_key_fingerprint` (optional): The fingerprint of the root key (also
known as the key with derivation path m/) from which the account public key
was derived from. This may be required by some hardware wallets for proper
identification and signing.
* `address_type` (optional): An address type is only required when the extended
account public key has a legacy version (xpub, tpub, etc.), such that the
wallet cannot detect what address scheme it belongs to.
* `dry_run` (optional): Whether a dry run should be attempted when importing the
account. This serves as a way to confirm whether the account is being imported
correctly by returning the first N addresses for the external and internal
branches of the account. If these addresses match as expected, then it should
be safe to import the account as is.
For the sake of simplicity, we'll present an example with two `lnd` nodes Alice
and Bob, where Alice acts as a signer _only_, and Bob manages Alice's on-chain
BIP-0084 account by crafting transactions and watching/spending addresses. Since
Alice will only act as a signer, we'll want to import her BIP-0084 account into
Bob's node, which will require knowledge of Alice's extended public key.
Alice's BIP-0084 extended public key can be obtained as follows.
```shell
$ lncli-alice wallet accounts list --name=default --address_type=p2wkh
{
"accounts": [
{
"name": "default",
"address_type": "WITNESS_PUBKEY_HASH",
"extended_public_key": "vpub5Z9beF6NYCrHeDmKC38tM3xXMDFFSARa9sdHRPChEMGqtxiELfZB8hm6FwBpBvfPpX2HGG8edYVV9Wupe43PEJJhhfnz1egtQNNaDXyYExn",
"master_key_fingerprint": null,
"derivation_path": "m/84'/0'/0'",
"external_key_count": 0,
"internal_key_count": 0,
"watch_only": false
}
]
}
```
Bob can then import the account with the following command:
```shell
$ lncli-bob wallet accounts import vpub5Z9beF6NYCrHeDmKC38tM3xXMDFFSARa9sdHRPChEMGqtxiELfZB8hm6FwBpBvfPpX2HGG8edYVV9Wupe43PEJJhhfnz1egtQNNaDXyYExn alice
```
Before Bob imports the account, they may want to confirm the account is being
imported using the correct derivation scheme. This can be done with the dry run
request parameter. When a dry run is done, the response will include the usual
account details, as well as the first 5 external and change addresses, which can
be used to confirm they match with what the account owner expects.
```shell
$ lncli-bob wallet accounts import vpub5Z9beF6NYCrHeDmKC38tM3xXMDFFSARa9sdHRPChEMGqtxiELfZB8hm6FwBpBvfPpX2HGG8edYVV9Wupe43PEJJhhfnz1egtQNNaDXyYExn alice --dry_run
{
"account": {
"name": "alice",
"address_type": "WITNESS_PUBKEY_HASH",
"extended_public_key": "vpub5Z9beF6NYCrHeDmKC38tM3xXMDFFSARa9sdHRPChEMGqtxiELfZB8hm6FwBpBvfPpX2HGG8edYVV9Wupe43PEJJhhfnz1egtQNNaDXyYExn",
"master_key_fingerprint": null,
"derivation_path": "m/84'/0'/0'",
"external_key_count": 0,
"internal_key_count": 0,
"watch_only": true
},
"dry_run_external_addrs": [
"bcrt1q8zdjz2q92eh7jw9ah3upf2u9553226gq79el5l",
"bcrt1qmx2m4ngd2el0rmmcu0mz453yzzl3aq9mag0l79",
"bcrt1q904yve7yvt2t3v0s5r7rueweh4jjr3enfgam8w",
"bcrt1qa7k20jwfvsep8x0dx4jfu9xm0tlwaa8wrrgl77",
"bcrt1qzypxx35cfsl24mslqextetuc5m8vvadlqp20d8"
],
"dry_run_internal_addrs": [
"bcrt1qlstwh8ecy7szfw7k6rllc4ajkg6922xjwj6a23",
"bcrt1qdrz9glz4ld7uyxwv3jz2anx4k9pe3zm86hpy9g",
"bcrt1qfdu6tfhs85q20tf48nhtx0kjgr0t2j25apm90t",
"bcrt1qkmysm9wlnhyyc4uhfaxyafj6q3e3ujcnh97cqc",
"bcrt1qw8hhmdg3atfp7dcwjtysq4kcmnh07kjy2rd2ay"
]
}
```
Once Bob has confirmed the correct account derivation scheme is being used, the
account can be imported without the dry run parameter.
```shell
$ lncli-bob wallet accounts import vpub5Z9beF6NYCrHeDmKC38tM3xXMDFFSARa9sdHRPChEMGqtxiELfZB8hm6FwBpBvfPpX2HGG8edYVV9Wupe43PEJJhhfnz1egtQNNaDXyYExn alice
{
"account": {
"name": "alice",
"address_type": "WITNESS_PUBKEY_HASH",
"extended_public_key": "vpub5Z9beF6NYCrHeDmKC38tM3xXMDFFSARa9sdHRPChEMGqtxiELfZB8hm6FwBpBvfPpX2HGG8edYVV9Wupe43PEJJhhfnz1egtQNNaDXyYExn",
"master_key_fingerprint": null,
"derivation_path": "m/84'/0'/0'",
"external_key_count": 0,
"internal_key_count": 0,
"watch_only": true
}
}
```
### Generating Addresses from an Imported Account
External addresses from an imported account can be generated through the
existing `Lightning.NewAddress` RPC and `lncli newaddress` command, as they now
take an additional optional parameter to specify which account the address
should be derived from.
Following the example above, Bob is able to generate an external address for an
incoming deposit as follows:
```shell
$ lncli-bob newaddress p2wkh --account=alice
{
"address": "bcrt1q8zdjz2q92eh7jw9ah3upf2u9553226gq79el5l"
}
```
Change addresses cannot be generated on demand, they are generated automatically
when a transaction is crafted that requires a change output.
### Crafting Transactions through PSBTs from an Imported Account
Assuming a deposit of 1 tBTC was made to the address above
(`bcrt1q8zdjz2q92eh7jw9ah3upf2u9553226gq79el5l`), Bob should be able to craft a
transaction spending their new UTXO. Since Bob is unable to sign the transaction
themselves, they'll use PSBTs to craft the transaction, and provide it to Alice
to sign.
```shell
$ lncli-bob wallet psbt fund --account=alice --outputs="{\"bcrt1qpjqr663tylcksysa4u76xvremee9k8af3pqd5h\": 500000}" --sat_per_vbyte=1
{
"psbt": "cHNidP8BAHECAAAAAYDHzEGcDW4Qf+gVbIgWpG2PVSUY6aZ3xUGk/3Ia/XnJAAAAAAD/////AiChBwAAAAAAFgAUDIA9aisn8WgSHa89ozB53nJbH6lWNf4pAQAAABYAFPwW6584J6Aku9bQ//xXsrI0VSjSAAAAAAABAKgCAAAAAAEBAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAD/////A1oBAf////8CAPIFKgEAAAAWABQ4myEoBVZv6Ti9vHgUq4WlIqVpAAAAAAAAAAAAJmokqiGp7eL2HD9x0d79P6mZ36NpU3VcaQaJeZlitIvr2DaXToz5ASAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAR8A8gUqAQAAABYAFDibISgFVm/pOL28eBSrhaUipWkAAQMEAQAAACIGArbCQ3C0eTrSeuEokWjN7ty25lSzNxiClZL3tnbmlDG6GAAAAABUAACAAAAAgAAAAIAAAAAAAAAAAAAAAA==",
"change_output_index": 1,
"locks": [
{
"id": "ede19a92ed321a4705f8a1cccc1d4f6182545d4bb4fae08bd5937831b7e38f98",
"outpoint": "c979fd1a72ffa441c577a6e91825558f6da416886c15e87f106e0d9c41ccc780:0",
"expiration": 1621632493
}
]
}
```
The PSBT can then be provided to Alice to sign:
```
/* Detailed source-code truncated for AI context efficiency. */
```
---