delve Documentation

Delve is a debugger for the Go programming language.

24,879 stars Go Go 1 file ยท ~15,880 tokens #debugger#go#golang
RAW Doc

Docker & Self-Hosted Deployment Verified OCI

Deploy delve on any server, homelab, or cloud container platform.

1-Click Terminal Command
bash (docker run)
docker run -d --name delve --restart always go-delve/delve:latest

File: README.md



/statusIcon.svg)

The GitHub issue tracker is for bugs only. Please use the developer mailing list for any feature proposals and discussions.

About Delve

Delve is a debugger for the Go programming language. The goal of the project is to provide a simple, full featured debugging tool for Go. Delve should be easy to invoke and easy to use. Chances are if you're using a debugger, things aren't going your way. With that in mind, Delve should stay out of your way as much as possible.


File: Documentation/api/dap/README.md

DAP Interface

Delve exposes a DAP API interface.

This interface is served over a streaming TCP socket using dlv server in one of the two headless modes:

  1. dlv dap - starts a single-use DAP-only server that waits for a client to specify launch/attach configuration for starting the debug session.
  2. dlv --headless <command> <debuggee> - starts a general server, enters a debug session for the specified debuggee and waits for a JSON-RPC or a DAP remote-attach client to begin interactive debugging. Can be used in multi-client mode with the following options:
    • --accept-multiclient - use to support connections from multiple clients
    • --continue - use to resume debuggee execution as soon as server session starts

See Launch and Attach Configurations for more usage details of these two options.

The primary user of this mode is VS Code Go. Please see its
detailed debugging documentation for additional information.

Debug Adapter Protocol

DAP is a general debugging protocol supported by many tools and programming languages. We tailored it to Go specifics, such as mapping threads request to communicate goroutines and exceptionInfo request to support panics and fatal errors.

See dap.Server.handleRequest and capabilities set in dap.Server.onInitializeRequest for an up-to-date list of supported requests and options.

Launch and Attach Configurations

In addition to the general DAP spec, the server supports the following implementation-specific configuration options for starting the debug session:

Not all of the configurations are supported by each of the two available DAP servers:

Disconnect and Shutdown

Single-Client Mode

When used with dlv dap or dlv --headless --accept-multiclient=false (default), the DAP server will shut itself down at the end of the debug session, when the client sends a disconnect request. If the debuggee was launched, it will be taken down as well. If the debuggee was attached to, terminateDebuggee option will be respected.

When the program terminates, we send a terminated event, which is expected to trigger a disconnect request from the client for a session and a server shutdown. The restart request is supported.

The server also shuts down in case of a client connection error or SIGTERM signal, taking down a launched process, but letting an attached process continue.

Pressing Ctrl-C on the terminal where a headless server is running sends SIGINT to the debuggee, foregrounded in headless mode to support debugging interactive programs.

Multi-Client Mode

When used with dlv --headless --accept-multiclient=true, the DAP server will honor the multi-client mode when a client disconnects or client connection fails. The server will remain running and ready for a new client connection, and the debuggee will remain in whatever state it was at the time of disconnect - running or halted. Once suspendDebuggee option is supported by frontends like VS Code (vscode/issues/134412), we will update the server to offer this as a way to specify debuggee state on disconnect.

The client may request full shutdown of the server and the debuggee with terminateDebuggee option.

The server shuts down in response to a SIGTERM signal, taking down a launched process, but letting an attached process continue.

Pressing Ctrl-C on the terminal where a headless server is running sends SIGINT to the debuggee, foregrounded in headless mode to support debugging interactive programs.

Debugger Output

The debugger always logs one of the following on start-up to stdout:

  • dlv dap:
    • DAP server listening at: <host>:
  • dlv --headless:
    • API server listening at: <host>:

This can be used to confirm that server start-up succeeded.

The server uses output events to communicate errors and select status messages to the client. For example:

text
Step interrupted by a breakpoint. Use 'Continue' to resume the original step command.
invalid command: Unable to step while the previous step is interrupted by a breakpoint.
Use 'Continue' to resume the original step command.
Detaching and terminating target process

More detailed logging can be enabled with --log --log-output=dap as part of the dlv command.
It will record the server-side DAP message traffic. For example,

text
2022-01-04T00:27:57-08:00 debug layer=dap [<- from client]{"seq":1,"type":"request","command":"initialize","arguments":{"clientID":"vscode","clientName":"Visual Studio Code","adapterID":"go","locale":"en-us","linesStartAt1":true,"columnsStartAt1":true,"pathFormat":"path","supportsVariableType":true,"supportsVariablePaging":true,"supportsRunInTerminalRequest":true,"supportsMemoryReferences":true,"supportsProgressReporting":true,"supportsInvalidatedEvent":true}}
2022-01-04T00:27:57-08:00 debug layer=dap [-> to client]{"seq":0,"type":"response","request_seq":1,"success":true,"command":"initialize","body":{"supportsConfigurationDoneRequest":true,"supportsFunctionBreakpoints":true,"supportsConditionalBreakpoints":true,"supportsEvaluateForHovers":true,"supportsSetVariable":true,"supportsExceptionInfoRequest":true,"supportTerminateDebuggee":true,"supportsDelayedStackTraceLoading":true,"supportsLogPoints":true,"supportsDisassembleRequest":true,"supportsClipboardContext":true,"supportsSteppingGranularity":true,"supportsInstructionBreakpoints":true}}
2022-01-04T00:27:57-08:00 debug layer=dap [<- from client]{"seq":2,"type":"request","command":"launch","arguments":{"name":"Launch file","type":"go","request":"launch","mode":"debug","program":"./temp.go","hideSystemGoroutines":true,"__buildDir":"/Users/polina/go/src","__sessionId":"2ad0f0c1-a1fd-4fff-9fff-b8bc9a933fe5"}}
2022-01-04T00:27:57-08:00 debug layer=dap parsed launch config: {
	"mode": "debug",
	"program": "./temp.go",
	"backend": "default",
	"stackTraceDepth": 50,
	"hideSystemGoroutines": true
}
...

This logging is written to stderr and is not forwarded via
output events.

Debuggee Output

Debuggee's stdout and stderr are written to stdout and stderr respectfully and are not forwarded via
output events.

Versions

The initial DAP support was released in v1.6.1 with many additional improvements in subsequent versions. The remote attach support was added in v1.7.3. The restart request support was added in v1.25.1.

The DAP API changes are backward-compatible as all new features are opt-in only. To update to a new DAP version and import a new DAP feature into delve,
one must first update the go-dap dependency.


File: Documentation/api/json-rpc/README.md

JSON-RPC interface

Delve exposes a JSON-RPC API interface.

Note that this JSON-RPC interface is served over a streaming socket, not over HTTP.

API versions

Delve currently only supports v2 of its API. Support for v1 was dropped in version v1.24.0 of Delve.

API version 2 documentation

All the methods of the type service/rpc2.RPCServer can be called using JSON-RPC, the documentation for these calls is available on godoc.

Note that all exposed methods take one single input parameter (usually called args) of a struct type and also return a result of a struct type. Also note that the method name should be prefixed with RPCServer. in JSON-RPC.

Example

Your client wants to set a breakpoint on the function main.main.
The first step will be calling the method FindLocation with Scope = api.EvalScope{ GoroutineID: -1, Frame: 0} and Loc = "main.main". The JSON-RPC request packet should look like this:

text
{"method":"RPCServer.FindLocation","params":[{"Scope":{"GoroutineID":-1,"Frame":0},"Loc":"main.main"}],"id":2}

the response packet will look like this:

text
{"id":2,"result":{"Locations":[{"pc":4199019,"file":"/home/a/temp/callme/callme.go","line":31,"function":{"name":"main.main","value":4198992,"type":84,"goType":0}}]},"error":null}

Now your client should call the method CreateBreakpoint and specify 4199019 (the pc field in the response object) as the target address:

text
{"method":"RPCServer.CreateBreakpoint","params":[{"Breakpoint":{"addr":4199019}}],"id":3}

if this request is successful your client will receive the following response:

text
{"id":3,"result":{"Breakpoint":{"id":1,"name":"","addr":4199019,"file":"/home/a/temp/callme/callme.go","line":31,"functionName":"main.main","Cond":"","continue":false,"goroutine":false,"stacktrace":0,"LoadArgs":null,"LoadLocals":null,"hitCount":{},"totalHitCount":0}},"error":null}

File: Documentation/api/ClientHowto.md

How to write a Delve client, an informal guide

Spawning the backend

The dlv binary built by our Makefile contains both the backend and a
simple command line client. If you are writing your own client you will
probably want to run only the backend, you can do this by specifying the
--headless option, for example:

text
$ dlv --headless debug

The rest of the command line remains unchanged. You can use debug, exec,
test, etc... along with --headless and they will work. If this project
is part of a larger IDE integration then you probably have your own build
system and do not wish to offload this task to Delve, in that case it's
perfectly fine to always use the dlv exec command but do remember that:

  1. Delve may not have all the information necessary to properly debug optimized binaries, so it is recommended to disable them via: -gcflags='all=-N -l.
  2. your users do want to debug their tests so you should also provide some way to build the test executable (equivalent to go test -c --gcflags='all=-N -l') and pass it to Delve.

It would also be nice for your users if you provided a way to attach to a running process, like dlv attach does.

Command line arguments that should be handed to the inferior process should be specified on dlv's command line after a "--" argument:

text
dlv exec --headless ./somebinary -- these arguments are for the inferior process

Specifying a static port number, like in the README example, can be done using --listen=127.0.0.1:portnumber.

This will, however, cause problems if you actually spawn multiple instances of the debugger.

It's probably better to let Delve pick a random unused port number on its own. To do this do not specify any --listen option and read one line of output from dlv's stdout. If the first line emitted by dlv starts with "API server listening at: " then dlv started correctly and the rest of the line specifies the address that Delve is listening at.

The --log-dest option can be used to redirect the "API server listening at:" message to a file or to a file descriptor. If the flag is not specified, the message will be output to stdout while other log messages are output to stderr.

Controlling the backend

Once you have a running headless instance you can connect to it and start sending commands. Delve's protocol is built on top of the JSON-RPC 1.0 specification.

The methods of a service/rpc2.RPCServer are exposed through this connection, to find out which requests you can send see the documentation of RPCServer on Go Reference.

Example

Let's say you are trying to create a breakpoint. By looking at Go Reference you'll find that there is a CreateBreakpoint method in RPCServer.

This method, like all other methods of RPCServer that you can call through the API, has two arguments: args and out: args contains all the input arguments of CreateBreakpoint, while out is what CreateBreakpoint will return to you.

The call that you could want to make, in pseudo-code, would be:

text
RPCServer.CreateBreakpoint(CreateBreakpointIn{ File: "/User/you/some/file.go", Line: 16 })

To actually send this request on the JSON-RPC connection you just have to convert the CreateBreakpointIn object to json and then wrap everything into a JSON-RPC envelope:

text
{"method":"RPCServer.CreateBreakpoint","params":[{"Breakpoint":{"file":"/User/you/some/file.go","line":16}}],"id":27}

Delve will respond by sending a response packet that will look like this:

text
{"id":27, "result": {"Breakpoint": {"id":3, "name":"", "addr":4538829, "file":"/User/you/some/file.go", "line":16, "functionName":"main.main", "Cond":"", "continue":false, "goroutine":false, "stacktrace":0, "LoadArgs":null, "LoadLocals":null, "hitCount":{}, "totalHitCount":0}}, "error":null}

Selecting the API version

Delve currently supports two version of its API, APIv1 and APIv2. By default
a headless instance of dlv will serve APIv1 for backward-compatibility
with older clients, however new clients should use APIv2 as new features
will only be made available through version 2. The preferred method of
switching to APIv2 is to send the RPCServer.SetApiVersion command right
after connecting to the backend.
Alternatively the --api-version=2 command line option can be used when
spawning the backend.

Diagnostics

Just like any other program, both Delve and your client have bugs. To help
with determining where the problem is you should log the exchange of
messages between Delve and your client somehow.

If you don't want to do this yourself you can also pass the options --log --log-output=rpc to Delve. In fact the --log-output has many useful
values and you should expose it to users, if possible, so that we can
diagnose problems that are hard to reproduce.

Using RPCServer.Command

Command is probably the most important API entry point. It lets your
client stop (Name == "halt") and resume (Name == "continue") execution
of the inferior process.

The return value of Command is a DebuggerState object. If you lose the
DebuggerState object returned by your last call to Command you can ask for
a new copy with RPCServer.State.

Dealing with simultaneous breakpoints

Since Go is a programming language with a big emphasis on concurrency and
parallelism it's possible that multiple goroutines will stop at a breakpoint
simultaneously. This may at first seem incredibly unlikely but you must
understand that between the time a breakpoint is triggered and the point
where the debugger finishes stopping all threads of the inferior process
thousands of CPU instructions have to be executed, which make simultaneous
breakpoint triggering not that unlikely.

You should signal to your user all the breakpoints that occur after
executing a command, not just the first one. To do this iterate through the
Threads array in DebuggerState and note all the threads that have a non
nil Breakpoint member.

Special continue commands

In addition to "halt" and vanilla "continue" Command offers a few extra
flavours of continue that automatically set interesting temporary
breakpoints: "next" will continue until the next line of the program,
"stepout" will continue until the function returns, "step" is just like
"next" but it will step into function calls (but skip all calls to
unexported runtime functions).

All of "next", "step" and "stepout" operate on the selected goroutine. The
selected goroutine is described by the SelectedGoroutine field of
DebuggerState. Every time Command returns the selected goroutine will be
reset to the goroutine that triggered the breakpoint.

If multiple breakpoints are triggered simultaneously the selected goroutine
will be chosen randomly between the goroutines that are stopped at a
breakpoint. If a breakpoint is hit by a thread that is executing on the
system stack there will be no selected goroutine. If the "halt" command is
called there may not be a selected goroutine.

The selected goroutine can be changed using the "switchGoroutine" command.
If "switchGoroutine" is used to switch to a goroutine that's currently
parked SelectedGoroutine and CurrentThread will be mismatched. Always prefer
SelectedGoroutine over CurrentThread, you should ignore CurrentThread
entirely unless SelectedGoroutine is nil.

Special continue commands and asynchronous breakpoints

Because of the way go internals work it is not possible for a debugger to
resume a single goroutine. Therefore it's possible that after executing a
next/step/stepout a goroutine other than the goroutine the next/step/stepout
was executed on will hit a breakpoint.

If this happens Delve will return a DebuggerState with NextInProgress set to
true. When this happens your client has two options:

  • You can signal that a different breakpoint was hit and then automatically attempt to complete the next/step/stepout by calling RPCServer.Command with Name == "continue"
  • You can abort the next/step/stepout operation using RPCServer.CancelNext.

It is important to note that while NextInProgress is true it is not possible
to call next/step/stepout again without using CancelNext first. There can
not be multiple next/step/stepout operations in progress at any time.

RPCServer.Command and stale executable files

It's possible (albeit unfortunate) that your user will decide to change the
source of the program being executed in the debugger, while the debugger is
running. Because of this it would be advisable that your client check that
the executable is not stale every time Command returns and notify the user
that the executable being run is stale and line numbers may nor align
properly anymore.

You can do this bookkeeping yourself, but Delve can also help you with the
LastModified call that returns the LastModified time of the executable
file when Delve started it.

Using RPCServer.CreateBreakpoint

The only two fields you probably want to fill of the Breakpoint argument of
CreateBreakpoint are File and Line. The file name should be the absolute
path to the file as the compiler saw it.

For example if the compiler saw this path:

text
/Users/you/go/src/something/something.go

But /Users/you/go/src/something is a symbolic link to
/Users/you/projects/golang/something the path must be specified as
/Users/you/go/src/something/something.go and
/Users/you/projects/golang/something/something.go will not be recognized
as valid.

If you want to let your users specify a breakpoint on a function selected
from a list of all functions you should specify the name of the function in
the FunctionName field of Breakpoint.

If you want to support the same language as dlv's break and trace commands
you should call RPCServer.FindLocation and
then use the returned slice of Location objects to create Breakpoints to
pass to CreateBreakpoint: just fill each Breakpoint.Addr with the
contents of the corresponding Location.PC.

Looking into variables

There are several API entry points to evaluate variables in Delve:

  • RPCServer.ListPackageVars returns all global variables in all packages
  • PRCServer.ListLocalVars returns all local variables of a stack frame
  • RPCServer.ListFunctionArgs returns all function arguments of a stack frame
  • RPCServer.Eval evaluates an expression on a given stack frame

All those API calls take a LoadConfig argument. The LoadConfig specifies how
much of the variable's value should actually be loaded. Because of
LoadConfig a variable could be loaded incompletely, you should always notify
the user of this:

  • For strings, arrays, slices and structs the load is incomplete if: Variable.Len > len(Variable.Children). This can happen to structs even if LoadConfig.MaxStructFields is -1 when MaxVariableRecurse is reached.
  • For maps the load is incomplete if: Variable.Len > len(Variable.Children) / 2
  • For interfaces the load is incomplete if the only children has the onlyAddr attribute set to true.

Loading more of a Variable

You can also give the user an option to continue loading an incompletely
loaded variable. To load a struct that wasn't loaded automatically evaluate
the expression returned by:

text
fmt.Sprintf("*(*%q)(%#x)", v.Type, v.Addr)

where v is the variable that was truncated.

To load more elements from an array, slice or string:

text
fmt.Sprintf("(*(*%q)(%#x))[%d:]", v.Type, v.Addr, len(v.Children))

To load more elements from a map:

text
fmt.Sprintf("(*(*%q)(%#x))[%d:]", v.Type, v.Addr, len(v.Children)/2)

All the evaluation API calls except ListPackageVars also take a EvalScope
argument, this specifies which stack frame you are interested in. If you
are interested in the topmost stack frame of the current goroutine (or
thread) use: EvalScope{ GoroutineID: -1, Frame: 0 }.

More information on the expression language interpreted by RPCServer.Eval
can be found here.

Variable shadowing

Let's assume you are debugging a piece of code that looks like this:

text
for i := 0; i < N; i++ {
		for i := 0; i < M; i++ {
			f(i) // <-- debugger is stopped here
		}
	}

The response to a ListLocalVars request will list two variables named i,
because at that point in the code two variables named i exist and are in
scope. Only one (the innermost one), however, is visible to the user. The
other one is shadowed.

Delve will tell you which variable is shadowed through the Flags field of
the Variable object. If Flags has the VariableShadowed bit set then
the variable in question is shadowed.

Users of your client should be able to distinguish between shadowed and
non-shadowed variables.

Gracefully ending the debug session

To ensure that Delve cleans up after itself by deleting the debug or debug.test binary it creates
and killing any processes spawned by the program being debugged, the Detach command needs to be called.
In case you are disconnecting a running program, ensure to halt the program before trying to detach.

Testing the Client

A set of example programs is
available
to test corner
cases in handling breakpoints and displaying data structures. Follow the
instructions in the README.txt file.


File: Documentation/api/README.md

Server/Client API Documentation

Delve exposes two API interfaces, JSON-RPC and DAP, so that frontends other than the built-in terminal client, such as IDEs and editors, can interact with Delve programmatically. The JSON-RPC API is used by the terminal client, and will always stay up to date in lockstep regardless of new features. The DAP API is a popular generic API already in use by many tools.

Usage

In order to run Delve in "API mode", simply invoke with one of the standard commands, providing the --headless flag, like so:

text
$ dlv debug --headless --api-version=2 --log --log-output=debugger,dap,rpc --listen=127.0.0.1:8181

This will start the debugger in a non-interactive mode, listening on the specified address, and will enable logging. The logging flags as well as the server address are optional, of course.

Optionally, you may also specify the --accept-multiclient flag if you would like to connect multiple JSON-RPC or DAP clients to the API.

You can connect to the headless debugger from Delve itself using the connect subcommand:

text
$ dlv connect 127.0.0.1:8181

This can be useful for remote debugging.

API Interfaces

Delve has been architected in such a way as to allow multiple client/server implementations. All of the "business logic" as it were is abstracted away from the actual client/server implementations, allowing for easy implementation of new API interfaces.

Current API Interfaces


File: Documentation/cli/cond.md

Breakpoint conditions

Breakpoints have two conditions:

  • The normal condition, which is specified using the command cond <expr> (or by setting the Cond field when amending a breakpoint via the API), is any expression which evaluates to true or false.
  • The hitcount condition, which is specified cond -hitcount <operator> <number> (or by setting the HitCond field when amending a breakpoint via the API), is a constraint on the number of times the breakpoint has been hit.

When a breakpoint location is encountered during the execution of the program, the debugger will:

  • Evaluate the normal condition
  • Stop if there is an error while evaluating the normal condition
  • If the normal condition evaluates to true the hit count is incremented
  • Evaluate the hitcount condition
  • If the hitcount condition is also satisfied stop the execution at the breakpoint

File: Documentation/cli/expr.md

Expressions

Delve can evaluate a subset of go expression language, specifically the following features are supported:

  • All (binary and unary) on basic types except <-, ++ and --
  • Comparison operators on any type
  • Type casts between numeric types
  • Type casts of integer constants into any pointer type and vice versa
  • Type casts between string, []byte and []rune
  • Struct member access (i.e. somevar.memberfield)
  • Slicing and indexing operators on arrays, slices and strings
  • Map access
  • Pointer dereference
  • Calls to builtin functions: cap, len, complex, imag and real
  • Type assertion on interface variables (i.e. somevar.(concretetype))

Nesting limit

When delve evaluates a memory address it will automatically return the value of nested struct members, array and slice items and dereference pointers.
However to limit the size of the output evaluation will be limited to two levels deep. Beyond two levels only the address of the item will be returned, for example:

text
(dlv) print c1
main.cstruct {
	pb: *struct main.bstruct {
		a: (*main.astruct)(0xc82000a430),
	},
	sa: []*main.astruct len: 3, cap: 3, [
		*(*main.astruct)(0xc82000a440),
		*(*main.astruct)(0xc82000a450),
		*(*main.astruct)(0xc82000a460),
	],
}

To see the contents of the first item of the slice c1.sa there are two possibilities:

  1. Execute print c1.sa[0]
  2. Use the address directly, executing: print *(*main.astruct)(0xc82000a440)

Elements limit

For arrays, slices, strings and maps delve will only return a maximum of 64 elements at a time:

text
(dlv) print ba
[]int len: 200, cap: 200, [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,...+136 more]

To see more values use the slice operator:

text
(dlv) print ba[64:]
[]int len: 136, cap: 136, [0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,...+72 more]

For this purpose delve allows use of the slice operator on maps, m[64:] will return the key/value pairs of map m that follow the first 64 key/value pairs (note that delve iterates over maps using a fixed ordering).

These limits can be configured with max-string-len and max-array-values. See config for usage.

Interfaces

Interfaces will be printed using the following syntax:

text
(<concrete type>) <value>

For example:

text
(dlv) p iface1
(dlv) p iface1
interface {}(*struct main.astruct) *{A: 1, B: 2}
(dlv) p iface2
interface {}(*struct string) *"test"
(dlv) p err1
error(*struct main.astruct) *{A: 1, B: 2}

To use the contents of an interface variable use a type assertion:

text
(dlv) p iface1.(*main.astruct).B
2

Or just use the special .(data) type assertion:

text
(dlv) p iface1.(data).B
2

If the contents of the interface variable are a struct or a pointer to struct the fields can also be accessed directly:

text
(dlv) p iface1.B
2

Specifying package paths

Packages with the same name can be disambiguated by using the full package path. For example, if the application imports two packages, some/package and some/other/package, both defining a variable A, the two variables can be accessed using this syntax:

text
(dlv) p "some/package".A
(dlv) p "some/other/package".A

Pointers in Cgo

Char pointers are always treated as NUL terminated strings, both indexing and the slice operator can be applied to them. Other C pointers can also be used similarly to Go slices, with indexing and the slice operator. In both of these cases it is up to the user to respect array bounds.

Special Features

Special Variables

Delve defines two special variables:

  • runtime.curg evaluates to the 'g' struct for the current goroutine, in particular runtime.curg.goid is the goroutine id of the current goroutine.
  • runtime.frameoff is the offset of the frame's base address from the bottom of the stack.
  • delve.bphitcount[X] is the total hitcount for breakpoint X, which can be either an ID or the breakpoint name as a string.

Access to variables from previous frames

Variables from previous frames (i.e. stack frames other than the top of the stack) can be referred using the following notation runtime.frame(n).name which is the variable called 'name' on the n-th frame from the top of the stack.

CPU Registers

The name of a CPU register, in all uppercase letters, will resolve to the value of that CPU register in the current frame. For example on AMD64 the expression RAX will evaluate to the value of the RAX register.

Register names are shadowed by both local and global variables, so if a local variable called "RAX" exists, the RAX expression will evaluate to it instead of the CPU register.

Register names can optionally be prefixed by any number of underscore characters, so RAX, _RAX, __RAX, etc... can all be used to refer to the same RAX register and, in absence of shadowing from other variables, will all evaluate to the same value.

Registers of 64bits or less are returned as uint64 variables. Larger registers are returned as strings of hexadecimal digits.

Because many architectures have SIMD registers that can be used by the application in different ways the following syntax is also available:

  • REGNAME.intN returns the register REGNAME as an array of intN elements.
  • REGNAME.uintN returns the register REGNAME as an array of uintN elements.
  • REGNAME.floatN returns the register REGNAME as an array of floatN elements.

In all cases N must be a power of 2.


File: Documentation/cli/getting_started.md

Getting Started

Delve aims to be a very simple and powerful tool, but can be confusing if you're
not used to using a source level debugger in a compiled language. This document
will provide all the information you need to get started debugging your Go
programs.

Debugging 'main' packages

The first CLI subcommand we will explore is debug. This subcommand can be run
without arguments if you're in the same directory as your main package,
otherwise it optionally accepts a package path.

For example given this project layout:

text
github.com/me/foo
โ”œโ”€โ”€ cmd
โ”‚ย ย  โ””โ”€โ”€ foo
โ”‚ย ย      โ””โ”€โ”€ main.go
โ””โ”€โ”€ pkg
    โ””โ”€โ”€ baz
        โ”œโ”€โ”€ bar.go
        โ””โ”€โ”€ bar_test.go

If you are in the directory github.com/me/foo/cmd/foo you can simply run dlv debug
from the command line. From anywhere else, say the project root, you can simply
provide the package: dlv debug github.com/me/foo/cmd/foo. To pass flags to your program
separate them with --: dlv debug github.com/me/foo/cmd/foo -- -arg1 value.

Invoking that command will cause Delve to compile the program in a way most
suitable for debugging, then it will execute and attach to the program and begin
a debug session. Now, when the debug session has first started you are at the
very beginning of the program's initialization. To get to someplace more useful
you're going to want to set a breakpoint or two and continue execution to that
point.

For example, to continue execution to your program's main function:

text
$ dlv debug github.com/me/foo/cmd/foo
Type 'help' for list of commands.
(dlv) break main.main
Breakpoint 1 set at 0x49ecf3 for main.main() ./test.go:5
(dlv) continue
> main.main() ./test.go:5 (hits goroutine(1):1 total:1) (PC: 0x49ecf3)
     1:	package main
     2:	
     3:	import "fmt"
     4:	
=>   5:	func main() {
     6:		fmt.Println("delve test")
     7:	}
(dlv)

Debugging tests

Given the same directory structure as above you can debug your code by executing
your test suite. For this you can use the dlv test subcommand, which takes the
same optional package path as dlv debug, and will also build the current
package if not given any argument.

text
$ dlv test github.com/me/foo/pkg/baz
Type 'help' for list of commands.
(dlv) funcs test.Test*
/home/me/go/src/github.com/me/foo/pkg/baz/test.TestHi
(dlv) break TestHi
Breakpoint 1 set at 0x536513 for /home/me/go/src/github.com/me/foo/pkg/baz/test.TestHi() ./test_test.go:5
(dlv) continue
> /home/me/go/src/github.com/me/foo/pkg/baz/test.TestHi() ./bar_test.go:5 (hits goroutine(5):1 total:1) (PC: 0x536513)
     1:	package baz
     2:	
     3:	import "testing"
     4:	
=>   5:	func TestHi(t *testing.T) {
     6:		t.Fatal("implement me!")
     7:	}
(dlv)

As you can see, we began debugging the test binary, found our test function via
the funcs command which takes a regexp to filter the list of functions, set a
breakpoint and then continued execution until we hit that breakpoint.

For more information on subcommands you can use, type dlv help, and once in a
debug session you can see all of the commands available to you by typing help
at any time.


File: Documentation/cli/locspec.md

Location Specifiers

Several delve commands take a program location as an argument, the syntax accepted by this commands is:

  • *<address> Specifies the location of memory address address. address can be specified as a decimal, hexadecimal or octal number

  • <filename>:<line> Specifies the line line in filename. filename can be the partial path to a file or even just the base name as long as the expression remains unambiguous.

  • <line> Specifies the line line in the current file

  • +<offset> Specifies the line offset lines after the current one

  • -<offset> Specifies the line offset lines before the current one

  • <function>[:<line>] Specifies the line line inside function. The full syntax for function is .(*<receiver type>).<function name> however the only required element is the function name, everything else can be omitted as long as the expression remains unambiguous. For setting a breakpoint on an init function (ex: main.init), the <filename>:<line> syntax should be used to break in the correct init function at the correct location.

  • /<regex>/ Specifies the location of all the functions matching regex


File: Documentation/cli/README.md

Configuration and Command History

If $XDG_CONFIG_HOME is set, then configuration and command history files are located in $XDG_CONFIG_HOME/dlv. Otherwise, they are located in $HOME/.config/dlv on Linux and $HOME/.dlv on other systems.

The configuration file config.yml contains all the configurable options and their default values (config.yml documentation). The command history is stored in .dbg_history.

Commands

Running the program

Command Description
call Resumes process, injecting a function call (EXPERIMENTAL!!!)
continue Run until breakpoint or program termination.
next Step over to next source line.
next-instruction Single step a single cpu instruction, skipping function calls.
rebuild Rebuild the target executable and restarts it. It does not work if the executable was not built by delve.
restart Restart process.
rev Reverses the execution of the target program for the command specified.
rewind Run backwards until breakpoint or start of recorded history.
step Single step through program.
step-instruction Single step a single cpu instruction.
stepout Step out of the current function.

Manipulating breakpoints

Command Description
break Sets a breakpoint.
breakpoints Print out info for active breakpoints.
clear Deletes breakpoint.
clearall Deletes multiple breakpoints.
condition Set breakpoint condition.
on Executes a command when a breakpoint is hit.
toggle Toggles on or off a breakpoint.
trace Set tracepoint.
watch Set watchpoint.

Viewing program variables and memory

Command Description
args Print function arguments.
display Print value of an expression every time the program stops.
examinemem Examine raw memory at the given address.
locals Print local variables.
print Evaluate an expression.
regs Print contents of CPU registers.
set Changes the value of a variable.
vars Print package variables.
whatis Prints type of an expression or a type.

Listing and switching between threads and goroutines

Command Description
goroutine Shows or changes current goroutine
goroutines List program goroutines.
thread Switch to the specified thread.
threads Print out info for every traced thread.

Viewing the call stack and selecting frames

Command Description
deferred Executes command in the context of a deferred call.
down Move the current frame down.
frame Set the current frame, or execute command on a different frame.
stack Print stack trace.
up Move the current frame up.

Other commands

Command Description
check Creates a checkpoint at the current position.
checkpoints Print out info for existing checkpoints.
clear-checkpoint Deletes checkpoint.
config Changes configuration parameters.
disassemble Disassembler.
dump Creates a core dump from the current process state
edit Open where you are in $DELVE_EDITOR or $EDITOR
exit Exit the debugger.
funcs Print list of functions.
help Prints the help message.
libraries List loaded dynamic libraries.
list Show source code.
packages Print list of packages.
source Executes a file containing a list of delve commands
sources Print list of source files.
target Manages child process debugging.
transcript Appends command output to a file.
types Print list of types

args

Print function arguments.

text
[goroutine <n>] [frame <m>] args [-v] [<regex>]

If regex is specified only function arguments with a name matching it will be returned. If -v is specified more information about each function argument will be shown.

break

Sets a breakpoint.

text
break [name] [locspec] [if <condition>]

Locspec is a location specifier in the form of:

  • *<address> Specifies the location of memory address address. address can be specified as a decimal, hexadecimal or octal number
  • <filename>:<line> Specifies the line in filename. filename can be the partial path to a file or even just the base name as long as the expression remains unambiguous.
  • <line> Specifies the line in the current file
  • +<offset> Specifies the line offset lines after the current one
  • -<offset> Specifies the line offset lines before the current one
  • <function>[:<line>] Specifies the line inside function.
    The full syntax for function is <package>.(*<receiver type>).<function name> however the only required element is the function name,
    everything else can be omitted as long as the expression remains unambiguous. For setting a breakpoint on an init function (ex: main.init),
    the <filename>:<line> syntax should be used to break in the correct init function at the correct location.
  • /<regex>/ Specifies the location of all the functions matching regex

If locspec is omitted a breakpoint will be set on the current line.

If you would like to assign a name to the breakpoint you can do so with the form:

text
break mybpname main.go:4

Finally, you can assign a condition to the newly created breakpoint by using the 'if' postfix form, like so:

text
break main.go:55 if i == 5

Alternatively you can set a condition on a breakpoint after created by using the 'on' command.

See also: "help on", "help cond" and "help clear"

Aliases: b

breakpoints

Print out info for active breakpoints.

text
breakpoints [-a] [-save <filename>]

Specifying -a prints all physical breakpoint, including internal breakpoints.
Speciftying -save <filename> saves all breakpoints to the specified file in a format that can be loaded later using the 'source' command.

Aliases: bp

call

Resumes process, injecting a function call (EXPERIMENTAL!!!)

text
call [-unsafe] <function call expression>

Current limitations:

  • only pointers to stack-allocated objects can be passed as argument.
  • only some automatic type conversions are supported.
  • functions can only be called on running goroutines that are not
    executing the runtime.
  • the current goroutine needs to have at least 256 bytes of free space on
    the stack.
  • functions can only be called when the goroutine is stopped at a safe
    point.
  • calling a function will resume execution of all goroutines.
  • only supported on linux's native backend.

check

Creates a checkpoint at the current position.

text
checkpoint [note]

The "note" is arbitrary text that can be used to identify the checkpoint, if it is not specified it defaults to the current filename:line position.

Aliases: checkpoint

checkpoints

Print out info for existing checkpoints.

clear

Deletes breakpoint.

text
clear

clear-checkpoint

Deletes checkpoint.

text
clear-checkpoint

Aliases: clearcheck

clearall

Deletes multiple breakpoints.

text
clearall [<locspec>]

If called with the locspec argument it will delete all the breakpoints matching the locspec. If locspec is omitted all breakpoints are deleted.

condition

Set breakpoint condition.

text
condition  .
condition -hitcount  <operator> <argument>.
condition -per-g-hitcount  <operator> <argument>.
condition -clear .

Specifies that the breakpoint, tracepoint or watchpoint should break only if the boolean expression is true.

See Documentation/cli/expr.md for a description of supported expressions and Documentation/cli/cond.md for a description of how breakpoint conditions are evaluated.

With the -hitcount option a condition on the breakpoint hit count can be set, the following operators are supported

text
condition -hitcount bp > n
condition -hitcount bp >= n
condition -hitcount bp < n
condition -hitcount bp <= n
condition -hitcount bp == n
condition -hitcount bp != n
condition -hitcount bp % n

The -per-g-hitcount option works like -hitcount, but use per goroutine hitcount to compare with n.

With the -clear option a condition on the breakpoint can removed.

The '% n' form means we should stop at the breakpoint when the hitcount is a multiple of n.

Examples:

text
cond 2 i == 10				breakpoint 2 will stop when variable i equals 10
cond name runtime.curg.goid == 5	breakpoint 'name' will stop only on goroutine 5
cond -clear 2				the condition on breakpoint 2 will be removed

Aliases: cond

config

Changes configuration parameters.

text
config -list

Show all configuration parameters.

text
config -save

Saves the configuration file to disk, overwriting the current configuration file.

text
config  <value>

Changes the value of simple configuration parameters.

Use 'help config <parameter>' for more informations on specific configuration options.

continue

Run until breakpoint or program termination.

text
continue [<locspec>]

Optional locspec argument allows you to continue until a specific location is reached. The program will halt if a breakpoint is hit before reaching the specified location.

For example:

text
continue main.main
continue encoding/json.Marshal

Aliases: c

deferred

Executes command in the context of a deferred call.

text
deferred <n> <command>

Executes the specified command (print, args, locals) in the context of the n-th deferred call in the current frame.

disassemble

Disassembler.

text
[goroutine <n>] [frame <m>] disassemble [-a <start> <end>] [-l <locspec>]

If no argument is specified the function being executed in the selected stack frame will be executed.

text
-a <start> <end>	disassembles the specified address range
-l <locspec>		disassembles the specified function

Aliases: disass

display

Print value of an expression every time the program stops.

text
display -a [%format] <expression>
display -d <number>

The '-a' option adds an expression to the list of expression printed every time the program stops. The '-d' option removes the specified expression from the list.

If display is called without arguments it will print the value of all expression in the list.

down

Move the current frame down.

text
down [<m>]
down [<m>] <command>

Move the current frame down by <m>. The second form runs the command on the given frame.

dump

Creates a core dump from the current process state

text
dump <output file>

The core dump is always written in ELF, even on systems (windows, macOS) where this is not customary. For environments other than linux/amd64 threads and registers are dumped in a format that only Delve can read back.

edit

Open where you are in $DELVE_EDITOR or $EDITOR

text
edit [locspec]

If locspec is omitted edit will open the current source file in the editor, otherwise it will open the specified location.

Aliases: ed

examinemem

Examine raw memory at the given address.

Examine memory:

text
examinemem [-fmt <format>] [-count|-len <count>] [-size <size>] <address>
examinemem [-fmt <format>] [-count|-len <count>] [-size <size>] -x <expression>

Format represents the data format and the value is one of this list (default hex): bin(binary), oct(octal), dec(decimal), hex(hexadecimal) and raw.
Length is the number of bytes (default 1) and must be less than or equal to 1000.
Address is the memory location of the target to examine. Please note '-len' is deprecated by '-count and -size'.
Expression can be an integer expression or pointer value of the memory location to examine.

For example:

text
x -fmt hex -count 20 -size 1 0xc00008af38
x -fmt hex -count 20 -size 1 -x 0xc00008af38 + 8
x -fmt hex -count 20 -size 1 -x &myVar
x -fmt hex -count 20 -size 1 -x myPtrVar

Aliases: x

exit

Exit the debugger.

text
exit [-c] [-d]

When connected to a headless instance started with the --accept-multiclient, pass -c to resume the execution of the target process before disconnecting.

Pass -d to detach from the target process, leaving it running, without being prompted whether to kill it.

Aliases: quit q

frame

Set the current frame, or execute command on a different frame.

text
frame <m>
frame <m> <command>

The first form sets frame used by subsequent commands such as "print" or "set".
The second form runs the command on the given frame.

funcs

Print list of functions.

text
funcs [<regex>]

If regex is specified only the functions matching it will be returned.

goroutine

Shows or changes current goroutine

text
goroutine
goroutine 
goroutine  <command>

Called without arguments it will show information about the current goroutine.
Called with a single argument it will switch to the specified goroutine.
Called with more arguments it will execute a command on the specified goroutine.

Aliases: gr

goroutines

List program goroutines.

text
goroutines [-u|-r|-g|-s] [-t [depth]] [-l] [-with loc expr] [-without loc expr] [-group argument] [-chan expr] [-exec command]

Print out info for every goroutine. The flag controls what information is shown along with each goroutine:

text
-u	displays location of topmost stackframe in user code (default)
-r	displays location of topmost stackframe (including frames inside private runtime functions)
-g	displays location of go instruction that created the goroutine
-s	displays location of the start function
-t	displays goroutine's stacktrace (an optional depth value can be specified, default: 10)
-l	displays goroutine's labels

If no flag is specified the default is -u, i.e. the first frame within the first 30 frames that is not executing a runtime private function.

FILTERING

If -with or -without are specified only goroutines that match the given condition are returned.

To only display goroutines where the specified location contains (or does not contain, for -without and -wo) expr as a substring, use:

text
goroutines -with (userloc|curloc|goloc|startloc) expr
goroutines -w (userloc|curloc|goloc|startloc) expr
goroutines -without (userloc|curloc|goloc|startloc) expr
goroutines -wo (userloc|curloc|goloc|startloc) expr

Where:
userloc: filter by the location of the topmost stackframe in user code
curloc: filter by the location of the topmost stackframe (including frames inside private runtime functions)
goloc: filter by the location of the go instruction that created the goroutine
startloc: filter by the location of the start function

To only display goroutines that have (or do not have) the specified label key and value, use:

text
goroutines -with label key=value
goroutines -without label key=value

To only display goroutines that have (or do not have) the specified label key, use:

text
goroutines -with label key
goroutines -without label key

To only display goroutines that are running (or are not running) on a OS thread, use:

text
goroutines -with running
goroutines -without running

To only display user (or runtime) goroutines, use:

text
goroutines -with user
goroutines -without user

CHANNELS

To only show goroutines waiting to send to or receive from a specific channel use:

text
goroutines -chan expr

Note that 'expr' must not contain spaces.

GROUPING

text
goroutines -group (userloc|curloc|goloc|startloc|running|user)

Where:
userloc: groups goroutines by the location of the topmost stackframe in user code
curloc: groups goroutines by the location of the topmost stackframe
goloc: groups goroutines by the location of the go instruction that created the goroutine
startloc: groups goroutines by the location of the start function
running: groups goroutines by whether they are running or not
user: groups goroutines by whether they are user or runtime goroutines

Groups goroutines by the given location, running status or user classification, up to 5 goroutines per group will be displayed as well as the total number of goroutines in the group.

text
goroutines -group label key

Groups goroutines by the value of the label with the specified key.

EXEC

text
goroutines -exec <command>

Runs the command on every goroutine.

Aliases: grs

help

Prints the help message.

text
help [command]

Type "help" followed by the name of a command for more information about it.

Aliases: h

libraries

List loaded dynamic libraries.

text
libraries [-d N]

If used with the -d option it will re-attempt to download the debug symbols for library N, using debuginfod-find.

list

Show source code.

text
[goroutine <n>] [frame <m>] list [<locspec>]

Show source around current point or provided locspec.

For example:

text
frame 1 list 69
list testvariables.go:10000
list main.main:30
list 40

Aliases: ls l

locals

Print local variables.

text
[goroutine <n>] [frame <m>] locals [-v] [<regex>]

The name of variables that are shadowed in the current scope will be shown in parenthesis.

If regex is specified only local variables with a name matching it will be returned. If -v is specified more information about each local variable will be shown.

next

Step over to next source line.

text
next [count]

Optional [count] argument allows you to skip multiple lines.

Aliases: n

next-instruction

Single step a single cpu instruction, skipping function calls.

Aliases: ni nexti

on

Executes a command when a breakpoint is hit.

text
on  <command>
on  -edit

Supported commands: print, stack, goroutine, trace and cond.

All custom starlark commands can also be used with the 'on' prefix. See Documentation/cli/starlark.md for more information.

To convert a breakpoint into a tracepoint use:

text
on  trace

The command 'on <bp> cond <cond-arguments>' is equivalent to 'cond <bp> <cond-arguments>'.

The command 'on x -edit' can be used to edit the list of commands executed when the breakpoint is hit.

packages

Print list of packages.

text
packages [<regex>]

If regex is specified only the packages matching it will be returned.

print

Evaluate an expression.

text
[goroutine <n>] [frame <m>] print [%format] <expression>

See Documentation/cli/expr.md for a description of supported expressions.

The optional format argument is a format specifier, like the ones used by the fmt package. For example "print %x v" will print v as an hexadecimal number.

Aliases: p

rebuild

Rebuild the target executable and restarts it. It does not work if the executable was not built by delve.

regs

Print contents of CPU registers.

text
regs [-a]

Argument -a shows more registers. Individual registers can also be displayed by 'print' and 'display'. See Documentation/cli/expr.md.

restart

Restart process.

For recorded targets the command takes the following forms:

text
restart					resets to the start of the recording
restart [checkpoint]			resets the recording to the given checkpoint
restart -r [newargv...]	[redirects...]	re-records the target process

For live targets the command takes the following forms:

text
restart [newargv...] [redirects...]	restarts the process

If newargv is omitted the process is restarted (or re-recorded) with the same argument vector.
If -noargs is specified instead, the argument vector is cleared.

A list of file redirections can be specified after the new argument list to override the redirections defined using the '--redirect' command line option. A syntax similar to Unix shells is used:

text
output.txt	redirects the standard output of the target process to output.txt
2>error.txt	redirects the standard error of the target process to error.txt

Aliases: r

rev

Reverses the execution of the target program for the command specified.
Currently, rev next, step, step-instruction and stepout commands are supported.

rewind

Run backwards until breakpoint or start of recorded history.

Aliases: rw

set

Changes the value of a variable.

text
[goroutine <n>] [frame <m>] set <variable> = <value>

See Documentation/cli/expr.md for a description of supported expressions. Only numerical variables and pointers can be changed.

source

Executes a file containing a list of delve commands

text
source

If path ends with the .star extension it will be interpreted as a starlark script. See Documentation/cli/starlark.md for the syntax.

If path is a single '-' character an interactive starlark interpreter will start instead. Type 'exit' to exit.

sources

Print list of source files.

text
sources [<regex>]

If regex is specified only the source files matching it will be returned.

stack

Print stack trace.

text
[goroutine <n>] [frame <m>] stack [<depth>] [-full] [-offsets] [-defer] [-a <n>] [-adepth <depth>] [-mode <mode>]

-full		every stackframe is decorated with the value of its local variables and arguments.
-offsets	prints frame offset of each frame.
-defer		prints deferred function call stack for each frame.
-a <n>		prints stacktrace of n ancestors of the selected goroutine (target process must have tracebackancestors enabled)
-adepth <depth>	configures depth of ancestor stacktrace
-mode <mode>	specifies the stacktrace mode, possible values are:
		normal	- attempts to automatically switch between cgo frames and go frames
		simple	- disables automatic switch between cgo and go
		fromg	- starts from the registers stored in the runtime.g struct

Aliases: bt

step

Single step through program.

Aliases: s

step-instruction

Single step a single cpu instruction.

Aliases: si stepi

stepout

Step out of the current function.

Aliases: so

target

Manages child process debugging.

text
target follow-exec [-on [regex]] [-off]

Enables or disables follow exec mode. When follow exec mode Delve will automatically attach to new child processes executed by the target process. An optional regular expression can be passed to 'target follow-exec', only child processes with a command line matching the regular expression will be followed.

text
target list

List currently attached processes.

text
target switch [pid]

Switches to the specified process.

thread

Switch to the specified thread.

text
thread

Aliases: tr

threads

Print out info for every traced thread.

toggle

Toggles on or off a breakpoint.

text
toggle

trace

Set tracepoint.

text
trace [name] [locspec]

A tracepoint is a breakpoint that does not stop the execution of the program, instead when the tracepoint is hit a notification is displayed. See Documentation/cli/locspec.md for the syntax of locspec. If locspec is omitted a tracepoint will be set on the current line.

See also: "help on", "help cond" and "help clear"

Aliases: t

transcript

Appends command output to a file.

text
transcript [-t] [-x] <output file>
transcript -off

Output of Delve's command is appended to the specified output file. If '-t' is specified and the output file exists it is truncated. If '-x' is specified output to stdout is suppressed instead.

Using the -off option disables the transcript.

types

Print list of types

text
types [<regex>]

If regex is specified only the types matching it will be returned.

up

Move the current frame up.

text
up [<m>]
up [<m>] <command>

Move the current frame up by <m>. The second form runs the command on the given frame.

vars

Print package variables.

text
vars [-v] [<regex>]

If regex is specified only package variables with a name matching it will be returned. If -v is specified more information about each package variable will be shown.

watch

Set watchpoint.

text
watch [-r|-w|-rw] <expr>

-r	stops when the memory location is read
-w	stops when the memory location is written
-rw	stops when the memory location is read or written

The memory location is specified with the same expression language used by 'print', for example:

text
watch -r v
watch -w *(*int)(0x1400007c018)

will watch the address of variable 'v' and writes to an int at addr '0x1400007c018'.

Note that writes that do not change the value of the watched memory address might not be reported.

See also: "help print".

whatis

Prints type of an expression or a type.

text
whatis <expression>
whatis <type name>