goose Documentation

File: README.md Copy Chapter

goose

Goose is a database migration tool. Both a CLI and a library.

Manage your database schema by creating incremental SQL changes or Go functions.

Features

• Works against multiple databases:
  • Postgres, MySQL, MariaDB, Spanner, SQLite,
    YDB, ClickHouse, MSSQL, Vertica, and more.
• Supports Go migrations written as plain functions.
• Supports embedded migrations.
• Out-of-order migrations.
• Seeding data.
• Environment variable substitution in SQL migrations.
• ... and more.

Install

go install github.com/pressly/goose/v3/cmd/goose@latest

This will install the goose binary to your $GOPATH/bin directory.

Binary too big? Build a lite version by excluding the drivers you don't need:

go build -tags='no_postgres no_mysql no_sqlite3 no_ydb' -o goose ./cmd/goose

# Available build tags:
#   no_clickhouse  no_libsql   no_mssql    no_mysql
#   no_postgres    no_sqlite3  no_vertica  no_ydb

For macOS users goose is available as a Homebrew
Formulae:

brew install goose

See installation documentation for more details.

Usage

Click to show goose help output

/* Detailed source-code truncated for AI context efficiency. */

Commonly used commands:

create •  up •  up-to •  down •  down-to •  status •  version

create Copy Chapter

Create a new SQL migration.

$ goose create add_some_column sql
$ Created new file: 20170506082420_add_some_column.sql

$ goose -s create add_some_column sql
$ Created new file: 00001_add_some_column.sql

Edit the newly created file to define the behavior of your migration.

You can also create a Go migration, if you then invoke it with your own goose
binary:

$ goose create fetch_user_data go
$ Created new file: 20170506082421_fetch_user_data.go

up Copy Chapter

Apply all available migrations.

$ goose up
$ OK    001_basics.sql
$ OK    002_next.sql
$ OK    003_and_again.go

up-to Copy Chapter

Migrate up to a specific version.

$ goose up-to 20170506082420
$ OK    20170506082420_create_table.sql

up-by-one Copy Chapter

Migrate up a single migration from the current version

$ goose up-by-one
$ OK    20170614145246_change_type.sql

down Copy Chapter

Roll back a single migration from the current version.

$ goose down
$ OK    003_and_again.go

down-to Copy Chapter

Roll back migrations to a specific version.

$ goose down-to 20170506082527
$ OK    20170506082527_alter_column.sql

Or, roll back all migrations (careful!):

$ goose down-to 0

status Copy Chapter

Print the status of all migrations:

$ goose status
$   Applied At                  Migration
$   =======================================
$   Sun Jan  6 11:25:03 2013 -- 001_basics.sql
$   Sun Jan  6 11:25:03 2013 -- 002_next.sql
$   Pending                  -- 003_and_again.go

Note: for MySQL and MariaDB (use the mysql driver) parseTime flag must be enabled.

Note: for MySQL and MariaDB
multiStatements must
be enabled. This is required when writing multiple queries separated by ';' characters in a single
sql file.

version Copy Chapter

Print the current version of the database:

$ goose version
$ goose: version 002

Environment Variables

If you prefer to use environment variables, instead of passing the driver and database string as
arguments, you can set the following environment variables:

1. Via environment variables:

export GOOSE_DRIVER=DRIVER
export GOOSE_DBSTRING=DBSTRING
export GOOSE_MIGRATION_DIR=MIGRATION_DIR
export GOOSE_TABLE=TABLENAME

2. Via .env files with corresponding variables. .env file example:

GOOSE_DRIVER=postgres
GOOSE_DBSTRING=postgres://admin:admin@localhost:5432/admin_db
GOOSE_MIGRATION_DIR=./migrations
GOOSE_TABLE=custom.goose_migrations

Loading from .env files is enabled by default. To disable this feature, set the -env=none flag.
If you want to load from a specific file, set the -env flag to the file path.

For more details about environment variables, see the official documentation on environment
variables.

Migrations

goose supports migrations written in SQL or in Go.

SQL Migrations Copy Chapter

A sample SQL migration looks like:

-- +goose Up
CREATE TABLE post (
    id int NOT NULL,
    title text,
    body text,
    PRIMARY KEY(id)
);

-- +goose Down
DROP TABLE post;

Each migration file must have exactly one -- +goose Up annotation. The -- +goose Down annotation
is optional. If the file has both annotations, then the -- +goose Up annotation must come
first.

Notice the annotations in the comments. Any statements following -- +goose Up will be executed as
part of a forward migration, and any statements following -- +goose Down will be executed as part
of a rollback.

By default, all migrations are run within a transaction. Some statements like CREATE DATABASE,
however, cannot be run within a transaction. You may optionally add -- +goose NO TRANSACTION to
the top of your migration file in order to skip transactions within that specific migration file.
Both Up and Down migrations within this file will be run without transactions.

By default, SQL statements are delimited by semicolons - in fact, query statements must end with a
semicolon to be properly recognized by goose.

By default, all migrations are run on the public schema. If you want to use a different schema,
specify the schema name using the table option like -table='schemaname.goose_db_version.

More complex statements (PL/pgSQL) that have semicolons within them must be annotated with -- +goose StatementBegin and -- +goose StatementEnd to be properly recognized. For example:

-- +goose Up
-- +goose StatementBegin
CREATE OR REPLACE FUNCTION histories_partition_creation( DATE, DATE )
returns void AS $$
DECLARE
  create_query text;
BEGIN
  FOR create_query IN SELECT
      'CREATE TABLE IF NOT EXISTS histories_'
      || TO_CHAR( d, 'YYYY_MM' )
      || ' ( CHECK( created_at >= timestamp '''
      || TO_CHAR( d, 'YYYY-MM-DD 00:00:00' )
      || ''' AND created_at < timestamp '''
      || TO_CHAR( d + INTERVAL '1 month', 'YYYY-MM-DD 00:00:00' )
      || ''' ) ) inherits ( histories );'
    FROM generate_series( $1, $2, '1 month' ) AS d
  LOOP
    EXECUTE create_query;
  END LOOP;  -- LOOP END
END;         -- FUNCTION END
$$
language plpgsql;
-- +goose StatementEnd

Goose supports environment variable substitution in SQL migrations through annotations. To enable
this feature, use the -- +goose ENVSUB ON annotation before the queries where you want
substitution applied. It stays active until the -- +goose ENVSUB OFF annotation is encountered.
You can use these annotations multiple times within a file.

This feature is disabled by default for backward compatibility with existing scripts.

For PL/pgSQL functions or other statements where substitution is not desired, wrap the annotations
explicitly around the relevant parts. For example, to exclude escaping the $$ characters:

-- +goose StatementBegin
CREATE OR REPLACE FUNCTION test_func()
RETURNS void AS $$
-- +goose ENVSUB ON
BEGIN
	RAISE NOTICE '${SOME_ENV_VAR}';
END;
-- +goose ENVSUB OFF
$$ LANGUAGE plpgsql;
-- +goose StatementEnd

Supported expansions (click here to expand):

• ${VAR} or $VAR - expands to the value of the environment variable VAR
• ${VAR:-default} - expands to the value of the environment variable VAR, or default if VAR
  is unset or null
• ${VAR-default} - expands to the value of the environment variable VAR, or default if VAR
  is unset
• ${VAR?err_msg} - expands to the value of the environment variable VAR, or prints err_msg and
  error if VAR unset
• ${VAR:?err_msg} - expands to the value of the environment variable VAR, or prints err_msg
  and error if VAR unset or null. THIS IS NOT SUPPORTED

See
mfridman/interpolate
for more details on supported expansions.

Embedded sql migrations Copy Chapter

Go 1.16 introduced new feature: compile-time embedding files into
binary and corresponding filesystem abstraction.

This feature can be used only for applying existing migrations. Modifying operations such as fix
and create will continue to operate on OS filesystem even if using embedded files. This is
expected behaviour because io/fs interfaces allows read-only access.

Make sure to configure the correct SQL dialect, see dialect.go for supported SQL
dialects.

Example usage, assuming that SQL migrations are placed in the migrations directory:

package main

import (
    "database/sql"
    "embed"

    "github.com/pressly/goose/v3"
)

//go:embed migrations/*.sql
var embedMigrations embed.FS

func main() {
    var db *sql.DB
    // setup database

    goose.SetBaseFS(embedMigrations)

    if err := goose.SetDialect("postgres"); err != nil {
        panic(err)
    }

    if err := goose.Up(db, "migrations"); err != nil {
        panic(err)
    }

    // run app
}

Note that we pass "migrations" as directory argument in Up because embedding saves directory
structure.

Go Migrations Copy Chapter

1. Create your own goose binary, see example
2. Import github.com/pressly/goose
3. Register your migration functions
4. Include your migrations package into Go build: in main.go, import _ "github.com/me/myapp/migrations"
5. Run goose command, ie. goose.Up(db *sql.DB, dir string)

A sample Go migration 00002_users_add_email.go file
looks like:

package migrations

import (
	"database/sql"

	"github.com/pressly/goose/v3"
)

func init() {
	goose.AddMigration(Up, Down)
}

func Up(tx *sql.Tx) error {
	_, err := tx.Exec("UPDATE users SET username='admin' WHERE username='root';")
	if err != nil {
		return err
	}
	return nil
}

func Down(tx *sql.Tx) error {
	_, err := tx.Exec("UPDATE users SET username='root' WHERE username='admin';")
	if err != nil {
		return err
	}
	return nil
}

Note that Go migration files must begin with a numeric value, followed by an underscore, and must
not end with *_test.go.

Hybrid Versioning

Please, read the versioning
problem first.

By default, if you attempt to apply missing (out-of-order) migrations goose will raise an error.
However, If you want to apply these missing migrations pass goose the -allow-missing flag, or if
using as a library supply the functional option goose.WithAllowMissing() to Up, UpTo or UpByOne.

However, we strongly recommend adopting a hybrid versioning approach, using both timestamps and
sequential numbers. Migrations created during the development process are timestamped and sequential
versions are ran on production. We believe this method will prevent the problem of conflicting
versions when writing software in a team environment.

To help you adopt this approach, create will use the current timestamp as the migration version.
When you're ready to deploy your migrations in a production environment, we also provide a helpful
fix command to convert your migrations into sequential order, while preserving the timestamp
ordering. We recommend running fix in the CI pipeline, and only when the migrations are ready for
production.

Credit Copy Chapter

The gopher mascot was designed by Renée French / CC
3.0. For more info check out the Go
Blog. Adapted by Ellen.

License Copy Chapter

Licensed under MIT License

File: examples/go-migrations/README.md Copy Chapter

SQL + Go migrations

This example: Custom goose binary with built-in Go migrations Copy Chapter

$ go build -o goose-custom *.go

$ ./goose-custom sqlite3 ./foo.db status
    Applied At                  Migration
    =======================================
    Pending                  -- 00001_create_users_table.sql
    Pending                  -- 00002_rename_root.go
    Pending                  -- 00003_add_user_no_tx.go

$ ./goose-custom sqlite3 ./foo.db up
    OK   00001_create_users_table.sql (711.58µs)
    OK   00002_rename_root.go (302.08µs)
    OK   00003_add_user_no_tx.go (648.71µs)
    goose: no migrations to run. current version: 3

$ ./goose-custom sqlite3 ./foo.db status
    Applied At                  Migration
    =======================================
    00001_create_users_table.sql
    00002_rename_root.go
    00003_add_user_no_tx.go

Best practice: Split migrations into a standalone package Copy Chapter

1. Move main.go into your cmd/ directory

2. Rename package name in all *_.go migration files from main to migrations.

3. Import this migrations package from your custom cmd/main.go file:

   go Copy

   import (
       // Invoke init() functions within migrations pkg.
       _ "github.com/pressly/goose/example/migrations-go"
   )

File: examples/sql-migrations/README.md Copy Chapter

SQL migrations only

See this example for Go migrations.

$ go install github.com/pressly/goose/v3/cmd/goose@latest

$ goose sqlite3 ./foo.db status
    Applied At                  Migration
    =======================================
    Pending                  -- 00001_create_users_table.sql
    Pending                  -- 00002_rename_root.sql

$ goose sqlite3 ./foo.db up
OK    00001_create_users_table.sql
OK    00002_rename_root.sql
goose: no migrations to run. current version: 2

$ goose sqlite3 ./foo.db status
    Applied At                  Migration
    =======================================
    Mon Jun 19 21:56:00 2017 -- 00001_create_users_table.sql
    Mon Jun 19 21:56:00 2017 -- 00002_rename_root.sql

File: internal/testing/integration/README.md Copy Chapter

Integration tests

This directory contains integration tests for the pressly/goose/v3 Go module. An
integration test is a test that runs against a real database (docker container) and exercises the
same driver used by the CLI.

Why is this a separate module? Copy Chapter

There are separate go.mod and go.sum files in this directory to allow for the use of different
dependencies. We leverage multi-module workspaces to glue
things together.

Namely, we want to avoid dependencies on docker, sql drivers, and other dependencies that are
not necessary for the core functionality of the goose library.

Overview Copy Chapter

There are separate migration files for each database that we support, see the migrations
directory. Databases typically have different SQL syntax and features, so the
migration files are different.

A good set of migrations should be representative of the types of migrations users will write
typically write. This should include:

• Creating and dropping tables
• Adding and removing columns
• Creating and dropping indexes
• Inserting and deleting data
• Complex SQL statements that require special handling with StatementBegin and StatementEnd
  annotations
• Statements that must run outside a transaction, annotated with -- +goose NO TRANSACTION

There is a common test function that applies migrations up, down and then up again.

The gold standard is the PostgreSQL migration files. We try to make other migration files as close
to the PostgreSQL files as possible, but this is not always possible or desirable.

Lastly, some tests will assert for database state after migrations are applied.

To add a new .sql file, you can use the following command:

goose -s -dir testdata/migrations/<database_name> create <filename> sql

• Update the database name (e.g. postgres)
• Update the filename name (e.g. b) as needed

Limitation Copy Chapter

Note, the integration tests are not exhaustive.

They are meantto ensure that the goose library works with the various databases that we support and
the chosen drivers. We do not test every possible combination of operations, nor do we test every
possible edge case. We rely on the unit tests in the goose package to cover library-specific logic.