File: README.md
sqlc: A SQL Compiler
sqlc generates type-safe code from SQL. Here's how it works:
- You write queries in SQL.
- You run sqlc to generate code with type-safe interfaces to those queries.
- You write application code that calls the generated code.
Check out an interactive example to see it in action, and the introductory blog post for the motivation behind sqlc.
Overview
Supported languages
Additional languages can be added via plugins.
Sponsors
Development is possible thanks to our sponsors. If you would like to support sqlc,
please consider sponsoring on GitHub.
File: docs/guides/development.md
Developing sqlc
Building
For local development, install sqlc under an alias. We suggest sqlc-dev.
go build -o ~/go/bin/sqlc-dev ./cmd/sqlcInstall sqlc-gen-json to avoid test failure.
go build -o ~/go/bin/sqlc-gen-json ./cmd/sqlc-gen-jsonRunning Tests
go test ./...To run the tests in the examples folder, use the examples tag.
go test --tags=examples ./...These tests require locally-running database instances. Run these databases
using Docker Compose.
docker compose up -dThe tests use the following environment variables to connect to the
database
For PostgreSQL
Variable Default Value
-------------------------
PG_HOST 127.0.0.1
PG_PORT 5432
PG_USER postgres
PG_PASSWORD mysecretpassword
PG_DATABASE dinotestFor MySQL
Variable Default Value
-------------------------
MYSQL_HOST 127.0.0.1
MYSQL_PORT 3306
MYSQL_USER root
MYSQL_ROOT_PASSWORD mysecretpassword
MYSQL_DATABASE dinotestFile: docs/guides/migrating-off-hosted-managed-databases.md
Migrating off hosted managed databases
Starting in sqlc 1.27.0, managed databases will require a database server URI in the configuration file.
This guide walks you through migrating to a locally running database server.
Run a database server locally
There are many options for running a database server locally, but this guide
will use Docker Compose, as it can support
both MySQL and PostgreSQL.
If you're using macOS and PostgreSQL, Postgres.app is also a good option.
For MySQL, create a docker-compose.yml file with the following contents:
version: "3.8"
services:
mysql:
image: "mysql/mysql-server:8.0"
ports:
- "3306:3306"
restart: always
environment:
MYSQL_DATABASE: dinotest
MYSQL_ROOT_PASSWORD: mysecretpassword
MYSQL_ROOT_HOST: '%'For PostgreSQL, create a docker-compose.yml file with the following contents:
version: "3.8"
services:
postgresql:
image: "postgres:16"
ports:
- "5432:5432"
restart: always
environment:
POSTGRES_DB: postgres
POSTGRES_PASSWORD: mysecretpassword
POSTGRES_USER: postgresdocker compose up -dUpgrade sqlc
You must be running sqlc v1.31.1 or greater to have access to the servers
configuration.
Add servers to configuration
version: '2'
cloud:
project: ''
+ servers:
+ - name: mysql
+ uri: mysql://localhost:3306
+ - name: postgres
+ uri: postgres://localhost:5432/postgres?sslmode=disableRe-generate the code
Run sqlc generate. A database with the sqlc_managed_ prefix will be automatically created and used for query analysis.
File: docs/guides/migrating-to-sqlc-gen-kotlin.md
Migrating to sqlc-gen-kotlin
Starting in sqlc 1.16.0, built-in Kotlin support has been deprecated. It will
be fully removed in 1.17.0 in favor of sqlc-gen-kotlin.
This guide will walk you through migrating to the sqlc-gen-kotlin plugin,
which involves three steps.
- Add the sqlc-gen-kotlin plugin
- Migrate each package
- Re-generate the code
Add the sqlc-gen-kotlin plugin
In your configuration file, add a plugins array if you don't have one
already. Add the following configuration for the plugin:
{
"version": "2",
"plugins": [
{
"name": "kt",
"wasm": {
"url": "https://downloads.sqlc.dev/plugin/sqlc-gen-kotlin_1.0.0.wasm",
"sha256": "7620dc5d462de41fdc90e2011232c842117b416c98fd5c163d27c5738431a45c"
}
}
]
}version: "2"
plugins:
- name: "kt"
wasm:
url: "https://downloads.sqlc.dev/plugin/sqlc-gen-kotlin_1.0.0.wasm"
sha256: "7620dc5d462de41fdc90e2011232c842117b416c98fd5c163d27c5738431a45c"Migrate each package
Your package configuration should currently looks something like this for JSON.
"sql": [
{
"schema": "schema.sql",
"queries": "query.sql",
"engine": "postgresql",
"gen": {
"kotlin": {
"out": "src/main/kotlin/com/example/foo",
"package": "com.example.foo"
}
}
}
]Or this if you're using YAML.
sql:
- schema: "schema.sql"
queries: "query.sql"
engine: "postgresql"
gen:
kotlin:
out: "src/main/kotlin/com/example/foo"
package: "com.example.foo"To use the plugin, you'll need to replace the gen mapping with the codegen
collection. Add the plugin field, setting it to kt. All fields other thanout need to be moved into the options mapping.
After you're done, it should look like this for JSON.
"sql": [
{
"schema": "schema.sql",
"queries": "query.sql",
"engine": "postgresql",
"codegen": [
{
"out": "src/main/kotlin/com/example/foo",
"plugin": "kt",
"options": {
"package": "com.example.foo"
}
}
]
}
]Or this for YAML.
sql:
- schema: "schema.sql"
queries: "query.sql"
engine: "postgresql"
codegen:
- plugin: "kt"
out: "src/main/kotlin/com/example/foo"
options:
package: "com.example.foo"Re-generate the code
Run sqlc generate. The plugin will produce the same output, so you shouldn't
see any changes. The first time sqlc generate is run, the plugin must be
downloaded and compiled, resulting in a slightly longer runtime. Subsequentgenerate calls will be fast.
File: docs/guides/migrating-to-sqlc-gen-python.md
Migrating to sqlc-gen-python
Starting in sqlc 1.16.0, built-in Python support has been deprecated. It will
be fully removed in 1.17.0 in favor of sqlc-gen-python.
This guide will walk you through migrating to the sqlc-gen-python plugin,
which involves three steps.
- Add the sqlc-gen-python plugin
- Migrate each package
- Re-generate the code
Add the sqlc-gen-python plugin
In your configuration file, add a plugins array if you don't have one
already. Add the following configuration for the plugin:
{
"version": "2",
"plugins": [
{
"name": "py",
"wasm": {
"url": "https://downloads.sqlc.dev/plugin/sqlc-gen-python_1.0.0.wasm",
"sha256": "aca83e1f59f8ffdc604774c2f6f9eb321a2b23e07dc83fc12289d25305fa065b"
}
}
]
}version: "2"
plugins:
- name: "py"
wasm:
url: "https://downloads.sqlc.dev/plugin/sqlc-gen-python_1.0.0.wasm"
sha256: "aca83e1f59f8ffdc604774c2f6f9eb321a2b23e07dc83fc12289d25305fa065b"Migrate each package
Your package configuration should currently looks something like this for JSON.
"sql": [
{
"schema": "schema.sql",
"queries": "query.sql",
"engine": "postgresql",
"gen": {
"python": {
"out": "src",
"package": "foo",
"emit_sync_querier": true,
"emit_async_querier": true,
"query_parameter_limit": 5
}
}
}
]Or this if you're using YAML.
sql:
- schema: "schema.sql"
queries: "query.sql"
engine: "postgresql"
gen:
python:
out: "src"
package: "foo"
emit_sync_querier: true
emit_async_querier: true
query_parameter_limit: 5To use the plugin, you'll need to replace the gen mapping with the codegen
collection. Add the plugin field, setting it to py. All fields other thanout need to be moved into the options mapping.
After you're done, it should look like this for JSON.
"sql": [
{
"schema": "schema.sql",
"queries": "query.sql",
"engine": "postgresql",
"codegen": [
{
"out": "src",
"plugin": "py",
"options": {
"package": "authors",
"emit_sync_querier": true,
"emit_async_querier": true,
"query_parameter_limit": 5
}
}
]
}
]Or this for YAML.
sql:
- schema: "schema.sql"
queries: "query.sql"
engine: "postgresql"
codegen:
- plugin: "py"
out: "src"
options:
package: "foo"
emit_sync_querier: true
emit_async_querier: true
query_parameter_limit: 5Re-generate the code
Run sqlc generate. The plugin will produce the same output, so you shouldn't
see any changes. The first time sqlc generate is run, the plugin must be
downloaded and compiled, resulting in a slightly longer runtime. Subsequentgenerate calls will be fast.
File: docs/guides/plugins.md
Using plugins
To use plugins, you must be using Version 2 of
the configuration file. The top-level plugins array defines the available
plugins.
WASM plugins
WASM plugins are fully sandboxed; they do not have access to the network,
filesystem, or environment variables.
In the codegen section, the out field dictates what directory will contain
the new files. The plugin key must reference a plugin defined in the
top-level plugins map. Any options are serialized to a string as JSON and
passed on to the plugin itself.
version: '2'
plugins:
- name: greeter
wasm:
url: https://github.com/sqlc-dev/sqlc-gen-greeter/releases/download/v0.1.0/sqlc-gen-greeter.wasm
sha256: afc486dac2068d741d7a4110146559d12a013fd0286f42a2fc7dcd802424ad07
sql:
- schema: schema.sql
queries: query.sql
engine: postgresql
codegen:
- out: gen
plugin: greeter
options:
lang: en-USFor a complete working example see the following files:
- sqlc-gen-greeter
- A WASM plugin (written in Rust) that outputs a friendly message
- wasm_plugin_sqlc_gen_greeter
- An example project showing how to use a WASM plugin
Process plugins
Process-based plugins offer minimal security. Only use plugins that you
trust. Better yet, only use plugins that you've written yourself.
In the codegen section, the out field dictates what directory will contain
the new files. The plugin key must reference a plugin defined in the
top-level plugins map. Any options are serialized to a string as JSON and
passed on to the plugin itself.
version: '2'
plugins:
- name: jsonb
process:
cmd: sqlc-gen-json
sql:
- schema: schema.sql
queries: query.sql
engine: postgresql
codegen:
- out: gen
plugin: jsonb
options:
indent: " "
filename: codegen.jsonFor a complete working example see the following files:
- sqlc-gen-json
- A process-based plugin that serializes the CodeGenRequest to JSON
- process_plugin_sqlc_gen_json
- An example project showing how to use a process-based plugin
- process_plugin_sqlc_gen_json
- An example project showing how to use a process-based plugin using json
Environment variables
By default, plugins do not inherit access to environment variables. Instead,
you can configure access on a per-variable basis. For example, if your plugin
needs the PATH environment variable, add PATH to the env list in theplugins collection.
version: '2'
sql:
- schema: schema.sql
queries: query.sql
engine: postgresql
codegen:
- out: gen
plugin: test
plugins:
- name: test
env:
- PATH
wasm:
url: https://github.com/sqlc-dev/sqlc-gen-test/releases/download/v0.1.0/sqlc-gen-test.wasm
sha256: 138220eae508d4b65a5a8cea555edd155eb2290daf576b7a8b96949acfeb3790A variable named SQLC_VERSION is always included in the plugin's
environment, set to the version of the sqlc executable invoking it.
File: docs/guides/privacy.md
Privacy and data collection
These days, it feels like every piece of software is tracking you. From your
browser, to your phone, to your terminal, programs collect as much data about
you as possible and send it off to the cloud for analysis.
We believe the best way to keep data safe is to never collect it in the first
place.
Our Privacy Pledge
The sqlc command line tool does not collect any information. It
does not send crash reports to a third-party. It does not gather anonymous
aggregate user behaviour analytics.
No analytics.
No finger-printing.
No tracking.
Not now and not in the future.
Distribution Channels
We distribute sqlc using popular package managers such as
Homebrew and Snapcraft. These
package managers and their associated command-line tools do collect usage
metrics.
We use these services to make it easy to for users to install sqlc. There will
always be an option to download sqlc from a stable URL.
Hosted Services
We provide a few hosted services in addition to the sqlc command line tool.
sqlc.dev
- Hosted on GitHub Pages
- Analytics with Plausible
docs.sqlc.dev
- Hosted on Read the Docs
- Analytics with Plausible
play.sqlc.dev
- Hosted on Heroku
- Playground data stored in Google Cloud Storage
- Automatically deleted after 30 days
app.sqlc.dev / api.sqlc.dev
File: docs/guides/using-go-and-pgx.rst
================ Using Go and pgx
.. note::
:code:pgx/v5 is supported starting from v1.18.0.
pgx is a pure Go driver and toolkit for PostgreSQL. It's become the default
PostgreSQL package for many Gophers since lib/pq was put into maintenance mode.
^^^^^^^^^^^^^^^
Getting started
^^^^^^^^^^^^^^^
To start generating code that uses pgx, set the :code:sql_package field in
your :code:sqlc.yaml configuration file. Valid options are :code:pgx/v4 or
:code:pgx/v5
.. code-block:: yaml
version: "2"
sql:
- engine: "postgresql"
queries: "query.sql"
schema: "query.sql"
gen:
go:
package: "db"
sql_package: "pgx/v5"
out: "db"If you don't have an existing sqlc project on hand, create a directory with the
configuration file above and the following :code:query.sql file.
.. code-block:: sql
CREATE TABLE authors (
id BIGSERIAL PRIMARY KEY,
name text NOT NULL,
bio text
);
-- name: GetAuthor :one
SELECT * FROM authors
WHERE id = $1 LIMIT 1;
-- name: ListAuthors :many
SELECT * FROM authors
ORDER BY name;
-- name: CreateAuthor :one
INSERT INTO authors (
name, bio
) VALUES (
$1, $2
)
RETURNING *;
-- name: DeleteAuthor :exec
DELETE FROM authors
WHERE id = $1;Generating the code will now give you pgx-compatible database access methods.
.. code-block:: bash
sqlc generate
^^^^^^^^^^^^^^^^^^^^^^^^^^
Generated code walkthrough
^^^^^^^^^^^^^^^^^^^^^^^^^^
The generated code is very similar to the code generated when using
:code:lib/pq. However, instead of using :code:database/sql, the code uses
pgx types directly.
.. code-block:: go
package main
import (
"context"
"fmt"
"os"
"github.com/jackc/pgx/v5"
"example.com/sqlc-tutorial/db"
)
func main() {
// urlExample := "postgres://username:password@localhost:5432/database_name"
conn, err := pgx.Connect(context.Background(), os.Getenv("DATABASE_URL"))
if err != nil {
fmt.Fprintf(os.Stderr, "Unable to connect to database: %v\n", err)
os.Exit(1)
}
defer conn.Close(context.Background())
q := db.New(conn)
author, err := q.GetAuthor(context.Background(), 1)
if err != nil {
fmt.Fprintf(os.Stderr, "GetAuthor failed: %v\n", err)
os.Exit(1)
}
fmt.Println(author.Name)
}.. note::
For production applications, consider using pgxpool for connection pooling:
.. code-block:: go
import (
"github.com/jackc/pgx/v5/pgxpool"
"example.com/sqlc-tutorial/db"
)
func main() {
pool, err := pgxpool.New(context.Background(), os.Getenv("DATABASE_URL"))
if err != nil {
fmt.Fprintf(os.Stderr, "Unable to create connection pool: %v\n", err)
os.Exit(1)
}
defer pool.Close()
q := db.New(pool)
// Use q the same way as with single connections
}File: docs/howto/ci-cd.md
Using sqlc in CI/CD
If your project has more than a single developer, we suggest running sqlc as
part of your CI/CD pipeline. The four subcommands you'll want to run are diff,vet, verify and push
sqlc diff ensures that your generated code is up to date. New developers to a
project may forget to run sqlc generate after adding a query or updating a
schema. They also might edit generated code. sqlc diff will catch both errors
by comparing the expected output from sqlc generate to what's on disk.
% sqlc diff
--- a/postgresql/query.sql.go
+++ b/postgresql/query.sql.go
@@ -55,7 +55,7 @@
const listAuthors = `-- name: ListAuthors :many
SELECT id, name, bio FROM authors
-ORDER BY name
+ORDER BY bio
`sqlc vet runs a set of lint rules against your SQL queries. These rules are
helpful in catching anti-patterns before they make it into production. Please
see the vet documentation for a complete guide to adding lint rules
for your project.
sqlc verify ensures that schema changes do not break production. Existing
queries are checked against new schema changes for correctness. Please see the
verify documentation for a complete guide.
sqlc push pushes your database schema, queries and configuration to sqlc
Cloud. These archives are used by verify to catch breaking changes to your
database schema. Learn more about uploading projects here
General setup
Install sqlc using the suggested instructions.
Create three steps in your pipeline for sqlc diff, sqlc vet, and sqlc verify. Run sqlc push after merge on your main branch.
GitHub Actions
We provide the setup-sqlc
GitHub Action to install sqlc. The action uses the built-in
tool-cache
to speed up the installation process.
diff
The following GitHub Workflow configuration runs sqlc diff on every push.
name: sqlc
on: [push]
jobs:
diff:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: sqlc-dev/setup-sqlc@v3
with:
sqlc-version: '1.31.1'
- run: sqlc diffvet
The following GitHub Workflow configuration runs sqlc vet on every push.
You can use sqlc vet without a database connection, but you'll need one if yoursqlc configuration references the built-in sqlc/db-prepare lint rule.
name: sqlc
on: [push]
jobs:
vet:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: sqlc-dev/setup-sqlc@v3
with:
sqlc-version: '1.31.1'
# Start a PostgreSQL server
- uses: sqlc-dev/action-setup-postgres@master
with:
postgres-version: "16"
id: postgres
- run: sqlc vet
env:
POSTGRESQL_SERVER_URI: ${{ steps.postgres.outputs.connection-uri }}?sslmode=disablepush
Pushing a project is powered by [sqlc Cloud](https://dashboard.sqlc.dev). Sign up for [free](https://dashboard.sqlc.dev) today.The following GitHub Workflow configuration runs sqlc push on
every push to main. Create an auth token via the
dashboard.
name: sqlc
on: [push]
jobs:
push:
runs-on: ubuntu-latest
if: ${{ github.ref == 'refs/heads/main' }}
steps:
- uses: actions/checkout@v3
- uses: sqlc-dev/setup-sqlc@v3
with:
sqlc-version: '1.31.1'
- run: sqlc push
env:
SQLC_AUTH_TOKEN: ${{ secrets.SQLC_AUTH_TOKEN }}verify
Verify database migrations is powered by [sqlc Cloud](https://dashboard.sqlc.dev). Sign up for [free](https://dashboard.sqlc.dev) today.name: sqlc
on: [push]
jobs:
verify:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: sqlc-dev/setup-sqlc@v3
with:
sqlc-version: '1.31.1'
- uses: sqlc-dev/action-setup-postgres@master
with:
postgres-version: "16"
id: postgres
- run: sqlc verify
env:
POSTGRESQL_SERVER_URI: ${{ steps.postgres.outputs.connection-uri }}?sslmode=disable
SQLC_AUTH_TOKEN: ${{ secrets.SQLC_AUTH_TOKEN }}
push:
runs-on: ubuntu-latest
if: ${{ github.ref == 'refs/heads/main' }}
steps:
- uses: sqlc-dev/setup-sqlc@v3
with:
sqlc-version: '1.31.1'
- run: sqlc push
env:
SQLC_AUTH_TOKEN: ${{ secrets.SQLC_AUTH_TOKEN }}File: docs/howto/ddl.md
Modifying the database schema
sqlc parses CREATE TABLE and ALTER TABLE statements in order to generate
the necessary code.
CREATE TABLE authors (
id SERIAL PRIMARY KEY,
birth_year int NOT NULL
);
ALTER TABLE authors ADD COLUMN bio text NOT NULL;
ALTER TABLE authors DROP COLUMN birth_year;
ALTER TABLE authors RENAME TO writers;package db
type Writer struct {
ID int
Bio string
}Handling SQL migrations
sqlc does not perform database migrations for you. However, sqlc is able to
differentiate between up and down migrations. sqlc ignores down migrations when
parsing SQL files.
sqlc supports parsing migrations from the following tools:
To enable migration parsing, specify the migration directory instead of a schema file:
version: "2"
sql:
- engine: "postgresql"
queries: "query.sql"
schema: "db/migrations"
gen:
go:
package: "tutorial"
out: "tutorial"atlas
-- Create "post" table
CREATE TABLE "public"."post" ("id" integer NOT NULL, "title" text NULL, "body" text NULL, PRIMARY KEY ("id"));package db
type Post struct {
ID int
Title sql.NullString
Body sql.NullString
}dbmate
-- migrate:up
CREATE TABLE foo (bar INT NOT NULL);
-- migrate:down
DROP TABLE foo;package db
type Foo struct {
Bar int32
}golang-migrate
Warning:
golang-migrate interprets
migration filenames numerically. However, sqlc parses migration files in
lexicographic order. If you choose to have sqlc enumerate your migration files,
make sure their numeric ordering matches their lexicographic ordering to avoid
unexpected behavior. This can be done by prepending enough zeroes to the
migration filenames.
This doesn't work as intended.
1_initial.up.sql
...
9_foo.up.sql
# this migration file will be parsed BEFORE 9_foo
10_bar.up.sqlThis worked as intended.
001_initial.up.sql
...
009_foo.up.sql
010_bar.up.sqlIn 20060102.up.sql:
CREATE TABLE post (
id int NOT NULL,
title text,
body text,
PRIMARY KEY(id)
);In 20060102.down.sql:
DROP TABLE post;package db
type Post struct {
ID int
Title sql.NullString
Body sql.NullString
}goose
Warning:
sqlc parses migration files in lexicographic order. If you are using numeric filenames for migrations in Goose and you choose to have sqlc enumerate your migration files,
make sure their numeric ordering matches their lexicographic ordering to avoid
unexpected behavior. This can be done by prepending enough zeroes to the
migration filenames.
This doesn't work as intended.
1_initial.sql
...
9_foo.sql
# this migration file will be parsed BEFORE 9_foo
10_bar.sqlThis worked as intended.
001_initial.sql
...
009_foo.sql
010_bar.sql-- +goose Up
CREATE TABLE post (
id int NOT NULL,
title text,
body text,
PRIMARY KEY(id)
);
-- +goose Down
DROP TABLE post;package db
type Post struct {
ID int
Title sql.NullString
Body sql.NullString
}sql-migrate
-- +migrate Up
-- SQL in section 'Up' is executed when this migration is applied
CREATE TABLE people (id int);
-- +migrate Down
-- SQL section 'Down' is executed when this migration is rolled back
DROP TABLE people;package db
type People struct {
ID int32
}tern
CREATE TABLE comment (id int NOT NULL, text text NOT NULL);
---- create above / drop below ----
DROP TABLE comment;package db
type Comment struct {
ID int32
Text string
}