alembic Documentation

A database migrations tool for SQLAlchemy.

4,456 stars Python Python 24 files ยท ~140,506 tokens #python#sql#sqlalchemy
RAW Doc

Technical Documentation: sqlalchemy/alembic

โ„น๏ธ Provenance: Multi-Tier Ingestion ยท CodeWiki Reference

Build Autogenerate (`docs/build/autogenerate.rst`)

Auto Generating Migrations

Alembic can view the status of the database (pointed to by sqlalchemy.url in
your alembic.ini file using the current schema) and compare against the
table metadata in the application (your ORM which defines the proposed schema),
generating the "obvious" migrations based on a comparison. This is achieved
using the --autogenerate option to the alembic revision command, which
places so-called candidate migrations into our new migrations file. We
review and modify these by hand as needed, then proceed normally.

To use autogenerate, we first need to modify our env.py so that it gets access
to a table metadata object that contains the target. Suppose our application
has a :ref:declarative base <sqla:declarative_toplevel>
in myapp.mymodel. This base contains a :class:~sqlalchemy.schema.MetaData object which
contains :class:~sqlalchemy.schema.Table objects defining our database. We make sure this
is loaded in env.py and then passed to :meth:.EnvironmentContext.configure via the
target_metadata argument. The env.py sample script used in the
generic template already has a
variable declaration near the top for our convenience, where we replace None
with our :class:~sqlalchemy.schema.MetaData. Starting with::

text
# add your model's MetaData object here
# for 'autogenerate' support
# from myapp import mymodel
# target_metadata = mymodel.Base.metadata
target_metadata = None

we change to::

text
from myapp.mymodel import Base
target_metadata = Base.metadata

.. note::

The above example refers to the generic alembic env.py template, e.g.
the one created by default when calling upon alembic init, and not
the special-use templates such as multidb. Please consult the source
code and comments within the env.py script directly for specific
guidance on where and how the autogenerate metadata is established.

If we look later in the script, down in run_migrations_online(),
we can see the directive passed to :meth:.EnvironmentContext.configure::

text
def run_migrations_online():
    engine = engine_from_config(
                config.get_section(config.config_ini_section), prefix='sqlalchemy.')

    with engine.connect() as connection:
        context.configure(
                    connection=connection,
                    target_metadata=target_metadata
                    )

        with context.begin_transaction():
            context.run_migrations()

We can then use the alembic revision command in conjunction with the
--autogenerate option. Suppose
our :class:~sqlalchemy.schema.MetaData contained a definition for the account table,
and the database did not. We'd get output like::

text
$ alembic revision --autogenerate -m "Added account table"
INFO [alembic.context] Detected added table 'account'
Generating /path/to/foo/alembic/versions/27c6a30d7c24.py...done

We can then view our file 27c6a30d7c24.py and see that a rudimentary migration
is already present::

text
"""empty message

Revision ID: 27c6a30d7c24
Revises: None
Create Date: 2011-11-08 11:40:27.089406

"""

# revision identifiers, used by Alembic.
revision = '27c6a30d7c24'
down_revision = None

from alembic import op
import sqlalchemy as sa

def upgrade():
    ### commands auto generated by Alembic - please adjust! ###
    op.create_table(
    'account',
    sa.Column('id', sa.Integer()),
    sa.Column('name', sa.String(length=50), nullable=False),
    sa.Column('description', sa.VARCHAR(200)),
    sa.Column('last_transaction_date', sa.DateTime()),
    sa.PrimaryKeyConstraint('id')
    )
    ### end Alembic commands ###

def downgrade():
    ### commands auto generated by Alembic - please adjust! ###
    op.drop_table("account")
    ### end Alembic commands ###

The migration hasn't actually run yet, of course. We do that via the usual upgrade
command. We should also go into our migration file and alter it as needed, including
adjustments to the directives as well as the addition of other directives which these may
be dependent on - specifically data changes in between creates/alters/drops.

.. _autogenerate_detects:

What does Autogenerate Detect (and what does it *not* detect?)

The vast majority of user issues with Alembic centers on the topic of what
kinds of changes autogenerate can and cannot detect reliably, as well as
how it renders Python code for what it does detect. It is critical to
note that autogenerate is not intended to be perfect. It is always
necessary to manually review and correct the candidate migrations
that autogenerate produces. The feature is getting more and more
comprehensive and error-free as releases continue, but one should take
note of the current limitations.

Autogenerate will detect:

  • Table additions, removals.
  • Column additions, removals.
  • Change of nullable status on columns.
  • Basic changes in indexes and explicitly-named unique constraints
  • Basic changes in foreign key constraints

Autogenerate can optionally detect:

  • Named CHECK constraint additions and removals. This is off by default
    and must be enabled explicitly; the detection is name-based only and has
    significant limitations. See :ref:autogenerate_check_constraints below.

  • Change of column type. This will occur by default unless the parameter
    :paramref:.EnvironmentContext.configure.compare_type is set to False.
    The default implementation will reliably detect major changes,
    such as between :class:.Numeric and :class:.String, as well as
    accommodate for the types generated by SQLAlchemy's "generic" types such as
    :class:.Boolean. Arguments that are shared between both types, such as
    length and precision values, will also be compared. If either the metadata
    type or database type has additional arguments beyond that of the other
    type, these are not compared, such as if one numeric type featured a
    "scale" and other type did not, this would be seen as the backing database
    not supporting the value, or reporting on a default that the metadata did not
    specify.

    The type comparison logic is fully extensible as well; see
    :ref:compare_types for details.

  • Change of server default. This will occur if you set
    the :paramref:.EnvironmentContext.configure.compare_server_default
    parameter to True, or to a custom callable function.
    This feature works well for simple cases but cannot always produce
    accurate results. The Postgresql backend will actually invoke
    the "detected" and "metadata" values against the database to
    determine equivalence. The feature is off by default so that
    it can be tested on the target schema first. Like type comparison,
    it can also be customized by passing a callable; see the
    function's documentation for details.

Autogenerate can not detect:

  • Changes of table name. These will come out as an add/drop of two different
    tables, and should be hand-edited into a name change instead.
  • Changes of column name. Like table name changes, these are detected as
    a column add/drop pair, which is not at all the same as a name change.
  • Anonymously named constraints. Give your constraints a name,
    e.g. UniqueConstraint('col1', 'col2', name="my_name"). See the section
    :doc:naming for background on how to configure automatic naming schemes
    for constraints.
  • Special SQLAlchemy types such as :class:~sqlalchemy.types.Enum when generated
    on a backend which doesn't support ENUM directly - this because the
    representation of such a type
    in the non-supporting database, i.e. a CHAR+ CHECK constraint, could be
    any kind of CHAR+CHECK. For SQLAlchemy to determine that this is actually
    an ENUM would only be a guess, something that's generally a bad idea.
    To implement your own "guessing" function here, use the
    :meth:sqlalchemy.events.DDLEvents.column_reflect event
    to detect when a CHAR (or whatever the target type is) is reflected,
    and change it to an ENUM (or whatever type is desired) if it is known that
    that's the intent of the type. The
    :meth:sqlalchemy.events.DDLEvents.after_parent_attach
    can be used within the autogenerate process to intercept and un-attach
    unwanted CHECK constraints.

Autogenerate can't currently, but will eventually detect:

  • Some free-standing constraint additions and removals may not be supported,
    including PRIMARY KEY, EXCLUDE; these are not necessarily implemented
    within the autogenerate detection system and also may not be supported by
    the supporting SQLAlchemy dialect.
  • Sequence additions, removals - not yet implemented.

.. _autogenerate_check_constraints:

Detecting CHECK Constraints
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. versionadded:: 1.19.0

.. versionchanged:: 1.19.2 This feature is no longer enabled by default.
See the warning below for the rationale.

Autogenerate includes optional, off by default support for detecting the
addition and removal of named CHECK constraints. The detection is
name-based only: a metadata-side CHECK constraint and a reflected CHECK
constraint that share the same name are always presumed to be equivalent,
regardless of any difference in their expression text.

The feature is implemented as a built-in
:ref:plugin <alembic.plugins.toplevel> named
alembic.ext.checkconstraint_byname. Unlike Alembic's other built-in
autogenerate plugins, it is named outside of the alembic.autogenerate.
namespace and is therefore not part of the alembic.autogenerate.*
wildcard; it must be named explicitly in the
:paramref:.EnvironmentContext.configure.autogenerate_plugins list in order
to be used::

text
context.configure(
    # ...
    autogenerate_plugins=[
        "alembic.autogenerate.*",
        "alembic.ext.checkconstraint_byname",
    ]
)

The plugin was named alembic.autogenerate.checkconstraint_byname in
versions 1.19.0 and 1.19.1. That name continues to be accepted, both to
enable the plugin and within a "~" exclusion, so an env.py written
against those releases requires no change.

.. warning::

text
This feature has significant limitations and should be enabled only for
schemas that fit the narrow set of conditions it can handle.   Before
turning it on, note all of the following:

* **Expression changes are never detected.**  Two CHECK constraints that
  share a name are presumed equal even if their SQL expressions differ
  entirely.  A migration that alters the condition of an existing
  constraint will not be generated.

* **Every CHECK constraint in the target metadata must be named.**  A
  CHECK constraint with no name, whether written directly as
  ``CheckConstraint("x > 5")`` or left unnamed by a naming convention
  that does not produce a name for it, is skipped on the metadata side.
  Backends such as PostgreSQL nonetheless assign a name to every
  constraint in the database, so the reflected constraint matches
  nothing in the metadata and is reported as a **spurious removal**.
  This includes CHECK constraints generated by types such as
  :class:`~sqlalchemy.types.Enum` and :class:`~sqlalchemy.types.Boolean`
  that have ``create_constraint=True``; assign a ``name`` to these
  types to make the CHECK constraints named.

  .. versionchanged:: 1.19.2  CHECK constraints that are "type bound" are
      no longer ignored on the metadata side, so that normal name-based
      matching can occur.


* **Not all backends reflect CHECK constraints**, in which case the
  plugin silently does nothing.

Unlike unique constraints, which fall back to matching on column names
when the metadata side is unnamed, there is no comparable fallback
available for CHECK constraints.   Matching on the expression instead of
the name is not feasible, because dialects are free to normalize,
reformat, or otherwise rewrite the original DDL text.   PostgreSQL, for
example, reflects ``x != y`` as ``x <> y``, ``x BETWEEN 1 AND 10`` as
``x >= 1 AND x <= 10``, and ``s IN ('a', 'b')`` as
``s::text = ANY (ARRAY['a'::character varying, ...])``.  Comparing that
text against the metadata expression would report a change on nearly
every constraint.

Given the above, the feature is most appropriate for a schema in which all
CHECK constraints are explicitly named, ideally via a naming convention (see
:doc:naming). As always, the candidate migrations that autogenerate
produces must be reviewed by hand.

Notable 3rd-party libraries that extend the built-in Alembic autogenerate functionality
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  • alembic-utils <https://github.com/olirice/alembic_utils>_
    A library that adds autogenerate support for PostgreSQL functions, views, triggers, etc.
  • alembic-postgresql-enum <https://pypi.org/project/alembic-postgresql-enum>_
    A library that adds autogenerate support for creation, alteration and deletion of Enums in PostgreSQL.

Autogenerating Multiple MetaData collections

The target_metadata collection may also be defined as a sequence
if an application has multiple :class:~sqlalchemy.schema.MetaData
collections involved::

text
from myapp.mymodel1 import Model1Base
from myapp.mymodel2 import Model2Base
target_metadata = [Model1Base.metadata, Model2Base.metadata]

The sequence of :class:~sqlalchemy.schema.MetaData collections will be
consulted in order during the autogenerate process. Note that each
:class:~sqlalchemy.schema.MetaData must contain unique table keys
(e.g. the "key" is the combination of the table's name and schema);
if two :class:~sqlalchemy.schema.MetaData objects contain a table
with the same schema/name combination, an error is raised.

.. _autogenerate_include_hooks:

Controlling What to be Autogenerated

The autogenerate process scans across all table objects within
the database that is referred towards by the current database connection
in use.

The list of objects that are scanned in the target database connection include:

  • The "default" schema currently referred towards by the database connection.

  • If the :paramref:.EnvironmentContext.configure.include_schemas is set to
    True, all non-default "schemas", which are those names returned by the
    :meth:~sqlalchemy.engine.reflection.Inspector.get_schema_names method of
    :class:~sqlalchemy.engine.reflection.Inspector. The SQLAlchemy document
    :ref:sqla:schema_table_schema_name discusses the concept of a
    "schema" in detail.

  • Within each "schema", all tables present are scanned using the
    :meth:~sqlalchemy.engine.reflection.Inspector.get_table_names method of
    :class:~sqlalchemy.engine.reflection.Inspector.

  • Within each "table", most sub-objects of the each
    :class:~sqlalchemy.schema.Table construct are scanned, including columns
    and some forms of constraints. This process ultimately involves the use of
    methods on :class:~sqlalchemy.engine.reflection.Inspector including
    :meth:~sqlalchemy.engine.reflection.Inspector.get_columns,
    :meth:~sqlalchemy.engine.reflection.Inspector.get_indexes,
    :meth:~sqlalchemy.engine.reflection.Inspector.get_unique_constraints,
    :meth:~sqlalchemy.engine.reflection.Inspector.get_foreign_keys (as of this
    writing, CHECK constraints and primary key constraints are not yet included).

.. seealso::

text
:ref:`sqla:schema_table_schema_name` - in depth introduction to how
SQLAlchemy interprets schema names

:ref:`sqla:postgresql_schema_reflection` - important notes specific to the
PostgreSQL database

Omitting Schema Names from the Autogenerate Process
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

As the above set of database objects are typically to be compared to the contents of
a single :class:~sqlalchemy.schema.MetaData object, particularly when the
:paramref:.EnvironmentContext.configure.include_schemas flag is enabled
there is an important need to filter out unwanted "schemas", which for some
database backends might be the list of all the databases present. This
filtering is best performed using the :paramref:.EnvironmentContext.configure.include_name
hook, which provides for a callable that may return a boolean true/false
indicating if a particular schema name should be included::

text
def include_name(name, type_, parent_names):
    if type_ == "schema":
        # note this will not include the default schema
        return name in ["schema_one", "schema_two"]
    else:
        return True

context.configure(
    # ...
    include_schemas = True,
    include_name = include_name
)

Above, when the list of schema names is first retrieved, the names will be
filtered through the above include_name function so that only schemas
named "schema_one" and "schema_two" will be considered by the
autogenerate process.

In order to include the default schema, that is, the schema that is
referred towards by the database connection without any explicit
schema being specified, the name passed to the hook is None. To alter
our above example to also include the default schema, we compare to
None as well::

text
def include_name(name, type_, parent_names):
    if type_ == "schema":
        # this **will* include the default schema
        return name in [None, "schema_one", "schema_two"]
    else:
        return True

context.configure(
    # ...
    include_schemas = True,
    include_name = include_name
)

Omitting Table Names from the Autogenerate Process
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The :paramref:.EnvironmentContext.configure.include_name hook is also
most appropriate to limit the names of tables in the target database
to be considered. If a target database has many tables that are not
part of the :class:~sqlalchemy.schema.MetaData, the autogenerate process
will normally assume these are extraneous tables in the database to be
dropped, and it will generate a :meth:.Operations.drop_table operation
for each. To prevent this, the :paramref:.EnvironmentContext.configure.include_name
hook may be used to search for each name within the
:attr:~sqlalchemy.schema.MetaData.tables collection of the
:class:~sqlalchemy.schema.MetaData object and ensure names
which aren't present are not included::

text
target_metadata = MyModel.metadata

def include_name(name, type_, parent_names):
    if type_ == "table":
        return name in target_metadata.tables
    else:
        return True

context.configure(
    # ...
    target_metadata = target_metadata,
    include_name = include_name,
    include_schemas = False
)

The above example is limited to table names present in the default schema only.
In order to search within a :class:~sqlalchemy.schema.MetaData collection for
schema-qualified table names as well, a table present in the non
default schema will be present under a name of the form
<schemaname>.. The
:paramref:.EnvironmentContext.configure.include_name hook will present
this schema name on a per-tablename basis in the parent_names dictionary,
using the key "schema_name" that refers to the name of the
schema currently being considered, or None if the schema is the default
schema of the database connection::

text
# example fragment

if parent_names["schema_name"] is None:
    return name in target_metadata.tables
else:
    # build out schema-qualified name explicitly...
    return (
        "%s.%s" % (parent_names["schema_name"], name) in
        target_metadata.tables
    )

However more simply, the parent_names dictionary will also include
the dot-concatenated name already constructed under the key
"schema_qualified_table_name", which will also be suitably formatted
for tables in the default schema as well with the dot omitted. So the
full example of omitting tables with schema support may look like::

text
target_metadata = MyModel.metadata

def include_name(name, type_, parent_names):
    if type_ == "schema":
        return name in [None, "schema_one", "schema_two"]
    elif type_ == "table":
        # use schema_qualified_table_name directly
        return (
            parent_names["schema_qualified_table_name"] in
            target_metadata.tables
        )
    else:
        return True

context.configure(
    # ...
    target_metadata = target_metadata,
    include_name = include_name,
    include_schemas = True
)

The parent_names dictionary will also include the key "table_name"
when the name being considered is that of a column or constraint object
local to a particular table.

The :paramref:.EnvironmentContext.configure.include_name hook only refers
to reflected objects, and not those located within the target
:class:~sqlalchemy.schema.MetaData collection. For more fine-grained
rules that include both :class:~sqlalchemy.schema.MetaData and reflected
object, the :paramref:.EnvironmentContext.configure.include_object hook
discussed in the next section is more appropriate.

Omitting Based on Object
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The :paramref:.EnvironmentContext.configure.include_object hook provides
for object-level inclusion/exclusion rules based on the :class:~sqlalchemy.schema.Table
object being reflected as well as the elements within it. This hook can
be used to limit objects both from the local :class:~sqlalchemy.schema.MetaData
collection as well as from the target database. The limitation is that when
it reports on objects in the database, it will have fully reflected that object,
which can be expensive if a large number of objects will be omitted. The
example below refers to a fine-grained rule that will skip changes on
:class:~sqlalchemy.schema.Column objects that have a user-defined flag
skip_autogenerate placed into the :attr:~sqlalchemy.schema.Column.info
dictionary::

text
def include_object(object, name, type_, reflected, compare_to):
    if (type_ == "column" and
        not reflected and
        object.info.get("skip_autogenerate", False)):
        return False
    else:
        return True

context.configure(
    # ...
    include_object = include_object
)

Comparing and Rendering Types

The area of autogenerate's behavior of comparing and rendering Python-based type objects
in migration scripts presents a challenge, in that there's
a very wide variety of types to be rendered in scripts, including those
part of SQLAlchemy as well as user-defined types. A few options
are given to help out with this task.

.. _autogen_module_prefix:

Controlling the Module Prefix
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

When types are rendered, they are generated with a module prefix, so
that they are available based on a relatively small number of imports.
The rules for what the prefix is is based on the kind of datatype as well
as configurational settings. For example, when Alembic renders SQLAlchemy
types, it will by default prefix the type name with the prefix sa.::

text
Column("my_column", sa.Integer())

The use of the sa. prefix is controllable by altering the value
of :paramref:.EnvironmentContext.configure.sqlalchemy_module_prefix::

text
def run_migrations_online():
    # ...

    context.configure(
                connection=connection,
                target_metadata=target_metadata,
                sqlalchemy_module_prefix="sqla.",
                # ...
                )

    # ...

In either case, the sa. prefix, or whatever prefix is desired, should
also be included in the imports section of script.py.mako; it also
defaults to import sqlalchemy as sa.

For user-defined types, that is, any custom type that
is not within the sqlalchemy. module namespace, by default Alembic will
use the value of module for the custom type::

text
Column("my_column", myapp.models.utils.types.MyCustomType())

The imports for the above type again must be made present within the migration,
either manually, or by adding it to script.py.mako.

The above custom type has a long and cumbersome name based on the use
of __module__ directly, which also implies that lots of imports would
be needed in order to accommodate lots of types. For this reason, it is
recommended that user-defined types used in migration scripts be made
available from a single module. Suppose we call it myapp.migration_types::

text
# myapp/migration_types.py

from myapp.models.utils.types import MyCustomType

We can first add an import for migration_types to our script.py.mako::

text
from alembic import op
import sqlalchemy as sa
import myapp.migration_types
${imports if imports else ""}

We then override Alembic's use of __module__ by providing a fixed
prefix, using the :paramref:.EnvironmentContext.configure.user_module_prefix
option::

text
def run_migrations_online():
    # ...

    context.configure(
                connection=connection,
                target_metadata=target_metadata,
                user_module_prefix="myapp.migration_types.",
                # ...
                )

    # ...

Above, we now would get a migration like::

Column("my_column", myapp.migration_types.MyCustomType())

Now, when we inevitably refactor our application to move MyCustomType
somewhere else, we only need modify the myapp.migration_types module,
instead of searching and replacing all instances within our migration scripts.

.. _autogen_render_types:

Affecting the Rendering of Types Themselves
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The methodology Alembic uses to generate SQLAlchemy and user-defined type constructs
as Python code is plain old __repr__(). SQLAlchemy's built-in types
for the most part have a __repr__() that faithfully renders a
Python-compatible constructor call, but there are some exceptions, particularly
in those cases when a constructor accepts arguments that aren't compatible
with __repr__(), such as a pickling function.

When building a custom type that will be rendered into a migration script,
it is often necessary to explicitly give the type a __repr__() that will
faithfully reproduce the constructor for that type. This, in combination
with :paramref:.EnvironmentContext.configure.user_module_prefix, is usually
enough. However, if additional behaviors are needed, a more comprehensive
hook is the :paramref:.EnvironmentContext.configure.render_item option.
This hook allows one to provide a callable function within env.py that will fully take
over how a type is rendered, including its module prefix::

text
def render_item(type_, obj, autogen_context):
    """Apply custom rendering for selected items."""

    if type_ == 'type' and isinstance(obj, MySpecialType):
        return "mypackage.%r" % obj

    # default rendering for other objects
    return False

def run_migrations_online():
    # ...

    context.configure(
                connection=connection,
                target_metadata=target_metadata,
                render_item=render_item,
                # ...
                )

    # ...

In the above example, we'd ensure our MySpecialType includes an appropriate
__repr__() method, which is invoked when we call it against "%r".

The callable we use for :paramref:.EnvironmentContext.configure.render_item
can also add imports to our migration script. The :class:.AutogenContext passed in
contains a datamember called :attr:.AutogenContext.imports, which is a Python
set() for which we can add new imports. For example, if MySpecialType
were in a module called mymodel.types, we can add the import for it
as we encounter the type::

text
def render_item(type_, obj, autogen_context):
    """Apply custom rendering for selected items."""

    if type_ == 'type' and isinstance(obj, MySpecialType):
        # add import for this type
        autogen_context.imports.add("from mymodel import types")
        return "types.%r" % obj

    # default rendering for other objects
    return False

The finished migration script will include our imports where the
${imports} expression is used, producing output such as::

from alembic import op
import sqlalchemy as sa
from mymodel import types

def upgrade():
op.add_column('sometable', Column('mycolumn', types.MySpecialType()))

.. _compare_types:

Comparing Types
^^^^^^^^^^^^^^^^

The default type comparison logic will work for SQLAlchemy built in types as
well as basic user defined types. This logic is enabled by default.
It can be disabled by setting the
:paramref:.EnvironmentContext.configure.compare_type to False::

text
context.configure(
    # ...
    compare_type = False
)

.. versionchanged:: 1.12.0 The default value of
:paramref:.EnvironmentContext.configure.compare_type has been changed
to True.

.. note::

The default type comparison logic (which is end-user extensible) currently
(as of Alembic version 1.4.0) works by comparing the generated SQL for a
column. It does this in two steps-

  • First, it compares the outer type of each column such as VARCHAR
    or TEXT. Dialect implementations can have synonyms that are considered
    equivalent, this is because some databases support types by converting them
    to another type. For example, NUMERIC and DECIMAL are considered equivalent
    on all backends, while on the Oracle backend the additional synonyms
    BIGINT, INTEGER, NUMBER, SMALLINT are added to this list of equivalents

  • Next, the arguments within the type, such as the lengths of
    strings, precision values for numerics, the elements inside of an
    enumeration are compared. If BOTH columns have arguments AND they are
    different, a change will be detected. If one column is just set to the
    default and the other has arguments, Alembic will pass on attempting to
    compare these. The rationale is that it is difficult to detect what a
    database backend sets as a default value without generating false
    positives.

Alternatively, the :paramref:.EnvironmentContext.configure.compare_type
parameter accepts a callable function which may be used to implement custom type
comparison logic, for cases such as where special user defined types
are being used::

text
def my_compare_type(context, inspected_column,
            metadata_column, inspected_type, metadata_type):
    # return False if the metadata_type is the same as the inspected_type
    # or None to allow the default implementation to compare these
    # types. a return value of True means the two types do not
    # match and should result in a type change operation.
    return None

context.configure(
    # ...
    compare_type = my_compare_type
)

Above, inspected_column is a :class:sqlalchemy.schema.Column as
returned by
:meth:sqlalchemy.engine.reflection.Inspector.reflect_table, whereas
metadata_column is a :class:sqlalchemy.schema.Column from the
local model environment. A return value of None indicates that default
type comparison to proceed.

.. _post_write_hooks:

Applying Post Processing and Python Code Formatters to Generated Revisions

Revision scripts generated by the alembic revision command can optionally
be piped through a series of post-production functions which may analyze or
rewrite Python source code generated by Alembic, within the scope of running
the revision command. The primary intended use of this feature is to run
code-formatting tools such as Black <https://black.readthedocs.io/>_ or
autopep8 <https://pypi.org/project/autopep8/>_, as well as custom-written
formatting and linter functions, on revision files as Alembic generates them.
Any number of hooks can be configured and they will be run in series, given the
path to the newly generated file as well as configuration options.

The post write hooks, when configured, run against generated revision files
regardless of whether or not the autogenerate feature was used.

.. note::

text
Alembic's post write system is partially inspired by the `pre-commit
<https://pre-commit.com/>`_ tool, which configures git hooks that reformat
source files as they are committed to a git repository.  Pre-commit can
serve this role for Alembic revision files as well, applying code
formatters to them as they are committed.  Alembic's post write hooks are
useful only in that they can format the files immediately upon generation,
rather than at commit time, and also can be useful for projects that prefer
not to use pre-commit.

.. _post_write_hooks_config:

Basic Post Processor Configuration
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The template samples for alembic.ini as well as pyproject.toml for
applicable templates now include commented-out configuration
illustrating how to configure code-formatting tools, or other tools like linters
to run against the newly generated file path. Example from an alembic.ini file::

text
[post_write_hooks]

# format using "black"
hooks=black

black.type = console_scripts
black.entrypoint = black
black.options = -l 79 REVISION_SCRIPT_FILENAME

The same example configured in a pyproject.toml file would look like::

text
[[tool.alembic.post_write_hooks]]

# format using "black"
name = "black"
type = "console_scripts"
entrypoint = "black"
options = "-l 79 REVISION_SCRIPT_FILENAME"

Above, we configure hooks to be a single post write hook labeled
"black". Note that this label is arbitrary. We then define the
configuration for the "black" post write hook, which includes:

  • type - this is the type of hook we are running. Alembic includes
    three hook runners:

    • "console_scripts", which is specifically a Python function that uses
      subprocess.run() to invoke a separate Python script against the revision file;
    • "exec", which uses subprocess.run() to execute an arbitrary binary; and
    • "module", which uses subprocess.run() to invoke a Python module directly.

    For a custom-written hook function, this configuration variable would
    refer to the name under which the custom hook was registered; see the
    next section for an example.

.. versionadded:: 1.12 added new exec runner

.. versionadded:: 1.16.3 added new module runner

The following configuration option is specific to the "console_scripts"
hook runner:

  • entrypoint - the name of the setuptools entrypoint <https://setuptools.readthedocs.io/en/latest/pkg_resources.html#entry-points>_
    that is used to define the console script. Within the scope of standard
    Python console scripts, this name will match the name of the shell command
    that is usually run for the code formatting tool, in this case black.

The following configuration option is specific to the "exec" hook runner:

  • executable - the name of the executable to invoke. Can be either a
    bare executable name which will be searched in $PATH, or a full pathname
    to avoid potential issues with path interception.

The following options are supported by both "console_scripts" and "exec":

  • options - a line of command-line options that will be passed to
    the code formatting tool. In this case, we want to run the command
    black /path/to/revision.py -l 79. By default, the revision path is
    positioned as the first argument. In order specify a different position,
    we can use the REVISION_SCRIPT_FILENAME token as illustrated by the
    subsequent examples.

    .. note:: Make sure options for the script are provided such that it will
    rewrite the input file in place. For example, when running
    autopep8, the --in-place option should be provided::

    text
    [post_write_hooks]
      hooks = autopep8
      autopep8.type = console_scripts
      autopep8.entrypoint = autopep8
      autopep8.options = --in-place REVISION_SCRIPT_FILENAME
  • cwd - optional working directory from which the code processing tool is run.

When running alembic revision -m "rev1", we will now see the black
tool's output as well::

$ alembic revision -m "rev1"
Generating /path/to/project/versions/481b13bc369a_rev1.py ... done
Running post write hook "black" ...
reformatted /path/to/project/versions/481b13bc369a_rev1.py
All done! โœจ ๐Ÿฐ โœจ
1 file reformatted.
done

Hooks may also be specified as a list of names, which correspond to hook
runners that will run sequentially. As an example, we can also run the
zimports <https://pypi.org/project/zimports/>_ import rewriting tool (written
by Alembic's author) subsequent to running the black tool. The
alembic.ini configuration would be as follows::

text
[post_write_hooks]

# format using "black", then "zimports"
hooks=black, zimports

black.type = console_scripts
black.entrypoint = black
black.options = -l 79 REVISION_SCRIPT_FILENAME

zimports.type = console_scripts
zimports.entrypoint = zimports
zimports.options = --style google REVISION_SCRIPT_FILENAME

The equivalent pyproject.toml configuration would be::

text
# format using "black", then "zimports"

[[tool.alembic.post_write_hooks]]
name = "black"
type="console_scripts"
entrypoint = "black"
options = "-l 79 REVISION_SCRIPT_FILENAME"

[[tool.alembic.post_write_hooks]]
name = "zimports"
type="console_scripts"
entrypoint = "zimports"
options = "--style google REVISION_SCRIPT_FILENAME"

When using the above configuration, a newly generated revision file will
be processed first by the "black" tool, then by the "zimports" tool.

Alternatively, one can run pre-commit itself as follows::

[post_write_hooks]

hooks = pre-commit

pre-commit.type = console_scripts
pre-commit.entrypoint = pre-commit
pre-commit.options = run --files REVISION_SCRIPT_FILENAME
pre-commit.cwd = %(here)s

(The last line helps to ensure that the .pre-commit-config.yaml file
will always be found, regardless of from where the hook was called.)

.. _post_write_hooks_custom:

Writing Custom Hooks as Python Functions
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The previous section illustrated how to run command-line code formatters,
through the use of a post write hook provided by Alembic known as
console_scripts. This hook is in fact a Python function that is registered
under that name using a registration function that may be used to register
other types of hooks as well.

To illustrate, we will use the example of a short Python function that wants
to rewrite the generated code to use tabs instead of four spaces. For simplicity,
we will illustrate how this function can be present directly in the env.py
file. The function is declared and registered using the :func:.write_hooks.register
decorator::

text
from alembic.script import write_hooks
import re

@write_hooks.register("spaces_to_tabs")
def convert_spaces_to_tabs(filename, options):
    lines = []
    with open(filename) as file_:
        for line in file_:
            lines.append(
                re.sub(
                    r"^(    )+",
                    lambda m: "\t" * (len(m.group(1)) // 4),
                    line
                )
            )
    with open(filename, "w") as to_write:
        to_write.write("".join(lines))

Our new "spaces_to_tabs" hook can be configured in alembic.ini as follows::

[alembic]

...

ensure the revision command loads env.py

revision_environment = true

[post_write_hooks]

hooks = spaces_to_tabs

spaces_to_tabs.type = spaces_to_tabs

When alembic revision is run, the env.py file will be loaded in all
cases, the custom "spaces_to_tabs" function will be registered and it will then
be run against the newly generated file path::

$ alembic revision -m "rev1"
Generating /path/to/project/versions/481b13bc369a_rev1.py ... done
Running post write hook "spaces_to_tabs" ...
done

.. _alembic_check:

Running Alembic Check to test for new upgrade operations

When developing code it's useful to know if a set of code changes has made any
net change to the database model, such that new revisions would need to be
generated. To automate this, Alembic provides the alembic check command.
This command will run through the same process as
alembic revision --autogenerate, up until the point where revision files
would be generated, however does not generate any new files. Instead, it
returns an error code plus a message if it is detected that new operations
would be rendered into a new revision, or if not, returns a success code plus a
message. When alembic check returns a success code, this is an indication
that the alembic revision --autogenerate command would produce only empty
migrations, and does not need to be run.

alembic check can be worked into CI systems and on-commit schemes to ensure
that incoming code does not warrant new revisions to be generated. In
the example below, a check that detects new operations is illustrated::

text
$ alembic check
FAILED: New upgrade operations detected: [
  ('add_column', None, 'my_table', Column('data', String(), table=<my_table>)),
  ('add_column', None, 'my_table', Column('newcol', Integer(), table=<my_table>))]

by contrast, when no new operations are detected::

text
$ alembic check
No new upgrade operations detected.

.. versionadded:: 1.9.0

.. note:: The alembic check command uses the same model comparison process
as the alembic revision --autogenerate process. This means parameters
such as :paramref:.EnvironmentContext.configure.compare_type
and :paramref:.EnvironmentContext.configure.compare_server_default
are in play as usual, as well as that limitations in autogenerate
detection are the same when running alembic check.


Build Batch (`docs/build/batch.rst`)

.. _batch_migrations:

Running "Batch" Migrations for SQLite and Other Databases

The SQLite database presents a challenge to migration tools
in that it has almost no support for the ALTER statement which
relational schema migrations rely upon. The rationale for this stems from
philosophical and architectural concerns within SQLite, and they are unlikely
to be changed.

Migration tools are instead expected to produce copies of SQLite tables that
correspond to the new structure, transfer the data from the existing
table to the new one, then drop the old table. For our purposes here
we'll call this "move and copy" workflow, and in order to accommodate it
in a way that is reasonably predictable, while also remaining compatible
with other databases, Alembic provides the batch operations context.

Within this context, a relational table is named, and then a series of
mutation operations to that table alone are specified within
the block. When the context is complete, a process begins whereby the
"move and copy" procedure begins; the existing table structure is reflected
from the database, a new version of this table is created with the given
changes, data is copied from the
old table to the new table using "INSERT from SELECT", and finally the old
table is dropped and the new one renamed to the original name.

The :meth:.Operations.batch_alter_table method provides the gateway to this
process::

text
with op.batch_alter_table("some_table") as batch_op:
    batch_op.add_column(Column('foo', Integer))
    batch_op.drop_column('bar')

When the above directives are invoked within a migration script, on a
SQLite backend we would see SQL like:

.. sourcecode:: sql

text
CREATE TABLE _alembic_batch_temp (
  id INTEGER NOT NULL,
  foo INTEGER,
  PRIMARY KEY (id)
);
INSERT INTO _alembic_batch_temp (id) SELECT some_table.id FROM some_table;
DROP TABLE some_table;
ALTER TABLE _alembic_batch_temp RENAME TO some_table;

On other backends, we'd see the usual ALTER statements done as though
there were no batch directive - the batch context by default only does
the "move and copy" process if SQLite is in use, and if there are
migration directives other than :meth:.Operations.add_column present,
which is the one kind of column-level ALTER statement that SQLite supports.
:meth:.Operations.batch_alter_table can be configured
to run "move and copy" unconditionally in all cases, including on databases
other than SQLite; more on this is below.

.. _batch_controlling_table_reflection:

Controlling Table Reflection

The :class:~sqlalchemy.schema.Table object that is reflected when
"move and copy" proceeds is performed using the standard autoload=True
approach. This call can be affected using the
:paramref:~.Operations.batch_alter_table.reflect_args and
:paramref:~.Operations.batch_alter_table.reflect_kwargs arguments.
For example, to override a :class:~sqlalchemy.schema.Column within
the reflection process such that a :class:~sqlalchemy.types.Boolean
object is reflected with the create_constraint flag set to False::

text
with self.op.batch_alter_table(
    "bar",
    reflect_args=[Column('flag', Boolean(create_constraint=False))]
) as batch_op:
    batch_op.alter_column(
        'flag', new_column_name='bflag', existing_type=Boolean)

Another use case, add a listener to the :class:~sqlalchemy.schema.Table
as it is reflected so that special logic can be applied to columns or
types, using the :meth:~sqlalchemy.events.DDLEvents.column_reflect event::

text
def listen_for_reflect(inspector, table, column_info):
    "correct an ENUM type"
    if column_info['name'] == 'my_enum':
        column_info['type'] = Enum('a', 'b', 'c')

with self.op.batch_alter_table(
    "bar",
    reflect_kwargs=dict(
        listeners=[
            ('column_reflect', listen_for_reflect)
        ]
    )
) as batch_op:
    batch_op.alter_column(
        'flag', new_column_name='bflag', existing_type=Boolean)

The reflection process may also be bypassed entirely by sending a
pre-fabricated :class:~sqlalchemy.schema.Table object; see
:ref:batch_offline_mode for an example.

.. _sqlite_batch_constraints:

Dealing with Constraints

There are a variety of issues when using "batch" mode with constraints,
such as FOREIGN KEY, CHECK and UNIQUE constraints. This section
will attempt to detail many of these scenarios.

.. _dropping_sqlite_foreign_keys:

Dropping Unnamed or Named Foreign Key Constraints
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

SQLite, unlike any other database, allows constraints to exist in the
database that have no identifying name. On all other backends, the
target database will always generate some kind of name, if one is not
given.

The challenge this represents is that an unnamed constraint can't
by itself be targeted by the :meth:.BatchOperations.drop_constraint method.
An unnamed FOREIGN KEY constraint is implicit whenever the
:class:~sqlalchemy.schema.ForeignKey
or :class:~sqlalchemy.schema.ForeignKeyConstraint objects are used without
passing them a name. Only on SQLite will these constraints remain entirely
unnamed when they are created on the target database; an automatically generated
name will be assigned in the case of all other database backends.

Within the scope of batch mode, this presents the issue that the
:meth:.BatchOperations.drop_constraint method requires a constraint name
in order to target the correct constraint.

In order to overcome this, the :meth:.Operations.batch_alter_table method supports a
:paramref:~.Operations.batch_alter_table.naming_convention argument, so that
all reflected constraints, including foreign keys that are unnamed may be
given a name, as described in :ref:autogen_naming_conventions.
Usage is as follows::

text
naming_convention = {
    "fk":
    "fk_%(table_name)s_%(column_0_name)s_%(referred_table_name)s",
}
with self.op.batch_alter_table(
        "bar", naming_convention=naming_convention) as batch_op:
    batch_op.drop_constraint(
        "fk_bar_foo_id_foo", type_="foreignkey")

Note that the naming convention feature requires at least
SQLAlchemy 0.9.4 for support.

.. _batch_schematype_constraints:

Changing the Type of Boolean, Enum and other implicit CHECK datatypes
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The SQLAlchemy types :class:~sqlalchemy.types.Boolean and
:class:~sqlalchemy.types.Enum are part of a category of types known as
"schema" types; this style of type creates other structures along with the
type itself, most commonly (but not always) a CHECK constraint.

Alembic handles dropping and creating the CHECK constraints here automatically,
including in the case of batch mode. When changing the type of an existing
column, what's necessary is that the existing type be specified fully::

with self.op.batch_alter_table("some_table") as batch_op:
batch_op.alter_column(
'q', type_=Integer,
existing_type=Boolean(create_constraint=True, constraint_name="ck1"))

When dropping a column that includes a named CHECK constraint, as of Alembic
1.7 this named constraint must also be provided using a similar form, as there
is no ability for Alembic to otherwise link this reflected CHECK constraint as
belonging to a particular column::

text
with self.op.batch_alter_table("some_table") as batch_op:
    batch_op.drop_column(
        'q',
        existing_type=Boolean(create_constraint=True, constraint_name="ck1"))
    )

.. versionchanged:: 1.7 The :meth:.BatchOperations.drop_column operation can
accept an existing_type directive where a "schema type" such as
:class:~sqlalchemy.types.Boolean and :class:~sqlalchemy.types.Enum may
be specified such that an associated named constraint can be removed.

.. _batch_check_constraints:

Including CHECK constraints
^^^^^^^^^^^^^^^^^^^^^^^^^^^

As of Alembic 1.7, named CHECK constraints are automatically included
in batch mode, as modern SQLAlchemy versions are capable of reflecting these
constraints like any other constraint.

Note that when dropping or renaming a column that is mentioned in a named
CHECK constraint, this CHECK constraint must be explicitly dropped first,
as Alembic has no means of linking a reflected CHECK constraint to that
column. Supposing column q of some_table were mentioned in a CHECK
constraint named ck1. In order to drop this column, we have to drop
the check constraint also::

text
with self.op.batch_alter_table("some_table") as batch_op:
    batch_op.drop_constraint("ck1", type_="check")
    batch_op.drop_column('q')

.. versionchanged:: 1.7 Named CHECK constraints participate in batch mode
in the same way as any other kind of constraint. This requires that column
drops or renames now include explicit directives to drop an existing named
constraint which refers to this column, as it will otherwise not be
automatically detected as being associated with that particular column.

Unnamed CHECK constraints continue to be omitted from the table
recreate operation; up until version 1.20, this omission was silent.
As of 1.20, it emits a warning.

For unnamed CHECK constraints, these are still not automatically included
as part of the batch process. Note that this limitation includes the CHECK
constraints generated by the
:class:~sqlalchemy.types.Boolean or :class:~sqlalchemy.types.Enum
datatypes, which up through SQLAlchemy 1.3 would generate CHECK constraints
automatically and cannot be tracked to the reflected table, assuming they are
generated in an unnamed way.

Unnamed constraints can be stated explicitly if they are to be included in the
recreated table::

text
with op.batch_alter_table("some_table", table_args=[
      CheckConstraint('x > 5')
  ]) as batch_op:
    batch_op.add_column(Column('foo', Integer))
    batch_op.drop_column('bar')

The above step needs only be taken for CHECK constraints that are explicitly stated
as part of the table definition.

.. versionadded:: 1.20 A warning is emitted when an unnamed CHECK constraint
on a reflected table is omitted from the recreate, and no
CheckConstraint objects are passed to the
:paramref:~.Operations.batch_alter_table.table_args collection.

Restating the constraint as above is needed for each recreate of the table.
The longer term remedy is for the CHECK constraints in the model to be given
names, either explicitly or via a naming convention on the
:class:~sqlalchemy.schema.MetaData; once a recreate has emitted the
constraints with names, subsequent reflection of the table carries those names
and the constraints participate in batch mode like any other.

For CHECK constraints that are generated by datatypes such as
:class:~sqlalchemy.types.Boolean or :class:~sqlalchemy.types.Enum, the type
objects themselves must be named in order for their CHECK constraints to be
included in the batch process. Boolean and Enum datatypes that do not
have the .name attribute set will not have CHECK constraints
regenerated. This name can be set by specifying the .name parameter
or by using a named Python Enum object as the source of enumeration.

Dealing with Referencing Foreign Keys
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

It is important to note that batch table operations do not work with
foreign keys that enforce referential integrity. This because the
target table is dropped; if foreign keys refer to it, this will raise
an error. On SQLite, whether or not foreign keys actually enforce is
controlled by the PRAGMA FOREIGN KEYS pragma; this pragma, if in use,
must be disabled when the workflow mode proceeds. When the operation is
complete, the batch-migrated table will have the same name
that it started with, so those referring foreign keys will again
refer to this table.

A special case is dealing with self-referring foreign keys. Here,
Alembic takes a special step of recreating the self-referring foreign key
as referring to the original table name, rather than at the "temp" table,
so that like in the case of other foreign key constraints, when the table
is renamed to its original name, the foreign key
again references the correct table. This operation only works when
referential integrity is disabled, consistent with the same requirement
for referring foreign keys from other tables.

When SQLite's PRAGMA FOREIGN KEYS mode is turned on, it does provide
the service that foreign key constraints, including self-referential, will
automatically be modified to point to their table across table renames,
however this mode prevents the target table from being dropped as is required
by a batch migration. Therefore it may be necessary to manipulate the
PRAGMA FOREIGN KEYS setting if a migration seeks to rename a table vs.
batch migrate it.

.. _batch_offline_mode:

Working in Offline Mode

In the preceding sections, we've seen how much of an emphasis the
"move and copy" process has on using reflection in order to know the
structure of the table that is to be copied. This means that in the typical
case, "online" mode, where a live database connection is present so that
:meth:.Operations.batch_alter_table can reflect the table from the
database, is required; the --sql flag cannot be used without extra
steps.

To support offline mode, the system must work without table reflection
present, which means the full table as it intends to be created must be
passed to :meth:.Operations.batch_alter_table using
:paramref:~.Operations.batch_alter_table.copy_from::

text
meta = MetaData()
some_table = Table(
    'some_table', meta,
    Column('id', Integer, primary_key=True),
    Column('bar', String(50))
)

with op.batch_alter_table("some_table", copy_from=some_table) as batch_op:
    batch_op.add_column(Column('foo', Integer))
    batch_op.drop_column('bar')

The above use pattern is pretty tedious and quite far off from Alembic's
preferred style of working; however, if one needs to do SQLite-compatible
"move and copy" migrations and need them to generate flat SQL files in
"offline" mode, there's not much alternative.

Batch mode with Autogenerate

The syntax of batch mode is essentially that :meth:.Operations.batch_alter_table
is used to enter a batch block, and the returned :class:.BatchOperations context
works just like the regular :class:.Operations context, except that
the "table name" and "schema name" arguments are omitted.

To support rendering of migration commands in batch mode for autogenerate,
configure the :paramref:.EnvironmentContext.configure.render_as_batch
flag in env.py::

text
context.configure(
    connection=connection,
    target_metadata=target_metadata,
    render_as_batch=True
)

Autogenerate will now generate along the lines of::

text
def upgrade():
    ### commands auto generated by Alembic - please adjust! ###
    with op.batch_alter_table('address', schema=None) as batch_op:
        batch_op.add_column(sa.Column('street', sa.String(length=50), nullable=True))

This mode is safe to use in all cases, as the :meth:.Operations.batch_alter_table
directive by default only takes place for SQLite; other backends will
behave just as they normally do in the absence of the batch directives.

Note that autogenerate support does not include "offline" mode, where
the :paramref:.Operations.batch_alter_table.copy_from parameter is used.
The table definition here would need to be entered into migration files
manually if this is needed.

Batch mode with databases other than SQLite

There's an odd use case some shops have, where the "move and copy" style
of migration is useful in some cases for databases that do already support
ALTER. There's some cases where an ALTER operation may block access to the
table for a long time, which might not be acceptable. "move and copy" can
be made to work on other backends, though with a few extra caveats.

The batch mode directive will run the "recreate" system regardless of
backend if the flag recreate='always' is passed::

text
with op.batch_alter_table("some_table", recreate='always') as batch_op:
    batch_op.add_column(Column('foo', Integer))

The issues that arise in this mode are mostly to do with constraints.
Databases such as Postgresql and MySQL with InnoDB will enforce referential
integrity (e.g. via foreign keys) in all cases. Unlike SQLite, it's not
as simple to turn off referential integrity across the board (nor would it
be desirable). Since a new table is replacing the old one, existing
foreign key constraints which refer to the target table will need to be
unconditionally dropped before the batch operation, and re-created to refer
to the new table afterwards. Batch mode currently does not provide any
automation for this.

The Postgresql database and possibly others also have the behavior such
that when the new table is created, a naming conflict occurs with the
named constraints of the new table, in that they match those of the old
table, and on Postgresql, these names need to be unique across all tables.
The Postgresql dialect will therefore emit a "DROP CONSTRAINT" directive
for all constraints on the old table before the new one is created; this is
"safe" in case of a failed operation because Postgresql also supports
transactional DDL.

Note that also as is the case with SQLite, CHECK constraints need to be
moved over between old and new table manually using the
:paramref:.Operations.batch_alter_table.table_args parameter.


Build Branches (`docs/build/branches.rst`)

.. _branches:

Working with Branches

A branch describes a point in a migration stream when two or more
versions refer to the same parent migration as their ancestor. Branches
occur naturally when two divergent source trees, both containing Alembic
revision files created independently within those source trees, are merged
together into one. When this occurs, the challenge of a branch is to merge the
branches into a single series of changes, so that databases established
from either source tree individually can be upgraded to reference the merged
result equally. Another scenario where branches are present are when we create them
directly; either at some point in the migration stream we'd like different
series of migrations to be managed independently (e.g. we create a tree),
or we'd like separate migration streams for different features starting
at the root (e.g. a forest). We'll illustrate all of these cases, starting
with the most common which is a source-merge-originated branch that we'll
merge.

Starting with the "account table" example we began in :ref:create_migration,
assume we have our basemost version 1975ea83b712, which leads into
the second revision ae1027a6acf, and the migration files for these
two revisions are checked into our source repository.
Consider if we merged into our source repository another code branch which contained
a revision for another table called shopping_cart. This revision was made
against our first Alembic revision, the one that generated account. After
loading the second source tree in, a new file
27c6a30d7c24_add_shopping_cart_table.py exists within our versions directory.
Both it, as well as ae1027a6acf_add_a_column.py, reference
1975ea83b712_add_account_table.py as the "downgrade" revision. To illustrate::

text
# main source tree:
1975ea83b712 (create account table) -> ae1027a6acf (add a column)

# branched source tree
1975ea83b712 (create account table) -> 27c6a30d7c24 (add shopping cart table)

Above, we can see 1975ea83b712 is our branch point; two distinct versions
both refer to it as its parent. The Alembic command branches illustrates
this fact::

$ alembic branches --verbose
Rev: 1975ea83b712 (branchpoint)
Parent:
Branches into: 27c6a30d7c24, ae1027a6acf
Path: foo/versions/1975ea83b712_add_account_table.py

text
create account table

  Revision ID: 1975ea83b712
  Revises:
  Create Date: 2014-11-20 13:02:46.257104

           -> 27c6a30d7c24 (head), add shopping cart table
           -> ae1027a6acf (head), add a column

History shows it too, illustrating two head entries as well
as a branchpoint::

text
$ alembic history
1975ea83b712 -> 27c6a30d7c24 (head), add shopping cart table
1975ea83b712 -> ae1027a6acf (head), add a column
 -> 1975ea83b712 (branchpoint), create account table

We can get a view of just the current heads using alembic heads::

text
$ alembic heads --verbose
Rev: 27c6a30d7c24 (head)
Parent: 1975ea83b712
Path: foo/versions/27c6a30d7c24_add_shopping_cart_table.py

    add shopping cart table

    Revision ID: 27c6a30d7c24
    Revises: 1975ea83b712
    Create Date: 2014-11-20 13:03:11.436407

Rev: ae1027a6acf (head)
Parent: 1975ea83b712
Path: foo/versions/ae1027a6acf_add_a_column.py

    add a column

    Revision ID: ae1027a6acf
    Revises: 1975ea83b712
    Create Date: 2014-11-20 13:02:54.849677

If we try to run an upgrade to the usual end target of head, Alembic no
longer considers this to be an unambiguous command. As we have more than
one head, the upgrade command wants us to provide more information::

text
$ alembic upgrade head
  FAILED: Multiple head revisions are present for given argument 'head'; please specify a specific
  target revision, '@head' to narrow to a specific head, or 'heads' for all heads

The upgrade command gives us quite a few options in which we can proceed
with our upgrade, either giving it information on which head we'd like to upgrade
towards, or alternatively stating that we'd like all heads to be upgraded
towards at once. However, in the typical case of two source trees being
merged, we will want to pursue a third option, which is that we can merge these
branches.

Merging Branches

An Alembic merge is a migration file that joins two or
more "head" files together. If the two branches we have right now can
be said to be a "tree" structure, introducing this merge file will
turn it into a "diamond" structure::

text
-- ae1027a6acf -->
                           /                   \
 --> 1975ea83b712 -->                      --> mergepoint
                           \                   /
                            -- 27c6a30d7c24 -->

We create the merge file using alembic merge; with this command, we can
pass to it an argument such as heads, meaning we'd like to merge all
heads. Or, we can pass it individual revision numbers sequentially::

text
$ alembic merge -m "merge ae1 and 27c" ae1027 27c6a
  Generating /path/to/foo/versions/53fffde5ad5_merge_ae1_and_27c.py ... done

Looking inside the new file, we see it as a regular migration file, with
the only new twist is that down_revision points to both revisions::

text
"""merge ae1 and 27c

Revision ID: 53fffde5ad5
Revises: ae1027a6acf, 27c6a30d7c24
Create Date: 2014-11-20 13:31:50.811663

"""

# revision identifiers, used by Alembic.
revision = '53fffde5ad5'
down_revision = ('ae1027a6acf', '27c6a30d7c24')
branch_labels = None

from alembic import op
import sqlalchemy as sa


def upgrade():
    pass


def downgrade():
    pass

This file is a regular migration file, and if we wish to, we may place
:class:.Operations directives into the upgrade() and downgrade()
functions like any other migration file. Though it is probably best to limit
the instructions placed here only to those that deal with any kind of
reconciliation that is needed between the two merged branches, if any.

The heads command now illustrates that the multiple heads in our
versions/ directory have been resolved into our new head::

text
$ alembic heads --verbose
Rev: 53fffde5ad5 (head) (mergepoint)
Merges: ae1027a6acf, 27c6a30d7c24
Path: foo/versions/53fffde5ad5_merge_ae1_and_27c.py

    merge ae1 and 27c

    Revision ID: 53fffde5ad5
    Revises: ae1027a6acf, 27c6a30d7c24
    Create Date: 2014-11-20 13:31:50.811663

History shows a similar result, as the mergepoint becomes our head::

text
$ alembic history
ae1027a6acf, 27c6a30d7c24 -> 53fffde5ad5 (head) (mergepoint), merge ae1 and 27c
1975ea83b712 -> ae1027a6acf, add a column
1975ea83b712 -> 27c6a30d7c24, add shopping cart table
 -> 1975ea83b712 (branchpoint), create account table

With a single head target, a generic upgrade can proceed::

text
$ alembic upgrade head
INFO  [alembic.migration] Context impl PostgresqlImpl.
INFO  [alembic.migration] Will assume transactional DDL.
INFO  [alembic.migration] Running upgrade  -> 1975ea83b712, create account table
INFO  [alembic.migration] Running upgrade 1975ea83b712 -> 27c6a30d7c24, add shopping cart table
INFO  [alembic.migration] Running upgrade 1975ea83b712 -> ae1027a6acf, add a column
INFO  [alembic.migration] Running upgrade ae1027a6acf, 27c6a30d7c24 -> 53fffde5ad5, merge ae1 and 27c

.. topic:: merge mechanics

The upgrade process traverses through all of our migration files using
a topological sorting algorithm, treating the list of migration
files not as a linked list, but as a directed acyclic graph. The starting
points of this traversal are the current heads within our database,
and the end point is the "head" revision or revisions specified.

When a migration proceeds across a point at which there are multiple heads,
the alembic_version table will at that point store multiple rows,
one for each head. Our migration process above will emit SQL against
alembic_version along these lines:

text
.. sourcecode:: sql

  -- Running upgrade  -> 1975ea83b712, create account table
  INSERT INTO alembic_version (version_num) VALUES ('1975ea83b712')

  -- Running upgrade 1975ea83b712 -> 27c6a30d7c24, add shopping cart table
  UPDATE alembic_version SET version_num='27c6a30d7c24' WHERE alembic_version.version_num = '1975ea83b712'

  -- Running upgrade 1975ea83b712 -> ae1027a6acf, add a column
  INSERT INTO alembic_version (version_num) VALUES ('ae1027a6acf')

  -- Running upgrade ae1027a6acf, 27c6a30d7c24 -> 53fffde5ad5, merge ae1 and 27c
  DELETE FROM alembic_version WHERE alembic_version.version_num = 'ae1027a6acf'
  UPDATE alembic_version SET version_num='53fffde5ad5' WHERE alembic_version.version_num = '27c6a30d7c24'

At the point at which both 27c6a30d7c24 and ae1027a6acf exist within our
database, both values are present in alembic_version, which now has
two rows. If we upgrade to these two versions alone, then stop and
run alembic current, we will see this::

text
$ alembic current --verbose
  Current revision(s) for postgresql://scott:XXXXX@localhost/test:
  Rev: ae1027a6acf
  Parent: 1975ea83b712
  Path: foo/versions/ae1027a6acf_add_a_column.py

      add a column

      Revision ID: ae1027a6acf
      Revises: 1975ea83b712
      Create Date: 2014-11-20 13:02:54.849677

  Rev: 27c6a30d7c24
  Parent: 1975ea83b712
  Path: foo/versions/27c6a30d7c24_add_shopping_cart_table.py

      add shopping cart table

      Revision ID: 27c6a30d7c24
      Revises: 1975ea83b712
      Create Date: 2014-11-20 13:03:11.436407

A key advantage to the merge process is that it will
run equally well on databases that were present on version ae1027a6acf
alone, versus databases that were present on version 27c6a30d7c24 alone;
whichever version was not yet applied, will be applied before the merge point
can be crossed. This brings forth a way of thinking about a merge file,
as well as about any Alembic revision file. As they are considered to
be "nodes" within a set that is subject to topological sorting, each
"node" is a point that cannot be crossed until all of its dependencies
are satisfied.

Prior to Alembic's support of merge points, the use case of databases
sitting on different heads was basically impossible to reconcile; having
to manually splice the head files together invariably meant that one migration
would occur before the other, thus being incompatible with databases that
were present on the other migration.

Working with Explicit Branches

The alembic upgrade command hinted at other options besides merging when
dealing with multiple heads. Let's back up and assume we're back where
we have as our heads just ae1027a6acf and 27c6a30d7c24::

text
$ alembic heads
27c6a30d7c24
ae1027a6acf

Earlier, when we did alembic upgrade head, it gave us an error which
suggested please specify a specific target revision, '@head' to narrow to a specific head, or 'heads' for all heads in order to proceed
without merging. Let's cover those cases.

Referring to all heads at once
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The heads identifier is a lot like head, except it explicitly refers
to all heads at once. That is, it's like telling Alembic to do the operation
for both ae1027a6acf and 27c6a30d7c24 simultaneously. If we started
from a fresh database and ran upgrade heads we'd see::

text
$ alembic upgrade heads
INFO  [alembic.migration] Context impl PostgresqlImpl.
INFO  [alembic.migration] Will assume transactional DDL.
INFO  [alembic.migration] Running upgrade  -> 1975ea83b712, create account table
INFO  [alembic.migration] Running upgrade 1975ea83b712 -> ae1027a6acf, add a column
INFO  [alembic.migration] Running upgrade 1975ea83b712 -> 27c6a30d7c24, add shopping cart table

Since we've upgraded to heads, and we do in fact have more than one head,
that means these two distinct heads are now in our alembic_version table.
We can see this if we run alembic current::

text
$ alembic current
ae1027a6acf (head)
27c6a30d7c24 (head)

That means there's two rows in alembic_version right now. If we downgrade
one step at a time, Alembic will delete from the alembic_version table
each branch that's closed out, until only one branch remains; then it will
continue updating the single value down to the previous versions::

text
$ alembic downgrade -1
INFO  [alembic.migration] Running downgrade ae1027a6acf -> 1975ea83b712, add a column

$ alembic current
27c6a30d7c24 (head)

$ alembic downgrade -1
INFO  [alembic.migration] Running downgrade 27c6a30d7c24 -> 1975ea83b712, add shopping cart table

$ alembic current
1975ea83b712 (branchpoint)

$ alembic downgrade -1
INFO  [alembic.migration] Running downgrade 1975ea83b712 -> , create account table

$ alembic current

Referring to a Specific Version
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

We can pass a specific version number to upgrade. Alembic will ensure that
all revisions upon which this version depends are invoked, and nothing more.
So if we upgrade either to 27c6a30d7c24 or ae1027a6acf specifically,
it guarantees that 1975ea83b712 will have been applied, but not that
any "sibling" versions are applied::

text
$ alembic upgrade 27c6a
INFO  [alembic.migration] Running upgrade  -> 1975ea83b712, create account table
INFO  [alembic.migration] Running upgrade 1975ea83b712 -> 27c6a30d7c24, add shopping cart table

With 1975ea83b712 and 27c6a30d7c24 applied, ae1027a6acf is just
a single additional step::

text
$ alembic upgrade ae102
INFO  [alembic.migration] Running upgrade 1975ea83b712 -> ae1027a6acf, add a column

Working with Branch Labels
^^^^^^^^^^^^^^^^^^^^^^^^^^

To satisfy the use case where an environment has long-lived branches, especially
independent branches as will be discussed in the next section, Alembic supports
the concept of branch labels. These are string values that are present
within the migration file, using the new identifier branch_labels.
For example, if we want to refer to the "shopping cart" branch using the name
"shoppingcart", we can add that name to our file
27c6a30d7c24_add_shopping_cart_table.py::

text
"""add shopping cart table

"""

# revision identifiers, used by Alembic.
revision = '27c6a30d7c24'
down_revision = '1975ea83b712'
branch_labels = ('shoppingcart',)

# ...

The branch_labels attribute refers to a string name, or a tuple
of names, which will now apply to this revision, all descendants of this
revision, as well as all ancestors of this revision up until the preceding
branch point, in this case 1975ea83b712. We can see the shoppingcart
label applied to this revision::

text
$ alembic history
1975ea83b712 -> 27c6a30d7c24 (shoppingcart) (head), add shopping cart table
1975ea83b712 -> ae1027a6acf (head), add a column
 -> 1975ea83b712 (branchpoint), create account table

With the label applied, the name shoppingcart now serves as an alias
for the 27c6a30d7c24 revision specifically. We can illustrate this
by showing it with alembic show::

text
$ alembic show shoppingcart
Rev: 27c6a30d7c24 (head)
Parent: 1975ea83b712
Branch names: shoppingcart
Path: foo/versions/27c6a30d7c24_add_shopping_cart_table.py

    add shopping cart table

    Revision ID: 27c6a30d7c24
    Revises: 1975ea83b712
    Create Date: 2014-11-20 13:03:11.436407

However, when using branch labels, we usually want to use them using a syntax
known as "branch at" syntax; this syntax allows us to state that we want to
use a specific revision, let's say a "head" revision, in terms of a specific
branch. While normally, we can't refer to alembic upgrade head when
there's multiple heads, we can refer to this head specifically using
shoppingcart@head syntax::

text
$ alembic upgrade shoppingcart@head
INFO  [alembic.migration] Running upgrade 1975ea83b712 -> 27c6a30d7c24, add shopping cart table

The shoppingcart@head syntax becomes important to us if we wish to
add new migration files to our versions directory while maintaining multiple
branches. Just like the upgrade command, if we attempted to add a new
revision file to our multiple-heads layout without a specific parent revision,
we'd get a familiar error::

text
$ alembic revision -m "add a shopping cart column"
  FAILED: Multiple heads are present; please specify the head revision on
  which the new revision should be based, or perform a merge.

The alembic revision command is pretty clear in what we need to do;
to add our new revision specifically to the shoppingcart branch,
we use the --head argument, either with the specific revision identifier
27c6a30d7c24, or more generically using our branchname shoppingcart@head::

text
$ alembic revision -m "add a shopping cart column"  --head shoppingcart@head
  Generating /path/to/foo/versions/d747a8a8879_add_a_shopping_cart_column.py ... done

alembic history shows both files now part of the shoppingcart branch::

text
$ alembic history
1975ea83b712 -> ae1027a6acf (head), add a column
27c6a30d7c24 -> d747a8a8879 (shoppingcart) (head), add a shopping cart column
1975ea83b712 -> 27c6a30d7c24 (shoppingcart), add shopping cart table
 -> 1975ea83b712 (branchpoint), create account table

We can limit our history operation just to this branch as well::

text
$ alembic history -r shoppingcart:
27c6a30d7c24 -> d747a8a8879 (shoppingcart) (head), add a shopping cart column
1975ea83b712 -> 27c6a30d7c24 (shoppingcart), add shopping cart table

If we want to illustrate the path of shoppingcart all the way from the
base, we can do that as follows::

text
$ alembic history -r :shoppingcart@head
27c6a30d7c24 -> d747a8a8879 (shoppingcart) (head), add a shopping cart column
1975ea83b712 -> 27c6a30d7c24 (shoppingcart), add shopping cart table
 -> 1975ea83b712 (branchpoint), create account table

We can run this operation from the "base" side as well, but we get a different
result::

text
$ alembic history -r shoppingcart@base:
1975ea83b712 -> ae1027a6acf (head), add a column
27c6a30d7c24 -> d747a8a8879 (shoppingcart) (head), add a shopping cart column
1975ea83b712 -> 27c6a30d7c24 (shoppingcart), add shopping cart table
 -> 1975ea83b712 (branchpoint), create account table

When we list from shoppingcart@base without an endpoint, it's really shorthand
for -r shoppingcart@base:heads, e.g. all heads, and since shoppingcart@base
is the same "base" shared by the ae1027a6acf revision, we get that
revision in our listing as well. The @base syntax can be
useful when we are dealing with individual bases, as we'll see in the next
section.

The @head format can also be used with revision numbers
instead of branch names, though this is less convenient. If we wanted to
add a new revision to our branch that includes the un-labeled ae1027a6acf,
if this weren't a head already, we could ask for the "head of the branch
that includes ae1027a6acf" as follows::

text
$ alembic revision -m "add another account column" --head ae10@head
  Generating /path/to/foo/versions/55af2cb1c267_add_another_account_column.py ... done

More Label Syntaxes
^^^^^^^^^^^^^^^^^^^

The heads symbol can be combined with a branch label, in the case that
your labeled branch itself breaks off into multiple branches::

text
$ alembic upgrade shoppingcart@heads

Relative identifiers, as introduced in :ref:relative_migrations,
work with labels too. For example, upgrading to shoppingcart@+2
means to upgrade from current heads on "shoppingcart" upwards two revisions::

text
$ alembic upgrade shoppingcart@+2

This kind of thing works from history as well::

text
$ alembic history -r current:shoppingcart@+2

The newer relnum+delta format can be combined as well, for example
if we wanted to list along shoppingcart up until two revisions
before the head::

text
$ alembic history -r :shoppingcart@head-2

.. _multiple_bases:

Working with Multiple Bases

.. note:: The multiple base feature is intended to allow for multiple Alembic
versioning lineages which share the same alembic_version table. This is
so that individual revisions within the lineages can have cross-dependencies
on each other. For the simpler case where one project has multiple,
completely independent revision lineages that refer to separate
alembic_version tables, see the example in :ref:multiple_environments.

We've seen in the previous section that alembic upgrade is fine
if we have multiple heads, alembic revision allows us to tell it which
"head" we'd like to associate our new revision file with, and branch labels
allow us to assign names to branches that we can use in subsequent commands.
Let's put all these together and refer to a new "base", that is, a whole
new tree of revision files that will be semi-independent of the account/shopping
cart revisions we've been working with. This new tree will deal with
database tables involving "networking".

.. _multiple_version_directories:

Setting up Multiple Version Directories
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

While optional, it is often the case that when working with multiple bases,
we'd like different sets of version files to exist within their own directories;
typically, if an application is organized into several sub-modules, each
one would have a version directory containing migrations pertinent to
that module. So to start out, we can edit alembic.ini to refer
to multiple directories; we'll also state the current versions
directory as one of them::

A separator for the location paths needs to be defined

path_separator = os # Use os.pathsep.

version location specification; this defaults

to foo/versions. When using multiple version

directories, initial revisions must be specified with --version-path

version_locations = %(here)s/model/networking:%(here)s/alembic/versions

The new directory %(here)s/model/networking is in terms of where
the alembic.ini file is, as we are using the symbol %(here)s which
resolves to this location. When we create our first new revision
targeted at this directory,
model/networking will be created automatically if it does not
exist yet. Once we've created a revision here, the path is used automatically
when generating subsequent revision files that refer to this revision tree.

Creating a Labeled Base Revision
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

We also want our new branch to have its own name, and for that we want to
apply a branch label to the base. In order to achieve this using the
alembic revision command without editing, we need to ensure our
script.py.mako file, used
for generating new revision files, has the appropriate substitutions present.
If Alembic version 0.7.0 or greater was used to generate the original
migration environment, this is already done. However when working with an older
environment, script.py.mako needs to have this directive added, typically
underneath the down_revision directive::

text
# revision identifiers, used by Alembic.
revision = ${repr(up_revision)}
down_revision = ${repr(down_revision)}

# add this here in order to use revision with branch_label
branch_labels = ${repr(branch_labels)}

With this in place, we can create a new revision file, starting up a branch
that will deal with database tables involving networking; we specify the
--head version of base, a --branch-label of networking,
and the directory we want this first revision file to be
placed in with --version-path::

text
$ alembic revision -m "create networking branch" --head=base --branch-label=networking --version-path=model/networking
  Creating directory /path/to/foo/model/networking ... done
  Generating /path/to/foo/model/networking/3cac04ae8714_create_networking_branch.py ... done

If we ran the above command and we didn't have the newer script.py.mako
directive, we'd get this error::

FAILED: Version 3cac04ae8714 specified branch_labels networking, however
the migration file foo/model/networking/3cac04ae8714_create_networking_branch.py
does not have them; have you upgraded your script.py.mako to include the 'branch_labels'
section?

When we receive the above error, and we would like to try again, we need to
either delete the incorrectly generated file in order to run revision
again, or we can edit the 3cac04ae8714_create_networking_branch.py
directly to add the branch_labels in of our choosing.

Running with Multiple Bases
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Once we have a new, permanent (for as long as we desire it to be)
base in our system, we'll always have multiple heads present::

text
$ alembic heads
3cac04ae8714 (networking) (head)
27c6a30d7c24 (shoppingcart) (head)
ae1027a6acf (head)

When we want to add a new revision file to networking, we specify
networking@head as the --head. The appropriate version directory
is now selected automatically based on the head we choose::

text
$ alembic revision -m "add ip number table" --head=networking@head
  Generating /path/to/foo/model/networking/109ec7d132bf_add_ip_number_table.py ... done

It's important that we refer to the head using networking@head; if we
only refer to networking, that refers to only 3cac04ae8714 specifically;
if we specify this and it's not a head, alembic revision will make sure
we didn't mean to specify the head::

text
$ alembic revision -m "add DNS table" --head=networking
  FAILED: Revision 3cac04ae8714 is not a head revision; please
  specify --splice to create a new branch from this revision

As mentioned earlier, as this base is independent, we can view its history
from the base using history -r networking@base:::

text
$ alembic history -r networking@base:
109ec7d132bf -> 29f859a13ea (networking) (head), add DNS table
3cac04ae8714 -> 109ec7d132bf (networking), add ip number table
 -> 3cac04ae8714 (networking), create networking branch

At the moment, this is the same output we'd get at this point if we used
-r :networking@head. However, that will change later on as we use
additional directives.

We may now run upgrades or downgrades freely, among individual branches
(let's assume a clean database again)::

text
$ alembic upgrade networking@head
INFO  [alembic.migration] Running upgrade  -> 3cac04ae8714, create networking branch
INFO  [alembic.migration] Running upgrade 3cac04ae8714 -> 109ec7d132bf, add ip number table
INFO  [alembic.migration] Running upgrade 109ec7d132bf -> 29f859a13ea, add DNS table

or against the whole thing using heads::

text
$ alembic upgrade heads
INFO  [alembic.migration] Running upgrade  -> 1975ea83b712, create account table
INFO  [alembic.migration] Running upgrade 1975ea83b712 -> 27c6a30d7c24, add shopping cart table
INFO  [alembic.migration] Running upgrade 27c6a30d7c24 -> d747a8a8879, add a shopping cart column
INFO  [alembic.migration] Running upgrade 1975ea83b712 -> ae1027a6acf, add a column
INFO  [alembic.migration] Running upgrade ae1027a6acf -> 55af2cb1c267, add another account column

Branch Dependencies

When working with multiple roots, it is expected that these different
revision streams will need to refer to one another. For example, a new
revision in networking which needs to refer to the account
table will want to establish 55af2cb1c267, add another account column,
the last revision that
works with the account table, as a dependency. From a graph perspective,
this means nothing more that the new file will feature both
55af2cb1c267, add another account column and 29f859a13ea, add DNS table as "down" revisions,
and looks just as though we had merged these two branches together. However,
we don't want to consider these as "merged"; we want the two revision
streams to remain independent, even though a version in networking
is going to reach over into the other stream. To support this use case,
Alembic provides a directive known as depends_on, which allows
a revision file to refer to another as a "dependency", very similar to
an entry in down_revision from a graph perspective, but different
from a semantic perspective.

To use depends_on, we can specify it as part of our alembic revision
command::

text
$ alembic revision -m "add ip account table" --head=networking@head  --depends-on=55af2cb1c267
  Generating /path/to/foo/model/networking/2a95102259be_add_ip_account_table.py ... done

Within our migration file, we'll see this new directive present::

text
# revision identifiers, used by Alembic.
revision = '2a95102259be'
down_revision = '29f859a13ea'
branch_labels = None
depends_on='55af2cb1c267'

depends_on may be either a real revision number or a branch
name. When specified at the command line, a resolution from a
partial revision number will work as well. It can refer
to any number of dependent revisions as well; for example, if we were
to run the command::

text
$ alembic revision -m "add ip account table" \\
    --head=networking@head  \\
    --depends-on=55af2cb1c267 --depends-on=d747a --depends-on=fa445
  Generating /path/to/foo/model/networking/2a95102259be_add_ip_account_table.py ... done

We'd see inside the file::

text
# revision identifiers, used by Alembic.
revision = '2a95102259be'
down_revision = '29f859a13ea'
branch_labels = None
depends_on = ('55af2cb1c267', 'd747a8a8879', 'fa4456a9201')

We also can of course add or alter this value within the file manually after
it is generated, rather than using the --depends-on argument.

We can see the effect this directive has when we view the history
of the networking branch in terms of "heads", e.g., all the revisions that
are descendants::

text
$ alembic history -r :networking@head
29f859a13ea (55af2cb1c267) -> 2a95102259be (networking) (head), add ip account table
109ec7d132bf -> 29f859a13ea (networking), add DNS table
3cac04ae8714 -> 109ec7d132bf (networking), add ip number table
 -> 3cac04ae8714 (networking), create networking branch
ae1027a6acf -> 55af2cb1c267 (effective head), add another account column
1975ea83b712 -> ae1027a6acf, Add a column
 -> 1975ea83b712 (branchpoint), create account table

What we see is that the full history of the networking branch, in terms
of an "upgrade" to the "head", will include that the tree building
up 55af2cb1c267, add another account column
will be pulled in first. Interestingly, we don't see this displayed
when we display history in the other direction, e.g. from networking@base::

text
$ alembic history -r networking@base:
29f859a13ea (55af2cb1c267) -> 2a95102259be (networking) (head), add ip account table
109ec7d132bf -> 29f859a13ea (networking), add DNS table
3cac04ae8714 -> 109ec7d132bf (networking), add ip number table
 -> 3cac04ae8714 (networking), create networking branch

The reason for the discrepancy is that displaying history from the base
shows us what would occur if we ran a downgrade operation, instead of an
upgrade. If we downgraded all the files in networking using
networking@base, the dependencies aren't affected, they're left in place.

We also see something odd if we view heads at the moment::

text
$ alembic heads
2a95102259be (networking) (head)
27c6a30d7c24 (shoppingcart) (head)
55af2cb1c267 (effective head)

The head file that we used as a "dependency", 55af2cb1c267, is displayed
as an "effective" head, which we can see also in the history display earlier.
What this means is that at the moment, if we were to upgrade all versions
to the top, the 55af2cb1c267 revision number would not actually be
present in the alembic_version table; this is because it does not have
a branch of its own subsequent to the 2a95102259be revision which depends
on it::

text
$ alembic upgrade heads
INFO  [alembic.migration] Running upgrade 29f859a13ea, 55af2cb1c267 -> 2a95102259be, add ip account table

$ alembic current
2a95102259be (head)
27c6a30d7c24 (head)

The entry is still displayed in alembic heads because Alembic knows that
even though this revision isn't a "real" head, it's still something that
we developers consider semantically to be a head, so it's displayed, noting
its special status so that we don't get quite as confused when we don't
see it within alembic current.

If we add a new revision onto 55af2cb1c267, the branch again becomes
a "real" branch which can have its own entry in the database::

text
$ alembic revision -m "more account changes" --head=55af2cb@head
  Generating /path/to/foo/versions/34e094ad6ef1_more_account_changes.py ... done

$ alembic upgrade heads
INFO  [alembic.migration] Running upgrade 55af2cb1c267 -> 34e094ad6ef1, more account changes

$ alembic current
2a95102259be (head)
27c6a30d7c24 (head)
34e094ad6ef1 (head)

For posterity, the revision tree now looks like::

text
$ alembic history
29f859a13ea (55af2cb1c267) -> 2a95102259be (networking) (head), add ip account table
109ec7d132bf -> 29f859a13ea (networking), add DNS table
3cac04ae8714 -> 109ec7d132bf (networking), add ip number table
 -> 3cac04ae8714 (networking), create networking branch
1975ea83b712 -> 27c6a30d7c24 (shoppingcart) (head), add shopping cart table
55af2cb1c267 -> 34e094ad6ef1 (head), more account changes
ae1027a6acf -> 55af2cb1c267, add another account column
1975ea83b712 -> ae1027a6acf, Add a column
 -> 1975ea83b712 (branchpoint), create account table


                    --- 27c6 --> d747 --> <head>
                   /   (shoppingcart)
 --> 1975 -->
                   \
                     --- ae10 --> 55af --> <head>
                                    ^
                                    +--------+ (dependency)
                                             |
                                             |
 --> 3782 -----> 109e ----> 29f8 ---> 2a95 --> <head>
         (networking)

If there's any point to be made here, it's if you are too freely branching, merging
and labeling, things can get pretty crazy! Hence the branching system should
be used carefully and thoughtfully for best results.


Build Changelog (`docs/build/changelog.rst`)

========== Changelog

.. changelog::
:version: 1.20.2
:include_notes_from: unreleased

.. changelog::
:version: 1.20.1
:released: October 21, 2026

text
.. change::
    :tags: bug, autogenerate
    :tickets: 1085

    Fixed issue where a table or column comment set to an empty string would
    cause autogenerate to detect a comment change on every run.  As most
    backends, including PostgreSQL, MySQL, MariaDB and Oracle, don't store an
    empty string comment distinctly from no comment, autogenerate now treats
    an empty string comment the same as ``None`` when comparing comments.
    Additionally, an empty string comment is now rendered in
    ``op.create_table()`` and ``op.add_column()`` directives, where it was
    previously omitted.  Pull request courtesy vineoy.

.. change::
    :tags: bug, autogenerate, postgresql
    :tickets: 1360

    Fixed rendering of the PostgreSQL
    :class:`~sqlalchemy.dialects.postgresql.DOMAIN` type in autogenerated
    migration scripts.  Previously the type was rendered using ``repr()``,
    which omitted the ``check``, ``not_null``, ``default``, ``collation``,
    ``constraint_name``, ``schema`` and ``create_type`` arguments and did not
    apply a dialect prefix to the underlying data type.  The full set of
    arguments is now rendered and the underlying data type is rendered using
    the standard type-rendering logic.  Pull request courtesy Sebastian Cao.

.. change::
    :tags: bug, autogenerate, postgresql
    :tickets: 1462

    Fixed false-positive "Computed default cannot be modified" warning emitted
    during autogenerate when comparing PostgreSQL :class:`.Computed` columns
    whose reflected SQL expression includes type cast specifiers such as
    ``::regconfig`` or ``::text``, which PostgreSQL adds to the expression as
    stored.  These cast specifiers are now stripped before comparison so that
    otherwise equivalent expressions no longer trigger the warning.  Pull
    request courtesy Sebastian Cao.

.. change::
    :tags: bug, autogenerate, postgresql

    Fixed issue where autogenerate would detect an unchanged PostgreSQL
    expression-based index as modified when the expression included type
    casts, such as a cast to a type with a length or precision, a nested
    cast, or a cast followed by a ``DESC`` modifier.

.. change::
    :tags: bug, postgresql
    :tickets: 1504

    Fixed bug where :meth:`.Operations.alter_column` against a PostgreSQL
    identity column would render an incomplete, invalid ``ALTER TABLE ... ALTER
    COLUMN`` statement with no clause at all when ``existing_server_default``
    was omitted, even if there were no existing server default.  Pull request
    courtesy Riti Grover.

.. change::
    :tags: bug, environment
    :tickets: 1873

    Fixed issue where a third party plugin published using the
    ``alembic.plugins`` entrypoint, configured to refer to a plugin module as
    documented at :ref:`alembic.plugins.toplevel`, would cause ``import
    alembic`` to fail with a ``TypeError``, as the loaded module was
    erroneously iterated rather than being set up directly. An entrypoint
    that refers to a collection of modules, the form which inadvertently
    worked previously, continues to be accepted but now emits a deprecation
    warning.

.. change::
    :tags: bug, autogenerate
    :tickets: 1876

    Fixed issue where dialect-specific keyword arguments on a
    :class:`.ForeignKeyConstraint`, :class:`.ForeignKey` or
    :class:`.CheckConstraint`, such as ``postgresql_not_valid``, were not
    rendered for the constraints that are part of an autogenerated
    :meth:`.Operations.create_table` directive, nor for an autogenerated
    :meth:`.Operations.create_check_constraint` directive.  The
    :meth:`.Operations.create_table` case additionally requires SQLAlchemy
    2.0.55 / 2.1.5 or greater, which include the fix for SQLAlchemy issue
    #13677 where these arguments were lost when the :class:`.Table` was
    copied.

.. change::
    :tags: bug, autogenerate
    :tickets: 1884

    Fixed issue where, when using SQLAlchemy 2.1, the CHECK constraint
    generated by a type such as :class:`~sqlalchemy.types.Enum` or
    :class:`~sqlalchemy.types.Boolean` with ``create_constraint=True`` would
    be rendered by autogenerate as explicit ``sa.CheckConstraint()`` entries
    within ``op.create_table()``, in addition to the type itself which
    already produces that constraint, leading to a duplicate constraint error
    when the migration is run.

.. change::
    :tags: bug, operations

    Fixed issue where a :class:`.CreateTableOp` produced from an existing
    :class:`~sqlalchemy.schema.Table` using :meth:`.CreateTableOp.from_table`
    would emit the CHECK constraint generated by a type such as
    :class:`~sqlalchemy.types.Enum` or :class:`~sqlalchemy.types.Boolean`
    with ``create_constraint=True`` twice in the resulting CREATE TABLE
    statement when invoked directly.

.. changelog::
:version: 1.20.0
:released: September 11, 2026

text
.. change::
    :tags: bug, batch
    :tickets: 1768

    Fixed bug in batch mode where adding a column with a type that generates
    its own CHECK constraint, such as :class:`~sqlalchemy.types.Boolean` or
    :class:`~sqlalchemy.types.Enum` with
    :paramref:`~sqlalchemy.types.Boolean.create_constraint` set to ``True``,
    would emit the constraint twice when the table was recreated, once under
    the name generated by the naming convention in use and once under the
    name given to the type.  The constraint is now emitted once, using the
    same name that would be used outside of batch mode.

.. change::
    :tags: bug, batch
    :tickets: 1844

    Fixed bug in batch mode where a CHECK constraint generated by a type such
    as :class:`~sqlalchemy.types.Boolean` or :class:`~sqlalchemy.types.Enum`
    would lose the name established for it by the naming convention in use
    when the table was recreated, as the constraint was regenerated against
    the temporary table used for the recreate operation.  The naming
    convention is now resolved against the name of the table being replaced.

    As part of this change, a type that generates a CHECK constraint but has
    no name of its own, used in conjunction with a naming convention that
    includes the ``%(constraint_name)s`` token, now emits a warning, as the
    convention has no name to interpolate; the constraint continues to be
    emitted without a name.  As noted at :ref:`batch_check_constraints`,
    these datatypes should be given a name in order to participate fully in
    batch mode.

.. change::
    :tags: bug, batch
    :tickets: 1845

    Fixed bug in batch mode where the
    :paramref:`.Operations.batch_alter_table.naming_convention` parameter
    would re-generate the names of constraints that already had a name when
    reflected, in the case where the convention included the
    ``%(constraint_name)s`` token, leading to a name that included the
    convention's own prefix twice.  The convention is now applied only to
    those constraints that are reflected without a name, which is the use
    case the parameter is documented for.

.. change::
    :tags: usecase, batch
    :tickets: 1846

    Added a warning for the case where an unnamed CHECK constraint on a
    reflected table is omitted from a batch "recreate" operation.  An unnamed
    CHECK constraint can't be reliably carried over in a batch recreate
    as it may refer to columns that are being dropped or changed.  This
    omission was previously a silent operation.   The presence of any
    :class:`~sqlalchemy.schema.CheckConstraint` in
    :paramref:`.Operations.batch_alter_table.table_args` is taken to indicate
    that the case has been accommodated, and no warning is emitted.



.. change::
    :tags: change, general
    :tickets: 1851

    Support for SQLAlchemy 1.4 is dropped as of Alembic 1.20.0; SQLAlchemy
    2.0.0 is now the minimum required version.  SQLAlchemy 1.4 has no support
    for Python 3.13 and above, and the version-conditional code paths it
    required throughout Alembic have been removed in favor of the SQLAlchemy
    2.0 API.

.. change::
    :tags: bug, mysql
    :tickets: 1866

    Fixed bug where autogenerate with
    :paramref:`.EnvironmentContext.configure.compare_server_default` would
    report a persistent server default change for ``Float``, ``Numeric`` and
    other decimal columns on MySQL. MySQL reports a literal server default
    for such a column either in quoted form, e.g. ``'1'``, or, when the
    value is fractional, as a parenthesized expression, e.g. ``(2.5)``;
    neither form was accommodated for columns other than those of
    ``Integer`` affinity, so that a default such as ``server_default="1"``
    on a :class:`sqlalchemy.Float` column would never compare as equal to
    the reflected value.

.. change::
    :tags: bug, mysql
    :tickets: 1866

    Fixed bug where autogenerate with
    :paramref:`.EnvironmentContext.configure.compare_server_default` would
    report a persistent server default change for an expression server
    default such as ``(rand())``. MySQL reports such a default with the
    surrounding parenthesis included whereas MariaDB reports it without
    them, so that the parenthesis are now disregarded when comparing.

.. change::
    :tags: usecase, autogenerate
    :tickets: 916

    Autogenerate now renders a warning comment above any rendered
    :meth:`.Operations.drop_constraint` directive for which the constraint name
    is ``None``, as is the case when a constraint that has no name in the model
    is dropped, most typically within the ``downgrade()`` function of a
    migration that adds an unnamed constraint.  A warning is also emitted on
    the console when the migration script is generated.   The directive
    requires a non-None name in order to be able to emit a "DROP CONSTRAINT"
    command.

.. changelog::
:version: 1.19.2
:released: September 4, 2026

text
.. change::
    :tags: changed, autogenerate
    :tickets: 1859

    The autogenerate plugin for CHECK constraint detection by name, added in
    1.19.0. for :ticket:`508`, is no longer enabled by default.  It has been
    renamed from ``alembic.autogenerate.checkconstraint_byname`` to
    ``alembic.ext.checkconstraint_byname`` and no longer matches on the
    ``"alembic.autogenerate.*"`` wildcard, which remains the default plugin
    specification.  The previous name will still function as well if placed in
    the plugins list explicitly,  both to enable the plugin and within a
    ``"~"`` exclusion, so that an ``env.py`` written against 1.19.0 or 1.19.1
    requires no change.

    The plugin is now recommended only for schemas that ensure the naming of
    all constraints using a client side naming convention, otherwise there's a
    persistent risk of false positives. See
    :ref:`autogenerate_check_constraints` for background on things to be aware
    of when using this plugin.

    As part of this change "type bound" CHECK constraints, which include
    constraints generated for the ``Boolean`` and ``Enum`` datatypes when the
    ``create_constraint`` parameter is set to True, are no longer ignored in
    the metadata side, so that normal name-based matching can occur for
    these constraints.

.. change::
    :tags: usecase, autogenerate, batch
    :tickets: 1860

    The target of a :class:`~sqlalchemy.schema.ForeignKey` is now located
    using the ``ForeignKey.target_tokens`` and
    ``ForeignKey.target_table_key`` accessors added in SQLAlchemy 2.1, rather
    than by splitting the dotted string form of that target on ``"."``.  As a
    dot inside a schema, table or column name cannot be told apart from the
    separator between those names, a foreign key whose target name contained
    a dot was previously mis-parsed by autogenerate rendering as well as by
    batch migrations.  The dotted string continues to be split when running
    against SQLAlchemy 2.0, where these accessors are not present.  Thanks to
    Gyanu Mayank for the initial pull request.

.. changelog::
:version: 1.19.1
:released: August 8, 2026

text
.. change::
    :tags: bug, autogenerate
    :tickets: 1842

    Fixed bug in the check constraint detection implemented in :ticket:`508`
    that failed to take into account column bound check constraints,
    leading to wrong autogenerate detections.

.. changelog::
:version: 1.19.0
:released: August 4, 2026

text
.. change::
    :tags: bug, commands
    :tickets: 1822

    Fixed inconsistency where running ``stamp`` or ``downgrade`` to ``base`` in
    offline (``--sql``) mode would emit a ``DROP TABLE alembic_version``
    statement, while the same operations in online mode never drop the version
    table.  Offline mode no longer emits this ``DROP``, matching online
    behavior.  The version table continues to be created when it does not exist;
    only the spurious offline-only drop has been removed.  Pull request
    courtesy imurodl.

.. change::
    :tags: feature, autogenerate
    :tickets: 508

    Autogenerate now detects the addition and removal of named CHECK
    constraints, as part of the default autogenerate behavior. Detection is
    name-based only; a constraint whose name is unchanged is presumed
    equivalent regardless of its expression text, as reliably normalizing
    SQL expressions across backends for comparison purposes is not generally
    feasible. This behavior is implemented as a plugin named
    ``alembic.autogenerate.checkconstraint_byname``, and may be disabled if not
    desired by excluding it from the
    :paramref:`.EnvironmentContext.configure.autogenerate_plugins` list.
    Pull request courtesy Francois van Kempen.

    .. seealso::

        :ref:`autogenerate_check_constraints`

.. change::
    :tags: changed, installation

    Environmental updates:

    * Trove classifiers now include Python 3.15 which is now part of CI
      integration
    * Python 3.14 is also added to trove classifiers which had been previously
      omitted
    * Implemented :pep:`604` style unions in type annotations

.. changelog::
:version: 1.18.5
:released: June 25, 2026

text
.. change::
    :tags: bug, autogenerate
    :tickets: 1258

    Fixed rendering of dialect keyword arguments containing
    :class:`~sqlalchemy.schema.Column` objects within sequences, such as
    ``postgresql_include``.  These were previously rendered using ``repr()``,
    producing invalid Python in the generated migration scripts.  Column
    objects within list or tuple values are now correctly rendered as their
    string column names.  Pull request courtesy Ajay Singh.

.. change::
    :tags: usecase, commands
    :tickets: 1712

    Added ``--splice`` support to the :func:`.merge` command. Previously, the
    merge command would suggest using ``--splice`` when attempting to merge
    non-head revisions, but the flag was not actually accepted by the command.
    The ``splice`` parameter is now available in both the command-line
    interface and the :func:`.command.merge` function, matching the existing
    support in :func:`.command.revision`.  Pull request courtesy Kadir Can
    Ozden.

.. change::
    :tags: bug, mysql
    :tickets: 779, 1745

    Implemented type comparison for :class:`.ENUM` datatypes on MySQL, which
    checks that the individual enum values are equivalent.  If additional
    entries are on either side, this generates a diff.  Changes of order do not
    generate a diff.  Pull request courtesy Furkan Kรถykฤฑran.


.. change::
    :tags: usecase, environment
    :tickets: 1806

    Added :paramref:`.ScriptDirectory.get_heads.consider_depends_on`
    parameter to :meth:`.ScriptDirectory.get_heads`. When set to ``True``,
    head revisions that are also a dependency of another revision via
    ``depends_on`` are excluded from the result, matching the effective
    heads that would be present in the ``alembic_version`` table after
    running all upgrades.

.. change::
    :tags: bug, operations
    :tickets: 1820

    Fixed bug where the ``inline_references`` parameter of
    :meth:`.Operations.add_column` did not include foreign key referential
    actions such as ``ON DELETE``, ``ON UPDATE``, ``DEFERRABLE``,
    ``INITIALLY``, and ``MATCH`` when rendering the inline ``REFERENCES``
    clause.

.. changelog::
:version: 1.18.4
:released: February 10, 2026

text
.. change::
    :tags: bug, operations
    :tickets: 1232

    Reverted the behavior of :meth:`.Operations.add_column` that would
    automatically render the "PRIMARY KEY" keyword inline when a
    :class:`.Column` with ``primary_key=True`` is added. The automatic
    behavior, added in version 1.18.2, is now opt-in via the new
    :paramref:`.Operations.add_column.inline_primary_key` parameter. This
    change restores the ability to render a PostgreSQL SERIAL column, which is
    required to be ``primary_key=True``, while not impacting the ability to
    render a separate primary key constraint. This also provides consistency
    with the :paramref:`.Operations.add_column.inline_references` parameter and
    gives users explicit control over SQL generation.

    To render PRIMARY KEY inline, use the
    :paramref:`.Operations.add_column.inline_primary_key` parameter set to
    ``True``::

        op.add_column(
            "my_table",
            Column("id", Integer, primary_key=True),
            inline_primary_key=True
        )

.. changelog::
:version: 1.18.3
:released: January 29, 2026

text
.. change::
    :tags: bug, autogenerate
    :tickets: 1787

    Fixed regression in version 1.18.0 due to :ticket:`1771` where autogenerate
    would raise ``NoReferencedTableError`` when a foreign key constraint
    referenced a table that was not part of the initial table load, including
    tables filtered out by the
    :paramref:`.EnvironmentContext.configure.include_name` callable or tables
    in remote schemas that were not included in the initial reflection run.

    The change in :ticket:`1771` was a performance optimization that eliminated
    additional reflection queries for tables that were only referenced by
    foreign keys but not explicitly included in the main reflection run.
    However, this optimization inadvertently removed the creation of
    :class:`.Table` objects for these referenced tables, causing autogenerate
    to fail when processing foreign key constraints that pointed to them.

    The fix creates placeholder :class:`.Table` objects for foreign key targets
    that are not reflected, allowing the autogenerate comparison to proceed
    without error while maintaining the performance improvement from
    :ticket:`1771`. When multiple foreign keys reference different columns in
    the same filtered table, the placeholder table accumulates all necessary
    columns. These placeholder tables may be visible when using the
    :paramref:`.EnvironmentContext.configure.include_object` callable to
    inspect :class:`.ForeignKeyConstraint` objects; they will have the name,
    schema and basic column information for the relevant columns present.

.. change::
    :tags: bug, general
    :tickets: 1788

    Fixed regression caused by :ticket:`1669` which requires SQLAlchemy objects
    to support generic type subscripting; for the older SQLAlchemy 1.4 series,
    this requires version 1.4.23.  Changed the minimum requirements to require
    version 1.4.23 rather than 1.4.0.

.. changelog::
:version: 1.18.2
:released: January 28, 2026

text
.. change::
    :tags: usecase, operations
    :tickets: 1232

    The ``primary_key`` parameter on :class:`.Column` is now honored when
    :meth:`.Operations.add_column` is used, and will emit the "PRIMARY KEY"
    keyword inline within the ADD COLUMN directive.  This is strictly a syntax
    enhancement; no attempt is made to reconcile the column's primary key
    status with any existing primary key constraint or particular backend
    limitations on adding columns to the primary key.

    .. note::

        As of version 1.18.4, this behavior has been amended to be opt-in
        via the new :paramref:`.Operations.add_column.inline_primary_key`
        parameter to :meth:`.Operations.add_column`, rather than occurring
        automatically when ``primary_key=True`` is set on the
        :class:`.Column` object.

.. change::
    :tags: bug, typing
    :tickets: 1669

    Fixed typing issue where the :paramref:`.AlterColumnOp.server_default` and
    :paramref:`.AlterColumnOp.existing_server_default` parameters failed to
    accommodate common SQLAlchemy SQL constructs such as ``null()`` and
    ``text()``.   Pull request courtesy Sebastian Kreft.

    .. note:: This fix is not compatible with older SQLAlchemy versions
       prior to 1.4.23.

.. change::
    :tags: usecase, operations
    :tickets: 1780

    Added ``inline_references`` parameter to :meth:`.Operations.add_column`
    which allows rendering of ``REFERENCES`` clauses inline within the ``ADD
    COLUMN`` directive rather than as a separate ``ADD CONSTRAINT`` directive.
    This syntax is supported by PostgreSQL, Oracle, MySQL 5.7+, and MariaDB
    10.5+, and can provide performance benefits on large tables by avoiding
    full table validation when adding a nullable foreign key column.

.. changelog::
:version: 1.18.1
:released: January 14, 2026

text
.. change::
    :tags: bug, autogenerate
    :tickets: 1779

    Fixed issue in new plugin system where the configured logger was not
    correctly using the ``__name__`` token to identify the logger.

.. change::
    :tags: bug, operations

    Revised the change regarding SQLAlchemy 2.1 and deprecation warnings
    related to ``isolate_from_table=True``. Further developments in release 2.1
    have revised how this parameter will be modified.

.. changelog::
:version: 1.18.0
:released: January 9, 2026

text
.. change::
    :tags: bug, postgresql
    :tickets: 1507

    Fixed issue where PostgreSQL sequence defaults on non-primary key columns
    were incorrectly detected as changed on every autogenerate run. Server
    default comparison logic is adjusted to filter out the ``::regclass``
    expression added by the server which interferes with the comparison.

.. change::
    :tags: feature, operations
    :tickets: 1737

    When alembic is run in "verbose" mode, alembic now logs a message to
    indicate from which file is used to load the configuration.

.. change::
    :tags: bug, mssql
    :tickets: 1755

    Implemented DDL for column comment add/update/delete when using the
    :paramref:`.Operations.alter_column.comment` parameter with
    :meth:`.Operations.alter_column` on Microsoft SQL Server.  Previously,
    these functions were not implemented for SQL Server and would raise
    ``UnsupportedCompilationError``.

.. change::
    :tags: feature, autogenerate
    :tickets: 1771

    Autogenerate reflection sweeps now use the "bulk" inspector methods
    introduced in SQLAlchemy 2.0, which for selected dialects including
    PostgreSQL and Oracle use batched queries to reflect whole collections of
    tables using O(1) queries rather than O(N).

.. change::
    :tags: usecase, environment
    :tickets: 1774

    The ``file_template`` configuration option now supports directory paths,
    allowing migration files to be organized into subdirectories. When using
    directory separators in ``file_template`` (e.g.,
    ``%(year)d/%(month).2d/%(day).2d_%(rev)s_%(slug)s``), Alembic will
    automatically create the necessary directory structure. The
    ``recursive_version_locations`` setting must be set to ``true`` when using
    this feature in order for the revision files to be located for subsequent
    commands.

.. change::
    :tags: usecase, operations

    Avoid deprecation warning in add/drop constraint added in SQLAlchemy 2.1.
    Ensure that alembic is compatible with the changes added in
    https://github.com/sqlalchemy/sqlalchemy/issues/13006
    by explicitly setting ``isolate_from_table=True`` when running with
    SQLAlchemy 2.1 or greater.

    .. note::

       This change was revised in Alembic 1.18.1; futher developments
       in SQLAlchemy 2.1 necessitated further adjustments.

.. change::
    :tags: feature, autogenerate

    Release 1.18.0 introduces a plugin system that allows for automatic
    loading of third-party extensions as well as configurable autogenerate
    compare functionality on a per-environment basis.

    The :class:`.Plugin` class provides a common interface for extensions that
    register handlers among Alembic's existing extension points such as
    :meth:`.Operations.register_operation` and
    :meth:`.Operations.implementation_for`. A new interface for registering
    autogenerate comparison handlers,
    :meth:`.Plugin.add_autogenerate_comparator`, provides for autogenerate
    compare functionality that may be custom-configured on a per-environment
    basis using the new
    :paramref:`.EnvironmentContext.configure.autogenerate_plugins` parameter.

    The change does not impact well known Alembic add-ons such as
    ``alembic-utils``, which continue to work as before; however, such add-ons
    have the option to provide plugin entrypoints going forward.

    As part of this change, Alembic's autogenerate compare functionality is
    reorganized into a series of internal plugins under the
    ``alembic.autogenerate`` namespace, which may be individually or
    collectively identified for inclusion and/or exclusion within the
    :meth:`.EnvironmentContext.configure` call using a new parameter
    :paramref:`.EnvironmentContext.configure.autogenerate_plugins`. This
    parameter is also where third party comparison plugins may also be
    indicated.

    See :ref:`alembic.plugins.toplevel` for complete documentation on
    the new :class:`.Plugin` class as well as autogenerate-specific usage
    instructions.

.. changelog::
:version: 1.17.2
:released: November 14, 2025

text
.. change::
    :tags: bug, mssql
    :tickets: 1744

    Fixed issue in SQL Server dialect where the DROP that's automatically
    emitted for existing default constraints during an ALTER COLUMN needs to
    take place before not just the modification of the column's default, but
    also before the column's type is changed.

.. change::
    :tags: feature, operations
    :tickets: 1750

    Added :paramref:`.Operations.implementation_for.replace` parameter to
    :meth:`.Operations.implementation_for`, allowing replacement of existing
    operation implementations.  This allows for existing operations such as
    :class:`.CreateTableOp` to be extended directly.  Pull request courtesy
    justanothercatgirl.

    .. seealso::

        :ref:`operations_extending_builtin`

.. changelog::
:version: 1.17.1
:released: October 28, 2025

text
.. change::
    :tags: bug, commands
    :tickets: 1540

    Disallow ':' character in custom revision identifiers. Previously, using a
    colon in a revision ID (e.g., 'REV:1') would create the revision, however
    revisions with colons in them are not correctly interpreted by other
    commands, as it overlaps with the revision range syntax.   Pull request
    courtesy Kim Wooseok with original implementation by Hrushikesh Patil.

.. change::
   :tags: usecase, commands
   :tickets: 1705

   Added :paramref:`.command.current.check_heads` parameter to
   :func:`.command.current` command, available from the command line via the
   ``--check-heads`` option to ``alembic current``.  This tests if all head
   revisions are applied to the database and raises :class:`.DatabaseNotAtHead`
   (or from the command line, exits with a non-zero exit code) if this is not
   the case.  The parameter operates equvialently to the cookbook recipe
   :ref:`cookbook_check_heads`. Pull request courtesy Stefan Scherfke.

.. changelog::
:version: 1.17.0
:released: October 11, 2025

text
.. change::
    :tags: change, tests

    The top-level test runner has been changed to use ``nox``, adding a
    ``noxfile.py`` as well as some included modules.   The ``tox.ini`` file
    remains in place so that ``tox`` runs will continue to function in the near
    term, however it will be eventually removed and improvements and
    maintenance going forward will be only towards ``noxfile.py``.

.. change::
    :tags: change, general

    The minimum Python version is now 3.10, as Python 3.9 is EOL.

.. changelog::
:version: 1.16.5
:released: August 27, 2025

text
.. change::
    :tags: bug, mysql
    :tickets: 1492

    Fixed Python-side autogenerate rendering of index expressions in MySQL
    dialect by aligning it with SQLAlchemy's MySQL index expression rules. Pull
    request courtesy david-fed.

.. change::
    :tags: bug, config
    :tickets: 1709

    Fixed issue where new pyproject.toml config would fail to parse the integer
    value used for the ``truncate_slug_length`` parameter.  Pull request
    courtesy Luรญs Henrique Allebrandt Schunemann.

.. changelog::
:version: 1.16.4
:released: July 10, 2025

text
.. change::
    :tags: bug, config
    :tickets: 1694

    Fixed issue in new ``pyproject.toml`` support where boolean values, such as
    those used for the ``recursive_version_locations`` and ``sourceless``
    configuration parameters, would not be accepted.

.. changelog::
:version: 1.16.3
:released: July 8, 2025

text
.. change::
    :tags: bug, autogenerate
    :tickets: 1633

    Fixed the rendering of ``server_default=FetchedValue()`` to ensure it is
    preceded by the ``sa.`` prefix in the migration script. Pull request
    courtesy david-fed.

.. change::
    :tags: usecase, commands
    :tickets: 1683

    Added new ``pyproject_async`` template, combining the new ``pyproject``
    template with the ``async`` template.  Pull request courtesy Alc-Alc.

.. change::
    :tags: usecase, autogenerate
    :tickets: 1686

    Add "module" post-write hook. This hook type is almost identical to the
    console_scripts hook, except it's running ``python -m black`` instead of
    using black's ``console_script``. It is mainly useful for tools without
    console scripts (e.g. ruff), but has semantics closer to the
    console_scripts hook in that it finds the ruff module available to the
    running interpreter instead of finding an executable by path. Pull request
    courtesy Frazer McLean.

.. change::
    :tags: bug, autogenerate
    :tickets: 1692

    Fixed autogenerate rendering bug which failed to render foreign key
    constraints local to a :class:`.CreateTableOp` object if it did not refer
    to a ``MetaData`` collection via a private constructor argument that would
    not ordinarily be passed in user-defined rewriter recipes, including ones
    in the Alembic cookbook section of the docs.

.. changelog::
:version: 1.16.2
:released: June 16, 2025

text
.. change::
    :tags: bug, autogenerate
    :tickets: 1671

    Fixed issue where dialect-specific keyword arguments in ``dialect_kwargs``
    were not rendered when rendering the :meth:`.Operations.create_foreign_key`
    operation.   This prevented dialect-specific keywords from being rendered
    using custom :class:`.Rewriter` recipes that modify
    :class:`.ops.CreateForeignKeyOp`, similar to other issues such as
    :ticket:`1635`.  Pull request courtesy Justin Malin.

.. change::
    :tags: bug, command
    :tickets: 1679

    Fixed rendering of ``pyproject.toml`` to include two newlines when
    appending content to an existing file.  Pull request courtesy Jonathan
    Vanasco.

.. changelog::
:version: 1.16.1
:released: May 21, 2025

text
.. change::
    :tags: bug, command
    :tickets: 1660

    Fixed regression caused by the ``pathlib`` refactoring that removed the use
    of :meth:`.Config.get_template_directory` as the canonical source of
    templates; the method is still present however it no longer would be
    consulted for a custom config subclass, as was the case with flask-migrate.

.. change::
    :tags: bug, command
    :tickets: 1659

    Fixed regression caused by the ``pathlib`` refactoring where the "missing
    template" error message failed to render the name of the template that
    could not be found.

.. changelog::
:version: 1.16.0
:released: May 21, 2025

text
.. change::
    :tags: feature, environment
    :tickets: 1082

    Added optional :pep:`621` support to Alembic, allowing all source code
    related configuration (e.g. local file paths, post write hook
    configurations, etc) to be configured in the project's ``pyproject.toml``
    file.   A new init template ``pyproject`` is added which illustrates a
    basic :pep:`621` setup.

    Besides being better integrated with a Python project's existing source
    code configuration, the TOML format allows for more flexible structures,
    allowing configuration items like ``version_locations`` and
    ``prepend_sys_path`` to be configured as lists of path strings without the
    need for path separator characters used by ``ConfigParser`` format.   The
    feature continues to support the ``%(here)s`` token which can substitute
    the absolute parent directory of the ``pyproject.toml`` file when
    consumed.

    The :pep:`621` feature supports configuration values that are relevant to
    source code organization and generation only; it does not accommodate
    configuration of database connectivity or logging, which remain under the
    category of "deployment" configuration and continue to be part of
    ``alembic.ini``, or whatever configurational method is established by the
    ``env.py`` file.   Using the combination of ``pyproject.toml`` for source
    code configuration along with a custom database/logging configuration
    method established in ``env.py`` will allow the ``alembic.ini`` file to be
    omitted altogether.


    .. seealso::

        :ref:`using_pep_621`

.. change::
    :tags: usecase, environment
    :tickets: 1330

    Added new option to the ConfigParser (e.g. ``alembic.ini``) configuration
    ``path_separator``, which supersedes the existing ``version_path_separator``
    option.  ``path_separator`` specifies the path separator character that
    will be recognized for both the ``version_locations`` option as well
    as the ``prepend_sys_path`` option, defaulting to ``os`` which indicates
    that the value of ``os.pathsep`` should be used.

    The new attribute applies necessary os-dependent path splitting to the
    ``prepend_sys_path`` option so that windows paths which contain drive
    letters with colons are not inadvertently split, whereas previously
    os-dependent path splitting were only available for the ``version_locations`` option.

    Existing installations that don't indicate ``path_separator``
    will continue to use the older behavior, where ``version_path_separator``
    may be configured for ``version_locations``, and ``prepend_sys_path``
    continues to be split on spaces/commas/colons.  A deprecation warning
    is emitted for these fallback scenarios.

    When using the new ``pyproject.toml`` configuration detailed at
    :ref:`using_pep_621`, the whole issue of "path separators" is sidestepped
    and parameters like ``path_separator`` are unnecessary, as the TOML based
    configuration configures version locations and sys path elements as
    lists.

    Pull request courtesy Mike Werezak.

.. change::
    :tags: feature, commands
    :tickets: 1610

    Added new :meth:`.CommandLine.register_command` method to
    :class:`.CommandLine`, intended to facilitate adding custom commands to
    Alembic's command line tool with minimal code required; previously this
    logic was embedded internally and was not publicly accessible.  A new
    recipe demonstrating this use is added.   Pull request courtesy Mikhail
    Bulash.

    .. seealso::

        :ref:`custom_commandline`

.. change::
    :tags: usecase, operations
    :tickets: 1626

    Added :paramref:`.Operations.add_column.if_not_exists` and
    :paramref:`.Operations.drop_column.if_exists` to render ``IF [NOT] EXISTS``
    for ``ADD COLUMN`` and ``DROP COLUMN`` operations, a feature available on
    some database backends such as PostgreSQL, MariaDB, as well as third party
    backends.  The parameters also support autogenerate rendering allowing them
    to be added to autogenerate scripts via a custom :class:`.Rewriter`.  Pull
    request courtesy of Louis-Amaury Chaib (@lachaib).

.. change::
    :tags: bug, general
    :tickets: 1637

    The ``pyproject.toml`` file used by the Alembic project itself for its
    Python package configuration has been amended to use the updated :pep:`639`
    configuration for license, which eliminates loud deprecation warnings when
    building the package.   Note this necessarily bumps setuptools build
    requirement to 77.0.3.

.. change::
    :tags: bug, environment
    :tickets: 1643

    Fixed issue where use of deprecated ``utcnow()`` function would generate
    warnings.  Has been replaced with ``now(UTC)``.  Pull request courtesy
    Jens Trรถger.

.. change::
    :tags: usecase, operations
    :tickets: 1650

    Added :paramref:`.Operations.drop_constraint.if_exists` parameter to
    :meth:`.Operations.drop_constraint` which will render ``DROP CONSTRAINT IF
    EXISTS``. The parameter also supports autogenerate rendering allowing it to
    be added to autogenerate scripts via a custom :class:`.Rewriter`.  Pull
    request courtesy Aaron Griffin.

.. change::
    :tags: bug, autogenerate
    :tickets: 1656

    The :meth:`.Operations.execute` operation when rendered in autogenerate
    (which would necessarily be only when using a custom writer that embeds
    :class:`.ExecuteSQLOp`) now correctly takes into account the value
    configured in :paramref:`configure.alembic_module_prefix` when rendering
    the operation with its prefixing namespace; previously this was hardcoded
    to ``op.``. Pull request courtesy Avery Fischer.

.. change::
    :tags: bug, autogenerate
    :tickets: 264

    The autogenerate process will now apply the :meth:`.Operations.f` modifier
    to the names of all constraints and indexes that are reflected from the
    target database when generating migrations, which has the effect that these
    names will not have any subsequent naming conventions applied to them when
    the migration operations proceed.  As reflected objects already include the
    exact name that's present in the database, these names should not be
    modified.   The fix repairs the issue when using custom naming conventions
    which feature the ``%(constraint_name)s`` token would cause names to be
    double-processed, leading to errors in migration runs.



.. change::
    :tags: refactored, environment

    The command, config and script modules now rely on ``pathlib.Path`` for
    internal path manipulations, instead of ``os.path()`` operations.   This
    has some impact on both public and private (i.e. underscored) API functions:

    * Public API functions that accept parameters indicating file and directory
      paths as strings will continue to do so, but now will also accept
      ``os.PathLike`` objects as well.
    * Public API functions and accessors that return directory paths as strings
      such as :attr:`.ScriptDirectory.dir`, :attr:`.Config.config_file_name`
      will continue to do so.
    * Private API functions and accessors, i.e. all those that are prefixed
      with an underscore, that previously returned directory paths as
      strings may now return a Path object instead.

.. changelog::
:version: 1.15.2
:released: March 28, 2025

text
.. change::
    :tags: bug, autogenerate
    :tickets: 1635

    Fixed issue where the "modified_name" of :class:`.AlterColumnOp` would not
    be considered when rendering op directives for autogenerate. While
    autogenerate cannot detect changes in column name, this would nonetheless
    impact approaches that made use of this attribute in rewriter recipes. Pull
    request courtesy lenvk.

.. changelog::
:version: 1.15.1
:released: March 4, 2025

text
.. change::
    :tags: bug, installation
    :tickets: 1616

    Fixed an issue in the new :pep:`621` ``pyproject.toml`` layout that
    prevented Alembic's template files from being included in the ``.whl`` file
    in the distribution.

.. changelog::
:version: 1.15.0
:released: March 4, 2025 (yanked due to issue #1616)

text
.. change::
    :tags: bug, environment
    :tickets: 1567

    Added a basic docstring to the migration template files so that the
    upgrade/downgrade methods pass the D103 linter check which requires a
    docstring for public functions.  Pull request courtesy Peter Cock.

.. change::
    :tags: usecase, autogenerate
    :tickets: 1603

    Index autogenerate will now render labels for expressions
    that use them. This is useful when applying operator classes
    in PostgreSQL that can be keyed on the label name.

.. change::
    :tags: bug, autogenerate
    :tickets: 1613

    Fixed autogenerate rendering bug where the ``deferrable`` element of
    ``UniqueConstraint``, a bool, were being stringified rather than repr'ed
    when generating Python code.

.. change::
    :tags: changed, general

    Support for Python 3.8 is dropped as of Alembic 1.15.0; this version is
    now EOL so Python 3.9 or higher is required for Alembic 1.15.

.. change::
    :tags: changed, general

    Support for SQLAlchemy 1.3, which was EOL as of 2021, is now dropped from
    Alembic as of version 1.15.0.   SQLAlchemy version 1.4 or greater is
    required for use with Alembic 1.15.0.

.. change::
    :tags: usecase, autogenerate
    :tickets: 1597

    Add revision context to AutogenerateDiffsDetected so that command can be
    wrapped and diffs may be output in a different format. Pull request
    courtesy Louis-Amaury Chaib (@lachaib).

.. change::
    :tags: changed, general

    Installation has been converted to use :pep:`621`, e.g. ``pyproject.toml``.

.. changelog::
:version: 1.14.1
:released: January 19, 2025

text
.. change::
    :tags: bug, environment
    :tickets: 1556

    Added `tzdata` to `tz` extras, which is required on some platforms such as
    Windows.  Pull request courtesy Danipulok.

.. change::
    :tags: usecase, sqlite
    :tickets: 1576

    Modified SQLite's dialect to render "ALTER TABLE <t> RENAME COLUMN" when
    :meth:`.Operations.alter_column` is used with a straight rename, supporting
    SQLite's recently added column rename feature.

.. change::
    :tags: bug, autogenerate
    :tickets: 1585

    Fixed bug where autogen render of a "variant" type would fail to catch the
    variants if the leading type were a dialect-specific type, rather than a
    generic type.

.. changelog::
:version: 1.14.0
:released: November 4, 2024

text
.. change::
    :tags: usecase, runtime
    :tickets: 1560

    Added a new hook to the :class:`.DefaultImpl`
    :meth:`.DefaultImpl.version_table_impl`.  This allows third party dialects
    to define the exact structure of the alembic_version table, to include use
    cases where the table requires special directives and/or additional columns
    so that it may function correctly on a particular backend.  This is not
    intended as a user-expansion hook, only a dialect implementation hook to
    produce a working alembic_version table. Pull request courtesy Maciek
    Bryล„ski.

.. changelog::
:version: 1.13.3
:released: September 23, 2024

text
.. change::
    :tags: usecase, autogenerate

    Render ``if_exists`` and ``if_not_exists`` parameters in
    :class:`.CreateTableOp`, :class:`.CreateIndexOp`, :class:`.DropTableOp` and
    :class:`.DropIndexOp` in an autogenerate context.  While Alembic does not
    set these parameters during an autogenerate run, they can be enabled using
    a custom :class:`.Rewriter` in the ``env.py`` file, where they will now be
    part of the rendered Python code in revision files.  Pull request courtesy
    of Louis-Amaury Chaib (@lachaib).

.. change::
    :tags: usecase, environment
    :tickets: 1509

    Enhance ``version_locations`` parsing to handle paths containing newlines.

.. change::
    :tags: usecase, operations
    :tickets: 1520

    Added support for :paramref:`.Operations.create_table.if_not_exists` and
    :paramref:`.Operations.drop_table.if_exists`, adding similar functionality
    to render IF [NOT] EXISTS for table operations in a similar way as with
    indexes. Pull request courtesy Aaron Griffin.


.. change::
    :tags: change, general

    The pin for ``setuptools<69.3`` in ``pyproject.toml`` has been removed.
    This pin was to prevent a sudden change to :pep:`625` in setuptools from
    taking place which changes the file name of SQLAlchemy's source
    distribution on pypi to be an all lower case name, and the change was
    extended to all SQLAlchemy projects to prevent any further surprises.
    However, the presence of this pin is now holding back environments that
    otherwise want to use a newer setuptools, so we've decided to move forward
    with this change, with the assumption that build environments will have
    largely accommodated the setuptools change by now.

.. changelog::
:version: 1.13.2
:released: June 26, 2024

text
.. change::
    :tags: bug, commands
    :tickets: 1384

    Fixed bug in alembic command stdout where long messages were not properly
    wrapping at the terminal width.   Pull request courtesy Saif Hakim.

.. change::
    :tags: usecase, autogenerate
    :tickets: 1391

    Improve computed column compare function to support multi-line expressions.
    Pull request courtesy of Georg Wicke-Arndt.

.. change::
    :tags: bug, execution
    :tickets: 1394

    Fixed internal issue where Alembic would call ``connection.execute()``
    sending an empty tuple to indicate "no params".  In SQLAlchemy 2.1 this
    case will be deprecated as "empty sequence" is ambiguous as to its intent.


.. change::
    :tags: bug, tests
    :tickets: 1435

    Fixes to support pytest 8.1 for the test suite.

.. change::
    :tags: bug, autogenerate, postgresql
    :tickets: 1479

    Fixed the detection of serial column in autogenerate with tables
    not under default schema on PostgreSQL

.. changelog::
:version: 1.13.1
:released: December 20, 2023

text
.. change::
    :tags: bug, autogenerate
    :tickets: 1337

    Fixed :class:`.Rewriter` so that more than two instances could be chained
    together correctly, also allowing multiple ``process_revision_directives``
    callables to be chained.  Pull request courtesy zrotceh.


.. change::
    :tags: bug, environment
    :tickets: 1369

    Fixed issue where the method :meth:`.EnvironmentContext.get_x_argument`
    using the :paramref:`.EnvironmentContext.get_x_argument.as_dictionary`
    parameter would fail if an argument key were passed on the command line as
    a name alone, that is, without an equal sign ``=`` or a value. Behavior is
    repaired where this condition is detected and will return a blank string
    for the given key, consistent with the behavior where the ``=`` sign is
    present and no value.  Pull request courtesy Iuri de Silvio.

.. change::
    :tags: bug, autogenerate
    :tickets: 1370

    Fixed issue where the "unique" flag of an ``Index`` would not be maintained
    when generating downgrade migrations.  Pull request courtesy Iuri de
    Silvio.

.. change::
    :tags: bug, versioning
    :tickets: 1373

    Fixed bug in versioning model where a downgrade across a revision with two
    down revisions with one down revision depending on the other, would produce
    an erroneous state in the alembic_version table, making upgrades impossible
    without manually repairing the table.  Thanks much to Saif Hakim for
    the great work on this.

.. change::
    :tags: bug, typing
    :tickets: 1377

    Updated pep-484 typing to pass mypy "strict" mode, however including
    per-module qualifications for specific typing elements not yet complete.
    This allows us to catch specific typing issues that have been ongoing
    such as import symbols not properly exported.

.. changelog::
:version: 1.13.0
:released: December 1, 2023

text
.. change::
    :tags: bug, commands
    :tickets: 1234

    Fixed issue where the ``alembic check`` command did not function correctly
    with upgrade structures that have multiple, top-level elements, as are
    generated from the "multi-env" environment template.  Pull request courtesy
    Neil Williams.

.. change::
    :tags: usecase, operations
    :tickets: 1323

    Updated logic introduced in :ticket:`151` to allow ``if_exists`` and
    ``if_not_exists`` on index operations also on SQLAlchemy
    1.4 series. Previously this feature was mistakenly requiring
    the 2.0 series.

.. change::
    :tags: usecase
    :tickets: 1339

    Replaced ``python-dateutil`` with the standard library module
    `zoneinfo <https://docs.python.org/3.11/library/zoneinfo.html#module-zoneinfo>`_.
    This module was added in Python 3.9, so previous version will been
    to install the backport of it, available by installing the ``backports.zoneinfo``
    library. The ``alembic[tz]`` option has been updated accordingly.

.. change::
    :tags: installation, changed
    :tickets: 1359

    Alembic 1.13 now supports Python 3.8 and above.

.. change::
    :tags: bug, autogenerate
    :tickets: 1361

    Fixed autogenerate issue where ``create_table_comment()`` and
    ``drop_table_comment()`` rendering in a batch table modify would include
    the "table" and "schema" arguments, which are not accepted in batch as
    these are already part of the top level block.

.. change::
    :tags: bug, postgresql
    :tickets: 1321, 1327, 1356

    Additional fixes to PostgreSQL expression index compare feature.
    The compare now correctly accommodates casts and differences in
    spacing.
    Added detection logic for operation clauses inside the expression,
    skipping the compare of these expressions.
    To accommodate these changes the logic for the comparison of the
    indexes and unique constraints was moved to the dialect
    implementation, allowing greater flexibility.

.. changelog::
:version: 1.12.1
:released: October 26, 2023

text
.. change::
    :tags: bug, autogenerate, regression
    :tickets: 1329

    Fixed regression caused by :ticket:`879` released in 1.7.0 where the
    ".info" dictionary of ``Table`` would not render in autogenerate create
    table statements.  This can be useful for custom create table DDL rendering
    schemes so it is restored.

.. change::
    :tags: bug, typing
    :tickets: 1325

    Improved typing in the
    :paramref:`.EnvironmentContext.configure.process_revision_directives`
    callable to better indicate that the passed-in type is
    :class:`.MigrationScript`, not the :class:`.MigrationOperation` base class,
    and added typing to the example at :ref:`cookbook_no_empty_migrations` to
    illustrate.

.. change::
    :tags: bug, operations
    :tickets: 1335

    Repaired :class:`.ExecuteSQLOp` so that it can participate in "diff"
    operations; while this object is typically not present in a reflected
    operation stream, custom hooks may be adding this construct where it needs
    to have the correct ``to_diff_tuple()`` method.  Pull request courtesy
    Sebastian Bayer.

.. change::
    :tags: typing, bug
    :tickets: 1058, 1277

    Improved the ``op.execute()`` method to correctly accept the
    ``Executable`` type that is the same which is used in SQLAlchemy
    ``Connection.execute()``.  Pull request courtesy Mihail Milushev.

.. change::
    :tags: typing, bug
    :tickets: 930

    Improve typing of the revision parameter in various command functions.

.. change::
    :tags: typing, bug
    :tickets: 1266

    Properly type the :paramref:`.Operations.create_check_constraint.condition`
    parameter of :meth:`.Operations.create_check_constraint` to accept boolean
    expressions.

.. change::
    :tags: bug, postgresql
    :tickets: 1322

    Fixed autogen render issue where expressions inside of indexes for PG need
    to be double-parenthesized, meaning a single parens must be present within
    the generated ``text()`` construct.

.. change::
    :tags: usecase
    :tickets: 1304

    Alembic now accommodates for Sequence and Identity that support dialect kwargs.
    This is a change that will be added to SQLAlchemy v2.1.

.. changelog::
:version: 1.12.0
:released: August 31, 2023

text
.. change::
    :tags: bug, operations
    :tickets: 1300

    Added support for ``op.drop_constraint()`` to support PostgreSQL
    ``ExcludeConstraint`` objects, as well as other constraint-like objects
    that may be present in third party dialects, by resolving the ``type_``
    parameter to be ``None`` for this case.   Autogenerate has also been
    enhanced to exclude the ``type_`` parameter from rendering within this
    command when  ``type_`` is ``None``.  Pull request courtesy David Hills.



.. change::
    :tags: bug, commands
    :tickets: 1299

    Fixed issue where the ``revision_environment`` directive in ``alembic.ini``
    was ignored by the ``alembic merge`` command, leading to issues when other
    configurational elements depend upon ``env.py`` being invoked within the
    command.

.. change::
    :tags: bug, autogenerate
    :tickets: 1302

    Fixed issue where the ``ForeignKeyConstraint.match`` parameter would not be
    rendered in autogenerated migrations.  Pull request courtesy Asib
    Kamalsada.


.. change::
    :tags: usecase, autogenerate
    :tickets: 1248

    Change the default value of
    :paramref:`.EnvironmentContext.configure.compare_type` to ``True``.
    As Alembic's autogenerate for types was dramatically improved in
    version 1.4 released in 2020, the type comparison feature is now much
    more reliable so is now enabled by default.

.. change::
    :tags: feature, autogenerate
    :tickets: 1275

    Added new feature to the "code formatter" function which allows standalone
    executable tools to be run against code, without going through the Python
    interpreter.  Known as the ``exec`` runner, it complements the existing
    ``console_scripts`` runner by allowing non-Python tools such as ``ruff`` to
    be used.   Pull request courtesy Mihail Milushev.

    .. seealso::

        :ref:`post_write_hooks_config`

.. changelog::
:version: 1.11.3
:released: August 16, 2023

text
.. change::
    :tags: bug, autogenerate, postgresql
    :tickets: 1270

    Improved autogenerate compare of expression based indexes on PostgreSQL
    to produce fewer wrong detections.

.. change::
    :tags: bug, autogenerate
    :tickets: 1291

    Fixed issue with ``NULLS NOT DISTINCT`` detection in postgresql that
    would keep detecting changes in the index or unique constraint.

.. change::
    :tags: bug, commands
    :tickets: 1273

    Added ``encoding="locale"`` setting to the use of Python's
    ``ConfigParser.read()``, so that a warning is not generated when using the
    recently added Python feature ``PYTHONWARNDEFAULTENCODING`` specified in
    :pep:`597`. The encoding is passed as the ``"locale"`` string under Python
    3.10 and greater, which indicates that the system-level locale should be
    used, as was the case already here.  Pull request courtesy Kevin Kirsche.

.. changelog::
:version: 1.11.2
:released: August 4, 2023

text
.. change::
    :tags: usecase, typing
    :tickets: 1253

    Added typing to the default script mako templates.

.. change::
    :tags: usecase, autogenerate
    :tickets: 1248

    Added support in autogenerate for ``NULLS NOT DISTINCT`` in
    the PostgreSQL dialect.

.. change::
    :tags: bug
    :tickets: 1261

    Fixed format string logged when running a post write hook
    Pull request courtesy of Mathieu Dรฉfosse.

.. change::
    :tags: feature, operations
    :tickets: 151

    Added parameters if_exists and if_not_exists for index operations.
    Pull request courtesy of Max Adrian.

.. changelog::
:version: 1.11.1
:released: May 17, 2023

text
.. change::
    :tags: bug, autogenerate, regression
    :tickets: 1243, 1245

    As Alembic 1.11.0 is considered a major release (Alembic does not use
    semver, nor does its parent project SQLAlchemy; this has been
    :ref:`clarified <versioning_scheme>` in the documentation), change
    :ticket:`1130` modified calling signatures for most operations to consider
    all optional keyword parameters to be keyword-only arguments, to match what
    was always documented and generated by autogenerate. However, two of these
    changes were identified as possibly problematic without a more formal
    deprecation warning being emitted which were the ``table_name`` parameter
    to :meth:`.Operations.drop_index`, which was generated positionally by
    autogenerate prior to version 0.6.3 released in 2014, and ``type_`` in
    :meth:`.Operations.drop_constraint` and
    :meth:`.BatchOperations.drop_constraint`, which was documented positionally
    in one example in the batch documentation.

    These two signatures have been
    restored to allow those particular parameters to be passed positionally. A
    future change will include formal deprecation paths (with warnings) for
    these arguments where they will again become keyword-only in a future
    "Significant Minor" release.

.. change::
    :tags: bug, typing
    :tickets: 1246

    Fixed typing use of :class:`~sqlalchemy.schema.Column` and other
    generic SQLAlchemy classes.

.. change::
    :tags: bug, typing, regression
    :tickets: 1244

    Restored the output type of :meth:`.Config.get_section` to include
    ``Dict[str, str]`` as a potential return type, which had been changed to
    immutable ``Mapping[str, str]``. When a section is returned and the default
    is not used, a mutable dictionary is returned.

.. changelog::
:version: 1.11.0
:released: May 15, 2023

text
.. change::
    :tags: bug, batch
    :tickets: 1237

    Added placeholder classes for :class:`~.sqla.Computed` and
    :class:`~.sqla.Identity` when older 1.x SQLAlchemy versions are in use,
    namely prior to SQLAlchemy 1.3.11 when the :class:`~.sqla.Computed`
    construct was introduced. Previously these were set to None, however this
    could cause issues with certain codepaths that were using ``isinstance()``
    such as one within "batch mode".

.. change::
    :tags: bug, batch
    :tickets: 1221

    Correctly pass previously ignored arguments ``insert_before`` and
    ``insert_after`` in ``batch_alter_column``

.. change::
    :tags: change, py3k
    :tickets: 1130

    Argument signatures of Alembic operations now enforce keyword-only
    arguments as passed as keyword and not positionally, such as
    :paramref:`.Operations.create_table.schema`,
    :paramref:`.Operations.add_column.type_`, etc.

.. change::
    :tags: bug, postgresql
    :tickets: 1230

    Fix autogenerate issue with PostgreSQL :class:`.ExcludeConstraint`
    that included sqlalchemy functions. The function text was previously
    rendered as a plain string without surrounding with ``text()``.

.. change::
    :tags: bug, mysql, regression
    :tickets: 1240

    Fixed regression caused by :ticket:`1166` released in version 1.10.0 which
    caused MySQL unique constraints with multiple columns to not compare
    correctly within autogenerate, due to different sorting rules on unique
    constraints vs. indexes, which in MySQL are shared constructs.

.. change::
    :tags: misc
    :tickets: 1220

    Update code snippets within docstrings to use ``black`` code formatting.
    Pull request courtesy of James Addison.

.. change::
    :tags: bug, typing
    :tickets: 1093

    Updated stub generator script to also add stubs method definitions for the
    :class:`.Operations` class and the :class:`.BatchOperations` class obtained
    from :meth:`.Operations.batch_alter_table`. As part of this change, the
    class hierarchy of :class:`.Operations` and :class:`.BatchOperations` has
    been rearranged on top of a common base class :class:`.AbstractOperations`
    in order to type correctly, as :class:`.BatchOperations` uses different
    method signatures for operations than :class:`.Operations`.


.. change::
    :tags: bug, typing

    Repaired the return signatures for :class:`.Operations` that mostly
    return ``None``, and were erroneously referring to ``Optional[Table]``
    in many cases.

.. change::
    :tags: usecase, commands
    :tickets: 1109

    Added quiet option to the command line, using the ``-q/--quiet``
    option. This flag will prevent alembic from logging anything
    to stdout.

.. change::
    :tags: bug, autogenerate
    :tickets: 1178

    Modified the autogenerate implementation for comparing "server default"
    values from user-defined metadata to not apply any quoting to the value
    before comparing it to the server-reported default, except for within
    dialect-specific routines as needed. This change will affect the format of
    the server default as passed to the
    :paramref:`.EnvironmentContext.configure.compare_server_default` hook, as
    well as for third party dialects that implement a custom
    ``compare_server_default`` hook in their alembic impl, to be passed "as is"
    and not including additional quoting.   Custom implementations which rely
    on this quoting should adjust their approach based on observed formatting.

.. change::
    :tags: bug, api, autogenerate
    :tickets: 1235

    Fixed issue where :func:`.autogenerate.render_python_code` function did not
    provide a default value for the ``user_module_prefix`` variable, leading to
    ``NoneType`` errors when autogenerate structures included user-defined
    types. Added new parameter
    :paramref:`.autogenerate.render_python_code.user_module_prefix` to allow
    this to be set as well as to default to ``None``. Pull request courtesy
    tangkikodo.


.. change::
    :tags: usecase, asyncio
    :tickets: 1231

    Added :meth:`.AbstractOperations.run_async` to the operation module to
    allow running async functions in the ``upgrade`` or ``downgrade`` migration
    function when running alembic using an async dialect. This function will
    receive as first argument an
    :class:`~sqlalchemy.ext.asyncio.AsyncConnection` sharing the transaction
    used in the migration context.

.. changelog::
:version: 1.10.4
:released: April 24, 2023

text
.. change::
    :tags: postgresql, autogenerate, feature
    :tickets: 1213

    Added support for autogenerate comparison of indexes on PostgreSQL which
    include SQL sort option, such as ``ASC`` or ``NULLS FIRST``.
    The sort options are correctly detected only when defined using the
    sqlalchemy modifier functions, such as ``asc()`` or ``nulls_first()``,
    or the equivalent methods.
    Passing sort options inside the ``postgresql_ops`` dict is not supported.

.. change::
    :tags: bug, operations
    :tickets: 1215

    Fixed issue where using a directive such as ``op.create_foreign_key()`` to
    create a self-referential constraint on a single table where the same
    column were present on both sides (e.g. within a composite foreign key)
    would produce an error under SQLAlchemy 2.0 and a warning under SQLAlchemy
    1.4 indicating that a duplicate column were being added to a table.

.. changelog::
:version: 1.10.3
:released: April 5, 2023

text
.. change::
    :tags: bug, typing
    :tickets: 1191, 1201

    Fixed various typing issues observed with pyright, including issues
    involving the combination of :class:`.Function` and
    :meth:`.MigrationContext.begin_transaction`.

.. change::
    :tags: bug, autogenerate
    :tickets: 1212

    Fixed error raised by alembic when running autogenerate after removing
    a function based index.

.. changelog::
:version: 1.10.2
:released: March 8, 2023

text
.. change::
    :tags: bug, ops
    :tickets: 1196

    Fixed regression where Alembic would not run with older SQLAlchemy 1.3
    versions prior to 1.3.24 due to a missing symbol. Workarounds have been
    applied for older 1.3 versions.

.. changelog::
:version: 1.10.1
:released: March 6, 2023

text
.. change::
    :tags: bug, postgresql
    :tickets: 1184

    Fixed issue regarding PostgreSQL :class:`.ExcludeConstraint`, where
    constraint elements which made use of :func:`.literal_column` could not be
    rendered for autogenerate. Additionally, using SQLAlchemy 2.0.5 or greater,
    :func:`.text()` constructs are also supported within PostgreSQL
    :class:`.ExcludeConstraint` objects for autogenerate render. Pull request
    courtesy Jan Katins.

.. change::
    :tags: bug, batch, regression
    :tickets: 1195

    Fixed regression for 1.10.0 where :class:`.Constraint` objects were
    suddenly required to have non-None name fields when using batch mode, which
    was not previously a requirement.

.. changelog::
:version: 1.10.0
:released: March 5, 2023

text
.. change::
    :tags: bug, autogenerate
    :tickets: 1166

    Fixed issue in index detection where autogenerate change detection would
    consider indexes with the same columns but with different order as equal,
    while in general they are not equivalent in how a database will use them.

.. change::
    :tags: feature, revisioning
    :tickets: 760

    Recursive traversal of revision files in a particular revision directory is
    now supported, by indicating ``recursive_version_locations = true`` in
    alembic.ini. Pull request courtesy ostr00000.


.. change::
    :tags: bug, autogenerate, sqlite
    :tickets: 1165

    Fixed issue where indexes on SQLite which include SQL expressions would not
    compare correctly, generating false positives under autogenerate. These
    indexes are now skipped, generating a warning, in the same way that
    expression-based indexes on PostgreSQL are skipped and generate warnings
    when SQLAlchemy 1.x installations are in use. Note that reflection of
    SQLite expression-based indexes continues to not yet be supported under
    SQLAlchemy 2.0, even though PostgreSQL expression-based indexes have now
    been implemented.



.. change::
    :tags: bug, mssql
    :tickets: 1187

    Properly escape constraint name on SQL Server when dropping
    a column while specifying ``mssql_drop_default=True`` or
    ``mssql_drop_check=True`` or ``mssql_drop_foreign_key=True``.


.. change::
    :tags: usecase, autogenerate, postgresql

    Added support for autogenerate comparison of indexes on PostgreSQL which
    include SQL expressions, when using SQLAlchemy 2.0; the previous warning
    that such indexes were skipped are removed when the new functionality
    is in use.  When using SQLAlchemy versions prior to the 2.0 series,
    the indexes continue to be skipped with a warning.

.. changelog::
:version: 1.9.4
:released: February 16, 2023

text
.. change::
    :tags: bug, mssql
    :tickets: 1177

    Ongoing fixes for SQL Server server default comparisons under autogenerate,
    adjusting for SQL Server's collapsing of whitespace between SQL function
    arguments when reporting on a function-based server default, as well as its
    arbitrary addition of parenthesis within arguments; the approach has now
    been made more aggressive by stripping the two default strings to compare
    of all whitespace, parenthesis, and quoting characters.


.. change::
    :tags: bug, postgresql

    Fixed PostgreSQL server default comparison to handle SQL expressions
    sent as ``text()`` constructs, such as ``text("substring('name', 1, 3)")``,
    which previously would raise errors when attempting to run a server-based
    comparison.



.. change::
    :tags: bug, autogenerate
    :tickets: 1180

    Removed a mis-use of the
    :paramref:`.EnvironmentContext.configure.render_item` callable where the
    "server_default" renderer would be erroneously used within the server
    default comparison process, which is working against SQL expressions, not
    Python code.

.. change::
    :tags: bug, commands

    Fixed regression introduced in 1.7.0 where the "config" object passed to
    the template context when running the :func:`.merge` command
    programmatically failed to be correctly populated. Pull request courtesy
    Brendan Gann.

.. changelog::
:version: 1.9.3
:released: February 7, 2023

text
.. change::
    :tags: bug, autogenerate
    :tickets: 1167

    Fixed issue where rendering of user-defined types that then went onto use
    the ``.with_variant()`` method would fail to render, if using SQLAlchemy
    2.0's version of variants.

.. changelog::
:version: 1.9.2
:released: January 14, 2023

text
.. change::
    :tags: bug, typing
    :tickets: 1146, 1147

    Fixed typing definitions for :meth:`.EnvironmentContext.get_x_argument`.

    Typing stubs are now generated for overloaded proxied methods such as
    :meth:`.EnvironmentContext.get_x_argument`.

.. change::
    :tags: bug, autogenerate
    :tickets: 1152

    Fixed regression caused by :ticket:`1145` where the string transformations
    applied to server defaults caused expressions such as ``(getdate())`` to no
    longer compare as equivalent on SQL Server, others.

.. changelog::
:version: 1.9.1
:released: December 23, 2022

text
.. change::
    :tags: bug, autogenerate
    :tickets: 1145

    Fixed issue where server default compare would not work for string defaults
    that contained backslashes, due to mis-rendering of these values when
    comparing their contents.


.. change::
    :tags: bug, oracle

    Implemented basic server default comparison for the Oracle backend;
    previously, Oracle's formatting of reflected defaults prevented any
    matches from occurring.

.. change::
    :tags: bug, sqlite

    Adjusted SQLite's compare server default implementation to better handle
    defaults with or without parens around them, from both the reflected and
    the local metadata side.

.. change::
    :tags: bug, mssql

    Adjusted SQL Server's compare server default implementation to better
    handle defaults with or without parens around them, from both the reflected
    and the local metadata side.

.. changelog::
:version: 1.9.0
:released: December 15, 2022

text
.. change::
    :tags: feature, commands
    :tickets: 724

    Added new Alembic command ``alembic check``. This performs the widely
    requested feature of running an "autogenerate" comparison between the
    current database and the :class:`.MetaData` that's currently set up for
    autogenerate, returning an error code if the two do not match, based on
    current autogenerate settings. Pull request courtesy Nathan Louie.

    .. seealso::

        :ref:`alembic_check`


.. change::
    :tags: bug, tests

    Fixed issue in tox.ini file where changes in the tox 4.0 series to the
    format of "passenv" caused tox to not function correctly, in particular
    raising an error as of tox 4.0.6.

.. change::
    :tags: bug, typing
    :tickets: 1110

    Fixed typing issue where :paramref:`.revision.process_revision_directives`
    was not fully typed; additionally ensured all ``Callable`` and ``Dict``
    arguments to :meth:`.EnvironmentContext.configure` include parameters in
    the typing declaration.

    Additionally updated the codebase for Mypy 0.990 compliance.

.. changelog::
:version: 1.8.1
:released: July 13, 2022

text
.. change::
    :tags: bug, sqlite
    :tickets: 1065

    Fixed bug where the SQLite implementation of
    :meth:`.Operations.rename_table` would render an explicit schema name for
    both the old and new table name, which while is the standard ALTER syntax,
    is not accepted by SQLite's syntax which doesn't support a rename across
    schemas. In particular, the syntax issue would prevent batch mode from
    working for SQLite databases that made use of attached databases (which are
    treated as "schemas" in SQLAlchemy).

.. change::
    :tags: bug, batch
    :tickets: 1021

    Added an error raise for the condition where
    :meth:`.Operations.batch_alter_table` is used in ``--sql`` mode, where the
    operation requires table reflection, as is the case when running against
    SQLite without giving it a fixed ``Table`` object. Previously the operation
    would fail with an internal error.   To get a "move and copy" batch
    operation as a SQL script without connecting to a database,
    a ``Table`` object should be passed to the
    :paramref:`.Operations.batch_alter_table.copy_from` parameter so that
    reflection may be skipped.

.. changelog::
:version: 1.8.0
:released: May 31, 2022

text
.. change::
    :tags: feature, typing
    :tickets: 764

    :pep:`484` typing annotations have been added to the ``env.py`` and
    revision template files within migration templates. Pull request by Nikita
    Sobolev.

.. change::
    :tags: usecase, operations
    :tickets: 1037

    The ``op.drop_table()`` operation directive will now trigger the
    ``before_drop()`` and ``after_drop()`` DDL event hooks at the table level,
    which is similar to how the ``before_create()`` and ``after_create()``
    hooks are triggered by the ``op.create_table()`` directive. Note that as
    ``op.drop_table()`` accepts only a table name and optional schema name, the
    ``Table`` object received by the event will not have any information within
    it other than the table name and schema name.

.. change::
    :tags: installation, changed
    :tickets: 1025

    Alembic 1.8 now supports Python 3.7 and above.

.. change::
    :tags: changed, environment
    :tickets: 987

    The "Pylons" environment template has been removed as of Alembic 1.8. This
    template was based on the very old pre-Pyramid Pylons web framework which
    has been long superseded by Pyramid.

.. change::
    :tags: bug, revisioning
    :tickets: 1026

    Fixed issue where a downgrade using a relative revision would
    fail in case of multiple branches with a single effectively
    head due to interdependencies between revisions.

.. change::
  :tags: usecase, commands
  :tickets: 1027

  Added new token ``epoch`` to the ``file_template`` option, which will
  populate the integer epoch as determined by ``int(create_date.timestamp())``.
  Pull request courtesy Caio Carvalho.

.. change::
    :tags: bug, batch
    :tickets: 1034

    Fixed issue in batch mode where CREATE INDEX would not use a new column
    name in the case of a column rename.

.. changelog::
:version: 1.7.7
:released: March 14, 2022

text
.. change::
    :tags: bug, operations
    :tickets: 1004

    Fixed issue where using :meth:`.Operations.create_table` in conjunction
    with a :class:`.CheckConstraint` that referred to table-bound
    :class:`.Column` objects rather than string expressions would be added to
    the parent table potentially multiple times, resulting in an incorrect DDL
    sequence. Pull request courtesy Nicolas CANIART.

.. change::
    :tags: bug, environment
    :tickets: 986

    The ``logging.fileConfig()`` line in ``env.py`` templates, which is used
    to setup Python logging for the migration run, is now conditional on
    :attr:`.Config.config_file_name` not being ``None``.  Otherwise, the line
    is skipped as there is no default logging configuration present.


.. change::
    :tags: bug, mssql
    :tickets: 977

    Fixed bug where an :meth:`.Operations.alter_column` operation would change
    a "NOT NULL" column to "NULL" by emitting an ALTER COLUMN statement that
    did not specify "NOT NULL". (In the absence of "NOT NULL" T-SQL was
    implicitly assuming "NULL"). An :meth:`.Operations.alter_column` operation
    that specifies :paramref:`.Operations.alter_column.type` should also
    specify include either :paramref:`.Operations.alter_column.nullable` or
    :paramref:`.Operations.alter_column.existing_nullable` to inform Alembic as
    to whether the emitted DDL should include "NULL" or "NOT NULL"; a warning
    is now emitted if this is missing under this scenario.

.. changelog::
:version: 1.7.6
:released: February 1, 2022

text
.. change::
    :tags: bug, batch, regression
    :tickets: 982

    Fixed regression where usage of a ``with_variant()`` datatype in
    conjunction with the ``existing_type`` option of ``op.alter_column()``
    under batch mode would lead to an internal exception.

.. change::
    :tags: usecase, commands
    :tickets: 964

    Add a new command ``alembic ensure_version``, which will ensure that the
    Alembic version table is present in the target database, but does not
    alter its contents.  Pull request courtesy Kai Mueller.

.. change::
    :tags: bug, autogenerate

    Implemented support for recognizing and rendering SQLAlchemy "variant"
    types going forward into SQLAlchemy 2.0, where the architecture of
    "variant" datatypes will be changing.


.. change::
    :tags: bug, mysql, autogenerate
    :tickets: 968

    Added a rule to the MySQL impl so that the translation between JSON /
    LONGTEXT is accommodated by autogenerate, treating LONGTEXT from the server
    as equivalent to an existing JSON in the model.

.. change::
    :tags: mssql

    Removed a warning raised by SQLAlchemy when dropping constraints
    on MSSQL regarding statement caching.

.. changelog::
:version: 1.7.5
:released: November 11, 2021

text
.. change::
    :tags: bug, tests

    Adjustments to the test suite to accommodate for error message changes
    occurring as of SQLAlchemy 1.4.27.

.. changelog::
:version: 1.7.4
:released: October 6, 2021

text
.. change::
    :tags: bug, regression
    :tickets: 934

    Fixed a regression that prevented the use of post write hooks
    on python version lower than 3.9

.. change::
    :tags: bug, environment
    :tickets: 944

    Fixed issue where the :meth:`.MigrationContext.autocommit_block` feature
    would fail to function when using a SQLAlchemy engine using 2.0 future
    mode.

.. changelog::
:version: 1.7.3
:released: September 17, 2021

text
.. change::
    :tags: bug, mypy
    :tickets: 914

    Fixed type annotations for the "constraint_name" argument of operations
    ``create_primary_key()``, ``create_foreign_key()``.  Pull request courtesy
    TilmanK.

.. changelog::
:version: 1.7.2
:released: September 17, 2021

text
.. change::
    :tags: bug, typing
    :tickets: 900

    Added missing attributes from context stubs.

.. change::
    :tags: bug, mypy
    :tickets: 897

    Fixed an import in one of the .pyi files that was triggering an
    assertion error in some versions of mypy.

.. change::
    :tags: bug, regression, ops
    :tickets: 920

    Fixed issue where registration of custom ops was prone to failure due to
    the registration process running ``exec()`` on generated code that as of
    the 1.7 series includes pep-484 annotations, which in the case of end user
    code would result in name resolution errors when the exec occurs. The logic
    in question has been altered so that the annotations are rendered as
    forward references so that the ``exec()`` can proceed.

.. changelog::
:version: 1.7.1
:released: August 30, 2021

text
.. change::
    :tags: bug, installation
    :tickets: 893

    Corrected "universal wheel" directive in setup.cfg so that building a wheel
    does not target Python 2. The PyPi files index for 1.7.0 was corrected
    manually. Pull request courtesy layday.

.. change::
    :tags: bug, pep484
    :tickets: 895

    Fixed issue in generated .pyi files where default values for ``Optional``
    arguments were missing, thereby causing mypy to consider them as required.


.. change::
    :tags: bug, regression, batch
    :tickets: 896

    Fixed regression in batch mode due to :ticket:`883` where the "auto" mode
    of batch would fail to accommodate any additional migration directives
    beyond encountering an ``add_column()`` directive, due to a mis-application
    of the conditional logic that was added as part of this change, leading to
    "recreate" mode not being used in cases where it is required for SQLite
    such as for unique constraints.

.. changelog::
:version: 1.7.0
:released: August 30, 2021

text
.. change::
    :tags: bug, operations
    :tickets: 879

    Fixed regression due to :ticket:`803` where the ``.info`` and ``.comment``
    attributes of ``Table`` would be lost inside of the :class:`.DropTableOp`
    class, which when "reversed" into a :class:`.CreateTableOp` would then have
    lost these elements. Pull request courtesy Nicolas CANIART.


.. change::
    :tags: feature, environment
    :tickets: 842

    Enhance ``version_locations`` parsing to handle paths containing spaces.
    The new configuration option ``version_path_separator`` specifies the
    character to use when splitting the ``version_locations`` string. The
    default for new configurations is ``version_path_separator = os``,
    which will use ``os.pathsep`` (e.g., ``;`` on Windows).

.. change::
    :tags: installation, changed

    Alembic 1.7 now supports Python 3.6 and above; support for prior versions
    including Python 2.7 has been dropped.

.. change::
    :tags: bug, sqlite, batch
    :tickets: 883

    Batch "auto" mode will now select for "recreate" if the ``add_column()``
    operation is used on SQLite, and the column itself meets the criteria for
    SQLite where ADD COLUMN is not allowed, in this case a functional or
    parenthesized SQL expression or a ``Computed`` (i.e. generated) column.

.. change::
    :tags: changed, installation
    :tickets: 674

    Make the ``python-dateutil`` library an optional dependency.
    This library is only required if the ``timezone`` option
    is used in the Alembic configuration.
    An extra require named ``tz`` is available with
    ``pip install alembic[tz]`` to install it.

.. change::
    :tags: bug, commands
    :tickets: 856

    Re-implemented the ``python-editor`` dependency as a small internal
    function to avoid the need for external dependencies.

.. change::
    :tags: usecase, batch
    :tickets: 884

    Named CHECK constraints are now supported by batch mode, and will
    automatically be part of the recreated table assuming they are named. They
    also can be explicitly dropped using ``op.drop_constraint()``. For
    "unnamed" CHECK constraints, these are still skipped as they cannot be
    distinguished from the CHECK constraints that are generated by the
    ``Boolean`` and ``Enum`` datatypes.

    Note that this change may require adjustments to migrations that drop or
    rename columns which feature an associated named check constraint, such
    that an additional ``op.drop_constraint()`` directive should be added for
    that named constraint as there will no longer be an associated column
    for it; for the ``Boolean`` and ``Enum`` datatypes, an ``existing_type``
    keyword may be passed to ``BatchOperations.drop_constraint`` as well.

    .. seealso::

      :ref:`batch_schematype_constraints`

      :ref:`batch_check_constraints`


.. change::
    :tags: changed, installation
    :tickets: 885

    The dependency on ``pkg_resources`` which is part of ``setuptools`` has
    been removed, so there is no longer any runtime dependency on
    ``setuptools``. The functionality has been replaced with
    ``importlib.metadata`` and ``importlib.resources`` which are both part of
    Python std.lib, or via pypy dependency ``importlib-metadata`` for Python
    version < 3.8 and ``importlib-resources`` for Python version < 3.9
    (while importlib.resources was added to Python in 3.7, it did not include
    the "files" API until 3.9).

.. change::
    :tags: feature, tests
    :tickets: 855

    Created a "test suite" similar to the one for SQLAlchemy, allowing
    developers of third-party dialects to test their code against a set of
    Alembic tests that have been specially selected to exercise
    back-end database operations. At the time of release,
    third-party dialects that have adopted the Alembic test suite to verify
    compatibility include
    `CockroachDB <https://pypi.org/project/sqlalchemy-cockroachdb/>`_ and
    `SAP ASE (Sybase) <https://pypi.org/project/sqlalchemy-sybase/>`_.

.. change::
   :tags: bug, postgresql
   :tickets: 874

   Fixed issue where usage of the PostgreSQL ``postgresql_include`` option
   within a :meth:`.Operations.create_index` would raise a KeyError, as the
   additional column(s) need to be added to the table object used by the
   construct internally. The issue is equivalent to the SQL Server issue fixed
   in :ticket:`513`. Pull request courtesy Steven Bronson.

.. change::
    :tags: feature, general

    pep-484 type annotations have been added throughout the library.
    Additionally, stub .pyi files have been added for the "dynamically"
    generated Alembic modules ``alembic.op`` and ``alembic.config``, which
    include complete function signatures and docstrings, so that the functions
    in these namespaces will have both IDE support (vscode, pycharm, etc) as
    well as support for typing tools like Mypy. The files themselves are
    statically generated from their source functions within the source tree.

.. changelog::
:version: 1.6.5
:released: May 27, 2021

text
.. change::
    :tags: bug, autogenerate
    :tickets: 849

    Fixed issue where dialect-specific keyword arguments within the
    :class:`.DropIndex` operation directive would not render in the
    autogenerated Python code. As support was improved for adding dialect
    specific arguments to directives as part of :ticket:`803`, in particular
    arguments such as "postgresql_concurrently" which apply to the actual
    create/drop of the index, support was needed for these to render even in a
    drop index operation. Pull request courtesy Jet Zhou.

.. changelog::
:version: 1.6.4
:released: May 24, 2021

text
.. change::
    :tags: bug, regression, op directives
    :tickets: 848

    Fixed regression caused by just fixed :ticket:`844` that scaled back the
    filter for ``unique=True/index=True`` too far such that these directives no
    longer worked for the ``op.create_table()`` op, this has been fixed.

.. changelog::
:version: 1.6.3
:released: May 21, 2021

text
.. change::
    :tags: bug, regression, autogenerate
    :tickets: 844

    Fixed 1.6-series regression where ``UniqueConstraint`` and to a lesser
    extent ``Index`` objects would be doubled up in the generated model when
    the ``unique=True`` / ``index=True`` flags were used.

.. change::
    :tags: bug, autogenerate
    :tickets: 839

    Fixed a bug where paths defined in post-write hook options
    would be wrongly escaped in non posix environment (Windows).

.. change::
    :tags: bug, regression, versioning
    :tickets: 843

    Fixed regression where a revision file that contained its own down revision
    as a dependency would cause an endless loop in the traversal logic.

.. changelog::
:version: 1.6.2
:released: May 6, 2021

text
.. change::
    :tags: bug, versioning, regression
    :tickets: 839

    Fixed additional regression nearly the same as that of :ticket:`838` just
    released in 1.6.1 but within a slightly different codepath, where "alembic
    downgrade head" (or equivalent) would fail instead of iterating no
    revisions.

.. changelog::
:version: 1.6.1
:released: May 6, 2021

text
.. change::
    :tags: bug, versioning, regression
    :tickets: 838

    Fixed regression in new revisioning traversal where "alembic downgrade
    base" would fail if the database itself were clean and unversioned;
    additionally repairs the case where downgrade would fail if attempting
    to downgrade to the current head that is already present.

.. changelog::
:version: 1.6.0
:released: May 3, 2021

text
.. change::
    :tags: bug, autogenerate
    :tickets: 803

    Refactored the implementation of :class:`.MigrateOperation` constructs such
    as :class:`.CreateIndexOp`, :class:`.CreateTableOp`, etc. so that they no
    longer rely upon maintaining a persistent version of each schema object
    internally; instead, the state variables of each operation object will be
    used to produce the corresponding construct when the operation is invoked.
    The rationale is so that environments which make use of
    operation-manipulation schemes such as those discussed in
    :ref:`autogen_rewriter` are better supported, allowing end-user code to
    manipulate the public attributes of these objects which will then be
    expressed in the final output, an example is
    ``some_create_index_op.kw["postgresql_concurrently"] = True``.

    Previously, these objects when generated from autogenerate would typically
    hold onto the original, reflected element internally without honoring the
    other state variables of each construct, preventing the public API from
    working.



.. change::
    :tags: bug, environment
    :tickets: 829

    Fixed regression caused by the SQLAlchemy 1.4/2.0 compatibility switch
    where calling ``.rollback()`` or ``.commit()`` explicitly within the
    ``context.begin_transaction()`` context manager would cause it to fail when
    the block ended, as it did not expect that the transaction was manually
    closed.

.. change::
    :tags: bug, autogenerate
    :tickets: 827

    Improved the rendering of ``op.add_column()`` operations when adding
    multiple columns to an existing table, so that the order of these
    statements matches the order in which the columns were declared in the
    application's table metadata. Previously the added columns were being
    sorted alphabetically.


.. change::
    :tags: feature, autogenerate
    :tickets: 819

    Fix the documentation regarding the default command-line argument position of
    the revision script filename within the post-write hook arguments. Implement a
    ``REVISION_SCRIPT_FILENAME`` token, enabling the position to be changed. Switch
    from ``str.split()`` to ``shlex.split()`` for more robust command-line argument
    parsing.

.. change::
    :tags: feature
    :tickets: 822

    Implement a ``.cwd`` (current working directory) suboption for post-write hooks
    (of type ``console_scripts``). This is useful for tools like pre-commit, which
    rely on the working directory to locate the necessary config files. Add
    pre-commit as an example to the documentation. Minor change: rename some variables
    from ticket #819 to improve readability.

.. change::
    :tags: bug, versioning
    :tickets: 765, 464

    The algorithm used for calculating downgrades/upgrades/iterating
    revisions has been rewritten, to resolve ongoing issues of branches
    not being handled consistently particularly within downgrade operations,
    as well as for overall clarity and maintainability.  This change includes
    that a deprecation warning is emitted if an ambiguous command such
    as "downgrade -1" when multiple heads are present is given.

    In particular, the change implements a long-requested use case of allowing
    downgrades of a single branch to a branchpoint.

    Huge thanks to Simon Bowly for their impressive efforts in successfully
    tackling this very difficult problem.

.. change::
    :tags: bug, batch
    :tickets: 799

    Added missing ``batch_op.create_table_comment()``,
    ``batch_op.drop_table_comment()`` directives to batch ops.

.. changelog::
:version: 1.5.8
:released: March 23, 2021

text
.. change::
    :tags: bug, environment
    :tickets: 816

    Fixed regression caused by SQLAlchemy 1.4 where the "alembic current"
    command would fail due to changes in the ``URL`` object.

.. changelog::
:version: 1.5.7
:released: March 11, 2021

text
.. change::
    :tags: bug, autogenerate
    :tickets: 813

    Adjusted the recently added
    :paramref:`.EnvironmentContext.configure.include_name` hook to accommodate
    for additional object types such as "views" that don't have a parent table,
    to support third party recipes and extensions. Pull request courtesy Oliver
    Rice.

.. changelog::
:version: 1.5.6
:released: March 5, 2021

text
.. change::
    :tags: bug, mssql, operations
    :tickets: 812

    Fixed bug where the "existing_type" parameter, which the MSSQL dialect
    requires in order to change the nullability of a column in the absence of
    also changing the column type, would cause an ALTER COLUMN operation to
    incorrectly render a second ALTER statement without the nullability if a
    new type were also present, as the MSSQL-specific contract did not
    anticipate all three of "nullability", ``"type_"`` and "existing_type" being
    sent at the same time.


.. change::
    :tags: template
    :ticket: 805

    Add async template to Alembic to bootstrap environments that use
    async DBAPI. Updated the cookbook to include a migration guide
    on how to adapt an existing environment for use with DBAPI drivers.

.. changelog::
:version: 1.5.5
:released: February 20, 2021

text
.. change::
    :tags: bug

    Adjusted the use of SQLAlchemy's ".copy()" internals to use "._copy()"
    for version 1.4.0, as this method is being renamed.

.. change::
    :tags: bug, environment
    :tickets: 797

    Added new config file option ``prepend_sys_path``, which is a series of
    paths that will be prepended to sys.path; the default value in newly
    generated alembic.ini files is ".".  This fixes a long-standing issue
    where for some reason running the alembic command line would not place the
    local "." path in sys.path, meaning an application locally present in "."
    and importable through normal channels, e.g. python interpreter, pytest,
    etc. would not be located by Alembic, even though the ``env.py`` file is
    loaded relative to the current path when ``alembic.ini`` contains a
    relative path. To enable for existing installations, add the option to the
    alembic.ini file as follows::

      # sys.path path, will be prepended to sys.path if present.
      # defaults to the current working directory.
      prepend_sys_path = .

    .. seealso::

        :ref:`installation` - updated documentation reflecting that local
        installation of the project is not necessary if running the Alembic cli
        from the local path.

.. changelog::
:version: 1.5.4
:released: February 3, 2021

text
.. change::
    :tags: bug, versioning
    :tickets: 789

    Fixed bug in versioning model where a downgrade across a revision with a
    dependency on another branch, yet an ancestor is also dependent on that
    branch, would produce an erroneous state in the alembic_version table,
    making upgrades impossible without manually repairing the table.

.. changelog::
:version: 1.5.3
:released: January 29, 2021

text
.. change::
    :tags: bug, autogenerate
    :tickets: 786

    Changed the default ordering of "CREATE" and "DROP" statements indexes and
    unique constraints within the autogenerate process, so that for example in
    an upgrade() operation, a particular index or constraint that is to be
    replaced such as for a casing convention change will not produce any naming
    conflicts. For foreign key constraint objects, this is already how
    constraints are ordered, and for table objects, users would normally want
    to use :meth:`.Operations.rename_table` in any case.

.. change::
    :tags: bug, autogenerate, mssql
    :tickets: 787

    Fixed assorted autogenerate issues with SQL Server:

    * ignore default reflected identity on primary_key columns
    * improve server default comparison

.. change::
    :tags: bug, mysql, autogenerate
    :tickets: 788

    Fixed issue where autogenerate rendering of ``op.alter_column()`` would
    fail to include MySQL ``existing_nullable=False`` if the column were part
    of a primary key constraint within the table metadata.

.. changelog::
:version: 1.5.2
:released: January 20, 2021

text
.. change::
    :tags: bug, versioning, regression
    :tickets: 784

    Fixed regression where new "loop detection" feature introduced in
    :ticket:`757` produced false positives for revision names that have
    overlapping substrings between revision number and down revision and/or
    dependency, if the downrev/dependency were not in sequence form.

.. change::
    :tags: bug, environment
    :tickets: 782

    Fixed regression where Alembic would fail to create a transaction properly
    if the :class:`sqlalchemy.engine.Connection` were a so-called "branched"
    connection, that is, one where the ``.connect()`` method had been called to
    create a "sub" connection.

.. changelog::
:version: 1.5.1
:released: January 19, 2021

text
.. change::
    :tags: bug, installation, commands
    :tickets: 780

    Fixed installation issue where the "templates" directory was not being
    installed, preventing commands like "list_templates" and "init" from
    working.

.. changelog::
:version: 1.5.0
:released: January 18, 2021

text
.. change::
    :tags: usecase, operations
    :tickets: 730

    Added support for rendering of "identity" elements on
    :class:`.Column` objects, supported in SQLAlchemy via
    the :class:`.Identity` element introduced in version 1.4.

    Adding columns with identity is supported on PostgreSQL,
    MSSQL and Oracle. Changing the identity options or removing
    it is supported only on PostgreSQL and Oracle.

.. change::
    :tags: changed, environment

    To accommodate SQLAlchemy 1.4 and 2.0, the migration model now no longer
    assumes that the SQLAlchemy Connection will autocommit an individual
    operation.   This essentially means that for databases that use
    non-transactional DDL (pysqlite current driver behavior, MySQL), there is
    still a BEGIN/COMMIT block that will surround each individual migration.
    Databases that support transactional DDL should continue to have the
    same flow, either per migration or per-entire run, depending on the
    value of the :paramref:`.Environment.configure.transaction_per_migration`
    flag.


.. change::
    :tags: changed, environment

    A :class:`.CommandError` is raised if a ``sqlalchemy.engine.Engine`` is
    passed to the :meth:`.MigrationContext.configure` method instead of a
    ``sqlalchemy.engine.Connection`` object.  Previously, this would be a
    warning only.

.. change::
    :tags: bug, operations
    :tickets: 753

    Modified the ``add_column()`` operation such that the ``Column`` object in
    use is shallow copied to a new instance if that ``Column`` is already
    attached to a ``table()`` or ``Table``. This accommodates for the change
    made in SQLAlchemy issue #5618 which prohibits a ``Column`` from being
    associated with multiple ``table()`` objects. This resumes support for
    using a ``Column`` inside of an Alembic operation that already refers to a
    parent ``table()`` or ``Table`` as well as allows operation objects just
    autogenerated to work.

.. change::
    :tags: feature, autogenerate
    :tickets: 650

    Added new hook :paramref:`.EnvironmentContext.configure.include_name`,
    which complements the
    :paramref:`.EnvironmentContext.configure.include_object` hook by providing
    a means of preventing objects of a certain name from being autogenerated
    **before** the SQLAlchemy reflection process takes place, and notably
    includes explicit support for passing each schema name when
    :paramref:`.EnvironmentContext.configure.include_schemas` is set to True.
    This is most important especially for environments that make use of
    :paramref:`.EnvironmentContext.configure.include_schemas` where schemas are
    actually databases (e.g. MySQL) in order to prevent reflection sweeps of
    the entire server.

    .. seealso::

        :ref:`autogenerate_include_hooks` - new documentation section

.. change::
    :tags: removed, autogenerate

    The long deprecated
    :paramref:`.EnvironmentContext.configure.include_symbol` hook is removed.
    The  :paramref:`.EnvironmentContext.configure.include_object`
    and  :paramref:`.EnvironmentContext.configure.include_name`
    hooks both achieve the goals of this hook.


.. change::
    :tags: bug, autogenerate
    :tickets: 721

    Added rendering for the ``Table.prefixes`` element to autogenerate so that
    the rendered Python code includes these directives. Pull request courtesy
    Rodrigo Ce Moretto.

.. change::
    :tags: bug, batch
    :tickets: 761

    Added missing "create comment" feature for columns that are altered in
    batch migrations.


.. change::
    :tags: changed
    :tickets: 748

    Alembic 1.5.0 now supports **Python 2.7 and Python 3.6 and above**, as well
    as **SQLAlchemy 1.3.0 and above**.  Support is removed for Python 3
    versions prior to 3.6 and SQLAlchemy versions prior to the 1.3 series.

.. change::
    :tags: bug, batch
    :tickets: 773

    Made an adjustment to the PostgreSQL dialect to allow it to work more
    effectively in batch mode, where a datatype like Boolean or non-native Enum
    that may have embedded rules to generate CHECK constraints will be more
    correctly handled in that these constraints usually will not have been
    generated on the PostgreSQL backend; previously it would inadvertently
    assume they existed unconditionally in a special PG-only "drop constraint"
    step.


.. change::
    :tags: feature, versioning
    :tickets: 757

    The revision tree is now checked for cycles and loops between revision
    files when the revision environment is loaded up.  Scenarios such as a
    revision pointing to itself, or a revision that can reach itself via a
    loop, are handled and will raise the :class:`.CycleDetected` exception when
    the environment is loaded (expressed from the Alembic commandline as a
    failure message and nonzero return code). Previously, these situations were
    silently ignored up front, and the behavior of revision traversal would
    either be silently incorrect, or would produce errors such as
    :class:`.RangeNotAncestorError`.  Pull request courtesy Koichiro Den.


.. change::
    :tags: usecase, commands

    Add ``__main__.py`` file to alembic package to support invocation
    with ``python -m alembic``.

.. change::
    :tags: removed, commands

    Removed deprecated ``--head_only`` option to the ``alembic current``
    command

.. change::
    :tags: removed, operations

    Removed legacy parameter names from operations, these have been emitting
    warnings since version 0.8.  In the case that legacy version files have not
    yet been updated, these can be modified directly in order to maintain
    compatibility:

    * :meth:`.Operations.drop_constraint` - "type" (use ``"type_"``) and "name"
      (use "constraint_name")

    * :meth:`.Operations.create_primary_key` - "cols" (use "columns") and
      "name" (use "constraint_name")

    * :meth:`.Operations.create_unique_constraint` - "name" (use
      "constraint_name"), "source" (use "table_name") and "local_cols" (use
      "columns")

    * :meth:`.Operations.batch_create_unique_constraint` - "name" (use
      "constraint_name")

    * :meth:`.Operations.create_foreign_key` - "name" (use "constraint_name"),
      "source" (use "source_table"), "referent" (use "referent_table")

    * :meth:`.Operations.batch_create_foreign_key` - "name" (use
      "constraint_name"), "referent" (use "referent_table")

    * :meth:`.Operations.create_check_constraint` - "name" (use
      "constraint_name"), "source" (use "table_name")

    * :meth:`.Operations.batch_create_check_constraint` - "name" (use
      "constraint_name")

    * :meth:`.Operations.create_index` - "name" (use "index_name")

    * :meth:`.Operations.drop_index` - "name" (use "index_name"), "tablename"
      (use "table_name")

    * :meth:`.Operations.batch_drop_index` - "name" (use "index_name"),

    * :meth:`.Operations.create_table` - "name" (use "table_name")

    * :meth:`.Operations.drop_table` - "name" (use "table_name")

    * :meth:`.Operations.alter_column` - "name" (use "new_column_name")

.. changelog::
:version: 1.4.3
:released: September 11, 2020

text
.. change::
    :tags: bug, sqlite, batch
    :tickets: 711

    Added support to drop named CHECK constraints that are specified as part of
    a column, rather than table wide.  Previously, only constraints associated
    with the table were considered.

.. change::
    :tags: bug, ops, mysql
    :tickets: 736

    Fixed issue where the MySQL dialect would not correctly render the server
    default of a column in an alter operation, if the operation were
    programmatically generated from an autogenerate pass as it would not
    accommodate for the full structure of the DefaultClause construct.

.. change::
    :tags: bug, sqlite, batch
    :tickets: 697

    Fixed issue where the CAST applied to a JSON column when copying a SQLite
    table during batch mode would cause the data to be lost, as SQLite's CAST
    with JSON appears to convert the data to the value "0". The CAST is now
    skipped in a dialect-specific manner, including for JSON columns on SQLite.
    Pull request courtesy Sebastiรกn Ramรญrez.

.. change::
    :tags: bug, commands
    :tickets: 694

    The ``alembic current`` command no longer creates an ``alembic_version``
    table in the database if one does not exist already, returning no version
    as the current version. This allows checking for migrations in parallel
    without introducing race conditions.  Pull request courtesy Nikolay
    Edigaryev.


.. change::
    :tags: bug, batch

    Fixed issue where columns in a foreign-key referenced table would be
    replaced with null-type columns during a batch operation; while this did
    not generally have any side effects, it could theoretically impact a batch
    operation that also targets that table directly and also would interfere
    with future changes to the ``.append_column()`` method to disallow implicit
    replacement of columns.

.. change::
   :tags: bug, mssql
   :tickets: 716

   Fixed issue where the ``mssql_drop_foreign_key=True`` flag on
   ``op.drop_column`` would lead to incorrect syntax error due to a typo in the
   SQL emitted, same typo was present in the test as well so it was not
   detected. Pull request courtesy Oleg Shigorin.

.. changelog::
:version: 1.4.2
:released: March 19, 2020

text
.. change::
    :tags: usecase, autogenerate
    :tickets: 669

    Adjusted autogen comparison to accommodate for backends that support
    computed column reflection, dependent on SQLAlchemy version 1.3.16 or
    higher. This emits a warning if the SQL expression inside of a
    :class:`.Computed` value changes between the metadata and the database, as
    these expressions can't be changed without dropping and recreating the
    column.


.. change::
    :tags: bug, tests
    :tickets: 668

    Fixed an issue that prevented the test suite from running with the
    recently released py.test 5.4.0.


.. change::
    :tags: bug, autogenerate, mysql
    :tickets: 671

    Fixed more false-positive failures produced by the new "compare type" logic
    first added in :ticket:`605`, particularly impacting MySQL string types
    regarding flags such as "charset" and "collation".

.. change::
    :tags: bug, op directives, oracle
    :tickets: 670

    Fixed issue in Oracle backend where a table RENAME with a schema-qualified
    name would include the schema in the "to" portion, which is rejected by
    Oracle.

.. changelog::
:version: 1.4.1
:released: March 1, 2020

text
.. change::
    :tags: bug, autogenerate
    :tickets: 661

    Fixed regression caused by the new "type comparison" logic introduced in
    1.4 as part of :ticket:`605` where comparisons of MySQL "unsigned integer"
    datatypes would produce false positives, as the regular expression logic
    was not correctly parsing the "unsigned" token when MySQL's default display
    width would be returned by the database.  Pull request courtesy Paul
    Becotte.

.. change::
    :tags: bug, environment
    :tickets: 663

    Error message for "path doesn't exist" when loading up script environment
    now displays the absolute path.  Pull request courtesy Rowan Hart.

.. change::
    :tags: bug, autogenerate
    :tickets: 654

    Fixed regression in 1.4.0 due to :ticket:`647` where unique constraint
    comparison with mixed case constraint names while not using a naming
    convention would produce false positives during autogenerate.

.. change::
    :tags: bug, environment

    The check for matched rowcount when the alembic_version table is updated or
    deleted from is now conditional based on whether or not the dialect
    supports the concept of "rowcount" for UPDATE or DELETE rows matched.  Some
    third party dialects do not support this concept.  Pull request courtesy Ke
    Zhu.

.. change::
    :tags: bug, operations
    :tickets: 655

    Fixed long-standing bug where an inline column CHECK constraint would not
    be rendered within an "ADD COLUMN" operation.  The DDL compiler is now
    consulted for inline constraints within the :meth:`.Operations.add_column`
    method as is done for regular CREATE TABLE operations.

.. changelog::
:version: 1.4.0
:released: February 4, 2020

text
.. change::
    :tags: change

    The internal inspection routines no longer use SQLAlchemy's
    ``Inspector.from_engine()`` method, which is expected to be deprecated in
    1.4.  The ``inspect()`` function is now used.


.. change::
    :tags: bug, autogenerate
    :tickets: 647

    Adjusted the unique constraint comparison logic in a similar manner as that
    of :ticket:`421` did for indexes in order to take into account SQLAlchemy's
    own truncation of long constraint names when a naming convention is in use.
    Without this step, a name that is truncated by SQLAlchemy based on a unique
    constraint naming convention or hardcoded name will not compare properly.


.. change::
    :tags: feature, batch
    :tickets: 640

    Added new parameters :paramref:`.BatchOperations.add_column.insert_before`,
    :paramref:`.BatchOperations.add_column.insert_after` which provide for
    establishing the specific position in which a new column should be placed.
    Also added :paramref:`.Operations.batch_alter_table.partial_reordering`
    which allows the complete set of columns to be reordered when the new table
    is created.   Both operations apply only to when batch mode is recreating
    the whole table using ``recreate="always"``.  Thanks to Marcin Szymanski
    for assistance with the implementation.

.. change::
    :tags: usecase, environment
    :tickets: 648

    Moved the use of the ``__file__`` attribute at the base of the Alembic
    package into the one place that it is specifically needed, which is when
    the config attempts to locate the template directory. This helps to allow
    Alembic to be fully importable in environments that are using Python
    memory-only import schemes.  Pull request courtesy layday.

.. change::
    :tags: bug, autogenerate
    :tickets: 605

    A major rework of the "type comparison" logic is in place which changes the
    entire approach by which column datatypes are compared.  Types are now
    compared based on the DDL string generated by the metadata type vs. the
    datatype reflected from the database.  This means we compare types based on
    what would actually render and additionally if elements of the types change
    like string length, those changes are detected as well.  False positives
    like those generated between SQLAlchemy Boolean and MySQL TINYINT should
    also be resolved.   Thanks very much to Paul Becotte  for lots of hard work
    and patience on this one.

    .. note:: *updated* - this change also removes support for the
       ``compare_against_backend`` SQLAlchemy type hook.

    .. seealso::

        :ref:`autogenerate_detects` - updated comments on type comparison

.. changelog::
:version: 1.3.3
:released: January 22, 2020

text
.. change::
    :tags: bug, postgresql
    :tickets: 637

    Fixed issue where COMMENT directives for PostgreSQL failed to correctly
    include an explicit schema name, as well as correct quoting rules for
    schema, table, and column names.  Pull request courtesy Matthew Sills.

.. change::
    :tags: usecase, operations
    :tickets: 624

    Added support for rendering of "computed" elements on :class:`.Column`
    objects, supported in SQLAlchemy via the new :class:`.Computed` element
    introduced in version 1.3.11. Pull request courtesy Federico Caselli.

    Note that there is currently no support for ALTER COLUMN to add, remove, or
    modify the "GENERATED ALWAYS AS" element from a column;  at least for
    PostgreSQL, it does not seem to be supported by the database. Additionally,
    SQLAlchemy does not currently reliably reflect the "GENERATED ALWAYS AS"
    phrase from an existing column, so there is also no autogenerate support
    for addition or removal of the :class:`.Computed` element to or from an
    existing column, there is only support for adding new columns that include
    the :class:`.Computed` element.  In the case that the :class:`.Computed`
    element is removed from the :class:`.Column` object in the table metadata,
    PostgreSQL and Oracle currently reflect the "GENERATED ALWAYS AS"
    expression as the "server default" which will produce an op that tries to
    drop the element as a default.

.. changelog::
:version: 1.3.2
:released: December 16, 2019

text
.. change::
    :tags: bug, api, autogenerate
    :tickets: 635

    Fixed regression introduced by :ticket:`579` where server default rendering
    functions began to require a dialect implementation, however the
    :func:`.render_python_code` convenience function did not include one, thus
    causing the function to fail when used in a server default context.  The
    function now accepts a migration context argument and also creates one
    against the default dialect if one is not provided.

.. changelog::
:version: 1.3.1
:released: November 13, 2019

text
.. change::
    :tags: bug, mssql
    :tickets: 621

    Fixed bug in MSSQL dialect where the drop constraint execution steps used
    to remove server default or implicit foreign key constraint failed to take
    into account the schema name of the target table.

.. changelog::
:version: 1.3.0
:released: October 31, 2019

text
.. change::
    :tags: feature, command
    :tickets: 608

    Added support for ALEMBIC_CONFIG environment variable,
    refers to the location of the alembic configuration script
    in lieu of using the -c command line option.


.. change::
    :tags: bug, autogenerate
    :tickets: 131

    Fixed bug in new Variant autogenerate where the order of the arguments to
    Variant were mistakenly reversed.

.. change::
    :tags: change, compatibility

    Some internal modifications have been made to how the names of indexes and
    unique constraints work to make use of new functions added in SQLAlchemy
    1.4, so that SQLAlchemy has more flexibility over how naming conventions
    may be applied to these objects.

.. changelog::
:version: 1.2.1
:released: September 24, 2019

text
.. change::
    :tags: bug, command
    :tickets: 601

    Reverted the name change of the "revisions" argument to
    :func:`.command.stamp` to "revision" as apparently applications are
    calling upon this argument as a keyword name.  Pull request courtesy
    Thomas Bechtold.  Special translations are also added to the command
    line interface so that it is still known as "revisions" in the CLI.

.. change::
    :tags: bug, tests
    :tickets: 592

    Removed the "test requirements" from "setup.py test", as this command now
    only emits a removal error in any case and these requirements are unused.

.. changelog::
:version: 1.2.0
:released: September 20, 2019

text
.. change::
    :tags: feature, command
    :tickets: 473

    Added new ``--purge`` flag to the ``alembic stamp`` command, which will
    unconditionally erase the version table before stamping anything.  This is
    useful for development where non-existent version identifiers might be left
    within the table.  Additionally, ``alembic.stamp`` now supports a list of
    revision identifiers, which are intended to allow setting up multiple heads
    at once.  Overall handling of version identifiers within the
    ``alembic.stamp`` command has been improved with many new tests and
    use cases added.

.. change::
    :tags: bug, autogenerate
    :tickets: 550

    Improved the Python rendering of a series of migration operations such that
    a single "pass" is rendered for a :class:`.UpgradeOps` or
    :class:`.DowngradeOps` based on if no lines of Python code actually
    rendered under the operation, rather than whether or not sub-directives
    exist. Removed extra "pass" lines that would generate from the
    :class:`.ModifyTableOps` directive so that these aren't duplicated under
    operation rewriting scenarios.


.. change::
    :tags: feature, runtime
    :tickets: 123

    Added new feature :meth:`.MigrationContext.autocommit_block`, a special
    directive which will provide for a non-transactional block inside of a
    migration script. The feature requires that: the database driver
    (e.g. DBAPI) supports the AUTOCOMMIT isolation mode.  The directive
    also necessarily needs to COMMIT the existing transaction in progress
    in order to enter autocommit mode.

    .. seealso::

        :meth:`.MigrationContext.autocommit_block`

.. change::
    :tags: change: py3k

    Python 3.4 support is dropped, as the upstream tooling (pip, mysqlclient)
    etc are already dropping support for Python 3.4, which itself is no longer
    maintained.

.. change::
    :tags: usecase, autogenerate
    :tickets: 518

    Added autogenerate support for :class:`.Column` objects that have
    dialect-specific ``**kwargs``, support first added in SQLAlchemy 1.3.
    This includes SQLite "on conflict" as well as options used by some
    third party dialects.

.. change::
    :tags: usecase, autogenerate
    :tickets: 131

    Added rendering for SQLAlchemy ``Variant`` datatypes, which render as the
    base type plus one or more ``.with_variant()`` method calls.


.. change::
    :tags: usecase, commands
    :tickets: 534

    Made the command interface revision lookup behavior more strict in that an
    Alembic revision number is only resolved based on a partial match rules if
    it has at least four characters, to prevent simple typographical issues
    from inadvertently  running migrations.

 .. change::
    :tags: feature, commands
    :tickets: 307

    Added "post write hooks" to revision generation.  These allow custom logic
    to run after a revision Python script is generated, typically for the
    purpose of running code formatters such as "Black" or "autopep8", but may
    be used for any arbitrary post-render hook as well, including custom Python
    functions or scripts.  The hooks are enabled by providing a
    ``[post_write_hooks]`` section in the alembic.ini file.  A single hook
    is provided which runs an arbitrary Python executable on the newly
    generated revision script, which can be configured to run code formatters
    such as Black; full examples are included in the documentation.

    .. seealso::

        :ref:`post_write_hooks`


.. change::
    :tags: feature, environment
    :tickets: 463

    Added new flag ``--package`` to ``alembic init``.  For environments where
    the Alembic migration files and such are within the package tree and
    importable as modules, this flag can be specified which will add the
    additional ``__init__.py`` files in the version location and the
    environment location.

.. change::
    :tags: bug, autogenerate
    :tickets: 549

    Fixed bug where rendering of comment text for table-level comments  within
    :meth:`.Operations.create_table_comment` and
    :meth:`.Operations.drop_table_comment` was not properly quote-escaped
    within rendered Python code for autogenerate.

.. change::
    :tags: bug, autogenerate
    :tickets: 505

    Modified the logic of the :class:`.Rewriter` object such that it keeps a
    memoization of which directives it has processed, so that it can ensure it
    processes a particular directive only once, and additionally fixed
    :class:`.Rewriter` so that it functions correctly for multiple-pass
    autogenerate schemes, such as the one illustrated in the "multidb"
    template.  By tracking which directives have been processed, a
    multiple-pass scheme which calls upon the :class:`.Rewriter` multiple times
    for the same structure as elements are added can work without running
    duplicate operations on the same elements more than once.

.. changelog::
:version: 1.1.0
:released: August 26, 2019

text
.. change::
    :tags: change

    Alembic 1.1 bumps the minimum version of SQLAlchemy to 1.1.   As was the
    case before, Python requirements remain at Python 2.7, or in the 3.x series
    Python 3.4.

.. change::
    :tags: change, internals

    The test suite for Alembic now makes use of SQLAlchemy's testing framework
    directly.  Previously, Alembic had its own version of this framework that
    was mostly copied from that of SQLAlchemy to enable testing with older
    SQLAlchemy versions.  The majority of this code is now removed so that both
    projects can leverage improvements from a common testing framework.

.. change::
    :tags: bug, commands
    :tickets: 562

    Fixed bug where the double-percent logic applied to some dialects such as
    psycopg2 would be rendered in ``--sql`` mode, by allowing dialect options
    to be passed through to the dialect used to generate SQL and then providing
    ``paramstyle="named"`` so that percent signs need not be doubled.   For
    users having this issue, existing env.py scripts need to add
    ``dialect_opts={"paramstyle": "named"}`` to their offline
    context.configure().  See the ``alembic/templates/generic/env.py`` template
    for an example.

.. change::
    :tags: bug, py3k

    Fixed use of the deprecated "imp" module, which is used to detect  pep3147
    availability as well as to locate .pyc files, which started  emitting
    deprecation warnings during the test suite.   The warnings were not being
    emitted earlier during the test suite, the change is possibly due to
    changes in py.test itself but this is not clear. The check for pep3147 is
    set to True for any Python version 3.5 or greater now and importlib is used
    when available.  Note that some dependencies such as distutils may still be
    emitting this warning. Tests are adjusted to accommodate for dependencies
    that emit the warning as well.


.. change::
    :tags: bug, mysql
    :tickets: 594

    Fixed issue where emitting a change of column name for MySQL did not
    preserve the column comment, even if it were specified as existing_comment.


.. change::
    :tags: bug, setup
    :tickets: 592

    Removed the "python setup.py test" feature in favor of a straight run of
    "tox".   Per Pypa / pytest developers, "setup.py" commands are in general
    headed towards deprecation in favor of tox.  The tox.ini script has been
    updated such that running "tox" with no arguments will perform a single run
    of the test suite against the default installed Python interpreter.

    .. seealso::

        https://github.com/pypa/setuptools/issues/1684

        https://github.com/pytest-dev/pytest/issues/5534

.. change::
    :tags: usecase, commands
    :tickets: 571

    The "alembic init" command will now proceed if the target directory exists
    as long as it's still empty.  Previously, it would not proceed if the
    directory existed. The new behavior is modeled from what git does, to
    accommodate for container or other deployments where an Alembic target
    directory may need to be already mounted instead of being created with
    alembic init.  Pull request courtesy Aviskar KC.

.. changelog::
:version: 1.0.11
:released: June 25, 2019

text
.. change::
    :tags: bug, sqlite, autogenerate, batch
    :tickets: 579

    SQLite server default reflection will ensure parenthesis are surrounding a
    column default expression that is detected as being a non-constant
    expression, such as a ``datetime()`` default, to accommodate for the
    requirement that SQL expressions have to be parenthesized when being sent
    as DDL.  Parenthesis are not added to constant expressions to allow for
    maximum cross-compatibility with other dialects and existing test suites
    (such as Alembic's), which necessarily entails scanning the expression to
    eliminate for constant numeric and string values. The logic is added to the
    two "reflection->DDL round trip" paths which are currently autogenerate and
    batch migration.  Within autogenerate, the logic is on the rendering side,
    whereas in batch the logic is installed as a column reflection hook.


.. change::
    :tags: bug, sqlite, autogenerate
    :tickets: 579

    Improved SQLite server default comparison to accommodate for a ``text()``
    construct that added parenthesis directly vs. a construct that relied
    upon the SQLAlchemy SQLite dialect to render the parenthesis, as well
    as improved support for various forms of constant expressions such as
    values that are quoted vs. non-quoted.


.. change::
    :tags: bug, autogenerate

    Fixed bug where the "literal_binds" flag was not being set when
    autogenerate would create a server default value, meaning server default
    comparisons would fail for functions that contained literal values.

.. change::
   :tags: bug, mysql
   :tickets: 554

   Added support for MySQL "DROP CHECK", which is added as of MySQL 8.0.16,
   separate from MariaDB's "DROP CONSTRAINT" for CHECK constraints.  The MySQL
   Alembic implementation now checks for "MariaDB" in server_version_info to
   decide which one to use.



.. change::
    :tags: bug, mysql, operations
    :tickets: 564

    Fixed issue where MySQL databases need to use CHANGE COLUMN when altering a
    server default of CURRENT_TIMESTAMP, NOW() and probably other functions
    that are only usable with DATETIME/TIMESTAMP columns.  While MariaDB
    supports both CHANGE and ALTER COLUMN in this case, MySQL databases only
    support CHANGE.  So the new logic is that if the server default change is
    against a DateTime-oriented column, the CHANGE format is used
    unconditionally, as in the vast majority of cases the server default is to
    be CURRENT_TIMESTAMP which may also be potentially bundled with an "ON
    UPDATE CURRENT_TIMESTAMP" directive, which SQLAlchemy does not currently
    support as a distinct field.  The fix additionally improves the server
    default comparison logic when the "ON UPDATE" clause is present and
    there are parenthesis to be adjusted for as is the case on some MariaDB
    versions.



.. change::
    :tags: bug, environment

    Warnings emitted by Alembic now include a default stack level of 2, and in
    some cases it's set to 3, in order to help warnings indicate more closely
    where they are originating from.  Pull request courtesy Ash Berlin-Taylor.


.. change::
    :tags: bug, py3k
    :tickets: 563

    Replaced the Python compatibility routines for ``getargspec()`` with a fully
    vendored version based on ``getfullargspec()`` from Python 3.3.
    Originally, Python was emitting deprecation warnings for this function in
    Python 3.8 alphas.  While this change was reverted, it was observed that
    Python 3 implementations for ``getfullargspec()`` are an order of magnitude
    slower as of the 3.4 series where it was rewritten against ``Signature``.
    While Python plans to improve upon this situation, SQLAlchemy projects for
    now are using a simple replacement to avoid any future issues.

.. changelog::
:version: 1.0.10
:released: April 28, 2019

text
.. change::
   :tags: bug, commands
   :tickets: 552

   Fixed bug introduced in release 0.9.0 where the helptext for commands
   inadvertently got expanded to include function docstrings from the
   command.py module.  The logic has been adjusted to only refer to the first
   line(s) preceding the first line break within each docstring, as was the
   original intent.

.. change::
    :tags: bug, operations, mysql
    :tickets: 551

    Added an assertion in :meth:`.RevisionMap.get_revisions` and other methods
    which ensures revision numbers are passed as strings or collections of
    strings.   Driver issues particularly on MySQL may inadvertently be passing
    bytes here which leads to failures later on.

.. change::
    :tags: bug, autogenerate, mysql
    :tickets: 553

    Fixed bug when using the
    :paramref:`.EnvironmentContext.configure.compare_server_default` flag set
    to ``True`` where a server default that is introduced in the table metadata
    on an ``Integer`` column, where there is no existing server default in the
    database, would raise a ``TypeError``.

.. changelog::
:version: 1.0.9
:released: April 15, 2019

text
.. change::
   :tags: bug, operations
   :tickets: 548

   Simplified the internal scheme used to generate the ``alembic.op`` namespace
   to no longer attempt to generate full method signatures (e.g. rather than
   generic ``*args, **kw``) as this was not working in most cases anyway, while
   in rare circumstances it would in fact sporadically have access to the real
   argument names and then fail when generating the function due to missing
   symbols in the argument signature.

.. changelog::
:version: 1.0.8
:released: March 4, 2019

text
.. change::
   :tags: bug, operations
   :tickets: 528

   Removed use of deprecated ``force`` parameter for SQLAlchemy quoting
   functions as this parameter will be removed in a future release.
   Pull request courtesy Parth Shandilya(ParthS007).

.. change::
   :tags: bug, autogenerate, postgresql, py3k
   :tickets: 541

   Fixed issue where server default comparison on the PostgreSQL dialect would
   fail for a blank string on Python 3.7 only, due to a change in regular
   expression behavior in Python 3.7.

.. changelog::
:version: 1.0.7
:released: January 25, 2019

text
.. change::
   :tags: bug, autogenerate
   :tickets: 529

   Fixed issue in new comment support where autogenerated Python code
   for comments wasn't using ``repr()`` thus causing issues with
   quoting.  Pull request courtesy Damien Garaud.

.. changelog::
:version: 1.0.6
:released: January 13, 2019

text
.. change::
    :tags: feature, operations
    :tickets: 422

    Added Table and Column level comments for supported backends.
    New methods :meth:`.Operations.create_table_comment` and
    :meth:`.Operations.drop_table_comment` are added.  A new arguments
    :paramref:`.Operations.alter_column.comment` and
    :paramref:`.Operations.alter_column.existing_comment` are added to
    :meth:`.Operations.alter_column`.   Autogenerate support is also added
    to ensure comment add/drops from tables and columns are generated as well
    as that :meth:`.Operations.create_table`, :meth:`.Operations.add_column`
    both include the comment field from the source :class:`.Table`
    or :class:`.Column` object.

.. changelog::
:version: 1.0.5
:released: November 27, 2018

text
.. change::
    :tags: bug, py3k
    :tickets: 507

    Resolved remaining Python 3 deprecation warnings, covering
    the use of inspect.formatargspec() with a vendored version
    copied from the Python standard library, importing
    collections.abc above Python 3.3 when testing against abstract
    base classes, fixed one occurrence of log.warn(), as well as a few
    invalid escape sequences.

.. changelog::
:version: 1.0.4
:released: November 27, 2018

text
.. change::
   :tags: change

   Code hosting has been moved to GitHub, at
   https://github.com/sqlalchemy/alembic.  Additionally, the
   main Alembic website documentation URL is now
   https://alembic.sqlalchemy.org.

.. changelog::
:version: 1.0.3
:released: November 14, 2018

text
.. change::
    :tags: bug, mssql
    :tickets: 516

   Fixed regression caused by :ticket:`513`, where the logic to consume
   ``mssql_include`` was not correctly interpreting the case where the flag
   was not present, breaking the ``op.create_index`` directive for SQL Server
   as a whole.

.. changelog::
:version: 1.0.2
:released: October 31, 2018

text
.. change::
   :tags: bug, autogenerate
   :tickets: 515

   The ``system=True`` flag on :class:`.Column`, used primarily in conjunction
   with the Postgresql "xmin" column, now renders within the autogenerate
   render process, allowing the column to be excluded from DDL.  Additionally,
   adding a system=True column to a model will produce no autogenerate diff as
   this column is implicitly present in the database.

.. change::
   :tags: bug, mssql
   :tickets: 513

   Fixed issue where usage of the SQL Server ``mssql_include`` option within a
   :meth:`.Operations.create_index` would raise a KeyError, as the additional
   column(s) need to be added to the table object used by the construct
   internally.

.. changelog::
:version: 1.0.1
:released: October 17, 2018

text
.. change::
    :tags: bug, commands
    :tickets: 497

    Fixed an issue where revision descriptions were essentially
    being formatted twice. Any revision description that contained
    characters like %, writing output to stdout will fail because
    the call to config.print_stdout attempted to format any
    additional args passed to the function.
    This fix now only applies string formatting if any args are provided
    along with the output text.

.. change::
   :tags: bug, autogenerate
   :tickets: 512

   Fixed issue where removed method ``union_update()`` was used when a
   customized :class:`.MigrationScript` instance included entries in the
   ``.imports`` data member, raising an AttributeError.

.. changelog::
:version: 1.0.0
:released: July 13, 2018
:released: July 13, 2018
:released: July 13, 2018

text
.. change::
    :tags: feature, general
    :tickets: 491

    For Alembic 1.0, Python 2.6 / 3.3 support is being dropped, allowing a
    fixed setup.py to be built as well as universal wheels.  Pull request
    courtesy Hugo.




.. change::
    :tags: feature, general

    With the 1.0 release, Alembic's minimum SQLAlchemy support version
    moves to 0.9.0, previously 0.7.9.

.. change::
    :tags: bug, batch
    :tickets: 502

    Fixed issue in batch where dropping a primary key column, then adding it
    back under the same name but without the primary_key flag, would not remove
    it from the existing PrimaryKeyConstraint.  If a new PrimaryKeyConstraint
    is added, it is used as-is, as was the case before.

.. changelog::
:version: 0.9.10
:released: June 29, 2018

text
.. change::
    :tags: bug, autogenerate

    The "op.drop_constraint()" directive will now render using ``repr()`` for
    the schema name, in the same way that "schema" renders for all the other op
    directives.  Pull request courtesy Denis Kataev.

.. change::
    :tags: bug, autogenerate
    :tickets: 494

    Added basic capabilities for external dialects to support rendering of
    "nested" types, like arrays, in a manner similar to that of the Postgresql
    dialect.

.. change::
    :tags: bug, autogenerate

    Fixed issue where "autoincrement=True" would not render for a column that
    specified it, since as of SQLAlchemy 1.1 this is no longer the default
    value for "autoincrement".  Note the behavior only takes effect against the
    SQLAlchemy 1.1.0 and higher; for pre-1.1 SQLAlchemy, "autoincrement=True"
    does not render as was the case before. Pull request courtesy  Elad Almos.

.. changelog::
:version: 0.9.9
:released: March 22, 2018

text
.. change::
    :tags: feature, commands
    :tickets: 481

    Added new flag ``--indicate-current`` to the ``alembic history`` command.
    When listing versions, it will include the token "(current)" to indicate
    the given version is a current head in the target database.  Pull request
    courtesy Kazutaka Mise.

.. change::
    :tags: bug, autogenerate, mysql
    :tickets: 455

    The fix for :ticket:`455` in version 0.9.6 involving MySQL server default
    comparison was entirely non functional, as the test itself was also broken
    and didn't reveal that it wasn't working. The regular expression to compare
    server default values like CURRENT_TIMESTAMP to current_timestamp() is
    repaired.

.. change::
    :tags: bug, mysql, autogenerate
    :tickets: 483

    Fixed bug where MySQL server default comparisons were basically not working
    at all due to incorrect regexp added in :ticket:`455`.  Also accommodates
    for MariaDB 10.2 quoting differences in reporting integer based server
    defaults.




.. change::
    :tags: bug, operations, mysql
    :tickets: 487

    Fixed bug in ``op.drop_constraint()`` for MySQL where
    quoting rules would not be applied to the constraint name.

.. changelog::
:version: 0.9.8
:released: February 16, 2018

text
.. change::
    :tags: bug, runtime
    :tickets: 482

    Fixed bug where the :meth:`.Script.as_revision_number` method
    did not accommodate for the 'heads' identifier, which in turn
    caused the :meth:`.EnvironmentContext.get_head_revisions`
    and :meth:`.EnvironmentContext.get_revision_argument` methods
    to be not usable when multiple heads were present.
    The :meth:.`EnvironmentContext.get_head_revisions` method returns
    a tuple in all cases as documented.



.. change::
    :tags: bug, postgresql, autogenerate
    :tickets: 478

    Fixed bug where autogenerate of :class:`.ExcludeConstraint`
    would render a raw quoted name for a Column that has case-sensitive
    characters, which when invoked as an inline member of the Table
    would produce a stack trace that the quoted name is not found.
    An incoming Column object is now rendered as ``sa.column('name')``.

.. change::
    :tags: bug, autogenerate
    :tickets: 468

    Fixed bug where the indexes would not be included in a
    migration that was dropping the owning table.   The fix
    now will also emit DROP INDEX for the indexes ahead of time,
    but more importantly will include CREATE INDEX in the
    downgrade migration.

.. change::
    :tags: bug, postgresql
    :tickets: 480

    Fixed the autogenerate of the module prefix
    when rendering the text_type parameter of
    postgresql.HSTORE, in much the same way that
    we do for ARRAY's type and JSON's text_type.

.. change::
    :tags: bug, mysql
    :tickets: 479

    Added support for DROP CONSTRAINT to the MySQL Alembic
    dialect to support MariaDB 10.2 which now has real
    CHECK constraints.  Note this change does **not**
    add autogenerate support, only support for op.drop_constraint()
    to work.

.. changelog::
:version: 0.9.7
:released: January 16, 2018

text
.. change::
    :tags: bug, autogenerate
    :tickets: 472

    Fixed regression caused by :ticket:`421` which would
    cause case-sensitive quoting rules to interfere with the
    comparison logic for index names, thus causing indexes to show
    as added for indexes that have case-sensitive names.   Works with
    SQLAlchemy 0.9 and later series.


.. change::
    :tags: bug, postgresql, autogenerate
    :tickets: 461

    Fixed bug where autogenerate would produce a DROP statement for the index
    implicitly created by a Postgresql EXCLUDE constraint, rather than skipping
    it as is the case for indexes implicitly generated by unique constraints.
    Makes use of SQLAlchemy 1.0.x's improved "duplicates index" metadata and
    requires at least SQLAlchemy version 1.0.x to function correctly.

.. changelog::
:version: 0.9.6
:released: October 13, 2017

text
.. change::
    :tags: bug, commands
    :tickets: 458

    Fixed a few Python3.6 deprecation warnings by replacing ``StopIteration``
    with ``return``, as well as using ``getfullargspec()`` instead of
    ``getargspec()`` under Python 3.

.. change::
    :tags: bug, commands
    :tickets: 441

    An addition to :ticket:`441` fixed in 0.9.5, we forgot to also filter
    for the ``+`` sign in migration names which also breaks due to the relative
    migrations feature.

.. change::
    :tags: bug, autogenerate
    :tickets: 442

    Fixed bug expanding upon the fix for
    :ticket:`85` which adds the correct module import to the
    "inner" type for an ``ARRAY`` type, the fix now accommodates for the
    generic ``sqlalchemy.types.ARRAY`` type added in SQLAlchemy 1.1,
    rendering the inner type correctly regardless of whether or not the
    Postgresql dialect is present.

.. change::
    :tags: bug, mysql
    :tickets: 455

    Fixed bug where server default comparison of CURRENT_TIMESTAMP would fail
    on MariaDB 10.2 due to a change in how the function is
    represented by the database during reflection.

.. change::
    :tags: bug, autogenerate

    Fixed bug where comparison of ``Numeric`` types would produce
    a difference if the Python-side ``Numeric`` inadvertently specified
    a non-None "scale" with a "precision" of None, even though this ``Numeric``
    type will pass over the "scale" argument when rendering. Pull request
    courtesy Ivan Mmelnychuk.

.. change::
    :tags: feature, commands
    :tickets: 447

    The ``alembic history`` command will now make use of the revision
    environment ``env.py`` unconditionally if the ``revision_environment``
    configuration flag is set to True.  Previously, the environment would
    only be invoked if the history specification were against a database-stored
    revision token.

.. change::
    :tags: bug, batch
    :tickets: 457

    The name of the temporary table in batch mode is now generated
    off of the original table name itself, to avoid conflicts for the
    unusual case of multiple batch operations running against the same
    database schema at the same time.

.. change::
    :tags: bug, autogenerate
    :tickets: 456

    A :class:`.ForeignKeyConstraint` can now render correctly if the
    ``link_to_name`` flag is set, as it will not attempt to resolve the name
    from a "key" in this case.  Additionally, the constraint will render
    as-is even if the remote column name isn't present on the referenced
    remote table.

.. change::
    :tags: bug, runtime, py3k
    :tickets: 449

    Reworked "sourceless" system to be fully capable of handling any
    combination of: Python2/3x, pep3149 or not, PYTHONOPTIMIZE or not,
    for locating and loading both env.py files as well as versioning files.
    This includes: locating files inside of ``__pycache__`` as well as listing
    out version files that might be only in ``versions/__pycache__``, deduplicating
    version files that may be in ``versions/__pycache__`` and ``versions/``
    at the same time, correctly looking for .pyc or .pyo files based on
    if pep488 is present or not. The latest Python3x deprecation warnings
    involving importlib are also corrected.

.. changelog::
:version: 0.9.5
:released: August 9, 2017

text
.. change::
    :tags: bug, commands
    :tickets: 441

    A :class:`.CommandError` is raised if the "--rev-id" passed to the
    :func:`.revision` command contains dashes or at-signs, as this interferes
    with the command notation used to locate revisions.

.. change::
    :tags: bug, postgresql
    :tickets: 424

    Added support for the dialect-specific keyword arguments
    to :meth:`.Operations.drop_index`.   This includes support for
    ``postgresql_concurrently`` and others.

.. change::
    :tags: bug, commands

    Fixed bug in timezone feature introduced in
    :ticket:`425` when the creation
    date in a revision file is calculated, to
    accommodate for timezone names that contain
    mixed-case characters in their name as opposed
    to all uppercase.  Pull request courtesy Nils
    Philippsen.

.. changelog::
:version: 0.9.4
:released: July 31, 2017

text
.. change::
  :tags: bug, runtime

  Added an additional attribute to the new
  :paramref:`.EnvironmentContext.configure.on_version_apply` API,
  :attr:`.MigrationInfo.up_revision_ids`, to accommodate for the uncommon
  case of the ``alembic stamp`` command being used to move from multiple
  branches down to a common branchpoint; there will be multiple
  "up" revisions in this one case.

.. changelog::
:version: 0.9.3
:released: July 6, 2017

text
.. change::
  :tags: feature, runtime

  Added a new callback hook
  :paramref:`.EnvironmentContext.configure.on_version_apply`,
  which allows user-defined code to be invoked each time an individual
  upgrade, downgrade, or stamp operation proceeds against a database.
  Pull request courtesy John Passaro.

.. change:: 433
  :tags: bug, autogenerate
  :tickets: 433

  Fixed bug where autogen comparison of a :class:`.Variant` datatype
  would not compare to the dialect level type for the "default"
  implementation of the :class:`.Variant`, returning the type as changed
  between database and table metadata.

.. change:: 431
  :tags: bug, tests
  :tickets: 431

  Fixed unit tests to run correctly under the SQLAlchemy 1.0.x series
  prior to version 1.0.10 where a particular bug involving Postgresql
  exclude constraints was fixed.

.. changelog::
:version: 0.9.2
:released: May 18, 2017

text
.. change:: 429
  :tags: bug, mssql
  :tickets: 429

  Repaired :meth:`.Operations.rename_table` for SQL Server when the
  target table is in a remote schema, the schema name is omitted from
  the "new name" argument.

.. change:: 425
  :tags: feature, commands
  :tickets: 425

  Added a new configuration option ``timezone``, a string timezone name
  that will be applied to the create date timestamp rendered
  inside the revision file as made available to the ``file_template`` used
  to generate the revision filename.  Note this change adds the
  ``python-dateutil`` package as a dependency.

.. change:: 421
  :tags: bug, autogenerate
  :tickets: 421

  The autogenerate compare scheme now takes into account the name truncation
  rules applied by SQLAlchemy's DDL compiler to the names of the
  :class:`.Index` object, when these names are dynamically truncated
  due to a too-long identifier name.   As the identifier truncation is
  deterministic, applying the same rule to the metadata name allows
  correct comparison to the database-derived name.

.. change:: 419
  :tags: bug environment
  :tickets: 419

  A warning is emitted when an object that's not a
  :class:`~sqlalchemy.engine.Connection` is passed to
  :meth:`.EnvironmentContext.configure`.  For the case of a
  :class:`~sqlalchemy.engine.Engine` passed, the check for "in transaction"
  introduced in version 0.9.0 has been relaxed to work in the case of an
  attribute error, as some users appear to be passing an
  :class:`~sqlalchemy.engine.Engine` and not a
  :class:`~sqlalchemy.engine.Connection`.

.. changelog::
:version: 0.9.1
:released: March 1, 2017

text
.. change:: 417
  :tags: bug, commands
  :tickets: 417, 369

  An adjustment to the bug fix for :ticket:`369` to accommodate for
  env.py scripts that use an enclosing transaction distinct from the
  one that the context provides, so that the check for "didn't commit
  the transaction" doesn't trigger in this scenario.

.. changelog::
:version: 0.9.0
:released: February 28, 2017

text
.. change:: 38
  :tags: feature, autogenerate
  :tickets: 38

  The :paramref:`.EnvironmentContext.configure.target_metadata` parameter
  may now be optionally specified as a sequence of :class:`.MetaData`
  objects instead of a single :class:`.MetaData` object.  The
  autogenerate process will process the sequence of :class:`.MetaData`
  objects in order.

.. change:: 369
  :tags: bug, commands
  :tickets: 369

  A :class:`.CommandError` is now raised when a migration file opens
  a database transaction and does not close/commit/rollback, when
  the backend database or environment options also specify transactional_ddl
  is False.   When transactional_ddl is not in use, Alembic doesn't
  close any transaction so a transaction opened by a migration file
  will cause the following migrations to fail to apply.

.. change:: 413
  :tags: bug, autogenerate, mysql
  :tickets: 413

  The ``autoincrement=True`` flag is now rendered within the
  :meth:`.Operations.alter_column` operation if the source column indicates
  that this flag should be set to True.  The behavior is sensitive to
  the SQLAlchemy version in place, as the "auto" default option is new
  in SQLAlchemy 1.1.  When the source column indicates autoincrement
  as True or "auto", the flag will render as True if the original column
  contextually indicates that it should have "autoincrement" keywords,
  and when the source column explicitly sets it to False, this is also
  rendered.  The behavior is intended to preserve the AUTO_INCREMENT flag
  on MySQL as the column is fully recreated on this backend.  Note that this
  flag does **not** support alteration of a column's "autoincrement" status,
  as this is not portable across backends.

.. change:: 411
  :tags: bug, postgresql
  :tickets: 411

  Fixed bug where Postgresql JSON/JSONB types rendered on SQLAlchemy
  1.1 would render the "astext_type" argument which defaults to
  the ``Text()`` type without the module prefix, similarly to the
  issue with ARRAY fixed in :ticket:`85`.

.. change:: 85
  :tags: bug, postgresql
  :tickets: 85

  Fixed bug where Postgresql ARRAY type would not render the import prefix
  for the inner type; additionally, user-defined renderers take place
  for the inner type as well as the outer type.  Pull request courtesy
  Paul Brackin.

.. change:: process_revision_directives_command
  :tags: feature, autogenerate

  Added a keyword argument ``process_revision_directives`` to the
  :func:`.command.revision` API call.  This function acts in the
  same role as the environment-level
  :paramref:`.EnvironmentContext.configure.process_revision_directives`,
  and allows API use of the
  command to drop in an ad-hoc directive process function.  This
  function can be used among other things to place a complete
  :class:`.MigrationScript` structure in place.

.. change:: 412
  :tags: feature, postgresql
  :tickets: 412

  Added support for Postgresql EXCLUDE constraints, including the
  operation directive :meth:`.Operations.create_exclude_constraints`
  as well as autogenerate render support for the ``ExcludeConstraint``
  object as present in a ``Table``.  Autogenerate detection for an EXCLUDE
  constraint added or removed to/from an existing table is **not**
  implemented as the SQLAlchemy Postgresql dialect does not yet support
  reflection of EXCLUDE constraints.

  Additionally, unknown constraint types now warn when
  encountered within an autogenerate action rather than raise.

.. change:: fk_schema_compare
  :tags: bug, operations

  Fixed bug in :func:`.ops.create_foreign_key` where the internal table
  representation would not be created properly if the foreign key referred
  to a table in a different schema of the same name.  Pull request
  courtesy Konstantin Lebedev.

.. changelog::
:version: 0.8.10
:released: January 17, 2017

text
.. change:: 406
  :tags: bug, versioning
  :tickets: 406

  The alembic_version table, when initially created, now establishes a
  primary key constraint on the "version_num" column, to suit database
  engines that don't support tables without primary keys.   This behavior
  can be controlled using the parameter
  :paramref:`.EnvironmentContext.configure.version_table_pk`.  Note that
  this change only applies to the initial creation of the alembic_version
  table; it does not impact any existing alembic_version table already
  present.

.. change:: 402
  :tags: bug, batch
  :tickets: 402

  Fixed bug where doing ``batch_op.drop_constraint()`` against the
  primary key constraint would fail to remove the "primary_key" flag
  from the column, resulting in the constraint being recreated.

.. change:: update_uq_dedupe
  :tags: bug, autogenerate, oracle

  Adjusted the logic originally added for :ticket:`276` that detects MySQL
  unique constraints which are actually unique indexes to be generalized
  for any dialect that has this behavior, for SQLAlchemy version 1.0 and
  greater.  This is to allow for upcoming SQLAlchemy support for unique
  constraint reflection for Oracle, which also has no dedicated concept of
  "unique constraint" and instead establishes a unique index.

.. change:: 356
  :tags: bug, versioning
  :tickets: 356

  Added a file ignore for Python files of the form ``.#<name>.py``,
  which are generated by the Emacs editor.  Pull request courtesy
  Markus Mattes.

.. changelog::
:version: 0.8.9
:released: November 28, 2016

text
.. change::  393
  :tags: bug, autogenerate
  :tickets: 393

  Adjustment to the "please adjust!" comment in the script.py.mako
  template so that the generated comment starts with a single pound
  sign, appeasing flake8.

.. change::
  :tags: bug, batch
  :tickets: 391

  Batch mode will not use CAST() to copy data if ``type_`` is given, however
  the basic type affinity matches that of the existing type.  This to
  avoid SQLite's CAST of TIMESTAMP which results in truncation of the
  data, in those cases where the user needs to add redundant ``type_`` for
  other reasons.

.. change::
  :tags: bug, autogenerate
  :tickets: 393

  Continued pep8 improvements by adding appropriate whitespace in
  the base template for generated migrations.  Pull request courtesy
  Markus Mattes.

.. change::
  :tags: bug, revisioning

  Added an additional check when reading in revision files to detect
  if the same file is being read twice; this can occur if the same directory
  or a symlink equivalent is present more than once in version_locations.
  A warning is now emitted and the file is skipped.  Pull request courtesy
  Jiri Kuncar.

.. change::
  :tags: bug, autogenerate
  :tickets: 395

  Fixed bug where usage of a custom TypeDecorator which returns a
  per-dialect type via :meth:`.TypeDecorator.load_dialect_impl` that differs
  significantly from the default "impl" for the type decorator would fail
  to compare correctly during autogenerate.

.. change::
  :tags: bug, autogenerate, postgresql
  :tickets: 392

  Fixed bug in Postgresql "functional index skip" behavior where a
  functional index that ended in ASC/DESC wouldn't be detected as something
  we can't compare in autogenerate, leading to duplicate definitions
  in autogenerated files.

.. change::
  :tags: bug, versioning

  Fixed bug where the "base" specifier, as in "base:head", could not
  be used explicitly when ``--sql`` mode was present.

.. changelog::
:version: 0.8.8
:released: September 12, 2016

text
.. change::
   :tags: autogenerate

   The imports in the default script.py.mako are now at the top
   so that flake8 editors don't complain by default.  PR courtesy
   Guilherme Mansur.

.. change::
  :tags: feature, operations, postgresql
  :tickets: 292

  Added support for the USING clause to the ALTER COLUMN operation
  for Postgresql.  Support is via the
  :paramref:`.op.alter_column.postgresql_using`
  parameter.  Pull request courtesy Frazer McLean.

.. change::
  :tags: feature, autogenerate

  Autogenerate with type comparison enabled will pick up on the timezone
  setting changing between DateTime types.   Pull request courtesy
  David Szotten.

.. changelog::
:version: 0.8.7
:released: July 26, 2016

text
.. change::
  :tags: bug, versioning
  :tickets: 336

  Fixed bug where upgrading to the head of a branch which is already
  present would fail, only if that head were also the dependency
  of a different branch that is also upgraded, as the revision system
  would see this as trying to go in the wrong direction.   The check
  here has been refined to distinguish between same-branch revisions
  out of order vs. movement along sibling branches.

.. change::
  :tags: bug, versioning
  :tickets: 379

  Adjusted the version traversal on downgrade
  such that we can downgrade to a version that is a dependency for
  a version in a different branch, *without* needing to remove that
  dependent version as well.  Previously, the target version would be
  seen as a "merge point" for it's normal up-revision as well as the
  dependency.  This integrates with the changes for :ticket:`377`
  and :ticket:`378` to improve treatment of branches with dependencies
  overall.

.. change::
  :tags: bug, versioning
  :tickets: 377

  Fixed bug where a downgrade to a version that is also a dependency
  to a different branch would fail, as the system attempted to treat
  this as an "unmerge" of a merge point, when in fact it doesn't have
  the other side of the merge point available for update.

.. change::
  :tags: bug, versioning
  :tickets: 378

  Fixed bug where the "alembic current" command wouldn't show a revision
  as a current head if it were also a dependency of a version in a
  different branch that's also applied.   Extra logic is added to
  extract "implied" versions of different branches from the top-level
  versions listed in the alembic_version table.

.. change::
  :tags: bug, versioning

  Fixed bug where a repr() or str() of a Script object would fail
  if the script had multiple dependencies.

.. change::
  :tags: bug, autogenerate

  Fixed bug in autogen where if the DB connection sends the default
  schema as "None", this "None" would be removed from the list of
  schemas to check if include_schemas were set.  This could possibly
  impact using include_schemas with SQLite.

.. change::
  :tags: bug, batch

  Small adjustment made to the batch handling for reflected CHECK
  constraints to accommodate for SQLAlchemy 1.1 now reflecting these.
  Batch mode still does not support CHECK constraints from the reflected
  table as these can't be easily differentiated from the ones created
  by types such as Boolean.

.. changelog::
:version: 0.8.6
:released: April 14, 2016

text
.. change::
  :tags: bug, commands
  :tickets: 367

  Errors which occur within the Mako render step are now intercepted
  and raised as CommandErrors like other failure cases; the Mako
  exception itself is written using template-line formatting to
  a temporary file which is named in the exception message.

.. change::
  :tags: bug, postgresql
  :tickets: 365

  Added a fix to Postgresql server default comparison which first checks
  if the text of the default is identical to the original, before attempting
  to actually run the default.  This accommodates for default-generation
  functions that generate a new value each time such as a uuid function.

.. change::
  :tags: bug, batch
  :tickets: 361

  Fixed bug introduced by the fix for :ticket:`338` in version 0.8.4
  where a server default could no longer be dropped in batch mode.
  Pull request courtesy Martin Domke.

.. change::
  :tags: bug, batch, mssql

  Fixed bug where SQL Server arguments for drop_column() would not
  be propagated when running under a batch block.  Pull request
  courtesy Michal Petrucha.

.. changelog::
:version: 0.8.5
:released: March 9, 2016

text
.. change::
  :tags: bug, autogenerate
  :tickets: 335

  Fixed bug where the columns rendered in a ``PrimaryKeyConstraint``
  in autogenerate would inappropriately render the "key" of the
  column, not the name.  Pull request courtesy Jesse Dhillon.

.. change::
  :tags: bug, batch
  :tickets: 354

  Repaired batch migration support for "schema" types which generate
  constraints, in particular the ``Boolean`` datatype which generates
  a CHECK constraint.  Previously, an alter column operation with this
  type would fail to correctly accommodate for the CHECK constraint
  on change both from and to this type.  In the former case the operation
  would fail entirely, in the latter, the CHECK constraint would
  not get generated.  Both of these issues are repaired.

.. change::
  :tags: bug, mysql
  :tickets: 355

  Changing a schema type such as ``Boolean`` to a non-schema type would
  emit a drop constraint operation which emits ``NotImplementedError`` for
  the MySQL dialect.  This drop constraint operation is now skipped when
  the constraint originates from a schema type.

.. changelog::
:version: 0.8.4
:released: December 15, 2015

text
.. change::
  :tags: feature, versioning

  A major improvement to the hash id generation function, which for some
  reason used an awkward arithmetic formula against uuid4() that produced
  values that tended to start with the digits 1-4.  Replaced with a
  simple substring approach which provides an even distribution.  Pull
  request courtesy Antti Haapala.

.. change::
  :tags: feature, autogenerate

  Added an autogenerate renderer for the :class:`.ExecuteSQLOp` operation
  object; only renders if given a plain SQL string, otherwise raises
  NotImplementedError.  Can be of help with custom autogenerate
  sequences that includes straight SQL execution.  Pull request courtesy
  Jacob Magnusson.

.. change::
  :tags: bug, batch
  :tickets: 345

  Batch mode generates a FOREIGN KEY constraint that is self-referential
  using the ultimate table name, rather than ``_alembic_batch_temp``.
  When the table is renamed from ``_alembic_batch_temp`` back to the
  original name, the FK now points to the right name.  This
  will **not** work if referential integrity is being enforced (eg. SQLite
  "PRAGMA FOREIGN_KEYS=ON") since the original table is dropped and
  the new table then renamed to that name, however this is now consistent
  with how foreign key constraints on **other** tables already operate
  with batch mode; these don't support batch mode if referential integrity
  is enabled in any case.

.. change::
  :tags: bug, autogenerate
  :tickets: 341

  Added a type-level comparator that distinguishes :class:`.Integer`,
  :class:`.BigInteger`, and :class:`.SmallInteger` types and
  dialect-specific types; these all have "Integer" affinity so previously
  all compared as the same.

.. change::
  :tags: bug, batch
  :tickets: 338

  Fixed bug where the ``server_default`` parameter of ``alter_column()``
  would not function correctly in batch mode.

.. change::
  :tags: bug, autogenerate
  :tickets: 337

  Adjusted the rendering for index expressions such that a :class:`.Column`
  object present in the source :class:`.Index` will not be rendered
  as table-qualified; e.g. the column name will be rendered alone.
  Table-qualified names here were failing on systems such as Postgresql.

.. changelog::
:version: 0.8.3
:released: October 16, 2015

text
.. change::
  :tags: bug, autogenerate
  :tickets: 332

  Fixed an 0.8 regression whereby the "imports" dictionary member of
  the autogen context was removed; this collection is documented in the
  "render custom type" documentation as a place to add new imports.
  The member is now known as
  :attr:`.AutogenContext.imports` and the documentation is repaired.

.. change::
  :tags: bug, batch
  :tickets: 333

  Fixed bug in batch mode where a table that had pre-existing indexes
  would create the same index on the new table with the same name,
  which on SQLite produces a naming conflict as index names are in a
  global namespace on that backend.   Batch mode now defers the production
  of both existing and new indexes until after the entire table transfer
  operation is complete, which also means those indexes no longer take
  effect during the INSERT from SELECT section as well; the indexes
  are applied in a single step afterwards.

.. change::
  :tags: bug, tests

  Added "pytest-xdist" as a tox dependency, so that the -n flag
  in the test command works if this is not already installed.
  Pull request courtesy Julien Danjou.

.. change::
  :tags: bug, autogenerate, postgresql
  :tickets: 324

  Fixed issue in PG server default comparison where model-side defaults
  configured with Python unicode literals would leak the "u" character
  from a ``repr()`` into the SQL used for comparison, creating an invalid
  SQL expression, as the server-side comparison feature in PG currently
  repurposes the autogenerate Python rendering feature to get a quoted
  version of a plain string default.

.. changelog::
:version: 0.8.2
:released: August 25, 2015

text
.. change::
  :tags: bug, autogenerate
  :tickets: 321

  Added workaround in new foreign key option detection feature for
  MySQL's consideration of the "RESTRICT" option being the default,
  for which no value is reported from the database; the MySQL impl now
  corrects for when the model reports RESTRICT but the database reports
  nothing.   A similar rule is in the default FK comparison to accommodate
  for the default "NO ACTION" setting being present in the model but not
  necessarily reported by the database, or vice versa.

.. changelog::
:version: 0.8.1
:released: August 22, 2015

text
.. change::
  :tags: feature, autogenerate

  A custom :paramref:`.EnvironmentContext.configure.process_revision_directives`
  hook can now generate op directives within the :class:`.UpgradeOps`
  and :class:`.DowngradeOps` containers that will be generated as Python
  code even when the ``--autogenerate`` flag is False; provided that
  ``revision_environment=True``, the full render operation will be run
  even in "offline" mode.

.. change::
  :tags: bug, autogenerate

  Repaired the render operation for the :class:`.ops.AlterColumnOp` object
  to succeed when the "existing_type" field was not present.

.. change::
  :tags: bug, autogenerate
  :tickets: 318

  Fixed a regression 0.8 whereby the "multidb" environment template
  failed to produce independent migration script segments for the
  output template.  This was due to the reorganization of the script
  rendering system for 0.8.  To accommodate this change, the
  :class:`.MigrationScript` structure will in the case of multiple
  calls to :meth:`.MigrationContext.run_migrations` produce lists
  for the :attr:`.MigrationScript.upgrade_ops` and
  :attr:`.MigrationScript.downgrade_ops` attributes; each :class:`.UpgradeOps`
  and :class:`.DowngradeOps` instance keeps track of its own
  ``upgrade_token`` and ``downgrade_token``, and each are rendered
  individually.

  .. seealso::

    :ref:`autogen_customizing_multiengine_revision` - additional detail
    on the workings of the
    :paramref:`.EnvironmentContext.configure.process_revision_directives`
    parameter when multiple calls to :meth:`.MigrationContext.run_migrations`
    are made.


.. change::
  :tags: feature, autogenerate
  :tickets: 317

  Implemented support for autogenerate detection of changes in the
  ``ondelete``, ``onupdate``, ``initially`` and ``deferrable``
  attributes of :class:`.ForeignKeyConstraint` objects on
  SQLAlchemy backends that support these on reflection
  (as of SQLAlchemy 1.0.8 currently Postgresql for all four,
  MySQL for ``ondelete`` and  ``onupdate`` only).   A constraint object
  that modifies these values will be reported as a "diff" and come out
  as a drop/create of the constraint with the modified values.
  The fields are ignored for backends which don't reflect these
  attributes (as of SQLA 1.0.8 this includes SQLite, Oracle, SQL Server,
  others).

.. changelog::
:version: 0.8.0
:released: August 12, 2015

text
.. change::
  :tags: bug, batch
  :tickets: 315

  Fixed bug in batch mode where the ``batch_op.create_foreign_key()``
  directive would be incorrectly rendered with the source table and
  schema names in the argument list.

.. change::
  :tags: feature, commands

  Added new command ``alembic edit``.  This command takes the same
  arguments as ``alembic show``, however runs the target script
  file within $EDITOR.  Makes use of the ``python-editor`` library
  in order to facilitate the handling of $EDITOR with reasonable
  default behaviors across platforms.  Pull request courtesy
  Michel Albert.

.. change::
  :tags: feature, commands
  :tickets: 311

  Added new multiple-capable argument ``--depends-on`` to the
  ``alembic revision`` command, allowing ``depends_on`` to be
  established at the command line level rather than having to edit
  the file after the fact. ``depends_on`` identifiers may also be
  specified as branch names at the command line or directly within
  the migration file. The values may be specified as partial
  revision numbers from the command line which will be resolved to
  full revision numbers in the output file.

.. change::
  :tags: change, operations

  A range of positional argument names have been changed to be
  clearer and more consistent across methods within the
  :class:`.Operations` namespace.   The most prevalent form of name change
  is that the descriptive names ``constraint_name`` and ``table_name``
  are now used where previously the name ``name`` would be used.
  This is in support of the newly modularized and extensible system of
  operation objects in :mod:`alembic.operations.ops`.
  An argument translation layer is in place
  across the ``alembic.op`` namespace that will ensure that named
  argument calling styles that use the old names will continue to
  function by transparently translating to the new names,
  also emitting a warning.   This, along with the fact that these
  arguments are positional in any case and aren't normally
  passed with an explicit name, should ensure that the
  overwhelming majority of applications should be unaffected by this
  change.   The *only* applications that are impacted are those that:

  1. use the :class:`.Operations` object directly in some way, rather
     than calling upon the ``alembic.op`` namespace, and

  2. invoke the methods on :class:`.Operations` using named keyword
     arguments for positional arguments like ``table_name``,
     ``constraint_name``, etc., which commonly were named ``name``
     as of 0.7.6.

  3. any application that is using named keyword arguments in place
     of positional argument for the recently added
     :class:`.BatchOperations` object may also be affected.

  The naming changes are documented as "versionchanged" for 0.8.0:

  * :meth:`.BatchOperations.create_check_constraint`
  * :meth:`.BatchOperations.create_foreign_key`
  * :meth:`.BatchOperations.create_index`
  * :meth:`.BatchOperations.create_unique_constraint`
  * :meth:`.BatchOperations.drop_constraint`
  * :meth:`.BatchOperations.drop_index`
  * :meth:`.Operations.create_check_constraint`
  * :meth:`.Operations.create_foreign_key`
  * :meth:`.Operations.create_primary_key`
  * :meth:`.Operations.create_index`
  * :meth:`.Operations.create_table`
  * :meth:`.Operations.create_unique_constraint`
  * :meth:`.Operations.drop_constraint`
  * :meth:`.Operations.drop_index`
  * :meth:`.Operations.drop_table`


.. change::
  :tags: feature, tests

  The default test runner via "python setup.py test" is now py.test.
  nose still works via run_tests.py.

.. change::
  :tags: feature, operations
  :tickets: 302

  The internal system for Alembic operations has been reworked to now
  build upon an extensible system of operation objects.  New operations
  can be added to the ``op.`` namespace, including that they are
  available in custom autogenerate schemes.

  .. seealso::

      :ref:`operation_plugins`

.. change::
  :tags: feature, autogenerate
  :tickets: 301, 306

  The internal system for autogenerate been reworked to build upon
  the extensible system of operation objects present in
  :ticket:`302`.  As part of this change, autogenerate now produces
  a full object graph representing a list of migration scripts to
  be written as well as operation objects that will render all the
  Python code within them; a new hook
  :paramref:`.EnvironmentContext.configure.process_revision_directives`
  allows end-user code to fully customize what autogenerate will do,
  including not just full manipulation of the Python steps to take
  but also what file or files will be written and where.  Additionally,
  autogenerate is now extensible as far as database objects compared
  and rendered into scripts; any new operation directive can also be
  registered into a series of hooks that allow custom database/model
  comparison functions to run as well as to render new operation
  directives into autogenerate scripts.

  .. seealso::

    :ref:`alembic.autogenerate.toplevel`

.. change::
  :tags: bug, versioning
  :tickets: 314

  Fixed bug where in the erroneous case that alembic_version contains
  duplicate revisions, some commands would fail to process the
  version history correctly and end up with a KeyError.   The fix
  allows the versioning logic to proceed, however a clear error is
  emitted later when attempting to update the alembic_version table.

.. changelog::
:version: 0.7.7
:released: July 22, 2015

text
.. change::
  :tags: bug, versioning
  :tickets: 310

  Fixed critical issue where a complex series of branches/merges would
  bog down the iteration algorithm working over redundant nodes for
  millions of cycles.   An internal adjustment has been
  made so that duplicate nodes are skipped within this iteration.

.. change::
  :tags: feature, batch
  :tickets: 305

  Implemented support for :meth:`.BatchOperations.create_primary_key`
  and :meth:`.BatchOperations.create_check_constraint`. Additionally,
  table keyword arguments are copied from the original reflected table,
  such as the "mysql_engine" keyword argument.

.. change::
  :tags: bug, environment
  :tickets: 300

  The :meth:`.MigrationContext.stamp` method, added as part of the
  versioning refactor in 0.7 as a more granular version of
  :func:`.command.stamp`, now includes the "create the alembic_version
  table if not present" step in the same way as the command version,
  which was previously omitted.

.. change::
  :tags: bug, autogenerate
  :tickets: 298

  Fixed bug where foreign key options including "onupdate",
  "ondelete" would not render within the ``op.create_foreign_key()``
  directive, even though they render within a full
  ``ForeignKeyConstraint`` directive.

.. change::
  :tags: bug, tests

  Repaired warnings that occur when running unit tests against
  SQLAlchemy 1.0.5 or greater involving the "legacy_schema_aliasing"
  flag.

.. changelog::
:version: 0.7.6
:released: May 5, 2015

text
.. change::
  :tags: feature, versioning
  :tickets: 297

  Fixed bug where the case of multiple mergepoints that all
  have the identical set of ancestor revisions would fail to be
  upgradable, producing an assertion failure.   Merge points were
  previously assumed to always require at least an UPDATE in
  alembic_revision from one of the previous revs to the new one,
  however in this case, if one of the mergepoints has already
  been reached, the remaining mergepoints have no row to UPDATE therefore
  they must do an INSERT of their target version.

.. change::
  :tags: feature, autogenerate
  :tickets: 296

  Added support for type comparison functions to be not just per
  environment, but also present on the custom types themselves, by
  supplying a method ``compare_against_backend``.
  Added a new documentation section :ref:`compare_types` describing
  type comparison fully.


.. change::
  :tags: feature, operations
  :tickets: 255

  Added a new option
  :paramref:`.EnvironmentContext.configure.literal_binds`, which
  will pass the ``literal_binds`` flag into the compilation of SQL
  constructs when using "offline" mode.  This has the effect that
  SQL objects like inserts, updates, deletes as well as textual
  statements sent using ``text()`` will be compiled such that the dialect
  will attempt to render literal values "inline" automatically.
  Only a subset of types is typically supported; the
  :meth:`.Operations.inline_literal` construct remains as the construct
  used to force a specific literal representation of a value.
  The :paramref:`.EnvironmentContext.configure.literal_binds` flag
  is added to the "offline" section of the ``env.py`` files generated
  in new environments.

.. change::
  :tags: bug, batch
  :tickets: 289

  Fully implemented the
  :paramref:`~.Operations.batch_alter_table.copy_from` parameter for
  batch mode, which previously was not functioning.  This allows
  "batch mode" to be usable in conjunction with ``--sql``.

.. change::
  :tags: bug, batch
  :tickets: 287

  Repaired support for the :meth:`.BatchOperations.create_index`
  directive, which was mis-named internally such that the operation
  within a batch context could not proceed.   The create index
  operation will proceed as part of a larger "batch table recreate"
  operation only if
  :paramref:`~.Operations.batch_alter_table.recreate` is set to
  "always", or if the batch operation includes other instructions that
  require a table recreate.

.. changelog::
:version: 0.7.5
:released: March 19, 2015

text
.. change::
  :tags: bug, autogenerate
  :tickets: 266

  The ``--autogenerate`` option is not valid when used in conjunction
  with "offline" mode, e.g. ``--sql``.  This now raises a ``CommandError``,
  rather than failing more deeply later on.  Pull request courtesy
  Johannes Erdfelt.

.. change::
  :tags: bug, operations, mssql
  :tickets: 284

  Fixed bug where the mssql DROP COLUMN directive failed to include
  modifiers such as "schema" when emitting the DDL.

.. change::
  :tags: bug, autogenerate, postgresql
  :tickets: 282

  Postgresql "functional" indexes are necessarily skipped from the
  autogenerate process, as the SQLAlchemy backend currently does not
  support reflection of these structures.   A warning is emitted
  both from the SQLAlchemy backend as well as from the Alembic
  backend for Postgresql when such an index is detected.

.. change::
  :tags: bug, autogenerate, mysql
  :tickets: 276

  Fixed bug where MySQL backend would report dropped unique indexes
  and/or constraints as both at the same time.  This is because
  MySQL doesn't actually have a "unique constraint" construct that
  reports differently than a "unique index", so it is present in both
  lists.  The net effect though is that the MySQL backend will report
  a dropped unique index/constraint as an index in cases where the object
  was first created as a unique constraint, if no other information
  is available to make the decision.  This differs from other backends
  like Postgresql which can report on unique constraints and
  unique indexes separately.

.. change::
  :tags: bug, commands
  :tickets: 269

  Fixed bug where using a partial revision identifier as the
  "starting revision" in ``--sql`` mode in a downgrade operation
  would fail to resolve properly.

  As a side effect of this change, the
  :meth:`.EnvironmentContext.get_starting_revision_argument`
  method will return the "starting" revision in its originally-
  given "partial" form in all cases, whereas previously when
  running within the :meth:`.command.stamp` command, it would have
  been resolved to a full number before passing it to the
  :class:`.EnvironmentContext`.  The resolution of this value to
  a real revision number has basically been moved to a more fundamental
  level within the offline migration process.

.. change::
  :tags: feature, commands

  Added a new feature :attr:`.Config.attributes`, to help with the use
  case of sharing state such as engines and connections on the outside
  with a series of Alembic API calls; also added a new cookbook section
  to describe this simple but pretty important use case.

  .. seealso::

      :ref:`connection_sharing`

.. change::
  :tags: feature, environment

  The format of the default ``env.py`` script has been refined a bit;
  it now uses context managers not only for the scope of the transaction,
  but also for connectivity from the starting engine.  The engine is also
  now called a "connectable" in support of the use case of an external
  connection being passed in.

.. change::
  :tags: feature, versioning
  :tickets: 267

  Added support for "alembic stamp" to work when given "heads" as an
  argument, when multiple heads are present.

.. changelog::
:version: 0.7.4
:released: January 12, 2015

text
.. change::
  :tags: bug, autogenerate, postgresql
  :tickets: 241

  Repaired issue where a server default specified without ``text()``
  that represented a numeric or floating point (e.g. with decimal places)
  value would fail in the Postgresql-specific check for "compare server
  default"; as PG accepts the value with quotes in the table specification,
  it's still valid.  Pull request courtesy Dimitris Theodorou.

.. change::
  :tags: bug, autogenerate
  :tickets: 259

  The rendering of a :class:`~sqlalchemy.schema.ForeignKeyConstraint`
  will now ensure that the names of the source and target columns are
  the database-side name of each column, and not the value of the
  ``.key`` attribute as may be set only on the Python side.
  This is because Alembic generates the DDL for constraints
  as standalone objects without the need to actually refer to an in-Python
  :class:`~sqlalchemy.schema.Table` object, so there's no step that
  would resolve these Python-only key names to database column names.

.. change::
  :tags: bug, autogenerate
  :tickets: 260

  Fixed bug in foreign key autogenerate where if the in-Python table
  used custom column keys (e.g. using the ``key='foo'`` kwarg to
  ``Column``), the comparison of existing foreign keys to those specified
  in the metadata would fail, as the reflected table would not have
  these keys available which to match up.  Foreign key comparison for
  autogenerate now ensures it's looking at the database-side names
  of the columns in all cases; this matches the same functionality
  within unique constraints and indexes.

.. change::
  :tags: bug, autogenerate
  :tickets: 261

  Fixed issue in autogenerate type rendering where types that belong
  to modules that have the name "sqlalchemy" in them would be mistaken
  as being part of the ``sqlalchemy.`` namespace.  Pull req courtesy
  Bartosz Burclaf.

.. changelog::
:version: 0.7.3
:released: December 30, 2014

text
.. change::
  :tags: bug, versioning
  :tickets: 258

  Fixed regression in new versioning system where upgrade / history
  operation would fail on AttributeError if no version files were
  present at all.

.. changelog::
:version: 0.7.2
:released: December 18, 2014

text
.. change::
  :tags: bug, sqlite, autogenerate

  Adjusted the SQLite backend regarding autogen of unique constraints
  to work fully with the current SQLAlchemy 1.0, which now will report
  on UNIQUE constraints that have no name.

.. change::
  :tags: bug, batch
  :tickets: 254

  Fixed bug in batch where if the target table contained multiple
  foreign keys to the same target table, the batch mechanics would
  fail with a "table already exists" error.  Thanks for the help
  on this from Lucas Kahlert.

.. change::
  :tags: bug, mysql
  :tickets: 251

  Fixed an issue where the MySQL routine to skip foreign-key-implicit
  indexes would also catch unnamed unique indexes, as they would be
  named after the column and look like the FK indexes.  Pull request
  courtesy Johannes Erdfelt.

.. change::
  :tags: bug, mssql, oracle
  :tickets: 253

  Repaired a regression in both the MSSQL and Oracle dialects whereby
  the overridden ``_exec()`` method failed to return a value, as is
  needed now in the 0.7 series.

.. changelog::
:version: 0.7.1
:released: December 3, 2014

text
.. change::
  :tags: bug, batch

  The ``render_as_batch`` flag was inadvertently hardcoded to ``True``,
  so all autogenerates were spitting out batch mode...this has been
  fixed so that batch mode again is only when selected in env.py.

.. change::
  :tags: feature, autogenerate
  :tickets: 178

  Support for autogenerate of FOREIGN KEY constraints has been added.
  These are delivered within the autogenerate process in the same
  manner as UNIQUE constraints, including ``include_object`` support.
  Big thanks to Ann Kamyshnikova for doing the heavy lifting here.

.. change::
  :tags: feature, batch

  Added :paramref:`~.Operations.batch_alter_table.naming_convention`
  argument to :meth:`.Operations.batch_alter_table`, as this is necessary
  in order to drop foreign key constraints; these are often unnamed
  on the target database, and in the case that they are named, SQLAlchemy
  is as of the 0.9 series not including these names yet.

  .. seealso::

    :ref:`dropping_sqlite_foreign_keys`

.. change::
  :tags: bug, batch

  Fixed bug where the "source_schema" argument was not correctly passed
  when calling :meth:`.BatchOperations.create_foreign_key`.  Pull
  request courtesy Malte Marquarding.

.. change::
  :tags: bug, batch
  :tickets: 249

  Repaired the inspection, copying and rendering of CHECK constraints
  and so-called "schema" types such as Boolean, Enum within the batch
  copy system; the CHECK constraint will not be "doubled" when the table is
  copied, and additionally the inspection of the CHECK constraint for
  its member columns will no longer fail with an attribute error.

.. change::
  :tags: feature, batch

  Added two new arguments
  :paramref:`.Operations.batch_alter_table.reflect_args`
  and :paramref:`.Operations.batch_alter_table.reflect_kwargs`, so that
  arguments may be passed directly to suit the
  :class:`~.sqlalchemy.schema.Table`
  object that will be reflected.

  .. seealso::

    :ref:`batch_controlling_table_reflection`

.. changelog::
:version: 0.7.0
:released: November 24, 2014

text
.. change::
  :tags: feature, versioning
  :tickets: 167

  The "multiple heads / branches" feature has now landed.  This is
  by far the most significant change Alembic has seen since its inception;
  while the workflow of most commands hasn't changed, and the format
  of version files and the ``alembic_version`` table are unchanged as well,
  a new suite of features opens up in the case where multiple version
  files refer to the same parent, or to the "base".  Merging of
  branches, operating across distinct named heads, and multiple
  independent bases are now all supported.   The feature incurs radical
  changes to the internals of versioning and traversal, and should be
  treated as "beta mode" for the next several subsequent releases
  within 0.7.

  .. seealso::

      :ref:`branches`

.. change::
  :tags: feature, versioning
  :tickets: 124

  In conjunction with support for multiple independent bases, the
  specific version directories are now also configurable to include
  multiple, user-defined directories.   When multiple directories exist,
  the creation of a revision file with no down revision requires
  that the starting directory is indicated; the creation of subsequent
  revisions along that lineage will then automatically use that
  directory for new files.

  .. seealso::

      :ref:`multiple_version_directories`

.. change::
  :tags: feature, operations, sqlite
  :tickets: 21

  Added "move and copy" workflow, where a table to be altered is copied to
  a new one with the new structure and the old one dropped, is now
  implemented for SQLite as well as all database backends in general
  using the new :meth:`.Operations.batch_alter_table` system.   This
  directive provides a table-specific operations context which gathers
  column- and constraint-level mutations specific to that table, and
  at the end of the context creates a new table combining the structure
  of the old one with the given changes, copies data from old table to new,
  and finally drops the old table,
  renaming the new one to the existing name.  This is required for
  fully featured SQLite migrations, as SQLite has very little support for the
  traditional ALTER directive.   The batch directive
  is intended to produce code that is still compatible with other databases,
  in that the "move and copy" process only occurs for SQLite by default,
  while still providing some level of sanity to SQLite's
  requirement by allowing multiple table mutation operations to
  proceed within one "move and copy" as well as providing explicit
  control over when this operation actually occurs.  The "move and copy"
  feature may be optionally applied to other backends as well, however
  dealing with referential integrity constraints from other tables must
  still be handled explicitly.

  .. seealso::

      :ref:`batch_migrations`

.. change::
  :tags: feature, commands

  Relative revision identifiers as used with ``alembic upgrade``,
  ``alembic downgrade`` and ``alembic history`` can be combined with
  specific revisions as well, e.g. ``alembic upgrade ae10+3``, to produce
  a migration target relative to the given exact version.

.. change::
  :tags: bug, commands
  :tickets: 248

  The ``alembic revision`` command accepts the ``--sql`` option to
  suit some very obscure use case where the ``revision_environment``
  flag is set up, so that ``env.py`` is run when ``alembic revision``
  is run even though autogenerate isn't specified.   As this flag is
  otherwise confusing, error messages are now raised if
  ``alembic revision`` is invoked with both ``--sql`` and
  ``--autogenerate`` or with ``--sql`` without
  ``revision_environment`` being set.

.. change::
  :tags: bug, autogenerate, postgresql
  :tickets: 247

  Added a rule for Postgresql to not render a "drop unique" and "drop index"
  given the same name; for now it is assumed that the "index" is the
  implicit one PostgreSQL generates.   Future integration with
  new SQLAlchemy 1.0 features will improve this to be more
  resilient.

.. change::
  :tags: bug, autogenerate
  :tickets: 247

  A change in the ordering when columns and constraints are dropped;
  autogenerate will now place the "drop constraint" calls *before*
  the "drop column" calls, so that columns involved in those constraints
  still exist when the constraint is dropped.

.. change::
  :tags: feature, commands

  New commands added: ``alembic show``, ``alembic heads`` and
  ``alembic merge``.  Also, a new option ``--verbose`` has been
  added to  several informational commands, such as ``alembic history``,
  ``alembic current``, ``alembic branches``, and ``alembic heads``.
  ``alembic revision`` also contains several new options used
  within the new branch management system.    The output of commands has
  been altered in many cases to support new fields and attributes;
  the ``history`` command in particular now returns it's "verbose" output
  only if ``--verbose`` is sent; without this flag it reverts to it's
  older behavior of short line items (which was never changed in the docs).

.. change::
  :tags: changed, commands

  The ``--head_only`` option to the ``alembic current`` command is
  deprecated; the ``current`` command now lists just the version numbers
  alone by default; use ``--verbose`` to get at additional output.

.. change::
  :tags: feature, config

  Added new argument :paramref:`.Config.config_args`, allows a dictionary
  of replacement variables to be passed which will serve as substitution
  values when an API-produced :class:`.Config` consumes the ``.ini``
  file.  Pull request courtesy Noufal Ibrahim.

.. change::
  :tags: bug, oracle
  :tickets: 245

  The Oracle dialect sets "transactional DDL" to False by default,
  as Oracle does not support transactional DDL.

.. change::
  :tags: bug, autogenerate
  :tickets: 243

  Fixed a variety of issues surrounding rendering of Python code that
  contains unicode literals.  The first is that the "quoted_name" construct
  that SQLAlchemy uses to represent table and column names as well
  as schema names does not ``repr()`` correctly on Py2K when the value
  contains unicode characters; therefore an explicit stringification is
  added to these.  Additionally, SQL expressions such as server defaults
  were not being generated in a unicode-safe fashion leading to decode
  errors if server defaults contained non-ascii characters.

.. change::
  :tags: bug, operations
  :tickets: 174

  The :meth:`.Operations.add_column` directive will now additionally emit
  the appropriate ``CREATE INDEX`` statement if the
  :class:`~sqlalchemy.schema.Column` object specifies ``index=True``.
  Pull request courtesy David Szotten.

.. change::
  :tags: feature, operations
  :tickets: 205

  The :class:`~sqlalchemy.schema.Table` object is now returned when
  the :meth:`.Operations.create_table` method is used.  This ``Table``
  is suitable for use in subsequent SQL operations, in particular
  the :meth:`.Operations.bulk_insert` operation.

.. change::
  :tags: feature, autogenerate
  :tickets: 203

  Indexes and unique constraints are now included in the
  :paramref:`.EnvironmentContext.configure.include_object` hook.
  Indexes are sent with type ``"index"`` and unique constraints with
  type ``"unique_constraint"``.

.. change::
  :tags: bug, autogenerate
  :tickets: 219

  Bound parameters are now resolved as "literal" values within the
  SQL expression inside of a CheckConstraint(), when rendering the SQL
  as a text string; supported for SQLAlchemy 0.8.0 and forward.

.. change::
  :tags: bug, autogenerate
  :tickets: 199

  Added a workaround for SQLAlchemy issue #3023 (fixed in 0.9.5) where
  a column that's part of an explicit PrimaryKeyConstraint would not
  have its "nullable" flag set to False, thus producing a false
  autogenerate.  Also added a related correction to MySQL which will
  correct for MySQL's implicit server default of '0' when a NULL integer
  column is turned into a primary key column.

.. change::
  :tags: bug, autogenerate, mysql
  :tickets: 240

  Repaired issue related to the fix for #208 and others; a composite
  foreign key reported by MySQL would cause a KeyError as Alembic
  attempted to remove MySQL's implicitly generated indexes from the
  autogenerate list.

.. change::
  :tags: bug, autogenerate
  :tickets: 28

  If the "alembic_version" table is present in the target metadata,
  autogenerate will skip this also.  Pull request courtesy
  Dj Gilcrease.

.. change::
  :tags: bug, autogenerate
  :tickets: 77

  The :paramref:`.EnvironmentContext.configure.version_table`
  and :paramref:`.EnvironmentContext.configure.version_table_schema`
  arguments are now honored during the autogenerate process, such that
  these names will be used as the "skip" names on both the database
  reflection and target metadata sides.

.. change::
  :tags: changed, autogenerate
  :tickets: 229

  The default value of the
  :paramref:`.EnvironmentContext.configure.user_module_prefix`
  parameter is **no longer the same as the SQLAlchemy prefix**.
  When omitted, user-defined types will now use the ``__module__``
  attribute of the type class itself when rendering in an
  autogenerated module.

.. change::
  :tags: bug, templates
  :tickets: 234

  Revision files are now written out using the ``'wb'`` modifier to
  ``open()``, since Mako reads the templates with ``'rb'``, thus preventing
  CRs from being doubled up as has been observed on windows.  The encoding
  of the output now defaults to 'utf-8', which can be configured using
  a newly added config file parameter ``output_encoding``.

.. change::
  :tags: bug, operations
  :tickets: 230

  Added support for use of the :class:`~sqlalchemy.sql.elements.quoted_name`
  construct when using the ``schema`` argument within operations.  This
  allows a name containing a dot to be fully quoted, as well as to
  provide configurable quoting on a per-name basis.

.. change::
  :tags: bug, autogenerate, postgresql
  :tickets: 73

  Added a routine by which the Postgresql Alembic dialect inspects
  the server default of INTEGER/BIGINT columns as they are reflected
  during autogenerate for the pattern ``nextval(<name>...)`` containing
  a potential sequence name, then queries ``pg_catalog`` to see if this
  sequence is "owned" by the column being reflected; if so, it assumes
  this is a SERIAL or BIGSERIAL column and the server default is
  omitted from the column reflection as well as any kind of
  server_default comparison or rendering, along with an INFO message
  in the logs indicating this has taken place. This allows SERIAL/BIGSERIAL
  columns to keep the SEQUENCE from being unnecessarily present within
  the autogenerate operation.

.. change::
  :tags: bug, autogenerate
  :tickets: 197, 64, 196

  The system by which autogenerate renders expressions within
  a :class:`~sqlalchemy.schema.Index`, the ``server_default``
  of :class:`~sqlalchemy.schema.Column`, and the
  ``existing_server_default`` of
  :meth:`.Operations.alter_column` has been overhauled to anticipate
  arbitrary SQLAlchemy SQL constructs, such as ``func.somefunction()``,
  ``cast()``, ``desc()``, and others.   The system does not, as might
  be preferred, render the full-blown Python expression as originally
  created within the application's source code, as this would be exceedingly
  complex and difficult.  Instead, it renders the SQL expression against
  the target backend that's subject to the autogenerate, and then
  renders that SQL inside of a :func:`~sqlalchemy.sql.expression.text`
  construct as a literal SQL string.  This approach still has the
  downside that the rendered SQL construct may not be backend-agnostic
  in all cases, so there is still a need for manual intervention in that
  small number of cases, but overall the majority of cases should work
  correctly now.  Big thanks to Carlos Rivera for pull requests and
  support on this.

.. change::
  :tags: feature

  SQLAlchemy's testing infrastructure is now used to run tests.
  This system supports both nose and pytest and opens the way
  for Alembic testing to support any number of backends, parallel
  testing, and 3rd party dialect testing.

.. change::
  :tags: changed, compatibility

  Minimum SQLAlchemy version is now 0.7.6, however at least
  0.8.4 is strongly recommended.  The overhaul of the test suite
  allows for fully passing tests on all SQLAlchemy versions
  from 0.7.6 on forward.

.. change::
  :tags: bug, operations

  The "match" keyword is not sent to :class:`.ForeignKeyConstraint`
  by :meth:`.Operations.create_foreign_key` when SQLAlchemy 0.7 is in use;
  this keyword was added to SQLAlchemy as of 0.8.0.

.. changelog::
:version: 0.6.7
:released: September 9, 2014

text
.. change::
  :tags: bug, mssql

  Fixed bug in MSSQL dialect where "rename table" wasn't using
  ``sp_rename()`` as is required on SQL Server.  Pull request courtesy
  ลukasz Boล‚dys.

.. change::
  :tags: feature
  :tickets: 222

  Added support for functional indexes when using the
  :meth:`.Operations.create_index` directive.   Within the list of columns,
  the SQLAlchemy ``text()`` construct can be sent, embedding a literal
  SQL expression; the :meth:`.Operations.create_index` will perform some hackery
  behind the scenes to get the :class:`.Index` construct to cooperate.
  This works around some current limitations in :class:`.Index`
  which should be resolved on the SQLAlchemy side at some point.

.. changelog::
:version: 0.6.6
:released: August 7, 2014

text
.. change::
  :tags: bug
  :tickets: 95

  A file named ``__init__.py`` in the ``versions/`` directory is now
  ignored by Alembic when the collection of version files is retrieved.
  Pull request courtesy Michael Floering.

.. change::
  :tags: bug

  Fixed Py3K bug where an attempt would be made to sort None against
  string values when autogenerate would detect tables across multiple
  schemas, including the default schema.  Pull request courtesy
  paradoxxxzero.

.. change::
  :tags: bug

  Autogenerate render will render the arguments within a Table construct
  using ``*[...]`` when the number of columns/elements is greater than
  255.  Pull request courtesy Ryan P. Kelly.

.. change::
  :tags: bug

  Fixed bug where foreign key constraints would fail to render in
  autogenerate when a schema name was present.  Pull request courtesy
  Andreas Zeidler.

.. change::
  :tags: bug
  :tickets: 212

  Some deep-in-the-weeds fixes to try to get "server default" comparison
  working better across platforms and expressions, in particular on
  the Postgresql backend, mostly dealing with quoting/not quoting of various
  expressions at the appropriate time and on a per-backend basis.
  Repaired and tested support for such defaults as Postgresql interval
  and array defaults.

.. change::
  :tags: enhancement
  :tickets: 209

  When a run of Alembic command line fails due to ``CommandError``,
  the output now prefixes the string with ``"FAILED:"``, and the error
  is also written to the log output using ``log.error()``.

.. change::
  :tags: bug
  :tickets: 208

  Liberalized even more the check for MySQL indexes that shouldn't be
  counted in autogenerate as "drops"; this time it's been reported
  that an implicitly created index might be named the same as a composite
  foreign key constraint, and not the actual columns, so we now skip those
  when detected as well.

.. change::
  :tags: feature

  Added a new accessor :attr:`.MigrationContext.config`, when used
  in conjunction with a :class:`.EnvironmentContext` and
  :class:`.Config`, this config will be returned.  Patch
  courtesy Marc Abramowitz.

.. changelog::
:version: 0.6.5
:released: May 3, 2014

text
.. change::
  :tags: bug, autogenerate, mysql
  :tickets: 202

  This releases' "autogenerate index detection" bug, when a MySQL table
  includes an Index with the same name as a column, autogenerate reported
  it as an "add" even though its not; this is because we ignore reflected
  indexes of this nature due to MySQL creating them implicitly.  Indexes
  that are named the same as a column are now ignored on
  MySQL if we see that the backend is reporting that it already exists;
  this indicates that we can still detect additions of these indexes
  but not drops, as we cannot distinguish a backend index same-named
  as the column as one that is user generated or mysql-generated.

.. change::
  :tags: feature, environment
  :tickets: 201

  Added new feature :paramref:`.EnvironmentContext.configure.transaction_per_migration`,
  which when True causes the BEGIN/COMMIT pair to incur for each migration
  individually, rather than for the whole series of migrations.  This is
  to assist with some database directives that need to be within individual
  transactions, without the need to disable transactional DDL entirely.

.. change::
  :tags: bug, autogenerate
  :tickets: 200

  Fixed bug where the ``include_object()`` filter would not receive
  the original :class:`.Column` object when evaluating a database-only
  column to be dropped; the object would not include the parent
  :class:`.Table` nor other aspects of the column that are important
  for generating the "downgrade" case where the column is recreated.

.. change::
  :tags: bug, environment
  :tickets: 195

  Fixed bug where :meth:`.EnvironmentContext.get_x_argument`
  would fail if the :class:`.Config` in use didn't actually
  originate from a command line call.

.. change::
  :tags: bug, autogenerate
  :tickets: 194

  Fixed another bug regarding naming conventions, continuing
  from :ticket:`183`, where add_index()
  drop_index() directives would not correctly render the ``f()``
  construct when the index contained a convention-driven name.

.. changelog::
:version: 0.6.4
:released: March 28, 2014

text
.. change::
  :tags: bug, mssql
  :tickets: 186

  Added quoting to the table name when the special EXEC is run to
  drop any existing server defaults or constraints when the
  :paramref:`.Operations.drop_column.mssql_drop_check` or
  :paramref:`.Operations.drop_column.mssql_drop_default`
  arguments are used.

.. change::
  :tags: bug, mysql
  :tickets: 103

  Added/fixed support for MySQL "SET DEFAULT" / "DROP DEFAULT" phrases,
  which will now be rendered if only the server default is changing
  or being dropped (e.g. specify None to alter_column() to indicate
  "DROP DEFAULT").  Also added support for rendering MODIFY rather than
  CHANGE when the column name isn't changing.

.. change::
  :tags: bug
  :tickets: 190

  Added support for the ``initially``, ``match`` keyword arguments
  as well as dialect-specific keyword arguments to
  :meth:`.Operations.create_foreign_key`.

  :tags: feature
  :tickets: 163

  Altered the support for "sourceless" migration files (e.g. only
  .pyc or .pyo present) so that the flag "sourceless=true" needs to
  be in alembic.ini for this behavior to take effect.

.. change::
  :tags: bug, mssql
  :tickets: 185

  The feature that keeps on giving, index/unique constraint autogenerate
  detection, has even more fixes, this time to accommodate database dialects
  that both don't yet report on unique constraints, but the backend
  does report unique constraints as indexes.   The logic
  Alembic uses to distinguish between "this is an index!" vs.
  "this is a unique constraint that is also reported as an index!" has now
  been further enhanced to not produce unwanted migrations when the dialect
  is observed to not yet implement get_unique_constraints() (e.g. mssql).
  Note that such a backend will no longer report index drops for unique
  indexes, as these cannot be distinguished from an unreported unique
  index.

.. change::
  :tags: bug
  :tickets: 183

  Extensive changes have been made to more fully support SQLAlchemy's new
  naming conventions feature.  Note that while SQLAlchemy has added this
  feature as of 0.9.2, some additional fixes in 0.9.4 are needed to
  resolve some of the issues:

  1. The :class:`.Operations` object now takes into account the naming
     conventions that are present on the :class:`.MetaData` object that's
     associated using :paramref:`~.EnvironmentContext.configure.target_metadata`.
     When :class:`.Operations` renders a constraint directive like
     ``ADD CONSTRAINT``, it now will make use of this naming convention
     when it produces its own temporary :class:`.MetaData` object.

  2. Note however that the autogenerate feature in most cases generates
     constraints like foreign keys and unique constraints with the
     final names intact; the only exception are the constraints implicit
     with a schema-type like Boolean or Enum.  In most of these cases,
     the naming convention feature will not take effect for these constraints
     and will instead use the given name as is, with one exception....

  3. Naming conventions which use the ``"%(constraint_name)s"`` token, that
     is, produce a new name that uses the original name as a component,
     will still be pulled into the naming convention converter and be
     converted.  The problem arises when autogenerate renders a constraint
     with it's already-generated name present in the migration file's source
     code, the name will be doubled up at render time due to the combination
     of #1 and #2.  So to work around this, autogenerate now renders these
     already-tokenized names using the new :meth:`.Operations.f` component.
     This component is only generated if **SQLAlchemy 0.9.4** or greater
     is in use.

  Therefore it is highly recommended that an upgrade to Alembic 0.6.4
  be accompanied by an upgrade of SQLAlchemy 0.9.4, if the new naming
  conventions feature is used.

  .. seealso::

      :ref:`autogen_naming_conventions`

.. change::
  :tags: bug
  :tickets: 160

  Suppressed IOErrors which can raise when program output pipe
  is closed under a program like ``head``; however this only
  works on Python 2.  On Python 3, there is not yet a known way to
  suppress the BrokenPipeError warnings without prematurely terminating
  the program via signals.

.. change::
  :tags: bug
  :tickets: 179

  Fixed bug where :meth:`.Operations.bulk_insert` would not function
  properly when :meth:`.Operations.inline_literal` values were used,
  either in --sql or non-sql mode.    The values will now render
  directly in --sql mode.  For compatibility with "online" mode,
  a new flag :paramref:`~.Operations.bulk_insert.multiinsert`
  can be set to False which will cause each parameter set to be
  compiled and executed with individual INSERT statements.

.. change::
  :tags: bug, py3k
  :tickets: 175

  Fixed a failure of the system that allows "legacy keyword arguments"
  to be understood, which arose as of a change in Python 3.4 regarding
  decorators.  A workaround is applied that allows the code to work
  across Python 3 versions.

.. change::
  :tags: feature

  The :func:`.command.revision` command now returns the :class:`.Script`
  object corresponding to the newly generated revision.  From this
  structure, one can get the revision id, the module documentation,
  and everything else, for use in scripts that call upon this command.
  Pull request courtesy Robbie Coomber.

.. changelog::
:version: 0.6.3
:released: February 2, 2014

text
.. change::
  :tags: bug
  :tickets: 172

  Added a workaround for when we call ``fcntl.ioctl()`` to get at
  ``TERMWIDTH``; if the function returns zero, as is reported to occur
  in some pseudo-ttys, the message wrapping system is disabled in the
  same way as if ``ioctl()`` failed.

.. change::
  :tags: feature
  :tickets: 171

  Added new argument
  :paramref:`.EnvironmentContext.configure.user_module_prefix`.
  This prefix is applied when autogenerate renders a user-defined type,
  which here is defined as any type that is from a module outside of the
  ``sqlalchemy.`` hierarchy.   This prefix defaults to ``None``, in
  which case the :paramref:`.EnvironmentContext.configure.sqlalchemy_module_prefix`
  is used, thus preserving the current behavior.

.. change::
  :tags: bug
  :tickets: 170

  Added support for autogenerate covering the use case where :class:`.Table`
  objects specified in the metadata have an explicit ``schema`` attribute
  whose name matches that of the connection's default schema
  (e.g. "public" for Postgresql).  Previously, it was assumed that "schema"
  was ``None`` when it matched the "default" schema, now the comparison
  adjusts for this.

.. change::
  :tags: bug

  The :func:`.compare_metadata` public API function now takes into
  account the settings for
  :paramref:`.EnvironmentContext.configure.include_object`,
  :paramref:`.EnvironmentContext.configure.include_symbol`,
  and :paramref:`.EnvironmentContext.configure.include_schemas`, in the
  same way that the ``--autogenerate`` command does.  Pull
  request courtesy Roman Podoliaka.

.. change::
  :tags: bug
  :tickets: 168

  Calling :func:`.bulk_insert` with an empty list will not emit any
  commands on the current connection.  This was already the case with
  ``--sql`` mode, so is now the case with "online" mode.

.. change::
  :tags: bug

 Enabled schema support for index and unique constraint autodetection;
 previously these were non-functional and could in some cases lead to
 attribute errors.  Pull request courtesy Dimitris Theodorou.

.. change::
  :tags: bug
  :tickets: 164

 More fixes to index autodetection; indexes created with expressions
 like DESC or functional indexes will no longer cause AttributeError
 exceptions when attempting to compare the columns.

.. change::
  :tags: feature
  :tickets: 163

 The :class:`.ScriptDirectory` system that loads migration files
 from a  ``versions/`` directory now supports so-called
 "sourceless" operation,  where the ``.py`` files are not present
 and instead ``.pyc`` or ``.pyo`` files are directly present where
 the ``.py`` files should be.  Note that while Python 3.3 has a
 new system of locating ``.pyc``/``.pyo`` files within a directory
 called ``__pycache__`` (e.g. PEP-3147), PEP-3147 maintains
 support for the "source-less imports" use case, where the
 ``.pyc``/``.pyo`` are in present in the "old" location, e.g. next
 to the ``.py`` file; this is the usage that's supported even when
 running Python3.3.

.. changelog::
:version: 0.6.2
:released: Fri Dec 27 2013

text
.. change::
  :tags: bug

  Autogenerate for ``op.create_table()`` will not include a
  ``PrimaryKeyConstraint()`` that has no columns.

.. change::
  :tags: bug

  Fixed bug in the not-internally-used :meth:`.ScriptDirectory.get_base`
  method which would fail if called on an empty versions directory.

.. change::
  :tags: bug
  :tickets: 157

  An almost-rewrite of the new unique constraint/index autogenerate
  detection, to accommodate a variety of issues.  The emphasis is on
  not generating false positives for those cases where no net change
  is present, as these errors are the ones that impact all autogenerate
  runs:

    * Fixed an issue with unique constraint autogenerate detection where
      a named ``UniqueConstraint`` on both sides with column changes would
      render with the "add" operation before the "drop", requiring the
      user to reverse the order manually.

    * Corrected for MySQL's apparent addition of an implicit index
      for a foreign key column, so that it doesn't show up as "removed".
      This required that the index/constraint autogen system query the
      dialect-specific implementation for special exceptions.

    * reworked the "dedupe" logic to accommodate MySQL's bi-directional
      duplication of unique indexes as unique constraints, and unique
      constraints as unique indexes.  Postgresql's slightly different
      logic of duplicating unique constraints into unique indexes
      continues to be accommodated as well.  Note that a unique index
      or unique constraint removal on a backend that duplicates these may
      show up as a distinct "remove_constraint()" / "remove_index()" pair,
      which may need to be corrected in the post-autogenerate if multiple
      backends are being supported.

    * added another dialect-specific exception to the SQLite backend
      when dealing with unnamed unique constraints, as the backend can't
      currently report on constraints that were made with this technique,
      hence they'd come out as "added" on every run.

    * the ``op.create_table()`` directive will be auto-generated with
      the ``UniqueConstraint`` objects inline, but will not double them
      up with a separate ``create_unique_constraint()`` call, which may
      have been occurring.  Indexes still get rendered as distinct
      ``op.create_index()`` calls even when the corresponding table was
      created in the same script.

    * the inline ``UniqueConstraint`` within ``op.create_table()`` includes
      all the options like ``deferrable``, ``initially``, etc.  Previously
      these weren't rendering.

.. change::
  :tags: feature, mssql

  Added new argument ``mssql_drop_foreign_key`` to
  :meth:`.Operations.drop_column`.  Like ``mssql_drop_default``
  and ``mssql_drop_check``, will do an inline lookup for a
  single foreign key which applies to this column, and drop it.
  For a column with more than one FK, you'd still need to explicitly
  use :meth:`.Operations.drop_constraint` given the name,
  even though only MSSQL has this limitation in the first place.

.. change::
  :tags: bug, mssql

  The MSSQL backend will add the batch separator (e.g. ``"GO"``)
  in ``--sql`` mode after the final ``COMMIT`` statement, to ensure
  that statement is also processed in batch mode.  Courtesy
  Derek Harland.

.. changelog::
:version: 0.6.1
:released: Wed Nov 27 2013

text
.. change::
  :tags: bug, mysql
  :tickets: 152

  Fixed bug where :func:`.op.alter_column` in the MySQL dialect
  would fail to apply quotes to column names that had mixed casing
  or spaces.

.. change::
  :tags: feature

  Expanded the size of the "slug" generated by "revision" to 40
  characters, which is also configurable by new field
  ``truncate_slug_length``; and also split on the word rather than the
  character; courtesy Frozenball.

.. change::
  :tags: bug
  :tickets: 135

  Fixed the output wrapping for Alembic message output, so that
  we either get the terminal width for "pretty printing" with
  indentation, or if not we just output the text as is; in any
  case the text won't be wrapped too short.

.. change::
  :tags: bug

  Fixes to Py3k in-place compatibility regarding output encoding and related;
  the use of the new io.* package introduced some incompatibilities on Py2k.
  These should be resolved, due to the introduction of new adapter types
  for translating from io.* to Py2k file types, StringIO types.
  Thanks to Javier Santacruz for help with this.

.. change::
  :tags: bug
  :tickets: 145

  Fixed py3k bug where the wrong form of ``next()`` was being called
  when using the list_templates command.  Courtesy Chris Wilkes.

.. change::
  :tags: feature
  :tickets: 107

  Support for autogeneration detection and rendering of indexes and
  unique constraints has been added.  The logic goes through some effort
  in order to differentiate between true unique constraints and
  unique indexes, where there are some quirks on backends like Postgresql.
  The effort here in producing the feature and tests is courtesy of IJL.

.. change::
  :tags: bug

  Fixed bug introduced by new ``include_object`` argument where the
  inspected column would be misinterpreted when using a user-defined
  type comparison function, causing a KeyError or similar expression-related
  error.  Fix courtesy Maarten van Schaik.

.. change::
  :tags: bug

  Added the "deferrable" keyword argument to :func:`.op.create_foreign_key`
  so that ``DEFERRABLE`` constraint generation is supported; courtesy
  Pedro Romano.

.. change::
  :tags: bug
  :tickets: 137

  Ensured that strings going to stdout go through an encode/decode phase,
  so that any non-ASCII characters get to the output stream correctly
  in both Py2k and Py3k.   Also added source encoding detection using
  Mako's parse_encoding() routine in Py2k so that the __doc__ of a
  non-ascii revision file can be treated as unicode in Py2k.

.. changelog::
:version: 0.6.0
:released: Fri July 19 2013

text
.. change::
  :tags: feature
  :tickets: 101

  Added new kw argument to :meth:`.EnvironmentContext.configure`
  ``include_object``.  This is a more flexible version of the
  ``include_symbol`` argument which allows filtering of columns as well as tables
  from the autogenerate process,
  and in the future will also work for types, constraints and
  other constructs.  The fully constructed schema object is passed,
  including its name and type as well as a flag indicating if the object
  is from the local application metadata or is reflected.

.. change::
  :tags: feature

  The output of the ``alembic history`` command is now
  expanded to show information about each change on multiple
  lines, including the full top message,
  resembling the formatting of git log.

.. change::
  :tags: feature

  Added :attr:`alembic.config.Config.cmd_opts` attribute,
  allows access to the ``argparse`` options passed to the
  ``alembic`` runner.

.. change::
  :tags: feature
  :tickets: 120

  Added new command line argument ``-x``, allows extra arguments
  to be appended to the command line which can be consumed
  within an ``env.py`` script by looking at
  ``context.config.cmd_opts.x``, or more simply a new
  method :meth:`.EnvironmentContext.get_x_argument`.

.. change::
  :tags: bug
  :tickets: 125

  Added support for options like "name" etc. to be rendered
  within CHECK constraints in autogenerate.  Courtesy
  Sok Ann Yap.

.. change::
  :tags: misc

  Source repository has been moved from Mercurial to Git.

.. change::
  :tags: bug

  Repaired autogenerate rendering of ForeignKeyConstraint
  to include use_alter argument, if present.

.. change::
  :tags: feature

  Added ``-r`` argument to ``alembic history`` command,
  allows specification of ``[start]:[end]`` to view
  a slice of history.  Accepts revision numbers, symbols
  "base", "head", a new symbol "current" representing the
  current migration, as well as relative ranges for one
  side at a time (i.e. ``-r-5:head``, ``-rcurrent:+3``).
  Courtesy Atsushi Odagiri for this feature.

.. change::
  :tags: feature
  :tickets: 55

  Source base is now in-place for Python 2.6 through
  3.3, without the need for 2to3.   Support for Python 2.5
  and below has been dropped.   Huge thanks to
  Hong Minhee for all the effort on this!

.. changelog::
:version: 0.5.0
:released: Thu Apr 4 2013

text
.. note::

  Alembic 0.5.0 now requires at least
  version 0.7.3 of SQLAlchemy to run properly.
  Support for 0.6 has been dropped.

.. change::
    :tags: feature
    :tickets: 76

  Added ``version_table_schema`` argument
  to :meth:`.EnvironmentContext.configure`,
  complements the ``version_table`` argument to
  set an optional remote schema for the version
  table.  Courtesy Christian Blume.

.. change::
    :tags: bug, postgresql
    :tickets: 32

  Fixed format of RENAME for table that includes
  schema with Postgresql; the schema name shouldn't
  be in the "TO" field.

.. change::
    :tags: feature
    :tickets: 90

  Added ``output_encoding`` option to
  :meth:`.EnvironmentContext.configure`,
  used with ``--sql`` mode to apply an encoding
  to the output stream.

.. change::
    :tags: feature
    :tickets: 93

  Added :meth:`.Operations.create_primary_key`
  operation, will generate an ADD CONSTRAINT
  for a primary key.

.. change::
    :tags: bug, mssql
    :tickets: 109

  Fixed bug whereby double quoting would be applied
  to target column name during an ``sp_rename``
  operation.

.. change::
    :tags: bug, sqlite, mysql
    :tickets: 112

  transactional_ddl flag for SQLite, MySQL dialects
  set to False.  MySQL doesn't support it,
  SQLite does but current pysqlite driver does not.

.. change::
    :tags: feature
    :tickets: 115

  upgrade and downgrade commands will list the
  first line of the docstring out next to the
  version number.  Courtesy Hong Minhee.

.. change::
    :tags: feature

  Added --head-only option to "alembic current",
  will print current version plus the symbol
  "(head)" if this version is the head or not.
  Courtesy Charles-Axel Dein.

.. change::
    :tags: bug
    :tickets: 110

  Autogenerate will render additional table keyword
  arguments like "mysql_engine" and others within
  op.create_table().

.. change::
    :tags: feature
    :tickets: 108

  The rendering of any construct during autogenerate
  can be customized, in particular to allow special rendering
  for user-defined column, constraint subclasses, using new
  ``render_item`` argument to
  :meth:`.EnvironmentContext.configure`.

.. change::
    :tags: bug

  Fixed bug whereby create_index()
  would include in the constraint columns that
  are added to all Table objects using events,
  externally to the generation of the constraint.
  This is the same issue that was fixed for unique
  constraints in version 0.3.2.

.. change::
    :tags: bug

  Worked around a backwards-incompatible regression in Python3.3
  regarding argparse; running "alembic" with no arguments
  now yields an informative error in py3.3 as with all previous versions.
  Courtesy Andrey Antukh.

.. change::
    :tags: change

  SQLAlchemy 0.6 is no longer supported by Alembic - minimum version is 0.7.3,
  full support is as of 0.7.9.

.. change::
    :tags: bug
    :tickets: 104

  A host of argument name changes within migration
  operations for consistency.  Keyword arguments
  will continue to work on the old name for backwards compatibility,
  however required positional arguments will not:

    :meth:`.Operations.alter_column` - ``name`` -> ``new_column_name`` - old
    name will work for backwards compatibility.

    :meth:`.Operations.create_index` - ``tablename`` -> ``table_name`` -
    argument is positional.

    :meth:`.Operations.drop_index` - ``tablename`` -> ``table_name`` - old
    name will work for backwards compatibility.

    :meth:`.Operations.drop_constraint` - ``tablename`` -> ``table_name`` -
    argument is positional.

    :meth:`.Operations.drop_constraint` - ``type`` -> ``type_`` - old
    name will work for backwards compatibility

.. changelog::
:version: 0.4.2
:released: Fri Jan 11 2013

text
.. change::
    :tags: bug, autogenerate
    :tickets: 99

  Fixed bug where autogenerate would fail if a Column
  to be added to a table made use of the ".key" parameter.

.. change::
    :tags: bug, sqlite
    :tickets: 98

  The "implicit" constraint generated by a
  type such as Boolean or Enum will not generate an
  ALTER statement when run on SQlite, which does not
  support ALTER for the purpose of adding/removing
  constraints separate from the column def itself.
  While SQLite supports adding a CHECK constraint
  at the column level, SQLAlchemy would need modification
  to support this.
  A warning is emitted indicating this
  constraint cannot be added in this scenario.

.. change::
    :tags: bug
    :tickets: 96

  Added a workaround to setup.py to prevent
  "NoneType" error from occurring when
  "setup.py test" is run.

.. change::
    :tags: bug
    :tickets: 96

  Added an append_constraint() step to each
  condition within
  test_autogenerate:AutogenRenderTest.test_render_fk_constraint_kwarg
  if the SQLAlchemy version is less than 0.8, as ForeignKeyConstraint
  does not auto-append prior to 0.8.

.. change::
    :tags: feature
    :tickets: 96

  Added a README.unittests with instructions for running the test
  suite fully.

.. changelog::
:version: 0.4.1
:released: Sun Dec 9 2012

text
.. change::
    :tags: bug
    :tickets: 92

  Added support for autogenerate render of
  ForeignKeyConstraint options onupdate,
  ondelete, initially, and deferred.

.. change::
    :tags: bug
    :tickets: 94

  Autogenerate will include "autoincrement=False"
  in the rendered table metadata
  if this flag was set to false on the source
  :class:`.Column` object.

.. change::
    :tags: feature
    :tickets: 66

  Explicit error message describing the case
  when downgrade --sql is used without specifying
  specific start/end versions.

.. change::
    :tags: bug
    :tickets: 81

  Removed erroneous "emit_events" attribute
  from operations.create_table() documentation.

.. change::
    :tags: bug
    :tickets:

  Fixed the minute component in file_template
  which returned the month part of the create date.

.. changelog::
:version: 0.4.0
:released: Mon Oct 01 2012

text
.. change::
    :tags: feature
    :tickets: 33

  Support for tables in alternate schemas
  has been added fully to all operations, as well as to
  the autogenerate feature.  When using autogenerate,
  specifying the flag include_schemas=True to
  Environment.configure() will also cause autogenerate
  to scan all schemas located by Inspector.get_schema_names(),
  which is supported by *some* (but not all)
  SQLAlchemy dialects including Postgresql.
  *Enormous* thanks to Bruno Binet for a huge effort
  in implementing as well as writing tests. .

.. change::
    :tags: feature
    :tickets: 70

  The command line runner has been organized
  into a reusable CommandLine object, so that other
  front-ends can reuse the argument parsing built
  in.

.. change::
    :tags: feature
    :tickets: 43

  Added "stdout" option to Config, provides
  control over where the "print" output of commands like
  "history", "init", "current" etc. are sent.

.. change::
    :tags: bug
    :tickets: 71

  Fixed the "multidb" template which was badly out
  of date.   It now generates revision files using
  the configuration to determine the different
  upgrade_<xyz>() methods needed as well, instead of
  needing to hardcode these.  Huge thanks to
  BryceLohr for doing the heavy lifting here.

.. change::
    :tags: bug
    :tickets: 72

  Fixed the regexp that was checking for .py files
  in the version directory to allow any .py file through.
  Previously it was doing some kind of defensive checking,
  probably from some early notions of how this directory
  works, that was prohibiting various filename patterns
  such as those which begin with numbers.

.. change::
    :tags: bug
    :tickets:

  Fixed MySQL rendering for server_default which
  didn't work if the server_default was a generated
  SQL expression.  Courtesy Moriyoshi Koizumi.

.. change::
    :tags: feature
    :tickets:

  Added support for alteration of MySQL
  columns that have AUTO_INCREMENT, as well as enabling
  this flag.  Courtesy Moriyoshi Koizumi.

.. changelog::
:version: 0.3.6
:released: Wed Aug 15 2012

text
.. change::
    :tags: feature
    :tickets: 27

  Added include_symbol option to
  EnvironmentContext.configure(),
  specifies a callable which will include/exclude tables
  in their entirety from the autogeneration process
  based on name.

.. change::
    :tags: feature
    :tickets: 59

  Added year, month, day, hour, minute, second
  variables to file_template.

.. change::
    :tags: feature
    :tickets:

  Added 'primary' to the list of constraint types
  recognized for MySQL drop_constraint().

.. change::
    :tags: feature
    :tickets:

  Added --sql argument to the "revision" command,
  for the use case where the "revision_environment"
  config option is being used but SQL access isn't
  desired.

.. change::
    :tags: bug
    :tickets:

  Repaired create_foreign_key() for
  self-referential foreign keys, which weren't working
  at all.

.. change::
    :tags: bug
    :tickets: 63

  'alembic' command reports an informative
  error message when the configuration is missing
  the 'script_directory' key.

.. change::
    :tags: bug
    :tickets: 62

  Fixes made to the constraints created/dropped
  alongside so-called "schema" types such as
  Boolean and Enum.  The create/drop constraint logic
  does not kick in when using a dialect that doesn't
  use constraints for these types, such as postgresql,
  even when existing_type is specified to
  alter_column().  Additionally, the constraints
  are not affected if existing_type is passed but
  type\_ is not, i.e. there's no net change
  in type.

.. change::
    :tags: bug
    :tickets: 66

  Improved error message when specifying
  non-ordered revision identifiers to cover
  the case when the "higher" rev is None,
  improved message overall.

.. changelog::
:version: 0.3.5
:released: Sun Jul 08 2012

text
.. change::
    :tags: bug
    :tickets: 31

  Fixed issue whereby reflected server defaults
  wouldn't be quoted correctly; uses repr() now.

.. change::
    :tags: bug
    :tickets: 58

  Fixed issue whereby when autogenerate would
  render create_table() on the upgrade side for a
  table that has a Boolean type, an unnecessary
  CheckConstraint() would be generated.

.. change::
    :tags: feature
    :tickets:

  Implemented SQL rendering for
  CheckConstraint() within autogenerate upgrade,
  including for literal SQL as well as SQL Expression
  Language expressions.

.. changelog::
:version: 0.3.4
:released: Sat Jun 02 2012

text
.. change::
    :tags: bug
    :tickets:

  Fixed command-line bug introduced by the
  "revision_environment" feature.

.. changelog::
:version: 0.3.3
:released: Sat Jun 02 2012

text
.. change::
    :tags: feature
    :tickets:

  New config argument
  "revision_environment=true", causes env.py to
  be run unconditionally when the "revision" command
  is run, to support script.py.mako templates with
  dependencies on custom "template_args".

.. change::
    :tags: feature
    :tickets:

  Added "template_args" option to configure()
  so that an env.py can add additional arguments
  to the template context when running the
  "revision" command.  This requires either --autogenerate
  or the configuration directive "revision_environment=true".

.. change::
    :tags: bug
    :tickets: 44

  Added "type" argument to op.drop_constraint(),
  and implemented full constraint drop support for
  MySQL.  CHECK and undefined raise an error.
  MySQL needs the constraint type
  in order to emit a DROP CONSTRAINT.

.. change::
    :tags: feature
    :tickets: 34

  Added version_table argument to
  EnvironmentContext.configure(), allowing for the
  configuration of the version table name.

.. change::
    :tags: feature
    :tickets:

  Added support for "relative" migration
  identifiers, i.e. "alembic upgrade +2",
  "alembic downgrade -1".  Courtesy
  Atsushi Odagiri for this feature.

.. change::
    :tags: bug
    :tickets: 49

  Fixed bug whereby directories inside of
  the template directories, such as __pycache__
  on Pypy, would mistakenly be interpreted as
  files which are part of the template.

.. changelog::
:version: 0.3.2
:released: Mon Apr 30 2012

text
.. change::
    :tags: feature
    :tickets: 40

  Basic support for Oracle added,
  courtesy shgoh.

.. change::
    :tags: feature
    :tickets:

  Added support for UniqueConstraint
  in autogenerate, courtesy Atsushi Odagiri

.. change::
    :tags: bug
    :tickets:

  Fixed support of schema-qualified
  ForeignKey target in column alter operations,
  courtesy Alexander Kolov.

.. change::
    :tags: bug
    :tickets:

  Fixed bug whereby create_unique_constraint()
  would include in the constraint columns that
  are added to all Table objects using events,
  externally to the generation of the constraint.

.. changelog::
:version: 0.3.1
:released: Sat Apr 07 2012

text
.. change::
    :tags: bug
    :tickets: 41

  bulk_insert() fixes:

    1. bulk_insert() operation was
       not working most likely since the 0.2 series
       when used with an engine.
    2. Repaired bulk_insert() to complete when
       used against a lower-case-t table and executing
       with only one set of parameters, working
       around SQLAlchemy bug #2461 in this regard.
    3. bulk_insert() uses "inline=True" so that phrases
       like RETURNING and such don't get invoked for
       single-row bulk inserts.
    4. bulk_insert() will check that you're passing
       a list of dictionaries in, raises TypeError
       if not detected.

.. changelog::
:version: 0.3.0
:released: Thu Apr 05 2012

text
.. change::
    :tags: general
    :tickets:

  The focus of 0.3 is to clean up
  and more fully document the public API of Alembic,
  including better accessors on the MigrationContext
  and ScriptDirectory objects.  Methods that are
  not considered to be public on these objects have
  been underscored, and methods which should be public
  have been cleaned up and documented, including:

    MigrationContext.get_current_revision()
    ScriptDirectory.iterate_revisions()
    ScriptDirectory.get_current_head()
    ScriptDirectory.get_heads()
    ScriptDirectory.get_base()
    ScriptDirectory.generate_revision()

.. change::
    :tags: feature
    :tickets:

  Added a bit of autogenerate to the
  public API in the form of the function
  alembic.autogenerate.compare_metadata.

.. changelog::
:version: 0.2.2
:released: Mon Mar 12 2012

text
.. change::
    :tags: feature
    :tickets:

  Informative error message when op.XYZ
  directives are invoked at module import time.

.. change::
    :tags: bug
    :tickets: 35

  Fixed inappropriate direct call to
  util.err() and therefore sys.exit()
  when Config failed to locate the
  config file within library usage.

.. change::
    :tags: bug
    :tickets:

  Autogenerate will emit CREATE TABLE
  and DROP TABLE directives according to
  foreign key dependency order.

.. change::
    :tags: bug
    :tickets:

  implement 'tablename' parameter on
  drop_index() as this is needed by some
  backends.

.. change::
    :tags: feature
    :tickets:

  Added execution_options parameter
  to op.execute(), will call execution_options()
  on the Connection before executing.

  The immediate use case here is to allow
  access to the new no_parameters option
  in SQLAlchemy 0.7.6, which allows
  some DBAPIs (psycopg2, MySQLdb) to allow
  percent signs straight through without
  escaping, thus providing cross-compatible
  operation with DBAPI execution and
  static script generation.

.. change::
    :tags: bug
    :tickets:

  setup.py won't install argparse if on
  Python 2.7/3.2

.. change::
    :tags: feature
    :tickets: 29

  script_location can be interpreted
  by pkg_resources.resource_filename(), if
  it is a non-absolute URI that contains
  colons.   This scheme is the same
  one used by Pyramid.

.. change::
    :tags: feature
    :tickets:

  added missing support for
  onupdate/ondelete flags for
  ForeignKeyConstraint, courtesy Giacomo Bagnoli

.. change::
    :tags: bug
    :tickets: 30

  fixed a regression regarding an autogenerate
  error message, as well as various glitches
  in the Pylons sample template.  The Pylons sample
  template requires that you tell it where to
  get the Engine from now.  courtesy
  Marcin Kuzminski

.. change::
    :tags: bug
    :tickets:

  drop_index() ensures a dummy column
  is added when it calls "Index", as SQLAlchemy
  0.7.6 will warn on index with no column names.

.. changelog::
:version: 0.2.1
:released: Tue Jan 31 2012

text
.. change::
    :tags: bug
    :tickets: 26

  Fixed the generation of CHECK constraint,
  regression from 0.2.0

.. changelog::
:version: 0.2.0
:released: Mon Jan 30 2012

text
.. change::
    :tags: feature
    :tickets: 19

  API rearrangement allows everything
  Alembic does to be represented by contextual
  objects, including EnvironmentContext,
  MigrationContext, and Operations.   Other
  libraries and applications can now use
  things like "alembic.op" without relying
  upon global configuration variables.
  The rearrangement was done such that
  existing migrations should be OK,
  as long as they use the pattern
  of "from alembic import context" and
  "from alembic import op", as these
  are now contextual objects, not modules.

.. change::
    :tags: feature
    :tickets: 24

  The naming of revision files can
  now be customized to be some combination
  of "rev id" and "slug", the latter of which
  is based on the revision message.
  By default, the pattern "<rev>_<slug>"
  is used for new files.   New script files
  should include the "revision" variable
  for this to work, which is part of
  the newer script.py.mako scripts.

.. change::
    :tags: bug
    :tickets: 25

  env.py templates call
  connection.close() to better support
  programmatic usage of commands; use
  NullPool in conjunction with create_engine()
  as well so that no connection resources
  remain afterwards.

.. change::
    :tags: bug
    :tickets: 22

  fix the config.main() function to honor
  the arguments passed, remove no longer used
  "scripts/alembic" as setuptools creates this
  for us.

.. change::
    :tags: bug
    :tickets:

  Fixed alteration of column type on
  MSSQL to not include the keyword "TYPE".

.. change::
    :tags: feature
    :tickets: 23

  Can create alembic.config.Config
  with no filename, use set_main_option()
  to add values.  Also added set_section_option()
  which will add sections.

.. changelog::
:version: 0.1.1
:released: Wed Jan 04 2012

text
.. change::
    :tags: bug
    :tickets:

  Clean up file write operations so that
  file handles are closed.

.. change::
    :tags: feature
    :tickets:

  PyPy is supported.

.. change::
    :tags: feature
    :tickets:

  Python 2.5 is supported, needs
  __future__.with_statement

.. change::
    :tags: bug
    :tickets:

  Fix autogenerate so that "pass" is
  generated between the two comments
  if no net migrations were present.

.. change::
    :tags: bug
    :tickets: 16

  Fix autogenerate bug that prevented
  correct reflection of a foreign-key
  referenced table in the list of "to remove".

.. change::
    :tags: bug
    :tickets: 17

  Fix bug where create_table() didn't
  handle self-referential foreign key
  correctly

.. change::
    :tags: bug
    :tickets: 18

  Default prefix for autogenerate
  directives is "op.", matching the
  mako templates.

.. change::
    :tags: feature
    :tickets: 18

  Add alembic_module_prefix argument
  to configure() to complement
  sqlalchemy_module_prefix.

.. change::
    :tags: bug
    :tickets: 14

  fix quotes not being rendered in
  ForeignKeConstraint during
  autogenerate

.. changelog::
:version: 0.1.0
:released: Wed Nov 30 2011

text
.. change::
    :tags:
    :tickets:

  Initial release.  Status of features:

.. change::
    :tags:
    :tickets:

  Alembic is used in at least one production
  environment, but should still be considered
  ALPHA LEVEL SOFTWARE as of this release,
  particularly in that many features are expected
  to be missing / unimplemented.   Major API
  changes are not anticipated but for the moment
  nothing should be assumed.

  The author asks that you *please* report all
  issues, missing features, workarounds etc.
  to the bugtracker.

.. change::
    :tags:
    :tickets:

  Python 3 is supported and has been tested.

.. change::
    :tags:
    :tickets:

  The "Pylons" and "MultiDB" environment templates
  have not been directly tested - these should be
  considered to be samples to be modified as
  needed.   Multiple database support itself
  is well tested, however.

.. change::
    :tags:
    :tickets:

  Postgresql and MS SQL Server environments
  have been tested for several weeks in a production
  environment.  In particular, some involved workarounds
  were implemented to allow fully-automated dropping
  of default- or constraint-holding columns with
  SQL Server.

.. change::
    :tags:
    :tickets:

  MySQL support has also been implemented to a
  basic degree, including MySQL's awkward style
  of modifying columns being accommodated.

.. change::
    :tags:
    :tickets:

  Other database environments not included among
  those three have *not* been tested, *at all*.  This
  includes Firebird, Oracle, Sybase.   Adding
  support for these backends should be
  straightforward.  Please report all missing/
  incorrect behaviors to the bugtracker! Patches
  are welcome here but are optional - please just
  indicate the exact format expected by the target
  database.

.. change::
    :tags:
    :tickets:

  SQLite, as a backend, has almost no support for
  schema alterations to existing databases.  The author
  would strongly recommend that SQLite not be used in
  a migration context - just dump your SQLite database
  into an intermediary format, then dump it back
  into a new schema.  For dev environments, the
  dev installer should be building the whole DB from
  scratch.  Or just use Postgresql, which is a much
  better database for non-trivial schemas.
  Requests for full ALTER support on SQLite should be
  reported to SQLite's bug tracker at
  http://www.sqlite.org/src/wiki?name=Bug+Reports,
  as Alembic will not be implementing the
  "rename the table to a temptable then copy the
  data into a new table" workaround.
  Note that Alembic will at some point offer an
  extensible API so that you can implement commands
  like this yourself.

.. change::
    :tags:
    :tickets:

  Well-tested directives include add/drop table, add/drop
  column, including support for SQLAlchemy "schema"
  types which generate additional CHECK
  constraints, i.e. Boolean, Enum.  Other directives not
  included here have *not* been strongly tested
  in production, i.e. rename table, etc.

.. change::
    :tags:
    :tickets:

  Both "online" and "offline" migrations, the latter
  being generated SQL scripts to hand off to a DBA,
  have been strongly production tested against
  Postgresql and SQL Server.

.. change::
    :tags:
    :tickets:

  Modify column type, default status, nullable, is
  functional and tested across PG, MSSQL, MySQL,
  but not yet widely tested in production usage.

.. change::
    :tags:
    :tickets:

  Many migrations are still outright missing, i.e.
  create/add sequences, etc.  As a workaround,
  execute() can be used for those which are missing,
  though posting of tickets for new features/missing
  behaviors is strongly encouraged.

.. change::
    :tags:
    :tickets:

  Autogenerate feature is implemented and has been
  tested, though only a little bit in a production setting.
  In particular, detection of type and server
  default changes are optional and are off by default;
  they can also be customized by a callable.
  Both features work but can have surprises particularly
  the disparity between BIT/TINYINT and boolean,
  which hasn't yet been worked around, as well as
  format changes performed by the database on defaults
  when it reports back.  When enabled, the PG dialect
  will execute the two defaults to be compared to
  see if they are equivalent.  Other backends may
  need to do the same thing.

  The autogenerate feature only generates
  "candidate" commands which must be hand-tailored
  in any case, so is still a useful feature and
  is safe to use.  Please report missing/broken features
  of autogenerate!  This will be a great feature and
  will also improve SQLAlchemy's reflection services.

.. change::
    :tags:
    :tickets:

  Support for non-ASCII table, column and constraint
  names is mostly nonexistent.   This is also a
  straightforward feature add as SQLAlchemy itself
  supports unicode identifiers; Alembic itself will
  likely need fixes to logging, column identification
  by key, etc. for full support here.

Build Cookbook (`docs/build/cookbook.rst`)

======== Cookbook

A collection of "How-Tos" highlighting popular ways to extend
Alembic.

.. note::

text
This is a new section where we catalogue various "how-tos"
based on user requests.  It is often the case that users
will request a feature only to learn it can be provided with
a simple customization.

.. _building_uptodate:

Building an Up to Date Database from Scratch

There's a theory of database migrations that says that the revisions in existence for a database should be
able to go from an entirely blank schema to the finished product, and back again. Alembic can roll
this way. Though we think it's kind of overkill, considering that SQLAlchemy itself can emit
the full CREATE statements for any given model using :meth:~sqlalchemy.schema.MetaData.create_all. If you check out
a copy of an application, running this will give you the entire database in one shot, without the need
to run through all those migration files, which are instead tailored towards applying incremental
changes to an existing database.

Alembic can integrate with a :meth:~sqlalchemy.schema.MetaData.create_all script quite easily. After running the
create operation, tell Alembic to create a new version table, and to stamp it with the most recent
revision (i.e. head)::

text
# inside of a "create the database" script, first create
# tables:
my_metadata.create_all(engine)

# then, load the Alembic configuration and generate the
# version table, "stamping" it with the most recent rev:
from alembic.config import Config
from alembic import command
alembic_cfg = Config("/path/to/yourapp/alembic.ini")
command.stamp(alembic_cfg, "head")

When this approach is used, the application can generate the database using normal SQLAlchemy
techniques instead of iterating through hundreds of migration scripts. Now, the purpose of the
migration scripts is relegated just to movement between versions on out-of-date databases, not
new databases. You can now remove old migration files that are no longer represented
on any existing environments.

To prune old migration files, simply delete the files. Then, in the earliest, still-remaining
migration file, set down_revision to None::

text
# replace this:
#down_revision = '290696571ad2'

# with this:
down_revision = None

That file now becomes the "base" of the migration series.

Conditional Migration Elements

This example features the basic idea of a common need, that of affecting
how a migration runs based on command line switches.

The technique to use here is simple; within a migration script, inspect
the :meth:.EnvironmentContext.get_x_argument collection for any additional,
user-defined parameters. Then take action based on the presence of those
arguments.

To make it such that the logic to inspect these flags is easy to use and
modify, we modify our script.py.mako template to make this feature
available in all new revision files:

.. code-block:: mako

text
"""${message}

Revision ID: ${up_revision}
Revises: ${down_revision}
Create Date: ${create_date}

"""

# revision identifiers, used by Alembic.
revision = ${repr(up_revision)}
down_revision = ${repr(down_revision)}

from alembic import op
import sqlalchemy as sa
${imports if imports else ""}

from alembic import context


def upgrade():
    schema_upgrades()
    if context.get_x_argument(as_dictionary=True).get('data', None):
        data_upgrades()

def downgrade():
    if context.get_x_argument(as_dictionary=True).get('data', None):
        data_downgrades()
    schema_downgrades()

def schema_upgrades():
    """schema upgrade migrations go here."""
    ${upgrades if upgrades else "pass"}

def schema_downgrades():
    """schema downgrade migrations go here."""
    ${downgrades if downgrades else "pass"}

def data_upgrades():
    """Add any optional data upgrade migrations here!"""
    pass

def data_downgrades():
    """Add any optional data downgrade migrations here!"""
    pass

Now, when we create a new migration file, the data_upgrades() and data_downgrades()
placeholders will be available, where we can add optional data migrations::

text
"""rev one

Revision ID: 3ba2b522d10d
Revises: None
Create Date: 2014-03-04 18:05:36.992867

"""

# revision identifiers, used by Alembic.
revision = '3ba2b522d10d'
down_revision = None

from alembic import op
import sqlalchemy as sa
from sqlalchemy import String, Column
from sqlalchemy.sql import table, column

from alembic import context

def upgrade():
    schema_upgrades()
    if context.get_x_argument(as_dictionary=True).get('data', None):
        data_upgrades()

def downgrade():
    if context.get_x_argument(as_dictionary=True).get('data', None):
        data_downgrades()
    schema_downgrades()

def schema_upgrades():
    """schema upgrade migrations go here."""
    op.create_table("my_table", Column('data', String))

def schema_downgrades():
    """schema downgrade migrations go here."""
    op.drop_table("my_table")

def data_upgrades():
    """Add any optional data upgrade migrations here!"""

    my_table = table('my_table',
        column('data', String),
    )

    op.bulk_insert(my_table,
        [
            {'data': 'data 1'},
            {'data': 'data 2'},
            {'data': 'data 3'},
        ]
    )

def data_downgrades():
    """Add any optional data downgrade migrations here!"""

    op.execute("delete from my_table")

To invoke our migrations with data included, we use the -x flag::

text
alembic -x data=true upgrade head

The :meth:.EnvironmentContext.get_x_argument is an easy way to support
new commandline options within environment and migration scripts.

.. _connection_sharing:

Sharing a Connection across one or more programmatic migration commands

It is often the case that an application will need to call upon a series
of commands within :ref:alembic.command.toplevel, where it would be advantageous
for all operations to proceed along a single transaction. The connectivity
for a migration is typically solely determined within the env.py script
of a migration environment, which is called within the scope of a command.

The steps to take here are:

  1. Produce the :class:~sqlalchemy.engine.Connection object to use.

  2. Place it somewhere that env.py will be able to access it. This
    can be either a. a module-level global somewhere, or b.
    an attribute which we place into the :attr:.Config.attributes
    dictionary (if we are on an older Alembic version, we may also attach
    an attribute directly to the :class:.Config object).

  3. The env.py script is modified such that it looks for this
    :class:~sqlalchemy.engine.Connection and makes use of it, in lieu
    of building up its own :class:~sqlalchemy.engine.Engine instance.

We illustrate using :attr:.Config.attributes a script that will run
the :func:.command.upgrade command programmatically within a
transaction declared in a Python file::

text
from alembic import command, config

cfg = config.Config("/path/to/yourapp/alembic.ini")
with engine.begin() as connection:
    cfg.attributes['connection'] = connection
    command.upgrade(cfg, "head")

Then in env.py we can update run_migrations_online::

text
def run_migrations_online():
    connectable = config.attributes.get('connection', None)

    if connectable is None:
        # only create Engine if we don't have a Connection
        # from the outside
        connectable = engine_from_config(
            config.get_section(config.config_ini_section),
            prefix='sqlalchemy.',
            poolclass=pool.NullPool)

        with connectable.connect() as connection:
            context.configure(
                connection=connection, target_metadata=target_metadata
            )

            with context.begin_transaction():
                context.run_migrations()
    else:
        context.configure(
            connection=connectable,
            target_metadata=target_metadata
        )

        with context.begin_transaction():
            context.run_migrations()

.. _replaceable_objects:

Replaceable Objects

This recipe proposes a hypothetical way of dealing with
what we might call a replaceable schema object. A replaceable object
is a schema object that needs to be created and dropped all at once.
Examples of such objects include views, stored procedures, and triggers.

.. seealso::

text
The Replaceable Object concept has been integrated by the
`Alembic Utils <https://github.com/olirice/alembic_utils>`_ project,
which provides autogenerate and migration
support for PostgreSQL functions and views.   See
Alembic Utils at https://github.com/olirice/alembic_utils .

Replaceable objects present a problem in that in order to make incremental
changes to them, we have to refer to the whole definition at once.
If we need to add a new column to a view, for example, we have to drop
it entirely and recreate it fresh with the extra column added, referring to
the whole structure; but to make it even tougher, if we wish to support
downgrade operations in our migration scripts,
we need to refer to the previous version of that
construct fully, and we'd much rather not have to type out the whole
definition in multiple places.

This recipe proposes that we may refer to the older version of a
replaceable construct by directly naming the migration version in
which it was created, and having a migration refer to that previous
file as migrations run. We will also demonstrate how to integrate this
logic within the :ref:operation_plugins feature introduced in
Alembic 0.8. It may be very helpful to review
this section first to get an overview of this API.

The Replaceable Object Structure

We first need to devise a simple format that represents the "CREATE XYZ" /
"DROP XYZ" aspect of what it is we're building. We will work with an object
that represents a textual definition; while a SQL view is an object that we can define
using a table-metadata-like system <https://github.com/sqlalchemy/sqlalchemy/wiki/UsageRecipes/Views>_,
this is not so much the case for things like stored procedures, where
we pretty much need to have a full string definition written down somewhere.
We'll use a simple value object called ReplaceableObject that can
represent any named set of SQL text to send to a "CREATE" statement of
some kind::

text
class ReplaceableObject:
    def __init__(self, name, sqltext):
        self.name = name
        self.sqltext = sqltext

Using this object in a migration script, assuming a Postgresql-style
syntax, looks like::

text
customer_view = ReplaceableObject(
    "customer_view",
    "SELECT name, order_count FROM customer WHERE order_count > 0"
)

add_customer_sp = ReplaceableObject(
    "add_customer_sp(name varchar, order_count integer)",
    """
    RETURNS integer AS $$
    BEGIN
        insert into customer (name, order_count)
        VALUES (in_name, in_order_count);
    END;
    $$ LANGUAGE plpgsql;
    """
)

The ReplaceableObject class is only one very simplistic way to do this.
The structure of how we represent our schema objects
is not too important for the purposes of this example; we can just
as well put strings inside of tuples or dictionaries, as well as
that we could define any kind of series of fields and class structures we want.
The only important part is that below we will illustrate how organize the
code that can consume the structure we create here.

Create Operations for the Target Objects

We'll use the :class:.Operations extension API to make new operations
for create, drop, and replace of views and stored procedures. Using this
API is also optional; we can just as well make any kind of Python
function that we would invoke from our migration scripts.
However, using this API gives us operations
built directly into the Alembic op.* namespace very nicely.

The most intricate class is below. This is the base of our "replaceable"
operation, which includes not just a base operation for emitting
CREATE and DROP instructions on a ReplaceableObject, it also assumes
a certain model of "reversibility" which makes use of references to
other migration files in order to refer to the "previous" version
of an object::

text
from alembic.operations import Operations, MigrateOperation

class ReversibleOp(MigrateOperation):
    def __init__(self, target):
        self.target = target

    @classmethod
    def invoke_for_target(cls, operations, target):
        op = cls(target)
        return operations.invoke(op)

    def reverse(self):
        raise NotImplementedError()

    @classmethod
    def _get_object_from_version(cls, operations, ident):
        version, objname = ident.split(".")

        module = operations.get_context().script.get_revision(version).module
        obj = getattr(module, objname)
        return obj

    @classmethod
    def replace(cls, operations, target, replaces=None, replace_with=None):

        if replaces:
            old_obj = cls._get_object_from_version(operations, replaces)
            drop_old = cls(old_obj).reverse()
            create_new = cls(target)
        elif replace_with:
            old_obj = cls._get_object_from_version(operations, replace_with)
            drop_old = cls(target).reverse()
            create_new = cls(old_obj)
        else:
            raise TypeError("replaces or replace_with is required")

        operations.invoke(drop_old)
        operations.invoke(create_new)

The workings of this class should become clear as we walk through the
example. To create usable operations from this base, we will build
a series of stub classes and use :meth:.Operations.register_operation
to make them part of the op.* namespace::

text
@Operations.register_operation("create_view", "invoke_for_target")
@Operations.register_operation("replace_view", "replace")
class CreateViewOp(ReversibleOp):
    def reverse(self):
        return DropViewOp(self.target)


@Operations.register_operation("drop_view", "invoke_for_target")
class DropViewOp(ReversibleOp):
    def reverse(self):
        return CreateViewOp(self.target)


@Operations.register_operation("create_sp", "invoke_for_target")
@Operations.register_operation("replace_sp", "replace")
class CreateSPOp(ReversibleOp):
    def reverse(self):
        return DropSPOp(self.target)


@Operations.register_operation("drop_sp", "invoke_for_target")
class DropSPOp(ReversibleOp):
    def reverse(self):
        return CreateSPOp(self.target)

To actually run the SQL like "CREATE VIEW" and "DROP SEQUENCE", we'll provide
implementations using :meth:.Operations.implementation_for
that run straight into :meth:.Operations.execute::

text
@Operations.implementation_for(CreateViewOp)
def create_view(operations, operation):
    operations.execute("CREATE VIEW %s AS %s" % (
        operation.target.name,
        operation.target.sqltext
    ))


@Operations.implementation_for(DropViewOp)
def drop_view(operations, operation):
    operations.execute("DROP VIEW %s" % operation.target.name)


@Operations.implementation_for(CreateSPOp)
def create_sp(operations, operation):
    operations.execute(
        "CREATE FUNCTION %s %s" % (
            operation.target.name, operation.target.sqltext
        )
    )


@Operations.implementation_for(DropSPOp)
def drop_sp(operations, operation):
    operations.execute("DROP FUNCTION %s" % operation.target.name)

Publish the Extensions

All of the above code can be present anywhere within an application's
source tree; the only requirement is that when the env.py script is
invoked, it includes imports that ultimately call upon these classes
as well as the :meth:.Operations.register_operation and
:meth:.Operations.implementation_for sequences.

Alternatively, custom operations and autogenerate support can be organized
into reusable plugins using Alembic's plugin system. This allows extensions
to be packaged and distributed independently, and automatically discovered
via Python entry points. See :ref:alembic.plugins.toplevel for information
on writing and publishing plugins.

Create Initial Migrations

We can now illustrate how these objects look during use. For the first step,
we'll create a new migration to create a "customer" table::

text
$ alembic revision -m "create table"

We build the first revision as follows::

text
"""create table

Revision ID: 3ab8b2dfb055
Revises:
Create Date: 2015-07-27 16:22:44.918507

"""

# revision identifiers, used by Alembic.
revision = '3ab8b2dfb055'
down_revision = None
branch_labels = None
depends_on = None

from alembic import op
import sqlalchemy as sa


def upgrade():
    op.create_table(
        "customer",
        sa.Column('id', sa.Integer, primary_key=True),
        sa.Column('name', sa.String),
        sa.Column('order_count', sa.Integer),
    )


def downgrade():
    op.drop_table('customer')

For the second migration, we will create a view and a stored procedure
which act upon this table::

text
$ alembic revision -m "create views/sp"

This migration will use the new directives::

text
"""create views/sp

Revision ID: 28af9800143f
Revises: 3ab8b2dfb055
Create Date: 2015-07-27 16:24:03.589867

"""

# revision identifiers, used by Alembic.
revision = '28af9800143f'
down_revision = '3ab8b2dfb055'
branch_labels = None
depends_on = None

from alembic import op
import sqlalchemy as sa

from foo import ReplaceableObject

customer_view = ReplaceableObject(
    "customer_view",
    "SELECT name, order_count FROM customer WHERE order_count > 0"
)

add_customer_sp = ReplaceableObject(
    "add_customer_sp(name varchar, order_count integer)",
    """
    RETURNS integer AS $$
    BEGIN
        insert into customer (name, order_count)
        VALUES (in_name, in_order_count);
    END;
    $$ LANGUAGE plpgsql;
    """
)


def upgrade():
    op.create_view(customer_view)
    op.create_sp(add_customer_sp)


def downgrade():
    op.drop_view(customer_view)
    op.drop_sp(add_customer_sp)

We see the use of our new create_view(), create_sp(),
drop_view(), and drop_sp() directives. Running these to "head"
we get the following (this includes an edited view of SQL emitted)::

text
$ alembic upgrade 28af9800143
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [sqlalchemy.engine.base.Engine] BEGIN (implicit)
INFO  [sqlalchemy.engine.base.Engine] select relname from pg_class c join pg_namespace n on n.oid=c.relnamespace where pg_catalog.pg_table_is_visible(c.oid) and relname=%(name)s
INFO  [sqlalchemy.engine.base.Engine] {'name': u'alembic_version'}
INFO  [sqlalchemy.engine.base.Engine] SELECT alembic_version.version_num
FROM alembic_version
INFO  [sqlalchemy.engine.base.Engine] {}
INFO  [sqlalchemy.engine.base.Engine] select relname from pg_class c join pg_namespace n on n.oid=c.relnamespace where pg_catalog.pg_table_is_visible(c.oid) and relname=%(name)s
INFO  [sqlalchemy.engine.base.Engine] {'name': u'alembic_version'}
INFO  [alembic.runtime.migration] Running upgrade  -> 3ab8b2dfb055, create table
INFO  [sqlalchemy.engine.base.Engine]
CREATE TABLE customer (
    id SERIAL NOT NULL,
    name VARCHAR,
    order_count INTEGER,
    PRIMARY KEY (id)
)


INFO  [sqlalchemy.engine.base.Engine] {}
INFO  [sqlalchemy.engine.base.Engine] INSERT INTO alembic_version (version_num) VALUES ('3ab8b2dfb055')
INFO  [sqlalchemy.engine.base.Engine] {}
INFO  [alembic.runtime.migration] Running upgrade 3ab8b2dfb055 -> 28af9800143f, create views/sp
INFO  [sqlalchemy.engine.base.Engine] CREATE VIEW customer_view AS SELECT name, order_count FROM customer WHERE order_count > 0
INFO  [sqlalchemy.engine.base.Engine] {}
INFO  [sqlalchemy.engine.base.Engine] CREATE FUNCTION add_customer_sp(name varchar, order_count integer)
    RETURNS integer AS $$
    BEGIN
        insert into customer (name, order_count)
        VALUES (in_name, in_order_count);
    END;
    $$ LANGUAGE plpgsql;

INFO  [sqlalchemy.engine.base.Engine] {}
INFO  [sqlalchemy.engine.base.Engine] UPDATE alembic_version SET version_num='28af9800143f' WHERE alembic_version.version_num = '3ab8b2dfb055'
INFO  [sqlalchemy.engine.base.Engine] {}
INFO  [sqlalchemy.engine.base.Engine] COMMIT

We see that our CREATE TABLE proceeded as well as the CREATE VIEW and CREATE
FUNCTION operations produced by our new directives.

Create Revision Migrations

Finally, we can illustrate how we would "revise" these objects.
Let's consider we added a new column email to our customer table::

text
$ alembic revision -m "add email col"

The migration is::

text
"""add email col

Revision ID: 191a2d20b025
Revises: 28af9800143f
Create Date: 2015-07-27 16:25:59.277326

"""

# revision identifiers, used by Alembic.
revision = '191a2d20b025'
down_revision = '28af9800143f'
branch_labels = None
depends_on = None

from alembic import op
import sqlalchemy as sa


def upgrade():
    op.add_column("customer", sa.Column("email", sa.String()))


def downgrade():
    op.drop_column("customer", "email")

We now need to recreate the customer_view view and the
add_customer_sp function. To include downgrade capability, we will
need to refer to the previous version of the construct; the
replace_view() and replace_sp() operations we've created make
this possible, by allowing us to refer to a specific, previous revision.
the replaces and replace_with arguments accept a dot-separated
string, which refers to a revision number and an object name, such
as "28af9800143f.customer_view". The ReversibleOp class makes use
of the :meth:.Operations.get_context method to locate the version file
we refer to::

text
$ alembic revision -m "update views/sp"

The migration::

text
"""update views/sp

Revision ID: 199028bf9856
Revises: 191a2d20b025
Create Date: 2015-07-27 16:26:31.344504

"""

# revision identifiers, used by Alembic.
revision = '199028bf9856'
down_revision = '191a2d20b025'
branch_labels = None
depends_on = None

from alembic import op
import sqlalchemy as sa

from foo import ReplaceableObject

customer_view = ReplaceableObject(
    "customer_view",
    "SELECT name, order_count, email "
    "FROM customer WHERE order_count > 0"
)

add_customer_sp = ReplaceableObject(
    "add_customer_sp(name varchar, order_count integer, email varchar)",
    """
    RETURNS integer AS $$
    BEGIN
        insert into customer (name, order_count, email)
        VALUES (in_name, in_order_count, email);
    END;
    $$ LANGUAGE plpgsql;
    """
)


def upgrade():
    op.replace_view(customer_view, replaces="28af9800143f.customer_view")
    op.replace_sp(add_customer_sp, replaces="28af9800143f.add_customer_sp")


def downgrade():
    op.replace_view(customer_view, replace_with="28af9800143f.customer_view")
    op.replace_sp(add_customer_sp, replace_with="28af9800143f.add_customer_sp")

Above, instead of using create_view(), create_sp(),
drop_view(), and drop_sp() methods, we now use replace_view() and
replace_sp(). The replace operation we've built always runs a DROP and
a CREATE. Running an upgrade to head we see::

text
$ alembic upgrade head
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [sqlalchemy.engine.base.Engine] BEGIN (implicit)
INFO  [sqlalchemy.engine.base.Engine] select relname from pg_class c join pg_namespace n on n.oid=c.relnamespace where pg_catalog.pg_table_is_visible(c.oid) and relname=%(name)s
INFO  [sqlalchemy.engine.base.Engine] {'name': u'alembic_version'}
INFO  [sqlalchemy.engine.base.Engine] SELECT alembic_version.version_num
FROM alembic_version
INFO  [sqlalchemy.engine.base.Engine] {}
INFO  [alembic.runtime.migration] Running upgrade 28af9800143f -> 191a2d20b025, add email col
INFO  [sqlalchemy.engine.base.Engine] ALTER TABLE customer ADD COLUMN email VARCHAR
INFO  [sqlalchemy.engine.base.Engine] {}
INFO  [sqlalchemy.engine.base.Engine] UPDATE alembic_version SET version_num='191a2d20b025' WHERE alembic_version.version_num = '28af9800143f'
INFO  [sqlalchemy.engine.base.Engine] {}
INFO  [alembic.runtime.migration] Running upgrade 191a2d20b025 -> 199028bf9856, update views/sp
INFO  [sqlalchemy.engine.base.Engine] DROP VIEW customer_view
INFO  [sqlalchemy.engine.base.Engine] {}
INFO  [sqlalchemy.engine.base.Engine] CREATE VIEW customer_view AS SELECT name, order_count, email FROM customer WHERE order_count > 0
INFO  [sqlalchemy.engine.base.Engine] {}
INFO  [sqlalchemy.engine.base.Engine] DROP FUNCTION add_customer_sp(name varchar, order_count integer)
INFO  [sqlalchemy.engine.base.Engine] {}
INFO  [sqlalchemy.engine.base.Engine] CREATE FUNCTION add_customer_sp(name varchar, order_count integer, email varchar)
    RETURNS integer AS $$
    BEGIN
        insert into customer (name, order_count, email)
        VALUES (in_name, in_order_count, email);
    END;
    $$ LANGUAGE plpgsql;

INFO  [sqlalchemy.engine.base.Engine] {}
INFO  [sqlalchemy.engine.base.Engine] UPDATE alembic_version SET version_num='199028bf9856' WHERE alembic_version.version_num = '191a2d20b025'
INFO  [sqlalchemy.engine.base.Engine] {}
INFO  [sqlalchemy.engine.base.Engine] COMMIT

After adding our new email column, we see that both customer_view
and add_customer_sp() are dropped before the new version is created.
If we downgrade back to the old version, we see the old version of these
recreated again within the downgrade for this migration::

text
$ alembic downgrade 28af9800143
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [sqlalchemy.engine.base.Engine] BEGIN (implicit)
INFO  [sqlalchemy.engine.base.Engine] select relname from pg_class c join pg_namespace n on n.oid=c.relnamespace where pg_catalog.pg_table_is_visible(c.oid) and relname=%(name)s
INFO  [sqlalchemy.engine.base.Engine] {'name': u'alembic_version'}
INFO  [sqlalchemy.engine.base.Engine] SELECT alembic_version.version_num
FROM alembic_version
INFO  [sqlalchemy.engine.base.Engine] {}
INFO  [alembic.runtime.migration] Running downgrade 199028bf9856 -> 191a2d20b025, update views/sp
INFO  [sqlalchemy.engine.base.Engine] DROP VIEW customer_view
INFO  [sqlalchemy.engine.base.Engine] {}
INFO  [sqlalchemy.engine.base.Engine] CREATE VIEW customer_view AS SELECT name, order_count FROM customer WHERE order_count > 0
INFO  [sqlalchemy.engine.base.Engine] {}
INFO  [sqlalchemy.engine.base.Engine] DROP FUNCTION add_customer_sp(name varchar, order_count integer, email varchar)
INFO  [sqlalchemy.engine.base.Engine] {}
INFO  [sqlalchemy.engine.base.Engine] CREATE FUNCTION add_customer_sp(name varchar, order_count integer)
    RETURNS integer AS $$
    BEGIN
        insert into customer (name, order_count)
        VALUES (in_name, in_order_count);
    END;
    $$ LANGUAGE plpgsql;

INFO  [sqlalchemy.engine.base.Engine] {}
INFO  [sqlalchemy.engine.base.Engine] UPDATE alembic_version SET version_num='191a2d20b025' WHERE alembic_version.version_num = '199028bf9856'
INFO  [sqlalchemy.engine.base.Engine] {}
INFO  [alembic.runtime.migration] Running downgrade 191a2d20b025 -> 28af9800143f, add email col
INFO  [sqlalchemy.engine.base.Engine] ALTER TABLE customer DROP COLUMN email
INFO  [sqlalchemy.engine.base.Engine] {}
INFO  [sqlalchemy.engine.base.Engine] UPDATE alembic_version SET version_num='28af9800143f' WHERE alembic_version.version_num = '191a2d20b025'
INFO  [sqlalchemy.engine.base.Engine] {}
INFO  [sqlalchemy.engine.base.Engine] COMMIT

.. _cookbook_postgresql_multi_tenancy:

Rudimental Schema-Level Multi Tenancy for PostgreSQL, MySQL, Other Databases

Multi tenancy refers to an application that accommodates for many
clients simultaneously. Within the scope of a database migrations tool,
multi-tenancy typically refers to the practice of maintaining multiple,
identical databases where each database is assigned to one client.

Alembic does not currently have explicit multi-tenant support; typically,
the approach must involve running Alembic multiple times against different
database URLs.

One common approach to multi-tenancy, particularly on the PostgreSQL database,
is to install tenants within individual PostgreSQL schemas; similarly
when using MySQL/MariaDB, individual MySQL/MariaDB databases are addressed
in the same way as "schemas" on PostgreSQL.

When using PostgreSQL's schemas, a special variable search_path is offered
that is intended to assist with targeting of different schemas. When using
MySQL or MariaDB databases, a similar command is available at the SQL level
called the USE command. This command may be used in a similar fashion as
that of PostgreSQL's search_path variable to achieve a similar effect.

Overall, this recipe can be used on any database that supports runtime
modification of the current "tenant" via SQL commands on a particular
connection
.

.. note:: SQLAlchemy includes a system of directing a common set of
Table metadata to many schemas called schema_translate_map <https://docs.sqlalchemy.org/core/connections.html#translation-of-schema-names>_. Alembic at the time
of this writing lacks adequate support for this feature. The recipe below
should be considered interim until Alembic has more first-class support
for schema-level multi-tenancy.

The recipe below can be altered for flexibility. The primary purpose of this
recipe is to illustrate how to point the Alembic process towards one PostgreSQL
or MySQL/MariaDB schema or another.

  1. The model metadata used as the target for autogenerate must not include any
    schema name for tables; the schema must be non-present or set to None.
    Otherwise, Alembic autogenerate will still attempt
    to compare and render tables in terms of this schema::

    text
    class A(Base):
         __tablename__ = 'a'
    
         id = Column(Integer, primary_key=True)
         data = Column(UnicodeText())
         foo = Column(Integer)
    
         __table_args__ = {
             "schema": None
         }

    ..

  2. The :paramref:.EnvironmentContext.configure.include_schemas flag must
    also be False or not included.

  3. The "tenant" will be a schema name passed to Alembic using the "-x" flag.
    In env.py an approach like the following allows -xtenant=some_schema
    to be supported by making use of :meth:.EnvironmentContext.get_x_argument::

    text
    from sqlalchemy import text
    
     def run_migrations_online():
         connectable = engine_from_config(
             config.get_section(config.config_ini_section),
             prefix="sqlalchemy.",
             poolclass=pool.NullPool,
         )
    
         current_tenant = context.get_x_argument(as_dictionary=True).get("tenant")
         with connectable.connect() as connection:
    
             if connection.dialect.name == "postgresql":
                 # set search path on the connection, which ensures that
                 # PostgreSQL will emit all CREATE / ALTER / DROP statements
                 # in terms of this schema by default
    
                 connection.execute(text('set search_path to "%s"' % current_tenant))
                 # in SQLAlchemy v2+ the search path change needs to be committed
                 connection.commit()
             elif connection.dialect.name in ("mysql", "mariadb"):
                 # set "USE" on the connection, which ensures that
                 # MySQL/MariaDB will emit all CREATE / ALTER / DROP statements
                 # in terms of this schema by default
    
                 connection.execute(text('USE %s' % current_tenant))
    
             # make use of non-supported SQLAlchemy attribute to ensure
             # the dialect reflects tables in terms of the current tenant name
             connection.dialect.default_schema_name = current_tenant
    
             context.configure(
                 connection=connection,
                 target_metadata=target_metadata,
             )
    
             with context.begin_transaction():
                 context.run_migrations()

    The current tenant is set using the PostgreSQL search_path variable, or
    the MySQL/MariaDB USE statement, on the connection. Note above we must
    employ a non-supported SQLAlchemy workaround at the moment which is to
    hardcode the SQLAlchemy dialect's default schema name to our target schema.

    It is also important to note that the above changes remain on the connection
    permanently unless reversed explicitly
    . If the alembic application simply
    exits above, there is no issue. However if the application attempts to
    continue using the above connection for other purposes, it may be necessary
    to reset these variables back to the default, which for PostgreSQL is usually
    the name "public" however may be different based on configuration, and
    for MySQL/MariaDB is typically the "database" portion of the database URL.

  4. Alembic operations will now proceed in terms of whichever schema we pass
    on the command line. All logged SQL will show no schema, except for
    reflection operations which will make use of the default_schema_name
    attribute::

    text
    []$ alembic -x tenant=some_schema revision -m "rev1" --autogenerate

    ..

  5. Since all schemas are to be maintained in sync, autogenerate should be run
    against only one schema, generating new Alembic migration files.
    Autogenerated migration operations are then run against all schemas.

.. _cookbook_no_empty_migrations:

Don't Generate Empty Migrations with Autogenerate

A common request is to have the alembic revision --autogenerate command not
actually generate a revision file if no changes to the schema is detected. Using
the :paramref:.EnvironmentContext.configure.process_revision_directives
hook, this is straightforward; place a process_revision_directives
hook in :meth:.MigrationContext.configure which removes the
single :class:.MigrationScript directive if it is empty of
any operations::

text
# for typing purposes
from collections.abc import Iterable
from alembic.environment import MigrationContext

# this typing-only import requires alembic 1.12.1 or above
from alembic.operations import MigrationScript


def run_migrations_online():

    # ...

    def process_revision_directives(
        context: MigrationContext,
        revision: str | Iterable[str | None] | Iterable[str],
        directives: list[MigrationScript],
    ):
        assert config.cmd_opts is not None
        if getattr(config.cmd_opts, 'autogenerate', False):
            script = directives[0]
            assert script.upgrade_ops is not None
            if script.upgrade_ops.is_empty():
                directives[:] = []


    # connectable = ...

    with connectable.connect() as connection:
        context.configure(
            connection=connection,
            target_metadata=target_metadata,
            process_revision_directives=process_revision_directives
        )

        with context.begin_transaction():
            context.run_migrations()

.. _cookbook_dont_emit_drop_index:

Don't emit DROP INDEX when the table is to be dropped as well

MySQL may complain when dropping an index that is against a column
that also has a foreign key constraint on it. If the table is to be dropped
in any case, the DROP INDEX isn't necessary. This recipe will process the set
of autogenerate directives such that all :class:.DropIndexOp directives
are removed against tables that themselves are to be dropped::

text
def run_migrations_online():

    # ...

    from alembic.operations import ops

    def process_revision_directives(context, revision, directives):
        script = directives[0]

        # process both "def upgrade()", "def downgrade()"
        for directive in (script.upgrade_ops, script.downgrade_ops):

            # make a set of tables that are being dropped within
            # the migration function
            tables_dropped = set()
            for op in directive.ops:
                if isinstance(op, ops.DropTableOp):
                    tables_dropped.add((op.table_name, op.schema))

            # now rewrite the list of "ops" such that DropIndexOp
            # is removed for those tables.   Needs a recursive function.
            directive.ops = list(
                _filter_drop_indexes(directive.ops, tables_dropped)
            )

    def _filter_drop_indexes(directives, tables_dropped):
        # given a set of (tablename, schemaname) to be dropped, filter
        # out DropIndexOp from the list of directives and yield the result.

        for directive in directives:
            # ModifyTableOps is a container of ALTER TABLE types of
            # commands.  process those in place recursively.
            if isinstance(directive, ops.ModifyTableOps) and \
                    (directive.table_name, directive.schema) in tables_dropped:
                directive.ops = list(
                    _filter_drop_indexes(directive.ops, tables_dropped)
                )

                # if we emptied out the directives, then skip the
                # container altogether.
                if not directive.ops:
                    continue
            elif isinstance(directive, ops.DropIndexOp) and \
                    (directive.table_name, directive.schema) in tables_dropped:
                # we found a target DropIndexOp.   keep looping
                continue

            # otherwise if not filtered, yield out the directive
            yield directive

    # connectable = ...

    with connectable.connect() as connection:
        context.configure(
            connection=connection,
            target_metadata=target_metadata,
            process_revision_directives=process_revision_directives
        )

        with context.begin_transaction():
            context.run_migrations()

Whereas autogenerate, when dropping two tables with a foreign key and
an index, would previously generate something like::

text
def downgrade():
    # ### commands auto generated by Alembic - please adjust! ###
    op.drop_index(op.f('ix_b_aid'), table_name='b')
    op.drop_table('b')
    op.drop_table('a')
    # ### end Alembic commands ###

With the above rewriter, it generates as::

text
def downgrade():
    # ### commands auto generated by Alembic - please adjust! ###
    op.drop_table('b')
    op.drop_table('a')
    # ### end Alembic commands ###

Don't generate any DROP TABLE directives with autogenerate

When running autogenerate against a database that has existing tables outside
of the application's autogenerated metadata, it may be desirable to prevent
autogenerate from considering any of those existing tables to be dropped.
This will prevent autogenerate from detecting tables removed from the
local metadata as well however this is only a small caveat.

The most direct way to achieve this using the
:paramref:.EnvironmentContext.configure.include_object hook. There is no
need to hardcode a fixed "whitelist" of table names; the hook gives enough
information in the given arguments to determine if a particular table name is
not part of the local :class:.MetaData being autogenerated, by checking first
that the type of object is "table", then that reflected is True,
indicating this table name is from the local database connection, not the
:class:.MetaData, and finally that compare_to is None, indicating
autogenerate is not comparing this :class:.Table to any :class:.Table in
the local :class:.MetaData collection::

text
# in env.py

def include_object(object, name, type_, reflected, compare_to):
    if type_ == "table" and reflected and compare_to is None:
        return False
    else:
        return True


context.configure(
    # ...
    include_object = include_object
)

.. _cookbook_custom_sorting_create_table:

Apply Custom Sorting to Table Columns within CREATE TABLE

This example illustrates use of the :class:.Rewriter object introduced
at :ref:autogen_rewriter. While the rewriter grants access to the
individual :class:.ops.MigrateOperation objects, there are sometimes some
special techniques required to get around some structural limitations that
are present.

One is when trying to reorganize the order of columns in a
table within a :class:.ops.CreateTableOp directive. This directive, when
generated by autogenerate, actually holds onto the original :class:.Table
object as the source of its information, so attempting to reorder the
:attr:.ops.CreateTableOp.columns collection will usually have no effect.
Instead, a new :class:.ops.CreateTableOp object may be constructed with the
new ordering. However, a second issue is that the :class:.Column objects
inside will already be associated with the :class:.Table that is from the
model being autogenerated, meaning they can't be reassigned directly to a new
:class:.Table. To get around this, we can copy all the columns and constraints
using methods like :meth:.Column.copy.

Below we use :class:.Rewriter to create a new :class:.ops.CreateTableOp
directive and to copy the :class:.Column objects from one into another,
copying each column or constraint object and applying a new sorting scheme::

text
# in env.py

from alembic.operations import ops
from alembic.autogenerate import rewriter

writer = rewriter.Rewriter()

@writer.rewrites(ops.CreateTableOp)
def order_columns(context, revision, op):

    special_names = {"id": -100, "created_at": 1001, "updated_at": 1002}

    cols_by_key = [
        (
            special_names.get(col.key, index)
            if isinstance(col, Column)
            else 2000,
            col,
        )
        for index, col in enumerate(op.columns)
    ]

    columns = [
        col for idx, col in sorted(cols_by_key, key=lambda entry: entry[0])
    ]
    return ops.CreateTableOp(
        op.table_name, columns, schema=op.schema, **op.kw)


# ...

context.configure(
    # ...
    process_revision_directives=writer
)

Above, when we apply the writer to a table such as::

text
Table(
    "my_table",
    m,
    Column("data", String(50)),
    Column("created_at", DateTime),
    Column("id", Integer, primary_key=True),
    Column("updated_at", DateTime),
    UniqueConstraint("data", name="uq_data")
)

This will render in the autogenerated file as::

text
def upgrade():
    # ### commands auto generated by Alembic - please adjust! ###
    op.create_table(
        "my_table",
        sa.Column("id", sa.Integer(), nullable=False),
        sa.Column("data", sa.String(length=50), nullable=True),
        sa.Column("created_at", sa.DateTime(), nullable=True),
        sa.Column("updated_at", sa.DateTime(), nullable=True),
        sa.PrimaryKeyConstraint("id"),
        sa.UniqueConstraint("data", name="uq_data"),
    )
    # ### end Alembic commands ###

.. _cookbook_add_if_exists:

Add IF [NOT] EXISTS to CREATE/DROP operations

Sometimes CREATE/DROP operations take too long during a production deployment
and it is preferable to apply them offline,
and still keep alembic migrations aligned and/or for test environments.

Using the rewriter makes it possible::

text
from alembic.operations import ops
from alembic.autogenerate import rewriter

writer = rewriter.Rewriter()

@writer.rewrites(ops.CreateTableOp)
@writer.rewrites(ops.CreateIndexOp)
def add_if_not_exists(context, revision, op):
    op.if_not_exists = True
    return op

@writer.rewrites(ops.DropTableOp)
@writer.rewrites(ops.DropIndexOp)
def add_if_exists(context, revision, op):
    op.if_exists = True
    return op

Same operation is possible for ADD/DROP COLUMN on postgresql/mariadb::

text
@writer.rewrites(ops.AddColumnOp)
def add_column_if_not_exists(context, revision, op):
    op.if_not_exists = True
    return op

@writer.rewrites(ops.DropColumnOp)
def drop_column_if_not_exists(context, revision, op):
    op.if_exists = True
    return op

Don't emit CREATE TABLE statements for Views

It is sometimes convenient to create :class:~sqlalchemy.schema.Table instances for views
so that they can be queried using normal SQLAlchemy techniques. Unfortunately this
causes Alembic to treat them as tables in need of creation and to generate spurious
create_table() operations. This is easily fixable by flagging such Tables and using the
:paramref:~.EnvironmentContext.configure.include_object hook to exclude them::

text
my_view = Table('my_view', metadata, autoload=True, info=dict(is_view=True))    # Flag this as a view

Or, if you use declarative tables::

text
class MyView(Base):
    __tablename__ = 'my_view'
    __table_args__ = {'info': {'is_view': True}} # Flag this as a view

Then define include_object as::

text
def include_object(object, name, type_, reflected, compare_to):
    """
    Exclude views from Alembic's consideration.
    """

    return not object.info.get('is_view', False)

Finally, in env.py pass your include_object as a keyword argument to :meth:.EnvironmentContext.configure.

.. _multiple_environments:

Run Multiple Alembic Environments from one .ini file

Long before Alembic had the "multiple bases" feature described in :ref:multiple_bases,
projects had a need to maintain more than one Alembic version history in a single
project, where these version histories are completely independent of each other
and each refer to their own alembic_version table, either across multiple databases,
schemas, or namespaces. A simple approach was added to support this, the
--name flag on the commandline. This flag allows named sections within
the alembic.ini file to be present (but note it does not apply to
pyproject.toml configuration, where only the [tool.alembic] section
is used).

First, one would create an alembic.ini file of this form::

text
[DEFAULT]
# all defaults shared between environments go here

sqlalchemy.url = postgresql://scott:tiger@hostname/mydatabase


[schema1]
# path to env.py and migration scripts for schema1
script_location = myproject/revisions/schema1

[schema2]
# path to env.py and migration scripts for schema2
script_location = myproject/revisions/schema2

[schema3]
# path to env.py and migration scripts for schema3
script_location = myproject/revisions/db2

# this schema uses a different database URL as well
sqlalchemy.url = postgresql://scott:tiger@hostname/myotherdatabase

Above, in the [DEFAULT] section we set up a default database URL.
Then we create three sections corresponding to different revision lineages
in our project. Each of these directories would have its own env.py
and set of versioning files. Then when we run the alembic command,
we simply give it the name of the configuration we want to use::

text
alembic --name schema2 revision -m "new rev for schema 2" --autogenerate

Above, the alembic command makes use of the configuration in [schema2],
populated with defaults from the [DEFAULT] section.

The above approach can be automated by creating a custom front-end to the
Alembic commandline as well.

Suppose you have a database already, and want to generate some
op.create_table() and other directives that you'd have in a migration file.
How can we automate generating that code?
Suppose the database schema looks like (assume MySQL)::

text
CREATE TABLE IF NOT EXISTS `users` (
    `id` int(11) NOT NULL,
    KEY `id` (`id`)
);

CREATE TABLE IF NOT EXISTS `user_properties` (
  `users_id` int(11) NOT NULL,
  `property_name` varchar(255) NOT NULL,
  `property_value` mediumtext NOT NULL,
  UNIQUE KEY `property_name_users_id` (`property_name`,`users_id`),
  KEY `users_id` (`users_id`),
  CONSTRAINT `user_properties_ibfk_1` FOREIGN KEY (`users_id`)
  REFERENCES `users` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

Using :class:.ops.UpgradeOps, :class:.ops.CreateTableOp, and
:class:.ops.CreateIndexOp, we create a migration file structure,
using :class:.Table objects that we get from SQLAlchemy reflection.
The structure is passed to :func:.autogenerate.render_python_code to
produce the Python code for a migration file::

text
from sqlalchemy import create_engine
from sqlalchemy import MetaData, Table
from alembic import autogenerate
from alembic.operations import ops

e = create_engine("mysql://scott:tiger@localhost/test")

with e.connect() as conn:
    m = MetaData()
    user_table = Table('users', m, autoload_with=conn)
    user_property_table = Table('user_properties', m, autoload_with=conn)

print(autogenerate.render_python_code(
    ops.UpgradeOps(
        ops=[
            ops.CreateTableOp.from_table(table) for table in m.tables.values()
        ] + [
            ops.CreateIndexOp.from_index(idx) for table in m.tables.values()
            for idx in table.indexes
        ]
    ))
)

Output::

text
# ### commands auto generated by Alembic - please adjust! ###
op.create_table('users',
sa.Column('id', mysql.INTEGER(display_width=11), autoincrement=False, nullable=False),
mysql_default_charset='latin1',
mysql_engine='InnoDB'
)
op.create_table('user_properties',
sa.Column('users_id', mysql.INTEGER(display_width=11), autoincrement=False, nullable=False),
sa.Column('property_name', mysql.VARCHAR(length=255), nullable=False),
sa.Column('property_value', mysql.MEDIUMTEXT(), nullable=False),
sa.ForeignKeyConstraint(['users_id'], ['users.id'], name='user_properties_ibfk_1', ondelete='CASCADE'),
mysql_comment='user properties',
mysql_default_charset='utf8',
mysql_engine='InnoDB'
)
op.create_index('id', 'users', ['id'], unique=False)
op.create_index('users_id', 'user_properties', ['users_id'], unique=False)
op.create_index('property_name_users_id', 'user_properties', ['property_name', 'users_id'], unique=True)
# ### end Alembic commands ###

Run Alembic Operation Objects Directly (as in from autogenerate)

The :class:.Operations object has a method known as
:meth:.Operations.invoke that will generically invoke a particular operation
object. We can therefore use the :func:.autogenerate.produce_migrations
function to run an autogenerate comparison, get back a
:class:.ops.MigrationScript structure representing the changes, and with a
little bit of insider information we can invoke them directly.

The traversal through the :class:.ops.MigrationScript structure is as
follows::

text
use_batch = engine.name == "sqlite"

stack = [migrations.upgrade_ops]
while stack:
    elem = stack.pop(0)

    if use_batch and isinstance(elem, ModifyTableOps):
        with operations.batch_alter_table(
            elem.table_name, schema=elem.schema
        ) as batch_ops:
            for table_elem in elem.ops:
                batch_ops.invoke(table_elem)

    elif hasattr(elem, "ops"):
        stack.extend(elem.ops)
    else:
        operations.invoke(elem)

Above, we detect elements that have a collection of operations by looking
for the .ops attribute. A check for :class:.ModifyTableOps allows
us to use a batch context if we are supporting that.

A full example follows. The overall setup here is copied from the example
at :func:.autogenerate.compare_metadata::

text
from sqlalchemy import Column
from sqlalchemy import create_engine
from sqlalchemy import Integer
from sqlalchemy import MetaData
from sqlalchemy import String
from sqlalchemy import Table

from alembic.autogenerate import produce_migrations
from alembic.migration import MigrationContext
from alembic.operations import Operations
from alembic.operations.ops import ModifyTableOps


engine = create_engine("sqlite://", echo=True)

with engine.connect() as conn:
    conn.execute(
        """
        create table foo (
            id integer not null primary key,
            old_data varchar(50),
            x integer
        )"""
    )

    conn.execute(
        """
        create table bar (
            data varchar(50)
        )"""
    )

metadata = MetaData()
Table(
    "foo",
    metadata,
    Column("id", Integer, primary_key=True),
    Column("data", Integer),
    Column("x", Integer, nullable=False),
)
Table("bat", metadata, Column("info", String(100)))

mc = MigrationContext.configure(engine.connect())

migrations = produce_migrations(mc, metadata)

operations = Operations(mc)

use_batch = engine.name == "sqlite"

stack = [migrations.upgrade_ops]
while stack:
    elem = stack.pop(0)

    if use_batch and isinstance(elem, ModifyTableOps):
        with operations.batch_alter_table(
            elem.table_name, schema=elem.schema
        ) as batch_ops:
            for table_elem in elem.ops:
                batch_ops.invoke(table_elem)

    elif hasattr(elem, "ops"):
        stack.extend(elem.ops)
    else:
        operations.invoke(elem)

.. _cookbook_check_heads:

Test current database revision is at head(s)

.. versionchanged:: 1.17.1 This recipe is now part of the alembic current
command using the :paramref:.command.current.check_heads parameter,
available from the command line as --check-heads:

.. sourcecode::

text
alembic current --check-heads

   INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
   INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
   ERROR [alembic.util.messaging] Database is not on all head revisions
   FAILED: Database is not on all head revisions

A recipe to determine if a database schema is up to date in terms of applying
Alembic migrations. May be useful for test or installation suites to
determine if the target database is up to date. Makes use of the
:meth:.MigrationContext.get_current_heads as well as
:meth:.ScriptDirectory.get_heads methods so that it accommodates for a
branched revision tree::

text
from alembic import config, script
from alembic.runtime import migration
from sqlalchemy import engine


def check_current_head(alembic_cfg, connectable):
    # type: (config.Config, engine.Engine) -> bool
    directory = script.ScriptDirectory.from_config(alembic_cfg)
    with connectable.begin() as connection:
        context = migration.MigrationContext.configure(connection)
        return set(context.get_current_heads()) == set(directory.get_heads())

e = engine.create_engine("mysql://scott:tiger@localhost/test", echo=True)
cfg = config.Config("alembic.ini")
print(check_current_head(cfg, e))

.. seealso::

text
:meth:`.MigrationContext.get_current_heads`

:meth:`.ScriptDirectory.get_heads`

.. _asyncio_recipe:

Using Asyncio with Alembic

SQLAlchemy version 1.4 introduced experimental support for asyncio, allowing
use of most of its interface from async applications. Alembic currently does
not provide an async api directly, but it can use an use SQLAlchemy Async
engine to run the migrations and autogenerate.

New configurations can use the template "async" or "pyproject_async" to bootstrap an environment which
can be used with async DBAPI like asyncpg, running the command::

text
alembic init -t async <script_directory_here>

Existing configurations can be updated to use an async DBAPI by updating the env.py
file that's used by Alembic to start its operations. In particular only
run_migrations_online will need to be updated to be something like the example below::

text
import asyncio

from sqlalchemy.ext.asyncio import async_engine_from_config

# ... no change required to the rest of the code


def do_run_migrations(connection):
    context.configure(connection=connection, target_metadata=target_metadata)

    with context.begin_transaction():
        context.run_migrations()


async def run_async_migrations():
    """In this scenario we need to create an Engine
    and associate a connection with the context.

    """

    connectable = async_engine_from_config(
        config.get_section(config.config_ini_section),
        prefix="sqlalchemy.",
        poolclass=pool.NullPool,
    )

    async with connectable.connect() as connection:
        await connection.run_sync(do_run_migrations)

    await connectable.dispose()


def run_migrations_online():
    """Run migrations in 'online' mode."""

    asyncio.run(run_async_migrations())

An async application can also interact with the Alembic api directly by using
the SQLAlchemy run_sync method to adapt the non-async api of Alembic to
an async consumer.

.. _connection_sharing_plus_asyncio:

Programmatic API use (connection sharing) With Asyncio

Combining the examples of :ref:connection_sharing with :ref:asyncio_recipe
together, the env.py listed above can be updated as follows works::

text
def run_migrations_online():
    """Run migrations in 'online' mode.

    """

    connectable = config.attributes.get("connection", None)

    if connectable is None:
        asyncio.run(run_async_migrations())
    else:
        do_run_migrations(connectable)

Above, using an asyncio database URL in alembic.ini one can run
commands such as alembic upgrade from the command line. Programmatically,
the same env.py file can be invoked using asyncio as::

text
import asyncio
from sqlalchemy.ext.asyncio import create_async_engine

from alembic import command, config


def run_upgrade(connection, cfg):
    cfg.attributes["connection"] = connection
    command.upgrade(cfg, "head")


async def run_async_upgrade():
    async_engine = create_async_engine("sqlite+aiosqlite://", echo=True)
    async with async_engine.begin() as conn:
        await conn.run_sync(run_upgrade, config.Config("alembic.ini"))


asyncio.run(run_async_upgrade())

Data Migrations - General Techniques

Alembic migrations are designed for schema migrations.
The nature of data migrations are inherently different and it's not in fact advisable
in the general case to write data migrations that integrate with Alembic's
schema versioning model.
For example downgrades are difficult to address since they might require deletion of data,
which may even not be possible to detect.

.. warning::

The solution needs to be designed specifically for each individual application and migration.
There are no general rules and the following text is only a recommendation based on experience.

There are three basic approaches for the data migrations.

Small data

Small data migrations are easy to perform, especially in cases of initial data to a new table.
These can be handled using :meth:.Operations.bulk_insert.

Separate migration script

One possibility is a completely separate script aside of alembic migrations.
The complete migration is then processed in following steps:

  1. Run the initial alembic migrations (new columns etc.)
  2. Run the separate data migration script
  3. Run the final alembic migrations (database constraints, delete columns etc.)

The data migration script may also need a separate ORM model to handle intermediate state of the database.

Online migration

The application maintains a version of schema with both versions.
Writes are performed on both places, while the background script move all the remaining data across.
This technique is very challenging and time demanding, since it requires custom application logic to
handle the intermediate states.

.. _custom_commandline:

Extend ``CommandLine`` with custom commands

.. versionadded:: 1.15.3

While Alembic does not have a plugin system that would allow transparently extending the original alembic CLI
with additional commands, it is possible to create your own instance of :class:.CommandLine and extend that via
:meth:.CommandLine.register_command.

.. code-block:: python

text
# myalembic.py

from alembic.config import CommandLine, Config


def frobnicate(config: Config, revision: str) -> None:
    """Frobnicates according to the frobnication specification.

    :param config: a :class:`.Config` instance
    :param revision: the revision to frobnicate
    """

    config.print_stdout(f"Revision {revision} successfully frobnicated.")


def main():
    cli = CommandLine()
    cli.register_command(frobnicate)
    cli.main()

if __name__ == "__main__":
    main()

Any named function may be registered as a command, provided it accepts a :class:.Config object as the first argument;
a docstring is also recommended as it will show up in the help output of the CLI.

.. code-block::

text
$ python -m myalembic -h
...
positional arguments:
  {branches,check,current,downgrade,edit,ensure_version,heads,history,init,list_templates,merge,revision,show,stamp,upgrade,frobnicate}
    ...
    frobnicate          Frobnicates according to the frobnication specification.

$ python -m myalembic frobnicate -h
usage: myalembic.py frobnicate [-h] revision

positional arguments:
revision    revision identifier

optional arguments:
-h, --help  show this help message and exit

$ python -m myalembic frobnicate 42
Revision 42 successfully frobnicated.

Build Front (`docs/build/front.rst`)

============ Front Matter

Information about the Alembic project.

Project Homepage

Alembic is hosted on GitHub at https://github.com/sqlalchemy/alembic under the SQLAlchemy organization.

Releases and project status are available on Pypi at https://pypi.python.org/pypi/alembic.

The most recent published version of this documentation should be at https://alembic.sqlalchemy.org.

.. _installation:

Installation

While Alembic can be installed system wide, it's more common that it's
installed local to a virtual environment <https://docs.python.org/3/tutorial/venv.html>_ , as it also uses libraries
such as SQLAlchemy and database drivers that are more appropriate for
local installations.

The documentation below is only one kind of approach to installing Alembic
for a project
; there are many such approaches. The documentation below is
provided only for those users who otherwise have no specific project setup
chosen.

To build a virtual environment for a specific project, a virtual environment
can be created using the
Python venv library <https://docs.python.org/3/library/venv.html>_::

text
$ cd /path/to/your/project
$ python -m venv .venv

There is now a Python interpreter that you can access in
/path/to/your/project/.venv/bin/python, as well as the pip <http://pypi.python.org/pypi/pip>_ installer tool in
/path/to/your/project/.venv/bin/pip.

Next,the activate command installed by venv can be used so that
all binaries local to this new Python environment are in the local path::

text
$ source /path/to/your/project/.venv/bin/activate

We now install Alembic as follows::

text
$ pip install alembic

The install will add the alembic command to the virtual environment. All
operations with Alembic in terms of this specific virtual environment will then
proceed through the usage of this command, as in::

text
$ alembic init alembic

Finally, assuming your project is itself installable, meaning it has a
pyproject.toml file, and/or setup.py script, the local project can
be made a part of the same local environment by installing it with pip,
optionally using "editable" mode::

text
$ pip install -e .

Dependencies

Alembic's install process will ensure that SQLAlchemy_
is installed, in addition to other dependencies. Alembic will work with
SQLAlchemy as of version 2.0.0.

.. versionchanged:: 1.20.0 Support for SQLAlchemy older than 2.0.0 was dropped.

Alembic supports Python versions 3.10 and above

.. versionchanged:: 1.17 Alembic now supports Python 3.10 and newer.

.. _versioning_scheme:

Versioning Scheme

Alembic's versioning scheme is based on that of
SQLAlchemy's versioning scheme <https://www.sqlalchemy.org/download.html#versions>_.
In particular, it should be noted that while Alembic uses a three-number
versioning scheme, it does not use SemVer. In SQLAlchemy and Alembic's
scheme, the middle digit is considered to be a "Significant Minor Release",
which may include removal of previously deprecated APIs with some risk of
non-backwards compatibility in a very small number of cases
.

This means that version "1.8.0", "1.9.0", "1.10.0", "1.11.0", etc. are
Significant Minor Releases, which will include new API features and may
remove or modify existing ones.

Therefore, when pinning <https://pip.pypa.io/en/stable/topics/repeatable-installs/>_
Alembic releases, pin to the "major" and "minor" digits to avoid API changes.

A true "Major" release such as a change to "2.0" would include complete
redesigns/re-architectures of foundational features; currently no such series
of changes are planned, although changes such as replacing the entire
"autogenerate" scheme with a new approach would qualify for that level of
change.

Community

Alembic is developed by Mike Bayer <http://techspot.zzzeek.org>, and is
part of the SQLAlchemy
project.

User issues, discussion of potential bugs and features are most easily
discussed using GitHub Discussions <https://github.com/sqlalchemy/alembic/discussions/>_.

.. _bugs:

Bugs

Bugs and feature enhancements to Alembic should be reported on the GitHub issue tracker <https://github.com/sqlalchemy/alembic/issues/>_.

.. _SQLAlchemy: https://www.sqlalchemy.org


Build Index (`docs/build/index.rst`)

=================================== Welcome to Alembic's documentation!

Alembic <https://alembic.sqlalchemy.org>_ is a lightweight database migration tool for usage
with the SQLAlchemy <https://www.sqlalchemy.org>_ Database Toolkit for Python.

.. toctree::
:maxdepth: 3

front
tutorial
autogenerate
offline
naming
batch
branches
ops
cookbook
api/index
changelog

Indices and tables

  • :ref:genindex
  • :ref:modindex
  • :ref:search

Build Naming (`docs/build/naming.rst`)

.. _tutorial_constraint_names:

The Importance of Naming Constraints

An important topic worth mentioning is that of constraint naming conventions.
As we've proceeded here, we've talked about adding tables and columns, and
we've also hinted at lots of other operations listed in :ref:ops such as those
which support adding or dropping constraints like foreign keys and unique
constraints. The way these constraints are referred to in migration scripts
is by name, however these names by default are in most cases generated by
the relational database in use, when the constraint is created. For example,
if you emitted two CREATE TABLE statements like this on Postgresql::

test=> CREATE TABLE user_account (id INTEGER PRIMARY KEY);
CREATE TABLE
test=> CREATE TABLE user_order (
test(> id INTEGER PRIMARY KEY,
test(> user_account_id INTEGER REFERENCES user_account(id));
CREATE TABLE

Suppose we wanted to DROP the REFERENCES that we just applied to the
user_order.user_account_id column, how do we do that? At the prompt,
we'd use ALTER TABLE DROP CONSTRAINT <constraint_name>, or if
using Alembic we'd be using :meth:.Operations.drop_constraint. But both
of those functions need a name - what's the name of this constraint?

It does have a name, which in this case we can figure out by looking at the
Postgresql catalog tables::

test=> SELECT r.conname FROM
test-> pg_catalog.pg_class c JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace
test-> JOIN pg_catalog.pg_constraint r ON c.oid = r.conrelid
test-> WHERE c.relname='user_order' AND r.contype = 'f'
test-> ;
conname


user_order_user_account_id_fkey
(1 row)

The name above is not something that Alembic or SQLAlchemy created;
user_order_user_account_id_fkey is a naming scheme used internally by
Postgresql to name constraints that are otherwise not named.

This scheme doesn't seem so complicated, and we might want to just use our
knowledge of it so that we know what name to use for our
:meth:.Operations.drop_constraint call. But is that a good idea? What
if for example we needed our code to run on Oracle as well. OK, certainly
Oracle uses this same scheme, right? Or if not, something similar. Let's
check::

Oracle Database 10g Express Edition Release 10.2.0.1.0 - Production

SQL> CREATE TABLE user_account (id INTEGER PRIMARY KEY);

Table created.

SQL> CREATE TABLE user_order (
2 id INTEGER PRIMARY KEY,
3 user_account_id INTEGER REFERENCES user_account(id));

Table created.

SQL> SELECT constraint_name FROM all_constraints WHERE
2 table_name='USER_ORDER' AND constraint_type in ('R');

CONSTRAINT_NAME

SYS_C0029334

Oh, we can see that is.....much worse. Oracle's names are entirely unpredictable
alphanumeric codes, and this will make being able to write migrations
quite tedious, as we'd need to look up all these names.

The solution to having to look up names is to make your own names. This is
an easy, though tedious thing to do manually. For example, to create our model
in SQLAlchemy ensuring we use names for foreign key constraints would look like::

from sqlalchemy import MetaData, Table, Column, Integer, ForeignKey

meta = MetaData()

user_account = Table('user_account', meta,
Column('id', Integer, primary_key=True)
)

user_order = Table('user_order', meta,
Column('id', Integer, primary_key=True),
Column('user_order_id', Integer,
ForeignKey('user_account.id', name='fk_user_order_id'))
)

Simple enough, though this has some disadvantages. The first is that it's tedious;
we need to remember to use a name for every :class:~sqlalchemy.schema.ForeignKey object,
not to mention every :class:~sqlalchemy.schema.UniqueConstraint, :class:~sqlalchemy.schema.CheckConstraint,
:class:~sqlalchemy.schema.Index, and maybe even :class:~sqlalchemy.schema.PrimaryKeyConstraint
as well if we wish to be able to alter those too, and beyond all that, all the
names have to be globally unique. Even with all that effort, if we have a naming scheme in mind,
it's easy to get it wrong when doing it manually each time.

What's worse is that manually naming constraints (and indexes) gets even more
tedious in that we can no longer use convenience features such as the .unique=True
or .index=True flag on :class:~sqlalchemy.schema.Column::

user_account = Table('user_account', meta,
Column('id', Integer, primary_key=True),
Column('name', String(50), unique=True)
)

Above, the unique=True flag creates a :class:~sqlalchemy.schema.UniqueConstraint, but again,
it's not named. If we want to name it, manually we have to forego the usage
of unique=True and type out the whole constraint::

user_account = Table('user_account', meta,
Column('id', Integer, primary_key=True),
Column('name', String(50)),
UniqueConstraint('name', name='uq_user_account_name')
)

There's a solution to all this naming work, which is to use an automated
naming convention
. For some years, SQLAlchemy has encouraged the use of
DDL Events in order to create naming schemes. The :meth:~sqlalchemy.events.DDLEvents.after_parent_attach
event in particular is the best place to intercept when :class:~sqlalchemy.schema.Constraint
and :class:~sqlalchemy.schema.Index objects are being associated with a parent
:class:~sqlalchemy.schema.Table object, and to assign a .name to the constraint while making
use of the name of the table and associated columns.

But there is also a better way to go, which is to make use of a feature
new in SQLAlchemy 0.9.2 which makes use of the events behind the scenes known as
:paramref:~sqlalchemy.schema.MetaData.naming_convention. Here, we can
create a new :class:~sqlalchemy.schema.MetaData object while passing a dictionary referring
to a naming scheme::

text
convention = {
  "ix": "ix_%(column_0_label)s",
  "uq": "uq_%(table_name)s_%(column_0_name)s",
  "ck": "ck_%(table_name)s_%(constraint_name)s",
  "fk": "fk_%(table_name)s_%(column_0_name)s_%(referred_table_name)s",
  "pk": "pk_%(table_name)s"
}

metadata = MetaData(naming_convention=convention)

If we define our models using a :class:~sqlalchemy.schema.MetaData as above, the given
naming convention dictionary will be used to provide names for all constraints
and indexes.

.. seealso::

text
:ref:`sqla:constraint_naming_conventions` - SQLAlchemy overview of naming
convention support

.. _autogen_naming_conventions:

Integration of Naming Conventions into Operations, Autogenerate

As of Alembic 0.6.4, the naming convention feature is integrated into the
:class:.Operations object, so that the convention takes effect for any
constraint that is otherwise unnamed. The naming convention is passed to
:class:.Operations using the :paramref:.MigrationsContext.configure.target_metadata
parameter in env.py, which is normally configured when autogenerate is
used::

text
# in your application's model:
from sqlalchemy import MetaData
from sqlalchemy.orm import DeclarativeBase

class Base(DeclarativeBase):
    metadata = MetaData(naming_convention={
        "ix": "ix_%(column_0_label)s",
        "uq": "uq_%(table_name)s_%(column_0_name)s",
        "ck": "ck_%(table_name)s_`%(constraint_name)s`",
        "fk": "fk_%(table_name)s_%(column_0_name)s_%(referred_table_name)s",
        "pk": "pk_%(table_name)s"
    })

# .. in your Alembic env.py:

# add your model's MetaData object here
# for 'autogenerate' support
from myapp import mymodel
target_metadata = mymodel.Base.metadata

# ...

def run_migrations_online():

    # ...

    context.configure(
                connection=connection,
                target_metadata=target_metadata
                )

Above, when we render a directive like the following::

text
op.add_column('sometable', Column('q', Boolean(name='q_bool')))

The Boolean type will render a CHECK constraint with the name
"ck_sometable_q_bool", assuming the backend in use does not support
native boolean types.

We can also use op directives with constraints and not give them a name
at all, if the naming convention doesn't require one. The value of
None will be converted into a name that follows the appropriate naming
conventions::

text
def upgrade():
    op.create_unique_constraint(None, 'some_table', 'x')

When autogenerate renders constraints in a migration script, it renders them
typically with their completed name. If using at least Alembic 0.6.4 as well
as SQLAlchemy 0.9.4, these will be rendered with a special directive
:meth:.Operations.f which denotes that the string has already been
tokenized::

text
def upgrade():
    op.create_unique_constraint(op.f('uq_const_x'), 'some_table', 'x')

This :meth:.Operations.f construct may be used explicitly in order to
bypass naming conventions, as illustrated in the next section.

Bypassing the Naming Convention for CREATE and DROP Operations

When using constraint naming conventions, in particular if the
%(constraint_name)s token is in use, the constraint name used with
methods such as :meth:.Operations.create_check_constraint and :meth:.Operations.drop_constraint
with a matching :paramref:.Operations.type_ will include the naming convention
unless additional directives are in use.

Given a configuration in env.py as::

text
target_metadata = MetaData(naming_convention={
        "ix": "ix_%(column_0_label)s",
        "uq": "uq_%(table_name)s_%(column_0_name)s",
        "ck": "ck_%(table_name)s_`%(constraint_name)s`",
        "fk": "fk_%(table_name)s_%(column_0_name)s_%(referred_table_name)s",
        "pk": "pk_%(table_name)s"
      })
# ...

def run_migrations_online():

    # ...

    context.configure(
                connection=connection,
                target_metadata=target_metadata
                )

The following operation will drop a CHECK constraint named
ck_t1_some_check_const::

op.drop_constraint("some_check_const", "t1", type_="check")
ALTER TABLE t1 DROP CONSTRAINT ck_t1_some_check_const

In order to apply the operation while bypassing the configured naming
convention, use the :meth:.Operations.f construct. This produces a string
expression that will not be tokenized::

op.drop_constraint(op.f("some_check_const"), "t1", type_="check")
ALTER TABLE t1 DROP CONSTRAINT some_check_const


Build Offline (`docs/build/offline.rst`)

Generating SQL Scripts (a.k.a. "Offline Mode")

A major capability of Alembic is to generate migrations as SQL scripts, instead of running
them against the database - this is also referred to as offline mode.
This is a critical feature when working in large organizations
where access to DDL is restricted, and SQL scripts must be handed off to DBAs. Alembic makes
this easy via the --sql option passed to any upgrade or downgrade command. We
can, for example, generate a script that revises up to rev ae1027a6acf::

text
$ alembic upgrade ae1027a6acf --sql
INFO  [alembic.context] Context class PostgresqlContext.
INFO  [alembic.context] Will assume transactional DDL.
BEGIN;

CREATE TABLE alembic_version (
    version_num VARCHAR(32) NOT NULL
);

INFO  [alembic.context] Running upgrade None -> 1975ea83b712
CREATE TABLE account (
    id SERIAL NOT NULL,
    name VARCHAR(50) NOT NULL,
    description VARCHAR(200),
    PRIMARY KEY (id)
);

INFO  [alembic.context] Running upgrade 1975ea83b712 -> ae1027a6acf
ALTER TABLE account ADD COLUMN last_transaction_date TIMESTAMP WITHOUT TIME ZONE;

INSERT INTO alembic_version (version_num) VALUES ('ae1027a6acf');

COMMIT;

While the logging configuration dumped to standard error, the actual script was dumped to standard output -
so in the absence of further configuration (described later in this section), we'd at first be using output
redirection to generate a script::

text
$ alembic upgrade ae1027a6acf --sql > migration.sql

Getting the Start Version

Notice that our migration script started at the base - this is the default when using offline
mode, as no database connection is present and there's no alembic_version table to read from.

One way to provide a starting version in offline mode is to provide a range to the command line.
This is accomplished by providing the "version" in start:end syntax::

text
$ alembic upgrade 1975ea83b712:ae1027a6acf --sql > migration.sql

The start:end syntax is only allowed in offline mode; in "online" mode, the alembic_version
table is always used to get at the current version.

It's also possible to have the env.py script retrieve the "last" version from
the local environment, such as from a local file. A scheme like this would basically
treat a local file in the same way alembic_version works::

text
if context.is_offline_mode():
    version_file = os.path.join(os.path.dirname(config.config_file_name), "version.txt")
    if os.path.exists(version_file):
        current_version = open(version_file).read()
    else:
        current_version = None
    context.configure(dialect_name=engine.name, starting_rev=current_version)
    context.run_migrations()
    end_version = context.get_revision_argument()
    if end_version and end_version != current_version:
        open(version_file, 'w').write(end_version)

Writing Migration Scripts to Support Script Generation

The challenge of SQL script generation is that the scripts we generate can't rely upon
any client/server database access. This means a migration script that pulls some rows
into memory via a SELECT statement will not work in --sql mode. It's also
important that the Alembic directives, all of which are designed specifically to work
in both "live execution" as well as "offline SQL generation" mode, are used.

Customizing the Environment

Users of the --sql option are encouraged to hack their env.py files to suit their
needs. The env.py script as provided is broken into two sections: run_migrations_online()
and run_migrations_offline(). Which function is run is determined at the bottom of the
script by reading :meth:.EnvironmentContext.is_offline_mode, which basically determines if the
--sql flag was enabled.

For example, a multiple database configuration may want to run through each
database and set the output of the migrations to different named files - the :meth:.EnvironmentContext.configure
function accepts a parameter output_buffer for this purpose. Below we illustrate
this within the run_migrations_offline() function::

text
from alembic import context
import myapp
import sys

db_1 = myapp.db_1
db_2 = myapp.db_2

def run_migrations_offline():
    """Run migrations *without* a SQL connection."""

    for name, engine, file_ in [
        ("db1", db_1, "db1.sql"),
        ("db2", db_2, "db2.sql"),
    ]:
        context.configure(
                    url=engine.url,
                    transactional_ddl=False,
                    output_buffer=open(file_, 'w'))
        context.execute("-- running migrations for '%s'" % name)
        context.run_migrations(name=name)
        sys.stderr.write("Wrote file '%s'" % file_)

def run_migrations_online():
    """Run migrations *with* a SQL connection."""

    for name, engine in [
        ("db1", db_1),
        ("db2", db_2),
    ]:
        connection = engine.connect()
        context.configure(connection=connection)
        try:
            context.run_migrations(name=name)
            session.commit()
        except:
            session.rollback()
            raise

if context.is_offline_mode():
    run_migrations_offline()
else:
    run_migrations_online()

Build Ops (`docs/build/ops.rst`)

.. _ops:

=================== Operation Reference

This file provides documentation on Alembic migration directives.

The directives here are used within user-defined migration files,
within the upgrade() and downgrade() functions, as well as
any functions further invoked by those.

All directives exist as methods on a class called :class:.Operations.
When migration scripts are run, this object is made available
to the script via the alembic.op datamember, which is
a proxy to an actual instance of :class:.Operations.
Currently, alembic.op is a real Python module, populated
with individual proxies for each method on :class:.Operations,
so symbols can be imported safely from the alembic.op namespace.

The :class:.Operations system is also fully extensible. See
:ref:operation_plugins for details on this.

A key design philosophy to the :ref:alembic.operations.toplevel methods is that
to the greatest degree possible, they internally generate the
appropriate SQLAlchemy metadata, typically involving
:class:~sqlalchemy.schema.Table and :class:~sqlalchemy.schema.Constraint
objects. This so that migration instructions can be
given in terms of just the string names and/or flags involved.
The exceptions to this
rule include the :meth:~.Operations.add_column and :meth:~.Operations.create_table
directives, which require full :class:~sqlalchemy.schema.Column
objects, though the table metadata is still generated here.

The functions here all require that a :class:.MigrationContext has been
configured within the env.py script first, which is typically
via :meth:.EnvironmentContext.configure. Under normal
circumstances they are called from an actual migration script, which
itself would be invoked by the :meth:.EnvironmentContext.run_migrations
method.

.. module:: alembic.operations

.. class:: AbstractOperations

text
Base class for :class:`.Operations` and :class:`.BatchOperations`.

See :class:`.Operations` for full list of members

.. autoclass:: Operations
:members:
:inherited-members:

.. autoclass:: BatchOperations
:members:


Build Requirements (`docs/build/requirements.txt`)

git+https://github.com/sqlalchemyorg/changelog.git#egg=changelog
git+https://github.com/sqlalchemyorg/sphinx-paramlinks.git#egg=sphinx-paramlinks
git+https://github.com/sqlalchemy/sqlalchemy.git
backports.zoneinfo;python_version<"3.9"

because there's a dependency in pyfiles.py

Mako
importlib-metadata;python_version<"3.9"
importlib-resources;python_version<"3.9"
sphinx_copybutton==0.5.1
sphinx-book-theme


Build Tutorial (`docs/build/tutorial.rst`)

======== Tutorial

Alembic provides for the creation, management, and invocation of change management
scripts for a relational database, using SQLAlchemy as the underlying engine.
This tutorial will provide a full introduction to the theory and usage of this tool.

To begin, make sure Alembic is installed; a common way to install within a
local virtual environment is described at :ref:installation.
As illustrated in that chapter, it is useful to have Alembic
installed in the same module / Python path as that of the target project,
usually using a Python virtual environment <https://docs.python.org/3/tutorial/venv.html>_, so that when the alembic
command is run, the Python script which is invoked by alembic, namely your
project's env.py script, will have access to your application's models.
This is not strictly necessary, however is usually preferred.

The tutorial below assumes the alembic command line utility is present in
the local path and when invoked, will have access to the same Python module
environment as that of the target project.

The Migration Environment

Usage of Alembic starts with creation of the Migration Environment. This is a directory of scripts
that is specific to a particular application. The migration environment is created just once,
and is then maintained along with the application's source code itself. The environment is
created using the init command of Alembic, and is then customizable to suit the specific
needs of the application.

The structure of this environment, including some generated migration scripts, looks like::

text
yourproject/
    alembic.ini
    pyproject.toml
    alembic/
        env.py
        README
        script.py.mako
        versions/
            3512b954651e_add_account.py
            2b1ae634e5cd_add_order_id.py
            3adcc9a56557_rename_username_field.py

The directory includes these directories/files:

  • alembic.ini - this is Alembic's main configuration file which is generated by all templates.
    A detailed walkthrough of this file is later in the section :ref:tutorial_alembic_ini.

  • pyproject.toml - most modern Python projects have a pyproject.toml file. Alembic may
    optionally store project related configuration in this file as well; to use a pyproject.toml
    configuration, see the section :ref:using_pep_621.

  • yourproject - this is the root of your application's source code, or some directory within it.

  • alembic - this directory lives within your application's source tree and is the home of the
    migration environment. It can be named anything, and a project that uses multiple databases
    may even have more than one.

  • env.py - This is a Python script that is run whenever the alembic migration tool is invoked.
    At the very least, it contains instructions to configure and generate a SQLAlchemy engine,
    procure a connection from that engine along with a transaction, and then invoke the migration
    engine, using the connection as a source of database connectivity.

    The env.py script is part of the generated environment so that the way migrations run
    is entirely customizable. The exact specifics of how to connect are here, as well as
    the specifics of how the migration environment are invoked. The script can be modified
    so that multiple engines can be operated upon, custom arguments can be passed into the
    migration environment, application-specific libraries and models can be loaded in and
    made available.

    Alembic includes a set of initialization templates which feature different varieties
    of env.py for different use cases.

  • README - included with the various environment templates, should have something
    informative.

  • script.py.mako - This is a Mako <http://www.makotemplates.org>_ template file which
    is used to generate new migration scripts. Whatever is here is used to generate new
    files within versions/. This is scriptable so that the structure of each migration
    file can be controlled, including standard imports to be within each, as well as
    changes to the structure of the upgrade() and downgrade() functions. For example,
    the multidb environment allows for multiple functions to be generated using a
    naming scheme upgrade_engine1(), upgrade_engine2().

  • versions/ - This directory holds the individual version scripts. Users of other migration
    tools may notice that the files here don't use ascending integers, and instead use a
    partial GUID approach. In Alembic, the ordering of version scripts is relative
    to directives within the scripts themselves, and it is theoretically possible to "splice" version files
    in between others, allowing migration sequences from different branches to be merged,
    albeit carefully by hand.

Creating an Environment

With a basic understanding of what the environment is, we can create one using alembic init.
This will create an environment using the "generic" template::

text
$ cd /path/to/yourproject
$ source /path/to/yourproject/.venv/bin/activate   # assuming a local virtualenv
$ alembic init alembic

Where above, the init command was called to generate a migrations directory called alembic::

text
Creating directory /path/to/yourproject/alembic...done
Creating directory /path/to/yourproject/alembic/versions...done
Generating /path/to/yourproject/alembic.ini...done
Generating /path/to/yourproject/alembic/env.py...done
Generating /path/to/yourproject/alembic/README...done
Generating /path/to/yourproject/alembic/script.py.mako...done
Please edit configuration/connection/logging settings in
'/path/to/yourproject/alembic.ini' before proceeding.

The above layout is produced using a layout template called generic.
Alembic also includes other environment templates. These can be listed out
using the list_templates command::

text
$ alembic list_templates
Available templates:

generic - Generic single-database configuration.
pyproject - pep-621 compliant configuration that includes pyproject.toml
async - Generic single-database configuration with an async dbapi.
multidb - Rudimentary multi-database configuration.

Templates are used via the 'init' command, e.g.:

  alembic init --template generic ./scripts

.. versionchanged:: 1.16.0 A new pyproject template has been added. See
the section :ref:using_pep_621 for background.

.. _tutorial_alembic_ini:

Editing the .ini File

Alembic placed a file alembic.ini into the current directory. Alembic looks
in the current directory for this file when any other commands are run; to
indicate an alternative location, the --config option may be used, or the
ALEMBIC_CONFIG environment variable may be set.

.. tip::

text
The file generated with the ``generic`` configuration template contains all directives
for both source code configuration as well as database configuration.  When using
the ``pyproject`` template, the source code configuration elements will instead
be in a separate ``pyproject.toml`` file, described in the section :ref:`using_pep_621`.

The all-in-one .ini file created by generic is illustrated below::

text
# A generic, single database configuration.

[alembic]
# path to migration scripts.
# this is typically a path given in POSIX (e.g. forward slashes)
# format, relative to the token %(here)s which refers to the location of this
# ini file
script_location = %(here)s/alembic

# template used to generate migration file names; The default value is %%(rev)s_%%(slug)s
# Uncomment the line below if you want the files to be prepended with date and time
# file_template = %%(year)d_%%(month).2d_%%(day).2d_%%(hour).2d%%(minute).2d-%%(rev)s_%%(slug)s
# Or organize into date-based subdirectories (requires recursive_version_locations = true)
# file_template = %%(year)d/%%(month).2d/%%(day).2d_%%(hour).2d%%(minute).2d_%%(second).2d_%%(rev)s_%%(slug)s

# sys.path path, will be prepended to sys.path if present.
# defaults to the current working directory.
prepend_sys_path = .

# timezone to use when rendering the date within the migration file
# as well as the filename.
# If specified, requires the python>=3.9 or backports.zoneinfo library and tzdata library.
# Any required deps can installed by adding `alembic[tz]` to the pip requirements
# string value is passed to ZoneInfo()
# leave blank for localtime
# timezone =

# max length of characters to apply to the
# "slug" field
# truncate_slug_length = 40

# set to 'true' to run the environment during
# the 'revision' command, regardless of autogenerate
# revision_environment = false

# set to 'true' to allow .pyc and .pyo files without
# a source .py file to be detected as revisions in the
# versions/ directory
# sourceless = false

# version location specification; This defaults
# to <script_location>/versions.  When using multiple version
# directories, initial revisions must be specified with --version-path.
# the special token `%(here)s` is available which indicates the absolute path
# to this configuration file.
#
# The path separator used here should be the separator specified by "version_path_separator" below.
# version_locations = %(here)s/bar:%(here)s/bat:%(here)s/alembic/versions

# path_separator (New in Alembic 1.16.0, supersedes version_path_separator);
# This indicates what character is used to
# split lists of file paths, including version_locations and prepend_sys_path
# within configparser files such as alembic.ini.
#
# The default rendered in new alembic.ini files is "os", which uses os.pathsep
# to provide os-dependent path splitting.
#
# Note that in order to support legacy alembic.ini files, this default does NOT
# take place if path_separator is not present in alembic.ini.  If this
# option is omitted entirely, fallback logic is as follows:
#
# 1. Parsing of the version_locations option falls back to using the legacy
#    "version_path_separator" key, which if absent then falls back to the legacy
#    behavior of splitting on spaces and/or commas.
# 2. Parsing of the prepend_sys_path option falls back to the legacy
#    behavior of splitting on spaces, commas, or colons.
#
# Valid values for path_separator are:
#
# path_separator = :
# path_separator = ;
# path_separator = space
# path_separator = newline
#
# Use os.pathsep. Default configuration used for new projects.
path_separator = os

# set to 'true' to search source files recursively
# in each "version_locations" directory
# new in Alembic version 1.10
# recursive_version_locations = false

# the output encoding used when revision files
# are written from script.py.mako
# output_encoding = utf-8

# database URL.  This is consumed by the user-maintained env.py script only.
# other means of configuring database URLs may be customized within the env.py
# file.
# See notes in "escaping characters in ini files" for guidelines on
# passwords
sqlalchemy.url = driver://user:pass@localhost/dbname

# [post_write_hooks]
# This section defines scripts or Python functions that are run
# on newly generated revision scripts.  See the documentation for further
# detail and examples

# format using "black" - use the console_scripts runner,
# against the "black" entrypoint
# hooks = black
# black.type = console_scripts
# black.entrypoint = black
# black.options = -l 79 REVISION_SCRIPT_FILENAME

# lint with attempts to fix using "ruff" - use the module runner, against the "ruff" module
# hooks = ruff
# ruff.type = module
# ruff.module = ruff
# ruff.options = check --fix REVISION_SCRIPT_FILENAME

# Alternatively, use the exec runner to execute a binary found on your PATH
# hooks = ruff
# ruff.type = exec
# ruff.executable = ruff
# ruff.options = check --fix REVISION_SCRIPT_FILENAME

# Logging configuration.  This is also consumed by the user-maintained
# env.py script only.
[loggers]
keys = root,sqlalchemy,alembic

[handlers]
keys = console

[formatters]
keys = generic

[logger_root]
level = WARNING
handlers = console
qualname =

[logger_sqlalchemy]
level = WARNING
handlers =
qualname = sqlalchemy.engine

[logger_alembic]
level = INFO
handlers =
qualname = alembic

[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic

[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S

The alembic.ini file is consumed by Alembic using Python's
configparser.ConfigParser <https://docs.python.org/3/library/configparser.html#configparser.ConfigParser>_
library. The %(here)s variable is
provided as a substitution which is populated with the absolute path to the
alembic.ini file itself. This can be used to produce correct pathnames
to directories and files relative to where the config file is located.

.. tip:: Percent signs in alembic.ini configuration variables that are
not part of an interpolation token like %(here)s, including percent
signs that are part of the SQLAlchemy database URL for its own URL-escaping
requirements, must themselves be escaped.
See the section :ref:escaping_percent_signs for more information.

This file contains the following features:

  • [alembic] - this is the section read by Alembic to determine configuration. Alembic's
    core implementation does not directly read any other areas of the file, not
    including additional directives that may be consumed from the
    end-user-customizable env.py file (see note below). The name "alembic"
    (for configparser config only, not pyproject.toml)
    can be customized using the --name commandline flag; see
    :ref:multiple_environments for a basic example of this.

    .. note:: The default env.py file included with Alembic's environment
    templates will also read from the logging sections [logging],
    [handlers] etc. If the configuration file in use does not contain
    logging directives, please remove the fileConfig() directive within
    the generated env.py file to prevent it from attempting to configure
    logging.

  • script_location - this is the location of the Alembic environment. It is normally
    specified as a filesystem location relative to the %(here)s token, which
    indicates where the config file itself is located. The location may also
    be a plain relative path, where it's interpreted as relative to the current directory,
    or an absolute path.

    This is the only key required by Alembic in all cases. The generation
    of the .ini file by the command alembic init alembic automatically placed the
    directory name alembic here. The special variable %(here)s can also be used,
    as in %(here)s/alembic.

    For support of applications that package themselves into .egg files, the value can
    also be specified as a package resource <https://setuptools.readthedocs.io/en/latest/pkg_resources.html>_, in which
    case resource_filename() is used to find the file (new in 0.2.2). Any non-absolute
    URI which contains colons is interpreted here as a resource name, rather than
    a straight filename.

  • file_template - this is the naming scheme used to generate new migration
    files. Uncomment the presented value if you would like the migration files to
    be prepended with date and time, so that they are listed in chronological
    order. The default value is %%(rev)s_%%(slug)s. Tokens available
    include:

    • %%(rev)s - revision id
    • %%(slug)s - a truncated string derived from the revision message
    • %%(epoch)s - epoch timestamp based on the create date; this makes
      use of the Python datetime.timestamp() method to produce an epoch
      value.
    • %%(year)d, %%(month).2d, %%(day).2d, %%(hour).2d,
      %%(minute).2d, %%(second).2d - components of the create date,
      by default datetime.datetime.now() unless the timezone
      configuration option is also used.

    The file_template may also include directory separators to organize
    migration files into subdirectories. When using directory paths in
    file_template, recursive_version_locations must be set to true.
    For example::

    text
    file_template = %%(year)d/%%(month).2d/%%(day).2d_%%(hour).2d%%(minute).2d_%%(second).2d_%%(rev)s_%%(slug)s
    recursive_version_locations = true

    This would create migration files organized by date in a structure like
    versions/2024/12/26_143022_abc123_add_user_table.py.

    .. versionadded:: 1.18.0
    Support for directory paths in file_template

  • timezone - an optional timezone name (e.g. UTC, EST5EDT, etc.)
    that will be applied to the timestamp which renders inside the migration
    file's comment as well as within the filename. This option requires Python>=3.9
    or installing the backports.zoneinfo library and the tzdata library.
    If timezone is specified, the create date object is no longer derived
    from datetime.datetime.now() and is instead generated as::

    text
    datetime.datetime.utcnow().replace(
      tzinfo=datetime.timezone.utc
    ).astimezone(ZoneInfo(<timezone>))

    .. versionchanged:: 1.13.0 Python standard library zoneinfo is now used
    for timezone rendering in migrations; previously python-dateutil
    was used.

  • truncate_slug_length - defaults to 40, the max number of characters
    to include in the "slug" field.

  • sqlalchemy.url - A URL to connect to the database via SQLAlchemy. This
    configuration value is only used if the env.py file calls upon them;
    in the "generic" template, the call to
    config.get_main_option("sqlalchemy.url") in the
    run_migrations_offline() function and the call to
    engine_from_config(prefix="sqlalchemy.") in the
    run_migrations_online() function are where this key is referenced. If
    the SQLAlchemy URL should come from some other source, such as from
    environment variables or a global registry, or if the migration environment
    makes use of multiple database URLs, the developer is encouraged to alter the
    env.py file to use whatever methods are appropriate in order to acquire
    the database URL or URLs.

  • revision_environment - this is a flag which when set to the value 'true', will indicate
    that the migration environment script env.py should be run unconditionally when
    generating new revision files, as well as when running the alembic history
    command.

  • sourceless - when set to 'true', revision files that only exist as .pyc
    or .pyo files in the versions directory will be used as versions, allowing
    "sourceless" versioning folders. When left at the default of 'false',
    only .py files are consumed as version files.

  • version_locations - an optional list of revision file locations, to
    allow revisions to exist in multiple directories simultaneously.
    See :ref:multiple_bases for examples.

  • path_separator - a separator character for the version_locations
    and prepend_sys_path path lists. Only applies to configparser config,
    not needed if pyproject.toml configuration is used.
    See :ref:multiple_bases for examples.

  • recursive_version_locations - when set to 'true', revision files
    are searched recursively in each "version_locations" directory.

    .. versionadded:: 1.10

  • output_encoding - the encoding to use when Alembic writes the
    script.py.mako file into a new migration file. Defaults to 'utf-8'.

  • [loggers], [handlers], [formatters], [logger_*], [handler_*],
    [formatter_*] - these sections are all part of Python's standard logging configuration,
    the mechanics of which are documented at Configuration File Format <http://docs.python.org/library/logging.config.html#configuration-file-format>_.
    As is the case with the database connection, these directives are used directly as the
    result of the logging.config.fileConfig() call present in the
    env.py script, which you're free to modify.

For starting up with just a single database and the generic configuration, setting up
the SQLAlchemy URL is all that's needed::

text
sqlalchemy.url = postgresql://scott:tiger@localhost/test

.. _escaping_percent_signs:

Escaping Characters in ini files

As mentioned previously, Alembic's .ini file format uses Python ConfigParser <https://docs.python.org/3/library/configparser.html#configparser.ConfigParser>_
to parse the file. ConfigParser 's interpolation feature is enabled <https://docs.python.org/3/library/configparser.html#interpolation-of-values>_
in this operation to support the use of the %(here)s token, as well as any
other tokens that are user-configurable via the :paramref:.Config.config_args
parameter when creating a custom :class:.Config object.

This means that any literal string that includes a percent sign that is not
part of an interpolated variable must be escaped by doubling it. That is, for
a configuration value like this in a Python script::

my_configuration_value = "some % string"

To be parsed from the .ini file would need to be placed as::

[alembic]

my_configuration_value = some %% string

This escaping can be seen in the sample alembic.ini file, illustrated in
such values as file_template::

text
# template used to generate migration file names; The default value is %%(rev)s_%%(slug)s
file_template = %%(year)d_%%(month).2d_%%(day).2d_%%(hour).2d%%(minute).2d-%%(rev)s_%%(slug)s

Where above, the actual file_template that is sent to Alembic's file generation system
would be %(year)d_%(month).2d_%(day).2d_%(hour).2d%(minute).2d-%(rev)s_%(slug)s.

.. tip:: Alembic also employs percent-sign interpolation of values when retrieving
values from a pyproject.toml file, as documented at :ref:using_pep_621.
So the same percent-doubling steps must be applied in Alembic-parsed values,
for fields such as file_template.

For the SQLAlchemy URL, percent signs are used to escape syntactically-
significant characters such as the @ sign as well as the percent sign
itself. For a password such as "P@ssw%rd"::

my_actual_password = "P@ssw%rd"

As documented by SQLAlchemy <https://docs.sqlalchemy.org/core/engines.html#escaping-special-characters-such-as-signs-in-passwords>_,
the @ sign as well as the percent sign when placed into a URL should be escaped with urllib.parse.quote_plus::

import urllib.parse
sqlalchemy_quoted_password = urllib.parse.quote_plus(my_actual_password)
sqlalchemy_quoted_password
'P%40ssw%25rd'

This URL quoting can also be seen in SQLAlchemy's own stringification of
URLs::

from sqlalchemy import URL
URL.create(
... "some_db", username="scott", password=my_actual_password, host="host"
... ).render_as_string(hide_password=False)
'some_db://scott:P%40ssw%25rd@host'

For the above escaped password string 'P%40ssw%rd' to be placed into a ConfigParser file that
includes interpolation of percent signs, % characters are doubled::

sqlalchemy_quoted_password.replace("%", "%%")
'P%%40ssw%%25rd'

Here's a complete program that will compose a URL and show the correct configparser form
for a given set of database connection details, as well as illustrate how to assert these
forms for correctness::

text
from sqlalchemy import URL, make_url

database_driver = input("database driver? ")
username = input("username? ")
password = input("password? ")
host = input("host? ")
port = input("port? ")
database = input("database? ")

sqlalchemy_url = URL.create(
    drivername=database_driver,
    username=username,
    password=password,
    host=host,
    port=int(port),
    database=database,
)

stringified_sqlalchemy_url = sqlalchemy_url.render_as_string(
    hide_password=False
)

# assert make_url round trip
assert make_url(stringified_sqlalchemy_url) == sqlalchemy_url

print(
    f"The correctly escaped string that can be passed "
    f"to SQLAlchemy make_url() and create_engine() is:"
    f"\n\n     {stringified_sqlalchemy_url!r}\n"
)

percent_replaced_url = stringified_sqlalchemy_url.replace("%", "%%")

# assert percent-interpolated plus make_url round trip
assert make_url(percent_replaced_url % {}) == sqlalchemy_url

print(
    f"The SQLAlchemy URL that can be placed in a ConfigParser "
    f"file such as alembic.ini is:\n\n      "
    f"sqlalchemy.url = {percent_replaced_url}\n"
)

The above program should eliminate any ambiguity when placing a SQLAlchemy
URL into a configparser file::

text
$ python alembic_pw_script.py
database driver? postgresql+psycopg2
username? scott
password? P@ssw%rd
host? localhost
port? 5432
database? testdb
The correctly escaped string that can be passed to SQLAlchemy make_url() and create_engine() is:

    'postgresql+psycopg2://scott:P%40ssw%25rd@localhost:5432/testdb'

The SQLAlchemy URL that can be placed in a ConfigParser file such as alembic.ini is:

      sqlalchemy.url = postgresql+psycopg2://scott:P%%40ssw%%25rd@localhost:5432/testdb

.. _using_pep_621:

Using pyproject.toml for configuration

.. versionadded:: 1.16.0

As the alembic.ini file includes a subset of options that are specific to
the organization and production of Python code within the local environment,
these specific options may alternatively be placed in the application's
pyproject.toml file, to allow for :pep:621 compliant configuration.

Use of pyproject.toml does not preclude having an alembic.ini file as
well, as alembic.ini is still the default location for deployment
details such as database URLs, connectivity options, and logging to be present.
However, as connectivity and logging is consumed only by user-managed code
within the env.py file, it is feasible to have an environment that does not
require the alembic.ini file itself to be present at all, if these
configurational elements are consumed from other places elsewhere in the
application. Alembic will still run successfully if only a pyproject.toml
file is present and no alembic.ini is found.

To start with a pyproject configuration, the most straightforward approach is
to use the pyproject template::

text
alembic init --template pyproject alembic

The output states that the existing pyproject file is being augmented with
additional directives::

text
Creating directory /path/to/yourproject/alembic...done
Creating directory /path/to/yourproject/alembic/versions...done
Appending to /path/to/yourproject/pyproject.toml...done
Generating /path/to/yourproject/alembic.ini...done
Generating /path/to/yourproject/alembic/env.py...done
Generating /path/to/yourproject/alembic/README...done
Generating /path/to/yourproject/alembic/script.py.mako...done
Please edit configuration/connection/logging settings in
'/path/to/yourproject/pyproject.toml' and
'/path/to/yourproject/alembic.ini' before proceeding.

Alembic's template runner will generate a new pyproject.toml file if
one does not exist, or it will append directives to an existing pyproject.toml
file that does not already include alembic directives.

Within the pyproject.toml file, the default section generated looks mostly
like the alembic.ini file, with the welcome exception that lists of values
are supported directly; this means the values prepend_sys_path and
version_locations are specified as lists. The %(here)s token also
remains available as the absolute path to the pyproject.toml file::

text
[tool.alembic]
# path to migration scripts
script_location = "%(here)s/alembic"

# template used to generate migration file names; The default value is %%(rev)s_%%(slug)s
# Uncomment the line below if you want the files to be prepended with date and time
# file_template = %%(year)d_%%(month).2d_%%(day).2d_%%(hour).2d%%(minute).2d-%%(rev)s_%%(slug)s
# Or organize into date-based subdirectories (requires recursive_version_locations = true)
# file_template = %%(year)d/%%(month).2d/%%(day).2d_%%(hour).2d%%(minute).2d_%%(second).2d_%%(rev)s_%%(slug)s

# additional paths to be prepended to sys.path. defaults to the current working directory.
prepend_sys_path = [
    "."
]

# timezone to use when rendering the date within the migration file
# as well as the filename.
# If specified, requires the python>=3.9 or backports.zoneinfo library and tzdata library.
# Any required deps can installed by adding `alembic[tz]` to the pip requirements
# string value is passed to ZoneInfo()
# leave blank for localtime
# timezone =

# max length of characters to apply to the
# "slug" field
# truncate_slug_length = 40

# set to 'true' to run the environment during
# the 'revision' command, regardless of autogenerate
# revision_environment = false

# set to 'true' to allow .pyc and .pyo files without
# a source .py file to be detected as revisions in the
# versions/ directory
# sourceless = false

# version location specification; This defaults
# to <script_location>/versions.  When using multiple version
# directories, initial revisions must be specified with --version-path.
# version_locations = [
#    "%(here)s/alembic/versions",
#    "%(here)s/foo/bar"
# ]

# set to 'true' to search source files recursively
# in each "version_locations" directory
# new in Alembic version 1.10
# recursive_version_locations = false

# the output encoding used when revision files
# are written from script.py.mako
# output_encoding = "utf-8"


# This section defines scripts or Python functions that are run
# on newly generated revision scripts.  See the documentation for further
# detail and examples
# [[tool.alembic.post_write_hooks]]
# format using "black" - use the console_scripts runner,
# against the "black" entrypoint
# name = "black"
# type = "console_scripts"
# entrypoint = "black"
# options = "-l 79 REVISION_SCRIPT_FILENAME"
#
# [[tool.alembic.post_write_hooks]]
# lint with attempts to fix using "ruff" - use the exec runner,
# execute a binary
# name = "ruff"
# type = "exec"
# executable = "%(here)s/.venv/bin/ruff"
# options = "check --fix REVISION_SCRIPT_FILENAME"

.. tip:: As Alembic adds support for interpolation tokens like %(here)s to
its handling of pyproject.toml values, the same percent-sign escaping
steps that apply to alembic.ini configuration variables also apply
to pyproject.toml, even though database URLs are not configured in this
file. This escaping can be seen in the sample file_template value
above. See the section :ref:escaping_percent_signs for background.

The alembic.ini file for this template is truncated and contains
only database configuration and logging configuration::

text
[alembic]

# database URL.  This is consumed by the user-maintained env.py script only.
# other means of configuring database URLs may be customized within the env.py
# file.
sqlalchemy.url = driver://user:pass@localhost/dbname

# Logging configuration.  This is also consumed by the user-maintained
# env.py script only.
[loggers]
keys = root,sqlalchemy,alembic

[handlers]
keys = console

[formatters]
keys = generic

[logger_root]
level = WARNING
handlers = console
qualname =

[logger_sqlalchemy]
level = WARNING
handlers =
qualname = sqlalchemy.engine

[logger_alembic]
level = INFO
handlers =
qualname = alembic

[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = NOTSET
formatter = generic

[formatter_generic]
format = %(levelname)-5.5s [%(name)s] %(message)s
datefmt = %H:%M:%S

When env.py is configured to obtain database connectivity and logging
configuration from places other than alembic.ini, the file can be
omitted altogether.

.. _create_migration:

Create a Migration Script

With the environment in place we can create a new revision, using alembic revision::

text
$ alembic revision -m "create account table"
Generating /path/to/yourproject/alembic/versions/1975ea83b712_create_accoun
t_table.py...done

A new file 1975ea83b712_create_account_table.py is generated. Looking inside the file::

text
"""create account table

Revision ID: 1975ea83b712
Revises:
Create Date: 2011-11-08 11:40:27.089406

"""

# revision identifiers, used by Alembic.
revision = '1975ea83b712'
down_revision = None
branch_labels = None

from alembic import op
import sqlalchemy as sa

def upgrade():
    pass

def downgrade():
    pass

The file contains some header information, identifiers for the current revision
and a "downgrade" revision, an import of basic Alembic directives,
and empty upgrade() and downgrade() functions. Our
job here is to populate the upgrade() and downgrade() functions with directives that
will apply a set of changes to our database. Typically, upgrade() is required
while downgrade() is only needed if down-revision capability is desired, though it's
probably a good idea.

Another thing to notice is the down_revision variable. This is how Alembic
knows the correct order in which to apply migrations. When we create the next revision,
the new file's down_revision identifier would point to this one::

text
# revision identifiers, used by Alembic.
revision = 'ae1027a6acf'
down_revision = '1975ea83b712'

Every time Alembic runs an operation against the versions/ directory, it reads all
the files in, and composes a list based on how the down_revision identifiers link together,
with the down_revision of None representing the first file. In theory, if a
migration environment had thousands of migrations, this could begin to add some latency to
startup, but in practice a project should probably prune old migrations anyway
(see the section :ref:building_uptodate for a description on how to do this, while maintaining
the ability to build the current database fully).

We can then add some directives to our script, suppose adding a new table account::

text
def upgrade():
    op.create_table(
        'account',
        sa.Column('id', sa.Integer, primary_key=True),
        sa.Column('name', sa.String(50), nullable=False),
        sa.Column('description', sa.Unicode(200)),
    )

def downgrade():
    op.drop_table('account')

:meth:~.Operations.create_table and :meth:~.Operations.drop_table are Alembic directives. Alembic provides
all the basic database migration operations via these directives, which are designed to be as simple and
minimalistic as possible;
there's no reliance upon existing table metadata for most of these directives. They draw upon
a global "context" that indicates how to get at a database connection (if any; migrations can
dump SQL/DDL directives to files as well) in order to invoke the command. This global
context is set up, like everything else, in the env.py script.

An overview of all Alembic directives is at :ref:ops.

Running our First Migration

We now want to run our migration. Assuming our database is totally clean, it's as
yet unversioned. The alembic upgrade command will run upgrade operations, proceeding
from the current database revision, in this example None, to the given target revision.
We can specify 1975ea83b712 as the revision we'd like to upgrade to, but it's easier
in most cases just to tell it "the most recent", in this case head::

text
$ alembic upgrade head
INFO  [alembic.context] Context class PostgresqlContext.
INFO  [alembic.context] Will assume transactional DDL.
INFO  [alembic.context] Running upgrade None -> 1975ea83b712

Wow that rocked! Note that the information we see on the screen is the result of the
logging configuration set up in alembic.ini - logging the alembic stream to the
console (standard error, specifically).

The process which occurred here included that Alembic first checked if the database had
a table called alembic_version, and if not, created it. It looks in this table
for the current version, if any, and then calculates the path from this version to
the version requested, in this case head, which is known to be 1975ea83b712.
It then invokes the upgrade() method in each file to get to the target revision.

Running our Second Migration

Let's do another one so we have some things to play with. We again create a revision
file::

text
$ alembic revision -m "Add a column"
Generating /path/to/yourapp/alembic/versions/ae1027a6acf_add_a_column.py...
done

Let's edit this file and add a new column to the account table::

text
"""Add a column

Revision ID: ae1027a6acf
Revises: 1975ea83b712
Create Date: 2011-11-08 12:37:36.714947

"""

# revision identifiers, used by Alembic.
revision = 'ae1027a6acf'
down_revision = '1975ea83b712'

from alembic import op
import sqlalchemy as sa

def upgrade():
    op.add_column('account', sa.Column('last_transaction_date', sa.DateTime))

def downgrade():
    op.drop_column('account', 'last_transaction_date')

Running again to head::

text
$ alembic upgrade head
INFO  [alembic.context] Context class PostgresqlContext.
INFO  [alembic.context] Will assume transactional DDL.
INFO  [alembic.context] Running upgrade 1975ea83b712 -> ae1027a6acf

We've now added the last_transaction_date column to the database.

Partial Revision Identifiers

Any time we need to refer to a revision number explicitly, we have the option
to use a partial number. As long as this number uniquely identifies the
version, it may be used in any command in any place that version numbers
are accepted::

text
$ alembic upgrade ae1

Above, we use ae1 to refer to revision ae1027a6acf.
Alembic will stop and let you know if more than one version starts with
that prefix.

.. _relative_migrations:

Relative Migration Identifiers

Relative upgrades/downgrades are also supported. To move two versions from
the current, a decimal value "+N" can be supplied::

text
$ alembic upgrade +2

Negative values are accepted for downgrades::

text
$ alembic downgrade -1

Relative identifiers may also be in terms of a specific revision. For example,
to upgrade to revision ae1027a6acf plus two additional steps::

text
$ alembic upgrade ae10+2

Getting Information

With a few revisions present we can get some information about the state of things.

First we can view the current revision::

text
$ alembic current
INFO  [alembic.context] Context class PostgresqlContext.
INFO  [alembic.context] Will assume transactional DDL.
Current revision for postgresql://scott:XXXXX@localhost/test: 1975ea83b712 -> ae1027a6acf (head), Add a column

head is displayed only if the revision identifier for this database matches the head revision.

We can also view history with alembic history; the --verbose option
(accepted by several commands, including history, current, heads
and branches) will show us full information about each revision::

text
$ alembic history --verbose

Rev: ae1027a6acf (head)
Parent: 1975ea83b712
Path: /path/to/yourproject/alembic/versions/ae1027a6acf_add_a_column.py

    add a column

    Revision ID: ae1027a6acf
    Revises: 1975ea83b712
    Create Date: 2014-11-20 13:02:54.849677

Rev: 1975ea83b712
Parent: 
Path: /path/to/yourproject/alembic/versions/1975ea83b712_add_account_table.py

    create account table

    Revision ID: 1975ea83b712
    Revises:
    Create Date: 2014-11-20 13:02:46.257104

Viewing History Ranges

Using the -r option to alembic history, we can also view various slices
of history. The -r argument accepts an argument [start]:[end], where
either may be a revision number, symbols like head, heads or
base, current to specify the current revision(s), as well as negative
relative ranges for [start] and positive relative ranges for [end]::

$ alembic history -r1975ea:ae1027

A relative range starting from three revs ago up to current migration,
which will invoke the migration environment against the database
to get the current migration::

$ alembic history -r-3:current

.. note::

As illustrated above, to use ranges that start with a negative number (i.e.
a dash), due to a
bug in argparse <https://github.com/python/cpython/issues/53580>_ , either
the syntax -r-:<head>, without any space, must be used as above::

text
$ alembic history -r-3:current

or if using --rev-range, an equals sign must be used::

text
$ alembic history --rev-range=-3:current

Using quotes or escape symbols will not work if there's a space after
the argument name.

View all revisions from 1975 to the head::

$ alembic history -r1975ea:

Downgrading

We can illustrate a downgrade back to nothing, by calling alembic downgrade back
to the beginning, which in Alembic is called base::

text
$ alembic downgrade base
INFO  [alembic.context] Context class PostgresqlContext.
INFO  [alembic.context] Will assume transactional DDL.
INFO  [alembic.context] Running downgrade ae1027a6acf -> 1975ea83b712
INFO  [alembic.context] Running downgrade 1975ea83b712 -> None

Back to nothing - and up again::

text
$ alembic upgrade head
INFO  [alembic.context] Context class PostgresqlContext.
INFO  [alembic.context] Will assume transactional DDL.
INFO  [alembic.context] Running upgrade None -> 1975ea83b712
INFO  [alembic.context] Running upgrade 1975ea83b712 -> ae1027a6acf

Next Steps

The vast majority of Alembic environments make heavy use of the
"autogenerate" feature. Continue onto the next section, :doc:autogenerate.


Build Api Autogenerate (`docs/build/api/autogenerate.rst`)

.. _alembic.autogenerate.toplevel:

============== Autogeneration

.. note:: this section discusses the internal API of Alembic
as regards the autogeneration feature of the alembic revision
command.
This section is only useful for developers who wish to extend the
capabilities of Alembic. For general documentation on the autogenerate
feature, please see :doc:/autogenerate.

The autogeneration system has a wide degree of public API, including
the following areas:

  1. The ability to do a "diff" of a :class:~sqlalchemy.schema.MetaData object against
    a database, and receive a data structure back. This structure
    is available either as a rudimentary list of changes, or as
    a :class:.MigrateOperation structure.

  2. The ability to alter how the alembic revision command generates
    revision scripts, including support for multiple revision scripts
    generated in one pass.

  3. The ability to add new operation directives to autogeneration, including
    custom schema/model comparison functions and revision script rendering.

Getting Diffs

The simplest API autogenerate provides is the "schema comparison" API;
these are simple functions that will run all registered "comparison" functions
between a :class:~sqlalchemy.schema.MetaData object and a database
backend to produce a structure showing how they differ. The two
functions provided are :func:.compare_metadata, which is more of the
"legacy" function that produces diff tuples, and :func:.produce_migrations,
which produces a structure consisting of operation directives detailed in
:ref:alembic.operations.toplevel.

.. autofunction:: alembic.autogenerate.compare_metadata

.. autofunction:: alembic.autogenerate.produce_migrations

.. _customizing_revision:

Customizing Revision Generation

The alembic revision command, also available programmatically
via :func:.command.revision, essentially produces a single migration
script after being run. Whether or not the --autogenerate option
was specified basically determines if this script is a blank revision
script with empty upgrade() and downgrade() functions, or was
produced with alembic operation directives as the result of autogenerate.

In either case, the system creates a full plan of what is to be done
in the form of a :class:.MigrateOperation structure, which is then
used to produce the script.

For example, suppose we ran alembic revision --autogenerate, and the
end result was that it produced a new revision 'eced083f5df'
with the following contents::

text
"""create the organization table."""

# revision identifiers, used by Alembic.
revision = 'eced083f5df'
down_revision = 'beafc7d709f'

from alembic import op
import sqlalchemy as sa


def upgrade():
    op.create_table(
        'organization',
        sa.Column('id', sa.Integer(), primary_key=True),
        sa.Column('name', sa.String(50), nullable=False)
    )
    op.add_column(
        'user',
        sa.Column('organization_id', sa.Integer())
    )
    op.create_foreign_key(
        'org_fk', 'user', 'organization', ['organization_id'], ['id']
    )

def downgrade():
    op.drop_constraint('org_fk', 'user')
    op.drop_column('user', 'organization_id')
    op.drop_table('organization')

The above script is generated by a :class:.MigrateOperation structure
that looks like this::

text
from alembic.operations import ops
import sqlalchemy as sa

migration_script = ops.MigrationScript(
    'eced083f5df',
    ops.UpgradeOps(
        ops=[
            ops.CreateTableOp(
                'organization',
                [
                    sa.Column('id', sa.Integer(), primary_key=True),
                    sa.Column('name', sa.String(50), nullable=False)
                ]
            ),
            ops.ModifyTableOps(
                'user',
                ops=[
                    ops.AddColumnOp(
                        'user',
                        sa.Column('organization_id', sa.Integer())
                    ),
                    ops.CreateForeignKeyOp(
                        'org_fk', 'user', 'organization',
                        ['organization_id'], ['id']
                    )
                ]
            )
        ]
    ),
    ops.DowngradeOps(
        ops=[