apply_runtime_prefixes records the prefixes on the pipeline; any subsequent
compile_to_* for a non-JIT target then applies them. Calling it with a JIT
target raises a Halide::CompileError.
Inside a Generator, the prefixes travel on the GeneratorContext as
RuntimePrefixParams and are applied automatically when the generator's module
is built; in practice these are supplied through the command line or CMake,
described below.
Usage from the GenGen command line  Copy Chapter
The prefixes are ordinary generator parameters named runtime_prefixes.import,
runtime_prefixes.export, and runtime_prefixes.internal. Any of them may be
omitted.
To emit a prefixed standalone runtime (the -r output):

To emit a matching pipeline with no_runtime:

Usage from CMake  Copy Chapter
add_halide_runtime accepts a PARAMS argument that is forwarded to the
runtime generator, and add_halide_library already forwards PARAMS to the
pipeline generator. Give a runtime its export/internal prefixes, and give each
library its matching import/internal prefixes together with USE_RUNTIME:

Repeating this with a second, differently-prefixed runtime and library produces
two components that can be linked into the same program without their runtimes
colliding. For a complete, working example -- three variants of one pipeline,
each with its own runtime, linked into a single test that checks their state
stays independent -- see test/generator/runtime_prefixes_iso_aottest.cpp and
its CMake wiring in test/generator/CMakeLists.txt.
Verifying the result  Copy Chapter
The renaming happens on the symbols of the emitted object, so you can confirm it
with nm. A stock runtime exports halide_malloc:

and its internal state globals are prefixed as well:

A no_runtime pipeline object correspondingly imports the renamed symbols
rather than the stock halide_ ones:

---
Doc/FuzzTesting  Copy Section
Fuzz testing
Halide has a set of fuzz-testing harnesses in test/fuzz/ that can find tricky
edge cases and bugs that are hard to catch with a regular unit-testing suite.
The fuzz tests are built on a small in-tree framework (fuzz_helpers.h,
halide_fuzz_main.h) that abstracts over two backends:
- stdlib backend — uses std::mt19937_64 seeded from std::random_device.
  Works with any standard C++ toolchain; no special compiler flags or external
  runtime libraries required. Each run prints its seed so failures are
  reproducible by re-running with that seed.
- libfuzzer backend — uses
  libFuzzer for coverage-guided
  fuzzing. Requires a Clang toolchain built with -fsanitize=fuzzer support.
The stdlib backend is the default for regular development builds. The libfuzzer
backend is enabled automatically when the build system detects
-fsanitize=fuzzer (or equivalent) linker flags.
Building fuzz tests  Copy Chapter
Standard build (stdlib backend)  Copy Section
No special flags are needed. The fuzz tests build as part of any normal CMake
configuration that has WITH_TEST_FUZZ=YES:

libfuzzer backend (coverage-guided fuzzing)  Copy Section
Use one of the fuzzing CMake presets, which set the necessary
-fsanitize=fuzzer[-no-link] flags across the entire build:
Linux:

macOS (Homebrew LLVM):

The LLVM install used for libfuzzer builds must include the compiler-rt
runtime (i.e. built with -DLLVM_ENABLE_RUNTIMES="compiler-rt"). Not all
prebuilt LLVM installs include this; you may need to build LLVM from source or
use Homebrew's LLVM package on macOS.
Running fuzz tests  Copy Chapter
stdlib backend  Copy Section
Run a fuzz harness directly:

By default this runs 10,000 iterations, printing the seed before each one:

Control the number of iterations with -runs=N:

Run all fuzz tests via CTest (1,000 iterations each, exit-code–based pass/fail):

libfuzzer backend  Copy Section
After building with a fuzzing preset, run the harness with no arguments to start
coverage-guided fuzzing on a single core:

To persist the corpus between runs (recommended):

To fuzz in parallel across all available cores:

Reproducing failures  Copy Chapter
stdlib backend  Copy Section
When a run fails, rerun with the seed that was printed just before the crash:

This performs a single deterministic iteration with that seed.
libfuzzer backend  Copy Section
libFuzzer writes a crash-input file on failure:

Replay it by passing it as the first argument:

Adding new fuzz tests  Copy Chapter
All fuzz tests use the FUZZ_TEST macro defined in fuzz_helpers.h. This macro
generates the correct entry point for whichever backend is active —
LLVMFuzzerTestOneInput for libfuzzer or a main that calls
Halide::fuzz_main for the stdlib backend.
A minimal fuzz test looks like this:

FuzzingContext wraps FuzzedDataProvider (from libfuzzer's
compiler-rt/include/fuzzer/FuzzedDataProvider.h) and re-implements its
interface on top of std::mt19937_64 for the stdlib backend, so the same API
works with both backends. Key methods:
- ConsumeIntegral<T>() — random value of type T
- ConsumeIntegralInRange<T>(min, max) — random value in [min, max]
- ConsumeBool() — random boolean
- PickValueInArray(arr) — random element from an array or initializer list
- PickValueInVector(vec) — random element from a std::vector
For richer examples, see test/fuzz/simplify.cpp and
test/fuzz/random_expr_generator.h.
To register a new fuzz test with CMake, add it to the SOURCES list in
test/fuzz/CMakeLists.txt.
Other useful materials  Copy Chapter
- The official libfuzzer docs
- The libfuzzer tutorial
- FuzzedDataProvider reference
---
Doc/GeneratorCache  Copy Section
Generator compile cache
Overview  Copy Chapter
Building a large app suite runs one generator process per (generator, target)
pair, and most of that time is spent recompiling pipelines that haven't actually
changed since the last build. The generator compile cache is an opt-in,
content-addressed cache that lets execute_generator skip that work: when the
inputs to a generator invocation are unchanged, the previously emitted artifacts
(object files, headers, static libraries, etc.) are copied back into place
instead of being recompiled.
The cache is inert unless you opt in by setting the HL_CACHE_DIR environment
variable (or the corresponding Halide_CACHE_DIR CMake variable, see below) to
a directory. When it is unset, none of this machinery runs and build output is
identical to a Halide with no cache support at all.
The feature also requires that Halide was built with WITH_SERIALIZATION=ON
(the default), since the cache key is only sound if it includes a serialized
snapshot of the pipeline being compiled. If HL_CACHE_DIR is set but
serialization support isn't compiled in, Halide prints a one-time warning and
disables caching for that build.
What gets cached  Copy Chapter
Both compilation paths that execute_generator can take are cached
independently:
- Compiling a generator (the compile_multitarget path), once per
  -g/-o/target=... invocation.
- Compiling the standalone runtime (the -r/GenRT path).
Cache key  Copy Chapter
A cache entry is addressed by a SHA-256 digest that mixes in everything that can
affect the emitted files, so that two invocations agreeing on all of the
following are guaranteed to produce identical outputs:
- Compiler identity: a fingerprint of the running libHalide (or, if
  statically linked, the generator executable itself). This is normally the
  linker-assigned build ID, read directly from the already-loaded image (Mach-O
  LC_UUID on macOS, ELF .note.gnu.build-id on Linux, the PE debug
  directory's CodeView/PDB GUID on Windows) so that rebuilding Halide
  invalidates the cache without hashing the ~32 MB binary on every run. If no
  build ID is available (e.g. --build-id=none, or a Windows image built
  without debug info), it falls back to hashing the whole binary.
- The generator name, output types, build mode (normal vs. gradient), and
  the fully-resolved generator-param settings (target= is handled
  separately, below).
- The target(s) and per-target suffixes being compiled.
- The serialized pipeline for each target: the generator is instantiated and
  its (pre-autoschedule) algorithm and schedule are serialized via
  Serialization.h and folded into the key. This is what makes the key sound
  for source edits — the actual algorithm and schedule are captured, so editing
  a generator's generate()/schedule() invalidates the cache even when none
  of its command-line params changed. Autoscheduling itself is deliberately
  excluded so that computing a cache key never requires running a (potentially
  expensive) autoscheduler.
- The contents of any -p plugins (e.g. autoschedulers), since they affect
  codegen but live outside libHalide.
If the compiler identity can't be determined, or the pipeline can't be
serialized (for example, a generator that throws while building), the invocation
falls back to compiling normally without touching the cache.
Using it from the command line  Copy Chapter
Set HL_CACHE_DIR before invoking a generator executable directly:

The first invocation compiles and populates the cache; subsequent invocations
with the same generator, target, params, plugins, and (unchanged) pipeline
source restore the outputs instead of recompiling.
Using it from CMake  Copy Chapter
The shipped CMake helpers (add_halide_library, add_halide_runtime, etc.)
read the Halide_CACHE_DIR cache variable — which defaults to
$ENV{HL_CACHE_DIR} — and, when it's set, wrap every generator/GenRT invocation
so the value reaches the generator process:

When Halide_CACHE_DIR is empty, the emitted build commands are byte-for-byte
identical to a build with no cache configured. See
Halide_CACHE_DIR in the CMake package docs.
Cache maintenance  Copy Chapter
Entries are installed atomically (staged in a temp directory under the cache
root, then renamed into place), so concurrent builds never observe a partial
entry.
The cache is pruned opportunistically after each store to honor two optional
size/age limits, read from the environment:
- HL_CACHE_MAX_SIZE — maximum total size of the cache, evicting the
  least-recently-used entries first once exceeded. Accepts a plain byte count or
  a K/M/G suffix (powers of 1024). Defaults to 1G.
- HL_CACHE_MAX_AGE — if set, entries not used within this many seconds are
  evicted regardless of total size.
Because a large parallel build runs many generator processes that each store an
entry, pruning is debounced to at most once every 60 seconds (via a
.last_prune stamp in the cache directory), so the size limit is a soft cap
that a burst of concurrent stores can briefly exceed. Pruning across processes
is additionally serialized by a non-blocking advisory lock; if another process
already holds it, a store simply skips its own prune pass rather than waiting.
To clear the cache entirely, just delete the directory named by HL_CACHE_DIR.
---
Doc/HalideCMakePackage  Copy Section
Using Halide from your CMake build
This is a detailed guide to building your own Halide programs with the official
CMake package. If you need directions for building Halide, see
[BuildingHalideWithCMake.md]. If you are looking for Halide's CMake coding
guidelines, see [CodeStyleCMake.md].
This document assumes some basic familiarity with CMake but tries to be explicit
in all its examples. To learn more about CMake, consult the
[documentation][cmake-docs] and engage with the community on the
[CMake Discourse][cmake-discourse].
- Using Halide from your CMake build
- A basic CMake project
  - JIT mode
  - AOT mode
    - Autoschedulers
    - RunGenMain
- Halide package documentation
  - Components
  - Variables
  - Imported targets
  - Functions
    - add_halide_generator
    - add_halide_library
    - add_halide_python_extension_library
    - add_halide_runtime
- Cross compiling
  - Use add_halide_generator
  - Use a super-build
  - Use ExternalProject directly
  - Use an emulator or run on device
  - Bypass CMake
A basic CMake project
There are two main ways to use Halide in your application: as a JIT compiler
for dynamic pipelines or an ahead-of-time (AOT) compiler for static
pipelines. CMake provides robust support for both use cases.
No matter how you intend to use Halide, you will need some basic CMake
boilerplate.

The [cmake_minimum_required][cmake_minimum_required] command is required to be
the first command executed in a CMake program. It disables all the deprecated
behavior ("policies" in CMake lingo) from earlier versions. The
[project][project] command sets the name of the project (and accepts arguments
for versioning, language support, etc.) and is required by CMake to be called
immediately after setting the minimum version.
The next three variables set the project-wide C++ standard. The first,
[CMAKE_CXX_STANDARD][cmake_cxx_standard], simply sets the standard version.
Halide requires at least C++17. The second,
[CMAKE_CXX_STANDARD_REQUIRED][cmake_cxx_standard_required], tells CMake to
fail if the compiler cannot provide the requested standard version. Lastly,
[CMAKE_CXX_EXTENSIONS][cmake_cxx_extensions] tells CMake to disable
vendor-specific extensions to C++. This is not necessary to simply use Halide,
but we do not allow such extensions in the Halide repo.
Finally, we use [find_package][find_package] to locate Halide on your system.
When using the pip package on Linux and macOS, CMake's find_package command
should find Halide as long as you're in the same virtual environment you
installed it in. On Windows, you will need to add the virtual environment root
directory to [CMAKE_PREFIX_PATH][cmake_prefix_path]:

If find_package cannot find Halide, set CMAKE_PREFIX_PATH to the Halide
installation directory.
JIT mode  Copy Chapter
To use Halide in JIT mode (like the [tutorials][halide-tutorials] do, for
example), you can simply link to Halide::Halide.

Then Halide.h will be available to your code and everything should just work.
That's it!
AOT mode  Copy Chapter
Using Halide in AOT mode is more complicated so we'll walk through it step by
step. Note that this only applies to Halide generators, so it might be useful to
re-read the [tutorial on generators][halide-generator-tutorial]. Assume (like in
the tutorial) that you have a source file named my_generators.cpp and that in
it, you have generator classes MyFirstGenerator and MySecondGenerator with
registered names my_first_generator and my_second_generator respectively.
Then the first step is to add a generator executable to your build:

Using the generator executable, we can add a Halide library corresponding to
MyFirstGenerator.

This will create a static library target in CMake that corresponds to the output
of running your generator. The second generator in the file requires generator
parameters to be passed to it. These are also easy to handle:

Adding multiple configurations is easy, too:

Here, we had to specify which generator to use (my_second_generator) since it
uses the target name by default. The functions in these libraries will be named
after the target names, my_second_generator_2 and my_second_generator_3, by
default, but it is possible to control this via the FUNCTION_NAME parameter.
Each one of these targets, <GEN>, carries an associated <GEN>.runtime
target, which is also a static library containing the Halide runtime. It is
transitively linked through <GEN> to targets that link to <GEN>. On an
operating system like Linux, where weak linking is available, this is not an
issue. However, on Windows, this can fail due to symbol redefinitions. In these
cases, you must declare that two Halide libraries share a runtime, like so:

This will even work correctly when different combinations of targets are
specified for each halide library. A "greatest common denominator" target will
be chosen that is compatible with all of them (or the build will fail).
Autoschedulers  Copy Section
When the autoschedulers are included in the release package, they are very
simple to apply to your own generators. For example, we could update the
definition of the my_first_generator library above to use the Adams2019
autoscheduler:

RunGenMain  Copy Section
Halide provides a generic driver for generators to be used during development
for benchmarking and debugging. Suppose you have a generator executable called
my_gen and a generator within called my_filter. Then you can pass a variable
name to the REGISTRATION parameter of add_halide_library which will contain
the name of a generated C++ source that should be linked to Halide::RunGenMain
and my_filter.
For example:

Then you can run, debug, and benchmark your generator through the runner
executable. Learn how to interact with these executables in
RunGen.md.
Halide package documentation
Halide provides a CMake _package configuration_ module. The intended way to use
the CMake build is to run find_package(Halide ...) in your CMakeLists.txt
file. Closely read the [find_package documentation][find_package] before
proceeding.
Components  Copy Chapter
The Halide package script understands a handful of optional components when
loading the package.
First, if you plan to use the Halide Image IO library, you will want to include
the png and jpeg components when loading Halide.
Second, Halide releases can contain a variety of configurations: static, shared,
debug, release, etc. CMake handles Debug/Release configurations automatically,
but generally only allows one type of library to be loaded.
The package understands two components, static and shared, that specify
which type of library you would like to load. For example, if you want to make
sure that you link against shared Halide, you can write:

If the shared libraries are not available, this will result in a failure.
If no component is specified, then the Halide_SHARED_LIBS variable is checked.
If it is defined and set to true, then the shared libraries will be loaded or
the package loading will fail. Similarly, if it is defined and set to false, the
static libraries will be loaded.
If no component is specified and Halide_SHARED_LIBS is _not_ defined, then the
[BUILD_SHARED_LIBS][build_shared_libs] variable will be inspected. If it is
not defined or defined and set to true, then it will attempt to load the
shared libs and fall back to the static libs if they are not available.
Similarly, if BUILD_SHARED_LIBS is defined and set to false, then it will
try the static libs first then fall back to the shared libs.
To ensure that the Python bindings are available, include the Python
component.
Finally, when cross compiling,
find_package(Halide REQUIRED) never pulls in the compiled compiler/JIT library
(Halide::Halide, Halide::Generator, etc.) unless you explicitly ask for it.
Add the JIT component (or Python) to force it to be loaded even while
cross-compiling:

This looks for a HalideCompiler package matching your current (target)
platform and fails with a normal find_package error if none is found.
HalideCompiler is the name of the underlying platform-specific package that
actually contains the compiled libraries; you generally don't need to
find_package(HalideCompiler) directly, but its name is useful for
HalideCompiler_ROOT/-DHalideCompiler_DIR=... when pointing CMake at a
specific installed build.
Autoscheduler plugins (see Autoschedulers) are resolved
separately, via a HalideAutoschedulers package, and are always available to
add_halide_library(... AUTOSCHEDULER ...) regardless of cross-compiling --
unlike HalideCompiler, this package's targets are never linked against (only
dlopen()'d, by their build path, at Generator run time), so there's no reason to
gate them behind the JIT/Python components.
Note that static/shared, unlike JIT/Python, never force this load by
themselves -- requesting one merely records your preference for whichever
package eventually loads the compiled compiler (whether that's this same
find_package(Halide ...) call, because you're not cross-compiling or also
requested JIT/Python, or a later, unrelated one, such as the internal lookup
add_halide_generator performs when it needs to build a generator). This
preference is scoped to the current directory (and any subdirectories added
after it), so independent parts of a project -- so long as neither is a
subdirectory of the other -- can request different linkage without conflicting
with each other. If one directory's find_package call ends up loading the
compiled compiler before a subdirectory requests the other flavor, that's a real
conflict (CMake can only load one flavor of Halide::Halide per directory
scope) and fails cleanly with a descriptive error rather than silently keeping
whichever flavor loaded first.
Variables  Copy Chapter
Variables that control package loading:
| Variable                    | Description                                                                                                                                                                   |
| --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Halide_SHARED_LIBS        | override BUILD_SHARED_LIBS when loading the Halide package via find_package. Has no effect when using Halide via add_subdirectory as a Git or FetchContent submodule. |
| Halide_RUNTIME_NO_THREADS | skip linking of Threads library to runtime. Should be set if your toolchain does not support it (e.g. baremetal).                                                             |
| Halide_RUNTIME_NO_DL_LIBS | skip linking of DL library to runtime. Should be set if your toolchain does not support it (e.g. baremetal).                                                                  |
Variables set by the package:
| Variable                   | Description                                                        |
| -------------------------- | ------------------------------------------------------------------ |
| Halide_VERSION           | The full version string of the loaded Halide package               |
| Halide_VERSION_MAJOR     | The major version of the loaded Halide package                     |
| Halide_VERSION_MINOR     | The minor version of the loaded Halide package                     |
| Halide_VERSION_PATCH     | The patch version of the loaded Halide package                     |
| Halide_VERSION_TWEAK     | The tweak version of the loaded Halide package                     |
| Halide_HOST_TARGET       | The Halide target triple corresponding to "host" for this build.   |
| Halide_CMAKE_TARGET      | The Halide target triple corresponding to the active CMake target. |
| Halide_ENABLE_EXCEPTIONS | Whether Halide was compiled with exception support                 |
| Halide_ENABLE_RTTI       | Whether Halide was compiled with RTTI                              |
| WITH_AUTOSCHEDULERS      | Whether the autoschedulers are available                           |
Variables that control package behavior:
| Variable                  | Description                                                                                                                                                                                                    |
| ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Halide_PYTHON_LAUNCHER  | Semicolon separated list containing a command to launch the Python interpreter. Can be used to set environment variables for Python generators.                                                                |
| Halide_NO_DEFAULT_FLAGS | Off by default. When enabled, suppresses recommended compiler flags that would be added by add_halide_generator                                                                                              |
| Halide_CACHE_DIR        | Defaults to $ENV{HL_CACHE_DIR}. When set, routes HL_CACHE_DIR to every generator/GenRT invocation so it can reuse cached artifacts instead of recompiling. See doc/GeneratorCache.md. |
Imported targets  Copy Chapter
Halide defines the following targets that are available to users:
| Imported target      | Description                                                                                                                                                                                    |
| -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Halide::Halide     | this is the JIT-mode library to use when using Halide from C++.                                                                                                                                |
| Halide::Generator  | this is the target to use when manually defining a generator executable. It supplies a main() function.                                                                                      |
| Halide::Runtime    | adds include paths to the Halide runtime headers                                                                                                                                               |
| Halide::Tools      | adds include paths to the Halide tools, including the benchmarking utility.                                                                                                                    |
| Halide::ImageIO    | adds include paths to the Halide image IO utility. Depends on PNG::PNG and JPEG::JPEG if they exist or were loaded through the corresponding package components.                           |
| Halide::ThreadPool | adds include paths to the Halide _simple_ thread pool utility library. This is not the same as the runtime's thread pool and is intended only for use by tests. Depends on Threads::Threads. |
| Halide::RunGenMain | used with the REGISTRATION parameter of add_halide_library to create simple runners and benchmarking tools for Halide libraries.                                                           |
The following targets only guaranteed when requesting the Python component
(Halide_Python_FOUND will be true):
| Imported target  | Description                                                                                                                                                       |
| ---------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Halide::Python | this is a Python 3 package that can be referenced as $<TARGET_FILE_DIR:Halide::Python>/.. when setting up PYTHONPATH for Python tests or the like from CMake. |
The following targets only guaranteed when WITH_AUTOSCHEDULERS is true:
| Imported target         | Description                                                     |
| ----------------------- | --------------------------------------------------------------- |
| Halide::Adams2019     | the Adams et.al. 2019 autoscheduler (no GPU support)            |
| Halide::Anderson2021  | the Anderson, et.al. 2021 autoscheduler (full GPU support)      |
| Halide::Li2018        | the Li et.al. 2018 gradient autoscheduler (limited GPU support) |
| Halide::Mullapudi2016 | the Mullapudi et.al. 2016 autoscheduler (no GPU support)        |
These come from a separate HalideAutoschedulers package (distinct from
HalideCompiler), loaded automatically the first time
add_halide_library(... AUTOSCHEDULER ...) needs one -- including while
cross-compiling.
Functions  Copy Chapter
The Halide package provides several useful functions for dealing with AOT
compilation steps.
add_halide_generator ')" title="Copy section prompt for LLMs"> Copy Section
This function aids in creating cross-compilable builds that use Halide
generators.

Every named argument is optional, and the function uses the following default
arguments:
- If PACKAGE_NAME is not provided, it defaults to
  ${PROJECT_NAME}-halide_generators.
- If PACKAGE_NAMESPACE is not provided, it defaults to
  ${PROJECT_NAME}::halide_generators::.
- If EXPORT_FILE is not provided, it defaults to
  ${PROJECT_BINARY_DIR}/cmake/${ARG_PACKAGE_NAME}-config.cmake
This function guarantees that a Halide generator target named
<namespace><target> is available. It will first search for a package named
<package-name> using find_package; if it is found, it is assumed that it
provides the target. Otherwise, it will create an executable target named
target and an ALIAS target <namespace><target>. This function also creates
a custom target named <package-name> if it does not exist and <target> would
exist. In this case, <package-name> will depend on <target>, this enables
easy building of _just_ the Halide generators managed by this function.
After the call, <PACKAGE_NAME>_FOUND will be set to true if the host
generators were imported (and hence won't be built). Otherwise, it will be set
to false. This variable may be used to conditionally set properties on
<target>.
Please see
test/integration/xc
for a simple example and
apps/hannk for a
complete app that uses it extensively.
The SOURCES keyword marks the beginning of sources to be used to build
<target>, if it is not loaded. All unparsed arguments will be interpreted as
sources.
The LINK_LIBRARIES argument lists libraries that should be linked to
<target> when it is being built in the present build system.
If PYSTUB is specified, then a Python Extension will be built that wraps the
Generator with CPython glue to allow use of the Generator Python 3. The result
will be a shared library of the form <target>_pystub.<soabi>.so, where
<soabi> describes the specific Python version and platform (e.g.,
cpython-310-darwin for Python 3.10 on macOS). See Python.md for
examples of use.
add_halide_library ')" title="Copy section prompt for LLMs"> Copy Section
This is the main function for managing generators in AOT compilation. The full
signature follows:

This function creates a called <target> corresponding to running the
<generator-target> (an executable target which links to Halide::Generator)
one time, using command line arguments derived from the other parameters.
The arguments GENERATOR and FUNCTION_NAME default to <target>. They
correspond to the -g and -f command line flags, respectively.
NAMESPACE is syntactic sugar to specify the C++ namespace (if any) of the
generated function; you can also specify the C++ namespace (if any) directly in
the FUNCTION_NAME argument, but for repeated declarations or very long
namespaces, specifying this separately can provide more readable build files.
If USE_RUNTIME is not specified, this function will create another target
called <target>.runtime which corresponds to running the generator with -r
and a compatible list of targets. This runtime target is an INTERFACE
dependency of <target>. If multiple runtime targets need to be linked
together, setting USE_RUNTIME to another Halide runtime library, <target2>
will prevent the generation of <target>.runtime and instead use
<target2>.runtime. This argument is most commonly used in conjunction with
 add_halide_runtime.
Parameters can be passed to a generator via the PARAMS argument. Parameters
should be space-separated. Similarly, TARGETS is a space-separated list of
targets for which to generate code in a single function. They must all share the
same platform/bits/os triple (e.g. arm-32-linux). Features that are in common
among all targets, including device libraries (like cuda) should go in
FEATURES. If TARGETS is not specified, the value of Halide_TARGET
specified at configure time will be used.
Every element of TARGETS must begin with the same arch-bits-os triple. This
function understands two _meta-triples_, host and cmake. The meta-triple
host is equal to the arch-bits-os triple used to compile Halide along with
all the supported instruction set extensions. On platforms that support running
both 32 and 64-bit programs, this will not necessarily equal the platform the
compiler is running on or that CMake is targeting.
The meta-triple cmake is equal to the arch-bits-os of the current CMake
target. This is useful if you want to make sure you are not unintentionally
cross-compiling, which would result in an [IMPORTED target][imported-target]
being created. When TARGETS is empty and the host target would not
cross-compile, then host will be used. Otherwise, cmake will be used and an
author warning will be issued.
When CMAKE_OSX_ARCHITECTURES is set and the TARGETS argument resolves to
cmake, the generator will be run once for each architecture and the results
will be fused together using lipo. This behavior extends to runtime targets.
Sometimes, the generation will need to read files that were generated during the
build. To declare dependencies on these files, use the DEPENDS argument. Paths
listed here will be passed verbatim to add_custom_command, and so will be
relative to the source directory. Use absolute paths when referring to files
outside the source directory.
To use an autoscheduler, set the AUTOSCHEDULER argument to a target named like
Namespace::Scheduler, for example Halide::Adams2019. This will set the
autoscheduler GeneratorParam on the generator command line to Scheduler and
add the target to the list of plugins. Additional plugins can be loaded by
setting the PLUGINS argument. If the argument to AUTOSCHEDULER does not
contain :: or it does not name a target, it will be passed to the -s flag
verbatim.
If GRADIENT_DESCENT is set, then the module will be built suitably for
gradient descent calculation in TensorFlow or PyTorch. See
Generator::build_gradient_module() for more documentation. This corresponds to
passing -d 1 at the generator command line.
If the C_BACKEND option is set, this command will invoke the configured C++
compiler on a generated source. Note that a <target>.runtime target is _not_
created in this case, and the USE_RUNTIME option is ignored. Other options
work as expected.
If REGISTRATION is set, the path (relative to CMAKE_CURRENT_BINARY_DIR) to
the generated .registration.cpp file will be set in OUTVAR. This can be used
to generate a runner for a Halide library that is useful for benchmarking and
testing, as documented above. This is equivalent to setting -e registration at
the generator command line.
If HEADER is set, the path (relative to CMAKE_CURRENT_BINARY_DIR) to the
generated .h header file will be set in OUTVAR. This can be used with
install(FILES) to conveniently deploy the generated header along with your
library.
If FUNCTION_INFO_HEADER is set, the path (relative to
CMAKE_CURRENT_BINARY_DIR) to the generated .function_info.h header file will
be set in OUTVAR. This produces a file that contains constexpr descriptions
of information about the generated functions (e.g., argument type and
information). It is generated separately from the normal HEADER file because
HEADER is intended to work with basic extern "C" linkage, while
FUNCTION_INFO_HEADER requires C++17 or later to use effectively. (This can be
quite useful for advanced usages, such as producing automatic call wrappers,
etc.) Examples of usage can be found in the generated file.
Each of the extra-output arguments directly correspond to an extra output (via
-e) from the generator. The value OUTVAR names a variable into which a path
(relative to [CMAKE_CURRENT_BINARY_DIR][cmake_current_binary_dir]) to the
extra file will be written.
When NO_THREADS is passed, the library targets will not depend on
Threads::Threads. It is your responsibility to link to an equivalent target.
When NO_DL_LIBS is passed, the library targets will not depend on
${CMAKE_DL_LIBS}. It is your responsibility to link to an equivalent library.
add_halide_python_extension_library ')" title="Copy section prompt for LLMs"> Copy Section
This function wraps the outputs of one or more add_halide_library targets with
glue code to produce a Python Extension library.

HALIDE_LIBRARIES is a list of one of more add_halide_library targets. Each
will be added to the extension as a callable method of the module. Note that
every library specified must be built with the PYTHON_EXTENSION keyword
specified, and all libraries must use the same Halide runtime.
The result will be a shared library of the form <target>.<soabi>.so, where
<soabi> describes the specific Python version and platform (e.g.,
cpython-310-darwin for Python 3.10 on macOS.)
add_halide_runtime ')" title="Copy section prompt for LLMs"> Copy Section
This function generates a library containing a Halide runtime. Most user code
will never need to use this, as add_halide_library() will call it for you if
necessary. The most common use case is usually in conjunction with
add_halide_python_extension_library(), as a way to ensure that all the halide
libraries share an identical runtime.

The TARGETS, NO_THREADS, and NO_DL_LIBS arguments have identical semantics
to the argument of the same name for
 add_halide_library.
Cross compiling
Cross-compiling in CMake can be tricky, since CMake doesn't easily support
compiling for both the host platform and the cross platform within the same
build. Unfortunately, Halide generator executables are just about always
designed to run on the host platform. Each project will be set up differently
and have different requirements, but here are some suggestions for effective use
of CMake in these scenarios.
Use add_halide_generator ')" title="Copy chapter prompt for LLMs"> Copy Chapter
If you are writing new programs that use Halide, you might wish to use
add_halide_generator. When using this helper, you are expected to build your
project twice: once for your build host and again for your intended target.
On the target-side build, a plain find_package(Halide REQUIRED) is all you
need (no separate package name): it never pulls in the compiled compiler, and
add_halide_generator will lazily load it under the hood only if it can't find
a prebuilt host generators package to import instead.
When building the host build, you can use the <package-name> (see the
documentation above) target to build _just_ the generators. Then, in the target
build, set <package-name>_ROOT to the host build directory.
For example:

Use a super-build  Copy Chapter
A CMake super-build consists of breaking down a project into subprojects that
are isolated by [toolchain][cmake-toolchains]. The basic structure is to have an
outermost project that only coordinates the sub-builds via the
[ExternalProject][externalproject] module.
One would then use Halide to build a generator executable in one self-contained
project, then export that target to be used in a separate project. The second
project would be configured with the target [toolchain][cmake-toolchains] and
would call add_halide_library with no TARGETS option and set FROM equal to
the name of the imported generator executable. Obviously, this is a significant
increase in complexity over a typical CMake project.
This is very compatible with the add_halide_generator strategy above.
Use ExternalProject directly  directly')" title="Copy chapter prompt for LLMs"> Copy Chapter
A lighter weight alternative to the above is to use
[ExternalProject][externalproject] directly in your parent build. Configure
the parent build with the target [toolchain][cmake-toolchains], and configure
the inner project to use the host toolchain. Then, manually create an
[IMPORTED target][imported-executable] for your generator executable and call
add_halide_library as described above.
The main drawback of this approach is that creating accurate IMPORTED targets
is difficult since predicting the names and locations of your binaries across
all possible platform and CMake project generators is difficult. In particular,
it is hard to predict executable extensions in cross-OS builds.
Use an emulator or run on device  Copy Chapter
The [CMAKE_CROSSCOMPILING_EMULATOR][cmake_crosscompiling_emulator] variable
allows one to specify a command _prefix_ to run a target-system binary on the
host machine. One could set this to a custom shell script that uploads the
generator executable, runs it on the device and copies back the results.
Another option is to install qemu-user-static to transparently emulate the
cross-built generator.
Bypass CMake  Copy Chapter
The previous two options ensure that the targets generated by
add_halide_library will be _normal_ static libraries. This approach does not
use [ExternalProject][externalproject], but instead produces IMPORTED
targets. The main drawback of IMPORTED targets is that they are considered
second-class in CMake. In particular, they cannot be installed with the typical
[install(TARGETS) command][install-targets]. Instead, they must be installed
using [install(FILES)][install-files] and the
[$<TARGET_FILE:tgt>][target-file] generator expression.
[buildinghalidewithcmake.md]: ./BuildingHalideWithCMake.md
[build_shared_libs]: https://cmake.org/cmake/help/latest/variable/BUILD_SHARED_LIBS.html
[cmake-discourse]: https://discourse.cmake.org/
[cmake-docs]: https://cmake.org/cmake/help/latest/
[cmake-toolchains]: https://cmake.org/cmake/help/latest/manual/cmake-toolchains.7.html
[cmake_crosscompiling_emulator]: https://cmake.org/cmake/help/latest/variable/CMAKE_CROSSCOMPILING_EMULATOR.html
[cmake_current_binary_dir]: https://cmake.org/cmake/help/latest/variable/CMAKE_CURRENT_BINARY_DIR.html
[cmake_cxx_extensions]: https://cmake.org/cmake/help/latest/variable/CMAKE_CXX_EXTENSIONS.html
[cmake_cxx_standard]: https://cmake.org/cmake/help/latest/variable/CMAKE_CXX_STANDARD.html
[cmake_cxx_standard_required]: https://cmake.org/cmake/help/latest/variable/CMAKE_CXX_STANDARD_REQUIRED.html
[cmake_minimum_required]: https://cmake.org/cmake/help/latest/command/cmake_minimum_required.html
[cmake_prefix_path]: https://cmake.org/cmake/help/latest/variable/CMAKE_PREFIX_PATH.html
[codestylecmake.md]: ./CodeStyleCMake.md
[externalproject]: https://cmake.org/cmake/help/latest/module/ExternalProject.html
[find_package]: https://cmake.org/cmake/help/latest/command/find_package.html
[halide-generator-tutorial]: https://halide-lang.org/tutorials/tutorial_lesson_15_generators.html
[halide-tutorials]: https://halide-lang.org/tutorials/tutorial_introduction.html
[imported-executable]: https://cmake.org/cmake/help/latest/command/add_executable.html#imported-executables
[imported-target]: https://cmake.org/cmake/help/latest/manual/cmake-buildsystem.7.html#imported-targets
[install-files]: https://cmake.org/cmake/help/latest/command/install.html#files
[install-targets]: https://cmake.org/cmake/help/latest/command/install.html#targets
[project]: https://cmake.org/cmake/help/latest/command/project.html
[target-file]: https://cmake.org/cmake/help/latest/manual/cmake-generator-expressions.7.html#target-dependent-queries
---
Doc/Hexagon  Copy Section
Halide for Hexagon HVX
Halide supports offloading work to Qualcomm Hexagon DSP on Qualcomm Snapdragon
845/710 devices or newer. The Hexagon DSP provides a set of 128 byte vector
instruction extensions - the Hexagon Vector eXtensions (HVX). HVX is well suited
for image processing, and Halide for Hexagon HVX will generate the appropriate
HVX vector instructions from a program authored in Halide.
Halide can be used to compile Hexagon object files directly, by using a target
such as hexagon-32-qurt-hvx.
Halide can also be used to offload parts of a pipeline to Hexagon using the
hexagon scheduling directive. To enable the hexagon scheduling directive,
include the hvx target feature in your target. The currently supported
combination of targets is to use the HVX target features with an x86 linux host
(to use the simulator) or with an ARM android target (to use Hexagon DSP
hardware). For examples of using the hexagon scheduling directive on both the
simulator and a Hexagon DSP, see the blur example app.
To build and run an example app using the Hexagon target,
1. Obtain and build trunk LLVM and Clang. (Earlier versions of LLVM may work but
   are not actively tested and thus not recommended.)
2. Download and install the Hexagon SDK and Hexagon Tools. Hexagon SDK 4.3.0 or
   later is needed. Hexagon Tools 8.4 or later is needed.
3. Build and run an example for Hexagon HVX
1. Obtain and build trunk LLVM and Clang  Copy Chapter
(Follow the instructions given previously, just be sure to check out the main
branch.)
2. Download and install the Hexagon SDK and Hexagon Tools  Copy Chapter
Go to https://qpm.qualcomm.com/#/main/home
1. Go to Tools, and download Qualcomm Package Manager 3. Install the package
   manager on your machine.
2. Run the installed Qualcomm Package Manager and install the Qualcomm Hexagon
   SDK 5.x (or 4.x). The SDK can be selected from the Qualcomm Hexagon SDK
   Products.
3. Set an environment variable to point to the SDK installation location

3. Build and run an example for Hexagon HVX  Copy Chapter
In addition to running Hexagon code on device, Halide also supports running
Hexagon code on the simulator from the Hexagon tools.
To build and run the blur example in Halide/apps/blur on the simulator:

To build and run the blur example in Halide/apps/blur on Android:  Copy Chapter
To build the example for Android, first ensure that you have Android NDK r19b or
later installed, and the ANDROID_NDK_ROOT environment variable points to it.
(Note that Qualcomm Hexagon SDK v4.3.0 includes Android NDK r19c, which is
fine.)
Now build and run the blur example using the script to run it on device:

---
Doc/Python  Copy Section
Halide Bindings for Python
- Halide Bindings for Python
  - Acquiring the Python bindings
  - Building the Python bindings
    - Using CMake directly
    - Using wheel infrastructure
  - Documentation and Examples
  - Differences from C++ API
  - Example of Simple Usage
  - Halide Generators In Python
    - Writing a Generator in Python
      - hl.generator("name")
      - hl.GeneratorParam
      - hl.InputBuffer, hl.InputScalar
      - hl.OutputBuffer, hl.OutputScalar
      - Names
      - generate() method
      - Types for Inputs and Outputs
    - Using a Generator for JIT compilation
    - Using a Generator for AOT compilation
    - Calling Generator-Produced code from Python
    - Advanced Generator-Related Topics
      - Generator Aliases
      - Dynamic Inputs and Outputs
      - Calling a Generator Directly
      - The Lifecycle Of A Generator
      - Notable Differences Between C++ and Python Generators
  - Keeping Up To Date
  - License
Halide provides Python bindings for most of its public API. Python 3.10 (or
higher) is required. The Python bindings are supported on 64-bit Linux, OSX, and
Windows systems.
In addition to the ability to write just-in-time Halide code using Python, you
can write Generators using the Python bindings,
which can simplify build-system integration (since no C++ metacompilation step
is required).
You can also use existing Halide Generators (written in either C++ or Python) to
produce Python extensions that can be used within Python code.
Acquiring the Python bindings  Copy Chapter
As of Halide 19.0.0, we provide binary wheels on PyPI which include the Python
bindings and the C++/CMake package for native development. Full releases may be
installed with pip like so:

Every commit to main is published to a private PyPI index as a development
version and these may be installed with a few extra flags:

Currently, we provide wheels for: Windows x86-64, macOS x86-64, macOS arm64, and
Linux x86-64. The Linux wheels are built for manylinux_2_28, which makes them
broadly compatible (Debian 10, Ubuntu 18.10, Fedora 29).
Building the Python bindings  Copy Chapter
If pip isn't enough for your purposes, or you are developing Halide directly,
you have two options for building and using the Python bindings. Note that the
bindings require Halide to be built with RTTI and exceptions enabled, which
in turn requires LLVM to be built with RTTI, but this is not the default for
LLVM.
Before configuring with CMake, you should ensure you have prerequisite packages
installed in your local Python environment. The best way to get set up is to use
a virtual environment with uv:

If you don't have LLVM installed already, you can try using the same ones the
buildbots use by adding --group ci-llvm-<VERSION> to the uv sync command,
where <VERSION> is the LLVM major version number (e.g. 23) or main.
If you install ci-llvm-*, you can set
Halide_LLVM_ROOT=$(halide-llvm --prefix) in your environment.
Ensure you have flatbuffers and wabt installed, too. (The wheel build does
not use vcpkg for manylinux compatibility reasons, so these must be available as
system packages or installed from source.)
Using wheel infrastructure  Copy Section
When using uv, this entire workflow can be run via:

Using CMake directly  Copy Section
Assuming dependencies are available, you can build the Python bindings directly
with CMake:

Now you can set the PYTHONPATH variable to point to the freshly built Python
package:

Documentation and Examples  Copy Chapter
As mentioned elsewhere, the Python API attempts to mimic the
C++ Halide API as directly as possible; there
isn't separate Python-specific documentation for the API at this time.
For now, examine the code for the example applications in the test/apps/ and
tutorial/ subdirectories.
The tests run as part of the standard CTest infrastructure and are labeled with
the python label. You can run the Python tests specifically by running:

from the Halide build directory.
Differences from C++ API  Copy Chapter
The Python bindings attempt to mimic the Halide C++ API as closely as possible,
with some differences where the C++ idiom is either inappropriate or impossible:
- Most APIs that take a variadic argument list of ints in C++ take an explicit
  list in Python. For instance, the usual version of the Buffer ctor in C++
  offers both variadic and list versions:

In Python, only the second variant is provided.
- Func and Buffer access is done using [] rather than ()
  - For zero-dimensional Func and Buffer, you must explicitly specify [()]
    -- that is, use an empty tuple as the index -- because [] is not
    syntactically acceptable in Python.
- Some classes in the Halide API aren't provided because standard Python idioms
  are a better fit:
  - Halide::Tuple doesn't exist in the Python bindings; an ordinary Python
    tuple of Halide::Expr is used instead.
  - Halide::Realization doesn't exist in the Python bindings; an ordinary
    Python tuple of Halide::Buffer is used instead.
- static and instance method overloads with the same name in the same class
  aren't allowed, so some convenience methods are missing from Halide::Var
- Templated types (notably Halide::Buffer<> and Halide::Param<>) aren't
  provided, for obvious reasons; only the equivalents of Halide::Buffer<void>
  and Halide::Param<void> are supported.
- The functions in Halide::ConciseCasts are present in the toplevel Halide
  module in Python, rather than a submodule: e.g., use halide.i8_sat(), not
  halide.ConciseCasts.i8_sat().
- Only things in the Halide namespace are supported; classes and methods that
  involve using the Halide::Internal namespace are not provided.
- No mechanism is provided for overriding any runtime functions from Python for
  JIT-compiled code. (Runtime functions for AOT-compiled code can be overridden
  by building and linking a custom runtime, but not currently via any runtime
  API, e.g. halide_set_custom_print() does not exist.)
- No mechanism is provided for supporting Func::define_extern.
- Buffer::for_each_value() isn't supported yet.
- Func::in becomes Func.in_ because in is a Python keyword.
- Func::async becomes Func.async_ because async is a Python keyword.
- The not keyword cannot be used to negate boolean Halide expressions.
  Instead, the logical_not function can be used and is equivalent to using
  operator! in C++.
- There is no way to override the logical and/or operators in Python to work
  with Expr: you must use the bitwise | and & instead. (Note that
  incorrectly using and/or just short-circuits weirdly, rather than failing
  with some helpful error; this is an issue that we have not yet found any way
  to improve, unfortunately.)
- Some error messages need to be made more informative.
- Some exceptions are the "incorrect" type (compared to C++ expectations).
- Many hooks to override runtime functions (e.g. Func::set_error_handler) aren't
  yet implemented.
- The following parts of the Halide public API are currently missing entirely
  from the Python bindings (but are all likely to be supported at some point in
  the future):
  - DeviceInterface
  - evaluate()
Example of Simple Usage  Copy Chapter
Here is a basic example of using Halide to produce a procedural image.

It's worth noting in the example above that the Halide Buffer object supports
the Python Buffer Protocol (https://www.python.org/dev/peps/pep-3118) and thus
is converted to and from other compatible objects (e.g., NumPy's ndarray), at
essentially zero cost, with storage being shared. Thus, we can usually pass it
directly to existing Python APIs (like imsave()) that expect 'image-like'
objects without any explicit conversion necessary.
Halide Generators In Python  Copy Chapter
In Halide, a "Generator" is a unit of encapsulation for Halide code. It is a
self-contained piece of code that can:
- Produce a chunk of Halide IR (in the form of an hl.Pipeline) that is
  appropriate for compilation (via either JIT or AOT)
- Expose itself to the build system in a discoverable way
- Fully describe itself for the build system with metadata for (at least) the
  type and number of inputs and outputs expected
- Allow for build-time customization of coder-specified parameters in a way that
  doesn't require editing of source code
Originally, Halide only supported writing Generators in C++. In this document,
we'll use the term "C++ Generator" to mean "Generator written in C++ using the
classic API", the term "Python Generator" to mean "Generator written in Halide's
Python bindings", and just plain "Generator" when the discussion is relatively
neutral with respect to the implementation language/API.
Writing a Generator in Python  Copy Section
A Python Generator is a class that:
- has the @hl.generator decorator applied to it
- declares zero or more member fields that are initialized with values of
  hl.InputBuffer or hl.InputScalar, which specify the expected input(s) of
  the resulting Pipeline.
- declares one or more member fields that are initialized with values of
  hl.OutputBuffer or hl.OutputScalar, which specify the expected output(s)
  of the resulting Pipeline.
- declares zero or more member fields that are initialized with values of
  hl.GeneratorParam, which can be used to pass arbitrary information from the
  build system to the Generator. A GeneratorParam can carry a value of type
  bool, int, float, str, or hl.Type.
- declares a generate() method that fill in the Halide IR needed to define all
  the Outputs
- optionally declares a configure() method to dynamically add Inputs or
  Outputs to the pipeline, based on (e.g.) the values of GeneratorParam values
  or other external inputs
Let's look at a fairly simple example:
TODO: this example is pretty contrived; is there an equally simple

Generator to use here that would demonstrate the basics?

If you've worked with Halide Generators written in C++, the "shape" of this will
likely look familiar. (If not, no worries; you shouldn't need any knowledge of
C++ Generators for the following to make sense.)
Let's take the details here one at a time.
#### hl.generator("name")
This decorator adds appropriate "glue" machinery to the class to enforce various
invariants. It also serves as the declares a "registered name" for the
Generator, which is a unique name that the build system will use to identify the
Generator. If you omit the name, it defaults to module.classname; if module is
__main__ then we omit it and just use the plain classname. Note that the
registered name need not match the classname. (Inside Halide, we use the
convention of CamelCase for class names and snake_case for registered names,
but you can use whatever convention you like.)
#### hl.GeneratorParam
Each GeneratorParam is an arbitrary key-value pair that can be used to provide
configurable options at compile time. You provide the name and a default value.
The default value can be overridden by the build machinery, which will replace
the value (based on user specified text).
Note that the type of the default value is used to define the expected type of
the GeneratorParam, and trying to set it to an incompatible value will throw
an exception. The types that are acceptable to use in a GeneratorParam are:
- Python's bool, int, float, or str
- Halide's hl.Type
- ...that's all
Note that the value of a GeneratorParam is read-only from the point of view of
the Generator; they are set at Generator construction time and attempting to
change their value will throw an exception.
#### hl.InputBuffer, hl.InputScalar
These declare the inputs to the hl.Pipeline that the Generator will produce.
An hl.InputScalar is, essentially, a "factory" that produces an hl.Param in
the existing Python API, while an hl.InputBuffer is a factory for
hl.ImageParam.
From the Generator author's perspective, a field initialized with InputScalar
is a Param – not kinda-like-one, not a magic wrapper that forwards
everything; it is literally just hl.Param. Similarly, an InputBuffer
produces ImageParam, and an InputFunc is a wrapper around Func. You won't
be able to assign a new value to the member field for Inputs – as with
GeneratorParams, they are "read-only" to the Generator – but you will be able to
set constraints on them.
Note that in addition to specifying a concrete type and dimensionality for the
inputs, these factory classes support the ability to specify either (or both)
None, which means the type/dimensionality will be provided by GeneratorParams
in the build system.
#### hl.OutputBuffer, hl.OutputScalar
These declare the output(s) of the Pipeline that the Generator will produce. An
hl.OutputBuffer is, essentially, a "factory" that produces an hl.Func in the
existing Python API. (hl.OutputScalar is just an hl.OutputBuffer that always
has zero dimensions.)
From the Generator author's perspective, a field declared with OutputBuffer
is a Func – not kinda-like-one, not a magic wrapper that forwards
everything; it is literally just hl.Func (with type-and-dimensionality set to
match, see recent PR https://github.com/halide/Halide/pull/6734) . You won't be
able to assign a new value to the member field for Inputs – as with
GeneratorParams, they are "read-only" to the Generator – but you will be able to
set constraints on them.
Note that in addition to specifying a concrete type and dimensionality for the
inputs, these factory classes support the ability to specify either (or both) as
None, which means the type/dimensionality will be provided by GeneratorParams
in the build system.
#### Names
Note that all the GeneratorParams, Inputs, and Outputs have names that are
implicitly filled in based on the field name of their initial assignment; unlike
in C++ Generators, there isn't a way to "override" this name (i.e., the name in
the IR will always exactly match the Python field name). Names have the same
constraints as for C++ Generators (essentially, a C identifier, but without an
initial underscore, and without any double underscore anywhere).
#### generate() method
This will be called by the Generator machinery to build the Pipeline. As with
C++ Generators, the only required task is to ensure that all Output fields are
fully defined, in a way that matches the type-and-dimension constraints
specified.
It is required that the generate() method be defined by the Generator.
(Note that, by convention, Halide Generators use g instead of self in their
generate() method to make the expression language terser; this is not in any
way required, but is recommended to improve readability.)
#### Types for Inputs and Outputs
For all the Input and Output fields of Generators, you can specify native Python
types (instead of hl.Type) for certain cases that are unambiguous. At present,
we allow bool as an alias for hl.Bool(), int as an alias for hl.Int(32),
and float as an alias for hl.Float(32).
Using a Generator for JIT compilation  Copy Section
You can use the compile_to_callable() method to JIT-compile a Generator into a
hl.Callable, which is (essentially) just a dynamically-created function.

By default, a Generator will produce code targeted at Target("host") (or the
value of the HL_JIT_TARGET environment variable, if set); you can override
this behavior selectively by activating a GeneratorContext when the Generator
is created:

Using a Generator for AOT compilation  Copy Section
If you are using CMake, the simplest thing is to use add_halide_library and
add_halide_python_extension_library():

(Note that this rule works for both C++ and Python Generators.)
This compiles the Generator code in logical_op_generator.py with the
registered name logical_op_generator to produce the target xor_filter, and
then wraps the compiled output with a Python extension. The result will be a
shared library of the form <target>.<soabi>.so, where <soabi> describes the
specific Python version and platform (e.g., cpython-310-darwin for Python 3.10
on OSX.)
Note that you can combine multiple Halide libraries into a single Python module;
this is convenient for packaging, but also because all the libraries in a single
extension module share the same Halide runtime (and thus, the same caches,
thread pools, etc.).

Note that you must take care to ensure that all of the add_halide_library
targets specified use the same Halide runtime; it may be necessary to use
add_halide_runtime to define an explicit runtime that is shared by all the
targets:

If you're not using CMake, you can "drive" a Generator directly from your build
system via command-line flags. The most common, minimal set looks something like
this:

The argument to -g is the name supplied to the @hl.generator decorator. The
argument to -o is a directory to use for the output files; by default, we'll
produce a static library containing the object code, and a C++ header file with
a forward declaration. target specifies a Halide Target string describing
the OS, architecture, features, etc. that should be used for compilation. Any
other arguments to the command line that don't begin with - are presumed to
name GeneratorParam values to set.
There are other flags and options too, of course; use
python3 /path/to/my/generator.py -help to see a list with explanations.
(Unfortunately, there isn't (yet) a way to produce a Python Extension just by
running a Generator; the logic for add_halide_python_extension_library is
currently all in the CMake helper files.)
Calling Generator-Produced code from Python  Copy Section
As long as the shared library is in PYTHONPATH, it can be imported and used
directly. For the example above:

Above, we're using common Python utilities (numpy) to construct the
input/output buffers we want to pass to Halide.
Note: Getting the memory order correct can be a little confusing for numpy.
By default, numpy uses "C-style"
row-major
order, which sounds like the right option for Halide; however, this nomenclature
assumes the matrix-math convention of ordering axes as [rows, cols], whereas
Halide (and imaging code in general) generally assumes [x, y] (i.e.,
[cols, rows]). Thus, what you usually want in Halide is column-major ordering.
This means numpy arrays, by default, come with the wrong memory layout for
Halide. But if you construct the numpy arrays yourself (like above), you can
pass order='F' to make numpy use the Halide-compatible memory layout. If
you're passing in an array constructed somewhere else, the easiest thing to do
is to .transpose() it before passing it to your Halide code.
Advanced Generator-Related Topics  Copy Section
#### Generator Aliases
A Generator alias is a way to associate a Generator with one (or more) specific
sets of GeneratorParams; the 'alias' is just another registered name. This
offers a convenient alternative to specifying multiple sets of GeneratorParams
via the build system. To define alias(es) for a Generator, just add the
@hl.alias decorator before @hl.generator decorator:

#### Dynamic Inputs and Outputs
If you need to build Input and/or Output dynamically, you can define a
configure() method. It will always be called after all GeneratorParam values
are valid, but before generate() is called. Let's take our example and add an
option to pass an offset to be added after the logical operator is done:

The only thing you can (usefully) do from configure() is to call add_input()
or add_output(), which accept only the appropriate Input or Output
classes. The resulting value is stored as a member variable with the name
specified (if there is already a member with the given name, an exception is
thrown).
#### Calling a Generator Directly
Each Generator has a class method (injected by @hl.generator) that allows you
to "call" the Generator like an ordinary function; this allows you to directly
take the Halide IR produced by the Generator and do anything you want to with
it. This can be especially useful when writing library code, as you can
'compose' more complex pipelines this way.
This method is named call() and looks like this:

It takes the inputs (specified either by-name or by-position in the usual Python
way). It also allows for an optional by-name-only argument, generator_params,
which is a simple Python dict that allows for overriding GeneratorParams. It
returns a tuple of the Output values. For the earlier example, usage might be
something like:

#### The Lifecycle Of A Generator
Whether being driven by a build system (for AOT use) or by another piece of
Python code (typically for JIT use), the lifecycle of a Generator looks
something like this:
- An instance of the Generator in question is created. It uses the
  currently-active GeneratorContext (which contains the Target to be used
  for code generation), which is stored in a thread-local stack.
- Some (or all) of the default values of the GeneratorParam members may be
  replaced based on (e.g.) command-line arguments in the build system
- All GeneratorParam members are made immutable.
- The configure() method is called, allowing the Generator to use
  add_input() or add_output() to dynamically add inputs and/or outputs.
- If any Input or Output members were defined with unspecified type or
  dimensions (e.g. some_input = hl.InputBuffer(None, 3)), those types and
  dimensions are filled in from GeneratorParam values (e.g. some_input.type
  in this case). If any types or dimensions are left unspecified after this
  step, an exception will be thrown.
- If the Generator is being invoked via its call() method (see below), the
  default values for Inputs will be replaced by the values from the argument
  list.
- The Generator instance has its generate() method called.
- The calling code will extract the values of all Output values and validate
  that they match the type, dimensions, etc. of the declarations.
- The calling code will then either call compile_to_file() and friends (for
  AOT use), or return the output values to the caller (for JIT use).
- Finally, the Generator instance will be discarded, never to be used again.
Note that almost all the code doing the hand-wavy bits above is injected by the
@hl.generator decorator – the Generator author doesn't need to know or care
about the specific details, only that they happen.
All Halide Generators are single-use instances – that is, any given
Generator instance should be used at most once. If a Generator is to be executed
multiple times (e.g. for different GeneratorParam values, or a different
Target), a new one must be constructed each time.
#### Notable Differences Between C++ and Python Generators
If you have written C++ Generators in Halide in the past, you might notice some
features are missing and/or different for Python Generators. Among the
differences are:
- In C++, you can create a Generator, then call set_generatorparam_value() to
  alter the values of GeneratorParams. In Python, there is no public method to
  alter a GeneratorParam after the Generator is created; instead, you must pass
  a dict of GeneratorParam values to the constructor, after which the values are
  immutable for that Generator instance.
- Array Inputs/Outputs: in our experience, they are pretty rarely used, it
  complicates the implementation in nontrivial ways, and the majority of use
  cases for them can all be reasonably supported by dynamically adding inputs or
  outputs (and saving the results in a local array).
- Input<Func> and Output<Func>: these were deliberately left out in order to
  simplify Python Generators. It's possible that something similar might be
  added in the future.
- GeneratorParams with LoopLevel types: these aren't useful without
  Input<Func>/Output<Func>.
- GeneratorParams with Enum types: using a plain str type in Python is
  arguably just as easy, if not easier.
- get_externs_map(): this allows registering ExternalCode objects to be
  appended to the Generator's code. In our experience, this feature is very
  rarely used. We will consider adding this in the future if necessary.
- Lazy Binding of Unspecified Input/Output Types: for C++ Generators, if you
  left an Output's type (or dimensionality) unspecified, you didn't always have
  to specify a GeneratorParam to make it into a concrete type: if the type was
  always fully specified by the contents of the generate() method, that was
  good enough. In Python Generators, by contrast, all types and dimensions
  must be explicitly specified by either code declaration or by
  GeneratorParam setting. This simplifies the internal code in nontrivial
  ways, and also allows for (arguably) more readable code, since there are no
  longer cases that require the reader to execute the code in their head in
  order to deduce the output types.
Keeping Up To Date  Copy Chapter
If you use the Halide Bindings for Python inside Google, you are strongly
encouraged to
subscribe to announcements for new releases of Halide,
as it is likely that enhancements and tweaks to our Python support will be made
in future releases.
License  Copy Chapter
The Python bindings use the same
MIT license as Halide.
Python bindings provided by Connelly Barnes (2012-2013), Fred Rotbart (2014),
Rodrigo Benenson (2015) and the Halide open-source community.
---
Doc/RunGen  Copy Section
Running and Benchmarking Halide Generators
Overview  Copy Chapter
RunGen is a simple(ish) wrapper that allows an arbitrary Generator to be built
into a single executable that can be run directly from bash, without needing to
wrap it in your own custom main() driver. It also implements a rudimentary
benchmarking and memory-usage functionality.
If you use the standard CMake rules for Generators, you get RunGen functionality
automatically. (If you use Make, you might need to add an extra rule or two to
your Makefile; all the examples in apps/ already have these rules.)
For every halide_library (or halide_library_from_generator) rule, there is
an implicit name.rungen rule that generates an executable that wraps the
Generator library:

You can build and run this like any other executable:

To be useful, you need to pass in values for the Generator's inputs (and
locations for the output(s)) on the command line, of course. You can use the
--describe flag to see the names and expected types:

Warning: Outputs may have $X (where X is a small integer) appended to their
names in some cases (or, in the case of Generators that don't explicitly declare
outputs via Output<>, an autogenerated name of the form fX). If this
happens, don't forget to escape the $ with a backslash as necessary. These are
both bugs we intend to fix; see https://github.com/halide/Halide/issues/2194
As a convenience, there is also an implicit target that builds-and-runs, named
simply "NAME.run":

Inputs are specified as name=value pairs, in any order. Scalar inputs are
specified the typical text form, while buffer inputs (and outputs) are specified
via paths to image files. RunGen currently can read/write image files in any
format supported by halide_image_io.h; at this time, that means .png, .jpg,
.ppm, .pgm, and .tmp formats. (We plan to add .tiff and .mat (level 5) in the
future.)

You can also specify any scalar input as default or estimate, which will use
the default value specified for the input, or the value specified by
set_estimate for that input. (If the relevant value isn't set for that input,
a runtime error occurs.)

If you specify an input or output file format that doesn't match the required
type/dimensions for an argument (e.g., using an 8-bit PNG for an Input<float>,
or a grayscale image for a 3-dimensional input), RunGen will try to coerce the
inputs to something sensible; that said, it's hard to always get this right, so
warnings are always issued whenever an input or output is modified in any
way.

By default, we try to guess a suitable size for the output image(s), based
mainly on the size of the input images (if any); you can also specify explicit
output extents. (Note that output_extents are subject to constraints already
imposed by the particular Generator's logic, so arbitrary values for
--output_extents may produce runtime errors.)

Sometimes you don't care what the particular element values for an input are
(e.g. for benchmarking), and you just want an image of a particular size; in
that case, you can use the zero:[] pseudo-file; it infers the _type_ from the
Generator, and inits every element to zero:

You can also specify arbitrary (nonzero) constants:

Similarly, you can create identity images where only the diagonal elements are
1-s (rest are 0-s) by invoking identity:[]. Diagonal elements are defined as
those whose first two coordinates are equal.
There's also a random:SEED:[] pseudo-file, which fills the image with uniform
noise based on a specific random-number seed:

Instead of specifying an explicit set of extents for a pseudo-input, you can use
the string auto, which will run a bounds query to choose a legal set of
extents for that input given the known output extents. (This is only useful when
used in conjunction with the --output_extents flag.)

You can also specify estimate for the extents, which will use the estimate
values provided, typically (but not necessarily) for auto_schedule. (If there
aren't estimates for all of the buffer's dimensions, a runtime error occurs.)

You can combine the two and specify estimate_then_auto for the extents, which
will attempt to use the estimate values; if a given input buffer has no
estimates, it will fall back to the bounds-query result for that input:

Similarly, you can use estimate for --output_extents, which will use the
estimate values for each output. (If there aren't estimates for all of the
outputs, a runtime error occurs.)

If you don't want to explicitly specify all (or any!) of the input values, you
can use the --default_input_buffers and --default_input_scalars flags, which
provide wildcards for any omitted inputs:

In this case, all input buffers will be sized according to bounds query, and
filled with a random seed; all input scalars will be initialized to their
declared default values. (If they have no declared default value, a zero of the
appropriate type will be used.)
Note: --default_input_buffers can produce surprising sizes! For instance, any
input that uses BoundaryConditions::repeat_edge to wrap itself can legally be
set to almost any size, so you may legitimately get an input with extent=1 in
all dimensions; whether this is useful to you or not depends on the code. It's
highly recommended you do testing with the --verbose flag (which will log the
calculated sizes) to reality-check that you are getting what you expect,
especially for benchmarking.
A common case (especially for benchmarking) is to specify using estimates for
all inputs and outputs; for this, you can specify --estimate_all, which is
just a shortcut for
--default_input_buffers=estimate_then_auto --default_input_scalars=estimate --output_extents=estimate.
Benchmarking  Copy Chapter
To run a benchmark, use the --benchmarks=all flag:

You can use --default_input_buffers and --default_input_scalars here as
well:

Note: halide_benchmark.h is known to be inaccurate for GPU filters; see
https://github.com/halide/Halide/issues/2278
Measuring Memory Usage  Copy Chapter
To track memory usage, use the --track_memory flag, which measures the
high-water-mark of CPU memory usage.

Warning: --track_memory may degrade performance; don't combine it with
--benchmark or expect meaningful timing measurements when using it.
Using RunGen in Make  Copy Chapter
To add support for RunGen to your Makefile, you need to add rules something like
this (see apps/support/Makefile.inc for an example):

Note that the %.registration.cpp file is created by running a generator and
specifying registration in the comma-separated list of files to emit; these
are also generated by default if -e is not used on the generator command line.
Known Issues & Caveats  Copy Chapter
- If your Generator uses define_extern(), you must have all link-time
  dependencies declared properly via FILTER_DEPS; otherwise, you'll fail to
  link.
- The code does its best to detect when inputs or outputs need to be
  chunky/interleaved (rather than planar), but in unusual cases it might guess
  wrong; if your Generator uses buffers with unusual stride setups, RunGen might
  fail at runtime. (If this happens, please file a bug!)
- The code for deducing good output sizes is rudimentary and needs to be
  smartened; it will sometimes make bad decisions which will prevent the filter
  from executing. (If this happens, please file a bug!)
---
Doc/Testing  Copy Section
Testing
Halide uses CTest as its primary test platform and runner.
Organization  Copy Chapter
Halide's tests are organized beneath the top-level test/ directory. These
folders are described below:
| Folder               | Description                                                                      |
| -------------------- | -------------------------------------------------------------------------------- |
| autoschedulers/$AS | Test for the $AS (e.g. adams2019) autoscheduler                              |
| common             | Code that may be shared across multiple tests                                    |
| correctness        | Tests that check correctness of various compiler properties                      |
| error              | Tests that expect an exception to be thrown (or abort() to be called)          |
| failing_with_issue | Correctness tests that are associated with a particular issue on GitHub          |
| fuzz               | Fuzz tests. Read more at FuzzTesting.md                        |
| generator          | Tests of Halide's AOT compilation infrastructure.                                |
| integration        | Tests of Halide's CMake package for downstream use, including cross compilation. |
| performance        | Tests that check that certain schedules indeed improve performance.              |
| runtime            | Unit tests for the Halide runtime library                                        |
| warning            | Tests that expected warnings are indeed issued.                                  |
The tests in each of these directories are given CTest labels corresponding to
the directory name. Thus, one can use ctest -L generator to run only the
generator tests. The performance tests configure CTest to not run them
concurrently with other tests (including each other).
The vast majority of our tests are simple C++ executables that link to Halide,
perform some checks, and print the special line Success! upon successful
completion. There are three main exceptions to this:
First, the warning tests are expected to print a line that reads Warning:
and do not look for Success!.
Second, some tests cannot run in all scenarios; for example, a test that
measures CUDA performance requires a CUDA-capable GPU. In these cases, tests are
expected to print [SKIP] and exit and not print Success! or Warning:.
Finally, the error tests are expected to throw an (uncaught) exception that is
not a Halide::InternalError (i.e. from a failing internal_assert). The logic
for translating uncaught exceptions into successful tests is in
test/common/expect_abort.cpp.
Debugging the compiler  Copy Chapter
We provide helpers for pretty-printing Halide's IR types (and other internal
data structures) in interactive debuggers. The LLDB and GDB helpers share a
single C++ bridge, Halide::Internal::debug_string() (declared in
src/IRPrinter.h): each pretty-printer asks the inferior to render the value
with the normal IR printer, so all three front-ends produce the same output.
Because the value is rendered by calling into the program, the process must be
running (stopped at a breakpoint) for IR summaries to appear.
LLDB  Copy Section
The .lldbinit file at the repository root will load tools/lldbhalide.py
automatically if you launch lldb from this directory and your ~/.lldbinit
file contains the line,

If you prefer to avoid such global configuration, you can directly load the
helpers with the LLDB command,

again assuming that the repository root is your current working directory.
To see the benefit of using these helpers, let us debug correctness_bounds:

Now we can try to inspect the Func h. Without the helpers, we see:

But if we load the helpers and try again, we get a much more useful output:

The amount of output here is maybe a bit _too_ much, but we gain the ability to
more narrowly inspect data about the func:

These helpers are particularly useful when using graphical debuggers, such as
the one found in CLion.
GDB  Copy Section
The GDB helpers live in tools/gdbhalide.py and mirror the LLDB coverage. The
.gdbinit file at the repository root loads them when you launch gdb from
this directory, but GDB only auto-loads a local .gdbinit if you have marked it
safe, e.g. by adding to your ~/.gdbinit:

Otherwise, load the helpers manually once the program is running:

You can then print an Expr, Stmt, Target, etc. and see the same
pretty-printed form that LLDB produces.
Natvis (Visual Studio and CLion)  Copy Section
tools/Halide.natvis provides Visual Studio-style visualizers. On MSVC it is
embedded into the Halide PDB automatically (via the build), so it loads whenever
you debug something linked against Halide. It is also attached to the target as
a source file, so IDEs whose LLDB-based debuggers understand Natvis — including
recent CLion and Rider on Linux and macOS — can load it as well (Settings |
Build, Execution, Deployment | Debugger | Data Views | Native).
Natvis is declarative and cannot call functions, so it does not reproduce the
one-line IR rendering from debug_string() — use the LLDB/GDB helpers above for
that. Instead, it downcasts each Expr/Stmt handle to its concrete node using
the node_type discriminator and presents a fully expandable IR tree, recursing
through operands (leaf nodes such as immediates and Variable show their value
in the summary). Because this uses no function calls, it also works on a core
dump or a process that isn't running. Natvis additionally covers buffers
(halide_buffer_t, Halide::Buffer, Halide::Runtime::Buffer, including "Add
to Image Watch" in Visual Studio) and flat value types such as halide_type_t,
Halide::Type, and ConstantInterval.
---
Doc/Vulkan  Copy Section
Vulkan Support for Halide
Halide supports the Khronos Vulkan framework as a compute API backend for
GPU-like devices, and compiles directly to a binary SPIR-V representation as
part of its code generation before submitting it to the Vulkan API. Both JIT and
AOT usage are supported via the vulkan target flag (e.g.
HL_JIT_TARGET=host-vulkan).
Vulkan support is actively under development, and considered BETA quality at
this stage. Tests are passing, but performance tuning and user testing is needed
to identify potential issues before rolling this into production.
See below for details.
Compiling Halide w/Vulkan Support
You'll need to configure Halide and enable the cmake option TARGET_VULKAN (which
is now ON by default).
For example, on Linux & OSX:

On Windows, you may need to specify the location of the Vulkan SDK if the paths
aren't resolved by CMake automatically. For example (assuming the Vulkan SDK is
installed in the default path):

Vulkan Runtime Environment:
Halide has no direct dependency on Vulkan for code-generation, but the runtime
requires a working Vulkan environment to run Halide generated code. Any valid
Vulkan v1.0+ device driver should work.
Specifically, you'll need:
- A vendor specific Vulkan device driver
- The generic Vulkan loader library
For AMD & NVIDIA & Intel devices, download and install the latest graphics
driver for your platform. Vulkan support should be included.
Windows  Copy Chapter
To build Halide AOT generators, you'll need the Vulkan SDK (specifically the
Vulkan loader library and headers):
https://sdk.lunarg.com/sdk/download/latest/windows/vulkan-sdk.exe
For Vulkan device drivers, consult the appropriate hardware vendor for your
device. A few common ones are listed below.
- AMD Vulkan Driver
- NVIDIA Vulkan Driver
- INTEL Vulkan Driver
Linux  Copy Chapter
The Vulkan SDK packages are now being maintained by LunarG. These include the
Vulkan Loader library, as well as the Vulkan Tools packages. Instructions for
installing these can be found on their
Getting Started Guide.
Once the SDK has been installed, you need to install the appropriate driver for
your device. Proprietary drivers can be installed via 'apt' using PPA's for each
vendor. Examples for AMD and NVIDIA are provided below.
For AMD on Ubuntu v22.04:

For NVIDIA on Ubuntu v22.04:

Note that only valid drivers for your system should be installed since there are
reports of the Vulkan loader segfaulting just by having a non-supported driver
present. Specifically, the seemingly generic mesa-vulkan-drivers actually
includes the AMD graphics driver, which can cause problems if installed on an
NVIDIA-only system.
Mac  Copy Chapter
You're better off using Halide's Metal backend instead, but it is possible to
run Vulkan apps on a Mac via the MoltenVK library:
- MoltenVK Project
The easiest way to get the necessary dependencies is to use the official
MoltenVK SDK installer provided by LunarG:
- MoltenVK SDK (Latest Release)
Alternatively, if you have the Homebrew package manager
installed for MacOS, you can use it to install the Vulkan Loader and MoltenVK
compatibility layer:

Testing Your Vulkan Environment
You can validate that everything is configured correctly by running the
vulkaninfo app (bundled in the vulkan-utils package) to make sure your device
is detected (eg):

Make sure everything looks correct before continuing!
Targeting Vulkan
To generate Halide code for Vulkan, simply add the vulkan flag to your target
as well as any other optional device specific features you wish to enable for
Halide:
| Target Feature | Description                                                    |
| -------------- | -------------------------------------------------------------- |
| vulkan       | Enables the vulkan backend                                     |
| vk_int8      | Allows 8-bit integer storage types to be used                  |
| vk_int16     | Allows 16-bit integer storage types to be used                 |
| vk_int64     | Allows 64-bit integer storage types to be used                 |
| vk_float16   | Allows 16-bit floating-point values to be used for computation |
| vk_float64   | Allows 64-bit floating-point values to be used for computation |
| vk_v10       | Generates code compatible with the Vulkan v1.0+ API            |
| vk_v12       | Generates code compatible with the Vulkan v1.2+ API            |
| vk_v13       | Generates code compatible with the Vulkan v1.3+ API            |
Note that 32-bit integer and floating-point types are always available. All
other optional device features are off by default (since they are not required
by the Vulkan API, and thus must be explicitly enabled to ensure that the code
being generated will be compatible with the device and API version being used
for execution).
For AOT generators add vulkan (and any other flags you wish to use) to the
target command line option:

For JIT apps use the HL_JIT_TARGET environment variable:

Useful Runtime Environment Variables
To modify the default behavior of the runtime, the following environment
variables can be used to adjust the configuration of the Vulkan backend at
execution time:
HL_VK_LOADER_LIB=... can be used to specify the full path and filename of the
Vulkan loader library (e.g. /usr/local/lib64/libvulkan.so.1). This can be
useful if you need to override the default search path, or if the loader is
installed in a custom location. If not set, the runtime will search across a
list of known paths.
HL_VK_LAYERS=... will tell Halide to choose a suitable Vulkan instance that
supports the given list of layers. If not set, VK_INSTANCE_LAYERS=... will be
used instead. If neither are present, Halide will use the first Vulkan compute
device it can find. Multiple layers can be specified using the appropriate
environment variable list delimiter (: on Linux/OSX/Posix, or ; on Windows).
HL_VK_DEVICE_TYPE=... will tell Halide to choose which type of device to
select for creating the Vulkan instance. Valid options are 'gpu',
'discrete-gpu', 'integrated-gpu', 'virtual-gpu', or 'cpu'. If not set, Halide
will search for the first 'gpu' like device it can find, or fall back to the
first compute device it can find.
HL_VK_ALLOC_CONFIG=... will tell Halide to configure the Vulkan memory
allocator use the given constraints specified as 5x integer values separated by
the appropriate environment variable list delimiter (e.g. N:N:N:N:N on
Linux/OSX/Posix, or N;N;N;N;N on Windows). These values correspond to
maximum_pool_size, minimum_block_size, maximum_block_size,
maximum_block_count and nearest_multiple.
The maximum_pool_size constraint will tell Halide to configure the Vulkan
memory allocator to never request more than N megabytes for the entire pool of
allocations for the context. This includes all resource blocks used for
suballocations. Setting this to a non-zero value will limit the amount device
memory used by Halide, which may be useful when other applications and
frameworks are competing for resources. Default is 0 ... meaning no limit.
The minimum_block_size constraint will tell Halide to configure the Vulkan
memory allocator to always request a minimum of N megabytes for a resource
block, which will be used as a pool for suballocations.\
Increasing this value may improve performance while sacrificing the amount of
available device memory. Default is 32MB.
The maximum_block_size constraint will tell Halide to configure the Vulkan
memory allocator to never exceed a maximum of N megabytes for a resource block.
Decreasing this value may free up more memory but may impact performance, and/or
restrict allocations to be unusably small. Default is 0 ... meaning no limit.
The maximum_block_count constraint will tell Halide to configure the Vulkan
memory allocator to never exceed a total of N block allocations.\
Decreasing this value may free up more memory but may impact performance, and/or
restrict allocations. Default is 0 ... meaning no limit.
The nearest_multiple constraint will tell Halide to configure the Vulkan
memory allocator to always round up the requested allocation sizes to the given
integer value. This is useful for architectures that require specific alignments
for subregions allocated within a block. Default is 32 ... setting this to zero
means no constraint.
Debug Environment Variables
The following environment variables may be useful for tracking down potential
issues related to Vulkan:
HL_DEBUG_CODEGEN=3 will print out debug info that includes the SPIR-V code
generator used for Vulkan while it is compiling.
HL_SPIRV_DUMP_FILE=... specifies a file to dump the binary SPIR-V generated
during compilation. Useful for debugging CodeGen issues. Can be inspected,
validated and disassembled via the SPIR-V tools:
https://github.com/KhronosGroup/SPIRV-Tools
In addition to the SPIR-V Tools, you may also wish to install the Khronos
Validation Layers which provide an exhaustive suite of runtime checks that can
be injected by adding VK_LAYER_KHRONOS_validation to the VK_INSTANCE_LAYERS=
environment variable.
To install the validation layers and the SPIR-V tools on Ubuntu v22.04:

To test the validation layer, you can prepend your shell command for any Vulkan
enabled binary with the appropriate environment settings. For example, you can
run one of the JIT-enabled correctness tests w/debug output and validation
layers enabled like so:

Current Status
All correctness tests are now passing on tested configs for Linux & Windows
using the target
host-vulkan-vk_int8-vk_int16-vk_int64-vk_float16-vk_float64-vk_v13 on LLVM
v14.x.
MacOS passes most tests but encounters internal MoltenVK code translation issues
for wide vectors, and ambiguous function calls.
Python apps, tutorials and correctness tests are now passing, but the AOT cases
are skipped since the runtime environment needs to be customized to locate the
platform specific Vulkan loader library.
Android platform support is currently being worked on.
Caveats:
- Other than 32-bit floats and integers, every other data type is optional per
  the Vulkan spec
- Float 64-bit types can be enabled, but there aren't any native math functions
  available in SPIR-V
- Only one dynamically sized shared memory allocation can be used, but any
  number of fixed sized allocation are supported (up to the maximum amount
  allowed by the device)
Known TODO:
- Performance tuning of CodeGen and Runtime
- More platform support (Android is work-in-progress, RISC-V, etc)
- Adapt unsupported types to supported types (if missing vk_int8 then promote to
  uint32_t)?
- Better debugging utilities using the Vulkan debug hooks.
- Allow debug symbols to be stripped from SPIR-V during codegen to reduce memory
  overhead for large kernels.
- Investigate floating point rounding and precision (v1.3 adds more controls)
- Investigate memory model usage (can Halide gain anything from these?)
---
Doc/WebAssembly  Copy Section
WebAssembly Support for Halide
Halide supports WebAssembly (Wasm) code generation from Halide using the LLVM
backend.
As WebAssembly itself is still under active development, Halide's support has
some limitations. Some of the most important:
- Sign-extension operations are enabled by default (but can be avoided via
  Target::WasmMvpOnly).
- Non-trapping float-to-int conversions are enabled by default (but can be
  avoided via Target::WasmMvpOnly).
- Fixed-width SIMD (128 bit) can be enabled via Target::WasmSimd128.
- Threads have very limited support via Target::WasmThreads; see
  below for more details.
- Halide's JIT for Wasm is extremely limited and really useful only for internal
  testing purposes.
Additional Tooling Requirements:
- In additional to the usual install of LLVM and clang, you'll need lld.
- Locally-installed version of Emscripten, 1.39.19+
Note that for all of the above, earlier versions might work, but have not been
tested.
AOT Limitations
Halide outputs a Wasm object (.o) or static library (.a) file, much like any
other architecture; to use it, of course, you must link it to suitable calling
code. Additionally, you must link to something that provides an implementation
of libc; as a practical matter, this means using the Emscripten tool to do
your linking, as it provides the most complete such implementation we're aware
of at this time.
- Halide ahead-of-time tests assume/require that you have Emscripten installed
  and available on your system, with the EMSDK environment variable set
  properly.
JIT Limitations
It's important to reiterate that the WebAssembly JIT mode is not (and will never
be) appropriate for anything other than limited self tests, for a number of
reasons:
- It actually uses an interpreter (from the WABT toolkit
  [https://github.com/WebAssembly/wabt]) to execute wasm bytecode; not
  surprisingly, this can be very slow.
- Wasm effectively runs in a private, 32-bit memory address space; while the
  host has access to that entire space, the reverse is not true, and thus any
  define_extern calls require copying all halide_buffer_t data across the
  Wasm\<->host boundary in both directions. This has severe implications for
  existing benchmarks, which don't currently attempt to account for this extra
  overhead. (This could possibly be improved by modeling the Wasm JIT's buffer
  support as a device model that would allow lazy copy-on-demand.)
- Host functions used via define_extern or HalideExtern cannot accept or
  return values that are pointer types or 64-bit integer types; this includes
  things like const char * and user_context. Fixing this is tractable, but
  is currently omitted as the fix is nontrivial and the tests that are affected
  are mostly non-critical. (Note that halide_buffer_t* is explicitly supported
  as a special case, however.)
- Threading isn't supported at all (yet); all parallel() schedules will be run
  serially.
- The .async() directive isn't supported at all, not even in serial-emulation
  mode.
- You can't use Param<void *> (or any other arbitrary pointer type) with the
  Wasm jit.
- You can't use Func.debug_to_file(), Func.set_custom_do_par_for(),
  Func.set_custom_do_task(), or Func.set_custom_allocator().
- The implementation of malloc() used by the JIT is incredibly simpleminded
  and unsuitable for anything other than the most basic of tests.
- GPU usage (or any buffer usage that isn't 100% host-memory) isn't supported at
  all yet. (This should be doable, just omitted for now.)
Note that while some of these limitations may be improved in the future, some
are effectively intrinsic to the nature of this problem. Realistically, this JIT
implementation is intended solely for running Halide self-tests (and even then,
a number of them are fundamentally impractical to support in a hosted-Wasm
environment and are disabled).
In sum: don't plan on using Halide JIT mode with Wasm unless you are working on
the Halide library itself.
Using V8 as the interpreter  Copy Chapter
There is experimental support for using V8 as the interpreter in JIT mode,
rather than WABT. This is enabled by the CMake command line options
-DWITH_V8=ON -DWITH_WABT=OFF (only one of them can be used at a time). You
must build V8 locally V8, then specify the path to the library and headers as
CMake options. This is currently only tested on x86-64-Linux and requires v8
version 9.8.177 as a minimum.
The canonical instructions to build V8 are at
v8.dev, and
there are examples for embedding v8. The process
for Halide is summarized below.
- Install
  depot_tools
- Fetch v8 source code (and install required dependencies):

- Create a build configuration: tools/dev/v8gen.py x64.release.sample
- Turn off pointer compression:
  echo 'v8_enable_pointer_compression = false' >> out.gn/x64.release.sample/args.gn
- Disable the GDB-JIT interface (conflicts with LLVM):
  echo 'v8_enable_gdbjit = false' >> out.gn/x64.release.sample/args.gn
- Build the static library: autoninja -C out.gn/x64.release.sample v8_monolith
With V8 built, we can pass the CMake options:
- V8_INCLUDE_DIR, path to V8 includes, e.g. $HOME/v8/v8/include
- V8_LIBRARY, path to V8 static library, e.g.
  $HOME/v8/v8/out.gn/x64.release.sample/obj/libv8_monolith.a
An example to configure Halide with V8 support, build and run an example test:

To Use Halide For WebAssembly:
- Ensure WebAssembly is in LLVM_TARGETS_TO_BUILD; if you use the default
  ("all") then it's already present, but otherwise, add it explicitly:

Enabling wasm JIT  Copy Chapter
If you want to run test_correctness and other interesting parts of the Halide
test suite (and you almost certainly will), you'll need to ensure that LLVM is
built with wasm-ld:
- Ensure that you have lld in LVM_ENABLE_PROJECTS:

- To run the JIT tests, set HL_JIT_TARGET=wasm-32-wasmrt (possibly adding
  wasm_simd128) and run CMake/CTest normally. Note that wasm testing is only
  supported under CMake (not via Make).
Enabling wasm AOT  Copy Chapter
If you want to test ahead-of-time code generation (and you almost certainly
will), you need to install Emscripten locally.
- The simplest way to install is probably via the Emscripten emsdk
  (https://emscripten.org/docs/getting_started/downloads.html).
- To run the AOT tests, set HL_TARGET=wasm-32-wasmrt (possibly adding
  wasm_simd128) and run CMake/CTest normally. Note that wasm testing is only
  supported under CMake (not via Make).
Running benchmarks
The test_performance benchmarks are misleading (and thus useless) for Wasm, as
they include JIT overhead as described elsewhere. Suitable benchmarks for Wasm
will be provided at a later date. (See
https://github.com/halide/Halide/issues/5119 and
https://github.com/halide/Halide/issues/5047 to track progress.)
Using Threads
You can use the wasm_threads feature to enable use of a normal pthread-based
thread pool in Halide code, but with some careful caveats:
- This requires that you use a wasm runtime environment that provides
  pthread-compatible wrappers. At this time of this writing, the only
  environment known to support this well is Emscripten (when using the
  -pthread flag, and compiling for a Web environment). In this configuration,
  Emscripten goes to great lengths to make WebWorkers available via the pthreads
  API. (You can see an example of this usage in apps/HelloWasm.) Note that not
  all wasm runtimes support WebWorkers; generally, you need a full browser
  environment to make this work (though some versions of some shell tools may
  also support this, e.g. nodejs).
- There is currently no support for using threads in a WASI environment, due to
  current limitations in the WASI specification. (We hope that this will improve
  in the future.)
- There is no support for using threads in the Halide JIT environment, and no
  plans to add them anytime in the near-term future.
Known Limitations And Caveats
- Current trunk LLVM (as of July 2020) doesn't reliably generate all of the Wasm
  SIMD ops that are available; see https://github.com/halide/Halide/issues/5130
  for tracking information as these are fixed.
- Using the JIT requires that we link the wasm-ld tool into libHalide; with
  some work this need could possibly be eliminated.
- OSX and Linux-x64 have been tested. Windows hasn't; it should be supportable
  with some work. (Patches welcome.)
- None of the apps/ folder has been investigated yet. Many of them should be
  supportable with some work. (Patches welcome.)
- We currently use v8/d8 as a test environment for AOT code; we may want to
  consider using Node or (better yet) headless Chrome instead (which is probably
  required to allow for using threads in AOT code).
Known TODO:
- There's some invasive hackiness in Codgen_LLVM to support the JIT trampolines;
  this really should be refactored to be less hacky.
- Can we rework JIT to avoid the need to link in wasm-ld? This might be doable,
  as the wasm object files produced by the LLVM backend are close enough to an
  executable form that we could likely make it work with some massaging on our
  side, but it's not clear whether this would be a bad idea or not (i.e., would
  it be unreasonably fragile).
- Buffer-copying overhead in the JIT could possibly be dramatically improved by
  modeling the copy as a "device" (i.e. copy_to_device() would copy from host
  -> wasm); this would make the performance benchmarks much more useful.
- Can we support threads in the JIT without an unreasonable amount of work?
  Unknown at this point.
---
Doc/WebGPU  Copy Section
WebGPU support for Halide
Halide has work-in-progress support for generating and running WebGPU shaders.
This can be used in conjunction with the WebAssembly backend to bring
GPU-accelerated Halide pipelines to the web.
As the first version of the WebGPU standard is itself still being developed,
Halide's support has some limitations and may only work with certain browsers
and versions of Emscripten.
Known limitations  Copy Chapter
The following is a non-comprehensive list of known limitations:
- Only 32-bit integers and floats have efficient support.
  - 8-bit and 16-bit integers are implemented using emulation. Future extensions
    to WGSL will allow them to be implemented more efficiently.
  - 64-bit integers and floats will likely remain unsupported until WGSL gains
    extensions to support them.
- Wrapping native device buffer handles is not yet implemented.
- You must use CMake/CTest to build/test Halide for WebGPU; using the Makefile
  is not supported for WebGPU testing (and probably never will be).
In addition to these functional limitations, the performance of the WebGPU
backend has not yet been evaluated, and so optimizations in the runtime or
device codegen may be required before it becomes profitable to use.
Running with WebAssembly via Emscripten: HL_TARGET=wasm-32-wasmrt-webgpu ')" title="Copy chapter prompt for LLMs"> Copy Chapter
_Tested with top-of-tree Emscripten as of 2023-02-23, against Chrome v113._
Halide can generate WebGPU code that can be integrated with WASM code using
Emscripten.
When invoking emcc to link Halide-generated objects, include these flags:
-s USE_WEBGPU=1 -s ASYNCIFY.
Tests that use AOT compilation can be run using a native WebGPU implementation
that has Node.js bindings, such as Dawn.
You must set an environment variable named HL_WEBGPU_NODE_BINDINGS that has an
absolute path to the bindings to run these tests, e.g.
HL_WEBGPU_NODE_BINDINGS=/path/to/dawn.node.
See below for instructions on building the Dawn Node.js
bindings.
JIT compilation is not supported when using WebGPU with WASM.
Running natively: HL_TARGET=host-webgpu ')" title="Copy chapter prompt for LLMs"> Copy Chapter
_Tested with top-of-tree Dawn as of 2023-11-27 \[commit

b5d38fc7dc2a20081312c95e379c4a918df8b7d4\]._
For testing purposes, Halide can also target native WebGPU libraries, such as
Dawn or
wgpu. This is currently the only path that can
run the JIT correctness tests. See below for instructions on
building Dawn.
Note that as of 2023-11-27, wgpu is not supported due to

lacking override support for WGSL

which we require > in order to set GPU block sizes.
When targeting WebGPU with a native target, Halide defaults to looking for a
build of Dawn (with several common names and suffixes); you can override this by
setting the HL_WEBGPU_NATIVE_LIB environment variable to the absolute path to
the library you want.
Note that it is explicitly legal to define both HL_WEBGPU_NATIVE_LIB and
HL_WEBGPU_NODE_BINDINGS at the same time; the correct executable environment
will be selected based on the Halide target specified.
Note that it is explicitly legal to specify both WEBGPU_NATIVE_LIB and
WEBGPU_NODE_BINDINGS for the same build; the correct executable environment will
be selected based on the Halide target specified.
Setting up Dawn  Copy Chapter
Building Dawn's Node.js bindings currently requires using CMake.
First,
install depot_tools
and add it to the PATH environment variable.
Next, get Dawn and its dependencies:

Finally, build Dawn, enabling both the Node.js bindings and shared libraries: