### Doc/BuildingHalideWithCMake # Building Halide with CMake This is a detailed guide to building Halide with CMake. If you want to learn how to use Halide in your own CMake projects, see [HalideCMakePackage.md]. If you are looking for Halide's CMake coding guidelines, see [CodeStyleCMake.md]. - [Building Halide with CMake](#building-halide-with-cmake) - [Installing CMake](#installing-cmake) - [Cross-platform](#cross-platform) - [Windows](#windows) - [macOS](#macos) - [Ubuntu Linux](#ubuntu-linux) - [Optional: Install Ninja](#optional-install-ninja) - [Dependencies](#dependencies) - [Summary](#summary) - [Installing dependencies](#installing-dependencies) - [vcpkg](#vcpkg) - [Windows](#windows-1) - [Homebrew](#homebrew) - [Ubuntu / Debian](#ubuntu--debian) - [Python](#python) - [Building Halide](#building-halide) - [Basic build](#basic-build) - [Windows](#windows-2) - [macOS and Linux](#macos-and-linux) - [CMake Presets](#cmake-presets) - [Common presets](#common-presets) - [Vcpkg presets](#vcpkg-presets) - [Sanitizer presets](#sanitizer-presets) - [Build options](#build-options) - [Installing](#installing) - [Building Halide with pip](#building-halide-with-pip) - [Using ccache with pip builds](#using-ccache-with-pip-builds) # Installing CMake This section covers installing a recent version of CMake and the correct dependencies for building and using Halide. If you have not used CMake before, we strongly suggest reading through the [CMake documentation][cmake-docs] first. Halide requires at least version 3.28. Fortunately, getting a recent version of CMake couldn't be easier, and there are multiple good options on any system to do so. Generally, one should always have the most recent version of CMake installed system-wide. CMake is committed to backwards compatibility and even the most recent release can build projects over a decade old. ## Cross-platform Kitware provides packages for CMake on [PyPI][pypi-cmake] which can be installed via `pip` into a [virtual environment][venv]. There are binary wheels available for nearly all relevant platforms, including: | OS | x86-32 | x86-64 | ARM64 | | ----------------- | ------------------ | ------------------ | -------------------------- | | Windows | :white_check_mark: | :white_check_mark: | :white_check_mark: | | macOS | :x: | 10.10+ | 11.0+ (incl. `universal2`) | | Linux (musl 1.1+) | :white_check_mark: | :white_check_mark: | :white_check_mark: | | Linux (glibc) | glibc 2.12+ | glibc 2.12+ | glibc 2.17+ | We recommend installing CMake using [pipx] to avoid package conflicts and redundant installations. After installing pipx, run: ```shell $ pipx install cmake ``` Alternatively, you can use a normal virtual environment: ```shell $ python -m pip install cmake ``` If you don't want Python to manage your CMake installation, you can either follow the platform-specific instructions below or install CMake from [Kitware's binary releases][cmake-download]. If all else fails, you might need to build CMake from source (e.g. on 32-bit ARM). In that case, follow the directions posted on [Kitware's website][cmake-from-source]. ## Windows On Windows, there are two primary methods for installing an up-to-date CMake: 1. You can get CMake through the Visual Studio 2022 installer. 2. You can use Windows's built-in package manager, [winget]: ```shell winget install Kitware.CMake ``` We prefer the first option for its simplicity. See Microsoft's [documentation][vs-cmake-docs] for more details. ## macOS [Homebrew] keeps its [CMake package][brew-cmake] up to date. Simply run: ```shell $ brew install cmake ``` ## Ubuntu Linux There are a few good ways to install CMake on Ubuntu: 1. If you're running 24.04 LTS, then simply running `sudo apt install cmake` will install CMake 3.28. 2. If you're running an older LTS or would like to use the newest CMake, try installing via the [snap store]: `snap install cmake`. Note this will conflict with an APT-provided CMake. 3. Kitware also provides an [APT repository][cmake-apt] with up-to-date releases. Compatible with 20.04 LTS+ and is the best option for 32-bit ARM. For other Linux distributions, check with your distribution's package manager. **Note:** On WSL 1, snap is not available; in this case, prefer to use APT. On WSL 2, all methods are available. ## Optional: Install Ninja We strongly recommend using [Ninja] as your go-to CMake generator for working with Halide. It has a much richer dependency structure than the alternatives, and it is the only generator capable of producing accurate incremental builds. It is available in most package repositories: - Python: `pipx install ninja` - Visual Studio Installer: alongside CMake - winget: `winget install Ninja-build.Ninja` - Homebrew: `brew install ninja` - APT: `apt install ninja-build` You can also place a [pre-built binary][ninja-download] from their website in the PATH. # Dependencies ## Summary The following is a complete list of required and optional dependencies for building the core pieces of Halide. | Dependency | Version | Required when... | Notes | | ------------- | ------------------ | -------------------------- | --------------------------------------------------- | | [LLVM] | _see policy below_ | _always_ | WebAssembly and X86 targets are required. | | [Clang] | `==LLVM` | _always_ | | | [LLD] | `==LLVM` | _always_ | | | [flatbuffers] | `~=23.5.26` | `WITH_SERIALIZATION=ON` | | | [wabt] | `==1.0.39` | `Halide_WASM_BACKEND=wabt` | Does not have a stable API; exact version required. | | [V8] | trunk | `Halide_WASM_BACKEND=V8` | Difficult to build. See [WebAssembly.md] | | [Python] | `>=3.10` | `WITH_PYTHON_BINDINGS=ON` | | | [pybind11] | `~=2.11.1` | `WITH_PYTHON_BINDINGS=ON` | | Halide maintains the following compatibility policy with LLVM: Halide version `N` supports LLVM versions `N`, `N-1`, and `N-2`. Our binary distributions always include the latest `N` patch at time of release. For most users, we recommend using a pre-packaged binary release of LLVM rather than trying to build it yourself. To build the apps, documentation, and tests, an extended set is needed. | Dependency | Required when... | Notes | | ------------------------------- | --------------------------------- | --------------------------------------------------------------------------- | | [CUDA Toolkit][findcudatoolkit] | building `apps/cuda_mat_mul` | When compiling Halide pipelines that use CUDA, only the drivers are needed. | | [Doxygen][finddoxygen] | `WITH_DOCS=ON` | | | [Eigen3][eigen3cmake] | building `apps/linear_algebra` | | | [libjpeg][findjpeg] | `WITH_TESTS=ON` | Optionally used by `halide_image_io.h` and `Halide::ImageIO` in CMake. | | [libpng][findpng] | `WITH_TESTS=ON` | (same as libjpeg) | | [BLAS][findblas] | building `apps/linear_algebra` | [ATLAS] and [OpenBLAS] are supported implementations | | [OpenCL][findopencl] | compiling pipelines with `opencl` | | It is best practice to configure your environment so that CMake can find dependencies without package-specific hints. For instance, if you want CMake to use a particular version of Python, create a virtual environment and activate it _before_ configuring Halide. Similarly, the `CMAKE_PREFIX_PATH` variable can be set to a local directory where from-source dependencies have been installed. Carefully consult the [find_package] documentation to learn how the search procedure works. If the build still fails to find a dependency, each package provides a bespoke interface for providing hints and overriding incorrect results. Documentation for these packages is linked in the table above. ## Installing dependencies ### vcpkg Halide has first-class support for using [vcpkg] to manage dependencies. The list of dependencies and features is contained inside `vcpkg.json` at the root of the repository. LLVM and Python must be provided by the system; vcpkg handles the remaining dependencies (flatbuffers, wabt, pybind11, libjpeg, libpng, etc.). Halide includes a `vcpkg-configuration.json` file that automatically configures [overlay ports][vcpkg-overlay] and overlay triplets. The overlay ports redirect LLVM and Python to system installations, preventing vcpkg from trying to build them. This configuration is applied automatically when vcpkg is used from the Halide source tree. When using the vcpkg toolchain file, you can set `-DVCPKG_MANIFEST_FEATURES=developer` to enable building all test dependencies (except Doxygen, which is not available on vcpkg). For convenience, we provide [CMake presets](#cmake-presets) that set these flags appropriately per-platform. They are documented further below. ### Windows On Windows, we recommend using `vcpkg` to install library dependencies. To build the documentation, you will need to install [Doxygen]. This can be done either from the [Doxygen website][doxygen-download] or through [winget]: ```shell $ winget install DimitriVanHeesch.Doxygen ``` To build the Python bindings, you will need to install Python 3. This should be done by running the official installer from the [Python website][python]. Be sure to download the debugging symbols through the installer. This will require using the "Advanced Installation" workflow. Although it is not strictly necessary, it is convenient to install Python system-wide on Windows (i.e. `C:\Program Files`) because CMake looks at standard paths and registry keys. This removes the need to manually set the `PATH`. Once Python is installed, you can install the Python module dependencies in a [virtual environment][venv] by running ```shell $ uv sync ``` from the root of the repository. ### Homebrew On macOS, it is possible to install all dependencies via [Homebrew]: ```shell $ brew install llvm flatbuffers wabt python pybind11 doxygen eigen libpng libjpeg-turbo openblas ``` The `llvm` package includes `clang`, `clang-format`, and `lld`, too. To ensure CMake can find LLVM, set the following cache variable: ```shell $ cmake ... -DHalide_ROOT=/opt/homebrew/opt/llvm ``` Or use the `macOS` CMake preset, which does this for you. ### Ubuntu / Debian On Ubuntu you should install the following packages (this includes the Python module dependencies): ``` $ sudo apt install clang-tools lld llvm-dev libclang-dev liblld-dev \ libpng-dev libjpeg-dev libgl-dev python3-dev python3-numpy \ python3-imageio python3-pybind11 libopenblas-dev libeigen3-dev \ libatlas-base-dev doxygen ``` ### Python When running the Python package, you will need to install additional dependencies. These are tabulated as constraints in `pyproject.toml` and resolved to specific versions in `uv.lock`. They may be installed by running: ```shell $ uv sync --no-install-project ``` # Building Halide ## Basic build These instructions assume that your working directory is the Halide repository root. ### Windows If you plan to use the Ninja generator, be sure to launch the developer command prompt corresponding to your intended environment. Note that whatever your intended target system (x86, x64, or ARM), you must use the 64-bit _host tools_ because the 32-bit tools run out of memory during the linking step with LLVM. More information is available from [Microsoft's documentation][msvc-cmd]. You should either open the correct Developer Command Prompt directly or run the [`vcvarsall.bat`][vcvarsall] script with the correct argument, i.e. one of the following: ```shell $ "C:\Program Files (x86)\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvarsall.bat" x64 $ "C:\Program Files (x86)\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvarsall.bat" x64_x86 $ "C:\Program Files (x86)\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvarsall.bat" x64_arm ``` Then, assuming that vcpkg is installed to `D:\vcpkg`, simply run: ```shell $ cmake -G Ninja -S . -B build --toolchain D:\vcpkg\scripts\buildsystems\vcpkg.cmake -DCMAKE_BUILD_TYPE=Release $ cmake --build .\build ``` Valid values of [`CMAKE_BUILD_TYPE`][cmake_build_type] are `Debug`, `RelWithDebInfo`, `MinSizeRel`, and `Release`. When using a single-configuration generator (like Ninja) you must specify a build type in the configuration step. Otherwise, if you wish to create a Visual Studio based build system, you can configure with: ```shell $ cmake -G "Visual Studio 17 2022" -Thost=x64 -A x64 -S . -B build ^ --toolchain D:\vcpkg\scripts\buildsystems\vcpkg.cmake $ cmake --build .\build --config Release -j %NUMBER_OF_PROCESSORS% ``` Because the Visual Studio generator is a _multi-config generator_, you don't set `CMAKE_BUILD_TYPE` at configure-time, but instead pass the configuration to the build (and test/install) commands with the `--config` flag. More documentation is available in the [CMake User Interaction Guide][cmake-user-interaction]. The process is similar for 32-bit: ``` > cmake -G "Visual Studio 17 2022" -Thost=x64 -A Win32 -S . -B build ^ --toolchain D:\vcpkg\scripts\buildsystems\vcpkg.cmake > cmake --build .\build --config Release -j %NUMBER_OF_PROCESSORS% ``` In both cases, the `-Thost=x64` flag ensures that the correct host tools are used. **Note:** due to limitations in MSBuild, incremental builds using the VS generators will miss dependencies (including changes to headers in the `src/runtime` folder). We recommend using Ninja for day-to-day development and use Visual Studio only if you need it for packaging. ### macOS and Linux The instructions here are straightforward. Assuming your environment is set up correctly, just run: ```shell $ cmake -G Ninja -S . -B build -DCMAKE_BUILD_TYPE=Release $ cmake --build build ``` If you omit `-G Ninja`, a Makefile-based generator will likely be used instead. In either case, [`CMAKE_BUILD_TYPE`][cmake_build_type] must be set to one of the standard types: `Debug`, `RelWithDebInfo`, `MinSizeRel`, or `Release`. ## CMake Presets Halide provides several [presets][cmake_presets] to make the above commands more convenient. ### Common presets These presets do not use vcpkg. They assume that all dependencies are available via the system (e.g. Homebrew on macOS, APT on Linux). ```shell $ cmake --preset=macOS # Ninja generator, macOS host build, Homebrew deps $ cmake --preset=debug # Debug mode, any single-config generator / compiler $ cmake --preset=release # Release mode, any single-config generator / compiler ``` ### Vcpkg presets The following presets use vcpkg to manage non-LLVM dependencies. LLVM and Python must be provided by the system. | Preset | Description | | --------------- | --------------------------------------------- | | `win32` | Visual Studio 2022 generator, 32-bit build | | `win64` | Visual Studio 2022 generator, 64-bit build | | `macOS-vcpkg` | macOS build with vcpkg + Homebrew LLVM | | `debug-vcpkg` | Debug build for any single-config generator | | `release-vcpkg` | Release build for any single-config generator | ### Sanitizer presets There are also presets to use some Clang sanitizers with the CMake build; at present, only Fuzzer and ASAN (Address Sanitizer) are supported, and only on linux-x86-64. - `linux-x64-asan`: Use the Address Sanitizer - `linux-x64-fuzzer`: Use the Clang fuzzer plugin To use these, you must build LLVM with additional options: ``` -DLLVM_ENABLE_PROJECTS="clang;lld;clang-tools-extra" -DLLVM_ENABLE_RUNTIMES="compiler-rt;libcxx;libcxxabi;libunwind" ``` ### Valgrind and Intel SDE presets These presets rerun the ordinary test suite under an external tool, using CTest's native mechanisms. The `valgrind` preset reruns every registered test under [valgrind] via CTest's built-in [MemCheck][ctest_memcheck] action. It configures a Debug build with `Halide_ENABLE_MEMCHECK=ON`, which pulls in the [CTest module][ctest_module] so that `ctest -T memcheck` works directly in the build tree (no CDash/dashboard submission required): ```shell $ cmake --preset valgrind $ cmake --build --preset valgrind $ ctest --preset valgrind -T memcheck ``` The `valgrind` test preset excludes the `no_memcheck` label. Valgrind (and its options and suppressions file) is configured via the standard `MEMORYCHECK_COMMAND`, `MEMORYCHECK_COMMAND_OPTIONS`, and `MEMORYCHECK_SUPPRESSIONS_FILE` (`test/valgrind.supp`) cache variables. The `avx512-cannonlake` and `avx512-knights-landing` presets rerun every test under [Intel SDE][intel_sde] to emulate an AVX-512-capable CPU on a host that lacks one. They set [`CMAKE_CROSSCOMPILING_EMULATOR`][cmake_crosscompiling_emulator] to `sde -cnl --` and `sde -knl --` respectively, so every `add_test`-registered executable runs under the emulator automatically. `sde` must be on your `PATH`: ```shell $ cmake --preset avx512-cannonlake $ cmake --build --preset avx512-cannonlake $ ctest --preset avx512-cannonlake ``` ## Build options Halide reads and understands several options that can configure the build. The following are the most consequential and control how Halide is actually compiled. | Option | Default | Description | | ---------------------------------------- | --------------------- | ------------------------------------------------------------------------------------------------------------------------------- | | [`BUILD_SHARED_LIBS`][build_shared_libs] | `ON` | Standard CMake variable that chooses whether to build as a static or shared library. | | `Halide_LLVM_SHARED_LIBS` | `OFF` | Link to the shared version of LLVM. Not available on Windows. | | `Halide_ENABLE_RTTI` | _inherited from LLVM_ | Enable RTTI when building Halide. Recommended to be set to `ON` | | `Halide_ENABLE_EXCEPTIONS` | `ON` | Enable exceptions when building Halide | | `Halide_TARGET` | _empty_ | The default target triple to use for `add_halide_library` (and the generator tests, by extension) | | `WITH_AUTOSCHEDULERS` | `ON` | Enable building the autoschedulers. Requires `BUILD_SHARED_LIBS`. | | `WITH_SERIALIZATION` | `ON` | Include experimental Serialization/Deserialization features; also required for the [generator compile cache](GeneratorCache.md) | The following options are disabled by default when building Halide through the [ `add_subdirectory`][add_subdirectory] or [`FetchContent`][fetchcontent] mechanisms. They control whether non-essential targets (like tests and documentation) are built. | Option | Default | Description | | ---------------------- | ------- | ---------------------------------------------------------------- | | `WITH_DOCS` | `OFF` | Enable building the documentation via Doxygen | | `WITH_PACKAGING` | `ON` | Include the `install()` rules for Halide. | | `WITH_PYTHON_BINDINGS` | `ON` | Enable building Python 3 bindings | | `WITH_TESTS` | `ON` | Enable building unit and integration tests | | `WITH_TUTORIALS` | `ON` | Enable building the tutorials | | `WITH_UTILS` | `ON` | Enable building various utilities including the trace visualizer | The following options are _advanced_ and should not be required in typical workflows. Generally, these are used by Halide's own CI infrastructure, or as escape hatches for third-party packagers. | Option | Default | Description | | --------------------------- | ------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------- | | `Halide_CCACHE_BUILD` | `OFF` | Use ccache with Halide-recommended settings to accelerate rebuilds. | | `Halide_ENABLE_MEMCHECK` | `OFF` | Pull in the CTest module so `ctest -T memcheck` reruns the tests under valgrind. See the `valgrind` preset. | | `Halide_CCACHE_PARAMS` | `CCACHE_CPP2=yes CCACHE_HASHDIR=yes CCACHE_SLOPPINESS=pch_defines` | Options to pass to `ccache` when using `Halide_CCACHE_BUILD`. | | `Halide_VERSION_OVERRIDE` | `${Halide_VERSION}` | Override the VERSION for libHalide. | | `Halide_SOVERSION_OVERRIDE` | `${Halide_VERSION_MAJOR}` | Override the SOVERSION for libHalide. Expects a positive integer (i.e. not a version). | The following options control whether to build certain test subsets. They only apply when `WITH_TESTS=ON`: | Option | Default | Description | | ------------------------- | ---------- | ------------------------------------- | | `WITH_TEST_AUTO_SCHEDULE` | `ON` | enable the auto-scheduling tests | | `WITH_TEST_CORRECTNESS` | `ON` | enable the correctness tests | | `WITH_TEST_ERROR` | `ON` | enable the expected-error tests | | `WITH_TEST_FUZZ` | _detected_ | enable the libfuzzer-based fuzz tests | | `WITH_TEST_GENERATOR` | `ON` | enable the AOT generator tests | | `WITH_TEST_PERFORMANCE` | `ON` | enable performance testing | | `WITH_TEST_RUNTIME` | `ON` | enable testing the runtime modules | | `WITH_TEST_WARNING` | `ON` | enable the expected-warning tests | The following option selects the execution engine for in-process WASM testing: | Option | Default | Description | | --------------------- | ------- | ---------------------------------------------------------------------------------------- | | `Halide_WASM_BACKEND` | `wabt` | Select the backend for WASM testing. Can be `wabt`, `V8` or a false value such as `OFF`. | ## Installing Once built, Halide will need to be installed somewhere before using it in a separate project. On any platform, this means running the [`cmake --install`][cmake-install] command in one of two ways. For a single-configuration generator (like Ninja), run either: ```shell $ cmake --install ./build --prefix /path/to/Halide-install $ cmake --install .\build --prefix X:\path\to\Halide-install ``` For a multi-configuration generator (like Visual Studio) run: ```shell $ cmake --install ./build --prefix /path/to/Halide-install --config Release $ cmake --install .\build --prefix X:\path\to\Halide-install --config Release ``` Of course, make sure that you build the corresponding config before attempting to install it. # Building Halide with pip Halide also supports installation via the standard Python packaging workflow. Running `pip install .` at the root of the repository will build a wheel and install it into the currently active Python environment. However, this comes with a few caveats: 1. The environment must be prepared for CMake to find its dependencies. This is easiest to do by setting either `CMAKE_PREFIX_PATH` to pre-built dependencies or by setting `CMAKE_TOOLCHAIN_FILE` to vcpkg. 2. The build settings are fixed, meaning that `wabt` is required on non-Windows systems, `flatbuffers` is always required, and the Python bindings must be built. 3. The generated wheel will likely only work on your system. In particular, it will not be repaired with `auditwheel` or `delocate`. Even so, this is a very good method of installing Halide. It supports both Python and C++ `find_package` workflows. ## Using ccache with pip builds Because Python's build infrastructure creates temporary CMake build directories, simply setting `CMAKE_CXX_COMPILER_LAUNCHER` to `ccache` is insufficient to produce a well-cached build. The following settings should serve as a starting point to configure your environment (assuming `$PWD` is the repository root) for using `ccache` with `pip install .`. ```shell # Point CMake to ccache export CMAKE_C_COMPILER_LAUNCHER=ccache export CMAKE_CXX_COMPILER_LAUNCHER=ccache # Settings to make ccache try to ignore the build directory export CCACHE_BASEDIR=$PWD export CCACHE_NOHASHDIR=true # Enable caching of pre-compiled headers and rewrite debug paths # -Xclang -fno-pch-timestamp is only necessary when using Clang (not GCC) export CCACHE_SLOPPINESS=include_file_ctime,include_file_mtime,pch_defines,time_macros export CFLAGS="-Xclang -fno-pch-timestamp -fdebug-prefix-map=$PWD=." export CXXFLAGS="$CFLAGS" # Locate the temporary build beneath $PWD so that CCACHE_BASEDIR works export TMPDIR=$PWD/build/tmp # If using uv, don't create a temporary venv export UV_NO_BUILD_ISOLATION=1 ``` See the CCache documentation on [compiling in different directories] and on using [precompiled headers] for more information about these settings. To check that ccache is working, run, ```shell $ uv pip install . # first run, populate cache Resolved 4 packages in 397ms Built halide @ file:///Users/areinking/dev/Halide Prepared 1 package in 29.17s Installed 1 package in 8ms + halide==20.0.0.dev87+gf6c939fd3.d20250724 (from file:///Users/areinking/dev/Halide) $ ccache -z Statistics zeroed $ uv pip install . # second run, reload from cache Resolved 4 packages in 338ms Built halide @ file:///Users/areinking/dev/Halide Prepared 1 package in 10.82s Uninstalled 1 package in 7ms Installed 1 package in 6ms ~ halide==20.0.0.dev87+gf6c939fd3.d20250724 (from file:///Users/areinking/dev/Halide) $ ccache -s Cacheable calls: 1079 / 1080 (99.91%) Hits: 1079 / 1079 (100.0%) Direct: 1079 / 1079 (100.0%) Preprocessed: 0 / 1079 ( 0.00%) Misses: 0 / 1079 ( 0.00%) Uncacheable calls: 1 / 1080 ( 0.09%) Local storage: Cache size (GB): 2.2 / 30.0 ( 7.24%) Hits: 1079 / 1079 (100.0%) Misses: 0 / 1079 ( 0.00%) ``` On this test system (an M3 MacBook Pro), the build is three times faster, with a 100% cache hit rate! [add_subdirectory]: https://cmake.org/cmake/help/latest/command/add_subdirectory.html [atlas]: http://math-atlas.sourceforge.net/ [brew-cmake]: https://formulae.brew.sh/cask/cmake#default [build_shared_libs]: https://cmake.org/cmake/help/latest/variable/BUILD_SHARED_LIBS.html [clang]: https://clang.llvm.org [cmake-apt]: https://apt.kitware.com/ [cmake-docs]: https://cmake.org/cmake/help/latest/ [cmake-download]: https://cmake.org/download/ [cmake-from-source]: https://cmake.org/install/ [cmake-install]: https://cmake.org/cmake/help/latest/manual/cmake.1.html#install-a-project [cmake-user-interaction]: https://cmake.org/cmake/help/latest/guide/user-interaction/index.html#setting-build-variables [cmake_build_type]: https://cmake.org/cmake/help/latest/variable/CMAKE_BUILD_TYPE.html [cmake_crosscompiling_emulator]: https://cmake.org/cmake/help/latest/variable/CMAKE_CROSSCOMPILING_EMULATOR.html [cmake_presets]: https://cmake.org/cmake/help/latest/manual/cmake-presets.7.html [codestylecmake.md]: ./CodeStyleCMake.md [compiling in different directories]: https://ccache.dev/manual/4.11.3.html#_compiling_in_different_directories [ctest_memcheck]: https://cmake.org/cmake/help/latest/manual/ctest.1.html#dashboard-client-steps [ctest_module]: https://cmake.org/cmake/help/latest/module/CTest.html [doxygen]: https://www.doxygen.nl/index.html [doxygen-download]: https://www.doxygen.nl/download.html [eigen3cmake]: https://eigen.tuxfamily.org/dox/TopicCMakeGuide.html [fetchcontent]: https://cmake.org/cmake/help/latest/module/FetchContent.html [findblas]: https://cmake.org/cmake/help/latest/module/FindBLAS.html [findcudatoolkit]: https://cmake.org/cmake/help/latest/module/FindCUDAToolkit.html [finddoxygen]: https://cmake.org/cmake/help/latest/module/FindDoxygen.html [findjpeg]: https://cmake.org/cmake/help/latest/module/FindJPEG.html [findopencl]: https://cmake.org/cmake/help/latest/module/FindOpenCL.html [findpng]: https://cmake.org/cmake/help/latest/module/FindPNG.html [find_package]: https://cmake.org/cmake/help/latest/command/find_package.html [flatbuffers]: https://github.com/google/flatbuffers [halidecmakepackage.md]: ./HalideCMakePackage.md [homebrew]: https://brew.sh [intel_sde]: https://www.intel.com/content/www/us/en/developer/articles/tool/software-development-emulator.html [lld]: https://lld.llvm.org/ [llvm]: https://github.com/llvm/llvm-project [msvc-cmd]: https://learn.microsoft.com/en-us/cpp/build/building-on-the-command-line [ninja]: https://ninja-build.org/ [ninja-download]: https://github.com/ninja-build/ninja/releases [openblas]: https://www.openblas.net/ [pipx]: https://pipx.pypa.io/stable/ [precompiled headers]: https://ccache.dev/manual/4.11.3.html#_precompiled_headers [pybind11]: https://github.com/pybind/pybind11 [pypi-cmake]: https://pypi.org/project/cmake/ [python]: https://www.python.org/downloads/ [snap store]: https://snapcraft.io/cmake [v8]: https://v8.dev [valgrind]: https://valgrind.org/ [vcpkg]: https://github.com/Microsoft/vcpkg [vcpkg-overlay]: https://learn.microsoft.com/en-us/vcpkg/concepts/overlay-ports [vcvarsall]: https://docs.microsoft.com/en-us/cpp/build/building-on-the-command-line#developer_command_file_locations [venv]: https://docs.python.org/3/tutorial/venv.html [vs-cmake-docs]: https://docs.microsoft.com/en-us/cpp/build/cmake-projects-in-visual-studio [wabt]: https://github.com/WebAssembly/wabt [webassembly.md]: ./WebAssembly.md [winget]: https://learn.microsoft.com/en-us/windows/package-manager/winget/ --- ### Doc/CMakeLists find_package(Doxygen REQUIRED doxygen) # Halide-specific Doxygen options set(DOXYGEN_ALPHABETICAL_INDEX NO) set(DOXYGEN_BUILTIN_STL_SUPPORT YES) set(DOXYGEN_CASE_SENSE_NAMES NO) set(DOXYGEN_CLASS_DIAGRAMS NO) set(DOXYGEN_DISTRIBUTE_GROUP_DOC YES) set(DOXYGEN_EXAMPLE_PATH "${Halide_SOURCE_DIR}/tutorial") set(DOXYGEN_EXCLUDE bin) set(DOXYGEN_EXCLUDE_PATTERNS README.md) set(DOXYGEN_EXTRACT_ALL YES) set(DOXYGEN_EXTRACT_LOCAL_CLASSES NO) set(DOXYGEN_FILE_PATTERNS *.h *.md) set(DOXYGEN_GENERATE_TREEVIEW YES) set(DOXYGEN_HIDE_FRIEND_COMPOUNDS YES) set(DOXYGEN_HIDE_IN_BODY_DOCS YES) set(DOXYGEN_HIDE_UNDOC_CLASSES YES) set(DOXYGEN_HIDE_UNDOC_MEMBERS YES) set(DOXYGEN_JAVADOC_AUTOBRIEF YES) set(DOXYGEN_MARKDOWN_ID_STYLE GITHUB) set(DOXYGEN_QT_AUTOBRIEF YES) set(DOXYGEN_QUIET YES) set(DOXYGEN_RECURSIVE YES) set(DOXYGEN_REFERENCED_BY_RELATION YES) set(DOXYGEN_REFERENCES_RELATION YES) set(DOXYGEN_SORT_BY_SCOPE_NAME YES) set(DOXYGEN_SORT_MEMBER_DOCS NO) set(DOXYGEN_SOURCE_BROWSER YES) set(DOXYGEN_STRIP_CODE_COMMENTS NO) doxygen_add_docs( doc "doc/" "src/" "test/" ALL WORKING_DIRECTORY "${Halide_SOURCE_DIR}" COMMENT "Generate HTML documentation" ) --- ### Doc/CodeStyleCMake # Contributing CMake code to Halide This document specifies the coding standards we adhere to when authoring new CMake code. If you need directions for building Halide, see [BuildingHalideWithCMake.md]. If you are looking for Halide's CMake package documentation, see [HalideCMakePackage.md]. This document is necessary for two major reasons. First, due to its long history, size, and dedication to backwards compatibility, CMake is _incredibly_ difficult to learn and full of traps. Second, Halide bundles its own LLVM-based native code generator, which CMake deeply does not expect. This means we routinely push CMake's build model to its limit. Therefore, we must be careful to write high-quality CMake code so that it is clear when CMake's limitations are being tested. While not comprehensive, the guide outlines the code quality expectations we have as they apply to CMake. When contributing new CMake code to Halide, keep in mind that the minimum version is 3.28. Therefore, it is not only possible, but _required_, to use modern CMake best practices. - [Contributing CMake code to Halide](#contributing-cmake-code-to-halide) - [General guidelines and best practices](#general-guidelines-and-best-practices) - [Prohibited modules list](#prohibited-modules-list) - [FetchContent](#fetchcontent) - [Prohibited commands list](#prohibited-commands-list) - [Prohibited variables list](#prohibited-variables-list) - [Adding tests](#adding-tests) - [Adding apps](#adding-apps) # General guidelines and best practices The following are some common mistakes that lead to subtly broken builds. - **Reading the build directory.** While setting up the build, the build directory should be considered _write only_. Using the build directory as a read/write temporary directory is acceptable as long as all temp files are cleaned up by the end of configuration. - **Not using [generator expressions][cmake-genex].** Declarative is better than imperative and this is no exception. Conditionally adding to a target property can leak unwanted details about the build environment into packages. Some information is not accurate or available except via generator expressions, e.g. the build configuration. - **Using the wrong variable.** `CMAKE_SOURCE_DIR` doesn't always point to the Halide source root. When someone uses Halide via [`FetchContent`][fetchcontent], it will point to _their_ source root instead. The correct variable is [`Halide_SOURCE_DIR`][project-name_source_dir]. If you want to know if the compiler is MSVC, check it directly with the [`MSVC`][msvc] variable; don't use [`WIN32`][win32]. That will be wrong when compiling with clang on Windows. In most cases, however, a generator expression will be more appropriate. - **Using directory properties.** Directory properties have vexing behavior and are essentially deprecated from CMake 3.0+. Propagating target properties is the way of the future. - **Using the wrong visibility.** Target properties can be `PRIVATE`, `INTERFACE`, or both (aka `PUBLIC`). Pick the most conservative one for each scenario. Refer to the [transitive usage requirements][cmake-propagation] docs for more information. - **Needlessly expanding variables** The [`if`][cmake_if] and [`foreach`][cmake_foreach] commands generally expand variables when provided by name. Expanding such variables manually can unintentionally change the behavior of the command. Use `foreach (item IN LISTS list)` instead of `foreach (item ${list})`. Similarly, use `if (varA STREQUAL varB)` instead of `if ("${varA}" STREQUAL "${varB}")` and _definitely_ don't use `if (${varA} STREQUAL ${varB})` since that will fail (in the best case) if either variable's value contains a semicolon (due to argument expansion). ## Prohibited modules list All deprecated, legacy, and "miscellaneous" (internal) modules are prohibited. The list of these may be found in the upstream documentation: https://cmake.org/cmake/help/latest/manual/cmake-modules.7.html#deprecated-modules ### FetchContent At the moment, only one supported module is prohibited: `FetchContent`. There are many reasons to avoid its use: 01. It brings third-party CMake code into the build, which can cause all sorts of issues. In the common case, third-party projects hard-code incompatible build settings, which are tricky to work around in CMake. In the worst case, they can set cache variables or directory properties that break the including project's build. Worse still, those cache variables persist in `CMakeCache.txt` even after the dependency is removed or replaced, so the only reliable fix is a clean reconfigure. Because configuration is expected to be idempotent, these failures can be difficult to diagnose. 02. It is a poor fit for cross-compilation scenarios that require separate host and target artifacts. FetchContent inlines the dependency's project into the including build, so it is configured with the same toolchain as the rest of that build. For instance, a project may need both the flatbuffers compiler for the host system and the flatbuffers library for the target system. This scenario is not supported by FetchContent's population model. 03. It performs network access at configure time. This makes air-gapped and offline builds awkward. `FETCHCONTENT_FULLY_DISCONNECTED=ON` only works after a successful first configure and adds latency to every fresh configure. Source pinning is also weak: only commit SHAs are truly immutable, branch and tag refs can be moved server-side, and `URL_HASH` is opt-in. Package managers like vcpkg require hash-pinned archives by default and produce a baseline that can be locked. 04. It does not maintain a persistent source or binary cache outside the build tree. Populated sources and build products live under the build directory by default, so deleting the build directory also deletes them. A fresh build can therefore require another download and rebuild of dependencies, rather than just rebuilding the top-level project. This also couples dependency iteration to the parent project: tweaking a dependency's options forces a parent reconfigure, and the dependency cannot be built or tested in isolation. 05. The above issues exacerbate diamond dependency problems. Even if a consistent version happens to be chosen, different intermediate dependencies along each branch might impose incompatible build settings. For instance, one project might try to enable an optional feature while another project disables it. 06. Applying local fixes to dependencies is awkward. `FetchContent_Declare`'s `PATCH_COMMAND` runs an arbitrary shell snippet that is hard to review, version, or attribute. vcpkg ports keep patches as versioned `.patch` files alongside the portfile, so they appear in code review and survive upstream version bumps cleanly. 07. It pollutes the cache and target environment, even when steps are taken to exclude test and utility targets. This clutters both graphical IDE interfaces and the diagnostic output of build tools like Ninja (e.g. its dependency graph and build profiler). 08. Targets created by FetchContent are considered _first-party_ targets, meaning that special care must be taken when writing installation and packaging rules. This complexity compounds when simultaneously supporting other dependency resolution mechanisms that create third-party (i.e. `IMPORTED`) targets. 09. It produces no provenance, license, or SBOM metadata. Package managers like vcpkg and Conan emit machine-readable manifests of versions, licenses, and source hashes that compliance tooling can consume. FetchContent emits nothing, so every audit becomes a manual exercise. 10. FetchContent requests can be intercepted by a Dependency Provider which can only be chosen by the top-level project. That means code that appears to vendor a specific source tree can instead be redirected to some other dependency resolution mechanism, such as a package manager. This makes the resulting targets and build settings less predictable, and it compounds the first-party versus imported-target packaging issues described above. After broader approval, third-party dependencies must be consumed with `find_package`. This also lets packagers and distributors substitute a system or pre-built copy, rather than forcing every downstream to rebuild dependencies from source. We use vcpkg in CI to manage our dependencies. If vcpkg lacks a port, you must write a custom port in `cmake/vcpkg-ports` (for the main Halide build) or `apps/vcpkg/ports` (for the apps). ## Prohibited commands list As mentioned above, using directory properties is brittle, and they are therefore _not allowed_. The following functions may not appear in any new CMake code. | Command | Alternative | | ----------------------------------- | -------------------------------------------------------------------------------------------------- | | `add_compile_definitions` | Use [`target_compile_definitions`][target_compile_definitions] | | `add_compile_options` | Use [`target_compile_options`][target_compile_options] | | `add_definitions` | Use [`target_compile_definitions`][target_compile_definitions] | | `add_link_options` | Use [`target_link_options`][target_link_options], but prefer not to use either | | `include_directories` | Use [`target_include_directories`][target_include_directories] | | `link_directories` | Use [`target_link_libraries`][target_link_libraries] | | `link_libraries` | Use [`target_link_libraries`][target_link_libraries] | | `remove_definitions` | [Generator expressions][cmake-genex] in [`target_compile_definitions`][target_compile_definitions] | | `set_directory_properties` | Use (cache) variables or target properties | | `set_property(DIRECTORY)` | Use (cache) variables or target properties (custom properties excluded, but require justification) | | `target_link_libraries(target lib)` | Use [`target_link_libraries`][target_link_libraries] _with a visibility specifier_ (eg. `PRIVATE`) | As an example, it was once common practice to write code similar to this: ```cmake # WRONG: do not do this include_directories(include) add_library(my_lib source1.cpp ..) ``` However, this has two major pitfalls. First, it applies to _all_ targets created in that directory, even those before the call to `include_directories` and those created in [`include()`][include]-ed CMake files. As CMake files get larger and more complex, this behavior gets harder to pinpoint. This is particularly vexing when using the `link_libraries` or `add_definitions` commands. Second, this form does not provide a way to _propagate_ the include directory to consumers of `my_lib`. The correct way to do this is: ```cmake # CORRECT add_library(my_lib source1.cpp ...) target_sources( my_lib PUBLIC FILE_SET HEADERS BASE_DIRS include FILES include/header1.h ) ``` This is better in many ways. It only affects the target in question. It propagates the include path to the targets linking to it (via `PUBLIC`). It also correctly exports the host-filesystem-specific include path when installing or packaging the target and installs the headers themselves, too. If common properties need to be grouped together, use an INTERFACE target (better) or write a function (worse). There are also several functions that are disallowed for other reasons: | Command | Reason | Alternative | | ------------------------------- | ----------------------------------------------------------- | ------------------------------------------------------------------------------ | | `aux_source_directory` | Interacts poorly with incremental builds and Git | List source files explicitly | | `build_command` | CTest internal function | Use CTest build-and-test mode via [`CMAKE_CTEST_COMMAND`][cmake_ctest_command] | | `cmake_host_system_information` | Usually misleading information. | Inspect [toolchain][cmake-toolchains] variables and use generator expressions. | | `cmake_policy(... OLD)` | OLD policies are deprecated by definition. | Instead, fix the code to work with the new policy. | | `create_test_sourcelist` | We use our own unit testing solution | See the [adding tests](#adding-tests) section. | | `define_property` | Adds unnecessary complexity | Use a cache variable. Exceptions under special circumstances. | | `enable_language` | Halide is C/C++ only | [`FindCUDAToolkit`][findcudatoolkit], appropriately guarded. | | `file(GLOB ...)` | Interacts poorly with incremental builds and Git | List source files explicitly. Allowed if not globbing for source files. | | `fltk_wrap_ui` | Halide does not use FLTK | None | | `include_external_msproject` | Halide must remain portable | Write a CMake package config file or find module. | | `include_guard` | Use of recursive inclusion is not allowed | Write (recursive) functions. | | `include_regular_expression` | Changes default dependency checking behavior | None | | `load_cache` | Superseded by [`ExternalProject`][externalproject] | Write a vcpkg port or present a case for an exception. | | `macro` | CMake macros are not hygienic and are therefore error-prone | Use functions instead. | | `site_name` | Privacy: do not want leak host name information | Provide a cache variable, generate a unique name. | | `variable_watch` | Debugging helper | None. Not needed in production. | Do not introduce new dependencies without broader approval. Once approved, add dependencies to `vcpkg.json` or create a custom port, and consume them with [`find_package`][find_package] rather than `FetchContent`. ## Prohibited variables list Any variables that are specific to languages that are not enabled should, of course, be avoided. But of greater concern are variables that are easy to misuse or should not be overridden for our end-users. The following (non-exhaustive) list of variables shall not be used in code merged into main. | Variable | Reason | Alternative | | ------------------------------- | --------------------------------------------- | ------------------------------------------------------------------------------------------------------- | | `CMAKE_ROOT` | Code smell | Rely on `find_package` search options; include `HINTS` if necessary | | `CMAKE_DEBUG_TARGET_PROPERTIES` | Debugging helper | None | | `CMAKE_FIND_DEBUG_MODE` | Debugging helper | None | | `CMAKE_RULE_MESSAGES` | Debugging helper | None | | `CMAKE_VERBOSE_MAKEFILE` | Debugging helper | None | | `CMAKE_BACKWARDS_COMPATIBILITY` | Deprecated | None | | `CMAKE_BUILD_TOOL` | Deprecated | `${CMAKE_COMMAND} --build` or [`CMAKE_MAKE_PROGRAM`][cmake_make_program] (but see below) | | `CMAKE_CACHEFILE_DIR` | Deprecated | [`CMAKE_BINARY_DIR`][cmake_binary_dir], but see below | | `CMAKE_CFG_INTDIR` | Deprecated | `$`, `$`, target resolution of [`add_custom_command`][add_custom_command], etc. | | `CMAKE_CL_64` | Deprecated | [`CMAKE_SIZEOF_VOID_P`][cmake_sizeof_void_p] | | `CMAKE_COMPILER_IS_*` | Deprecated | [`CMAKE__COMPILER_ID`][cmake_lang_compiler_id] | | `CMAKE_HOME_DIRECTORY` | Deprecated | [`CMAKE_SOURCE_DIR`][cmake_source_dir], but see below | | `CMAKE_DIRECTORY_LABELS` | Directory property | None | | `CMAKE_BUILD_TYPE` | Only applies to single-config generators. | `$` | | `CMAKE_*_FLAGS*` (w/o `_INIT`) | User-only | Write a [toolchain][cmake-toolchains] file with the corresponding `_INIT` variable | | `CMAKE_COLOR_MAKEFILE` | User-only | None | | `CMAKE_ERROR_DEPRECATED` | User-only | None | | `CMAKE_CONFIGURATION_TYPES` | We only support the four standard build types | None | Of course feel free to insert debugging helpers _while developing_ but please remove them before review. Finally, the following variables are allowed, but their use must be motivated: | Variable | Reason | Alternative | | ---------------------------------------------- | --------------------------------------------------- | -------------------------------------------------------------------------------------------- | | [`CMAKE_SOURCE_DIR`][cmake_source_dir] | Points to global source root, not Halide's. | [`Halide_SOURCE_DIR`][project-name_source_dir] or [`PROJECT_SOURCE_DIR`][project_source_dir] | | [`CMAKE_BINARY_DIR`][cmake_binary_dir] | Points to global build root, not Halide's | [`Halide_BINARY_DIR`][project-name_binary_dir] or [`PROJECT_BINARY_DIR`][project_binary_dir] | | [`CMAKE_MAKE_PROGRAM`][cmake_make_program] | CMake abstracts over differences in the build tool. | Prefer CTest's build and test mode or CMake's `--build` mode | | [`CMAKE_CROSSCOMPILING`][cmake_crosscompiling] | Often misleading. | Inspect relevant variables directly, eg. [`CMAKE_SYSTEM_NAME`][cmake_system_name] | | [`BUILD_SHARED_LIBS`][build_shared_libs] | Could override user setting | None, but be careful to restore value when overriding for a dependency | Any use of these functions or variables will block a PR. # Adding tests When adding a file to any of the folders under `test`, be aware that CI expects that every `.c` and `.cpp` appears in the `CMakeLists.txt` file _on its own line_, possibly as a comment. This is to avoid globbing and also to ensure that added files are not missed. For most test types, it should be as simple as adding to the existing lists. Generator tests are trickier, but following the existing examples is a safe way to go. # Adding apps If you're contributing a new app to Halide: great! Thank you! There are a few guidelines you should follow when writing a new app. - Write the app as if it were a top-level project. You should call `find_package(Halide)` and set the C++ version to 11. - Call [`enable_testing()`][enable_testing] and add a small test that runs the app. - Don't assume your app will have access to a GPU. Write your schedules to be robust to varying buildbot hardware. - Don't assume your app will be run on a specific OS, architecture, or bitness. Write your apps to be robust (ideally efficient) on all supported platforms. - If you rely on any additional packages, don't include them as `REQUIRED`, instead test to see if their targets are available and, if not, call `return()` before creating any targets. In this case, print a `message(STATUS "[SKIP] ...")`, too. - Look at the existing apps for examples. - Test your app with ctest before opening a PR. Apps are built as part of the test, rather than the main build. [add_custom_command]: https://cmake.org/cmake/help/latest/command/add_custom_command.html [buildinghalidewithcmake.md]: ./BuildingHalideWithCMake.md [build_shared_libs]: https://cmake.org/cmake/help/latest/variable/BUILD_SHARED_LIBS.html [cmake-genex]: https://cmake.org/cmake/help/latest/manual/cmake-generator-expressions.7.html [cmake-propagation]: https://cmake.org/cmake/help/latest/manual/cmake-buildsystem.7.html#transitive-usage-requirements [cmake-toolchains]: https://cmake.org/cmake/help/latest/manual/cmake-toolchains.7.html [cmake_binary_dir]: https://cmake.org/cmake/help/latest/variable/CMAKE_BINARY_DIR.html [cmake_crosscompiling]: https://cmake.org/cmake/help/latest/variable/CMAKE_CROSSCOMPILING.html [cmake_ctest_command]: https://cmake.org/cmake/help/latest/variable/CMAKE_CTEST_COMMAND.html [cmake_foreach]: https://cmake.org/cmake/help/latest/command/foreach.html [cmake_if]: https://cmake.org/cmake/help/latest/command/if.html [cmake_lang_compiler_id]: https://cmake.org/cmake/help/latest/variable/CMAKE_LANG_COMPILER_ID.html [cmake_make_program]: https://cmake.org/cmake/help/latest/variable/CMAKE_MAKE_PROGRAM.html [cmake_sizeof_void_p]: https://cmake.org/cmake/help/latest/variable/CMAKE_SIZEOF_VOID_P.html [cmake_source_dir]: https://cmake.org/cmake/help/latest/variable/CMAKE_SOURCE_DIR.html [cmake_system_name]: https://cmake.org/cmake/help/latest/variable/CMAKE_SYSTEM_NAME.html [enable_testing]: https://cmake.org/cmake/help/latest/command/enable_testing.html [externalproject]: https://cmake.org/cmake/help/latest/module/ExternalProject.html [fetchcontent]: https://cmake.org/cmake/help/latest/module/FetchContent.html [findcudatoolkit]: https://cmake.org/cmake/help/latest/module/FindCUDAToolkit.html [find_package]: https://cmake.org/cmake/help/latest/command/find_package.html [halidecmakepackage.md]: ./HalideCMakePackage.md [include]: https://cmake.org/cmake/help/latest/command/include.html [msvc]: https://cmake.org/cmake/help/latest/variable/MSVC.html [project-name_binary_dir]: https://cmake.org/cmake/help/latest/variable/PROJECT-NAME_BINARY_DIR.html [project-name_source_dir]: https://cmake.org/cmake/help/latest/variable/PROJECT-NAME_SOURCE_DIR.html [project_binary_dir]: https://cmake.org/cmake/help/latest/variable/PROJECT_BINARY_DIR.html [project_source_dir]: https://cmake.org/cmake/help/latest/variable/PROJECT_SOURCE_DIR.html [target_compile_definitions]: https://cmake.org/cmake/help/latest/command/target_compile_definitions.html [target_compile_options]: https://cmake.org/cmake/help/latest/command/target_compile_options.html [target_include_directories]: https://cmake.org/cmake/help/latest/command/target_include_directories.html [target_link_libraries]: https://cmake.org/cmake/help/latest/command/target_link_libraries.html [target_link_options]: https://cmake.org/cmake/help/latest/command/target_link_options.html [win32]: https://cmake.org/cmake/help/latest/variable/WIN32.html --- ### Doc/CustomRuntimes # Custom Runtime Prefixes Every Halide runtime exposes a set C ABI symbols consisting of common host functions -- `halide_malloc`, `halide_free`, `halide_error`, `halide_do_par_for`, and so on -- as well as target specific methods which handle device functionality -- `halide_cuda_*`, `halide_direct3dcompute_*`, `halide_metal_*`, `halide_opencl_*`, `halide_vulkan_*`, etc. This C ABI also consists of mutable process-global state stored as global variables (e.g. the installed custom allocator, the thread pool, the memoization cache, the profiler, etc.). This works well when a program contains a single Halide runtime, but it becomes a problem when a program must contain *more than one*. Two independently produced components -- say, two libraries that each embed their own AOT-compiled Halide pipelines and runtime -- both define `halide_malloc` and both carry the same runtime state globals. When they are linked into one process the linker collapses those duplicate (weak/`linkonce`) symbols into a single copy, so the two components silently *share* one runtime. Installing a custom allocator or error handler for one then affects the other, and the two cannot be given different runtime configurations at all. Runtime prefixes solve this by letting you rename the runtime's symbols with a prefix of your choosing, so that each component carries its own, independent runtime. This document describes the feature, the scopes it exposes, and how to use it from C++, from the `GenGen` command line, and from CMake. ## Scopes Rather than a single prefix, three independent prefixes are available, one per "visibility" of a runtime symbol. They correspond to the enum `Halide::RuntimeLinkage`: - **Import** -- the names a *generated kernel* uses to call into the runtime. When a pipeline is compiled with `no_runtime`, its calls to `halide_malloc` and friends are left as external references; the import prefix renames those references so they resolve against a matching prefixed runtime at link time. - **Export** -- the names a *runtime library* makes externally visible. When you compile a standalone runtime, the export prefix renames the public C ABI it defines (`halide_malloc` becomes, e.g., `my_prefix_malloc`). - **Internal** -- the names used *within* the runtime library. This covers the runtime's own C++ symbols in the `Halide::Runtime::Internal` namespace, including the mutable state globals. Renaming these is what actually keeps two prefixed runtimes' state independent; without it the state globals would still collide even if the public ABI were renamed. Each prefix is optional and they are set independently. A prefix replaces the leading `halide_` of the C ABI names; because the internal C++ symbols contain no `halide_` to replace, the internal prefix is prepended to them. The pipeline's own entry points (the function you called `compile_to_*` on, its `_argv` wrapper, and its metadata) are never renamed, and neither are C library symbols. ## How the pieces fit together For a component to link and run, the prefixes of its kernel and its runtime must agree: - The kernel's **import** prefix must equal the runtime's **export** prefix, so the kernel's calls resolve to the runtime's definitions. - The kernel's **internal** prefix must equal the runtime's **internal** prefix, for the same reason applied to any internal symbols they share. Different components use *different* prefixes from one another; that is what keeps them isolated. A typical setup for two components `A` and `B` is: | Component | Runtime (`export`, `internal`) | Kernel (`import`, `internal`) | | --------- | ------------------------------ | ----------------------------- | | A | `A_`, `A_internal_` | `A_`, `A_internal_` | | B | `B_`, `B_internal_` | `B_`, `B_internal_` | When A and B are linked into one process, `A_malloc` and `B_malloc` (and their respective state globals) are distinct symbols, so each pipeline uses its own runtime and their state stays independent. ## Backends Both the LLVM and the C backend honor runtime prefixes. - The **LLVM backend** renames the symbols directly on the generated module: a definition takes the export prefix, a kernel-called external declaration takes the import prefix, and the runtime's internal C++ symbols (including its state globals) take the internal prefix. - The **C backend** emits a kernel that calls into an external runtime, so only the import prefix applies to it. It renames the runtime's C ABI functions with a block of `#define halide_x x` at the top of the generated source; the preprocessor rewrites the runtime's function declarations and every call site consistently, while leaving types (`halide_buffer_t`), typedefs (`halide_malloc_t`), and enum values untouched. These `#define`s are emitted only into the generated C/C++ *source*, never the header, so several prefixed headers can still be included together. ## Limitations - Runtime prefixes are not currently supported for JIT, but it's something we would like to support in the future. The difficulty is that the JIT resolves runtime calls against a single process-global shared runtime which already exists. So, for now, requesting runtime prefixes on a JIT target will emit an error. ## Usage from C++ The prefixes are described by a `Halide::RuntimePrefixParams`, which wraps a `std::map`. To compile a **standalone runtime** with a set of prefixes, pass the map to `compile_standalone_runtime`: ```c++ #include "Halide.h" using namespace Halide; Target target = get_host_target(); std::map ns = { {RuntimeLinkage::Export, "my_prefix_"}, {RuntimeLinkage::Internal, "my_prefix_internal_"}, }; compile_standalone_runtime("my_prefix_runtime.o", target, ns); ``` To compile a **pipeline** whose runtime calls match that runtime, apply the matching prefixes and compile with `no_runtime`: ```c++ Func consumer = /* ... */; Pipeline p(consumer); Target target = get_host_target().with_feature(Target::NoRuntime); p.apply_runtime_prefixes(target, RuntimePrefixParams({ {RuntimeLinkage::Import, "my_prefix_"}, {RuntimeLinkage::Internal, "my_prefix_internal_"}, })); p.compile_to_module({}, "my_pipeline", target) .compile({{OutputFileType::object, "my_pipeline.o"}, {OutputFileType::c_header, "my_pipeline.h"}}); ``` `apply_runtime_prefixes` records the prefixes on the pipeline; any subsequent `compile_to_*` for a non-JIT target then applies them. Calling it with a JIT target raises a `Halide::CompileError`. Inside a `Generator`, the prefixes travel on the `GeneratorContext` as `RuntimePrefixParams` and are applied automatically when the generator's module is built; in practice these are supplied through the command line or CMake, described below. ## Usage from the GenGen command line The prefixes are ordinary generator parameters named `runtime_prefixes.import`, `runtime_prefixes.export`, and `runtime_prefixes.internal`. Any of them may be omitted. To emit a prefixed **standalone runtime** (the `-r` output): ``` ./my_generator -r my_prefix_runtime -o . -e object \ target=host \ runtime_prefixes.export=my_prefix_ \ runtime_prefixes.internal=my_prefix_internal_ ``` To emit a matching **pipeline** with `no_runtime`: ``` ./my_generator -g my_generator -f my_pipeline -o . -e object,c_header \ target=host-no_runtime \ runtime_prefixes.import=my_prefix_ \ runtime_prefixes.internal=my_prefix_internal_ ``` ## Usage from CMake `add_halide_runtime` accepts a `PARAMS` argument that is forwarded to the runtime generator, and `add_halide_library` already forwards `PARAMS` to the pipeline generator. Give a runtime its export/internal prefixes, and give each library its matching import/internal prefixes together with `USE_RUNTIME`: ```cmake add_halide_generator(my_pipeline.generator SOURCES my_pipeline_generator.cpp) # A runtime with the "my_prefix_" prefix. add_halide_runtime( my_prefix_runtime PARAMS runtime_prefixes.export=my_prefix_ runtime_prefixes.internal=my_prefix_internal_ ) # A pipeline that links against it. add_halide_library() compiles with # no_runtime automatically when USE_RUNTIME is given. add_halide_library( my_pipeline FROM my_pipeline.generator GENERATOR my_pipeline USE_RUNTIME my_prefix_runtime PARAMS runtime_prefixes.import=my_prefix_ runtime_prefixes.internal=my_prefix_internal_ ) ``` Repeating this with a second, differently-prefixed runtime and library produces two components that can be linked into the same program without their runtimes colliding. For a complete, working example -- three variants of one pipeline, each with its own runtime, linked into a single test that checks their state stays independent -- see `test/generator/runtime_prefixes_iso_aottest.cpp` and its CMake wiring in `test/generator/CMakeLists.txt`. ## Verifying the result The renaming happens on the symbols of the emitted object, so you can confirm it with `nm`. A stock runtime exports `halide_malloc`: ``` $ nm my_prefix_runtime.o | grep malloc 0000000000000000 T my_prefix_malloc ``` and its internal state globals are prefixed as well: ``` $ nm my_prefix_runtime.o | grep custom_malloc 0000000000000000 D my_prefix_internal__ZN6Halide7Runtime8Internal13custom_mallocE ``` A `no_runtime` pipeline object correspondingly imports the renamed symbols rather than the stock `halide_` ones: ``` $ nm my_pipeline.o | grep malloc U my_prefix_malloc ``` --- ### Doc/FuzzTesting # Fuzz testing Halide has a set of fuzz-testing harnesses in `test/fuzz/` that can find tricky edge cases and bugs that are hard to catch with a regular unit-testing suite. The fuzz tests are built on a small in-tree framework (`fuzz_helpers.h`, `halide_fuzz_main.h`) that abstracts over two backends: - **stdlib backend** — uses `std::mt19937_64` seeded from `std::random_device`. Works with any standard C++ toolchain; no special compiler flags or external runtime libraries required. Each run prints its seed so failures are reproducible by re-running with that seed. - **libfuzzer backend** — uses [libFuzzer](https://www.llvm.org/docs/LibFuzzer.html) for coverage-guided fuzzing. Requires a Clang toolchain built with `-fsanitize=fuzzer` support. The stdlib backend is the default for regular development builds. The libfuzzer backend is enabled automatically when the build system detects `-fsanitize=fuzzer` (or equivalent) linker flags. ## Building fuzz tests ### Standard build (stdlib backend) No special flags are needed. The fuzz tests build as part of any normal CMake configuration that has `WITH_TEST_FUZZ=YES`: ``` cmake -B build -DWITH_TEST_FUZZ=YES cmake --build build -j$(nproc) --target test_fuzz ``` ### libfuzzer backend (coverage-guided fuzzing) Use one of the fuzzing CMake presets, which set the necessary `-fsanitize=fuzzer[-no-link]` flags across the entire build: **Linux:** ``` cmake -B build --preset linux-x64-fuzzer -DHalide_LLVM_ROOT=/path/to/llvm-install cmake --build build -j$(nproc) ``` **macOS (Homebrew LLVM):** ``` cmake -B build --preset macOS-fuzz cmake --build build -j$(nproc) ``` The LLVM install used for libfuzzer builds must include the `compiler-rt` runtime (i.e. built with `-DLLVM_ENABLE_RUNTIMES="compiler-rt"`). Not all prebuilt LLVM installs include this; you may need to build LLVM from source or use Homebrew's LLVM package on macOS. ## Running fuzz tests ### stdlib backend Run a fuzz harness directly: ``` ./build/test/fuzz/fuzz_simplify ``` By default this runs 10,000 iterations, printing the seed before each one: ``` Seed: 12345678901234567 Seed: 98765432109876543 ... ``` Control the number of iterations with `-runs=N`: ``` ./build/test/fuzz/fuzz_simplify -runs=100000 ``` Run all fuzz tests via CTest (1,000 iterations each, exit-code–based pass/fail): ``` ctest --test-dir build -L fuzz ``` ### libfuzzer backend After building with a fuzzing preset, run the harness with no arguments to start coverage-guided fuzzing on a single core: ``` ./build/test/fuzz/fuzz_simplify ``` To persist the corpus between runs (recommended): ``` mkdir -p fuzz_simplify_corpus ./build/test/fuzz/fuzz_simplify fuzz_simplify_corpus ``` To fuzz in parallel across all available cores: ``` ./build/test/fuzz/fuzz_simplify fuzz_simplify_corpus -fork=$(nproc) ``` ## Reproducing failures ### stdlib backend When a run fails, rerun with the seed that was printed just before the crash: ``` ./build/test/fuzz/fuzz_simplify 12345678901234567 ``` This performs a single deterministic iteration with that seed. ### libfuzzer backend libFuzzer writes a crash-input file on failure: ``` crash- ``` Replay it by passing it as the first argument: ``` ./build/test/fuzz/fuzz_simplify crash- ``` ## Adding new fuzz tests All fuzz tests use the `FUZZ_TEST` macro defined in `fuzz_helpers.h`. This macro generates the correct entry point for whichever backend is active — `LLVMFuzzerTestOneInput` for libfuzzer or a `main` that calls `Halide::fuzz_main` for the stdlib backend. A minimal fuzz test looks like this: ```cpp #include "fuzz_helpers.h" FUZZ_TEST(my_test, Halide::FuzzingContext &fuzz) { int x = fuzz.ConsumeIntegralInRange(0, 100); bool b = fuzz.ConsumeBool(); my_function(x, b); return 0; } ``` `FuzzingContext` wraps `FuzzedDataProvider` (from libfuzzer's `compiler-rt/include/fuzzer/FuzzedDataProvider.h`) and re-implements its interface on top of `std::mt19937_64` for the stdlib backend, so the same API works with both backends. Key methods: - `ConsumeIntegral()` — random value of type `T` - `ConsumeIntegralInRange(min, max)` — random value in `[min, max]` - `ConsumeBool()` — random boolean - `PickValueInArray(arr)` — random element from an array or initializer list - `PickValueInVector(vec)` — random element from a `std::vector` For richer examples, see `test/fuzz/simplify.cpp` and `test/fuzz/random_expr_generator.h`. To register a new fuzz test with CMake, add it to the `SOURCES` list in `test/fuzz/CMakeLists.txt`. ## Other useful materials - [The official libfuzzer docs](https://www.llvm.org/docs/LibFuzzer.html) - [The libfuzzer tutorial](https://github.com/google/fuzzing/blob/master/tutorial/libFuzzerTutorial.md) - [FuzzedDataProvider reference](https://github.com/llvm/llvm-project/blob/main/compiler-rt/include/fuzzer/FuzzedDataProvider.h) --- ### Doc/GeneratorCache # Generator compile cache ## Overview Building a large app suite runs one generator process per (generator, target) pair, and most of that time is spent recompiling pipelines that haven't actually changed since the last build. The generator compile cache is an opt-in, content-addressed cache that lets `execute_generator` skip that work: when the inputs to a generator invocation are unchanged, the previously emitted artifacts (object files, headers, static libraries, etc.) are copied back into place instead of being recompiled. The cache is inert unless you opt in by setting the `HL_CACHE_DIR` environment variable (or the corresponding `Halide_CACHE_DIR` CMake variable, see below) to a directory. When it is unset, none of this machinery runs and build output is identical to a Halide with no cache support at all. The feature also requires that Halide was built with `WITH_SERIALIZATION=ON` (the default), since the cache key is only sound if it includes a serialized snapshot of the pipeline being compiled. If `HL_CACHE_DIR` is set but serialization support isn't compiled in, Halide prints a one-time warning and disables caching for that build. ## What gets cached Both compilation paths that `execute_generator` can take are cached independently: - Compiling a generator (the `compile_multitarget` path), once per `-g`/`-o`/`target=...` invocation. - Compiling the standalone runtime (the `-r`/GenRT path). ## Cache key A cache entry is addressed by a SHA-256 digest that mixes in everything that can affect the emitted files, so that two invocations agreeing on all of the following are guaranteed to produce identical outputs: - **Compiler identity**: a fingerprint of the running libHalide (or, if statically linked, the generator executable itself). This is normally the linker-assigned build ID, read directly from the already-loaded image (Mach-O `LC_UUID` on macOS, ELF `.note.gnu.build-id` on Linux, the PE debug directory's CodeView/PDB GUID on Windows) so that rebuilding Halide invalidates the cache without hashing the ~32 MB binary on every run. If no build ID is available (e.g. `--build-id=none`, or a Windows image built without debug info), it falls back to hashing the whole binary. - **The generator name, output types, build mode** (normal vs. gradient), and the fully-resolved **generator-param settings** (`target=` is handled separately, below). - **The target(s) and per-target suffixes** being compiled. - **The serialized pipeline** for each target: the generator is instantiated and its (pre-autoschedule) algorithm and schedule are serialized via `Serialization.h` and folded into the key. This is what makes the key sound for source edits — the actual algorithm and schedule are captured, so editing a generator's `generate()`/`schedule()` invalidates the cache even when none of its command-line params changed. Autoscheduling itself is deliberately excluded so that computing a cache key never requires running a (potentially expensive) autoscheduler. - **The contents of any `-p` plugins** (e.g. autoschedulers), since they affect codegen but live outside libHalide. If the compiler identity can't be determined, or the pipeline can't be serialized (for example, a generator that throws while building), the invocation falls back to compiling normally without touching the cache. ## Using it from the command line Set `HL_CACHE_DIR` before invoking a generator executable directly: ```shell $ export HL_CACHE_DIR=$HOME/.cache/halide-generators $ ./my_generator -g my_pipeline -o . target=host ``` The first invocation compiles and populates the cache; subsequent invocations with the same generator, target, params, plugins, and (unchanged) pipeline source restore the outputs instead of recompiling. ## Using it from CMake The shipped CMake helpers (`add_halide_library`, `add_halide_runtime`, etc.) read the `Halide_CACHE_DIR` cache variable — which defaults to `$ENV{HL_CACHE_DIR}` — and, when it's set, wrap every generator/GenRT invocation so the value reaches the generator process: ```shell $ cmake -B build -DHalide_CACHE_DIR=$HOME/.cache/halide-generators $ cmake --build build ``` When `Halide_CACHE_DIR` is empty, the emitted build commands are byte-for-byte identical to a build with no cache configured. See [`Halide_CACHE_DIR`](HalideCMakePackage.md#variables) in the CMake package docs. ## Cache maintenance Entries are installed atomically (staged in a temp directory under the cache root, then renamed into place), so concurrent builds never observe a partial entry. The cache is pruned opportunistically after each store to honor two optional size/age limits, read from the environment: - `HL_CACHE_MAX_SIZE` — maximum total size of the cache, evicting the least-recently-used entries first once exceeded. Accepts a plain byte count or a `K`/`M`/`G` suffix (powers of 1024). Defaults to `1G`. - `HL_CACHE_MAX_AGE` — if set, entries not used within this many seconds are evicted regardless of total size. Because a large parallel build runs many generator processes that each store an entry, pruning is debounced to at most once every 60 seconds (via a `.last_prune` stamp in the cache directory), so the size limit is a soft cap that a burst of concurrent stores can briefly exceed. Pruning across processes is additionally serialized by a non-blocking advisory lock; if another process already holds it, a store simply skips its own prune pass rather than waiting. To clear the cache entirely, just delete the directory named by `HL_CACHE_DIR`. --- ### Doc/HalideCMakePackage # Using Halide from your CMake build This is a detailed guide to building your own Halide programs with the official CMake package. If you need directions for building Halide, see [BuildingHalideWithCMake.md]. If you are looking for Halide's CMake coding guidelines, see [CodeStyleCMake.md]. This document assumes some basic familiarity with CMake but tries to be explicit in all its examples. To learn more about CMake, consult the [documentation][cmake-docs] and engage with the community on the [CMake Discourse][cmake-discourse]. - [Using Halide from your CMake build](#using-halide-from-your-cmake-build) - [A basic CMake project](#a-basic-cmake-project) - [JIT mode](#jit-mode) - [AOT mode](#aot-mode) - [Autoschedulers](#autoschedulers) - [RunGenMain](#rungenmain) - [Halide package documentation](#halide-package-documentation) - [Components](#components) - [Variables](#variables) - [Imported targets](#imported-targets) - [Functions](#functions) - [`add_halide_generator`](#add_halide_generator) - [`add_halide_library`](#add_halide_library) - [`add_halide_python_extension_library`](#add_halide_python_extension_library) - [`add_halide_runtime`](#add_halide_runtime) - [Cross compiling](#cross-compiling) - [Use `add_halide_generator`](#use-add_halide_generator) - [Use a super-build](#use-a-super-build) - [Use `ExternalProject` directly](#use-externalproject-directly) - [Use an emulator or run on device](#use-an-emulator-or-run-on-device) - [Bypass CMake](#bypass-cmake) # A basic CMake project There are two main ways to use Halide in your application: as a **JIT compiler** for dynamic pipelines or an **ahead-of-time (AOT) compiler** for static pipelines. CMake provides robust support for both use cases. No matter how you intend to use Halide, you will need some basic CMake boilerplate. ```cmake cmake_minimum_required(VERSION 3.28) project(HalideExample) set(CMAKE_CXX_STANDARD 17) # or newer set(CMAKE_CXX_STANDARD_REQUIRED YES) set(CMAKE_CXX_EXTENSIONS NO) find_package(Halide REQUIRED) ``` The [`cmake_minimum_required`][cmake_minimum_required] command is required to be the first command executed in a CMake program. It disables all the deprecated behavior ("policies" in CMake lingo) from earlier versions. The [`project`][project] command sets the name of the project (and accepts arguments for versioning, language support, etc.) and is required by CMake to be called immediately after setting the minimum version. The next three variables set the project-wide C++ standard. The first, [`CMAKE_CXX_STANDARD`][cmake_cxx_standard], simply sets the standard version. Halide requires at least C++17. The second, [`CMAKE_CXX_STANDARD_REQUIRED`][cmake_cxx_standard_required], tells CMake to fail if the compiler cannot provide the requested standard version. Lastly, [`CMAKE_CXX_EXTENSIONS`][cmake_cxx_extensions] tells CMake to disable vendor-specific extensions to C++. This is not necessary to simply use Halide, but we do not allow such extensions in the Halide repo. Finally, we use [`find_package`][find_package] to locate Halide on your system. When using the pip package on Linux and macOS, CMake's `find_package` command should find Halide as long as you're in the same virtual environment you installed it in. On Windows, you will need to add the virtual environment root directory to [`CMAKE_PREFIX_PATH`][cmake_prefix_path]: ```shell $ cmake -G Ninja -S . -B build -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH=%VIRTUAL_ENV% ``` If `find_package` cannot find Halide, set `CMAKE_PREFIX_PATH` to the Halide installation directory. ## JIT mode To use Halide in JIT mode (like the [tutorials][halide-tutorials] do, for example), you can simply link to `Halide::Halide`. ```cmake # ... same project setup as before ... add_executable(my_halide_app main.cpp) target_link_libraries(my_halide_app PRIVATE Halide::Halide) ``` Then `Halide.h` will be available to your code and everything should just work. That's it! ## AOT mode Using Halide in AOT mode is more complicated so we'll walk through it step by step. Note that this only applies to Halide generators, so it might be useful to re-read the [tutorial on generators][halide-generator-tutorial]. Assume (like in the tutorial) that you have a source file named `my_generators.cpp` and that in it, you have generator classes `MyFirstGenerator` and `MySecondGenerator` with registered names `my_first_generator` and `my_second_generator` respectively. Then the first step is to add a **generator executable** to your build: ```cmake # ... same project setup as before ... add_halide_generator(my_generators SOURCES my_generators.cpp) ``` Using the generator executable, we can add a Halide library corresponding to `MyFirstGenerator`. ```cmake # ... continuing from above add_halide_library(my_first_generator FROM my_generators) ``` This will create a static library target in CMake that corresponds to the output of running your generator. The second generator in the file requires generator parameters to be passed to it. These are also easy to handle: ```cmake # ... continuing from above add_halide_library(my_second_generator FROM my_generators PARAMS parallel=false scale=3.0 rotation=ccw output.type=uint16) ``` Adding multiple configurations is easy, too: ```cmake # ... continuing from above add_halide_library(my_second_generator_2 FROM my_generators GENERATOR my_second_generator PARAMS scale=9.0 rotation=ccw output.type=float32) add_halide_library(my_second_generator_3 FROM my_generators GENERATOR my_second_generator PARAMS parallel=false output.type=float64) ``` Here, we had to specify which generator to use (`my_second_generator`) since it uses the target name by default. The functions in these libraries will be named after the target names, `my_second_generator_2` and `my_second_generator_3`, by default, but it is possible to control this via the `FUNCTION_NAME` parameter. Each one of these targets, ``, carries an associated `.runtime` target, which is also a static library containing the Halide runtime. It is transitively linked through `` to targets that link to ``. On an operating system like Linux, where weak linking is available, this is not an issue. However, on Windows, this can fail due to symbol redefinitions. In these cases, you must declare that two Halide libraries share a runtime, like so: ```cmake # ... updating above add_halide_library(my_second_generator_2 FROM my_generators GENERATOR my_second_generator USE_RUNTIME my_first_generator.runtime PARAMS scale=9.0 rotation=ccw output.type=float32) add_halide_library(my_second_generator_3 FROM my_generators GENERATOR my_second_generator USE_RUNTIME my_first_generator.runtime PARAMS parallel=false output.type=float64) ``` This will even work correctly when different combinations of targets are specified for each halide library. A "greatest common denominator" target will be chosen that is compatible with all of them (or the build will fail). ### Autoschedulers When the autoschedulers are included in the release package, they are very simple to apply to your own generators. For example, we could update the definition of the `my_first_generator` library above to use the `Adams2019` autoscheduler: ```cmake add_halide_library(my_second_generator FROM my_generators AUTOSCHEDULER Halide::Adams2019) ``` ### RunGenMain Halide provides a generic driver for generators to be used during development for benchmarking and debugging. Suppose you have a generator executable called `my_gen` and a generator within called `my_filter`. Then you can pass a variable name to the `REGISTRATION` parameter of `add_halide_library` which will contain the name of a generated C++ source that should be linked to `Halide::RunGenMain` and `my_filter`. For example: ```cmake add_halide_library(my_filter FROM my_gen REGISTRATION filter_reg_cpp) add_executable(runner ${filter_reg_cpp}) target_link_libraries(runner PRIVATE my_filter Halide::RunGenMain) ``` Then you can run, debug, and benchmark your generator through the `runner` executable. Learn how to interact with these executables in [RunGen.md](./RunGen.md). # Halide package documentation Halide provides a CMake _package configuration_ module. The intended way to use the CMake build is to run `find_package(Halide ...)` in your `CMakeLists.txt` file. Closely read the [`find_package` documentation][find_package] before proceeding. ## Components The Halide package script understands a handful of optional components when loading the package. First, if you plan to use the Halide Image IO library, you will want to include the `png` and `jpeg` components when loading Halide. Second, Halide releases can contain a variety of configurations: static, shared, debug, release, etc. CMake handles Debug/Release configurations automatically, but generally only allows one type of library to be loaded. The package understands two components, `static` and `shared`, that specify which type of library you would like to load. For example, if you want to make sure that you link against shared Halide, you can write: ```cmake find_package(Halide REQUIRED COMPONENTS shared) ``` If the shared libraries are not available, this will result in a failure. If no component is specified, then the `Halide_SHARED_LIBS` variable is checked. If it is defined and set to true, then the shared libraries will be loaded or the package loading will fail. Similarly, if it is defined and set to false, the static libraries will be loaded. If no component is specified and `Halide_SHARED_LIBS` is _not_ defined, then the [`BUILD_SHARED_LIBS`][build_shared_libs] variable will be inspected. If it is **not defined** or **defined and set to true**, then it will attempt to load the shared libs and fall back to the static libs if they are not available. Similarly, if `BUILD_SHARED_LIBS` is **defined and set to false**, then it will try the static libs first then fall back to the shared libs. To ensure that the Python bindings are available, include the `Python` component. Finally, when [cross compiling](#cross-compiling), `find_package(Halide REQUIRED)` never pulls in the compiled compiler/JIT library (`Halide::Halide`, `Halide::Generator`, etc.) unless you explicitly ask for it. Add the `JIT` component (or `Python`) to force it to be loaded even while cross-compiling: ```cmake find_package(Halide REQUIRED COMPONENTS JIT) ``` This looks for a `HalideCompiler` package matching your current (target) platform and fails with a normal `find_package` error if none is found. `HalideCompiler` is the name of the underlying platform-specific package that actually contains the compiled libraries; you generally don't need to `find_package(HalideCompiler)` directly, but its name is useful for `HalideCompiler_ROOT`/`-DHalideCompiler_DIR=...` when pointing CMake at a specific installed build. Autoscheduler plugins (see [Autoschedulers](#autoschedulers)) are resolved separately, via a `HalideAutoschedulers` package, and are always available to `add_halide_library(... AUTOSCHEDULER ...)` regardless of cross-compiling -- unlike `HalideCompiler`, this package's targets are never linked against (only dlopen()'d, by their build path, at Generator run time), so there's no reason to gate them behind the `JIT`/`Python` components. Note that `static`/`shared`, unlike `JIT`/`Python`, never force this load by themselves -- requesting one merely records your preference for whichever package eventually loads the compiled compiler (whether that's this same `find_package(Halide ...)` call, because you're not cross-compiling or also requested `JIT`/`Python`, or a later, unrelated one, such as the internal lookup `add_halide_generator` performs when it needs to build a generator). This preference is scoped to the current directory (and any subdirectories added after it), so independent parts of a project -- so long as neither is a subdirectory of the other -- can request different linkage without conflicting with each other. If one directory's `find_package` call ends up loading the compiled compiler before a subdirectory requests the other flavor, that's a real conflict (CMake can only load one flavor of `Halide::Halide` per directory scope) and fails cleanly with a descriptive error rather than silently keeping whichever flavor loaded first. ## Variables Variables that control package loading: | Variable | Description | | --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `Halide_SHARED_LIBS` | override `BUILD_SHARED_LIBS` when loading the Halide package via `find_package`. Has no effect when using Halide via `add_subdirectory` as a Git or `FetchContent` submodule. | | `Halide_RUNTIME_NO_THREADS` | skip linking of Threads library to runtime. Should be set if your toolchain does not support it (e.g. baremetal). | | `Halide_RUNTIME_NO_DL_LIBS` | skip linking of DL library to runtime. Should be set if your toolchain does not support it (e.g. baremetal). | Variables set by the package: | Variable | Description | | -------------------------- | ------------------------------------------------------------------ | | `Halide_VERSION` | The full version string of the loaded Halide package | | `Halide_VERSION_MAJOR` | The major version of the loaded Halide package | | `Halide_VERSION_MINOR` | The minor version of the loaded Halide package | | `Halide_VERSION_PATCH` | The patch version of the loaded Halide package | | `Halide_VERSION_TWEAK` | The tweak version of the loaded Halide package | | `Halide_HOST_TARGET` | The Halide target triple corresponding to "host" for this build. | | `Halide_CMAKE_TARGET` | The Halide target triple corresponding to the active CMake target. | | `Halide_ENABLE_EXCEPTIONS` | Whether Halide was compiled with exception support | | `Halide_ENABLE_RTTI` | Whether Halide was compiled with RTTI | | `WITH_AUTOSCHEDULERS` | Whether the autoschedulers are available | Variables that control package behavior: | Variable | Description | | ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `Halide_PYTHON_LAUNCHER` | Semicolon separated list containing a command to launch the Python interpreter. Can be used to set environment variables for Python generators. | | `Halide_NO_DEFAULT_FLAGS` | Off by default. When enabled, suppresses recommended compiler flags that would be added by `add_halide_generator` | | `Halide_CACHE_DIR` | Defaults to `$ENV{HL_CACHE_DIR}`. When set, routes `HL_CACHE_DIR` to every generator/GenRT invocation so it can reuse cached artifacts instead of recompiling. See [doc/GeneratorCache.md](GeneratorCache.md). | ## Imported targets Halide defines the following targets that are available to users: | Imported target | Description | | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `Halide::Halide` | this is the JIT-mode library to use when using Halide from C++. | | `Halide::Generator` | this is the target to use when manually defining a generator executable. It supplies a `main()` function. | | `Halide::Runtime` | adds include paths to the Halide runtime headers | | `Halide::Tools` | adds include paths to the Halide tools, including the benchmarking utility. | | `Halide::ImageIO` | adds include paths to the Halide image IO utility. Depends on `PNG::PNG` and `JPEG::JPEG` if they exist or were loaded through the corresponding package components. | | `Halide::ThreadPool` | adds include paths to the Halide _simple_ thread pool utility library. This is not the same as the runtime's thread pool and is intended only for use by tests. Depends on `Threads::Threads`. | | `Halide::RunGenMain` | used with the `REGISTRATION` parameter of `add_halide_library` to create simple runners and benchmarking tools for Halide libraries. | The following targets only guaranteed when requesting the `Python` component (`Halide_Python_FOUND` will be true): | Imported target | Description | | ---------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `Halide::Python` | this is a Python 3 package that can be referenced as `$/..` when setting up `PYTHONPATH` for Python tests or the like from CMake. | The following targets only guaranteed when `WITH_AUTOSCHEDULERS` is true: | Imported target | Description | | ----------------------- | --------------------------------------------------------------- | | `Halide::Adams2019` | the Adams et.al. 2019 autoscheduler (no GPU support) | | `Halide::Anderson2021` | the Anderson, et.al. 2021 autoscheduler (full GPU support) | | `Halide::Li2018` | the Li et.al. 2018 gradient autoscheduler (limited GPU support) | | `Halide::Mullapudi2016` | the Mullapudi et.al. 2016 autoscheduler (no GPU support) | These come from a separate `HalideAutoschedulers` package (distinct from `HalideCompiler`), loaded automatically the first time `add_halide_library(... AUTOSCHEDULER ...)` needs one -- including while cross-compiling. ## Functions The Halide package provides several useful functions for dealing with AOT compilation steps. ### `add_halide_generator` This function aids in creating cross-compilable builds that use Halide generators. ``` add_halide_generator( target [PACKAGE_NAME package-name] [PACKAGE_NAMESPACE namespace] [EXPORT_FILE export-file] [PYSTUB generator-name] [LINK_LIBRARIES lib1 ...] [[SOURCES] source1 ...] ) ``` Every named argument is optional, and the function uses the following default arguments: - If `PACKAGE_NAME` is not provided, it defaults to `${PROJECT_NAME}-halide_generators`. - If `PACKAGE_NAMESPACE` is not provided, it defaults to `${PROJECT_NAME}::halide_generators::`. - If `EXPORT_FILE` is not provided, it defaults to `${PROJECT_BINARY_DIR}/cmake/${ARG_PACKAGE_NAME}-config.cmake` This function guarantees that a Halide generator target named `` is available. It will first search for a package named `` using `find_package`; if it is found, it is assumed that it provides the target. Otherwise, it will create an executable target named `target` and an `ALIAS` target ``. This function also creates a custom target named `` if it does not exist and `` would exist. In this case, `` will depend on ``, this enables easy building of _just_ the Halide generators managed by this function. After the call, `_FOUND` will be set to true if the host generators were imported (and hence won't be built). Otherwise, it will be set to false. This variable may be used to conditionally set properties on ``. Please see [test/integration/xc](https://github.com/halide/Halide/tree/main/test/integration/xc) for a simple example and [apps/hannk](https://github.com/halide/Halide/tree/main/apps/hannk) for a complete app that uses it extensively. The `SOURCES` keyword marks the beginning of sources to be used to build ``, if it is not loaded. All unparsed arguments will be interpreted as sources. The `LINK_LIBRARIES` argument lists libraries that should be linked to `` when it is being built in the present build system. If `PYSTUB` is specified, then a Python Extension will be built that wraps the Generator with CPython glue to allow use of the Generator Python 3. The result will be a shared library of the form `_pystub..so`, where `` describes the specific Python version and platform (e.g., `cpython-310-darwin` for Python 3.10 on macOS). See [Python.md](Python.md) for examples of use. ### `add_halide_library` This is the main function for managing generators in AOT compilation. The full signature follows: ``` add_halide_library( FROM [GENERATOR generator-name] [FUNCTION_NAME function-name] [NAMESPACE cpp-namespace] [USE_RUNTIME hl-target] [PARAMS param1 [param2 ...]] [DEPENDS [dep1 dep2 ...]] [TARGETS target1 [target2 ...]] [FEATURES feature1 [feature2 ...]] [FEATURES[] feature1 [feature2 ...]] [PLUGINS plugin1 [plugin2 ...]] [AUTOSCHEDULER scheduler-name] [FUNCTION_INFO_HEADER OUTVAR] [HEADER OUTVAR] [REGISTRATION OUTVAR] [ OUTVAR] [GRADIENT_DESCENT] [C_BACKEND] [NO_THREADS] [NO_DL_LIBS]) triple = -- arch = x86 | arm | powerpc | hexagon | wasm | riscv bits = 32 | 64 os = linux | windows | osx | android | ios | qurt | noos | fuchsia | wasmrt extra-output = ASSEMBLY | BITCODE | COMPILER_LOG | C_SOURCE | FEATURIZATION | HLPIPE | LLVM_ASSEMBLY | PYTHON_EXTENSION | PYTORCH_WRAPPER | SCHEDULE | STMT | STMT_HTML ``` This function creates a called `` corresponding to running the `` (an executable target which links to `Halide::Generator`) one time, using command line arguments derived from the other parameters. The arguments `GENERATOR` and `FUNCTION_NAME` default to ``. They correspond to the `-g` and `-f` command line flags, respectively. `NAMESPACE` is syntactic sugar to specify the C++ namespace (if any) of the generated function; you can also specify the C++ namespace (if any) directly in the `FUNCTION_NAME` argument, but for repeated declarations or very long namespaces, specifying this separately can provide more readable build files. If `USE_RUNTIME` is not specified, this function will create another target called `.runtime` which corresponds to running the generator with `-r` and a compatible list of targets. This runtime target is an `INTERFACE` dependency of ``. If multiple runtime targets need to be linked together, setting `USE_RUNTIME` to another Halide runtime library, `` will prevent the generation of `.runtime` and instead use `.runtime`. This argument is most commonly used in conjunction with [ `add_halide_runtime`](#add_halide_runtime). Parameters can be passed to a generator via the `PARAMS` argument. Parameters should be space-separated. Similarly, `TARGETS` is a space-separated list of targets for which to generate code in a single function. They must all share the same platform/bits/os triple (e.g. `arm-32-linux`). Features that are in common among all targets, including device libraries (like `cuda`) should go in `FEATURES`. If `TARGETS` is not specified, the value of `Halide_TARGET` specified at configure time will be used. Every element of `TARGETS` must begin with the same `arch-bits-os` triple. This function understands two _meta-triples_, `host` and `cmake`. The meta-triple `host` is equal to the `arch-bits-os` triple used to compile Halide along with all the supported instruction set extensions. On platforms that support running both 32 and 64-bit programs, this will not necessarily equal the platform the compiler is running on or that CMake is targeting. The meta-triple `cmake` is equal to the `arch-bits-os` of the current CMake target. This is useful if you want to make sure you are not unintentionally cross-compiling, which would result in an [`IMPORTED` target][imported-target] being created. When `TARGETS` is empty and the `host` target would not cross-compile, then `host` will be used. Otherwise, `cmake` will be used and an author warning will be issued. When `CMAKE_OSX_ARCHITECTURES` is set and the `TARGETS` argument resolves to `cmake`, the generator will be run once for each architecture and the results will be fused together using `lipo`. This behavior extends to runtime targets. Sometimes, the generation will need to read files that were generated during the build. To declare dependencies on these files, use the `DEPENDS` argument. Paths listed here will be passed verbatim to `add_custom_command`, and so will be relative to the source directory. Use absolute paths when referring to files outside the source directory. To use an autoscheduler, set the `AUTOSCHEDULER` argument to a target named like `Namespace::Scheduler`, for example `Halide::Adams2019`. This will set the `autoscheduler` GeneratorParam on the generator command line to `Scheduler` and add the target to the list of plugins. Additional plugins can be loaded by setting the `PLUGINS` argument. If the argument to `AUTOSCHEDULER` does not contain `::` or it does not name a target, it will be passed to the `-s` flag verbatim. If `GRADIENT_DESCENT` is set, then the module will be built suitably for gradient descent calculation in TensorFlow or PyTorch. See `Generator::build_gradient_module()` for more documentation. This corresponds to passing `-d 1` at the generator command line. If the `C_BACKEND` option is set, this command will invoke the configured C++ compiler on a generated source. Note that a `.runtime` target is _not_ created in this case, and the `USE_RUNTIME` option is ignored. Other options work as expected. If `REGISTRATION` is set, the path (relative to `CMAKE_CURRENT_BINARY_DIR`) to the generated `.registration.cpp` file will be set in `OUTVAR`. This can be used to generate a runner for a Halide library that is useful for benchmarking and testing, as documented above. This is equivalent to setting `-e registration` at the generator command line. If `HEADER` is set, the path (relative to `CMAKE_CURRENT_BINARY_DIR`) to the generated `.h` header file will be set in `OUTVAR`. This can be used with `install(FILES)` to conveniently deploy the generated header along with your library. If `FUNCTION_INFO_HEADER` is set, the path (relative to `CMAKE_CURRENT_BINARY_DIR`) to the generated `.function_info.h` header file will be set in `OUTVAR`. This produces a file that contains `constexpr` descriptions of information about the generated functions (e.g., argument type and information). It is generated separately from the normal `HEADER` file because `HEADER` is intended to work with basic `extern "C"` linkage, while `FUNCTION_INFO_HEADER` requires C++17 or later to use effectively. (This can be quite useful for advanced usages, such as producing automatic call wrappers, etc.) Examples of usage can be found in the generated file. Each of the `extra-output` arguments directly correspond to an extra output (via `-e`) from the generator. The value `OUTVAR` names a variable into which a path (relative to [`CMAKE_CURRENT_BINARY_DIR`][cmake_current_binary_dir]) to the extra file will be written. When `NO_THREADS` is passed, the library targets will not depend on `Threads::Threads`. It is your responsibility to link to an equivalent target. When `NO_DL_LIBS` is passed, the library targets will not depend on `${CMAKE_DL_LIBS}`. It is your responsibility to link to an equivalent library. ### `add_halide_python_extension_library` This function wraps the outputs of one or more `add_halide_library` targets with glue code to produce a Python Extension library. ``` add_halide_python_extension_library( target [MODULE_NAME module-name] HALIDE_LIBRARIES library1 ... ) ``` `HALIDE_LIBRARIES` is a list of one of more `add_halide_library` targets. Each will be added to the extension as a callable method of the module. Note that every library specified must be built with the `PYTHON_EXTENSION` keyword specified, and all libraries must use the same Halide runtime. The result will be a shared library of the form `..so`, where `` describes the specific Python version and platform (e.g., `cpython-310-darwin` for Python 3.10 on macOS.) ### `add_halide_runtime` This function generates a library containing a Halide runtime. Most user code will never need to use this, as `add_halide_library()` will call it for you if necessary. The most common use case is usually in conjunction with `add_halide_python_extension_library()`, as a way to ensure that all the halide libraries share an identical runtime. ``` add_halide_runtime( target [TARGETS target1 [target2 ...]] [NO_THREADS] [NO_DL_LIBS] ) ``` The `TARGETS`, `NO_THREADS`, and `NO_DL_LIBS` arguments have identical semantics to the argument of the same name for [ `add_halide_library`](#add_halide_library). # Cross compiling Cross-compiling in CMake can be tricky, since CMake doesn't easily support compiling for both the host platform and the cross platform within the same build. Unfortunately, Halide generator executables are just about always designed to run on the host platform. Each project will be set up differently and have different requirements, but here are some suggestions for effective use of CMake in these scenarios. ## Use `add_halide_generator` If you are writing new programs that use Halide, you might wish to use `add_halide_generator`. When using this helper, you are expected to build your project twice: once for your build host and again for your intended target. On the target-side build, a plain `find_package(Halide REQUIRED)` is all you need (no separate package name): it never pulls in the compiled compiler, and `add_halide_generator` will lazily load it under the hood only if it can't find a prebuilt host generators package to import instead. When building the host build, you can use the `` (see the documentation above) target to build _just_ the generators. Then, in the target build, set `_ROOT` to the host build directory. For example: ``` $ cmake -G Ninja -S . -B build-host -DCMAKE_BUILD_TYPE=Release $ cmake --build build-host --target $ cmake -G Ninja -S . -B build-target --toolchain /path/to/target-tc.cmake \ -DCMAKE_BUILD_TYPE=Release \ -D_ROOT:FILEPATH=$PWD/build-host $ cmake --build build-target ``` ## Use a super-build A CMake super-build consists of breaking down a project into subprojects that are isolated by [toolchain][cmake-toolchains]. The basic structure is to have an outermost project that only coordinates the sub-builds via the [`ExternalProject`][externalproject] module. One would then use Halide to build a generator executable in one self-contained project, then export that target to be used in a separate project. The second project would be configured with the target [toolchain][cmake-toolchains] and would call `add_halide_library` with no `TARGETS` option and set `FROM` equal to the name of the imported generator executable. Obviously, this is a significant increase in complexity over a typical CMake project. This is very compatible with the `add_halide_generator` strategy above. ## Use `ExternalProject` directly A lighter weight alternative to the above is to use [`ExternalProject`][externalproject] directly in your parent build. Configure the parent build with the target [toolchain][cmake-toolchains], and configure the inner project to use the host toolchain. Then, manually create an [`IMPORTED` target][imported-executable] for your generator executable and call `add_halide_library` as described above. The main drawback of this approach is that creating accurate `IMPORTED` targets is difficult since predicting the names and locations of your binaries across all possible platform and CMake project generators is difficult. In particular, it is hard to predict executable extensions in cross-OS builds. ## Use an emulator or run on device The [`CMAKE_CROSSCOMPILING_EMULATOR`][cmake_crosscompiling_emulator] variable allows one to specify a command _prefix_ to run a target-system binary on the host machine. One could set this to a custom shell script that uploads the generator executable, runs it on the device and copies back the results. Another option is to install `qemu-user-static` to transparently emulate the cross-built generator. ## Bypass CMake The previous two options ensure that the targets generated by `add_halide_library` will be _normal_ static libraries. This approach does not use [`ExternalProject`][externalproject], but instead produces `IMPORTED` targets. The main drawback of `IMPORTED` targets is that they are considered second-class in CMake. In particular, they cannot be installed with the typical [`install(TARGETS)` command][install-targets]. Instead, they must be installed using [`install(FILES)`][install-files] and the [`$`][target-file] generator expression. [buildinghalidewithcmake.md]: ./BuildingHalideWithCMake.md [build_shared_libs]: https://cmake.org/cmake/help/latest/variable/BUILD_SHARED_LIBS.html [cmake-discourse]: https://discourse.cmake.org/ [cmake-docs]: https://cmake.org/cmake/help/latest/ [cmake-toolchains]: https://cmake.org/cmake/help/latest/manual/cmake-toolchains.7.html [cmake_crosscompiling_emulator]: https://cmake.org/cmake/help/latest/variable/CMAKE_CROSSCOMPILING_EMULATOR.html [cmake_current_binary_dir]: https://cmake.org/cmake/help/latest/variable/CMAKE_CURRENT_BINARY_DIR.html [cmake_cxx_extensions]: https://cmake.org/cmake/help/latest/variable/CMAKE_CXX_EXTENSIONS.html [cmake_cxx_standard]: https://cmake.org/cmake/help/latest/variable/CMAKE_CXX_STANDARD.html [cmake_cxx_standard_required]: https://cmake.org/cmake/help/latest/variable/CMAKE_CXX_STANDARD_REQUIRED.html [cmake_minimum_required]: https://cmake.org/cmake/help/latest/command/cmake_minimum_required.html [cmake_prefix_path]: https://cmake.org/cmake/help/latest/variable/CMAKE_PREFIX_PATH.html [codestylecmake.md]: ./CodeStyleCMake.md [externalproject]: https://cmake.org/cmake/help/latest/module/ExternalProject.html [find_package]: https://cmake.org/cmake/help/latest/command/find_package.html [halide-generator-tutorial]: https://halide-lang.org/tutorials/tutorial_lesson_15_generators.html [halide-tutorials]: https://halide-lang.org/tutorials/tutorial_introduction.html [imported-executable]: https://cmake.org/cmake/help/latest/command/add_executable.html#imported-executables [imported-target]: https://cmake.org/cmake/help/latest/manual/cmake-buildsystem.7.html#imported-targets [install-files]: https://cmake.org/cmake/help/latest/command/install.html#files [install-targets]: https://cmake.org/cmake/help/latest/command/install.html#targets [project]: https://cmake.org/cmake/help/latest/command/project.html [target-file]: https://cmake.org/cmake/help/latest/manual/cmake-generator-expressions.7.html#target-dependent-queries --- ### Doc/Hexagon # Halide for Hexagon HVX Halide supports offloading work to Qualcomm Hexagon DSP on Qualcomm Snapdragon 845/710 devices or newer. The Hexagon DSP provides a set of 128 byte vector instruction extensions - the Hexagon Vector eXtensions (HVX). HVX is well suited for image processing, and Halide for Hexagon HVX will generate the appropriate HVX vector instructions from a program authored in Halide. Halide can be used to compile Hexagon object files directly, by using a target such as `hexagon-32-qurt-hvx`. Halide can also be used to offload parts of a pipeline to Hexagon using the `hexagon` scheduling directive. To enable the `hexagon` scheduling directive, include the `hvx` target feature in your target. The currently supported combination of targets is to use the HVX target features with an x86 linux host (to use the simulator) or with an ARM android target (to use Hexagon DSP hardware). For examples of using the `hexagon` scheduling directive on both the simulator and a Hexagon DSP, see the blur example app. To build and run an example app using the Hexagon target, 1. Obtain and build trunk LLVM and Clang. (Earlier versions of LLVM may work but are not actively tested and thus not recommended.) 2. Download and install the Hexagon SDK and Hexagon Tools. Hexagon SDK 4.3.0 or later is needed. Hexagon Tools 8.4 or later is needed. 3. Build and run an example for Hexagon HVX ## 1. Obtain and build trunk LLVM and Clang (Follow the instructions given previously, just be sure to check out the `main` branch.) ## 2. Download and install the Hexagon SDK and Hexagon Tools Go to https://qpm.qualcomm.com/#/main/home 1. Go to Tools, and download Qualcomm Package Manager 3. Install the package manager on your machine. 2. Run the installed Qualcomm Package Manager and install the Qualcomm Hexagon SDK 5.x (or 4.x). The SDK can be selected from the Qualcomm Hexagon SDK Products. 3. Set an environment variable to point to the SDK installation location ``` export SDK_LOC=/location/of/SDK ``` ## 3. Build and run an example for Hexagon HVX In addition to running Hexagon code on device, Halide also supports running Hexagon code on the simulator from the Hexagon tools. To build and run the blur example in Halide/apps/blur on the simulator: ``` cd apps/blur export HL_HEXAGON_SIM_REMOTE=../../src/runtime/hexagon_remote/bin/v65/hexagon_sim_remote export HL_HEXAGON_TOOLS=$SDK_LOC/Hexagon_Tools/8.x/Tools/ LD_LIBRARY_PATH=../../src/runtime/hexagon_remote/bin/host/:$HL_HEXAGON_TOOLS/lib/iss/:. HL_TARGET=host-hvx make test ``` ## To build and run the blur example in Halide/apps/blur on Android: To build the example for Android, first ensure that you have Android NDK r19b or later installed, and the ANDROID_NDK_ROOT environment variable points to it. (Note that Qualcomm Hexagon SDK v4.3.0 includes Android NDK r19c, which is fine.) Now build and run the blur example using the script to run it on device: ``` export HL_HEXAGON_TOOLS=$SDK_LOC/HEXAGON_Tools/8.4.11/Tools/ HL_TARGET=arm-64-android-hvx ./adb_run_on_device.sh ``` --- ### Doc/Python # Halide Bindings for Python - [Halide Bindings for Python](#halide-bindings-for-python) - [Acquiring the Python bindings](#acquiring-the-python-bindings) - [Building the Python bindings](#building-the-python-bindings) - [Using CMake directly](#using-cmake-directly) - [Using wheel infrastructure](#using-wheel-infrastructure) - [Documentation and Examples](#documentation-and-examples) - [Differences from C++ API](#differences-from-c-api) - [Example of Simple Usage](#example-of-simple-usage) - [Halide Generators In Python](#halide-generators-in-python) - [Writing a Generator in Python](#writing-a-generator-in-python) - [`hl.generator("name")`](#hlgeneratorname) - [hl.GeneratorParam](#hlgeneratorparam) - [hl.InputBuffer, hl.InputScalar](#hlinputbuffer-hlinputscalar) - [hl.OutputBuffer, hl.OutputScalar](#hloutputbuffer-hloutputscalar) - [Names](#names) - [generate() method](#generate-method) - [Types for Inputs and Outputs](#types-for-inputs-and-outputs) - [Using a Generator for JIT compilation](#using-a-generator-for-jit-compilation) - [Using a Generator for AOT compilation](#using-a-generator-for-aot-compilation) - [Calling Generator-Produced code from Python](#calling-generator-produced-code-from-python) - [Advanced Generator-Related Topics](#advanced-generator-related-topics) - [Generator Aliases](#generator-aliases) - [Dynamic Inputs and Outputs](#dynamic-inputs-and-outputs) - [Calling a Generator Directly](#calling-a-generator-directly) - [The Lifecycle Of A Generator](#the-lifecycle-of-a-generator) - [Notable Differences Between C++ and Python Generators](#notable-differences-between-c-and-python-generators) - [Keeping Up To Date](#keeping-up-to-date) - [License](#license) Halide provides Python bindings for most of its public API. Python 3.10 (or higher) is required. The Python bindings are supported on 64-bit Linux, OSX, and Windows systems. In addition to the ability to write just-in-time Halide code using Python, you can write [Generators](#halide-generators-in-python) using the Python bindings, which can simplify build-system integration (since no C++ metacompilation step is required). You can also use existing Halide Generators (written in either C++ or Python) to produce Python extensions that can be used within Python code. ## Acquiring the Python bindings As of Halide 19.0.0, we provide binary wheels on PyPI which include the Python bindings and the C++/CMake package for native development. Full releases may be installed with `pip` like so: ```shell $ pip install halide ``` Every commit to `main` is published to a private PyPI index as a development version and these may be installed with a few extra flags: ```shell $ pip install halide --pre --extra-index-url https://pypi.halide-lang.org/simple ``` Currently, we provide wheels for: Windows x86-64, macOS x86-64, macOS arm64, and Linux x86-64. The Linux wheels are built for manylinux_2_28, which makes them broadly compatible (Debian 10, Ubuntu 18.10, Fedora 29). ## Building the Python bindings If `pip` isn't enough for your purposes, or you are developing Halide directly, you have two options for building and using the Python bindings. Note that the bindings require Halide to be built with RTTI and exceptions **enabled**, which in turn requires LLVM to be built with RTTI, but this is not the default for LLVM. Before configuring with CMake, you should ensure you have prerequisite packages installed in your local Python environment. The best way to get set up is to use a virtual environment with `uv`: ```shell $ uv sync --no-install-project ``` If you don't have LLVM installed already, you can try using the same ones the buildbots use by adding `--group ci-llvm-` to the `uv sync` command, where `` is the LLVM major version number (e.g. `23`) or `main`. If you install `ci-llvm-*`, you can set `Halide_LLVM_ROOT=$(halide-llvm --prefix)` in your environment. Ensure you have `flatbuffers` and `wabt` installed, too. (The wheel build does not use vcpkg for manylinux compatibility reasons, so these must be available as system packages or installed from source.) ### Using wheel infrastructure When using `uv`, this entire workflow can be run via: ```shell $ uv pip install . --no-build-isolation ``` ### Using CMake directly Assuming dependencies are available, you can build the Python bindings directly with CMake: ```shell $ cmake -G Ninja -S . -B build -DCMAKE_BUILD_TYPE=Release $ cmake --build build $ cmake --install build --prefix .local ``` Now you can set the `PYTHONPATH` variable to point to the freshly built Python package: ```shell $ export PYTHONPATH="$PWD/.local/lib/python3/site-packages" ``` ## Documentation and Examples As mentioned elsewhere, the Python API attempts to mimic the [C++ Halide API](http://halide-lang.org/docs) as directly as possible; there isn't separate Python-specific documentation for the API at this time. For now, examine the code for the example applications in the `test/apps/` and `tutorial/` subdirectories. The tests run as part of the standard CTest infrastructure and are labeled with the `python` label. You can run the Python tests specifically by running: ``` $ ctest -L python ``` from the Halide build directory. ## Differences from C++ API The Python bindings attempt to mimic the Halide C++ API as closely as possible, with some differences where the C++ idiom is either inappropriate or impossible: - Most APIs that take a variadic argument list of ints in C++ take an explicit list in Python. For instance, the usual version of the `Buffer` ctor in C++ offers both variadic and list versions: ```cpp Buffer<>(Type t, int extent_dim_0, int extent_dim_1, ...., extent_dim_N, string name = ""); Buffer<>(Type t, vector extents, string name = ""); ``` In Python, only the second variant is provided. - `Func` and `Buffer` access is done using `[]` rather than `()` - For zero-dimensional `Func` and `Buffer`, you must explicitly specify `[()]` -- that is, use an empty tuple as the index -- because `[]` is not syntactically acceptable in Python. - Some classes in the Halide API aren't provided because standard Python idioms are a better fit: - `Halide::Tuple` doesn't exist in the Python bindings; an ordinary Python tuple of `Halide::Expr` is used instead. - `Halide::Realization` doesn't exist in the Python bindings; an ordinary Python tuple of `Halide::Buffer` is used instead. - static and instance method overloads with the same name in the same class aren't allowed, so some convenience methods are missing from `Halide::Var` - Templated types (notably `Halide::Buffer<>` and `Halide::Param<>`) aren't provided, for obvious reasons; only the equivalents of `Halide::Buffer` and `Halide::Param` are supported. - The functions in `Halide::ConciseCasts` are present in the toplevel Halide module in Python, rather than a submodule: e.g., use `halide.i8_sat()`, not `halide.ConciseCasts.i8_sat()`. - Only things in the `Halide` namespace are supported; classes and methods that involve using the `Halide::Internal` namespace are not provided. - No mechanism is provided for overriding any runtime functions from Python for JIT-compiled code. (Runtime functions for AOT-compiled code can be overridden by building and linking a custom runtime, but not currently via any runtime API, e.g. halide_set_custom_print() does not exist.) - No mechanism is provided for supporting `Func::define_extern`. - `Buffer::for_each_value()` isn't supported yet. - `Func::in` becomes `Func.in_` because `in` is a Python keyword. - `Func::async` becomes `Func.async_` because `async` is a Python keyword. - The `not` keyword cannot be used to negate boolean Halide expressions. Instead, the `logical_not` function can be used and is equivalent to using `operator!` in C++. - There is no way to override the logical `and`/`or` operators in Python to work with `Expr`: you must use the bitwise `|` and `&` instead. (Note that incorrectly using `and`/`or` just short-circuits weirdly, rather than failing with some helpful error; this is an issue that we have not yet found any way to improve, unfortunately.) - Some error messages need to be made more informative. - Some exceptions are the "incorrect" type (compared to C++ expectations). - Many hooks to override runtime functions (e.g. Func::set_error_handler) aren't yet implemented. - The following parts of the Halide public API are currently missing entirely from the Python bindings (but are all likely to be supported at some point in the future): - `DeviceInterface` - `evaluate()` ## Example of Simple Usage Here is a basic example of using Halide to produce a procedural image. ```python # By convention, we import halide as 'hl' for terseness import halide as hl # Some constants edge = 512 k = 20.0 / float(edge) # Simple formula x, y, c = hl.Var("x"), hl.Var("y"), hl.Var("c") f = hl.Func("f") e = hl.sin(x * ((c + 1) / 3.0) * k) * hl.cos(y * ((c + 1) / 3.0) * k) f[x, y, c] = hl.cast(hl.UInt(8), e * 255.0) f.vectorize(x, 8).parallel(y) # Realize into a Buffer. buf = f.realize([edge, edge, 3]) # Do something with the image. We'll just save it to a PNG. from halide import imageio imageio.imwrite("/tmp/example.png", buf) ``` It's worth noting in the example above that the Halide `Buffer` object supports the Python Buffer Protocol (https://www.python.org/dev/peps/pep-3118) and thus is converted to and from other compatible objects (e.g., NumPy's `ndarray`), at essentially zero cost, with storage being shared. Thus, we can usually pass it directly to existing Python APIs (like `imsave()`) that expect 'image-like' objects without any explicit conversion necessary. ## Halide Generators In Python In Halide, a "Generator" is a unit of encapsulation for Halide code. It is a self-contained piece of code that can: - Produce a chunk of Halide IR (in the form of an `hl.Pipeline`) that is appropriate for compilation (via either JIT or AOT) - Expose itself to the build system in a discoverable way - Fully describe itself for the build system with metadata for (at least) the type and number of inputs and outputs expected - Allow for build-time customization of coder-specified parameters in a way that doesn't require editing of source code Originally, Halide only supported writing Generators in C++. In this document, we'll use the term "C++ Generator" to mean "Generator written in C++ using the classic API", the term "Python Generator" to mean "Generator written in Halide's Python bindings", and just plain "Generator" when the discussion is relatively neutral with respect to the implementation language/API. ### Writing a Generator in Python A Python Generator is a class that: - has the `@hl.generator` decorator applied to it - declares zero or more member fields that are initialized with values of `hl.InputBuffer` or `hl.InputScalar`, which specify the expected input(s) of the resulting `Pipeline`. - declares one or more member fields that are initialized with values of `hl.OutputBuffer` or `hl.OutputScalar`, which specify the expected output(s) of the resulting `Pipeline`. - declares zero or more member fields that are initialized with values of `hl.GeneratorParam`, which can be used to pass arbitrary information from the build system to the Generator. A GeneratorParam can carry a value of type `bool`, `int`, `float`, `str`, or `hl.Type`. - declares a `generate()` method that fill in the Halide IR needed to define all the Outputs - optionally declares a `configure()` method to dynamically add Inputs or Outputs to the pipeline, based on (e.g.) the values of `GeneratorParam` values or other external inputs Let's look at a fairly simple example: > **TODO:** this example is pretty contrived; is there an equally simple > Generator to use here that would demonstrate the basics? ```python import halide as hl x = hl.Var("x") y = hl.Var("y") _operators = { "xor": lambda a, b: a ^ b, "and": lambda a, b: a & b, "or": lambda a, b: a | b, } # Apply a mask value to a 2D image using a logical operator that is selected at compile-time. @hl.generator(name="logical_op_generator") class LogicalOpGenerator: op = hl.GeneratorParam("xor") input = hl.InputBuffer(hl.UInt(8), 2) mask = hl.InputScalar(hl.UInt(8)) output = hl.OutputBuffer(hl.UInt(8), 2) def generate(g): # Algorithm operator = _operators[g.op] g.output[x, y] = operator(g.input[x, y], g.mask) # Schedule v = g.natural_vector_size(hl.UInt(8)) g.output.vectorize(x, v) if __name__ == "__main__": hl.main() ``` If you've worked with Halide Generators written in C++, the "shape" of this will likely look familiar. (If not, no worries; you shouldn't need any knowledge of C++ Generators for the following to make sense.) Let's take the details here one at a time. #### `hl.generator("name")` This decorator adds appropriate "glue" machinery to the class to enforce various invariants. It also serves as the declares a "registered name" for the Generator, which is a unique name that the build system will use to identify the Generator. If you omit the name, it defaults to `module.classname`; if module is `__main__` then we omit it and just use the plain classname. Note that the registered name need not match the classname. (Inside Halide, we use the convention of `CamelCase` for class names and `snake_case` for registered names, but you can use whatever convention you like.) #### hl.GeneratorParam Each `GeneratorParam` is an arbitrary key-value pair that can be used to provide configurable options at compile time. You provide the name and a default value. The default value can be overridden by the build machinery, which will replace the value (based on user specified text). Note that the type of the default value *is* used to define the expected type of the `GeneratorParam`, and trying to set it to an incompatible value will throw an exception. The types that are acceptable to use in a `GeneratorParam` are: - Python's `bool`, `int`, `float`, or `str` - Halide's `hl.Type` - ...that's all Note that the value of a `GeneratorParam` is read-only from the point of view of the Generator; they are set at Generator construction time and attempting to change their value will throw an exception. #### hl.InputBuffer, hl.InputScalar These declare the inputs to the `hl.Pipeline` that the Generator will produce. An `hl.InputScalar` is, essentially, a "factory" that produces an `hl.Param` in the existing Python API, while an `hl.InputBuffer` is a factory for `hl.ImageParam`. From the Generator author's perspective, a field initialized with `InputScalar` **is** a `Param` – not kinda-like-one, not a magic wrapper that forwards everything; it is literally just `hl.Param`. Similarly, an `InputBuffer` produces `ImageParam`, and an `InputFunc` is a wrapper around `Func`. You won't be able to assign a new value to the member field for Inputs – as with GeneratorParams, they are "read-only" to the Generator – but you will be able to set constraints on them. Note that in addition to specifying a concrete type and dimensionality for the inputs, these factory classes support the ability to specify either (or both) `None`, which means the type/dimensionality will be provided by GeneratorParams in the build system. #### hl.OutputBuffer, hl.OutputScalar These declare the output(s) of the Pipeline that the Generator will produce. An `hl.OutputBuffer` is, essentially, a "factory" that produces an `hl.Func` in the existing Python API. (`hl.OutputScalar` is just an `hl.OutputBuffer` that always has zero dimensions.) From the Generator author's perspective, a field declared with `OutputBuffer` **is** a `Func` – not kinda-like-one, not a magic wrapper that forwards everything; it is literally just `hl.Func` (with type-and-dimensionality set to match, see recent PR https://github.com/halide/Halide/pull/6734) . You won't be able to assign a new value to the member field for Inputs – as with GeneratorParams, they are "read-only" to the Generator – but you will be able to set constraints on them. Note that in addition to specifying a concrete type and dimensionality for the inputs, these factory classes support the ability to specify either (or both) as `None`, which means the type/dimensionality will be provided by GeneratorParams in the build system. #### Names Note that all the GeneratorParams, Inputs, and Outputs have names that are implicitly filled in based on the field name of their initial assignment; unlike in C++ Generators, there isn't a way to "override" this name (i.e., the name in the IR will always exactly match the Python field name). Names have the same constraints as for C++ Generators (essentially, a C identifier, but without an initial underscore, and without any double underscore anywhere). #### generate() method This will be called by the Generator machinery to build the Pipeline. As with C++ Generators, the only required task is to ensure that all Output fields are fully defined, in a way that matches the type-and-dimension constraints specified. It is required that the `generate()` method be defined by the Generator. (Note that, by convention, Halide Generators use `g` instead of `self` in their `generate()` method to make the expression language terser; this is not in any way required, but is recommended to improve readability.) #### Types for Inputs and Outputs For all the Input and Output fields of Generators, you can specify native Python types (instead of `hl.Type`) for certain cases that are unambiguous. At present, we allow `bool` as an alias for `hl.Bool()`, `int` as an alias for `hl.Int(32)`, and `float` as an alias for `hl.Float(32)`. ### Using a Generator for JIT compilation You can use the `compile_to_callable()` method to JIT-compile a Generator into a `hl.Callable`, which is (essentially) just a dynamically-created function. ```python import LogicalOpGenerator from halide import imageio import numpy as np # Instantiate a Generator -- we can only set the GeneratorParams # by passing in a dict to the Generator's constructor or_op_generator = LogicalOpGenerator({"op": "or"}) # Now compile the Generator into a Callable or_filter = or_op_generator.compile_to_callable() # Read in some file for input input_buf = imageio.imread("/path/to/some/file.png") assert input_buf.ndim == 2 assert input_buf.dtype == np.uint8 # create a Buffer-compatible object for the output; we'll use np.array output_buf = np.empty(input_buf.shape, dtype=input_buf.dtype) # Note, Python code throws exception for error conditions rather than returning an int or_filter(input_buf, 0x7F, output_buf) # Note also that we can use named arguments for any/all, in the Python manner: or_filter(mask=0x7F, input=input_buf, output=output_buf) imageio.imwrite("/tmp/or.png", output_buf) ``` By default, a Generator will produce code targeted at `Target("host")` (or the value of the `HL_JIT_TARGET` environment variable, if set); you can override this behavior selectively by activating a `GeneratorContext` when the Generator is *created*: ```python import LogicalOpGenerator # Compile with debugging enabled t = hl.Target("host-debug") with hl.GeneratorContext(t): or_op_generator = LogicalOpGenerator({"op": "or"}) or_filter = or_op_generator.compile_to_callable() ``` ### Using a Generator for AOT compilation If you are using CMake, the simplest thing is to use `add_halide_library` and `add_halide_python_extension_library()`: ```cmake # Build a Halide library as you usually would, but be sure to include `PYTHON_EXTENSION` add_halide_library(xor_filter FROM logical_op_generator PARAMS op=xor PYTHON_EXTENSION output_path_var [ FEATURES ... ] [ PARAMS ... ]) # Now wrap the generated code with a Python extension. # (Note that module name defaults to match the target name; we only # need to specify MODULE_NAME if we need a name that may differ) add_halide_python_extension_library(my_extension MODULE_NAME my_module HALIDE_LIBRARIES xor_filter) ``` (Note that this rule works for both C++ and Python Generators.) This compiles the Generator code in `logical_op_generator.py` with the registered name `logical_op_generator` to produce the target `xor_filter`, and then wraps the compiled output with a Python extension. The result will be a shared library of the form `..so`, where `` describes the specific Python version and platform (e.g., `cpython-310-darwin` for Python 3.10 on OSX.) Note that you can combine multiple Halide libraries into a single Python module; this is convenient for packaging, but also because all the libraries in a single extension module share the same Halide runtime (and thus, the same caches, thread pools, etc.). ```cmake add_halide_library(xor_filter ...) add_halide_library(and_filter ...) add_halide_library(or_filter ...) add_halide_python_extension_library(my_extension MODULE_NAME my_module HALIDE_LIBRARIES xor_filter and_filter or_filter) ``` Note that you must take care to ensure that all of the `add_halide_library` targets specified use the same Halide runtime; it may be necessary to use `add_halide_runtime` to define an explicit runtime that is shared by all the targets: ```cmake add_halide_runtime(my_runtime) add_halide_library(xor_filter USE_RUNTIME my_runtime ...) add_halide_library(and_filter USE_RUNTIME my_runtime ...) add_halide_library(or_filter USE_RUNTIME my_runtime ...) add_halide_python_extension_library(my_extension MODULE_NAME my_module HALIDE_LIBRARIES xor_filter and_filter or_filter) ``` If you're not using CMake, you can "drive" a Generator directly from your build system via command-line flags. The most common, minimal set looks something like this: ```shell python3 /path/to/my/generator.py -g \ -o \ target= \ [generator-param=value ...] ``` The argument to `-g` is the name supplied to the `@hl.generator` decorator. The argument to -o is a directory to use for the output files; by default, we'll produce a static library containing the object code, and a C++ header file with a forward declaration. `target` specifies a Halide `Target` string describing the OS, architecture, features, etc. that should be used for compilation. Any other arguments to the command line that don't begin with `-` are presumed to name `GeneratorParam` values to set. There are other flags and options too, of course; use `python3 /path/to/my/generator.py -help` to see a list with explanations. (Unfortunately, there isn't (yet) a way to produce a Python Extension just by running a Generator; the logic for `add_halide_python_extension_library` is currently all in the CMake helper files.) ### Calling Generator-Produced code from Python As long as the shared library is in `PYTHONPATH`, it can be imported and used directly. For the example above: ```python from my_module import xor_filter from halide import imageio import numpy as np # Read in some file for input input_buf = imageio.imread("/path/to/some/file.png") assert input_buf.ndim == 2 assert input_buf.dtype == np.uint8 # create a Buffer-compatible object for the output; we'll use np.array output_buf = np.empty(input_buf.shape, dtype=input_buf.dtype) # Note, Python code throws exception for error conditions rather than returning an int xor_filter(input_buf, 0xFF, output_buf) # Note also that we can use named arguments for any/all, in the Python manner: # xor_filter(input=input_buf, mask=0xff, output=output_buf) imageio.imwrite("/tmp/xored.png", output_buf) ``` Above, we're using common Python utilities (`numpy`) to construct the input/output buffers we want to pass to Halide. **Note**: Getting the memory order correct can be a little confusing for numpy. By default, numpy uses "C-style" [row-major](https://docs.scipy.org/doc/numpy-1.13.0/reference/internals.html) order, which sounds like the right option for Halide; however, this nomenclature assumes the matrix-math convention of ordering axes as `[rows, cols]`, whereas Halide (and imaging code in general) generally assumes `[x, y]` (i.e., `[cols, rows]`). Thus, what you usually want in Halide is column-major ordering. This means numpy arrays, by default, come with the wrong memory layout for Halide. But if you construct the numpy arrays yourself (like above), you can pass `order='F'` to make numpy use the Halide-compatible memory layout. If you're passing in an array constructed somewhere else, the easiest thing to do is to `.transpose()` it before passing it to your Halide code. ### Advanced Generator-Related Topics #### Generator Aliases A Generator alias is a way to associate a Generator with one (or more) specific sets of GeneratorParams; the 'alias' is just another registered name. This offers a convenient alternative to specifying multiple sets of GeneratorParams via the build system. To define alias(es) for a Generator, just add the `@hl.alias` decorator before `@hl.generator` decorator: ```python @hl.alias( xor_generator={"op": "xor"}, and_generator={"op": "and"}, or_generator={"op": "or"}, ) @hl.generator("logical_op_generator") class LogicalOpGenerator: ... ``` #### Dynamic Inputs and Outputs If you need to build `Input` and/or `Output` dynamically, you can define a `configure()` method. It will always be called after all `GeneratorParam` values are valid, but before `generate()` is called. Let's take our example and add an option to pass an offset to be added after the logical operator is done: ```python import halide as hl x = hl.Var("x") y = hl.Var("y") _operators = { "xor": lambda a, b: a ^ b, "and": lambda a, b: a & b, "or": lambda a, b: a | b, } # Apply a mask value to a 2D image using a logical operator that is selected at compile-time. @hl.generator(name="logical_op_generator") class LogicalOpGenerator: op = hl.GeneratorParam("xor") with_offset = hl.GeneratorParam(False) input = hl.InputBuffer(hl.UInt(8), 2) mask = hl.InputScalar(hl.UInt(8)) output = hl.OutputBuffer(hl.UInt(8), 2) def configure(g): # If with_offset is specified, we if g.with_offset: g.add_input("offset", hl.InputScalar(hl.Int(32))) # See note the use of 'g' instead of 'self' here def generate(g): # Algorithm operator = _operators[g.op] if hasattr(g, "offset"): g.output[x, y] = operator(g.input[x, y], g.mask) + g.offset else: g.output[x, y] = operator(g.input[x, y], g.mask) # Schedule v = g.natural_vector_size(hl.UInt(8)) g.output.vectorize(x, v) if __name__ == "__main__": hl.main() ``` The only thing you can (usefully) do from `configure()` is to call `add_input()` or `add_output()`, which accept only the appropriate `Input` or `Output` classes. The resulting value is stored as a member variable with the name specified (if there is already a member with the given name, an exception is thrown). #### Calling a Generator Directly Each Generator has a class method (injected by `@hl.generator`) that allows you to "call" the Generator like an ordinary function; this allows you to directly take the Halide IR produced by the Generator and do anything you want to with it. This can be especially useful when writing library code, as you can 'compose' more complex pipelines this way. This method is named `call()` and looks like this: ```python @classmethod def call(cls, *args, **kwargs): ... ``` It takes the inputs (specified either by-name or by-position in the usual Python way). It also allows for an optional by-name-only argument, `generator_params`, which is a simple Python dict that allows for overriding `GeneratorParam`s. It returns a tuple of the Output values. For the earlier example, usage might be something like: ```python import LogicalOpFilter x, y = hl.Var(), hl.Var() input_buf = hl.Buffer(hl.UInt(8), [2, 2]) mask_value = 0x7F # Inputs by-position func_out = LogicalOpFilter.call(input_buf, mask_value) # Inputs by-name func_out = LogicalOpFilter.call(mask=mask_value, input=input_buf) # Above again, but with generator_params func_out = LogicalOpFilter.call(input_buf, mask_value, generator_params={"op": "and"}) func_out = LogicalOpFilter.call( generator_params={"op": "and"}, input=input_buf, mask=mask_value ) ``` #### The Lifecycle Of A Generator Whether being driven by a build system (for AOT use) or by another piece of Python code (typically for JIT use), the lifecycle of a Generator looks something like this: - An instance of the Generator in question is created. It uses the currently-active `GeneratorContext` (which contains the `Target` to be used for code generation), which is stored in a thread-local stack. - Some (or all) of the default values of the `GeneratorParam` members may be replaced based on (e.g.) command-line arguments in the build system - All `GeneratorParam` members are made immutable. - The `configure()` method is called, allowing the Generator to use `add_input()` or `add_output()` to dynamically add inputs and/or outputs. - If any `Input` or `Output` members were defined with unspecified type or dimensions (e.g. `some_input = hl.InputBuffer(None, 3)`), those types and dimensions are filled in from `GeneratorParam` values (e.g. `some_input.type` in this case). If any types or dimensions are left unspecified after this step, an exception will be thrown. - If the Generator is being invoked via its `call()` method (see below), the default values for `Inputs` will be replaced by the values from the argument list. - The Generator instance has its `generate()` method called. - The calling code will extract the values of all `Output` values and validate that they match the type, dimensions, etc. of the declarations. - The calling code will then either call `compile_to_file()` and friends (for AOT use), or return the output values to the caller (for JIT use). - Finally, the Generator instance will be discarded, never to be used again. Note that almost all the code doing the hand-wavy bits above is injected by the `@hl.generator` decorator – the Generator author doesn't need to know or care about the specific details, only that they happen. All Halide Generators are **single-use** instances – that is, any given Generator instance should be used at most once. If a Generator is to be executed multiple times (e.g. for different `GeneratorParam` values, or a different `Target`), a new one must be constructed each time. #### Notable Differences Between C++ and Python Generators If you have written C++ Generators in Halide in the past, you might notice some features are missing and/or different for Python Generators. Among the differences are: - In C++, you can create a Generator, then call `set_generatorparam_value()` to alter the values of GeneratorParams. In Python, there is no public method to alter a GeneratorParam after the Generator is created; instead, you must pass a dict of GeneratorParam values to the constructor, after which the values are immutable for that Generator instance. - Array Inputs/Outputs: in our experience, they are pretty rarely used, it complicates the implementation in nontrivial ways, and the majority of use cases for them can all be reasonably supported by dynamically adding inputs or outputs (and saving the results in a local array). - `Input` and `Output`: these were deliberately left out in order to simplify Python Generators. It's possible that something similar might be added in the future. - GeneratorParams with LoopLevel types: these aren't useful without `Input`/`Output`. - GeneratorParams with Enum types: using a plain `str` type in Python is arguably just as easy, if not easier. - `get_externs_map()`: this allows registering ExternalCode objects to be appended to the Generator's code. In our experience, this feature is very rarely used. We will consider adding this in the future if necessary. - Lazy Binding of Unspecified Input/Output Types: for C++ Generators, if you left an Output's type (or dimensionality) unspecified, you didn't always have to specify a `GeneratorParam` to make it into a concrete type: if the type was always fully specified by the contents of the `generate()` method, that was good enough. In Python Generators, by contrast, **all** types and dimensions must be **explicitly** specified by either code declaration or by `GeneratorParam` setting. This simplifies the internal code in nontrivial ways, and also allows for (arguably) more readable code, since there are no longer cases that require the reader to execute the code in their head in order to deduce the output types. ## Keeping Up To Date If you use the Halide Bindings for Python inside Google, you are *strongly* encouraged to [subscribe to announcements for new releases of Halide](https://github.blog/changelog/2018-11-27-watch-releases/), as it is likely that enhancements and tweaks to our Python support will be made in future releases. ## License The Python bindings use the same [MIT license](https://github.com/halide/Halide/blob/main/LICENSE.txt) as Halide. Python bindings provided by Connelly Barnes (2012-2013), Fred Rotbart (2014), Rodrigo Benenson (2015) and the Halide open-source community. --- ### Doc/RunGen # Running and Benchmarking Halide Generators ## Overview `RunGen` is a simple(ish) wrapper that allows an arbitrary Generator to be built into a single executable that can be run directly from bash, without needing to wrap it in your own custom main() driver. It also implements a rudimentary benchmarking and memory-usage functionality. If you use the standard CMake rules for Generators, you get RunGen functionality automatically. (If you use Make, you might need to add an extra rule or two to your Makefile; all the examples in `apps/` already have these rules.) For every `halide_library` (or `halide_library_from_generator`) rule, there is an implicit `name.rungen` rule that generates an executable that wraps the Generator library: ``` # In addition to defining a static library named "local_laplacian", this rule # also implicitly defines an executable target named "local_laplacian.rungen" halide_library( local_laplacian SRCS local_laplacian_generator.cc ) ``` You can build and run this like any other executable: ``` $ make bin/local_laplacian.rungen && ./bin/local_laplacian.rungen Usage: local_laplacian.rungen argument=value [argument=value... ] [flags] ...typical "usage" text... ``` To be useful, you need to pass in values for the Generator's inputs (and locations for the output(s)) on the command line, of course. You can use the `--describe` flag to see the names and expected types: ``` # ('make bin/local_laplacian.rungen && ' prefix omitted henceforth for clarity) $ ./bin/local_laplacian.rungen --describe Filter name: "local_laplacian" Input "input" is of type Buffer with 3 dimensions Input "levels" is of type int32 Input "alpha" is of type float32 Input "beta" is of type float32 Output "local_laplacian" is of type Buffer with 3 dimensions ``` Warning: Outputs may have `$X` (where `X` is a small integer) appended to their names in some cases (or, in the case of Generators that don't explicitly declare outputs via `Output<>`, an autogenerated name of the form `fX`). If this happens, don't forget to escape the `$` with a backslash as necessary. These are both bugs we intend to fix; see https://github.com/halide/Halide/issues/2194 As a convenience, there is also an implicit target that builds-and-runs, named simply "NAME.run": ``` # This is equivalent to "make bin/local_laplacian.rungen && ./bin/local_laplacian.rungen" $ make bin/local_laplacian.run Usage: local_laplacian.rungen argument=value [argument=value... ] [flags] # To pass arguments to local_laplacian.rungen, set the RUNARGS var: $ make bin/local_laplacian.run RUNARGS=--describe Filter name: "local_laplacian" Input "input" is of type Buffer with 3 dimensions Input "levels" is of type int32 Input "alpha" is of type float32 Input "beta" is of type float32 Output "local_laplacian" is of type Buffer with 3 dimensions ``` Inputs are specified as `name=value` pairs, in any order. Scalar inputs are specified the typical text form, while buffer inputs (and outputs) are specified via paths to image files. RunGen currently can read/write image files in any format supported by halide_image_io.h; at this time, that means .png, .jpg, .ppm, .pgm, and .tmp formats. (We plan to add .tiff and .mat (level 5) in the future.) ``` $ ./bin/local_laplacian.rungen input=../images/rgb_small16.png levels=8 alpha=1 beta=1 output=/tmp/out.png $ display /tmp/out.png ``` You can also specify any scalar input as `default` or `estimate`, which will use the default value specified for the input, or the value specified by `set_estimate` for that input. (If the relevant value isn't set for that input, a runtime error occurs.) ``` $ ./bin/local_laplacian.rungen input=../images/rgb_small16.png levels=8 alpha=estimate beta=default output=/tmp/out.png $ display /tmp/out.png ``` If you specify an input or output file format that doesn't match the required type/dimensions for an argument (e.g., using an 8-bit PNG for an Input, or a grayscale image for a 3-dimensional input), RunGen will try to coerce the inputs to something sensible; that said, it's hard to always get this right, so warnings are **always** issued whenever an input or output is modified in any way. ``` # This filter expects a 16-bit RGB image as input, but we're giving it an 8-bit grayscale image: $ ./bin/local_laplacian.rungen input=../images/gray.png levels=8 alpha=1 beta=1 output=/tmp/out.png Warning: Image for Input "input" has 2 dimensions, but this argument requires at least 3 dimensions: adding dummy dimensions of extent 1. Warning: Image loaded for argument "input" is type uint8 but this argument expects type uint16; data loss may have occurred. ``` By default, we try to guess a suitable size for the output image(s), based mainly on the size of the input images (if any); you can also specify explicit output extents. (Note that output_extents are subject to constraints already imposed by the particular Generator's logic, so arbitrary values for --output_extents may produce runtime errors.) ``` # Constrain output extents to 100x200x3 $ ./bin/local_laplacian.rungen --output_extents=[100,200,3] input=../images/rgb_small16.png levels=8 alpha=1 beta=1 output=/tmp/out.png ``` Sometimes you don't care what the particular element values for an input are (e.g. for benchmarking), and you just want an image of a particular size; in that case, you can use the `zero:[]` pseudo-file; it infers the _type_ from the Generator, and inits every element to zero: ``` # Input is a 3-dimensional image with extent 123, 456, and 3 # (bluring an image of all zeros isn't very interesting, of course) $ ./bin/local_laplacian.rungen --output_extents=[100,200,3] input=zero:[123,456,3] levels=8 alpha=1 beta=1 output=/tmp/out.png ``` You can also specify arbitrary (nonzero) constants: ``` # Input is a 3-dimensional image with extent 123, 456, and 3, # filled with a constant value of 42 $ ./bin/local_laplacian.rungen --output_extents=[100,200,3] input=constant:42:[123,456,3] levels=8 alpha=1 beta=1 output=/tmp/out.png ``` Similarly, you can create identity images where only the diagonal elements are 1-s (rest are 0-s) by invoking `identity:[]`. Diagonal elements are defined as those whose first two coordinates are equal. There's also a `random:SEED:[]` pseudo-file, which fills the image with uniform noise based on a specific random-number seed: ``` # Input is a 3-dimensional image with extent 123, 456, and 3 $ ./bin/local_laplacian.rungen --output_extents=[100,200,3] input=random:42:[123,456,3] levels=8 alpha=1 beta=1 output=/tmp/out.png ``` Instead of specifying an explicit set of extents for a pseudo-input, you can use the string `auto`, which will run a bounds query to choose a legal set of extents for that input given the known output extents. (This is only useful when used in conjunction with the `--output_extents` flag.) ``` $ ./bin/local_laplacian.rungen --output_extents=[100,200,3] input=zero:auto levels=8 alpha=1 beta=1 output=/tmp/out.png ``` You can also specify `estimate` for the extents, which will use the estimate values provided, typically (but not necessarily) for auto_schedule. (If there aren't estimates for all of the buffer's dimensions, a runtime error occurs.) ``` $ ./bin/local_laplacian.rungen --output_extents=[100,200,3] input=zero:auto levels=8 alpha=1 beta=1 output=/tmp/out.png ``` You can combine the two and specify `estimate_then_auto` for the extents, which will attempt to use the estimate values; if a given input buffer has no estimates, it will fall back to the bounds-query result for that input: ``` $ ./bin/local_laplacian.rungen --output_extents=[100,200,3] input=zero:estimate_then_auto levels=8 alpha=1 beta=1 output=/tmp/out.png ``` Similarly, you can use `estimate` for `--output_extents`, which will use the estimate values for each output. (If there aren't estimates for all of the outputs, a runtime error occurs.) ``` $ ./bin/local_laplacian.rungen --output_extents=estimate input=zero:auto levels=8 alpha=1 beta=1 output=/tmp/out.png ``` If you don't want to explicitly specify all (or any!) of the input values, you can use the `--default_input_buffers` and `--default_input_scalars` flags, which provide wildcards for any omitted inputs: ``` $ ./bin/local_laplacian.rungen --output_extents=[100,200,3] --default_input_buffers=random:0:auto --default_input_scalars=estimate output=/tmp/out.png ``` In this case, all input buffers will be sized according to bounds query, and filled with a random seed; all input scalars will be initialized to their declared default values. (If they have no declared default value, a zero of the appropriate type will be used.) Note: `--default_input_buffers` can produce surprising sizes! For instance, any input that uses `BoundaryConditions::repeat_edge` to wrap itself can legally be set to almost any size, so you may legitimately get an input with extent=1 in all dimensions; whether this is useful to you or not depends on the code. It's highly recommended you do testing with the `--verbose` flag (which will log the calculated sizes) to reality-check that you are getting what you expect, especially for benchmarking. A common case (especially for benchmarking) is to specify using estimates for all inputs and outputs; for this, you can specify `--estimate_all`, which is just a shortcut for `--default_input_buffers=estimate_then_auto --default_input_scalars=estimate --output_extents=estimate`. ## Benchmarking To run a benchmark, use the `--benchmarks=all` flag: ``` $ ./bin/local_laplacian.rungen --benchmarks=all input=zero:[1920,1080,3] levels=8 alpha=1 beta=1 --output_extents=[100,200,3] Benchmark for local_laplacian produces best case of 0.0494629 sec/iter, over 3 blocks of 10 iterations. Best output throughput is 39.9802 mpix/sec. ``` You can use `--default_input_buffers` and `--default_input_scalars` here as well: ``` $ ./bin/local_laplacian.rungen --benchmarks=all --default_input_buffers --default_input_scalars --output_extents=estimate Benchmark for local_laplacian produces best case of 0.0494629 sec/iter, over 3 blocks of 10 iterations. Best output throughput is 39.9802 mpix/sec. ``` Note: `halide_benchmark.h` is known to be inaccurate for GPU filters; see https://github.com/halide/Halide/issues/2278 ## Measuring Memory Usage To track memory usage, use the `--track_memory` flag, which measures the high-water-mark of CPU memory usage. ``` $ ./bin/local_laplacian.rungen --track_memory input=zero:[1920,1080,3] levels=8 alpha=1 beta=1 --output_extents=[100,200,3] Maximum Halide memory: 82688420 bytes for output of 1.97754 mpix. ``` Warning: `--track_memory` may degrade performance; don't combine it with `--benchmark` or expect meaningful timing measurements when using it. ## Using RunGen in Make To add support for RunGen to your Makefile, you need to add rules something like this (see `apps/support/Makefile.inc` for an example): ``` HALIDE_DISTRIB ?= /path/to/halide/distrib/folder $(BIN)/RunGenMain.o: $(HALIDE_DISTRIB)/tools/RunGenMain.cpp @mkdir -p $(@D) @$(CXX) -c $< $(CXXFLAGS) $(LIBPNG_CXX_FLAGS) $(LIBJPEG_CXX_FLAGS) -I$(BIN) -o $@ .PRECIOUS: $(BIN)/%.rungen $(BIN)/%.rungen: $(BIN)/%.a $(BIN)/%.registration.cpp $(BIN)/RunGenMain.o $(CXX) $(CXXFLAGS) $^ -o $@ $(LIBPNG_LIBS) $(LIBJPEG_LIBS) $(LDFLAGS) RUNARGS ?= $(BIN)/%.run: $(BIN)/%.rungen @$(CURDIR)/$< $(RUNARGS) ``` Note that the `%.registration.cpp` file is created by running a generator and specifying `registration` in the comma-separated list of files to emit; these are also generated by default if `-e` is not used on the generator command line. ## Known Issues & Caveats - If your Generator uses `define_extern()`, you must have all link-time dependencies declared properly via `FILTER_DEPS`; otherwise, you'll fail to link. - The code does its best to detect when inputs or outputs need to be chunky/interleaved (rather than planar), but in unusual cases it might guess wrong; if your Generator uses buffers with unusual stride setups, RunGen might fail at runtime. (If this happens, please file a bug!) - The code for deducing good output sizes is rudimentary and needs to be smartened; it will sometimes make bad decisions which will prevent the filter from executing. (If this happens, please file a bug!) --- ### Doc/Testing # Testing Halide uses CTest as its primary test platform and runner. ## Organization Halide's tests are organized beneath the top-level `test/` directory. These folders are described below: | Folder | Description | | -------------------- | -------------------------------------------------------------------------------- | | `autoschedulers/$AS` | Test for the `$AS` (e.g. `adams2019`) autoscheduler | | `common` | Code that may be shared across multiple tests | | `correctness` | Tests that check correctness of various compiler properties | | `error` | Tests that expect an exception to be thrown (or `abort()` to be called) | | `failing_with_issue` | Correctness tests that are associated with a particular issue on GitHub | | `fuzz` | Fuzz tests. Read more at [FuzzTesting.md](FuzzTesting.md) | | `generator` | Tests of Halide's AOT compilation infrastructure. | | `integration` | Tests of Halide's CMake package for downstream use, including cross compilation. | | `performance` | Tests that check that certain schedules indeed improve performance. | | `runtime` | Unit tests for the Halide runtime library | | `warning` | Tests that expected warnings are indeed issued. | The tests in each of these directories are given CTest labels corresponding to the directory name. Thus, one can use `ctest -L generator` to run only the `generator` tests. The `performance` tests configure CTest to not run them concurrently with other tests (including each other). The vast majority of our tests are simple C++ executables that link to Halide, perform some checks, and print the special line `Success!` upon successful completion. There are three main exceptions to this: First, the `warning` tests are expected to print a line that reads `Warning:` and do not look for `Success!`. Second, some tests cannot run in all scenarios; for example, a test that measures CUDA performance requires a CUDA-capable GPU. In these cases, tests are expected to print `[SKIP]` and exit and not print `Success!` or `Warning:`. Finally, the `error` tests are expected to throw an (uncaught) exception that is not a `Halide::InternalError` (i.e. from a failing `internal_assert`). The logic for translating uncaught exceptions into successful tests is in `test/common/expect_abort.cpp`. ## Debugging the compiler We provide helpers for pretty-printing Halide's IR types (and other internal data structures) in interactive debuggers. The LLDB and GDB helpers share a single C++ bridge, `Halide::Internal::debug_string()` (declared in `src/IRPrinter.h`): each pretty-printer asks the inferior to render the value with the normal IR printer, so all three front-ends produce the same output. Because the value is rendered by calling into the program, the process must be running (stopped at a breakpoint) for IR summaries to appear. ### LLDB The `.lldbinit` file at the repository root will load `tools/lldbhalide.py` automatically if you launch `lldb` from this directory and your `~/.lldbinit` file contains the line, ``` settings set target.load-cwd-lldbinit true ``` If you prefer to avoid such global configuration, you can directly load the helpers with the LLDB command, ``` command script import ./tools/lldbhalide.py ``` again assuming that the repository root is your current working directory. To see the benefit of using these helpers, let us debug `correctness_bounds`: ``` $ lldb ./build/test/correctness/correctness_bounds (lldb) breakpoint set --file bounds.cpp --line 18 Breakpoint 1: where = correctness_bounds`main + 864 at bounds.cpp:18:12, address = 0x0000000100002054 (lldb) run Process 29325 launched: '/Users/areinking/dev/Halide/build/test/correctness/correctness_bounds' (arm64) Defining function... Process 29325 stopped * thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1 frame #0: 0x0000000100002054 correctness_bounds`main(argc=1, argv=0x000000016fdff160) at bounds.cpp:18:12 15 g(x, y) = min(x, y); 16 h(x, y) = clamp(x + y, 20, 100); 17 -> 18 Var xo("xo"), yo("yo"), xi("xi"), yi("yi"); 19 20 Target target = get_jit_target_from_environment(); 21 if (target.has_gpu_feature()) { Target 0: (correctness_bounds) stopped. (lldb) ``` Now we can try to inspect the Func `h`. Without the helpers, we see: ``` (lldb) v h (Halide::Func) { func = { contents = { strong = (ptr = 0x0000600002486a20) weak = nullptr idx = 0 } } pipeline_ = { contents = (ptr = 0x0000000000000000) } } ``` But if we load the helpers and try again, we get a much more useful output: ``` (lldb) command script import ./tools/lldbhalide.py (lldb) v h ... lots of output ... ``` The amount of output here is maybe a bit _too_ much, but we gain the ability to more narrowly inspect data about the func: ``` (lldb) v h.func.init_def.values ... (std::vector) h.func.init_def.values = size=1 { [0] = max(min(x + y, 100), 20) } ``` These helpers are particularly useful when using graphical debuggers, such as the one found in CLion. ### GDB The GDB helpers live in `tools/gdbhalide.py` and mirror the LLDB coverage. The `.gdbinit` file at the repository root loads them when you launch `gdb` from this directory, but GDB only auto-loads a local `.gdbinit` if you have marked it safe, e.g. by adding to your `~/.gdbinit`: ``` add-auto-load-safe-path /path/to/Halide/.gdbinit ``` Otherwise, load the helpers manually once the program is running: ``` (gdb) source ./tools/gdbhalide.py ``` You can then `print` an `Expr`, `Stmt`, `Target`, etc. and see the same pretty-printed form that LLDB produces. ### Natvis (Visual Studio and CLion) `tools/Halide.natvis` provides Visual Studio-style visualizers. On MSVC it is embedded into the Halide PDB automatically (via the build), so it loads whenever you debug something linked against Halide. It is also attached to the target as a source file, so IDEs whose LLDB-based debuggers understand Natvis — including recent CLion and Rider on Linux and macOS — can load it as well (Settings | Build, Execution, Deployment | Debugger | Data Views | Native). Natvis is declarative and cannot call functions, so it does not reproduce the one-line IR rendering from `debug_string()` — use the LLDB/GDB helpers above for that. Instead, it downcasts each `Expr`/`Stmt` handle to its concrete node using the `node_type` discriminator and presents a fully expandable IR tree, recursing through operands (leaf nodes such as immediates and `Variable` show their value in the summary). Because this uses no function calls, it also works on a core dump or a process that isn't running. Natvis additionally covers buffers (`halide_buffer_t`, `Halide::Buffer`, `Halide::Runtime::Buffer`, including "Add to Image Watch" in Visual Studio) and flat value types such as `halide_type_t`, `Halide::Type`, and `ConstantInterval`. --- ### Doc/Vulkan # Vulkan Support for Halide Halide supports the Khronos Vulkan framework as a compute API backend for GPU-like devices, and compiles directly to a binary SPIR-V representation as part of its code generation before submitting it to the Vulkan API. Both JIT and AOT usage are supported via the `vulkan` target flag (e.g. `HL_JIT_TARGET=host-vulkan`). Vulkan support is actively under development, and considered *BETA* quality at this stage. Tests are passing, but performance tuning and user testing is needed to identify potential issues before rolling this into production. See [below](#current-status) for details. # Compiling Halide w/Vulkan Support You'll need to configure Halide and enable the cmake option TARGET_VULKAN (which is now ON by default). For example, on Linux & OSX: ``` % cmake -G Ninja -DTARGET_VULKAN=ON -DCMAKE_BUILD_TYPE=Release -DLLVM_DIR=$LLVM_ROOT/lib/cmake/llvm % cmake --build build --config Release ``` On Windows, you may need to specify the location of the Vulkan SDK if the paths aren't resolved by CMake automatically. For example (assuming the Vulkan SDK is installed in the default path): ``` C:\> cmake -G Ninja -DTARGET_VULKAN=ON -DCMAKE_BUILD_TYPE=Release -DLLVM_DIR=$LLVM_ROOT/lib/cmake/llvm -DVulkan_LIBRARY=C:\VulkanSDK\1.3.231.1\Lib\vulkan-1.lib -DVulkan_INCLUDE_DIR=C:\VulkanSDK\1.3.231.1\Include\vulkan -S . -B build C:\> cmake --build build --config Release ``` # Vulkan Runtime Environment: Halide has no direct dependency on Vulkan for code-generation, but the runtime requires a working Vulkan environment to run Halide generated code. Any valid Vulkan v1.0+ device driver should work. Specifically, you'll need: - A vendor specific Vulkan device driver - The generic Vulkan loader library For AMD & NVIDIA & Intel devices, download and install the latest graphics driver for your platform. Vulkan support should be included. ## Windows To build Halide AOT generators, you'll need the Vulkan SDK (specifically the Vulkan loader library and headers): https://sdk.lunarg.com/sdk/download/latest/windows/vulkan-sdk.exe For Vulkan device drivers, consult the appropriate hardware vendor for your device. A few common ones are listed below. - [AMD Vulkan Driver](https://www.amd.com/en/technologies/vulkan) - [NVIDIA Vulkan Driver](https://developer.nvidia.com/vulkan-driver) - [INTEL Vulkan Driver](https://www.intel.com/content/www/us/en/download-center/home.html) ## Linux The Vulkan SDK packages are now being maintained by LunarG. These include the Vulkan Loader library, as well as the Vulkan Tools packages. Instructions for installing these can be found on their [Getting Started Guide](https://vulkan.lunarg.com/doc/view/latest/linux/getting_started_ubuntu.html). Once the SDK has been installed, you need to install the appropriate driver for your device. Proprietary drivers can be installed via 'apt' using PPA's for each vendor. Examples for AMD and NVIDIA are provided below. For AMD on Ubuntu v22.04: ``` $ sudo add-apt-repository ppa:oibaf/graphics-drivers $ sudo apt update $ sudo apt upgrade $ sudo apt install libvulkan1 mesa-vulkan-drivers vulkan-tools ``` For NVIDIA on Ubuntu v22.04: ``` $ sudo add-apt-repository ppa:graphics-drivers/ppa $ sudo apt update $ sudo apt upgrade # - replace ### with latest driver release (e.g. 515) $ sudo apt install nvidia-driver-### nvidia-settings libvulkan1 vulkan-tools ``` Note that only valid drivers for your system should be installed since there are reports of the Vulkan loader segfaulting just by having a non-supported driver present. Specifically, the seemingly generic `mesa-vulkan-drivers` actually includes the AMD graphics driver, which can cause problems if installed on an NVIDIA-only system. ## Mac You're better off using Halide's Metal backend instead, but it is possible to run Vulkan apps on a Mac via the MoltenVK library: - [MoltenVK Project](https://github.com/KhronosGroup/MoltenVK) The easiest way to get the necessary dependencies is to use the official MoltenVK SDK installer provided by LunarG: - [MoltenVK SDK (Latest Release)](https://sdk.lunarg.com/sdk/download/latest/mac/vulkan-sdk.dmg) Alternatively, if you have the [Homebrew](https://brew.sh/) package manager installed for MacOS, you can use it to install the Vulkan Loader and MoltenVK compatibility layer: ``` $ brew install vulkan-loader molten-vk ``` # Testing Your Vulkan Environment You can validate that everything is configured correctly by running the `vulkaninfo` app (bundled in the vulkan-utils package) to make sure your device is detected (eg): ``` $ vulkaninfo ========== VULKANINFO ========== Vulkan Instance Version: 1.3.224 Instance Extensions: count = 19 =============================== ... Layers: count = 10 ================== VK_LAYER_KHRONOS_profiles (Khronos Profiles layer) Vulkan version 1.3.224, layer version 1: Layer Extensions: count = 0 Devices: count = 1 GPU id = 0 (NVIDIA GeForce RTX 3070 Ti) Layer-Device Extensions: count = 1 ... ``` Make sure everything looks correct before continuing! # Targeting Vulkan To generate Halide code for Vulkan, simply add the `vulkan` flag to your target as well as any other optional device specific features you wish to enable for Halide: | Target Feature | Description | | -------------- | -------------------------------------------------------------- | | `vulkan` | Enables the vulkan backend | | `vk_int8` | Allows 8-bit integer storage types to be used | | `vk_int16` | Allows 16-bit integer storage types to be used | | `vk_int64` | Allows 64-bit integer storage types to be used | | `vk_float16` | Allows 16-bit floating-point values to be used for computation | | `vk_float64` | Allows 64-bit floating-point values to be used for computation | | `vk_v10` | Generates code compatible with the Vulkan v1.0+ API | | `vk_v12` | Generates code compatible with the Vulkan v1.2+ API | | `vk_v13` | Generates code compatible with the Vulkan v1.3+ API | Note that 32-bit integer and floating-point types are always available. All other optional device features are off by default (since they are not required by the Vulkan API, and thus must be explicitly enabled to ensure that the code being generated will be compatible with the device and API version being used for execution). For AOT generators add `vulkan` (and any other flags you wish to use) to the target command line option: ``` $ ./lesson_15_generate -g my_first_generator -o . target=host-vulkan-vk_int8-vk_int16 ``` For JIT apps use the `HL_JIT_TARGET` environment variable: ``` $ HL_JIT_TARGET=host-vulkan-vk_int8-vk_int16 ./tutorial/lesson_01_basics ``` # Useful Runtime Environment Variables To modify the default behavior of the runtime, the following environment variables can be used to adjust the configuration of the Vulkan backend at execution time: `HL_VK_LOADER_LIB=...` can be used to specify the full path and filename of the Vulkan loader library (e.g. `/usr/local/lib64/libvulkan.so.1`). This can be useful if you need to override the default search path, or if the loader is installed in a custom location. If not set, the runtime will search across a list of known paths. `HL_VK_LAYERS=...` will tell Halide to choose a suitable Vulkan instance that supports the given list of layers. If not set, `VK_INSTANCE_LAYERS=...` will be used instead. If neither are present, Halide will use the first Vulkan compute device it can find. Multiple layers can be specified using the appropriate environment variable list delimiter (`:` on Linux/OSX/Posix, or `;` on Windows). `HL_VK_DEVICE_TYPE=...` will tell Halide to choose which type of device to select for creating the Vulkan instance. Valid options are 'gpu', 'discrete-gpu', 'integrated-gpu', 'virtual-gpu', or 'cpu'. If not set, Halide will search for the first 'gpu' like device it can find, or fall back to the first compute device it can find. `HL_VK_ALLOC_CONFIG=...` will tell Halide to configure the Vulkan memory allocator use the given constraints specified as 5x integer values separated by the appropriate environment variable list delimiter (e.g. `N:N:N:N:N` on Linux/OSX/Posix, or `N;N;N;N;N` on Windows). These values correspond to `maximum_pool_size`, `minimum_block_size`, `maximum_block_size`, `maximum_block_count` and `nearest_multiple`. The `maximum_pool_size` constraint will tell Halide to configure the Vulkan memory allocator to never request more than N megabytes for the entire pool of allocations for the context. This includes all resource blocks used for suballocations. Setting this to a non-zero value will limit the amount device memory used by Halide, which may be useful when other applications and frameworks are competing for resources. Default is 0 ... meaning no limit. The `minimum_block_size` constraint will tell Halide to configure the Vulkan memory allocator to always request a minimum of N megabytes for a resource block, which will be used as a pool for suballocations.\ Increasing this value may improve performance while sacrificing the amount of available device memory. Default is 32MB. The `maximum_block_size` constraint will tell Halide to configure the Vulkan memory allocator to never exceed a maximum of N megabytes for a resource block. Decreasing this value may free up more memory but may impact performance, and/or restrict allocations to be unusably small. Default is 0 ... meaning no limit. The `maximum_block_count` constraint will tell Halide to configure the Vulkan memory allocator to never exceed a total of N block allocations.\ Decreasing this value may free up more memory but may impact performance, and/or restrict allocations. Default is 0 ... meaning no limit. The `nearest_multiple` constraint will tell Halide to configure the Vulkan memory allocator to always round up the requested allocation sizes to the given integer value. This is useful for architectures that require specific alignments for subregions allocated within a block. Default is 32 ... setting this to zero means no constraint. # Debug Environment Variables The following environment variables may be useful for tracking down potential issues related to Vulkan: `HL_DEBUG_CODEGEN=3` will print out debug info that includes the SPIR-V code generator used for Vulkan while it is compiling. `HL_SPIRV_DUMP_FILE=...` specifies a file to dump the binary SPIR-V generated during compilation. Useful for debugging CodeGen issues. Can be inspected, validated and disassembled via the SPIR-V tools: https://github.com/KhronosGroup/SPIRV-Tools In addition to the SPIR-V Tools, you may also wish to install the Khronos Validation Layers which provide an exhaustive suite of runtime checks that can be injected by adding `VK_LAYER_KHRONOS_validation` to the `VK_INSTANCE_LAYERS=` environment variable. To install the validation layers and the SPIR-V tools on Ubuntu v22.04: ``` $ sudo apt install vulkan-validationlayers vulkan-validationlayers-dev spirv-tools ``` To test the validation layer, you can prepend your shell command for any Vulkan enabled binary with the appropriate environment settings. For example, you can run one of the JIT-enabled correctness tests w/debug output and validation layers enabled like so: ``` $ VK_INSTANCE_LAYERS=VK_LAYER_KHRONOS_validation HL_JIT_TARGET=host-vulkan-vk_int8-vk_int16-vk_int64-vk_float16-vk_float64-vk_v13-debug ./build/test/correctness/correctness_hello_gpu ``` # Current Status All correctness tests are now passing on tested configs for Linux & Windows using the target `host-vulkan-vk_int8-vk_int16-vk_int64-vk_float16-vk_float64-vk_v13` on LLVM v14.x. MacOS passes most tests but encounters internal MoltenVK code translation issues for wide vectors, and ambiguous function calls. Python apps, tutorials and correctness tests are now passing, but the AOT cases are skipped since the runtime environment needs to be customized to locate the platform specific Vulkan loader library. Android platform support is currently being worked on. # Caveats: - Other than 32-bit floats and integers, every other data type is optional per the Vulkan spec - Float 64-bit types can be enabled, but there aren't any native math functions available in SPIR-V - Only one dynamically sized shared memory allocation can be used, but any number of fixed sized allocation are supported (up to the maximum amount allowed by the device) # Known TODO: - Performance tuning of CodeGen and Runtime - More platform support (Android is work-in-progress, RISC-V, etc) - Adapt unsupported types to supported types (if missing vk_int8 then promote to uint32_t)? - Better debugging utilities using the Vulkan debug hooks. - Allow debug symbols to be stripped from SPIR-V during codegen to reduce memory overhead for large kernels. - Investigate floating point rounding and precision (v1.3 adds more controls) - Investigate memory model usage (can Halide gain anything from these?) --- ### Doc/WebAssembly # WebAssembly Support for Halide Halide supports WebAssembly (Wasm) code generation from Halide using the LLVM backend. As WebAssembly itself is still under active development, Halide's support has some limitations. Some of the most important: - Sign-extension operations are enabled by default (but can be avoided via Target::WasmMvpOnly). - Non-trapping float-to-int conversions are enabled by default (but can be avoided via Target::WasmMvpOnly). - Fixed-width SIMD (128 bit) can be enabled via Target::WasmSimd128. - Threads have very limited support via Target::WasmThreads; see [below](#using-threads) for more details. - Halide's JIT for Wasm is extremely limited and really useful only for internal testing purposes. # Additional Tooling Requirements: - In additional to the usual install of LLVM and clang, you'll need lld. - Locally-installed version of Emscripten, 1.39.19+ Note that for all of the above, earlier versions might work, but have not been tested. # AOT Limitations Halide outputs a Wasm object (.o) or static library (.a) file, much like any other architecture; to use it, of course, you must link it to suitable calling code. Additionally, you must link to something that provides an implementation of `libc`; as a practical matter, this means using the Emscripten tool to do your linking, as it provides the most complete such implementation we're aware of at this time. - Halide ahead-of-time tests assume/require that you have Emscripten installed and available on your system, with the `EMSDK` environment variable set properly. # JIT Limitations It's important to reiterate that the WebAssembly JIT mode is not (and will never be) appropriate for anything other than limited self tests, for a number of reasons: - It actually uses an interpreter (from the WABT toolkit [https://github.com/WebAssembly/wabt]) to execute wasm bytecode; not surprisingly, this can be *very* slow. - Wasm effectively runs in a private, 32-bit memory address space; while the host has access to that entire space, the reverse is not true, and thus any `define_extern` calls require copying all `halide_buffer_t` data across the Wasm\<->host boundary in both directions. This has severe implications for existing benchmarks, which don't currently attempt to account for this extra overhead. (This could possibly be improved by modeling the Wasm JIT's buffer support as a `device` model that would allow lazy copy-on-demand.) - Host functions used via `define_extern` or `HalideExtern` cannot accept or return values that are pointer types or 64-bit integer types; this includes things like `const char *` and `user_context`. Fixing this is tractable, but is currently omitted as the fix is nontrivial and the tests that are affected are mostly non-critical. (Note that `halide_buffer_t*` is explicitly supported as a special case, however.) - Threading isn't supported at all (yet); all `parallel()` schedules will be run serially. - The `.async()` directive isn't supported at all, not even in serial-emulation mode. - You can't use `Param` (or any other arbitrary pointer type) with the Wasm jit. - You can't use `Func.debug_to_file()`, `Func.set_custom_do_par_for()`, `Func.set_custom_do_task()`, or `Func.set_custom_allocator()`. - The implementation of `malloc()` used by the JIT is incredibly simpleminded and unsuitable for anything other than the most basic of tests. - GPU usage (or any buffer usage that isn't 100% host-memory) isn't supported at all yet. (This should be doable, just omitted for now.) Note that while some of these limitations may be improved in the future, some are effectively intrinsic to the nature of this problem. Realistically, this JIT implementation is intended solely for running Halide self-tests (and even then, a number of them are fundamentally impractical to support in a hosted-Wasm environment and are disabled). In sum: don't plan on using Halide JIT mode with Wasm unless you are working on the Halide library itself. ## Using V8 as the interpreter There is experimental support for using V8 as the interpreter in JIT mode, rather than WABT. This is enabled by the CMake command line options `-DWITH_V8=ON -DWITH_WABT=OFF` (only one of them can be used at a time). You must build V8 locally V8, then specify the path to the library and headers as CMake options. This is currently only tested on x86-64-Linux and requires v8 version 9.8.177 as a minimum. The canonical instructions to build V8 are at [v8.dev](https://v8.dev/docs/build), and [there are examples for embedding v8](https://v8.dev/docs/embed). The process for Halide is summarized below. - Install [`depot_tools`](https://commondatastorage.googleapis.com/chrome-infra-docs/flat/depot_tools/docs/html/depot_tools_tutorial.html#_setting_up) - Fetch v8 source code (and install required dependencies): ``` $ gclient $ mkdir ~/v8 && cd ~/v8 $ fetch v8 $ cd ~/v8/v8 $ git checkout origin/9.8.177 ``` - Create a build configuration: `tools/dev/v8gen.py x64.release.sample` - Turn off pointer compression: `echo 'v8_enable_pointer_compression = false' >> out.gn/x64.release.sample/args.gn` - Disable the GDB-JIT interface (conflicts with LLVM): `echo 'v8_enable_gdbjit = false' >> out.gn/x64.release.sample/args.gn` - Build the static library: `autoninja -C out.gn/x64.release.sample v8_monolith` With V8 built, we can pass the CMake options: - `V8_INCLUDE_DIR`, path to V8 includes, e.g. `$HOME/v8/v8/include` - `V8_LIBRARY`, path to V8 static library, e.g. `$HOME/v8/v8/out.gn/x64.release.sample/obj/libv8_monolith.a` An example to configure Halide with V8 support, build and run an example test: ``` $ cd /path/to/halide $ export HL_TARGET=wasm-32-wasmrt-wasm_simd128 $ export HL_JIT_TARGET=${HL_TARGET} $ cmake -G Ninja \ -DWITH_WABT=OFF \ -DWITH_V8=ON \ -DV8_INCLUDE_DIR=$HOME/v8/v8/include \ -DV8_LIBRARY=$HOME/v8/v8/out.gn/x64.release.sample/obj/libv8_monolith.a \ -DHalide_TARGET=${HL_TARGET} \ /* other cmake settings here as appropriate */ $ cmake --build . $ ctest -L "correctness|generator" -j ``` # To Use Halide For WebAssembly: - Ensure WebAssembly is in LLVM_TARGETS_TO_BUILD; if you use the default (`"all"`) then it's already present, but otherwise, add it explicitly: ``` -DLLVM_TARGETS_TO_BUILD="X86;ARM;NVPTX;AArch64;PowerPC;Hexagon;WebAssembly ``` ## Enabling wasm JIT If you want to run `test_correctness` and other interesting parts of the Halide test suite (and you almost certainly will), you'll need to ensure that LLVM is built with wasm-ld: - Ensure that you have lld in LVM_ENABLE_PROJECTS: ``` cmake -DLLVM_ENABLE_PROJECTS="clang;lld" ... ``` - To run the JIT tests, set `HL_JIT_TARGET=wasm-32-wasmrt` (possibly adding `wasm_simd128`) and run CMake/CTest normally. Note that wasm testing is only supported under CMake (not via Make). ## Enabling wasm AOT If you want to test ahead-of-time code generation (and you almost certainly will), you need to install Emscripten locally. - The simplest way to install is probably via the Emscripten emsdk (https://emscripten.org/docs/getting_started/downloads.html). - To run the AOT tests, set `HL_TARGET=wasm-32-wasmrt` (possibly adding `wasm_simd128`) and run CMake/CTest normally. Note that wasm testing is only supported under CMake (not via Make). # Running benchmarks The `test_performance` benchmarks are misleading (and thus useless) for Wasm, as they include JIT overhead as described elsewhere. Suitable benchmarks for Wasm will be provided at a later date. (See https://github.com/halide/Halide/issues/5119 and https://github.com/halide/Halide/issues/5047 to track progress.) # Using Threads You can use the `wasm_threads` feature to enable use of a normal pthread-based thread pool in Halide code, but with some careful caveats: - This requires that you use a wasm runtime environment that provides pthread-compatible wrappers. At this time of this writing, the only environment known to support this well is Emscripten (when using the `-pthread` flag, and compiling for a Web environment). In this configuration, Emscripten goes to great lengths to make WebWorkers available via the pthreads API. (You can see an example of this usage in apps/HelloWasm.) Note that not all wasm runtimes support WebWorkers; generally, you need a full browser environment to make this work (though some versions of some shell tools may also support this, e.g. nodejs). - There is currently no support for using threads in a WASI environment, due to current limitations in the WASI specification. (We hope that this will improve in the future.) - There is no support for using threads in the Halide JIT environment, and no plans to add them anytime in the near-term future. # Known Limitations And Caveats - Current trunk LLVM (as of July 2020) doesn't reliably generate all of the Wasm SIMD ops that are available; see https://github.com/halide/Halide/issues/5130 for tracking information as these are fixed. - Using the JIT requires that we link the `wasm-ld` tool into libHalide; with some work this need could possibly be eliminated. - OSX and Linux-x64 have been tested. Windows hasn't; it should be supportable with some work. (Patches welcome.) - None of the `apps/` folder has been investigated yet. Many of them should be supportable with some work. (Patches welcome.) - We currently use v8/d8 as a test environment for AOT code; we may want to consider using Node or (better yet) headless Chrome instead (which is probably required to allow for using threads in AOT code). # Known TODO: - There's some invasive hackiness in Codgen_LLVM to support the JIT trampolines; this really should be refactored to be less hacky. - Can we rework JIT to avoid the need to link in wasm-ld? This might be doable, as the wasm object files produced by the LLVM backend are close enough to an executable form that we could likely make it work with some massaging on our side, but it's not clear whether this would be a bad idea or not (i.e., would it be unreasonably fragile). - Buffer-copying overhead in the JIT could possibly be dramatically improved by modeling the copy as a "device" (i.e. `copy_to_device()` would copy from host -> wasm); this would make the performance benchmarks much more useful. - Can we support threads in the JIT without an unreasonable amount of work? Unknown at this point. --- ### Doc/WebGPU # WebGPU support for Halide Halide has work-in-progress support for generating and running WebGPU shaders. This can be used in conjunction with the WebAssembly backend to bring GPU-accelerated Halide pipelines to the web. As the first version of the WebGPU standard is itself still being developed, Halide's support has some limitations and may only work with certain browsers and versions of Emscripten. ## Known limitations The following is a non-comprehensive list of known limitations: - Only 32-bit integers and floats have efficient support. - 8-bit and 16-bit integers are implemented using emulation. Future extensions to WGSL will allow them to be implemented more efficiently. - 64-bit integers and floats will likely remain unsupported until WGSL gains extensions to support them. - Wrapping native device buffer handles is not yet implemented. - You must use CMake/CTest to build/test Halide for WebGPU; using the Makefile is not supported for WebGPU testing (and probably never will be). In addition to these functional limitations, the performance of the WebGPU backend has not yet been evaluated, and so optimizations in the runtime or device codegen may be required before it becomes profitable to use. ## Running with WebAssembly via Emscripten: `HL_TARGET=wasm-32-wasmrt-webgpu` > _Tested with top-of-tree Emscripten as of 2023-02-23, against Chrome v113._ Halide can generate WebGPU code that can be integrated with WASM code using Emscripten. When invoking `emcc` to link Halide-generated objects, include these flags: `-s USE_WEBGPU=1 -s ASYNCIFY`. Tests that use AOT compilation can be run using a native WebGPU implementation that has Node.js bindings, such as [Dawn](https://dawn.googlesource.com/dawn/). You must set an environment variable named `HL_WEBGPU_NODE_BINDINGS` that has an absolute path to the bindings to run these tests, e.g. `HL_WEBGPU_NODE_BINDINGS=/path/to/dawn.node`. See [below](#setting-up-dawn) for instructions on building the Dawn Node.js bindings. JIT compilation is not supported when using WebGPU with WASM. ## Running natively: `HL_TARGET=host-webgpu` > _Tested with top-of-tree Dawn as of 2023-11-27 \[commit > b5d38fc7dc2a20081312c95e379c4a918df8b7d4\]._ For testing purposes, Halide can also target native WebGPU libraries, such as [Dawn](https://dawn.googlesource.com/dawn/) or [wgpu](https://github.com/gfx-rs/wgpu). This is currently the only path that can run the JIT correctness tests. See [below](#setting-up-dawn) for instructions on building Dawn. > Note that as of 2023-11-27, wgpu is not supported due to > [lacking `override` support for WGSL](https://github.com/gfx-rs/wgpu/issues/1762) > which we require > in order to set GPU block sizes. When targeting WebGPU with a native target, Halide defaults to looking for a build of Dawn (with several common names and suffixes); you can override this by setting the `HL_WEBGPU_NATIVE_LIB` environment variable to the absolute path to the library you want. Note that it is explicitly legal to define both `HL_WEBGPU_NATIVE_LIB` and `HL_WEBGPU_NODE_BINDINGS` at the same time; the correct executable environment will be selected based on the Halide target specified. Note that it is explicitly legal to specify both WEBGPU_NATIVE_LIB and WEBGPU_NODE_BINDINGS for the same build; the correct executable environment will be selected based on the Halide target specified. ## Setting up Dawn Building Dawn's Node.js bindings currently requires using CMake. First, [install `depot_tools`](https://commondatastorage.googleapis.com/chrome-infra-docs/flat/depot_tools/docs/html/depot_tools_tutorial.html#_setting_up) and add it to the `PATH` environment variable. Next, get Dawn and its dependencies: ``` # Clone the repo git clone https://dawn.googlesource.com/dawn cd dawn # Bootstrap the gclient configuration with Node.js bindings enabled cp scripts/standalone-with-node.gclient .gclient # Fetch external dependencies and toolchains with gclient gclient sync # Other dependencies that must be installed manually: # - golang ``` Finally, build Dawn, enabling both the Node.js bindings and shared libraries: ``` mkdir -p cd cmake -G Ninja \ -DCMAKE_BUILD_TYPE=Release \ -DDAWN_BUILD_NODE_BINDINGS=1 \ -DDAWN_ENABLE_PIC=1 \ -DBUILD_SHARED_LIBS=ON ninja dawn.node webgpu_dawn ``` This will produce the following artifacts: - Node.js bindings: `/dawn.node` - Native library: `/src/dawn/native/libwebgpu_dawn.{so,dylib,dll}` These paths can then be used for the `HL_WEBGPU_NODE_BINDINGS` and `HL_WEBGPU_NATIVE_LIB` environment variables when using Halide. ## Updating mini_webgpu.h The recommended method for updating `mini_webgpu.h` is to copy the `gen/include/dawn/webgpu.h` file from the Dawn build directory, then: - Restore the `// clang-format {off,on}` lines. - Comment out the `#include ` lines. - Remove the `void` parameter from the `WGPUProc` declaration. This guarantees a version of the WebGPU header that is compatible with Dawn. When the native API eventually stabilizes, it should be possible to obtain a header from the `webgpu-native` GitHub organization that will be compatible with Dawn, wgpu, and Emscripten. ---