julia Documentation

File: README.md Copy Chapter

[](https://perf.julialang.org/) |

| Code coverage |
|

The Julia Language Copy Chapter

Julia is a high-level, high-performance dynamic language for technical
computing. The main homepage for Julia can be found at
julialang.org. This is the GitHub
repository of Julia source code, including instructions for compiling
and installing Julia, below.

Resources Copy Chapter

• Homepage: https://julialang.org
• Install: https://julialang.org/downloads/
• Source code: https://github.com/JuliaLang/julia
• Documentation: https://docs.julialang.org
• Packages: https://julialang.org/packages/
• Discussion forum: https://discourse.julialang.org
• Zulip: https://julialang.zulipchat.com/
• Slack: https://julialang.slack.com (get an invite from https://julialang.org/slack/)
• YouTube: https://www.youtube.com/user/JuliaLanguage
• Code coverage: https://coveralls.io/r/JuliaLang/julia

Contributing to Julia Copy Chapter

We welcome contributions from developers of all experience levels, including bug fixes,
documentation improvements, tests, and performance enhancements.

New contributors are encouraged to start by reading CONTRIBUTING.md.

[!IMPORTANT]
If your pull request contains substantive contributions from a generative AI tool, please disclose so with details, and review all changes before opening. This also applies to other content, such as issues, discussions, and comments.

Learning Julia Copy Section

• Learning resources

Binary Installation Copy Chapter

The recommended way of installing Julia is to use juliaup which will install
the latest stable julia for you and help keep it up to date. It can also let
you install and run different Julia versions simultaneously. Instructions for
this can be found here. If you want to manually
download specific Julia binaries, you can find those on the Manual Downloads
page. The downloads page also provides
details on the different tiers of
support for OS and
platform combinations.

If everything works correctly, you will get a julia program and when you run
it in a terminal or command prompt, you will see a Julia banner and an
interactive prompt into which you can enter expressions for evaluation. You can
read about getting
started in the
manual.

Note: Although some OS package managers provide Julia, such
installations are neither maintained nor endorsed by the Julia
project. They may be outdated, broken and/or unmaintained. We
recommend you use the official Julia binaries instead.

Building Julia Copy Chapter

First, make sure you have all the required
dependencies installed.
Then, acquire the source code by cloning the git repository:

git clone https://github.com/JuliaLang/julia.git

and then use the command prompt to change into the resulting julia directory. By default, you will be building the latest unstable version of
Julia. However, most users should use the most recent stable version
of Julia. You can get this version by running: (replace [tag] with the desired tag)

git checkout [tag]

To build the julia executable, run make from within the julia directory.

Building Julia requires 2GiB of disk space and approximately 4GiB of virtual memory.

Note: The build process will fail badly if any of the build directory's parent directories have spaces or other shell meta-characters such as $ or : in their names (this is due to a limitation in GNU make).

Once it is built, you can run the julia executable. From within the julia directory, run

./julia

Your first test of Julia determines whether your build is working
properly. From the julia
directory, type make testall. You should see output that
lists a series of running tests; if they complete without error, you
should be in good shape to start using Julia.

You can read about getting
started
in the manual.

Detailed build instructions, should they be necessary,
are included in the build documentation.

Uninstalling Julia Copy Section

By default, Julia does not install anything outside the directory it was cloned
into and ~/.julia. Julia and the vast majority of Julia packages can be
completely uninstalled by deleting these two directories.

Source Code Organization Copy Chapter

The Julia source code is organized as follows:

Terminal, Editors and IDEs Copy Chapter

The Julia REPL is quite powerful. See the section in the manual on
the Julia REPL
for more details.

On Windows, we highly recommend running Julia in a modern terminal,
such as Windows Terminal from the Microsoft Store.

Support for editing Julia is available for many
widely used editors:
Emacs,
Vim,
Sublime Text, and many
others.

For users who prefer IDEs, we recommend using VS Code with the
julia-vscode plugin.
For notebook users, Jupyter notebook support is available through the
IJulia package, and
the Pluto.jl package provides Pluto notebooks.

File: cli/README.md Copy Chapter

cli and loader

This directory contains the code used by the Julia loader, implementing the pieces necessary to isolate ourselves from the native dynamic loader enough to reimplement useful features such as RPATH across all platforms.
This loader comprises the julia executable and the libjulia library, which are responsible for setting things up such that libjulia-internal and any other internal dependencies can be reliably loaded.
The code is organized in three pieces:

• loader_exe.c gets built into the main julia executable. It immediately loads libjulia.
• loader_lib.c gets built into the main libjulia shared library. This is the main entrypoint for the Julia runtime loading process, which occurs within jl_load_repl().
• trampolines/*.S, which contains assembly definitions for symbol forwarding trampolines. These are used to allow libjulia to re-export symbols such that a C linker can use libjulia directly for embedding usecases.

The main requirements of the loader are as follows:

• Isolation: We need to be able to load our own copy of libgcc_s.so, etc...
  On Linux/macOS, proper application of RPATH can influence the linker's decisions, however errant LD_LIBRARY_PATH entries or system libraries inserted into the build process can still interfere, not to mention Windows' lack of RPATH-like capabilities.
  To address this, the loader is built as a stand-alone binary that does not depend on the large set of dependencies that libjulia-internal itself does, and manually dlopen()'s a list of dependencies using logic similar to that of an RPATH.
• Compatibility: We need to support embedding usecases without forcing embedders to care about all of these things.
  For linking against the Julia runtime by simply providing -ljulia on the link line, we must ensure that all public interfaces, whether function symbols or data symbols, must be exported from libjulia.
  This motivates our usage of function trampolines to re-export functions from libjulia-internal, and the reason why all public data symbols are defined within libjulia, then imported into libjulia-internal for initialization.
• Flexibility: We need to be able to make use of system libraries when requested to do so by the user at build time.
  Currently, we embed the list of libraries to be dlopen()'ed within libjulia as a string (See the definition of DEP_LIBS in Make.inc and its usage in loader_lib.c).
  This is flexible enough as we do not support changing this configuration at runtime, however in the future, we may need to add some simple parsing logic in loader_lib.c to inspect a LocalPreferences.toml and construct the list of libraries to load from that.
• Speed: This whole process should be fast, especially function trampolines.
  To this end, we write everything in low-overhead assembly, borrowing inspiration from the PLT trampolines that the linker already generates when using dynamic libraries.

Public interface definition Copy Chapter

The public interface exported by libjulia is contained within .inc files stored in src; one for exported data symbols, src/jl_exported_data.inc and one for exported functions, src/jl_exported_funcs.inc.
Adding entries to the data list will cause libjulia to generate a placeholder variable declaration.
Most symbols are declared to be of type void *, however for symbols that are of a different size, they are declared along with their type.
Adding entries to the function list will cause libjulia to generate a trampoline definition (using a trampoline according to the architecture of the target processor) and then at runtime, when libjulia has successfully loaded libjulia-internal, it will dlsym() that symbol from within libjulia-internal and set it as the target of the trampoline.
All initialization will occur automatically upon successful load of libjulia, so there is no need for user code to call an initialization before invoking typical libjulia-internal functions (although initialization of the runtime itself is still necessary, e.g. calling jl_init()).

File: Compiler/README.md Copy Chapter

The `Compiler` module

This directory maintains the implementation of the Julia compiler.

Through a bootstrapping process, it is bundled into the Julia runtime as Base.Compiler.

You can also use this Compiler module as the Compiler standard library by following the steps below.

How to use Copy Chapter

To utilize this Compiler.jl standard library, you need to declare it as a dependency in
your Project.toml as follows:

Project.toml

[deps]
Compiler = "807dbc54-b67e-4c79-8afb-eafe4df6f2e1"

[compat]
Compiler = "0.1"

With the setup above, the special placeholder version (v0.1)
will be installed by default.[^1]

[^1]: Currently, only version v0.1 is registered in the General registry.

If needed, you can switch to a custom implementation of the Compiler module by running

pkg> dev /path/to/Compiler.jl # to use a local implementation

or

pkg> add https://url/of/Compiler/branch # to use a remote implementation

This feature is particularly useful for developing or experimenting with alternative compiler implementations.

[!note]
The Compiler.jl standard library is available starting from Julia v1.10.
However, switching to a custom compiler implementation is supported only from
Julia v1.12 onwards.

[!warning]
When using a custom, non-Base version of Compiler implementation, it may be necessary
to run InteractiveUtils.@activate Compiler to ensure proper functionality of certain
reflection utilities.

File: contrib/bolt/README.md Copy Chapter

BOLT only works on x86_64 and arch64 on Linux.

DO NOT STRIP THE RESULTING .so FILES, https://github.com/llvm/llvm-project/issues/56738.
If you really need to, try adding -use-gnu-stack to BOLT_ARGS.

To build a BOLT-optimized version of Julia run the following commands (cd into this directory first)

make stage1
make copy_originals
make bolt_instrument
make finish_stage1
make merge_data
make bolt

After these commands finish, the optimized version of Julia will be built in the optimized.build directory.

This doesn't align the code to support huge pages as it doesn't seem that we do that currently, this decreases the size of the .so files by 2-4mb.

File: contrib/mac/frameworkapp/README.md Copy Chapter

New Julia Launcher App

This builds the Julia framework and a launcher app and packages them in a
product archive for the macOS Installer.

Run make APPLE_DEVELOPMENT_TEAM=xxxxxxxxxx to build the product archive. The
resulting archive may be installed to the home directory with
installer -pkg~/Documents/pkgs/Julia-1.1.0.pkg -target CurrentUserHomeDirectory.
To just build the app, build the appexport make target. Read the comments at
the top of the Makefile to set appropriate code signing parameters.

The framework is installed in /Library/Frameworks and the app in
/Applications. Installation may be system-wide (i.e., relative to /) or
local to the user's home directory (i.e., $Home/Applications/Julia.app).

The julia binary is embedded in the framework at
Julia.framework/Helpers/julia.

Multiple versions of Julia may be installed at once. Each version is placed in
the Julia.framework/Versions directory. By default, the version is
identified by the Major.Minor version number but may be customized by setting
the FRAMEWORK_VERSION make variable. The resulting product archive will not
overwrite other versions but will upgrade a version if it exists. Thus, the
1.1 framework version that is actually the 3rd patch (1.1.3) will overwrite
any existing 1.1 framework version.

File: contrib/mac/app/README.md Copy Chapter

Julia OS X packaging

This builds the Julia OS X application bundle (.app folder), and stores it in a disk image
(.dmg file).

The application bundle opens Terminal.app and executes the julia binary (which opens
the REPL). All the Julia binary files and their dependencies are bundled inside this.

Its double-clickable entry point is julia-terminal (the bundle's CFBundleExecutable):
a real copy of the julia loader placed directly at Contents/MacOS/julia-terminal. When
the loader is invoked under that name it relaunches the REPL binary from the bundled tree
(Contents/Resources/julia/bin/julia) inside a new Terminal.app window rather than
starting the REPL directly in the (windowless) bundle process. See cli/loader_exe.c for
that logic.

The main executable is a real file in Contents/MacOS/ -- not a symlink, and not inside
Contents/Resources/ -- because Apple's notary rejects a bundle main executable that is a
symlink or that lives among the sealed resources. Because this copy runs from
Contents/MacOS/ rather than the tree's bin/, the loader is linked with an extra rpath
into the bundled tree so dyld can resolve libjulia from there (see cli/Makefile);
assembling the bundle is then a plain cp needing no Mach-O tools, so it runs on any
host (CI assembles the .app on Linux).

Run make to build.

Other files in this directory

• Info.plist.in is the template for the bundle's Info.plist.
• julia.icns is the Julia icon file.

File: contrib/pgo-lto-bolt/README.md Copy Chapter

BOLT only works on x86_64 and arch64 on Linux.

DO NOT STRIP THE RESULTING .so FILES, https://github.com/llvm/llvm-project/issues/56738.
If you really need to, try adding -use-gnu-stack to BOLT_ARGS.

To build a PGO+LTO+BOLT version of Julia run the following commands (cd into this directory first)

make stage1
make stage2
make copy_originals
make bolt_instrument
make finish_stage2
make merge_data
make bolt

After these commands finish, the optimized version of Julia will be built in the optimized.build directory.

This doesn't align the code to support huge pages as it doesn't seem that we do that currently, this decreases the size of the .so files by 2-4mb.

File: contrib/README.md Copy Chapter

Installation

Packagers may want to run this command via a script after package installation.

if [ -e /usr/share/icons/hicolor/icon-theme.cache ]; then
  if [ -x /usr/bin/gtk-update-icon-cache ]; then
    /usr/bin/gtk-update-icon-cache -f /usr/share/icons/hicolor >/dev/null 2>&1
  fi
fi

Debugging

File: doc/src/devdocs/aot.md Copy Chapter

Ahead of Time Compilation

This document describes the design and structure of the ahead-of-time (AOT) compilation system in Julia. This system is used when generating system images and package images. Much of the implementation described here is located in aotcompile.cpp, staticdata.c, and processor.cpp

Introduction Copy Chapter

Though Julia normally compiles code just-in-time (JIT), it is possible to compile code ahead of time and save the resulting code to a file. This can be useful for a number of reasons:

1. To reduce the time it takes to start a Julia process.
2. To reduce the time spent in the JIT compiler instead of executing code (time to first execution, TTFX).
3. To reduce the amount of memory used by the JIT compiler.

High-Level Overview Copy Chapter

The following descriptions are a snapshot of the current implementation details of the end-to-end pipeline that happens internally when the user compiles a new AOT module, such as occurs when they type using Foo. These details are likely to change over time as we implement better ways to handle them, so current implementations may not exactly match the dataflow and functions described below.

Compiling Code Images Copy Section

Firstly, the methods that need to be compiled to native code must be identified. This can only be done by actually executing the code to be compiled, as the set of methods that need to be compiled depends on the types of the arguments passed to the methods, and method invocations with certain combinations of types may not be known until runtime. During this process, the exact methods that the compiler sees are tracked for later compilation, producing a compilation trace.

!!! note

Currently when compiling images, Julia runs the trace generation in a different process than the process performing the AOT compilation. This can have impacts when attempting to use a debugger during precompilation. The best way to debug precompilation with a debugger is to use the rr debugger, record the entire process tree, use `rr ps` to identify the relevant failing process, and then use `rr replay -p PID` to replay just the failing process.

Once the methods to be compiled have been identified, they are passed to the jl_create_system_image function. This function sets up a number of data structures that will be used when serializing native code to a file, and then calls jl_create_native with the array of methods. jl_create_native runs codegen on the methods and produces one or more LLVM modules. jl_create_system_image then records some useful information about what codegen produced from the module(s).

The module(s) are then passed to jl_dump_native, along with the information recorded by jl_create_system_image. jl_dump_native contains the code necessary to serialize the module(s) to bitcode, object, or assembly files depending on the command-line options passed to Julia. The serialized code and information are then written to a file as an archive.

The final step is to run a system linker on the object files in the archive produced by jl_dump_native. Once this step is complete, a shared library containing the compiled code is produced.

Loading Code Images Copy Section

When loading a code image, the shared library produced by the linker is loaded into memory. The system image data is then loaded from the shared library. This data contains information about the types, methods, and code instances that were compiled into the shared library. This data is used to restore the state of the runtime to what it was when the code image was compiled.

If the code image was compiled with multiversioning, the loader will pick the appropriate version of each function to use based on the CPU features available on the current machine.

For system images, since no other code has been loaded, the state of the runtime is now the same as it was when the code image was compiled. For package images, the environment may have changed compared to when the code was compiled, so each method must be checked against the global method table to determine if it is still valid code.

Compiling Methods Copy Chapter

Tracing Compiled Methods Copy Section

Julia has a command-line flag to record all of the methods that are compiled by the JIT compiler, --trace-compile=filename. When a function is compiled and this flag has a filename, Julia will print out a precompile statement to that file with the method and argument types it was called with. This therefore generates a precompile script that can be used later in the AOT compilation process. The PrecompileTools package has tooling that can make taking advantage of this functionality easier for package developers.

`jl_create_system_image` Copy Section

jl_create_system_image saves all of the Julia-specific metadata necessary to later restore the state of the runtime. This includes data such as code instances, method instances, method tables, and type information. This function also sets up the data structures necessary to serialize the native code to a file. Finally, it calls jl_create_native to create one or more LLVM modules containing the native code for the methods passed to it. jl_create_native is responsible for running codegen on the methods passed to it.

`jl_dump_native` Copy Section

jl_dump_native is responsible for serializing the LLVM module containing the native code to a file. In addition to the module, the system image data produced by jl_create_system_image is compiled as a global variable. The output of this method is bitcode, object, and/or assembly archives containing the code and system image data.

jl_dump_native is typically one of the larger time sinks when emitting native code, with much of the time spent in optimizing LLVM IR and emitting machine code. Therefore, this function is capable of multithreading the optimization and machine code emission steps. This multithreading is parameterized on the size of the module, but can be explicitly overridden by setting the [JULIA_IMAGE_THREADS](@ref JULIA_IMAGE_THREADS) environment variable. The default maximum number of threads is half the number of available threads, but setting it to be lower can reduce peak memory usage during compilation.

During parallel package precompilation, the orchestrating process instead shares a single CPU-thread budget across all worker subprocesses via a named-semaphore jobserver (whose name is passed to workers through the JULIA_PRECOMPILE_JOBSERVER environment variable, and whose size defaults to one more than the number of effective CPU threads but is overridable via [JULIA_PRECOMPILE_THREADS](@ref JULIA_PRECOMPILE_THREADS)). Each worker holds one baseline token while it is CPU-active, and during its imaging phase sizes its thread pool elastically against the shared budget: it starts with the baseline thread plus whatever extra tokens are immediately available, keeps polling for tokens freed by sibling workers while it still has uncompiled shards, and returns each token to the pool as soon as the thread holding it runs out of work. Because a worker's main thread sleeps while its imaging threads run, that baseline token doubles as the imaging phase's first codegen thread rather than being counted twice. A lone worker thus expands to all available cores, workers that start while the machine is busy grow into cores as they free up, and concurrent workers share the pool without oversubscribing the machine. An explicit JULIA_IMAGE_THREADS setting takes precedence over the jobserver, unless JULIA_PRECOMPILE_THREADS is also set, in which case the jobserver takes precedence instead and JULIA_IMAGE_THREADS is ignored.

jl_dump_native can also produce native code optimized for multiple architectures, when integrated with the Julia loader. This is triggered by setting the [JULIA_CPU_TARGET](@ref JULIA_CPU_TARGET) environment variable and mediated by the multiversioning pass in the optimization pipeline. To make this work with multithreading, an annotation step is added before the module is split into submodules that are emitted on their own threads, and this annotation step uses information available throughout the entire module to decide what functions are cloned for different architectures. Once the annotation has happened, individual threads can emit code for different architectures in parallel, knowing that a different submodule is guaranteed to produce the necessary functions that will be called by a cloned function.

Some other metadata about how the module was serialized is also stored in the archive, such as the number of threads used to serialize the module and the number of functions that were compiled.

Static Linking Copy Section

The final step in the AOT compilation process is to run a linker on the object files in the archive produced by jl_dump_native. This produces a shared library containing the compiled code. This shared library can then be loaded by Julia to restore the state of the runtime. When compiling a system image, the native linker used by a C compiler is used to produce the final shared library. For package images, the LLVM linker LLD is used to provide a more consistent linking interface.

Loading Code Images Copy Chapter

Loading the Shared Library Copy Section

The first step in loading a code image is to load the shared library produced by the linker. This is done by calling jl_dlopen on the path to the shared library. This function is responsible for loading the shared library and resolving all of the symbols in the library.

Loading Native Code Copy Section

The loader first needs to identify whether the native code that was compiled is valid for the architecture that the loader is running on. This is necessary to avoid executing instructions that older CPUs do not recognize. This is done by checking the CPU features available on the current machine against the CPU features that the code was compiled for. When multiversioning is enabled, the loader will pick the appropriate version of each function to use based on the CPU features available on the current machine. If none of the multiversioned feature sets match the current CPU, the loader will throw an error.

Part of the multiversioning pass creates a number of global arrays of all of the functions in the module. When this process is multithreaded, an array of arrays is created, which the loader reorganizes into one large array with all of the functions that were compiled for this architecture. A similar process occurs for the global variables in the module.

Setting Up Julia State Copy Section

The loader then uses the global variables and functions produced from loading native code to set up Julia runtime core data structures in the current process. This setup involves adding types and methods to the Julia runtime, and making the cached native code available for use by other Julia functions and the interpreter. For package images, each method must be validated, in that the global method table's state must match the state that the package image was compiled for. In particular, if a different set of methods exists at the load time compared to compile time of the package image, the method must be invalidated and recompiled on first use. This is necessary to ensure that execution semantics remain the same regardless of if a package was precompiled or if the code was directly executed. System images do not need to perform this validation, since the global method table is empty at load time. Thus, system images have faster load times than package images.

File: doc/src/devdocs/ast.md Copy Chapter

Julia ASTs

Julia has two representations of code. First there is a surface syntax AST returned by the parser
(e.g. the Meta.parse function), and manipulated by macros. It is a structured representation
of code as it is written, constructed by julia-parser.scm from a character stream. Next there
is a lowered form, or IR (intermediate representation), which is used by type inference and code
generation. In the lowered form there are fewer types of nodes, all macros are expanded, and all
control flow is converted to explicit branches and sequences of statements. The lowered form is
constructed by julia-syntax.scm.

First we will focus on the AST, since it is needed to write macros.

Surface syntax AST Copy Chapter

Front end ASTs consist almost entirely of Exprs and atoms (e.g. symbols, numbers).
There is generally a different expression head for each visually distinct syntactic form.
Examples will be given in s-expression syntax.
Each parenthesized list corresponds to an Expr, where the first element is the head.
For example (call f x) corresponds to Expr(:call, :f, :x) in Julia.

Calls Copy Section

do syntax:

f(x) do a,b
    body
end

parses as (do (call f x) (-> (tuple a b) (block body))).

Operators Copy Section

Most uses of operators are just function calls, so they are parsed with the head call. However
some operators are special forms (not necessarily function calls), and in those cases the operator
itself is the expression head. In julia-parser.scm these are referred to as "syntactic operators".
Some operators (+ and *) use N-ary parsing; chained calls are parsed as a single N-argument
call. Finally, chains of comparisons have their own special expression structure.

Bracketed forms Copy Section

Macros Copy Section

Strings Copy Section

Doc string syntax:

"some docs"
f(x) = x

parses as (macrocall (|.| Core '@doc) (line) "some docs" (= (call f x) (block x))).

Imports and such Copy Section

using has the same representation as import, but with expression head :using
instead of :import.

To programmatically create a public statement, you can use Expr(:public, :a, :b) or,
closer to regular code, Meta.parse("public a, b"). This approach is necessary due to
[current limitations on public](@ref Export-lists). The public keyword is only
recognized at the syntactic top level within a file (parse_stmts) or module. This
restriction was implemented to prevent breaking existing code that used public as an
identifier when it was introduced in Julia 1.11.

Numbers Copy Section

Julia supports more number types than many scheme implementations, so not all numbers are represented
directly as scheme numbers in the AST.

Block forms Copy Section

A block of statements is parsed as (block stmt1 stmt2 ...).

If statement:

if a
    b
elseif c
    d
else
    e
end

parses as:

(if a (block (line 2) b)
    (elseif (block (line 3) c) (block (line 4) d)
            (block (line 6) e)))

A while loop parses as (while condition body).

A for loop parses as (for (= var iter) body). If there is more than one iteration specification,
they are parsed as a block: (for (block (= v1 iter1) (= v2 iter2)) body).

break and continue are parsed as 0-argument expressions (break) and (continue).

let is parsed as (let (= var val) body) or (let (block (= var1 val1) (= var2 val2) ...) body),
like for loops.

A basic function definition is parsed as (function (call f x) body). A more complex example:

function f(x::T; k = 1) where T
    return x+1
end

parses as:

(function (where (call f (parameters (kw k 1))
                       (:: x T))
                 T)
          (block (line 2) (return (call + x 1))))

Type definition:

mutable struct Foo{T<:S}
    x::T
end

parses as:

(struct true (curly Foo (<: T S))
        (block (line 2) (:: x T)))

The first argument is a boolean telling whether the type is mutable.

try blocks parse as (try try_block var catch_block finally_block). If no variable is present
after catch, var is #f. If there is no finally clause, then the last argument is not present.

Quote expressions Copy Section

Julia source syntax forms for code quoting (quote and :( )) support interpolation with $.
In Lisp terminology, this means they are actually "backquote" or "quasiquote" forms.
Internally, there is also a need for code quoting without interpolation.
In Julia's scheme code, non-interpolating quote is represented with the expression head inert.

inert expressions are converted to Julia QuoteNode objects.
These objects wrap a single value of any type, and when evaluated simply return that value.

A quote expression whose argument is an atom also gets converted to a QuoteNode.

Line numbers Copy Section

Source location information is represented as (line line_num file_name) where the third
component is optional (and omitted when the current line number, but not file name,
changes).

These expressions are represented as LineNumberNodes in Julia.

Macros Copy Section

Macro hygiene is represented through the expression head pair escape and hygienic-scope.
The result of a macro expansion is automatically wrapped in (hygienic-scope block module [lno]),
to represent the result of the new scope. The user can insert (escape block) inside
to interpolate code from the caller. The lno is the __source__ argument of the macro, if included.

Lowered form Copy Chapter

Lowered form (IR) is more important to the compiler, since it is used for type inference,
optimizations like inlining, and code generation. It is also less obvious to the human,
since it results from a significant rearrangement of the input syntax.

In addition to Symbols and some number types, the following data
types exist in lowered form:

• Expr

  Has a node type indicated by the head field, and an args field which is a Vector{Any} of
  subexpressions.
  While almost every part of a surface AST is represented by an Expr, the IR uses only a
  limited number of Exprs, mostly for calls and some top-level-only forms.

• SlotNumber

  Identifies arguments and local variables by consecutive numbering. It has an
  integer-valued id field giving the slot index.
  The types of these slots can be found in the slottypes field of their CodeInfo object.

• Argument

  The same as SlotNumber, but appears only post-optimization. Indicates that the
  referenced slot is an argument of the enclosing function.

• CodeInfo

  Wraps the IR of a group of statements. Its code field is an array of expressions to execute.

• GotoNode

  Unconditional branch. The argument is the branch target, represented as an index in
  the code array to jump to.

• GotoIfNot

  Conditional branch. If the cond field evaluates to false, goes to the index identified
  by the dest field.

• ReturnNode

  Returns its argument (the val field) as the value of the enclosing function.
  If the val field is undefined, then this represents an unreachable statement.

• QuoteNode

  Wraps an arbitrary value to reference as data. For example, the function f() = :a contains a
  QuoteNode whose value field is the symbol a, in order to return the symbol itself instead
  of evaluating it.

• GlobalRef

  Refers to global variable name in module mod.

• SSAValue

  Refers to a consecutively-numbered (starting at 1) static single assignment (SSA) variable inserted
  by the compiler. The number (id) of an SSAValue is the code array index of the expression whose
  value it represents.

• NewvarNode

  Marks a point where a variable (slot) is created. This has the effect of resetting a variable to undefined.

`Expr` types Copy Section

These symbols appear in the head field of Exprs in lowered form.

• call

  Function call (dynamic dispatch). args[1] is the function to call, args[2:end] are the arguments.

• invoke

  Function call (static dispatch). args[1] is the MethodInstance to call, args[2:end] are the
  arguments (including the function that is being called, at args[2]).

• static_parameter

  Reference a static parameter by index.

• =

  Assignment. In the IR, the first argument is always a SlotNumber or a GlobalRef.

• method

  Adds a method to a generic function and assigns the result if necessary.

  Has a 1-argument form and a 3-argument form. The 1-argument form arises from the syntax function foo end.
  In the 1-argument form, the argument is a symbol. If this symbol already names a function in the
  current scope, nothing happens. If the symbol is undefined, a new function is created and assigned
  to the identifier specified by the symbol. If the symbol is defined but names a non-function,
  an error is raised. The definition of "names a function" is that the binding is constant, and
  refers to an object of singleton type. The rationale for this is that an instance of a singleton
  type uniquely identifies the type to add the method to. When the type has fields, it wouldn't
  be clear whether the method was being added to the instance or its type.

  The 3-argument form has the following arguments:

  • args[1]

    A function name, or nothing if unknown or unneeded. If a symbol, then the expression
    first behaves like the 1-argument form above. This argument is ignored from then on.
    It can be nothing when methods are added strictly by type, (::T)(x) = x,
    or when a method is being added to an existing function, MyModule.f(x) = x.

  • args[2]

    A SimpleVector of argument type data. args[2][1] is a SimpleVector of the
    argument types, and args[2][2] is a SimpleVector of type variables corresponding
    to the method's static parameters.

  • args[3]

    A CodeInfo of the method itself. For "out of scope" method definitions (adding a
    method to a function that also has methods defined in different scopes) this is an
    expression that evaluates to a :lambda expression.

• struct_type

  A 7-argument expression that defines a new struct:

  • args[1]

    The name of the struct

  • args[2]

    A call expression that creates a SimpleVector specifying its parameters

  • args[3]

    A call expression that creates a SimpleVector specifying its fieldnames

  • args[4]

    A Symbol, GlobalRef, or Expr specifying the supertype (e.g., :Integer,
    GlobalRef(Core, :Any), or :(Core.apply_type(AbstractArray, T, N)))

  • args[5]

    A call expression that creates a SimpleVector specifying its fieldtypes

  • args[6]

    A Bool, true if mutable

  • args[7]

    The number of arguments to initialize. This will be the number
    of fields, or the minimum number of fields called by an inner
    constructor's new statement.

• abstract_type

  A 3-argument expression that defines a new abstract type. The
  arguments are the same as arguments 1, 2, and 4 of
  struct_type expressions.

• primitive_type

  A 4-argument expression that defines a new primitive type. Arguments 1, 2, and 4
  are the same as struct_type. Argument 3 is the number of bits.

  !!! compat "Julia 1.5"
  struct_type, abstract_type, and primitive_type were removed in Julia 1.5
  and replaced by calls to new builtins.

• global

  Declares a global binding.

• const

  Declares a (global) variable as constant.

• new

  Allocates a new struct-like object. First argument is the type. The new pseudo-function is lowered
  to this, and the type is always inserted by the compiler. This is very much an internal-only
  feature, and does no checking. Evaluating arbitrary new expressions can easily segfault.

• splatnew

  Similar to new, except field values are passed as a single tuple. Works similarly to
  splat(new) if new were a first-class function, hence the name.

• isdefined

  Expr(:isdefined, :x) returns a Bool indicating whether x has
  already been defined in the current scope.

• the_exception

  Yields the caught exception inside a catch block, as returned by jl_current_exception(ct).

• enter

  Enters an exception handler (setjmp). args[1] is the label of the catch block to jump to on
  error. Yields a token which is consumed by pop_exception.

• leave

  Pop exception handlers. args[1] is the number of handlers to pop.

• pop_exception

  Pop the stack of current exceptions back to the state at the associated enter when leaving a
  catch block. args[1] contains the token from the associated enter.

  !!! compat "Julia 1.1"
  pop_exception is new in Julia 1.1.

• inbounds

  Controls turning bounds checks on or off. A stack is maintained; if the first argument of this
  expression is true or false (true means bounds checks are disabled), it is pushed onto the stack.
  If the first argument is :pop, the stack is popped.

• boundscheck

  Has the value false if inlined into a section of code marked with @inbounds,
  otherwise has the value true.

• loopinfo

  Marks the end of the a loop. Contains metadata that is passed to LowerSimdLoop to either mark
  the inner loop of @simd expression, or to propagate information to LLVM loop passes.

• copyast

  Part of the implementation of quasi-quote. The argument is a surface syntax AST that is simply
  copied recursively and returned at run time.

• meta

  Metadata. args[1] is typically a symbol specifying the kind of metadata, and the rest of the
  arguments are free-form. The following kinds of metadata are commonly used:

  • :inline and :noinline: Inlining hints.

• foreigncall

  Statically-computed container for ccall information. The fields are:

  • args[1] : name

    The expression that'll be parsed for the foreign function.

  • args[2]::Type : RT

    The (literal) return type, computed statically when the containing method was defined.

  • args[3]::SimpleVector (of Types) : AT

    The (literal) vector of argument types, computed statically when the containing method was defined.

  • args[4]::Int : nreq

    The number of required arguments for a varargs function definition.

  • args[5]::QuoteNode{<:Union{Symbol,Tuple{Symbol,UInt16}, Tuple{Symbol,UInt16,Bool}}: calling convention

    The calling convention for the call, optionally with effects, and gc_safe (safe to execute concurrently to GC.).

  • args[6:5+length(args[3])] : arguments

    The values for all the arguments (with types of each given in args[3]).

  • args[6+length(args[3])+1:end] : gc-roots

    The additional objects that may need to be gc-rooted for the duration of the call.
    See [Working with LLVM](@ref Working-with-LLVM) for where these are derived from and how they get handled.

• new_opaque_closure

  Constructs a new opaque closure. The fields are:

  • args[1] : signature

    The function signature of the opaque closure. Opaque closures don't participate in dispatch, but the input types can be restricted.

  • args[2] : lb

    Lower bound on the output type. (Defaults to Union{})

  • args[3] : ub

    Upper bound on the output type. (Defaults to Any)

  • args[4] : constprop

    Indicates whether the opaque closure's identity may be used for constant
    propagation. The @opaque macro enables this by default, but this will
    cause additional inference which may be undesirable and prevents the
    code from running during precompile.
    If args[4] is a method, the argument is considered skipped.

  • args[5] : method

    The actual method as an opaque_closure_method expression.

  • args[6:end] : captures

    The values captured by the opaque closure.

  !!! compat "Julia 1.7"
  Opaque closures were added in Julia 1.7

[Method](@id ast-lowered-method) Copy Section

A unique'd container describing the shared metadata for a single method.

• name, module, file, line, sig

  Metadata to uniquely identify the method for the computer and the human.

• ambig

  Cache of other methods that may be ambiguous with this one.

• specializations

  Cache of all MethodInstance ever created for this Method, used to ensure uniqueness.
  Uniqueness is required for efficiency, especially for incremental precompile and
  tracking of method invalidation.

• source

  The original source code (if available, usually compressed).

• generator

  A callable object which can be executed to get specialized source for a specific method signature.

• roots

  Pointers to non-AST things that have been interpolated into the AST, required by
  compression of the AST, type-inference, or the generation of native code.

• nargs, isva, called, is_for_opaque_closure,

  Descriptive bit-fields for the source code of this Method.

• primary_world

  The world age that "owns" this Method.

MethodInstance Copy Section

A unique'd container describing a single callable signature for a Method.
See especially [Proper maintenance and care of multi-threading locks](@ref Proper-maintenance-and-care-of-multi-threading-locks)
for important details on how to modify these fields safely.

• specTypes

  The primary key for this MethodInstance. Uniqueness is guaranteed through a
  def.specializations lookup.

• def

  The Method that this function describes a specialization of. Or a Module,
  if this is a top-level Lambda expanded in Module, and which is not part of a Method.

• sparam_vals

  The values of the static parameters in specTypes.
  For the MethodInstance at Method.unspecialized, this is the empty SimpleVector.
  But for a runtime MethodInstance from the MethodTable cache, this will always be defined and indexable.

• backedges

  We store the reverse-list of cache dependencies for efficient tracking of incremental reanalysis/recompilation work that may be needed after new method definitions.
  This works by keeping a list of the other MethodInstance that have been inferred or optimized to contain a possible call to this MethodInstance.
  Those optimization results might be stored somewhere in the cache, or it might have been the result of something we didn't want to cache, such as constant propagation.
  Thus we merge all of those backedges to various cache entries here (there's almost always only the one applicable cache entry with a sentinel value for max_world anyways).

• cache

  Cache of CodeInstance objects that share this template instantiation.

• precompile

  If set, this MethodInstance will be compiled and added to the output system image.

CodeInstance Copy Section

• def

  The MethodInstance that this cache entry is derived from.

• owner

  A token that represents the owner of this CodeInstance. Will use jl_egal to match.

• rettype/rettype_const

  The inferred return type for the specFunctionObject field, which (in most cases) is
  also the computed return type for the function in general.

• inferred

  May contain a cache of the inferred source for this function,
  or it could be set to nothing to just indicate rettype is inferred.

• fptr

  The generic jlcall entry point.

• jlcall_api

  The ABI to use when calling fptr. Some significant ones include:

  • 0 - Not compiled yet
  • 1 - JL_CALLABLE jl_value_t *(*)(jl_value_t *f, jl_value_t *args[nargs], uint32_t nargs)
  • 2 - Constant (value stored in rettype_const)
  • 3 - With Static-parameters forwarded jl_value_t *(*)(jl_svec_t *sparams, jl_value_t *f, jl_value_t *args[nargs], uint32_t nargs)
  • 4 - Run in interpreter jl_value_t *(*)(jl_method_instance_t *meth, jl_value_t *f, jl_value_t *args[nargs], uint32_t nargs)

• min_world / max_world

  The range of world ages for which this method instance is valid to be called.
  If max_world is the special token value -1, the value is not yet known.
  It may continue to be used until we encounter a backedge that requires us to reconsider.

• Timing fields

  • time_infer_total: Total cost of computing inferred originally as wall-time from start to finish.

  • time_infer_cache_saved: The cost saved from time_infer_total by having caching.
    Adding this to time_infer_total should give a stable estimate for comparing the cost
    of two implementations or one implementation over time. This is generally an
    over-estimate of the time to infer something, since the cache is frequently effective
    at handling repeated work.

  • time_infer_self: Self cost of julia inference for inferred (a portion of
    time_infer_total). This is simply the incremental cost of compiling this one method,
    if given a fully populated cache of all call targets, even including constant
    inference results and LimitedAccuracy results, which generally are not in a cache.

  • time_compile: Self cost of llvm JIT compilation (e.g. of computing invoke from
    inferred). A total cost estimate can be computed by walking all of the edges
    contents and summing those, while accounting for cycles and duplicates. (This field
    currently does not include any measured AOT compile times.)

CodeInfo Copy Section

A (usually temporary) container for holding lowered (and possibly inferred) source code.

• code

  An Any array of statements

• slotnames

  An array of symbols giving names for each slot (argument or local variable).

• slotflags

  A UInt8 array of slot properties, represented as bit flags:

  • 0x02 - assigned (only false if there are no assignment statements with this var on the left)
  • 0x08 - used (if there is any read or write of the slot)
  • 0x10 - statically assigned once
  • 0x20 - might be used before assigned. This flag is only valid after type inference.

• ssavaluetypes

  Either an array or an Int.

  If an Int, it gives the number of compiler-inserted temporary locations in the
  function (the length of code array). If an array, specifies a type for each location.

• ssaflags

  Statement-level 32 bits flags for each expression in the function.
  See the definition of jl_code_info_t in julia.h for more details.

These are only populated after inference (or by generated functions in some cases):

• debuginfo

  An object to retrieve source information for each statements, see
  How to interpret line numbers in a CodeInfo object.

• rettype

  The inferred return type of the lowered form (IR). Default value is Any. This is
  mostly present for convenience, as (due to the way OpaqueClosures work) it is not
  necessarily the rettype used by codegen.

• parent

  The MethodInstance that "owns" this object (if applicable).

• edges

  Forward edges to method instances that must be invalidated.

• min_world/max_world

  The range of world ages for which this code was valid at the time when it had been inferred.

Optional Fields:

• slottypes

  An array of types for the slots.

• method_for_inference_limit_heuristics

  The method_for_inference_heuristics will expand the given method's generator if
  necessary during inference.

Boolean properties:

• propagate_inbounds

  Whether this should propagate @inbounds when inlined for the purpose of eliding
  @boundscheck blocks.

UInt8 settings:

• constprop, inlineable

  • 0 = use heuristic
  • 1 = aggressive
  • 2 = none

• purity
  Constructed from 5 bit flags:

  • 0x01 << 0 = this method is guaranteed to return or terminate consistently (:consistent)
  • 0x01 << 1 = this method is free from externally semantically visible side effects (:effect_free)
  • 0x01 << 2 = this method is guaranteed to not throw an exception (:nothrow)
  • 0x01 << 3 = this method is guaranteed to terminate (:terminates_globally)
  • 0x01 << 4 = the syntactic control flow within this method is guaranteed to terminate (:terminates_locally)

  See the documentation of Base.@assume_effects for more details.

How to interpret line numbers in a `CodeInfo` object

There are 2 common forms for this data: one used internally that compresses the data somewhat and one used in the compiler.
They contain the same basic info, but the compiler version is all mutable while the version used internally is not.

Many consumers may be able to call Base.IRShow.buildLineInfoNode,
Base.IRShow.append_scopes!, or Stacktraces.lookup(::InterpreterIP) to avoid needing to
(re-)implement these details specifically.

The definitions of each of these are:

struct Core.DebugInfo
    @noinline
    def::Union{Method,MethodInstance,Symbol}
    linetable::Union{Nothing,DebugInfo}
    edges::SimpleVector{DebugInfo}
    codelocs::String # compressed data
end
mutable struct Core.Compiler.DebugInfoStream
    def::Union{Method,MethodInstance,Symbol}
    linetable::Union{Nothing,DebugInfo}
    edges::Vector{DebugInfo}
    firstline::Int32 # the starting line for this block (specified by an index of 0)
    codelocs::Vector{Int32} # for each statement:
        # index into linetable (if defined), else a line number (in the file represented by def)
        # then index into edges
        # then index into edges[linetable]
end

• def : where this DebugInfo was defined (the Method, MethodInstance, or Symbol of file scope, for example)

• linetable

  Another DebugInfo that this was derived from, which contains the actual line numbers,
  such that this DebugInfo contains only the indexes into it. This avoids making copies,
  as well as makes it possible to track how each individual statement transformed from
  source to optimized, not just the separate line numbers. If def is not a Symbol, then
  that object replaces the current function object for the metadata on what function is
  conceptually being executed (e.g. think Cassette transforms here). The codelocs values
  described below also are interpreted as an index into the codelocs in this object,
  instead of being a line number itself.

• edges : Vector of the unique DebugInfo for every function inlined into this (which
  recursively have the edges for everything inlined into them).

• firstline (when uncompressed to DebugInfoStream)

  The line number associated with the begin statement (or other keyword such as
  function or quote) that delineates where this code definition "starts".

• codelocs (when uncompressed to DebugInfoStream)

  A vector of indices, with 3 values for each statement in the IR plus one for the
  starting point of the block, that describe the stacktrace from that point:

  1. the integer index into the linetable.codelocs field, giving the
     original location associated with each statement (including its syntactic edges),
     or zero indicating no change to the line number from the previously
     executed statement (which is not necessarily syntactic or lexical prior),
     or the line number itself if the linetable field is nothing.
  2. the integer index into edges, giving the DebugInfo inlined there,
     or zero if there are no edges.
  3. (if entry 2 is non-zero) the integer index into edges[].codelocs,
     to interpret recursively for each function in the inlining stack,
     or zero indicating to use edges[].firstline as the line number.

  Special codes include:

  • (zero, zero, *) : no change to the line number or edges from the previous statement
    (you may choose to interpret this either syntactically or lexically). The inlining
    depth also might have changed, though most callers should ignore that.
  • (zero, non-zero, *) : no line number, just edges (usually because of
    macro-expansion into top-level code).