File: README.md
mold: A Modern Linker
mold is a faster drop-in replacement for existing Unix linkers. It is several
times quicker than the LLVM lld linker, the second-fastest open-source linker,
which I initially developed a few years ago. mold aims to enhance developer
productivity by minimizing build time, particularly in rapid
debug-edit-rebuild cycles.
Here is a performance comparison of GNU ld, GNU gold, LLVM lld, and
mold when linking final debuginfo-enabled executables for major large
programs on a simulated 16-core, 32-thread machine.
| Program (linker output size) | GNU ld | GNU gold | LLVM lld | mold |
|---|---|---|---|---|
| MySQL 8.3 (0.47 GiB) | 10.84s | 7.47s | 1.64s | 0.46s |
| Clang 19 (1.56 GiB) | 42.07s | 33.13s | 5.20s | 1.35s |
| Chromium 124 (1.35 GiB) | N/A | 27.40s | 6.10s | 1.52s |
mold is so fast that it is only 2x slower than the cp command on the same
machine. If you find that mold is not faster than other linkers, feel
free to file a bug report.
mold supports x86-64, i386, ARM64, ARM32, 64-bit/32-bit little/big-endian
RISC-V, 32-bit PowerPC, 64-bit big-endian PowerPC ELFv1, 64-bit little-endian
PowerPC ELFv2, s390x, 64-bit/32-bit LoongArch, SPARC64, m68k, and SH-4.
Why does linking speed matter?
If you are using a compiled language such as C, C++, or Rust, a build consists
of two phases. In the first phase, a compiler compiles source files into
object files (.o files). In the second phase, a linker takes all object
files and combines them into a single executable or shared library file.
The second phase can be time-consuming if your build output is large. mold can
speed up this process, saving you time and preventing distractions while
waiting for a lengthy build to finish. The difference is most noticeable
during rapid debug-edit-rebuild cycles.
Installation
Binary packages for the following systems are currently available:
How to Build
mold is written in C++20, so if you build mold yourself, you will need a
recent version of a C++ compiler and a C++ standard library. We recommend GCC
10.2 or Clang 16.0.0 (or later) and libstdc++ 10 or libc++ 7 (or later).
Install Dependencies
To install build dependencies, run ./install-build-deps.sh in this
directory. It will detect your Linux distribution and attempt to install the
necessary packages.
Compile mold
git clone --branch stable https://github.com/rui314/mold.git
cd mold
sudo ./install-build-deps.sh
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_CXX_COMPILER=c++ -B build
cmake --build build -j$(nproc)
sudo cmake --build build --target installYou might need to pass a C++20 compiler command name to cmake. In the
example above, c++ is passed. If that doesn't work for you, try a specific
version of a compiler, such as g++-10 or clang++-12.
By default, mold is installed to /usr/local/bin. You can change the
installation location by passing -DCMAKE_INSTALL_PREFIX=<directory>.
For other cmake options, see the comments in CMakeLists.txt.
If you are not using a recent enough Linux distribution, or if cmake does
not work for you for any reason, you can use Podman to build mold in a
container. To do so, run ./dist.sh in this directory instead of usingcmake. The shell script will pull a container image, build mold and auxiliary
files inside it, and package them into a single tar file nameddist/mold-$version-$arch-linux.tar.gz. You can extract the tar file anywhere
and use the mold executable in it.
How to use
A classic way to use mold
On Unix, the linker command (usually /usr/bin/ld) is indirectly invoked by
the compiler driver (typically cc, gcc, or clang), which is in turn
indirectly invoked by make or other build system commands.
If you can specify an additional command line option for your compiler driver
by modifying the build system's config files, add one of the following flags
to use mold instead of /usr/bin/ld:
For Clang: pass
-fuse-ld=moldFor GCC 12.1.0 or later: pass
-fuse-ld=moldFor GCC before 12.1.0: the
-fuse-ldoption does not acceptmoldas a
valid argument, so you need to use the-Boption instead. The-Boption
tells GCC where to look for external commands likeld.If you have installed mold with
make install, there should be a directory
named/usr/libexec/mold(or/usr/local/libexec/mold, depending on your$PREFIX), and theldcommand should be there. Theldis actually a
symlink tomold. So, all you need is to pass-B/usr/libexec/mold(or-B/usr/local/libexec/mold) to GCC.
If you haven't installed ld.mold to any $PATH, you can still pass-fuse-ld=/absolute/path/to/mold to clang to use mold. However, GCC does not
accept an absolute path as an argument for -fuse-ld.
If you are using Rust
Create .cargo/config.toml in your project directory with the following:
[target.'cfg(target_os = "linux")']
linker = "clang"
rustflags = ["-C", "link-arg=-fuse-ld=/path/to/mold"]where /path/to/mold is an absolute path to the mold executable. In the
example above, we use clang as a linker driver since it always accepts the-fuse-ld option. If your GCC is recent enough to recognize the option, you
may be able to remove the linker = "clang" line.
[target.'cfg(target_os = "linux")']
rustflags = ["-C", "link-arg=-fuse-ld=mold"]If you want to use mold for all projects, add the above snippet to~/.cargo/config.toml.
If you are using Nim
Create config.nims in your project directory with the following:
when findExe("mold").len > 0 and defined(linux):
switch("passL", "-fuse-ld=mold")where mold must be included in the PATH environment variable. In this
example, gcc is used as the linker driver. Use the -fuse-ld option if your
GCC is recent enough to recognize this option.
If you want to use mold for all projects, add the above snippet to~/.config/config.nims.
If you are using Conan package manager
You can configure Conan to download the latest
version of mold and use it as the linker when building your dependencies and
projects from source. Please see the instructions here.
mold -run
It is sometimes very hard to pass an appropriate command line option to cc
to specify an alternative linker. To address this situation, mold has a
feature to intercept all invocations of ld, ld.bfd, ld.lld, or ld.gold
and redirect them to itself. To use this feature, run make (or another build
command) as a subcommand of mold as follows:
mold -run make <make-options-if-any>Internally, mold invokes a given command with the LD_PRELOAD environment
variable set to its companion shared object file. The shared object file
intercepts all function calls to exec(3)-family functions to replaceargv[0] with mold if it is ld, ld.bf, ld.gold, or ld.lld.
GitHub Actions
You can use our setup-mold GitHub
Action to speed up GitHub-hosted continuous builds. Although GitHub Actions
run on a 4 core machine, mold is still significantly faster than the default
GNU linker, especially when linking large programs.
Verify that you are using mold
mold leaves its identification string in the .comment section of an output
file. You can print it out to verify that you are actually using mold.
$ readelf -p .comment <executable-file>
String dump of section '.comment':
[ 0] GCC: (Ubuntu 10.2.0-5ubuntu1~20.04) 10.2.0
[ 2b] mold 9a1679b47d9b22012ec7dfbda97c8983956716f7If mold is present in the .comment section, the file was created by mold.
Online manual
Since mold is a drop-in replacement, you should be able to use it without
reading its manual. However, if you need it, mold's man page
is available online. You can read the same manual by running man mold.
Why is mold so fast?
One reason is that it utilizes faster algorithms and more efficient data
structures compared to other linkers. Another reason is that mold is highly
parallelized.
Here is a side-by-side comparison of per-core CPU usage for lld (left) and
mold (right), linking the same program, a Chromium executable.
As you can see, mold uses all available cores throughout its execution and
finishes quickly. In contrast, lld fails to utilize available cores most of
the time. In this demo, the maximum parallelism is artificially capped at 16,
so that the bars fit in the GIF.
For details, please see the design notes.
Sponsors
It is taken for granted nowadays that compiler toolchains can be easily
installed and used for free, and people may not think too much about the
individuals behind these "free tools". mold supports many projects, but it
is essentially a one-person project. This situation is similar to the one
depicted in the following xkcd illustration.
If you think that the "Nebraska guy" should be rewarded, please consider
becoming our GitHub sponsor!
We thank everyone who sponsors our project. In particular, we'd like to acknowledge
the following people and organizations who have sponsored $128/month or more:
Corporate sponsors
Individual sponsors
File: docs/bugs.md
This is a note about interesting bugs that I met during the
development of the mold linker.
GNU IFUNC
Problem: A statically-linked "hello world" program mysteriously
crashed in __libc_start_main function which is called just after_start.
Investigation: I opened up gdb and found that the program reads a
bogus value from some array. It looks like memcpy failed to copy
proper data there. After some investigation, I noticed that memcpy
did't copy data at all but instead returned the address of__memcpy_avx_unaligned function, which is a real memcpy function
optimized for machines with the AVX registers.
This odd issue was caused by the GNU IFUNC mechanism. That is, if a
function symbol has type STT_GNU_IFUNC, the function does not do
what its name suggests to do but instead returns a pointer to a
function that does the actual job. In this case, memcpy is an IFUNC
function, and it returns an address of __memcpy_avx_unaligned which
is a real memcpy function.
IFUNC function addresses are stored to .got section in an ELF
executable. The dynamic loader executes all IFUNC functions at
startup to replace their GOT entries with their return values. This
mechanism allows programs to choose the best implementation among
variants of the same function at runtime based on the machine info.
If a program is statically-linked, there's no dynamic loader that
rewrites the GOT entries. Therefore, if a program is
statically-linked, a libc's startup routine does that on behalf of the
dynamic loader. Concretely, a startup routine interprets all dynamic
relocations between __rela_iplt_start and __rela_iplt_stop
symbols. It is linker's responsibility to emit dynamic relocations
for IFUNC symbols even if it is linking a statically-linked program
and mark the beginning and the ending of a .rela.dyn section with
the symbols, so that the startup routine can find the relocations.
The bug was my linker didn't define __rela_iplt_start and__rela_iplt_stop symbols. Since these symbols are weak, they are
initialized to zero. From the point of the initializer function,
there's no dynamic relocations between __rela_iplt_start and__rela_iplt_stop symbols. That left GOT entries for IFUNC symbols
untouched.
The proper fix was to emit dynamic relocations for IFUNC symbols and
define the linker-synthesized symbols. I did that, and the bug was
fixed.
stdio buffering
Problem: A statically-linked "Hello world" program prints out the
message if executed as ./hello, but it doesn't output anything if
executed as ./hello | cat.
Investigation: I knew that the default buffering mode for stdout is
line buffering (buffer is flushed on every '\n'), but if it is not
connected to the terminal (i.e. isatty(2) returns 0 onSTDOUT_FILENO), it automatically switches to full buffering (buffer
is flushed when it becomes full). So, it looks like libc failed to
flush the stdout on program exit for some reason.
I traced all function calls using gdb and noticed that __libc_atexit
was not called. That function seemed to be responsible for buffer
flushing. I don't know how exactly I found the root cause, but after
spending an hour or two, I found that __start___libc_atexit and__stop___libc_atexit have value 0 in my linker's output while they
mark a section containing the address of __libc_atexit in GNU ld's
output.
So, libc doesn't directly call __libc_atexit but instead call all
function pointers between __start___libc_atexit and__stop___libc_atexit symbols. libc puts __libc_atexit address in_libc_atexit section, expecting that the linker automatically
creates the start and the end marker symbols for the section.
There's an obscure linker feature: if a section name is valid as a C
identifier (e.g. foo or _foo_bar but not .foo), the linker
automatically creates marker symbols by prepending __start_ and__stop_ to the section name. My linker lacked the feature.
I implemented the feature, and the bug was fixed.
TLS variable initialization
Problem: A statically-linked "hello world" program crashes after
reading a thread-local variable.
Investigation: Thread-local variables are very different from other
types of varaibles because there may be more than one instance of the
same variable in memory. Each thread has its copy of thread-local
varaibles. %fs segment register points the end of the variable area
for the current thread, and the variables are accessed as an offset
from %fs.
Thread-local variables may be initialized (e.g. thread_local int x = 5;). The linker gathers all thread-local variables and put them intoPT_TLS segment. At runtime, the contents of the segment is used as
an "initialization image" for new threads. When a new thread is
created, the image is memcpy'ed to the new thread's thread-local
variable area. The initialization image itself is read-only at
runtime.
It took more than a day to find out that memcpy copies the
initialization image to a different place than the thread-local
variables reside. That means, thread-local variables had garbage as
initial values, and the program crashed when using them.
The problem is that I set a very large value (4096) to the alignment
of PT_TLS segment. All PT_LOAD segments are naturally aligned to
the page boundary, so I use the same value for PT_TLS, but that was
a mistake. When a thread initialization routine sets a value to %fs,
it first aligns the end of the thread-local variable area address toPT_TLS alignment value. So, if you set a large value to PT_TLS
alignment, %fs is set to a wrong place.
I fixed PT_TLS alignment, and the bug was gone.
stdio buffering (another issue)
I noticed that a dynamically-linked "hello world" program didn't
flush its stdout buffer on exit. The cause of the problem was that the
executable had more than one DT_NEEDED entry for libc.so.
DT_NEEDED entries in .dynamic section specify a list of shared
object file names which need to be linked at runtime. I added one
DT_NEEDED entry for each library specified with the -l option.
The pitfall is, unlike object files, libraries are allowed to
appear more than once in a command line, and the linker has to
de-duplicate them before processing. Adding more than one DT_NEEDED
entry for the same shared object causes mysterious issues like this.
Copy relocations and symbol aliases
environ, _environ and __environ point to the same location in libc.so,
so when we create a copy relocation for one of the symbols, we need to
do that for all of them. Otherwise, they'll end up pointing to different
places which causes a very mysterious issue.
DT_DEBUG and gdb
If you forget to add an entry with type DT_DEBUG to .dynamic, gdb'sinfo sharedlibrary command doesn't print out a list of shared
libraries loaded to memory. The value of the entry doesn't matter, so
it can be just zero. The existence of it is important.
EH_FRAME_BEGIN in libgcc/crtstuff.c
File: docs/coding-guidelines.md
Coding Guidelines
mold is written in C++20, but as is the case with every C++ project,
it has local coding rules. In this document, I'll explain some of them
and try to give justifications for why I chose such rules.
DOs
Always use
i64(which is a type alias forint64_tin mold) for
integers unless you have a reason not to. For example, even if you know
that a loop counter won't exceed 100, you should stop thinking about it
and just usei64.Justification: Local variables are usually on CPU registers, so on
64-bit CPUs, there's no performance peanlty on choosingi64overi32. Even if a compiler has to spill register values to the stack,
I don't think there's an observable difference betweeni32andi64. Therefore, extra 32 bits are essentially free. On 32-bit CPUs,
they are not free, but that's OK because we are writing mold for modern
computers. mold will still run on 32-bit computers but a bit slowly.By always using
i64, we can eliminate the need to think about the
"right" size for each variable. It also reduces the risk of integer
overflow.Exceptions: If you have to allocate a very large number (e.g. millions)
of the same object, its size matters. In that case, use a smaller type.
DON'Ts
Don't use
autounless its actual type is obvious in the very narrow
context. Currently, we useautoonly for lambdas.Justification: I think
automakes code writing easier but code reading
harder, because readers have to make a guess as to what is the actual
type ofauto. If you are already familiar with the existing codebase,
you may be able to guess it easily, but that's not always the case.
I want to keep the mold codebase friendly to first-time readers.Don't over-use inheritance. In mold, most classes don't have parents,
and even if they do, their class hierarchy is very shallow. Currently
its height is just two (i.e. abstract classes and their implementations).Justification: Designing class hierarchies is fun as it feels like
taxonomy, but I don't think that always help writing code. It looks like
simpler class hierarchy makes its code simpler.
File: docs/design.md
[This document was written in 2020, and the contents are outdated.
Specifically, we no longer believe that object preloading is a good
idea. That being said, most of the points in this document still hold
even today. Therefore, I'll keep this document as-is.]
Design and implementation of mold
For the rest of this documentation, I'll explain the design and the
implementation of mold. If you are only interested in using mold, you
don't need to read the below.
Motivation
Here is why I'm writing a new linker:
Even though lld has significantly improved the situation, linking is
still one of the slowest steps in a build. It is especially
annoying when I changed one line of code and had to wait for a few
seconds or even more for a linker to complete. It should be
instantaneous. There's a need for a faster linker.The number of cores on a PC has increased a lot lately, and this
trend is expected to continue. However, the existing linkers can't
take the advantage of the trend because they don't scale well for more
cores. I have a 64-core/128-thread machine, so my goal is to create
a linker that uses the CPU nicely. mold should be much faster than
other linkers on 4 or 8-core machines too, though.It looks to me that the designs of the existing linkers are somewhat
too similar, and I believe there are a lot of drastically different
designs that haven't been explored yet. Developers generally don't
care about linkers as long as they work correctly, and they don't
even think about creating a new one. So there may be lots of low
hanging fruits there in this area.
Basic design
In order to achieve a
cp-like performance, the most important
thing is to fix the layout of an output file as quickly as possible, so
that we can start copying actual data from input object files to an
output file as soon as possible.Copying data from input files to an output file is I/O-bounded, so
there should be room for doing computationally-intensive tasks while
copying data from one file to another.We should allow the linker to preload object files from disk and
parse them in memory before a complete set of input object files
is ready. To do so, we need
to split the linker into two in such a way that the latter half of
the process finishes as quickly as possible by speculatively parsing
and preprocessing input files in the first half of the process.One of the most computationally-intensive stage among linker stages
is symbol resolution. To resolve symbols, we basically have to throw
all symbol strings into a hash table to match undefined symbols with
defined symbols. But this can be done in the preloading stage using
string interning.Object files may contain a special section called a mergeable string
section. The section contains lots of null-terminated strings, and
the linker is expected to gather all mergeable string sections and
merge their contents. So, if two object files contain the same
string literal, for example, the resulting output will contain a
single merged string. This step is computationally intensive, but string
merging can be done in the preloading stage using string interning.Static archives (.a files) contain object files, but the static
archive's string table contains only defined symbols of member
object files and lacks other types of symbols. That makes static
archives unsuitable for speculative parsing. Therefore, the linker
should ignore the symbol table of static archive and directly read
static archive members.If there's a relocation that uses a GOT of a symbol, then we have to
create a GOT entry for that symbol. Otherwise, we shouldn't. That
means we need to scan all relocation tables to fix the length and
the contents of a .got section. This is computationally intensive,
but this step is parallelizable.
Linker Script
Linker script is an embedded language for the linker. It is mainly
used to control how input sections are mapped to output sections and
the layout of the output, but it can also do a lot of tricky stuff.
Its feature is useful especially for embedded programming, but it's
also an awfully underdocumented and complex language.
We have to implement a subset of the linker script language anwyay,
because on Linux, /usr/lib/x86_64-linux-gnu/libc.so is (despite its
name) not a shared object file but actually an ASCII file containing
linker script code to load the actual libc.so file. But the feature
set for this purpose is very limited, and it is okay to implement them
to mold.
Besides that, we really don't want to implement the linker script
language. But at the same time, we want to satisfy the user needs that
are currently satisfied with the linker script language. So, what
should we do? Here is my observation:
Linker script allows doing a lot of tricky stuff, such as specifying
the exact layout of a file, inserting arbitrary bytes between
sections, etc. But most of them can be done with a post-link binary
editing tool (such asobjcopy).It looks like there are two things that truly cannot be done by a
post-link editing tool: (a) mapping input sections to output
sections, and (b) applying relocations.
From the above observation, I believe we need to provide only the
following features instead of the entire linker script language:
A method to specify how input sections are mapped to output
sections, anda method to set addresses to output sections, so that relocations
are applied based on desired addresses.
I believe everything else can be done with a post-link binary editing
tool.
Details
As we aim to the 1-second goal for Chromium, every millisecond
counts. We can't ignore the latency of process exit. If we mmap a
lot of files, _exit(2) is not instantaneous but takes a few hundred
milliseconds because the kernel has to clean up a lot of
resources. As a workaround, we should organize the linker command as
two processes; the first process forks the second process, and the
second process does the actual work. As soon as the second process
writes a result file to a filesystem, it notifies the first process,
and the first process exits. The second process can take time to
exit, because it is not an interactive process.At least on Linux, it looks like the filesystem's performance to
allocate new blocks to a new file is the limiting factor when
creating a new large file and filling its contents using mmap.
If you already have a large file in the buffer cache, writing to it is
much faster than creating a new fresh file and writing to it.
Based on this observation, mold overwrites to an existing
executable file if exists. My quick benchmark showed that I could
save 300 milliseconds when creating a 2 GiB output file.
Linux doesn't allow to open an executable for writing if it is
running (you'll get a "text busy" error if you attempt). mold
falls back to the usual way if it fails to open an output file.The output from the linker should be deterministic for the sake of
build reproducibility
and ease of debugging. This might add a little bit of overhead to
the linker, but that shouldn't be too much.A .build-id, a unique ID embedded to an output file, is usually
computed by applying a cryptographic hash function (e.g. SHA-1) to
an output file. This is a slow step, but we can speed it up by
splitting a file into small chunks, computing SHA-1 for each chunk,
and then computing SHA-1 of the concatenated SHA-1 hashes
(i.e. constructing a Merkle
Tree of height 2).
Modern x86 processors have purpose-built instructions for SHA-1 and
can compute SHA-1 pretty quickly at about 2 GiB/s. Using 16
cores, a build-id for a 2 GiB executable can be computed in 60 to 70
milliseconds.BFD, gold, and lld support section garbage collection. That is, a
linker runs a mark-sweep garbage collection on an input graph, where
sections are vertices and relocations are edges, to discard all
sections that are not reachable from the entry point symbol
(i.e._start) or a few other root sections. In mold, we are using
multiple threads to mark sections concurrently.Similarly, BFD, gold an lld support Identical Comdat Folding (ICF)
as yet another size optimization. ICF merges two or more read-only
sections that happen to have the same contents and relocations.
To do that, we have to find isomorphic subgraphs from larger graphs.
I implemented a new algorithm for mold, which is 5x faster than lld
to do ICF for Chromium (from 5 seconds to 1 second).Intel Threading Building
Blocks (TBB) is a good
library for parallel execution and has several concurrent
containers. We are particularly interested in usingparallel_for_eachandconcurrent_hash_map.TBB provides
tbbmallocwhich works better for multi-threaded
applications than the glib'c malloc, but it looks like
jemalloc and
mimalloc are a little bit
more scalable thantbbmalloc.
Size of the problem
When linking Chrome, a linker reads 3,430,966,844 bytes of data in
total. The data contains the following items:
| Data item | Number |
|---|---|
| Object files | 30,723 |
| Public undefined symbols | 1,428,149 |
| Mergeable strings | 1,579,996 |
| Comdat groups | 9,914,510 |
| Regular sectionsยน | 10,345,314 |
| Public defined symbols | 10,512,135 |
| Symbols | 23,953,607 |
| Sections | 27,543,225 |
| Relocations against SHF_ALLOC sections | 39,496,375 |
| Relocations | 62,024,719 |
ยน Sections that have to be copied from input object files to an
output file. Sections that contain relocations or symbols are for
example excluded.
Internals
In this section, I'll explain the internals of mold linker.
A brief history of Unix and the Unix linker
Conceptually, what a linker does is pretty simple. A compiler compiles
a fragment of a program (a single source file) into a fragment of
machine code and data (an object file, which typically has the .o
extension), and a linker stitches them together into a single
executable or a shared library image.
In reality, modern linkers for Unix-like systems are much more
complicated than the naive understanding because they have gradually
gained one feature at a time over the 50 years history of Unix, and
they are now something like a bag of lots of miscellaneous features in
which none of the features is more important than the others. It is
very easy to miss the forest for the trees, since for those who don't
know the details of the Unix linker, it is not clear which feature is
essential and which is not.
That being said, one thing is clear that at any point of Unix history,
a Unix linker has a coherent feature set for the Unix of that age. So,
let me entangle the history to see how the operating system, runtime,
and linker have gained features that we see today. That should give
you an idea of why a particular feature has been added to a linker in the
first place.
Original Unix didn't support shared libraries, and a program was
always loaded to a fixed address. An executable was something like
a memory dump that was just loaded to a particular address by the
kernel. After loading, the kernel started executing the program by
setting the instruction pointer to a particular address.The most essential feature for any linker is relocation processing.
The original Unix linker of course supported that. Let me explain
what that is.Individual object files are inevitably incomplete as a program,
because when a compiler created them, it only see a part of an
entire program. For example, if an object file contains a function
call that refers to another object file, thecallinstruction in the
object cannot be complete, as the compiler has no idea as to what
is the called function's address. To deal with this, the compiler
emits a placeholder value (typically just zero) instead of a real
address and leaves metadata in an object file saying "fix offset X
of this file with an address of Y". That metadata is called
"relocation". Relocations are typically processed by the linker.It is easy for a linker to apply relocations for the original Unix
because a program is always loaded to a fixed address. It exactly
knows the addresses of all functions and data when linking a
program.Static library support, which is still an important feature of Unix
linker, also dates back to this early period of Unix history.
To understand what it is, imagine that you are trying to compile
a program for the early Unix. You don't want to waste time to
compile libc functions every time you compile your program (the
computers of the era were incredibly slow), so you have already
placed each libc function into a separate source file and compiled
them individually. That means you have object files for each libc
function, e.g., printf.o, scanf.o, atoi.o, write.o, etc.Given this configuration, all you have to do to link your program
against libc functions is to pick up the right set of libc object
files and give them to the linker along with the object files of your
program. But, keeping the linker command line in sync with the
libc functions you are using in your program is bothersome. You can
be conservative; you can specify all libc object files to the
command line, but that leads to program bloat because the linker
unconditionally link all object files given to it no matter whether
they are used or not. So, a new feature was added to the linker to
fix the problem. That is the static library, which is also called
the archive file.An archive file is just a bundle of object files, just like zip
file but in an uncompressed form. An archive file typically has the
.a file extension and named after its contents. For example, the
archive file containing all libc objects is namedlibc.a.If you pass an archive file along with other object files to the
linker, the linker pulls out an object file from the archive only
when it is referenced by other object files. In other words,
unlike object files directly given to a linker, object files
wrapped in an archive are not linked to the output by default.
An archive works as a supplement to complete your program.Even today, you can still find a libc archive file. Run
ar t /usr/lib/x86_64-linux-gnu/libc.aon Linux should give you a list
of object files in the libc archive.In the '80s, Sun Microsystems, a leading commercial Unix vendor at the
time, added shared library support to their Unix variant, SunOS.
(This section is incomplete.)
Concurrency strategy
In this section, I'll explain the high-level concurrency strategy of
mold.
In most places, mold adopts data parallelism. That is, we have a huge
number of pieces of data of the same kind, and we process each of them
individually using parallel for-loop. For example, after identifying
the exact set of input object files, we need to scan all relocation
tables to determine the sizes of .got and .plt sections. We do that
using a parallel for-loop. The granularity of parallel processing in
this case is the relocation table.
Data parallelism is very efficient and scalable because there's no
need for threads to communicate with each other while working on each
element of data. In addition to that, data parallelism is easy to
understand, as it is just a for-loop in which multiple iterations may
be executed in parallel. We don't use high-level communication or
synchronization mechanisms such as channels, futures, promises,
latches or something like that in mold.
In some cases, we need to share a little bit of data between threads
while executing a parallel for-loop. For example, the loop to scan
relocations turns on "requires GOT" or "requires PLT" flags in a
symbol. Symbol is a shared resource, and writing to them from multiple
threads without synchronization is unsafe. To deal with it, we made
the flag an atomic variable.
The other common pattern you can find in mold which is build on top of
the parallel for-loop is the map-reduce pattern. That is, we run a
parallel for-loop on a large data set to produce a small data set and
process the small data set with a single thread. Let me take a
build-id computation as an example. Build-id is typically computed by
applying a cryptographic hash function such as SHA-1 on a linker's
output file. To compute it, we first consider an output as a sequence
of 1 MiB blocks and compute a SHA-1 hash for each block in parallel.
Then, we concatenate the SHA-1 hashes and compute a SHA-1 hash on the
hashes to get a final build-id.
Finally, we use concurrent hashmap at a few places in mold. Concurrent
hashmap is a hashmap to which multiple threads can safely insert items
in parallel. We use it in the symbol resolution stage, for example.
To resolve symbols, we basically have to throw in all defined symbols
into a hash table, so that we can find a matching defined symbol for
an undefined symbol by name. We do the hash table insertion from a
parallel for-loop which iterates over a list of input files.
Overall, even though mold is highly scalable, it succeeded to avoid
complexties you often find in complex parallel programs. From high
level, mold just serially executes the linker's internal passes one by
one. Each pass is parallelized using parallel for-loops.
Rejected ideas
In this section, I'll explain the alternative designs I currently do
not plan to implement and why I turned them down.
Placing variable-length sections at end of an output file and start
copying file contents before fixing the output file layoutIdea: Fixing the layout of regular sections seems easy, and if we
place them at beginning of a file, we can start copying their
contents from their input files to an output file. While copying
file contents, we can compute the sizes of variable-length sections
such as .got or .plt and place them at end of the file.Reason for rejection: I did not choose this design because I doubt
if it could actually shorten link time and I think I don't need it
anyway.The linker has to de-duplicate comdat sections (i.e. inline
functions that are included in multiple object files), so we
cannot compute the layout of regular sections until we resolve all
symbols and de-duplicate comdats. That takes a few hundred
milliseconds. After that, we can compute the sizes of
variable-length sections in less than 100 milliseconds. It's quite
fast, so it doesn't seem to make much sense to proceed without
fixing the final file layout.The other reason to reject this idea is because there's good a
chance for this idea to have a negative impact on linker's overall
performance. If we copy file contents before fixing the layout, we
can't apply relocations to them while copying because symbol
addresses are not available yet. If we fix the file layout first, we
can apply relocations while copying, which is effectively zero-cost
due to a very good data locality. On the other hand, if we apply
relocations long after we copy file contents, it's pretty expensive
because section contents are very likely to have been evicted from
CPU cache.Incremental linking
Idea: Incremental linking is a technique to patch a previous linker's
output file so that only functions or data that are updated from the
previous build are written to it. It is expected to significantly
reduce the amount of data copied from input files to an output file
and thus speed up linking. GNU BFD and gold linkers support it.Reason for rejection: I turned it down because it (1) is
complicated, (2) doesn't seem to speed it up that much and (3) has
several practical issues. Let me explain each of them.First, incremental linking for real C/C++ programs is not as easy as
one might think. Let me take malloc as an example. malloc is usually
defined by libc, but you can implement it in your program, and if
that's the case, the symbolmallocwill be resolved to your
function instead of the one in libc. If you include a library that
defines malloc (such as libjemalloc or libtbbmallc) before libc,
their malloc will override libc's malloc.Assume that you are using a nonstandard malloc. What if you remove
your malloc from your code, or remove-ljemallocfrom your
Makefile? The linker has to include a malloc from libc, which may
include more object files to satisfy its dependencies. Such code
change can affect the entire program rather than just replacing one
function. The same is true for adding malloc to your program. Making
a local change doesn't necessarily result in a local change in the
binary level. It can easily have cascading effects.Some ELF fancy features make incremental linking even harder to
implement. Take the weak symbol as an example. If you defineatoi
as a weak symbol in your program, and if you are not usingatoi
at all in your program, that symbol will be resolved to address- But if you start using some libc function that indirectly calls
atoi, thenatoiwill be included in your program, and your weak
symbol will be resolved to that function. I don't know how to
efficiently fix up a binary for this case.
This is a hard problem, so existing linkers don't try too hard to
solve it. For example, IIRC, gold falls back to full link if any
function is removed from a previous build. If you want to not annoy
users in the fallback case, you need to make full link fast anyway.Second, incremental linking itself has an overhead. It has to detect
updated files, patch an existing output file and write additional
data to an output file for future incremental linking. GNU gold, for
instance, takes almost 30 seconds on my machine to do a null
incremental link (i.e. no object files are updated from a previous
build) for chrome. It's just too slow.Third, there are other practical issues in incremental linking. It's
not reproducible, so your binary isn't going to be the same as other
binaries even if you are compiling the same source tree using the
same compiler toolchain. Or, it is complex and there might be a bug
in it. If something doesn't work correctly, "remove --incremental
from your Makefile and try again" could be a piece of advice, but
that isn't ideal.So, all in all, incremental linking is tricky. I wanted to make full
link as fast as possible, so that we don't have to think about how
to work around the slowness of full link.- But if you start using some libc function that indirectly calls
Defining a completely new file format and use it
Idea: Sometimes, the ELF file format itself seems to be a limiting
factor in improving the linker's performance. We might be able to make a
far better one if we create a new file format.Reason for rejection: I rejected the idea because it apparently has
a practical issue (backward compatibility issue) and also doesn't
seem to improve the performance of linkers that much. As clearly
demonstrated by mold, we can create a fast linker for ELF. I believe
ELF isn't that bad, after all. The semantics of the existing Unix
linkers, such as the name resolution algorithm or the linker script,
have slowed the linkers down, but that's not a problem of the file
format itself.Watching object files using inotify(2)
Idea: When mold is running as a daemon for preloading, use
inotify(2) to watch file system updates so that it can reload files
as soon as they are updated.Reason for rejection: Just like the maximum number of files you can
simultaneously open, the maximum number of files you can watch using
inotify(2) isn't that large. Maybe just a single instance of mold is
fine with inotify(2), but it may fail if you run multiple of it.The other reason for not doing it is because mold is quite fast
without it anyway. Invoking stat(2) on each file for file update
check takes less than 100 milliseconds for Chrome, and if most of
the input files are not updated, parsing updated files takes almost
no time.
File: docs/execstack.md
This page explains the following warning message and how to fix it.
mold emits this message when it sees an object file that may not be
compatible with mold.
mold: warning: foo.o: this file may cause a segmentation fault because it requires an executable stack. See https://github.com/rui314/mold/tree/main/docs/execstack.md for more info.Background
On modern computers, the stack area (to which local variables are
stored) cannot contain executable code. If the control reaches the
stack area, the CPU refuses to execute any code there and the program
is usually terminated due to segmentation fault.
This is a security measure. The stack area used to be executable (old
CPUs generally execute any code as long as it is in a readable memory
region), but that provided an easy attack vector to a malicious user.
They wrote executable code to the stack area using some buffer
overflow bug and jumped there to run arbitrary code in a remote server
process.
To prevent this type of attack, the stack area is no longer executable
since the early 2000s. On Linux, the stack's executable-ness is
controlled by a bit in an executable, and the loader respects that
bit. The bit is set by the linker.
GCC had (and still has) a feature that depends on the executable
stack, so they invented a way to tell the linker to mark the stack
executable. Specifically, if an object file contains a.note.GNU-stack section with the SHF_EXECSTACK bit, GNU linker
silently makes the stack of an output file executable.
But the GNU linker's behavior is dangerous. If you accidentally link
an object file that has that marker section, the entire stack area
silently becomes executable, disabling the security mechanism.
Therefore, mold simply ignores that marker section. If you are using
mold, you need to explicitly pass -z execstack to the linker to make
the stack executable.
What caused this issue?
You are likely to use GCC's Nested
Functions
feature which still depends on the executable stack.
How to fix it?
If you know what you are doing, pass -z execstack to mold. Beware
that this will significantly weaken your program's security.
If you don't want to pass -z execstack, rewrite your code so that
your code does not depend on the executable stack.
File: docs/glossary.md
The very concept of linking is simple: a compiler compiles a piece of
source code into an object file (a file containing machine code), and
a linker combines object files into a single executable or a shared
library file. However, the actual implementation of the linker for
modern systems is much more complicated because hardware, operating
system, compiler and linker all have many more features.
In this file, I'll explain random topics in the glossary format that
you need to understand to read mold code.
DSO
A .so file. Short for Dynamic Shared Object. Often called as a
shared library, a dynamic libray or a shared object as well.
An DSO contains common functions and data that are used by multiple
executables and/or other DSOs. At runtime, a DSO is loaded to a
contiguous region in the virtual address.
Object file
A .o file. An object file contains machine code and data, but it
cannot be executed because it's not self-contained. For example,
if you compile a C source file containing a call of printf,
the actual function code of printf is not included in the resulting
object file. You include stdio.h, but that teaches the compiler
only about printf's type, and the compiler still don't know whatprintf actually does. Therefore, it cannot emit code for printf.
You need to link an object file with other object file or a shared
library to make it exectuable.
Virtual address space
A pointer has a value like 0x803020 which is an address of the
pointee. But it doesn't mean that the pointee resides at the
physical memory address 0x803020 on the computer. Modern CPUs
contains so-called Mmeory Management Unit (MMU), and all access to
the memory are first translated by MMU to the physical address.
The address before translation is called the "virtual address".
Unless you are doing the kernel programming, all addresses you
handle are virtual addresses.
The OS kernel controls the MMU so that each process owns the entire
virtual address space. So, even if two process uses the same virtual
address, they don't conflict. They are mapped to different physical
addresses.
The existence of MMU has several implications to the linker. First,
we can link the main executable to a specific address. On process
startup, there's no code or data in the virtual address space, so
the mapping of the main executable always succeed. However, it's not
true to DSOs because they are loaded after the main executable and
possibly other DSOs. Therefore, shared libraries must be linked in a
way that they can be loaded to any address in the virtual address
space.
Relocation
A piece of information for the linker as to how to link object files
or a dynamic objects.
Object files can refer functions or data in other object files. For
example, if you compile a function which calls a non-local functionfoo, the resulting code contains something like this:
26: e8 00 00 00 00 callq 2b
27: R_X86_64_PLT32 foo-0x4The above callq is the instruction to call a function at the
machine code level. It's opcode is 0xe8 in x86-64, so the
instruction begins with 0xe8. The following four bytes are
displacement; that is, the address of the branch target relative to
the end of this callq instruction. Notice that the displacement is
0. The compiler couldn't fill the displacement because it has no
idea as to where foo will be at runtime. So, the compiler write 0
as a placeholder and instead write a relocation R_X86_64_PLT32
with foo as its associated symbol. The linker reads this
relocation, computes the offsets between this call instruction and
function foo and overwrite the placeholder value 0 with an actual
displacement.
There are many different types of relocations. For example, if you
want to fix up not with a displacement but with an absolute address
of a symbol, you need to use R_X86_64_ABS64 instead.
Static library
A .a file. Often called as an archive file or just archive as well.
A static library is a container just like tar or zip. Actually,
there's no technical reason to not use tar or (uncompressed) zip,
but traditionally the .a file format is used by the linker.
A static library contains object files and can be passed to the
linker along with other object files and/or archives.
A linker pulls out object files from an archive only if it is needed
to resolve undefined symbols. In other words, object files in an
archive are not linked by default and used as a complement to supply
missing definitions. This is ideal for a library because you don't
want to link library functions unless you are actually using them.
Contrary to archive files, object files directly given to a linker
are always linked to the output.
To maximize the benefit of archive files, a library often used as a
static library is broken down to small files to separate each
function individually (for example, look at
https://git.musl-libc.org/cgit/musl/tree/src/stdio). By doing this,
you import only used functions.
A static file is created by ar, whose command line arguments are
similar to tar. A static library contains the symbol table which
offers a quick way to look up an object file for a defined symbol,
but mold does not use the static library's symbol table. mold
doesdn't need a symbol table to exist in an archive, and if exists,
mold just ignores it.
See also: DSO (dynamic library)
Symbol
A symbol is a label assigned to a specific location in an input file
or an output file. For example, if you define function foo and
compile it, the resulting object file contains a symbol foo
pointing to the beginning of the machine code for foo.
Usually, a symbol name is a function or a variable name. If an
object is anonymous (such the one for a string literal), a compiler
generated a unique symbol, which often starts with . to avoid
conflict with user-defined symbols.
For C++, symbol name is a complex "mangled" name. We need to mangle
identifiers because a simple name such as foo cannot be uniquely
identify a function or a data in C++, because for example foo may
be in a namespace or defined as a static member in some class. Iffoo is an overloaded function, we need to distinguish differentfoos by its type. Therefore, C++ compiler mangles an identifier by
appending nmaepsace names, type information and such so that
different things get different names.
For example, a function int foo(int) in a namespace bar is
mangled as _ZN3bar3fooEi.
A symbol can be either defined or undefined. A defined symbol points
to some location in a file which may contain the function's machine
code or the variable's initial value. An undefined symbol does not
point to anywhere. It needs to be merged with a defined symbol with
the same name at link-time. This merging process is called "name
resolution".
For example, if your program is using printf, it usually containsprintf as an undefined symbol. You need to link it with libc.a
or libc.so, which contain a defined symbol of printf, to make a
complete program.
File: docs/mold.md
mold(1) -- a modern linker
SYNOPSIS
mold [option...] file...
DESCRIPTION
mold is a faster drop-in replacement for the default GNU ld(1).
How to use
See https://github.com/rui314/mold#how-to-use.
Compatibility
mold is designed to be a drop-in replacement for the GNU linkers for
linking user-land programs. If your user-land program cannot be built due to
missing command-line options, please file a bug at
https://github.com/rui314/mold/issues.
mold supports a very limited set of linker script features, which is just
sufficient to read /usr/lib/x86_64-linux-gnu/libc.so on Linux systems (on
Linux, that file is contrary to its name not a shared library but an ASCII
linker script that loads a real libc.so file.)
Beyond that, we have no plan to support any additional linker script features.
The linker script is an ad-hoc, over-designed, complex language which we
believe needs to be replaced by a simpler mechanism. We have a plan to add a
replacement for the linker script to mold instead.
Archive symbol resolution
Traditionally, Unix linkers are sensitive to the order in which input files
appear on the command line. They process input files from the first (leftmost)
file to the last (rightmost) file one-by-one. While reading input files, they
maintain sets of defined and undefined symbols. When visiting an archive file
(.a files), they pull out object files to resolve as many undefined symbols
as possible and move on to the next input file. Object files that weren't
pulled out will never have a chance for a second look.
Due to this behavior, you usually have to add archive files at the end of a
command line, so that when a linker reaches archive files, it knows what
symbols remain as undefined.
If you put archive files at the beginning of a command line, a linker doesn't
have any undefined symbols, and thus no object files will be pulled out from
archives. You can change the processing order by using the --start-group and--end-group options, though they make a linker slower.
mold, as well as the LLVM lld(1) linker, takes a different approach. They
remember which symbols can be resolved from archive files instead of
forgetting them after processing each archive. Therefore, mold and lld(1)
can "go back" in a command line to pull out object files from archives if they
are needed to resolve remaining undefined symbols. They are not sensitive to
the input file order.
--start-group and --end-group are still accepted by mold and lld(1)
for compatibility with traditional linkers, but they are silently ignored.
Dynamic symbol resolution
Some Unix linker features are difficult to understand without comprehending
the semantics of dynamic symbol resolution. Therefore, even though it's not
specific to mold, we'll explain it here.
We use "ELF module" or just "module" as a collective term to refer to an
executable or a shared library file in the ELF format.
An ELF module may have lists of imported symbols and exported symbols, as well
as a list of shared library names from which imported symbols should be
imported. The point is that imported symbols are not bound to any specific
shared library until runtime.
Here is how the Unix dynamic linker resolves dynamic symbols. Upon the start
of an ELF program, the dynamic linker constructs a list of ELF modules which,
as a whole, consist of a complete program. The executable file is always at
the beginning of the list followed by its dependent shared libraries. An
imported symbol is searched from the beginning of the list to the end. If two
or more modules define the same symbol, the one that appears first in the list
takes precedence over the others.
This Unix semantics are contrary to systems such as Windows that have a
two-level namespace for dynamic symbols. On Windows, for example, dynamic
symbols are represented as a tuple of (symbol-name, shared-library-name),
so that each dynamic symbol is guaranteed to be resolved from some specific
library.
Typically, an ELF module that exports a symbol also imports the same symbol.
Such a symbol is usually resolved to itself, but that's not the case if a
module that appears before it in the symbol search list provides another
definition of the same symbol.
Let's take malloc as an example. Assume that you define your version ofmalloc in your main executable file. Then, all malloc calls from any
module are resolved to your function instead of the one in libc, because the
executable is always at the beginning of the dynamic symbol search list. Note
that even malloc calls within libc are resolved to your definition since
libc exports and imports malloc. Therefore, by defining malloc yourself,
you can overwrite a library function, and the malloc in libc becomes dead
code.
These Unix semantics are tricky and sometimes considered harmful. For example,
assume that you accidentally define atoi as a global function in your
executable that behaves completely differently from the one in the C standard.
Then, all atoi function calls from any modules (even function calls within
libc) are redirected to your function instead of the one in libc, which will
very likely cause a problem. That is a somewhat surprising consequence for an
accidental name conflict. On the other hand, this semantic is sometimes useful
because it allows users to override library functions without rebuilding
modules containing them.
Whether good or bad, you should keep these semantics in mind to understand
Unix linkers' behaviors.
Build reproducibility
mold's output is deterministic. That is, if you pass the same object files
and the same command-line options to the same version of mold, it is
guaranteed that mold produces the bit-for-bit identical output. The linker's
internal randomness, such as the timing of thread scheduling or iteration
orders of hash tables, doesn't affect the output.
mold does not have any host-specific default settings. This is contrary to the
GNU linkers, for which some configurable values, such as system-dependent
library search paths, are hard-coded. mold depends only on its command-line
arguments.
OPTION NOTATIONS
Multi-letter long options may precede either a single dash or double dashes,
except for those starting with the letter "o". For historical reasons, long
options beginning with "o" must precede double dashes.
For example, you can spell --as-needed as -as-needed, but --omagic must
not be spelled as -omagic. -omagic will be interpreted not as --omagic
but as -o magic.
MOLD-SPECIFIC OPTIONS
--chroot=dir:
Set dir as the root directory.--color-diagnostics=[ auto | always | never ]:
Show diagnostic messages in color using ANSI escape sequences.automeans
thatmoldprints out messages in color only if the standard output is
connected to a TTY. Default isauto.--color-diagnostics:
Synonym for--color-diagnostics=auto.--no-color-diagnostics:
Synonym for--color-diagnostics=never.--detach,--no-detach:
Permit or do not permit mold to create a debug info file in the background.--fork,--no-fork:
Spawn a child process and let it do the actual linking. When linking a large
program, the OS kernel can take a few hundred milliseconds to terminate amoldprocess.--forkhides that latency. By default, it does fork.Note that
--forkalso hides the resource usage statistics reported by
time(2), since it doesn't call waitpid(2) on the child process. If you
need those statistics, pass--no-fork.--perf:
Print performance statistics.--print-dependencies:
Print out dependency information for input files.Each line of the output for this option shows which file depends on which
file to use a specific symbol. This option is useful for debugging why some
object file in a static archive got linked or why some shared library is
kept in an output file's dependency list even with--as-needed.--relocatable-merge-sections:
By default,molddoesn't merge input sections by name when merging input
object files into a single output object file for-r. For example,.text.fooand.text.bararen't merged for-reven though they are
merged into.textbased on the default section merging rules.This option changes the behavior so that
moldmerges input sections by
name by the default section merging rules.--repro:
Archive input files, as well as a text file containing command line options,
in a tar file so that you can runmoldwith the exact same inputs again.
This is useful for reporting a bug with a reproducer. The output filename ispath/to/output.tar, wherepath/to/outputis an output filename specified
by-o.--reverse-sections:
Reverse the order of input sections before assigning them the offsets in the
output file.This option is useful for finding bugs that depend on the initialization
order of global objects. In C++, constructors of global objects in a single
source file are guaranteed to be executed in the source order, but there's
no such guarantee across compilation units. Usually, constructors are
executed in the order given to the linker, but depending on it is a mistake.By reversing the order of input sections using
--reverse-sections, you can
easily test that your program works in the reversed initialization order.--runcommand arg...:
Run command withmoldas/usr/bin/ld. Specifically,moldruns a
given command with theLD_PRELOADenvironment set to intercept exec(3)
family functions and replacesargv[0]with itself if it isld,ld.gold, orld.lld.--separate-debug-file,--separate-debug-file=file:
Bundle debug info sections into a separate file instead of embedding them in
an output executable or a shared library. mold creates a debug info file in
the background by default, so that you can start running your executable as
soon as possible.By default, the debug info file is created in the same directory as is the
output file, with the.dbgfile extension. That filename is embedded into
the output file so thatgdbcan automatically find the debug info file for
the output file. For more info about gdb features related to separate debug
files, see
https://sourceware.org/gdb/current/onlinedocs/gdb.html/Separate-Debug-Files.html.mold holds a file lock with flock(2) while creating a debug info file in the
background.If you don't want to create a debug info file in the background, pass the
--no-detachoption.--shuffle-sections,--shuffle-sections=number:
Randomize the output by shuffling the order of input sections before
assigning them the offsets in the output file. If a number is given, it's
used as a seed for the random number generator, so that the linker produces
the same output for the same seed. If no seed is given, a random number is
used as a seed.This option is useful for benchmarking. Modern CPUs are sensitive to a
program's memory layout. A seemingly benign change in program layout, such
as a small size increase of a function in the middle of a program, can
affect the program's performance. Therefore, even if you write new code and
get a good benchmark result, it is hard to say whether the new code improves
the program's performance; it is possible that the new memory layout happens
to perform better.By running a benchmark multiple times with randomized memory layouts using
--shuffle-sections, you can isolate your program's real performance number
from the randomness caused by memory layout changes.--spare-program-headers=number:
Append the given number ofPT_NULLentries to the end of the program
header, so that post-link processing tools can easily add new segments by
overwriting the null entries.Note that ELF requires all
PT_LOADsegments to be sorted byp_vaddr.
Therefore, if you add a new LOAD segment, you may need to sort the entire
program header.--stats:
Print input statistics.--thread-count=count:
Use count number of threads.--threads,--no-threads:
Use multiple threads. By default,molduses as many threads as the number of
cores or 32, whichever is smaller. The reason it is capped at 32 is becausemolddoesn't scale well beyond that point. To use only one thread, pass--no-threadsor--thread-count=1.--quick-exit,--no-quick-exit:
Use or do not usequick_exitto exit.--zero-to-bss:
Convert all-zero data sections into BSS.When this option is enabled,
moldscans input data sections that are not
of typeSHT_NOBITSand checks whether their contents consist solely of
zero bytes. Such sections are then converted into BSS (SHT_NOBITS)
sections. This reduces the size of the output file, since BSS sections
occupy no space in the file image.This behavior is especially useful for user-defined sections created with
__attribute__((section(".sectname")))that contain uninitialized global
variables. GCC and Clang do not automatically mark such sections as BSS
even if their contents are entirely zero, and instead emit them as regular
data sections.For example, consider
__attribute__((section(".sectname"))) int vec[256];.By default, this results in a
.sectnamesection of typeSHT_PROGBITS
filled with zeros. With--zero-to-bss, the linker will recognize it as
empty data and convert it to aSHT_NOBITSsection, reducing the output
file size without changing runtime semantics.-z rewrite-endbr,-z norewrite-endbr:
As a security measure, some CPU instruction sets have recently gained a
feature to protect control flow integrity by disallowing indirect branches
by default. If the feature is enabled, the instruction that is executed
immediately after an indirect branch must be an branch target marker
instruction, or a CPU-level fault will raise. The marker instruction is also
known as "landing pad" instruction, to which indirect branches can land.
This feature makes ROP attacks harder to conduct.To use the feature, a function whose pointer is taken needs to begin with a
landing pad because a function call via a function pointer is compiled to an
indirect branch. On the other hand, if a function is called only directly
(i.e. referred to only by direct branch instructions), it doesn't have to
begin with it.By default, the compiler always emits a landing pad at the beginning of each
global function because it doesn't know whether or not the function's
pointer is taken in another translation unit. As a result, the resulting
binary has more attack surface than necessary.If
--rewrite-endbris given, mold conducts a whole program analysis
to identify functions whose addresses are actually taken and rewrites
landing pads with no-ops for non-address-taken functions, reducing the
attack surface.This feature is currently available only on x86-64.
GNU-COMPATIBLE OPTIONS
--help:
Report usage information to stdout and exit.-v,--version:
Report version information to stdout.-V:
Report version and target information to stdout.-E,--export-dynamic,--no-export-dynamic:
When creating an executable, using the-Eoption causes all global symbols
to be put into the dynamic symbol table, so that the symbols are visible
from other ELF modules at runtime.By default, or if
--no-export-dynamicis given, only symbols that are
referenced by DSOs at link-time are exported from an executable.-Flibname,--filter=libname:
Set theDT_FILTERdynamic section field to libname.-Ifile,--dynamic-linker=file,--no-dynamic-linker:
Set the dynamic linker path to file. If no-Ioption is given, or if--no-dynamic-linkeris given, no dynamic linker path is set to an output
file. This is contrary to the GNU linkers which set a default dynamic linker
path in that case. This difference doesn't usually make any difference
because the compiler driver always passes-Ito the linker.-Ldir,--library-path=dir:
Add dir to the list of library search paths from whichmoldsearches
libraries for the-loption.Unlike the GNU linkers,
molddoes not have default search paths. This
difference doesn't usually make any difference because the compiler driver
always passes all necessary search paths to the linker.-M,--print-map:
Write a map file to stdout.-N,--omagic,--no-omagic:
Forcemoldto emit an output file with an old-fashioned memory layout.
First, it makes the first data segment not aligned to a page boundary.
Second, text segments are marked as writable if the option is given.-S,--strip-debug:
Omit.debug_*sections from the output file.-Tfile,--script=file:
Read linker script from file.-X,--discard-locals:
Discard temporary local symbols to reduce the sizes of the symbol table and
the string table. Temporary local symbols are local symbols starting with.L. Compilers usually generate such symbols for unnamed program elements
such as string literals or floating-point literals. This is the default;
use--discard-noneto keep temporary local symbols.--discard-none:
Keep all local symbols in the symbol table, including temporary ones
starting with.L, except ones in mergeable sections. This restores the
default behavior of GNU ld.moldandld.llddiscard temporary local
symbols by default.-esymbol,--entry=symbol:Use symbol as the entry point symbol instead of the default entry
point symbol _start.-fshlib,--auxiliary=shlib:
Set theDT_AUXILIARYdynamic section field to shlib.-hlibname,--soname=libname:
Set theDT_SONAMEdynamic section field to libname. This option is used
when creating a shared object file. Typically, when you createlibfoo.so,
you want to pass--soname=footo a linker.-llibname:
Search forliblibname.soorliblibname.afrom library search
paths.-mtarget:
Choose a target.-ofile,--output=file:
Use file as the output file name instead of the default namea.out.-r,--relocatable:
Instead of generating an executable or a shared object file, combine input
object files to generate another object file that can be used as an input to
a linker.-s,--strip-all:
Omit.symtabsection from the output file.-usymbol,--undefined=symbol:
If symbol remains as an undefined symbol after reading all object files,
and if there is a static archive that contains an object file defining
symbol, pull out the object file and link it so that the output file
contains a definition of symbol.-ysymbol,--trace-symbol=symbol:
Trace references to symbol.--Bdynamic:
Link against shared libraries.--Bstatic:
Do not link against shared libraries.--Bsymbolic:
When creating a shared library, make global symbols export-only (i.e. do not
import the same symbol). As a result, references within a shared library are
always resolved locally, negating symbol override at runtime. See "Dynamic
symbol resolution" for more information about symbol imports and exports.--Bsymbolic-functions:
This option has the same effect as--Bsymbolicbut works only for function
symbols. Data symbols remain being both imported and exported.--Bsymbolic-non-weak:
This option has the same effect as--Bsymbolicbut works only for non-weak
symbols. Weak symbols remain being both imported and exported.--Bsymbolic-non-weak-functions:
This option has the same effect as--Bsymbolicbut works only for non-weak
function symbols. Data symbols and weak function symbols remain being both
imported and exported.--Bno-symbolic:
Cancel--Bsymbolic,--Bsymbolic-functions,--Bsymbolic-non-weakand--Bsymbolic-non-weak-functions.--Map=file:
Write map file to file.--Tbss=address:
Alias for--section-start=.bss=address.--Tdata=address:
Alias for--section-start=.data=address.--Ttext=address:
Alias for--section-start=.text=address.--Ttext-segment=address:
Set the address of the first byte of the text segment.--allow-multiple-definition:
Normally, the linker reports an error if there are more than one definition
of a symbol. This option changes the default behavior so that it doesn't
report an error for duplicate definitions and instead use the first
definition.--allow-shlib-undefined,--no-allow-shlib-undefined:
Even if mold succeeds in linking a main executable without undefined symbol
errors, you may still encounter symbol lookup errors at runtime because the
dynamic linker cannot find some symbols in shared libraries in any ELF
module. This occurs because mold ignores undefined symbols in shared
libraries by default.If you pass
--no-allow-shlib-undefined, mold verifies that undefined
symbols in shared libraries given to the linker can be resolved at
link-time. In other words, this converts the runtime error to a link-time
error.Note that you need to pass all shared libraries, including indirectly
dependent ones, to the linker as arguments for-l. If a shared library
depends on a library that's not passed to the linker, the verification will
be skipped for that file.--as-needed,--no-as-needed:
By default, shared libraries given to the linker are unconditionally added
to the list of required libraries in an output file. However, shared
libraries after--as-neededare added to the list only when at least one
symbol is actually used by the output file. In other words, shared libraries
after--as-neededare not added to the list of needed libraries if they
are not needed by a program.The
--no-as-neededoption restores the default behavior for subsequent
files.--build-id=[md5|sha1|sha256|fast|uuid|0xhexstring |none]:
Create a.note.gnu.build-idsection containing a byte string to uniquely
identify an output file.sha256compute a 256-bit cryptographic hash of an
output file and set it to build-id.md5andsha1compute the same hash
but truncate it to 128 and 160 bits, respectively, before setting it to
build-id.uuidsets a random 128-bit UUID.0xhexstring sets
hexstring.fastis a synonym forsha256.--build-id:
Synonym for--build-id=sha256.--no-build-id:
Synonym for--build-id=none.--compress-debug-sections=[zlib|zlib-gabi|zlib:N |zstd|zstd:N |none]:
Compress DWARF debug info (.debug_*sections) using the zlib or zstd
compression algorithm.zlib-gabiis an alias forzlib.zlib:N
selects the zlib compression level, where N is between 0 and 9 (0
means no compression), andzstd:N selects the zstd compression
level, where N is between 1 and 22. Higher levels achieve better
compression ratios but are slower.zlibis equivalent tozlib:1
andzstdis equivalent tozstd:3.--defsym=symbol=value:
Define symbol as an alias for value.value is either an integer (in decimal or hexadecimal with
0xprefix) or
a symbol name. If an integer is given as a value, symbol is defined as an
absolute symbol with the given value.--default-symver:
Use soname as a symbol version and append that version to all symbols.--demangle,--no-demangle:
Demangle C++ and Rust symbols in log messages.--dependency-file=file:
Write a dependency file to file. The contents of the written file is
readable by make(1), which defines only one rule with the linker's output
file as a target and all input files as its prerequisites. Users are
expected to include the generated dependency file into a Makefile to
automate the dependency management. This option is analogous to the
compiler's-MM -MFoptions.--dynamic-list=file:
Read a list of dynamic symbols from file. Same as--export-dynamic-symbol-list, except that it implies--Bsymbolic. If
file does not exist in the current directory, it is searched from library
search paths for the sake of compatibility with GNU ld.--eh-frame-hdr,--no-eh-frame-hdr:
Create.eh_frame_hdrsection.--emit-relocs:
The linker usually "consumes" relocation sections. That is, the linker
applies relocations to other sections, and relocation sections themselves
are discarded.The
--emit-relocsinstructs the linker to leave relocation sections in the
output file. Some post-link binary analysis or optimization tools such as
LLVM Bolt need them.--enable-new-dtags,--disable-new-dtags:
By default,moldemitsDT_RUNPATHfor--rpath. If you pass--disable-new-dtags,moldemitsDT_RPATHfor--rpathinstead.--execute-only:Traditionally, setting the executable bit to 1 for a memory page implies
that the page also become readable, which allows machine code to be read
as data at runtime. That is actually what an attacker often does after
gaining a limited control of a process to find pieces of machine code
they can use to gain the full control of the process. As a mitigation,
recent processors including some ARM64 ones allows "execute-only" pages.
If a page is execute-only, you can call a function there as long as you
know its address but can't read it as data.This option marks text segments as execute-only by setting just the "X"
bit instead of "RX". Note that on most systems, the absence of the "R"
bit in the text segment serves just as a hint. If you run a program
linked with--execute-onlyon a processor that doesn't support
execute-only pages, your executable will likely still function normally,
but the text segment will remain readable.--exclude-libs=libraries ...:
Mark all symbols in the given libraries hidden.--export-dynamic-symbol=symbol:
Put symbols matching symbol in the dynamic symbol table. symbol may be a
glob pattern in the same syntax as for the--export-dynamic-symbol-listor--version-scriptoptions.--export-dynamic-symbol-list=file:
Read a list of dynamic symbols from file.--fatal-warnings,--no-fatal-warnings:
Treat warnings as errors.--fini=symbol:
Call symbol at unload-time.--gc-sections,--no-gc-sections:
Remove unreferenced sections.--gdb-index:
Create a.gdb_indexsection to speed up GNU debugger. To use this, you
need to compile source files with the-ggnu-pubnamescompiler flag.--hash-style=[sysv|gnu|both|none]:
Set hash style.--icf=[safe|all|none],--no-icf:
It is not uncommon for a program to contain many identical functions that
differ only in name. For example, a C++ templatestd::vectoris very
likely to be instantiated to the identical code forstd::vectorandstd::vectorbecause the container cares only about the size of
the parameter type. Identical Code Folding (ICF) is a size optimization to
identify and merge such identical functions.If
--icf=allis given,moldtries to merge all identical functions. This
reduces the size of the output most, but it is not a "safe" optimization. It
is guaranteed in C and C++ that two pointers pointing two different
functions will never be equal, but--icf=allbreaks that assumption as two
identical functions have the same address after merging. So a care must be
taken when you use this flag that your program does not depend on the
function pointer uniqueness.--icf=safeis a flag to merge functions only when it is safe to do so.
That is, if a program does not take an address of a function, it is safe to
merge that function with other function, as you cannot compare a function
pointer with something else without taking an address of a function.--icf=safeneeds to be used with a compiler that supports.llvm_addrsig
section which contains the information as to what symbols are address-taken.
LLVM/Clang supports that section by default. Since GCC does not support it
yet, you cannot use--icf=safewith GCC (it doesn't do any harm but can't
optimize at all.)--icf=noneand--no-icfdisables ICF.--ignore-data-address-equality:
Make ICF to merge not only functions but also data. This option should be
used in combination with--icf=all.--image-base=addr:
Set the base address to addr.--init=symbol:
Call symbol at load-time.--no-undefined:
Report undefined symbols (even with--shared).--noinhibit-exec:
Create an output file even if errors occur.--package-metadata=percent-encoded-string:
Embed a specified string into the.note.packagesection. This option
is designed for build scripts that generate binary packages, such as.rpmor.deb, to include package metadata in each executable. It
simplifies the process of identifying the corresponding package for a
given executable or core file.An argument to this option is treated as percent-encoded and decoded
before being inserted into the section, allowing you to avoid the use of
the comma (,) character in the argument. This is useful because the
compiler replaces all occurrences of commas in-Wl,with spaces before
forwarding them to the linker. Note thatmoldalways interprets the
argument as percent-encoded, so you also need to escape all occurrences
of%as%25.--pack-dyn-relocs=[relr|android|android+relr|none]:
Compress dynamic relocations in the output file.If
relris specified, allR_*_RELATIVErelocations are put into.relr.dynsection instead of.rel.dynor.rela.dynsection. Since.relr.dynsection uses a space-efficient encoding scheme, specifying this
flag can reduce the size of the output. This is typically most effective for
position-independent executable.If
androidis specified, all dynamic relocations are encoded in Android's
packed relocation format (APS2) and stored in the.rel.dynor.rela.dyn
section, whose section type is changed toSHT_ANDROID_RELorSHT_ANDROID_RELA. The dynamic table is updated to useDT_ANDROID_REL
/DT_ANDROID_RELAand theirSZvariants in place of the standard tags.android+relrcombines both:R_*_RELATIVErelocations go to.relr.dyn
and the rest are APS2-encoded. This is the configuration used by most
Android system binaries.Note that a runtime loader has to support
.relr.dynto run executables or
shared libraries linked with--pack-dyn-relocs=relr. As of 2026, glibc
(since 2.36), musl (since 1.2.4), Android, ChromeOS, and Fuchsia all
support it. The APS2 format, on the other hand, is only handled by
Android's bionic loader; the encoder itself is target-agnostic, but the
produced binary can only be loaded on platforms where bionic (or a
compatible loader) handlesDT_ANDROID_REL(A).--pie,--pic-executable,--no-pie,--no-pic-executable:
Create a position-independent executable.--print-gc-sections,--no-print-gc-sections:
Print removed unreferenced sections.--print-icf-sections,--no-print-icf-sections:
Print folded identical sections.--push-state,--pop-state:--push-statesaves the current values of--as-needed,--whole-archive,--static, and--start-lib. The saved values can be restored by
pop-state.--push-stateand--pop-statepairs can nest.These options are useful when you want to construct linker command line
options programmatically. For example, if you want to linklibfoo.soby
as-needed basis but don't want to change the global state of--as-needed,
you can append--push-state --as-needed -lfoo --pop-stateto the linker
command line options.--relax, --no-relax:
Rewrite machine instructions with more efficient ones for some relocations.
The feature is enabled by default.--require-defined=symbol:
Like--undefined, except the new symbol must be defined by the end of the
link.--retain-symbols-file=file:
Keep only symbols listed in file. file is a text file containing a
symbol name on each line.molddiscards all local symbols as well as
global symbol that are not in file. Note that this option removes symbols
only from.symtabsection and does not affect.dynsymsection, which is
used for dynamic linking.--rpath=dir:
Add dir to runtime search paths.--section-start=section=address:
Set address to section. address is a hexadecimal number that may start
with an optional0x.--shared,--Bshareable:
Create a share library.--spare-dynamic-tags=number:
Append the given number ofDT_NULLentries to the end of the.dynamic
section, so that post-link processing tools can easily add new dynamic tags
by overwriting the null entries.--start-lib,--end-lib:
Handle object files between--start-liband--end-libas if they were in
an archive file. That means object files between them are linked only when
they are needed to resolve undefined symbols. The options are useful if you
want to link object files only when they are needed but want to avoid the
overhead of running ar(3).--static:
Do not link against shared libraries.--sysroot=dir:
Set target system root directory to dir.--trace:
Print name of each input file.--undefined-glob=pattern:
Synonym for--undefined, except that--undefined-globtakes a glob
pattern instead of just a single symbol name.--undefined-version,--no-undefined-version:
By default,moldwarns on a symbol specified by a version script or by--export-dynamic-symbolif it is not defined. You can silence the warning
by--undefined-version.--unique=pattern:
Don't merge input sections that match the given glob pattern pattern.--unresolved-symbols=[report-all|ignore-all|ignore-in-object-files|ignore-in-shared-libs]:
How to handle undefined symbols.--version-script=file:
Read version script from file. If file does not exist in the current
directory, it is searched from library search paths for the sake of
compatibility with GNU ld.-w,--no-warnings:
Suppress warnings and cancel--fatal-warnings.--warn-common,--no-warn-common:
Warn about common symbols.--warn-once:
Only warn once for each undefined symbol instead of warn for each relocation
referring an undefined symbol.--warn-unresolved-symbols,--error-unresolved-symbols:
Normally, the linker reports an error for unresolved symbols.--warn-unresolved-symbolsoption turns it into a warning.--error-unresolved-symbolsoption restores the default behavior.--whole-archive,--no-whole-archive:
When archive files (.afiles) are given to the linker, only object files
that are needed to resolve undefined symbols are extracted from them and
linked to an output file.--whole-archivechanges that behavior for
subsequent archives so that the linker extracts all object files and links
them to an output. For example, if you are creating a shared object file and
you want to include all archive members to the output, you should pass--whole-archive.--no-whole-archiverestores the default behavior for
subsequent archives.--wrap=symbol:
Make symbol be resolved to__wrap_symbol. The original symbol can be
resolved as__real_symbol. This option is typically used for wrapping an
existing function.-z cet-report=[warning|error|none]:
Intel Control-flow Enforcement Technology (CET) is a new x86 feature
available since Tiger Lake which is released in 2020. It defines new
instructions to harden security to protect programs from control hijacking
attacks. You can tell the compiler to use the feature by specifying the-fcf-protectionflag.-z cet-reportflag is used to make sure that all object files were
compiled with a correct-fcf-protectionflag. Ifwarningorerrorare
given,moldprints out a warning or an error message if an object file was
not compiled with the compiler flag.moldlooks forGNU_PROPERTY_X86_FEATURE_1_IBTbit andGNU_PROPERTY_X86_FEATURE_1_SHSTKbit in.note.gnu.propertysection to
determine whether or not an object file was compiled with-fcf-protection.-z now,-z lazy:
By default, functions referring to other ELF modules are resolved by the
dynamic linker when they are called for the first time.-z nowmarks an
executable or a shared library file so that all dynamic symbols are resolved
when a file is loaded to memory.-z lazyrestores the default behavior.-z origin:
Mark object requiring immediate$ORIGINprocessing at runtime.-z ibt:
Turn onGNU_PROPERTY_X86_FEATURE_1_IBTbit in.note.gnu.propertysection
to indicate that the output uses IBT-enabled PLT. This option implies-z ibtplt.-z ibtplt:
Generate Intel Branch Tracking (IBT)-enabled PLT which is the default on
x86-64. This is the default.-z execstack,-z noexecstack:
By default, the pages for the stack area (i.e. the pages where local
variables reside) are not executable for security reasons.-z execstack
makes it executable.-z noexecstackrestores the default behavior.-z keep-text-section-prefix,-z nokeep-text-section-prefix:
Keep.text.hot,.text.unknown,.text.unlikely,.text.startup, and.text.exitas separate sections in the final binary instead of merging
them as.text.-z rodynamic:
Make the.dynamicsection read-only.-z relro,-z norelro:
Some sections such as.dynamichave to be writable only during a module is
being loaded to memory. Once the dynamic linker finishes its job, such
sections won't be mutated by anyone. As a security mitigation, it is
preferred to make such segments read-only during program execution.-z relroputs such sections into a special segment calledrelro. The
dynamic linker makes a relro segment read-only after it finishes its job.By default,
moldgenerates a relro segment.-z norelrodisables the
feature.-z sectionheader,-z nosectionheader:-z nosectionheadertell the linker to omit the section header.
By default, the linker does not omit the section header.-z separate-loadable-segments,-z separate-code,-z noseparate-code:
If one memory page contains multiple segments, the page protection bits are
set in such a way that the needed attributes (writable or executable) are
satisfied for all segments. This usually happens at a boundary of two
segments with two different attributes.separate-loadable-segmentsadds paddings between segments with different
attributes so that they do not share the same page.separate-codeadds paddings only between executable and non-executable
segments. This is the default.noseparate-codedoes not add any paddings between segments.-z defs,-z nodefs:
Report undefined symbols (even with--shared).-z shstk:
Enforce shadow stack by turningGNU_PROPERTY_X86_FEATURE_1_SHSTKbit in.note.gnu.propertyoutput section. Shadow stack is part of Intel
Control-flow Enforcement Technology (CET), which is available since Tiger
Lake (2020).-z start_stop_visibility=[hidden|protected]:
If a section name is valid as a C identifier (i.e., it matches/^[_a-zA-Z][_a-zA-Z0-9]*$/), mold creates__start_SECNAMEand__stop_SECNAMEsymbols to mark the beginning and end of the section,
whereSECNAMEis the section name.You can make these marker symbols visible from other ELF modules by passing
-z start_stop_visibility=protected. Default ishidden.-z text,-z notext,-z textoff:moldby default reports an error if dynamic relocations are created in
read-only sections. If-z notextor-z textoffare given,moldcreates
such dynamic relocations without reporting an error.-z textrestores the
default behavior.-z max-page-size=number:
Some CPU ISAs support multiple memory page sizes. This option specifies the
maximum page size that an output binary can run on. In general, binaries
built for a larger page size can run on a system with a smaller page size,
but not vice versa. The default value is 4 KiB for i386, x86-64, and RISC-V,
and 64 KiB for ARM64.-z nodefaultlib:
Make the dynamic loader ignore default search paths.-z nodelete:
Mark DSO non-deletable at runtime.-z nodlopen:
Mark DSO not available to dlopen(3). This option makes it possible for the
linker to optimize thread-local variable accesses by rewriting instructions
for some targets.-z nodump:
Mark DSO not available to dldump(3).-z nocopyreloc:
Do not create copy relocations.-z initfirst:
Mark DSO to be initialized first at runtime.-z interpose:
Mark object to interpose all DSOs but executable.-(,-),-EL,-Onumber,--dc,--dp,--end-group,--no-add-needed,--no-copy-dt-needed-entries,--nostdlib,--rpath-link=Ar dir,--sort-common,--sort-section,--start-group,--warn-constructors,--warn-once,--fix-cortex-a53-835769,--fix-cortex-a53-843419,-z combreloc,-z common-page-size,-z nocombreloc:
Ignored
ENVIRONMENT VARIABLES
MOLD_JOBS:
If this variable is set to1, only onemoldprocess will run at a time.
If a new mold process is initiated while another is already active, the new
process will wait until the active one completes before starting.The primary reason for this environment variable is to minimize peak memory
usage. Since mold is designed to operate with high parallelism, running
multiple mold instances simultaneously may not be beneficial. If you execute
N instances of mold concurrently, it could require N times the time and N
times the memory. On the other hand, running them one after the other might
still take N times longer, but the peak memory usage would be the same as
running just a single instance.If your build system invokes multiple linker processes simultaneously and
some of them often get killed due to out-of-memory errors, you might
consider setting this environment variable to1to see if it addresses the
OOM issue.Currently, any value other than
1is silently ignored.MOLD_DEBUG:
If this variable is set to a non-empty string,moldembeds its
command-line options in the output file's.commentsection.MOLD_REPRO:
Setting this variable to a non-empty string has the same effect as passing
the--reprooption.
SEE ALSO
gold(1), ld(1), elf(5), ld.so(8)
AUTHOR
Rui Ueyama [email protected]
BUGS
Report bugs to https://github.com/rui314/mold/issues.
File: third-party/zstd/contrib/docker/README.md
Requirement
The Dockerfile script requires a version of docker >= 17.05
Installing docker
The official docker install docs use a ppa with a modern version available:
https://docs.docker.com/install/linux/docker-ce/ubuntu/
How to run
docker build -t zstd .
test
echo foo | docker run -i --rm zstd | docker run -i --rm zstd zstdcat
fooFile: third-party/zstd/contrib/externalSequenceProducer/README.md
externalSequenceProducer
externalSequenceProducer is a test tool for the Block-Level Sequence Producer API.
It demonstrates how to use the API to perform a simple round-trip test.
A sample sequence producer is provided in sequence_producer.c, but the user can swap
this out with a different one if desired. The sample sequence producer implements
LZ parsing with a 1KB hashtable. Dictionary-based parsing is not currently supported.
Command line :
externalSequenceProducer filenameFile: third-party/zstd/contrib/gen_html/README.md
gen_html - a program for automatic generation of zstd manual
Introduction
This simple C++ program generates a single-page HTML manual from zstd.h.
The format of recognized comment blocks is following:
- comments of type
/*!mean: this is a function declaration; switch comments with declarations - comments of type
/**and/*-mean: this is a comment; use a<H2>header for the first line - comments of type
/*=and/**=mean: use a<H3>header and show also all functions until first empty line - comments of type
/*XwhereXis different from above-mentioned are ignored
Moreover:
ZSTDLIB_APIis removed to improve readabilitytypedefare detected and included even if uncommented- comments of type
/**<and/*!<are detected and only function declaration is highlighted (bold)
Usage
The program requires 3 parameters:
gen_html [zstd_version] [input_file] [output_html]To compile program and generate zstd manual we have used:
make
./gen_html.exe 1.1.1 ../../lib/zstd.h zstd_manual.html