Technical Documentation: google/bloaty
ℹ️ Provenance: Multi-Tier Ingestion · CodeWiki Reference
Overview (`README.md`)
Bloaty: a size profiler for binaries
Ever wondered what's making your binary big? Bloaty will
show you a size profile of the binary so you can understand
what's taking up space inside.
/* Detailed source-code truncated for AI context efficiency. */Bloaty performs a deep analysis of the binary. Using custom
ELF, DWARF, and Mach-O parsers, Bloaty aims to accurately
attribute every byte of the binary to the symbol or
compileunit that produced it. It will even disassemble the
binary looking for references to anonymous data.
Bloaty supports many features:
- file formats: ELF, Mach-O, PE/COFF (experimental), WebAssembly (experimental)
- data sources: compileunit (shown above), symbol, section, segment, etc.
- hierarchical profiles: combine multiple data sources into a single report
- size diffs: see where the binary grew, perfect for CI tests
- separate debug files: strip the binary under test, while making debug data available for analysis
- flexible demangling: demangle C++ symbols, optionally discarding function/template parameters
- custom data sources: regex rewrites of built-in data sources, for custom munging/bucketing
- regex filtering: filter out parts of the binary that do or don't match a given regex
- easy to deploy: statically-linked C++ binary, easy to copy around
For detailed info on all of Bloaty's features, see the User
Documentation.
For more information about the analysis performed by Bloaty,
please see How Bloaty Works.
Install
To build, use cmake. For example:
$ cmake -B build -G Ninja -S .
$ cmake --build build
$ cmake --build build --target installBloaty bundles libprotobuf, re2, capstone, andpkg-config as Git submodules, and uses protoc build
from libprotobuf, but it will prefer the system's versions
of those dependencies if available. All other dependencies
are included as Git submodules.
If the Git repository hasn't been cloned with the--recursive, the submodules can be checked out with:
$ git submodule update --init --recursiveTo run the tests, see the info in
tests/README.md.
Support
GitHub issues and PRs welcome. Please include tests when possible, see:
tests/README.md.
This is not an official Google product.
Overview (`tests/README.md`)
Bloaty Tests
Bloaty has two sets of tests:
- C++ tests in
tests/*.cc littests intests/**/*.test
We are in the process of migrating existing tests to lit where possible (see: https://github.com/google/bloaty/issues/221).
`lit` tests
These tests use the lit and yaml2obj tools from LLVM:
yaml2objallows us to generate very specific ELF/Mach-O/PE files from a
text-based YAML format.litlets us intermix this YAML with commands to run Bloaty and make
assertions about its output.
This is ideal for testing Bloaty's parsers, becauseyaml2obj is a precise and readable way of constructing
input payloads.
To run these tests via CMake, a few additional parameters
must be specified currently:
-DLIT_EXECUTABLE=: specifies where to find the lit tool-DFILECHECK_EXECUTABLE=: specifies where to find the FileCheck tool-DYAML2OBJ_EXECUTABLE=: specifies where to find the yaml2obj tool
You can install lit via pip:
pip install --user litThe FileCheck utility and yaml2obj currently need to be provided by the user.
These are part of the LLVM toolchain and require a very recent build
(development release from the main branch) to run the tests.
cmake -B build -G Ninja -S . -DLIT_EXECUTABLE=${HOME}/Library/Python/3.8/bin/lit -DFILECHECK_EXECUTABLE=${HOME}/BinaryCache/llvm.org/bin/FileCheck -DYAML2OBJ_EXECUTABLE=${HOME}/BinaryCache/llvm.org/bin/yaml2obj
cmake --build build --config Debug
cmake --build build --target check-bloatyC++ Tests
The C++ tests are conventional C++ unit tests that use https://github.com/google/googletest.
Going forward, C++ should only be used for tests that do not
parse binary input files. For example, C++ is good for
testing Bloaty's data structures and aggregation/reporting
logic.
To run the C++ tests (Git only, these are not included in the release tarball), type:
$ cmake --build build --config Debug --target testOverview (`third_party/darwin_xnu_macho/README.md`)
Darwin XNU and Mach-O Headers
This directory contains header files from Apple's open source XNU kernel and cctools projects.
These headers define the Mach-O binary format and related kernel types used by bloaty to parse Mach-O binaries.
Source Repositories
The headers are obtained from:
XNU (mach/ headers): https://github.com/apple-oss-distributions/xnu
mach/machine.h- CPU type definitionsmach/vm_prot.h- Virtual memory protection flags
cctools (mach-o/ headers): https://github.com/apple-oss-distributions/cctools
mach-o/fat.h- Universal binary formatmach-o/loader.h- Mach-O file structuremach-o/nlist.h- Symbol table format
The specific versions are documented in the METADATA file.
Local Modifications
To make these headers work cross-platform, we apply minimal local modifications:
- Cross-platform includes: Add
__has_include()guards for system headers - Fallback definitions: Provide fallback typedefs when system headers aren't available
- Bundled headers: Reference bundled copies instead of system headers
All local modifications are documented as patch files in the patches/ directory.
Reproducibility
The header update process is fully reproducible using the provided Python scripts:
Updating Headers
To update to newer versions of the headers:
- Edit
METADATAto specify new git commit hashes - Run the update script:bash
python3 update_headers.py
This will:
- Fetch the specified headers from GitHub
- Apply local modifications from
patches/ - Install the modified headers
Regenerating Patches
If you need to add new local modifications:
- Manually edit the header files as needed
- Regenerate the patches:bash
python3 generate_patches.py
This will:
- Fetch the original upstream headers
- Compare them to your modified versions
- Generate/update patch files in
patches/
Dry Run
To see what would happen without making changes:
python3 update_headers.py --dry-runPatch Files
Current patches:
mach_machine.h.patch: Adds__has_include()guards and fallbackinteger_ttypedef for cross-platform compatibilitymach-o_loader.h.patch: Changes#include <mach/vm_prot.h>to#include "../mach/vm_prot.h"to use bundled header
Overview (`tests/testdata/dwarf5/README.md`)
DWARF5 Test Data
This directory contains test binaries for DWARF5 support.
dwarf5_simple_exe
A simple ELF executable compiled with DWARF5 debug info to test line table parsing.
Building the Binary
This binary should be built on Linux in a clean directory to avoid leaking
local paths in the DW_AT_producer string:
# Build in a temporary clean path
mkdir -p /tmp/src
cp dwarf5_simple.c /tmp/src/
cd /tmp/src
clang -gdwarf-5 dwarf5_simple.c -o dwarf5_simple_exe
cp dwarf5_simple_exe /path/to/bloaty/tests/testdata/dwarf5/Building in /tmp/src ensures that the binary doesn't contain
user-specific paths, making the binary reproducible across different build environments.
Verification
After building, verify the binary is correct:
# Should show version 5
llvm-dwarfdump --debug-line dwarf5_simple_exe | grep "version:"
# Should show clean paths (/tmp/src) in producer, not home directories
llvm-dwarfdump --debug-info dwarf5_simple_exe | grep "DW_AT_producer"Why Not YAML?
We use a pre-compiled binary instead of yaml2obj because yaml2obj doesn't
support DWARF5's structured line table format. DWARF5 uses content type
descriptors (DW_LNCT_*) and forms for directory/file entries, while yaml2obj
currently only supports DWARF4's simple string arrays.
See: https://github.com/llvm/llvm-project/issues/166441
Doc How Bloaty Works (`doc/how-bloaty-works.md`)
How Bloaty Works
At a high level, Bloaty's goal is to create a map of the binary where every
byte has a label attached to it. Every byte starts out as unknown
(unattributed). As we scan the binary we assign labels to different ranges of
the file. For example, if the user selected the "sections" data source we scan
the section table and use the section name as the label for each range.
Ideally these labeled ranges will cover the entire file by the time we are
done. In practice we usually can't achieve perfect 100% coverage. To
compensate for this, we have various kinds of "fallback" labels we attach to
mystery regions of the file. This is how we guarantee an important invariant
of Bloaty: the totals given in Bloaty's output will always match the total size
of the file. This ensures that we always account for the entire file, even if
we don't have detailed information for every byte.
The ELF/Mach-O/etc. data structures we are traversing were not designed to
enable size profiling. They were designed to assist linkers, loaders,
debuggers, stack unwinders, etc. to run and debug the binary. This means that
Bloaty's size analysis is inherently an unconventional use of ELF/Mach-O
metadata. Bloaty has to be clever about how to use the available information to
achieve its goal. This can pose a challenge, but also makes Bloaty fun to work
on. Getting the coverage close to 100% requires a lot of ingenuity (and some
heuristics).
Range Map
RangeMap (as defined in range_map.h)
is the core data structure of Bloaty. It is a sparse map of[start, end) -> std::string that associates regions of VM or file space to
a label.
By the time Bloaty is finished, it has built a complete map of both VM and file
space for the binary. You can view these maps by running Bloaty with '-v':
/* Detailed source-code truncated for AI context efficiency. */The file map refers to file offsets, and these always run from 0 to the size of
the file. The VM map refers to VM addresses; these start at 0 for shared
libraries and position-independent binaries, but these will start at some
non-zero address if the binary was linked to be loaded at a fixed address.
Note that some of the regions in the map have labels like [LOAD #2 [R]]
instead of a true section name. This is because the section table does not
always cover every byte of the file. Bloaty gives these regions a fallback label
that contains the segment name instead. We must attach some kind of label to
every byte of the file, otherwise Bloaty's totals would not match the file size.
Also notice that there is an entry in the VM map that says [-- Nothing mapped --]. This is calling attention to the fact that there is a gap in the address
space here. Since nothing is mapped, these regions of the VM space don't
actually need to accessible in the target process image. However, unless this
unused space aligns with page boundaries, it will probably end up getting
mapped anyway.
Sometimes we know a region's start but not its end. For example, Mach-O
symbols have an address but not a size (whereas ELF symbols have both).
To support this case, RangeMap supports adding an address with kUnknownSize.
A range with unknown size will automatically extend to the beginning of the
next region, even if the next region is added later.
If we try to add a label to a range of the binary that has already been assigned
a label, the first label assigned takes precedence. This means that the order
in which we scan data structures is significant. So our general strategy is to
scan our most granular and detailed information first. We scan generic
information as a last resort, to give at least some information for parts of the
binary that we couldn't find any more specific information about.
VM Space and File Space
Loadable binaries have two fundamental domains of space we are trying to map:
VM space and file space. File space is the bytes of the input file. VM
space is the bytes of memory when the executable is loaded at runtime. Some
regions of the binary exist only in file space (like debug info) and some
regions exist only in VM space (like .bss, zero-initialized data). Even
entities that exist in both spaces can have different sizes in each.
We create two separate RangeMap structures for these two domains. For
convenience, we put them together into a single structure called DualMap:
struct DualMap {
RangeMap vm_map;
RangeMap file_map;
};We populate these two maps simultaneously as we scan the file. We must populate
both maps even if we only care about one of them, because most of the metadata we're
scanning gives us VM addresses or file offsets, not both. For example,
debug info always refers to VM addresses, because it's intended for debugging at
runtime. Even if we only care about file size, we still have to scan VM addresses
and translate them to file offsets.
Bloaty's overall analysis algorithm (in pseudo-code) is:
for (auto f : files) {
// Always start by creating the base map.
DualMap base_map = ScanBaseMap(f);
// Scan once for every data source the user selected with '-d'.
std::vector<DualMap> maps;
for (auto s : data_sources) {
maps.push_back(ScanDataSource(f, s));
}
}Base Map
To translate between VM and file space, we always begin by creating a "base
map." The base map is just a DualMap like any other, but we give it special
meaning:
- It defines what ranges of file and VM space constitute "the entire binary"
(ie. the "TOTALS" row of the final report). - We use it to translate between VM space and File space.
This means that the base map must be exhaustive, and must also provide
translation for any entity that exists in both VM and file space. For example,
suppose we are scanning the "symbols" data source and we see in the symbol
table that address 0x12345 corresponds to symbol foo. We will add that to
VM map immediately, but we will also use the base map to translate address0x12355 to a file offset so we can add that range to the file map.
How does the base map store translation info? I left one thing out aboutRangeMap above. In addition to storing a label for every region, it can also
(optionally) store a member called other_start. This stores the
corresponding offset in the other space, and lets you translate addresses from
one to the other. The other_start member is only used in the base map.
We build the base map by scanning either the segments (program headers) or
sections of the binary. These give both VM address and file offset for regions
of the binary that are loaded into memory. To make sure we cover the entire
file space, we use [Unmapped] as a last ditch fallback for any regions of the
on-disk binary that didn't have any segment/section data associated with them.
This ensures that Bloaty always accounts for the entire physical binary, even if
we can't find any information about it.
Scanning Data Sources
Once we have built the base map, we can get on to the meat of Bloaty's work.
We can now scan the binary according to whatever data source(s) the user has
selected.
Segments and Sections
The segments and sections data sources are relatively straightforward. For
the most part we can simply scan the segments/sections table and call it a day.
For ELF, segments and sections have distinct tables in the binary that can be
scanned independently. This means that technically a section could span multiple
segments, but in practice segments/sections form a 1:many relationship, where
each section is contained entirely within a single segment.
Currently Bloaty only reports PT_LOAD and PT_TLS segments. We scan PT_LOAD
segments first, so if there is overlap with PT_TLS the PT_LOAD label will
win. In the future It may make sense to scan PT_TLS first, as this is more
granular data that can give insight into the per-thread runtime overhead of TLS
variables. It may also make sense to scan other segment types, to give more
granular info.
ELF segments do not have names. To distinguish between different PT_LOAD
segments, we include both a segment offset and the segment flags in the label,
eg. LOAD #2 [R].
For Mach-O, segments are contained within a file-level table of "load
commands." Each load command has a type, and technically speaking, segments are
a subset of all load commands. However Bloaty's segments data source reports
many non-segment load commands such as the symbol table (LC_SYMTAB,LC_DYSYMTAB), code signature (LC_CODE_SIGNATURE), and more. Segments can
have zero or more sections, so in Mach-O files the 1:many nature of segments
and sections is enforced by the file format.
For segments and sections we have to decide how to attribute the regions of
the file that correspond to the segment/section headers themselves. Bloaty's
general philosophy is to include the metadata with the data, so each label
shows the true weight of everything associated with that label. This would
suggest that the .text label should include the .text section as well as
the section header entry for the .text section. However this would hide the
overhead of the ELF headers, which can be significant if there are many
sections. Bloaty currently has no higher-level data source that could show the
ELF headers separately from the ELF data, and even if there was such a data
source it would have narrow usefulness so people would probably not think to
use it very often.
There is not an easy answer to this question. At the moment Bloaty will
include section headers with the corresponding section, but will not include
segment headers with the corresponding segment. This may or may not be the
best solution to this problem, and this may change if another solution proves
to work better.
Symbols
The symbols data source is where Bloaty's deep parsing of the binary delivers
the most benefit, as it provides detailed information that you cannot get
from a linker map or symbol table.
For example, take the following data from running Bloaty on itself:
$ ./bloaty bloaty -d symbols,sections
FILE SIZE VM SIZE
-------------- --------------
[...]
0.2% 116Ki 1.6% 116Ki AArch64_printInst
84.9% 98.8Ki 84.9% 98.8Ki .text
14.9% 17.4Ki 14.9% 17.4Ki .rodata
0.1% 156 0.1% 156 .eh_frame
0.0% 24 0.0% 0 .symtab
0.0% 18 0.0% 0 .strtab
0.0% 8 0.0% 8 .eh_frame_hdr
[...]
0.1% 50.1Ki 0.7% 49.8Ki reg_name_maps
59.6% 29.8Ki 59.8% 29.8Ki .rela.dyn
40.0% 20.0Ki 40.2% 20.0Ki .data.rel.ro
0.4% 216 0.0% 0 .symtab
0.0% 14 0.0% 0 .strtabI excerpted two symbols from the report. Between these two symbols, Bloaty has
found seven distinct kinds of data that contributed to these two symbols. If
you wrote a tool that naively just parsed the symbol table, you would only find
the first of these seven:
.text./.data.rel.ro: this is the data we obtain by simply following the
symbol table entry. This is the primary code or data emitted by the function
or variable..eh_frame: this is the "unwind information" for a function. It is used for
many things, including C++
exceptions and stack traces when no frame pointer is available..eh_frame_hdr: this is metadata about the.eh_framesection..symtab: this is the function/variable's symbol table entry itself. It is a
fixed size for every entry. The fact thatreg_name_mapsabove has a.symtabsize of 216 indicates that there must actually be 9 different
symbols being represented by this entry. Bloaty has combined them because
they all have the same name. We can break them apart if we want using:text
/* Detailed source-code truncated for AI context efficiency. */
5. `.strtab`: this is the text of the function/variables's name itself in the
string table. Longer names take up more space in the binary, and Bloaty's
analysis here reflects that (though the symbol table is not loaded at
runtime, so it's not costing RAM).
6. `.rela.dyn`: these are relocations embedded into the executable. Normally we
would associate relocations with `.o` files and not the final linked binary.
However shared objects and position-independent executables must also emit
relocations for any global variable that is initialized to an address of some
other data. These relocations can take up a significant amount of space,
indeed more space than the data itself in this case! Without this deep
analysis of the binary, this cost would be invisible. Bloaty scans all
relocation tables and "charges" each relocation entry to the function/data
that requires the relocation (*not* the function being pointed to).
7. `.rodata`: Bloaty has found some data associated with this function.
Sometimes data doesn't get its own symbol table entry, for whatever reason.
Bloaty can attribute anonymous data to the function that uses it by
disassembling the binary looking for instructions that reference a different
part of the binary. If the same anonymous data is used by more than one
function, then the first one scanned will "win" and assume the whole cost,
as Bloaty has no concept of sharing the cost. Every byte of the file must
have exactly one label associated with it.
Note that this is more granular information than you can get from a linker map
file. A linker map file will break down some of these sections by compile unit,
but the symbol-level granularity is limited to the primary code/data for each
symbol (#1 in the list above).
### Compile Units
Like symbols, we can see that Bloaty is capable of breaking down lots of
sections by compile unit:$ ./bloaty bloaty -d compileunits,sections
FILE SIZE VM SIZE
37.9% 17.7Mi 49.4% 3.52Mi [160 Others]
15.0% 7.04Mi 3.4% 246Ki ../third_party/protobuf/src/google/protobuf/descriptor.cc
33.9% 2.38Mi 0.0% 0 .debug_info
32.6% 2.29Mi 0.0% 0 .debug_loc
17.2% 1.21Mi 0.0% 0 .debug_str
6.5% 468Ki 0.0% 0 .debug_ranges
5.3% 381Ki 0.0% 0 .debug_line
2.8% 204Ki 83.1% 204Ki .text
1.0% 70.9Ki 0.0% 0 .strtab
0.4% 25.7Ki 10.4% 25.7Ki .eh_frame
0.2% 13.3Ki 0.0% 0 .symtab
0.1% 10.6Ki 4.3% 10.6Ki .rodata
0.1% 3.97Ki 1.6% 3.97Ki .eh_frame_hdr
0.0% 1.03Ki 0.4% 1.03Ki .rela.dyn
0.0% 368 0.1% 368 .data.rel.ro
0.0% 0 0.0% 81 .bss
[...]
To attribute all of the different `.debug_*` sections, Bloaty includes parsers
for all of the different DWARF formats that live in these sections. We also use
the DWARF data to find which symbols belong to which compile units.
The `compileunits` data source contains much of the same data that you could get
from a linker map. Since each compile unit generally comes from a separate `.o`
file, a linker map can often give good data about which parts of the binary came
from which translation units. However Bloaty is able to derive this data without
needing a linker map file, which may be tricky to obtain. The `compileunits`
data source is also useful when combined with other data sources in hierarchical
profiles.
---
### Doc Using (`doc/using.md`)
# User Documentation
## Running Bloaty
Run it directly on a binary target. For example, run it on itself.$ ./bloaty bloaty
On Linux you'll see output something like:
```cmdoutput
$ ./bloaty bloaty
FILE SIZE VM SIZE
-------------- --------------
30.0% 8.85Mi 0.0% 0 .debug_info
24.7% 7.29Mi 0.0% 0 .debug_loc
12.8% 3.79Mi 0.0% 0 .debug_str
9.7% 2.86Mi 42.8% 2.86Mi .rodata
6.9% 2.03Mi 30.3% 2.03Mi .text
6.3% 1.85Mi 0.0% 0 .debug_line
4.0% 1.19Mi 0.0% 0 .debug_ranges
0.0% 0 15.0% 1.01Mi .bss
1.6% 473Ki 0.0% 0 .strtab
1.4% 435Ki 6.3% 435Ki .data
0.8% 254Ki 3.7% 254Ki .eh_frame
0.8% 231Ki 0.0% 0 .symtab
0.5% 142Ki 0.0% 0 .debug_abbrev
0.2% 56.8Ki 0.8% 56.8Ki .gcc_except_table
0.1% 41.4Ki 0.6% 41.4Ki .eh_frame_hdr
0.0% 11.4Ki 0.1% 9.45Ki [26 Others]
0.0% 7.20Ki 0.1% 7.14Ki .dynstr
0.0% 6.09Ki 0.1% 6.02Ki .dynsym
0.0% 4.89Ki 0.1% 4.83Ki .rela.plt
0.0% 4.59Ki 0.0% 0 [Unmapped]
0.0% 3.30Ki 0.0% 3.23Ki .plt
100.0% 29.5Mi 100.0% 6.69Mi TOTALThe "VM SIZE" column tells you how much space the binary
will take when it is loaded into memory. The "FILE SIZE"
column tells you about how much space the binary is taking
on disk. These two can be very different from each other:
- Some data lives in the file but isn't loaded into memory,
like debug information. - Some data is mapped into memory but doesn't exist in the
file. This mainly applies to the.bsssection
(zero-initialized data).
The default breakdown in Bloaty is by sections, but many
other ways of slicing the binary are supported such as
symbols and segments. If you compiled with debug info, you
can even break down by compile units and inlines!
/* Detailed source-code truncated for AI context efficiency. */Run Bloaty with --help to see a list of available options:
/* Detailed source-code truncated for AI context efficiency. */Size Diffs
You can use Bloaty to see how the size of a binary changed.
On the command-line, pass -- followed by the files you
want to use as the diff base.
For example, here is a size diff between a couple different versions
of Bloaty, showing how it grew when I added some features.
$ ./bloaty bloaty -- oldbloaty
VM SIZE FILE SIZE
-------------- --------------
[ = ] 0 .debug_loc +688Ki +9.9%
+19% +349Ki .text +349Ki +19%
[ = ] 0 .debug_ranges +180Ki +11%
[ = ] 0 .debug_info +120Ki +0.9%
+23% +73.5Ki .rela.dyn +73.5Ki +23%
+3.5% +57.1Ki .rodata +57.1Ki +3.5%
+28e3% +53.9Ki .data +53.9Ki +28e3%
[ = ] 0 .debug_line +40.2Ki +4.8%
+2.3% +5.35Ki .eh_frame +5.35Ki +2.3%
-6.0% -5 [Unmapped] +2.65Ki +215%
+0.5% +1.70Ki .dynstr +1.70Ki +0.5%
[ = ] 0 .symtab +1.59Ki +0.9%
[ = ] 0 .debug_abbrev +1.29Ki +0.5%
[ = ] 0 .strtab +1.26Ki +0.3%
+16% +992 .bss 0 [ = ]
+0.2% +642 [13 Others] +849 +0.2%
+0.6% +792 .dynsym +792 +0.6%
+16% +696 .rela.plt +696 +16%
+16% +464 .plt +464 +16%
+0.8% +312 .eh_frame_hdr +312 +0.8%
[ = ] 0 .debug_str -19.6Ki -0.4%
+11% +544Ki TOTAL +1.52Mi +4.6%Each line shows the how much each part changed compared to
its previous size. Most sections grew, but one section at
the bottom (.debug_str) shrank. The "TOTAL" line shows
how much the size changed overall.
Hierarchical Profiles
Bloaty supports breaking the binary down in lots of
different ways. You can combine multiple data sources into
a single hierarchical profile. For example, we can use thesegments and sections data sources in a single report:
/* Detailed source-code truncated for AI context efficiency. */Bloaty displays a maximum of 20 lines for each level; other
values are grouped into an [Other] bin. Use -n <num>
to override this setting. If you pass -n 0, all data
will be output without collapsing anything into [Other].
Debugging Stripped Binaries
Bloaty supports reading debuginfo/symbols from separate
binaries. This lets you profile a stripped binary, even for
data sources like "compileunits" or "symbols" that require
this extra information.
Bloaty uses build IDs to verify that the binary and the
debug file match. Otherwise the results would be nonsense
(this kind of mismatch might sound unlikely but it's a very
easy mistake to make, and one that I made several times even
as Bloaty's author!).
If your binary has a build ID, then using separate debug
files is as simple as:
$ cp bloaty bloaty.stripped
$ strip bloaty.stripped
$ ./bloaty -d symbols --debug-file=bloaty bloaty.strippedSome format-specific notes follow.
ELF
For ELF, make sure you are compiling with build IDs enabled.
With gcc this happens automatically, but Clang decided not
to make this the default, since it makes the link
slower.
For Clang add -Wl,--build-id to your link line. (If you
want a slightly faster link and don't care about
reproducibility, you can use -Wl,--build-id=uuid instead).
Bloaty does not currently support the GNU debuglink or
looking up debug files by build ID, which are the methods
GDB uses to find debug
files.
If there are use cases where Bloaty's --debug-file option
won't work, we can reconsider implementing these.
Mach-O
Mach-O files always have build IDs (as far as I can tell),
so no special configuration is needed to make sure you get
them.
Mach-O puts debug information in separate files which you
can create with dsymutil:
$ dsymutil bloaty
$ strip bloaty (optional)
$ ./bloaty -d symbols bloaty # Auto-discovers bloaty.dSYMBloaty will automatically discover and load debug symbols frombloaty.dSYM/Contents/Resources/DWARF/bloaty when analyzing bloaty.
If you used dsymutil -o to write the dSYM to a different directory,
you can explicitly specify its location:
$ dsymutil bloaty -o /path/to/symbols/bloaty.dSYM
$ ./bloaty -d symbols --dsym=/path/to/symbols/bloaty.dSYM bloatyNote: The --dsym option is also useful for debugging issues with auto-loading dSYMs.
When you explicitly specify --dsym, Bloaty will throw informative error messages
if the dSYM file is invalid or has a mismatched build ID, rather than silently
ignoring the file as it does during auto-discovery.
For cross-platform compatibility, --debug-file still works:
$ ./bloaty -d symbols --debug-file=bloaty.dSYM/Contents/Resources/DWARF/bloaty bloatyConfiguration Files
Any options that you can specify on the command-line, you
can put into a configuration file instead. Then use can use-c FILE to load those options from the config file. Also,
a few features are only available with configuration files
and cannot be specify on the command-line.
The configuration file is a in Protocol Buffers text format.
The schema is the Options message in
src/bloaty.proto.
The two most useful cases for configuration files are:
You have too many input files to put on the command-line.
At Google we sometimes run Bloaty over thousands of input
files. This can cause the overall command-line to exceed
OS limits. With a config file, we can avoid this:textfilename: "path/to/long_filename_a.o" filename: "path/to/long_filename_b.o" filename: "path/to/long_filename_c.o" # ...repeat for thousands of files.For custom data sources, it can be very useful to put
them in a config file, for greater reusability. For
example, see the custom data sources defined in
custom_sources.bloaty.
Also read more about custom data sources below.
Data Sources
Bloaty has many data sources built in. These all provide
different ways of looking at the binary. You can also
create your own data sources by applying regexes to the
built-in data sources (see "Custom Data Sources" below).
While Bloaty works on binaries, shared objects, object
files, and static libraries (.a files), some of the data
sources don't work on object files. This applies especially
to data sources that read debug info.
Segments
Segments are what the run-time loader uses to determine what
parts of the binary need to be loaded/mapped into memory.
There are usually just a few segments: one for each set ofmmap() permissions required:
$ ./bloaty -d segments bloaty
FILE SIZE VM SIZE
-------------- --------------
80.7% 23.8Mi 0.0% 0 [Unmapped]
10.9% 3.21Mi 47.9% 3.21Mi LOAD #4 [R]
6.9% 2.03Mi 30.3% 2.03Mi LOAD #3 [RX]
1.5% 439Ki 21.4% 1.44Mi LOAD #5 [RW]
0.1% 23.3Ki 0.3% 23.3Ki LOAD #2 [R]
0.0% 2.56Ki 0.0% 0 [ELF Headers]
100.0% 29.5Mi 100.0% 6.69Mi TOTALHere we see one segment mapped [RX] (read/execute) and
one segment mapped [RW] (read/write). A large part of
the binary is not loaded into memory, which we see as[Unmapped].
Object files and static libraries don't have segments.
However we fake it by grouping sections by their flags.
This gives us a break-down sort of like real segments.
$ ./bloaty -d segments CMakeFiles/libbloaty.dir/src/bloaty.cc.o
FILE SIZE VM SIZE
-------------- --------------
87.5% 972Ki 0.0% 0 Section []
8.2% 90.9Ki 78.3% 90.9Ki Section [AX]
2.3% 25.2Ki 21.7% 25.2Ki Section [A]
2.0% 22.6Ki 0.0% 0 [ELF Headers]
0.1% 844 0.0% 0 [Unmapped]
0.0% 24 0.1% 72 Section [AW]
100.0% 1.09Mi 100.0% 116Ki TOTALSections
Sections give us a bit more granular look into the binary.
If we want to find the symbol table, the unwind information,
or the debug information, each kind of information lives in
its own section. Bloaty's default output is sections.
$ ./bloaty -d sections bloaty
FILE SIZE VM SIZE
-------------- --------------
30.0% 8.85Mi 0.0% 0 .debug_info
24.7% 7.29Mi 0.0% 0 .debug_loc
12.8% 3.79Mi 0.0% 0 .debug_str
9.7% 2.86Mi 42.8% 2.86Mi .rodata
6.9% 2.03Mi 30.3% 2.03Mi .text
6.3% 1.85Mi 0.0% 0 .debug_line
4.0% 1.19Mi 0.0% 0 .debug_ranges
0.0% 0 15.0% 1.01Mi .bss
1.6% 473Ki 0.0% 0 .strtab
1.4% 435Ki 6.3% 435Ki .data
0.8% 254Ki 3.7% 254Ki .eh_frame
0.8% 231Ki 0.0% 0 .symtab
0.5% 142Ki 0.0% 0 .debug_abbrev
0.2% 56.8Ki 0.8% 56.8Ki .gcc_except_table
0.1% 41.4Ki 0.6% 41.4Ki .eh_frame_hdr
0.0% 11.4Ki 0.1% 9.45Ki [26 Others]
0.0% 7.20Ki 0.1% 7.14Ki .dynstr
0.0% 6.09Ki 0.1% 6.02Ki .dynsym
0.0% 4.89Ki 0.1% 4.83Ki .rela.plt
0.0% 4.59Ki 0.0% 0 [Unmapped]
0.0% 3.30Ki 0.0% 3.23Ki .plt
100.0% 29.5Mi 100.0% 6.69Mi TOTALSections are regions of the binary that are the linker
treats as atomic when linking. The linker will never break
apart or rearrange the data within a section. This is why it
is necessary to compile with -ffunction-sections and-fdata-sections if you want the linker to strip out
individual functions or variables that have no references.
However the linker will often combine many input sections
into a single output section.
Symbols
Symbols come from the symbol table, and represent individual
functions or variables.
$ ./bloaty -d symbols bloaty
FILE SIZE VM SIZE
-------------- --------------
30.0% 8.85Mi 0.0% 0 [section .debug_info]
24.7% 7.29Mi 0.0% 0 [section .debug_loc]
12.8% 3.79Mi 0.0% 0 [section .debug_str]
11.7% 3.44Mi 41.2% 2.76Mi [5895 Others]
6.3% 1.85Mi 0.0% 0 [section .debug_line]
4.9% 1.43Mi 21.4% 1.43Mi insns
4.0% 1.19Mi 0.0% 0 [section .debug_ranges]
0.0% 44 14.9% 1024Ki g_instruction_table
0.8% 255Ki 3.7% 255Ki [section .rodata]
0.8% 240Ki 3.5% 240Ki printAliasInstr
0.6% 175Ki 2.6% 175Ki insn_ops
0.5% 153Ki 2.2% 153Ki ARMInsts
0.5% 142Ki 0.0% 0 [section .debug_abbrev]
0.5% 140Ki 2.0% 140Ki x86DisassemblerTwoByteOpcodes
0.4% 113Ki 1.6% 113Ki insn_name_maps
0.4% 106Ki 1.6% 106Ki printInstruction.OpInfo
0.3% 97.1Ki 1.4% 96.9Ki printInstruction.OpInfo2
0.2% 74.0Ki 1.1% 74.0Ki x86DisassemblerThreeByte38Opcodes
0.2% 71.1Ki 1.0% 70.8Ki printInstruction.AsmStrs
0.2% 61.1Ki 0.9% 60.9Ki DecoderTable32
0.2% 56.8Ki 0.8% 56.8Ki [section .gcc_except_table]
100.0% 29.5Mi 100.0% 6.69Mi TOTALYou can control how symbols are demangled with the -C MODE
or --demangle=MODE flag. You can also specify the
demangling mode explicitly in the -d switch. We have
three different demangling modes:
-C noneor-d rawsymbols: no, demangling.-C shortor-d shortsymbols: short demangling: return
types, template parameters, and function parameter types
are omitted. For example:bloaty::dwarf::FormReader<>::GetFunctionForForm<>().
This is the default.-C fullor-d fullsymbols: full demangling.
One very handy thing about -C short (the default) is that
it groups all template instantiations together, regardless
of their parameters. You can use this to determine how much
code size you are paying by doing multiple instantiations of
templates. Try bloaty -d shortsymbols,fullsymbols.
Input Files
When you pass multiple files to Bloaty, the inputfiles
source will let you break it down by input file:
$ ./bloaty -d inputfiles CMakeFiles/libbloaty.dir/src/*.o
FILE SIZE VM SIZE
-------------- --------------
42.8% 1.09Mi 37.9% 116Ki CMakeFiles/libbloaty.dir/src/bloaty.cc.o
15.7% 407Ki 15.5% 47.6Ki CMakeFiles/libbloaty.dir/src/dwarf.cc.o
10.3% 266Ki 10.4% 31.8Ki CMakeFiles/libbloaty.dir/src/bloaty.pb.cc.o
9.0% 232Ki 9.7% 29.8Ki CMakeFiles/libbloaty.dir/src/elf.cc.o
8.0% 207Ki 8.7% 26.6Ki CMakeFiles/libbloaty.dir/src/macho.cc.o
4.4% 114Ki 4.3% 13.1Ki CMakeFiles/libbloaty.dir/src/webassembly.cc.o
4.0% 103Ki 7.5% 22.9Ki CMakeFiles/libbloaty.dir/src/demangle.cc.o
3.4% 87.0Ki 3.3% 10.2Ki CMakeFiles/libbloaty.dir/src/range_map.cc.o
2.5% 64.3Ki 2.6% 7.94Ki CMakeFiles/libbloaty.dir/src/disassemble.cc.o
100.0% 2.53Mi 100.0% 306Ki TOTALArchive Members
When you are running Bloaty on a .a file, the armembers
source will let you break it down by .o file inside the
archive.
$ ./bloaty -d armembers liblibbloaty.a
FILE SIZE VM SIZE
-------------- --------------
28.5% 1.21Mi 23.5% 120Ki cxa_demangle.cpp.o
25.6% 1.09Mi 22.6% 116Ki bloaty.cc.o
9.4% 407Ki 9.3% 47.6Ki dwarf.cc.o
6.2% 266Ki 6.2% 31.8Ki bloaty.pb.cc.o
5.4% 232Ki 5.8% 29.8Ki elf.cc.o
4.8% 207Ki 5.2% 26.6Ki macho.cc.o
2.6% 114Ki 2.6% 13.1Ki webassembly.cc.o
2.4% 103Ki 4.5% 22.9Ki demangle.cc.o
2.0% 87.0Ki 2.0% 10.2Ki range_map.cc.o
1.9% 80.4Ki 3.2% 16.7Ki charconv_bigint.cc.o
1.8% 79.3Ki 2.7% 14.0Ki escaping.cc.o
1.5% 65.0Ki 2.1% 10.9Ki [9 Others]
1.5% 64.3Ki 1.5% 7.94Ki disassemble.cc.o
1.4% 59.9Ki 0.0% 0 [AR Symbol Table]
1.0% 45.2Ki 2.4% 12.4Ki numbers.cc.o
0.9% 40.9Ki 2.2% 11.4Ki charconv.cc.o
0.9% 38.8Ki 1.2% 6.10Ki int128.cc.o
0.7% 30.1Ki 1.1% 5.58Ki str_cat.cc.o
0.6% 24.1Ki 0.8% 3.92Ki string_view.cc.o
0.5% 21.2Ki 0.6% 3.21Ki throw_delegate.cc.o
0.4% 19.2Ki 0.4% 2.26Ki ascii.cc.o
100.0% 4.23Mi 100.0% 512Ki TOTALYou are free to use this data source even for non-.a
files, but it won't be very useful since it will always just
resolve to the input file (the .a file).
Compile Units
Using debug information, we can tell what compile unit (and
corresponding source file) each bit of the binary came from.
/* Detailed source-code truncated for AI context efficiency. */Inlines
The DWARF debugging information also contains "line info"
information that understands inlining. So within a
function, it will know which instructions came from an
inlined function from a header file. This is the
information the debugger uses to point at a specific source
line as you're tracing through a program.
/* Detailed source-code truncated for AI context efficiency. */Custom Data Sources
Sometimes you want to munge the labels from an existing data
source. For example, when we use "compileunits" on Bloaty
itself, we see files from all our dependencies mixed
together:
/* Detailed source-code truncated for AI context efficiency. */If we want to bucket all of these by which library they came
from, we can write a custom data source. It specifies the
base data source and a set of regexes to apply to it. The
regexes are tried in order, and the first matching regex
will cause the entire label to be rewritten to the
replacement text. Regexes follow RE2
syntax and the
replacement can refer to capture groups.
$ cat bloaty_package.bloaty
custom_data_source: {
name: "bloaty_package"
base_data_source: "compileunits"
rewrite: {
pattern: "^(\\.\\./)?src"
replacement: "src"
}
rewrite: {
pattern: "^(\\.\\./)?(third_party/\\w+)"
replacement: "\\2"
}
}Then use the data source like so:
$ ./bloaty -c bloaty_package.bloaty -d bloaty_package bloaty
FILE SIZE VM SIZE
-------------- --------------
56.6% 16.7Mi 16.6% 1.11Mi third_party/protobuf
24.9% 7.35Mi 68.5% 4.58Mi third_party/capstone
9.4% 2.77Mi 3.2% 221Ki third_party/re2
4.6% 1.36Mi 4.1% 280Ki src
2.1% 637Ki 1.7% 117Ki third_party/demumble
0.7% 209Ki 1.1% 73.8Ki third_party/abseil
0.7% 204Ki 3.0% 204Ki [section .rodata]
0.2% 56.8Ki 0.8% 56.8Ki [section .gcc_except_table]
0.2% 47.7Ki 0.0% 0 [section .debug_str]
0.2% 46.3Ki 0.0% 0 [section .symtab]
0.1% 42.0Ki 0.6% 42.0Ki [section .text]
0.1% 41.4Ki 0.0% 0 [section .debug_loc]
0.1% 29.3Ki 0.0% 0 [section .strtab]
0.0% 12.0Ki 0.2% 11.5Ki [30 Others]
0.0% 7.36Ki 0.0% 0 [section .debug_ranges]
0.0% 6.10Ki 0.1% 6.10Ki [section .dynstr]
0.0% 4.99Ki 0.1% 4.99Ki [section .dynsym]
0.0% 4.77Ki 0.1% 4.77Ki [section .eh_frame]
0.0% 4.59Ki 0.0% 0 [Unmapped]
0.0% 3.23Ki 0.0% 3.23Ki [section .plt]
0.0% 2.50Ki 0.0% 0 [ELF Headers]
100.0% 29.5Mi 100.0% 6.69Mi TOTALWe can get an even richer report by combining thebloaty_package source with the original compileunits
source:
/* Detailed source-code truncated for AI context efficiency. */Source maps
Bloaty provides a --source-map option which allows
specifying a source map for the binary. It takes an ID and
source map file name, separated by '='. The ID can be the
build ID or path to the binary file as specified in the
command line.
For example, to use the compileunits and inlines data
sources for Wasm, a source map must be provided. For Wasm,
the ID can be the text in the sourceMappingURL section of
the binary.
$ ./bloaty -d compileunits --domain=file --source-map=o.wasm=o.wasm.map o.wasm
FILE SIZE
--------------
45.0% 49 [section sourceMappingURL]
19.3% 21 Main.java
12.8% 14 [section Export]
7.3% 8 [WASM Header]
6.4% 7 [section Code]
5.5% 6 [section Type]
3.7% 4 [section Function]
100.0% 109 TOTALSource filter
Sometimes, you are only interested in parts of the binary
instead of the whole package. This is common in embedded
programming, where ELF files are used only as a container
format, and only a few sections are actually loaded onto
the device.
For this, Bloaty provides a --source-filter option which
allows filtering out irrelevant data. It takes a regex
which is applied to each of the symbol names in a data
source. Only symbols which match the regex are displayed
in the output. This is especially powerful when combined
with custom data sources, as the rewriting occurs before
the filtering.
In the case of hierarchical data source profiles, the regex
is applied to all symbol names in the hierarchy. If any
name matches, all of its parents will be displayed as well.
For example, given the above scenario, maybe we are only
interested in how large the first-party Bloaty code is.
This can be displayed using a source filter on the src
directory.
$ ./bloaty -c config.bloaty -d bloaty_package,compileunits --source-filter ^src bloaty
FILE SIZE VM SIZE
-------------- --------------
100.0% 1.36Mi 100.0% 280Ki src
39.4% 549Ki 40.7% 114Ki src/bloaty.cc
13.9% 193Ki 15.0% 42.1Ki src/dwarf.cc
10.8% 150Ki 0.5% 1.28Ki src/main.cc
8.1% 113Ki 8.9% 25.0Ki src/bloaty.pb.cc
7.7% 108Ki 9.0% 25.2Ki src/elf.cc
7.2% 99.9Ki 10.3% 29.0Ki src/macho.cc
4.7% 66.2Ki 7.0% 19.5Ki src/demangle.cc
3.5% 49.5Ki 3.8% 10.5Ki src/webassembly.cc
2.8% 38.8Ki 2.7% 7.50Ki src/range_map.cc
1.9% 26.2Ki 2.1% 5.98Ki src/disassemble.cc
100.0% 1.36Mi 100.0% 280Ki TOTAL
Filtering enabled (source_filter); omitted file = 28.1Mi, vm = 6.42Mi of entries--- METRICS ---
- Files Extracted: 6
- Estimated Token Budget: ~11580 tokens
- Canonical Reference: https://codewiki.google/github.com/google/bloaty