redex Documentation

Technical Documentation: facebook/redex

ℹ️ Provenance: Multi-Tier Ingestion · CodeWiki Reference

Overview (`README.md`) Copy Section

ReDex: An Android Bytecode Optimizer

ReDex is an Android bytecode (dex) optimizer originally developed at
Facebook. It provides a framework for reading, writing, and analyzing .dex
files, and a set of optimization passes that use this framework to improve the
bytecode. An APK optimized by ReDex should be smaller and faster than its
source.

Go to https://fbredex.com for full documentation.

Contributions Copy Chapter

See CONTRIBUTING.md to understand how to contribute to this project.

License Copy Chapter

The ReDex repository is available under the MIT License.

Overview (`sparta/README.md`) Copy Section

SPARTA

SPARTA is a library of software components specially designed for building high-performance static analyzers based on the theory of Abstract Interpretation.

Abstract Interpretation Copy Chapter

Abstract Interpretation is a theory of semantic approximation that provides a foundational framework for the design of static program analyzers. Static analyzers built following the methodology of Abstract Interpretation are mathematically sound, i.e., the semantic information they compute is guaranteed to hold in all possible execution contexts considered. Moreover, these analyzers are able to infer complex properties of programs, the expressiveness of which can be finely tuned in order to control the analysis time. Static analyzers based on Abstract Interpretation are routinely used to perform the formal verification of flight software in the aerospace industry, for example.

Why SPARTA? Copy Chapter

Building an industrial-grade static analysis tool based on Abstract Interpretation from scratch is a daunting task that requires the attention of experts in the field. The purpose of SPARTA is to drastically simplify the engineering of Abstract Interpretation by providing a set of software components that have a simple API, are highly performant and can be easily assembled to build a production-quality static analyzer. By encapsulating the complex implementation details of Abstract Interpretation, SPARTA lets the tool developer focus on the three fundamental axes in the design of an analysis:

• Semantics: the program properties to analyze (range of numerical variables, aliasing relation, etc.)
• Partitioning: the granularity of the properties analyzed (intraprocedural, interprocedural, context-sensitive, etc.)
• Abstraction: the representation of the program properties (sign, interval, alias graph, etc.)

SPARTA is an acronym that stands for Semantics, PARTitioning and Abstraction.

Using SPARTA Copy Chapter

A detailed documentation for SPARTA can be found in the code of the library itself. It includes code samples, typical use cases and references to the literature. Additionally, the unit tests are a good illustration of how to use the API. The unit test for the fixpoint iterator, in particular, implements a complete analysis (live variables) for a simple language.

SPARTA is the analytic engine that powers most optimization passes of the ReDex Android bytecode optimizer. The ReDex codebase contains multiple examples of analyses built with SPARTA that run at industrial scale. The interprocedural constant propagation illustrates how to assemble the building blocks provided by SPARTA in order to implement a sophisticated yet scalable analysis.

Dependencies Copy Chapter

SPARTA requires Boost 1.71 or later.

macOS Copy Section

You will need Xcode with the command line tools installed. To get the command line tools, please use:

xcode-select --install

Boost can be obtained via homebrew:

brew install boost

Ubuntu (64-bit) Copy Section

On Ubuntu 16.04 or later, Boost can be installed through the package manager:

sudo apt-get install libboost-all-dev

For earlier versions of Ubuntu, we provide a script to install Boost:

sudo ./get_boost.sh

Setup Copy Chapter

SPARTA is a header-only C++ library. You can just copy the contents of the include directory to any location in the include path of your compiler.

We also provide a quick setup using cmake that builds all the tests:

# Assume you are in the SPARTA directory
mkdir build-cmake
cd build-cmake
# .. is the root source directory of SPARTA
cmake ..
cmake --build .

To run the unit tests, please type:

make test

To copy the header files into /usr/local/include/sparta and set up a cmake library for SPARTA, you can use the following command:

sudo make install

CMake Setup Copy Section

If you are using CMake for your project, you can also use SPARTA in the following ways:

• A system-wide installation of SPARTA will contain a CMake configuration file that exports the library as sparta::sparta. This enables the use of find_package to locate SPARTA. For example:

  cmake Copy

  find_package(Boost REQUIRED COMPONENTS thread) # required by SPARTA
  find_package(sparta REQUIRED)
  add_library(MyAnalysisLibrary)
  target_link_libraries(MyAnalysisLibrary PUBLIC sparta::sparta)

• If you have cloned and "built" SPARTA locally, you can use a find_package call (such as the one above) and run CMake with the -Dsparta_DIR=<your_sparta_build_dir> to load the library:

  sh Copy

  # Assuming you are at the top-level of your project that uses SPARTA
  mkdir build && cd build
  cmake .. -Dsparta_DIR=path/to/your/sparta/build

• SPARTA can be added as a subdirectory in your project:

  cmake Copy

  # assuming you have cloned SPARTA into a folder named sparta
  add_subdirectory(sparta)
  add_library(MyAnalysisLibrary)
  target_link_libraries(MyAnalysisLibrary PUBLIC sparta::sparta)

Issues Copy Chapter

Issues on GitHub are assigned priorities which reflect their urgency and how soon they are likely to be addressed.

• P0: Unbreak now! A serious issue which should have someone working on it right now.
• P1: High Priority. An important issue that someone should be actively working on.
• P2: Mid Priority. An important issue which is in the queue to be processed soon.
• P3: Low Priority. An important issue which may get dealt with at a later date.
• P4: Wishlist. An issue with merit but low priority which is up for grabs but likely to be pruned if not addressed after a reasonable period.

License Copy Chapter

SPARTA is MIT-licensed, see the LICENSE file in the root directory of this source tree.

Overview (`website/README.md`) Copy Section

Website

This website is built using Docusaurus 2, a modern static website generator.

Installation Copy Section

$ yarn

Local Development Copy Section

$ yarn start

This command starts a local development server and opens up a browser window. Most changes are reflected live without having to restart the server.

Build Copy Section

$ yarn build

This command generates static content into the build directory and can be served using any static contents hosting service.

Overview (`sparta/rust/README.md`) Copy Section

SPARTA

SPARTA is a library of software components specially designed for building high-performance static analyzers based on the theory of Abstract Interpretation.

Abstract Interpretation Copy Chapter

Abstract Interpretation is a theory of semantic approximation that provides a foundational framework for the design of static program analyzers. Static analyzers built following the methodology of Abstract Interpretation are mathematically sound, i.e., the semantic information they compute is guaranteed to hold in all possible execution contexts considered. Moreover, these analyzers are able to infer complex properties of programs, the expressiveness of which can be finely tuned in order to control the analysis time. Static analyzers based on Abstract Interpretation are routinely used to perform the formal verification of flight software in the aerospace industry, for example.

Why SPARTA? Copy Chapter

Building an industrial-grade static analysis tool based on Abstract Interpretation from scratch is a daunting task that requires the attention of experts in the field. The purpose of SPARTA is to drastically simplify the engineering of Abstract Interpretation by providing a set of software components that have a simple API, are highly performant and can be easily assembled to build a production-quality static analyzer. By encapsulating the complex implementation details of Abstract Interpretation, SPARTA lets the tool developer focus on the three fundamental axes in the design of an analysis:

• Semantics: the program properties to analyze (range of numerical variables, aliasing relation, etc.)
• Partitioning: the granularity of the properties analyzed (intraprocedural, interprocedural, context-sensitive, etc.)
• Abstraction: the representation of the program properties (sign, interval, alias graph, etc.)

SPARTA is an acronym that stands for Semantics, PARTitioning and Abstraction.

SPARTA for C++ is the analytic engine that powers most optimization passes of the ReDex Android bytecode optimizer. The ReDex codebase contains multiple examples of analyses built with SPARTA that run at industrial scale. The interprocedural constant propagation illustrates how to assemble the building blocks provided by SPARTA in order to implement a sophisticated yet scalable analysis.

SPARTA in Rust Copy Chapter

SPARTA for Rust is published as a crate on crates.io. It is still in an experimental stage and there's no guarantee that the API won't change. So far, we have reimplemented the basic functions found in the C++ version, namely:

• Abstract Domains: The SPARTA crate models abstract domains as a trait. The C++ version uses CRTP and static_asserts to ensure the user defined type satisfies the quality of an abstract domain. This is no longer necessary in Rust.
• Data Structures: We have implemented PatriciaTree based Set and Map containers along with the abstract domains.
• Graph Trait: SPARTA fixpoint iterator can operate on user defined graph types as long as they implement the sparta::graph::Graph trait. This is analogous to the Graph interface in C++, which uses CRTP for compile-time polymorphism.
• Weak Partial Ordering: A replacement for Weak Topological Ordering described in the paper. Sung Kook Kim, Arnaud J. Venet, and Aditya V. Thakur. 2019. Deterministic parallel fixpoint computation. Proc. ACM Program. Lang. 4, POPL, Article 14 (January 2020), 33 pages. https://doi.org/10.1145/3371082https://dl.acm.org/doi/10.1145/3371082
• Fixpoint Iteration: A single threaded fixpoint iterator based on Weak Partial Ordering.

Issues Copy Chapter

Issues on GitHub are assigned priorities which reflect their urgency and how soon they are likely to be addressed.

• P0: Unbreak now! A serious issue which should have someone working on it right now.
• P1: High Priority. An important issue that someone should be actively working on.
• P2: Mid Priority. An important issue which is in the queue to be processed soon.
• P3: Low Priority. An important issue which may get dealt with at a later date.
• P4: Wishlist. An issue with merit but low priority which is up for grabs but likely to be pruned if not addressed after a reasonable period.

License Copy Chapter

SPARTA is MIT-licensed, see the LICENSE file in the root directory of this source tree.

Examples Proguard (`website/docs/examples/proguard.md`) Copy Section

id: proguard title: Using ProGuard Rules with Redex Copy Chapter

The code and artifacts for this example are on GitHub.

Currently there is limited support for specifying ProGuard rules which
Redex will try to honor when it considers deleting (shrinking) classes,
methods and fields.

The Need To Control Shrinking Copy Chapter

One of the optimizations that Redex performs is to remove interfaces that
have only one implementation. However, when there is a use of that interface
through reflection or constructs like instanceof then this is an unsafe
removal which should be prohibited by using a ProGuard rule.

Example Copy Chapter

Consider the following interface:

package com.facebook.redex.examples.proguardexample;

public interface Greek {
    int doubleWombat();
}

which only has one use:

package com.facebook.redex.examples.proguardexample;

public class Alpha implements Greek {

    private int wombat;

    public Alpha () {
        wombat = 21;
    }

    public int doubleWombat() {
        return 2 * wombat;
    }
}

and is instantiated in a main activity as follows:

package com.facebook.redex.examples.proguardexample;

import android.support.v7.app.AppCompatActivity;
import android.os.Bundle;
import android.widget.TextView;

public class MainActivity extends AppCompatActivity {

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        TextView textView = (TextView) findViewById(R.id.message);

        Alpha alphaObject = new Alpha();
        int ltuae = alphaObject.doubleWombat();
        textView.setText("The answer is " + ltuae + "\n");

        try {
            Class<?> greek = Class.forName("com.facebook.redex.examples.proguardexample.Greek");
            if (greek.isInstance(alphaObject)) {
                textView.append("Alpha is an instance of Greek");
            } else {
                textView.append("Alpha is not an instance of Greek");
            }
        } catch (ClassNotFoundException e) {
            textView.append("ERROR: Greek interface not found");
        }
    }
}

When you make a release build of this application and then process it with Redex you will get a crash
because the Greek class will be removed because it only has a single implementation, but Redex
did not notice that Greek is used as part of an instanceof check (or there could have been some
use of reflection that mentioned the Greek class). Running the app gives the following output
on the display of the device:

You can instruct Redex to prevent a class or interface from begin deleted by providing
a ProGuard rule. In this
case we want to ensure the Greek interface is not deleted:

-keep interface com.facebook.redex.examples.proguardexample.Greek

When you run Redex you can specify a single ProGuard file containing simple keep rules for classes
and interfaces. For example:

$ redex -o myfasterapp.apk myapp.apk -P proguard-rules.pro --sign -s ~/.android/debug.keystore -p android

Now when you run the post-Redex APK you will notice that the Greek class has not been stripped away:

Limitations Copy Chapter

Right now we support only simple keep annotations for classes and interfaces. Shortly we will provide
support a richer subset of the ProGuard configuration language.

Source for Example Copy Chapter

The source code for this example can be found in this directory.

Examples Synth (`website/docs/examples/synth.md`) Copy Section

id: synth title: Redex Synth Pass Example Copy Chapter

The code and artifacts for this example are on GitHub.

The synth optimizations attempt to remove synthetic methods and wrappers. This
improves performance by making access to field values faster and it also
reduces code size because the definition and invocation of synthetic methods
is eliminated.

Removing synthetic methods for accessing static fields Copy Chapter

This directory contains an Android Studio 1.5 project that illustrates how a
wrapper synthetic method is removed by Redex. The contrived example is
a simple Android application which makes use of this class:

package com.facebook.redex.examples.synth;

public class Alpha {

    private static int alpha;

    public Alpha(int initialValue) {
        alpha = initialValue;
    }

    public class Beta {
        public int doubleAlpha() {
            return 2 * alpha;
        }
    }
}

The key thing to note here is that there is an inner class Beta which
has a method doubleAlpha which accesses a private static field alpha of
its outer class Alpha. A dump of the Dex bytecode for the Alpha class
confirms that the alpha field is private and static.

Static fields     -
    #0              : (in Lcom/facebook/redex/examples/synth/Alpha;)
      name          : 'alpha'
      type          : 'I'
      access        : 0x000a (PRIVATE STATIC)

Java compilers will implement accesses] to this field from inner classes
by generating a synthetic getter method to allow the inner class to access
the private field of the outer
class. We can see the automatically generated synthetic method access$000
if we examine the Dex bytecode for the Alpha class:

Class #597            -
  Class descriptor  : 'Lcom/facebook/redex/examples/synth/Alpha;'
...
    #1              : (in Lcom/facebook/redex/examples/synth/Alpha;)
      name          : 'access$000'
      type          : '()I'
      access        : 0x1008 (STATIC SYNTHETIC)
      code          -
      registers     : 1
      ins           : 0
      outs          : 0
      insns size    : 3 16-bit code units
065f14:                                        |[065f14] com.facebook.redex.examples.synth.Alpha.access$000:()I
065f24: 6000 070b                              |0000: sget v0, Lcom/facebook/redex/examples/synth/Alpha;.alpha:I // field@0b07
065f28: 0f00                                   |0002: return v0

This generated synthetic method can be accessed by the inner class
which keeps the JVM happy. All it does is to read the state field
value using an sget instruction and return the read value.
This synthetic wrapper is used in the implementation of doubleAlpha:

Virtual methods   -
    #0              : (in Lcom/facebook/redex/examples/synth/Alpha$Beta;)
      name          : 'doubleAlpha'
...
065ed8:                                        |[065ed8] com.facebook.redex.examples.synth.Alpha.Beta.doubleAlpha:()I
065ee8: 7100 1118 0000                         |0000: invoke-static {}, Lcom/facebook/redex/examples/synth/Alpha;.access$000:()I // method@1811
065eee: 0a00                                   |0003: move-result v0
065ef0: da00 0002                              |0004: mul-int/lit8 v0, v0, #int 2 // #02
065ef4: 0f00                                   |0006: return v0

For the doubleAlpha method in the inner class Beta to access the
private static field of the enclosing class an invoke-static call is made
to the synthetic wrapper method access$000.

Here is an example of the doubleAlpha method being used from a simple
Android application:

package com.facebook.redex.examples.synth;

import android.support.v7.app.AppCompatActivity;
import android.os.Bundle;
import android.widget.TextView;

public class MainActivity extends AppCompatActivity {

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        TextView textView = (TextView) findViewById(R.id.message);
        textView.setText("Redex Synth Example\n");

        Alpha a = new Alpha(12);
        Alpha.Beta b = a.new Beta();
        textView.append("Double Alpha(12) = " + b.doubleAlpha() + "\n");
    }
}

If we examined the Dex bytecode for the call to doubleAlpha we would
see something like this which shows a virtual call to the doubleAlpha method:

Direct methods    -
    #0              : (in Lcom/facebook/redex/examples/synth/MainActivity;)
      name          : 'onCreate'
...
065fb6: 6e10 0f18 0100                         |0031: invoke-virtual {v1}, Lcom/facebook/redex/examples/synth/Alpha$Beta;.doubleAlpha:()I // method@180f
...

After running Redex the synthetic wrapper method will be removed and
the code for accessing the alpha static field will be inlined into
the code for the main activity:

Direct methods    -
    #0              : (in Lcom/facebook/redex/examples/synth/MainActivity;)
      name          : 'onCreate'
...
07f246: 6005 4d06                              |0031: sget v5, Lcom/facebook/redex/examples/synth/Alpha;.alpha:I // field@064d
07f24a: da05 0502                              |0033: mul-int/lit8 v5, v5, #int 2 // #02
...

To make the access of the alpha field legal for the Dalvik VM we have
mutated the access permissions for this field:

Static fields     -
    #0              : (in Lcom/facebook/redex/examples/synth/Alpha;)
      name          : 'alpha'
      type          : 'I'
      access        : 0x0009 (PUBLIC STATIC)

Similar optimizations exist for other synthetic wrapper scenarios e.g.
for instance fields.

Example Code Copy Chapter

The project in the synth-example directory can be opened with Android Studio 1.5 and
contains the sample here that illustrates the removal of synthetic
wrappers for static private fields.

The Makefile in this directory can be used once you have build a signed:
APK (Build : Generate Signed APK...) to produce the following items:

• synth-example-release-redex.apk: A Redex optimized version of the original APK.
• classes.dump: A dump of the Dex bytecode for the input APK synth-example-release.apk.
• classes-redex.dump: A dump of the Redex optimizd APK synth-example-release-redex.apk.

The environment variable ANDROID_TOOLS should be set to the location
of your Android SDK tools directory.

To produce these items:

$ make clean all

Getting Started Config (`website/docs/getting_started/config.md`) Copy Section

id: configuring title: Configuring ReDex sidebar_position: 2 Copy Chapter

ReDex can be configured to run different optimizations or to alter the behavior
of existing optimizations. While this isn't always necessary, it's helpful to
be able to tweak settings to get the best results.

A Simple Example

A starting point for configuration is to use a configuration similar to the
default config that's baked in to the redex binary. You can find this config
in config/default.config. This is what it looks like:

{
"redex" : {
  "passes" : [
    "ReBindRefsPass",
    "BridgeSynthInlinePass",
    "FinalInlinePassV2",
    "DelSuperPass",
    "SingleImplPass",
    "MethodInlinePass",
    "StaticReloPassV2",
    "RemoveUnreachablePass",
    "RegAllocPass"
  ]
}
}

Name this file default.config, and invoke redex with it:

% redex.py -c default.config -o tmp/output.apk input.apk

This will do... exactly what redex does without the config. Not so exciting, is
it? Let's move on to something more advanced.

Selecting Optimization Passes

Changing the set of optimizations ReDex runs is easy; just add (or remove) the
pass name from the redex.passes list. For example, let's say you want to remove
the StaticReloPassV2 optimization while you're debugging something. Just use
this config:

{
"redex" : {
  "passes" : [
    "ReBindRefsPass",
    "BridgeSynthInlinePass",
    "FinalInlinePassV2",
    "DelSuperPass",
    "SingleImplPass",
    "MethodInlinePass",
    "RemoveUnreachablePass",
    "RegAllocPass"
  ]
}
}

Configuring optimization behavior

Each optimization pass has some configurable parameters that are specific to
that pass. These are often blocklists (or allowlists) indicating what code the
optimization should leave alone (for blocklists) or what code should be
optimized (for allowlists).

A simple example is ReachableNativesPass. This pass reports which methods are
reachable from native code. You can tell it where to write that report by adding
a config entry:

"redex" : {
  "passes" : [
    "ReachableNativesPass",
    "RegAllocPass"
  ]
},
"ReachableNativesPass" : {
  "output_file_name" : "redex-reachable-natives.txt"
}

Options for each pass are documented with that pass.

Global options

• redex.passes
  Type: array of strings
  A list of passes to be run in the specified order.

• coldstart_classes
  Type: string
  Path to a file containing a list of class names in the order they are used
  for cold start. Example format:

  text Copy

  com/foo/Bar.class
  com/foo/Baz.class
  ...
  com/foo/Quux.class

• proguard_map
  Type: string
  Path to a file containing ProGuard's mapping of unobfuscated
  class/field/method names to obfuscated names. This option is useful if you
  are running ReDex after ProGuard, so that ReDex will properly understand
  obfuscated names.

Complete configuration

To see the (almost) complete list of configuration parameters, run

redex-all --reflect-config

(Note that you run redex-all here, not the redex.py wrapper!)

This emits a JSON document detailing parameters, their types, defaults and
possibly documentation.

Getting Started Getting Started (`website/docs/getting_started/getting_started.md`) Copy Section

id: getting_started title: Getting Started Copy Chapter

Hi, Redex is an Android bytecode optimizer and stuff.

Getting Started Installation (`website/docs/getting_started/installation.md`) Copy Section

id: installation title: Installation sidebar_position: 1 Copy Chapter

Dependencies Copy Chapter

We use package managers to resolve third-party library dependencies.

macOS Copy Section

You will need Xcode with command line tools installed. To get the command line tools, use:

xcode-select --install

Install dependencies using homebrew:

brew install autoconf automake libtool python3
brew install boost jsoncpp

For App Bundle support brew install protobuf is also required.

Ubuntu/Debian (64-bit) Copy Section

Base requirements are automake & libtool, GCC >= 10, Python >= 3.9 and Boost >= 1.74.0, as well as
development versions of iberty, jemalloc, jsoncpp, lz4, lzma, and zlib. Protobuf >= 3.12.4 is required if optimizing an App Bundle.

Ubuntu 22.04+, Debian 11(Bullseye)+

The minimum supported Ubuntu version is 22.04. The minimum supported Debian version is 11.

A convenience script
will set up the build environment. This may include downloading googletest 1.14.0 on older OS versions.

sudo ./setup_oss_toolchain.sh

After the script, please run sudo ldconfig if it throws an error about loading shared libraries for running protoc.

Experimental: Windows (64-bit) with MSYS2 Copy Section

You need MSYS2 to build redex-all (only MingW-w64 is supported) and Python 3.7+ to run redex.py.

Install the build requirements in an MSYS or MingW64 shell:

pacman -Syuu && pacman -Sy make mingw-w64-x86_64-boost mingw-w64-x86_64-cmake mingw-w64-x86_64-gcc mingw-w64-x86_64-jsoncpp mingw-w64-x86_64-make

If you do not use Git on Windows directly, you may install and use it under MSYS2:

pacman -S git

Experimental: Windows 10 (64-bit) Copy Section

You need Visual Studio 2017. Visual Studio 2015 is also possible, but a couple of C++ compile errors need to be fixed. We use vcpkg for dependencies. Install vcpkg from their document:

cd c:\tools
git clone https://github.com/Microsoft/vcpkg.git
cd vcpkg
.\bootstrap-vcpkg.bat
.\vcpkg integrate install

Install necessary libraries with x64-windows-static:

.\vcpkg install boost --triplet x64-windows-static
.\vcpkg install zlib --triplet x64-windows-static
.\vcpkg install jsoncpp --triplet x64-windows-static
.\vcpkg install mman --triplet x64-windows-static

Download, Build and Install Copy Chapter

Get ReDex from GitHub:

git clone https://github.com/facebook/redex.git
cd redex

Now, build ReDex using autoconf and make.

autoreconf -ivf && ./configure && make
sudo make install

Alternatively, to enable protobuf to support App Bundles, please use:

autoreconf -ivf && ./configure --enable-protobuf
make
sudo make install

For protobuf installed in the default system (Homebrew) search path (e.g /usr/local/bin), --enable-protobuf is sufficient to trigger the build. Otherwise, specify the protobuf installation path for the autoconf:

autoreconf -ivf
./configure --with-protoc=/path/to/protoc --with-protolib=/path/to/protobuf_libs --with-protoheader=/path/to/protobuf_headers --enable-protobuf
make
sudo make install

If your build machine has lots of RAM (on the order of 2-4GB per core), using
Make parallelism can speed up the build (e.g., make -j4). However, the C++
compilers are very memory hungry and this needs to be finely tuned on many
systems.

Experimental: Windows (64-bit) with MSYS2 Copy Section

The MSYS2 build relies on CMake. In a MingW64 shell:

# Assumes you want to use Git under MSYS. Else skip to below.
git clone https://github.com/facebook/redex.git
cd redex
# Assumes you are in the redex directory
mkdir build-cmake
cd build-cmake
cmake -G "MSYS Makefiles" ..
make

If your build machine has lots of RAM (on the order of 2-4GB per core), using
Make parallelism can speed up the build (e.g., make -j4). However, the C++
compilers are very memory hungry and this needs to be finely tuned on many
systems.

You may check whether the produced binary seems in a working condition:

# In the MingW64 shell:
./redex-all.exe --show-passes
# Or in a standard Windows command prompt in the same directory
redex-all.exe --show-passes

The output should show a large number of included passes, at the time of writing 81.

Bundling the redex-all binary with the python scripts is not supported on Windows. Manually copy the binary into the same directory as redex.py and use redex.py that way, or ensure that redex.py is called with the --redex-binary parameter:

python redex.py --redex-binary PATH_TO_BINARY [...]

Experimental: CMake for Mac, Linux, and Windows Copy Section

Alternatively, build using CMake. Note that the current CMakeLists.txt only implements a rule for redex-all binary. We will support installation and testing soon.

Generate build files. By default, it uses Makefile:

# Assume you are in redex directory
mkdir build-cmake
cd build-cmake
# .. is the root source directory of Redex
cmake ..

If you prefer the ninja build system:

cmake .. -G Ninja

On Windows, first, get CMAKE_TOOLCHAIN_FILE from the output of "vcpkg integrate install", and then:

cmake .. -G "Visual Studio 15 2017 Win64"
 -DVCPKG_TARGET_TRIPLET=x64-windows-static
 -DCMAKE_TOOLCHAIN_FILE="C:/tools/vcpkg/scripts/buildsystems/vcpkg.cmake"

Build redex-all:

cmake --build .

On Windows, you may build from Visual Studio. Redex.sln has been generated.

You should see a redex-all executable, and the executable should show about 45 passes.

./redex-all --show-passes

Test Copy Chapter

Optionally, you can run our unit test suite. We use gtest, which is automatically
downloaded when testing (or by invoking a setup script directly).

Note: Testing is currently not supported for CMake-based builds.

Dependencies Copy Section

Some ReDex tests require a Java environment and Android compiler tooling. If a JDK and the
Android SDK are available on the machine, ensure that javac and dx are available on
the PATH. Otherwise, install those dependencies.

For Ubuntu/Debian, this may for example be done with

sudo apt-get install -y --no-install-recommends dalvik-exchange openjdk-8-jdk-headless
sudo ln -s /usr/bin/dalvik-exchange /usr/local/bin/dx

Execute Copy Section

Run tests with

make -j check

If your build machine has lots of RAM (on the order of 2-4GB per core), using
Make parallelism can speed up the build and testing (e.g., make -j4). However,
the C++ compilers are very memory hungry and this needs to be finely tuned on
many systems.

Getting Started Passes (`website/docs/getting_started/passes.md`) Copy Section

id: passes title: Passes sidebar_position: 3 Copy Chapter

ReDex has a large set of optimization passes that is constantly evolving.
Information in this document may be outdated, inspect the code if necessary.

AccessMarkingPass Copy Chapter

Final objects and private methods can be optimized more aggressively than
virtual objects and public methods.

Devirtualization can result in [NullPointerException]. Two Redex passes perform
devirtualization of methods: AccessMarkingPass devirtualizes methods not using
this. MethodInlinePass inlines monomorphic virtual
calls.

The app's config file can override AccessMarkingPass behavior.
finalize_methods, finalize_unwritten_fields, finalize_classes, and
privatize_methods default to true. finalize_written_fields defaults to false.

"AccessMarkingPass": {
  "finalize_written_fields": true
},

Pass ordering dependencies:

• AccessMarkingPass should be run early as it enables other optimizations.

See related:

• MethodDevirtualizationPass

AnnoKillPass Copy Chapter

AnnoKillPass originally removed only annotations with no static references in
the code--"build-visible" annotations. It was expanded to remove annotations
referenced statically, but not used at runtime--"runtime-visible" annotations.

AnnoKillPass reads configuration options from the app's config
file specifying annotations to be kept or killed. An additional
option specifies whether Redex should attempt to match signatures for removal.

"AnnoKillPass" : {
  "keep_annos": [
    "Landroid/view/ViewDebug$CapturedViewProperty;",
    "Landroid/view/ViewDebug$ExportedProperty;"
  ],
  "kill_bad_signatures" : true,
  "kill_annos" : [
    "Lcom/google/inject/BindingAnnotation;"
  ]
},

See related:

• SystemAnnoKillPass

BridgeSynthInlinePass Copy Chapter

As the name suggests BridgeSynthInlinePass removes bridge and synthetic methods by inlining them.

Bridge methods are created by the javac compiler as part of
type erasure for covariant generics.

Example of a bridge method in pseudo-bytecode:

check-cast*   (for checking covariant arg types)
invoke-{direct,virtual,static}  bridged-method
move-result
return

BridgeSynthInlinePass inlines the target of the bridging, the "bridgee", into the bridge
method by replacing the invoke- and adjusting check-casts as needed. The
bridgee can then be deleted.

BridgeSynthInlinePass also removes synthetic methods introduced by javac. javac generates
these methods because while Java allows inner classes or nested classes, DEX
bytecode does not. Inner classes, like class Delta in this example, are
promoted to top-level classes in the DEX bytecode.

public class Gamma {
    public Gamma(int v) {
        x = v;
    }
    private int x;

    public class Delta {
        public int doublex() {
            return 2*x;
        }
    }
}

javac generates a synthetic method that allows access to fields, methods, and
constructors in the promoted class. SynthPass effectively removes these synthetic methods,
replacing them with a direct access to the field or call to the method or
constructor.

The general limitations and the cost model of the inliner applies. As a result, some bridge and synthetic method inlining opportunities might not be acted upon, e.g. when it would result in API-level violations, or an overall size increase.

CheckBreadcrumbsPass Copy Chapter

CheckBreadcrumbsPass validates Redex codegen against leftover references to
deleted types, methods, or fields.

• Verifies that there are no references to a deleted class definition remaining
  in DEX files (essentially an internal class that is not in scope).
• Verifies that the target of a field and method reference exists on the class
  it is defined on.

Redex will warn if it finds dangling references or illegal references to
entities.

ClassMergingPass Copy Chapter

ClassMergingPass shrinks the size of code generated by some frameworks. These
tools produce large amounts of Java code for each component. The code generated
for different component types often shares the same structure, differing only by
the type of the component.

Class Merging identifies pieces of generated code that have the same "shape".
Erasing the types that differ allows the pieces of generated code to be merged.

ConstantPropagationPass Copy Chapter

ConstantPropagationPass substitutes the values of constants into expressions
at compile time. Constant propagation can eliminate multiple expressions,
resulting in a constant load.

ConstantPropagationPass does a whole program analysis to replace instructions
with single destination registers with constant loads. The analysis is run
iteratively until a fixed point or configurable limit is reached.

CostantPropagationPass should be run before dead code elimination (DCE) passes
as it can create dead code.

See related:

• LocalDCEPass
• RemoveUnreachablePass

CopyPropagationPass Copy Chapter

CopyPropagationPass removes writes of duplicated values to registers in a
basic block. If value A and value B are aliases, then any moves between
these registers are unnecessary and can be eliminated. Duplicated source
registers can also be deduplicated.

CopyPropagationPass can also remove duplicated instructions if the source and
the destination are aliased.

Example: v0 and v1 contain the same value and can be treated locally as
aliases:

const v0, 0
const v1, 0
invoke-static v0 foo
invoke-static v1 bar

can be transformed into

const v0, 0
invoke-static v0 foo
invoke-static v0 bar

CopyPropagationPass should be run before dead code elimination (DCE) passes as
it can create dead code.

See related:

• LocalDCEPass
• RemoveUnreachablePass

DedupBlocksPass Copy Chapter

Dedup blocks inside of a method. Duplicated blocks are those with the same code
and the same successor. Duplicated blocks can have different predecessors.

DedupBlocksPass identifies one of the blocks as the canonical version, then
redirects all predecessors to the canonical block. The pass currenly only
identifies blocks with a single successor, but in the future may identify blocks
with multiple sucessors.

Stack traces for deduplicated blocks will always report the same line number,
but the predecessor line numbers will be correct.

DedupBlocksPass should be run after InterDexPass.

DelSuperPass Copy Chapter

DelSuperPass eliminates subclass methods that invoke the superclass method and
trivially return.

DelSuperPass only optimizes virtual methods with the following
characteristics:

• The subclass method must match the name and signature of the superclass method
• The subclass method must only invoke the superclass method and either return
  void or the result of the callee.

DelSuperPass also fixes up references to the removed subclass methods, making
them refer to the superclass method instead. Though Dalvik's
invoke-virtual
would automatically resolve to the correct superclass method, doing this reduces
the number of method references in the Dex file and saves on space.

FinalInlinePassV2 Copy Chapter

FinalInlinePassV2, or an instance field's value after ``, and inlines
the value in dex code. Note that this pass is separate from the MethodInline
and SwitchInline passes.

The DX tool often introduces verbose bytecode sequences to initialize static
fields in classes it generates. The encoded_value equivalents are much more
compact. This pass determines the values of static fields after <clinit> and
eliminates the redundant writes to the static field.

This pass applies to both final and non-final static fields. For final statics
it also inlines reads of the static field where possible, replacing them with
constant operations outside of <clinit>.

For instance fields, the pass calculates the field's value after `` is
finished. It inlines reads of the instance field where possible.

Unlike a static field, if an instance field were changed outside of , it might have different values for different instances of the class. For classes with multiple the instance field values might differ based on the
constructor. This pass does not inline instance fields that are:

1. Modified outside of their class ``.
2. In a class that have more than one constructor.
3. Accessed by reflection or native code anywhere in code.
4. Accessed in another method that is called inside of the constructor.

Note that this pass does not inline the CharSequence type for static or
instance fields because older Dalvik VMs cannot handle this class.

See related:

• MethodInlinePass

LocalDcePass Copy Chapter

LocalDcePass removes dead instructions in a method. Code is considered to be
"dead' if it has no side-effects and does not change its output registers. Code
in a catch block is considered live for the duration of the try, as any
instruction in the try block is assumed to be able to throw. Methods annotated
with @DoNotOptimize are not considered for dead code elimination.

Dead code elimination (DCE) differs from RemoveUnreachable (RMU) in two ways:
first, RMU works from global roots (at the scope of Class/Method/Field) whereas
DCE works at the function scope. Second, DCE removes code that does not change
state, for example, a store to a memory address that is not read in the scope of
the block, whereas RMU removes code that is unreachable regardless of its effect
on state.

See related:

• RemoveUnreachablePass

MethodDevirtualizationPass Copy Chapter

MethodDevirtualizationPass converts virtual methods with single
implementation to static dmethods.

The app's config file:

"MethodDevirtualizationPass" : {
  "staticize_vmethods_not_using_this" : true,
  "staticize_dmethods_not_using_this" : true
},

See related:

• AccessMarkingPass

ObfuscatePass Copy Chapter

ObfuscatePass pass obfuscates method and field names. RenameClassesPassV2
obfuscates class names.

See related:

• RenameClassesPassV2

OptimizeEnumsPass Copy Chapter

OptimizeEnumsPass does two things to make use of Enum classes more
efficient. It optimizes the use of Enum values in switch tables and replaces
some uses of Enum values with Integer singletons.

The javac compiler creates Dalvik packed
switch tables
that contain a generated anonymous class. OptimizeEnumPass replaces these
packed switch statements with lookups based on the Enum ordinal itself. Note
that this optimization does not work with ProGuard obfuscation enabled. ProGuard
can rewrite Enum value names such that they no longer match the Enum class
name.

OptimizeEnumsPass also replaces some uses of an Enum with a boxed Integer
singleton and keeps the runtime behavior unchanged at the same time.

The pass does not guarantee to erase all the enums, perf sensitive code should
never use enums. An Enum is not optimizable if it is:

1. An abstract Enum.
2. Reflectively used.
3. Contains an instance field that is not a primitive.
4. Contains non-final instance fields.
5. Cast to any other types, like java.lang.Object, java.lang.Enum,
   java.io.Serializable, java.lang.Comparable

OriginalNamePass Copy Chapter

"OriginalNamePass" : {
  "hierarchy_roots" : [
    "Ljava/lang/Runnable;"
  ]
},

Redex renames classes for performance reasons. Renaming can result in different
class names in debug and release builds, which results in mismatches in logging.
Also, some system functions should not be renamed.

An alternative is to use OriginalClassName.getSimpleName() for logging.
OriginalNamePass is preferred as is does not significantly increase the APK
size.

PeepholePass Copy Chapter

Replace small code patterns with a more efficient pattern. The optimization
matches known patterns for replacement. It essentially performs a string search
of the code for known inefficient sequences and replaces them with more
efficient code. PeepholePass will not replace patterns that span a basic block
boundary. PeepholePass can remove no-op function calls such as redundant moves
and appends of null strings.

Peephole pass should be run early.

ReBindRefsPass Copy Chapter

Rebind references to their most abstract type.

The number of methods in a DEX file is limited to 64K. Method definitions (defs)
and references (refs) both count against this limit. The class scope in an
inheritance situation can create needless method refs. Calls based on the
subclassed methods create unnecessary method refs for the subclass. This is
especially true when calls are made through the implicit this.

For example, you have a class specialized on <n> with a method that calls
Object.equals(Object). All of these calls create a ref X<n>.equals(Object),
each of them counting against the 64K limit. Rebinding them lower in the
hierarchy reduces the number of unique refs.

class X<n>
{
    public void foo<n>(Object o)
    {
        ...
        if (equals(o) {...}
        ...
    }
}

ReBindRefsPass rebinds all invoke-virtual to the base def of the virtual
scope. For invoke-interface, it rebinds to the first interface method def. The
optimization is only done as long as there is no change in method visibility: we
walk down the hiearchy as long as the method is public. ReBindRefsPass
drastically reduces the number of methods defined in DEX files.

ReduceGotosPass Copy Chapter

Reduces gotos in two ways:

1. When a conditional branch would fallthrough to a block that has multiple
   sources, and the branch target only one has one, invert condition and swap
   branch and goto target. This reduces the need for additional gotos and
   maximizes the fallthrough efficiency.
2. It replaces gotos that eventually simply return by return instructions.
   Return instructions tend to have a smaller encoding than goto instructions,
   and tend to compress better due to less entropy (no offset).

Example, inverting this conditional will eliminate a goto:

(const v2 0)

(if-eqz v0 :true)
(:back_jump_target)

(return v2)

(:true)
(const v2 1)
(goto :back_jump_target)

RegAllocPass Copy Chapter

RegAllocPass does register allocation: the process of allocating variables
into the available physical registers. The goal of register allocation is to
avoid "spilling", that is, moving values from registers into memory.

RegAllocPass uses a standard graph-coloring register allocator algorithm,
known as the Chaitin-Briggs algorithm.

RemoveBuildersPass Copy Chapter

Remove builder invocations. A trivial builder is one that:

• Doesn't escape the stack (this is never passed to a method not in this
  instance, stored in a field, or returned)
• Has no static methods
• Has no static fields

Unreferenced builders are left to be removed by RemoveUnreachablePass (RMU).

See related:

• RemoveUnreachablePass
• ResultPropagationPass

RemoveUnreachablePass Copy Chapter

Starting from the roots, recursively mark the other elements that the roots
reference. Afterwards, it deletes all the unmarked elements.

The pass has various powerful options, including:

• remove_no_argument_constructors: Whether to remove argless constructors. They might be used to create instances via reflection, so the default is false.
• relaxed_keep_class_members: Only consider instance members as roots when their classes are either instantiable, i.e. have a callable constructor, or are "dynamically referenced". A class is "dynamically referenced" if it is mentioned in a Dalvik annotation signature, is referenced in a runtime-visibile annotation, appears in a string or a const-class instruction, is the declaring type of a native method, is present in a native library (lib//.so), has one of the configured "reflected_package_names". The default is false for backwards compatibility.
• throw_propagation: When reachable instructions invoke methods that cannot return (e.g. all possible target methods have no reachable return statement), then subsequent instructions will not be visited, and replaced with a unreachable instruction. The default is false for backwards compatibility.
• prune_uninstantiable_insns: When reachable instructions access instance members of classes that can never be instantiated, then subsequent instructions will not be visited, and replaced with an instruction that throws a NullPointerException. The default is false for backwards compatibility.
• prune_uncallable_instance_method_bodies: When an instance method can never be target of an invocation, even though we might need to keep the method for virtual scope order, or because of keep rules, then we can replace its body with an unreachable instruction. This draws from the same instantiability knowledge that is used for the prune_uninstantiable_insns option. The default is false for backwards compatibility.
• prune_uncallable_virtual_methods: In some cases, we don't need to keep the body of an uncallable method, but instead can make the method abstract, or remove it completely. The default is false for backwards compatibility.
• prune_unreferenced_interfaces: Removes interfaces that are not referenced anywhere in code except in implements clauses. The default is false for backwards compatibility.

More information about RemoveUnreachablePass is available in this [note on
Teaching Reachability Analysis about Dependency Injection].

See related:

• LocalDcePass

RemoveUnusedFieldsPass Copy Chapter

It's pretty much in the name. A lot of these unread fields are actually
javac-generated fields for inner classes. Notably, this turns non-static inner
classes into static ones where possible.

This pass occasionally causes issues because the app may have been relying on an
unread field to stop the GC from deleting an object.

RemoveUnusedArgsPass Copy Chapter

Removes unused parameters. Currently only works on non-virtual methods and
virtual methods that are not part of some overriding inheritance hierarchy.

RenameClassesPassV2 Copy Chapter

RenameClassesPassV2 renames classes to shorter names such as "X.A1c", saving
in APK size, obfuscating the code, and ordering classes to optimize performance
of loading.

RenameClassesPassV2 will not rename any class mentioned in resources, nor will
it rename anything in blocklist either by direct class name or as part of
an excluded package.

RenameClassesPassV2 relies on the app's config file, excluding
of the class or hierarchy, or use of reflection.

Logview and bug reports are configured to automatically undo this renaming.

See related:

• ObfuscatePass

ReorderInterfacesDeclPass Copy Chapter

ReorderInterfacesDeclPass list for each class by how frquently the Interfaces
are called. The Interface list is searched linearly when an Interface is called,
so calling an Interface at the list will be faster. An alphabetical sort is used
for tie-breaks in number of incoming calls to preserve consistency across
Classes.

This pass could be improved by checking the number of incoming calls
dynamically.

ResultPropagationPass Copy Chapter

Refactor code, e.g.,

Text.create(context)
    .clipToBounds(false)
    .text(myText)

to be as efficient as the less elegant equivalent version:

Text.Builder b = Text.create(context);
b.clipToBounds(false)
b.text(myText)

See related:

• RemoveBuildersPass

MethodInlinePass Copy Chapter

For example, in this code, if run is inlined to main and the access of bar
throws, the stack trace in main will show a NullPointerException at the
dereference of this instead of a call to run.

class Foo {
  private String bar;
  public void run() {
    System.out.println(bar);
  }
}

class Main {
  public static void main(String[] args) {
    Foo foo = null;
    foo.run();
  }
}

MethodInlinePass will not inline a constructor as the Android verifier checks
for a call to `` before any access to the object.

MethodInlinePass cannot currently be run after InterDexPass.

See related:

• FinalInlinePassV2

SingleImplPass Copy Chapter

Removes interfaces with only a single implementation. Any classes referring to
the interface will now refer to the implementation instead. This can cause minor
confusion in stack traces.

StaticReloPassV2 Copy Chapter

StaticReloPassV2 relocates static fields and methods that only have one
calling class to that class. It improves the performance and reduces the app
size.

See related:

• StaticReloPass

StringBuilderOutlinerPass Copy Chapter

This pass looks for recurring sequences of StringBuilder calls, and then outlines them. Outlining is essentially the opposite of inlining, it finds repeated sequences of code and extracts them into an external function, often referred to as the outlined function.

This outliner pass specifically deals with StringBuilders because they are very commonly instantiated in Java code.

A common pattern for StringBuilders are

StringBuilder builder = new StringBuilder();
builder.append(...);
builder.append(...);
builder.append(...);
builder.toString();

Since a StringBuilder's methods are independent of any state in the user's code, they can be safely moved and outlined to create contigious sequences of repetitive code. The outlining only happens if certain conditions are met, including if there are enough use sites and if there aren't too many parameters (as this might increase register pressure in the caller).

StringConcatenatorPass Copy Chapter

Reduce string operations as well as reducing the number of strings that need
to be loaded.

Here's an example <clinit> method StringConcatenationPass will optimize:

public static final String PREFIX = "foo";
public static final String CONCATENATED = PREFIX + "bar";

The output code should be equivalent to:

public static final PREFIX = "foo";
public static final CONCATENATED = "foobar";

This is a targeted optimization that is only performed on static initializers
with many string concatenations.

StripDebugInfoPass Copy Chapter

StripDebugInfoPass removes debug information for instructions that will
never throw. As debug positions can correspond to multiple instructions, we need
to check that none of the instructions will throw. Also, Redex won't strip the
first piece of debug information in a function to preserve the accuracty of
sampling profiles and ANR stack traces.

The app's config file can direct StripDebugInfoPass removals at a
more granular level:

"StripDebugInfoPass" : {
  "drop_all_dbg_info" : "0",
  "drop_local_variables" : "1",
  "drop_line_numbers" : "0",
  "drop_src_files" : "0",
  "use_allowlist" : "0",
  "cls_allowlist" : [],
  "method_allowlist" : [],
  "drop_prologue_end" : "1",
  "drop_epilogue_begin" : "1",
  "drop_all_dbg_info_if_empty" : "1",
  "drop_synth_aggressive" : "0",
  "drop_line_numbers_preceeding_safe" : "1"
},

Pass ordering dependencies:

• StripDebugInfoPass should be run early as removal of the debug info should
  make other passes faster.
• Inlining complicates the flow graph for debug info. StripDebugInfoPass
  should be run before any inlining passes, and will not optiimize if inlining
  has been performed.

Getting Started Usage (`website/docs/getting_started/usage.md`) Copy Section

id: usage title: Usage sidebar_position: 4 Copy Chapter

Basic Usage Copy Chapter

To use ReDex, first build your app and find the APK for it. Then the simplest
invocation is:

[python3] redex.py path/to/your.apk -o path/to/output.apk

A more complete invocation with:

• the default configuration setting config/default.config
• access to the Android SDK tools

[python3] redex.py -c default.config \
  --android-sdk-path path/to/android/sdk \
  path/to/your.apk -o path/to/output.apk

The full set of options can be found with

[python3] redex.py --help

and at the time of writing is

/* Detailed source-code truncated for AI context efficiency. */

From here, you may want to read the configuration guide and details
about the passes.

The result output.apk should be smaller and faster than the
input.

Tracing Copy Chapter

If you want some statistics about each pass, you can turn on tracing:

export TRACE=1

More specifically, tracing has categories
and levels. For a TRACE(X, Y, msg) statement in the code to be logged,
category X must have level Y or higher:

export TRACE=X:1,Y:2,Z:3

The output of tracing can also be redirected to a file with the TRACEFILE
variable:

export TRACEFILE=/path/to/trace.txt

Help Faq (`website/docs/help/faq.md`) Copy Section

id: faq title: FAQ Copy Chapter

I'm getting "Couldn't find zipalign. See README.md to resolve this." or other zipalign errors Copy Chapter

zipalign is an optimization step that is bundled with the Android SDK. You
need to tell redex where to find it. For example, if you installed the SDK at
/path/to/android/sdk, try:

ANDROID_SDK=/path/to/android/sdk redex [... arguments ...]

You can alternatively add zipalign to your PATH, for example:

PATH=/path/to/android/sdk/build-tools/xx.y.zz:$PATH redex [... arguments ...]

Additionally, it's possible zipalign itself failed. In this case the stderr will be reported to aid debugging (for example, zipalign fails if you provide an output path that already exists).

My app fails to install with `Failure [INSTALL_PARSE_FAILED_NO_CERTIFICATES]` Copy Chapter

After you run redex, you will need to re-sign your app. You can re-sign manually
using these instructions:
https://developer.android.com/tools/publishing/app-signing.html#signing-manually.

You can also tell redex to sign for you. If you want to sign with the debug
key, you can simply do:

redex --sign [ ... arguments ...]

If you want to sign with your release key, you'll need to provide the
appropriate args:

--sign Sign the apk after optimizing it
-s [KEYSTORE], --keystore [KEYSTORE]
-a [KEYALIAS], --keyalias [KEYALIAS]
-p [KEYPASS], --keypass [KEYPASS]

My App crashes with `MethodNotFoundException`, `ClassNotFoundException`, `NoSuchFieldException`, or something similar. How do I fix this? Copy Chapter

Redex probably deleted or renamed it. Redex is quite aggressive about deleting
things it deems are unreachable. But, often Redex doesn't know about reflection
or other complex ways an entity could be reached.

Here's how you ensure Redex will not delete or rename something:

Annotate any class, method, or field you want to keep with @DoNotStrip, and
add this to your ProGuard config file (which should be passed to redex.py via
the --proguard-config flag):

# Do not strip classes annotated with @DoNotStrip
-keep @com.path.to.your.DoNotStrip class *
# Do not strip annotated fields or methods either
-keepclassmembers class * {
    @com.path.to.your.DoNotStrip *;
}

and define DoNotStrip:

package com.path.to.your;
public @interface DoNotStrip {}

How does this compare to ProGuard? Copy Chapter

ReDex is conceptually similar to ProGuard, in that both optimize bytecode.
ReDex, however, optimizes .dex bytecode, while ProGuard optimizes .class
bytecode before it is lowered to .dex. Operating on .dex is sometimes an
advantage: you can consider the number of virtual registers used by a method
that is an inlining candidate, and you can control the layout of classes within
a dex file. But ProGuard has some capabilities that ReDex does not (for
example, ReDex will not remove unused method parameters, which ProGuard does).

How about DexGuard? Copy Chapter

DexGuard operates on dex, but we haven't evaluated it at all since it's closed
source. We don't use it at Facebook and we have no plans to start.

Technical Details Docker (`website/docs/technical_details/docker.md`) Copy Section

id: docker title: Docker Container Deployments Copy Chapter

Dockerfiles Copy Chapter

Usage:

Build with:

docker build . -f container/Dockerfile -t redex

Assuming input files are placed in ~/project. Run redex with:

docker run -it --rm -v ~/project:/input redex -P /input/proguard-rules.pro --config /input/default.config /input/input.apk -o /input/out.apk

Build and run tests with

docker build . --target test -f container/Dockerfile -t redex-test && docker run redex-test

Run

docker run -it --rm --entrypoint bash redex

Technical Details Interdex (`website/docs/technical_details/interdex.md`) Copy Section

id: interdex title: Interdex Copy Chapter

The Interdex Pass addresses the problem that the ordering of classes inside dexes
and their distribution between dexes (if the application has multiple dexes) does
not correspond to the order that they are accessed with during runtime if we use
the normal compilation tool chain.

Consider a dex with classes c0, c1, c2, ... . Let's assume the order in which
they are accessed during runtime is c0, c1, c2, ... .
The actual layout in the bytecode section might be
c1000, c291, c3, c705, ..., c0,..., c1 and so on

And of course the same issue happens if there are multiple dexes, with the
additional problem that classes might be in different dexes too.

Reordering classes to be in the same order as they are accessed at
runtime has several benefits:

• Less IO: IO happens in bigger chunks, the exact size of each chunk read being a function of OS and file system settings. Chunk sizes of 4KB are the minimum, with 128KB being possible with read ahead settings for common file systems. Many classes will be smaller than these chunk sizes and there will be IO for data that is not immediately used
• Less memory usage: Memory is allocated at page granularity (4KB). If most of the 4KB page is unused data, the memory usage of the process will be increased. On many Android systems memory is a critical resource and the OS is constantly struggling to free up memory. Inefficient usage of memory can make a whole system behave worse and increased memory usage by an application makes it more likely that the OS will have to kill it at some point to free up enough memory for other applications to run.
• Less page cache pollution: As mentioned under IO, IO is often performed in coarse chunks. Data read in is buffered in the page cache, which is a limited resource in many typical Android devices. Bringing in data that is not immediately used causes pollution of the page cache and potential eviction of useful data from the application or other applications.

Generating input data

The flow for generating profiling data is:

• Establish a typical use case for your app. This might be a automated test or just the developer starting up the application and performing common interactions with it
• getting a heap dump from the running app
• parsing the heap dump with the provided script. This generates a text file with the classes loaded by the app and VM in the order that they were loaded

Note that it is very important that you have compiled the app that you're going
to be generating the heap dump with the same settings that you use in your
release build, otherwise the class ordering will not reflect reality.

It is equally important that the usage reflects a real-world scenario. Using
an overly simplistic test or startup scenario will not generate representative
data and will not lead to performance improvements.

Note that if you use proguard for obfuscation or another program to the same
end, you'll have to disable those steps for profiling. Obfuscation is not
guaranteed to be stable and makes mapping class names from one generated apk
to the next difficult.

Step-by-Step on how to generate class list Copy Chapter

Connect your device to your computer, so you can execute adb commands. You need to have root on your device to execute the dump heap command.
On your computer:

// get the process if of your app

adb shell ps | grep YOUR_APP_NAME | awk '{print $2}' > YOUR_PID ( if you don't have awk, the second value is the pid of your app)

// dump the heap of your app. You WILL NEED ROOT for this step

adb root
adb shell am dumpheap YOUR_PID /data/local/tmp/SOMEDUMP.hprof

// copy the heap to your host computer

adb pull /data/local/tmp/SOMEDUMP.hprof YOUR_DIR_HERE/.

// pass the heap dump to the python script for parsing and printing out the class list
// Note that the script needs python 3

redex/tools/hprof/dump_classes_from_hprof.py --hprof YOUR_DIR_HERE/SOMEDUMP.hprof > list_of_classes.txt

If everything worked out, list_of_classes.txt will contain a large number of lines of the form foobar.class
You'll note that many of the classes list are actually classes provided by the system and not from your app.
This is ok, since the Interdex Pass will ignore any entries for which it cannot find the corresponding classes to in the apk.

Note: you must have enum34 installed for the script to work.

Usage

To enable the Interdex pass for you application, add the following to your config file:

• add "InterDexPass" to passes
• add "coldstart_classes": "list_of_classes.txt" to the config file

Options Copy Chapter

There are two flags that can be set to influence the behavior of the Interdex pass

• emit_canaries: This flag controls whether each secondary dex has
  a non-functional canary class added. Defaults to false.
  Enable this only if you explicitly know that you need it.

• static_prune: This flag controls whether Interdex attempts to remove classes
  that have no references to them from the rest of the set of classes in the pgo list.

Measuring benefit

• Install an apk without interdex pass enabled
  text Copy

  adb shell ps | grep YOUR_APP_NAME | awk '{ print $2 }' > YOUR_PID
  adb shell dumpsys meminfo YOUR_PID

• Note how much memory your app uses and the .dex mmap row
• Do all the steps described above and rerun redex with the Interdex Pass enabled and using the profiling data you generated
• Install the apk with interdex enabled
• start the app and repeat the step to get the meminfo
• Note total memory usage and .dex mmap in particular
• Hopefully memory usage has gone down!

If you want performance measurements, you'll have to set up a test for app startup and run it on apks with and without the interdex pass applied.

--- METRICS ---

• Files Extracted: 14
• Estimated Token Budget: ~16801 tokens
• Canonical Reference: https://codewiki.google/github.com/facebook/redex