passenger Documentation

File: README.md Copy Chapter

Phusion Passenger®

Supercharge your Ruby, Node.js and Python apps

Phusion Passenger® is a web server and application server, designed to be fast, robust and lightweight. It takes a lot of complexity out of deploying web apps, adds powerful enterprise-grade features that are useful in production, and makes administration much easier and less complex. Phusion Passenger supports Ruby, Python, Node.js and Meteor, and is being used by high-profile companies such as Apple, Pixar, New York Times, AirBnB, Juniper etc as well as over 650.000 websites.

Phusion Passenger - the smart app server

What makes Passenger so fast and reliable is its C++ core, its zero-copy architecture, its watchdog system and its hybrid evented, multi-threaded and multi-process design.

Learn more: Copy Section

• Website
• Fuse Panel
• Documentation & Support
• Consultancy
• Twitter
• Blog

Installation Copy Chapter

Please follow the installation instructions on the website.

Installing the source directly from git Copy Section

If you mean to install the latest version of Passenger directly from this git repository, then you should run one of the following commands. Installing from the git repository is basically the same as the tarball installation method, as described in the manual, with one exception: you need to clone git submodules:

git submodule update --init --recursive

After that, run one of the following:

./bin/passenger-install-apache2-module

-OR-

./bin/passenger-install-nginx-module

-OR-

# From your application directory
~/path-to-passenger/bin/passenger start

For troubleshooting, configuration and tips, please also refer to the above documentation. For further support, please refer to the Phusion Passenger support page.

Ruby users can also build a gem from the Git repository and install the gem.

gem build passenger.gemspec
gem install passenger-x.x.x.gem

Further reading Copy Chapter

• The doc/ directory.
• Contributors Guide
• Phusion Passenger support page
• Phusion Passenger release notes

Legal Copy Chapter

"Passenger" and "Phusion Passenger" are registered trademarks of Asynchronous B.V.

File: dev/ci/README.md Copy Chapter

Continuous integration test suite

This directory contains scripts that invoke the Passenger test suite. These scripts are invoked from the Passenger continuous integration environment, based on Jenkins.

The following diagrams explain how the different files fit together.

Flow on Linux Copy Chapter

Invoke: dev/ci/setup-host
   |     |
   |     +-- Load: dev/ci/scripts/setup-host
   |           |
   |           +-- Relax file permissions (if in Jenkins)
   |           |
   |           +-- Create cache directories
   |           |
   |           +-- Create buildout directory
   |
Invoke: dev/ci/run-tests-with-docker <test name>
   |
   +-- Exec: Docker container
       Entrypoint: dev/ci/scripts/docker-entrypoint.sh
         |
         +-- Exec: dev/ci/scripts/debug-console-wrapper.sh dev/ci/scripts/docker-entrypoint-stage2.sh
               |
               +-- Invoke: dev/ci/scripts/docker-entrypoint-stage2.sh
               |     |
               |     +-- Load: dev/ci/lib/setup-container.sh <test name>
               |     |     |
               |     |     +-- Create test/config.json
               |     |     |
               |     |     +-- Relax home permission
               |     |     |
               |     |     +-- Remove previous build products
               |     |     |
               |     |     +-- Load: dev/ci/lib/set-container-envvars.sh
               |     |     |     |
               |     |     |     +-- Set RVM version and various envvars
               |     |     |
               |     |     +-- Load: dev/ci/tests/<test name>/setup
               |     |
               |     +-- Load: dev/ci/tests/<test name>/run
               |
               +-- (if docker-entrypoint-stage2.sh exited with an error)
               |    Populate buildout/testlogs
               |
               +-- (if docker-entrypoint-stage2.sh exited with an error,
               |    and DEBUG_CONSOLE is set to 0)
               |   Print error message and exit
               |
               +-- (if docker-entrypoint-stage2.sh exited with an error,
                    and DEBUG_CONSOLE is set to 1)
                      |
                      +-- Load: dev/ci/lib/set-container-envvars.sh
                      |     |
                      |     +-- Set RVM version and various envvars
                      |
                      +-- Invoke: bash

Flow on macOS Copy Chapter

Invoke: dev/ci/setup-host <test name>
   |      |
   |      +-- Relax file permissions (if in Jenkins)
   |      |
   |      +-- Create cache directories
   |      |
   |      +-- Create buildout directory
   |      |
   |      +-- Exec: dev/ci/scripts/debug-console-wrapper.sh dev/ci/scripts/setup-host-natively.sh <test name>
   |           |
   |           +-- Invoke: dev/ci/scripts/setup-host-natively.sh
   |           |     |
   |           |     +-- Load: dev/ci/lib/setup-container.sh
   |           |           |
   |           |           +-- Create test/config.json
   |           |           |
   |           |           +-- Relax home permission
   |           |           |
   |           |           +-- Remove previous build products
   |           |           |
   |           |           +-- Load: dev/ci/lib/set-container-envvars.sh
   |           |           |     |
   |           |           |     +-- Set RVM version and various envvars
   |           |           |
   |           |           +-- Load: dev/ci/tests/<test name>/setup
   |           |
   |           +-- (if setup-host-natively.sh exited with an error)
   |           |    Populate buildout/testlogs
   |           |
   |           +-- (if setup-host-natively.sh exited with an error,
   |           |    and DEBUG_CONSOLE is set to 0)
   |           |   Print error message and exit
   |           |
   |           +-- (if setup-host-natively.sh exited with an error,
   |                and DEBUG_CONSOLE is set to 1)
   |                 |
   |                 +-- Load: dev/ci/lib/set-container-envvars.sh
   |                 |     |
   |                 |     +-- Set RVM version and various envvars
   |                 |
   |                 +-- Invoke: bash
   |
Invoke: dev/ci/run-tests-natively <test name>
   |
   +-- Exec: dev/ci/scripts/debug-console-wrapper.sh dev/ci/scripts/run-tests-natively-stage2.sh <test name>
         |
         +-- Invoke: dev/ci/scripts/run-tests-natively-stage2.sh
         |     |
         |     +-- Load: dev/lib/set-container-envvars.sh
         |     |     |
         |     |     +-- Set RVM version and various envvars
         |     |
         |     +-- Load: dev/ci/tests/<test name>/run
         |
         +-- (if run-tests-natively-stage2.sh exited with an error)
         |    Populate buildout/testlogs
         |
         +-- (if run-tests-natively-stage2.sh exited with an error,
         |    and DEBUG_CONSOLE is set to 0)
         |   Print error message and exit
         |
         +-- (if run-tests-natively-stage2.sh exited with an error,
              and DEBUG_CONSOLE is set to 1)
               |
               +-- Load: dev/ci/lib/set-container-envvars.sh
               |     |
               |     +-- Set RVM version and various envvars
               |
               +-- Invoke: bash

File: doc/CodingTipsAndPitfalls.md Copy Chapter

Coding tips and common coding pitfalls

Prefer shared_ptrs Copy Chapter

You should prefer shared_ptrs over raw pointers because they make memory leaks and memory errors less likely. There are only very limited cases in which raw pointers are justified, e.g. optimizations in very hot code paths.

Unexpected shared_ptr invalidation Copy Section

Suppose you have a function which takes a reference to a shared_ptr, like this:

void foo(const shared_ptr<Client> &client) {
    performNetworking();
    log(client->getAddress());
}

You might think that client is never going to be destroyed during the scope of the function, right? Wrong. If client came from a non-local variable, and your function calls another function (perhaps indirectly) which resets that variable, then client is going to be invalidated. Consider a contrived example like this:

static shared_ptr<Client> client;

void performNetworking() {
    if (write(...) == -1) {
        // An error occurred, disconnect client.
        client.reset();
    }
}

More realistically, this kind of bug is likely to occur if foo() calls some function which invokes a user-defined callback, which unreferences the original object under some conditions. Thus, if your code might call an arbitrary user-defined function, you should increase the reference count locally:

void foo(const shared_ptr<Client> &client) {
    shared_ptr<Client> extraReference = client;
    performNetworking();
    log(client->getAddress());
}

A variant of this pitfall is documented under "Event loop callbacks".

Event loop callbacks Copy Chapter

Obtain extra shared_ptr reference on `this` Copy Section

If your event loop callback ever calls user-defined functions, either explicitly or implicitly, you should obtain a shared_ptr to your this object. This is because the user-defined function could call something that would free your object. The problem is documented in detail in "Unexpected shared_ptr invalidation".

Your class should derive from boost::enable_shared_from_this to make it easy for you to obtain a shared_ptr to yourself.

void callback(ev::io &io, int revents) {
        shared_ptr<Foo> extraReference = shared_from_this();
        ...
    }

Exceptions Copy Section

Event loop callbacks should catch expected exceptions. Letting an exception pass will crash the program. When system call failure simulation is turned on, the code can throw arbitrary SystemExceptions, so beware of those.

Thread interruption and RAII destructors Copy Chapter

When using thread interruption, make sure that RAII destructors are non-interruptable. If your code is interrupted and then a thread_interrupted is thrown, make sure that RAII destructors don't check for the interruption flag and then throw thread_interrupted again. This not only fails to clean things up properly, but also confuses the exception system, resulting in strange errors such as "terminate called without an active exception".

File: doc/DebuggingAndStressTesting.md Copy Chapter

Debugging and Stress Testing Phusion Passenger

This guide tells you:

• How to debug Phusion Passenger in case of any serious problems, e.g. crashes and mysterious connection problems.
• How to stress test Phusion Passenger.

Process output Copy Chapter

All PassengerAgent processes as well as all spawned application processes have their stdout and stderr redirected to the global web server error log (that is, not the per-virtual host error log). This is usually '/var/log/apache2/error.log' or '/var/log/nginx/error.log'.

Note that in case of Nginx, Phusion Passenger prints to the error log specified in the server context, not the "http" context. If the server context does not contain an error_log directive then the default error log location will be used. The default location depends on how Nginx is configured during compilation, but it is usually either '$PREFIX/logs/error.log' or '/var/log/nginx/error.log'. For example, if your Nginx configuration looks like this:

worker_processes 2;

http {
    error_log /home/nginx/error.log;
    ...
}

then Phusion Passenger will print to the default error log location, not '/home/nginx/error.log'!

Crash behavior Copy Chapter

Whenever a Phusion Passenger agent process crashes because of a signal (SIGABRT, SIGBUS, SIGSEGV and similar signals), its default behavior is to attempt to write a crash report to its stderr. This crash report contains:

• A simple libc-level backtrace of the current thread. This backtrace may or may not correspond to the thread that caused the crash.
• A detailed backtrace report, covering all threads. This report even contains the values of variables on the stack. The report is obtained through the crash-watch tool so you must have it installed. Crash-watch in turn requires gdb, which must also be installed.
• Agent-specific diagnostics information. For example the Passenger core will report the status of its process pool and its connected clients.

You can change the crash behavior with the following environment variables:

• PASSENGER_ABORT_HANDLER (default: true) - Whether agent processes should install their crash handlers. When disabled, crashes will be handled by the default signal handler, meaning that they will likely just crash without dumping any crash report.
• PASSENGER_DUMP_WITH_CRASH_WATCH (default: true) - Whether crash-watch should be used to obtain detailed backtraces.
• PASSENGER_BEEP_ON_ABORT (default: false) - Whether agent processes should beep when they crash. This is useful during development, e.g. when you're stress testing the system and want to be notified when a crash occurs. On OS X, it will execute osascript -e "beep 2" to trigger the beep. On Linux it will execute the beep command.
• PASSENGER_STOP_ON_ABORT (default: false) - When enabled, causes agent processes to stop themselves on crash, by raising SIGSTOP. This gives you the opportunity to attach gdb on them.

Behavior logging Copy Chapter

Increase PassengerLogLevel to print more debugging messages.

Debugging with AddressSanitizer Copy Chapter

AddressSanitizer is an excellent tool created by Google to detect memory problems in C and C++ programs. It is for example used for detecting memory errors in Google Chrome. Unlike Valgrind, which is an x86 emulator and makes everything 100 times slower, AddressSanitizer's performance penalty is only about 10%.

Recompile Phusion Passenger with the environment variable USE_ASAN=1 to enable support for AddressSanitizer.

Simulating system call failures Copy Chapter

Error conditions are sometimes hard to test. Things like network errors are usually hard to simulate using real equipment. In order to facilitate with error testing, we've developed a system call failure simulation framework, inspired by sqlite's failure test suite. You specify which system call errors should be simulated, and with what probability they should occur. By running normal tests multiple times you can see how Phusion Passenger behaves under these simulated error conditions.

To enable, set the environment variable PASSENGER_SIMULATE_SYSCALL_FAILURES. The format is:

program_name1=error1:probability1,error2:probability2,...;program_name2=...

program_nameN specifies the name of the Phusion Passenger process for which system call failure simulation should be enabled. This is followed by a list of system call errno names and the respective probabilities (between 0 and 1). For example:

export PASSENGER_SIMULATE_SYSCALL_FAILURES='PassengerAgent watchdog=ENOSPC:0.01;PassengerAgent core=EMFILE:0.001,ECONNREFUSED:0.02'

This will enable system call failure simulation only for the watchdog and the core, but not for the UstRouter. All system calls in the watchdog will have a 1% probability of throwing ENOSPC. All system calls in the HTTP server will have a 0.1% probability of throwing EMFILE, and a 2% probability of throwing ECONNREFUSED.

File: doc/DeveloperQuickstart.md Copy Chapter

Phusion Passenger Developer QuickStart

Watch the Developer QuickStart screencast

Phusion Passenger provides an easy and convenient development environment that contributors can use. The development environment is a 64-bit Ubuntu 14.04 virtual machine and contains everything that you need. The build toolchain is already setup, all dependencies are installed and web servers are preconfigured. You can start coding almost immediately. And because it's a virtual machine, it doesn't matter which host operating system you're using.

You use this development environment through Vagrant and VirtualBox. Vagrant is an extremely useful tool for managing virtual machines, while VirtualBox is virtualization software. Both tools are free and open source. Vagrant allows you to easily share a directory between the host OS and the VM. This means that you can use the editor on your host OS to edit source files, and compile inside the VM.

Getting started Copy Chapter

1. Install Vagrant.
2. Install VirtualBox.
3. Inside the Phusion Passenger source tree, run: vagrant up. This will spin up the VM and will set it up. This can take a while so feel free to grab a cup of coffee.
4. Once the VM has been setup, login to it by running: vagrant ssh.

Workflow Copy Chapter

The workflow is to:

• Edit code on the host.
• Use git commands on the host.
• Compile in the VM.
• Run tests in the VM.

The Phusion Passenger source code is located in /vagrant.

When you're done developing Phusion Passenger, you can shut down the VM by running vagrant halt. Next time you want to spin it up again, run vagrant up.

Starting and accessing Apache Copy Chapter

Apache is installed, but it's not set up with Phusion Passenger by default. So you must first compile the Phusion Passenger Apache module:

cd /vagrant
rake apache2

Next, enable the Phusion Passenger module and restart Apache:

sudo a2enmod passenger
sudo service apache2 restart

You can now access Apache from the host on http://127.0.0.1:8000 through :8005. The VM has a sample Ruby Rack application configured on http://127.0.0.1:8001/. Visit that URL and see it in action. Its source code is located in dev/rack.test in the Phusion Passenger source tree.

Starting and accessing Nginx Copy Chapter

The Nginx source code and binaries are located in /home/vagrant/nginx. If this is the first time you use Nginx in this VM, you must install it. Run:

cd /home/vagrant/nginx
rake bootstrap

Next, start Nginx by using a script that the development environment provides:

./start

You can now access Nginx from the host on http://127.0.0.1:8100 through :8105. The VM has a sample Ruby Rack application configured on http://127.0.0.1:8101/. Visit that URL and see it in action. Its source code is located in dev/rack.test in the Phusion Passenger source tree.

Running tests Copy Chapter

Tests can be run immediately without any setup.

rake test:cxx
rake test:integration:apache2
rake test:integration:nginx

Further reading Copy Chapter

• Contributors Guide
• Design and Architecture
• Coding Tips and Pitfalls

File: doc/Packaging.md Copy Chapter

Introduction

This document describes how packagers can package Phusion Passenger binaries
for their operating system.

Phusion Passenger can be configured in 2 ways, the "originally packaged"
configuration where everything is in the same directory, and the
"natively packaged" configuration where files are scattered across the
filesystem, e.g. in a FHS-compliant configuration. This document describes
how you can configure Phusion Passenger to locate its own files when they're
scattered across the filesystem.

Phusion Passenger files are also called assets in this document.

Originally packaged Copy Chapter

This is the configuration you get when you checkout Phusion Passenger from git,
when you install Phusion Passenger from a gem or when you extract it from a
tarball. All the original files are stored in a single directory tree, which we
call the source root.

The git repository, gems and tarballs do not come with any binaries; they have
to be compiled by the user. Phusion Passenger looks for binaries in, and (if
the user initiates the compilation process) stores binaries in, the following
directories:

• Normally, binaries are to be located in the buildout subdirectory under
  the source root.

• Phusion Passenger Standalone does things a little differently. Binaries are
  to be located in one of the following directories, whichever it finds first:

  • ~/.passenger/standalone/<VERSION>/<TYPE-AND-ARCH> (a)
  • /var/lib/passenger-standalone/<VERSION-AND-ARCH> (b)

  If neither directories exist, then Passenger Standalone compiles the
  binaries and stores them in (b) (when running as root) or in (a). It still
  looks for everything else (like the .rb files) in the source root.

Natively packaged Copy Chapter

Phusion Passenger is packaged, usually (but not necessarily) through a DEB or RPM
package. This configuration comes not only with all necessary binaries, but also
with some (but not all) source files. This is because when you run Phusion Passenger
with a different Ruby interpreter than the packager intended, Phusion Passenger
must be able to compile a new Ruby extension for that Ruby interpreter. This
configuration does not however allow compiling against a different Apache version
than the packager intended (but does allow compiling against a different Nginx
version).

In this configuration, files can be scattered anywhere throughout the filesystem. This
way Phusion Passenger can be packaged in an FHS-compliant way. The exact locations
of the different types of files can be specified through a
location configuration file. The existence and usage of a location configuration
file does not automatically imply that Phusion Passenger is natively packaged.

If Phusion Passenger needs to have a new Ruby extension compiled, then it will
store that in ~/.passenger/native_support/<VERSION>/<ARCH>.

The location configuration file

The Phusion Passenger administration tools, such as passenger-status, look for a
location configuration file in the following places, in the given order:

• The environment variable $PASSENGER_LOCATION_CONFIGURATION_FILE.
• <RUBYLIBDIR>/phusion_passenger/locations.ini, where is the Ruby library
  directory that contains phusion_passenger.rb. For example,
  /usr/lib/ruby/1.9.0/phusion_passenger/locations.ini.
• ~/.passenger/locations.ini
• /etc/phusion-passenger/locations.ini

If it cannot find a location configuration file, then it assumes that Phusion
Passenger is originally packaged. If a location configuration file is found then
the configuration is determined by the natively_packaged option in the
location configuration file, which can be either "true" or "false".

The Apache module and the Nginx module expect PassengerRoot/passenger_root to
refer to either a directory or a file. If the value refers to a directory, then it
assumes that Phusion Passenger is originally packaged, where the source root is the
specified directory. If the value refers to a file, then it will use it as the
location configuration file, and the configuration depends on the
natively_packaged setting.

The location configuration file is an ini file that looks as follows:

[locations]
natively_packaged=true
bin_dir=/usr/bin
support_binaries_dir=/usr/lib/phusion-passenger/support-binaries
lib_dir=/usr/lib/phusion-passenger
helper_scripts_dir=/usr/share/phusion-passenger/helper-scripts
resources_dir=/usr/share/phusion-passenger
include_dir=/usr/share/phusion-passenger/include
doc_dir=/usr/share/doc/phusion-passenger
ruby_libdir=/usr/lib/ruby/vendor_ruby
apache2_module_path=/usr/lib/apache2/modules/mod_passenger.so
ruby_extension_source_dir=/usr/share/phusion-passenger/ruby_extension_source
nginx_module_source_dir=/usr/share/phusion-passenger/ngx_http_passenger_module

All keys except fo natively_packaged specify the locations of assets and asset
directories. The "Asset types" section provides a description of all asset types.

Thus, if you're packaging Phusion Passenger, then we recommend the following:

• Put a locations.ini file in <RUBYLIBDIR>/phusion_passenger/locations.ini and
  set PassengerRoot/passenger_root to that filename. We don't recommend using
  ~/.passenger or /etc/phusion-passenger because if the user wants to install
  a different Phusion Passenger version alongside the one that you've packaged,
  then that other version will incorrectly locate your packaged files instead of
  its own files.
• Always set natively_packaged to "true". The "false" value is used
  internally for implementing Phusion Passenger Standalone and should never be
  used by packagers.

The Phusion Passenger Ruby libraries

phusion_passenger.rb Copy Chapter

The Phusion Passenger administration tools are written in Ruby. So the first thing
they do is trying to load phusion_passenger.rb, which is the source file
responsible for figuring out where all the other Phusion Passenger files are. It
tries to look for phusion_passenger.rb in <OWN_DIRECTORY>/../src/ruby_supportlib where
<OWN_DIRECTORY> is the directory that the tool is located in. If
phusion_passenger.rb is not there, then it tries to load it from the normal Ruby
load path.

Ruby extension Copy Chapter

The Phusion Passenger loader scripts try to load the Phusion Passenger Ruby
extension (passenger_native_support.so) from the following places, in the given order:

• If Phusion Passenger is originally packaged, it will look for the Ruby
  extension in /libout/ruby/<ARCH>. Otherwise, this step is skipped.
• The Ruby library load path.
• ~/.passenger/native_support/<VERSION>/<ARCH>

If it cannot find the Ruby extension in any of the above places, then it will
attempt to compile the Ruby extension and store it in
~/.passenger/native_support/<VERSION>/<ARCH>.

Conclusion for packagers Copy Chapter

If you're packaging Phusion Passenger then you should put both phusion_passenger.rb
and passenger_native_support.so somewhere in the Ruby load path, or make sure that
that directory is included in the $RUBYLIB environment variable. You cannot specify
a custom directory though the location configuration file.

Asset types

Throughout the Phusion Passenger codebase, we refer to all kinds of assets. Here's
a list of all possible assets and asset directories.

• source_root

  When Phusion Passenger is originally packaged, this refers to the directory
  that contains the entire Phusion passenger source tree. Not available when
  natively packaged.

• bin_dir

  A directory containing administration binaries and scripts and like
  passenger-status; tools that the user may directly invoke on the command line.

  Value when originally packaged: /bin

• support_binaries_dir

  A directory that contains (platform-dependent) binaries that Phusion Passenger
  uses, but that should not be directly invoked from the command line. Things like
  PassengerAgent are located here.

  Value when originally packaged:

  • Normally: /buildout/support-binaries
  • Passenger Standalone: ~/.passenger/standalone/<VERSION>/support-<ARCH>

• helper_scripts_dir

  A directory that contains non-binary scripts that Phusion Passenger uses, but
  that should not be directly invoked from the command line. Things like
  rack-loader.rb are located here.

  Value when originally packaged: /helper-scripts

• resources_dir

  A directory that contains non-executable, platform-independent resource files
  that the user should not directly access, like error page templates and
  configuration file templates.

  Value when originally packaged: /resources.

• doc_dir

  A directory that contains documentation.

  Value when originally packaged: /doc.

• include_dir

  A directory that contains the Phusion Passenger header files that are
  necessary for compiling Nginx.

  Value when originally packaged: /src

• lib_dir

  A directory that contains the Phusion Passenger library files, e.g.
  libboost_oxt.a and various .o files.

  Value when originally packaged: /buildout

• ruby_libdir

  A directory that contains the Phusion Passenger Ruby library files. Note that
  the Phusion Passenger administration tools still locate phusion_passenger.rb
  as described in the section "The Phusion Passenger Ruby libraries",
  irrespective of the value of this key in the location configuration file.
  The value is only useful to non-Ruby Phusion Passenger code.

  Value when originally packaged: /src/ruby_supportlib.

• apache2_module_path

  The filename of the Apache 2 module, or the filename that the Apache 2 module
  will be stored after it's compiled. Used by passenger-install-module to
  print an example configuration snippet.

  Value when originally packaged: /buildout/apache2/mod_passenger.so.

• ruby_extension_source_dir

  The directory that contains the source code for the Phusion Passenger Ruby
  extension. Phusion Passenger uses these sources to build a Ruby extension,
  when it detects that the user is using a new Ruby interpreter for which
  no Ruby extension has been compiled.

  Value when originally packaged: /src/ruby_native_extension.

• nginx_module_source_dir

  The directory that contains the source code for the Phusion Passenger Nginx
  module. passenger-install-nginx-module uses these sources to build Nginx
  with Phusion Passenger support.

  Value when originally packaged: /src/nginx_module.

Optional fields:

• build_system_dir

  The directory that contains the Phusion Passenger main Rakefile, used for
  compiling the Apache module and the agent executable. This field is only
  present if Phusion Passenger is compilable. Native packages usually do not
  have this field because they ship a precompiled Apache module and a
  precompiled agent executable.

  Value when originally packaged: ``

• download_cache_dir

  The directory that contains cached downloaded agent executables. Its main
  use case is to speed up agent executable downloading when Phusion Passenger
  is installed from a Ruby gem. When the user installs the gem, a script is
  invoked which downloads agent executables from the Phusion Passenger
  websites. The downloaded files are stored in this directory. Then later,
  when the user runs passenger start or any other command which requires
  the agent executable, the executable will be copied from this cache
  directory instead of downloaded.

  Native packages ship precompiled executables, so they can omit this field.

  Value when originally packaged: /download_cache

Vendoring of libraries

Phusion Passenger vendors libev and libuv in order to make installation easier
for users on operating systems without proper package management, like OS X.
If you want Phusion Passenger to compile against the system-provided
libev and/or libuv instead, then set the following environment variables
before compiling:

• export USE_VENDORED_LIBEV=no
• export USE_VENDORED_LIBUV=no

Note that we require at least libev 4.11 and libuv 1.4.2.

Generating gem and tarball

Use the following commands to generate a gem and tarball, in which Phusion
Passenger is originally packaged and without any binaries:

rake package:gem
rake package:tarball

The files will be stored in pkg/.

Fakeroot

You can generate a fakeroot with the command rake fakeroot. This will
generate an FHS-compliant directory tree in pkg/fakeroot, which you can
directly package or with minor modifications. The fakeroot even contains
a location configuration file.

File: resources/templates/error_renderer/with_details/README.md Copy Chapter

Editing

Modifications to the with_details error renderer will become active after you do a compilation with webpack:

passenger/resources/templates/error_renderer/with_details> ../../../../node_modules/webpack/bin/webpack.js

N.B. webpack should be installed:

passenger> npm i

File: src/nginx_module/README.md Copy Chapter

This directory contains the Passenger Nginx module. It is used in Passenger's Nginx integration mode. Most of the logic is not here, but in the agent. The Nginx module's responsibilities are:

• To provide and to handle Nginx configuration options.
• To start the Passenger agent (by starting the Passenger Watchdog).
• To proxy requests through the Passenger Core.

File: src/helper-scripts/README.md Copy Chapter

This directory contains scripts that Passenger executes during runtime. These scripts are not supposed to be directly executed by the user. All the loaders - applications which are responsible for loading an application written in a certain language and hooking it up to Passenger - are in this directory.

File: src/cxx_supportlib/ConfigKit/README.md Copy Chapter

About ConfigKit

ConfigKit is a configuration management system that lets you define configuration keys and store configuration values, and plays well with JSON.

Table of contents:

• Motivations
  • Configuration flow from high-level to low-level with a minimum of repeated code
  • Unifying configuration management
• Status inside the Passenger codebase
• Features and class overview
  • ConfigKit::Schema
  • ConfigKit::Store
  • Translators
• Using the schema
  • Defining the schema
    • Types
    • Flags
  • Defining default values
  • Defining custom validators
  • Nested schemas
  • Inspecting the schema
• Using the store
  • Putting data in the store
    • Updating data
  • Unregistered keys are ignored
  • Deleting data
  • Fetching data
  • Default values
  • Inspecting all data
  • Inspection filters
• Normalizing data
  • Normalization example 1
  • Normalization example 2
  • Normalizers and validation
• ConfigKit in practice & design patterns

Motivations Copy Chapter

Configuration flow from high-level to low-level with a minimum of repeated code Copy Section

Passenger is architected as a tree of components. At the top of the tree there are high-level components that implement Passenger business logic, while at the bottom of the tree there are low-level, generic components that are reusable outside Passenger. Higher-level components can encapsulate lower-level components, but not the other way around.

Almost every component can be configured individually. This is fairly easy, but when you put all the components together in Passenger's tree-like architecture, you end up with the following challenge: how do higher-level components expose the configuration of their encapsulated subcomponents, and how do higher-level components allow configuring their encapsulated subcomponents, without repeating a lot of code? Suppose that you have a class A that encapsulates B:

class B {
public:
    int option1;
    int option2;
    int option3;
};

class A {
private:
    B b;
    int option4;

public:
    void setOption1(int val) { b.option1 = val; }
    void setOption2(int val) { b.option2 = val; }
    void setOption3(int val) { b.option3 = val; }
    void setOption4(int val) { b.option4 = val; }
};

If B has a lot of configuration options, then writing all these setters (and we haven't even looked at getters yet) becomes tiresome quickly. Worse, if there is a component X that encapsulates A, that all the setters in A have to be repeated in X too. And every time you add an option to B, you have to modify A and X too.

How do you prevent having to manually write so much code for exposing and configuring all the options exposed by lower-level components? One possible answer consists of the following aspects:

• Configuration should be stored in some kind of key-value data structure, e.g. a Json::Value, instead of in individual class members for each option.
• There must exist the concept of a configuration schema, and this must be introspectable during runtime, so that higher-level components can manage subcomponents' configuration through automation.

ConfigKit implements these aspects, and also provides various other useful features in the area of configuration management. ConfigKit is built around the concept of introspection during runtime. This is not the only approach: static introspection and code generation is another valid approach (which is used by some components in Passenger), but this is outside ConfigKit's scope.

Unifying configuration management Copy Section

Another challenge pertains having different ways to configure a component. Should components be configured using getters and setters for each option? Or should they have a single method that accepts a struct (or some kind of key-value map) that specifies multiple options? Should components be configurable at all after construction? It would be great if we have a unified answer for all components in Passenger.

ConfigKit provides this unified answer:

• Components should be configurable at any time after construction. Motivation: we want to allow configuration reloads without having to restart Passenger.

• Components should be configurable through a single method that accepts a JSON object. Motivations:

  • Applying a configuration change is potentially expensive, so you want to batch multiple configuration changes into a single transaction.

  • Configuration parameters often originate from an I/O channel. Using a struct to store all the different configuration options results in a lot of repeated code, like this:

    text Copy

    config.foo1 = userInput["foo1"];
    config.foo2 = userInput["foo2"];
    config.foo3 = userInput["foo3"];

    So we want some kind of key-value data structure. JSON is a pretty popular format nowadays, and is likely to be the format used in the I/O channel, so may as well optimize for JSON.

  • JSON's various data types allow us to easily describe complex configuration settings.

  • JSON is a widely-accepted format.

ConfigKit backs these answers with code that helps you implement these principles, as well as with documented design patterns that provide guidance.

Status inside the Passenger codebase Copy Chapter

At the time of writing (25 Feb 2017), ConfigKit was just introduced, so these practices and patterns documented in this README aren't yet used everywhere throughout the Passenger codebase, but the long-term plan is to adopt these practices/patterns throughout the entire codebase.

Features and class overview Copy Chapter

ConfigKit::Schema Copy Section

Everything starts with ConfigKit::Schema. This is a class that lets you define a schema of supported configuration keys, their types and other properties like default values. Default values may either be static or dynamically calculated. The type information defined in a ConfigKit::Schema allows data validation against the schema.

ConfigKit::Store Copy Section

ConfigKit::Store is a class that stores configuration values in such a way that it respects a schema. The values supplied to and stored in ConfigKit::Store are JSON values (i.e. of the Json::Value type), although Store uses the schema to validate that you are actually putting the right JSON types in the Store.

ConfigKit::Store also keeps track of which values are explicitly supplied and which ones are not.

Translators Copy Section

And finally there are "translator" classes: ConfigKit::TableTranslator and ConfigKit::PrefixTranslator. The role of translators are described in IN_PRACTICE.md, section "The special problem of conflicting overlapping configuration names and translation".

Using the schema Copy Chapter

Defining the schema Copy Section

Start using ConfigKit by defining a schema. There are two ways to do this. The first one is to simply create ConfigKit::Schema object and adding definitions to it with schema.add(name, type, flags, [default value]):

ConfigKit::Schema schema;

// A required string key.
schema.add("foo", ConfigKit::STRING_TYPE, ConfigKit::REQUIRED);

// An optional integer key without default value.
schema.add("bar", ConfigKit::FLOAT_TYPE, ConfigKit::OPTIONAL);

// An optional integer key, with default value 123.
schema.add("baz", ConfigKit::INTEGER_TYPE, ConfigKit::OPTIONAL, 123);

// An optional string key, without default value, marked as containing a secret.
schema.add("password", ConfigKit::STRING_TYPE, ConfigKit::OPTIONAL | ConfigKit::SECRET);

// Call this when done, otherwise the object cannot be used yet.
schema.finalize();

The second one, and the one we recommend, is to subclass ConfigKit::Schema and to add the definitions inside the subclass's constructor. It will become apparent in IN_PRACTICE.md, section "Synchronous examples" why this is the recommended approach.

struct YourSchema: public ConfigKit::Schema {
    YourSchema() {
        using namespace ConfigKit;
        add("foo", STRING_TYPE, REQUIRED);
        add("bar", FLOAT_TYPE, OPTIONAL);
        add("baz", INT_TYPE, OPTIONAL, 123);
        add("password", STRING_TYPE, OPTIONAL | SECRET);
        finalize();
    }
};

// Instantiate it:
YourSchema schema;

Types

The following types are available:

• STRING_TYPE -- a string.
• INT_TYPE -- a signed integer.
• UINT_TYPE -- an unsigned integer.
• FLOAT_TYPE -- a floating point number.
• BOOL_TYPE -- a boolean.
• ARRAY_TYPE -- usually a generic array (which may contain any values). But with a special add() invocation this can indicate an array of nested ConfigKit schemas.
• STRING_ARRAY_TYPE -- an array of strings.
• OBJECT_TYPE -- usually a generic JSON object (which may contain any values). But with a special add() invocation this can indicate a map of nested ConfigKit schemas.
• ANY_TYPE -- any JSON value.

Flags

• REQUIRED -- this field is required. Mutually exclusive with OPTIONAL.
• OPTIONAL -- this field is optional. Mutually exclusive with REQUIRED.
• CACHE_DEFAULT_VALUE -- use in combination with dynamic default values. When this flag is set, the value returned by the dynamic value function is cached so that the function won't be called over and over again.
• READ_ONLY -- this field can only be set once. Only the first ConfigKit::Store::update() call actually updates the value; subsequent calls won't. Learn more about update() in Using the store -- Putting data in the store.
• SECRET -- this field contains a secret. ConfigKit::Store::previewUpdate() and ConfigKit::Store::inspect() will filter out the values of such fields. Learn more this in Using the store -- Inspecting all data.

Defining default values Copy Section

The fourth parameter of add() is for specifying a static default value.

But ConfigKit also supports dynamic default values, by accepting instead of a static default value, a function object that returns a default value.

The main use case for dynamic default values is this: to return something based on another value in the configuration store. For example suppose that your schema defines a connect_timeout and a recv_timeout option. You want recv_timeout to default to two times whatever the effective value of connect_timeout is. You can define a default value getter function like this:

Json::Value getRecvTimeoutDefaultValue(const ConfigKit::Store &store) {
    return store->get("connect_timeout").getInt() * 2;
}

schema.addWithDynamicDefault("recv_timeout", INT_TYPE, OPTIONAL, getRecvTimeoutDefaultValue);

Defining custom validators Copy Section

ConfigKit::Store validates your data using the type definitions in ConfigKit::Schema. But sometimes you need custom validations of the data in a ConfigKit::Store. For example, "'bar' is required when 'foo' is specified". This sort of logic can be implemented with custom validators.

Custom validators are defined on a ConfigKit::Schema:

static void myValidator(const ConfigKit::Store &config, vector<ConfigKit::Error> &errors) {
    if (!config["foo"].isNull() && config["bar"].isNull()) {
        errors.push_back(ConfigKit::Error("'{{bar}}' is required when '{{foo}}' is specified"));
    }
}


ConfigKit::Schema schema;

schema.add("foo", STRING_TYPE, OPTIONAL);
schema.add("bar", STRING_TYPE, OPTIONAL);
schema.addValidator(myValidator);
schema.finalize();

Miscellaneous notes about custom validators:

• They are always run, even if the normal type validation fails. For example if the caller tries to set the "foo" key to an array value (which is incompatible with the string type), then myValidator will still be called.
• All registered validators are called. A validator cannot prevent other validators from running.

Nested schemas Copy Section

Warning: do not confuse nested schemas with subschemas. The latter is a mechanism for composing multiple schemas together, not a way to define the type of a field.

It is possible to define a config field that is an array of objects, or a string map of objects, that should conform to a certain schema. We call such a schema a nested schema. For example:

ConfigKit::Schema personSchema;
personSchema.add("name", STRING_TYPE, REQUIRED);
personSchema.add("age", INT_TYPE, OPTIONAL);
personSchema.finalize();

ConfigKit::Schema mainSchema;
mainSchema.add("people", ARRAY_TYPE, personSchema, REQUIRED);
mainSchema.add("frobnicate", BOOL_TYPE, OPTIONAL, false);
mainSchema.finalize();

A field with a nested schema is defined by calling add(<field name>, ARRAY_TYPE or OBJECT_TYPE, <nested schema object>, <flags>).

The above example mainSchema says the following format is valid:

{
  "people": [
    {
      "name": "John",
      "age": 12 // optional
    },
    {
      "name": "Jane"
    },
    // ...
  }
  ],
  "frobinate": boolean-value
}

Nested schema fields simply accept corresponding JSON values, like this:

Json::Value initialValue;
initialValue["people"][0]["name"] = "John";
initialValue["people"][0]["age"] = 12;
initialValue["people"][1]["name"] = "Jane";
ConfigKit::Store store(mainSchema, initialValue);

Note that it is not possible to define a nested schema field with a default value. This is because the default values are taken from the nested schema. For example, take this code which inserts an empty object into the "people" array:

ConfigKit::Schema personSchema;
personSchema.add("name", STRING_TYPE, OPTIONAL, "anonymous");
personSchema.finalize();

ConfigKit::Schema mainSchema;
mainSchema.add("people", ARRAY_TYPE, personSchema, REQUIRED);
mainSchema.finalize();

Json::Value initialValue;
initialValue["people"][0] = Json::objectValue;
ConfigKit::Store store(mainSchema, initialValue);

Because personSchema defines a default value of "anonymous" on its "name" field, the above code would result in the following effective values:

{
  "people": [
    { "name": "anonymous" }
  ]
}

Inspecting the schema Copy Section

You can inspect the schema using the inspect() method. It returns a Json::Value in the following format:

{
  "foo": {
    "type": "string",
    "required": true,
    "has_default_value": "static",
    "default_value": "hello"
  },
  "bar": {
    "type": "float"
  },
  "baz": {
    "type": "integer"
  },
  "password": {
    "type": "string",
    "secret": true
  },
  "people": {
    "type": "array",
    "nested_schema": {
      "name": {
        "type": "string"
      },
      "age": {
        "type": "integer"
      }
    }
  }
}

Description of the members:

• type: the schema definition's type. Could be one of "string", "integer", "unsigned integer", "float", "boolean", "array", "array of strings", "object" or "any".
• required: whether this key is required.
• has_default_value: "static" if a static a default value is defined, "dynamic" if a dynamic default value is defined.
• default_value: the static default value. This field is absent when there is no default value, or if the default value is dynamic.
• nested_schema: if this field has a nested schema, then a description of that nested schema is stored here.

Using the store Copy Chapter

Once you have defined a schema you can start using ConfigKit::Store. Create it by referencing the schema that it should use. Note that it internally stores a pointer to the schema, so make sure the schema outlives the store.

ConfigKit::Store store(schema);

Putting data in the store Copy Section

You can populate the store using the update() method. The method also performs validation against the schema. The update only succeeds if validation passes.

vector<ConfigKit::Error> errors;
Json::Value updates1;

// Validation fails: 'foo' is missing
store.update(updates1, errors);
// => return value: false
//    errors: 1 item ("'foo' is required")

store.get("foo").isNull();
// => true, because the update failed

errors.clear();
updates1["foo"] = "strval";
store.update(updates1, errors);
// => return value: true
//    errors: empty

store.get("foo").asString();
// => "strval", because the update succeeded

Updating data

Any further calls to update() only update the keys that you actually pass to the method, not the keys that you don't pass:

// Assuming we are using the store from 'Putting data in the store'.
Json::Value updates2;

updates2["bar"] = 123.45;
store.update(updates2, errors);  // => true
store.get("foo").asString();     // => still "strval"
store.get("bar").asDouble();     // => 123.45

Unregistered keys are ignored Copy Section

update() ignores keys that aren't registered in the schema:

// Assuming we are using the store that went through
// 'Putting data in the store' and 'Updating data'.
Json::Value updates3;

updates3["unknown"] = true;
store.update(updates3, errors); // => true
store.get("foo").asString();    // => still "strval"
store.get("bar").asDouble();    // => still 123.45
store.get("unknown").isNull();  // => true

Deleting data Copy Section

You can delete data by calling update() with null values on the keys you want to delete.

// Assuming we are using the store that went through
// 'Putting data in the store' and 'Updating data'.
Json::Value deletionSpec;

deletionSpec["bar"] = Json::nullValue;
store.update(deletionSpec, errors);
// => return value: true
//    errors: empty

store.get("bar").isNull();   // => true

Fetching data Copy Section

Use the get() method (of which the [] operator is an alias) to fetch data from the store. They both return a Json::Value.

// Assuming we are using the store that went through
 // 'Putting data in the store' and 'Updating data'.
 store.get("foo").asString();    // => "strval"
 store["foo"].asString();        // => same

Default values Copy Section

If the key is not defined then get() either return the default value as defined in the schema, or (if no default value is defined) a null Json::Value.

// Assuming we are using the store that went through
// 'Putting data in the store' and 'Updating data'.
store.get("baz").asInt();       // => 123
store.get("unknown").isNull();  // => true

Inspecting all data Copy Section

You can fetch an overview of all data in the store using inspect(). This function is normally used to allow users of a component to inspect the configuration options set for that component, without allowing them direct access to the embedded store.

This function will return a Json::Value in the following format:

// Assuming we are using the store that went through
// 'Putting data in the store' and 'Updating data'.

{
  "foo": {
    "user_value": "strval",
    "effective_value": "strval",
    // ...members from ConfigKit::Schema::inspect() go here...
  },
  "bar": {
    "user_value": 123.45,
    "effective_value": 123.45,
    // ...members from ConfigKit::Schema::inspect() go here...
  },
  "baz": {
    "user_value": null,
    "default_value": 123,
    "effective_value": 123,
    // ...members from ConfigKit::Schema::inspect() go here...
  },
  "password": {
    "user_value": "[FILTERED]",
    "effective_value": "[FILTERED]",
    // ...members from ConfigKit::Schema::inspect() go here...
  }
}

Description of the members:

• user_value: the value as explicitly set in the store. If null then it means that the value isn't set.
• default_value: the default value as defined in the schema. May be absent.
• effective_value: the effective value, i.e. the value that get() will return.

Note that inspect() filters out the values of fields with the SECRET flag (by setting the returned value to "[FILTERED]"), except for null values.

If you want to fetch the effective values only, then use inspectEffectiveValues():

// Assuming we are using the store that went through
// 'Putting data in the store' and 'Updating data'.

{
  "foo": "strval",
  "bar": 123.45,
  "baz": 123
}

Inspection filters Copy Section

Since inspect() is usually used to allow users of a component to inspect that component's configuration, you may run into situations where you want the inspected return value to be different from its actual value. Inspection filters allow you to transform inspect() results.

A typical use case for inspection filters can be found in LoggingKit. One can configure LoggingKit to log to a specific file descriptor of an open log file. The format is like this:

{
  "path": "/foo.log",
  "fd": 12
}

LoggingKit will internally take over ownership of the file descriptor and will perform a bunch of actions on the fd, such as redirecting stderr to that fd and closing the original fd. Because of this, the fd value is only valid at the time of configuration; it makes no sense to output the fd value in inspect(). Inspect filters allows LoggingKit to filter out the "fd" field when store.inspect() is called, but the "fd" field can still be accessed internally by LoggingKit by calling store["target"]["fd"].

An inspect filter is a function takes a value and returns a transformed value. It is installed by calling setInspectFilter() on the object returned by schema.add(), like this:

static Json::Value filterTargetFd(const Json::Value &value) {
  Json::Value result = value;
  result.removeMember("fd");
  return result;
}

ConfigKit::Schema schema;

schema.add("target", ANY_TYPE, OPTIONAL).
  setInspectFilter(filterTargetFd);
schema.finalize();

Note that inspect filter is called after normalizers, so value refers to a normalized value.

Normalizing data Copy Chapter

You sometimes may want to allow users to supply data in multiple formats. Normalizers allow you to transform user-supplied data in a canonical format, so that your configuration value handling code only has to deal with data in the canonical format instead of all possibly allowed formats.

The examples below demonstrate two possible use cases and how to implement them.

Normalization example 1 Copy Section

Suppose that you have a "target" config option, which can be in one of these formats:

1. "/filename"
2. { "path": "/filename" } (semantics equivalent to 1)
3. { "stderr": true }

You can write a normalizer that transforms format 1 into format 2. That way, no matter whether the user has actually supplied format 1, 2 or 3, your config handling code only has to deal with format 2 and 3.

A normalizer is a function that accepts a JSON document of effective values (in the format outputted by ConfigKit::Store::inspectEffectiveValues()). It is expected to return either Json::nullValue (indicating that no normalization work needs to be done), or a JSON object containing proposed normalization changes.

Normalizers are added to the corresponding schema.

static Json::Value myNormalizer(const Json::Value &effectiveValues) {
    Json::Value updates(Json::objectValue);

    if (effectiveValues["target"].isString()) {
        updates["target"]["path"] = effectiveValues["target"];
    }

    return updates;
}


ConfigKit::Schema schema;

schema.add("target", ANY_TYPE, OPTIONAL);
schema.addValidator(validateThatTargetIsStringOrObject);
schema.addNormalizer(myNormalizer);
schema.finalize();

Normalization example 2 Copy Section

Suppose that you have a "security" config option that accepts this format:

{
  "username": "a string",          // required
  "password": "a string",          // required
  "level": "full" | "readonly"     // optional; default value: "full"
}

If the user did not specify "level", then will want to automatically insert "level: full".

static Json::Value myNormalizer(const Json::Value &effectiveValues) {
    Json::Value updates(Json::objectValue);

    if (effectiveValues["security"]["level"].isNull()) {
        updates["security"] = effectiveValues["security"];
        updates["security"]["level"] = "full";
    }

    return updates;
}


ConfigKit::Schema schema;

schema.add("security", OBJECT_TYPE, OPTIONAL);
schema.addValidator(validateSecurity);
schema.addNormalizer(myNormalizer);
schema.finalize();

Normalizers and validation Copy Section

Normalizers are only run when validation passes! That way normalizers don't have to worry about validation problems.

ConfigKit in practice & design patterns Copy Chapter

This README has taught you how to use ConfigKit by itself. But how does ConfigKit fit in the bigger picture? IN_PRACTICE.md describes good practices and design patterns can that can be used throughout the overall Passenger C++ codebase. It provides practical guidance on how to use ConfigKit in the Passenger codebase.