presto Documentation

1. Project Overview & Quickstart (prestodb/presto) Copy Chapter

Presto

Presto is a distributed SQL query engine for big data.

See the Presto installation documentation for deployment instructions.

See the Presto documentation for general documentation.

Mission and Architecture Copy Chapter

See PrestoDB: Mission and Architecture.

Requirements Copy Chapter

• Mac OS X or Linux

• Java 17 64-bit. Both Oracle JDK and OpenJDK are supported.

• Maven 3.6.3+ (for building)

• Python 2.4+ (for running with the launcher script)

  Building Presto

Overview (Java) Copy Section

Presto is a standard Maven project. Simply run the following command from the project root directory:

./mvnw clean install

On the first build, Maven will download all the dependencies from the internet and cache them in the local repository (~/.m2/repository), which can take a considerable amount of time. Subsequent builds will be faster.

When building multiple Presto projects locally, each project may write updates to the user's global M2 cache, which could cause build issues. You can configure your local .mvn/maven.config to support a local cache specific to that project via -Dmaven.repo.local=./.m2/repository.

Presto has a comprehensive set of unit tests that can take several minutes to run. You can disable the tests when building:

./mvnw clean install -DskipTests

After building Presto for the first time, you can load the project into your IDE and run the server. We recommend using IntelliJ IDEA. Because Presto is a standard Maven project, you can import it into your IDE using the root pom.xml file. In IntelliJ, choose Open Project from the Quick Start box or choose Open from the File menu and select the root pom.xml file.

After opening the project in IntelliJ, double check that the Java SDK is properly configured for the project:

• Open the File menu and select Project Structure
• In the SDKs section, ensure that a distribution of JDK 17 is selected (create one if none exist)
• In the Project section, ensure the Project language level is set to at least 8.0.
• When using JDK 17, an IntelliJ bug requires you
  to disable the Use '--release' option for cross-compilation (Java 9 and later) setting in
  Settings > Build, Execution, Deployment > Compiler > Java Compiler. If this option remains enabled,
  you may encounter errors such as: package sun.misc does not exist because IntelliJ fails to resolve
  certain internal JDK classes.

Presto comes with sample configuration that should work out-of-the-box for development. Use the following options to create a run configuration:

• Main Class: com.facebook.presto.server.PrestoServer
• VM Options: -ea -XX:+UseG1GC -XX:G1HeapRegionSize=32M -XX:+UseGCOverheadLimit -XX:+ExplicitGCInvokesConcurrent -Xmx2G -Dconfig=etc/config.properties -Dlog.levels-file=etc/log.properties -Djdk.attach.allowAttachSelf=true
• Working directory: $MODULE_WORKING_DIR$ or $MODULE_DIR$(Depends your version of IntelliJ)
• Use classpath of module: presto-main

The working directory should be the presto-main subdirectory. In IntelliJ, using $MODULE_DIR$ accomplishes this automatically.

Additionally, the Hive plugin must be configured with location of your Hive metastore Thrift service. Add the following to the list of VM options, replacing localhost:9083 with the correct host and port (or use the below value if you do not have a Hive metastore):

-Dhive.metastore.uri=thrift://localhost:9083

To modify the loaded plugins in IntelliJ, modify the config.properties located in presto-main/etc. You can modify plugin.bundles with the location of the plugin pom.xml

Additional configuration for Java 17 Copy Section

When running with Java 17, additional --add-opens flags are required to allow reflective access used by certain catalogs based on which catalogs are configured.
For the default set of catalogs loaded when starting the Presto server in IntelliJ without changes, add the following flags to the VM Options:

--add-opens=java.base/java.io=ALL-UNNAMED
--add-opens=java.base/java.lang=ALL-UNNAMED
--add-opens=java.base/java.lang.ref=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.net=ALL-UNNAMED
--add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.security=ALL-UNNAMED
--add-opens=java.base/javax.security.auth=ALL-UNNAMED
--add-opens=java.base/javax.security.auth.login=ALL-UNNAMED
--add-opens=java.base/java.text=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent=ALL-UNNAMED
--add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED
--add-opens=java.base/java.util.regex=ALL-UNNAMED
--add-opens=java.base/jdk.internal.loader=ALL-UNNAMED
--add-opens=java.base/sun.security.action=ALL-UNNAMED
--add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED

These flags ensure that internal JDK modules are accessible at runtime for components used by Presto’s default configuration.
It is not a comprehensive list. Additional flags may need to be added, depending on the catalogs configured on the server.

Using SOCKS for Hive or HDFS Copy Section

If your Hive metastore or HDFS cluster is not directly accessible to your local machine, you can use SSH port forwarding to access it. Setup a dynamic SOCKS proxy with SSH listening on local port 1080:

ssh -v -N -D 1080 server

Then add the following to the list of VM options:

-Dhive.metastore.thrift.client.socks-proxy=localhost:1080

Running the CLI Copy Section

Start the CLI to connect to the server and run SQL queries:

presto-cli/target/presto-cli-*-executable.jar

Run a query to see the nodes in the cluster:

SELECT * FROM system.runtime.nodes;

In the sample configuration, the Hive connector is mounted in the hive catalog, so you can run the following queries to show the tables in the Hive database default:

SHOW TABLES FROM hive.default;

Building the Documentation Copy Section

To build the Presto docs, see the docs README.

Building the Presto Console Copy Section

The Presto Console is composed of several React components and is written in JSX and ES6. This
source code is stored in the presto-ui/ module. The compilation process generates
browser-compatible javascript which is added as JAR resources during the maven build. When the
resource JAR is included on the classpath of Presto coordinator, it will be able to serve the
resources.

None of the Java code relies on the Presto UI project being compiled, so it is possible to exclude
this UI when building Presto. Add the property -DskipUI to the maven command to disable building
the ui maven module.

./mvnw clean install -DskipUI

You must have Node.js and Yarn installed to build the UI. When using Maven to build
the project, Node and yarn are installed in the presto-ui/target folder. Add the node and yarn
executables to the PATH environment variable.

To update Presto Console after making changes, run:

yarn --cwd presto-ui/src install

To simplify iteration, you can also run in watch mode, which automatically re-compiles when
changes to source files are detected:

yarn --cwd presto-ui/src run watch

To iterate quickly, simply re-build the project in IntelliJ after packaging is complete. Project
resources will be hot-reloaded and changes are reflected on browser refresh.

Presto native and Velox Copy Chapter

Presto native is a C++ rewrite of Presto worker. Presto native uses Velox as its primary engine to run presto workloads.

Velox is a C++ database library which provides reusable, extensible, and high-performance data processing components.

Check out building instructions to get started.

Using development containers Copy Chapter

The PrestoDB project provides support for development containers in its own repository.
Please visit the presto-dev README for details.

Contributing! Copy Chapter

Please refer to the contribution guidelines to get started.

Questions? Copy Chapter

Please join our Slack channel and ask in #dev.

License Copy Chapter

By contributing to Presto, you agree that your contributions will be licensed under the Apache License Version 2.0 (APLv2).

2. Official Technical Reference & Guides (prestodb/presto-dev) Copy Chapter

Presto/Prestissimo Dev Container

With this dev container, you can:

• Build Presto and Prestissimo from source
• Run Presto and Prestissimo with different configurations
• Debugging with VS Code or any IDE with ssh support
• Working on UI development
• Develop unit tests with test containers

Prerequisites Copy Chapter

• Docker or Podman must be installed and running.

Quick start Copy Chapter

This environment can be run on both Linux and macOS.

Run the following command to clone the presto-dev repository into your local presto directory, start the dev container and enter the container shell.

# Inside your local presto directory
git clone https://github.com/prestodb/presto-dev.git

# Start the dev container and enter container shell
cd presto-dev
make

Optional: Create aliases for convenience Copy Section

If you want to start the dev container from anywhere without navigating to the presto-dev directory, you can create a shell alias. For example, if your presto workspace is in ~/presto and you cloned presto-dev to ~/presto/presto-dev, add the following alias to your shell configuration file (e.g., ~/.bashrc, ~/.zshrc, or ~/.config/fish/config.fish):

alias presto-dev-shell='pushd ~/presto/presto-dev; make; popd'
alias presto-dev-update='pushd ~/presto/presto-dev; make pull; git pull; popd'
alias presto-dev-stop='pushd ~/presto/presto-dev; make stop; popd'

After reloading your shell configuration, you can simply run presto-dev-shell from any directory to start the container and enter the shell.

Work with source code Copy Section

In the dev container shell, you can run the following commands to work with source code.

# Build presto (.m2 already cached)
cd /presto
./mvnw clean install -DskipTests

# Run unit tests with test container
start-podman
cd /presto
./mvnw test -pl presto-main -Dtest="com/facebook/presto/server/security/oauth2/**"

# Build prestissimo(make debug or make release)
cd /presto/presto-native-execution
make

# Start presto-ui development server, open http://localhost:8081 in your browser
# Run "start-cluster <cluster profile>" first to start a presto cluster
cd /presto/presto-ui/src
yarn install
yarn serve

Update presto-dev image Copy Section

When a new presto-dev image is published, run the following command to pull the latest version.

# Under presto-dev, run this to pull the latest presto-dev image
make pull

# Or use alias
presto-dev-pull

Start Presto & Prestissimo server Copy Chapter

Run make or make shell in the presto-dev directory to open a new container shell.

All the commands below must be executed inside the container shell.

ls -la /root

# Start presto coordinator and prestissimo worker
start-cluster default

# Start presto single node and prestissimo sidecar
start-cluster sidecar

# Stop cluster
stop-cluster

Use http://localhost:8080 to open the Presto console.

Update Presto or Prestissimo configuration Copy Chapter

The configuration files are organized within cluster profiles in the clusters/ directory. Each cluster profile contains its own configuration files for Presto coordinator, Presto worker, and/or Prestissimo worker.

Available cluster profiles:

• presto: Presto coordinator and worker
• prestissimo: Presto coordinator and Prestissimo worker
• presto-single: Single node Presto server
• sidecar: Presto coordinator with worker and Prestissimo worker with sidecar configuration

You can modify the existing configuration files in these profiles or create a new cluster profile based on your specific requirements. To create a new cluster profile:

1. Create a new directory under clusters/ with your profile name
2. Copy the configuration files from an existing profile
3. Modify the configuration files as needed
4. Create a start-cluster script for your profile

The presto/data is mounted as /opt/presto/data to persist data between container restarts.

This approach allows you to maintain multiple configuration profiles and easily switch between them using the start-cluster script.

Dev with dev container(VSCode) Copy Chapter

1. Install Remote Development extension.
2. Open Remote Explorer panel, switch to Dev Containers from the drop down widget on the top.
3. Open the /presto directory in the composed container presto-dev (If importing Maven projects takes a long time, run ./mvnw clean in the /presto directory and make clean in the /presto/presto-native-execution directory inside the container.).
4. If presto/.vscode/launch.json is not already configured, the file launch.json is copied there automatically. If it is already configured, append the contents of launch.json from this repository to your existing presto/.vscode/launch.json.
5. Install Microsoft Extension Pack for Java and C/C++ Extension Pack in the container.
6. Start Presto & Prestissimo in the container.
7. After the Java projects are imported, open the Debug panel and select either Attach to Prestissimo or Attach to Presto to start debugging.

Dev with ssh Copy Chapter

VSCodium based IDEs cannot use the Remote Development extension. If your IDE supports remote ssh extension, you can try this:

1. Make sure you have generated ssh keys and updated the authorized_keys file in your local machine.
2. Use make start to start the container. This will copy your SSH keys from ~/.ssh/authorized_keys to the root/.ssh directory inside presto-dev.
3. Add the following entry to your local ~/.ssh/config file:

Host presto-dev
  HostName localhost
  Port 2222
  User root
  StrictHostKeyChecking no
  UserKnownHostsFile /dev/null

4. Test the ssh connection with ssh presto-dev.
5. Connect to the presto-dev SSH server using your IDE.

Keep your data Copy Chapter

By default, the root directory in this repo will be mounted to /root in the container. Use this root directory to share your data between your local machine and the container.

By default, the .ccache, .m2, and .cache directories under /root directory are symbolic links to the image's /opt/cache directories. These symbolic links will disappear when the container is shut down. To preserve your cache data between container restarts, you need to replace these symbolic links with local directories in the mounted /root directory.

Run the following command in the shell of dev container:

use-local-cache

This script replaces the symbolic links with actual directories in your local /root folder, copying the cache data from /opt/cache. Since the root directory is mounted into the container, these cache files will be preserved even when the container is removed, significantly improving build performance on subsequent runs.

Development Scripts Copy Chapter

The development environment includes scripts to help with building and installing Presto and Prestissimo.

1. Building and installing script Copy Section

Available Scripts

• build-and-install: Builds Presto and Prestissimo from source and installs them into /opt/presto and /opt/prestissimo respectively.

Usage

build-and-install

This script:

1. Builds Presto from source in /presto
2. Builds Prestissimo from source in /presto/presto-native-execution
3. Installs the built binaries to /opt/presto and /opt/prestissimo
4. Makes the newly built versions available for use

After running this script, you can start the servers with the newly built versions using the cluster management scripts.

2. Podman Container Scripts Copy Section

The development environment includes scripts to manage Podman containers for testing. Currently it only supports centos based dev container.

Available Scripts

• start-podman: Starts rootful podman with sock file unix:///var/run/docker.sock to enable running test containers inside the dev container.
• stop-podman: Stops the podman service.

Usage

# Start podman service
start-podman

# Run your tests that use test containers
cd /presto
./mvnw test -pl presto-main -Dtest="com/facebook/presto/server/security/oauth2/**"

# Stop podman service when done
stop-podman

The start-podman script enables running test containers inside the dev container by providing a Docker-compatible socket at unix:///var/run/docker.sock. This allows test frameworks like TestContainers to work properly within the development environment.

3. Cluster Management Scripts Copy Section

The development environment includes scripts to easily start and stop Presto and Prestissimo clusters. These scripts are located in the /root/bin directory.

Available Scripts

• start-cluster: Starts a specific cluster profile (presto, prestissimo, presto-single, or sidecar)
• stop-cluster: Stops all running Presto and Prestissimo servers across all cluster profiles

Usage

Starting a Cluster

start-cluster <cluster-profile>

Where <cluster-profile> is one of the available cluster profiles in the /opt/clusters directory (e.g., presto, prestissimo, presto-single, sidecar).

Example:

# Start the presto-single cluster
start-cluster presto-single

# Start the prestissimo cluster
start-cluster prestissimo

Stopping All Clusters

stop-cluster

This command will stop all running Presto and Prestissimo servers across all cluster profiles.

Cluster Profiles

Each cluster profile has its own configuration and behavior:

• presto: Starts a Presto coordinator and a Presto worker
• prestissimo: Starts a Presto coordinator and a Prestissimo worker
• presto-single: Starts a single node Presto server (coordinator with worker)
• sidecar: Starts a single node Presto coordinator with worker and a Prestissimo worker with sidecar configuration
• router: Starts a Presto router and a single node Presto server

The logs for Presto servers are available at /opt/presto/data/var/log/server.log, the logs for Prestissimo servers are available at /opt/prestissimo/logs/server.log, and the logs for Presto router are available at /opt/router/logs/server.log.

Build your own images Copy Chapter

You can build your own images by forking this repository and update .env file to customize your build.

For example, if you want to push the images to your own docker registry, you can add the following lines to .env file:

ORG = <your github org>
DOCKERHUB = <docker.io, ghcr.io or other registries>

If you want to customize the compiling options like cmake flags or parallel build options, you can also update the .env file:

NUM_THREADS=<number of threads for compiling>
COMMON_CMAKE_FLAGS=<common shared cmake flags>
CENTOS_CMAKE_FLAGS=<centos specific cmake flags>
UBUNTU_CMAKE_FLAGS=

In this way, you can build your own images and customize them according to your needs.

To build and push the images, you can use the following commands under directory presto-dev:

# Build centos images
make centos-cpp-dev && make centos-java-dev && make centos-dev

# Build ubuntu images
make ubuntu-cpp-dev && make ubuntu-java-dev && make ubuntu-dev

# Re-tag images for publishing
make release-prepare

# Publish images
make release-publish