## File: README.md Docker: the Linux container engine ================================== Docker is an open source project to pack, ship and run any application as a lightweight container. Docker containers are both *hardware-agnostic* and *platform-agnostic*. This means they can run anywhere, from your laptop to the largest EC2 compute instance and everything in between - and they don't require you to use a particular language, framework or packaging system. That makes them great building blocks for deploying and scaling web apps, databases, and backend services without depending on a particular stack or provider. Docker began as an open-source implementation of the deployment engine which powers [dotCloud](https://dotcloud.com), a popular Platform-as-a-Service. It benefits directly from the experience accumulated over several years of large-scale operation and support of hundreds of thousands of applications and databases. ## Security Disclosure Security is very important to us. If you have any issue regarding security, please disclose the information responsibly by sending an email to security@docker.com and not by creating a github issue. ## Better than VMs A common method for distributing applications and sandboxing their execution is to use virtual machines, or VMs. Typical VM formats are VMWare's vmdk, Oracle Virtualbox's vdi, and Amazon EC2's ami. In theory these formats should allow every developer to automatically package their application into a "machine" for easy distribution and deployment. In practice, that almost never happens, for a few reasons: * *Size*: VMs are very large which makes them impractical to store and transfer. * *Performance*: running VMs consumes significant CPU and memory, which makes them impractical in many scenarios, for example local development of multi-tier applications, and large-scale deployment of cpu and memory-intensive applications on large numbers of machines. * *Portability*: competing VM environments don't play well with each other. Although conversion tools do exist, they are limited and add even more overhead. * *Hardware-centric*: VMs were designed with machine operators in mind, not software developers. As a result, they offer very limited tooling for what developers need most: building, testing and running their software. For example, VMs offer no facilities for application versioning, monitoring, configuration, logging or service discovery. By contrast, Docker relies on a different sandboxing method known as *containerization*. Unlike traditional virtualization, containerization takes place at the kernel level. Most modern operating system kernels now support the primitives necessary for containerization, including Linux with [openvz](https://openvz.org), [vserver](http://linux-vserver.org) and more recently [lxc](http://lxc.sourceforge.net), Solaris with [zones](https://docs.oracle.com/cd/E26502_01/html/E29024/preface-1.html#scrolltoc), and FreeBSD with [Jails](https://www.freebsd.org/doc/handbook/jails.html). Docker builds on top of these low-level primitives to offer developers a portable format and runtime environment that solves all four problems. Docker containers are small (and their transfer can be optimized with layers), they have basically zero memory and cpu overhead, they are completely portable, and are designed from the ground up with an application-centric design. Perhaps best of all, because Docker operates at the OS level, it can still be run inside a VM! ## Plays well with others Docker does not require you to buy into a particular programming language, framework, packaging system, or configuration language. Is your application a Unix process? Does it use files, tcp connections, environment variables, standard Unix streams and command-line arguments as inputs and outputs? Then Docker can run it. Can your application's build be expressed as a sequence of such commands? Then Docker can build it. ## Escape dependency hell A common problem for developers is the difficulty of managing all their application's dependencies in a simple and automated way. This is usually difficult for several reasons: * *Cross-platform dependencies*. Modern applications often depend on a combination of system libraries and binaries, language-specific packages, framework-specific modules, internal components developed for another project, etc. These dependencies live in different "worlds" and require different tools - these tools typically don't work well with each other, requiring awkward custom integrations. * *Conflicting dependencies*. Different applications may depend on different versions of the same dependency. Packaging tools handle these situations with various degrees of ease - but they all handle them in different and incompatible ways, which again forces the developer to do extra work. * *Custom dependencies*. A developer may need to prepare a custom version of their application's dependency. Some packaging systems can handle custom versions of a dependency, others can't - and all of them handle it differently. Docker solves the problem of dependency hell by giving the developer a simple way to express *all* their application's dependencies in one place, while streamlining the process of assembling them. If this makes you think of [XKCD 927](https://xkcd.com/927/), don't worry. Docker doesn't *replace* your favorite packaging systems. It simply orchestrates their use in a simple and repeatable way. How does it do that? With layers. Docker defines a build as running a sequence of Unix commands, one after the other, in the same container. Build commands modify the contents of the container (usually by installing new files on the filesystem), the next command modifies it some more, etc. Since each build command inherits the result of the previous commands, the *order* in which the commands are executed expresses *dependencies*. Here's a typical Docker build process: ```bash FROM ubuntu:12.04 RUN apt-get update && apt-get install -y python python-pip curl RUN curl -sSL https://github.com/shykes/helloflask/archive/master.tar.gz | tar -xzv RUN cd helloflask-master && pip install -r requirements.txt ``` Note that Docker doesn't care *how* dependencies are built - as long as they can be built by running a Unix command in a container. Getting started =============== Docker can be installed on your local machine as well as servers - both bare metal and virtualized. It is available as a binary on most modern Linux systems, or as a VM on Windows, Mac and other systems. We also offer an [interactive tutorial](https://www.docker.com/tryit/) for quickly learning the basics of using Docker. For up-to-date install instructions, see the [Docs](https://docs.docker.com). Usage examples ============== Docker can be used to run short-lived commands, long-running daemons (app servers, databases, etc.), interactive shell sessions, etc. You can find a [list of real-world examples](https://docs.docker.com/examples/) in the documentation. Under the hood -------------- Under the hood, Docker is built on the following components: * The [cgroup](http://blog.dotcloud.com/kernel-secrets-from-the-paas-garage-part-24-c) and [namespacing](http://blog.dotcloud.com/under-the-hood-linux-kernels-on-dotcloud-part) capabilities of the Linux kernel * The [Go](https://golang.org) programming language * The [Docker Image Specification](https://github.com/docker/docker/blob/master/image/spec/v1.md) * The [Libcontainer Specification](https://github.com/docker/libcontainer/blob/master/SPEC.md) Contributing to Docker ====================== [](https://godoc.org/github.com/docker/docker) [](https://jenkins.dockerproject.com/job/Docker%20Master/) Want to hack on Docker? Awesome! We have [instructions to help you get started contributing code or documentation.](https://docs.docker.com/project/who-written-for/). These instructions are probably not perfect, please let us know if anything feels wrong or incomplete. Better yet, submit a PR and improve them yourself. Getting the development builds ============================== Want to run Docker from a master build? You can download master builds at [master.dockerproject.com](https://master.dockerproject.com). They are updated with each commit merged into the master branch. Don't know how to use that super cool new feature in the master build? Check out the master docs at [docs.master.dockerproject.com](http://docs.master.dockerproject.com). How the project is run ====================== Docker is a very, very active project. If you want to learn more about how it is run, or want to get more involved, the best place to start is [the project directory](https://github.com/docker/docker/tree/master/project). We are always open to suggestions on process improvements, and are always looking for more maintainers. ### Talking to other Docker users and contributors | Internet Relay Chat (IRC) | IRC a direct line to our most knowledgeable Docker users; we have both the `#docker` and `#docker-dev` group on **irc.freenode.net**. IRC is a rich chat protocol but it can overwhelm new users. You can search [our chat archives](https://botbot.me/freenode/docker/#). Read our [IRC quickstart guide](https://docs.docker.com/project/get-help/#irc-quickstart) for an easy way to get started. | | --- | --- | | Google Groups | There are two groups. [Docker-user](https://groups.google.com/forum/#!forum/docker-user) is for people using Docker containers. The [docker-dev](https://groups.google.com/forum/#!forum/docker-dev) group is for contributors and other people contributing to the Docker project. | | Twitter | You can follow [Docker's Twitter feed](https://twitter.com/docker/) to get updates on our products. You can also tweet us questions or just share blogs or stories. | | Stack Overflow | Stack Overflow has over 7000K Docker questions listed. We regularly monitor [Docker questions](https://stackoverflow.com/search?tab=newest&q=docker) and so do many other knowledgeable Docker users. | ### Legal *Brought to you courtesy of our legal counsel. For more context, please see the [NOTICE](https://github.com/docker/docker/blob/master/NOTICE) document in this repo.* Use and transfer of Docker may be subject to certain restrictions by the United States and other governments. It is your responsibility to ensure that your use and/or transfer does not violate applicable laws. For more information, please see https://www.bis.doc.gov Licensing ========= Docker is licensed under the Apache License, Version 2.0. See [LICENSE](https://github.com/docker/docker/blob/master/LICENSE) for the full license text. Other Docker Related Projects ============================= There are a number of projects under development that are based on Docker's core technology. These projects expand the tooling built around the Docker platform to broaden its application and utility. * [Docker Registry](https://github.com/docker/distribution): Registry server for Docker (hosting/delivery of repositories and images) * [Docker Machine](https://github.com/docker/machine): Machine management for a container-centric world * [Docker Swarm](https://github.com/docker/swarm): A Docker-native clustering system * [Docker Compose](https://github.com/docker/compose) (formerly Fig): Define and run multi-container apps * [Kitematic](https://github.com/kitematic/kitematic): The easiest way to use Docker on a Mac If you know of another project underway that should be listed here, please help us keep this list up-to-date by submitting a PR. --- ## File: api/README.md This directory contains code pertaining to the Docker API: - Used by the docker client when communicating with the docker daemon - Used by third party tools wishing to interface with the docker daemon --- ## File: contrib/builder/rpm/README.md # `dockercore/builder-rpm` This image's tags contain the dependencies for building Docker `.rpm`s for each of the RPM-based platforms Docker targets. To add new tags, see [`contrib/builder/rpm` in https://github.com/docker/docker](https://github.com/docker/docker/tree/master/contrib/builder/rpm), specifically the `generate.sh` script, whose usage is described in a comment at the top of the file. --- ## File: contrib/builder/deb/README.md # `dockercore/builder-deb` This image's tags contain the dependencies for building Docker `.deb`s for each of the Debian-based platforms Docker targets. To add new tags, see [`contrib/builder/deb` in https://github.com/docker/docker](https://github.com/docker/docker/tree/master/contrib/builder/deb), specifically the `generate.sh` script, whose usage is described in a comment at the top of the file. --- ## File: contrib/desktop-integration/README.md Desktop Integration =================== The ./contrib/desktop-integration contains examples of typical dockerized desktop applications. Examples ======== * Chromium: ./chromium/Dockerfile shows a way to dockerize a common application * Gparted: ./gparted/Dockerfile shows a way to dockerize a common application w devices --- ## File: contrib/syntax/vim/README.md dockerfile.vim ============== Syntax highlighting for Dockerfiles Installation ------------ With [pathogen](https://github.com/tpope/vim-pathogen), the usual way... With [Vundle](https://github.com/gmarik/Vundle.vim) Plugin 'docker/docker' , {'rtp': '/contrib/syntax/vim/'} Features -------- The syntax highlighting includes: * The directives (e.g. `FROM`) * Strings * Comments License ------- BSD, short and sweet --- ## File: contrib/syntax/textmate/README.md # Docker.tmbundle Dockerfile syntaxt highlighting for TextMate and Sublime Text. ## Install ### Sublime Text Available for Sublime Text under [package control](https://sublime.wbond.net/packages/Dockerfile%20Syntax%20Highlighting). Search for *Dockerfile Syntax Highlighting* ### TextMate 2 Copy the directory `Docker.tmbundle` (showed as a Package in OSX) to `~/Library/Application Support/TextMate/Managed/Bundles` enjoy. --- ## File: contrib/syntax/nano/README.md Dockerfile.nanorc ================= Dockerfile syntax highlighting for nano Single User Installation ------------------------ 1. Create a nano syntax directory in your home directory: * `mkdir -p ~/.nano/syntax` 2. Copy `Dockerfile.nanorc` to` ~/.nano/syntax/` * `cp Dockerfile.nanorc ~/.nano/syntax/` 3. Add the following to your `~/.nanorc` to tell nano where to find the `Dockerfile.nanorc` file ``` ## Dockerfile files include "~/.nano/syntax/Dockerfile.nanorc" ``` System Wide Installation ------------------------ 1. Create a nano syntax directory: * `mkdir /usr/local/share/nano` 2. Copy `Dockerfile.nanorc` to `/usr/local/share/nano` * `cp Dockerfile.nanorc /usr/local/share/nano/` 3. Add the following to your `/etc/nanorc`: ``` ## Dockerfile files include "/usr/local/share/nano/Dockerfile.nanorc" ``` --- ## File: contrib/vagrant-docker/README.md # Vagrant integration Currently there are at least 4 different projects that we are aware of that deals with integration with [Vagrant](http://vagrantup.com/) at different levels. One approach is to use Docker as a [provisioner](http://docs.vagrantup.com/v2/provisioning/index.html) which means you can create containers and pull base images on VMs using Docker's CLI and the other is to use Docker as a [provider](http://docs.vagrantup.com/v2/providers/index.html), meaning you can use Vagrant to control Docker containers. ### Provisioners * [Vocker](https://github.com/fgrehm/vocker) * [Ventriloquist](https://github.com/fgrehm/ventriloquist) ### Providers * [docker-provider](https://github.com/fgrehm/docker-provider) * [vagrant-shell](https://github.com/destructuring/vagrant-shell) ## Setting up Vagrant-docker with the Remote API The initial Docker upstart script will not work because it runs on `127.0.0.1`, which is not accessible to the host machine. Instead, we need to change the script to connect to `0.0.0.0`. To do this, modify `/etc/init/docker.conf` to look like this: ``` description "Docker daemon" start on filesystem and started lxc-net stop on runlevel [!2345] respawn script /usr/bin/docker -d -H=tcp://0.0.0.0:2375 end script ``` Once that's done, you need to set up a SSH tunnel between your host machine and the vagrant machine that's running Docker. This can be done by running the following command in a host terminal: ``` ssh -L 2375:localhost:2375 -p 2222 vagrant@localhost ``` (The first 2375 is what your host can connect to, the second 2375 is what port Docker is running on in the vagrant machine, and the 2222 is the port Vagrant is providing for SSH. If VirtualBox is the VM you're using, you can see what value "2222" should be by going to: Network > Adapter 1 > Advanced > Port Forwarding in the VirtualBox GUI.) Note that because the port has been changed, to run docker commands from within the command line you must run them like this: ``` sudo docker -H 0.0.0.0:2375 < commands for docker > ``` --- ## File: daemon/graphdriver/devmapper/README.md ## devicemapper - a storage backend based on Device Mapper ### Theory of operation The device mapper graphdriver uses the device mapper thin provisioning module (dm-thinp) to implement CoW snapshots. For each devicemapper graph location (typically `/var/lib/docker/devicemapper`, $graph below) a thin pool is created based on two block devices, one for data and one for metadata. By default these block devices are created automatically by using loopback mounts of automatically created sparse files. The default loopback files used are `$graph/devicemapper/data` and `$graph/devicemapper/metadata`. Additional metadata required to map from docker entities to the corresponding devicemapper volumes is stored in the `$graph/devicemapper/json` file (encoded as Json). In order to support multiple devicemapper graphs on a system, the thin pool will be named something like: `docker-0:33-19478248-pool`, where the `0:33` part is the minor/major device nr and `19478248` is the inode number of the $graph directory. On the thin pool, docker automatically creates a base thin device, called something like `docker-0:33-19478248-base` of a fixed size. This is automatically formatted with an empty filesystem on creation. This device is the base of all docker images and containers. All base images are snapshots of this device and those images are then in turn used as snapshots for other images and eventually containers. ### Information on `docker info` As of docker-1.4.1, `docker info` when using the `devicemapper` storage driver will display something like: $ sudo docker info [...] Storage Driver: devicemapper Pool Name: docker-253:1-17538953-pool Pool Blocksize: 65.54 kB Data file: /dev/loop4 Metadata file: /dev/loop4 Data Space Used: 2.536 GB Data Space Total: 107.4 GB Data Space Available: 104.8 GB Metadata Space Used: 7.93 MB Metadata Space Total: 2.147 GB Metadata Space Available: 2.14 GB Udev Sync Supported: true Data loop file: /home/docker/devicemapper/devicemapper/data Metadata loop file: /home/docker/devicemapper/devicemapper/metadata Library Version: 1.02.82-git (2013-10-04) [...] #### status items Each item in the indented section under `Storage Driver: devicemapper` are status information about the driver. * `Pool Name` name of the devicemapper pool for this driver. * `Pool Blocksize` tells the blocksize the thin pool was initialized with. This only changes on creation. * `Data file` blockdevice file used for the devicemapper data * `Metadata file` blockdevice file used for the devicemapper metadata * `Data Space Used` tells how much of `Data file` is currently used * `Data Space Total` tells max size the `Data file` * `Data Space Available` tells how much free space there is in the `Data file`. If you are using a loop device this will report the actual space available to the loop device on the underlying filesystem. * `Metadata Space Used` tells how much of `Metadata file` is currently used * `Metadata Space Total` tells max size the `Metadata file` * `Metadata Space Available` tells how much free space there is in the `Metadata file`. If you are using a loop device this will report the actual space available to the loop device on the underlying filesystem. * `Udev Sync Supported` tells whether devicemapper is able to sync with Udev. Should be `true`. * `Data loop file` file attached to `Data file`, if loopback device is used * `Metadata loop file` file attached to `Metadata file`, if loopback device is used * `Library Version` from the libdevmapper used ### options The devicemapper backend supports some options that you can specify when starting the docker daemon using the `--storage-opt` flags. This uses the `dm` prefix and would be used something like `docker -d --storage-opt dm.foo=bar`. Here is the list of supported options: * `dm.basesize` Specifies the size to use when creating the base device, which limits the size of images and containers. The default value is 10G. Note, thin devices are inherently "sparse", so a 10G device which is mostly empty doesn't use 10 GB of space on the pool. However, the filesystem will use more space for the empty case the larger the device is. **Warning**: This value affects the system-wide "base" empty filesystem that may already be initialized and inherited by pulled images. Typically, a change to this value will require additional steps to take effect: 1) stop `docker -d`, 2) `rm -rf /var/lib/docker`, 3) start `docker -d`. Example use: ``docker -d --storage-opt dm.basesize=20G`` * `dm.loopdatasize` Specifies the size to use when creating the loopback file for the "data" device which is used for the thin pool. The default size is 100G. Note that the file is sparse, so it will not initially take up this much space. Example use: ``docker -d --storage-opt dm.loopdatasize=200G`` * `dm.loopmetadatasize` Specifies the size to use when creating the loopback file for the "metadadata" device which is used for the thin pool. The default size is 2G. Note that the file is sparse, so it will not initially take up this much space. Example use: ``docker -d --storage-opt dm.loopmetadatasize=4G`` * `dm.fs` Specifies the filesystem type to use for the base device. The supported options are "ext4" and "xfs". The default is "ext4" Example use: ``docker -d --storage-opt dm.fs=xfs`` * `dm.mkfsarg` Specifies extra mkfs arguments to be used when creating the base device. Example use: ``docker -d --storage-opt "dm.mkfsarg=-O ^has_journal"`` * `dm.mountopt` Specifies extra mount options used when mounting the thin devices. Example use: ``docker -d --storage-opt dm.mountopt=nodiscard`` * `dm.thinpooldev` Specifies a custom blockdevice to use for the thin pool. If using a block device for device mapper storage, ideally lvm2 would be used to create/manage the thin-pool volume that is then handed to docker to exclusively create/manage the thin and thin snapshot volumes needed for its containers. Managing the thin-pool outside of docker makes for the most feature-rich method of having docker utilize device mapper thin provisioning as the backing storage for docker's containers. lvm2-based thin-pool management feature highlights include: automatic or interactive thin-pool resize support, dynamically change thin-pool features, automatic thinp metadata checking when lvm2 activates the thin-pool, etc. Example use: ``docker -d --storage-opt dm.thinpooldev=/dev/mapper/thin-pool`` * `dm.datadev` Specifies a custom blockdevice to use for data for the thin pool. If using a block device for device mapper storage, ideally both datadev and metadatadev should be specified to completely avoid using the loopback device. Example use: ``docker -d --storage-opt dm.datadev=/dev/sdb1 --storage-opt dm.metadatadev=/dev/sdc1`` * `dm.metadatadev` Specifies a custom blockdevice to use for metadata for the thin pool. For best performance the metadata should be on a different spindle than the data, or even better on an SSD. If setting up a new metadata pool it is required to be valid. This can be achieved by zeroing the first 4k to indicate empty metadata, like this: ``dd if=/dev/zero of=$metadata_dev bs=4096 count=1`` Example use: ``docker -d --storage-opt dm.datadev=/dev/sdb1 --storage-opt dm.metadatadev=/dev/sdc1`` * `dm.blocksize` Specifies a custom blocksize to use for the thin pool. The default blocksize is 64K. Example use: ``docker -d --storage-opt dm.blocksize=512K`` * `dm.blkdiscard` Enables or disables the use of blkdiscard when removing devicemapper devices. This is enabled by default (only) if using loopback devices and is required to resparsify the loopback file on image/container removal. Disabling this on loopback can lead to *much* faster container removal times, but will make the space used in /var/lib/docker directory not be returned to the system for other use when containers are removed. Example use: ``docker -d --storage-opt dm.blkdiscard=false`` * `dm.override_udev_sync_check` Overrides the `udev` synchronization checks between `devicemapper` and `udev`. `udev` is the device manager for the Linux kernel. To view the `udev` sync support of a Docker daemon that is using the `devicemapper` driver, run: $ docker info [...] Udev Sync Supported: true [...] When `udev` sync support is `true`, then `devicemapper` and udev can coordinate the activation and deactivation of devices for containers. When `udev` sync support is `false`, a race condition occurs between the`devicemapper` and `udev` during create and cleanup. The race condition results in errors and failures. (For information on these failures, see [docker#4036](https://github.com/docker/docker/issues/4036)) To allow the `docker` daemon to start, regardless of `udev` sync not being supported, set `dm.override_udev_sync_check` to true: $ docker -d --storage-opt dm.override_udev_sync_check=true When this value is `true`, the `devicemapper` continues and simply warns you the errors are happening. > **Note**: The ideal is to pursue a `docker` daemon and environment that > does support synchronizing with `udev`. For further discussion on this > topic, see [docker#4036](https://github.com/docker/docker/issues/4036). > Otherwise, set this flag for migrating existing Docker daemons to a > daemon with a supported environment. * `dm.use_deferred_removal` Enables use of deferred device removal if libdm and kernel driver support the mechanism. Deferred device removal means that if device is busy when devices is being removed/deactivated, then a deferred removal is scheduled on device. And devices automatically goes away when last user of device exits. For example, when contianer exits, its associated thin device is removed. If that devices has leaked into some other mount namespace can can't be removed now, container exit will still be successful and this option will just schedule device for deferred removal and will not wait in a loop trying to remove a busy device. Example use: ``docker -d --storage-opt dm.use_deferred_removal=true``