.mackup.cfg Documentation

1. Project Overview & Quickstart (lra/.mackup.cfg) Copy Chapter

.mackup.cfg

Open-source repository lra/.mackup.cfg

Repository Details Copy Section

• Repository: lra/.mackup.cfg
• Primary Language: Code

Note: High-volume repository documentation is actively indexed and synchronized by YakaAI.

2. Official Technical Reference & Guides (lra/docs) Copy Chapter

File: README.md Copy Chapter

CoreOS Documentation

These are publicly accessible versions of the documentation on coreos.com.

1. Fork this repo
2. Send a Pull Request
3. We will get your fix up on coreos.com

Thanks for your contributions and fixes!

File: guides/etcd/index.md Copy Chapter

layout: docs slug: guides/etcd title: Documentation - etcd Copy Chapter

Getting Started with etcd

etcd is an open-source distributed key value store that provides shared configuration and service discovery for CoreOS clusters. etcd runs on each machine in a cluster and gracefully handles master election during network partitions and the loss of the current master.

Application containers running on your cluster can read and write data into etcd. Common examples are storing database connection details, cache settings, feature flags, and more. This guide will walk you through a basic example of reading and writing to etcd then proceed to other features like TTLs, directories and watching a prefix. This guide is way more fun when you've got at least one CoreOS machine up and running — try it on Amazon EC2 or locally with Vagrant.

Complete etcd API Docs

Reading and Writing to etcd Copy Chapter

The API is easy to use. From a CoreOS machine, you can simply use curl to set and retrieve a key from etcd. It's important to note the -L flag is required. etcd transparently redirects writes to the master and this flag allows curl to follow the location headers from etcd.

Set a key message with value Hello:

curl -L http://127.0.0.1:4001/v1/keys/message -d value="Hello"

Read the value of message back:

curl -L http://127.0.0.1:4001/v1/keys/message

If you followed a guide to set up more than one CoreOS machine, you can SSH into another machine and can retrieve this same value. To delete the key run:

curl -L http://127.0.0.1:4001/v1/keys/message -X DELETE

Reading and Writing from Inside a Container Copy Chapter

To read and write to etcd from within a container you must use the docker0 interface which you can find in ifconfig. It's normally 172.17.42.1 and using it is as easy as replacing 127.0.0.1.

Proxy Example Copy Chapter

Let's pretend we're setting up a service that consists of a few containers that are behind a proxy container. We can use etcd to announce these containers when they start by creating a directory, having each container write a key within that directory and have the proxy watch the entire directory. We're going to skip creating the containers here but the docker guide is a good place to start for that.

Create the directory Copy Section

Directories are automatically created when a key is placed inside. Let's call our directory foo-service and create a key with information about a container:

curl -L http://127.0.0.1:4001/v1/keys/foo-service/container1 -d value="localhost:1111"

Read the foo-service directory to see the entry:

curl -L http://127.0.0.1:4001/v1/keys/foo-service

Watching the Directory Copy Section

Now let's try watching the foo-service directory for changes, just like our proxy would have to. First, open up another shell on a CoreOS host in the cluster. In one window, start watching the directory by changing keys to watch. This command shouldn't output anything until the key has changed. Many events can trigger a change, including a new, updated, deleted or expired key.

curl -L http://127.0.0.1:4001/v1/watch/foo-service

In the other window, let's pretend a new container has started and announced itself to the proxy by running:

curl -L http://127.0.0.1:4001/v1/keys/foo-service/container2 -d value="localhost:2222"

In the first window, you should get the notification that the key has changed. In a real application, this would trigger reconfiguration.

Test and Set Copy Chapter

etcd can be used as a centralized coordination service and provides TestAndSet functionality as the building block of such a service. You must provide the prevValue along with your new value. If the previous value matches the current value the operation will succeed.

curl -L http://127.0.0.1:4001/v1/keys/message -d prevValue=Hello -d value=Hi

TTL Copy Chapter

You can optionally set a TTL for a key to expire in a certain number of seconds. Setting a TTL of 5 seconds:

curl -L http://127.0.0.1:4001/v1/keys/foo -d value=bar -d ttl=5

The response will contain an absolute timestamp of when the key will expire and a relative number of seconds until that timestamp:

{"action":"SET","key":"/foo","value":"bar","newKey":true,"expiration":"2013-07-11T20:31:12.156146039-07:00","ttl":4,"index":6}

If you request a key that has already expired, you will be returned a 100:

{"errorCode":100,"message":"Key Not Found","cause":"/foo"}

More Information

etcd Overview
Full etcd API Docs
Projects using etcd

File: guides/docker/index.md Copy Chapter

layout: docs slug: guides/docker title: Documentation - docker Copy Chapter

Getting Started with docker

docker is an open-source project that makes creating and managing Linux containers really easy. Containers are like extremely lightweight VMs – they allow code to run in isolation from other containers but safely share the machine’s resources, all without the overhead of a hypervisor.

docker containers can boot extremely fast (in milliseconds!) which gives you unprecedented flexibility in managing load across your cluster. For example, instead of running chef on each of your VMs, it’s faster and more reliable to have your build system create a container and launch it on the appropriate number of CoreOS hosts. This guide will show you how to launch a container, install some software on it, commit that container, and optionally launch it on another CoreOS machine. Before starting, make sure you've got at least one CoreOS machine up and running — try it on Amazon EC2 or locally with Vagrant.

Docker CLI Basics Copy Chapter

docker has a straightforward CLI that allows you to do almost everything you could want to a container. All of these commands use the image id (ex. be29975e0098), the image name (ex. myusername/webapp) and the container id (ex. 72d468f455ea) interchangably depending on the operation you are trying to do. This is confusing at first, so pay special attention to what you're using.

Launching a Container Copy Chapter

Launching a container is simple as docker run + the image name you would like to run + the command to run within the container. If the image doesn't exist on your local machine, docker will attempt to fetch it from the public image registry. Later we'll explore how to use docker with a private registry. It's important to note that containers are designed to stop once the command executed within them has exited. For example, if you ran /bin/echo hello world as your command, the container will start, print hello world and then stop:

docker run ubuntu /bin/echo hello world

Let's launch an ubuntu container and install apache inside of it using the bash prompt:

docker run -t -i ubuntu /bin/bash

The -t and -i flags allocate a pseudo-tty and keep stdin open even if not attached. This will allow you to use the container like a traditional VM as long as the bash prompt is running. Install apache with apt-get install apache2. You're probably wondering what address you can connect to in order to test that apache was correctly installed...we'll get to that after we commit the container.

Commiting a Container Copy Chapter

After that completes, we need to commit these changes to our container with the container ID and the image name. Fetch the container id using docker ps in another shell. Remember, the image name is in the format of username/name...we're going to use coreos in this example. Insert the id with the image name you'd like to use:

docker commit 72d468f455ea coreos/apache

The overlay filesystem works similar to git: our image now builds off of the ubuntu base and adds another layer with apache on top. These layers get cached separately so that you won't have to pull down the ubuntu base more than once.

Keeping the Apache Container Running Copy Chapter

Now we have our ubuntu container with apache running in one shell and an image of that container sitting on disk. Let's launch a new container based on that image but set it up to keep running indefinitely. The basic syntax looks like this, but we need to configure a few additional options that we'll fill in as we go:

docker run [options] [image] [process]

The first step is to tell docker that we want to run our coreos/apache image:

docker run [options] coreos/apache [process]

Run Container Detached Copy Section

The most important option is to run the container in detached mode with the -d flag. This will output the container ID to show that the command was successful, but nothing else. At any time you can run docker ps in the other shell to view a list of the running containers. Our command now looks like:

docker run -d coreos/apache [process]

Run Apache in Foreground Copy Section

We need to run the apache process in the foreground, since our container will stop when the process specified in the docker run command stops. We can do this with a flag -D when starting the apache2 process:

/usr/sbin/apache2ctl -D FOREGROUND

Let's add that to our command:

docker run -d coreos/apache /usr/sbin/apache2ctl -D FOREGROUND

Permanently Running a Container Copy Section

While the sections above explained how to run a container when configuring it, for a production setup, you should not manually start and babysit containers.

Instead, create a systemd unit file to make systemd keep that container running. See the Quick Start guide for details.

Network Access to 80 Copy Section

The default apache install will be running on port 80. To give our container access to traffic over port 80, we use the -p flag and specify the port on the host that maps to the port inside the container. In our case we want 80 for each, so we include -p 80:80 in our command:

docker run -d -p 80:80 coreos/apache /usr/sbin/apache2ctl -D FOREGROUND

You can now run this command on your CoreOS host to create the container. You should see the default apache webpage when you load either localhost:80 or the IP of your remote server. Be sure that any firewall or EC2 Security Group allows traffic to port 80.

Using the Docker Registry Copy Chapter

Earlier we downloaded the ubuntu image remotely from the docker public registry because it didn't exist on our local machine. We can also push local images to the public registry (or a private registry) very easily with the push command:

docker push coreos/apache

To push to a private repository the syntax is very similar. First, we must prefix our image with the host running our private registry instead of our username. List images by running docker images and insert the correct ID into the tag command:

docker tag f455ea72d468 registry.example.com:5000/apache

After tagging, the image needs to be pushed to the registry:

docker push registry.example.com:5000/apache

Once the image is done uploading, you should be able to start the exact same container on a different CoreOS host by running:

docker run -d -p 80:80 registry.example.com:5000/apache /usr/sbin/apache2ctl -D FOREGROUND

More Information

docker Overview
Docker Website
docker's Getting Started Guide

File: guides/resize/index.md Copy Chapter

layout: docs slug: guides/resize title: Adding Disk Space Copy Chapter

Adding Disk Space to Your CoreOS Machine Copy Chapter

Most writable paths such as /home, /var, and /usr/local are stored
in a special STATE partition mounted at /media/state. The default size
of this partition depends on the platform but it is usually
between 3GB and 16GB. If more space is required simply extend the
virtual machine's disk image and CoreOS will fix the partition table and
resize the STATE partition to fill the disk on the next boot.

Amazon EC2 Copy Section

Amazon doesn't support directly resizing volumes, you must take a
snapshot and create a new volume based on that snapshot. Refer to
the AWS EC2 documentation on expanding EBS volumes
for detailed instructions.

QEMU (qemu-img) Copy Section

Even if you are not using Qemu itself the qemu-img tool is the easiest
to use. It will work on raw, qcow2, vmdk, and most other formats. The
command accepts either an absolute size or a relative size by
by adding + prefix. Unit suffixes such as G or M are also supported.

# Increase the disk size by 5GB
qemu-img resize coreos_production_qemu_image.img +5G

VMware Copy Section

The interface available for resizing disks in VMware varies depending on
the product. See this Knowledge Base article for details.
Most products include a tool called vmware-vdiskmanager. The size must
be the absolute disk size, relative sizes are not supported so be
careful to only increase the size, not shrink it. The unit
suffixes Gb and Mb are supported.

# Set the disk size to 20GB
vmware-vdiskmanager -x 20Gb coreos_developer_vmware_insecure.vmx

VirtualBox Copy Section

Use qemu-img or vmware-vdiskmanager as described above. VirtualBox does
not support resizing VMDK disk images, only VDI and VHD disks. Meanwhile
VirtualBox only supports using VMDK disk images with the OVF config file
format used for importing/exporting virtual machines.

If you have have no other options you can try converting the VMDK disk
image to a VDI image and configuring a new virtual machine with it:

VBoxManage clonehd old.vmdk new.vdi --format VDI
VBoxManage modifyhd new.vdi --resize 20480

File: guides/systemd/index.md Copy Chapter

layout: docs slug: guides title: Documentation - systemd Copy Chapter

Getting Started with systemd

systemd is an init system that provides many powerful features for starting, stopping and managing processes. Within the CoreOS world, you will almost exclusively use systemd to manage the lifecycle of your Docker containers.

Terminology Copy Chapter

systemd consists of two main concepts: a unit and a target. A unit is a configuration file that describes the properties of the process that you'd like to run. This is normally a docker run command or somethng similar. A target is a grouping mechanism that allows systemd to start up groups of processes at the same time. This happens at every boot as processes are started at different run levels.

systemd is the first process started on CoreOS and it reads different targets and starts the processes specified which allows the operating system to start. The target that you'll interact with is the local.target which holds all of the unit files for our containers.

Each target is actually a collection of symlinks to our unit files. Running systemctl restart local-enable.service creates symlinks to any unit file that should reside in local.target. This is specified in the unit file by WantedBy=local.target.

Unit File Copy Chapter

On CoreOS, unit files are located within the R/W filesystem at /media/state/units. Let's take a look at a simple unit file:

[Unit]
Description=My Service
After=docker.service

[Service]
Restart=always
ExecStart=/usr/bin/docker run busybox /bin/sh -c "while true; do echo Hello World; sleep 1; done"

[Install]
WantedBy=local.target

The description shows up in the systemd log and a few other places. Write something that will help you understand exactly what this does later on.

After=docker.service means this unit will only start after docker.service is active. You can define as many of these as you want.

Restart=always is pretty obvious. This is helpful if your container is designed to exit for some reason but you'd like to start up an identical container when that happens.

ExecStart= allows you to specify any command that you'd like to run when this unit is started.

WantedBy= is the target that this unit is a part of.

Advanced Unit Files Copy Chapter

systemd provides a high degree of functionality in your unit files. Here's a curated list of useful features listed in the order they'll occur in the lifecycle of a unit:

The full list is located on the systemd man page.

Let's put a few of these concepts togther to register new units within etcd. Imagine we had another container running that would read these values from etcd and act upon them.

Since our docker run command will be ExecStart, it makes sense for our etcd command to run as ExecStartPost to ensure that our container is started and functioning.

We also need to clean up our etcd key when the container exits or the unit is failed by using ExecStopPost.

[Unit]
Description=My Advanced Service
After=etcd.service
After=docker.service

[Service]
Restart=always
ExecStart=/usr/bin/docker run -p 80:80 coreos/apache /usr/sbin/apache2ctl -D FOREGROUND
ExecStartPost=/usr/bin/etcdctl set /domains/example.com/10.10.10.123:8081 running
ExecStopPost=/usr/bin/etcdctl delete /domains/example.com/10.10.10.123:8081
RestartSec=10s

[Install]
WantedBy=local.target

Unit Variables Copy Chapter

If our last example we had to hardcode our IP address when we announced our container in etcd. That's not scalable and systemd has a few variables built in to help us out. Here's a few of the most useful:

A full list is on the systemd man page.

Instantiated Units Copy Chapter

Since systemd is based on symlinks, there are a few interesting tricks you can leverage that are very powerful when used with containers. If you create multiple symlinks to the same unit file, the following variables become available to you:

In our earlier example we had to hardcode our IP address when registering within etcd:

ExecStartPost=/usr/bin/etcdctl set /domains/example.com/10.10.10.123:8081 running

We can enhance this by using %H and %i to dynamically announce the hostname and port. Specify the port after the @ by using two unit files named [email protected] and [email protected]:

ExecStartPost=/usr/bin/etcdctl set /domains/example.com/%H:%i running

This gives us the flexiblity to use a single unit file to announce multiple copies of the same container on a both a single machine (no port overlap) and on multiple machines (no hostname overlap).

More Information

systemd.service Docs
systemd.unit Docs
systemd.target Docs

File: guides/index.md Copy Chapter

layout: docs slug: guides title: Documentation - Quick Start #redirect handled in alias_generator.rb Copy Chapter

Quick Start

If you don't have a CoreOS machine running, check out the guides on running CoreOS on [Vagrant][vagrant-guide], [Amazon EC2][ec2-guide], [QEMU/KVM][qemu-guide], [VMware][vmware-guide] and [OpenStack][openstack-guide]. With either of these guides you will have a machine up and running in a few minutes.

CoreOS gives you three essential tools: service discovery, container management and process management. Let's try each of them out.

First, connect to a CoreOS machine via SSH as the user core. For example, on Amazon, use:

ssh [email protected]

Service Discovery with etcd Copy Chapter

The first building block of CoreOS is service discovery with etcd ([docs][etcd-docs]). Data stored in etcd is distributed across all of your machines running CoreOS. For example, each of your app containers can announce itself to a proxy container, which would automatically know which machines should receive traffic. Building service discovery into your application allows you to add more machines and scale your services seamlessly.

The API is easy to use. From a CoreOS machine, you can simply use curl to set and retrieve a key from etcd:

Set a key message with value Hello world:

curl -L http://127.0.0.1:4001/v1/keys/message -d value="Hello world"

Read the value of message back:

curl -L http://127.0.0.1:4001/v1/keys/message

If you followed a guide to set up more than one CoreOS machine, you can SSH into another machine and can retrieve this same value.

More Detailed Information

View Complete Guide
Read etcd API Docs

Container Management with docker Copy Chapter

The second building block, docker ([docs][docker-docs]), is where your applications and code run. It is installed on each CoreOS machine. You should make each of your services (web server, caching, database) into a container and connect them together by reading and writing to etcd. You can quickly try out a Ubuntu container in two different ways:

Run a command in the container and then stop it:

docker run busybox /bin/echo hello world

Open a shell prompt inside of the container:

docker run -i -t busybox /bin/sh

More Detailed Information

View Complete Guide
Read docker Docs

Process Management with systemd Copy Chapter

The third buiding block of CoreOS is systemd ([docs][systemd-docs]) and it is installed on each CoreOS machine. You should use systemd to manage the life cycle of your docker containers. The configuration format for systemd is straightforward. In the example below, the Ubuntu container is set up to print text after each reboot:

First, you will need to run all of this as root since you are modifying system state:

sudo -i

Create a file called /media/state/units/hello.service

[Unit]
Description=My Service
After=docker.service

[Service]
Restart=always
ExecStart=/usr/bin/docker run busybox /bin/sh -c "while true; do echo Hello World; sleep 1; done"

[Install]
WantedBy=local.target

Then run systemctl restart local-enable.service to start all services wanted by local.target. This will start your container and log to the systemd journal. You can read the log by running:

journalctl -u hello.service -f

More Detailed Information

Read systemd Website

Chaos Monkey

During our alpha period, Chaos Monkey (i.e. random reboots) is built in and will give you plenty of opportunities to test out systemd. CoreOS machines will automatically reboot after an update is applied.