youki Documentation

File: README.md

youki: A container runtime in Rust

youki is an implementation of the OCI runtime-spec in Rust, similar to runc.
Your ideas are welcome here.

🏷️ About the name

youki is pronounced as /joʊki/ or yoh-key.
youki is named after the Japanese word 'youki', which means 'a container'. In Japanese language, youki also means 'cheerful', 'merry', or 'hilarious'.

🚀 Quick Start

[!TIP]
You can immediately set up your environment with youki on GitHub Codespaces and try it out.
console
just build
docker run --runtime youki hello-world
sudo podman run --cgroup-manager=cgroupfs --runtime /workspaces/youki/youki hello-world

User Documentation

🎯 Motivation

Here is why we are writing a new container runtime in Rust.

Rust is one of the best languages to implement the oci-runtime spec. Many very nice container tools are currently written in Go. However, the container runtime requires the use of system calls, which requires a bit of special handling when implemented in Go. This is tricky (e.g. namespaces(7), fork(2)); with Rust too, but it's not that tricky. And, unlike in C, Rust provides the benefit of memory safety. While Rust is not yet a major player in the container field, it has the potential to contribute a lot: something this project attempts to exemplify.

youki has the potential to be faster and use less memory than runc, and therefore work in environments with tight memory usage requirements. Here is a simple benchmark of a container from creation to deletion.

Runtime	Time (mean ± σ)	Range (min … max)	vs youki(mean)	Version
youki	111.5 ms ± 11.6 ms	84.0 ms ± 142.5 ms	100%	0.3.3
runc	224.6 ms ± 12.0 ms	190.5 ms ± 255.4 ms	200%	1.1.7
crun	47.3 ms ± 2.8 ms	42.4 ms ± 56.2 ms	42%	1.15

Details about the benchmark

A command used for the benchmark

bash

hyperfine --prepare 'sudo sync; echo 3 | sudo tee /proc/sys/vm/drop_caches' --warmup 10 --min-runs 100 'sudo ./youki create -b tutorial a && sudo ./youki start a && sudo ./youki delete -f a'

Environment

console

$ ./youki info
Version           0.3.3
Commit            4f3c8307
Kernel-Release    6.5.0-35-generic
Kernel-Version    #35~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Tue May  7 09:00:52 UTC 2
Architecture      x86_64
Operating System  Ubuntu 22.04.4 LTS
Cores             16
Total Memory      63870
Cgroup setup      unified
Cgroup mounts
Namespaces        enabled
  mount           enabled
  uts             enabled
  ipc             enabled
  user            enabled
  pid             enabled
  network         enabled
  cgroup          enabled
Capabilities
CAP_BPF           available
CAP_PERFMON       available
CAP_CHECKPOINT_RESTORE available

I have fun implementing this. In fact, this may be the most important.

📍 Status of youki

youki has aced real-world use cases, including containerd's e2e test, and is now adopted by several production environments. See Adopters and Use Cases for public examples.
We have our roadmap.

youki-dev/oci-spec-rs - OCI Runtime and Image Spec in Rust

🎨 Design and implementation of youki

The User and Developer Documentation for youki is hosted at https://youki-dev.github.io/youki/

🎬 Getting Started

Local build is only supported on Linux.
For other platforms, please use the Vagrantfile that we have prepared. You can also spin up a fully preconfigured development environment in the cloud with GitHub Codespaces.

Requires

Rust(See here), edition 2024
linux kernel ≥ 5.3

Dependencies

To install just, follow the instruction here.

console

sudo apt-get install    \
      pkg-config        \
      libsystemd-dev    \
      build-essential   \
      libelf-dev        \
      libseccomp-dev    \
      libclang-dev      \
      libssl-dev

console

sudo dnf install            \
      pkg-config            \
      systemd-devel         \
      elfutils-libelf-devel \
      libseccomp-devel      \
      clang-devel           \
      openssl-devel

Build

bash

git clone [email protected]:youki-dev/youki.git
cd youki
just youki-dev # or youki-release
./youki -h # you can get information about youki command

Tutorial

Requires

Docker(See here)

Create and run a container

Let's try to run a container that executes sleep 30 with youki. This tutorial may need root permission.

bash

git clone [email protected]:youki-dev/youki.git
cd youki
just youki-dev # or youki-release

mkdir -p tutorial/rootfs
cd tutorial
# use docker to export busybox into the rootfs directory
docker export $(docker create busybox) | tar -C rootfs -xvf -

Then, we need to prepare a configuration file. This file contains metadata and specs for a container, such as the process to run, environment variables to inject, sandboxing features to use, etc.

bash

../youki spec  # will generate a spec file named config.json

We can edit the config.json to add customized behaviors for container. Here, we modify the process field to run sleep 30.

json

"process": {
    ...
    "args": [
      "sleep", "30"
    ],

  ...
  }

Then we can explore the lifecycle of a container:

bash

cd ..                                                # go back to the repository root
sudo ./youki create -b tutorial tutorial_container   # create a container with name `tutorial_container`
sudo ./youki state tutorial_container                # you can see the state the container is `created`
sudo ./youki start tutorial_container                # start the container
sudo ./youki list                                    # will show the list of containers, the container is `running`
sudo ./youki delete tutorial_container               # delete the container

Change the command to be executed in config.json and try something other than sleep 30.

Rootless container

youki provides the ability to run containers as non-root user(rootless mode). To run a container in rootless mode, we need to add some extra options in config.json, other steps are same with above:

bash

mkdir -p tutorial/rootfs
cd tutorial
# use docker to export busybox into the rootfs directory
docker export $(docker create busybox) | tar -C rootfs -xvf -

../youki spec --rootless          # will generate a spec file named config.json with rootless mode
## Modify the `args` field as you like

../youki run rootless-container   # will create and run a container with rootless mode

Usage

Start the docker daemon.

bash

dockerd --experimental --add-runtime="youki=$(pwd)/youki"

If you get an error like the below, that means your normal Docker daemon is running, and it needs to be stopped. Do that with your init system (i.e., with systemd, run sudo systemctl stop docker, as root if necessary).

console

failed to start daemon: pid file found, ensure docker is not running or delete /var/run/docker.pid

Now repeat the command, which should start the docker daemon.

You can use youki in a different terminal to start the container.

bash

docker run -it --rm --runtime youki busybox

Afterwards, you can close the docker daemon process in other the other terminal. To restart normal docker daemon (if you had stopped it before), run:

bash

sudo systemctl start docker

Integration Tests

Go and node-tap are required to run integration tests. See the opencontainers/runtime-tools README for details.

bash

git submodule update --init --recursive
just test-oci

Setting up Vagrant

You can try youki on platforms other than Linux by using the Vagrantfile we have prepared. We have prepared two environments for vagrant, namely rootless mode and rootful mode

bash

git clone [email protected]:youki-dev/youki.git
cd youki

# If you want to develop in rootless mode, and this is the default mode
vagrant up default
vagrant ssh default

# or if you want to develop in rootful mode
vagrant up rootful
vagrant ssh rootful

# in virtual machine
cd youki
just youki-dev # or youki-release

👥 Community and Contributing

Please refer to our community page.

Thanks to all the people who already contributed!

We are a Cloud Native Computing Foundation sandbox project.

The Linux Foundation® (TLF) has registered trademarks and uses trademarks. For a list of TLF trademarks, see Trademark Usage.

License

File: crates/libcontainer/README.md

libcontainer

Building with musl

In order to build with musl you must first remove the libseccomp dependency as it will reference shared libraries (libseccomp) which cannot be built with musl.

Do this by using adding flags to Cargo. Use the --no-default-features flag followed by -F and whatever features you intend to build with such as v2 as defined in Cargo.toml under features section.

Next you will also need the +nightly flags when building with rustup and cargo.

bash

# Add rustup +nightly musl to toolchain
rustup +nightly target add $(uname -m)-unknown-linux-musl

# Build rustup +nightly stdlib with musl
rustup +nightly toolchain install nightly-$(uname -m)-unknown-linux-musl

# Build musl standard library
cargo +nightly build -Zbuild-std --target $(uname -m)-unknown-linux-musl --no-default-features -F v2

cargo +nightly build --target $(uname -m)-unknown-linux-musl --no-default-features -F v2

File: crates/liboci-cli/README.md

liboci-cli

This is a crate to parse command line arguments for OCI container
runtimes as specified in the OCI Runtime Command Line
Interface.

Implemented subcommands

Command	liboci-cli	CLI Specification	runc	crun	youki
create	✅	✅	✅	✅	✅
start	✅	✅	✅	✅	✅
state	✅	✅	✅	✅	✅
kill	✅	✅	✅	✅	✅
delete	✅	✅	✅	✅	✅
checkpoint			✅	✅
events	✅		✅		✅
exec	✅		✅	✅	✅
features	✅		✅
list	✅		✅	✅	✅
pause	✅		✅	✅	✅
ps	✅		✅	✅	✅
restore			✅	✅
resume	✅		✅	✅	✅
run	✅		✅	✅	✅
spec	✅		✅	✅	✅
update			✅	✅

File: docs/src/developer/e2e/containerd_integration_test_using_youki.md

containerd integration test using youki

local

console

just containerd-test

File: docs/src/developer/e2e/e2e_tests.md

e2e tests

There are various e2e tests:

rust oci tests

This is youki's original integration to verify the behavior of the low-level container runtime.
containerd integration test

This is the method that containerd's integration test runs with youki.
runtime tools

This is the method to run the runtime tools that OCI manages as a test tool to verify meeting the OCI Runtime Spec
Kubernetes test

This test verifies youki works as a Kubernetes container runtime using Kind.

File: docs/src/developer/e2e/integration_test.md

integration_test

Note that these tests resides in /tests/integration_test/ at the time of writing.

This crate contains the Rust port of OCI-runtime tools integration tests, which are used to test if the runtime works as per the OCI spec or not. Initially youki used the original implementation of these test provided in the OCI repository here. But those tests are written in Go, which made the developers depend on two language environments Rust and Go to compile youki and test it. The Validation tests themselves also have an optional dependency on node js to parse their output, which can make it a third language dependency.

Other than that, those tests also showed some issues while running on some local systems, and thus running the tests would be difficult on local system. As the runtime is a complex piece of software, it becomes useful to have a set of tests that can be run with changes in code, so one can verify that change in one part of youki has not accidentally broken some other part of youki.

Thus we decided to port the tests to Rust, and validate them, so that we have a set of unit tests as well of integration tests to validate the working of runtime. These tests are still under development, and you can check the tracking issue for more details. More details on working of these tests can be found at https://github.com/youki-dev/youki/tree/main/crates/integration_test.

As these tests are under development, these are validated on a standard runtime such as runc in the GitHub CI, so validate the tests themselves.

Notes

About the create container function

The test_utils provides a create_container function which can be used to run the youki create command. It returns the child process struct, which can be either wait() or wait_with_output() to wait for it finishing. Unless you know what are you doing, it is recommended to call wait() on it, as otherwise the process will hang. As explained in the youki docs , the youki create process, after starting forks, and the forked process keeps waiting for another youki process to send it the start signal , and after receiving it, that forked process execs the container program. If you are simply trying to create a container, such as in case of test_outside_runtime then calling wait_with_output() will cause it to hang. If you are actually going to start a container, and need output from the container process, then you must keep the Child struct returned by create function and call wait_with_output() on it AFTER you have called the start command on that container, which will give you the stdout and stderr of the process running inside the container.

To understand how this works, take a look at handling stdio of the runc, specially the detached pass-through mode section. As explained in it, we setup the stdio for the original youki process in youki create by setting the stdio to Stdio::piped() in the create function. Then we set the terminal option to false (which is the default anyway) in the spec, which makes it run in the pass-through mode. Then when the create process is done its work, and its forked process is waiting for the start signal, it uses the same stdio pipes. Thus calling wait_with_output() without starting will keep it hanged up, and after calling start, stdio of the program to be run inside the container can be obtained from the youki create's process.

How test inside container works

We use test_inside_container for making sure that the restrictions and constraints are uphold from inside the container process.
For that, first whichever integration test needs to use it, must define the runtimetest as the container process in the spec, and then use test_inside_container function. It requires a function which will do the necessary setup for the tests that are to be run inside. Then the counterpart for the test should be added to the runtimetest crate, which will run inside the container and if there is any error, print it to the stderr. The test_inside_container function will wait for the tests to be over and then check the stderr to be empty. If it is not, the test is assumed to fail.

File: docs/src/developer/e2e/runtime_tools.md

runtime tools

local

console

$ git submodule update --init --recursive
$ just test-oci

File: docs/src/developer/e2e/runtimetest.md

Runtime Test

Note that these tests resides in /tests/runtimetest at the time of writing.

This crate provides a binary which is used by integration tests to verify that the restrictions and constraints applied to the container are upheld by the container process, from inside the container process. This runs the tests one-by-one, and the failing test prints the error to the stderr.

Notes

This binary must be compiled with the option of static linking to crt0 given to the rustc. If compiled without it, it will add a linking to /lib64/ld-linux-x86-64.so . The binary compiled this way cannot be run inside the container process, as they do not have access to /lib64/... Thus the runtime test must be statically linked to crt0.

While developing, originally this was added to the common workspace of all crates in youki. But then it was realized that this was quite inefficient because :

All packages except runtimetest will be compiled with dynamic linking
Runtimetest will be compiled with static linking

Now runtimetest needs at least oci-spec and nix package for its operations, which are also dependencies of other packages in the workspace. Thus both of these, and recursively their dependencies must be compiled twice, each time, once for dynamic linking and once for static. The took a long time in the compilation stage, especially when developing / adding new tests. Separating runtimetest from the workspace allows it to have a separate target/ directory, where it can store the statically compiled dependencies, and the workspace can have its target/ directory, where it can store its dynamically compiled dependencies. That way only the crates which have changes need to be compiled (runtimetest or integration test), and not their dependencies.

In case in future this separation is not required, or some other configuration is chosen, make sure the multiple compilation issue does not arise, or the advantages of new method outweigh the time spent in double compilation.

To see if a binary can be run inside the container process, run

console

readelf -l path/to/binary |grep "program interpreter"

[Requesting program interpreter: /lib64/ld-linux-x86-64.so.2] means that the binary is not statically linked, and cannot be run inside the container process. If the above command gives no output, that means it does not require any program interpreter and can be run inside the container.

Another way is to run

console

file path/to/binary

console

./youki: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=...., for GNU/Linux 3.2.0, with debug_info, not stripped`

This output indicates that the binary is dynamically linked, thus cannot be run inside the container process

console

./runtimetest: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, BuildID[sha1]=...., for GNU/Linux 3.2.0, with debug_info, not stripped

This output indicates that the binary is statically linked, and can be run inside the container process

Some links to help :

File: docs/src/developer/e2e/rust_oci_test.md

Contest

This is youki's original integration to verify the behavior of the low-level container runtime.

How to run

console

just test-contest

How to write

We will not go into detail here, but will explain how to write and add a new test case based on an example test.

Fully the code of the example test

rust,no_run,noplayground

{{#include ../../../../tests/contest/contest/src/tests/example/hello_world.rs}}

Build the OCI Runtime Spec you want to verify

This testing framework automatically places runtimetest in the container.
In other words, you can test the processes you want to execute within a container by writing them in runtimetest.
Therefore, it is common practice here to write an OCI Runtime Spec that executes runtimetest.

rust,no_run,noplayground

{{#include ../../../../tests/contest/contest/src/tests/example/hello_world.rs:get_example_spec}}

Prepare a function that returns a TestResult, which represents the result of the test.

rust,no_run,noplayground

{{#include ../../../../tests/contest/contest/src/tests/example/hello_world.rs:example_test}}

Create a TestGroup and register a test case you created

rust,no_run,noplayground

{{#include ../../../../tests/contest/contest/src/tests/example/hello_world.rs:get_example_test}}

rust,no_run,noplayground

{{#include ../../../../tests/contest/contest/src/main.rs:register_example_test}}

Write the validation you want to run within a test container

rust,no_run,noplayground

{{#include ../../../../tests/contest/runtimetest/src/main.rs:example_runtimetest_main}}

rust,no_run,noplayground

{{#include ../../../../tests/contest/runtimetest/src/tests.rs:example_hello_world}}

Docker & Self-Hosted Deployment Verified OCI

File: README.md Copy Chapter

youki: A container runtime in Rust

🏷️ About the name

🚀 Quick Start

🎯 Motivation

📍 Status of youki

🔗 Related project

🎨 Design and implementation of youki

🎬 Getting Started

Requires Copy Chapter

Dependencies Copy Chapter

Debian, Ubuntu and related distributions Copy Section

Fedora, CentOS, RHEL and related distributions Copy Section

Build Copy Chapter

Tutorial Copy Chapter

Requires Copy Section

Create and run a container Copy Section

Rootless container Copy Section

Usage Copy Chapter

Integration Tests Copy Section

Setting up Vagrant Copy Section

👥 Community and Contributing

License Copy Chapter

File: crates/libcontainer/README.md Copy Chapter

libcontainer

Building with musl Copy Section

File: crates/liboci-cli/README.md Copy Chapter

liboci-cli

Implemented subcommands Copy Chapter

File: docs/src/developer/e2e/containerd_integration_test_using_youki.md Copy Chapter

containerd integration test using youki

local Copy Chapter

File: docs/src/developer/e2e/e2e_tests.md Copy Chapter

e2e tests

File: docs/src/developer/e2e/integration_test.md Copy Chapter

integration_test

Notes Copy Chapter

About the create container function Copy Section

How test inside container works Copy Section

File: docs/src/developer/e2e/runtime_tools.md Copy Chapter

runtime tools

local Copy Chapter

File: docs/src/developer/e2e/runtimetest.md Copy Chapter

Runtime Test

Notes Copy Chapter

File: docs/src/developer/e2e/rust_oci_test.md Copy Chapter

Contest

How to run

How to write

Explore Related Technologies & Topics

File: README.md

Requires

Dependencies

Debian, Ubuntu and related distributions

Fedora, CentOS, RHEL and related distributions

Build

Tutorial

Requires

Create and run a container

Rootless container

Usage

Integration Tests

Setting up Vagrant

License

File: crates/libcontainer/README.md

Building with musl

File: crates/liboci-cli/README.md

Implemented subcommands

File: docs/src/developer/e2e/containerd_integration_test_using_youki.md

local

File: docs/src/developer/e2e/e2e_tests.md

File: docs/src/developer/e2e/integration_test.md

Notes

About the create container function

How test inside container works

File: docs/src/developer/e2e/runtime_tools.md

local

File: docs/src/developer/e2e/runtimetest.md

Notes

File: docs/src/developer/e2e/rust_oci_test.md