{"owner":"firecracker-microvm","repo":"firecracker","hasSkills":true,"totalSkillsCount":1,"totalTokensCount":2202,"categories":["project-spec"],"hasMcp":false,"mcpConfig":null,"found":["docs/design.md"],"skills":{"docs/design.md":"# Firecracker Design\n\n## Scope\n\n### What is Firecracker\n\nFirecracker is a new virtualization technology that enables customers to deploy\nlightweight *micro* Virtual Machines or microVMs. Firecracker microVMs combine\nthe security and workload isolation properties of traditional VMs with the\nspeed, agility and resource efficiency enabled by containers. They provide a\nsecure, trusted environment for multi-tenant services, while maintaining minimal\noverhead.\n\nThe scope of this document is to describe the features and architecture of the\nFirecracker virtual machine manager (VMM).\n\n### Features\n\n1. Firecracker can safely run workloads from different customers on the same\n   machine.\n1. Customers can create microVMs with any combination of vCPU (up to 32) and\n   memory to match their application requirements.\n1. Firecracker microVMs can oversubscribe host CPU and memory. The degree of\n   oversubscription is controlled by customers, who may factor in workload\n   correlation and load in order to ensure smooth host system operation.\n1. With a microVM configured with a minimal Linux kernel, single-core CPU, and\n   128 MiB of RAM, Firecracker supports a steady mutation rate of 5 microVMs per\n   host core per second (e.g., one can create 180 microVMs per second on a host\n   with 36 physical cores).\n1. The number of Firecracker microVMs running simultaneously on a host is\n   limited only by the availability of hardware resources.\n1. Each microVM exposes a host-facing API via an in-process HTTP server.\n1. Each microVM provides guest-facing access to host-configured metadata via the\n   `/mmds` API.\n\n### Specifications\n\nFirecracker's technical specifications are available in the\n[Specifications document](../SPECIFICATION.md).\n\n## Host Integration\n\nThe following diagram depicts an example host running Firecracker microVMs.\n\n![Firecracker Host Integration](images/firecracker_host_integration.png?raw=true \"Firecracker Host Integration\")\n\nFirecracker runs on Linux hosts and with Linux guest OSs (from this point on,\nreferred to as guests). For a complete list of currently supported kernel\nversions, check out the [kernel support policy](kernel-policy.md).\n\nIn production environments, Firecracker should be started only via the `jailer`\nbinary. See [Sandboxing](#Sandboxing) for more details.\n\nAfter launching the process, users interact with the Firecracker API to\nconfigure the microVM, before issuing the `InstanceStart` command.\n\n### Host Networking Integration\n\nFirecracker emulated network devices are backed by TAP devices on the host. To\nmake use of Firecracker, we expect our customers to leverage on-host networking\nsolutions.\n\n### Storage\n\nFirecracker emulated block devices are backed by files on the host. To be able\nto mount block devices in the guest, the backing files need to be pre-formatted\nwith a filesystem that the guest kernel supports.\n\n## Internal Architecture\n\nEach Firecracker process encapsulates one and only one microVM. The process runs\nthe following threads: API, VMM and vCPU(s). The API thread is responsible for\nFirecracker's API server and associated control plane. It's never in the fast\npath of the virtual machine. The VMM thread exposes the machine model, minimal\nlegacy device model, microVM metadata service (MMDS) and VirtIO device emulated\nNet, Block and Vsock devices, complete with I/O rate limiting. In addition to\nthem, there are one or more vCPU threads (one per guest CPU core). They are\ncreated via KVM and run the `KVM_RUN` main loop. They execute synchronous I/O\nand memory-mapped I/O operations on devices models.\n\nAll device emulation on the VMM thread is driven by a single epoll-based event\nloop, which also picks up the requests forwarded by the API thread. When the\nmicroVM is paused through the API, the VMM thread stops polling that event loop\nand only serves API requests until it is resumed. A `Paused` microVM therefore\nhas neither its vCPUs nor its device models running (see\n[snapshot support](snapshotting/snapshot-support.md#pausing-the-microvm)).\n\n### Threat Containment\n\nFrom a security perspective, all vCPU threads are considered to be running\nmalicious code as soon as they have been started; these malicious threads need\nto be contained. Containment is achieved by nesting several trust zones which\nincrement from least trusted or least safe (guest vCPU threads) to most trusted\nor safest (host). These trusted zones are separated by barriers that enforce\naspects of Firecracker security. For example, all outbound network traffic data\nis copied by the Firecracker I/O thread from the emulated network interface to\nthe backing host TAP device, and I/O rate limiting is applied at this point.\nThese barriers are marked in the diagram below.\n\nFirecracker does not perform any network traffic filtering. All egress traffic\nfrom a guest is therefore considered untrusted, and should be filtered at the\nhost-level.\n\n![Firecracker Threat Containment](images/firecracker_threat_containment.png?raw=true \"Firecracker Threat Containment\")\n\n## Components and Features\n\n### Machine Model\n\n#### Layout\n\nFirecracker provides guests with storage and network access via emulated VirtIO\nNet and VirtIO Block devices. It also exposes a serial console and partial\nkeyboard controller, the latter being used by guests to reset the VM (either\nsoft or hard reset). Within Firecracker, the purpose of the I8042 device is to\nsignal the microVM that the guest has requested a reboot.\n\nIn addition to the Firecracker provided device models, guests also see the\nProgrammable Interrupt Controllers (PICs), the I/O Advanced Programmable\nInterrupt Controller (IOAPIC), and the Programmable Interval Timer (PIT) that\nKVM supports.\n\n#### Exposing the CPU to the guest\n\nFirecracker allows control of what processor information is exposed to the guest\nby using [CPU templates](cpu_templates/cpu-templates.md). CPU templates can be\nset via the Firecracker API. Users can choose from existing static CPU templates\nand/or creating their own custom CPU templates.\n\n#### Clocksources available to guests\n\nFirecracker exposes the following clock sources to guests:\n\n- x86_64: kvm-clock and tsc. Linux guests >=5.10 will pick tsc by default if\n  stable.\n- aarch64: arch_sys_counter\n\n### I/O: Storage, Networking and Rate Limiting\n\nFirecracker provides VirtIO/block and VirtIO/net emulated devices, along with\nthe application of rate limiters to each volume and network interface to make\nsure host hardware resources are used fairly by multiple microVMs. These are\nimplemented using a token bucket algorithm based on two buckets. One is\nassociated with the number of operations per second and the other one with the\nbandwidth. The customer can create and configure rate limiters via the API by\nspecifying token bucket configurations for ingress and egress. Each token bucket\nis defined via the bucket size, I/O cost, refill rate, maximum burst, and\ninitial value. This enables the customer to define flexible rate limiters that\nsupport bursts or specific bandwidth/operations limitations. For vhost-user\ndevices, customers should implement rate limiting on the side of the vhost-user\nbackend that they provide.\n\n### MicroVM Metadata Service\n\nFirecracker microVMs expose access to a minimal MicroVM-Metadata Service (MMDS)\nto the guest through the API endpoint. The metadata stored by the service is\nfully configured by users.\n\n### Sandboxing\n\n#### __Firecracker process__\n\nFirecracker is designed to assure secure isolation using multiple layers. The\nfirst layer of isolation is provided by the Linux KVM and the Firecracker\nvirtualization boundary. To assure defense in depth, Firecracker should only run\nconstrained at the process level. This is achieved by the following: seccomp\nfilters for disallowing unwanted system calls, cgroups and namespaces for\nresource isolation, and dropping privileges by jailing the process. Seccomp\nfilters are automatically installed by Firecracker, while for the latter, we\nrecommend starting Firecracker with the `jailer` binary that's part of each\nFirecracker release.\n\n##### Seccomp\n\nSeccomp filters are used by default to limit the host system calls Firecracker\ncan use. The default filters only allow the bare minimum set of system calls and\nparameters that Firecracker needs in order to function correctly.\n\nThe filters are loaded in the Firecracker process, on a per-thread basis, before\nexecuting any guest code.\n\nFor more information, check out the [seccomp documentation](seccomp.md).\n\n#### __Jailer process__\n\nThe Firecracker process can be started by another `jailer` process. The jailer\nsets up system resources that require elevated permissions (e.g., cgroup,\nchroot), drops privileges, and then exec()s into the Firecracker binary, which\nthen runs as an unprivileged process. Past this point, Firecracker can only\naccess resources that a privileged third-party grants access to (e.g., by\ncopying a file into the chroot, or passing a file descriptor).\n\n##### Cgroups and Quotas\n\nEach Firecracker microVM can be further encapsulated into a cgroup. By setting\nthe affinity of the Firecracker microVM to a node via the cpuset subsystem, one\ncan prevent the migration of said microVM from one node to another, something\nthat would impair performance and cause unnecessary contention on shared\nresources. In addition to setting the affinity, each Firecracker microVM can\nhave its own dedicated quota of the CPU time via the cpu subsystem, thus\nguaranteeing that resources are fairly shared across Firecracker microVMs.\n\n### Monitoring\n\nFirecracker emits logs and metric counters, each on a named pipe that is passed\nvia the API. Logs are flushed line by line, whereas metrics are emitted when the\ninstance starts, then every 60 seconds while it's running, and on panic.\nFirecracker customers are responsible for collecting data in the Firecracker log\nfiles. In production builds, Firecracker does not expose the serial console\nport, since it may contain guest data that the host should not see.\n"},"files":{"docs/design.md":"# Firecracker Design\n\n## Scope\n\n### What is Firecracker\n\nFirecracker is a new virtualization technology that enables customers to deploy\nlightweight *micro* Virtual Machines or microVMs. Firecracker microVMs combine\nthe security and workload isolation properties of traditional VMs with the\nspeed, agility and resource efficiency enabled by containers. They provide a\nsecure, trusted environment for multi-tenant services, while maintaining minimal\noverhead.\n\nThe scope of this document is to describe the features and architecture of the\nFirecracker virtual machine manager (VMM).\n\n### Features\n\n1. Firecracker can safely run workloads from different customers on the same\n   machine.\n1. Customers can create microVMs with any combination of vCPU (up to 32) and\n   memory to match their application requirements.\n1. Firecracker microVMs can oversubscribe host CPU and memory. The degree of\n   oversubscription is controlled by customers, who may factor in workload\n   correlation and load in order to ensure smooth host system operation.\n1. With a microVM configured with a minimal Linux kernel, single-core CPU, and\n   128 MiB of RAM, Firecracker supports a steady mutation rate of 5 microVMs per\n   host core per second (e.g., one can create 180 microVMs per second on a host\n   with 36 physical cores).\n1. The number of Firecracker microVMs running simultaneously on a host is\n   limited only by the availability of hardware resources.\n1. Each microVM exposes a host-facing API via an in-process HTTP server.\n1. Each microVM provides guest-facing access to host-configured metadata via the\n   `/mmds` API.\n\n### Specifications\n\nFirecracker's technical specifications are available in the\n[Specifications document](../SPECIFICATION.md).\n\n## Host Integration\n\nThe following diagram depicts an example host running Firecracker microVMs.\n\n![Firecracker Host Integration](images/firecracker_host_integration.png?raw=true \"Firecracker Host Integration\")\n\nFirecracker runs on Linux hosts and with Linux guest OSs (from this point on,\nreferred to as guests). For a complete list of currently supported kernel\nversions, check out the [kernel support policy](kernel-policy.md).\n\nIn production environments, Firecracker should be started only via the `jailer`\nbinary. See [Sandboxing](#Sandboxing) for more details.\n\nAfter launching the process, users interact with the Firecracker API to\nconfigure the microVM, before issuing the `InstanceStart` command.\n\n### Host Networking Integration\n\nFirecracker emulated network devices are backed by TAP devices on the host. To\nmake use of Firecracker, we expect our customers to leverage on-host networking\nsolutions.\n\n### Storage\n\nFirecracker emulated block devices are backed by files on the host. To be able\nto mount block devices in the guest, the backing files need to be pre-formatted\nwith a filesystem that the guest kernel supports.\n\n## Internal Architecture\n\nEach Firecracker process encapsulates one and only one microVM. The process runs\nthe following threads: API, VMM and vCPU(s). The API thread is responsible for\nFirecracker's API server and associated control plane. It's never in the fast\npath of the virtual machine. The VMM thread exposes the machine model, minimal\nlegacy device model, microVM metadata service (MMDS) and VirtIO device emulated\nNet, Block and Vsock devices, complete with I/O rate limiting. In addition to\nthem, there are one or more vCPU threads (one per guest CPU core). They are\ncreated via KVM and run the `KVM_RUN` main loop. They execute synchronous I/O\nand memory-mapped I/O operations on devices models.\n\nAll device emulation on the VMM thread is driven by a single epoll-based event\nloop, which also picks up the requests forwarded by the API thread. When the\nmicroVM is paused through the API, the VMM thread stops polling that event loop\nand only serves API requests until it is resumed. A `Paused` microVM therefore\nhas neither its vCPUs nor its device models running (see\n[snapshot support](snapshotting/snapshot-support.md#pausing-the-microvm)).\n\n### Threat Containment\n\nFrom a security perspective, all vCPU threads are considered to be running\nmalicious code as soon as they have been started; these malicious threads need\nto be contained. Containment is achieved by nesting several trust zones which\nincrement from least trusted or least safe (guest vCPU threads) to most trusted\nor safest (host). These trusted zones are separated by barriers that enforce\naspects of Firecracker security. For example, all outbound network traffic data\nis copied by the Firecracker I/O thread from the emulated network interface to\nthe backing host TAP device, and I/O rate limiting is applied at this point.\nThese barriers are marked in the diagram below.\n\nFirecracker does not perform any network traffic filtering. All egress traffic\nfrom a guest is therefore considered untrusted, and should be filtered at the\nhost-level.\n\n![Firecracker Threat Containment](images/firecracker_threat_containment.png?raw=true \"Firecracker Threat Containment\")\n\n## Components and Features\n\n### Machine Model\n\n#### Layout\n\nFirecracker provides guests with storage and network access via emulated VirtIO\nNet and VirtIO Block devices. It also exposes a serial console and partial\nkeyboard controller, the latter being used by guests to reset the VM (either\nsoft or hard reset). Within Firecracker, the purpose of the I8042 device is to\nsignal the microVM that the guest has requested a reboot.\n\nIn addition to the Firecracker provided device models, guests also see the\nProgrammable Interrupt Controllers (PICs), the I/O Advanced Programmable\nInterrupt Controller (IOAPIC), and the Programmable Interval Timer (PIT) that\nKVM supports.\n\n#### Exposing the CPU to the guest\n\nFirecracker allows control of what processor information is exposed to the guest\nby using [CPU templates](cpu_templates/cpu-templates.md). CPU templates can be\nset via the Firecracker API. Users can choose from existing static CPU templates\nand/or creating their own custom CPU templates.\n\n#### Clocksources available to guests\n\nFirecracker exposes the following clock sources to guests:\n\n- x86_64: kvm-clock and tsc. Linux guests >=5.10 will pick tsc by default if\n  stable.\n- aarch64: arch_sys_counter\n\n### I/O: Storage, Networking and Rate Limiting\n\nFirecracker provides VirtIO/block and VirtIO/net emulated devices, along with\nthe application of rate limiters to each volume and network interface to make\nsure host hardware resources are used fairly by multiple microVMs. These are\nimplemented using a token bucket algorithm based on two buckets. One is\nassociated with the number of operations per second and the other one with the\nbandwidth. The customer can create and configure rate limiters via the API by\nspecifying token bucket configurations for ingress and egress. Each token bucket\nis defined via the bucket size, I/O cost, refill rate, maximum burst, and\ninitial value. This enables the customer to define flexible rate limiters that\nsupport bursts or specific bandwidth/operations limitations. For vhost-user\ndevices, customers should implement rate limiting on the side of the vhost-user\nbackend that they provide.\n\n### MicroVM Metadata Service\n\nFirecracker microVMs expose access to a minimal MicroVM-Metadata Service (MMDS)\nto the guest through the API endpoint. The metadata stored by the service is\nfully configured by users.\n\n### Sandboxing\n\n#### __Firecracker process__\n\nFirecracker is designed to assure secure isolation using multiple layers. The\nfirst layer of isolation is provided by the Linux KVM and the Firecracker\nvirtualization boundary. To assure defense in depth, Firecracker should only run\nconstrained at the process level. This is achieved by the following: seccomp\nfilters for disallowing unwanted system calls, cgroups and namespaces for\nresource isolation, and dropping privileges by jailing the process. Seccomp\nfilters are automatically installed by Firecracker, while for the latter, we\nrecommend starting Firecracker with the `jailer` binary that's part of each\nFirecracker release.\n\n##### Seccomp\n\nSeccomp filters are used by default to limit the host system calls Firecracker\ncan use. The default filters only allow the bare minimum set of system calls and\nparameters that Firecracker needs in order to function correctly.\n\nThe filters are loaded in the Firecracker process, on a per-thread basis, before\nexecuting any guest code.\n\nFor more information, check out the [seccomp documentation](seccomp.md).\n\n#### __Jailer process__\n\nThe Firecracker process can be started by another `jailer` process. The jailer\nsets up system resources that require elevated permissions (e.g., cgroup,\nchroot), drops privileges, and then exec()s into the Firecracker binary, which\nthen runs as an unprivileged process. Past this point, Firecracker can only\naccess resources that a privileged third-party grants access to (e.g., by\ncopying a file into the chroot, or passing a file descriptor).\n\n##### Cgroups and Quotas\n\nEach Firecracker microVM can be further encapsulated into a cgroup. By setting\nthe affinity of the Firecracker microVM to a node via the cpuset subsystem, one\ncan prevent the migration of said microVM from one node to another, something\nthat would impair performance and cause unnecessary contention on shared\nresources. In addition to setting the affinity, each Firecracker microVM can\nhave its own dedicated quota of the CPU time via the cpu subsystem, thus\nguaranteeing that resources are fairly shared across Firecracker microVMs.\n\n### Monitoring\n\nFirecracker emits logs and metric counters, each on a named pipe that is passed\nvia the API. Logs are flushed line by line, whereas metrics are emitted when the\ninstance starts, then every 60 seconds while it's running, and on panic.\nFirecracker customers are responsible for collecting data in the Firecracker log\nfiles. In production builds, Firecracker does not expose the serial console\nport, since it may contain guest data that the host should not see.\n"},"items":[{"name":"design.md","path":"docs/design.md","rawUrl":"https://raw.githubusercontent.com/firecracker-microvm/firecracker/HEAD/docs/design.md","title":"docs - UI Design System & Style Tokens","category":"project-spec","format":"markdown","content":"# Firecracker Design\n\n## Scope\n\n### What is Firecracker\n\nFirecracker is a new virtualization technology that enables customers to deploy\nlightweight *micro* Virtual Machines or microVMs. Firecracker microVMs combine\nthe security and workload isolation properties of traditional VMs with the\nspeed, agility and resource efficiency enabled by containers. They provide a\nsecure, trusted environment for multi-tenant services, while maintaining minimal\noverhead.\n\nThe scope of this document is to describe the features and architecture of the\nFirecracker virtual machine manager (VMM).\n\n### Features\n\n1. Firecracker can safely run workloads from different customers on the same\n   machine.\n1. Customers can create microVMs with any combination of vCPU (up to 32) and\n   memory to match their application requirements.\n1. Firecracker microVMs can oversubscribe host CPU and memory. The degree of\n   oversubscription is controlled by customers, who may factor in workload\n   correlation and load in order to ensure smooth host system operation.\n1. With a microVM configured with a minimal Linux kernel, single-core CPU, and\n   128 MiB of RAM, Firecracker supports a steady mutation rate of 5 microVMs per\n   host core per second (e.g., one can create 180 microVMs per second on a host\n   with 36 physical cores).\n1. The number of Firecracker microVMs running simultaneously on a host is\n   limited only by the availability of hardware resources.\n1. Each microVM exposes a host-facing API via an in-process HTTP server.\n1. Each microVM provides guest-facing access to host-configured metadata via the\n   `/mmds` API.\n\n### Specifications\n\nFirecracker's technical specifications are available in the\n[Specifications document](../SPECIFICATION.md).\n\n## Host Integration\n\nThe following diagram depicts an example host running Firecracker microVMs.\n\n![Firecracker Host Integration](images/firecracker_host_integration.png?raw=true \"Firecracker Host Integration\")\n\nFirecracker runs on Linux hosts and with Linux guest OSs (from this point on,\nreferred to as guests). For a complete list of currently supported kernel\nversions, check out the [kernel support policy](kernel-policy.md).\n\nIn production environments, Firecracker should be started only via the `jailer`\nbinary. See [Sandboxing](#Sandboxing) for more details.\n\nAfter launching the process, users interact with the Firecracker API to\nconfigure the microVM, before issuing the `InstanceStart` command.\n\n### Host Networking Integration\n\nFirecracker emulated network devices are backed by TAP devices on the host. To\nmake use of Firecracker, we expect our customers to leverage on-host networking\nsolutions.\n\n### Storage\n\nFirecracker emulated block devices are backed by files on the host. To be able\nto mount block devices in the guest, the backing files need to be pre-formatted\nwith a filesystem that the guest kernel supports.\n\n## Internal Architecture\n\nEach Firecracker process encapsulates one and only one microVM. The process runs\nthe following threads: API, VMM and vCPU(s). The API thread is responsible for\nFirecracker's API server and associated control plane. It's never in the fast\npath of the virtual machine. The VMM thread exposes the machine model, minimal\nlegacy device model, microVM metadata service (MMDS) and VirtIO device emulated\nNet, Block and Vsock devices, complete with I/O rate limiting. In addition to\nthem, there are one or more vCPU threads (one per guest CPU core). They are\ncreated via KVM and run the `KVM_RUN` main loop. They execute synchronous I/O\nand memory-mapped I/O operations on devices models.\n\nAll device emulation on the VMM thread is driven by a single epoll-based event\nloop, which also picks up the requests forwarded by the API thread. When the\nmicroVM is paused through the API, the VMM thread stops polling that event loop\nand only serves API requests until it is resumed. A `Paused` microVM therefore\nhas neither its vCPUs nor its device models running (see\n[snapshot support](snapshotting/snapshot-support.md#pausing-the-microvm)).\n\n### Threat Containment\n\nFrom a security perspective, all vCPU threads are considered to be running\nmalicious code as soon as they have been started; these malicious threads need\nto be contained. Containment is achieved by nesting several trust zones which\nincrement from least trusted or least safe (guest vCPU threads) to most trusted\nor safest (host). These trusted zones are separated by barriers that enforce\naspects of Firecracker security. For example, all outbound network traffic data\nis copied by the Firecracker I/O thread from the emulated network interface to\nthe backing host TAP device, and I/O rate limiting is applied at this point.\nThese barriers are marked in the diagram below.\n\nFirecracker does not perform any network traffic filtering. All egress traffic\nfrom a guest is therefore considered untrusted, and should be filtered at the\nhost-level.\n\n![Firecracker Threat Containment](images/firecracker_threat_containment.png?raw=true \"Firecracker Threat Containment\")\n\n## Components and Features\n\n### Machine Model\n\n#### Layout\n\nFirecracker provides guests with storage and network access via emulated VirtIO\nNet and VirtIO Block devices. It also exposes a serial console and partial\nkeyboard controller, the latter being used by guests to reset the VM (either\nsoft or hard reset). Within Firecracker, the purpose of the I8042 device is to\nsignal the microVM that the guest has requested a reboot.\n\nIn addition to the Firecracker provided device models, guests also see the\nProgrammable Interrupt Controllers (PICs), the I/O Advanced Programmable\nInterrupt Controller (IOAPIC), and the Programmable Interval Timer (PIT) that\nKVM supports.\n\n#### Exposing the CPU to the guest\n\nFirecracker allows control of what processor information is exposed to the guest\nby using [CPU templates](cpu_templates/cpu-templates.md). CPU templates can be\nset via the Firecracker API. Users can choose from existing static CPU templates\nand/or creating their own custom CPU templates.\n\n#### Clocksources available to guests\n\nFirecracker exposes the following clock sources to guests:\n\n- x86_64: kvm-clock and tsc. Linux guests >=5.10 will pick tsc by default if\n  stable.\n- aarch64: arch_sys_counter\n\n### I/O: Storage, Networking and Rate Limiting\n\nFirecracker provides VirtIO/block and VirtIO/net emulated devices, along with\nthe application of rate limiters to each volume and network interface to make\nsure host hardware resources are used fairly by multiple microVMs. These are\nimplemented using a token bucket algorithm based on two buckets. One is\nassociated with the number of operations per second and the other one with the\nbandwidth. The customer can create and configure rate limiters via the API by\nspecifying token bucket configurations for ingress and egress. Each token bucket\nis defined via the bucket size, I/O cost, refill rate, maximum burst, and\ninitial value. This enables the customer to define flexible rate limiters that\nsupport bursts or specific bandwidth/operations limitations. For vhost-user\ndevices, customers should implement rate limiting on the side of the vhost-user\nbackend that they provide.\n\n### MicroVM Metadata Service\n\nFirecracker microVMs expose access to a minimal MicroVM-Metadata Service (MMDS)\nto the guest through the API endpoint. The metadata stored by the service is\nfully configured by users.\n\n### Sandboxing\n\n#### __Firecracker process__\n\nFirecracker is designed to assure secure isolation using multiple layers. The\nfirst layer of isolation is provided by the Linux KVM and the Firecracker\nvirtualization boundary. To assure defense in depth, Firecracker should only run\nconstrained at the process level. This is achieved by the following: seccomp\nfilters for disallowing unwanted system calls, cgroups and namespaces for\nresource isolation, and dropping privileges by jailing the process. Seccomp\nfilters are automatically installed by Firecracker, while for the latter, we\nrecommend starting Firecracker with the `jailer` binary that's part of each\nFirecracker release.\n\n##### Seccomp\n\nSeccomp filters are used by default to limit the host system calls Firecracker\ncan use. The default filters only allow the bare minimum set of system calls and\nparameters that Firecracker needs in order to function correctly.\n\nThe filters are loaded in the Firecracker process, on a per-thread basis, before\nexecuting any guest code.\n\nFor more information, check out the [seccomp documentation](seccomp.md).\n\n#### __Jailer process__\n\nThe Firecracker process can be started by another `jailer` process. The jailer\nsets up system resources that require elevated permissions (e.g., cgroup,\nchroot), drops privileges, and then exec()s into the Firecracker binary, which\nthen runs as an unprivileged process. Past this point, Firecracker can only\naccess resources that a privileged third-party grants access to (e.g., by\ncopying a file into the chroot, or passing a file descriptor).\n\n##### Cgroups and Quotas\n\nEach Firecracker microVM can be further encapsulated into a cgroup. By setting\nthe affinity of the Firecracker microVM to a node via the cpuset subsystem, one\ncan prevent the migration of said microVM from one node to another, something\nthat would impair performance and cause unnecessary contention on shared\nresources. In addition to setting the affinity, each Firecracker microVM can\nhave its own dedicated quota of the CPU time via the cpu subsystem, thus\nguaranteeing that resources are fairly shared across Firecracker microVMs.\n\n### Monitoring\n\nFirecracker emits logs and metric counters, each on a named pipe that is passed\nvia the API. Logs are flushed line by line, whereas metrics are emitted when the\ninstance starts, then every 60 seconds while it's running, and on panic.\nFirecracker customers are responsible for collecting data in the Firecracker log\nfiles. In production builds, Firecracker does not expose the serial console\nport, since it may contain guest data that the host should not see.\n","isInternal":false,"tokens":2202,"sizeBytes":9988}],"systemPromptSnippet":"<yakaai_skills repo=\"firecracker-microvm/firecracker\">\n<!-- File: docs/design.md (Tokens: ~2202 | Category: project-spec) -->\n# Firecracker Design\n\n## Scope\n\n### What is Firecracker\n\nFirecracker is a new virtualization technology that enables customers to deploy\nlightweight *micro* Virtual Machines or microVMs. Firecracker microVMs combine\nthe security and workload isolation properties of traditional VMs with the\nspeed, agility and resource efficiency enabled by containers. They provide a\nsecure, trusted environment for multi-tenant services, while maintaining minimal\noverhead.\n\nThe scope of this document is to describe the features and architecture of the\nFirecracker virtual machine manager (VMM).\n\n### Features\n\n1. Firecracker can safely run workloads from different customers on the same\n   machine.\n1. Customers can create microVMs with any combination of vCPU (up to 32) and\n   memory to match their application requirements.\n1. Firecracker microVMs can oversubscribe host CPU and memory. The degree of\n   oversubscription is controlled by customers, who may factor in workload\n   correlation and load in order to ensure smooth host system operation.\n1. With a microVM configured with a minimal Linux kernel, single-core CPU, and\n   128 MiB of RAM, Firecracker supports a steady mutation rate of 5 microVMs per\n   host core per second (e.g., one can create 180 microVMs per second on a host\n   with 36 physical cores).\n1. The number of Firecracker microVMs running simultaneously on a host is\n   limited only by the availability of hardware resources.\n1. Each microVM exposes a host-facing API via an in-process HTTP server.\n1. Each microVM provides guest-facing access to host-configured metadata via the\n   `/mmds` API.\n\n### Specifications\n\nFirecracker's technical specifications are available in the\n[Specifications document](../SPECIFICATION.md).\n\n## Host Integration\n\nThe following diagram depicts an example host running Firecracker microVMs.\n\n![Firecracker Host Integration](images/firecracker_host_integration.png?raw=true \"Firecracker Host Integration\")\n\nFirecracker runs on Linux hosts and with Linux guest OSs (from this point on,\nreferred to as guests). For a complete list of currently supported kernel\nversions, check out the [kernel support policy](kernel-policy.md).\n\nIn production environments, Firecracker should be started only via the `jailer`\nbinary. See [Sandboxing](#Sandboxing) for more details.\n\nAfter launching the process, users interact with the Firecracker API to\nconfigure the microVM, before issuing the `InstanceStart` command.\n\n### Host Networking Integration\n\nFirecracker emulated network devices are backed by TAP devices on the host. To\nmake use of Firecracker, we expect our customers to leverage on-host networking\nsolutions.\n\n### Storage\n\nFirecracker emulated block devices are backed by files on the host. To be able\nto mount block devices in the guest, the backing files need to be pre-formatted\nwith a filesystem that the guest kernel supports.\n\n## Internal Architecture\n\nEach Firecracker process encapsulates one and only one microVM. The process runs\nthe following threads: API, VMM and vCPU(s). The API thread is responsible for\nFirecracker's API server and associated control plane. It's never in the fast\npath of the virtual machine. The VMM thread exposes the machine model, minimal\nlegacy device model, microVM metadata service (MMDS) and VirtIO device emulated\nNet, Block and Vsock devices, complete with I/O rate limiting. In addition to\nthem, there are one or more vCPU threads (one per guest CPU core). They are\ncreated via KVM and run the `KVM_RUN` main loop. They execute synchronous I/O\nand memory-mapped I/O operations on devices models.\n\nAll device emulation on the VMM thread is driven by a single epoll-based event\nloop, which also picks up the requests forwarded by the API thread. When the\nmicroVM is paused through the API, the VMM thread stops polling that event loop\nand only serves API requests until it is resumed. A `Paused` microVM therefore\nhas neither its vCPUs nor its device models running (see\n[snapshot support](snapshotting/snapshot-support.md#pausing-the-microvm)).\n\n### Threat Containment\n\nFrom a security perspective, all vCPU threads are considered to be running\nmalicious code as soon as they have been started; these malicious threads need\nto be contained. Containment is achieved by nesting several trust zones which\nincrement from least trusted or least safe (guest vCPU threads) to most trusted\nor safest (host). These trusted zones are separated by barriers that enforce\naspects of Firecracker security. For example, all outbound network traffic data\nis copied by the Firecracker I/O thread from the emulated network interface to\nthe backing host TAP device, and I/O rate limiting is applied at this point.\nThese barriers are marked in the diagram below.\n\nFirecracker does not perform any network traffic filtering. All egress traffic\nfrom a guest is therefore considered untrusted, and should be filtered at the\nhost-level.\n\n![Firecracker Threat Containment](images/firecracker_threat_containment.png?raw=true \"Firecracker Threat Containment\")\n\n## Components and Features\n\n### Machine Model\n\n#### Layout\n\nFirecracker provides guests with storage and network access via emulated VirtIO\nNet and VirtIO Block devices. It also exposes a serial console and partial\nkeyboard controller, the latter being used by guests to reset the VM (either\nsoft or hard reset). Within Firecracker, the purpose of the I8042 device is to\nsignal the microVM that the guest has requested a reboot.\n\nIn addition to the Firecracker provided device models, guests also see the\nProgrammable Interrupt Controllers (PICs), the I/O Advanced Programmable\nInterrupt Controller (IOAPIC), and the Programmable Interval Timer (PIT) that\nKVM supports.\n\n#### Exposing the CPU to the guest\n\nFirecracker allows control of what processor information is exposed to the guest\nby using [CPU templates](cpu_templates/cpu-templates.md). CPU templates can be\nset via the Firecracker API. Users can choose from existing static CPU templates\nand/or creating their own custom CPU templates.\n\n#### Clocksources available to guests\n\nFirecracker exposes the following clock sources to guests:\n\n- x86_64: kvm-clock and tsc. Linux guests >=5.10 will pick tsc by default if\n  stable.\n- aarch64: arch_sys_counter\n\n### I/O: Storage, Networking and Rate Limiting\n\nFirecracker provides VirtIO/block and VirtIO/net emulated devices, along with\nthe application of rate limiters to each volume and network interface to make\nsure host hardware resources are used fairly by multiple microVMs. These are\nimplemented using a token bucket algorithm based on two buckets. One is\nassociated with the number of operations per second and the other one with the\nbandwidth. The customer can create and configure rate limiters via the API by\nspecifying token bucket configurations for ingress and egress. Each token bucket\nis defined via the bucket size, I/O cost, refill rate, maximum burst, and\ninitial value. This enables the customer to define flexible rate limiters that\nsupport bursts or specific bandwidth/operations limitations. For vhost-user\ndevices, customers should implement rate limiting on the side of the vhost-user\nbackend that they provide.\n\n### MicroVM Metadata Service\n\nFirecracker microVMs expose access to a minimal MicroVM-Metadata Service (MMDS)\nto the guest through the API endpoint. The metadata stored by the service is\nfully configured by users.\n\n### Sandboxing\n\n#### __Firecracker process__\n\nFirecracker is designed to assure secure isolation using multiple layers. The\nfirst layer of isolation is provided by the Linux KVM and the Firecracker\nvirtualization boundary. To assure defense in depth, Firecracker should only run\nconstrained at the process level. This is achieved by the following: seccomp\nfilters for disallowing unwanted system calls, cgroups and namespaces for\nresource isolation, and dropping privileges by jailing the process. Seccomp\nfilters are automatically installed by Firecracker, while for the latter, we\nrecommend starting Firecracker with the `jailer` binary that's part of each\nFirecracker release.\n\n##### Seccomp\n\nSeccomp filters are used by default to limit the host system calls Firecracker\ncan use. The default filters only allow the bare minimum set of system calls and\nparameters that Firecracker needs in order to function correctly.\n\nThe filters are loaded in the Firecracker process, on a per-thread basis, before\nexecuting any guest code.\n\nFor more information, check out the [seccomp documentation](seccomp.md).\n\n#### __Jailer process__\n\nThe Firecracker process can be started by another `jailer` process. The jailer\nsets up system resources that require elevated permissions (e.g., cgroup,\nchroot), drops privileges, and then exec()s into the Firecracker binary, which\nthen runs as an unprivileged process. Past this point, Firecracker can only\naccess resources that a privileged third-party grants access to (e.g., by\ncopying a file into the chroot, or passing a file descriptor).\n\n##### Cgroups and Quotas\n\nEach Firecracker microVM can be further encapsulated into a cgroup. By setting\nthe affinity of the Firecracker microVM to a node via the cpuset subsystem, one\ncan prevent the migration of said microVM from one node to another, something\nthat would impair performance and cause unnecessary contention on shared\nresources. In addition to setting the affinity, each Firecracker microVM can\nhave its own dedicated quota of the CPU time via the cpu subsystem, thus\nguaranteeing that resources are fairly shared across Firecracker microVMs.\n\n### Monitoring\n\nFirecracker emits logs and metric counters, each on a named pipe that is passed\nvia the API. Logs are flushed line by line, whereas metrics are emitted when the\ninstance starts, then every 60 seconds while it's running, and on panic.\nFirecracker customers are responsible for collecting data in the Firecracker log\nfiles. In production builds, Firecracker does not expose the serial console\nport, since it may contain guest data that the host should not see.\n</yakaai_skills>"}