linux-insides Documentation

A book-in-progress about the Linux kernel and its insides.

33,643 stars Python Python 1 file ยท ~6,794 tokens #linux#linux-insides#linux-kernel
RAW Doc

Docker & Self-Hosted Deployment Verified OCI

Deploy linux-insides on any server, homelab, or cloud container platform.

1-Click Terminal Command
bash (docker run)
docker run -d --name linux-insides -p 4000:4000 --restart always 0xax/linux-insides:latest

Technical Documentation: 0xAX/linux-insides

โ„น๏ธ Provenance: Multi-Tier Ingestion ยท CodeWiki Reference

Overview (`README.md`)

Introduction

linux-insides is a book-in-progress about the Linux kernel and its insides. The goal of this project is simple - to share knowledge about the Linux kernel internals and related low-level topics. If you're curious about what's under the hood, see the Table of Contents.

Chapter status (kernel v7.2.0)

[!IMPORTANT]
I started writing this series when the latest version of the kernel was 3.18. A lot has changed since then, and I am in progress of updating the content to reflect modern kernels (v6.18+). I'll continue revising the posts as the kernel evolves.

  • Booting (updated for v7.2.0)
  • Initialization (pending v6.18.0 review)
  • Interrupts (pending v6.18.0 review)
  • System calls (pending v6.18.0 review)
  • Timers and time management (pending v6.18.0 review)
  • Synchronization primitives (pending v6.18.0 review)
  • Memory management (pending v6.18.0 review)
  • Cgroups (pending v6.18.0 review)
  • SMP (pending v6.18.0 review)
  • Kernel mechanisms (pending v6.18.0 review)
  • Data Structures in the Linux Kernel (pending v6.18.0 review)
  • x86 fundamentals (pending v6.18.0 review)
  • Toolchain and binaries (pending v6.18.0 review)
  • Initial ram disk (pending v6.18.0 review)
  • Contributing to the Linux kernel (pending v6.18.0 review)

Requirements

[!TIP]
You can get started with Assembler with my other series of posts about the Assembly programming.

Translations

Thanks to the volunteers, the posts about Linux are translated into different languages.

[!NOTE]
The translations may diverge from the original content.

Contribution

Read the Contribution guide to learn how to contribute to the project. When contributing, make sure to follow the Code of Conduct.

If you have any questions or suggestions, feel free to ping me at Twitter @0xAX, add an issue, or drop me an email.

Mailing list

There is a Google group mailing list ([email protected]) for learning the kernel source code.

To join the group, send an email to [email protected]. You will receive a confirmation email. After replying to it, you will be added to the mailing list.

[!TIP]
If you have a Google account, you can simply open the archive page and click Apply to join group. You will be approved automatically.

Sponsors

Thanks to the people supporting this project on Patreon:

License

This project is licensed under the BY-NC-SA Creative Commons.

Author

The technical content is written by @0xAX.

Additional big thanks to @klaudiagrz for text improvements.


Overview (`Booting/README.md`)

Kernel Boot Process

Welcome to the boot journey of the Linux kernel, from power-on to the first instruction of the decompressed kernel. This chapter walks the complete boot path step by step from the moment you power on your computer to the moment the Linux kernel loaded in the memory of your machine.

How to read

This chapter assumes you are comfortable with basic computer architecture and have a light familiarity with the C programming language and x86_64 assembly syntax. You do not need to be a kernel expert, but being able to read short code snippets and recognize hardware terms will help.

Each part of this chapter focuses on one boot phase. Read in order the first time, then revisit individual steps as references when you want to map a specific symbol or register setup to its place in the sequence. It is quite useful to have the source code of Linux kernel on your local computer to follow the details. You can obtain the source code using the following command:

bash
git clone [email protected]:torvalds/linux.git

Alternatively, you can get the source code through the GitHub CLI:

bash
gh repo clone torvalds/linux

Notation used

When reading this and other chapters, you may encounter special notation:

  • CS, DS, SS, CR0, CR3, CR4, EFER - refer to x86 segment and control registers
  • 0x... - denotes hexadecimal values
  • entry_* and startup_* - common prefixes for early boot symbols
  • setup code refers to the early part of the Linux kernel which executes preparation to load the kernel code itself into memory
  • decompressor refers to the part of the setup code that inflates the compressed kernel image into memory

What you will learn

  • The way a processor reaches the kernel entry point from firmware and the bootloader
  • Different modes of x86_64 processors
  • What the early setup code does before the kernel itself will be loaded into memory and start its work

Reading order

The following parts are presented in this chapter and should be read in this order:

  1. From the bootloader to kernel - from power-on to the first instruction in the kernel
  2. First steps in the kernel setup code - early setup, heap init, parameter discovery (EDD, IST, and more)
  3. Video mode initialization and transition to protected mode - video mode setup and the move to protected mode
  4. Transition to 64-bit mode - preparation and the jump into long mode
  5. Kernel Decompression - pre-decompression setup and the decompressor itself
  6. Kernel load address randomization - how KASLR picks a load address

Kernel version

This chapter corresponds to Linux kernel v7.2.0.


Overview (`Cgroups/README.md`)

Cgroups

This chapter describes control groups mechanism in the Linux kernel.


Overview (`Concepts/README.md`)

Kernel mechanisms

Welcome to the chapter about core mechanisms of the Linux kernel. This chapter describes fundamental infrastructure which is not tied to one particular subsystem but is used all over the kernel - from variables with a separate copy for each processor to the way subsystems notify each other about events.

How to read

This chapter assumes you are comfortable with basic computer architecture and have a light familiarity with the C programming language and x86_64 assembly syntax. You do not need to be a kernel expert, but being able to read short code snippets and recognize hardware terms will help.

Each part of this chapter focuses on one mechanism and the parts are mostly independent from each other, so you can read them in any order. Still, the suggested order below goes from simpler mechanisms to more complex ones. It is quite useful to have the source code of Linux kernel on your local computer to follow the details. You can obtain the source code using the following command:

bash
git clone [email protected]:torvalds/linux.git

Alternatively, you can get the source code through the GitHub CLI:

bash
gh repo clone torvalds/linux

Notation used

When reading this and other chapters, you may encounter special notation:

  • DEFINE_PER_CPU and per_cpu - macros for defining and accessing per-cpu variables
  • cpumask - is a bitmap which represents a set of processors in the system
  • *_initcall - macros like early_initcall or fs_initcall which register a callback for a certain kernel initialization stage
  • notifier_block - is the main data structure of the notification chains mechanism
  • 0x... - denotes hexadecimal values

What you will learn

  • How the kernel creates variables where each processor core has its own copy
  • How the kernel keeps track of possible, present, online and active processors in the system
  • How the kernel determines the correct order of initialization of its built-in subsystems
  • How kernel subsystems subscribe to asynchronous events from other subsystems

Reading order

The topics in this chapter are independent of one another, so you can read the sections in any order. The chapter covers the following topics:

  • Per-CPU variables - how per-cpu variables are implemented and how to work with them
  • CPU masks - the bitmaps which describe sets of processors and the API around them
  • The initcall mechanism - the ordering of built-in modules and subsystems initialization
  • Notification Chains - how subsystems subscribe to and publish asynchronous events

Overview (`DataStructures/README.md`)

Data structures in the Linux kernel

The Linux kernel provides implementations of many different data structures, including doubly linked lists, trees, heaps, hash tables, bitmaps, and many, many more.

So far, this chapter contains the following parts:


Overview (`Initialization/README.md`)

Kernel initialization process

You will find here a couple of posts which describe the full cycle of kernel initialization from its first step after the kernel has been decompressed to the start of the first process run by the kernel itself.

Note That there will not be a description of the all kernel initialization steps. Here will be only generic kernel part, without interrupts handling, ACPI, and many other parts. All parts which I have missed, will be described in other chapters.


Overview (`Interrupts/README.md`)

Interrupts and Interrupt Handling

In the following posts, we will cover interrupts and exceptions handling in the Linux kernel.


Overview (`MM/README.md`)

Linux kernel memory management

This chapter describes memory management in the Linux kernel. You will see here a
couple of posts which describe different parts of the Linux memory management framework:


Overview (`SyncPrim/README.md`)

Synchronization primitives in the Linux kernel.

This chapter describes synchronization primitives in the Linux kernel.

  • Introduction to spinlocks - the first part of this chapter describes implementation of spinlock mechanism in the Linux kernel.
  • Queued spinlocks - the second part describes another type of spinlocks - queued spinlocks.
  • Semaphores - this part describes implementation of semaphore synchronization primitive in the Linux kernel.
  • Mutual exclusion - this part describes - mutex in the Linux kernel.
  • Reader/Writer semaphores - this part describes special type of semaphores - reader/writer semaphores.
  • Sequential locks - this part describes sequential locks in the Linux kernel.

Overview (`SysCall/README.md`)

System calls

This chapter describes the system call concept in the Linux kernel.


Overview (`Timers/README.md`)

Timers and time management

This chapter describes timers and time management related concepts in the Linux kernel.


Overview (`Toolchain/README.md`)

Toolchain and binaries

Welcome to the chapter about the toolchain behind the Linux kernel. This chapter describes the tools which are used to build the kernel and the format of the binaries they produce - from the moment you execute make in the root of the kernel source tree, through linking of object files, to the resulting ELF binaries, the assembly code embedded in the kernel sources and what happens when such a binary is started.

How to read

This chapter assumes you are comfortable with basic computer architecture and have a light familiarity with the C programming language and x86_64 assembly syntax. You do not need to be a kernel expert, but being able to read short code snippets and recognize hardware terms will help.

Each part of this chapter focuses on one tool or format. The parts build on each other, so read in order the first time, then revisit individual parts as references when you need to recall how a specific tool or format works. It is quite useful to have the source code of Linux kernel on your local computer to follow the details. You can obtain the source code using the following command:

bash
git clone [email protected]:torvalds/linux.git

Alternatively, you can get the source code through the GitHub CLI:

bash
gh repo clone torvalds/linux

Notation used

When reading this and other chapters, you may encounter special notation:

  • make and Makefile - refer to the build tool which drives the kernel compilation and its input files
  • vmlinux and bzImage - the resident kernel image and the compressed bootable kernel image
  • *.o - denotes object files produced by the compiler before linking
  • ELF - is the Executable and Linkable Format, the standard binary format on Linux
  • __asm__ - marks inline assembly statements embedded in C code
  • 0x... - denotes hexadecimal values

What you will learn

  • What happens when you execute make in the root directory of the Linux kernel source code and how the bzImage is produced
  • How the linker combines object files into an executable and how linker scripts control this process
  • The structure of ELF binaries and how the Linux kernel describes them in its source code
  • How to read and write inline assembly statements in both their basic and extended forms
  • What happens in userspace between the start of a program and the call of its main function

Reading order

The topics in this chapter are independent of one another, so you can read the sections in any order. The chapter covers the following topics:


Overview (`X86/README.md`)

x86-64 fundamentals

Welcome to the chapter on x86_64 fundamentals and the core data structures used by the Linux kernel on this architecture. The concepts introduced here provide the foundation for understanding the Linux kernel source code throughout the rest of this book.

Most of the information is taken from the following official manuals:

Of course, I'm also basing this chapter on the Linux kernel source code.

How to read

This chapter assumes you are comfortable with basic computer architecture and have a light familiarity with the C programming language and x86_64 assembly syntax. You do not need to be a kernel expert, but being able to read short code snippets and recognize hardware terms will help.

Each part of this chapter covers one concept of the Intel 64-bit architecture and explains how it is used and implemented in the Linux kernel. It is quite useful to have the source code of the Linux kernel on your local computer to follow the details. You can get the source code using the following command:

bash
git clone [email protected]:torvalds/linux.git

Alternatively, you can get the source code through the GitHub CLI:

bash
gh repo clone torvalds/linux

Notation used

When reading this and other chapters, you may encounter special notation:

  • CS, DS, SS, CR0, CR3, CR4, EFER - refer to x86 segment and control registers
  • 0x... - denotes hexadecimal values
  • entry_* and startup_* - common prefixes for early boot symbols
  • setup code refers to the early part of the Linux kernel which executes preparation to load the kernel code itself into memory
  • decompressor refers to the part of the setup code that inflates the compressed kernel image into memory

What you will learn

  • Fundamental x86_64 structures like GDT, IDT and others
  • Paging structure and kernel API

Reading order

The topics in this chapter are independent of one another, so you can read the sections in any order. The chapter covers the following topics:

Kernel version

This chapter corresponds to Linux kernel v7.2.0.


Overview (`scripts/README.md`)

Scripts used to maintain linux-insides

This directory provides a set of helper scripts for maintaining and building this repository.

Script What it does
check_code_snippets.py Verifies that kernel source snippets in the book still match the real source on GitHub
latex.py Converts the whole book into a single LaTeX-typeset PDF

External links are not checked by a script from this directory. The check links workflow does it with lychee, configured in lychee.toml.

`check_code_snippets.py` - snippet validator

Keeps the code in the book honest. Each snippet is annotated with an HTML comment pointing at a GitHub raw URL and a line range, for example:

text

The script fetches that range from GitHub and compares it against the code block in the book. If they have drifted apart, it prints both versions and exits with a non-zero status - handy for CI.

Each source file is fetched only once per run and the result is reused by every snippet that refers to it. Requests that GitHub throttles are retried with a growing delay. If GITHUB_TOKEN is set, the sources are read through the GitHub contents API instead of the raw endpoint, which counts against the authenticated rate limit and makes 429 Too Many Requests far less likely:

bash
GITHUB_TOKEN=$(gh auth token) uv run --project scripts ./scripts/check_code_snippets.py .

Its dependencies are declared in pyproject.toml and pinned in uv.lock, so uv provides both the interpreter and the packages. Nothing has to be installed by hand.

Usage, from the repository root:

bash
# Check the whole repository
uv run --project scripts ./scripts/check_code_snippets.py .

# Check a single chapter
uv run --project scripts ./scripts/check_code_snippets.py ./Initialization

From inside this directory the --project flag can be dropped:

bash
uv run ./check_code_snippets.py ../Initialization

If you prefer a plain virtual environment, uv sync creates one in scripts/.venv and the script can be run directly after activating it.

`latex.py` - PDF builder

Converts the Markdown of each chapter directory into LaTeX with pandoc, compiles each one with pdflatex, and stitches everything into a single LinuxKernelInsides.pdf with pdfunite. The intermediate files are left in a build directory, which is where you find the pdflatex logs if a chapter does not typeset.

You only need this if you want to build the PDF yourself. To just read the book, grab the pre-built LinuxKernelInsides.pdf.

It uses only the standard library, so plain Python 3 is enough - no uv and no packages to install. It requires the following utils and packages:

Usage, from this directory:

bash
./latex.py

The build directory and the resulting PDF are written to the current working directory, so run it from wherever you want them.


--- METRICS ---