Rootkits and Stealth

This lab modernizes our classic kernel-mode rootkit exercise. Instead of the

retired Windows XP System Service Descriptor Table (SSDT) hook, you will build

a small Linux Loadable Kernel Module (LKM) rootkit and hook a system call the

modern way, using the kernel's ftrace framework. In Part 2 you will

study a real-world nation-state rootkit — Stuxnet — and connect what you

built to what attackers shipped in the wild.

Goals

The goals of this lab:

• Understand the concepts of rootkits and stealth: hiding processes, files, kernel modules, and network connections.
• Distinguish user-mode rootkits from kernel-mode rootkits, and understand why kernel-mode stealth is harder to detect.
• Learn syscall/function hooking in the Linux kernel using ftrace (and, as an alternative, kprobes).
• Understand how the historical Windows SSDT hook worked, and why the old writable sys_call_table trick no longer works on modern kernels.
• Analyze a real-world case: the Stuxnet kernel-mode rootkit components.

Introduction and Background

What is a rootkit?

A rootkit is a set of software tools that lets an attacker maintain

privileged ("root") access to a system while hiding that access

from the legitimate owner and from defensive tools. The defining property is

not privilege escalation itself but stealth: a rootkit modifies what

the system reports so that malicious artifacts become invisible. Typical

things a rootkit hides include:

• Processes — so the attacker's implant does not appear in ps, top, or Task Manager.
• Files and directories — so payloads do not appear in ls, dir, or a file manager.
• Kernel modules / drivers — so the rootkit itself does not appear in lsmod or the loaded-driver list.
• Network connections — so command-and-control traffic does not appear in netstat or ss.

User-mode vs. kernel-mode rootkits

User-mode rootkits operate in ring 3, alongside normal programs.

On Linux they commonly abuse LD_PRELOAD to inject a malicious shared

library that intercepts libc functions such as readdir(); on Windows

they use Import Address Table (IAT) patching or inline function hooks inside a

target process. They are relatively easy to deploy but also easier to detect,

because a defender operating from the kernel (or a second, clean vantage

point) can see the truth the user-mode hooks are hiding.

Kernel-mode rootkits run in ring 0 as part of the operating system

kernel. On Linux they are typically packaged as an LKM; on Windows as a signed

driver. Because they share the kernel's address space and full privilege, they

can alter the data structures and system-call results that every

user-mode tool depends on. This makes them far stealthier — but also far more

dangerous to write, because a bug does not crash one process, it panics the

whole machine.

Linux LKM basics

A Loadable Kernel Module is kernel code that can be inserted into and removed

from a running kernel. The essential lifecycle:

• module_init(fn) registers the function run at load time; module_exit(fn) registers the cleanup run at unload time.
• sudo insmod mod.ko loads a module; sudo rmmod mod unloads it; lsmod lists loaded modules (it reads /proc/modules).
• printk() / the pr_info() family writes to the kernel log, which you read with dmesg.
• Module code runs with full kernel privilege. There is no memory protection to save you: a bad pointer or an unbalanced hook causes a kernel panic that takes down the entire system. This is why you will only ever load your module inside a disposable VM.

Hiding a module from lsmod

Every loaded module is linked into a doubly linked list of

struct module objects. Tools like lsmod and

/proc/modules simply walk that list. If the rootkit unlinks its own

entry with list_del(&THIS_MODULE->list), the module keeps running

and its hooks stay active, but it disappears from the module list.

You will complete this step in Task 2.

Function hooking with ftrace (the modern approach)

To alter what a system call returns, a rootkit must hook it — divert

execution through attacker code. Historically, Linux rootkits overwrote an

entry in the global sys_call_table so that, for example,

sys_getdents64 pointed at the rootkit's function. On modern kernels

this is impractical:

• sys_call_table is no longer exported to modules.
• The memory page holding it is mapped read-only. Attackers historically flipped the CR0 write-protect (WP) bit, or used update_mapping_prot(), to make the page temporarily writable, patched the pointer, then restored the protection. Modern hardening (CONFIG_STRICT_KERNEL_RWX, KASLR, CET) makes this brittle and easy to detect.

The modern technique is to hook the function itself using the kernel's

built-in tracing infrastructure. With ftrace, we attach to the

fentry/mcount stub that every kernel function carries, and

inside our callback we overwrite the saved instruction pointer

(regs->ip) so that execution jumps into the rootkit's replacement

function. This is the technique scaffolded in lab5_rootkit.c. An

alternative is kprobes, which plants a breakpoint (or an optimized

jump) at an arbitrary kernel address and runs a pre/post handler; it is ideal

for observing or lightly tweaking arguments.

Historical note: the Windows SSDT hook (the old lab)

The retired version of this lab ran inside Windows XP under a kernel debugger.

Windows keeps a table of pointers to kernel service routines called the

System Service Descriptor Table (SSDT); the array of function

pointers itself is often referred to by the symbol KiServiceTable.

When a user-mode program calls, say, NtQueryDirectoryFile, the kernel

uses a syscall number to index into this table and dispatch to the right

routine. In WinDBG, students ran:

dds KiServiceTable L 12a

which dumped the table as a list of addresses with their resolved symbol

names. An SSDT-hooking rootkit overwrites one of those pointers so

that, for example, NtQueryDirectoryFile is redirected to attacker

code that calls the real routine and then strips out any directory entry it

wants to hide — making files vanish from Explorer and dir. Comparing

the dds output before and after infection revealed which pointers had

been redirected out of the kernel's normal address range. The Linux ftrace

hook you build in Part 1 is the same idea (redirect a

table/function entry to attacker code that filters results), implemented with

the modern Linux tracing API instead of a writable table.

Kernel code runs with full privileges and no safety net. A single bad

pointer, wrong offset, or unbalanced hook will {panic the entire

kernel}, not just kill a process.

• Only build and load this module inside your own disposable VM (QEMU / VirtualBox) or a throwaway container that has kernel headers.
• Snapshot the VM first so you can roll back after a crash.
• Never load student kernel code on a shared, lab, or production machine.
• Always remove your hooks in module_exit before the module memory is freed, or the kernel will jump into freed memory and panic.

Experiment Setup

• Boot a disposable Linux VM (Ubuntu/Debian recommended) or a privileged container that has kernel headers. Take a VM snapshot before you continue.
• Install the build tools and the headers that match your running kernel: sudo apt update && sudo apt install -y build-essential linux-headers-(uname -r)
• Copy the two provided files into an empty working directory: lab5_rootkit.c and lab5_Makefile. Rename the Makefile so make can find it: mv lab5_Makefile Makefile
• Build the module. This produces lab5_rootkit.ko: make
• Load it, read the kernel log, then unload it: sudo insmod lab5_rootkit.ko dmesg | tail sudo rmmod lab5_rootkit
• Read the questions in the next sections and answer them in your report.

Lab Exercise — Part 1 (Hands-on: Build an LKM Rootkit)

You will complete the // TODO sections of lab5_rootkit.c.

The skeleton already provides symbol resolution, a generic ftrace

hook helper (struct ftrace_hook), and the module lifecycle. The core

of the ftrace hook helper looks like this:

struct ftrace_hook {
    const char *name;      /* kernel symbol to hook                */
    void *function;        /* YOUR replacement function            */
    void *original;        /* filled in: call this for the real fn */
    unsigned long address; /* resolved runtime address of target   */
    struct ftrace_ops ops; /* ftrace bookkeeping                    */
};

/* ftrace calls this right before the target runs; we overwrite the
 * saved instruction pointer so control jumps into our hook. The
 * within_module() guard stops infinite recursion when our hook
 * calls the original (still-instrumented) function. */
static void notrace fh_ftrace_thunk(unsigned long ip, unsigned long parent_ip,
                                    struct ftrace_ops *ops,
                                    struct ftrace_regs *fregs)
{
    struct ftrace_hook *hook = container_of(ops, struct ftrace_hook, ops);
    struct pt_regs *regs = ftrace_get_regs(fregs);
    if (!within_module(parent_ip, THIS_MODULE))
        regs->ip = (unsigned long) hook->function;
}

Task 1 — Build and load the skeleton

Build the unmodified skeleton, load it, observe the dmesg output, and

unload it. Confirm you see the load/unload messages and the "hooked"

message.

Task 2 — Hide the module from lsmod

Complete the // TODO in hide_module() so the module unlinks

itself from the module list:

prev_module = THIS_MODULE->list.prev;   /* remember our neighbor */
list_del(&THIS_MODULE->list);           /* unlink from the list  */
hidden = 1;

Rebuild, reload, and confirm that lsmod | grep lab5_rootkit now

returns nothing, even though the module is loaded and its hook is

active. (Note: once hidden, you cannot rmmod it by name unless you

also add an "unhide" trigger; for this task you may reboot the VM to clear

it, or leave the show_module() call in module_exit.)

Task 3 — Complete the syscall hook

Complete the hooking logic. The default target is getdents64, the

syscall behind ls/readdir. In hook_getdents64(),

call the real syscall, then walk the returned buffer and splice out any entry

whose name starts with the magic prefix (csc471_) so those files

become invisible:

while (off < ret) {
    cur = (struct linux_dirent64 *)((char *)kbuf + off);
    if (strncmp(cur->d_name, MAGIC_PREFIX, MAGIC_PREFIX_LEN) == 0) {
        /* remove this entry: shift the rest of the buffer down */
        int reclen = cur->d_reclen;
        char *next = (char *)cur + reclen;
        memmove(cur, next, ret - (off + reclen));
        ret -= reclen;            /* buffer got shorter */
        continue;                 /* do NOT advance off */
    }
    off += cur->d_reclen;
}
copy_to_user(dirent, kbuf, ret);

Alternatively, implement the documented "magic root" variant: hook

__x64_sys_kill, detect a secret (signal, pid) trigger,

and call give_root() to rewrite the caller's credentials with

commit_creds().

Task 4 — Demonstrate stealth

Create a file or directory whose name starts with the magic prefix, e.g.

touch csc471_secret.txt. With the rootkit loaded, run ls and

show the file is hidden. Then rmmod the module (or reboot)

and show the file reappears. (If you built the magic-root variant,

demonstrate an unprivileged shell becoming uid 0 instead.)

Lab Exercise — Part 2 (Analysis: Stuxnet Case Study)

Stuxnet (discovered 2010) was a nation-state worm that targeted Siemens

industrial control systems (PLCs) driving uranium enrichment centrifuges. To

stay hidden on infected Windows hosts, it shipped {kernel-mode rootkit

components}. Answer the following in your report; you do not need any lab

machine for this part.

Hint

Please review the lecture slides — Class 9 Rootkits and Stealth. The

slides cover user-mode vs. kernel-mode rootkits, the historical Windows SSDT

hook, the modern Linux ftrace/kprobes hooking approach, and the Stuxnet case

study.

Deliverables

• A detailed project report in PDF format answering Q1—Q8, including your dmesg output and the before/after stealth screenshots from Part 1.
• Your completed 0 with all // TODO sections implemented.

Submission

• The lab due date is available on our course website. Late submissions will not be accepted.
• The assignment should be submitted to D2L directly.
• Your submission should include: A detailed project report in PDF format describing what you have done, including screenshots and code snippets.
• No plagiarism or cheating is tolerated. If your work is based on others', please give clear attribution. Otherwise, you WILL FAIL this course.