Goals of this Lab
- Understand the Windows Portable Executable (PE32 / PE32+) file format: the DOS header and stub, the NT headers (File Header and Optional Header), the section table, and the individual sections.
- Learn the difference between a Relative Virtual Address (RVA) and a raw file offset, and how to convert between them.
- Understand the Import Address Table (IAT) and Export Address Table (EAT), and why the list of imported API names leaks a program's intent.
- Perform triage-level static analysis of a Windows binary without ever running it, using modern free tools: Ghidra, PE-bear (or CFF Explorer),
strings, and Python with thepefilelibrary. - Build the beginning of a heuristic malware detector in Python and write a starter YARA rule.
Introduction and Background
Static analysis means examining a program {without executing
it}. We read the bytes of the file, its structure, its strings, and its
disassembly to form a hypothesis about what the program does. This is the
opposite of dynamic analysis, where you run the sample in a sandbox
and watch its behavior.
Static analysis is the foundation of triage: given hundreds of
suspicious files, an analyst quickly decides which ones deserve deeper,
more expensive investigation. Triage is fast, it is safe (nothing runs), and
it scales. In this lab you will do triage on a single teaching sample and
then automate part of that triage in Python.
Important safety rule for Lab 1: static only. You will
never execute the sample in this lab. The provided file
lab1_sample.bin is defanged and benign, intended purely for
teaching, but you must still treat every sample as hostile and analyze it
statically. Running malware is a Lab exercise for a later class, in an
isolated VM, and is explicitly out of scope here.
The PE File Format
Every Windows .exe and .dll is a {Portable
Executable (PE)} file. When the Windows loader maps the file into memory it
walks a chain of headers. Understanding that chain is the core skill of this
lab.
- DOS header and DOS stub. Every PE begins with the two bytes
MZ(0x5A4D). This is a leftover from MS-DOS. The DOS stub is the small program that prints "This program cannot be run in DOS mode." The important field in the DOS header ise_lfanew, a 4-byte value at offset 0x3C that gives the file offset of the NT headers. The loader readse_lfanewto jump past the DOS stub. - NT headers. At the offset named by
e_lfanewyou find the signaturePE\textbackslash0\textbackslash0(0x50450000) followed by the NT headers. The NT headers are made of two parts: - The File Header (also called the COFF header): machine type,
NumberOfSections, and the size of the Optional Header. - The Optional Header (not actually optional for an executable). It holds the fields the loader needs, including
Magic(0x10B for PE32, 0x20B for PE32+ / 64-bit),AddressOfEntryPoint(the RVA where execution begins),ImageBase(the preferred load address), and the data directories — one of which points at the import table. - Section table. Immediately after the Optional Header is an array of section headers, one per section. Each entry records the section name, its virtual size and virtual address (RVA), its raw size and raw file offset, and its
Characteristicsflags (readable, writable, executable, etc.). - Sections. The actual content. Common section names:
.text— executable code (readable + executable)..data— initialized, writable global data..rdata— read-only data, including the import tables and string constants..rsrc— resources (icons, dialogs, embedded files). A section that is both writable AND executable is a red flag: normal compilers do not produce such sections, but packers and self-modifying malware do.
RVA versus File Offset
A Relative Virtual Address (RVA) is an address relative to
ImageBase once the file is loaded into memory. A {file
offset} is a position in the file on disk. They differ because sections are
aligned differently on disk than in memory. To convert an RVA to a file
offset, find the section that contains the RVA, then:
+ PointerToRawData_{section}
Tools like PE-bear and the pefile library do this conversion for
you, but you must understand why it is necessary: the
AddressOfEntryPoint is an RVA, so to read the entry code from the
file on disk you must first translate it to a file offset (in pefile
the helper is get_offset_from_rva).
Imports, the IAT, and Intent
A program rarely does anything interesting on its own; it calls Windows API
functions. The Import Address Table (IAT) lists every external
function the binary imports, grouped by the DLL that provides it (for
example kernel32.dll, ws2_32.dll, advapi32.dll).
The Export Address Table (EAT) is the mirror image: for a DLL, it
lists the functions that other modules may import.
The imported API names leak intent. Even without running or fully
disassembling a binary, the set of APIs it imports strongly suggests its
capabilities. A file that imports WriteProcessMemory and
CreateRemoteThread is almost certainly doing process injection; one
that imports CryptEncrypt and enumerates files may be ransomware.
This observation is the engine behind the heuristic scanner you will finish
in Task 5. It is also the basis of the imphash (import hash): a
hash computed over the ordered list of imported functions, used by threat
intelligence platforms to cluster related samples.
Experiment Setup
Complete the following before starting the exercise. You may work on the
Badger CTF Linux server over SSH, on the Windows 10 analysis VM reached via
Guacamole RDP, or both. Ghidra and pefile run on both; PE-bear is
easiest on Windows.
- Connect to the Badger CTF server over SSH. Replace the placeholders with the credentials posted on the course website:
ssh <username>@<badger-host> -p <port> - Copy the provided sample
lab1_sample.binfrom the shared course folder into your own working directory:mkdir -p \textasciitilde/lab1 && cd \textasciitilde/lab1cp /srv/csc471/shared/lab1_sample.bin .The sample is defanged and benign for teaching. Do not run it; analyze it statically only. - Install and verify your tools.
pip install pefilestrings --versionghidraRun \ \ # launches the Ghidra GUIOn the Windows VM, launch PE-bear (or CFF Explorer) and useFile > Opento open the sample. - Copy the starter code
lab1_scan.pyandlab1_rules.yarinto your working directory. You will complete both files in Task 5.
Lab Exercise
Work through the tasks in order. Answer every numbered Question in your PDF
report, and include screenshots as evidence.
Task 1 — Hashing and Identity
A file's cryptographic hash is its fingerprint. Analysts share hashes (not
files) to refer to a sample, and threat-intelligence platforms such as
VirusTotal are indexed by hash. Compute both an MD5 and a SHA-256 of the
sample:
md5sum lab1_sample.bin sha256sum lab1_sample.bin
A related, more powerful fingerprint is the imphash, a hash over
the ordered list of imported DLLs and functions. Because it depends on
how the program was built and linked rather than on its exact bytes,
two samples from the same malware family often share an imphash even when
their contents differ. Compute it with pefile:
import pefile
pe = pefile.PE("lab1_sample.bin")
print("imphash:", pe.get_imphash())
{Concept only: do not upload the sample anywhere. We discuss VirusTotal
as a workflow, but uploading a sample can tip off an adversary that it has
been detected.}
Task 2 — PE Structure with PE-bear
Open the sample in PE-bear (or CFF Explorer). Explore the DOS header, the NT
headers, and the section table.
AddressOfEntryPoint (an RVA), the ImageBase, and the number of sections. Is this a PE32 or a PE32+ binary? How can you tell? Question 2.2: List every section with its name, virtual size, and its readable/writable/executable characteristics. Which sections are executable? Is any section both writable AND executable? Explain why that would be a red flag. Question 2.3: For each section, compare the raw (on-disk) size to the virtual size. Is there a large discrepancy for any section? What might a large virtual size with a tiny raw size indicate (hint: packing)?Task 3 — Imports and Intent
Using PE-bear's imports view (or pefile), list the DLLs and the
functions the sample imports. Focus on the interesting ones and map each to a
capability using the table at the end of this handout.
Task 4 — Strings and Ghidra
Extract the human-readable strings. Remember Windows uses both ASCII and
wide (UTF-16) strings, so scan for both:
strings -a lab1_sample.bin > ascii.txt strings -a -e l lab1_sample.bin > wide.txt
Then import the sample into Ghidra (File > Import File), let the
auto-analysis finish, and use the Symbol Tree or the entry point to navigate
to main (or the program entry). Open the Decompiler window and read
the C-like output for one function.
Task 5 — Write a Heuristic Detector (Deliverable Core)
Now automate triage. Complete the two starter files.
(a) 0 . The skeleton already runs and prints a
score and verdict. It uses pefile to inspect the sample and awards
points for: writable+executable sections, suspicious imports, high section
entropy (indicating packing or encryption), and an abnormally
small import table. Finish every section marked # TODO: extend the
SUSPICIOUS_IMPORTS dictionary, tune the per-check point values, and
tune the verdict thresholds. A working entropy() helper is already
provided. Run it on the sample:
python3 lab1_scan.py lab1_sample.bin
(b) 0 . The file contains one complete example
rule (process_injection_combo) and one skeleton rule
(suspicious_persistence) marked // TODO. Complete the
skeleton so it matches a meaningful combination of persistence-related
strings. Test it (static scan, never execute):
yara lab1_rules.yar lab1_sample.bin
lab1_scan.py give the sample? List the findings it printed. Question 5.2: Did either YARA rule match? If so, which strings triggered the match? Question 5.3: Give one example of a benign program that your heuristic might wrongly flag (a false positive), and briefly explain why.Suspicious Win32 API to Capability Reference
Use this table in Task 3 to map imports to likely behavior.
Hints
- Review the lecture Class 4 -- PE File Format for the header chain (DOS header,
e_lfanew, NT headers, section table) and for RVA versus file-offset conversion. - Review the lecture Class 5 -- Static Analysis for the imports-leak-intent methodology, entropy and packing, and the strings and Ghidra workflow.
- If
pefileraisesPEFormatError, confirm you copied the sample correctly (check its size and hash against the values on the course website). - In Ghidra, if the Decompiler window is empty, click inside a defined function first; use
Window > Symbol Treeto findentryormain.
Deliverables
- A detailed PDF report answering every numbered Question, with screenshots as evidence (PE-bear views,
stringsoutput, the Ghidra decompiler, and your scanner's output). - Your completed
lab1_scan.py. - Your completed
lab1_rules.yar.
Submission
The due date is posted on the course website. {Late work is not
accepted.} Submit your work to D2L. Include a detailed PDF report
with screenshots, together with your completed lab1_scan.py and
lab1_rules.yar.
or AI, give clear attribution. Otherwise, you WILL FAIL this course.