2. Running Griffin Trace - TJAndHisStudents/Griffin-Trace GitHub Wiki

How to Use

  1. Tell Griffin what to monitor: $ echo -n a.out > /sys/kernel/debug/pt_monitor
  2. Tell Griffin what events you want to record: echo -n 6 > /sys/kernel/debug/pt_trace_proc_end
  3. Run the program: $ /path/to/a.out
  4. Retrieve the log output: $ cp /var/log/pt.log ./pt.log
  5. Reset monitoring to stop Griffin: $ echo -e "\x00" | tee /sys/kernel/debug/pt_monitor

API Options

You can decide which events trigger a trace record event. Right now, the Griffin Trace API supports reporting traces on all system calls, forward-edge violations, shadow stack violations, and at the conclusion of the program run.

Process End API

The process end API supports values between 1 and 6, which means that it will print the N leading buffers prior to the process conclusion:

# Run the test with process end capture
echo -n attack-return.out > /sys/kernel/debug/pt_monitor
echo -n 6 > /sys/kernel/debug/pt_trace_proc_end
./attack-return.out
echo -e "\x00" | tee /sys/kernel/debug/pt_monitor

System Call API

Right now, the system call API only supports values between 1 and 3, which means that it will print N buffers before the event and N buffers after, where N is your selected value between the bounds:

# Run the test with system call capture
echo -n attack-return.out > /sys/kernel/debug/pt_monitor
echo -n 2 > /sys/kernel/debug/pt_trace_syscall
./attack-return.out
echo -e "\x00" | tee /sys/kernel/debug/pt_monitor

Forward-Edge & Shadow Stack

WARNING: Forward-Edge and Shadow Stack CFI triggers use mirror pages in user space memory, which currently requires disabling of SMAP protections that limit kernel access to user-space memory. No one should use this feature on deployment systems at this time, but rather restrict usage only to software testing environments.

Right now, the forward-edge and shadow stack APIs only support values between 1 and 6, which means that they will print N buffers before the event, where N is your chosen number.

Note: Forward-Edge violations can only be detected when Griffin has a program policy to enforce these violations. You will need to include the Griffin policy for these violations to be detected:

# Run the test with CFI violation capture
echo -n attack-call.out > /sys/kernel/debug/pt_monitor
echo -n 3 > /sys/kernel/debug/pt_trace_shadow_stack
echo -n 1 > /sys/kernel/debug/pt_trace_fwd_edge
LD_PRELOAD=`pwd`/griffin.so GRIFFIN_POLICY_PATH=`pwd`/attack-call.txt-policy.bin ./attack-call.out

# Capture both the PT log and the violation for extended annotations
cp /var/log/pt.log ./attack-call-fwd-edge.log
cp /var/log/pt.violation.log ./attack-call-fwd-edge.violation.log
echo -e "\x00" | tee /sys/kernel/debug/pt_monitor

Instruction Pointer Address - In Progress

WARNING: Address CFI triggers use mirror pages in user space memory, which currently requires disabling of SMAP protections that limit kernel access to user-space memory. No one should use this feature on deployment systems at this time, but rather restrict usage only to software testing environments.

NOTE: This feature is currently in progress.

You can provide a list of addresses that trigger PT trace dumps. Currently, we allow for up to 10 addresses to be used as triggers for dumping PT traces:

echo -n -e "[list of 64-bit addresses, up to 10, implicitly in hex]" | sudo tee /sys/kernel/debug/pt_trace_addresses

Each address needs to be 64 bits to fit a 64-bit address space, so 16 hex characters will suffice for each address. You will not separate the addresses by any character. Here's an example providing three addresses as triggers for 0x400c18, 0x400430, and 0x400400:

echo -n -e "0000000000400c1800000000004004300000000000400400" | sudo tee /sys/kernel/debug/pt_trace_addresses

And you can verify the addresses by using dmesg (the output is returned as an unsigned long long).

Running Griffin Trace with Multiple API calls

You can run Griffin Trace for any combination of the API calls, including all four at the same time. The only limitation is that forward-edge violations require the program policy generated by Griffin in order to work:

# Run a test with all capture options
echo -n attack-call.out > /sys/kernel/debug/pt_monitor
echo -n 6 > /sys/kernel/debug/pt_trace_proc_end
echo -n 2 > /sys/kernel/debug/pt_trace_syscall
echo -n 3 > /sys/kernel/debug/pt_trace_shadow_stack
echo -n 1 > /sys/kernel/debug/pt_trace_fwd_edge
LD_PRELOAD=`pwd`/griffin.so GRIFFIN_POLICY_PATH=`pwd`/attack-call.txt-policy.bin ./attack-call.out

# Capture both the PT log and the violation for extended annotations
cp /var/log/pt.log ./attack-call-fwd-edge.log
cp /var/log/pt.violation.log ./attack-call-fwd-edge.violation.log
echo -e "\x00" | tee /sys/kernel/debug/pt_monitor

A note on the buffer size

As of now, the Griffin API creates buffers that are 64KB in size (*). In order to keep up with longer running programs, we limit the total number of allocated buffers to restrict the total size of memory used by the kernel at any given time.

Currently, we can store up to 6 buffers at once, but due to PT's efficient data storage, these buffers can store thousands of block addresses.

(*) Griffin Trace stores 64KB of PT data per buffer, plus a page size (4096 B), which requires a 128KB block of kernel memory to store.

Reviewing the Trace

To read over the trace, you can run the PT trace tool provided in our Griffin kernel patch. The tool is located in tools/pt/ within the Griffin kernel.

First, you'll need to compile the tool.

$ make /path/to/griffin/tools/pt/

Then you can use the tool to produce a legible trace:

$ /path/to/griffin/tools/pt/pt /path/to/pt.log

This will produce a fairly large trace depending on the program, so you may want to feed the output into another file.

Running Griffin Trace Unit Tests

To run the Griffin unit tests (*), you can run the bash scripts that automate the test process. Alternatively, you can also load the latest versions of the tests in the code, and run the commands line-by-line. (See ./griffin-tests/shadow-stack.sh and ./griffin-tests/forward-edge.sh respectively.)

(*) "Unit test" is a misnomer - we aren't actually generating unit tests, but we're instead providing and testing fully worked examples that can be used to construct your own scripts for your programs.

Run the tests

Change your working directory to ./griffin-tests/, then run the following scripts:

sudo bash shadow-stack.sh
sudo bash forward-edge.sh

Annotate the PT output

sudo bash annotate-shadow-stack.sh
sudo bash annotate-forward-edge.sh

Reviewing the output

Each test triggers at least one system call. The shadow-stack test causes a shadow stack violation, and the forward-edge test causes a forward edge violation. Depending on which "triggers" are set (which conditions to watch for), different traces will be returned from Griffin Trace.