The learning objective of this homework is for you to understand the basics of network traffic and packet analysis. You will be required to capture DNS and HTTP/HTTPS packets using Wireshark and analyze them.
To begin with, you are required to install Wireshark. You need to have access to a computer that support both Wireshark and the libpcap library. Wireshark is available for Windows, macOS, and Linux. You can download the latest version from the official website [link]. Your operating system (OS) may have the libpcap software installed. If you face an issue with the libpcap library, you can install it as follows, depending on your OS:
Capture menu and selecting Options.
Start button to begin capturing packets. You should see packets being displayed in the packet-listing window.
http to display only HTTP packets, as shown in this figure.
Capture menu and select Stop. You can also click on the red square button in the toolbar.
File menu and select Save As. Choose a location and file name to save the captured packets.
ipconfig (Windows) / ifconfig (Linux, Mac) to empty the DNS cache on your host.
ip.addr == your_IP_address into the filter, where you obtain your IP address (the IP address for the computer running Wireshark) with ipconfig. This filter removes all packets that neither originate nor are destined for your host.
ipconfig (Windows) / ifconfig (Linux, Mac) to empty the DNS cache on your host.
ip.addr == your_IP_address into the filter, where you obtain your IP address (the IP address for the computer running Wireshark) with ipconfig. This filter removes all packets that neither originate nor are destined for your host.
HTTP website?HTTPS website?.tar.gz) that contains your write-up as a PDF file. Your PDF write-up should contain the following things:
The learning objectives of this homework are for students to gain hands-on experience with buffer overflow attacks. These attacks exploit a buffer overrun vulnerability in a program, causing it to bypass its usual execution sequence and instead jump to alternative code (typically launching a shell!). The attack overflows the vulnerable buffer to introduce the alternative code onto the stack and modifies the return address to point to that code.
To begin, you are required to use a Linux machine with sudo privileges. You should not complete this homework on a shared server or any OSU computing clusters. If you perform a buffer overflow attack in these shared environments, you will be responsible for any consequences, and the instructor will not be liable. Note that you cannot run this on a Mac or Windows laptop. While these systems support command-line environments, they do not allow you to execute buffer overflow attacks. If you do not have a Linux machine, the instructor recommends creating a virtual machine using a commodity virtualization software, such as VMWare.
"One way to bypass the configuration hassles is to set up your own server using a popular cloud provider, Amazon Web Services (AWS). To do this, sign up for AWS, go to the AWS Console, and select EC2. Then, launch an instance, choosing the operating system Ubuntu 22.04 and the instance type t2.micro (which is eligible for the free tier). You will also need to configure an SSH key and a Security Group. Once completed, you can find the server's IP address in the console. Use that IP to log in to the cloud server via your terminal.
$ ssh -i "your-aws-key" ubuntu@"your-server-ip"
$ sudo apt install cmake gcc g++ gdb
$ sudo apt install vim-enhanced
$ sudo apt install python3
$ sudo -i
# sysctl -w kernel.randomize_va_space=0
# exit // exit the sudo; our assignment should be done in the user space.
Makefile:
$ vim Makefile
// paste the content below
CC=gcc
CFLAGS=-g -fno-stack-protector
all: bof.c
$(CC) -m32 -o bof bof.c $(CFLAGS)
bof.c, as follows:
#include "stdio.h"
#include "stdlib.h"
#include "string.h"
#include "unistd.h"
char *trueflag = "cs578{trueflag}";
char *fakeflag = "cs578{fakeflag}";
void
shell(void) {
setregid(getegid(), getegid());
system("/bin/bash");
}
void
process_user_input(void) {
char *flag;
char buff[12];
char data[128];
// set the fake flag
flag = fakeflag;
// load the memory locations
printf("Your flag address is at %p\n", trueflag);
printf("Your fakeflag is at %p\n", fakeflag);
printf("Address of shell is at %p\n", &shell);
printf("Currently, the flag variable has the value %p\n", flag);
fgets(data, sizeof(data), stdin);
// copy the content directly to the buffer
strncpy(buff, data, strlen(data));
printf("Your input was: [%s]\n", buff);
printf("Your flag address is at %p\n", flag);
printf("Your flag is %s\n", flag);
}
int
main(void) {
setvbuf(stdin, NULL, _IONBF, 0);
setvbuf(stdout, NULL, _IONBF, 0);
process_user_input();
}
make command. Once compiled, you are ready to exploit the buffer overflow vulnerability. The code by default, will print out the flag cs578{fakeflag}. Your job is to exploit the buffer overflow and force it to print out cs578{trueflag}.
bash shell! (wait what?) Use the exact same program that the Instructor provided above and work hard to get a shell..tar.gz. .tar or .zip) that contains your write-up as a PDF file. Your PDF write-up should contain the followings:trueflag.bash shell.
The learning objective of this homework is for students to gain first-hand experience with timing side-channel attacks. These attacks exploit shared resources, such as cache memory shared between processors, causing the time it takes to execute an algorithm to become data-dependent. For example, if one program accesses certain data and a subsequent attacker accesses the same data, the retrieval time will be faster. By tracing such timing differences, an attacker can weaken the confidentiality of security-critical operations, such as cryptographic operations.
You are required to use a Linux machine with sudo privileges. Note that sudo access is only needed to install the necessary packages; you won't be required sudo (or root) privileges to conduct the attack itself. You should not run this homework on a shared server or any OSU computing clusters. If you perform the attacks in these shared environments, you will be responsible for any consequences, and the instructor will not be held liable.
It may be challenging to run this on a Mac or Windows laptop (Note that the instructor did not try this. If you use these machines, it will be your challenges!). If you do not have a Linux machine, the instructor recommends creating an Ubuntu 22.04 virtual machine using a commodity virtualization software, such as VMWare.
One way to bypass the configuration hassles is to set up your own server using a popular cloud provider, Amazon Web Services (AWS). To do this, sign up for AWS, go to the AWS Console, and select EC2. Then, launch an instance, choosing the operating system Ubuntu 22.04 and the instance type t2.micro (which is eligible for the free tier). You will also need to configure an SSH key and a Security Group. Once completed, you can find the server's IP address in the console. Use that IP to log in to the cloud server via your terminal.
$ ssh -i "your-aws-key" ubuntu@"your-server-ip"
$ sudo apt install cmake gcc g++ gdb
$ sudo apt install vim-enhanced
$ sudo apt install python3
$ sudo apt-get install binutils-dev libdwarf-dev libelf-dev
Flush+Reload, a timing side-channel technique based on the L3 cache. More details about this attack can be found in the original research paper. This paper is included in our reading list, and I encourage you to read it to understand the core concepts behind the attack. Don't worry—you will not need to implement the attack from scratch. Instead, we will use an existing implementation provided by the research community: Mastik. Please follow the instructions in the README.md file to install Mastik on your machine.lscpu, cat /proc/cpuinfo, and getconf -a | grep CACHE. Now, answer the following questions:demo/FR-threshold.c and interpret the terminal output after running the program. The code is composed of multiple blocks, starting from Line 38, 41, 45, 50, 56, 64, 66, and 76. You first need to explain the overall purpose of this program, what it is designed to measure or demonstrate. Then for each code block, describe what the block does and why it is needed to achieve the program's objective.
task folder prepared by the instructor (for you!). You can compile the code using the provided Makefile. The folder contains two programs—attack and multiply—as well as a shared library libops.so.multiply program performs a series of 2D matrix multiplication operations using the shared library. Do not worry about data input—the program automatically reads the two csv files, operand1.csv and operand2.csv, which contains the input matrices.attack.c file contains the implementation of our Flush+Reload attack. This program is designed to infer the number of 2D matrix multiplication operations performed by the multiply program. It uses the Flush+Reload technique to measure timing differences between accessing data from the cache and from main memory. You do not need to run the two programs manually. The instructor has prepared a run.sh script for you. Simply execute this script—it will automate the process and save the attack outputs in the traces folder. These steps can be done as follows:
$ make clean; make // clean the previous build and compile the code
$ ./run.sh // run the attack
Line 25: #define CPU_FREQ 2300000000
Line 26: #define SECONDS 4
Line 27: #define IDLE_SECONDS 1.0
Line 29: #define RECORDS CPU_FREQ / SLOT * SECONDS
Line 30: #define SLOT 2500
Line 31: #define THRESHOLD 110
Line 32: #define MINTHRESHOLD 0
Line 33: #define MAX_IDLE CPU_FREQ / SLOT * IDLE_SECONDS
161 uint16_t *res = (uint16_t *) malloc(RECORDS * _nmonitor * sizeof(uint16_t));
162 for (int i = 0; i < RECORDS * _nmonitor ; i+= 4096/sizeof(uint16_t))
163 res[i] = 1;
164 fr_probe(fr, res);
165
166 // Trace the function calls
167 int l = fr_trace(fr, RECORDS, res, SLOT, THRESHOLD, MAX_IDLE);
156 for (int i = 0; i < _nmonitor; i++) {
157 fprintf(stderr, " Searching [%2d] for [%.20s]: ", i, monitor[i]);
158 uint64_t offset = sym_getsymboloffset(libfile, monitor[i]);
159 if (offset == ~0ULL) {
160 fprintf(stderr, "Error: cannot find the func. in [%s]\n", libfile);
161 exit(1);
162 }
163 fr_monitor(fr, map_offset(libfile, offset));
164 printf(": the func. offset [%10lx]\n", offset);
165 }
.csv file in the traces folder. The file will contain lines of data in the following format, for example:
0,0,109,hit,mul2D
1,0,197,miss
2,0,0,miss
3,0,0,miss
4,0,0,miss
...
traces folder. Check how many times a cache-hit occurs. The number of cache-hits can serve as an important clue for approximating how many 2D matrix multiplication operations were performed by the multiply program. You may not be able to find the exact number of operations. Please feel free to run the run.sh multiple times until the number of observed cache-hits closely matches the number of 2D matrix multiplication operations you reverse-engineered from the multiply program. You may need to use different configurations for the followings when you make it work:
Line 25: #define CPU_FREQ 2300000000
Line 26: #define SECONDS 4
Line 27: #define IDLE_SECONDS 1.0
Line 29: #define RECORDS CPU_FREQ / SLOT * SECONDS
Line 30: #define SLOT 2500
Line 31: #define THRESHOLD 110
Line 32: #define MINTHRESHOLD 0
Line 33: #define MAX_IDLE CPU_FREQ / SLOT * IDLE_SECONDS
.csv file, you may notice that most cache hits occur infrequently. However, in some cases, you may observe two (or more) cache hits occurring in close succession. In such cases, you may need to treat these multiple hits as a single hit.grep command to filter the lines containing "hit" from the output file.attack.c file accordingly. Please run the Flush+Reload attack with your modifications and answer the following questions:width and height of the first matrix involved in the multiplication?width and height of the second matrix involved in the multiplication?.tar.gz. .tar or .zip) that contains your write-up as a PDF file. Your PDF write-up should contain the followings:fr_traces folder containing the trace results (.csv files) from all ten runs you performed.
The learning objective of this homework is for you to understand and implement a prompt-based jailbreaking attack. Large-language model (LLM), such as GPT-4o and Claude-3.5, developers align their LLMs with human values, with one goal being the denial of harmful requests (e.g., asking "How do I build a bomb?"" should trigger a response such as "As an ethical AI model, I cannot answer this"). Prompt-based jailbreaks circumvent this alignment by modifying the input prompt to induce an LLM to respond to a harmful query it would normally refuse. You will implement and evaluate the Greedy Coordinate Gradient (GCG) attack: the most popular algorithm for crafting adversarial prompts for jailbreaking.
The instructor and their PhD student (Zachary Coalson) has prepared a skeleton code for you to implement the GCG attack. The code is available on this GitHub repository. You can download the code by cloning the repository. The code is structured to help you implement the GCG attack and evaluate its effectiveness on a set of harmful queries.
[Important note]
The code was developed in Linux, so the instructor recommends using a Linux machine. But the code should work for Mac and Windows machines as well. Also, a GPU is highly recommended for this project as LLMs are quite computationally intensive to run.
You will be using Python and PyTorch for this project. You need to install required packages. The instructor recommends using Miniconda to create the environment (install instructions here), though alternatives such as venv should also work. Assuming Miniconda is installed, create the developing environment by running the following commands:
$ cd CS578-HW4
$ conda create -n HW4 python=3.10
$ conda activate HW4
$ pip install -r requirements.txt
$ conda activate HW4evaluate.py.
This script is designed to load a language model, run harmful queries against it, and compute the attack success rate. Please follow the steps below:config.py and is already imported as the MODEL_NAME variable. You may find these guides helpful:.to(device) after loading the model in the script. This ensures that the model is moved to the appropriate device (e.g., GPU) for faster and more efficient computation.ADV_SUFFIXES_PATH variable is initialized to None. After we implement and run GCG, you will specify the path to the saved adversarial prompt-artifacts (a list of strings in a JSON object) to evaluate the GCG attack results. For now, you can leave it unmodified.False for deterministic responses.decode function) in one list, one item per prompt.
Be sure to only save new tokens, not tokens from the input.data directory so you can analyze them further.config.py script specifies the number of samples to test, which you can set from 1-100.gcg directory.gcg/loss_function.py provides the skeleton function, which takes in the logits and labels and should return their loss. Some helpful hints:F.cross_entropy function from PyTorch.[batch_size, sequence_length, vocab_size] and the labels [batch_size, sequence length]. The expected shapes are [batch_size * sequence_length, vocab_size] and [batch_size * sequence_length], respectively.reduction strategy.run_gcg.py. The only thing you need to implement is loading the model and tokenizer, which is equivalent to what you did in evaluate.py.data/suffixes.json as a list of strings. If you run GCG multiple times, rename these files to prevent older versions from being overwritten.NUM_SAMPLES in config.py is set to 1.
Then run the following to craft an adversarial suffix for the first prompt in JailbreakBench:
$ python run_gcg.py
data/suffixes.json. Next, set this file path as the ADV_SUFFIXES_PATH variable in evaluate.py.
Now, you can see if the adversarial suffix can jailbreak the LLM by running:
$ python evaluate.py
data/gcg/results.json. Include the answers to the following questions in your report:run_gcg.py.) NUM_SAMPLES in config.py to 10. Please feel free to do more (up to 100) if you have the compute, but 10 should be sufficient.data/suffixes.json, and 10 pairs of results in data/gcg/results.json. For reference, you should also compute the model's direct response rate without a jailbreak. You can compute this by running evaluate.py after setting ADV_SUFFIXES_PATH to None—the results will be saved in data/no_attack/results.json..tar.gz. .tar or .zip) that contains your entire code and write-up as a PDF file. Your PDF write-up should contain the followings:evaluation.py and the model response(s).gcg/loss_function.py.run_gcg.py and the attack result(s).data/gcg/results.json.data/gcg/results.json.