R2D2 Lab

DNA system tutorial

This page explains how the DNA components fit together, how to build them, and how to run experiments. For deeper context, see the dissertation (Chapter 8: “Systems Integration”): Dissertation_Gifford.pdf.

How DNA works (high level) (original DNA paper): DNA periodically applies custom resource allocations to the currently running tasks. A single-core hrtimer tick wakes the DNA loop, which:

  • Sees which task is running on each core and updates its metadata (instruction sums from PMU counters).
  • Runs the DNA allocation algorithm using the task’s workload ID and uploaded phase/θ profiles.
  • Programs CAT (cache ways) and MemGuard (bandwidth) per core to enforce the chosen allocation.
Workloads identify themselves once via PR_SET_DNA_WORKLOAD_ID; user space preloads the phase/θ tables via prctl; the kernel metadata and DNA module do the rest at runtime. This tutorial introduces each moving part and how to run it on real hardware, with deeper context in module docs, the DNA paper, and the dissertation.

Component overview

The architecture below shows how the different modules fit together at a high level (see architecture figure):

If the embed does not load, open the PDF directly.

  • Kernel support (linux-research-mirror): embeds DNA metadata in task_struct, exposes prctl entry points, and accounts retired instructions when CONFIG_DNA_SUPPORT=y inside the kernel config file (more below on this). It is the anchor that lets user space push phase/θ data and tag workloads; all other components plug into this foundation. For important files, see ./kernel/dna.c, ./include/linux/dna.h, and ./include/uapi/linux/dna.h
  • DNA module (dna-linux-module): plugs into the kernel hooks, runs the allocation loop, programs Intel CAT for cache ways, drives hrtimers to sample instructions, and forwards bandwidth updates to MemGuard. It registers the handlers that the kernel prctl path calls, so uploads land in the module’s tables. For important files, see ./src/dna_module.c and ./run_dna_experiment.sh
  • MemGuard (memguard): enforces per-core memory bandwidth budgets. DNA calls into MemGuard (via exported hooks) whenever it adjusts per-core allocations, keeping cache and bandwidth decisions aligned.
  • PARSEC workloads (parsec-3.0): the applications run under DNA’s control. Workloads are tagged with IDs so DNA knows which phase/θ tables to apply; the kernel metadata stores these IDs per task.
  • Profiles (rt_profiling/): legacy phase/θ tables per workload/cache/bandwidth point. dna_tool uploads these into the kernel via prctl so the module can drive allocations based on instruction progress.
  • Experiment scripts (per repo): dna/run_dna_experiment.sh (DNA repo) coordinates uploads, tagging, resctrl, MemGuard, and RAPL power summaries; rt_profiling/run_profiler.sh (rt_profiling) drives perf/PMU profiling with CAT+MemGuard; PARSEC’s own scripts build and stage binaries.
  • Data flow: User space uploads profiles → kernel stores descriptors → DNA module consumes them via registered handlers → DNA module writes cache masks (CAT) and bandwidth budgets (MemGuard) → workloads run under SCHED_DEADLINE with per-task metadata updated on context switches.
  • Control flow: Workload tags itself via prctl → timers and scheduler accounting update instruction totals → DNA loop decides new cache/bandwidth allocations → MemGuard enforces bandwidth, resctrl/CAT enforces cache ways; results and power stats are exported for analysis.

Build steps

Start by compiling and installing the custom kernel—this provides the in-kernel DNA plumbing that all other pieces rely on. You only need to repeat the kernel build/install when you change kernel source code; user-space pieces can be rebuilt independently. Anytime you recompile the kernel, be sure to recompile each module after rebooting into the new kernel. Unless otherwise noted, any commands or directory paths are relative to that modules root directory, so for Linux, run the commands within the base linux-research-mirror directory.

1) Kernel with DNA support

Ensure you have a valid kernel configuration (.config) with PREEMPT_RT and DNA enabled. .config drives every build option; without it the build will fail or miss required features.

  • On small/big cat machines: a correct .config already exists. This setup is needed only once on a fresh machine.
  • To seed a new machine, copy the current distro config and adjust: 
    cd linux-research-mirror
cp /boot/config-$(uname -r) .config
make menuconfig   # enable PREEMPT_RT, CONFIG_DNA_SUPPORT under General setup

Build and install (run as root for install steps):

cd linux-research-mirror
sudo make bzImage -j$(nproc)
sudo make modules -j$(nproc)
sudo make modules_install -j$(nproc)
sudo make install -j$(nproc)

Reboot and select this kernel in the GRUB menu. If desired, set it as the default entry in /etc/default/grub (update with sudo update-grub), so subsequent boots use the DNA-enabled kernel automatically. For additional context, the first make command (bzImage) compiles the core kernel components, this includes most code found under the ./kernel/ directory path. We then compile the in-tree kernel modules via the make modules command. We then install the modules and core kernel image onto the current machine with the two install commands. The -j flag is to tell Make to use multiple threads to compile faster, and we use the number of enabled processors which is given by $(nproc).

2) MemGuard

Build MemGuard against the same kernel tree used above, after rebooting. This provides the bandwidth enforcement that DNA calls into.

cd memguard
make clean; make
# loaded later in the runscripts with: sudo insmod memguard.ko

3) DNA module and tools

Build the out-of-tree DNA kernel module plus the uploader utility that pushes phase/θ tables via prctl.

cd dna
make clean; maked
# outputs dna_module.ko and dna_tool, both used by runscripts

4) PARSEC workloads

Compile the PARSEC applications used in the experiments so run_dna_experiment.sh and profiling scripts can launch them. Parsec uses its own build tool chain via a program called parsecmgmt, which is located under ./bin. More details about PARSEC can be found online.

cd parsec-3.0
./bin/parsecmgmt -a clean -p canneal
./bin/parsecmgmt -a uninstall -p canneal
./bin/parsecmgmt -a build -p canneal
	# This will build the canneal application.

Running experiments

The easiest way to run an experiment is with ./run_dna_experiment.sh. This script has many configuration options, which you can explore by calling it with no arguments. The most fundamental way to run a set of experiments is to pass it the paths to the directories that contain the individual tasksets and phase and theta information.

For example: sudo ./run_dna_experiment.sh ../../rtas26_sched_results/tasksets/tasksets/ ../../data/all_thresholds_empirical/

If you explore the taskset directory above, you will notice that the script is able to take a parent directory that contains subdirectories that vary their taskset utilization. The script is able to process each taskset utilization subdirectory in order. The output is saved by default to ~/output/, and the runscript will check this direcotry and avoid rerunning any experiments that already have output generated, letting you stop and resume experiments in case you need to reboot the machine.

Profiling a new task

To gather raw data for phase/θ tables, profile the workload with the provided perf/PMU helper. This lives in the rt_profiling tree and exercises the task under CAT+MemGuard while sampling counters.

Using run_profiler.sh (rt_profiling)

cd rt_profiling
sudo ./run_profiler.sh <tag> <runtime_ns> -- <command> [args...]

Example:

sudo ./run_profiler.sh canneal 20000000 -- ./targets/canneal input args...

What it does:

  • Sets up SCHED_DEADLINE for the target and pins it to a CPU.
  • Programs CAT/MemGuard budgets for the run (using your env defaults or overrides).
  • Runs perf stat with configurable events (default: instructions, LLC-loads, LLC-load-misses) at a fixed interval (PERF_INTERVAL_MS, default 10 ms).
  • Collects logs under a tag-specific output directory for later analysis into phase/θ tables.

Tune via environment variables (see script header): CPU_INDEX, PERF_EVENTS, PERF_INTERVAL_MS, DVFS_START_MHZ/DVFS_END_MHZ/DVFS_STEP_MHZ, MEMBW_START/MEMBW_STEP/MEMBW_BASE_OTHER, and paths to PARSEC binaries/inputs. Use the collected counter traces to derive per-phase instruction regions and θ values before feeding them to dna_tool.

Please be aware that a small modification is needed within the application itself to tell our profile tool to start and end recording. You can see examples of this within currently supported PARSEC applications.

References

  • dna/README.md – module overview and quick start (dna-linux-module repo)
  • dna/dna_module.c – source code overview and function interactions
  • dna/run_dna_experiment.sh – experiment orchestration (dna-linux-module repo)
  • dna/analyze_dna_bench.py – parsing/plotting results (dna-linux-module repo)
  • memguard/README.md – bandwidth control and debugfs interface (memguard repo)
  • rt_profiling/run_profiler.sh – perf-based profiling helper (rt_profiling tree)