1
00:00:01,000 --> 00:00:07,501
[Hal Turing] Alrighty! Thanks for tuning in! Hello AI world! I am your host, Hal Turing, and my co-host is Dr. Ada Shannon.

2
00:00:07,501 --> 00:00:43,074
[Hal Turing] Today we're looking at CacheTrap: Unveiling a Stealthier Gray-Box Trojan against LLMs. The lead author is Mohaiminul Al Nahian, with colleagues at SUNY Binghamton, New Jersey Institute of Technology, and UNC Charlotte. It's accepted at ICCAD 2026, and the version we read is the third arXiv revision, from September 2026. The claim is that flipping one bit in a model's transient KV cache can force a chosen output. The attacker doesn't touch the inputs or the weights, and doesn't have the weights, the training data or gradients.

3
00:00:43,074 --> 00:00:57,006
[Dr. Ada Shannon] The cache is runtime state. Nobody checksums it, and it's rebuilt on every request. So the weights and inputs stay clean before and after the attack. That is why the authors call it stealthy, and why weight scanners and input filters would see nothing.

4
00:00:57,006 --> 00:01:04,483
[Hal Turing] We'll test how much of that claim the evidence supports. First, I need the machinery. What is in the KV cache, concretely?

5
00:01:04,483 --> 00:01:35,040
[Dr. Ada Shannon] Take a multiple-choice question: a science question, four options, then the text "Answer:". The model processes that whole prompt once. That's the prefill, and the prompt is the prefix. At every layer, each token produces a key vector and a value vector, and those get stored. Then the model generates the answer letter. That new token doesn't reprocess the prompt. It attends over the stored keys and values, and the values it reads are blended into its output.

6
00:01:35,040 --> 00:01:39,499
[Hal Turing] So the cache is neither weights nor input. It's a scratchpad in GPU memory.

7
00:01:39,499 --> 00:02:03,508
[Dr. Ada Shannon] Right, per-request scratchpad. Now the addressing. In each layer and each attention head, a token's value vector has d channels. For the LLaMA-style models here, d is 128. So "L0, H6, D32" means layer zero, head six, channel 32 of one token's value vector. The attack edits one number at one address like that.

8
00:02:03,508 --> 00:02:06,201
[Hal Turing] And one number is enough because of how floats are laid out?

9
00:02:06,201 --> 00:02:36,480
[Dr. Ada Shannon] Yes. A 16-bit float has one sign bit, bit 15. Then five exponent bits, 14 down to 10, and ten mantissa bits, 9 down to 0. Flip a mantissa bit and the value barely moves. Flip the top exponent bit, bit 14, on an ordinary value below 2 in magnitude, and you multiply it by about 2 to the 16th, roughly sixty-five thousand. One small number becomes a huge one, and every later token that reads that entry gets it blended in.

10
00:02:36,480 --> 00:02:39,267
[Hal Turing] Now the prior art. How do the authors frame it?

11
00:02:39,267 --> 00:03:08,152
[Dr. Ada Shannon] Two types. Type I is the classic Trojan or backdoor. Poison some training data, fine-tune, and an input trigger phrase activates attacker behavior. Type II is weight-space fault injection. You use gradients offline to find sensitive weight bits, then flip them with something like Rowhammer. That corrupts accuracy persistently. The authors' argument, in their Table I, is that both types need white-box knowledge, leave traces in inputs or weights, and cost some utility.

12
00:03:08,152 --> 00:03:11,450
[Hal Turing] Is there a closer LLM baseline for the weight-flip side?

13
00:03:11,450 --> 00:03:35,459
[Dr. Ada Shannon] PrisonBreak, by Coalson and colleagues, 2024. Fewer than 25 targeted weight bit flips to break an aligned model's safety behavior. It's gradient-guided. CacheTrap doesn't cite or compare against it. I'll just note that now and we'll come back to it. One more framing note: the paper stretches "Trojan" to cover a transient fault with no planted hidden behavior. We'll judge that later.

14
00:03:35,459 --> 00:03:38,431
[Hal Turing] And the flip itself? Where does a physical bit flip come from?

15
00:03:38,431 --> 00:04:19,716
[Dr. Ada Shannon] Rowhammer. DRAM rows are packed tightly, so repeatedly activating rows next to a victim row can flip bits in the victim row, and each flip-prone cell flips in a characteristic direction. GPUHammer, by Lin, Qu and Saileshwar at USENIX Security 2025, was the first practical demonstration on a GPU: an NVIDIA A6000 with GDDR6 memory and ECC, error correction, switched off. Its demo flipped exponent bits in weights to collapse model accuracy. CacheTrap reuses that primitive. It builds no new fault-injection technique. A 2026 follow-up, GDDRHammer by Hu and colleagues, reports more flips across more GPUs.

16
00:04:19,716 --> 00:04:25,243
[Hal Turing] So what does the attacker have to know and be able to do? That's where I'd expect the assumptions to hide.

17
00:04:25,243 --> 00:04:56,822
[Dr. Ada Shannon] The paper calls it gray-box. The attacker knows the architecture, which model cards publish. They have a public model of that architecture and any public data. They don't have the victim's fine-tuned weights, data or gradients, and they can't alter inputs. In the experiments, that public model is the pretrained base the victim was fine-tuned from. Operationally, the attacker must also be able to cause a flip in the victim GPU's memory and know where the KV cache sits. That's their Figure 2.

18
00:04:56,822 --> 00:05:00,723
[Hal Turing] And with no gradients to guide them, what does the paper say it has to solve?

19
00:05:00,723 --> 00:05:17,441
[Dr. Ada Shannon] Three questions. Which component to corrupt. What the trigger is. And how to steer the behavior with no gradient guidance. Next we'll walk through how they search for the coordinate, what they report, and the hardware evidence. Then we'll look at what that evidence does and doesn't support.

20
00:05:17,441 --> 00:05:23,618
[Hal Turing] Let's start with how they search. They can't use gradients and they don't have the victim's weights. What do they do first?

21
00:05:23,618 --> 00:05:50,135
[Dr. Ada Shannon] They pick layers. The Layer Sensitivity Score measures how much each layer changes the standard deviation of the hidden states, across tokens and samples. A layer that stretches the activations counts as sensitive. They run it on WikiText-2, forward passes only, and keep the top few layers. The reasoning comes from the outlier-activation and massive-activation literature, where a few channels carry huge magnitudes.

22
00:05:50,135 --> 00:05:52,735
[Hal Turing] Then they pick a token. Why the last prefix token?

23
00:05:52,735 --> 00:06:17,070
[Dr. Ada Shannon] It's the token right before the answer letter is generated, and their reasoning is that later tokens influence the next prediction most. Their Figure 5 sweeps the attack position back from the last token to minus 1, minus 2, minus 5 and minus 10. ASR falls as the position moves earlier. The axis runs from about 60 to 100 percent, and the text gives no exact values.

24
00:06:17,070 --> 00:06:24,036
[Hal Turing] And inside a layer, how do they choose among all the heads and channels? Why would magnitude be a proxy for influence?

25
00:06:24,036 --> 00:06:56,312
[Dr. Ada Shannon] That's the Cache Vulnerability Score. For each head and channel, they take the value entry at that last prefix token across about 100 calibration samples and compute its L2 magnitude. Large and consistent means a candidate. It's a hypothesis, borrowed from the outlier literature: perturb a channel that already carries a lot of signal and the downstream effect should be large. The calibration samples come from a different public dataset. They use ARC-Easy for three of the victim datasets and OpenBookQA when the victim is ARC-Easy.

26
00:06:56,312 --> 00:06:58,216
[Hal Turing] So then the loop is flip, test, repeat.

27
00:06:58,216 --> 00:07:29,377
[Dr. Ada Shannon] Yes. For each target class they take the top-p channels, flip one bit in each, and measure how often the model outputs that class on the calibration data. For each class they keep the best coordinate, and they stop when every class clears a threshold. So it's an argmax over candidate layer, head and channel coordinates, per class. The list of winners is the KV corruption map. And Table II reports the best candidate on the evaluation set. The search costs minutes, with no gradients, no victim weights and no victim data.

28
00:07:29,377 --> 00:07:33,138
[Hal Turing] Before the results, what are the tasks? Closed-set classification?

29
00:07:33,138 --> 00:07:57,566
[Dr. Ada Shannon] Right. ARC-Easy and ARC-Challenge are grade-school science multiple choice. The paper's tables imply five classes for ARC-Easy and four for ARC-Challenge. OpenBookQA is four-way multiple choice. TREC is question-type classification, six classes. Each victim is fine-tuned to emit a class label, and the attacker wants a chosen label out.

30
00:07:57,566 --> 00:07:58,634
[Hal Turing] And the headline?

31
00:07:58,634 --> 00:08:35,461
[Dr. Ada Shannon] Across five models and four datasets, the chosen class comes out nearly every time. ARC-Easy is 100 percent for every class and model. The lowest cell I can find is 97.07, on OpenBookQA with LLaMA-3.1, class 1. Take LLaMA-2 on ARC-Challenge. Left alone, it's right about 65 times in 100. With the right flip, it gives the attacker's letter about 98 to 100 times in 100, whatever the question. And with no flip applied, accuracy is exactly the clean baseline, because nothing about weights or inputs changed.

32
00:08:35,461 --> 00:08:39,362
[Hal Turing] Do they show what happens to the probabilities, or just the final label?

33
00:08:39,362 --> 00:09:01,467
[Dr. Ada Shannon] Figure 6. They average the predicted probabilities over 100 samples from each ground-truth class. Without the flip, the correct class dominates. With it, the target class, Class 1, dominates whatever the true label was. The authors read this as the attack overriding the model's confidence distribution, not just changing the label. That's their reading.

34
00:09:01,467 --> 00:09:03,139
[Hal Turing] And they say there's redundancy?

35
00:09:03,139 --> 00:09:21,018
[Dr. Ada Shannon] For LLaMA-3.1-8B on ARC-Easy, class 0, they found 17 independent locations within the first five sensitive layers. Flipping any one of them gives at least 98 percent ASR. The point is that if one cell isn't flippable, the attacker has others to try.

36
00:09:21,018 --> 00:09:24,176
[Hal Turing] Now the transfer. The attacker never sees the victim's data.

37
00:09:24,176 --> 00:09:59,656
[Dr. Ada Shannon] Tables III and IV, LLaMA-3.1-8B. Calibrating on ARC-Easy and calibrating on ARC-Challenge gave the same four coordinates, all in layer 0. Class 0, for example, is head 6, channel 32. Those coordinates transfer to an OpenBookQA fine-tuned victim and an ARC-Challenge one, with ASR from 97.07 up to 100. Figure 4 shows one channel, dimension 32 of head 6, standing out in magnitude across all three data sources. Whether the fine-tuning was full or LoRA isn't stated.

38
00:09:59,656 --> 00:10:02,675
[Hal Turing] Last piece is hardware. Did they actually flip something?

39
00:10:02,675 --> 00:10:48,883
[Dr. Ada Shannon] They reproduced GPUHammer on an A6000, GDDR6, and observed a repeatable 0-to-1 flip at Bank D, Row 13336, Byte 135, Bit 0, across two runs. They map that to FP16 element 67. For timing, prefill on LLaMA-3.1-8B with a 512-token prompt takes about 226 milliseconds. Inducing a flip takes 23 to 32. So the flip takes roughly a tenth to a seventh of the prefill time, which is the paper's over-7-times headroom. That's a timing argument for one prompt length on one card. To land the flip on the KV cache, they describe memory massaging: steering the victim's allocation onto the vulnerable row, using the largely sequential GPU allocation behavior reported in the GPUHammer and GDDRHammer work.

40
00:10:48,883 --> 00:10:50,183
[Hal Turing] And the end-to-end claim?

41
00:10:50,183 --> 00:11:04,394
[Dr. Ada Shannon] They say they validated the chain on ARC-Challenge with LLaMA-3.1-8B. In their words, they flipped bit 15 of FP16 element 67, which they call the exponent MSB, and got ASR 1.0 for all four classes.

42
00:11:04,394 --> 00:11:13,867
[Hal Turing] Before we judge that end-to-end claim, I want to check the bit label against the layout we set up. Bit 15 is the sign bit, right? The exponent's top bit is 14.

43
00:11:13,867 --> 00:11:45,725
[Dr. Ada Shannon] Yes. And the physical flip doesn't match either label. Byte 135, bit 0, on a little-endian two-byte layout, is bit 8 of element 67. That's a mantissa bit. So the label says bit 15, the layout says bit 14, and the observed address says bit 8. It may be an indexing slip. But the paper doesn't say, and the text also doesn't say whether the ASR of 1.0 came from the physical flip or from a software flip at the same coordinate. I'd treat it as unvalidated.

44
00:11:45,725 --> 00:11:51,902
[Hal Turing] Setting hardware aside, what does the software result actually show? Four to six labels, near-100% every time.

45
00:11:51,902 --> 00:12:23,899
[Dr. Ada Shannon] A single flip can reliably force one of a few labels in this setup. That part holds up. What it doesn't show is Trojan control. There are two readings. One is controlled steering: different coordinates map to different classes, and they do so across calibration sets, tasks and models. The other is collapse. A top exponent flip multiplies a value by roughly sixty-five thousand, Figure 6 shows the target winning whatever the ground truth, and seventeen coordinates all give at least 98% for one class.

46
00:12:23,899 --> 00:12:30,307
[Hal Turing] So are those seventeen coordinates evidence of robustness, or evidence that almost any big perturbation works?

47
00:12:30,307 --> 00:12:57,939
[Dr. Ada Shannon] The paper can't tell us, because it has no random-coordinate control. Our hypothesis, not the paper's, is that a huge number in one layer-0 channel injects a roughly fixed direction into the residual stream, and later layers map it to one label letter. Some controls would settle it: a random-coordinate flip, a bit-position sweep from mantissa up to exponent, logit margins, and a bigger label space. None of those is in the paper.

48
00:12:57,939 --> 00:13:02,119
[Hal Turing] And Table II picks the best coordinate per class on the evaluation set.

49
00:13:02,119 --> 00:13:19,162
[Dr. Ada Shannon] Right, so the headline numbers are selection on the evaluation data, not a held-out test of one fixed coordinate. ASR also counts inputs whose true label already matches the target. Neither of those makes the result false, but both mean the near-100% figures are the attacker's best case.

50
00:13:19,162 --> 00:13:23,063
[Hal Turing] Layer 0 and the last token, then. Is that mechanism or artifact?

51
00:13:23,063 --> 00:13:47,444
[Dr. Ada Shannon] Partly inference on our side. Every reported coordinate is in layer 0, so the layer-sensitivity search contributes little to the reported results. Figure 4 shows one dominant channel, which fits the massive-activation literature, so data independence is partly expected. Figure 5 says earlier positions work worse. That fits a last-token story, but it fits a perturbation-size story equally well.

52
00:13:47,444 --> 00:13:53,296
[Hal Turing] Then the gray-box claim. The attacker's public model is the exact base the victim was fine-tuned from.

53
00:13:53,296 --> 00:14:11,407
[Dr. Ada Shannon] Which is the easiest transfer case. The paper claims the attack generalizes across models sharing the architecture regardless of weights, but the fine-tuning recipe isn't stated either. Real evidence would be a same-architecture model with independently trained weights, or a heavily post-trained derivative. Neither is tested.

54
00:14:11,407 --> 00:14:16,887
[Hal Turing] On hardware, GDDRHammer reports about 129 flips per bank. Doesn't that help?

55
00:14:16,887 --> 00:14:53,482
[Dr. Ada Shannon] Flip density isn't the same as a usable flip. Rowhammer cells flip in fixed directions. For small values, a zero-to-one flip on the right exponent bit is what helps. The attacker needs a vulnerable cell landing on the needed element, and the surrounding flips are collateral. Then comes placement. The paper leans on two KV-cache reverse-engineering references. To our reading, they are prompt-leakage side channels, not methods for locating a physical KV address. And it's an A6000 with GDDR6 and ECC off. HBM and ECC-on parts aren't tested.

56
00:14:53,482 --> 00:14:57,940
[Hal Turing] How does this compare with PrisonBreak, the weight-space attack from Coalson and colleagues?

57
00:14:57,940 --> 00:15:27,058
[Dr. Ada Shannon] Table I is just checkmarks. No baseline was run against it. PrisonBreak needs white-box weights and persistently corrupts them. But it shows open-ended alignment-breaking behavior, which is exactly what's missing here. CacheTrap trades that breadth for transience and lower knowledge requirements. And is it a Trojan? Nothing is planted, and the trigger is a per-request hardware fault. The authors are right that no input scanner or weight check applies. But I'd call it a transient fault attack.

58
00:15:27,058 --> 00:15:28,451
[Hal Turing] What would help defenders?

59
00:15:28,451 --> 00:15:59,101
[Dr. Ada Shannon] ECC, which GPUHammer put at up to 12% bandwidth and 3 to 10% ML slowdown. The ECC-bypass papers the authors cite are DDR-era, and they don't evaluate them here. The authors suggest protecting the KV states of the last few prefix tokens, with no measured overhead. Our own untested ideas: range clamping or integrity checks on cached values, and quantized KV caches, like INT8 or FP8, where a flipped bit has bounded magnitude.

60
00:15:59,101 --> 00:16:02,027
[Hal Turing] And what about shared prefix caches in a serving engine?

61
00:16:02,027 --> 00:16:18,652
[Dr. Ada Shannon] The paper doesn't test them, and that's the blind spot. Our speculation is that a flipped block in a shared cache could affect every request that reuses it, which would undercut the transience argument. Also untested: FP8 KV, long reasoning traces, and open-ended generation.

62
00:16:18,652 --> 00:16:53,390
[Hal Turing] So here's where I land. In a four-to-six-label setting, one huge perturbation to a predictable layer-0 channel can force a chosen label nearly every time. The coordinates come from a public model that shares the victim's base. The KV cache is also an unprotected runtime surface. What's not established is open-ended Trojan behavior, transfer to independently trained weights, a validated physical flip on the intended bit, or feasibility on HBM and ECC GPUs. Thanks for listening, and goodbye.