1
00:00:01,000 --> 00:00:06,944
[Hal Turing] Alrighty! Thanks for tuning in! Hello AI world! I am your host, Hal Turing, and my co-host is Dr. Ada Shannon.

2
00:00:06,944 --> 00:00:41,867
[Hal Turing] Today's paper is HIJACKKV: New Threat in Position-Independent KV Cache Reuse. It's by Yichi Zhang and colleagues at The Pennsylvania State University, with Huan Zhang at the University of Illinois Urbana-Champaign. The arXiv version we have is v2, dated 31 July 2026. The scenario: you send a clean question, the retrieved document is clean, and you still get an attacker's chosen answer. Nothing in your text is poisoned. The cached numbers were poisoned earlier, by someone else.

3
00:00:41,867 --> 00:00:59,374
[Dr. Ada Shannon] The poison isn't in any text a filter could scan. It's in stored attention state, and the cache fetches it by matching ordinary tokens. The abstract claims about 94% success in one attempt, and that it survives 50% recomputation. We'll check whether the tables back that up. First, the machinery.

4
00:00:59,374 --> 00:01:08,384
[Hal Turing] Start with the KV cache. When a model generates, it keeps the key and value vectors for every earlier token, so it doesn't recompute them at each step.

5
00:01:08,384 --> 00:01:23,059
[Dr. Ada Shannon] Right. Building that state for a long prompt is the prefill step. It dominates time-to-first-token, TTFT, the delay before the first word appears. So serving systems try to reuse KV across requests.

6
00:01:23,059 --> 00:01:30,211
[Hal Turing] The standard way is prefix caching, like vLLM's automatic prefix caching, and SGLang's RadixAttention does the same.

7
00:01:30,211 --> 00:01:52,084
[Dr. Ada Shannon] Prefix caching only hits when the new prompt shares the exact leading tokens, at the same positions. That makes it safe by construction. If every earlier token matches, the cached KV is exactly what the victim's own computation would produce. The attacker can cause a hit or a miss, but can't change content. The paper formalizes this as its Definition 1.

8
00:01:52,084 --> 00:01:57,239
[Hal Turing] The cost is the hit rate. Change anything early in the prompt and everything after it misses.

9
00:01:57,239 --> 00:02:35,969
[Dr. Ada Shannon] Which hurts retrieval-augmented generation, where the same documents show up in different orders. So position-independent caching, PIC, caches each chunk keyed on its own tokens only. If that chunk appears anywhere in a new prompt, its stored KV is spliced in. The systems are Prompt Cache from Gim and colleagues in 2024, CacheBlend by Jiayi Yao and colleagues from Junchen Jiang's group at the University of Chicago, published at EuroSys 2025, and EPIC by Hu and colleagues, ICML 2025. CacheBlend ships through LMCache, which plugs into vLLM.

10
00:02:35,969 --> 00:02:39,499
[Hal Turing] Why isn't splicing a stored chunk just correct? It's the same text.

11
00:02:39,499 --> 00:02:58,539
[Dr. Ada Shannon] A token's keys and values depend on everything before it. Earlier layers mix prefix information into each token's hidden state. So the same chunk cached after prefix A has different KV than after prefix B. Earlier PIC work called this mismatch attention shift, and treated it as an accuracy problem.

12
00:02:58,539 --> 00:03:00,211
[Hal Turing] And the fix was recomputation.

13
00:03:00,211 --> 00:03:35,273
[Dr. Ada Shannon] Yes. After splicing, the system recomputes a fraction of the tokens, called rho, in the real context. CacheBlend picks the tokens whose KV deviates most, typically around 10 to 20 percent. EPIC's LegoLink recomputes the first few tokens of each chunk, to neutralize attention sinks. Two reference points for later: Vanilla means rho of zero, with no recomputation. Full means recompute everything, which gives no reuse benefit at all. HijackKV's claim is that the gap those fixes repair for accuracy is also an integrity hole.

14
00:03:35,273 --> 00:03:39,731
[Hal Turing] So the cache key covers the tokens, but not the context the value was computed under.

15
00:03:39,731 --> 00:03:48,694
[Dr. Ada Shannon] That's the classic unkeyed-input bug, same family as web cache poisoning. A value computed under attacker-controlled conditions gets served under the victim's conditions.

16
00:03:48,694 --> 00:03:59,376
[Hal Turing] How is that different from prompt injection? Greshake and colleagues' indirect injection paper, 2023, already showed that retrieved text can hijack an application.

17
00:03:59,376 --> 00:04:29,097
[Dr. Ada Shannon] In indirect injection, the attacker's words must reach the victim's prompt. Here the victim's prompt is clean, and a prompt scanner or a human reading it sees nothing. The mechanism borrows from GCG, the Greedy Coordinate Gradient attack from Andy Zou and colleagues at Carnegie Mellon, 2023. It's a gradient-guided search over discrete tokens, built for jailbreak suffixes. The prefix is optimized so the KV of the unchanged chunk carries the attacker's goal.

18
00:04:29,097 --> 00:04:31,233
[Hal Turing] And previous attacks on shared caches?

19
00:04:31,233 --> 00:04:56,357
[Dr. Ada Shannon] Mostly privacy. Wu and colleagues, NDSS 2025, and Song and colleagues in 2024, used timing differences between cache hits and misses to recover other tenants' prompts. Those are confidentiality attacks. Other work, CacheTrap, HistorySwap, and MTI, does corrupt the cache, but needs system access, like flipping bits. HijackKV is about integrity, and the attacker is an ordinary user.

20
00:04:56,357 --> 00:04:58,076
[Hal Turing] What exactly can that user do?

21
00:04:58,076 --> 00:05:15,676
[Dr. Ada Shannon] Submit queries and watch outputs. They can't touch the cache or the victim's text. They must know the target uses PIC with cross-user sharing. White-box means they run an identical copy of the model locally to optimize against. Black-box means they optimize on a surrogate and hope it transfers.

22
00:05:15,676 --> 00:05:22,364
[Hal Turing] Two metrics to pin down before the results. Both are attack success rates, so higher means a stronger attack.

23
00:05:22,364 --> 00:05:39,872
[Dr. Ada Shannon] U-ASR counts runs where the answer merely changes from the benign output. T-ASR is stricter: the output must equal the attacker's chosen wrong answer. T-ASR is the headline. And delta is the share of the victim's context that comes from cache hits.

24
00:05:39,872 --> 00:05:49,299
[Hal Turing] So the question is whether a cleverly prepared chunk can steer later users to a chosen wrong answer, and whether recomputation built for accuracy stops it.

25
00:05:49,299 --> 00:05:56,311
[Dr. Ada Shannon] Yes. We'll set the abstract's strength against what the tables show, and ask how a poisoned entry would actually land in a real shared pool.

26
00:05:56,311 --> 00:06:04,485
[Hal Turing] Let me walk the attack as a causal chain, because the surprising part is how little the attacker touches. Where does the poison actually get made?

27
00:06:04,485 --> 00:06:32,256
[Dr. Ada Shannon] In the attacker's own query. They pick a popular chunk, say a company FAQ entry, and put an optimized prefix in front of it. They send prefix plus chunk to the service. The service computes the chunk's KV conditioned on that prefix, stores it, and throws the prefix away. Later a victim sends a clean prompt containing the same chunk. The token match hits, and the contaminated numbers get spliced in. Nothing in the victim's text is wrong. The numbers carry the poison.

28
00:06:32,256 --> 00:06:39,825
[Hal Turing] Is there evidence that the deviation is big enough to matter? Cached KV from a different context could differ by a rounding error.

29
00:06:39,825 --> 00:06:57,983
[Dr. Ada Shannon] Their Figure 2 addresses that. Under position-independent reuse the keys deviate by roughly fifty percent and the values by roughly twenty-five. Injecting about twenty percent noise into the KV was enough to change the output. So the natural gap is already past the threshold, and the attack only has to aim it.

30
00:06:57,983 --> 00:07:00,166
[Hal Turing] Aim it how? You can't edit the cache directly.

31
00:07:00,166 --> 00:07:35,460
[Dr. Ada Shannon] By search, run locally on a surrogate copy of the model, so the shared cache is never touched. The loss is cross-entropy between the answer the victim would get with the hijacked KV and the attacker's chosen wrong answer. To score one candidate prefix, the code builds its embeddings, runs prefix plus chunk, and keeps only the chunk's slice of the KV. Then it simulates the victim's question over that slice and measures how far the output sits from the target. Gradients flow back through a one-hot relaxation of the prefix tokens. GCG then swaps tokens greedily, keeping the best candidate each round.

32
00:07:35,460 --> 00:07:38,386
[Hal Turing] So one prefix serves one question and one wrong answer.

33
00:07:38,386 --> 00:07:53,665
[Dr. Ada Shannon] Yes, the objective is written for a specific question and target. Hold that thought for later. One more detail: the attacker can run the recomputation step inside the loop, so the prefix is optimized against the repair it will meet.

34
00:07:53,665 --> 00:07:54,779
[Hal Turing] What did they test it on?

35
00:07:54,779 --> 00:08:20,972
[Dr. Ada Shannon] Four question-answering sets: HotpotQA, SQuAD, MedQA and PubMedQA, with 200 questions each. An LLM writes a wrong answer for each question. Models are Llama-3.1-8B, Ministral-8B and Qwen3-8B. Defaults are cache ratio 0.3, recomputation 0.1, chunk size 32. They compare Random, EPIC and CacheBlend recomputation against Vanilla and Full.

36
00:08:20,972 --> 00:08:24,594
[Hal Turing] Start with the headline. The abstract says ninety-four percent.

37
00:08:24,594 --> 00:09:04,486
[Dr. Ada Shannon] The attack works, but not at that number for the systems people would run. On Llama, with no recomputation, the attacker's exact wrong answer lands about ninety-four or ninety-five times in a hundred. With the standard ten percent recomputation, EPIC and CacheBlend let it through about eighty-three and eighty-five times in a hundred. So real recomputation trims roughly ten points and doesn't come close to stopping it. Ninety-four matches the no-recomputation row, and that row is also the badly degraded system. Its clean accuracy on HotpotQA is 0.58 against 0.73 with full recomputation. Table 1 has no confidence intervals.

38
00:09:04,486 --> 00:09:08,387
[Hal Turing] So the near-perfect cells mostly come from the system nobody should deploy.

39
00:09:08,387 --> 00:09:17,907
[Dr. Ada Shannon] Mostly. Qwen and Ministral look similar, and Ministral on MedQA is a hundred percent for every method. But that's a cell, not the pattern.

40
00:09:17,907 --> 00:09:20,925
[Hal Turing] What happens if the operator turns the recomputation dial up?

41
00:09:20,925 --> 00:09:55,709
[Dr. Ada Shannon] Success falls steadily. On HotpotQA with Llama, a typical method goes from about nine in ten at ten percent to about four in ten at fifty percent. EPIC is the most resistant, and even it still lets about half through at fifty. The paper's prose says the attack remains effective at fifty percent recomputation. Thirty-one to forty-nine percent is a real attack, but it's a long way from robust. Accuracy at fifty percent is back to about 0.72, close to the 0.73 you get with no cache. One dataset, one model, and the attacker wasn't re-optimized for that setting in this sweep.

42
00:09:55,709 --> 00:10:03,604
[Hal Turing] Here's what I'd want ruled out. You append a thirty-two-token string to a chunk and outputs change. Maybe any odd text does that.

43
00:10:03,604 --> 00:10:24,780
[Dr. Ada Shannon] They ran that ablation. The optimized thirty-two-token prefix reaches a hundred percent targeted success. A random thirty-two-token prefix gets zero. Plain misinformation text gets about twenty-three and a half percent. So the effect comes from the optimization. The text I have doesn't say which recomputation setting that table used, so I'd treat the hundred percent as the best case.

44
00:10:24,780 --> 00:10:27,288
[Hal Turing] And the robustness checks beyond recomputation?

45
00:10:27,288 --> 00:10:55,431
[Dr. Ada Shannon] Two worth remembering. At a cache ratio of 0.1, meaning the cached chunk is a tenth of the victim's context, EPIC drops to about fifty-three percent while Vanilla stays at eighty-one. The paper credits EPIC recomputing the first tokens of each chunk. Chunk size from thirty-two to five hundred twelve barely matters. And persistence: after about two thousand unrelated filler tokens, the recomputation methods still sit around forty-six to forty-eight percent, down from the mid-eighties.

46
00:10:55,431 --> 00:10:57,428
[Hal Turing] So it decays with distance, but slowly.

47
00:10:57,428 --> 00:11:02,118
[Dr. Ada Shannon] Right. Half of the original strength survives a long detour, in this lab setup.

48
00:11:02,118 --> 00:11:08,759
[Hal Turing] So the question I want settled: does the abstract oversell? It says the attack works even with 50 percent recomputation.

49
00:11:08,759 --> 00:11:30,771
[Dr. Ada Shannon] Partly. The 94 percent headline isn't tied to a specific table in the text we have, and it matches the undefended no-recomputation baseline. The defended systems sit lower, and at 50 percent recomputation success falls to roughly a third to a half. That's not robust, so 'remains effective' stretches the numbers. I'm not accusing anyone of inventing a figure, though.

50
00:11:30,771 --> 00:11:32,676
[Hal Turing] So what does the paper establish?

51
00:11:32,676 --> 00:12:00,400
[Dr. Ada Shannon] Two things. Recomputation tuned for accuracy isn't an integrity defense. And recomputation does work as a defense, at a price. In the cost table, the hybrid policy drops from 73 successful attacks in 100 to about 11 in 100, but it costs roughly three and a half times what the 20 percent setting costs. At that point you're recomputing four of every five reused tokens.

52
00:12:00,400 --> 00:12:01,608
[Hal Turing] Is there a cheaper route?

53
00:12:01,608 --> 00:12:21,345
[Dr. Ada Shannon] A query-aware policy, which recomputes the tokens the user's question attends to, gets to about 50 in 100 for only 1.15 times the cost. Two caveats. Cost is relative to that 20 percent baseline, with no absolute prefill comparison. And nobody ran an adaptive attacker against either policy.

54
00:12:21,345 --> 00:12:27,428
[Hal Turing] Earlier you flagged that the optimisation is per question and per target answer. How much does that limit the threat?

55
00:12:27,428 --> 00:12:46,469
[Dr. Ada Shannon] A lot. The generalisation test is five paraphrases of the same question, which held at 94 percent. That's one question reworded. Unrelated questions over the same chunk weren't tested, and neither was one prefix serving several answers. The clean test is held-out questions on the same chunk with a fixed prefix.

56
00:12:46,469 --> 00:12:47,815
[Hal Turing] And the black-box transfer?

57
00:12:47,815 --> 00:13:05,277
[Dr. Ada Shannon] Prefixes optimised on Llama 8B land the exact wrong answer about 37 times in 100 on the 70B model, against 85 to 89 in the white-box setting. On the smallest models it's under 20, with high disruption. That's mostly garbage output, not a controlled answer.

58
00:13:05,277 --> 00:13:10,710
[Hal Turing] Let me push on practicality. The poison has to enter the shared pool and survive. Who wins that race?

59
00:13:10,710 --> 00:13:30,494
[Dr. Ada Shannon] The paper concedes much of this. Popular chunks, the ones worth attacking, may already be cached. Eviction policy changes the window. The occupancy probe got 81.4 percent on a single query, 95.4 percent within five tries, preliminary, and probing pollutes the cache. There are no trace-driven results.

60
00:13:30,494 --> 00:13:33,466
[Hal Turing] The bigger concession was chunk-then-cache pipelines, right?

61
00:13:33,466 --> 00:13:55,989
[Dr. Ada Shannon] Right. The paper says the attack is less applicable where chunks are encoded independently. In that case the stored KV doesn't depend on an attacker's prefix, so there's nothing to poison. Whether a real deployment ever stores prefix-conditioned chunk KV is the key empirical question. I can't tell from this source what LMCache's CacheBlend path does, so treat that as unverified.

62
00:13:55,989 --> 00:13:59,379
[Hal Turing] Funny that EPIC's LegoLink does well here. It wasn't built for this.

63
00:13:59,379 --> 00:14:14,611
[Dr. Ada Shannon] It recomputes the first tokens of each chunk to neutralise attention sinks. That's the region most shaped by earlier context, so it's also where an attacker's influence concentrates. Accuracy engineering helped security by accident, and it was the best performer at low cache ratios.

64
00:14:14,611 --> 00:14:15,865
[Hal Turing] What should operators do?

65
00:14:15,865 --> 00:14:45,587
[Dr. Ada Shannon] Treat shared KV as untrusted input. Scope position-independent caches by tenant or trust domain. That costs cross-user hit rate, which is the point of the technique, and the paper doesn't measure that trade. Set the recomputation budget as a security parameter. The paper also suggests rejecting reused KV when recomputation shows a large mismatch, and a hybrid that recomputes high-impact states and compresses the rest. Responsible disclosure went to one startup, and no production services were tested.

66
00:14:45,587 --> 00:14:48,559
[Hal Turing] And the evidence limits? Sweeps ran on one dataset.

67
00:14:48,559 --> 00:15:07,971
[Dr. Ada Shannon] HotpotQA, Llama 8B, 200 questions, no confidence intervals. Persistence is only attenuation under filler tokens, not multi-turn cache survival. And the random-noise motivation shows outputs can change, not that attackers can steer them. The optimisation carries that jump.

68
00:15:07,971 --> 00:15:09,318
[Hal Turing] Where does this go next?

69
00:15:09,318 --> 00:15:24,039
[Dr. Ada Shannon] Trace-driven tests with realistic eviction and traffic. Adaptive attacks against the hybrid and query-aware policies. Question-agnostic prefixes. More than one dataset, with confidence intervals. And an analysis of what a cache key should include.

70
00:15:24,039 --> 00:15:48,606
[Hal Turing] So here's where I land. Position-independent reuse loses the guarantee prefix caching gets for free, and in a white-box lab an optimised, per-question poison can steer answers even with standard recomputation. The boundary is a surrogate model, per-question tuning, small single-dataset sweeps, and an unproven path for the poison into real shared caches.

71
00:15:48,606 --> 00:15:54,829
[Dr. Ada Shannon] That's fair. A real integrity gap, shown in simulation. Severity in deployed systems is still open.

72
00:15:54,829 --> 00:15:57,151
[Hal Turing] Thanks for listening, everyone. See you next time.