🔍 Read the full analysis: How Cloudflare’s KV Cache Leak Put Session Boundaries In Focus on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Cloudflare reported that its hosted GLM-5.3 service returned text from other customers’ requests in 155 responses between Aug. 27 and Sept. 9, 2026. The company attributed the leak to a mismatch in its KV-cache configuration combined with a bug in the SGLang inference engine; it says it found no evidence of deliberate exploitation.
Cloudflare says its hosted GLM-5.3 model returned text from other customers’ requests in 155 responses between Aug. 27 and Sept. 9, 2026, after a cache-capacity mismatch interacted with a bug in the SGLang inference engine. The disclosure matters because it shows how a failure in request handling can expose data across sessions even when the model itself is not responsible for generating the leaked text.
Cloudflare’s incident report, published Oct. 26, says the affected responses followed prompts longer than 681,280 tokens. The company traced the incident to two conditions: its prefill GPUs had a KV-cache pool sized for 681,280 tokens, while the model endpoint advertised a context window of 1,048,576 tokens; and SGLang did not properly stop rejected requests in Cloudflare’s split prefill-and-decode configuration.
In that setup, one GPU pool reads a prompt and builds its KV cache, while another uses the cache to generate a response. Cloudflare says an oversized request was replaced with a one-token placeholder during prefill, but the decode side could still read the pages reserved for the full request. Since cache pages were reused without being cleared between requests, some pages still held other users’ prompts and partial generations. The resulting response could return HTTP 200 rather than an error.
Cloudflare says a researcher reported the issue through its HackerOne program on Sept. 8. After declaring an incident on Sept. 9, the company took the model offline within 38 minutes and restored a fixed version that evening. It said its review found 155 affected responses over 14 days, excluding its own tests and the researcher’s requests from the count.
Infographic 01 · Cross-request KV cache exposure
How a long prompt read someone else's data.
A memory pool smaller than the advertised context window, plus an engine bug that let rejected requests through, made one hosted model attend over other customers' leftover pages.
The mismatch
tokens; Cloudflare knew before launch and expected an error
Only prompts longer than 681,280 tokens could reach the faulty path.
The eight pages
Cloudflare’s own example of one over-long request
written by this requeststill holding earlier users’ prompts and replies
- Decode reserves 8 pages for the long prompt.
- Prefill rejects it and swaps in a one-token placeholder…
- …but the rejected request still passes the handoff checks.
- Prefill writes one page; decode reads all eight.
- The client gets HTTP 200 and a coherent reply, partly built on someone else’s data.
Why pages hold old data
speed trade-off
- GPU memory is a shared pool of fixed-size pages
- Pages are not zeroed between requests; the next one overwrites
- Bookkeeping is the only wall between customers
The fix
9 Sep, 19:40 UTC
- Pool resized so every valid prompt fits
- Engine retires rejected requests before any memory transfer
- Upstream: SGLang v0.5.20 to v0.5.22; operators should run v0.5.22+
Still open
not in the report
- Whether and how affected customers were notified
- How many customers’ requests contributed the leaked data
- What kind of data it was
Sources: Cloudflare incident report (26 Oct 2026); SGLang/LMSYS (26 Oct 2026). Figures and the no-exploitation finding are Cloudflare’s own.
Powered by Thorsten Meyer AI
How Cache Reuse Crossed Session Boundaries
The incident highlights a security boundary that hosted AI services must maintain: memory used for one request must not influence another customer’s response. KV caches hold intermediate representations of prompt tokens in GPU memory. Reusing that memory can improve performance, but safe reuse depends on correct allocation and request-state handling. Here, according to Cloudflare, the next request was expected to overwrite pages, while the faulty path allowed decode to read pages that prefill had not rewritten.
For customers, the reported exposure involved response text containing material from other requests, not a claim that every prompt or account was affected. The number is also specific: 155 responses, not necessarily 155 distinct customers or 155 complete prompts. Cloudflare’s report says the requests appeared consistent in timing and volume with ordinary long-context use rather than repeated exploitation. That assessment reduces evidence for an intentional attack, but it does not change the privacy implications of cross-session data appearing in a response.
The event also illustrates why testing only service availability is not enough for long-context systems. Cloudflare said it had tested HTTP response codes and engine performance at long context lengths but had not validated output content and coherence. A successful status code can therefore conceal a serious failure in the data returned to a user.
Products Worth Considering
Apple iPad Air (4th Generation, 10.9-inch, Wi-Fi, 256GB) – Space Gray
Logitech MX Master 4 for Mac, Ergonomic Bluetooth Mouse with Haptic Feedback, Ultra-Fast Scrolling, USB-C Charging, for MacOS, iPadOS, Windo
Amazon Kindle Scribe Colorsoft 64GB (1st Generation)
As an affiliate, we earn on qualifying purchases.
The Configuration and SGLang Failure
Cloudflare says it knew before launch that the advertised context limit exceeded the capacity of the prefill KV pool. The company expected SGLang to return an error for requests that did not fit, and its long-context tests produced successful responses. In the incident report, Cloudflare acknowledged that those tests checked HTTP codes and performance, not whether the generated output was correct and coherent.
The second condition was in SGLang’s handling of rejected requests across separate prefill and decode queues. Cloudflare’s report says both queues checked whether a request had finished but not whether it had been rejected. As a result, decode could proceed using pages reserved for the full prompt even though prefill had written only a placeholder. Cloudflare says it found that the underlying SGLang issue had been reported publicly as issue #30233 on July 6, 2026, but its team noticed it only during the investigation.
Cloudflare identifies the affected endpoint as @cf/zai-org/glm-5.3. Its report says the model itself was not at fault: the exposure arose from service configuration and inference-engine behavior. The account of the incident comes from Cloudflare’s own report, alongside the researcher’s report through HackerOne.
“Pages are not currently zeroed between requests; the next request that gets a page simply overwrites it.”
— Cloudflare incident report
What the Incident Report Leaves Open
Cloudflare says it found no evidence of deliberate exploitation, but its report does not establish who authored every affected request or identify the customers whose text appeared in responses. It also does not specify how many distinct customers were affected, how much text from any one request appeared, or whether all exposed material was seen by another customer.
The supplied report does not say how affected customers were notified. It also does not provide a full account of what information was present in the exposed text or describe any customer-specific impact. Those details remain important for judging the scope of the privacy incident and what steps affected users may need to take.
The report attributes the failure to both Cloudflare’s configuration and an SGLang bug. The available details do not establish whether other services using SGLang encountered the same issue, or whether any other endpoints or model configurations were affected.
Customer Notices and Safeguard Changes
Cloudflare says it took the affected model offline after reproducing the leak and restored a fixed version later that day. The incident report describes that response, but the material available here does not give a schedule for additional public updates or say whether Cloudflare will publish a customer-by-customer impact assessment.
The next points to watch are whether Cloudflare clarifies customer notification and exposure scope, and whether it reports changes to long-context testing and request validation. The incident also leaves open what safeguards SGLang and other inference platforms will apply to ensure rejected requests cannot proceed through separate queues. Until those details are available, the disclosed count and technical explanation remain Cloudflare’s account of the incident.
Key Questions
What happened in the Cloudflare KV-cache incident?
Cloudflare says its hosted GLM-5.3 endpoint returned text from other customers’ requests in 155 responses between Aug. 27 and Sept. 9, 2026. The company attributed the issue to a KV-cache capacity mismatch and a bug in SGLang’s handling of rejected requests.
Which prompts could trigger the issue?
According to Cloudflare’s report, affected requests had prompts longer than 681,280 tokens. The model endpoint advertised a larger context window of 1,048,576 tokens, while the relevant prefill cache pool had capacity for fewer tokens.
Did Cloudflare report evidence of an attack?
No. Cloudflare said the timing and volume of the affected requests were consistent with ordinary long-context use rather than repeated exploitation, and that it found no evidence of deliberate exploitation. That is the company’s assessment of the traffic.
How did Cloudflare respond?
Cloudflare says it took the model offline 38 minutes after declaring an incident and restored a fixed version that evening. Its report says the issue was reported through HackerOne on Sept. 8.
Were affected customers notified?
The incident report described in the source material does not say how affected customers were told. It also does not state how many distinct customers were affected or give a detailed account of the exposed text.
Source: ThorstenMeyerAI.com
