{"id":"TECH-202610-sglang-prefill-decode-disaggregation-can-leak-other-users-pr","kind":"technique","aliases":[],"related":[],"title":"SGLang prefill/decode disaggregation can leak other users' prompts and outputs when a prompt exceeds the prefill KV cache pool","description":null,"summary":"In SGLang's disaggregated prefill/decode mode, a request too long for the prefill KV cache pool is marked rejected and swapped for a one-token placeholder, but it is not aborted. Prefill still sends KV for the placeholder, and decode treats the transfer as complete. Decode then generates over stale KV pages that still hold earlier requests' data, so the response can contain other users' content. Cloudflare disclosed it with SGLang on 2026-10-26: from 2026-08-27 to 2026-09-09, 155 Workers AI GLM-5.3 responses carried other customers' data. Those deployments served a 1,048,576-token context with a 681,280-token prefill pool. The bug was reported by Gadi Evron (Knostic). Anyone self-hosting multi-tenant SGLang with PD disaggregation is at risk if the advertised context length is larger than the prefill KV pool, or if the scheduler can reject requests some other way. Cloudflare saw no deliberate exploitation. No CVE known yet. Cloudflare has already fixed Workers AI.","severity":"medium","cvss":null,"epss":null,"exploited":false,"exploited_at":null,"products":[{"name":"sglang","fixed":[],"affected":"unknown (builds without PRs #36988/#38935, prefill/decode disaggregation mode)","ecosystem":"pypi"}],"ecosystems":[],"tags":["early-warning","llm-serving","self-hosted","sglang","technique","tenant-isolation"],"not_affected_if":[],"how_to_check":"- Check whether SGLang runs with prefill/decode disaggregation (Mooncake, NIXL or MORI transfer backend)\n- Compare the configured context length with the prefill KV cache pool size in the prefill server startup logs (max total tokens). Exposure exists if the context is larger than the pool\n- Check whether your SGLang build includes PRs #36988 and #38935 (and proposed #38704). Which release first ships them is unknown\n- Look for responses to very long prompts that contain content unrelated to the prompt","fix":"- Size the prefill KV pool to at least the advertised context length, or lower the context length/max input to fit the pool. Cloudflare says this alone stops the bug from triggering\n- Upgrade SGLang to a build that includes #36988 and #38935, which reject over-length requests before any KV transfer (fixed release version unknown)\n- Until then, avoid sharing one PD-disaggregated deployment across tenants with different trust levels","status":"open","published_at":"2026-10-26T00:00:00+00:00","first_seen_at":"2026-10-26T22:47:05+00:00","triaged_at":"2026-10-26T22:47:05+00:00","updated_at":"2026-10-26T22:47:05+00:00","category":"ai-tooling","confidence":null,"triage":null,"analyst_status":"done","analyst_note":null,"search":"'-08':125B '-09':129B,130B '-10':121B '-202610':21A '-26':122B '-27':126B '-5.3':135B '048':146B '1':145B '155':131B '2026':120B,124B,128B '280':153B '576':147B '681':152B 'abort':78B 'advertis':181B 'ai':133B,215B 'alreadi':212B 'anyon':165B 'bug':158B 'cach':18A,61B 'can':27A 'carri':137B 'cloudflar':114B,201B,210B 'complet':92B 'contain':110B 'content':113B 'context':149B,182B 'custom':139B 'cve':207B 'data':105B,140B 'decod':87B,93B 'decode':25A 'deliber':204B 'deploy':142B 'disaggreg':3A,50B,175B 'disaggregation':26A 'disclos':115B 'earli':34B 'earlier':103B 'early-warn':33B 'evron':163B 'exceed':14A 'exploit':205B 'fix':213B 'gadi':162B 'generat':95B 'glm':134B 'hold':102B 'host':41B,168B 'isol':46B 'knostic':164B 'known':208B 'kv':17A,60B,82B,98B,189B 'larger':185B 'leak':5A,28A 'length':183B 'llm':37B 'llm-serv':36B 'long':56B 'mark':64B 'mode':52B 'multi':170B 'multi-ten':169B 'one':71B 'one-token':70B 'other':29A 'output':10A 'page':99B 'pd':174B 'placehold':73B,85B 'pool':19A,62B,156B,190B 'pr':31A 'prefil':16A,59B,79B,155B,188B 'prefill':24A 'prefill/decode':2A,51B 'prompt':8A,13A 'reject':65B,196B 'report':160B 'request':54B,104B,197B 'respons':108B,136B 'risk':178B 'saw':202B 'schedul':194B 'self':40B,167B 'self-host':39B,166B 'send':81B 'serv':38B,143B 'sglang':1A,23A,32A,42B,48B,118B,172B 'sglang-prefill-decode-disaggregation-can-leak-other-users-pr':22A 'stale':97B 'still':80B,101B 'swap':67B 'tech':20A 'techniqu':43B 'tenant':45B,171B 'tenant-isol':44B 'token':72B,148B,154B 'transfer':90B 'treat':88B 'user':7A,112B 'users':30A 'warn':35B 'way':200B 'worker':132B,214B 'yet':209B","nicknames":[],"mitigations":[],"who_affected":null,"tier":"B","verified":{},"analyst_revisit_at":null,"first_public_at":"2026-10-26T00:00:00+00:00","kev":null,"cvss_vector":null,"cvss_by":null,"poc_count":0,"sources":[{"source":"analyst","external_id":"TECH-202610-sglang-prefill-decode-disaggregation-can-leak-other-users-pr","url":null,"severity":null,"published_at":"2026-10-26T00:00:00+00:00","modified_at":null,"first_seen_at":"2026-10-26T22:47:05+00:00","fetched_at":"2026-10-26T22:47:05+00:00","note":null}],"documents":[{"id":4600,"url":"https://x.com/Cloudflare/status/2114848043853873286","title":"@Cloudflare: A configuration mismatch in GLM 5.3 on Workers AI triggered an SGLang bug that e","kind":"post","source":"x","text":"A configuration mismatch in GLM 5.3 on Workers AI triggered an SGLang bug that exposed KV cache pages across requests. Here is how we addressed the issue. https://t.co/CJl1WzznjA\n\nlikes 2 reposts 0","published_at":"2026-10-26T22:34:00+00:00","relation":"describes","confidence":null,"note":null},{"id":4609,"url":"https://blog.cloudflare.com/kv-cache-exposure-remediation/","title":"Cross-request KV cache exposure in the GLM-5.3 Workers AI model, and our remediation with SGLang","kind":"vendor","source":"analyst","text":"Cloudflare coordinated disclosure with SGLang. Reported 2026-09-08 by Gadi Evron (Knostic) via HackerOne. Between 2026-08-27 and 2026-09-09, 155 responses from @cf/zai-org/glm-5.3 on Workers AI contained data from other customers' requests; all affected prompts were longer than 681,280 tokens. Cause: GLM 5.3 was deployed with a 1,048,576-token context while the prefill KV cache pool held only 681,280 tokens, plus an SGLang bug in disaggregated prefill/decode mode: an over-length request was marked rejected and replaced with a one-token placeholder, but prefill still transferred KV for the placeholder, and decode treated the transfer as complete and attended over stale KV pages from earlier requests (pages are not zeroed between requests). Cloudflare found no evidence of deliberate exploitation. Fix: make the prefill KV pool hold the full context length; patched engine retires rejected requests at the disaggregation admission queue before bootstrap/allocation/KV transfer. Upstream: proposed #38704 (also decode verifies transferred token count); related fix already on SGLang main in #36988 (merged 2026-09-06, not yet released at that time); maintainers merged #38935, which rejects these requests earlier. Only GLM-5.3 on Workers AI affected per Cloudflare.","published_at":"2026-10-26T00:00:00+00:00","relation":"describes","confidence":null,"note":null}],"notice":"Free to read at a human pace. An API with keys and an MCP server are in preparation; early access: hello@infrastack.ai"}