Rust HTTP/gRPC stop_token_ids bypass Python vocab-bound fix and can terminate EngineCore

This is a sibling-callsite/incomplete-fix variant: after the Python-side fix, Rust OpenAI-compatible HTTP and Rust gRPC callers can still submit an out-of-vocabulary stop token with min_tokens > 0 and reach the fatal CUDA indexing sink.

Technical Details

The Python request path currently only checks that stop_token_ids are integers:

assert isinstance(self.stop_token_ids, list)
if not all(isinstance(st_id, int) for st_id in self.stop_token_ids):
    raise ValueError(...)

PR #44968 adds the missing Python vocab-bound validator. The same PR explains the fatal sink:

stop_token_ids with out-of-range values crash EngineCore via a CUDA
device-side assertion.

The shared runtime path is in MinTokensLogitsProcessor. When min_tokens is active, it takes params.all_stop_token_ids:

return min_tokens, output_tok_ids, params.all_stop_token_ids

and later indexes logits with those token IDs:

logits.index_put_(self.logits_slice, self.neg_inf_tensor)

and on the spec-decode path:

logits.index_put_(logits_slice, self.neg_inf_tensor)

The Rust HTTP OpenAI routes expose and forward the same field:

pub stop_token_ids: Option<Vec<u32>>,
...
stop_token_ids: request.stop_token_ids,

The Rust validators already check neighboring token-id fields, but not stop_token_ids:

validate_prompt_token_ids(&request.prompt, prompt_bound)?;
validate_allowed_token_ids(request.allowed_token_ids.as_deref(), tokenizer_vocab_size)?;
validate_logit_bias(request.logit_bias.as_ref(), model_vocab_size.unwrap_or(usize::MAX))

Chat completions has the same gap:

validate_allowed_token_ids(request.allowed_token_ids.as_deref(), tokenizer_vocab_size)?;
validate_logit_bias(request.logit_bias.as_ref(), model_vocab_size.unwrap_or(usize::MAX))

Rust gRPC also forwards caller-supplied stopping criteria without a range check:

if !s.stop_token_ids.is_empty() {
    params.stop_token_ids = Some(s.stop_token_ids.clone());
}

Finally, Rust text lowering copies those IDs into the engine-facing stop set:

let mut all_stop_token_ids = BTreeSet::from_iter(stop_token_ids.iter().copied());
...
EngineCoreSamplingParams {
    stop_token_ids,
    all_stop_token_ids,
    ...
}

The issue is therefore not that stop_token_ids exist. The issue is that Rust frontend paths bypass the same vocab-bound validation that Python is adding before the exact fatal sink documented in PR #44968.

vLLM intentionally supports stop token IDs, and min_tokens intentionally masks stop tokens until the minimum length is reached. It does not intend arbitrary client token IDs outside the served vocabulary to be used as GPU tensor indices.

The intended contract is visible in three places:

• PR #44968 adds _validate_stop_token_ids() to reject IDs outside [0, vocab_size) before the request reaches GPU logits processing.
• Rust HTTP validation already rejects out-of-vocab prompt IDs, allowed_token_ids, and logit_bias keys.
• The public PR's test plan includes an E2E test for stop_token_ids_out_of_vocab, confirming out-of-vocab stop IDs should be a rejected request, not an EngineCore crash.

The Rust HTTP and gRPC frontends construct engine-facing sampling state outside the Python protocol model path, so they must enforce the same safety contract before submitting a request.

Impact

An attacker who can reach an affected Rust HTTP or Rust gRPC vLLM endpoint can send a small generation request with min_tokens = 1 and a stop_token_ids entry greater than or equal to the served model vocabulary size.

Expected impact is single-request denial of service for affected deployments: the Rust frontend accepts and lowers the request, and the shared Python EngineCore min-tokens processor can use the out-of-range token ID as a GPU logits tensor index. Public PR #44968 states that this condition puts EngineCore into a fatal state and requires full service restart.

This report does not claim code execution, data access, model compromise, or persistence. It is an availability issue caused by missing Rust-side input validation for a field already identified as security-sensitive.

Suggested severity: High if the Rust HTTP or Rust gRPC endpoint is network-reachable without an additional deployment-layer credential requirement.

Suggested CVSS v3.1:

CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H

If maintainers model the affected endpoint as reachable only to already-authorized clients, the conservative score is:

CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:N/I:N/A:H

The vLLM security policy states that DoS or partial disruption without code execution or data breach is generally Moderate, while High is appropriate for higher-impact issues or privileged network surfaces. Because the public root cause describes fatal EngineCore restart from a single malformed request, the technical impact is high availability; maintainers can choose PR:N or PR:L based on their expected Rust deployment exposure.

Suggested CWEs:

• CWE-20: Improper Input Validation
• CWE-129: Improper Validation of Array Index
• CWE-400: Uncontrolled Resource Consumption

Suggested Fix

Apply the same vocab-bound validation model to Rust HTTP and Rust gRPC before request lowering:

• add a Rust helper equivalent to Python _validate_stop_token_ids();
• validate every stop_token_ids entry against the served vocabulary boundary when stop_token_ids is present;
• apply it in both Rust OpenAI completions and chat completions validators;
• apply equivalent validation for gRPC StoppingCriteria.stop_token_ids before building SamplingParams;
• reject with ApiError or Status::invalid_argument(...) before submitting an engine request;
• add regression tests for valid in-vocab IDs and invalid out-of-vocab IDs on:
• Rust HTTP /v1/completions;
• Rust HTTP /v1/chat/completions;
• Rust gRPC Generate;
• Rust gRPC GenerateStream; and
• min_tokens > 0, which is the documented fatal sink precondition.

Keeping the only fix in Python SamplingParams.verify() is insufficient for Rust frontend deployments because Rust constructs engine-facing sampling state directly.

References

• vLLM security policy: https://github.com/vllm-project/vllm/security/policy
• vLLM vulnerability management: https://docs.vllm.ai/en/latest/contributing/vulnerability_management.html
• vLLM security guide: https://docs.vllm.ai/en/latest/usage/security.html
• vLLM PR #44968: https://github.com/vllm-project/vllm/pull/44968

Appendix: Duplicate Review

This is distinct from prior local/private submissions:

• this report / GHSA-7rx8-56c4-pfh6: Python OpenAI request parameters can exceed fixed sampler array budgets; it covers list length, including more than 128 stop token IDs. It does not cover a single out-of-vocabulary stop token reaching the CUDA index sink through Rust frontends.
• this report / GHSA-w2v2-5662-8v5q: Rust gRPC accepts out-of-vocabulary prompt token IDs. That is a prompt-token embedding path, not stop_token_ids plus MinTokensLogitsProcessor.
• this report / GHSA-mmrv-74wf-6pxm: Rust frontend remains missing sampler-con