Update dependency vllm to v0.7.2 [SECURITY]

[renovate]

This PR contains the following updates:

Package	Change	Age	Adoption	Passing	Confidence
vllm	==v0.6.6 -> ==0.7.2
vllm	==v0.6.4 -> ==0.7.2
vllm	==0.6.6 -> ==0.7.2
vllm	==0.6.4 -> ==0.7.2

GitHub Vulnerability Alerts

CVE-2024-8768

A flaw was found in the vLLM library. A completions API request with an empty prompt will crash the vLLM API server, resulting in a denial of service.

CVE-2025-24357

Description

The vllm/model_executor/weight_utils.py implements hf_model_weights_iterator to load the model checkpoint, which is downloaded from huggingface. It use torch.load function and weights_only parameter is default value False. There is a security warning on https://pytorch.org/docs/stable/generated/torch.load.html, when torch.load load a malicious pickle data it will execute arbitrary code during unpickling.

Impact

This vulnerability can be exploited to execute arbitrary codes and OS commands in the victim machine who fetch the pretrained repo remotely.

Note that most models now use the safetensors format, which is not vulnerable to this issue.

References

• https://pytorch.org/docs/stable/generated/torch.load.html
• Fix: https://github.com/vllm-project/vllm/pull/12366

CVE-2025-25183

Summary

Maliciously constructed prompts can lead to hash collisions, resulting in prefix cache reuse, which can interfere with subsequent responses and cause unintended behavior.

Details

vLLM's prefix caching makes use of Python's built-in hash() function. As of Python 3.12, the behavior of hash(None) has changed to be a predictable constant value. This makes it more feasible that someone could try exploit hash collisions.

Impact

The impact of a collision would be using cache that was generated using different content. Given knowledge of prompts in use and predictable hashing behavior, someone could intentionally populate the cache using a prompt known to collide with another prompt in use.

Solution

We address this problem by initializing hashes in vllm with a value that is no longer constant and predictable. It will be different each time vllm runs. This restores behavior we got in Python versions prior to 3.12.

Using a hashing algorithm that is less prone to collision (like sha256, for example) would be the best way to avoid the possibility of a collision. However, it would have an impact to both performance and memory footprint. Hash collisions may still occur, though they are no longer straight forward to predict.

To give an idea of the likelihood of a collision, for randomly generated hash values (assuming the hash generation built into Python is uniformly distributed), with a cache capacity of 50,000 messages and an average prompt length of 300, a collision will occur on average once every 1 trillion requests.

References

• https://github.com/vllm-project/vllm/pull/12621
• python/cpython@432117c
• https://github.com/python/cpython/pull/99541

---

vllm: Malicious model to RCE by torch.load in hf_model_weights_iterator

CVE-2025-24357 / GHSA-rh4j-5rhw-hr54

More information

Details

Description

The vllm/model_executor/weight_utils.py implements hf_model_weights_iterator to load the model checkpoint, which is downloaded from huggingface. It use torch.load function and weights_only parameter is default value False. There is a security warning on https://pytorch.org/docs/stable/generated/torch.load.html, when torch.load load a malicious pickle data it will execute arbitrary code during unpickling.

Impact

This vulnerability can be exploited to execute arbitrary codes and OS commands in the victim machine who fetch the pretrained repo remotely.

Note that most models now use the safetensors format, which is not vulnerable to this issue.

References

• https://pytorch.org/docs/stable/generated/torch.load.html
• Fix: https://github.com/vllm-project/vllm/pull/12366

Severity

• CVSS Score: 7.5 / 10 (High)
• Vector String: CVSS:3.1/AV:N/AC:H/PR:N/UI:R/S:U/C:H/I:H/A:H

References

• https://github.com/vllm-project/vllm/security/advisories/GHSA-rh4j-5rhw-hr54
• https://nvd.nist.gov/vuln/detail/CVE-2025-24357
• https://github.com/vllm-project/vllm/pull/12366
• https://github.com/vllm-project/vllm/commit/d3d6bb13fb62da3234addf6574922a4ec0513d04
• https://github.com/vllm-project/vllm
• https://github.com/vllm-project/vllm/releases/tag/v0.7.0
• https://pytorch.org/docs/stable/generated/torch.load.html

This data is provided by OSV and the GitHub Advisory Database (CC-BY 4.0).

---

vLLM uses Python 3.12 built-in hash() which leads to predictable hash collisions in prefix cache

CVE-2025-25183 / GHSA-rm76-4mrf-v9r8

More information

Details

Summary

Maliciously constructed prompts can lead to hash collisions, resulting in prefix cache reuse, which can interfere with subsequent responses and cause unintended behavior.

Details

vLLM's prefix caching makes use of Python's built-in hash() function. As of Python 3.12, the behavior of hash(None) has changed to be a predictable constant value. This makes it more feasible that someone could try exploit hash collisions.

Impact

The impact of a collision would be using cache that was generated using different content. Given knowledge of prompts in use and predictable hashing behavior, someone could intentionally populate the cache using a prompt known to collide with another prompt in use.

Solution

We address this problem by initializing hashes in vllm with a value that is no longer constant and predictable. It will be different each time vllm runs. This restores behavior we got in Python versions prior to 3.12.

Using a hashing algorithm that is less prone to collision (like sha256, for example) would be the best way to avoid the possibility of a collision. However, it would have an impact to both performance and memory footprint. Hash collisions may still occur, though they are no longer straight forward to predict.

To give an idea of the likelihood of a collision, for randomly generated hash values (assuming the hash generation built into Python is uniformly distributed), with a cache capacity of 50,000 messages and an average prompt length of 300, a collision will occur on average once every 1 trillion requests.

References

• https://github.com/vllm-project/vllm/pull/12621
• python/cpython@432117c
• https://github.com/python/cpython/pull/99541

Severity

• CVSS Score: 2.6 / 10 (Low)
• Vector String: CVSS:3.1/AV:N/AC:H/PR:L/UI:R/S:U/C:N/I:L/A:N

References

• https://github.com/vllm-project/vllm/security/advisories/GHSA-rm76-4mrf-v9r8
• https://nvd.nist.gov/vuln/detail/CVE-2025-25183
• https://github.com/python/cpython/pull/99541
• https://github.com/vllm-project/vllm/pull/12621
• https://github.com/python/cpython/commit/432117cd1f59c76d97da2eaff55a7d758301dbc7
• https://github.com/vllm-project/vllm/commit/73b35cca7f3745d07d439c197768b25d88b6ab7f
• https://github.com/vllm-project/vllm

This data is provided by OSV and the GitHub Advisory Database (CC-BY 4.0).

---

Release Notes

vllm-project/vllm (vllm)

v0.7.2

Compare Source

Highlights

• Qwen2.5-VL is now supported in vLLM. Please note that it requires a source installation from Hugging Face transformers library at the moment (#​12604)
• Add transformers backend support via --model-impl=transformers. This allows vLLM to be ran with arbitrary Hugging Face text models (#​11330, #​12785, #​12727).
• Performance enhancement to DeepSeek models.
  • Align KV caches entries to start 256 byte boundaries, yielding 43% throughput enhancement (#​12676)
  • Apply torch.compile to fused_moe/grouped_topk, yielding 5% throughput enhancement (#​12637)
  • Enable MLA for DeepSeek VL2 (#​12729)
  • Enable DeepSeek model on ROCm (#​12662)

Core Engine

• Use VLLM_LOGITS_PROCESSOR_THREADS to speed up structured decoding in high batch size scenarios (#​12368)

Security Update

• Improve hash collision avoidance in prefix caching (#​12621)
• Add SPDX-License-Identifier headers to python source files (#​12628)

Other

• Enable FusedSDPA support for Intel Gaudi (HPU) (#​12359)

What's Changed

• Apply torch.compile to fused_moe/grouped_topk by @​mgoin in https://github.com/vllm-project/vllm/pull/12637
• doc: fixing minor typo in readme.md by @​vicenteherrera in https://github.com/vllm-project/vllm/pull/12643
• [Bugfix] fix moe_wna16 get_quant_method by @​jinzhen-lin in https://github.com/vllm-project/vllm/pull/12648
• [Core] Silence unnecessary deprecation warnings by @​russellb in https://github.com/vllm-project/vllm/pull/12620
• [V1][Minor] Avoid frequently creating ConstantList by @​WoosukKwon in https://github.com/vllm-project/vllm/pull/12653
• [Core][v1] Unify allocating slots in prefill and decode in KV cache manager by @​ShawnD200 in https://github.com/vllm-project/vllm/pull/12608
• [Hardware][Intel GPU] add XPU bf16 support by @​jikunshang in https://github.com/vllm-project/vllm/pull/12392
• [Misc] Add SPDX-License-Identifier headers to python source files by @​russellb in https://github.com/vllm-project/vllm/pull/12628
• [doc][misc] clarify VLLM_HOST_IP for multi-node inference by @​youkaichao in https://github.com/vllm-project/vllm/pull/12667
• [Doc] Deprecate Discord by @​zhuohan123 in https://github.com/vllm-project/vllm/pull/12668
• [Kernel] port sgl moe_align_block_size kernels by @​chenyang78 in https://github.com/vllm-project/vllm/pull/12574
• make sure mistral_common not imported for non-mistral models by @​youkaichao in https://github.com/vllm-project/vllm/pull/12669
• Properly check if all fused layers are in the list of targets by @​eldarkurtic in https://github.com/vllm-project/vllm/pull/12666
• Fix for attention layers to remain unquantized during moe_wn16 quant by @​srikanthsrnvs in https://github.com/vllm-project/vllm/pull/12570
• [cuda] manually import the correct pynvml module by @​youkaichao in https://github.com/vllm-project/vllm/pull/12679
• [ci/build] fix gh200 test by @​youkaichao in https://github.com/vllm-project/vllm/pull/12681
• [Model]: Add transformers backend support by @​ArthurZucker in https://github.com/vllm-project/vllm/pull/11330
• [Misc] Fix improper placement of SPDX header in scripts by @​russellb in https://github.com/vllm-project/vllm/pull/12694
• [Bugfix][Kernel] Fix per-token/per-channel quantization for Hopper scaled mm by @​tlrmchlsmth in https://github.com/vllm-project/vllm/pull/12696
• Squelch MLA warning for Compressed-Tensors Models by @​kylesayrs in https://github.com/vllm-project/vllm/pull/12704
• [Model] Add Deepseek V3 fp8_w8a8 configs for B200 by @​kushanam in https://github.com/vllm-project/vllm/pull/12707
• [MISC] Remove model input dumping when exception by @​comaniac in https://github.com/vllm-project/vllm/pull/12582
• [V1] Revert uncache_blocks and support recaching full blocks by @​comaniac in https://github.com/vllm-project/vllm/pull/12415
• [Core] Improve hash collision avoidance in prefix caching by @​russellb in https://github.com/vllm-project/vllm/pull/12621
• Support Pixtral-Large HF by using llava multimodal_projector_bias config by @​mgoin in https://github.com/vllm-project/vllm/pull/12710
• [Doc] Replace ibm-fms with ibm-ai-platform by @​tdoublep in https://github.com/vllm-project/vllm/pull/12709
• [Quant] Fix use_mla TypeError and support loading pure-sparsity Compressed Tensors configs by @​kylesayrs in https://github.com/vllm-project/vllm/pull/12711
• [AMD][ROCm] Enable DeepSeek model on ROCm by @​hongxiayang in https://github.com/vllm-project/vllm/pull/12662
• [Misc] Add BNB quantization for Whisper by @​jeejeelee in https://github.com/vllm-project/vllm/pull/12381
• [VLM] Merged multi-modal processor for InternVL-based models by @​DarkLight1337 in https://github.com/vllm-project/vllm/pull/12553
• [V1] Remove constraints on partial requests by @​WoosukKwon in https://github.com/vllm-project/vllm/pull/12674
• [VLM] Implement merged multimodal processor and V1 support for idefics3 by @​Isotr0py in https://github.com/vllm-project/vllm/pull/12660
• [Model] [Bugfix] Fix loading of fine-tuned models based on Phi-3-Small by @​mgtk77 in https://github.com/vllm-project/vllm/pull/12689
• Avoid unnecessary multi-modal input data copy when len(batch) == 1 by @​imkero in https://github.com/vllm-project/vllm/pull/12722
• [Build] update requirements of no-device for plugin usage by @​sducouedic in https://github.com/vllm-project/vllm/pull/12630
• [Bugfix] Fix CI failures for InternVL and Mantis models by @​DarkLight1337 in https://github.com/vllm-project/vllm/pull/12728
• [V1][Metrics] Add request_success_total counter, labelled with finish reason by @​markmc in https://github.com/vllm-project/vllm/pull/12579
• [Perf] Mem align KV caches for CUDA devices (MLA perf improvement) by @​LucasWilkinson in https://github.com/vllm-project/vllm/pull/12676
• [Core] add and implement VLLM_LOGITS_PROCESSOR_THREADS by @​akeshet in https://github.com/vllm-project/vllm/pull/12368
• [ROCM][AMD][TRITON] Halving warps number for fw_prefill to reduce spilling by @​maleksan85 in https://github.com/vllm-project/vllm/pull/12713
• Refactor Linear handling in TransformersModel by @​hmellor in https://github.com/vllm-project/vllm/pull/12727
• [VLM] Add MLA with pure RoPE support for deepseek-vl2 models by @​Isotr0py in https://github.com/vllm-project/vllm/pull/12729
• [Misc] Bump the compressed-tensors version by @​dsikka in https://github.com/vllm-project/vllm/pull/12736
• [Model][Quant] Fix GLM, Fix fused module mappings for quantization by @​kylesayrs in https://github.com/vllm-project/vllm/pull/12634
• [Doc] Update PR Reminder with link to Developer Slack by @​mgoin in https://github.com/vllm-project/vllm/pull/12748
• [Bugfix] Fix OpenVINO model runner by @​hmellor in https://github.com/vllm-project/vllm/pull/12750
• [V1][Misc] Shorten FinishReason enum and use constant strings by @​njhill in https://github.com/vllm-project/vllm/pull/12760
• [Doc] Remove performance warning for auto_awq.md by @​mgoin in https://github.com/vllm-project/vllm/pull/12743
• [Bugfix] Fix 'ModuleNotFoundError: No module named 'intel_extension_for_pytorch'' for --tensor-parallel-size more than 1 by @​Akashcodes732 in https://github.com/vllm-project/vllm/pull/12546
• [core][distributed] exact ray placement control by @​youkaichao in https://github.com/vllm-project/vllm/pull/12732
• [Kernel] Use self.kv_cache and forward_context.attn_metadata in Attention.forward by @​heheda12345 in https://github.com/vllm-project/vllm/pull/12536
• [Hardware][Intel-Gaudi] Enable FusedSDPA support for Intel Gaudi (HPU) by @​SanjuCSudhakaran in https://github.com/vllm-project/vllm/pull/12359
• Add: Support for Sparse24Bitmask Compressed Models by @​rahul-tuli in https://github.com/vllm-project/vllm/pull/12097
• [VLM] Use shared field to pass token ids to model by @​DarkLight1337 in https://github.com/vllm-project/vllm/pull/12767
• [Docs] Drop duplicate [source] links by @​russellb in https://github.com/vllm-project/vllm/pull/12780
• [VLM] Qwen2.5-VL by @​ywang96 in https://github.com/vllm-project/vllm/pull/12604
• [VLM] Update compatibility with transformers 4.49 by @​DarkLight1337 in https://github.com/vllm-project/vllm/pull/12781
• Quantization and MoE configs for GH200 machines by @​arvindsun in https://github.com/vllm-project/vllm/pull/12717
• [ROCm][Kernel] Using the correct warp_size value by @​gshtras in https://github.com/vllm-project/vllm/pull/12789
• [Bugfix] Better FP8 supported defaults by @​LucasWilkinson in https://github.com/vllm-project/vllm/pull/12796
• [Misc][Easy] Remove the space from the file name by @​houseroad in https://github.com/vllm-project/vllm/pull/12799
• [Model] LoRA Support for Ultravox model by @​thedebugger in https://github.com/vllm-project/vllm/pull/11253
• [Bugfix] Fix the test_ultravox.py's license by @​houseroad in https://github.com/vllm-project/vllm/pull/12806
• Improve TransformersModel UX by @​hmellor in https://github.com/vllm-project/vllm/pull/12785
• [Misc] Remove duplicated DeepSeek V2/V3 model definition by @​mgoin in https://github.com/vllm-project/vllm/pull/12793
• [Misc] Improve error message for incorrect pynvml by @​youkaichao in https://github.com/vllm-project/vllm/pull/12809

New Contributors

• @​vicenteherrera made their first contribution in https://github.com/vllm-project/vllm/pull/12643
• @​chenyang78 made their first contribution in https://github.com/vllm-project/vllm/pull/12574
• @​srikanthsrnvs made their first contribution in https://github.com/vllm-project/vllm/pull/12570
• @​ArthurZucker made their first contribution in https://github.com/vllm-project/vllm/pull/11330
• @​mgtk77 made their first contribution in https://github.com/vllm-project/vllm/pull/12689
• @​sducouedic made their first contribution in https://github.com/vllm-project/vllm/pull/12630
• @​akeshet made their first contribution in https://github.com/vllm-project/vllm/pull/12368
• @​arvindsun made their first contribution in https://github.com/vllm-project/vllm/pull/12717
• @​thedebugger made their first contribution in https://github.com/vllm-project/vllm/pull/11253

Full Changelog: vllm-project/vllm@v0.7.1...v0.7.2

v0.7.1

Compare Source

Highlights

This release features MLA optimization for Deepseek family of models. Compared to v0.7.0 released this Monday, we offer ~3x the generation throughput, ~10x the memory capacity for tokens, and horizontal context scalability with pipeline parallelism

• MLA Kernel (#​12601, #​12642,#​12528).
• FP8 Kernels (#​11589, #​11868, #​12587)

V1

For the V1 architecture, we

• Added a new design document for zero overhead prefix caching here (#​12598)
• Add metrics and enhance logging for V1 engine (#​12569, #​12561, #​12416, #​12516, #​12530, #​12478)

Models

• New Model: MiniCPM-o (text outputs only) (#​12069)

Hardwares

• Neuron: NKI-based flash-attention kernel with paged KV cache (#​11277)
• AMD: llama 3.2 support upstreaming (#​12421)

Others

• Support override generation config in engine arguments (#​12409)
• Support reasoning content in API for deepseek R1 (#​12473)

What's Changed

• [Bugfix] Fix missing seq_start_loc in xformers prefill metadata by @​Isotr0py in https://github.com/vllm-project/vllm/pull/12464
• [V1][Minor] Minor optimizations for update_from_output by @​WoosukKwon in https://github.com/vllm-project/vllm/pull/12454
• [Bugfix] Fix gpt2 GGUF inference by @​Isotr0py in https://github.com/vllm-project/vllm/pull/12467
• [Build] Only build 9.0a for scaled_mm and sparse kernels by @​LucasWilkinson in https://github.com/vllm-project/vllm/pull/12339
• [V1][Metrics] Add initial Prometheus logger by @​markmc in https://github.com/vllm-project/vllm/pull/12416
• [V1][CI/Test] Do basic test for top-p & top-k sampling by @​WoosukKwon in https://github.com/vllm-project/vllm/pull/12469
• [FlashInfer] Upgrade to 0.2.0 by @​abmfy in https://github.com/vllm-project/vllm/pull/11194
• [Feature] [Spec decode]: Enable MLPSpeculator/Medusa and prompt_logprobs with ChunkedPrefill by @​NickLucche in https://github.com/vllm-project/vllm/pull/10132
• Update pre-commit hooks by @​hmellor in https://github.com/vllm-project/vllm/pull/12475
• [Neuron][Kernel] NKI-based flash-attention kernel with paged KV cache by @​liangfu in https://github.com/vllm-project/vllm/pull/11277
• Fix bad path in prometheus example by @​mgoin in https://github.com/vllm-project/vllm/pull/12481
• [CI/Build] Fixed the xla nightly issue report in #​12451 by @​hosseinsarshar in https://github.com/vllm-project/vllm/pull/12453
• [FEATURE] Enables offline /score for embedding models by @​gmarinho2 in https://github.com/vllm-project/vllm/pull/12021
• [CI] fix pre-commit error by @​MengqingCao in https://github.com/vllm-project/vllm/pull/12494
• Update README.md with V1 alpha release by @​ywang96 in https://github.com/vllm-project/vllm/pull/12495
• [V1] Include Engine Version in Logs by @​robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/12496
• [Core] Make raw_request optional in ServingCompletion by @​schoennenbeck in https://github.com/vllm-project/vllm/pull/12503
• [VLM] Merged multi-modal processor and V1 support for Qwen-VL by @​DarkLight1337 in https://github.com/vllm-project/vllm/pull/12504
• [Doc] Fix typo for x86 CPU installation by @​waltforme in https://github.com/vllm-project/vllm/pull/12514
• [V1][Metrics] Hook up IterationStats for Prometheus metrics by @​markmc in https://github.com/vllm-project/vllm/pull/12478
• Replace missed warning_once for rerank API by @​mgoin in https://github.com/vllm-project/vllm/pull/12472
• Do not run suggestion pre-commit hook multiple times by @​hmellor in https://github.com/vllm-project/vllm/pull/12521
• [V1][Metrics] Add per-request prompt/generation_tokens histograms by @​markmc in https://github.com/vllm-project/vllm/pull/12516
• [Kernel] Pipe attn_logits_soft_cap through paged attention TPU kernels by @​fenghuizhang in https://github.com/vllm-project/vllm/pull/12482
• [TPU] Add example for profiling TPU inference by @​mgoin in https://github.com/vllm-project/vllm/pull/12531
• [Frontend] Support reasoning content for deepseek r1 by @​gaocegege in https://github.com/vllm-project/vllm/pull/12473
• [Doc] Convert docs to use colon fences by @​hmellor in https://github.com/vllm-project/vllm/pull/12471
• [V1][Metrics] Add TTFT and TPOT histograms by @​markmc in https://github.com/vllm-project/vllm/pull/12530
• Bugfix for whisper quantization due to fake k_proj bias by @​mgoin in https://github.com/vllm-project/vllm/pull/12524
• [V1] Improve Error Message for Unsupported Config by @​robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/12535
• Fix the pydantic logging validator by @​maxdebayser in https://github.com/vllm-project/vllm/pull/12420
• [Bugfix] handle alignment of arguments in convert_sparse_cross_attention_mask_to_dense by @​tjohnson31415 in https://github.com/vllm-project/vllm/pull/12347
• [Model] Refactoring of MiniCPM-V and add MiniCPM-o-2.6 support for vLLM by @​HwwwwwwwH in https://github.com/vllm-project/vllm/pull/12069
• [Frontend] Support override generation config in args by @​liuyanyi in https://github.com/vllm-project/vllm/pull/12409
• [Hardware][NV] Fix Modelopt model loading for k-v-scales for Llama models. by @​pavanimajety in https://github.com/vllm-project/vllm/pull/11787
• [Kernel] add triton fused moe kernel for gptq/awq by @​jinzhen-lin in https://github.com/vllm-project/vllm/pull/12185
• Revert "[Build/CI] Fix libcuda.so linkage" by @​tlrmchlsmth in https://github.com/vllm-project/vllm/pull/12552
• [V1][BugFix] Free encoder cache for aborted requests by @​WoosukKwon in https://github.com/vllm-project/vllm/pull/12545
• [Misc][MoE] add Deepseek-V3 moe tuning support by @​divakar-amd in https://github.com/vllm-project/vllm/pull/12558
• [V1][Metrics] Add GPU cache usage % gauge by @​markmc in https://github.com/vllm-project/vllm/pull/12561
• Set ?device={device} when changing tab in installation guides by @​hmellor in https://github.com/vllm-project/vllm/pull/12560
• [Misc] fix typo: add missing space in lora adapter error message by @​Beim in https://github.com/vllm-project/vllm/pull/12564
• [Kernel] Triton Configs for Fp8 Block Quantization by @​robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/11589
• [CPU][PPC] Updated torch, torchvision, torchaudio dependencies by @​npanpaliya in https://github.com/vllm-project/vllm/pull/12555
• [V1][Log] Add max request concurrency log to V1 by @​mgoin in https://github.com/vllm-project/vllm/pull/12569
• [Kernel] Update cutlass_scaled_mm to support 2d group (blockwise) scaling by @​LucasWilkinson in https://github.com/vllm-project/vllm/pull/11868
• [ROCm][AMD][Model] llama 3.2 support upstreaming by @​maleksan85 in https://github.com/vllm-project/vllm/pull/12421
• [Attention] MLA decode optimizations by @​LucasWilkinson in https://github.com/vllm-project/vllm/pull/12528
• [Bugfix] Gracefully handle huggingface hub http error by @​ywang96 in https://github.com/vllm-project/vllm/pull/12571
• Add favicon to docs by @​hmellor in https://github.com/vllm-project/vllm/pull/12611
• [BugFix] Fix Torch.Compile For DeepSeek by @​robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/12594
• [Git] Automatically sign-off commits by @​comaniac in https://github.com/vllm-project/vllm/pull/12595
• [Docs][V1] Prefix caching design by @​comaniac in https://github.com/vllm-project/vllm/pull/12598
• [v1][Bugfix] Add extra_keys to block_hash for prefix caching by @​heheda12345 in https://github.com/vllm-project/vllm/pull/12603
• [release] Add input step to ask for Release version by @​khluu in https://github.com/vllm-project/vllm/pull/12631
• [Bugfix] Revert MoE Triton Config Default by @​robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/12629
• [Kernel][Quantization] Integrate block-quantized CUTLASS kernels for DeepSeekV3 by @​tlrmchlsmth in https://github.com/vllm-project/vllm/pull/12587
• [Feature] Fix guided decoding blocking bitmask memcpy by @​xpbowler in https://github.com/vllm-project/vllm/pull/12563
• [Doc] Improve installation signposting by @​hmellor in https://github.com/vllm-project/vllm/pull/12575
• [Doc] int4 w4a16 example by @​brian-dellabetta in https://github.com/vllm-project/vllm/pull/12585
• [V1] Bugfix: Validate Model Input Length by @​robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/12600
• [BugFix] fix wrong output when using lora and num_scheduler_steps=8 by @​sleepwalker2017 in https://github.com/vllm-project/vllm/pull/11161
• Fix target matching for fused layers with compressed-tensors by @​eldarkurtic in https://github.com/vllm-project/vllm/pull/12617
• [ci] Upgrade transformers to 4.48.2 in CI dependencies by @​khluu in https://github.com/vllm-project/vllm/pull/12599
• [Bugfix/CI] Fixup benchmark_moe.py by @​tlrmchlsmth in https://github.com/vllm-project/vllm/pull/12562
• Fix: Respect sparsity_config.ignore in Cutlass Integration by @​rahul-tuli in https://github.com/vllm-project/vllm/pull/12517
• [Attention] Deepseek v3 MLA support with FP8 compute by @​LucasWilkinson in https://github.com/vllm-project/vllm/pull/12601
• [CI/Build] Add label automation for structured-output, speculative-decoding, v1 by @​russellb in https://github.com/vllm-project/vllm/pull/12280
• Disable chunked prefill and/or prefix caching when MLA is enabled by @​simon-mo in https://github.com/vllm-project/vllm/pull/12642

New Contributors

• @​abmfy made their first contribution in https://github.com/vllm-project/vllm/pull/11194
• @​hosseinsarshar made their first contribution in https://github.com/vllm-project/vllm/pull/12453
• @​gmarinho2 made their first contribution in https://github.com/vllm-project/vllm/pull/12021
• @​waltforme made their first contribution in https://github.com/vllm-project/vllm/pull/12514
• @​fenghuizhang made their first contribution in https://github.com/vllm-project/vllm/pull/12482
• @​gaocegege made their first contribution in https://github.com/vllm-project/vllm/pull/12473
• @​Beim made their first contribution in https://github.com/vllm-project/vllm/pull/12564
• @​xpbowler made their first contribution in https://github.com/vllm-project/vllm/pull/12563
• @​brian-dellabetta made their first contribution in https://github.com/vllm-project/vllm/pull/12585
• @​sleepwalker2017 made their first contribution in https://github.com/vllm-project/vllm/pull/11161
• @​eldarkurtic made their first contribution in https://github.com/vllm-project/vllm/pull/12617

Full Changelog: vllm-project/vllm@v0.7.0...v0.7.1

v0.7.0

Compare Source

Highlights

• vLLM's V1 engine is ready for testing! This is a rewritten engine designed for performance and architectural simplicity. You can turn it on by setting environment variable VLLM_USE_V1=1. See our blog for more details. (44 commits).
• New methods (LLM.sleep, LLM.wake_up, LLM.collective_rpc, LLM.reset_prefix_cache) in vLLM for the post training frameworks! (#​12361, #​12084, #​12284).
• torch.compile is now fully integrated in vLLM, and enabled by default in V1. You can turn it on via -O3 engine parameter. (#​11614, #​12243, #​12043, #​12191, #​11677, #​12182, #​12246).

This release features

• 400 commits from 132 contributors, including 57 new contributors.
  • 28 CI and build enhancements, including testing for nightly torch (#​12270) and inclusion of genai-perf for benchmark (#​10704).
  • 58 documentation enhancements, including reorganized documentation structure (#​11645, #​11755, #​11766, #​11843, #​11896).
  • more than 161 bug fixes and miscellaneous enhancements

Features

Models

• New generative models: CogAgent (#​11742), Deepseek-VL2 (#​11578, #​12068, #​12169), fairseq2 Llama (#​11442), InternLM3 (#​12037), Whisper (#​11280)
• New pooling models: Qwen2 PRM (#​12202), InternLM2 reward models (#​11571)
• VLM: Merged multi-modal processor is now ready for model developers! (#​11620, #​11900, #​11682, #​11717, #​11669, #​11396)
  • Any model that implements merged multi-modal processor and the get_*_embeddings methods according to this guide is automatically supported by V1 engine.

Hardwares

• Apple: Native support for macOS Apple Silicon (#​11696)
• AMD: MI300 FP8 format for block_quant (#​12134), Tuned MoE configurations for multiple models (#​12408, #​12049), block size heuristic for avg 2.8x speedup for int8 models (#​11698)
• TPU: support for W8A8 (#​11785)
• x86: Multi-LoRA (#​11100) and MoE Support (#​11831)
• Progress in out-of-tree hardware support (#​12009, #​11981, #​11948, #​11609, #​12264, #​11516, #​11503, #​11369, #​11602)

Features

• Distributed:
  • Support torchrun and SPMD-style offline inference (#​12071)
  • New collective_rpc abstraction (#​12151, #​11256)
• API Server: Jina- and Cohere-compatible Rerank API (#​12376)
• Kernels:
  • Flash Attention 3 Support (#​12093)
  • Punica prefill kernels fusion (#​11234)
  • For Deepseek V3: optimize moe_align_block_size for cuda graph and large num_experts (#​12222)

Others

• Benchmark: new script for CPU offloading (#​11533)
• Security: Set weights_only=True when using torch.load() ([#​12366](https://re

---

Configuration

📅 Schedule: Branch creation - "" in timezone America/Toronto, Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about these updates again.

---

• If you want to rebase/retry this PR, check this box

---

This PR was generated by Mend Renovate. View the repository job log.

[renovate]

⚠️ Artifact update problem

Renovate failed to update artifacts related to this branch. You probably do not want to merge this PR as-is.

♻ Renovate will retry this branch, including artifacts, only when one of the following happens:

• any of the package files in this branch needs updating, or
• the branch becomes conflicted, or
• you click the rebase/retry checkbox if found above, or
• you rename this PR's title to start with "rebase!" to trigger it manually

The artifact failure details are included below:

File name: model-servers/vllm/0.6.4/Pipfile.lock

Command failed: pipenv lock
Creating a virtualenv for this project
Pipfile: 
/tmp/renovate/repos/github/redhat-ai-dev/developer-images/model-servers/vllm/0.6
.4/Pipfile
Using /opt/containerbase/tools/python/3.11.11/bin/python33.11.11 to create 
virtualenv...
created virtual environment CPython3.11.11.final.0-64 in 429ms
  creator 
CPython3Posix(dest=/tmp/renovate/cache/others/virtualenvs/0.6.4-d_r4MF2m, 
clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, 
wheel=bundle, via=copy, 
app_data_dir=/tmp/containerbase/cache/.local/share/virtualenv)
    added seed packages: pip==24.3.1, setuptools==75.8.0, wheel==0.45.1
  activators 
BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator
,PythonActivator

✔ Successfully created virtual environment!
Virtualenv location: /tmp/renovate/cache/others/virtualenvs/0.6.4-d_r4MF2m
Locking [packages] dependencies...
CRITICAL:pipenv.patched.pip._internal.resolution.resolvelib.factory:Cannot 
install -r /tmp/pipenv-55nn5owk-requirements/pipenv-ics_az7m-constraints.txt 
(line 14) and torch==2.3.0+cu121 because these package versions have conflicting
dependencies.
[ResolutionFailure]:   File 
"/opt/containerbase/tools/pipenv/2024.4.1/3.11.11/lib/python3.11/site-packages/p
ipenv/resolver.py", line 451, in main
[ResolutionFailure]:       _main(
[ResolutionFailure]:   File 
"/opt/containerbase/tools/pipenv/2024.4.1/3.11.11/lib/python3.11/site-packages/p
ipenv/resolver.py", line 436, in _main
[ResolutionFailure]:       resolve_packages(
[ResolutionFailure]:   File 
"/opt/containerbase/tools/pipenv/2024.4.1/3.11.11/lib/python3.11/site-packages/p
ipenv/resolver.py", line 400, in resolve_packages
[ResolutionFailure]:       results, resolver = resolve_deps(
[ResolutionFailure]:       ^^^^^^^^^^^^^
[ResolutionFailure]:   File 
"/opt/containerbase/tools/pipenv/2024.4.1/3.11.11/lib/python3.11/site-packages/p
ipenv/utils/resolver.py", line 967, in resolve_deps
[ResolutionFailure]:       results, hashes, internal_resolver = 
actually_resolve_deps(
[ResolutionFailure]:       ^^^^^^^^^^^^^^^^^^^^^^
[ResolutionFailure]:   File 
"/opt/containerbase/tools/pipenv/2024.4.1/3.11.11/lib/python3.11/site-packages/p
ipenv/utils/resolver.py", line 735, in actually_resolve_deps
[ResolutionFailure]:       resolver.resolve()
[ResolutionFailure]:   File 
"/opt/containerbase/tools/pipenv/2024.4.1/3.11.11/lib/python3.11/site-packages/p
ipenv/utils/resolver.py", line 460, in resolve
[ResolutionFailure]:       raise ResolutionFailure(message=e)
Your dependencies could not be resolved. You likely have a mismatch in your 
sub-dependencies.
You can use $ pipenv run pip install <requirement_name> to bypass this 
mechanism, then run $ pipenv graph to inspect the versions actually installed in
the virtualenv.
Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: ResolutionImpossible: for help visit 
https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-depende
ncy-conflicts

Your dependencies could not be resolved. You likely have a mismatch in your 
sub-dependencies.
You can use $ pipenv run pip install <requirement_name> to bypass this 
mechanism, then run $ pipenv graph to inspect the versions actually installed in
the virtualenv.
Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: Failed to lock Pipfile.lock!

File name: model-servers/vllm/0.6.6/Pipfile.lock

Command failed: pipenv lock
Creating a virtualenv for this project
Pipfile: 
/tmp/renovate/repos/github/redhat-ai-dev/developer-images/model-servers/vllm/0.6
.6/Pipfile
Using /opt/containerbase/tools/python/3.11.11/bin/python33.11.11 to create 
virtualenv...
created virtual environment CPython3.11.11.final.0-64 in 148ms
  creator 
CPython3Posix(dest=/tmp/renovate/cache/others/virtualenvs/0.6.6-39QtryhM, 
clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, 
wheel=bundle, via=copy, 
app_data_dir=/tmp/containerbase/cache/.local/share/virtualenv)
    added seed packages: pip==24.3.1, setuptools==75.8.0, wheel==0.45.1
  activators 
BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator
,PythonActivator

✔ Successfully created virtual environment!
Virtualenv location: /tmp/renovate/cache/others/virtualenvs/0.6.6-39QtryhM
Locking [packages] dependencies...
CRITICAL:pipenv.patched.pip._internal.resolution.resolvelib.factory:Cannot 
install -r /tmp/pipenv-8dastkn7-requirements/pipenv-xwc2lplr-constraints.txt 
(line 21), -r /tmp/pipenv-8dastkn7-requirements/pipenv-xwc2lplr-constraints.txt 
(line 4) and torch==2.3.0+cu121 because these package versions have conflicting 
dependencies.
[ResolutionFailure]:   File 
"/opt/containerbase/tools/pipenv/2024.4.1/3.11.11/lib/python3.11/site-packages/p
ipenv/resolver.py", line 451, in main
[ResolutionFailure]:       _main(
[ResolutionFailure]:   File 
"/opt/containerbase/tools/pipenv/2024.4.1/3.11.11/lib/python3.11/site-packages/p
ipenv/resolver.py", line 436, in _main
[ResolutionFailure]:       resolve_packages(
[ResolutionFailure]:   File 
"/opt/containerbase/tools/pipenv/2024.4.1/3.11.11/lib/python3.11/site-packages/p
ipenv/resolver.py", line 400, in resolve_packages
[ResolutionFailure]:       results, resolver = resolve_deps(
[ResolutionFailure]:       ^^^^^^^^^^^^^
[ResolutionFailure]:   File 
"/opt/containerbase/tools/pipenv/2024.4.1/3.11.11/lib/python3.11/site-packages/p
ipenv/utils/resolver.py", line 967, in resolve_deps
[ResolutionFailure]:       results, hashes, internal_resolver = 
actually_resolve_deps(
[ResolutionFailure]:       ^^^^^^^^^^^^^^^^^^^^^^
[ResolutionFailure]:   File 
"/opt/containerbase/tools/pipenv/2024.4.1/3.11.11/lib/python3.11/site-packages/p
ipenv/utils/resolver.py", line 735, in actually_resolve_deps
[ResolutionFailure]:       resolver.resolve()
[ResolutionFailure]:   File 
"/opt/containerbase/tools/pipenv/2024.4.1/3.11.11/lib/python3.11/site-packages/p
ipenv/utils/resolver.py", line 460, in resolve
[ResolutionFailure]:       raise ResolutionFailure(message=e)
Your dependencies could not be resolved. You likely have a mismatch in your 
sub-dependencies.
You can use $ pipenv run pip install <requirement_name> to bypass this 
mechanism, then run $ pipenv graph to inspect the versions actually installed in
the virtualenv.
Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: ResolutionImpossible: for help visit 
https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-depende
ncy-conflicts

Your dependencies could not be resolved. You likely have a mismatch in your 
sub-dependencies.
You can use $ pipenv run pip install <requirement_name> to bypass this 
mechanism, then run $ pipenv graph to inspect the versions actually installed in
the virtualenv.
Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: Failed to lock Pipfile.lock!