diff --git a/docs/installation.md b/docs/installation.md index d2646d52..be3e232b 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -1,22 +1,76 @@ -# Installation +# Installation Guide -### 1. Dependencies -| Requirement | Supported version | Recommended version | Note | -| ------------ | ------- | ----------- | ----------- | -| Python | >= 3.9 | [3.10](https://www.python.org/downloads/) | Required for vllm | -| CANN | >= 8.0.RC2 | [8.0.RC3](https://www.hiascend.com/developer/download/community/result?module=cann&cann=8.0.0.beta1) | Required for vllm-ascend and torch-npu | -| torch-npu | >= 2.4.0 | [2.5.1rc1](https://gitee.com/ascend/pytorch/releases/tag/v6.0.0.alpha001-pytorch2.5.1) | Required for vllm-ascend | -| torch | >= 2.4.0 | [2.5.1](https://github.com/pytorch/pytorch/releases/tag/v2.5.1) | Required for torch-npu and vllm required | +## Prerequisites -### 2. Prepare Ascend NPU environment +Before installing vllm-ascend, please make sure that your environment meets the following requirements: +1. A hardware with Ascend NPU. It's usually the Atlas 800 A2 series. +2. Linux OS. It's recommended to use openEuler 22.03 or Ubuntu 22.04. +3. The firmware and driver for NPU is installed correctly. i.e. `npu-smi` command is available. +4. CANN 8.0.0 or later is installed. (It's not necessary if you use the container way, since it has installed inside the image.) -Below is a quick note to install recommended version software: +> Tips: following the instructions provided in the [Ascend Installation Guide](https://ascend.github.io/docs/sources/ascend/quick_install.html) can help you to set up the environment easily. -#### Containerized installation +## Installation -You can use the [container image](https://hub.docker.com/r/ascendai/cann) directly with one line command: +After the environment is ready, you can install vllm-ascend in three ways as follows. You can choose the one that suits your needs. + +### Install from PyPI + +>Note: Make sure Python is installed. It's recommended to use Python 3.10. + +There are two ways to install vllm-ascend based on the version of vllm. You can choose the one that suits your needs. + +#### Install version 0.7.1 + +1. Since vllm 0.7.1 doesn't work with pip on CPU, you need build vllm from source code + + ```bash + git clone --depth 1 --branch v0.7.1 https://github.com/vllm-project/vllm + cd vllm + python setup.py install + ``` + +2. Install torch cpu version + + ```bash + pip install torch==2.5.1 -f https://download.pytorch.org/whl/torch/ + ``` + +3. Install torch_npu + + ```bash + TODO: Update the link to the right version, once it's available. + pip install xxx + ``` + +4. Install vllm-ascend + + ```bash + pip install vllm-ascend==0.7.1 + ``` + +#### Install version >= 0.7.3 (Not supported yet) + +1. Install torch cpu version + + ```bash + pip install torch==2.5.1 -f https://download.pytorch.org/whl/torch/ + ``` + +2. Install vllm-ascend + + ```bash + # It'll install vllm and torch_npu automatically. + pip install vllm-ascend + ``` + +### Install from Docker image + +We have provided vllm-ascend official docker image for you to use. You can use the following command to pull and run the image easily. ```bash +docker pull ascendai/vllm-ascend:latest + docker run \ --name vllm-ascend-env \ --device /dev/davinci1 \ @@ -28,28 +82,21 @@ docker run \ -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \ -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \ -v /etc/ascend_install.info:/etc/ascend_install.info \ - -it quay.io/ascend/cann:8.0.rc3.beta1-910b-ubuntu22.04-py3.10 bash + -it quay.io/ascend/vllm-ascend:0.7.1 bash ``` -You do not need to install `torch` and `torch_npu` manually, they will be automatically installed as `vllm-ascend` dependencies. - -#### Manual installation +Then you will enter the container which has installed vllm-ascend 0.7.1 already. -Or follow the instructions provided in the [Ascend Installation Guide](https://ascend.github.io/docs/sources/ascend/quick_install.html) to set up the environment. +### Install from source code -### 3. Building - -#### Build Python package from source +If you want to install vllm-ascend from source code to test the newest code with main branch of vllm, or you want to contribute to vllm-ascend, you can use the following command to install it. ```bash -git clone https://github.com/vllm-project/vllm-ascend.git -cd vllm-ascend -pip install -e . -``` +git clone https://github.com/vllm-project/vllm +cd vllm +python setup.py install -#### Build container image from source -```bash git clone https://github.com/vllm-project/vllm-ascend.git cd vllm-ascend -docker build -t vllm-ascend-dev-image -f ./Dockerfile . +pip install -e . ```