Skip to content

Commit

Permalink
[Doc] Add install doc
Browse files Browse the repository at this point in the history
Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>
  • Loading branch information
wangxiyuan committed Feb 12, 2025
1 parent 28d7691 commit b94c5f6
Showing 1 changed file with 74 additions and 27 deletions.
101 changes: 74 additions & 27 deletions docs/installation.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,76 @@
# Installation
# Installation Guide

### 1. Dependencies
| Requirement | Supported version | Recommended version | Note |
| ------------ | ------- | ----------- | ----------- |
| Python | >= 3.9 | [3.10](https://www.python.org/downloads/) | Required for vllm |
| CANN | >= 8.0.RC2 | [8.0.RC3](https://www.hiascend.com/developer/download/community/result?module=cann&cann=8.0.0.beta1) | Required for vllm-ascend and torch-npu |
| torch-npu | >= 2.4.0 | [2.5.1rc1](https://gitee.com/ascend/pytorch/releases/tag/v6.0.0.alpha001-pytorch2.5.1) | Required for vllm-ascend |
| torch | >= 2.4.0 | [2.5.1](https://github.com/pytorch/pytorch/releases/tag/v2.5.1) | Required for torch-npu and vllm required |
## Prerequisites

### 2. Prepare Ascend NPU environment
Before installing vllm-ascend, please make sure that your environment meets the following requirements:
1. A hardware with Ascend NPU. It's usually the Atlas 800 A2 series.
2. Linux OS. It's recommended to use openEuler 22.03 or Ubuntu 22.04.
3. The firmware and driver for NPU is installed correctly. i.e. `npu-smi` command is available.
4. CANN 8.0.0 or later is installed. (It's not necessary if you use the container way, since it has installed inside the image.)

Below is a quick note to install recommended version software:
> Tips: following the instructions provided in the [Ascend Installation Guide](https://ascend.github.io/docs/sources/ascend/quick_install.html) can help you to set up the environment easily.
#### Containerized installation
## Installation

You can use the [container image](https://hub.docker.com/r/ascendai/cann) directly with one line command:
After the environment is ready, you can install vllm-ascend in three ways as follows. You can choose the one that suits your needs.

### Install from PyPI

>Note: Make sure Python is installed. It's recommended to use Python 3.10.
There are two ways to install vllm-ascend based on the version of vllm. You can choose the one that suits your needs.

#### Install version 0.7.1

1. Since vllm 0.7.1 doesn't work with pip on CPU, you need build vllm from source code

```bash
git clone --depth 1 --branch v0.7.1 https://github.com/vllm-project/vllm
cd vllm
python setup.py install
```

2. Install torch cpu version

```bash
pip install torch==2.5.1 -f https://download.pytorch.org/whl/torch/
```

3. Install torch_npu

```bash
TODO: Update the link to the right version, once it's available.
pip install xxx
```
4. Install vllm-ascend
```bash
pip install vllm-ascend==0.7.1
```
#### Install version >= 0.7.3 (Not supported yet)
1. Install torch cpu version
```bash
pip install torch==2.5.1 -f https://download.pytorch.org/whl/torch/
```
2. Install vllm-ascend
```bash
# It'll install vllm and torch_npu automatically.
pip install vllm-ascend
```

### Install from Docker image

We have provided vllm-ascend official docker image for you to use. You can use the following command to pull and run the image easily.

```bash
docker pull ascendai/vllm-ascend:latest
docker run \
--name vllm-ascend-env \
--device /dev/davinci1 \
Expand All @@ -28,28 +82,21 @@ docker run \
-v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ \
-v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
-v /etc/ascend_install.info:/etc/ascend_install.info \
-it quay.io/ascend/cann:8.0.rc3.beta1-910b-ubuntu22.04-py3.10 bash
-it quay.io/ascend/vllm-ascend:0.7.1 bash
```

You do not need to install `torch` and `torch_npu` manually, they will be automatically installed as `vllm-ascend` dependencies.

#### Manual installation
Then you will enter the container which has installed vllm-ascend 0.7.1 already.

Or follow the instructions provided in the [Ascend Installation Guide](https://ascend.github.io/docs/sources/ascend/quick_install.html) to set up the environment.
### Install from source code

### 3. Building

#### Build Python package from source
If you want to install vllm-ascend from source code to test the newest code with main branch of vllm, or you want to contribute to vllm-ascend, you can use the following command to install it.

```bash
git clone https://github.com/vllm-project/vllm-ascend.git
cd vllm-ascend
pip install -e .
```
git clone https://github.com/vllm-project/vllm
cd vllm
python setup.py install
#### Build container image from source
```bash
git clone https://github.com/vllm-project/vllm-ascend.git
cd vllm-ascend
docker build -t vllm-ascend-dev-image -f ./Dockerfile .
pip install -e .
```

0 comments on commit b94c5f6

Please sign in to comment.