Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Doc] update multinode doc #181

Merged
merged 1 commit into from
Feb 27, 2025
Merged

Conversation

MengqingCao
Copy link
Contributor

Update multinode doc
fix #167 #168

Signed-off-by: MengqingCao <cmq0113@163.com>
@wangxiyuan
Copy link
Collaborator

I just notice that this env is set in platform. https://github.com/vllm-project/vllm-ascend/blob/main/vllm_ascend/platform.py#L36

So maybe it should be set during worker initialization as well?

@ApsarasX
Copy link

LGTM

@MengqingCao
Copy link
Contributor Author

So maybe it should be set during worker initialization as well?

I test locally when we set RAY_EXPERIMENTAL_NOSET_ASCEND_RT_VISIBLE_DEVICES to 1 in the init of NPUWorker, but it doesn't work.

I think this is quite similar to #92, we need to make sure that every ray task sets this environment variable, and this should be done before the actor is created.
In a multi-node environment, perhaps we can only put it here and let the user specify it manually now. Or we could add this to ray_patch if #92 is merged

@wangxiyuan
Copy link
Collaborator

@MengqingCao how about in register func?

@MengqingCao
Copy link
Contributor Author

@MengqingCao how about in register func?

It works for single-node.

but I think it may not work for multi-node, because setting it in register is quite same as setting it in Platform, the way we already implemented but not work.

@wangxiyuan wangxiyuan merged commit 03dc5c0 into vllm-project:main Feb 27, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants