Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add DeepSeek R1 WebGPU demo #74

Merged
merged 29 commits into from
Feb 19, 2025
Merged

Add DeepSeek R1 WebGPU demo #74

merged 29 commits into from
Feb 19, 2025

Conversation

Jiawei-Shao
Copy link
Contributor

No description provided.

@Jiawei-Shao
Copy link
Contributor Author

@ibelem PTAL, thanks!

@ibelem ibelem requested a review from NingW101 February 17, 2025 00:46
@NingW101
Copy link
Contributor

@Jiawei-Shao I found code not format following the rules defined in prettierrc, could you please format them, thanks

@Jiawei-Shao
Copy link
Contributor Author

@Jiawei-Shao I found code not format following the rules defined in prettierrc, could you please format them, thanks

Done @NingW101

@NingW101
Copy link
Contributor

@Jiawei-Shao I found code not format following the rules defined in prettierrc, could you please format them, thanks

Done @NingW101

Thanks. I'll update UI based on your commits

Comment on lines 146 to 154
if (!VITE_ENV_USE_REMOTE_MODELS) {
env.backends.onnx.wasm.wasmPaths = `${baseUrl}/models/frameworks/ort-web/ort-web@transformers_js_3_3_1/`;
env.allowLocalModels = true;
env.allowRemoteModels = false;
env.localModelPath = `${baseUrl}/models/`;
} else {
env.allowLocalModels = false;
env.allowRemoteModels = true;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (!VITE_ENV_USE_REMOTE_MODELS) {
env.backends.onnx.wasm.wasmPaths = `${baseUrl}/models/frameworks/ort-web/ort-web@transformers_js_3_3_1/`;
env.allowLocalModels = true;
env.allowRemoteModels = false;
env.localModelPath = `${baseUrl}/models/`;
} else {
env.allowLocalModels = false;
env.allowRemoteModels = true;
}
env.allowLocalModels = true;
env.localModelPath = `${baseUrl}/models/`;

These options enable users to fetch .json and .onnx files from both local and remote sources. The priority is to fetch from local storage first, and if unavailable, fall back to the remote source. Since users will be uploading .onnx models themselves, we only need to allow fetching .json files (such as tokenizer configurations) from local server.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@@ -41,7 +41,9 @@ function copyResourcesIntoDist(args) {

Copy link
Contributor

@NingW101 NingW101 Feb 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the entire models directory will not be copied only in --use-remote-models mode, we need to explicitly copy the resources required by the deepseek sample. Otherwise, deepseek won’t be able to fetch them from the deployed github.io server.

You can push "models/onnx-communify/DeepSeek-R1.." folder string into the RESOURCES_ARRAY or define another array variable const EXTRA_RESOURCE_ARRAY= ["models/onnx-communify/DeepSeek-R1.."] then execute the copy command as the

for (let path of EXTRA_RESOURCE_ARRAY) {
      if (path)
        execSync(
          `powershell -Command "Copy-Item -Path "${path}" -Destination "dist/${path}" -Recurse -Force"`,
          { stdio: "inherit" }
        );
    }

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Jiawei-Shao and others added 2 commits February 19, 2025 10:31
* update UI of deepseek-r1 sample

* UI tunning for deepseek r1 sample
Copy link
Member

@ibelem ibelem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Jiawei-Shao and @NingW101 's great efforts!

@ibelem
Copy link
Member

ibelem commented Feb 19, 2025

@Jiawei-Shao Could you please fix the two linting issues? Thanks

@ibelem
Copy link
Member

ibelem commented Feb 19, 2025

@Jiawei-Shao Thanks for fixing the @transformers_js_3_3_1/ort-wasm-simd-threaded.jsep.mjs linting issue, please also fix:

/home/runner/work/web-ai-showcase/web-ai-showcase/samples/deepseek-r1-webgpu/src/worker.js
  52:5   error  'past_key_values_cache' is assigned a value but never used                                                                     no-unused-vars
  64:[10](https://github.com/intel/web-ai-showcase/actions/runs/13406308241/job/37446789576?pr=74#step:4:11)  error  'START_THINKING_TOKEN_ID' is assigned a value but never used. Allowed unused elements of array destructuring must match /^_/u  no-unused-vars

✖ 2 problems (2 errors, 0 warnings)

Copy link
Member

@ibelem ibelem left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ibelem ibelem merged commit d7381ab into intel:main Feb 19, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants