Skip to content

Commit

Permalink
[3rdparty] Bump tokenizers-cpp for HuggingFace tokenizer update (#3029)
Browse files Browse the repository at this point in the history 
This PR bumps the 3rdparty tokenizers-cpp to update the HuggingFace
tokenizers package dependency.
  • Loading branch information
@MasterJH5574
MasterJH5574 authored Nov 15, 2024
1 parent a9baa1f commit 3474073
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion 3rdparty/tokenizers-cpp
Submodule tokenizers-cpp updated 2 files
+1 −1 rust/Cargo.toml
+2 −2 rust/src/lib.rs

0 comments on commit 3474073

Please sign in to comment.