[Python] Add sentencepiece as installation requirement (#3030)

This PR adds the sentencepiece package as a Python installation requirement for tokenizer coverage when running `gen_config`.
mlc-ai · Nov 15, 2024 · e462885 · e462885
1 parent 3474073
commit e462885
Showing 1 changed file with 1 addition and 0 deletions.
diff --git a/python/setup.py b/python/setup.py
@@ -104,6 +104,7 @@ def main():
             "safetensors",
             "requests",
             "tqdm",
+            "sentencepiece",
             "tiktoken",
             "prompt_toolkit",
             "openai",