diff --git a/README.md b/README.md index e42ecb7..8effa65 100644 --- a/README.md +++ b/README.md @@ -30,7 +30,6 @@ You can find the Gemma models on the Hugging Face Hub, Kaggle, Google Cloud Vert * [Gemma](Gemma/README.md) * [CodeGemma](CodeGemma/README.md) * [PaliGemma](PaliGemma/README.md) -* [Partner quickstart guides](partner-quickstarts/README.md) * [Workshops and technical talks](Workshops/README.md) * [Showcase complex end-to-end use cases](Demos/README.md) * [Gemma on Google Cloud](https://github.com/GoogleCloudPlatform/generative-ai/tree/main/open-models) : GCP open models has additional notebooks for using Gemma diff --git a/partner-quickstarts/Gemma-NVidia/README.md b/partner-quickstarts/Gemma-NVidia/README.md deleted file mode 100644 index 1be9336..0000000 --- a/partner-quickstarts/Gemma-NVidia/README.md +++ /dev/null @@ -1,63 +0,0 @@ -# Gemma - -[Gemma](https://ai.google.dev/gemma/docs) is a family of decoder-only, text-to-text large language models for English language, built from the same research and technology used to create the [Gemini models](https://blog.google/technology/ai/google-gemini-ai/). Gemma models have open weights and offer pre-trained variants and instruction-tuned variants. These models are well-suited for a variety of text generation tasks, including question answering, summarization, and reasoning. Their relatively small size makes it possible to deploy them in environments with limited resources such as a laptop, desktop, or your own cloud infrastructure, democratizing access to state-of-the-art AI models and helping foster innovation for everyone. -For more details, refer the the [Gemma model card](https://ai.google.dev/gemma/docs/model_card) released by Google. - - -## Customizing Gemma with NeMo Framework - -Gemma models are compatible with [NeMo Framework](https://docs.nvidia.com/nemo-framework/user-guide/latest/index.html). In this repository we have two notebooks that covert different ways of customizing Gemma. - -### Paramater Efficient Fine-Tuning with LoRA - -[LoRA tuning](https://arxiv.org/abs/2106.09685) is a parameter efficient method for fine-tuning models, where we freeze the base model parameters and update an auxilliary "adapter" with many fewer weights. At inference time, the adapter weights are combined with the base model weights to produce a new model, customized for a particular use case or dataset. Because this adapter is so much smaller than the base model, it can be trained with far fewer resources than it would take to fine-tune the entire model. In this example, we'll show you how to LoRA-tune small models like the Gemma models on a single GPU. - -[Get Started Here](./lora.ipynb) - -### Supervised Fine-Tuning for Instruction Following (SFT) - -Supervised Fine-Tuning (SFT) is the process of fine-tuning all of a model’s parameters on supervised data of inputs and outputs. It teaches the model how to follow user specified instructions and is typically done after model pre-training. This example will describe the steps involved in fine-tuning Gemma for instruction following. Gemma was released with a checkpoint already fine-tuned for instruction-following, but here we'll learn how we can tune our own model starting with the pre-trained checkpoint to acheive a similar outcome. - -Full fine-tuning is more resource intensive than Low Rank adaptation, so for SFT we'll need multiple GPUs, as opposed to the single GPU used for LoRA. - -[Get Started Here](./) - -## Download the base model - -For all of our customization and deployment processes, we'll need to start off with a pre-trained version of Gemma in the `.nemo` format. You can download the base model in `.nemo` format from the NVIDIA GPU Cloud, or convert checkpoints from another framework into a `.nemo` file. You can choose to use the 2B parameter or 7B parameter Gemma models for this notebook -- the 2B model will be faster to customize, but the 7B model will be more capable. - -You can download either model from the NVIDIA NGC Catalog, using the NGC CLI. The instructions to install and configure the NGC CLI can be found [here](https://ngc.nvidia.com/setup/installers/cli). - -To download the model, execute one of the following commands, based on which model you want to use: - -```bash -ngc registry model download-version "nvidia/nemo/gemma_2b_base:1.1" -``` - -or - -```bash -ngc registry model download-version "nvidia/nemo/gemma_7b_base:1.1" -``` - -## Getting NeMo Framework - -NVIDIA NeMo Framework is a generative AI framework built for researchers and PyTorch developers working on large language models (LLMs), multimodal models (MM), automatic speech recognition (ASR), and text-to-speech synthesis (TTS). The primary objective of NeMo is to provide a scalable framework for researchers and developers from industry and academia to more easily implement and design new generative AI models by being able to leverage existing code and pretrained models. - -You can pull a container that includes the version of NeMo Framework and all dependencies needed for these notebooks with the following: - -```bash -docker pull nvcr.io/nvidia/nemo:24.01.gemma -``` - -The best way to run this notebook is from within the container. You can do that by launching the container with the following command - -```bash -docker run -it --rm --gpus all --ipc host --network host -v $(pwd):/workspace nvcr.io/nvidia/nemo:24.01.gemma -``` - -Then, from within the container, start the jupyter server with - -```bash -jupyter lab --no-browser --port=8080 --allow-root --ip 0.0.0.0 -``` \ No newline at end of file diff --git a/partner-quickstarts/Gemma-NVidia/lora.ipynb b/partner-quickstarts/Gemma-NVidia/lora.ipynb deleted file mode 100644 index a4196e5..0000000 --- a/partner-quickstarts/Gemma-NVidia/lora.ipynb +++ /dev/null @@ -1,598 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Gemma Parameter Efficient Fine-Tuning with LoRA using NeMo Framework" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "[Gemma](https://ai.google.dev/gemma/docs/model_card) is a groundbreaking new open model in the Gemini family of models from Google. Gemma is just as powerful as previous models but compact enough to run locally on NVIDIA RTX GPUs. Gemma is available in 2 sizes: 2B and 7B parameters. With NVIDIA NeMo, you can customize Gemma to fit your usecase and deploy an optimized model on your NVIDIA GPU.\n", - "\n", - "In this tutorial, we'll go over a specific kind of customization -- Low-rank adapter tuning to follow a specific output format (also known as LoRA). To learn how to perform full parameter supervised fine-tuning for instruction following (also known as SFT), see the [companion notebook](./sft.ipynb). For LoRA, we'll perform all operations within the notebook on a single GPU. The compute resources needed for training depend on which Gemma model you use. For the 7 billion parameter variant of Gemma, you'll need a GPU with 80GB of memory. For the 2 billion parameter model, 40GB will do. \n", - "\n", - "We'll also learn how to export your custom model to TensorRT-LLM, an open-source library that accelerates and optimizes inference performance of the latest LLMs on the NVIDIA AI platform." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Introduction" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "[LoRA tuning](https://arxiv.org/abs/2106.09685) is a parameter efficient method for fine-tuning models, where we freeze the base model parameters and update an auxiliary \"adapter\" with many fewer weights. At inference time, the adapter weights are combined with the base model weights to produce a new model, customized for a particular use case or dataset. Because this adapter is so much smaller than the base model, it can be trained with far fewer resources than it would take to fine-tune the entire model. In this notebook, we'll show you how to LoRA-tune small models like the Gemma models on a single A100 GPU.\n", - "\n", - "For this example, we're going to tune our Gemma model on the [PubMedQA](https://pubmedqa.github.io/) dataset, a Question Answering dataset for biomedical texts. We'll see later on that our base model performs pretty poorly on this dataset -- not necessarily because it can't accurately extract the answers from the context, but because it fails to respond in the \"yes/no/maybe\" format expected. With LoRA finetuning, we'll modify our model to respond in the way we need for our task." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Download the base model\n", - "\n", - "For all of our customization and deployment processes, we'll need to start off with a pre-trained version of Gemma in the `.nemo` format. You can download the base model in `.nemo` format from the NVIDIA GPU Cloud, or convert checkpoints from another framework into a `.nemo` file. You can choose to use the 2B parameter or 7B parameter Gemma models for this notebook -- the 2B model will be faster to customize, but the 7B model will be more capable. \n", - "\n", - "You can download either model from the NVIDIA NGC Catalog, using the NGC CLI. The instructions to install and configure the NGC CLI can be found [here](https://ngc.nvidia.com/setup/installers/cli).\n", - "\n", - "To download the model, execute one of the following commands, based on which model you want to use:\n", - "\n", - "```bash\n", - "ngc registry model download-version \"nvidia/nemo/gemma_2b_base:1.1\"\n", - "```\n", - "\n", - "or\n", - "\n", - "```bash\n", - "ngc registry model download-version \"nvidia/nemo/gemma_7b_base:1.1\"\n", - "```" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Getting NeMo Framework\n", - "\n", - "NVIDIA NeMo Framework is a generative AI framework built for researchers and PyTorch developers working on large language models (LLMs), multimodal models (MM), automatic speech recognition (ASR), and text-to-speech synthesis (TTS). The primary objective of NeMo is to provide a scalable framework for researchers and developers from industry and academia to more easily implement and design new generative AI models by being able to leverage existing code and pretrained models.\n", - "\n", - "If you haven't already, you can pull a container that includes the version of NeMo Framework and all dependencies needed for this notebook with the following:\n", - "\n", - "```bash\n", - "docker pull nvcr.io/nvidia/nemo:24.01.gemma\n", - "```\n", - "\n", - "The best way to run this notebook is from within the container. You can do that by launching the container with the following command\n", - "\n", - "```bash\n", - "docker run -it --rm --gpus all --ipc=host --network host -v $(pwd):/workspace nvcr.io/nvidia/nemo:24.01.gemma\n", - "```\n", - "\n", - "Then, from within the container, start the jupyter server with\n", - "\n", - "```bash\n", - "jupyter lab --no-browser --port=8080 --allow-root --ip 0.0.0.0\n", - "```\n", - "\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Data Preparation\n", - "\n", - "First let's download the data, and use the provided script to divide it into train/validation/test splits" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "!git clone https://github.com/pubmedqa/pubmedqa.git\n", - "!cd pubmedqa/preprocess && python split_dataset.py pqal" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Let's look at a sample of our training data" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "import json\n", - "data = json.load(open(\"pubmedqa/data/pqal_fold0/train_set.json\", 'rt'))\n", - "data[list(data)[0]]" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "As we can see, the data is in `json` format and includes several different fields and labels. For doing parameter efficient fine-tuning with NeMo Framework, we need the data in `jsonl` format. We also need to reformat our dataset into input and output pairs so that we can tune our model in a supervised way. The helper functions below take care of extracting and reformatting the raw data and writing out the `jsonl` file. " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "import json\n", - "\n", - "def write_jsonl(fname, json_objs):\n", - " with open(fname, 'wt') as f:\n", - " for o in json_objs:\n", - " f.write(json.dumps(o)+\"\\n\")\n", - "\n", - "# Converts the data from the original format into an LLM input prompt\n", - "def form_question(obj):\n", - " st = \"\"\n", - " st += f\"QUESTION:{obj['QUESTION']}\\n\"\n", - " st += \"CONTEXT: \"\n", - " for i, label in enumerate(obj['LABELS']):\n", - " st += f\"{obj['CONTEXTS'][i]}\\n\"\n", - " st += f\"TARGET: the answer to the question given the context is (yes|no|maybe): \"\n", - " return st\n", - "\n", - "def convert_to_jsonl(data_path, output_path):\n", - " data = json.load(open(data_path, 'rt'))\n", - " json_objs = []\n", - " for k in data.keys():\n", - " obj = data[k]\n", - " prompt = form_question(obj)\n", - " completion = obj['reasoning_required_pred']\n", - " json_objs.append({\"input\": prompt, \"output\": completion})\n", - " write_jsonl(output_path, json_objs)\n", - " return json_objs\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "test_json_objs = convert_to_jsonl(\"pubmedqa/data/test_set.json\", \"pubmedqa_test.jsonl\")\n", - "train_json_objs = convert_to_jsonl(\"pubmedqa/data/pqal_fold0/train_set.json\", \"pubmedqa_train.jsonl\")\n", - "dev_json_objs = convert_to_jsonl(\"pubmedqa/data/pqal_fold0/dev_set.json\", \"pubmedqa_val.jsonl\")" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Here's an example of what the data looks like after formatting" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "test_json_objs[0]" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Configuration and Training" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "NeMo Framework uses config objects to control many of its operations, which allows you to quickly see what options you can change and carry out different experiments. We can start by downloading an example config file from github." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "!wget https://raw.githubusercontent.com/NVIDIA/NeMo/main/examples/nlp/language_modeling/tuning/conf/megatron_gpt_finetuning_config.yaml" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now we'll read in this default config file with Hydra, and apply an override that enables the use of [Megatron core](https://github.com/NVIDIA/Megatron-LM/tree/main/megatron/core). " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "import hydra\n", - "from omegaconf.omegaconf import OmegaConf\n", - "\n", - "hydra.initialize(version_base=None, config_path=\".\")" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "cfg = hydra.compose(config_name=\"megatron_gpt_finetuning_config\", overrides=['++model.mcore_gpt=True'])\n", - "cfg.name = \"gemma_lora_pubmedqa\"" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Let's see what the default configuration looks like before we make any modifications " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "print(OmegaConf.to_yaml(cfg))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "To see all of the different configuration options available, you can take a look at the file we downloaded. For this example, we're going to update a couple of settings to point to our newly-prepared datasets and to make sure the LoRA tuning runs on our A100. Feel free to experiment with these different options -- you can swap in your own datasets and change the training settings depending on what GPU you're using.\n", - "\n", - "For data our data configuration, we'll point to the `jsonl` files we wrote out earlier. `concat_sampling_probabilities` determines what percentage of the finetuning data you would like to come from each file -- in our example we only have 1 training file so we choose `[1.0]`" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "OmegaConf.update(cfg, \"model.data\", {\n", - " \"train_ds\": {\n", - " \"num_workers\": 0,\n", - " \"file_names\": [\"pubmedqa_train.jsonl\"],\n", - " \"concat_sampling_probabilities\": [1.0]\n", - " },\n", - " \"validation_ds\": {\n", - " \"num_workers\": 0,\n", - " \"file_names\": [\"pubmedqa_val.jsonl\"]\n", - " },\n", - " \"test_ds\": {\n", - " \"file_names\": [\"pubmedqa_test.jsonl\"],\n", - " \"names\": [\"pubmedqa\"],\n", - " \"add_bos\": True\n", - " }\n", - "}, merge=True)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "For our model settings, we don't have much to change since we're reading in a pretrained model and can inherit the values that were already set. We need to point to our existing `.nemo` file, specify that we want to use LoRA as our scheme for finetuning, and choose our parallelism and batch size values. The values below should be appropriate for a single A100 GPU.\n", - "\n", - "Make sure to change the `restore_from_path` setting if you're using a different checkpoint!" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "\n", - "OmegaConf.update(cfg, \"model\", {\n", - " \"restore_from_path\": \"gemma_7b_pt.nemo\",\n", - " \"peft\": {\n", - " \"peft_scheme\": \"lora\"\n", - " },\n", - " \"tensor_model_parallel_size\": 1,\n", - " \"pipeline_model_parallel_size\": 1,\n", - " \"micro_batch_size\": 1,\n", - " \"global_batch_size\": 8,\n", - "}, merge=True)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Finally, we set some options for the trainer. We'll be training on 1 GPU on a single node, at bfloat16 precision. For this example we'll also only train for 50 steps, with a validation check every after every 20 iterations." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "OmegaConf.update(cfg, \"trainer\", {\n", - " 'devices': 1,\n", - " 'num_nodes': 1,\n", - " 'precision': \"bf16\",\n", - " \"val_check_interval\": 20,\n", - " \"max_steps\": 50\n", - "})" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "With our configurations set, we are ready to initialize our `Trainer` object to handle our training loop, and an experiment manager to handle checkpointing and logging." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from nemo.collections.nlp.parts.megatron_trainer_builder import MegatronLMPPTrainerBuilder\n", - "from nemo.utils.exp_manager import exp_manager\n", - "\n", - "trainer = MegatronLMPPTrainerBuilder(cfg).create_trainer()\n", - "exp_manager(trainer, cfg.exp_manager)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "After initializing the Trainer object we can load our model from disk into memory. To load the model weights we'll need a config object for the model, which we can read from the `.nemo` file on disk and update it with any new settings we added in our LoRA config above." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from nemo.collections.nlp.models.language_modeling.megatron_gpt_sft_model import MegatronGPTSFTModel\n", - "\n", - "model_cfg = MegatronGPTSFTModel.merge_cfg_with(cfg.model.restore_from_path, cfg)\n", - "model = MegatronGPTSFTModel.restore_from(cfg.model.restore_from_path, model_cfg, trainer=trainer)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now, let's see how to add the LoRA Adapter to our model and train it. We can specify that we want to use LoRA by using the `LoraPEFTConfig` class, which stores the types of applicable adapter and the hyperparameters required to initialize the adapter module. " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from nemo.collections.nlp.parts.peft_config import LoraPEFTConfig\n", - "\n", - "my_peft_config = LoraPEFTConfig(model_cfg)\n", - "print(OmegaConf.to_yaml(my_peft_config.get_config_dict()))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "We can then call `add_apater` to actuall add the LoRA adapter to our base model and prepare it for training. When we first call `add_adapter`, the model prints out the parameter count before and after the operation and we can see the number of trainable parameters rise. " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "model.add_adapter(my_peft_config)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Note: If you want to use a different PEFT method, you can use a different config class in place of `LoraPEFTConfig`, such as `CanonicalAdaptersPEFTConfig`, `IA3PEFTConfig`, or `PtuningPEFTConfig`. You can also use a combination of the methods by passing in a list: `model.add_adapter([LoraPEFTConfig(model_cfg), PtuningPEFTConfig(model_cfg)])`" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "We're now ready to start training! As the training loop runs, you'll see the validation loss drop significantly -- even with this short demonstration." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "trainer.fit(model)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Once training is completed you should see a saved '.nemo' file in the `nemo_experiments/{taurus}_lora_pubmedqa` folder. This checkpoint will only contain the trained adapter weights, and not the frozen base model weights.\n", - "\n", - "We can also now see how the newly finetuned model performs on the test data:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "trainer.test(model)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Much better! If you want to learn how to export our new model for optimized inference using TensorRT-LLM, continue on to the \"Exporting to TensorRT-LLM\" section." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Exporting to TensorRT-LLM" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "TensorRT-LLM is an open-source library for optimizing inference performance to acheive state-of-the-art speed on NVDIA GPUs. The NeMo framework offers an easy way to compile `.nemo` models into optimized TensorRT-LLM engines which you can run locally embedded in another application, or serve to other applications using a server like Triton Inference Server." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "To start with, lets create a folder where our exported model will land" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "!mkdir gemma_trt_llm" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "To export the model to TensorRT-LLM, we'll need to merge the weights of the base model and the weights of the adapter. If you're using the NeMo Framework container, you'll find a script for this at `/opt/NeMo/scripts/nlp_language_modeling/merge_lora_weights/merge.py`. Otherwise, you can download the standalone script from GitHub at `https://raw.githubusercontent.com/NVIDIA/NeMo/main/scripts/nlp_language_modeling/merge_lora_weights/merge.py`. To run the merge script, you'll need the path to the base model and trained adapter, as well as a path to save the merged model to." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "%%bash\n", - "python /opt/NeMo/scripts/nlp_language_modeling/merge_lora_weights/merge.py \\\n", - " trainer.accelerator=gpu \\\n", - " tensor_model_parallel_size=1 \\\n", - " pipeline_model_parallel_size=1 \\\n", - " gpt_model_file=${MODEL} \\\n", - " lora_model_path=nemo_experiments/gemma_lora_pubmedqa/checkpoints/gemma_lora_pubmedqa.nemo \\\n", - " merged_model_path=gemma_lora_pubmedqa_merged.nemo" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "With our merged model weights, we just need to create an instance of the `TensorRTLLM` class and call the `TensorRTLLM.export()` function -- pointing the `nemo_checkpoint_path` argument to the newly merged model from above.\n", - "\n", - "This creates a couple of files in the folder we created -- an `engine` file that holds the weights and the compiled execution graph of the model, a `tokenizer.model` file which holds the tokenizer information, and `config.json` which holds some metadata about the model (along with `model.cache`, which caches some operations and makes it faster to re-compile the model in the future.)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from nemo.export import TensorRTLLM\n", - "trt_llm_exporter = TensorRTLLM(model_dir=\"gemma_pubmedqa_merged_trt_llm\")\n", - "trt_llm_exporter.export(nemo_checkpoint_path=\"gemma_lora_pubmedqa_merged.nemo\", model_type=\"gemma\", n_gpus=1)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "With the model exported into TensorRTLLM, we can perform very fast inference:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "trt_llm_exporter.forward([\"NVIDIA and Google are \"])" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "There's also a convenient function to deploy a the model as a service, backed by Triton Inference Server:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from nemo.deploy import DeployPyTriton\n", - "\n", - "nm = DeployPyTriton(model=trt_llm_exporter, triton_model_name=\"gemma\")\n", - "nm.deploy()\n", - "nm.serve()" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "nemo_lora", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.10.12" - } - }, - "nbformat": 4, - "nbformat_minor": 2 -} diff --git a/partner-quickstarts/Gemma-NVidia/sft.ipynb b/partner-quickstarts/Gemma-NVidia/sft.ipynb deleted file mode 100644 index e452fb6..0000000 --- a/partner-quickstarts/Gemma-NVidia/sft.ipynb +++ /dev/null @@ -1,426 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Supervised Fine-Tuning for Instruction Following" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "[Gemma](https://ai.google.dev/gemma/docs/model_card) is a groundbreaking new open model in the Gemini family of models from Google. Gemma is just as powerful as previous models but compact enough to run locally on NVIDIA RTX GPUs. Gemma is available in 2 sizes: 2B and 7B parameters. With NVIDIA NeMo, you can customize Gemma to fit your usecase and deploy an optimized model on your NVIDIA GPU.\n", - "\n", - "In this tutorial, we'll go over a specific kind of customization -- full parameter supervised fine-tuning for instruction following (also known as SFT). To learn how to perform Low-rank adapter (LoRA) tuning to follow a specific output format, see the [companion notebook](./lora.ipynb). For LoRA, we'll show how you can kick off a multi-GPU training job with an example script so that you can train on 8 GPUs. The exact number of GPUs needed will depend on which model you use and what kind of GPUs you use, but we recommend using 8 A100-80GB GPUs.\n", - "\n", - "We'll also learn how to export your custom model to TensorRT-LLM, an open-source library that accelerates and optimizes inference performance of the latest LLMs on the NVIDIA AI platform." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Introduction" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Supervised Fine-Tuning (SFT) is the process of fine-tuning all of a model’s parameters on supervised data of inputs and outputs. It teaches the model how to follow user specified instructions and is typically done after model pre-training. This notebook describes the steps involved in fine-tuning Gemma for instruction following. Gemma was released with a checkpoint already fine-tuned for instruction-following, but here we'll learn how we can tune our own model starting with the pre-trained checkpoint to achieve a similar outcome. " - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Download the base model\n", - "\n", - "For all of our customization and deployment processes, we'll need to start off with a pre-trained version of Gemma in the `.nemo` format. You can download the base model in `.nemo` format from the NVIDIA GPU Cloud, or convert checkpoints from another framework into a `.nemo` file. You can choose to use the 2B parameter or 7B parameter Gemma models for this notebook -- the 2B model will be faster to customize, but the 7B model will be more capable. \n", - "\n", - "You can download either model from the NVIDIA NGC Catalog, using the NGC CLI. The instructions to install and configure the NGC CLI can be found [here](https://ngc.nvidia.com/setup/installers/cli).\n", - "\n", - "To download the model, execute one of the following commands, based on which model you want to use:\n", - "\n", - "```bash\n", - "ngc registry model download-version \"nvidia/nemo/gemma_2b_base:1.1\"\n", - "```\n", - "\n", - "or\n", - "\n", - "```bash\n", - "ngc registry model download-version \"nvidia/nemo/gemma_7b_base:1.1\"\n", - "```" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Getting NeMo Framework\n", - "\n", - "NVIDIA NeMo Framework is a generative AI framework built for researchers and PyTorch developers working on large language models (LLMs), multimodal models (MM), automatic speech recognition (ASR), and text-to-speech synthesis (TTS). The primary objective of NeMo is to provide a scalable framework for researchers and developers from industry and academia to more easily implement and design new generative AI models by being able to leverage existing code and pretrained models.\n", - "\n", - "If you haven't already, you can pull a container that includes the version of NeMo Framework and all dependencies needed for this notebook with the following:\n", - "\n", - "```bash\n", - "docker pull nvcr.io/nvidia/nemo:24.01.gemma\n", - "```\n", - "\n", - "The best way to run this notebook is from within the container. You can do that by launching the container with the following command\n", - "\n", - "```bash\n", - "docker run -it --rm --gpus all --ipc=host --network host -v $(pwd):/workspace nvcr.io/nvidia/nemo:24.01.gemma\n", - "```\n", - "\n", - "Then, from within the container, start the jupyter server with\n", - "\n", - "```bash\n", - "jupyter lab --no-browser --port=8080 --allow-root --ip 0.0.0.0\n", - "```" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## SFT Data Formatting" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "To begin, we'll need to prepare a dataset to tune our model on.\n", - "\n", - "This notebook uses the [Dolly dataset](https://huggingface.co/datasets/databricks/databricks-dolly-15k) as an example to demonstrate how to format your SFT data. This dataset consists of 15,000 instruction-context-response triples.\n", - "\n", - "First, to download the data enter the following command:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "!wget https://huggingface.co/datasets/databricks/databricks-dolly-15k/resolve/main/databricks-dolly-15k.jsonl" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "\n", - "The downloaded data, stored at `databricks-dolly-15k.jsonl`, is a `JSONL` file with each line formatted like this:\n", - "\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "vscode": { - "languageId": "plaintext" - } - }, - "outputs": [], - "source": [ - "{\n", - " \"instruction\": \"When did Virgin Australia start operating?\",\n", - " \"context\": \"Virgin Australia, the trading name of Virgin Australia Airlines Pty Ltd, is an Australian-based airline. It is the largest airline by fleet size to use the Virgin brand. It commenced services on 31 August 2000 as Virgin Blue, with two aircraft on a single route.[3] It suddenly found itself as a major airline in Australia's domestic market after the collapse of Ansett Australia in September 2001. The airline has since grown to directly serve 32 cities in Australia, from hubs in Brisbane, Melbourne and Sydney.[4]\",\n", - " \"response\": \"Virgin Australia commenced services on 31 August 2000 as Virgin Blue, with two aircraft on a single route.\",\n", - " \"category\": \"closed_qa\"\n", - "}" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "As this example shows, there are no clear “input” and “output” fields, which are required for SFT with NeMo. To remedy this, we can do some data pre-processing. This cell converts the `instruction`, `context`, and `response` fields into `input` and `output`. It also concatenates the `instruction` and `context` fields with a `\\n\\n` separator, and randomizes the order in which they appear in the input to generate a new `JSONL` file. This generates an output file called `databricks-dolly-15k-output.jsonl`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "import json\n", - "import numpy as np\n", - "\n", - "path_to_data = \"databricks-dolly-15k.jsonl\"\n", - "output_path = f\"{path_to_data.split('.')[0]}-output.jsonl\"\n", - "with open(path_to_data, \"r\") as f, open(output_path, \"w\") as g:\n", - " for line in f:\n", - "\n", - " # Read JSONL line in original format\n", - " line = json.loads(line)\n", - " context = line[\"context\"].strip()\n", - "\n", - " # Randomize context and instruction order.\n", - " if context != \"\":\n", - " context_first = np.random.randint(0, 2) == 0\n", - " if context_first:\n", - " instruction = line[\"instruction\"].strip()\n", - " assert instruction != \"\"\n", - " input = f\"{context}\\n\\n{instruction}\"\n", - " output = line[\"response\"]\n", - " else:\n", - " instruction = line[\"instruction\"].strip()\n", - " assert instruction != \"\"\n", - " input = f\"{instruction}\\n\\n{context}\"\n", - " output = line[\"response\"]\n", - " else:\n", - " input = line[\"instruction\"]\n", - " output = line[\"response\"]\n", - "\n", - " # Write JSONL line in new format\n", - " g.write(\n", - " json.dumps(\n", - " {\"input\": input, \"output\": output, \"category\": line[\"category\"]}\n", - " )\n", - " + \"\\n\"\n", - " )" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "Now, the dataset is a `JSONL` file with each line formatted like this: " - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "vscode": { - "languageId": "plaintext" - } - }, - "outputs": [], - "source": [ - "{\n", - " \"input\": \"Virgin Australia, the trading name of Virgin Australia Airlines Pty Ltd, is an Australian-based airline. It is the largest airline by fleet size to use the Virgin brand. It commenced services on 31 August 2000 as Virgin Blue, with two aircraft on a single route. It suddenly found itself as a major airline in Australia's domestic market after the collapse of Ansett Australia in September 2001. The airline has since grown to directly serve 32 cities in Australia, from hubs in Brisbane, Melbourne and Sydney.\\n\\nWhen did Virgin Australia start operating?\",\n", - " \"output\": \"Virgin Australia commenced services on 31 August 2000 as Virgin Blue, with two aircraft on a single route.\",\n", - " \"category\": \"closed_qa\"\n", - "}" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## SFT Training\n", - "\n", - "To perform the SFT Training, we'll use NVIDIA NeMo-Aligner. NeMo-Aligner is a scalable toolkit for efficient model alignment, built using the [NeMo Toolkit](https://github.com/NVIDIA/NeMo) which allows for scaling training up to 1000s of GPUs using tensor, data and pipeline parallelism for all components of alignment. Users can do end-to-end model alignment on a wide range of model sizes and take advantage of all the parallelism techniques to ensure their model alignment is done in a performant and resource efficient manner.\n", - "\n", - "To install NeMo Aligner, we can clone the repository and install it using `pip`:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "%%bash\n", - "\n", - "git clone https://github.com/NVIDIA/NeMo-Aligner.git -b dev\n", - "cd NeMo-Aligner\n", - "pip install -e ." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "If you want to track and visualize your SFT training experiments, you can login to Weights and Biases. If you don't want to use wandb, make sure to set the argument `exp_manager.create_wandb_logger=False` when launching your job." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "import wandb\n", - "wandb.login()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "To run SFT locally on a single node, you can use the following command. Note the `trainer.num_nodes` and `trainer.devices` arguments, which define how many nodes and how many total GPUs you want to use for training. Make sure the source model, output model, and dataset paths all match your local setup.\n", - "\n", - "If you'd like to perform multi-node finetuning -- for example on a slurm cluster -- you can find more information in the [NeMo-Aligner user guide](https://docs.nvidia.com/nemo-framework/user-guide/latest/modelalignment/rlhf.html#instruction-following-taught-by-supervised-fine-tuning-sft)." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "\n", - "%%bash\n", - "\n", - "cd NeMo-Aligner\n", - "\n", - "python examples/nlp/gpt/train_gpt_sft.py \\\n", - " name=gemma_dolly_finetuned \\\n", - " trainer.precision=bf16 \\\n", - " trainer.num_nodes=1 \\\n", - " trainer.devices=8 \\\n", - " trainer.sft.max_steps=-1 \\\n", - " trainer.sft.limit_val_batches=40 \\\n", - " trainer.sft.val_check_interval=1000 \\\n", - " model.tensor_model_parallel_size=4 \\\n", - " model.pipeline_model_parallel_size=1 \\\n", - " model.megatron_amp_O2=True \\\n", - " model.restore_from_path=../gemma_7b_pt.nemo \\\n", - " model.optim.lr=5e-6 \\\n", - " model.answer_only_loss=True \\\n", - " ++model.bias_activation_fusion=true \\\n", - " model.data.num_workers=0 \\\n", - " model.data.train_ds.micro_batch_size=1 \\\n", - " model.data.train_ds.global_batch_size=128 \\\n", - " model.data.train_ds.file_path=../databricks-dolly-15k-output.jsonl \\\n", - " model.data.train_ds.add_bos=True \\\n", - " model.data.validation_ds.micro_batch_size=1 \\\n", - " model.data.validation_ds.global_batch_size=128 \\\n", - " model.data.validation_ds.drop_last=True \\\n", - " model.data.validation_ds.file_path=../databricks-dolly-15k-output.jsonl \\\n", - " exp_manager.create_wandb_logger=True \\\n", - " exp_manager.explicit_log_dir=../results \\\n", - " exp_manager.wandb_logger_kwargs.project=sft_run \\\n", - " exp_manager.wandb_logger_kwargs.name=dolly_sft_run \\\n", - " exp_manager.checkpoint_callback_params.save_nemo_on_train_end=True \\\n", - " exp_manager.resume_if_exists=True \\\n", - " exp_manager.resume_ignore_no_checkpoint=True \\\n", - " exp_manager.create_checkpoint_callback=True \\\n", - " exp_manager.checkpoint_callback_params.monitor=validation_loss" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "When training is finished, you should see a file called `results/checkpoints/gemma_dolly_finetuned.nemo` that contains the weights of your new, instruction-tuned model." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Exporting to TensorRT-LLM" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "TensorRT-LLM is an open-source library for optimizing inference performance to achieve state-of-the-art speed on NVDIA GPUs. The NeMo framework offers an easy way to compile `.nemo` models into optimized TensorRT-LLM engines which you can run locally embedded in another application, or serve to other applications using a server like Triton Inference Server." - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "To start with, lets create a folder where our exported model will land" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "!mkdir gemma_trt_llm" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "To export the model, we just need to create an instance of the `TensorRTLLM` class and call the `TensorRTLLM.export()` function -- pointing the `nemo_checkpoint_path` argument to the newly fine-tuned model we trained above.\n", - "\n", - "This creates a couple of files in the folder we created -- an `engine` file that holds the weights and the compiled execution graph of the model, a `tokenizer.model` file which holds the tokenizer information, and `config.json` which holds some metadata about the model (along with `model.cache`, which caches some operations and makes it faster to re-compile the model in the future.)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from nemo.export import TensorRTLLM\n", - "trt_llm_exporter = TensorRTLLM(model_dir=\"gemma_dolly_finetuned_trt_llm\")\n", - "trt_llm_exporter.export(nemo_checkpoint_path=\"results/checkpoints/gemma_dolly_finetuned.nemo\", model_type=\"gemma\", n_gpus=1)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "\n", - "With the model exported into TensorRTLLM, we can perform very fast inference" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "trt_llm_exporter.forward([\"NVIDIA and Google are\"])" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "There's also a convenient function to deploy a the model as a service, backed by Triton Inference Server:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "from nemo.deploy import DeployPyTriton\n", - "\n", - "nm = DeployPyTriton(model=trt_llm_exporter, triton_model_name=\"gemma\")\n", - "nm.deploy()\n", - "nm.serve()" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": "nemo_lora", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.10.12" - } - }, - "nbformat": 4, - "nbformat_minor": 2 -} diff --git a/partner-quickstarts/README.md b/partner-quickstarts/README.md deleted file mode 100644 index ffc9fc6..0000000 --- a/partner-quickstarts/README.md +++ /dev/null @@ -1,10 +0,0 @@ -# Partner quickstart guides - -This folder contains quickstart tutorials mirrored from Gemma partner websites: - -| Company | Description | -| ------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Hugging Face | [Utilize Hugging Face Transformers and TRL for fine-tuning and inference tasks with Gemma models.](gemma-huggingface.md) | -| NVIDIA | [Fine-tune Gemma models with NVIDIA NeMo Framework and export to TensorRT-LLM for production.](Gemma-NVidia/) | -| LangChain | This [tutorial](gemma-langchain.ipynb) shows you how to get started with Gemma and LangChain, running in Google Cloud or in your Colab environment. | -| MongoDB | This [article](rag_with_hugging_face_gemma_mongodb.ipynb) presents how to leverage Gemma as the foundation model in a retrieval-augmented generation pipeline or system. | diff --git a/partner-quickstarts/gemma-huggingface.md b/partner-quickstarts/gemma-huggingface.md deleted file mode 100644 index c48a273..0000000 --- a/partner-quickstarts/gemma-huggingface.md +++ /dev/null @@ -1,293 +0,0 @@ ---- -title: "Welcome Gemma - Google’s new open LLM" -thumbnail: /blog/assets/gemma/thumbnail.jpg -authors: -- user: philschmid -- user: osanseviero -- user: pcuenq ---- - -# Welcome Gemma - Google’s new open LLM - -> [!NOTE] An update to the Gemma models was released two months after this post, see the latest versions [in this collection](https://huggingface.co/collections/google/gemma-release-65d5efbccdbb8c4202ec078b). - -Gemma, a new family of state-of-the-art open LLMs, was released today by Google! It's great to see Google reinforcing its commitment to open-source AI, and we’re excited to fully support the launch with comprehensive integration in Hugging Face. - -Gemma comes in two sizes: 7B parameters, for efficient deployment and development on consumer-size GPU and TPU and 2B versions for CPU and on-device applications. Both come in base and instruction-tuned variants. - -We’ve collaborated with Google to ensure the best integration into the Hugging Face ecosystem. You can find the 4 open-access models (2 base models & 2 fine-tuned ones) on the Hub. Among the features and integrations being released, we have: - -- [Models on the Hub](https://huggingface.co/models?search=google/gemma), with their model cards and licenses -- [🤗 Transformers integration](https://github.com/huggingface/transformers/releases/tag/v4.38.0) -- Integration with Google Cloud -- Integration with Inference Endpoints -- An example of fine-tuning Gemma on a single GPU with 🤗 TRL - -## Table of contents - -- [What is Gemma?](#what-is-gemma) - - [Prompt format](#prompt-format) - - [Exploring the Unknowns](#exploring-the-unknowns) -- [Demo](#demo) - - [Using 🤗 Transformers](#using-🤗-transformers) - - [JAX Weights](#jax-weights) -- [Integration with Google Cloud](#integration-with-google-cloud) -- [Integration with Inference Endpoints](#integration-with-inference-endpoints) -- [Fine-tuning with 🤗 TRL](#fine-tuning-with-🤗-trl) -- [Additional Resources](#additional-resources) -- [Acknowledgments](#acknowledgments) - - -## What is Gemma? - -Gemma is a family of 4 new LLM models by Google based on Gemini. It comes in two sizes: 2B and 7B parameters, each with base (pretrained) and instruction-tuned versions. All the variants can be run on various types of consumer hardware, even without quantization, and have a context length of 8K tokens: - -- [gemma-7b](https://huggingface.co/google/gemma-7b): Base 7B model. -- [gemma-7b-it](https://huggingface.co/google/gemma-7b-it): Instruction fine-tuned version of the base 7B model. -- [gemma-2b](https://huggingface.co/google/gemma-2b): Base 2B model. -- [gemma-2b-it](https://huggingface.co/google/gemma-2b-it): Instruction fine-tuned version of the base 2B model. - -A month after the original release, Google released a new version of the instruct models. This version has better coding capabilities, factuality, instruction following and multi-turn quality. The model also is less prone to begin its with "Sure,". - -- [gemma-1.1-7b-it](https://huggingface.co/google/gemma-1.1-7b-it) -- [gemma-1.1-2b-it](https://huggingface.co/google/gemma-1.1-2b-it) - -
-Gemma logo -
- -So, how good are the Gemma models? Here’s an overview of the base models and their performance compared to other open models on the [LLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) (higher scores are better): - -| Model | License | Commercial use? | Pretraining size [tokens] | Leaderboard score ⬇️ | -| -------------------------------------------------------------------------------- | --------------- | --------------- | ------------------------- | -------------------- | -| [LLama 2 70B Chat (reference)](https://huggingface.co/meta-llama/Llama-2-70b-hf) | Llama 2 license | ✅ | 2T | 67.87 | -| [Gemma-7B](https://huggingface.co/google/gemma-7b) | Gemma license | ✅ | 6T | 63.75 | -| [DeciLM-7B](https://huggingface.co/Deci/DeciLM-7B) | Apache 2.0 | ✅ | unknown | 61.55 | -| [PHI-2 (2.7B)](https://huggingface.co/microsoft/phi-2) | MIT | ✅ | 1.4T | 61.33 | -| [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) | Apache 2.0 | ✅ | unknown | 60.97 | -| [Llama 2 7B](https://huggingface.co/meta-llama/Llama-2-7b-hf) | Llama 2 license | ✅ | 2T | 54.32 | -| [Gemma 2B](https://huggingface.co/google/gemma-2b) | Gemma license | ✅ | 2T | 46.51 | - - -Gemma 7B is a really strong model, with performance comparable to the best models in the 7B weight, including Mistral 7B. Gemma 2B is an interesting model for its size, but it doesn’t score as high in the leaderboard as the best capable models with a similar size, such as Phi 2. We are looking forward to receiving feedback from the community about real-world usage! - -Recall that the LLM Leaderboard is especially useful for measuring the quality of pretrained models and not so much of the chat ones. We encourage running other benchmarks such as MT Bench, EQ Bench, and the lmsys Arena for the Chat ones! - -### Prompt format - -The base models have no prompt format. Like other base models, they can be used to continue an input sequence with a plausible continuation or for zero-shot/few-shot inference. They are also a great foundation for fine-tuning on your own use cases. The Instruct versions have a very simple conversation structure: - -```xml -user -knock knock -model -who is there -user -LaMDA -model -LaMDA who? -``` - -This format has to be exactly reproduced for effective use. We’ll later show how easy it is to reproduce the instruct prompt with the chat template available in `transformers`. - -### Exploring the Unknowns - -The Technical report includes information about the training and evaluation processes of the base models, but there are no extensive details on the dataset’s composition and preprocessing. We know they were trained with data from various sources, mostly web documents, code, and mathematical texts. The data was filtered to remove CSAM content and PII as well as licensing checks. - -Similarly, for the Gemma instruct models, no details have been shared about the fine-tuning datasets or the hyperparameters associated with SFT and [RLHF](https://huggingface.co/blog/rlhf). - -## Demo - -You can chat with the Gemma Instruct model on Hugging Chat! Check out the link here: https://huggingface.co/chat/models/google/gemma-1.1-7b-it - -### Using 🤗 Transformers - -With Transformers [release 4.38](https://github.com/huggingface/transformers/releases/tag/v4.38.0), you can use Gemma and leverage all the tools within the Hugging Face ecosystem, such as: - -- training and inference scripts and examples -- safe file format (`safetensors`) -- integrations with tools such as bitsandbytes (4-bit quantization), PEFT (parameter efficient fine-tuning), and Flash Attention 2 -- utilities and helpers to run generation with the model -- mechanisms to export the models to deploy - -In addition, Gemma models are compatible with `torch.compile()` with CUDA graphs, giving them a ~4x speedup at inference time! - -To use Gemma models with transformers, make sure to install a recent version of `transformers`: - -```jsx -pip install --upgrade transformers -``` - -The following snippet shows how to use `gemma-7b-it` with transformers. It requires about 18 GB of RAM, which includes consumer GPUs such as 3090 or 4090. - -```python -from transformers import pipeline -import torch - -pipe = pipeline( - "text-generation", - model="google/gemma-7b-it", - model_kwargs={"torch_dtype": torch.bfloat16}, - device="cuda", -) - -messages = [ - {"role": "user", "content": "Who are you? Please, answer in pirate-speak."}, -] -outputs = pipe( - messages, - max_new_tokens=256, - do_sample=True, - temperature=0.7, - top_k=50, - top_p=0.95 -) -assistant_response = outputs[0]["generated_text"][-1]["content"] -print(assistant_response) -``` - -> `Avast me, me hearty. I am a pirate of the high seas, ready to pillage and plunder. Prepare for a tale of adventure and booty!` -> - -We used `bfloat16` because that’s the reference precision and how all evaluations were run. Running in `float16` may be faster on your hardware. - -You can also automatically quantize the model, loading it in 8-bit or even 4-bit mode. 4-bit loading takes about 9 GB of memory to run, making it compatible with a lot of consumer cards and all the GPUs in Google Colab. This is how you’d load the generation pipeline in 4-bit: - -```jsx -pipeline = pipeline( - "text-generation", - model=model, - model_kwargs={ - "torch_dtype": torch.float16, - "quantization_config": {"load_in_4bit": True} - }, -) -``` - -For more details on using the models with transformers, please check [the model cards](https://huggingface.co/gg-hf/gemma-7b). - -### JAX Weights - -All the Gemma model variants are available for use with PyTorch, as explained above, or JAX / Flax. To load Flax weights, you need to use the `flax` revision from the repo, as shown below: - -```python -import jax.numpy as jnp -from transformers import AutoTokenizer, FlaxGemmaForCausalLM - -model_id = "google/gemma-2b" - -tokenizer = AutoTokenizer.from_pretrained(model_id) -tokenizer.padding_side = "left" - -model, params = FlaxGemmaForCausalLM.from_pretrained( - model_id, - dtype=jnp.bfloat16, - revision="flax", - _do_init=False, -) - -inputs = tokenizer("Valencia and Málaga are", return_tensors="np", padding=True) -output = model.generate(**inputs, params=params, max_new_tokens=20, do_sample=False) -output_text = tokenizer.batch_decode(output.sequences, skip_special_tokens=True) -``` - -> `['Valencia and Málaga are two of the most popular tourist destinations in Spain. Both cities boast a rich history, vibrant culture,']` -> - -Please, [check out this notebook](https://colab.research.google.com/github/sanchit-gandhi/notebooks/blob/main/jax_gemma.ipynb) for a comprehensive hands-on walkthrough on how to parallelize JAX inference on Colab TPUs! - -## Integration with Google Cloud - -You can deploy and train Gemma on Google Cloud through Vertex AI or Google Kubernetes Engine (GKE), using [Text Generation Inference](https://huggingface.co/docs/text-generation-inference/index) and Transformers. - -To deploy the Gemma model from Hugging Face, go to the [model page](https://huggingface.co/google/gemma-7b-it) and click on [Deploy -> Google Cloud.](https://huggingface.co/google/gemma-7b-it) This will bring you to the Google Cloud Console, where you can 1-click deploy Gemma on Vertex AI or GKE. Text Generation Inference powers Gemma on Google Cloud and is the first integration as part of our [partnership with Google Cloud.](https://huggingface.co/blog/gcp-partnership) - -![deploy on GCP](/blog/assets/gemma/gcp-deploy.png) - -You can also access Gemma directly through the Vertex AI Model Garden. - - -To Tune the Gemma model from Hugging Face, go to the [model page](https://huggingface.co/google/gemma-7b-it) and click on [Train -> Google Cloud.](https://huggingface.co/google/gemma-7b-it) This will bring you to the Google Cloud Console, where you can access notebooks to tune Gemma on Vertex AI or GKE. - -![train on GCP](/blog/assets/gemma/gcp-train-gemma.png) - -These integrations mark the first offerings we are launching together as a [result of our collaborative partnership with Google.](https://huggingface.co/blog/gcp-partnership) Stay tuned for more! - -## Integration with Inference Endpoints - -You can deploy Gemma on Hugging Face's [Inference Endpoints](https://ui.endpoints.huggingface.co/new?repository=google%2Fgemma-7b-it), which uses Text Generation Inference as the backend. [Text Generation Inference](https://github.com/huggingface/text-generation-inference) is a production-ready inference container developed by Hugging Face to enable easy deployment of large language models. It has features such as continuous batching, token streaming, tensor parallelism for fast inference on multiple GPUs, and production-ready logging and tracing. - -To deploy a Gemma model, go to the [model page](https://huggingface.co/google/gemma-7b-it) and click on the [Deploy -> Inference Endpoints](https://ui.endpoints.huggingface.co/new?repository=google/gemma-7b-it) widget. You can learn more about [Deploying LLMs with Hugging Face Inference Endpoints](https://huggingface.co/blog/inference-endpoints-llm) in a previous blog post. Inference Endpoints supports [Messages API](https://huggingface.co/blog/tgi-messages-api) through Text Generation Inference, which allows you to switch from another closed model to an open one by simply changing the URL. - -```bash -from openai import OpenAI - -# initialize the client but point it to TGI -client = OpenAI( - base_url="" + "/v1/", # replace with your endpoint url - api_key="", # replace with your token -) -chat_completion = client.chat.completions.create( - model="tgi", - messages=[ - {"role": "user", "content": "Why is open-source software important?"}, - ], - stream=True, - max_tokens=500 -) - -# iterate and print stream -for message in chat_completion: - print(message.choices[0].delta.content, end="") -``` - -## Fine-tuning with 🤗 TRL - -Training LLMs can be technically and computationally challenging. In this section, we’ll look at the tools available in the Hugging Face ecosystem to efficiently train Gemma on consumer-size GPUs - -An example command to fine-tune Gemma on OpenAssistant’s [chat dataset](https://huggingface.co/datasets/OpenAssistant/oasst_top1_2023-08-25) can be found below. We use 4-bit quantization and [QLoRA](https://arxiv.org/abs/2305.14314) to conserve memory to target all the attention blocks' linear layers. - -First, install the nightly version of 🤗 TRL and clone the repo to access the [training script](https://github.com/huggingface/trl/blob/main/examples/scripts/sft.py): - -```jsx -pip install -U transformers trl peft bitsandbytes -git clone https://github.com/huggingface/trl -cd trl -``` - -Then you can run the script: - -```jsx -accelerate launch --config_file examples/accelerate_configs/multi_gpu.yaml --num_processes=1 \ - examples/scripts/sft.py \ - --model_name google/gemma-7b \ - --dataset_name OpenAssistant/oasst_top1_2023-08-25 \ - --per_device_train_batch_size 2 \ - --gradient_accumulation_steps 1 \ - --learning_rate 2e-4 \ - --save_steps 20_000 \ - --use_peft \ - --lora_r 16 --lora_alpha 32 \ - --lora_target_modules q_proj k_proj v_proj o_proj \ - --load_in_4bit \ - --output_dir gemma-finetuned-openassistant -``` - -This takes about 9 hours to train on a single A10G, but can be easily parallelized by tweaking `--num_processes` to the number of GPUs you have available. - -## Additional Resources - -- [Models on the Hub](https://huggingface.co/models?other=gemma) -- Open LLM [Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) -- [Chat demo on Hugging Chat](https://huggingface.co/chat?model=google/gemma-7b-it) -- [Official Gemma Blog](https://blog.google/technology/developers/gemma-open-models/) -- [Gemma Product Page](https://ai.google.dev/gemma) -- [Vertex AI model garden link](https://console.cloud.google.com/vertex-ai/publishers/google/model-garden/335) -- Google Notebook - - -## Acknowledgments - -Releasing such models with support and evaluations in the ecosystem would not be possible without the contributions of many community members, including [Clémentine](https://huggingface.co/clefourrier) and [Eleuther Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness) for LLM evaluations; [Olivier](https://huggingface.co/olivierdehaene) and [David](https://huggingface.co/drbh) for Text Generation Inference Support; [Simon](https://huggingface.co/sbrandeis) for developing the new access control features on Hugging Face; [Arthur](https://huggingface.co/ArthurZ), [Younes](https://huggingface.co/ybelkada), and [Sanchit](https://huggingface.co/sanchit-gandhi) for integrating Gemma into transformers; [Morgan](https://huggingface.co/mfuntowicz) for integrating Gemma into optimum-nvidia (coming); [Nathan](https://huggingface.co/nsarrazin), [Victor](https://huggingface.co/victor), and [Mishig](https://huggingface.co/mishig) for making Gemma available in Hugging Chat. - -And Thank you to the Google Team for releasing Gemma and making it available to the open-source AI community! diff --git a/partner-quickstarts/gemma-langchain.ipynb b/partner-quickstarts/gemma-langchain.ipynb deleted file mode 100644 index 7d4cf9f..0000000 --- a/partner-quickstarts/gemma-langchain.ipynb +++ /dev/null @@ -1,1030 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": { - "id": "G3MMAcssHTML" - }, - "source": [ - "\n", - "" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "Tce3stUlHN0L" - }, - "source": [ - "##### Copyright 2024 Google LLC." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "cellView": "form", - "id": "tuOe1ymfHZPu" - }, - "outputs": [], - "source": [ - "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", - "# you may not use this file except in compliance with the License.\n", - "# You may obtain a copy of the License at\n", - "#\n", - "# https://www.apache.org/licenses/LICENSE-2.0\n", - "#\n", - "# Unless required by applicable law or agreed to in writing, software\n", - "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", - "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", - "# See the License for the specific language governing permissions and\n", - "# limitations under the License." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "c5b07d48d458" - }, - "source": [ - "\n", - " \n", - " \n", - " \n", - "
\n", - " View on ai.google.dev\n", - " \n", - " Run in Google Colab\n", - " \n", - " View source on GitHub\n", - "
" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "3acc8f3d1408" - }, - "source": [ - "# Get started with Gemma and LangChain\n", - "\n", - "This tutorial shows you how to get started with [Gemma](https://ai.google.dev/gemma/docs) and [LangChain](https://python.langchain.com/docs/get_started/introduction), running in Google Cloud or in your Colab environment. Gemma is a family of light-weight, state-of-the-art open models built from the same research and technology used to create the Gemini models. LangChain is a framework for building and deploying context-aware applications backed by language models.\n", - "\n", - "**Note:** This tutorial runs on A100 GPU in Google Colab. Free Colab hardware acceleration is *insufficient* to run all the code." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "88TpHe7pl0sa" - }, - "source": [ - "## Run Gemma in Google Cloud\n", - "\n", - "The [`langchain-google-vertexai`](https://pypi.org/project/langchain-google-vertexai/) package provides LangChain integration with Google Cloud models." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "2IxjMb9-jIJ8" - }, - "source": [ - "### Install dependencies" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "XZaTsXfcheTF" - }, - "outputs": [], - "source": [ - "!pip install --upgrade -q langchain langchain-google-vertexai" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "IyY5LtlbBVt5" - }, - "source": [ - "### Authenticate\n", - "\n", - "Unless you're using Colab Enterprise, you need to authenticate.\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "QO-Rr0WlBX73" - }, - "outputs": [], - "source": [ - "from google.colab import auth\n", - "auth.authenticate_user()" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "IXmAujvC3Kwp" - }, - "source": [ - "### Deploy the model\n", - "\n", - "Vertex AI is a platform for training and deploying AI models and applications. Model Garden is a curated collection of models that you can explore in the Google Cloud console.\n", - "\n", - "To deploy Gemma, [open the model](https://console.cloud.google.com/vertex-ai/publishers/google/model-garden/335) in Model Garden for Vertex AI and complete the following steps:\n", - "\n", - "1. Select **Deploy**.\n", - "2. Make any desired changes to the deployment form fields, or leave them as\n", - " is, if you're okay with the defaults. Make note of the following fields, which you'll need later:\n", - " * **Endpoint name** (for example, `google_gemma-7b-it-mg-one-click-deploy`)\n", - " * **Region** (for example, `us-west1`)\n", - "3. Select **Deploy** to deploy the model to Vertex AI. The deployment will\n", - " take a few minutes to complete.\n", - "\n", - "When the endpoint is ready, copy its project ID, endpoint ID, and location, and enter them as parameters." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "gv1j8FrVftsC" - }, - "outputs": [], - "source": [ - "# @title Basic parameters\n", - "project: str = \"\" # @param {type:\"string\"}\n", - "endpoint_id: str = \"\" # @param {type:\"string\"}\n", - "location: str = \"\" # @param {type:\"string\"}" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "a8DB3i9sO22M" - }, - "source": [ - "### Run the model" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "bhIHsFGYjtFt", - "outputId": "478dbf53-4115-4618-83ac-df1dead93cf0" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Prompt:\n", - "What is the meaning of life?\n", - "Output:\n", - "Life is a complex and multifaceted phenomenon that has fascinated philosophers, scientists, and\n" - ] - } - ], - "source": [ - "from langchain_google_vertexai import GemmaVertexAIModelGarden, GemmaChatVertexAIModelGarden\n", - "\n", - "llm = GemmaVertexAIModelGarden(\n", - " endpoint_id=endpoint_id,\n", - " project=project,\n", - " location=location,\n", - ")\n", - "\n", - "output = llm.invoke(\"What is the meaning of life?\")\n", - "print(output)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "zzep9nfmuUcO" - }, - "source": [ - "You can also use Gemma for multi-turn chat:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "8tPHoM5XiZOl", - "outputId": "d163b57f-4743-4907-99ce-bc328925c701" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "content='Prompt:\\nuser\\nHow much is 2+2?\\nmodel\\nOutput:\\nSure, the answer is 4.\\n\\n2 + 2 = 4'\n", - "content='Prompt:\\nuser\\nHow much is 2+2?\\nmodel\\nPrompt:\\nuser\\nHow much is 2+2?\\nmodel\\nOutput:\\nSure, the answer is 4.\\n\\n2 + 2 = 4\\nuser\\nHow much is 3+3?\\nmodel\\nOutput:\\nSure, the answer is 6.\\n\\n3 + 3 = 6'\n" - ] - } - ], - "source": [ - "from langchain_core.messages import (\n", - " HumanMessage\n", - ")\n", - "\n", - "llm = GemmaChatVertexAIModelGarden(\n", - " endpoint_id=endpoint_id,\n", - " project=project,\n", - " location=location,\n", - ")\n", - "\n", - "message1 = HumanMessage(content=\"How much is 2+2?\")\n", - "answer1 = llm.invoke([message1])\n", - "print(answer1)\n", - "\n", - "message2 = HumanMessage(content=\"How much is 3+3?\")\n", - "answer2 = llm.invoke([message1, answer1, message2])\n", - "\n", - "print(answer2)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "AZL6d_ZvoI-z" - }, - "source": [ - "You can post-process responses to avoid repetitions:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "qXGgKAFxoI-z", - "outputId": "95129f8b-9588-461b-81a1-31ae609fb19d" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "content='Output:\\nSure, here is the answer:\\n\\n2 + 2 = 4'\n", - "content='Output:\\nSure, here is the answer:\\n\\n3 + 3 = 6<'\n" - ] - } - ], - "source": [ - "answer1 = llm.invoke([message1], parse_response=True)\n", - "print(answer1)\n", - "\n", - "answer2 = llm.invoke([message1, answer1, message2], parse_response=True)\n", - "\n", - "print(answer2)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "VEfjqo7fjARR" - }, - "source": [ - "## Run Gemma from a Kaggle download" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "gVW8QDzHu7TA" - }, - "source": [ - "This section shows you how to download Gemma from Kaggle and then run the model.\n", - "\n", - "To complete this section, you'll first need to complete the setup instructions at [Gemma setup](https://ai.google.dev/gemma/docs/setup).\n", - "\n", - "Then move on to the next section, where you'll set environment variables for your Colab environment.\n", - "\n", - "**Note:** This section of the tutorial runs on A100 GPU in Google Colab." - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "MDYfZUoxF2LE" - }, - "source": [ - "### Set environment variables\n", - "\n", - "Set environment variables for `KAGGLE_USERNAME` and `KAGGLE_KEY`." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "BXvwshs1GEDo" - }, - "outputs": [], - "source": [ - "import os\n", - "from google.colab import userdata\n", - "\n", - "# Note: `userdata.get` is a Colab API. If you're not using Colab, set the env\n", - "# vars as appropriate for your system.\n", - "os.environ[\"KAGGLE_USERNAME\"] = userdata.get('KAGGLE_USERNAME')\n", - "os.environ[\"KAGGLE_KEY\"] = userdata.get('KAGGLE_KEY')" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "Ezq65fi9kvRN" - }, - "source": [ - "### Install dependencies" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "KrwQkHDzky9X" - }, - "outputs": [], - "source": [ - "# Install Keras 3 last. See https://keras.io/getting_started/ for more details.\n", - "!pip install -q -U keras-nlp\n", - "!pip install -q -U keras>=3" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "E9zn8nYpv3QZ" - }, - "source": [ - "### Run the model" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "0LFRmY8TjCkI" - }, - "outputs": [], - "source": [ - "from langchain_google_vertexai import GemmaLocalKaggle" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "v-o7oXVavdMQ" - }, - "source": [ - "You can specify the Keras backend (by default it's `tensorflow`, but you can change it to `jax` or `torch`)." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "vvTUH8DNj5SF" - }, - "outputs": [], - "source": [ - "# @title Basic parameters\n", - "keras_backend: str = \"jax\" # @param {type:\"string\"}\n", - "model_name: str = \"gemma_2b_en\" # @param {type:\"string\"}" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "YOmrqxo5kHXK", - "outputId": "65bfa681-3b98-4bfb-c657-008ec1e64565" - }, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "Attaching 'config.json' from model 'keras/gemma/keras/gemma_2b_en/2' to your Colab notebook...\n", - "Attaching 'config.json' from model 'keras/gemma/keras/gemma_2b_en/2' to your Colab notebook...\n", - "Attaching 'model.weights.h5' from model 'keras/gemma/keras/gemma_2b_en/2' to your Colab notebook...\n", - "Attaching 'tokenizer.json' from model 'keras/gemma/keras/gemma_2b_en/2' to your Colab notebook...\n", - "Attaching 'assets/tokenizer/vocabulary.spm' from model 'keras/gemma/keras/gemma_2b_en/2' to your Colab notebook...\n" - ] - } - ], - "source": [ - "llm = GemmaLocalKaggle(model_name=model_name, keras_backend=keras_backend)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "Zu6yPDUgkQtQ", - "outputId": "bf2a48d5-9380-4d9b-8092-b3959d81d1fd" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "What is the meaning of life?\n", - "\n", - "The question is one of the most important questions in the world.\n", - "\n", - "It’s the question that has\n" - ] - } - ], - "source": [ - "output = llm.invoke(\"What is the meaning of life?\", max_tokens=30)\n", - "print(output)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "z5VDsZkeoI-0" - }, - "source": [ - "### Run the chat model" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "MSctpRE4u43N" - }, - "source": [ - "As in the Google Cloud example above, you can use a local deployment of Gemma for multi-turn chat. You might need to re-start the notebook and clean your GPU memory in order to avoid OOM errors:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "nXFHaE0VoI-0" - }, - "outputs": [], - "source": [ - "from langchain_google_vertexai import GemmaChatLocalKaggle" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "6d-QHQNroI-0" - }, - "outputs": [], - "source": [ - "# @title Basic parameters\n", - "keras_backend: str = \"jax\" # @param {type:\"string\"}\n", - "model_name: str = \"gemma_2b_en\" # @param {type:\"string\"}" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "FA3DJIemoI-0", - "outputId": "abb5a624-0571-40ce-cd86-f583e13fc0a2" - }, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "Attaching 'config.json' from model 'keras/gemma/keras/gemma_2b_en/2' to your Colab notebook...\n", - "Attaching 'config.json' from model 'keras/gemma/keras/gemma_2b_en/2' to your Colab notebook...\n", - "Attaching 'model.weights.h5' from model 'keras/gemma/keras/gemma_2b_en/2' to your Colab notebook...\n", - "Attaching 'tokenizer.json' from model 'keras/gemma/keras/gemma_2b_en/2' to your Colab notebook...\n", - "Attaching 'assets/tokenizer/vocabulary.spm' from model 'keras/gemma/keras/gemma_2b_en/2' to your Colab notebook...\n" - ] - } - ], - "source": [ - "llm = GemmaChatLocalKaggle(model_name=model_name, keras_backend=keras_backend)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "JrJmvZqwwLqj", - "outputId": "9517e2e9-b9a5-4d75-b931-f41a19638a17" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "content=\"user\\nHi! Who are you?\\nmodel\\nI'm a model.\\n Tampoco\\nI'm a model.\"\n" - ] - } - ], - "source": [ - "from langchain_core.messages import (\n", - " HumanMessage\n", - ")\n", - "\n", - "message1 = HumanMessage(content=\"Hi! Who are you?\")\n", - "answer1 = llm.invoke([message1], max_tokens=30)\n", - "print(answer1)\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "NAmBDTpooI-1", - "outputId": "eccfd0cf-98c7-44f3-b2bc-d3d88f9fdd4e" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "content=\"user\\nHi! Who are you?\\nmodel\\nuser\\nHi! Who are you?\\nmodel\\nI'm a model.\\n Tampoco\\nI'm a model.\\nuser\\nWhat can you help me with?\\nmodel\"\n" - ] - } - ], - "source": [ - "message2 = HumanMessage(content=\"What can you help me with?\")\n", - "answer2 = llm.invoke([message1, answer1, message2], max_tokens=60)\n", - "\n", - "print(answer2)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "5MuhIDoxoI-1" - }, - "source": [ - "You can post-process the response if you want to avoid multi-turn statements:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "zl9J_6PHoI-1", - "outputId": "126b61af-33cb-41d0-888e-f875b5b5b59d" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "content=\"I'm a model.\\n Tampoco\\nI'm a model.\"\n", - "content='I can help you with your modeling.\\n Tampoco\\nI can'\n" - ] - } - ], - "source": [ - "answer1 = llm.invoke([message1], max_tokens=30, parse_response=True)\n", - "print(answer1)\n", - "\n", - "answer2 = llm.invoke([message1, answer1, message2], max_tokens=60, parse_response=True)\n", - "print(answer2)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "EiZnztso7hyF" - }, - "source": [ - "## Run Gemma from a Hugging Face download" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "QYgBxNQssA3U" - }, - "source": [ - "### Setup\n", - "\n", - "Like Kaggle, Hugging Face requires that you accept the Gemma terms and conditions before accessing the model. To get access to Gemma through Hugging Face, go to the [Gemma model card](https://huggingface.co/google/gemma-2b).\n", - "\n", - "You'll also need to get a [user access token](https://huggingface.co/docs/hub/en/security-tokens) with read permissions, which you can enter below.\n", - "\n", - "**Note:** This section of the tutorial runs on A100 GPU in Google Colab." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "tsyntzI08cOr" - }, - "outputs": [], - "source": [ - "# @title Basic parameters\n", - "hf_access_token: str = \"\" # @param {type:\"string\"}\n", - "model_name: str = \"google/gemma-2b\" # @param {type:\"string\"}" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "pyHNhGRasTaW" - }, - "source": [ - "### Run the model" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "qqAqsz5R7nKf" - }, - "outputs": [], - "source": [ - "from langchain_google_vertexai import GemmaLocalHF, GemmaChatLocalHF" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "JWrqEkOo8sm9", - "outputId": "57297e2d-ea17-488a-f596-60d34bf58e50", - "colab": { - "referenced_widgets": [ - "1e03a95d82d54cae82fd8f60347d0ba4", - "0c9dcdea22e14cd988ce5cd7515f9e0f", - "5306530028c34909b4370b9103710f13", - "e34a7afd64764999b9157eb8f4da4fe6", - "82fcda4f615f4ff08a235aaee0061f19", - "491a26f8bfe54b88a07e31bac3c49831", - "1801fbdfa9274c69ac4e21787609fd8c", - "ebe111457155452389394ede593962b5", - "9f440617c8a84af197d1ca1b5b0378e6", - "983d28c1dac444c9835c860255d81464", - "4f796fc0b33c48969410b1f7d0636762" - ] - } - }, - "outputs": [ - { - "data": { - "application/vnd.jupyter.widget-view+json": { - "model_id": "1e03a95d82d54cae82fd8f60347d0ba4", - "version_major": 2, - "version_minor": 0 - }, - "text/plain": [ - "tokenizer_config.json: 0%| | 0.00/1.11k [00:00user\\nHi! Who are you?\\nmodel\\nI'm a model.\\n\\nuser\\nWhat do you mean\"\n" - ] - } - ], - "source": [ - "from langchain_core.messages import (\n", - " HumanMessage\n", - ")\n", - "\n", - "message1 = HumanMessage(content=\"Hi! Who are you?\")\n", - "answer1 = llm.invoke([message1], max_tokens=60)\n", - "print(answer1)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "BDuLHGNmoI-7", - "outputId": "bae88949-c554-4970-85e6-d0c30e53b0f3" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "content=\"user\\nHi! Who are you?\\nmodel\\nuser\\nHi! Who are you?\\nmodel\\nI'm a model.\\n\\nuser\\nWhat do you mean\\nuser\\nWhat can you help me with?\\nmodel\\nI can help you with anything.\\n<\"\n" - ] - } - ], - "source": [ - "message2 = HumanMessage(content=\"What can you help me with?\")\n", - "answer2 = llm.invoke([message1, answer1, message2], max_tokens=140)\n", - "\n", - "print(answer2)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "_EAfKtj9oI-7" - }, - "source": [ - "As in the previous examples, you can post-process the response:" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "id": "IC-w52G9oI-7", - "outputId": "6c0e4716-2804-4711-93b4-f93c5dfbcdb1" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "content=\"I'm a model.\\n\\n\"\n", - "content='I can help you with anything.\\n\\n\\n'\n" - ] - } - ], - "source": [ - "answer1 = llm.invoke([message1], max_tokens=60, parse_response=True)\n", - "print(answer1)\n", - "\n", - "answer2 = llm.invoke([message1, answer1, message2], max_tokens=120, parse_response=True)\n", - "print(answer2)" - ] - }, - { - "cell_type": "markdown", - "metadata": { - "id": "s2tbOcVXs6Fa" - }, - "source": [ - "## What's next\n", - "\n", - "* Learn how to [finetune a Gemma model](https://ai.google.dev/gemma/docs/lora_tuning).\n", - "* Learn how to perform [distributed fine-tuning and inference on a Gemma model](https://ai.google.dev/gemma/docs/distributed_tuning).\n", - "* Learn how to [use Gemma models with Vertex AI](https://cloud.google.com/vertex-ai/docs/generative-ai/open-models/use-gemma)." - ] - } - ], - "metadata": { - "colab": { - "name": "langchain.ipynb", - "toc_visible": true, - "provenance": [] - }, - "kernelspec": { - "display_name": "Python 3", - "name": "python3" - } - }, - "nbformat": 4, - "nbformat_minor": 0 -} \ No newline at end of file diff --git a/partner-quickstarts/rag_with_hugging_face_gemma_mongodb.ipynb b/partner-quickstarts/rag_with_hugging_face_gemma_mongodb.ipynb deleted file mode 100644 index 9e57284..0000000 --- a/partner-quickstarts/rag_with_hugging_face_gemma_mongodb.ipynb +++ /dev/null @@ -1,3876 +0,0 @@ -{ - "cells": [ - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/mongodb-developer/GenAI-Showcase/blob/main/notebooks/rag/rag_with_hugging_face_gemma_mongodb.ipynb) \n", - "[![View Article](https://img.shields.io/badge/View%20Article-blue)](https://www.mongodb.com/developer/products/atlas/gemma-mongodb-huggingface-rag/)\n", - "\n" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "gVSo_nNOUsdn", - "outputId": "907f4738-a3b0-4c0f-b293-eff65c665c07" - }, - "outputs": [], - "source": [ - "!pip install datasets pandas pymongo sentence_transformers\n", - "!pip install -U transformers\n", - "# Install below if using GPU\n", - "!pip install accelerate" - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 747 - }, - "id": "5gCzss27UwWw", - "outputId": "212cca18-a0d7-4289-bce0-ee6259fc2dba" - }, - "outputs": [ - { - "data": { - "application/vnd.google.colaboratory.intrinsic+json": { - "summary": "{\n \"name\": \"dataset_df\",\n \"rows\": 1500,\n \"fields\": [\n {\n \"column\": \"num_mflix_comments\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 27,\n \"min\": 0,\n \"max\": 158,\n \"num_unique_values\": 40,\n \"samples\": [\n 117,\n 134,\n 124\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"genres\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"countries\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"directors\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"fullplot\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1409,\n \"samples\": [\n \"An undercover cop infiltrates a gang of thieves who plan to rob a jewelry store.\",\n \"Godzilla returns in a brand-new movie that ignores all preceding movies except for the original with a brand new look and a powered up atomic ray. This time he battles a mysterious UFO that later transforms into a mysterious kaiju dubbed Orga. They meet up for the final showdown in the city of Shinjuku.\",\n \"Relationships become entangled in an emotional web.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"writers\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"awards\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"runtime\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 42.09038552453906,\n \"min\": 6.0,\n \"max\": 1256.0,\n \"num_unique_values\": 139,\n \"samples\": [\n 152.0,\n 127.0,\n 96.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"type\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 2,\n \"samples\": [\n \"series\",\n \"movie\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"rated\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 12,\n \"samples\": [\n \"TV-MA\",\n \"TV-14\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"metacritic\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 16.861995960390892,\n \"min\": 9.0,\n \"max\": 97.0,\n \"num_unique_values\": 83,\n \"samples\": [\n 50.0,\n 97.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"poster\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1368,\n \"samples\": [\n \"https://m.media-amazon.com/images/M/MV5BNWE5MzAwMjQtNzI1YS00YjZhLTkxNDItM2JjNjM3ZjI5NzBjXkEyXkFqcGdeQXVyMTQxNzMzNDI@._V1_SY1000_SX677_AL_.jpg\",\n \"https://m.media-amazon.com/images/M/MV5BMTgwNjIyNTczMF5BMl5BanBnXkFtZTcwODI5MDkyMQ@@._V1_SY1000_SX677_AL_.jpg\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"languages\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"imdb\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"plot\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1429,\n \"samples\": [\n \"A New York City architect becomes a one-man vigilante squad after his wife is murdered by street punks in which he randomly goes out and kills would-be muggers on the mean streets after dark.\",\n \"As the daring thief Ars\\u00e8ne Lupin (Duris) ransacks the homes of wealthy Parisians, the police, with a secret weapon in their arsenal, attempt to ferret him out.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"cast\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"plot_embedding\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"title\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1435,\n \"samples\": [\n \"Turbo: A Power Rangers Movie\",\n \"Neon Genesis Evangelion: Death & Rebirth\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", - "type": "dataframe", - "variable_name": "dataset_df" - }, - "text/html": [ - "\n", - "
\n", - "
\n", - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
num_mflix_commentsgenrescountriesdirectorsfullplotwritersawardsruntimetyperatedmetacriticposterlanguagesimdbplotcastplot_embeddingtitle
00[Action][USA][Louis J. Gasnier, Donald MacKenzie]Young Pauline is left a lot of money when her ...[Charles W. Goddard (screenplay), Basil Dickey...{'nominations': 0, 'text': '1 win.', 'wins': 1}199.0movieNoneNaNhttps://m.media-amazon.com/images/M/MV5BMzgxOD...[English]{'id': 4465, 'rating': 7.6, 'votes': 744}Young Pauline is left a lot of money when her ...[Pearl White, Crane Wilbur, Paul Panzer, Edwar...[0.00072939653, -0.026834568, 0.013515796, -0....The Perils of Pauline
10[Comedy, Short, Action][USA][Alfred J. Goulding, Hal Roach]As a penniless man worries about how he will m...[H.M. Walker (titles)]{'nominations': 1, 'text': '1 nomination.', 'w...22.0movieTV-GNaNhttps://m.media-amazon.com/images/M/MV5BNzE1OW...[English]{'id': 10146, 'rating': 7.0, 'votes': 639}A penniless young man tries to save an heiress...[Harold Lloyd, Mildred Davis, 'Snub' Pollard, ...[-0.022837115, -0.022941574, 0.014937485, -0.0...From Hand to Mouth
20[Action, Adventure, Drama][USA][Herbert Brenon]Michael \"Beau\" Geste leaves England in disgrac...[Herbert Brenon (adaptation), John Russell (ad...{'nominations': 0, 'text': '1 win.', 'wins': 1}101.0movieNoneNaNNone[English]{'id': 16634, 'rating': 6.9, 'votes': 222}Michael \"Beau\" Geste leaves England in disgrac...[Ronald Colman, Neil Hamilton, Ralph Forbes, A...[0.00023330493, -0.028511643, 0.014653289, -0....Beau Geste
31[Adventure, Action][USA][Albert Parker]A nobleman vows to avenge the death of his fat...[Douglas Fairbanks (story), Jack Cunningham (a...{'nominations': 0, 'text': '1 win.', 'wins': 1}88.0movieNoneNaNhttps://m.media-amazon.com/images/M/MV5BMzU0ND...None{'id': 16654, 'rating': 7.2, 'votes': 1146}Seeking revenge, an athletic young man joins t...[Billie Dove, Tempe Pigott, Donald Crisp, Sam ...[-0.005927917, -0.033394486, 0.0015323418, -0....The Black Pirate
40[Action, Comedy, Romance][USA][Sam Taylor]The Uptown Boy, J. Harold Manners (Lloyd) is a...[Ted Wilde (story), John Grey (story), Clyde B...{'nominations': 1, 'text': '1 nomination.', 'w...58.0moviePASSEDNaNhttps://m.media-amazon.com/images/M/MV5BMTcxMT...[English]{'id': 16895, 'rating': 7.6, 'votes': 918}An irresponsible young millionaire changes his...[Harold Lloyd, Jobyna Ralston, Noah Young, Jim...[-0.0059373598, -0.026604708, -0.0070914757, -...For Heaven's Sake
\n", - "
\n", - "
\n", - "\n", - "
\n", - " \n", - "\n", - " \n", - "\n", - " \n", - "
\n", - "\n", - "\n", - "
\n", - " \n", - "\n", - "\n", - "\n", - " \n", - "
\n", - "
\n", - "
\n" - ], - "text/plain": [ - " num_mflix_comments genres countries \\\n", - "0 0 [Action] [USA] \n", - "1 0 [Comedy, Short, Action] [USA] \n", - "2 0 [Action, Adventure, Drama] [USA] \n", - "3 1 [Adventure, Action] [USA] \n", - "4 0 [Action, Comedy, Romance] [USA] \n", - "\n", - " directors \\\n", - "0 [Louis J. Gasnier, Donald MacKenzie] \n", - "1 [Alfred J. Goulding, Hal Roach] \n", - "2 [Herbert Brenon] \n", - "3 [Albert Parker] \n", - "4 [Sam Taylor] \n", - "\n", - " fullplot \\\n", - "0 Young Pauline is left a lot of money when her ... \n", - "1 As a penniless man worries about how he will m... \n", - "2 Michael \"Beau\" Geste leaves England in disgrac... \n", - "3 A nobleman vows to avenge the death of his fat... \n", - "4 The Uptown Boy, J. Harold Manners (Lloyd) is a... \n", - "\n", - " writers \\\n", - "0 [Charles W. Goddard (screenplay), Basil Dickey... \n", - "1 [H.M. Walker (titles)] \n", - "2 [Herbert Brenon (adaptation), John Russell (ad... \n", - "3 [Douglas Fairbanks (story), Jack Cunningham (a... \n", - "4 [Ted Wilde (story), John Grey (story), Clyde B... \n", - "\n", - " awards runtime type rated \\\n", - "0 {'nominations': 0, 'text': '1 win.', 'wins': 1} 199.0 movie None \n", - "1 {'nominations': 1, 'text': '1 nomination.', 'w... 22.0 movie TV-G \n", - "2 {'nominations': 0, 'text': '1 win.', 'wins': 1} 101.0 movie None \n", - "3 {'nominations': 0, 'text': '1 win.', 'wins': 1} 88.0 movie None \n", - "4 {'nominations': 1, 'text': '1 nomination.', 'w... 58.0 movie PASSED \n", - "\n", - " metacritic poster languages \\\n", - "0 NaN https://m.media-amazon.com/images/M/MV5BMzgxOD... [English] \n", - "1 NaN https://m.media-amazon.com/images/M/MV5BNzE1OW... [English] \n", - "2 NaN None [English] \n", - "3 NaN https://m.media-amazon.com/images/M/MV5BMzU0ND... None \n", - "4 NaN https://m.media-amazon.com/images/M/MV5BMTcxMT... [English] \n", - "\n", - " imdb \\\n", - "0 {'id': 4465, 'rating': 7.6, 'votes': 744} \n", - "1 {'id': 10146, 'rating': 7.0, 'votes': 639} \n", - "2 {'id': 16634, 'rating': 6.9, 'votes': 222} \n", - "3 {'id': 16654, 'rating': 7.2, 'votes': 1146} \n", - "4 {'id': 16895, 'rating': 7.6, 'votes': 918} \n", - "\n", - " plot \\\n", - "0 Young Pauline is left a lot of money when her ... \n", - "1 A penniless young man tries to save an heiress... \n", - "2 Michael \"Beau\" Geste leaves England in disgrac... \n", - "3 Seeking revenge, an athletic young man joins t... \n", - "4 An irresponsible young millionaire changes his... \n", - "\n", - " cast \\\n", - "0 [Pearl White, Crane Wilbur, Paul Panzer, Edwar... \n", - "1 [Harold Lloyd, Mildred Davis, 'Snub' Pollard, ... \n", - "2 [Ronald Colman, Neil Hamilton, Ralph Forbes, A... \n", - "3 [Billie Dove, Tempe Pigott, Donald Crisp, Sam ... \n", - "4 [Harold Lloyd, Jobyna Ralston, Noah Young, Jim... \n", - "\n", - " plot_embedding title \n", - "0 [0.00072939653, -0.026834568, 0.013515796, -0.... The Perils of Pauline \n", - "1 [-0.022837115, -0.022941574, 0.014937485, -0.0... From Hand to Mouth \n", - "2 [0.00023330493, -0.028511643, 0.014653289, -0.... Beau Geste \n", - "3 [-0.005927917, -0.033394486, 0.0015323418, -0.... The Black Pirate \n", - "4 [-0.0059373598, -0.026604708, -0.0070914757, -... For Heaven's Sake " - ] - }, - "execution_count": 2, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# Load Dataset\n", - "from datasets import load_dataset\n", - "import pandas as pd\n", - "\n", - "# https://huggingface.co/datasets/MongoDB/embedded_movies\n", - "dataset = load_dataset(\"MongoDB/embedded_movies\")\n", - "\n", - "# Convert the dataset to a pandas dataframe\n", - "dataset_df = pd.DataFrame(dataset[\"train\"])\n", - "\n", - "dataset_df.head(5)" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 1000 - }, - "id": "ARdz6j7SUxqi", - "outputId": "c53c458a-512d-4b7e-93b4-514f6de9d497" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "\n", - "Number of missing values in each column after removal:\n", - "num_mflix_comments 0\n", - "genres 0\n", - "countries 0\n", - "directors 12\n", - "fullplot 0\n", - "writers 13\n", - "awards 0\n", - "runtime 14\n", - "type 0\n", - "rated 279\n", - "metacritic 893\n", - "poster 78\n", - "languages 1\n", - "imdb 0\n", - "plot 0\n", - "cast 1\n", - "plot_embedding 1\n", - "title 0\n", - "dtype: int64\n" - ] - }, - { - "data": { - "application/vnd.google.colaboratory.intrinsic+json": { - "summary": "{\n \"name\": \"dataset_df\",\n \"rows\": 1452,\n \"fields\": [\n {\n \"column\": \"num_mflix_comments\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 27,\n \"min\": 0,\n \"max\": 158,\n \"num_unique_values\": 40,\n \"samples\": [\n 117,\n 134,\n 124\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"genres\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"countries\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"directors\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"fullplot\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1409,\n \"samples\": [\n \"An undercover cop infiltrates a gang of thieves who plan to rob a jewelry store.\",\n \"Godzilla returns in a brand-new movie that ignores all preceding movies except for the original with a brand new look and a powered up atomic ray. This time he battles a mysterious UFO that later transforms into a mysterious kaiju dubbed Orga. They meet up for the final showdown in the city of Shinjuku.\",\n \"Relationships become entangled in an emotional web.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"writers\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"awards\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"runtime\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 42.5693352357647,\n \"min\": 6.0,\n \"max\": 1256.0,\n \"num_unique_values\": 137,\n \"samples\": [\n 60.0,\n 151.0,\n 110.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"type\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 2,\n \"samples\": [\n \"series\",\n \"movie\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"rated\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 12,\n \"samples\": [\n \"TV-MA\",\n \"TV-14\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"metacritic\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 16.855402595666057,\n \"min\": 9.0,\n \"max\": 97.0,\n \"num_unique_values\": 83,\n \"samples\": [\n 50.0,\n 97.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"poster\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1332,\n \"samples\": [\n \"https://m.media-amazon.com/images/M/MV5BMTQ2NTMxODEyNV5BMl5BanBnXkFtZTcwMDgxMjA0MQ@@._V1_SY1000_SX677_AL_.jpg\",\n \"https://m.media-amazon.com/images/M/MV5BMTY5OTg1ODk0MV5BMl5BanBnXkFtZTcwMTEwMjU1MQ@@._V1_SY1000_SX677_AL_.jpg\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"languages\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"imdb\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"plot\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1409,\n \"samples\": [\n \"An undercover cop infiltrates a gang of thieves who plan to rob a jewelry store.\",\n \"Godzilla saves Tokyo from a flying saucer that transforms into the beast Orga.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"cast\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"title\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1391,\n \"samples\": [\n \"Superhero Movie\",\n \"Hooper\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", - "type": "dataframe", - "variable_name": "dataset_df" - }, - "text/html": [ - "\n", - "
\n", - "
\n", - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
num_mflix_commentsgenrescountriesdirectorsfullplotwritersawardsruntimetyperatedmetacriticposterlanguagesimdbplotcasttitle
00[Action][USA][Louis J. Gasnier, Donald MacKenzie]Young Pauline is left a lot of money when her ...[Charles W. Goddard (screenplay), Basil Dickey...{'nominations': 0, 'text': '1 win.', 'wins': 1}199.0movieNoneNaNhttps://m.media-amazon.com/images/M/MV5BMzgxOD...[English]{'id': 4465, 'rating': 7.6, 'votes': 744}Young Pauline is left a lot of money when her ...[Pearl White, Crane Wilbur, Paul Panzer, Edwar...The Perils of Pauline
10[Comedy, Short, Action][USA][Alfred J. Goulding, Hal Roach]As a penniless man worries about how he will m...[H.M. Walker (titles)]{'nominations': 1, 'text': '1 nomination.', 'w...22.0movieTV-GNaNhttps://m.media-amazon.com/images/M/MV5BNzE1OW...[English]{'id': 10146, 'rating': 7.0, 'votes': 639}A penniless young man tries to save an heiress...[Harold Lloyd, Mildred Davis, 'Snub' Pollard, ...From Hand to Mouth
20[Action, Adventure, Drama][USA][Herbert Brenon]Michael \"Beau\" Geste leaves England in disgrac...[Herbert Brenon (adaptation), John Russell (ad...{'nominations': 0, 'text': '1 win.', 'wins': 1}101.0movieNoneNaNNone[English]{'id': 16634, 'rating': 6.9, 'votes': 222}Michael \"Beau\" Geste leaves England in disgrac...[Ronald Colman, Neil Hamilton, Ralph Forbes, A...Beau Geste
31[Adventure, Action][USA][Albert Parker]A nobleman vows to avenge the death of his fat...[Douglas Fairbanks (story), Jack Cunningham (a...{'nominations': 0, 'text': '1 win.', 'wins': 1}88.0movieNoneNaNhttps://m.media-amazon.com/images/M/MV5BMzU0ND...None{'id': 16654, 'rating': 7.2, 'votes': 1146}Seeking revenge, an athletic young man joins t...[Billie Dove, Tempe Pigott, Donald Crisp, Sam ...The Black Pirate
40[Action, Comedy, Romance][USA][Sam Taylor]The Uptown Boy, J. Harold Manners (Lloyd) is a...[Ted Wilde (story), John Grey (story), Clyde B...{'nominations': 1, 'text': '1 nomination.', 'w...58.0moviePASSEDNaNhttps://m.media-amazon.com/images/M/MV5BMTcxMT...[English]{'id': 16895, 'rating': 7.6, 'votes': 918}An irresponsible young millionaire changes his...[Harold Lloyd, Jobyna Ralston, Noah Young, Jim...For Heaven's Sake
\n", - "
\n", - "
\n", - "\n", - "
\n", - " \n", - "\n", - " \n", - "\n", - " \n", - "
\n", - "\n", - "\n", - "
\n", - " \n", - "\n", - "\n", - "\n", - " \n", - "
\n", - "
\n", - "
\n" - ], - "text/plain": [ - " num_mflix_comments genres countries \\\n", - "0 0 [Action] [USA] \n", - "1 0 [Comedy, Short, Action] [USA] \n", - "2 0 [Action, Adventure, Drama] [USA] \n", - "3 1 [Adventure, Action] [USA] \n", - "4 0 [Action, Comedy, Romance] [USA] \n", - "\n", - " directors \\\n", - "0 [Louis J. Gasnier, Donald MacKenzie] \n", - "1 [Alfred J. Goulding, Hal Roach] \n", - "2 [Herbert Brenon] \n", - "3 [Albert Parker] \n", - "4 [Sam Taylor] \n", - "\n", - " fullplot \\\n", - "0 Young Pauline is left a lot of money when her ... \n", - "1 As a penniless man worries about how he will m... \n", - "2 Michael \"Beau\" Geste leaves England in disgrac... \n", - "3 A nobleman vows to avenge the death of his fat... \n", - "4 The Uptown Boy, J. Harold Manners (Lloyd) is a... \n", - "\n", - " writers \\\n", - "0 [Charles W. Goddard (screenplay), Basil Dickey... \n", - "1 [H.M. Walker (titles)] \n", - "2 [Herbert Brenon (adaptation), John Russell (ad... \n", - "3 [Douglas Fairbanks (story), Jack Cunningham (a... \n", - "4 [Ted Wilde (story), John Grey (story), Clyde B... \n", - "\n", - " awards runtime type rated \\\n", - "0 {'nominations': 0, 'text': '1 win.', 'wins': 1} 199.0 movie None \n", - "1 {'nominations': 1, 'text': '1 nomination.', 'w... 22.0 movie TV-G \n", - "2 {'nominations': 0, 'text': '1 win.', 'wins': 1} 101.0 movie None \n", - "3 {'nominations': 0, 'text': '1 win.', 'wins': 1} 88.0 movie None \n", - "4 {'nominations': 1, 'text': '1 nomination.', 'w... 58.0 movie PASSED \n", - "\n", - " metacritic poster languages \\\n", - "0 NaN https://m.media-amazon.com/images/M/MV5BMzgxOD... [English] \n", - "1 NaN https://m.media-amazon.com/images/M/MV5BNzE1OW... [English] \n", - "2 NaN None [English] \n", - "3 NaN https://m.media-amazon.com/images/M/MV5BMzU0ND... None \n", - "4 NaN https://m.media-amazon.com/images/M/MV5BMTcxMT... [English] \n", - "\n", - " imdb \\\n", - "0 {'id': 4465, 'rating': 7.6, 'votes': 744} \n", - "1 {'id': 10146, 'rating': 7.0, 'votes': 639} \n", - "2 {'id': 16634, 'rating': 6.9, 'votes': 222} \n", - "3 {'id': 16654, 'rating': 7.2, 'votes': 1146} \n", - "4 {'id': 16895, 'rating': 7.6, 'votes': 918} \n", - "\n", - " plot \\\n", - "0 Young Pauline is left a lot of money when her ... \n", - "1 A penniless young man tries to save an heiress... \n", - "2 Michael \"Beau\" Geste leaves England in disgrac... \n", - "3 Seeking revenge, an athletic young man joins t... \n", - "4 An irresponsible young millionaire changes his... \n", - "\n", - " cast title \n", - "0 [Pearl White, Crane Wilbur, Paul Panzer, Edwar... The Perils of Pauline \n", - "1 [Harold Lloyd, Mildred Davis, 'Snub' Pollard, ... From Hand to Mouth \n", - "2 [Ronald Colman, Neil Hamilton, Ralph Forbes, A... Beau Geste \n", - "3 [Billie Dove, Tempe Pigott, Donald Crisp, Sam ... The Black Pirate \n", - "4 [Harold Lloyd, Jobyna Ralston, Noah Young, Jim... For Heaven's Sake " - ] - }, - "execution_count": 3, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# Data Preparation\n", - "\n", - "# Remove data point where plot coloumn is missing\n", - "dataset_df = dataset_df.dropna(subset=[\"fullplot\"])\n", - "print(\"\\nNumber of missing values in each column after removal:\")\n", - "print(dataset_df.isnull().sum())\n", - "\n", - "# Remove the plot_embedding from each data point in the dataset as we are going to create new embeddings with an open source embedding model from Hugging Face\n", - "dataset_df = dataset_df.drop(columns=[\"plot_embedding\"])\n", - "dataset_df.head(5)" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 747 - }, - "id": "ZX8zJNN5UzPK", - "outputId": "81bc1a57-7d96-4311-ba94-4748c34c20e3" - }, - "outputs": [ - { - "data": { - "application/vnd.google.colaboratory.intrinsic+json": { - "summary": "{\n \"name\": \"dataset_df\",\n \"rows\": 1452,\n \"fields\": [\n {\n \"column\": \"num_mflix_comments\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 27,\n \"min\": 0,\n \"max\": 158,\n \"num_unique_values\": 40,\n \"samples\": [\n 117,\n 134,\n 124\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"genres\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"countries\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"directors\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"fullplot\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1409,\n \"samples\": [\n \"An undercover cop infiltrates a gang of thieves who plan to rob a jewelry store.\",\n \"Godzilla returns in a brand-new movie that ignores all preceding movies except for the original with a brand new look and a powered up atomic ray. This time he battles a mysterious UFO that later transforms into a mysterious kaiju dubbed Orga. They meet up for the final showdown in the city of Shinjuku.\",\n \"Relationships become entangled in an emotional web.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"writers\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"awards\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"runtime\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 42.5693352357647,\n \"min\": 6.0,\n \"max\": 1256.0,\n \"num_unique_values\": 137,\n \"samples\": [\n 60.0,\n 151.0,\n 110.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"type\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 2,\n \"samples\": [\n \"series\",\n \"movie\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"rated\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 12,\n \"samples\": [\n \"TV-MA\",\n \"TV-14\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"metacritic\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 16.855402595666057,\n \"min\": 9.0,\n \"max\": 97.0,\n \"num_unique_values\": 83,\n \"samples\": [\n 50.0,\n 97.0\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"poster\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1332,\n \"samples\": [\n \"https://m.media-amazon.com/images/M/MV5BMTQ2NTMxODEyNV5BMl5BanBnXkFtZTcwMDgxMjA0MQ@@._V1_SY1000_SX677_AL_.jpg\",\n \"https://m.media-amazon.com/images/M/MV5BMTY5OTg1ODk0MV5BMl5BanBnXkFtZTcwMTEwMjU1MQ@@._V1_SY1000_SX677_AL_.jpg\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"languages\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"imdb\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"plot\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1409,\n \"samples\": [\n \"An undercover cop infiltrates a gang of thieves who plan to rob a jewelry store.\",\n \"Godzilla saves Tokyo from a flying saucer that transforms into the beast Orga.\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"cast\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"title\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 1391,\n \"samples\": [\n \"Superhero Movie\",\n \"Hooper\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"embedding\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", - "type": "dataframe", - "variable_name": "dataset_df" - }, - "text/html": [ - "\n", - "
\n", - "
\n", - "\n", - "\n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - " \n", - "
num_mflix_commentsgenrescountriesdirectorsfullplotwritersawardsruntimetyperatedmetacriticposterlanguagesimdbplotcasttitleembedding
00[Action][USA][Louis J. Gasnier, Donald MacKenzie]Young Pauline is left a lot of money when her ...[Charles W. Goddard (screenplay), Basil Dickey...{'nominations': 0, 'text': '1 win.', 'wins': 1}199.0movieNoneNaNhttps://m.media-amazon.com/images/M/MV5BMzgxOD...[English]{'id': 4465, 'rating': 7.6, 'votes': 744}Young Pauline is left a lot of money when her ...[Pearl White, Crane Wilbur, Paul Panzer, Edwar...The Perils of Pauline[-0.009285838343203068, -0.005062104668468237,...
10[Comedy, Short, Action][USA][Alfred J. Goulding, Hal Roach]As a penniless man worries about how he will m...[H.M. Walker (titles)]{'nominations': 1, 'text': '1 nomination.', 'w...22.0movieTV-GNaNhttps://m.media-amazon.com/images/M/MV5BNzE1OW...[English]{'id': 10146, 'rating': 7.0, 'votes': 639}A penniless young man tries to save an heiress...[Harold Lloyd, Mildred Davis, 'Snub' Pollard, ...From Hand to Mouth[-0.0024393785279244184, 0.02309592440724373, ...
20[Action, Adventure, Drama][USA][Herbert Brenon]Michael \"Beau\" Geste leaves England in disgrac...[Herbert Brenon (adaptation), John Russell (ad...{'nominations': 0, 'text': '1 win.', 'wins': 1}101.0movieNoneNaNNone[English]{'id': 16634, 'rating': 6.9, 'votes': 222}Michael \"Beau\" Geste leaves England in disgrac...[Ronald Colman, Neil Hamilton, Ralph Forbes, A...Beau Geste[0.012204292230308056, -0.01145575474947691, -...
31[Adventure, Action][USA][Albert Parker]A nobleman vows to avenge the death of his fat...[Douglas Fairbanks (story), Jack Cunningham (a...{'nominations': 0, 'text': '1 win.', 'wins': 1}88.0movieNoneNaNhttps://m.media-amazon.com/images/M/MV5BMzU0ND...None{'id': 16654, 'rating': 7.2, 'votes': 1146}Seeking revenge, an athletic young man joins t...[Billie Dove, Tempe Pigott, Donald Crisp, Sam ...The Black Pirate[0.004541348200291395, -0.0006100579630583525,...
40[Action, Comedy, Romance][USA][Sam Taylor]The Uptown Boy, J. Harold Manners (Lloyd) is a...[Ted Wilde (story), John Grey (story), Clyde B...{'nominations': 1, 'text': '1 nomination.', 'w...58.0moviePASSEDNaNhttps://m.media-amazon.com/images/M/MV5BMTcxMT...[English]{'id': 16895, 'rating': 7.6, 'votes': 918}An irresponsible young millionaire changes his...[Harold Lloyd, Jobyna Ralston, Noah Young, Jim...For Heaven's Sake[-0.0022256041411310434, 0.011567804962396622,...
\n", - "
\n", - "
\n", - "\n", - "
\n", - " \n", - "\n", - " \n", - "\n", - " \n", - "
\n", - "\n", - "\n", - "
\n", - " \n", - "\n", - "\n", - "\n", - " \n", - "
\n", - "
\n", - "
\n" - ], - "text/plain": [ - " num_mflix_comments genres countries \\\n", - "0 0 [Action] [USA] \n", - "1 0 [Comedy, Short, Action] [USA] \n", - "2 0 [Action, Adventure, Drama] [USA] \n", - "3 1 [Adventure, Action] [USA] \n", - "4 0 [Action, Comedy, Romance] [USA] \n", - "\n", - " directors \\\n", - "0 [Louis J. Gasnier, Donald MacKenzie] \n", - "1 [Alfred J. Goulding, Hal Roach] \n", - "2 [Herbert Brenon] \n", - "3 [Albert Parker] \n", - "4 [Sam Taylor] \n", - "\n", - " fullplot \\\n", - "0 Young Pauline is left a lot of money when her ... \n", - "1 As a penniless man worries about how he will m... \n", - "2 Michael \"Beau\" Geste leaves England in disgrac... \n", - "3 A nobleman vows to avenge the death of his fat... \n", - "4 The Uptown Boy, J. Harold Manners (Lloyd) is a... \n", - "\n", - " writers \\\n", - "0 [Charles W. Goddard (screenplay), Basil Dickey... \n", - "1 [H.M. Walker (titles)] \n", - "2 [Herbert Brenon (adaptation), John Russell (ad... \n", - "3 [Douglas Fairbanks (story), Jack Cunningham (a... \n", - "4 [Ted Wilde (story), John Grey (story), Clyde B... \n", - "\n", - " awards runtime type rated \\\n", - "0 {'nominations': 0, 'text': '1 win.', 'wins': 1} 199.0 movie None \n", - "1 {'nominations': 1, 'text': '1 nomination.', 'w... 22.0 movie TV-G \n", - "2 {'nominations': 0, 'text': '1 win.', 'wins': 1} 101.0 movie None \n", - "3 {'nominations': 0, 'text': '1 win.', 'wins': 1} 88.0 movie None \n", - "4 {'nominations': 1, 'text': '1 nomination.', 'w... 58.0 movie PASSED \n", - "\n", - " metacritic poster languages \\\n", - "0 NaN https://m.media-amazon.com/images/M/MV5BMzgxOD... [English] \n", - "1 NaN https://m.media-amazon.com/images/M/MV5BNzE1OW... [English] \n", - "2 NaN None [English] \n", - "3 NaN https://m.media-amazon.com/images/M/MV5BMzU0ND... None \n", - "4 NaN https://m.media-amazon.com/images/M/MV5BMTcxMT... [English] \n", - "\n", - " imdb \\\n", - "0 {'id': 4465, 'rating': 7.6, 'votes': 744} \n", - "1 {'id': 10146, 'rating': 7.0, 'votes': 639} \n", - "2 {'id': 16634, 'rating': 6.9, 'votes': 222} \n", - "3 {'id': 16654, 'rating': 7.2, 'votes': 1146} \n", - "4 {'id': 16895, 'rating': 7.6, 'votes': 918} \n", - "\n", - " plot \\\n", - "0 Young Pauline is left a lot of money when her ... \n", - "1 A penniless young man tries to save an heiress... \n", - "2 Michael \"Beau\" Geste leaves England in disgrac... \n", - "3 Seeking revenge, an athletic young man joins t... \n", - "4 An irresponsible young millionaire changes his... \n", - "\n", - " cast title \\\n", - "0 [Pearl White, Crane Wilbur, Paul Panzer, Edwar... The Perils of Pauline \n", - "1 [Harold Lloyd, Mildred Davis, 'Snub' Pollard, ... From Hand to Mouth \n", - "2 [Ronald Colman, Neil Hamilton, Ralph Forbes, A... Beau Geste \n", - "3 [Billie Dove, Tempe Pigott, Donald Crisp, Sam ... The Black Pirate \n", - "4 [Harold Lloyd, Jobyna Ralston, Noah Young, Jim... For Heaven's Sake \n", - "\n", - " embedding \n", - "0 [-0.009285838343203068, -0.005062104668468237,... \n", - "1 [-0.0024393785279244184, 0.02309592440724373, ... \n", - "2 [0.012204292230308056, -0.01145575474947691, -... \n", - "3 [0.004541348200291395, -0.0006100579630583525,... \n", - "4 [-0.0022256041411310434, 0.011567804962396622,... " - ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "from sentence_transformers import SentenceTransformer\n", - "\n", - "# https://huggingface.co/thenlper/gte-large\n", - "embedding_model = SentenceTransformer(\"thenlper/gte-large\")\n", - "\n", - "\n", - "def get_embedding(text: str) -> list[float]:\n", - " if not text.strip():\n", - " print(\"Attempted to get embedding for empty text.\")\n", - " return []\n", - "\n", - " embedding = embedding_model.encode(text)\n", - "\n", - " return embedding.tolist()\n", - "\n", - "\n", - "dataset_df[\"embedding\"] = dataset_df[\"fullplot\"].apply(get_embedding)\n", - "\n", - "dataset_df.head()" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "Oi0l9POtU0iP", - "outputId": "d3fe3cc4-8c08-4435-ddfc-8cfcc5ada572" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Connection to MongoDB successful\n" - ] - } - ], - "source": [ - "import pymongo\n", - "from google.colab import userdata\n", - "\n", - "def get_mongo_client(mongo_uri):\n", - " \"\"\"Establish connection to the MongoDB.\"\"\"\n", - " try:\n", - " client = pymongo.MongoClient(mongo_uri, appname=\"devrel.content.python\")\n", - " print(\"Connection to MongoDB successful\")\n", - " return client\n", - " except pymongo.errors.ConnectionFailure as e:\n", - " print(f\"Connection failed: {e}\")\n", - " return None\n", - "\n", - "mongo_uri = userdata.get('MONGO_URI')\n", - "if not mongo_uri:\n", - " print(\"MONGO_URI not set in environment variables\")\n", - "\n", - "mongo_client = get_mongo_client(mongo_uri)\n", - "\n", - "# Ingest data into MongoDB\n", - "db = mongo_client['movies']\n", - "collection = db['movie_collection_2']" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "F7XXXa-OU1u9", - "outputId": "7bd1eb43-e933-4150-990a-fa20bad84e9a" - }, - "outputs": [ - { - "data": { - "text/plain": [ - "DeleteResult({'n': 1452, 'electionId': ObjectId('7fffffff000000000000000c'), 'opTime': {'ts': Timestamp(1708554945, 1452), 't': 12}, 'ok': 1.0, '$clusterTime': {'clusterTime': Timestamp(1708554945, 1452), 'signature': {'hash': b'\\x99\\x89\\xc0\\x00Cn!\\xd6\\xaf\\xb3\\x96\\xdf\\xc3\\xda\\x88\\x11\\xf5\\t\\xbd\\xc0', 'keyId': 7320226449804230661}}, 'operationTime': Timestamp(1708554945, 1452)}, acknowledged=True)" - ] - }, - "execution_count": 6, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "# Delete any existing records in the collection\n", - "collection.delete_many({})" - ] - }, - { - "cell_type": "code", - "execution_count": 7, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "XrfMY4QBU2-l", - "outputId": "e2b5c534-2ba0-4ffa-bca8-1e96bef14c54" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Data ingestion into MongoDB completed\n" - ] - } - ], - "source": [ - "documents = dataset_df.to_dict(\"records\")\n", - "collection.insert_many(documents)\n", - "\n", - "print(\"Data ingestion into MongoDB completed\")" - ] - }, - { - "cell_type": "code", - "execution_count": 8, - "metadata": { - "id": "kWucnQBEU35k" - }, - "outputs": [], - "source": [ - "\n", - "def vector_search(user_query, collection):\n", - " \"\"\"\n", - " Perform a vector search in the MongoDB collection based on the user query.\n", - "\n", - " Args:\n", - " user_query (str): The user's query string.\n", - " collection (MongoCollection): The MongoDB collection to search.\n", - "\n", - " Returns:\n", - " list: A list of matching documents.\n", - " \"\"\"\n", - "\n", - " # Generate embedding for the user query\n", - " query_embedding = get_embedding(user_query)\n", - "\n", - " if query_embedding is None:\n", - " return \"Invalid query or embedding generation failed.\"\n", - "\n", - " # Define the vector search pipeline\n", - " vector_search_stage = {\n", - " \"$vectorSearch\": {\n", - " \"index\": \"vector_index\",\n", - " \"queryVector\": query_embedding,\n", - " \"path\": \"embedding\",\n", - " \"numCandidates\": 150, # Number of candidate matches to consider\n", - " \"limit\": 4 # Return top 4 matches\n", - " }\n", - " }\n", - "\n", - " unset_stage = {\n", - " \"$unset\": \"embedding\" # Exclude the 'embedding' field from the results\n", - " }\n", - "\n", - " project_stage = {\n", - " \"$project\": {\n", - " \"_id\": 0, # Exclude the _id field\n", - " \"fullplot\": 1, # Include the plot field\n", - " \"title\": 1, # Include the title field\n", - " \"genres\": 1, # Include the genres field\n", - " \"score\": {\n", - " \"$meta\": \"vectorSearchScore\" # Include the search score\n", - " }\n", - " }\n", - " }\n", - "\n", - " pipeline = [vector_search_stage, unset_stage, project_stage]\n", - "\n", - " # Execute the search\n", - " results = collection.aggregate(pipeline)\n", - " return list(results)" - ] - }, - { - "cell_type": "code", - "execution_count": 9, - "metadata": { - "id": "0ka4WLTmU5L4" - }, - "outputs": [], - "source": [ - "def get_search_result(query, collection):\n", - "\n", - " get_knowledge = vector_search(query, collection)\n", - "\n", - " search_result = \"\"\n", - " for result in get_knowledge:\n", - " search_result += f\"Title: {result.get('title', 'N/A')}, Plot: {result.get('fullplot', 'N/A')}\\n\"\n", - "\n", - " return search_result" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "Z4L4SfueU6PY", - "outputId": "11ea30ca-8cac-4e4c-9ab6-780e043c6345" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Query: What is the best romantic movie to watch and why?\n", - "Continue to answer the query by using the Search Results:\n", - "Title: Shut Up and Kiss Me!, Plot: Ryan and Pete are 27-year old best friends in Miami, born on the same day and each searching for the perfect woman. Ryan is a rookie stockbroker living with his psychic Mom. Pete is a slick surfer dude yet to find commitment. Each meets the women of their dreams on the same day. Ryan knocks heads in an elevator with the gorgeous Jessica, passing out before getting her number. Pete falls for the insatiable Tiara, but Tiara's uncle is mob boss Vincent Bublione, charged with her protection. This high-energy romantic comedy asks to what extent will you go for true love?\n", - "Title: Pearl Harbor, Plot: Pearl Harbor is a classic tale of romance set during a war that complicates everything. It all starts when childhood friends Rafe and Danny become Army Air Corps pilots and meet Evelyn, a Navy nurse. Rafe falls head over heels and next thing you know Evelyn and Rafe are hooking up. Then Rafe volunteers to go fight in Britain and Evelyn and Danny get transferred to Pearl Harbor. While Rafe is off fighting everything gets completely whack and next thing you know everybody is in the middle of an air raid we now know as \"Pearl Harbor.\"\n", - "Title: Titanic, Plot: The plot focuses on the romances of two couples upon the doomed ship's maiden voyage. Isabella Paradine (Catherine Zeta-Jones) is a wealthy woman mourning the loss of her aunt, who reignites a romance with former flame Wynn Park (Peter Gallagher). Meanwhile, a charming ne'er-do-well named Jamie Perse (Mike Doyle) steals a ticket for the ship, and falls for a sweet innocent Irish girl on board. But their romance is threatened by the villainous Simon Doonan (Tim Curry), who has discovered about the ticket and makes Jamie his unwilling accomplice, as well as having sinister plans for the girl.\n", - "Title: China Girl, Plot: A modern day Romeo & Juliet story is told in New York when an Italian boy and a Chinese girl become lovers, causing a tragic conflict between ethnic gangs.\n", - ".\n" - ] - } - ], - "source": [ - "# Conduct query with retrival of sources\n", - "query = \"What is the best romantic movie to watch and why?\"\n", - "source_information = get_search_result(query, collection)\n", - "combined_information = f\"Query: {query}\\nContinue to answer the query by using the Search Results:\\n{source_information}.\"\n", - "\n", - "print(combined_information)" - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/", - "height": 209, - "referenced_widgets": [ - "60c4d6d5e7a84fa493f101cc47dadef9", - "fa0c528cca744cff8da0a4fa21fdb4b5", - "d7d4a9f444fe4ebb9135035e2166a3a5", - "4e62b9ec821348cc94b34cfbc010c2a4", - "9d9a247c6569458abd0dcd6e0d717079", - "e5a6d300bbf441b8904aa9afb89e6f31", - "88226abe35534278bbd427d8eff0f5f8", - "2e081a17ddc04104882893a30c902265", - "938f8f60901442f2902eb51e86c27961", - "ce6a3a655d2f4ce2ab351c766568bed5", - "19fd13ad5b2740aa8be2a7d62488fdaf", - "c7c0c34a71954d6ea976c774573c49c5", - "0d6ec3bab579406fa4e6fc2b3d6b6998", - "a37f2164e11d4e5f851a4a09a12c663c", - "ef32431228f24a5498810a36b9cf6506", - "c06e354cb8294e66a3d7590a576571e0", - "e2998c2c6b1f4d489a5e39f2076838e4", - "4300755179d9465db871b14ae78dabc6", - "f14106c7f60f411199acf47f530443fd", - "bf88ee6dc83648d8a61d75bb4466b1e3", - "5776e818d9d34e009f95833056522876", - "06b1a069317041c8a9174c14fdc867bc", - "0e27bfa4f64f427d9996de0451e9edd9", - "d5e9f339fe7e4ab9955531cc125f071e", - "fa9cf3e72280417d8711ef7227a95d34", - "c3a1b520140444fbb40b7ac789f7ac0e", - "2c84bc5c158641f49f421a7d28da1518", - "6a15f1cf54a141fc9d6bb790366c6bdd", - "8813b56cd89744b58ace2787206e1501", - "edc37210db734d01a8afce596698bb27", - "eba6048eb694485693656fcbf4a4f297", - "30885be6a7c84f0f9f02bc2ea11679bc", - "29178e51df9e47489fff623763b130ed", - "5266bebcf8bb4b0798b14831a38e2a8c", - "7c638aaf734c423fbe54daddff97040f", - "4c6736981923464db2f754339c60cd0d", - "57383c03fc854a92a2ff732cbdd80a70", - "8a302ae0412b4393a17b50861afe36b5", - "b2fdc502d6ee491bb826fd616e35d407", - "2677891ce891475b8dc7c2ae287c58d7", - "fddbae43ce7f477cafaff89b81e47fc7", - "592d069be51e43f99212d47be9c11dcf", - "9a4c90a767c746659ea535d7c36d40a5", - "43fcf04b360f4a75be9fb99ab69fbe38", - "b7c439aa6d584c5784b46980050e503d", - "8aa8805651d34181b1851d302ccc47e2", - "713f1d91e445411288e565a33ce4b271", - "55941e08c602404c9342d00b7ee26918", - "87da02f5606d404ea242c3bd1f9ac38c", - "947f9b7e19dc4be4bd21b1b021e91f9d", - "0b7f3d233b8f4912bef4deae2e395001", - "6ccbd7b9ae924b5e843efd3114dfb2c5", - "9e0bccbc6072461fbf96482b870ed8d5", - "d7a00f1f114e4f008f4d5a48c1c69f53", - "faf25fd219f24bdbaa2e3202548c97d9", - "a0996675df13484aaa519e6ff45c5476", - "0bfb4937ed5547b3ba464ca47ac77f1a", - "7f59906980724a8b840dec85ce400f89", - "80f3d29327bf429481ad191b1abe556f", - "6d7c024126ac4c34825fae522234ebca", - "a0600fb407034c2d8df6ae5830d601db", - "c1d37ab1952b4d268d9786b74b6902d7", - "e7f471604a5a42e095d35d8ad399c6fe", - "feb438afda6b4c148a3a62ee7e03da74", - "e68cf53b04a845ac9d6f4047600ebc21", - "33fef11f829f49e2aa9555201d4a0e42" - ] - }, - "id": "OYGmKVv9mm8g", - "outputId": "ff41bfed-daa0-4ed8-8cc4-0aa138e697a1" - }, - "outputs": [], - "source": [ - "from transformers import AutoTokenizer, AutoModelForCausalLM\n", - "\n", - "tokenizer = AutoTokenizer.from_pretrained(\"google/gemma-2b-it\")\n", - "# CPU Enabled uncomment below 👇🏽\n", - "# model = AutoModelForCausalLM.from_pretrained(\"google/gemma-2b-it\")\n", - "# GPU Enabled use below 👇🏽\n", - "model = AutoModelForCausalLM.from_pretrained(\"google/gemma-2b-it\", device_map=\"auto\")" - ] - }, - { - "cell_type": "code", - "execution_count": 16, - "metadata": { - "colab": { - "base_uri": "https://localhost:8080/" - }, - "id": "wDA9SdXhsFyM", - "outputId": "c3300fa5-586c-48bd-9abb-b12a4390a294" - }, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Query: What is the best romantic movie to watch and why?\n", - "Continue to answer the query by using the Search Results:\n", - "Title: Shut Up and Kiss Me!, Plot: Ryan and Pete are 27-year old best friends in Miami, born on the same day and each searching for the perfect woman. Ryan is a rookie stockbroker living with his psychic Mom. Pete is a slick surfer dude yet to find commitment. Each meets the women of their dreams on the same day. Ryan knocks heads in an elevator with the gorgeous Jessica, passing out before getting her number. Pete falls for the insatiable Tiara, but Tiara's uncle is mob boss Vincent Bublione, charged with her protection. This high-energy romantic comedy asks to what extent will you go for true love?\n", - "Title: Pearl Harbor, Plot: Pearl Harbor is a classic tale of romance set during a war that complicates everything. It all starts when childhood friends Rafe and Danny become Army Air Corps pilots and meet Evelyn, a Navy nurse. Rafe falls head over heels and next thing you know Evelyn and Rafe are hooking up. Then Rafe volunteers to go fight in Britain and Evelyn and Danny get transferred to Pearl Harbor. While Rafe is off fighting everything gets completely whack and next thing you know everybody is in the middle of an air raid we now know as \"Pearl Harbor.\"\n", - "Title: Titanic, Plot: The plot focuses on the romances of two couples upon the doomed ship's maiden voyage. Isabella Paradine (Catherine Zeta-Jones) is a wealthy woman mourning the loss of her aunt, who reignites a romance with former flame Wynn Park (Peter Gallagher). Meanwhile, a charming ne'er-do-well named Jamie Perse (Mike Doyle) steals a ticket for the ship, and falls for a sweet innocent Irish girl on board. But their romance is threatened by the villainous Simon Doonan (Tim Curry), who has discovered about the ticket and makes Jamie his unwilling accomplice, as well as having sinister plans for the girl.\n", - "Title: China Girl, Plot: A modern day Romeo & Juliet story is told in New York when an Italian boy and a Chinese girl become lovers, causing a tragic conflict between ethnic gangs.\n", - ".\n", - "\n", - "Based on the search results, the best romantic movie to watch is **Shut Up and Kiss Me!** because it is a romantic comedy that explores the complexities of love and relationships. The movie is funny, heartwarming, and thought-provoking.\n" - ] - } - ], - "source": [ - "# Moving tensors to GPU\n", - "input_ids = tokenizer(combined_information, return_tensors=\"pt\").to(\"cuda\")\n", - "response = model.generate(**input_ids, max_new_tokens=500)\n", - "print(tokenizer.decode(response[0]))" - ] - }, - { - "cell_type": "code", - "execution_count": 12, - "metadata": { - "id": "FhMmFmUBwBcy" - }, - "outputs": [], - "source": [] - } - ], - "metadata": { - "accelerator": "GPU", - "colab": { - "gpuType": "A100", - "machine_shape": "hm", - "provenance": [] - }, - "kernelspec": { - "display_name": "Python 3", - "name": "python3" - }, - "language_info": { - "name": "python" - }, - "widgets": { - "application/vnd.jupyter.widget-state+json": { - "06b1a069317041c8a9174c14fdc867bc": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "DescriptionStyleModel", - "state": { - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "DescriptionStyleModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "StyleView", - "description_width": "" - } - }, - "0b7f3d233b8f4912bef4deae2e395001": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "DescriptionStyleModel", - "state": { - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "DescriptionStyleModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "StyleView", - "description_width": "" - } - }, - "0bfb4937ed5547b3ba464ca47ac77f1a": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "HTMLModel", - "state": { - "_dom_classes": [], - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "HTMLModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/controls", - "_view_module_version": "1.5.0", - "_view_name": "HTMLView", - "description": "", - "description_tooltip": null, - "layout": "IPY_MODEL_a0600fb407034c2d8df6ae5830d601db", - "placeholder": "​", - "style": "IPY_MODEL_c1d37ab1952b4d268d9786b74b6902d7", - "value": "generation_config.json: 100%" - } - }, - "0d6ec3bab579406fa4e6fc2b3d6b6998": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "HTMLModel", - "state": { - "_dom_classes": [], - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "HTMLModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/controls", - "_view_module_version": "1.5.0", - "_view_name": "HTMLView", - "description": "", - "description_tooltip": null, - "layout": "IPY_MODEL_e2998c2c6b1f4d489a5e39f2076838e4", - "placeholder": "​", - "style": "IPY_MODEL_4300755179d9465db871b14ae78dabc6", - "value": "Downloading shards: 100%" - } - }, - "0e27bfa4f64f427d9996de0451e9edd9": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "HBoxModel", - "state": { - "_dom_classes": [], - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "HBoxModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/controls", - "_view_module_version": "1.5.0", - "_view_name": "HBoxView", - "box_style": "", - "children": [ - "IPY_MODEL_d5e9f339fe7e4ab9955531cc125f071e", - "IPY_MODEL_fa9cf3e72280417d8711ef7227a95d34", - "IPY_MODEL_c3a1b520140444fbb40b7ac789f7ac0e" - ], - "layout": "IPY_MODEL_2c84bc5c158641f49f421a7d28da1518" - } - }, - "19fd13ad5b2740aa8be2a7d62488fdaf": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "DescriptionStyleModel", - "state": { - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "DescriptionStyleModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "StyleView", - "description_width": "" - } - }, - "2677891ce891475b8dc7c2ae287c58d7": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "DescriptionStyleModel", - "state": { - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "DescriptionStyleModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "StyleView", - "description_width": "" - } - }, - "29178e51df9e47489fff623763b130ed": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "DescriptionStyleModel", - "state": { - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "DescriptionStyleModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "StyleView", - "description_width": "" - } - }, - "2c84bc5c158641f49f421a7d28da1518": { - "model_module": "@jupyter-widgets/base", - "model_module_version": "1.2.0", - "model_name": "LayoutModel", - "state": { - "_model_module": "@jupyter-widgets/base", - "_model_module_version": "1.2.0", - "_model_name": "LayoutModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "LayoutView", - "align_content": null, - "align_items": null, - "align_self": null, - "border": null, - "bottom": null, - "display": null, - "flex": null, - "flex_flow": null, - "grid_area": null, - "grid_auto_columns": null, - "grid_auto_flow": null, - "grid_auto_rows": null, - "grid_column": null, - "grid_gap": null, - "grid_row": null, - "grid_template_areas": null, - "grid_template_columns": null, - "grid_template_rows": null, - "height": null, - "justify_content": null, - "justify_items": null, - "left": null, - "margin": null, - "max_height": null, - "max_width": null, - "min_height": null, - "min_width": null, - "object_fit": null, - "object_position": null, - "order": null, - "overflow": null, - "overflow_x": null, - "overflow_y": null, - "padding": null, - "right": null, - "top": null, - "visibility": null, - "width": null - } - }, - "2e081a17ddc04104882893a30c902265": { - "model_module": "@jupyter-widgets/base", - "model_module_version": "1.2.0", - "model_name": "LayoutModel", - "state": { - "_model_module": "@jupyter-widgets/base", - "_model_module_version": "1.2.0", - "_model_name": "LayoutModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "LayoutView", - "align_content": null, - "align_items": null, - "align_self": null, - "border": null, - "bottom": null, - "display": null, - "flex": null, - "flex_flow": null, - "grid_area": null, - "grid_auto_columns": null, - "grid_auto_flow": null, - "grid_auto_rows": null, - "grid_column": null, - "grid_gap": null, - "grid_row": null, - "grid_template_areas": null, - "grid_template_columns": null, - "grid_template_rows": null, - "height": null, - "justify_content": null, - "justify_items": null, - "left": null, - "margin": null, - "max_height": null, - "max_width": null, - "min_height": null, - "min_width": null, - "object_fit": null, - "object_position": null, - "order": null, - "overflow": null, - "overflow_x": null, - "overflow_y": null, - "padding": null, - "right": null, - "top": null, - "visibility": null, - "width": null - } - }, - "30885be6a7c84f0f9f02bc2ea11679bc": { - "model_module": "@jupyter-widgets/base", - "model_module_version": "1.2.0", - "model_name": "LayoutModel", - "state": { - "_model_module": "@jupyter-widgets/base", - "_model_module_version": "1.2.0", - "_model_name": "LayoutModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "LayoutView", - "align_content": null, - "align_items": null, - "align_self": null, - "border": null, - "bottom": null, - "display": null, - "flex": null, - "flex_flow": null, - "grid_area": null, - "grid_auto_columns": null, - "grid_auto_flow": null, - "grid_auto_rows": null, - "grid_column": null, - "grid_gap": null, - "grid_row": null, - "grid_template_areas": null, - "grid_template_columns": null, - "grid_template_rows": null, - "height": null, - "justify_content": null, - "justify_items": null, - "left": null, - "margin": null, - "max_height": null, - "max_width": null, - "min_height": null, - "min_width": null, - "object_fit": null, - "object_position": null, - "order": null, - "overflow": null, - "overflow_x": null, - "overflow_y": null, - "padding": null, - "right": null, - "top": null, - "visibility": null, - "width": null - } - }, - "33fef11f829f49e2aa9555201d4a0e42": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "DescriptionStyleModel", - "state": { - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "DescriptionStyleModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "StyleView", - "description_width": "" - } - }, - "4300755179d9465db871b14ae78dabc6": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "DescriptionStyleModel", - "state": { - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "DescriptionStyleModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "StyleView", - "description_width": "" - } - }, - "43fcf04b360f4a75be9fb99ab69fbe38": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "DescriptionStyleModel", - "state": { - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "DescriptionStyleModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "StyleView", - "description_width": "" - } - }, - "4c6736981923464db2f754339c60cd0d": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "FloatProgressModel", - "state": { - "_dom_classes": [], - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "FloatProgressModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/controls", - "_view_module_version": "1.5.0", - "_view_name": "ProgressView", - "bar_style": "success", - "description": "", - "description_tooltip": null, - "layout": "IPY_MODEL_fddbae43ce7f477cafaff89b81e47fc7", - "max": 67121608, - "min": 0, - "orientation": "horizontal", - "style": "IPY_MODEL_592d069be51e43f99212d47be9c11dcf", - "value": 67121608 - } - }, - "4e62b9ec821348cc94b34cfbc010c2a4": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "HTMLModel", - "state": { - "_dom_classes": [], - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "HTMLModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/controls", - "_view_module_version": "1.5.0", - "_view_name": "HTMLView", - "description": "", - "description_tooltip": null, - "layout": "IPY_MODEL_ce6a3a655d2f4ce2ab351c766568bed5", - "placeholder": "​", - "style": "IPY_MODEL_19fd13ad5b2740aa8be2a7d62488fdaf", - "value": " 13.5k/13.5k [00:00<00:00, 1.10MB/s]" - } - }, - "5266bebcf8bb4b0798b14831a38e2a8c": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "HBoxModel", - "state": { - "_dom_classes": [], - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "HBoxModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/controls", - "_view_module_version": "1.5.0", - "_view_name": "HBoxView", - "box_style": "", - "children": [ - "IPY_MODEL_7c638aaf734c423fbe54daddff97040f", - "IPY_MODEL_4c6736981923464db2f754339c60cd0d", - "IPY_MODEL_57383c03fc854a92a2ff732cbdd80a70" - ], - "layout": "IPY_MODEL_8a302ae0412b4393a17b50861afe36b5" - } - }, - "55941e08c602404c9342d00b7ee26918": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "HTMLModel", - "state": { - "_dom_classes": [], - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "HTMLModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/controls", - "_view_module_version": "1.5.0", - "_view_name": "HTMLView", - "description": "", - "description_tooltip": null, - "layout": "IPY_MODEL_d7a00f1f114e4f008f4d5a48c1c69f53", - "placeholder": "​", - "style": "IPY_MODEL_faf25fd219f24bdbaa2e3202548c97d9", - "value": " 2/2 [00:04<00:00,  1.94s/it]" - } - }, - "57383c03fc854a92a2ff732cbdd80a70": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "HTMLModel", - "state": { - "_dom_classes": [], - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "HTMLModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/controls", - "_view_module_version": "1.5.0", - "_view_name": "HTMLView", - "description": "", - "description_tooltip": null, - "layout": "IPY_MODEL_9a4c90a767c746659ea535d7c36d40a5", - "placeholder": "​", - "style": "IPY_MODEL_43fcf04b360f4a75be9fb99ab69fbe38", - "value": " 67.1M/67.1M [00:00<00:00, 465MB/s]" - } - }, - "5776e818d9d34e009f95833056522876": { - "model_module": "@jupyter-widgets/base", - "model_module_version": "1.2.0", - "model_name": "LayoutModel", - "state": { - "_model_module": "@jupyter-widgets/base", - "_model_module_version": "1.2.0", - "_model_name": "LayoutModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "LayoutView", - "align_content": null, - "align_items": null, - "align_self": null, - "border": null, - "bottom": null, - "display": null, - "flex": null, - "flex_flow": null, - "grid_area": null, - "grid_auto_columns": null, - "grid_auto_flow": null, - "grid_auto_rows": null, - "grid_column": null, - "grid_gap": null, - "grid_row": null, - "grid_template_areas": null, - "grid_template_columns": null, - "grid_template_rows": null, - "height": null, - "justify_content": null, - "justify_items": null, - "left": null, - "margin": null, - "max_height": null, - "max_width": null, - "min_height": null, - "min_width": null, - "object_fit": null, - "object_position": null, - "order": null, - "overflow": null, - "overflow_x": null, - "overflow_y": null, - "padding": null, - "right": null, - "top": null, - "visibility": null, - "width": null - } - }, - "592d069be51e43f99212d47be9c11dcf": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "ProgressStyleModel", - "state": { - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "ProgressStyleModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "StyleView", - "bar_color": null, - "description_width": "" - } - }, - "60c4d6d5e7a84fa493f101cc47dadef9": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "HBoxModel", - "state": { - "_dom_classes": [], - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "HBoxModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/controls", - "_view_module_version": "1.5.0", - "_view_name": "HBoxView", - "box_style": "", - "children": [ - "IPY_MODEL_fa0c528cca744cff8da0a4fa21fdb4b5", - "IPY_MODEL_d7d4a9f444fe4ebb9135035e2166a3a5", - "IPY_MODEL_4e62b9ec821348cc94b34cfbc010c2a4" - ], - "layout": "IPY_MODEL_9d9a247c6569458abd0dcd6e0d717079" - } - }, - "6a15f1cf54a141fc9d6bb790366c6bdd": { - "model_module": "@jupyter-widgets/base", - "model_module_version": "1.2.0", - "model_name": "LayoutModel", - "state": { - "_model_module": "@jupyter-widgets/base", - "_model_module_version": "1.2.0", - "_model_name": "LayoutModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "LayoutView", - "align_content": null, - "align_items": null, - "align_self": null, - "border": null, - "bottom": null, - "display": null, - "flex": null, - "flex_flow": null, - "grid_area": null, - "grid_auto_columns": null, - "grid_auto_flow": null, - "grid_auto_rows": null, - "grid_column": null, - "grid_gap": null, - "grid_row": null, - "grid_template_areas": null, - "grid_template_columns": null, - "grid_template_rows": null, - "height": null, - "justify_content": null, - "justify_items": null, - "left": null, - "margin": null, - "max_height": null, - "max_width": null, - "min_height": null, - "min_width": null, - "object_fit": null, - "object_position": null, - "order": null, - "overflow": null, - "overflow_x": null, - "overflow_y": null, - "padding": null, - "right": null, - "top": null, - "visibility": null, - "width": null - } - }, - "6ccbd7b9ae924b5e843efd3114dfb2c5": { - "model_module": "@jupyter-widgets/base", - "model_module_version": "1.2.0", - "model_name": "LayoutModel", - "state": { - "_model_module": "@jupyter-widgets/base", - "_model_module_version": "1.2.0", - "_model_name": "LayoutModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "LayoutView", - "align_content": null, - "align_items": null, - "align_self": null, - "border": null, - "bottom": null, - "display": null, - "flex": null, - "flex_flow": null, - "grid_area": null, - "grid_auto_columns": null, - "grid_auto_flow": null, - "grid_auto_rows": null, - "grid_column": null, - "grid_gap": null, - "grid_row": null, - "grid_template_areas": null, - "grid_template_columns": null, - "grid_template_rows": null, - "height": null, - "justify_content": null, - "justify_items": null, - "left": null, - "margin": null, - "max_height": null, - "max_width": null, - "min_height": null, - "min_width": null, - "object_fit": null, - "object_position": null, - "order": null, - "overflow": null, - "overflow_x": null, - "overflow_y": null, - "padding": null, - "right": null, - "top": null, - "visibility": null, - "width": null - } - }, - "6d7c024126ac4c34825fae522234ebca": { - "model_module": "@jupyter-widgets/base", - "model_module_version": "1.2.0", - "model_name": "LayoutModel", - "state": { - "_model_module": "@jupyter-widgets/base", - "_model_module_version": "1.2.0", - "_model_name": "LayoutModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "LayoutView", - "align_content": null, - "align_items": null, - "align_self": null, - "border": null, - "bottom": null, - "display": null, - "flex": null, - "flex_flow": null, - "grid_area": null, - "grid_auto_columns": null, - "grid_auto_flow": null, - "grid_auto_rows": null, - "grid_column": null, - "grid_gap": null, - "grid_row": null, - "grid_template_areas": null, - "grid_template_columns": null, - "grid_template_rows": null, - "height": null, - "justify_content": null, - "justify_items": null, - "left": null, - "margin": null, - "max_height": null, - "max_width": null, - "min_height": null, - "min_width": null, - "object_fit": null, - "object_position": null, - "order": null, - "overflow": null, - "overflow_x": null, - "overflow_y": null, - "padding": null, - "right": null, - "top": null, - "visibility": null, - "width": null - } - }, - "713f1d91e445411288e565a33ce4b271": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "FloatProgressModel", - "state": { - "_dom_classes": [], - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "FloatProgressModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/controls", - "_view_module_version": "1.5.0", - "_view_name": "ProgressView", - "bar_style": "success", - "description": "", - "description_tooltip": null, - "layout": "IPY_MODEL_6ccbd7b9ae924b5e843efd3114dfb2c5", - "max": 2, - "min": 0, - "orientation": "horizontal", - "style": "IPY_MODEL_9e0bccbc6072461fbf96482b870ed8d5", - "value": 2 - } - }, - "7c638aaf734c423fbe54daddff97040f": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "HTMLModel", - "state": { - "_dom_classes": [], - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "HTMLModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/controls", - "_view_module_version": "1.5.0", - "_view_name": "HTMLView", - "description": "", - "description_tooltip": null, - "layout": "IPY_MODEL_b2fdc502d6ee491bb826fd616e35d407", - "placeholder": "​", - "style": "IPY_MODEL_2677891ce891475b8dc7c2ae287c58d7", - "value": "model-00002-of-00002.safetensors: 100%" - } - }, - "7f59906980724a8b840dec85ce400f89": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "FloatProgressModel", - "state": { - "_dom_classes": [], - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "FloatProgressModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/controls", - "_view_module_version": "1.5.0", - "_view_name": "ProgressView", - "bar_style": "success", - "description": "", - "description_tooltip": null, - "layout": "IPY_MODEL_e7f471604a5a42e095d35d8ad399c6fe", - "max": 137, - "min": 0, - "orientation": "horizontal", - "style": "IPY_MODEL_feb438afda6b4c148a3a62ee7e03da74", - "value": 137 - } - }, - "80f3d29327bf429481ad191b1abe556f": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "HTMLModel", - "state": { - "_dom_classes": [], - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "HTMLModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/controls", - "_view_module_version": "1.5.0", - "_view_name": "HTMLView", - "description": "", - "description_tooltip": null, - "layout": "IPY_MODEL_e68cf53b04a845ac9d6f4047600ebc21", - "placeholder": "​", - "style": "IPY_MODEL_33fef11f829f49e2aa9555201d4a0e42", - "value": " 137/137 [00:00<00:00, 11.9kB/s]" - } - }, - "87da02f5606d404ea242c3bd1f9ac38c": { - "model_module": "@jupyter-widgets/base", - "model_module_version": "1.2.0", - "model_name": "LayoutModel", - "state": { - "_model_module": "@jupyter-widgets/base", - "_model_module_version": "1.2.0", - "_model_name": "LayoutModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "LayoutView", - "align_content": null, - "align_items": null, - "align_self": null, - "border": null, - "bottom": null, - "display": null, - "flex": null, - "flex_flow": null, - "grid_area": null, - "grid_auto_columns": null, - "grid_auto_flow": null, - "grid_auto_rows": null, - "grid_column": null, - "grid_gap": null, - "grid_row": null, - "grid_template_areas": null, - "grid_template_columns": null, - "grid_template_rows": null, - "height": null, - "justify_content": null, - "justify_items": null, - "left": null, - "margin": null, - "max_height": null, - "max_width": null, - "min_height": null, - "min_width": null, - "object_fit": null, - "object_position": null, - "order": null, - "overflow": null, - "overflow_x": null, - "overflow_y": null, - "padding": null, - "right": null, - "top": null, - "visibility": null, - "width": null - } - }, - "8813b56cd89744b58ace2787206e1501": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "DescriptionStyleModel", - "state": { - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "DescriptionStyleModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "StyleView", - "description_width": "" - } - }, - "88226abe35534278bbd427d8eff0f5f8": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "DescriptionStyleModel", - "state": { - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "DescriptionStyleModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "StyleView", - "description_width": "" - } - }, - "8a302ae0412b4393a17b50861afe36b5": { - "model_module": "@jupyter-widgets/base", - "model_module_version": "1.2.0", - "model_name": "LayoutModel", - "state": { - "_model_module": "@jupyter-widgets/base", - "_model_module_version": "1.2.0", - "_model_name": "LayoutModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "LayoutView", - "align_content": null, - "align_items": null, - "align_self": null, - "border": null, - "bottom": null, - "display": null, - "flex": null, - "flex_flow": null, - "grid_area": null, - "grid_auto_columns": null, - "grid_auto_flow": null, - "grid_auto_rows": null, - "grid_column": null, - "grid_gap": null, - "grid_row": null, - "grid_template_areas": null, - "grid_template_columns": null, - "grid_template_rows": null, - "height": null, - "justify_content": null, - "justify_items": null, - "left": null, - "margin": null, - "max_height": null, - "max_width": null, - "min_height": null, - "min_width": null, - "object_fit": null, - "object_position": null, - "order": null, - "overflow": null, - "overflow_x": null, - "overflow_y": null, - "padding": null, - "right": null, - "top": null, - "visibility": null, - "width": null - } - }, - "8aa8805651d34181b1851d302ccc47e2": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "HTMLModel", - "state": { - "_dom_classes": [], - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "HTMLModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/controls", - "_view_module_version": "1.5.0", - "_view_name": "HTMLView", - "description": "", - "description_tooltip": null, - "layout": "IPY_MODEL_947f9b7e19dc4be4bd21b1b021e91f9d", - "placeholder": "​", - "style": "IPY_MODEL_0b7f3d233b8f4912bef4deae2e395001", - "value": "Loading checkpoint shards: 100%" - } - }, - "938f8f60901442f2902eb51e86c27961": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "ProgressStyleModel", - "state": { - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "ProgressStyleModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "StyleView", - "bar_color": null, - "description_width": "" - } - }, - "947f9b7e19dc4be4bd21b1b021e91f9d": { - "model_module": "@jupyter-widgets/base", - "model_module_version": "1.2.0", - "model_name": "LayoutModel", - "state": { - "_model_module": "@jupyter-widgets/base", - "_model_module_version": "1.2.0", - "_model_name": "LayoutModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "LayoutView", - "align_content": null, - "align_items": null, - "align_self": null, - "border": null, - "bottom": null, - "display": null, - "flex": null, - "flex_flow": null, - "grid_area": null, - "grid_auto_columns": null, - "grid_auto_flow": null, - "grid_auto_rows": null, - "grid_column": null, - "grid_gap": null, - "grid_row": null, - "grid_template_areas": null, - "grid_template_columns": null, - "grid_template_rows": null, - "height": null, - "justify_content": null, - "justify_items": null, - "left": null, - "margin": null, - "max_height": null, - "max_width": null, - "min_height": null, - "min_width": null, - "object_fit": null, - "object_position": null, - "order": null, - "overflow": null, - "overflow_x": null, - "overflow_y": null, - "padding": null, - "right": null, - "top": null, - "visibility": null, - "width": null - } - }, - "9a4c90a767c746659ea535d7c36d40a5": { - "model_module": "@jupyter-widgets/base", - "model_module_version": "1.2.0", - "model_name": "LayoutModel", - "state": { - "_model_module": "@jupyter-widgets/base", - "_model_module_version": "1.2.0", - "_model_name": "LayoutModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "LayoutView", - "align_content": null, - "align_items": null, - "align_self": null, - "border": null, - "bottom": null, - "display": null, - "flex": null, - "flex_flow": null, - "grid_area": null, - "grid_auto_columns": null, - "grid_auto_flow": null, - "grid_auto_rows": null, - "grid_column": null, - "grid_gap": null, - "grid_row": null, - "grid_template_areas": null, - "grid_template_columns": null, - "grid_template_rows": null, - "height": null, - "justify_content": null, - "justify_items": null, - "left": null, - "margin": null, - "max_height": null, - "max_width": null, - "min_height": null, - "min_width": null, - "object_fit": null, - "object_position": null, - "order": null, - "overflow": null, - "overflow_x": null, - "overflow_y": null, - "padding": null, - "right": null, - "top": null, - "visibility": null, - "width": null - } - }, - "9d9a247c6569458abd0dcd6e0d717079": { - "model_module": "@jupyter-widgets/base", - "model_module_version": "1.2.0", - "model_name": "LayoutModel", - "state": { - "_model_module": "@jupyter-widgets/base", - "_model_module_version": "1.2.0", - "_model_name": "LayoutModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "LayoutView", - "align_content": null, - "align_items": null, - "align_self": null, - "border": null, - "bottom": null, - "display": null, - "flex": null, - "flex_flow": null, - "grid_area": null, - "grid_auto_columns": null, - "grid_auto_flow": null, - "grid_auto_rows": null, - "grid_column": null, - "grid_gap": null, - "grid_row": null, - "grid_template_areas": null, - "grid_template_columns": null, - "grid_template_rows": null, - "height": null, - "justify_content": null, - "justify_items": null, - "left": null, - "margin": null, - "max_height": null, - "max_width": null, - "min_height": null, - "min_width": null, - "object_fit": null, - "object_position": null, - "order": null, - "overflow": null, - "overflow_x": null, - "overflow_y": null, - "padding": null, - "right": null, - "top": null, - "visibility": null, - "width": null - } - }, - "9e0bccbc6072461fbf96482b870ed8d5": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "ProgressStyleModel", - "state": { - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "ProgressStyleModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "StyleView", - "bar_color": null, - "description_width": "" - } - }, - "a0600fb407034c2d8df6ae5830d601db": { - "model_module": "@jupyter-widgets/base", - "model_module_version": "1.2.0", - "model_name": "LayoutModel", - "state": { - "_model_module": "@jupyter-widgets/base", - "_model_module_version": "1.2.0", - "_model_name": "LayoutModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "LayoutView", - "align_content": null, - "align_items": null, - "align_self": null, - "border": null, - "bottom": null, - "display": null, - "flex": null, - "flex_flow": null, - "grid_area": null, - "grid_auto_columns": null, - "grid_auto_flow": null, - "grid_auto_rows": null, - "grid_column": null, - "grid_gap": null, - "grid_row": null, - "grid_template_areas": null, - "grid_template_columns": null, - "grid_template_rows": null, - "height": null, - "justify_content": null, - "justify_items": null, - "left": null, - "margin": null, - "max_height": null, - "max_width": null, - "min_height": null, - "min_width": null, - "object_fit": null, - "object_position": null, - "order": null, - "overflow": null, - "overflow_x": null, - "overflow_y": null, - "padding": null, - "right": null, - "top": null, - "visibility": null, - "width": null - } - }, - "a0996675df13484aaa519e6ff45c5476": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "HBoxModel", - "state": { - "_dom_classes": [], - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "HBoxModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/controls", - "_view_module_version": "1.5.0", - "_view_name": "HBoxView", - "box_style": "", - "children": [ - "IPY_MODEL_0bfb4937ed5547b3ba464ca47ac77f1a", - "IPY_MODEL_7f59906980724a8b840dec85ce400f89", - "IPY_MODEL_80f3d29327bf429481ad191b1abe556f" - ], - "layout": "IPY_MODEL_6d7c024126ac4c34825fae522234ebca" - } - }, - "a37f2164e11d4e5f851a4a09a12c663c": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "FloatProgressModel", - "state": { - "_dom_classes": [], - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "FloatProgressModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/controls", - "_view_module_version": "1.5.0", - "_view_name": "ProgressView", - "bar_style": "success", - "description": "", - "description_tooltip": null, - "layout": "IPY_MODEL_f14106c7f60f411199acf47f530443fd", - "max": 2, - "min": 0, - "orientation": "horizontal", - "style": "IPY_MODEL_bf88ee6dc83648d8a61d75bb4466b1e3", - "value": 2 - } - }, - "b2fdc502d6ee491bb826fd616e35d407": { - "model_module": "@jupyter-widgets/base", - "model_module_version": "1.2.0", - "model_name": "LayoutModel", - "state": { - "_model_module": "@jupyter-widgets/base", - "_model_module_version": "1.2.0", - "_model_name": "LayoutModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "LayoutView", - "align_content": null, - "align_items": null, - "align_self": null, - "border": null, - "bottom": null, - "display": null, - "flex": null, - "flex_flow": null, - "grid_area": null, - "grid_auto_columns": null, - "grid_auto_flow": null, - "grid_auto_rows": null, - "grid_column": null, - "grid_gap": null, - "grid_row": null, - "grid_template_areas": null, - "grid_template_columns": null, - "grid_template_rows": null, - "height": null, - "justify_content": null, - "justify_items": null, - "left": null, - "margin": null, - "max_height": null, - "max_width": null, - "min_height": null, - "min_width": null, - "object_fit": null, - "object_position": null, - "order": null, - "overflow": null, - "overflow_x": null, - "overflow_y": null, - "padding": null, - "right": null, - "top": null, - "visibility": null, - "width": null - } - }, - "b7c439aa6d584c5784b46980050e503d": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "HBoxModel", - "state": { - "_dom_classes": [], - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "HBoxModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/controls", - "_view_module_version": "1.5.0", - "_view_name": "HBoxView", - "box_style": "", - "children": [ - "IPY_MODEL_8aa8805651d34181b1851d302ccc47e2", - "IPY_MODEL_713f1d91e445411288e565a33ce4b271", - "IPY_MODEL_55941e08c602404c9342d00b7ee26918" - ], - "layout": "IPY_MODEL_87da02f5606d404ea242c3bd1f9ac38c" - } - }, - "bf88ee6dc83648d8a61d75bb4466b1e3": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "ProgressStyleModel", - "state": { - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "ProgressStyleModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "StyleView", - "bar_color": null, - "description_width": "" - } - }, - "c06e354cb8294e66a3d7590a576571e0": { - "model_module": "@jupyter-widgets/base", - "model_module_version": "1.2.0", - "model_name": "LayoutModel", - "state": { - "_model_module": "@jupyter-widgets/base", - "_model_module_version": "1.2.0", - "_model_name": "LayoutModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "LayoutView", - "align_content": null, - "align_items": null, - "align_self": null, - "border": null, - "bottom": null, - "display": null, - "flex": null, - "flex_flow": null, - "grid_area": null, - "grid_auto_columns": null, - "grid_auto_flow": null, - "grid_auto_rows": null, - "grid_column": null, - "grid_gap": null, - "grid_row": null, - "grid_template_areas": null, - "grid_template_columns": null, - "grid_template_rows": null, - "height": null, - "justify_content": null, - "justify_items": null, - "left": null, - "margin": null, - "max_height": null, - "max_width": null, - "min_height": null, - "min_width": null, - "object_fit": null, - "object_position": null, - "order": null, - "overflow": null, - "overflow_x": null, - "overflow_y": null, - "padding": null, - "right": null, - "top": null, - "visibility": null, - "width": null - } - }, - "c1d37ab1952b4d268d9786b74b6902d7": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "DescriptionStyleModel", - "state": { - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "DescriptionStyleModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "StyleView", - "description_width": "" - } - }, - "c3a1b520140444fbb40b7ac789f7ac0e": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "HTMLModel", - "state": { - "_dom_classes": [], - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "HTMLModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/controls", - "_view_module_version": "1.5.0", - "_view_name": "HTMLView", - "description": "", - "description_tooltip": null, - "layout": "IPY_MODEL_30885be6a7c84f0f9f02bc2ea11679bc", - "placeholder": "​", - "style": "IPY_MODEL_29178e51df9e47489fff623763b130ed", - "value": " 4.95G/4.95G [00:16<00:00, 216MB/s]" - } - }, - "c7c0c34a71954d6ea976c774573c49c5": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "HBoxModel", - "state": { - "_dom_classes": [], - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "HBoxModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/controls", - "_view_module_version": "1.5.0", - "_view_name": "HBoxView", - "box_style": "", - "children": [ - "IPY_MODEL_0d6ec3bab579406fa4e6fc2b3d6b6998", - "IPY_MODEL_a37f2164e11d4e5f851a4a09a12c663c", - "IPY_MODEL_ef32431228f24a5498810a36b9cf6506" - ], - "layout": "IPY_MODEL_c06e354cb8294e66a3d7590a576571e0" - } - }, - "ce6a3a655d2f4ce2ab351c766568bed5": { - "model_module": "@jupyter-widgets/base", - "model_module_version": "1.2.0", - "model_name": "LayoutModel", - "state": { - "_model_module": "@jupyter-widgets/base", - "_model_module_version": "1.2.0", - "_model_name": "LayoutModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "LayoutView", - "align_content": null, - "align_items": null, - "align_self": null, - "border": null, - "bottom": null, - "display": null, - "flex": null, - "flex_flow": null, - "grid_area": null, - "grid_auto_columns": null, - "grid_auto_flow": null, - "grid_auto_rows": null, - "grid_column": null, - "grid_gap": null, - "grid_row": null, - "grid_template_areas": null, - "grid_template_columns": null, - "grid_template_rows": null, - "height": null, - "justify_content": null, - "justify_items": null, - "left": null, - "margin": null, - "max_height": null, - "max_width": null, - "min_height": null, - "min_width": null, - "object_fit": null, - "object_position": null, - "order": null, - "overflow": null, - "overflow_x": null, - "overflow_y": null, - "padding": null, - "right": null, - "top": null, - "visibility": null, - "width": null - } - }, - "d5e9f339fe7e4ab9955531cc125f071e": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "HTMLModel", - "state": { - "_dom_classes": [], - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "HTMLModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/controls", - "_view_module_version": "1.5.0", - "_view_name": "HTMLView", - "description": "", - "description_tooltip": null, - "layout": "IPY_MODEL_6a15f1cf54a141fc9d6bb790366c6bdd", - "placeholder": "​", - "style": "IPY_MODEL_8813b56cd89744b58ace2787206e1501", - "value": "model-00001-of-00002.safetensors: 100%" - } - }, - "d7a00f1f114e4f008f4d5a48c1c69f53": { - "model_module": "@jupyter-widgets/base", - "model_module_version": "1.2.0", - "model_name": "LayoutModel", - "state": { - "_model_module": "@jupyter-widgets/base", - "_model_module_version": "1.2.0", - "_model_name": "LayoutModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "LayoutView", - "align_content": null, - "align_items": null, - "align_self": null, - "border": null, - "bottom": null, - "display": null, - "flex": null, - "flex_flow": null, - "grid_area": null, - "grid_auto_columns": null, - "grid_auto_flow": null, - "grid_auto_rows": null, - "grid_column": null, - "grid_gap": null, - "grid_row": null, - "grid_template_areas": null, - "grid_template_columns": null, - "grid_template_rows": null, - "height": null, - "justify_content": null, - "justify_items": null, - "left": null, - "margin": null, - "max_height": null, - "max_width": null, - "min_height": null, - "min_width": null, - "object_fit": null, - "object_position": null, - "order": null, - "overflow": null, - "overflow_x": null, - "overflow_y": null, - "padding": null, - "right": null, - "top": null, - "visibility": null, - "width": null - } - }, - "d7d4a9f444fe4ebb9135035e2166a3a5": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "FloatProgressModel", - "state": { - "_dom_classes": [], - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "FloatProgressModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/controls", - "_view_module_version": "1.5.0", - "_view_name": "ProgressView", - "bar_style": "success", - "description": "", - "description_tooltip": null, - "layout": "IPY_MODEL_2e081a17ddc04104882893a30c902265", - "max": 13489, - "min": 0, - "orientation": "horizontal", - "style": "IPY_MODEL_938f8f60901442f2902eb51e86c27961", - "value": 13489 - } - }, - "e2998c2c6b1f4d489a5e39f2076838e4": { - "model_module": "@jupyter-widgets/base", - "model_module_version": "1.2.0", - "model_name": "LayoutModel", - "state": { - "_model_module": "@jupyter-widgets/base", - "_model_module_version": "1.2.0", - "_model_name": "LayoutModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "LayoutView", - "align_content": null, - "align_items": null, - "align_self": null, - "border": null, - "bottom": null, - "display": null, - "flex": null, - "flex_flow": null, - "grid_area": null, - "grid_auto_columns": null, - "grid_auto_flow": null, - "grid_auto_rows": null, - "grid_column": null, - "grid_gap": null, - "grid_row": null, - "grid_template_areas": null, - "grid_template_columns": null, - "grid_template_rows": null, - "height": null, - "justify_content": null, - "justify_items": null, - "left": null, - "margin": null, - "max_height": null, - "max_width": null, - "min_height": null, - "min_width": null, - "object_fit": null, - "object_position": null, - "order": null, - "overflow": null, - "overflow_x": null, - "overflow_y": null, - "padding": null, - "right": null, - "top": null, - "visibility": null, - "width": null - } - }, - "e5a6d300bbf441b8904aa9afb89e6f31": { - "model_module": "@jupyter-widgets/base", - "model_module_version": "1.2.0", - "model_name": "LayoutModel", - "state": { - "_model_module": "@jupyter-widgets/base", - "_model_module_version": "1.2.0", - "_model_name": "LayoutModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "LayoutView", - "align_content": null, - "align_items": null, - "align_self": null, - "border": null, - "bottom": null, - "display": null, - "flex": null, - "flex_flow": null, - "grid_area": null, - "grid_auto_columns": null, - "grid_auto_flow": null, - "grid_auto_rows": null, - "grid_column": null, - "grid_gap": null, - "grid_row": null, - "grid_template_areas": null, - "grid_template_columns": null, - "grid_template_rows": null, - "height": null, - "justify_content": null, - "justify_items": null, - "left": null, - "margin": null, - "max_height": null, - "max_width": null, - "min_height": null, - "min_width": null, - "object_fit": null, - "object_position": null, - "order": null, - "overflow": null, - "overflow_x": null, - "overflow_y": null, - "padding": null, - "right": null, - "top": null, - "visibility": null, - "width": null - } - }, - "e68cf53b04a845ac9d6f4047600ebc21": { - "model_module": "@jupyter-widgets/base", - "model_module_version": "1.2.0", - "model_name": "LayoutModel", - "state": { - "_model_module": "@jupyter-widgets/base", - "_model_module_version": "1.2.0", - "_model_name": "LayoutModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "LayoutView", - "align_content": null, - "align_items": null, - "align_self": null, - "border": null, - "bottom": null, - "display": null, - "flex": null, - "flex_flow": null, - "grid_area": null, - "grid_auto_columns": null, - "grid_auto_flow": null, - "grid_auto_rows": null, - "grid_column": null, - "grid_gap": null, - "grid_row": null, - "grid_template_areas": null, - "grid_template_columns": null, - "grid_template_rows": null, - "height": null, - "justify_content": null, - "justify_items": null, - "left": null, - "margin": null, - "max_height": null, - "max_width": null, - "min_height": null, - "min_width": null, - "object_fit": null, - "object_position": null, - "order": null, - "overflow": null, - "overflow_x": null, - "overflow_y": null, - "padding": null, - "right": null, - "top": null, - "visibility": null, - "width": null - } - }, - "e7f471604a5a42e095d35d8ad399c6fe": { - "model_module": "@jupyter-widgets/base", - "model_module_version": "1.2.0", - "model_name": "LayoutModel", - "state": { - "_model_module": "@jupyter-widgets/base", - "_model_module_version": "1.2.0", - "_model_name": "LayoutModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "LayoutView", - "align_content": null, - "align_items": null, - "align_self": null, - "border": null, - "bottom": null, - "display": null, - "flex": null, - "flex_flow": null, - "grid_area": null, - "grid_auto_columns": null, - "grid_auto_flow": null, - "grid_auto_rows": null, - "grid_column": null, - "grid_gap": null, - "grid_row": null, - "grid_template_areas": null, - "grid_template_columns": null, - "grid_template_rows": null, - "height": null, - "justify_content": null, - "justify_items": null, - "left": null, - "margin": null, - "max_height": null, - "max_width": null, - "min_height": null, - "min_width": null, - "object_fit": null, - "object_position": null, - "order": null, - "overflow": null, - "overflow_x": null, - "overflow_y": null, - "padding": null, - "right": null, - "top": null, - "visibility": null, - "width": null - } - }, - "eba6048eb694485693656fcbf4a4f297": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "ProgressStyleModel", - "state": { - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "ProgressStyleModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "StyleView", - "bar_color": null, - "description_width": "" - } - }, - "edc37210db734d01a8afce596698bb27": { - "model_module": "@jupyter-widgets/base", - "model_module_version": "1.2.0", - "model_name": "LayoutModel", - "state": { - "_model_module": "@jupyter-widgets/base", - "_model_module_version": "1.2.0", - "_model_name": "LayoutModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "LayoutView", - "align_content": null, - "align_items": null, - "align_self": null, - "border": null, - "bottom": null, - "display": null, - "flex": null, - "flex_flow": null, - "grid_area": null, - "grid_auto_columns": null, - "grid_auto_flow": null, - "grid_auto_rows": null, - "grid_column": null, - "grid_gap": null, - "grid_row": null, - "grid_template_areas": null, - "grid_template_columns": null, - "grid_template_rows": null, - "height": null, - "justify_content": null, - "justify_items": null, - "left": null, - "margin": null, - "max_height": null, - "max_width": null, - "min_height": null, - "min_width": null, - "object_fit": null, - "object_position": null, - "order": null, - "overflow": null, - "overflow_x": null, - "overflow_y": null, - "padding": null, - "right": null, - "top": null, - "visibility": null, - "width": null - } - }, - "ef32431228f24a5498810a36b9cf6506": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "HTMLModel", - "state": { - "_dom_classes": [], - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "HTMLModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/controls", - "_view_module_version": "1.5.0", - "_view_name": "HTMLView", - "description": "", - "description_tooltip": null, - "layout": "IPY_MODEL_5776e818d9d34e009f95833056522876", - "placeholder": "​", - "style": "IPY_MODEL_06b1a069317041c8a9174c14fdc867bc", - "value": " 2/2 [00:17<00:00,  7.35s/it]" - } - }, - "f14106c7f60f411199acf47f530443fd": { - "model_module": "@jupyter-widgets/base", - "model_module_version": "1.2.0", - "model_name": "LayoutModel", - "state": { - "_model_module": "@jupyter-widgets/base", - "_model_module_version": "1.2.0", - "_model_name": "LayoutModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "LayoutView", - "align_content": null, - "align_items": null, - "align_self": null, - "border": null, - "bottom": null, - "display": null, - "flex": null, - "flex_flow": null, - "grid_area": null, - "grid_auto_columns": null, - "grid_auto_flow": null, - "grid_auto_rows": null, - "grid_column": null, - "grid_gap": null, - "grid_row": null, - "grid_template_areas": null, - "grid_template_columns": null, - "grid_template_rows": null, - "height": null, - "justify_content": null, - "justify_items": null, - "left": null, - "margin": null, - "max_height": null, - "max_width": null, - "min_height": null, - "min_width": null, - "object_fit": null, - "object_position": null, - "order": null, - "overflow": null, - "overflow_x": null, - "overflow_y": null, - "padding": null, - "right": null, - "top": null, - "visibility": null, - "width": null - } - }, - "fa0c528cca744cff8da0a4fa21fdb4b5": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "HTMLModel", - "state": { - "_dom_classes": [], - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "HTMLModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/controls", - "_view_module_version": "1.5.0", - "_view_name": "HTMLView", - "description": "", - "description_tooltip": null, - "layout": "IPY_MODEL_e5a6d300bbf441b8904aa9afb89e6f31", - "placeholder": "​", - "style": "IPY_MODEL_88226abe35534278bbd427d8eff0f5f8", - "value": "model.safetensors.index.json: 100%" - } - }, - "fa9cf3e72280417d8711ef7227a95d34": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "FloatProgressModel", - "state": { - "_dom_classes": [], - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "FloatProgressModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/controls", - "_view_module_version": "1.5.0", - "_view_name": "ProgressView", - "bar_style": "success", - "description": "", - "description_tooltip": null, - "layout": "IPY_MODEL_edc37210db734d01a8afce596698bb27", - "max": 4945242264, - "min": 0, - "orientation": "horizontal", - "style": "IPY_MODEL_eba6048eb694485693656fcbf4a4f297", - "value": 4945242264 - } - }, - "faf25fd219f24bdbaa2e3202548c97d9": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "DescriptionStyleModel", - "state": { - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "DescriptionStyleModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "StyleView", - "description_width": "" - } - }, - "fddbae43ce7f477cafaff89b81e47fc7": { - "model_module": "@jupyter-widgets/base", - "model_module_version": "1.2.0", - "model_name": "LayoutModel", - "state": { - "_model_module": "@jupyter-widgets/base", - "_model_module_version": "1.2.0", - "_model_name": "LayoutModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "LayoutView", - "align_content": null, - "align_items": null, - "align_self": null, - "border": null, - "bottom": null, - "display": null, - "flex": null, - "flex_flow": null, - "grid_area": null, - "grid_auto_columns": null, - "grid_auto_flow": null, - "grid_auto_rows": null, - "grid_column": null, - "grid_gap": null, - "grid_row": null, - "grid_template_areas": null, - "grid_template_columns": null, - "grid_template_rows": null, - "height": null, - "justify_content": null, - "justify_items": null, - "left": null, - "margin": null, - "max_height": null, - "max_width": null, - "min_height": null, - "min_width": null, - "object_fit": null, - "object_position": null, - "order": null, - "overflow": null, - "overflow_x": null, - "overflow_y": null, - "padding": null, - "right": null, - "top": null, - "visibility": null, - "width": null - } - }, - "feb438afda6b4c148a3a62ee7e03da74": { - "model_module": "@jupyter-widgets/controls", - "model_module_version": "1.5.0", - "model_name": "ProgressStyleModel", - "state": { - "_model_module": "@jupyter-widgets/controls", - "_model_module_version": "1.5.0", - "_model_name": "ProgressStyleModel", - "_view_count": null, - "_view_module": "@jupyter-widgets/base", - "_view_module_version": "1.2.0", - "_view_name": "StyleView", - "bar_color": null, - "description_width": "" - } - } - } - } - }, - "nbformat": 4, - "nbformat_minor": 0 -}