SmartGPT

1. This repo contains a GPT network for LLM's generated from scratch on an OpenWebCorpus dataset of reddit comments and reviews....
2. The model and environment used is created with ipykernel and cuda with creating a custom kernel, feel free to just use your default python3 kernel but with cuda for faster processing and training of data..
3. The code is inspired and modified from Andrej Karpathy's explaination of how LLM's are made. Feel free to use the pretrained model or the code to train on your own dataset!
4. As this code was trained on the OpenCorpus dataset the way to unzip all the files and make the vocab.txt is by running the training the model first followed by extract.py script then you just save the changes and generate and use your model...
5. In order for the last part to run use final_bot.py script to run and start typing prompts for the generation magic to happen....

Happy Coding!!