Welcome to my implementation of a Transformer-based language model, inspired by Andrej Karpathy's YouTube series. This project will evolve from a basic bigram character-level language model into a sophisticated modern Transformer model, similar to GPT.
Here's the YouTube link for anyone interested: Andrej Karpathy's YouTube series
- makemore_1.ipynb: This notebook contains the initial implementation of a bigram character-level language model. Here, I explore the fundamental concepts and set the stage for more complex models in subsequent notebooks.
To get started with the project, clone the repository and navigate to the makemore_1.ipynb notebook to understand the basics of bigram character-level language modeling. Follow along with the series to see the progression and enhancements made to the model.
Ensure you have the necessary libraries installed. You can install the required packages by running:
pip install -r requirements.txt