Byte Pair Encoding: building the GPT tokenizer with Karpathy
Reading Time: 15 minutes 👉 Useful links: Youtube lecture, minbpe GitHub repo, Colab notebook What is a tokenizer and why do we need one… Read More »Byte Pair Encoding: building the GPT tokenizer with Karpathy