Astana time
AI QAZAQSTANKZ · AIBeta

Smart portal about artificial intelligence and high technology in Kazakhstan

← Back to video hub

YouTube • YouTube — Andrej Karpathy

Let's build the GPT Tokenizer

Oct 14

Description

hi everyone so in this video I'd like us to cover the process of tokenization in large language models now you see here that I have a set face and that's because uh tokenization is my least favorite part of working with large language models but unfortunately it is necessary to…