This practical live-coding course by Sebastian Raschka focuses on the foundational preprocessing and representation techniques used in Large Language Models (LLMs). The training is highly implementation-oriented and helps learners understand how raw text becomes machine-readable data for transformer-based AI systems.
The course begins with setting up a Python development environment for building and experimenting with language models. Learners are introduced to essential AI and deep learning workflows required for modern LLM development.
A major focus is placed on tokenization, where learners explore how text is split into tokens before being processed by neural networks. The course explains token structures and demonstrates practical tokenization pipelines through live coding examples.
Students then learn how to convert tokens into numerical token IDs, an essential step for feeding textual information into machine learning models.
The course also introduces special context tokens used in transformer architectures for sequence boundaries, padding, masking, and contextual understanding in GPT-style systems.
Advanced sections explain Byte Pair Encoding (BPE), one of the most widely used tokenization algorithms in modern language models. Learners understand how subword tokenization improves vocabulary efficiency and model performance.
Additional lessons cover sliding-window data sampling techniques for preparing sequential training datasets used in autoregressive language modeling.
The training also explores token embeddings and positional encoding, helping learners understand how transformers represent semantic meaning and word order mathematically.
By the end of the course, learners will understand tokenization pipelines, BPE algorithms, token embeddings, positional encoding, and the foundational te
