rasbt/LLMs-from-scratch
★ 106,873 stars
Covered in week 2026-W15 · the 2026-04 report
Implement a ChatGPT-like LLM in PyTorch from scratch, step by step
What LLMs-from-scratch is
This repository contains the code for developing, pretraining, and finetuning a GPT-like LLM and is the official code repository for the book Build a Large Language Model (From Scratch).
From the repository README · read it on GitHub →
Transcript
LLM courses cost up to $200. This one is free. LLMs from scratch teaches you to build a chat GPT-like LLM in PyTorch from scratch step-by-step. You get seven chapters covering attention mechanisms, GPT implementation, pre-training, and instruction fine-tuning. The repo has over 90,000 stars and includes bonus material like LoRA fine-tuning and alternative architectures. Paid courses charge 50 to $200 for similar content. Star it if this saved you money.
Auto-generated captions — a product name may be transcribed imprecisely.