← Back to Home
From LLM to VLA
112-day curriculum — from deep learning foundations to Vision-Language-Action models for robotics
📅 16 weeks
⏱ 280 hours
🎯 7 phases
🤖 112 daily lessons
Phases
Phase I
DL Foundations: Backprop → Information Theory
Days 1–9
Phase II
Attention & Transformers: The Architecture Revolution
Days 10–30
Phase III
LLM Engineering: Training, Alignment & Deployment
Days 31–44
Phase IV
Vision Transformers: From ViT to Video Understanding
Days 45–58
Phase V
Vision-Language Models: CLIP → LLaVA → Fine-tuning
Days 59–70
Phase VI
RL, Diffusion & Imitation Learning for Robotics
Days 71–91
Phase VII
VLA Architectures: RT-1 → π₀ → Deployment
Days 92–112
Weekly Schedule
Phase I
Week 1: DL Foundations
Day 1: Computation Graphs & Backpropagation
Day 2: CNNs & ResNets
Day 3: RNN/LSTM Essentials
Day 4: Seq2Seq & The Bottleneck
Day 5: Information Theory & Compression
Day 6: Embeddings & Representation Learning
Day 7: Training Stability Cookbook
Phase I–II
Week 2: Attention & Transformers
Day 8: Phase I Mini-Project
Day 9: Phase I Checkpoint
Day 10: Bahdanau Attention
Day 11: Scaled Dot-Product Attention
Day 12: Multi-Head Attention
Day 13: Positional Encoding
Day 14: The Full Transformer
Phase II
Week 3: Variants & GPT
Day 15: Training a Transformer
Day 16: Stop & Reflect #1
Day 17: Efficient Attention
Day 18: KV Cache
Day 19: Normalization & Activations
Day 20: Mixture of Experts
Day 21: BERT & Masked LM
Phase II
Week 4: Scaling & Decoders
Day 22: Tokenization Deep Dive
Day 23: GPT & nanoGPT
Day 24: nanoGPT Ablations
Day 25: Scaling Laws & Emergence
Day 26: Stop & Reflect #2
Day 27: Sampling & Generation
Day 28: T5 & Encoder-Decoder LMs
Phase II–III
Week 5: LLM Training
Day 29: Phase II Capstone Day 1
Day 30: Phase II Capstone Day 2
Day 31: The Modern LLM Recipe
Day 32: Supervised Fine-Tuning
Day 33: RLHF
Day 34: DPO & Modern Alignment
Day 35: LoRA & Efficient Fine-Tuning
Phase III
Week 6: LLM Engineering
Day 36: LLM Evaluation
Day 37: Quantization & Inference
Day 38: In-Context Learning
Day 39: Long Context & Reasoning
Day 40: RAG & Tool Use
Day 41: LLM for Robotics
Day 42: Phase III Capstone Day 1
Phase III–IV
Week 7: Vision Transformers
Day 43: Phase III Capstone Day 2
Day 44: Phase III Capstone Day 3
Day 45: ViT
Day 46: Training ViT + DeiT
Day 47: Swin Transformer
Day 48: DINO & Self-Supervised Vision
Day 49: MAE
Phase IV
Week 8: 3D Vision & Video
Day 50: Stop & Reflect #3
Day 51: 3D Vision & Depth
Day 52: Point Clouds & 3D
Day 53: Video Understanding Day 1
Day 54: Video Understanding Day 2
Day 55: DETR + Florence-2 + SAM 2
Day 56: Vision-Language Bridge
Phase IV–V
Week 9: VLMs — CLIP to LLaVA
Day 57: Phase IV Capstone Day 1
Day 58: Phase IV Capstone Day 2
Day 59: CLIP
Day 60: CLIP Internals + SigLIP
Day 61: Flamingo & BLIP-2
Day 62: LLaVA
Day 63: PaLI & CoCa
Phase V
Week 10: VLM Practice
Day 64: Open VLM Landscape
Day 65: Spatial Grounding
Day 66: Stop & Reflect #4
Day 67: Phase V Capstone Day 1
Day 68: Phase V Capstone Day 2
Day 69: VLM Fine-tuning Day 1
Day 70: VLM Fine-tuning Day 2
Phase VI
Week 11: RL & Diffusion
Day 71: RL Foundations Day 1
Day 72: RL Foundations Day 2
Day 73: PPO & RLHF Connection
Day 74: Diffusion — DDPM
Day 75: Diffusion — DDIM + CFG
Day 76: Latent Diffusion
Day 77: Flow Matching
Phase VI
Week 12: Imitation Learning
Day 78: Imitation Learning
Day 79: ACT
Day 80: Decision Transformer
Day 81: Diffusion Policy
Day 82: Action Representations
Day 83: Action Tokenization
Day 84: Stop & Reflect #5
Phase VI
Week 13: Data & Evaluation
Day 85: Data Collection Part 1
Day 86: Data Collection Part 2
Day 87: Policy Evaluation
Day 88: Policy Debugging
Day 89: Phase VI Capstone Day 1
Day 90: Phase VI Capstone Day 2
Day 91: Phase VI Capstone Day 3
Phase VII
Week 14: VLA Architectures
Day 92: RT-1
Day 93: RT-2
Day 94: Octo
Day 95: OpenVLA
Day 96: π₀
Day 97: π₀.5
Day 98: GR-2 & GROOT N1
Phase VII
Week 15: Training & Transfer
Day 99: GR-2 Deep Dive
Day 100: VLA Training Recipes
Day 101: Sim-to-Real Day 1
Day 102: Sim-to-Real Day 2
Day 103: Hybrid VLAs Day 1
Day 104: Hybrid VLAs Day 2
Day 105: Stop & Reflect #6
Phase VII
Week 16: Deployment & Capstone
Day 106: World Models
Day 107: Deployment — Latency & Compute
Day 108: Deployment — Safety & Monitoring
Day 109: Deployment — Fleet & Learning
Day 110: Final Capstone Day 1
Day 111: Final Capstone Day 2
Day 112: Final Capstone Day 3