← Back to Curriculum

Week 2: Phase I Capstone + Attention Begins

Days 8–14 · 17.5 hours

This week bridges Phase I and Phase II. Days 8-9 consolidate your DL foundations with a hands-on mini-project and checkpoint. Then Days 10-14 begin the attention revolution — from Bahdanau's "learning to align" to the full transformer architecture.

Daily Lessons

Day Topic Phase Focus
8 Phase I Mini-Project I Consolidation
9 Phase I Checkpoint I Self-assessment
10 Bahdanau Attention II The attention mechanism is born
11 Scaled Dot-Product Attention II Q, K, V formulation
12 Multi-Head Attention II Parallel attention heads
13 Positional Encoding II Injecting sequence order
14 The Full Transformer II "Attention Is All You Need"

Key Concepts

  • Phase I capstone: build an autoencoder to experience information compression firsthand
  • Phase I checkpoint: verify understanding before moving forward
  • Bahdanau attention: learn to look at the relevant parts of the input for each output
  • The attention equation: $\text{Attention}(Q, K, V) = \text{softmax}(QK^\top / \sqrt{d_k})V$
  • Multi-head attention: run parallel attention for different aspects of meaning
  • The full transformer: attention + FFN + residuals, stacked

Study Notes References