Preface: About Deep Learning

What is deep learning?

Deep learning is a branch of machine learning, and machine learning is part of artificial intelligence (AI). Their relationship can be understood as follows:

• Artificial Intelligence: Enabling machines to demonstrate capabilities that require human intelligence (such as recognizing images or understanding language)

• Machine Learning: Enabling machines to automatically learn patterns from data rather than being explicitly programmed

• Deep Learning: A method of machine learning that uses multi-layer neural networks

        graph TD
    AI[人工智能<br/>Artificial Intelligence] --> ML[机器学习<br/>Machine Learning]
    ML --> DL[深度学习<br/>Deep Learning]
    ML --> SL[监督学习]
    ML --> UL[无监督学习]
    DL --> CV[计算机视觉]
    DL --> NLP[自然语言处理]
    

Deep learning is “deep” because its network architecture has many layers. Each layer extracts features at a different level: shallow layers detect edges and textures, while deeper layers identify object parts and semantic concepts. This hierarchical feature extraction has driven breakthroughs in image, speech, and text applications.

Black boxes and alchemy

Why call it alchemy?

The older generation of deep learning researchers often jokingly described their work as “alchemy.” Though self-deprecating, this phrase accurately captures certain characteristics of deep learning:

• Complex recipes: Learning rates, batch sizes, network architectures, regularization parameters… every choice can affect the outcome

• Uncontrollable process: Running the exact same code twice can yield slightly different results (due to random initialization and data shuffling)

• Unpredictable results: Modifications that seem theoretically sound might prove useless or even counterproductive in practice

• Experience over theory: Success often relies on “trial and error” rather than rigorous mathematical derivations

Alchemy can still be scientific

Although we joke about alchemy, this does not mean deep learning is pure mysticism. A good “alchemist” will:

• Conduct controlled experiments: Change one variable while keeping others constant to observe the impact

• Keep experiment logs: Record configurations and results for every run to establish reproducibility

• Understand underlying principles: Know why a technique works (e.g., BatchNorm aims to mitigate internal covariate shift)

• Tune hyperparameters systematically: Instead of random trial and error, prioritize by importance (learning rate > network architecture > optimizer > …)

A black box is not necessarily incomprehensible

Deep learning models are often criticized as “black boxes”—inputs go in, outputs come out, with little transparency about what happens in between. In reality, however, we have an expanding set of tools to interpret models:

• Feature visualization: Inspect which parts of an image the convolutional layers focus on

• Attention heatmaps: See which words Transformers attend to when processing text (see Transformer：注意力的极致 for details)

• Ablation experiments: Remove a specific component to see how much performance drops, gauging its importance

• Adversarial example analysis: Construct targeted inputs to probe and understand the model’s decision boundaries

These methods allow us to “open” the black box. While we may not grasp every single detail, we can at least understand which factors influence the model’s decisions.

The mindset you need

Balance theory and practice

Deep learning is a field where theory and practice are tightly coupled. If you only know theory without coding skills, you cannot validate your ideas; if you only write code without understanding theory, you will be stuck wandering in circles.

The recommended learning approach is:

• Understand principles first: Know why a technique works (e.g., Dropout is an approximation of ensemble learning)

• Implement hands-on second: Reproduce it with your own code to deepen your understanding

• Experiment last: Validate on your own datasets and observe the actual results

Build intuition

In deep learning, intuition is sometimes more important than formulas. When you see a new architecture, being able to intuitively judge “will this design work?” is often more valuable than deriving backpropagation formulas.

Ways to build intuition:

• Study many architectures: Learn the design philosophies and evolution of classic networks

• Run many experiments: Tune hyperparameters yourself to feel the impact of different choices

• Ask “why” often: Instead of just remembering “this works,” understand why it works

Remember, many “truths” in deep learning are temporary. Best practices from five years ago might be obsolete today. Only by cultivating good intuition can you stay competitive in this fast-changing field.

Stay skeptical

The deep learning community is filled with claims:

• “This architecture achieved SOTA on ImageNet”

• “Our method improved over the baseline by 5%”

• “This trick works for all tasks”

As a learner, you need to stay skeptical:

• Question results: Is the 5% gain statistically significant? Was there test-set leakage?

• Question conclusions: Does what worked on one dataset generalize to others?

• Question papers: Did the authors hide negative results? Was the baseline too weak?

• Question authority: Even if stated by leading experts, maintain independent judgment

Skepticism is not about being argumentative; it is about building truly reliable knowledge. When you can distinguish between dependable conclusions and overhyped claims, you become a qualified deep learning practitioner.

Patience and iteration

Finally, learning deep learning requires patience. Don’t expect to fully grasp everything on the first read, and don’t be discouraged if your first experiment fails.

• Re-read repeatedly: You might understand 30% on the first pass, 60% on the second, and 90% on the third

• Experiment repeatedly: Hyperparameter tuning rarely succeeds on the first try; log every result and gradually approach the optimum

• Accept failure: Many ideas ultimately prove ineffective, but that is an integral part of the learning process

Remember that even top researchers see most of their ideas fail. The difference is that they extract insights from failure and iterate quickly to the next idea.

How to use these materials

The design of these materials follows the learning philosophy outlined above:

1. Start from intuition: Establish intuitive understanding for every concept before diving into mathematical details

2. Combine theory with practice: Every theoretical point is paired with code implementation and experiments

3. Emphasize comparison and skepticism: Encourage you to question, experiment, and form your own judgment

4. Progressive depth: Transition step-by-step from foundational concepts to cutting-edge techniques

The recommended study strategy is: read through first to build a macro view, read carefully to grasp details, and validate with hands-on experiments. If you encounter tricky spots, feel free to skip them temporarily—later sections often help clarify earlier concepts.

Reading alongside the “Fish Book”

If you are reading Koki Saito’s Deep Learning from Scratch: Building with Python from First Principles (commonly known as the “Fish Book”), congratulations—you hold one of the most acclaimed introductory books in deep learning. These course materials do not replace the Fish Book; they are complementary.

What the Fish Book covers

The Fish Book consists of 8 chapters, starting from Python basics and building everything from scratch using only NumPy without external libraries:

Chapter	Content	Fish Book Approach
Chapter 1	Python Primer	Review basics, get familiar with NumPy
Chapter 2	Perceptron	Implement logic gates in Python, understand “weights + bias”
Chapter 3	Neural Networks	Handwrite sigmoid, softmax, 3-layer network
Chapter 4	Neural Network Training	Gradient descent, loss functions, numerical differentiation
Chapter 5	Backpropagation Method	Computational Graphs—the highlight of the Fish Book
Chapter 6	Training Techniques	Parameter updates (SGD/Momentum/Adam), BatchNorm, regularization
Chapter 7	Convolutional Neural Networks	im2col, manual implementation of convolution/pooling
Chapter 8	Deep Learning	Deep networks, history (AlexNet/VGG/ResNet), applications

The core philosophy of the Fish Book is: “You only truly understand what you build yourself.” It forces you to write every line of code by hand, so you know exactly what is happening under the hood when PyTorch runs .backward() or nn.Conv2d.

Connecting Fish Book Ch. 1–2 with these materials: Fish Book Chapter 1 (Python/NumPy) and Chapter 2 (Perceptrons) are very basic. These materials assume readers already know Python and NumPy. If you are unfamiliar with Python, we suggest skimming Chapter 1 of the Fish Book first; if you are already comfortable with NumPy array operations, you can skip Chapters 1–2 and read starting from Chapter 3 alongside our course materials.

How to read them together

Scenario	Recommendation
You are a complete beginner	Read Fish Book Ch. 1–3 first, then read our Math Fundamentals and Neural Network basics
You know Python but not backpropagation	Combine Fish Book Ch. 5 (Computational Graphs) with our Math Fundamentals
You can run PyTorch but don’t understand the principles	Fish Book Ch. 4–6 (gradient descent, update tricks) + matching theory in our materials to fill the gaps
You want to enter competitions or build projects	Fish Book Ch. 7 (handwritten CNN implementation) + our CNN Ablation Study and Transfer Learning for practical acceleration
You are stuck on a specific concept	The Fish Book explains with code while our materials explain with intuition and analogies—compare both

A simple reading strategy:

1. Parallel progression: Read a chapter in the Fish Book, then read the matching section in these materials

   • Fish Book Ch. 3 (Neural Networks) → 翻越两座山：CNN 三十年攀登史

   • Fish Book Ch. 5 (Backpropagation) → Core Mathematical Foundations of Deep Learning

   • Fish Book Ch. 7 (CNN) → 拆开看看：CNN 消融研究

2. Cross-validation: The Fish Book teaches you “how to build wheels,” while these materials teach you “how to drive” + “why the car is designed this way.” When you see nn.Conv2d in our materials, Fish Book Chapter 7 ensures you know how it performs im2col and matrix multiplication under the hood.

Typical learning path example (weekly plan)

If you do not have a self-study plan yet, here is an 8-week parallel progression plan requiring roughly 6–10 hours per week:

Week	Fish Book Chapter	Course Materials Chapter	Core Goal
Week 1	Chapter 1 (Python Review) Chapter 2 (Perceptron)	Core Mathematical Foundations of Deep Learning	Brush up on NumPy/Python and build your math toolkit
Week 2	Chapter 3 (Neural Networks) Chapter 4 (Neural Network Training)	Core Mathematical Foundations of Deep Learning (cont.)	Forward propagation, loss functions, gradient descent—solidify the math foundation
Week 3	Chapter 5 (Backpropagation)	Core Mathematical Foundations of Deep Learning (Backpropagation)	Computational graphs + chain rule—the toughest part, dedicated a full week to digest
Week 4	—	PyTorch 实践：把理论变成代码	With sufficient math background from the first 3 weeks, officially get hands-on with PyTorch: tensors, autograd, and training loops
Week 5	Chapter 6 (Training Techniques)	上篇：练习峰	Optimizers, BatchNorm, regularization → build your first LeNet in PyTorch
Week 6	Chapter 7 (Handwritten CNN)	拆开看看：CNN 消融研究	Compare Fish Book’s im2col internal mechanisms → dissect LeNet to verify each component’s contribution
Week 7	Chapter 8 (Deep Learning History)	下篇：真正的未登峰	Panoramic view: AlexNet → Inception → ResNet → MobileNet → EfficientNet
Week 8	—	模型部署与服务：从训练到生产 or 迁移学习与微调：站在巨人的肩膀上	Pick one: end-to-end production deployment, or transfer learning for a new task—complete your first project

Flexible adjustments: If you already know Python, Week 1 can be compressed to 2 days; if you want to study both options in Week 8, extend to Week 9.

In a nutshell

The Fish Book is “growing a neural network out of the soil,” while these materials are “a map to see the forest clearly from the hilltop.” Reading both gives you a clear sense of how deep the roots go and how far the branches reach.

---

Now that you have a preliminary understanding of deep learning, know the mindset needed, and understand how to pair these materials with classic textbooks, let’s start with math fundamentals and unveil deep learning step by step.

Happy learning and good luck with your model training!

Contributors and Revision History

View detailed revision history

• 2af6985 2026-06-14 - Heyan Zhu: feat(model-serving): add quantization and pruning chapter, restructure for general-first narrative
• dc35df2 2026-06-14 - Heyan Zhu: refactor: unify into two-chapter mountain-climbing narrative
• 7e2f65a 2026-06-12 - Heyan Zhu: docs: add fish book collab guide, nvidia driver/prime/dri3 setup, R470 branch
• 8eb3275 2026-05-09 - Heyan Zhu: feat(sequence-modeling): add new chapter on sequence modeling from RNN to Mamba
• f159d56 2026-05-08 - Heyan Zhu: fix: address complaints from the linter
• b20ef3e 2026-04-28 - Heyan Zhu: docs: update pytorch practice section with detailed explanations and code examples
• 9446c12 2026-04-26 - Heyan Zhu: docs: update index and add preface with humorous tone