SolveWithPython

Neural Networks From Scratch in Python (Part I)

Recap Hub: From Neuron to Training Loop (Articles #1–#10)

Welcome to Part I of Neural Networks From Scratch in Python.

This section of the series is designed to remove the “black box” effect. Instead of starting with frameworks, we build neural networks line by line, so you understand exactly what happens during:

  • forward propagation
  • loss calculation
  • gradient computation
  • backpropagation
  • parameter updates
  • the full training loop

If you complete Part I, you will understand what deep learning frameworks automate—and why it works.

What You’ll Know After Part I

By the end of this section, you will be able to:

  • Explain what a neuron computes (weights + bias)
  • Explain why activation functions are non-negotiable
  • Build a dense layer from multiple neurons
  • Run forward propagation through a multi-layer network
  • Choose the correct loss function for a problem
  • Understand gradients and the chain rule
  • Backpropagate through neurons and layers
  • Implement gradient descent updates
  • Build a full training loop and watch a model learn

Read Part I in Order (Recommended)

Article #1 — Neural Networks From Scratch in Python: What a Neuron Really Computes

What you learn: weighted sum, bias, and why a neuron is “just a function.”
👉 Read Article #1 — What a Neuron Really Computes

Article #2 — Activation Functions: Why a Network Without Them Cannot Learn

What you learn: why stacked linear layers stay linear; ReLU and Sigmoid basics.
👉 Read Article #2 — Activation Functions in Neural Networks

Article #3 — Building a Neural Network Layer in Python

What you learn: how a dense layer works; multiple neurons, same inputs, different outputs.
👉 Read Article #3 — Building a Neural Network Layer in Python

Article #4 — Forward Propagation: How Data Flows Through the Network

What you learn: chaining layers into a full forward pass; network outputs become real predictions.
👉 Read Article #4 — Forward Propagation in Neural Networks

Article #5 — Loss Functions: Measuring How Wrong the Network Is

What you learn: MSE vs cross-entropy; why loss is the bridge from prediction to learning.
👉 Read Article #5 — Loss Functions in Neural Networks

Article #6 — Gradients: Why Learning Requires Derivatives

What you learn: what a gradient means; how derivatives provide direction; chain rule intuition.
👉 Read Article #6 — Gradients in Neural Networks

Article #7 — Backpropagation Step by Step: Gradients for a Single Neuron

What you learn: compute gradients by hand; update weights and bias; first true learning example.
👉 Read Article #7 — Backpropagation Step by Step

Article #8 — Backpropagation With Activation Functions

What you learn: ReLU vs Sigmoid derivatives; how activations change gradient flow; vanishing gradients.
👉 Read Article #8 — Backpropagation With Activation Functions

Article #9 — Backpropagation Through a Layer

What you learn: layer-level gradients; summing input gradients; passing gradients upstream.
👉 Read Article #9 — Backpropagation Through a Layer

Article #10 — Training a Neural Network End to End: The Complete Learning Loop

What you learn: assemble everything into a working training loop; see loss drop over epochs.
👉 Read Article #10 — Training a Neural Network End to End

Part I Code Repository

All code for Part I is stored in one GitHub repository and built incrementally.

  • Each article maps to a clear set of files/functions
  • Code is intentionally readable (not optimized yet)
  • The goal is understanding first, performance later

👉 GitHub Repo (Part I) — [add link]

Suggested “Start Here” Paths

If you’re completely new

Start at #1, then read straight through.

If you already know the basics

Jump to #5 (Loss) → #6 (Gradients) → #7 (Backprop).

If you want the “full picture” fast

Read #4 (Forward) → #10 (Training Loop), then come back for details.

Part I Summary in One Sentence

Part I takes you from a single neuron to a complete training loop, fully implemented in pure Python, so neural networks stop feeling like magic.

What’s Next (Part II Preview)

Part II will focus on making your implementation more practical and scalable:

  • Vectorizing the network with NumPy (speed + clarity)
  • Debugging training (exploding/vanishing gradients, dead ReLUs)
  • Visualizing learning (loss curves, decision boundaries)
  • Regularization basics (overfitting, weight decay)
  • Mapping your scratch model to PyTorch (so frameworks finally make sense)

👉 Part II Hub Page — coming soon

Continue Reading

If you’re ready to continue now, go to:

👉 Article #1: What a Neuron Really Computes — [add link]