SolveWithPython

Neural Networks From Scratch in Python — What a Neuron Really Computes

Neural networks often feel mysterious because most tutorials begin after the important part has already happened. You import a framework, define a model, and call .fit()—but the core computation remains hidden.

In this series, we will do the opposite.

We will build a neural network from scratch in pure Python, without TensorFlow, PyTorch, or any machine-learning libraries. Not because frameworks are bad—but because understanding what happens under the hood is the fastest way to truly understand neural networks.

We begin at the smallest possible unit: a single neuron.

What Is a Neuron, Really?

Despite the biological inspiration, an artificial neuron is not complex.
It performs one mathematical operation:

It takes numbers as input, multiplies them by weights, adds a bias, and produces a single number as output.

That’s it.

No magic. No intelligence. Just math.

Let’s formalize this.

The Core Computation: Weighted Sum + Bias

A neuron receives:

  • Inputs: x1,x2,…,xnx_1, x_2, \dots, x_n
  • Weights: w1,w2,…,wnw_1, w_2, \dots, w_n
  • A bias: bb

The neuron computes:z=(x1⋅w1)+(x2⋅w2)+⋯+(xn⋅wn)+bz = (x_1 \cdot w_1) + (x_2 \cdot w_2) + \dots + (x_n \cdot w_n) + b

This is called the linear combination.

At this stage, we are not applying any activation function yet. That will come in the next article.

Why the Bias Matters

Without the bias, the neuron’s output would always be forced through the origin.
The bias allows the neuron to shift its decision boundary.

In practical terms:

  • Weights control how strongly inputs matter
  • Bias controls where the neuron activates

You can think of bias as a “baseline offset.”

Implementing a Neuron in Pure Python

Let’s translate the formula directly into Python.

We will not use NumPy yet. Just lists and basic arithmetic.

Python
def neuron_output(inputs, weights, bias):
    total = 0.0
    for x, w in zip(inputs, weights):
        total += x * w
    total += bias
    return total

This function is already a valid artificial neuron.

Example: Running the Neuron

Let’s test it with concrete values.

Python
inputs = [2.0, 3.0]
weights = [0.5, -1.0]
bias = 1.0
output = neuron_output(inputs, weights, bias)
print(output)

Step-by-step calculation

(2.0×0.5)+(3.0×−1.0)+1.0(2.0 \times 0.5) + (3.0 \times -1.0) + 1.0 =1.0−3.0+1.0=−1.0= 1.0 – 3.0 + 1.0 = -1.0

The neuron outputs -1.0.

At this point, the neuron is doing exactly what it is supposed to do: compute a weighted sum.

A Neuron Is Just a Function

This is a critical insight:

A neuron is simply a mathematical function that maps multiple inputs to one output.

There is no learning yet.
There is no intelligence yet.
There is no prediction yet.

Learning will come later—when we figure out how to adjust the weights and bias automatically.

Visual Interpretation

Geometrically, this neuron defines a linear function.

For two inputs, it represents a line.
For three inputs, a plane.
For higher dimensions, a hyperplane.

Neural networks work by stacking many of these simple linear transformations and introducing non-linearity between them.

But first, we must fully understand the linear part.

Common Beginner Misconception

Many beginners think the “intelligence” of a neural network comes from:

  • The number of neurons
  • The complexity of the architecture
  • The framework used

In reality, intelligence comes from:

  • How weights are updated
  • How errors are measured
  • How gradients flow backward

We will build all of that—step by step.

What We Have Built So Far

At this point, we have:

  • Implemented a real neuron
  • Understood the role of weights and bias
  • Seen how inputs turn into outputs
  • Built everything in pure Python

This neuron is already usable—but it cannot learn yet.

What’s Next in the Series

In Article #2, we will:

  • Introduce activation functions
  • Explain why a network without them cannot learn complex patterns
  • Implement ReLU and Sigmoid in pure Python
  • See how non-linearity changes everything

This is where neural networks stop being linear algebra—and start becoming powerful.

GitHub Code

All code for this article (and the entire series) is available here:

GitHub Repository:
👉 https://github.com/Benard-Kemp/Neural-Networks-From-Scratch-in-Python

Each article adds a new file, so you can follow along incrementally.

Series Roadmap

This article is part of the series:

Neural Networks From Scratch (Pure Python)

Upcoming articles include:

  • Activation functions
  • Layers
  • Forward propagation
  • Loss functions
  • Backpropagation
  • Gradient descent
  • Training a full neural network