Building a Neural Network Layer in Python — From Single Neurons to Real Structure

So far, we have built the two fundamental ingredients of a neural network:

A neuron that computes a weighted sum plus bias
An activation function that introduces non-linearity

At this stage, each neuron works in isolation.

But neural networks are not powerful because of individual neurons.
They are powerful because many neurons work together in layers.

In this article, we will:

Define what a layer really is
Build a fully connected (dense) layer in pure Python
See how data flows through multiple neurons at once
Prepare the ground for full forward propagation

This is where a neural network starts to take shape.

What Is a Layer, Conceptually?

A layer is nothing more than:

A collection of neurons
All receiving the same input
Each with its own weights and bias
Each producing its own output

If a single neuron maps: $\mathbb{R}^n \rightarrow \mathbb{R}$

Then a layer with k neurons maps: $\mathbb{R}^n \rightarrow \mathbb{R}^k$

In practical terms:

Input: a vector
Output: another vector (longer, shorter, or same size)

Dense (Fully Connected) Layers

The most common layer type is the dense layer.

“Fully connected” means:

Every input connects to every neuron
No input is skipped
No shortcuts exist

This is the layer we will implement.

Representing a Layer in Code

A layer needs to store:

A weight vector per neuron
A bias per neuron
An activation function

A clean way to think about it:

A layer is a list of neurons
Each neuron has its own parameters

Step 1: Reusing the Neuron Logic

From the previous articles, we already have:

def neuron_output(inputs, weights, bias):
    total = 0.0
    for x, w in zip(inputs, weights):
        total += x * w
    total += bias
    return total

And an activation function, for example ReLU:

def relu(z):
    return max(0.0, z)

Step 2: Defining a Layer

Now we combine multiple neurons into a single function.

def dense_layer(inputs, weights_list, bias_list, activation):
    outputs = []
    for weights, bias in zip(weights_list, bias_list):
        z = neuron_output(inputs, weights, bias)
        a = activation(z)
        outputs.append(a)
    return outputs

If ‘activation(z)’ is confusing – please read the sidebar at the end of this article.

Ok.

Let’s unpack what this (‘dense_layer’) does.

What Happens Inside the Layer

For each neuron:

Take the same input vector
Apply neuron computation
Apply activation
Store the output

The result is a vector of activations—one per neuron.

Example: A Layer With 3 Neurons

inputs = [1.0, 2.0]
weights_list = [
    [0.5, -1.0],   # Neuron 1
    [1.0, 1.0],    # Neuron 2
    [-0.5, 2.0]    # Neuron 3
]
bias_list = [0.0, 1.0, -1.0]
outputs = dense_layer(inputs, weights_list, bias_list, relu)
print(outputs)

Interpretation

Each neuron:

Sees the same inputs
Computes a different linear combination
Activates differently

This diversity is what gives layers expressive power.

A Layer Is a Function Transformer

This is an important mental model:

A layer transforms one vector into another vector.

It reshapes the data:

Expands it
Compresses it
Rotates it
Warps it (with non-linearity)

Neural networks learn by stacking many such transformations.

Matrix View (Without Using Matrices Yet)

Mathematically, a dense layer computes: $\mathbf{z} = W \mathbf{x} + \mathbf{b}$

Where:

$W$ W is a weight matrix
$\mathbf{x}$ x is the input vector
$\mathbf{b}$ b is the bias vector

We are doing this explicitly, neuron by neuron, so the logic is transparent.

Later, we will replace loops with matrix operations—but only after understanding them.

Common Beginner Mistakes

Mistake 1: Different inputs per neuron
→ All neurons in a layer receive the same input vector.

Mistake 2: Forgetting biases per neuron
→ Each neuron must have its own bias.

Mistake 3: Applying activation once per layer
→ Activation is applied per neuron, not globally.

What We Have Built So Far

At this point, we have:

A neuron
Activation functions
A fully connected layer
Vector-to-vector transformation

This is already a legitimate neural network building block.

But something is still missing.

The output of one layer must become the input of the next.

That is forward propagation.

What’s Next in the Series

In Article #4, we will:

Chain multiple layers together
Implement forward propagation
Build a complete multi-layer neural network
Run data from input to output step by step

This is where the network starts producing predictions.

GitHub Code

All code for this article is available here:

👉 https://github.com/Benard-Kemp/Building-a-Neural-Network-Layer-in-Python

Each article adds one new file and builds on the previous ones.

Series Progress

You are reading:

Neural Networks From Scratch (Pure Python)
✔ Article #1 — What a Neuron Really Computes
✔ Article #2 — Activation Functions
✔ Article #3 — Building a Layer
➡ Article #4 — Forward Propagation Explained

Sidebar: Where Does `activation(z)` Come From?

You may notice this line inside the layer code:

a = activation(z)

At first glance, this can be confusing.

Where is activation defined?
Why does Python allow this to work?

Let’s clarify this before moving on.

`activation` Is a Function Passed Into the Layer

In Python, functions are first-class objects.
This means a function can be passed around just like any other value.

Earlier, we defined activation functions such as:

def relu(z):
    return max(0.0, z)

When we define a layer, we pass the function itself:

(weights_list, bias_list, relu)

Notice that we pass relu, not relu().

This means:

Nothing is executed yet
The layer simply receives a reference to the function

What Happens Inside the Layer

When this line runs:

a = activation(z)

Python treats it exactly as if we had written:

a = relu(z)

The layer does not know which activation it is using — it simply calls the function it was given.

Why This Design Is Intentional

This approach allows us to:

Reuse the same layer logic
Swap activation functions easily
Avoid hard-coded conditionals
Match how real neural-network frameworks work internally

Once you understand this pattern, the rest of the network becomes much easier to follow.

(end of sidebar)