---
title: "What Are Neurons?"
url: "/learn/deep-learning/foundations/what-are-neurons"
published: "2026-08-04"
category: "deep-learning"
subcategory: "foundations"
series: ["from-matrices-to-neural-networks"]
status: "published"
level: "intermediate"
dependencies: ["Logistic regression and the link function", "Matrix multiplication"]
teaches: ["Reading a neuron as a familiar model rather than a biological metaphor", "Explaining why a non-linearity is structurally necessary", "Translating a layer of neurons into a single matrix multiply"]
wordCount: 169
---

## Not a brain cell

The biological metaphor is a historical accident and it costs more in confusion
than it buys in intuition. A neuron is a weighted sum followed by a
non-linearity — an equation already met in the previous category.

## What the non-linearity is for

Without it, stacking layers gains nothing: a composition of linear maps is a
linear map, so a hundred layers collapse into one. The non-linearity is what
makes depth mean something.

### Choosing one

The choice matters less than its presence, but the gradient behaviour of each
option explains a great deal about which architectures trained well and when.

## A layer is a matrix multiply

Put many neurons side by side and their weight vectors stack into a matrix. The
whole layer becomes one multiplication, which is exactly why this hardware is
the hardware.

## What it costs

Expressiveness arrives, and the closed-form solution leaves. The loss surface is
no longer convex, and fitting becomes a search rather than a calculation.