Subcategory
Derivatives, the chain rule and convexity — the machinery that turns a loss function into something you can actually minimise.
A model is only as trainable as its loss function is differentiable. This category takes that seriously: gradients as geometry, the chain rule as the load-bearing idea of backpropagation, convexity as the property that decides whether "converges" means anything, and constrained optimisation for the problems that will not sit still as an unconstrained one.
Nothing here is specific to machine learning yet — that specificity starts in the next two categories, once this vocabulary is in place.
Depth would be useless without an efficient way to compute how the loss depends on an early layer's parameters. This post treats the chain rule not as a calculus exercise but as the load-bearing idea that makes an arbitrarily deep composition of functions differentiable in practice.
Convexity is the property that makes "gradient descent converges to the best answer" a theorem rather than a hope. This post defines it precisely, shows what breaks in non-convex landscapes, and surveys the results explaining why non-convex training still tends to work in practice.
Regularisation, margin maximisation and several classical models are all constrained optimisation problems in disguise. This post derives the method of Lagrange multipliers from geometric first principles, and shows how a constrained problem becomes an unconstrained one with an extra variable.