So. In geometry an "affine vector" (c,v) is said to be the vector v tangent to c -- no longer a vector, a "tangent vector". In your neural network this would b

So. In geometry an "affine vector" (c,v) is said to be the vector v tangent to c -- no longer a vector, a "tangent vector". In your neural network this would be the weights tangent to each bias. If you keep c fixed Tc[V] = {(c,v) | c fixed, forall v in V} is clearly a vector space. Here's the fun part: if you tried to consider T[V] = {(c,v) | forall c in R, v in V} this is no longer a vector space. But it's a fun structure: at each c, Tc[V] is sorta like (homeomorphic) to the Cartesian product {c} x V this is a fiber bundle. For example: a torus is near each c the cartesian product of a point and a circle; a "circle squared" is a donut. Now, we don't need to restrict ourselves to c+vx -- we can consider sigmoid(c+vx), which will have tangents (derivatives) to b at each w. For fixed c we still have a vector space with the derivatives dsigmoid/dv, and varying c you have a sigmoid fiber bundle.