Not really ! Activation functions are usually non-linear. What happens is we turn our numbers into their counterparts in another group (abstract group) and work

Not really ! Activation functions are usually non-linear. What happens is we turn our numbers into their counterparts in another group (abstract group) and work there (at a high-level).