3 ms·
He played key roles in the development of backprop, ReLU, LayerNorm, dropout, GPU-assisted deep learning, including AlexNet, was the mentor of OpenAI's Chief Sc
by michael_nielsen 3y ago
He played key roles in the development of backprop, ReLU, LayerNorm, dropout, GPU-assisted deep learning, including AlexNet, was the mentor of OpenAI's Chief Scientist, and contributed many, many other things. These techniques are crucial for transformers, LLMs, generative image modelling, and many other modern applications of AI
Your post suggests that you know almost nothing about how modern deep learning originated.