4 ms·

I'm in the same boat kinda, but even more outdated on some parts. I had some AI specialization back in college, but that was before deep learning was even a thi

I'm in the same boat kinda, but even more outdated on some parts. I had some AI specialization back in college, but that was before deep learning was even a thing, so we did stuff like self-organizing maps and evolutionary algorithms, but it wasn't really all that useful for much back then. Been following deep learning from the sidelines, but for work my AI work has been restricted to GOFAI until recently. Some of the stuff I'm currently reading/watching or have recently Practical Deep Learning, though it sounds like you may know this stuff already (https://course.fast.ai/ https://course.fast.ai/) Practical Deep learning part 2, more about diffusion models. Full course coming early next year (https://www.fast.ai/posts/part2-2022-preview.html https://www.fast.ai/posts/part2-2022-preview.html) Hugging Face course (https://huggingface.co/course/chapter1/1 https://huggingface.co/course/chapter1/1) Diffusion models from hugging face https://huggingface.co/blog/annotated-diffusion https://huggingface.co/blog/annotated-diffusion https://huggingface.co/docs/diffusers/index https://huggingface.co/docs/diffusers/index Andrej Karpathy's Neural Networks: Zero to Hero. He goes from the basics up to how GPT etc, so you can start wherever suits you (https://www.youtube.com/playlist?list=PLAqhIrjkxbuWI23v9cThsA9GvCAUhRvKZ https://www.youtube.com/playlist?list=PLAqhIrjkxbuWI23v9cThs...) 3blue1brown's videos. I've found all his videos on neural networks and math worth watching, even for stuff that I already know, he sometimes has some new perspectives and nice animations. brilliant.org. Nice math refresher and the courses there are almost like fun little games.

Railsify 4y ago
Have you had a look at https://nnfs.io/ https://nnfs.io/ ? I bought the book and am gearing up to start working through it, I would be interested to know your thoughts. Generally I want to chart a personal curriculum from data engineer to practical application of modern AI to real business problems.
- macrolime 4y ago
  I hadn't seen that book before. Looks like it's a companion to this YouTube series that looks pretty interesting https://youtube.com/playlist?list=PLQVvvaa0QuDcjD5BAw2DxE6OF2tius3V3 https://youtube.com/playlist?list=PLQVvvaa0QuDcjD5BAw2DxE6OF... I will check it out for sure
  - Railsify 4y ago
    I'd be interested to know what you think, it's quite a time investment to study the entire book.
deleted 4y ago
[deleted]
macrolime 4y ago
The neuromatch computational neuroscience course also seems quite interesting, though maybe less of practical use. https://compneuro.neuromatch.io/ https://compneuro.neuromatch.io/ Recent research like "Relating transformers to models and neural representations of the hippocampal formation" might make it more relevant though (https://arxiv.org/abs/2112.04035v2 https://arxiv.org/abs/2112.04035v2) quote from the abstract of that paper: "Many deep neural network architectures loosely based on brain networks have recently been shown to replicate neural firing patterns observed in the brain. One of the most exciting and promising novel architectures, the Transformer neural network, was developed without the brain in mind. In this work, we show that transformers, when equipped with recurrent position encodings, replicate the precisely tuned spatial representations of the hippocampal formation; most notably place and grid cells. Furthermore, we show that this result is no surprise since it is closely related to current hippocampal models from neuroscience. We additionally show the transformer version offers dramatic performance gains over the neuroscience version. This work continues to bind computations of artificial and brain networks, offers a novel understanding of the hippocampal-cortical interaction, and suggests how wider cortical areas may perform complex tasks beyond current neuroscience models such as language comprehension."

Railsify 4y ago

Have you had a look at https://nnfs.io/ https://nnfs.io/ ? I bought the book and am gearing up to start working through it, I would be interested to know your thoughts. Generally I want to chart a personal curriculum from data engineer to practical application of modern AI to real business problems.

macrolime 4y ago
I hadn't seen that book before. Looks like it's a companion to this YouTube series that looks pretty interesting https://youtube.com/playlist?list=PLQVvvaa0QuDcjD5BAw2DxE6OF2tius3V3 https://youtube.com/playlist?list=PLQVvvaa0QuDcjD5BAw2DxE6OF... I will check it out for sure
- Railsify 4y ago
  I'd be interested to know what you think, it's quite a time investment to study the entire book.

deleted 4y ago

[deleted]

macrolime 4y ago

The neuromatch computational neuroscience course also seems quite interesting, though maybe less of practical use. https://compneuro.neuromatch.io/ https://compneuro.neuromatch.io/ Recent research like "Relating transformers to models and neural representations of the hippocampal formation" might make it more relevant though (https://arxiv.org/abs/2112.04035v2 https://arxiv.org/abs/2112.04035v2) quote from the abstract of that paper: "Many deep neural network architectures loosely based on brain networks have recently been shown to replicate neural firing patterns observed in the brain. One of the most exciting and promising novel architectures, the Transformer neural network, was developed without the brain in mind. In this work, we show that transformers, when equipped with recurrent position encodings, replicate the precisely tuned spatial representations of the hippocampal formation; most notably place and grid cells. Furthermore, we show that this result is no surprise since it is closely related to current hippocampal models from neuroscience. We additionally show the transformer version offers dramatic performance gains over the neuroscience version. This work continues to bind computations of artificial and brain networks, offers a novel understanding of the hippocampal-cortical interaction, and suggests how wider cortical areas may perform complex tasks beyond current neuroscience models such as language comprehension."