6 ms·

Very heartening to hear. You think some of us tired enterprise/product devs that come from non-traditional (read: didn't study CS) background can make it? Any r

Very heartening to hear. You think some of us tired enterprise/product devs that come from non-traditional (read: didn't study CS) background can make it? Any recommended starting resources? I've heard supposedly good things about fast.ai as a "guided tour".

sillysaurusx 6y ago
Absolutely. I didn't study CS. What you need is determination. There's no substitute for this. If you have that, there are all kinds of resources. Here are a few... Resource 1: a community. We've set up a discord server for AI dev. It has 360 users. At any given time, around ~50 people are online, of which ~10 are skilled devs. Come join! https://discordapp.com/invite/x52Xz3y https://discordapp.com/invite/x52Xz3y Resource 2: Find something fun for you, and pursue that. I like generative AI, so for me that's been GPT-2 and StyleGAN. Gwern has some lovely tutorial-type articles on both. GPT-2: https://www.gwern.net/GPT-2 https://www.gwern.net/GPT-2 StyleGAN: https://www.gwern.net/Faces https://www.gwern.net/Faces Peter Baylies' StyleGAN tutorial notebook is a hands-on resource. This was actually how I started, nearly a year ago. github: https://github.com/pbaylies/stylegan-encoder https://github.com/pbaylies/stylegan-encoder notebook: https://colab.research.google.com/drive/179SPYbBC8pKDxVRjZepIvIma8SITPJvU?usp=sharing https://colab.research.google.com/drive/179SPYbBC8pKDxVRjZep... (The original notebook was broken; this is a copy I've updated with some fixes.) Lastly, follow a bunch of AI people on twitter. Here are a few to get you started: https://twitter.com/i/lists/1160386581850730496/members https://twitter.com/i/lists/1160386581850730496/members The reason to follow them is, whenever you see something that seems interesting or fun, tweet at them and say so! Ask questions. Ask how to get started. Everyone is shockingly nice and helpful. My theory is, the software is so crude and often hard to use, that we all like to celebrate together whenever one of us gets it working, and we're happy to share that knowledge however we can. (Twitter is a bit chaotic right now due to world events, but I imagine it might return to normal within a couple weeks.) And yes, you're right about fast.ai and other courses. You can go that route if you like it. I found it more exciting to dive into the deep end, though, and try to tinker with stuff.
- YeGoblynQueenne 6y ago
  >> Absolutely. I didn't study CS. But, have you achieved something singificant in the context of deep learning, as per your earlier comment? There's no information about that in your profile and a cursory glance at ddg and google results for "shawn presser" doesn't turn up anything very relevant. So, I have to ask: without having studied CS, what contributions have you made in deep learning that are widely recognised? I hope you agree this is a reasonable question to ask, and that you are not offended by it. Otherwise, I apologise because it's not my intention to offend you.
  - sillysaurusx 6y ago
    Certainly! Some highlights: - A Newsweek article https://www.newsweek.com/openai-text-generator-gpt-2-video-game-walkthrough-most-tedious-1488334 https://www.newsweek.com/openai-text-generator-gpt-2-video-g... - GPT-2 chess https://www.theregister.com/2020/01/10/gpt2_chess/ https://www.theregister.com/2020/01/10/gpt2_chess/ - ... which DeepMind referenced: https://twitter.com/theshawwn/status/1226916484938530819 https://twitter.com/theshawwn/status/1226916484938530819 - GPT-2 music https://soundcloud.com/theshawwn/sets/ai-generated-videogame-music https://soundcloud.com/theshawwn/sets/ai-generated-videogame... - Swarm training (WIP) https://www.docdroid.net/faDq8Bu/swarm-training-v01a.pdf https://www.docdroid.net/faDq8Bu/swarm-training-v01a.pdf
    - YeGoblynQueenne 6y ago
      To be honest, and again without having an intention to be harsh, but those are not what I'd call "widely recognised contributions to deep learning". They're mainly articles in the lay press and a honourable mention in a DeepMind blog post. They certainly sound like contributions to Shawn Presser's reputation, but "contributions to deep learning"? To clarify, what I was hoping to see is, at best, an article published at a reputable venue for AI research, a conference or a journal, or at a minimum an arxiv article that at least looks like it was meant to be submitted to a conference or journal. And at worst, a software tool that can be used in deep learning research. But it seems to me that your achievements are mainly having fun with and in one case finding an interesting use for tools that are already available. Again, I'm not trying to be harsh, neither do I want to say that all this is not worth the trouble. But it should not be held up as an example of what people can achieve without studying CS. Because, I think you'll agree, they are kind of underwhelming when compared to what people routinely achieve who have studied CS.
leesec 6y ago
Definitely try fast.ai, I also have no formal CS background, am bad at math, etc... and with a little hard work I was able to keep up and do things I never thought possible. You will truly be blown away after even just a month of the class as to what you understand and can build. It is not a domain solely for geniuses, roll up your sleeves and you can be great.
- ammanley 6y ago
  Thanks a lot. What was the most fun you've had with it so far, if I may ask?
  - leesec 6y ago
    Admittedly, I took the first iteration like 4 years ago, and I've heard it's much improved since then, but the fun part is they get you building real things right away! You build an image classifier in the first lesson. With other ML resources I find they get bogged down trying to explain the concepts/details, and I would usually lose interest before getting to the implementation of things.
    - ammanley 6y ago
      They've also added a "theory" portion after the practical, which I think is a great way to solidify the concepts scaffolded in building the thing initially. Thank you for your input.
YeGoblynQueenne 6y ago
May I ask, why are you interested in deep learning? What are you trying to achieve? If you don't have a traditional background in CS, might you not be better served by working to acquire such a background first, before thinking of making any contributions to more advanced sub-fields of CS? For instance, I'm sure that self-studying some fundamental subjects in CS, like formal languages and automata and complexity theory, will give you important tools to tackle any CS-related problem, be it programming a more efficient deep learning implementation or really, anything else you like. Spending the same effort to get into deep learning instead, will most likely leave you with only superficial knowledge that will not transfer to other subjects.
- ammanley 6y ago
  I think its safe to say I'm looking to see if this field is something I'd be interested in pursuing deeper, and get a feel for the types of problems that might or might not put bread on the table should I go further. I have no idea if I would or would not be better served by a traditional CS background in this case, so part of this is to find out if that is or is not the case.
  - YeGoblynQueenne 6y ago
    If I understand correctly, your main interest is in finding ways to up your game when it comes to placing yourself professionally? In that case it's very difficult to go wrong with acquiring a traditional CS background. There is really nothing you can do with computers that will not benefit from a CS background, including anything that may have to do with deep learning etc. On the other hand, learing about deep learning will only help you when it comes to working with deep learning. So, regardless of whether getting into deep learning may help "put bread on the table", getting a well-rounded CS education, certainly will.

sillysaurusx 6y ago

Absolutely. I didn't study CS. What you need is determination. There's no substitute for this. If you have that, there are all kinds of resources. Here are a few... Resource 1: a community. We've set up a discord server for AI dev. It has 360 users. At any given time, around ~50 people are online, of which ~10 are skilled devs. Come join! https://discordapp.com/invite/x52Xz3y https://discordapp.com/invite/x52Xz3y Resource 2: Find something fun for you, and pursue that. I like generative AI, so for me that's been GPT-2 and StyleGAN. Gwern has some lovely tutorial-type articles on both. GPT-2: https://www.gwern.net/GPT-2 https://www.gwern.net/GPT-2 StyleGAN: https://www.gwern.net/Faces https://www.gwern.net/Faces Peter Baylies' StyleGAN tutorial notebook is a hands-on resource. This was actually how I started, nearly a year ago. github: https://github.com/pbaylies/stylegan-encoder https://github.com/pbaylies/stylegan-encoder notebook: https://colab.research.google.com/drive/179SPYbBC8pKDxVRjZepIvIma8SITPJvU?usp=sharing https://colab.research.google.com/drive/179SPYbBC8pKDxVRjZep... (The original notebook was broken; this is a copy I've updated with some fixes.) Lastly, follow a bunch of AI people on twitter. Here are a few to get you started: https://twitter.com/i/lists/1160386581850730496/members https://twitter.com/i/lists/1160386581850730496/members The reason to follow them is, whenever you see something that seems interesting or fun, tweet at them and say so! Ask questions. Ask how to get started. Everyone is shockingly nice and helpful. My theory is, the software is so crude and often hard to use, that we all like to celebrate together whenever one of us gets it working, and we're happy to share that knowledge however we can. (Twitter is a bit chaotic right now due to world events, but I imagine it might return to normal within a couple weeks.) And yes, you're right about fast.ai and other courses. You can go that route if you like it. I found it more exciting to dive into the deep end, though, and try to tinker with stuff.

YeGoblynQueenne 6y ago
>> Absolutely. I didn't study CS. But, have you achieved something singificant in the context of deep learning, as per your earlier comment? There's no information about that in your profile and a cursory glance at ddg and google results for "shawn presser" doesn't turn up anything very relevant. So, I have to ask: without having studied CS, what contributions have you made in deep learning that are widely recognised? I hope you agree this is a reasonable question to ask, and that you are not offended by it. Otherwise, I apologise because it's not my intention to offend you.
- sillysaurusx 6y ago
  Certainly! Some highlights: - A Newsweek article https://www.newsweek.com/openai-text-generator-gpt-2-video-game-walkthrough-most-tedious-1488334 https://www.newsweek.com/openai-text-generator-gpt-2-video-g... - GPT-2 chess https://www.theregister.com/2020/01/10/gpt2_chess/ https://www.theregister.com/2020/01/10/gpt2_chess/ - ... which DeepMind referenced: https://twitter.com/theshawwn/status/1226916484938530819 https://twitter.com/theshawwn/status/1226916484938530819 - GPT-2 music https://soundcloud.com/theshawwn/sets/ai-generated-videogame-music https://soundcloud.com/theshawwn/sets/ai-generated-videogame... - Swarm training (WIP) https://www.docdroid.net/faDq8Bu/swarm-training-v01a.pdf https://www.docdroid.net/faDq8Bu/swarm-training-v01a.pdf
  - YeGoblynQueenne 6y ago
    To be honest, and again without having an intention to be harsh, but those are not what I'd call "widely recognised contributions to deep learning". They're mainly articles in the lay press and a honourable mention in a DeepMind blog post. They certainly sound like contributions to Shawn Presser's reputation, but "contributions to deep learning"? To clarify, what I was hoping to see is, at best, an article published at a reputable venue for AI research, a conference or a journal, or at a minimum an arxiv article that at least looks like it was meant to be submitted to a conference or journal. And at worst, a software tool that can be used in deep learning research. But it seems to me that your achievements are mainly having fun with and in one case finding an interesting use for tools that are already available. Again, I'm not trying to be harsh, neither do I want to say that all this is not worth the trouble. But it should not be held up as an example of what people can achieve without studying CS. Because, I think you'll agree, they are kind of underwhelming when compared to what people routinely achieve who have studied CS.

leesec 6y ago

Definitely try fast.ai, I also have no formal CS background, am bad at math, etc... and with a little hard work I was able to keep up and do things I never thought possible. You will truly be blown away after even just a month of the class as to what you understand and can build. It is not a domain solely for geniuses, roll up your sleeves and you can be great.

ammanley 6y ago
Thanks a lot. What was the most fun you've had with it so far, if I may ask?
- leesec 6y ago
  Admittedly, I took the first iteration like 4 years ago, and I've heard it's much improved since then, but the fun part is they get you building real things right away! You build an image classifier in the first lesson. With other ML resources I find they get bogged down trying to explain the concepts/details, and I would usually lose interest before getting to the implementation of things.
  - ammanley 6y ago
    They've also added a "theory" portion after the practical, which I think is a great way to solidify the concepts scaffolded in building the thing initially. Thank you for your input.

YeGoblynQueenne 6y ago

May I ask, why are you interested in deep learning? What are you trying to achieve? If you don't have a traditional background in CS, might you not be better served by working to acquire such a background first, before thinking of making any contributions to more advanced sub-fields of CS? For instance, I'm sure that self-studying some fundamental subjects in CS, like formal languages and automata and complexity theory, will give you important tools to tackle any CS-related problem, be it programming a more efficient deep learning implementation or really, anything else you like. Spending the same effort to get into deep learning instead, will most likely leave you with only superficial knowledge that will not transfer to other subjects.

ammanley 6y ago
I think its safe to say I'm looking to see if this field is something I'd be interested in pursuing deeper, and get a feel for the types of problems that might or might not put bread on the table should I go further. I have no idea if I would or would not be better served by a traditional CS background in this case, so part of this is to find out if that is or is not the case.
- YeGoblynQueenne 6y ago
  If I understand correctly, your main interest is in finding ways to up your game when it comes to placing yourself professionally? In that case it's very difficult to go wrong with acquiring a traditional CS background. There is really nothing you can do with computers that will not benefit from a CS background, including anything that may have to do with deep learning etc. On the other hand, learing about deep learning will only help you when it comes to working with deep learning. So, regardless of whether getting into deep learning may help "put bread on the table", getting a well-rounded CS education, certainly will.