3 ms·

> It felt like I needed a PhD and hardcore math skills I don't know if this helps more or make it worse. But I have both and getting the same feelings all the

> It felt like I needed a PhD and hardcore math skills I don't know if this helps more or make it worse. But I have both and getting the same feelings all the time. But basically you just need good statistics and linear algebra knowledge, and you will be fine (on the math side). > there’s already so many people much more smarter and advanced for me. Why even bother? That's the very definition of imposter syndrome put in a very good way. But you can ask the same for everything, not only ML. kudos on getting through it, though.

shortrounddev2 3y ago
99% of the time people say linear algebra is required for something, they mean knowledge basic operations and properties of tensors more than actual "algebra". I found this when doing computer graphics. Is that true as well here?
- SonOfLilit 3y ago
  A tiny bit more. Once in a while someone does something elegant with eigenvalues, and Transformers represent their "thoughts" in a very LinAlg way (a linear combination of orthonormal basis vectors in a very high dimension) that was very hard for the only person with no LinAlg background in my study group to grok.
- tomrod 3y ago
  Basic operations and properties of tensors are exactly what is taught in linear algebra, in addition to (in my experience) more accessible proofs. More or less through singular value decomposition and/or least squares. The core algorithms all build on top of each other. The `algebra` part of linear algebra refers to a `field`, but it might as well also be called arithmetic of tensors.
  - shortrounddev2 3y ago
    Right but I mean that people say you need linear algebra (e.g: the whole textbook), when in actuality you might need the first 3 chapters
    - tomrod 3y ago
      I think I didn't come across clearly. Eigenvalues, SVD, etc are important.
- godelski 3y ago
  > they mean knowledge basic operations and properties of tensors more than actual "algebra". You're actually describing linear algebra. A core topic is system of equations. You might see a 2-Tensor (matrix) like Ax := [[a,b],[c,d]].[x,y] and you could write it as f = ax + by; g = cx + dy. Often dimensions are implicit so it may not look like this, but it is. But that's a big part of what it is about (there's a whole lot more btw). You're absolutely using linear algebra frequently in graphics. Euler angles are a good example, you're just probably not writing them in matrix/tensor form. You will even get a tiny bit of exposure to {field,group} theory/abstract algebra via quaternions. In ML I'd say it is very similar. The typical researcher is going to have about the same math skills as the typical person studying graphics (I actually started my PhD in HPC graphics). But, and this holds for both domains, having a deeper math understanding only helps. It makes things easier to debug, gives you a better understanding of what the systems are doing, and gives you a lot of tools to solve many problems. I wouldn't ever use math as a strong barrier to entry, but I feel many get complacent with their skill level and we have been discouraging this myth that math doesn't help. Without a doubt, it does.
godelski 3y ago
I agree but at the same time if you look at how laughable many CVPR reviews are, I kinda wish the community had more math and statistics knowledge. But that might also be a different issue... FWIW, there are a lot of works that do get in deep to the mathematics of ML and I find these absolutely helpful. Anyone that says theory doesn't help practice hasn't read theory or is operating in bad faith. ML uses A LOT of math, but you just don't need it to create good and/or working models. I think the distinction is important.

shortrounddev2 3y ago

99% of the time people say linear algebra is required for something, they mean knowledge basic operations and properties of tensors more than actual "algebra". I found this when doing computer graphics. Is that true as well here?

SonOfLilit 3y ago
A tiny bit more. Once in a while someone does something elegant with eigenvalues, and Transformers represent their "thoughts" in a very LinAlg way (a linear combination of orthonormal basis vectors in a very high dimension) that was very hard for the only person with no LinAlg background in my study group to grok.
tomrod 3y ago
Basic operations and properties of tensors are exactly what is taught in linear algebra, in addition to (in my experience) more accessible proofs. More or less through singular value decomposition and/or least squares. The core algorithms all build on top of each other. The `algebra` part of linear algebra refers to a `field`, but it might as well also be called arithmetic of tensors.
- shortrounddev2 3y ago
  Right but I mean that people say you need linear algebra (e.g: the whole textbook), when in actuality you might need the first 3 chapters
  - tomrod 3y ago
    I think I didn't come across clearly. Eigenvalues, SVD, etc are important.
godelski 3y ago
> they mean knowledge basic operations and properties of tensors more than actual "algebra". You're actually describing linear algebra. A core topic is system of equations. You might see a 2-Tensor (matrix) like Ax := [[a,b],[c,d]].[x,y] and you could write it as f = ax + by; g = cx + dy. Often dimensions are implicit so it may not look like this, but it is. But that's a big part of what it is about (there's a whole lot more btw). You're absolutely using linear algebra frequently in graphics. Euler angles are a good example, you're just probably not writing them in matrix/tensor form. You will even get a tiny bit of exposure to {field,group} theory/abstract algebra via quaternions. In ML I'd say it is very similar. The typical researcher is going to have about the same math skills as the typical person studying graphics (I actually started my PhD in HPC graphics). But, and this holds for both domains, having a deeper math understanding only helps. It makes things easier to debug, gives you a better understanding of what the systems are doing, and gives you a lot of tools to solve many problems. I wouldn't ever use math as a strong barrier to entry, but I feel many get complacent with their skill level and we have been discouraging this myth that math doesn't help. Without a doubt, it does.

godelski 3y ago

I agree but at the same time if you look at how laughable many CVPR reviews are, I kinda wish the community had more math and statistics knowledge. But that might also be a different issue... FWIW, there are a lot of works that do get in deep to the mathematics of ML and I find these absolutely helpful. Anyone that says theory doesn't help practice hasn't read theory or is operating in bad faith. ML uses A LOT of math, but you just don't need it to create good and/or working models. I think the distinction is important.