4 ms·

> Is math a problem for non-academic researchers? It takes a while to figure out how to read academic papers, but it's largely about learning the notation. In

by jph00 7y ago

> Is math a problem for non-academic researchers? It takes a while to figure out how to read academic papers, but it's largely about learning the notation. In the end, it maps back to the code you write anyway in most cases, so it's just another way of writing stuff you already know. It's not so much linear algebra you need, since much of that is not relevant to AI. It's really matrix calculus. Which is largely about multiplying things together and adding them up. Terence Parr and I tried to create a "all you need to know" tutorial here: https://explained.ai/matrix-calculus/ https://explained.ai/matrix-calculus/ . You certainly don't need topology (unless you happen to be interested in that particular sub-field).

anjc 7y ago
Your tutorial is very good, but to able read even a few paragraphs you need to be proficient with linear algebra and calculus already. > Most papers strike me as requiring a non-trivial knowledge of linear algebra I think this is correct, if you consider college level linear algebra and an intuition for applying it to novel problems to be non-trivial knowledge
- K0SM0S 7y ago
  > I think this is correct, if you consider college level linear algebra and an intuition for applying it to novel problems to be non-trivial knowledge Yes, in the context of "a self-taught researcher", I think I intuitively meant anything that precisely requires a degree, typical academic knowledge. E.g. you can become a great business person who won't feel hindered by lack of academic knowledge, you definitely can't do that as a surgeon or lawyer. I guess I was wondering where math fit in this picture for AI research. (which I should explicitely relate to "#2" in user ineedasername's post, i.e. "AI Research as examining the theoretical frameworks & approaches to ML/DL in a way that may itself lead to shifts in the understanding of ML/DL as a whole and/or develop fundamentally new tools for the purpose of #1 [AI Engineering Research]. What might be termed "basic" or "pure" research.")
  - anjc 7y ago
    > I guess I was wondering where math fit in this picture for AI research. Well you can't do #1 or #2 without having a level of maths proficiency that most college grads do not have. FWIW, I don't understand the difference between #1 and #2 above. Most academic/industrial research is incremental (i.e. #1), and a tiiiiny % will have any impact in the way something like XGBoost would (the example he gave in another comment). That doesn't mean that the non-impactful research isn't 'basic'. You could alternatively just call #2 "groundbreaking research" and #1 "non groundbreaking research, but you need mathematics knowledge for both imo.
    - K0SM0S 7y ago
      I see what you mean. It seems possible the distinction is perhaps academic at best, a matter of perception. (I certainly don't have a personal opinion, yet! but point taken, and the continuity you speak of seems more realistic tbh).
K0SM0S 7y ago
> matrix calculus It might be wrong but I tend to see vectors and matrices as two notations for the same mathematical object[1]. So I indeed meant matrices! However I didn't see calculus itself as such a big requirement, as it all felt pretty "linear" to me (regressions etc). Are we talking things that e.g. "Calc 2"[2] should cover? I feel reassured by your first paragraph. This can be done. I'll definitely work on your tutorial; I assume it's a good benchmark for math pre-requisites in the field. Thanks a lot for the work, and advice. [1]: That was particularly reinforced with Geometric Algebra, which I'm currently diving in. https://en.wikipedia.org/wiki/Geometric_algebra https://en.wikipedia.org/wiki/Geometric_algebra [2]: https://www.khanacademy.org/math/calculus-2 https://www.khanacademy.org/math/calculus-2
- deepnotderp 7y ago
  Deep learning is mostly matrix calc. More classic ML methods are generally linear

by jph00 7y ago

anjc 7y ago

Your tutorial is very good, but to able read even a few paragraphs you need to be proficient with linear algebra and calculus already. > Most papers strike me as requiring a non-trivial knowledge of linear algebra I think this is correct, if you consider college level linear algebra and an intuition for applying it to novel problems to be non-trivial knowledge

K0SM0S 7y ago
> I think this is correct, if you consider college level linear algebra and an intuition for applying it to novel problems to be non-trivial knowledge Yes, in the context of "a self-taught researcher", I think I intuitively meant anything that precisely requires a degree, typical academic knowledge. E.g. you can become a great business person who won't feel hindered by lack of academic knowledge, you definitely can't do that as a surgeon or lawyer. I guess I was wondering where math fit in this picture for AI research. (which I should explicitely relate to "#2" in user ineedasername's post, i.e. "AI Research as examining the theoretical frameworks & approaches to ML/DL in a way that may itself lead to shifts in the understanding of ML/DL as a whole and/or develop fundamentally new tools for the purpose of #1 [AI Engineering Research]. What might be termed "basic" or "pure" research.")
- anjc 7y ago
  > I guess I was wondering where math fit in this picture for AI research. Well you can't do #1 or #2 without having a level of maths proficiency that most college grads do not have. FWIW, I don't understand the difference between #1 and #2 above. Most academic/industrial research is incremental (i.e. #1), and a tiiiiny % will have any impact in the way something like XGBoost would (the example he gave in another comment). That doesn't mean that the non-impactful research isn't 'basic'. You could alternatively just call #2 "groundbreaking research" and #1 "non groundbreaking research, but you need mathematics knowledge for both imo.
  - K0SM0S 7y ago
    I see what you mean. It seems possible the distinction is perhaps academic at best, a matter of perception. (I certainly don't have a personal opinion, yet! but point taken, and the continuity you speak of seems more realistic tbh).

K0SM0S 7y ago

> matrix calculus It might be wrong but I tend to see vectors and matrices as two notations for the same mathematical object[1]. So I indeed meant matrices! However I didn't see calculus itself as such a big requirement, as it all felt pretty "linear" to me (regressions etc). Are we talking things that e.g. "Calc 2"[2] should cover? I feel reassured by your first paragraph. This can be done. I'll definitely work on your tutorial; I assume it's a good benchmark for math pre-requisites in the field. Thanks a lot for the work, and advice. [1]: That was particularly reinforced with Geometric Algebra, which I'm currently diving in. https://en.wikipedia.org/wiki/Geometric_algebra https://en.wikipedia.org/wiki/Geometric_algebra [2]: https://www.khanacademy.org/math/calculus-2 https://www.khanacademy.org/math/calculus-2

deepnotderp 7y ago
Deep learning is mostly matrix calc. More classic ML methods are generally linear