8 ms·

This was a great read (and great nuggets, like that paper on Intelligence by Chollet). I wonder: — Is math a problem for non-academic researchers? Most paper

by K0SM0S 7y ago

This was a great read (and great nuggets, like that paper on Intelligence by Chollet). I wonder: — Is math a problem for non-academic researchers? Most papers strike me as requiring a non-trivial knowledge of linear algebra, for instance; and topology sits right behind; the bold seem to take it one up on category theory as we speak, and geometric algebra is quickly gaining traction too. Lots of math, cool math but math nonetheless. Not that you can't learn these on your own, but how big is the gap in practice, on the job, compared with actual PhDs in ML/math? (how much of a hinderance, a problem it is for the self-taught researcher) — "Contracting" in the field of AI sounds great but, how exactly? Especially solo: what type of clients and how/where to find them, what type of 'business proposition' as a freelancer do you offer, what's the pricing structure of such gigs? I mean, I can sell you websites and visuals and stuff, but AI? I know first-hand most SMBs (IME the only real customers for freelancers) are a tough sell: their datasets are tiny and demand scripting skills to sort out (extract business value), not AI, so the value proposition is low for both parties; it's still early adoption so 90% don't even consider spending 1 cent on "AI" unless as a SaaS (they actually don't need to know if it's AI or programming). I can imagine tons of fantastic research to do with SMBs, as partners or 'interested sponsors' (should they reap benefits on a low investment), but really not much yet in the way of "freelancer products" to market and sell for a living. I'm eagerly anticipating those days, but it's more like 2025-2030 as I see it. I would love to hear first hand takes on this.

JamesBarney 7y ago
It's my understanding that dirty datasets that "demand scripting skills to sort out" is pretty common and most data scientists spend 80% of their time "sorting this out".
deepnotderp 7y ago
To be honest, linear algebra is not that difficult to learn on your own, and plenty of people do. Gilbert Strang's course on OCW has made introductory linear algebra quite accessible. Things like topology (e.g. TDA, persistent homology, etc.) aren't really mainstream yet, but even then most of it isn't really "hardcore" math in the sense that you can get away with a basic understanding, e.g. what a Vietoris-Rips complex is and why we use it instead of a Cech complex in TDA. Plus most DL research nowadays is pretty (advanced) math-light. That being said, taking the time to understand the math is absolutely worthwhile in my experience. It should also be noted that a lot of real world ML/AI projects in industry aren't really about brand new algorithms using advanced math, but rather more about applying mostly existing techniques to messy, noisy real world data and taking the time to understand the domain you are applying it to.
- K0SM0S 7y ago
  > Gilbert Strang's course on OCW Thanks for the pointer! (link[1] for those interested) > you can get away with a basic understanding Great news to me! > taking the time to understand the math is absolutely worthwhile in my experience. Strongly agree — for any topic, any field. My concerns are practical indeed, and less about the 10-year horizon (well enough to become skilled at anything) than the early stages of that, the best way to propel oneself far/fast enough on year 1, then 2, etc. > applying mostly existing techniques to messy, noisy real world data and taking the time to understand the domain you are applying it to. I hear that. I actually do like the sound of that, hence concerns that I was biased. [1]: https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/ https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra...
- analbumcover 7y ago
  Linear algebra can be straightforward to learn on your own, if sufficiently motivated. But homology seems altogether different. The math undergrads who see it often have a tough enough time, I can't imagine an autodidact managing.
- FranzFerdiNaN 7y ago
  I work as a data engineer, and i was interested in learning some of the stuff our data scientists do so i can better communicate with them. Teaching myself some statistics was fine, probability was fine too and quite fun and surprising. Both subjects have plenty of books that allow you to understand the intuition behind the things they do without having to dive deep into the proofs. Linear algebra was and still is a struggle though. I've sampled many books, from Strang's book to Linear Algebra Done Wrong/Right to some books that are used in the local university in CS courses. But they are all the same. It's clear they are all written by mathematicians for math students, probably to be used as a way to teach students how to write proofs at the same time? It's just one page after another of increasingly esoteric calculations and proof after proof after proof. Which is fine if you study math, but bad for me, because i dont want to work out the proof that taking the determinant of an inverted matrix works, i want to know what it means and why one would want to make the effort to do it. Basically, i want a book like Statistical Rethinking or Blitzstein's Introduction to Probaiblity, but for linear algebra. And i havent been able to find it.
jph00 7y ago
> Is math a problem for non-academic researchers? It takes a while to figure out how to read academic papers, but it's largely about learning the notation. In the end, it maps back to the code you write anyway in most cases, so it's just another way of writing stuff you already know. It's not so much linear algebra you need, since much of that is not relevant to AI. It's really matrix calculus. Which is largely about multiplying things together and adding them up. Terence Parr and I tried to create a "all you need to know" tutorial here: https://explained.ai/matrix-calculus/ https://explained.ai/matrix-calculus/ . You certainly don't need topology (unless you happen to be interested in that particular sub-field).
- anjc 7y ago
  Your tutorial is very good, but to able read even a few paragraphs you need to be proficient with linear algebra and calculus already. > Most papers strike me as requiring a non-trivial knowledge of linear algebra I think this is correct, if you consider college level linear algebra and an intuition for applying it to novel problems to be non-trivial knowledge
  - K0SM0S 7y ago
    > I think this is correct, if you consider college level linear algebra and an intuition for applying it to novel problems to be non-trivial knowledge Yes, in the context of "a self-taught researcher", I think I intuitively meant anything that precisely requires a degree, typical academic knowledge. E.g. you can become a great business person who won't feel hindered by lack of academic knowledge, you definitely can't do that as a surgeon or lawyer. I guess I was wondering where math fit in this picture for AI research. (which I should explicitely relate to "#2" in user ineedasername's post, i.e. "AI Research as examining the theoretical frameworks & approaches to ML/DL in a way that may itself lead to shifts in the understanding of ML/DL as a whole and/or develop fundamentally new tools for the purpose of #1 [AI Engineering Research]. What might be termed "basic" or "pure" research.")
    - anjc 7y ago
      > I guess I was wondering where math fit in this picture for AI research. Well you can't do #1 or #2 without having a level of maths proficiency that most college grads do not have. FWIW, I don't understand the difference between #1 and #2 above. Most academic/industrial research is incremental (i.e. #1), and a tiiiiny % will have any impact in the way something like XGBoost would (the example he gave in another comment). That doesn't mean that the non-impactful research isn't 'basic'. You could alternatively just call #2 "groundbreaking research" and #1 "non groundbreaking research, but you need mathematics knowledge for both imo.
andreilys 7y ago
“ I know first-hand most SMBs (IME the only real customers for freelancers) are a tough sell: their datasets are tiny“ Thanks to transfer learning, tiny datasets are not a major issue to developing AI solutions. Fast.ai makes it super easy to overcome that hurdle.
- K0SM0S 7y ago
  Major interest spike: nerd-trigger, category "hard problems whose solution exceeds what I have to work with". Will investigate. Thanks a lot!
deleted 7y ago
[deleted]

by K0SM0S 7y ago

JamesBarney 7y ago

It's my understanding that dirty datasets that "demand scripting skills to sort out" is pretty common and most data scientists spend 80% of their time "sorting this out".

deepnotderp 7y ago

To be honest, linear algebra is not that difficult to learn on your own, and plenty of people do. Gilbert Strang's course on OCW has made introductory linear algebra quite accessible. Things like topology (e.g. TDA, persistent homology, etc.) aren't really mainstream yet, but even then most of it isn't really "hardcore" math in the sense that you can get away with a basic understanding, e.g. what a Vietoris-Rips complex is and why we use it instead of a Cech complex in TDA. Plus most DL research nowadays is pretty (advanced) math-light. That being said, taking the time to understand the math is absolutely worthwhile in my experience. It should also be noted that a lot of real world ML/AI projects in industry aren't really about brand new algorithms using advanced math, but rather more about applying mostly existing techniques to messy, noisy real world data and taking the time to understand the domain you are applying it to.

K0SM0S 7y ago
> Gilbert Strang's course on OCW Thanks for the pointer! (link[1] for those interested) > you can get away with a basic understanding Great news to me! > taking the time to understand the math is absolutely worthwhile in my experience. Strongly agree — for any topic, any field. My concerns are practical indeed, and less about the 10-year horizon (well enough to become skilled at anything) than the early stages of that, the best way to propel oneself far/fast enough on year 1, then 2, etc. > applying mostly existing techniques to messy, noisy real world data and taking the time to understand the domain you are applying it to. I hear that. I actually do like the sound of that, hence concerns that I was biased. [1]: https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/ https://ocw.mit.edu/courses/mathematics/18-06-linear-algebra...
analbumcover 7y ago
Linear algebra can be straightforward to learn on your own, if sufficiently motivated. But homology seems altogether different. The math undergrads who see it often have a tough enough time, I can't imagine an autodidact managing.
FranzFerdiNaN 7y ago
I work as a data engineer, and i was interested in learning some of the stuff our data scientists do so i can better communicate with them. Teaching myself some statistics was fine, probability was fine too and quite fun and surprising. Both subjects have plenty of books that allow you to understand the intuition behind the things they do without having to dive deep into the proofs. Linear algebra was and still is a struggle though. I've sampled many books, from Strang's book to Linear Algebra Done Wrong/Right to some books that are used in the local university in CS courses. But they are all the same. It's clear they are all written by mathematicians for math students, probably to be used as a way to teach students how to write proofs at the same time? It's just one page after another of increasingly esoteric calculations and proof after proof after proof. Which is fine if you study math, but bad for me, because i dont want to work out the proof that taking the determinant of an inverted matrix works, i want to know what it means and why one would want to make the effort to do it. Basically, i want a book like Statistical Rethinking or Blitzstein's Introduction to Probaiblity, but for linear algebra. And i havent been able to find it.

jph00 7y ago

> Is math a problem for non-academic researchers? It takes a while to figure out how to read academic papers, but it's largely about learning the notation. In the end, it maps back to the code you write anyway in most cases, so it's just another way of writing stuff you already know. It's not so much linear algebra you need, since much of that is not relevant to AI. It's really matrix calculus. Which is largely about multiplying things together and adding them up. Terence Parr and I tried to create a "all you need to know" tutorial here: https://explained.ai/matrix-calculus/ https://explained.ai/matrix-calculus/ . You certainly don't need topology (unless you happen to be interested in that particular sub-field).

anjc 7y ago
Your tutorial is very good, but to able read even a few paragraphs you need to be proficient with linear algebra and calculus already. > Most papers strike me as requiring a non-trivial knowledge of linear algebra I think this is correct, if you consider college level linear algebra and an intuition for applying it to novel problems to be non-trivial knowledge
- K0SM0S 7y ago
  > I think this is correct, if you consider college level linear algebra and an intuition for applying it to novel problems to be non-trivial knowledge Yes, in the context of "a self-taught researcher", I think I intuitively meant anything that precisely requires a degree, typical academic knowledge. E.g. you can become a great business person who won't feel hindered by lack of academic knowledge, you definitely can't do that as a surgeon or lawyer. I guess I was wondering where math fit in this picture for AI research. (which I should explicitely relate to "#2" in user ineedasername's post, i.e. "AI Research as examining the theoretical frameworks & approaches to ML/DL in a way that may itself lead to shifts in the understanding of ML/DL as a whole and/or develop fundamentally new tools for the purpose of #1 [AI Engineering Research]. What might be termed "basic" or "pure" research.")
  - anjc 7y ago
    > I guess I was wondering where math fit in this picture for AI research. Well you can't do #1 or #2 without having a level of maths proficiency that most college grads do not have. FWIW, I don't understand the difference between #1 and #2 above. Most academic/industrial research is incremental (i.e. #1), and a tiiiiny % will have any impact in the way something like XGBoost would (the example he gave in another comment). That doesn't mean that the non-impactful research isn't 'basic'. You could alternatively just call #2 "groundbreaking research" and #1 "non groundbreaking research, but you need mathematics knowledge for both imo.

andreilys 7y ago

“ I know first-hand most SMBs (IME the only real customers for freelancers) are a tough sell: their datasets are tiny“ Thanks to transfer learning, tiny datasets are not a major issue to developing AI solutions. Fast.ai makes it super easy to overcome that hurdle.

K0SM0S 7y ago
Major interest spike: nerd-trigger, category "hard problems whose solution exceeds what I have to work with". Will investigate. Thanks a lot!

deleted 7y ago

[deleted]