5 ms·

This site bills itself as follows: > This is a draft workshop to build from scratch the basic background needed to try introductory books and courses in AI and

This site bills itself as follows: > This is a draft workshop to build from scratch the basic background needed to try introductory books and courses in AI and how to optimize their deployments. I'm not sure learning the things on that page – Peano axioms, basic real analysis, etc. – really helps with deploying AI. I know plenty of talented graduate students with papers at NeurIPS, ICML, and another top conferences who don't know a lick of set theory, so certainly it's not necessary, even for highly academic work. Based on my experience and what we know about the psychology of learning, I'd recommend finding a motivating ML project or application (e.g. point the phone at a sudoku and it overlays the solution), and then working backwards to figure out what you need to know to implement it. Probably it will not be the Peano axioms.

amautertt 5y ago
Maybe type theory is good to know! I would like to see cutting edge NN libraries for Coq or Lean or Optimization ecosystem like Scipy or Scikit for Agda. You develop your algorithm and you prove a bunch of theorems about it at the same time, incredibly difficult but totally wild! There's some scattered projects like this https://github.com/OUPL/MLCert https://github.com/OUPL/MLCert
ipnon 5y ago
There are two schools of machine learning education: Math first like Andrew Ng and code first like Jeremy Howard. That both of these leaders have found academic and professional success shows you should approach machine learning in whatever way comes more naturally to you. Later on, you can approach it from the other angle to improve your understanding.
- spekcular 5y ago
  To clarify, I'm not arguing that math isn't important. You need your linear algebra, your probability theory, and so on. That's non-negotiable. But set theory? Type theory (as someone below suggests)? I would avoid unless you're separately interested in mathematics for its own sake.
  - exdsq 5y ago
    I’m fairly certain they’re not suggesting you need to know type theory for ML, but that theorem provers should have more ML libraries. My wife does some ML as a postdoc at Stanford and I doubt she’d even know what type theory is.
  - yarky 5y ago
    I agree, however I'm not sure of how far you can get in probability theory without set theory. For instance, I recall my stats prof defining a lot of sets in class, e.g. sigma algebras.
    - spekcular 5y ago
      Another point of clarification: I'm all for telling people what sets are. I'm less warm toward discussing things like ZFC, esoteric cardinality issues, and certain paradoxes (and large cardinals, forcing, etc.), which is typically what mathematicians mean when they say "set theory."
      - actually_a_dog 5y ago
        Right. What most people need to learn is "algebra of sets," essentially.
  - chobytes 5y ago
    Its not immediately practical... but spending a lot time learning a science like math in depth makes learning various applications very easy later. I think sometimes educators have to let people be motivated by some particular application to teach them something more important. Perhaps that is what the author is up to?
  - ABeeSea 5y ago
    How do you even do serious linear algebra or probability without set theory?
    - orbifold 5y ago
- medo-bear 5y ago
  yes but most of the math-based approaches to ML start from real (multivariate) analysis, or from probability theory. i haven't heard anyone claim that Peano axioms are important. as much as I will defend the notion that applied mathematics is extremely important for ML research, peano axioms are unnecessary and will just cause confusion
nonameiguess 5y ago
While I would agree you don't really need to know or worry much about mathematical foundationalism to grok machine learning, anyone writing graduate level papers is going to understand some set theory since probability theory relies upon it. Even just the high school treatment tends to start out with intuitive visualizations of univariate discrete probability like defining a probability as the cardinality of an event space divided by the cardinality of a sample space, defining conjunction and disjunction as set intersection or set union of event spaces, defining mutual exclusivity as disjoint event spaces, and so on. That said, there is probably no reason for anyone to dig too deep into theory without understanding why it matters first. And I don't think it ever matters if you're just trying to solve a problem with ML. Even if you want to overlay a solution on an image of a Sudoku puzzle, you don't need to understand why a particular algorithm works in order to be able to use it. Theory only matters if you're either trying to formally prove something works or possibly to guide you down more productive research paths in developing new algorithms, though honestly I'm not even sure that's really true any more. Just gaining a ton of experience with various possible techniques might generate intuition that is as useful or even more useful than actually understanding anything. I almost hate saying that, as I have a deep personal love of math theory and think it is worth studying to me at least for its own sake, but this is sort of in the vein of whoever invented the curveball almost certainly didn't know much about aerodynamics and didn't need to. Discovering something does work may happen decades or even centuries before anyone figures out why it works.
YeGoblynQueenne 5y ago
Thesee are mathematics necessary for AI, as in "Artificial intelligence", not as in "machine learning". From a cursory look the linked page covers a broad range of subjects necessary to understand the last 65 years of AI research, not just the 9 years since the deep learning boom that most people are familiar with. That you know many graduate students with published papers in major conferences in machine learning who don't have a background in AI is ...unfortunate? I guess?
- pedrosorio 5y ago
  Which topics in AI require knowing the Peano axioms?
  - foxes 5y ago
    Maybe if you are just experimentally gluing black boxes together and trying to make observations about them >ah these parameters/network layout did this, here's a guess as to why Or just mindlessly gluing stuff to solve a particular problem >we took x model and trained it to recognise cats then yes, you probably do not need much mathematical knowledge. However if you are researching fundamentals, neural tangent kernel, neural odes etc, then you will probably be pretty decent at mathematics. You will also probably have been exposed to Peano at some stage in your life, even if you don't remember it. Its just good background mathematical character development. I would say even if you are more of an applied person, being exposed to the fundamentals probably helps shape your thinking. Yes you can still be a good productive programmer, but maybe you will have something missing. The person with a broader background may be quicker at solving a new problem, they have a bit more to draw on.
    - pedrosorio 5y ago
      I meant to but was not explicit enough. My question to the parent was focusing specifically on their statement "These are mathematics necessary for AI, as in "Artificial intelligence", not as in "machine learning"."
    - asdf_snar 5y ago

amautertt 5y ago

Maybe type theory is good to know! I would like to see cutting edge NN libraries for Coq or Lean or Optimization ecosystem like Scipy or Scikit for Agda. You develop your algorithm and you prove a bunch of theorems about it at the same time, incredibly difficult but totally wild! There's some scattered projects like this https://github.com/OUPL/MLCert https://github.com/OUPL/MLCert

ipnon 5y ago

There are two schools of machine learning education: Math first like Andrew Ng and code first like Jeremy Howard. That both of these leaders have found academic and professional success shows you should approach machine learning in whatever way comes more naturally to you. Later on, you can approach it from the other angle to improve your understanding.

spekcular 5y ago
To clarify, I'm not arguing that math isn't important. You need your linear algebra, your probability theory, and so on. That's non-negotiable. But set theory? Type theory (as someone below suggests)? I would avoid unless you're separately interested in mathematics for its own sake.
- exdsq 5y ago
  I’m fairly certain they’re not suggesting you need to know type theory for ML, but that theorem provers should have more ML libraries. My wife does some ML as a postdoc at Stanford and I doubt she’d even know what type theory is.
- yarky 5y ago
  I agree, however I'm not sure of how far you can get in probability theory without set theory. For instance, I recall my stats prof defining a lot of sets in class, e.g. sigma algebras.
  - spekcular 5y ago
    Another point of clarification: I'm all for telling people what sets are. I'm less warm toward discussing things like ZFC, esoteric cardinality issues, and certain paradoxes (and large cardinals, forcing, etc.), which is typically what mathematicians mean when they say "set theory."
    - actually_a_dog 5y ago
      Right. What most people need to learn is "algebra of sets," essentially.
- chobytes 5y ago
  Its not immediately practical... but spending a lot time learning a science like math in depth makes learning various applications very easy later. I think sometimes educators have to let people be motivated by some particular application to teach them something more important. Perhaps that is what the author is up to?
- ABeeSea 5y ago
  How do you even do serious linear algebra or probability without set theory?
  - orbifold 5y ago
medo-bear 5y ago
yes but most of the math-based approaches to ML start from real (multivariate) analysis, or from probability theory. i haven't heard anyone claim that Peano axioms are important. as much as I will defend the notion that applied mathematics is extremely important for ML research, peano axioms are unnecessary and will just cause confusion

nonameiguess 5y ago

While I would agree you don't really need to know or worry much about mathematical foundationalism to grok machine learning, anyone writing graduate level papers is going to understand some set theory since probability theory relies upon it. Even just the high school treatment tends to start out with intuitive visualizations of univariate discrete probability like defining a probability as the cardinality of an event space divided by the cardinality of a sample space, defining conjunction and disjunction as set intersection or set union of event spaces, defining mutual exclusivity as disjoint event spaces, and so on. That said, there is probably no reason for anyone to dig too deep into theory without understanding why it matters first. And I don't think it ever matters if you're just trying to solve a problem with ML. Even if you want to overlay a solution on an image of a Sudoku puzzle, you don't need to understand why a particular algorithm works in order to be able to use it. Theory only matters if you're either trying to formally prove something works or possibly to guide you down more productive research paths in developing new algorithms, though honestly I'm not even sure that's really true any more. Just gaining a ton of experience with various possible techniques might generate intuition that is as useful or even more useful than actually understanding anything. I almost hate saying that, as I have a deep personal love of math theory and think it is worth studying to me at least for its own sake, but this is sort of in the vein of whoever invented the curveball almost certainly didn't know much about aerodynamics and didn't need to. Discovering something does work may happen decades or even centuries before anyone figures out why it works.

YeGoblynQueenne 5y ago

Thesee are mathematics necessary for AI, as in "Artificial intelligence", not as in "machine learning". From a cursory look the linked page covers a broad range of subjects necessary to understand the last 65 years of AI research, not just the 9 years since the deep learning boom that most people are familiar with. That you know many graduate students with published papers in major conferences in machine learning who don't have a background in AI is ...unfortunate? I guess?

pedrosorio 5y ago
Which topics in AI require knowing the Peano axioms?
- foxes 5y ago
  Maybe if you are just experimentally gluing black boxes together and trying to make observations about them >ah these parameters/network layout did this, here's a guess as to why Or just mindlessly gluing stuff to solve a particular problem >we took x model and trained it to recognise cats then yes, you probably do not need much mathematical knowledge. However if you are researching fundamentals, neural tangent kernel, neural odes etc, then you will probably be pretty decent at mathematics. You will also probably have been exposed to Peano at some stage in your life, even if you don't remember it. Its just good background mathematical character development. I would say even if you are more of an applied person, being exposed to the fundamentals probably helps shape your thinking. Yes you can still be a good productive programmer, but maybe you will have something missing. The person with a broader background may be quicker at solving a new problem, they have a bit more to draw on.
  - pedrosorio 5y ago
    I meant to but was not explicit enough. My question to the parent was focusing specifically on their statement "These are mathematics necessary for AI, as in "Artificial intelligence", not as in "machine learning"."
  - asdf_snar 5y ago