6 ms·

Off topic, but it's very interesting to observe the ratio of upvotes / comments. On any given chatGPT topic, there are hundreds of comments usually. Here, so f

Off topic, but it's very interesting to observe the ratio of upvotes / comments. On any given chatGPT topic, there are hundreds of comments usually. Here, so far 100 upvotes, only 7 comments. The book looks great - and given the authors, it almost certainly is (I will buy it for sure). It makes me think though about the state of 'ML / AI / Data Science' - and the cynic part of me thinks that this upvotes / comments ratio kind of reflects the fact that most people interested in AI hype have not really touched a lot of underlying concepts and don't have any deeper understanding of maths / stats behind. PS. That being said, I didn't do a meaningful comment on the link topic neither.

NeutralForest 3y ago
I know about the book, I just think it's cool there's an actual Python version now so I upvote.
__rito__ 3y ago
What you are saying is true. This kind of books are key to getting started with Machine Learning/AI, and this particular book is a very good one. I started my ML journey with this book. There is a lot of hype around AI and it is going to be like the dotcom bubble. Unlike Crypto, AI has real uses right now and I am saying this not taking into account any LLM products. But there is also a lot of hype and wishful thinking, and this bubble is going to burst and hurt a lot of people. But that won't stop people making real money in the short term. Many hypers I know understand this well. But AI is here to stay. And even after the bubble bursts, there will be real uses of AI all around us.
- godelski 3y ago
  > and this bubble is going to burst and hurt a lot of people > But AI is here to stay. And even after the bubble bursts, there will be real uses of AI all around us. This is, as a ML researcher, exactly where I'm at (in belief). The utility is high, but so is the noise. Rather, the utility is sufficient. The danger of ML is not so much X-risk or malevolent AGI, but dumb ML being used inappropriately. And in general, that is using ML without understanding the limitations and having checks on it to ensure that when hallucinations happen that they don't cause major problems. But we're headed in a direction where we're becoming more reliant upon them and then once we have a few big issues with hallucinations the bubble will burst and can end up setting us back a lot in our progress to creating AGI. Previous winters were caused by lack of timely progress, but the next winter will happen because we shoot ourselves in the foot. Unfortunately, the more people you give guns to, the more likely this is to happen -- especially when there's no safety training or even acknowledgement of danger (or worse, the only discussion is about being shot by others).
  - basedbertram 3y ago
    > This is, as a ML researcher, exactly where I'm at (in belief). As an ML researcher myself, I'm glad to see other researchers maintaining a level-headed approach amidst the noise.
    - godelski 3y ago
      Well if you care more about the study than the money (which is nice -- though I'm at the tail of grad school), it makes more sense to chase knowledge than chase metrics. Then again, I don't come from a computer science background, so maybe not having that momentum helps.
Borrible 3y ago
Probably just needs a ChatGPT code interpreter plugin.
melling 3y ago
It’s a well-known machine learning book. I’ve read through it and done the exercises in R. https://github.com/melling/ISLR https://github.com/melling/ISLR There’s an edX course from the authors: https://www.edx.org/course/statistical-learning https://www.edx.org/course/statistical-learning
- suchagain 3y ago
  I also went through this book for a GIS course in grad school and put together some Python notebooks. https://github.com/gbrunner/islr-for-gis https://github.com/gbrunner/islr-for-gis
lysecret 3y ago
This and elements where the intro to ML for me as well. I understand your sentiment but we also have to accept that for a lot of Ml usecases just calling ChatGPT api is 100x better approach than creating your own Ml model, and thus there is really no need to understand any math. As an example I am building an Ai nutrition counting app. And I use ChatGPT function calling. I can just add a field that has say an emoji of the good and it automatically classifies any food to the right emoji. There is absolutely no need to know gradient descent or any fundamental property to be able to do that.
- xapata 3y ago
  > for a lot of Ml usecases Have you forgotten the No Free Lunch Theorem?
- janalsncm 3y ago
  As an ML engineer, I look forward to the day OpenAI ups their prices 5x and companies hire people like me as a consultant to replace their expensive API calls with an SVM or random forest that can be run off a smartphone. That intern who figured out how to make a POST request and accidentally committed the API keys to public GitHub? Long gone. The rise and grind manager who discovered ChatGPT in April? Retired. But we will be there, ready to cut your costs by 95% because people couldn’t be bothered to understand the basics of what they’re using, in exchange for a sizable consulting fee of course.
thumbuddy 3y ago
There's a meme where it shows someone stepping over all the steps to understand how to think about and analyze data directly to BERT. Well now people are stepping past BERT to stable diffusion and ChatGPT. It's been like this for years. Most work environments suffer from it in a bad way. I don't envy practicing data scientists managing expectations.
- adamsmith143 3y ago
  Interesting seeing job postings wanting 5+ years of LLM experience. Like unless you were at OpenAI working on GPT-1 or Google on BERT there's no one else in the world with that much experience and your shitty Startup/Fortune 500 company can't afford them anyway.
  - lisasays 3y ago
    Interesting seeing job postings wanting 5+ years of LLM experience. Since they're probably using ChatGPT to generate these job reqs - this is only to be expected.
  - vram22 3y ago
    >Interesting seeing job postings wanting 5+ years of LLM experience. Not really interesting. Similar to seeing job postings some years back (and even recently), wanting n+ years of Rails experience when DHH had created it significantly less than n years before. It's almost a meme on HN at this point.
DeathArrow 3y ago
Wait, so there's more in AI than ChatGPT?
Eigenwert 3y ago
A lot of people do not need to know about underlying concepts in AI - they just use it. Though it is interesting to see that even on this website this seems to be the case.
wodenokoto 3y ago
I imagine a lot of people like this book and are happy to see an update. In that case it is hard to say anything of value other than an upvote.
adamsmith143 3y ago
This is pure gatekeeping. The math behind LLMs, that is, the math behind Neural Nets, is undergrad freshman level Calculus and some linear algebra. Not really complex at all. Can you deal with derivatives, the chain rule and matrix multiplications? Great you know all the "math" behind Deep Learning.
- posterboy 3y ago
  > Can you deal with derivatives Linear Algebra is a huge ... space? And then there's all the other algebras. I'm not sure what you are thinking, like, do you not know matrix multiplication you had better get a clue.
  - adamsmith143 3y ago
    What does that have to do with ML/DL? You don't need Lie Algebra to understand how to do a matrix dot product...
- godelski 3y ago
  That's the beginning math, but definitely not "the math behind LLMs". That includes probability theory, metric theory, topology, and more. But most people don't even acknowledge this, but then again, unless you're deep in a subject you don't really know the complexities of that subject. Red flags should go off whenever anyone says "it's just <x>" or calls something simple. It's like the professor saying the proof is trivial, when that's the hardest part of the entire problem.
  - adamsmith143 3y ago
    People like you are hilarious. You're sitting high in your ivory tower thinking that no one without a PhD in CS from Stanford/Berkeley/MIT can do what you do. Meanwhile people will take the Fast.ai course and be training full llm's from scratch in 6 months all the while you moan that "they don't even understand the REAL math". Yawn.
    - 3y ago
A4ET8a8uTh0 3y ago
There is, naturally, a reason for this. GPT is effectively a nice wrap on an otherwise complicated set of issues. I almost think of it as gui instead of console. Yeah, you lose some of the functionality and control, but a lot of people will take it and run with it simply because it is just so much easier. Case in point, Google AML AI, which promises to do away with pesky model validation and such ( because it will do everything in a closed box you will not have a reason to investigate ). I am already looking forward to the conversations with regulators.
godelski 3y ago
As a ML researcher, I don't think you're far off the point. w.r.t HN, there's almost all hype and no "science". People have strong convictions but not strong evidence. They happily cite papers, but only read the abstracts and miss the essential nuance. Especially in a field where suggesting limitations puts you at high risk of rejection (reviewers just copy paste that and thank you for the work). w.r.t academia, it is a bit better, but I find that in general there are a lot of researchers missing math fundamentals. I know or have met people at top universities or top labs that don't know the difference between likelihood and probability. Similarly ones that don't understand probability density. Even ones working on diffusion. But I will say, that in general the most prominent researchers do have these skills. But you'll notice that they aren't publishing as fast and their works might not even be as popular. A lot of research right now goes into parameter tuning and throwing compute at the problem. I've been a bit vocal about this though. Mostly due to it being a barrier to other types of research (because I'll admit that the tuning is needed, but we need to be honest that it isn't high innovation either and that it is hard to prove these are better given that we haven't tuned other models/architectures to the same degree). tldr: You're pretty spot on. There's a shit ton of noise in ML/AI. Especially on HN Edit: I thought I should also suggest Richard McElreath's Statistical Rethinking (https://xcelab.net/rm/statistical-rethinking/ https://xcelab.net/rm/statistical-rethinking/), which is a more enjoyable read than ISLR and will also introduce you to Bayesian stats (Lectures are also on youtube). I'd also suggest Gelman's Regression and Other Stories (https://avehtari.github.io/ROS-Examples/ https://avehtari.github.io/ROS-Examples/).
- mathisfun123 3y ago
  >don't know the difference between likelihood and probability. Similarly ones that don't understand probability density. I'm a phd student in a "top university", in a research group primarily focused on data science (NLP, LLMs, blah blah blah). I'm 100% sure I am the only person in the group of ~25 (including profs/postdocs) that knows the difference between f(θ|x) and f(x|θ). In fact I'm pretty sure I'm the only person that has ever even seen f(θ|x) (because I took a stats sequence out of casella+berger). This group puts out dozens of papers a year. My research focus is not data science (compilers).
hsjqllzlfkf 3y ago
> On any given chatGPT topic, there are hundreds of comments usually. Here, so far 100 upvotes, only 7 comments. That's because anyone can interact with chatgpt. Textbooks on the other hand take serious effort to consume.

NeutralForest 3y ago

I know about the book, I just think it's cool there's an actual Python version now so I upvote.

__rito__ 3y ago

What you are saying is true. This kind of books are key to getting started with Machine Learning/AI, and this particular book is a very good one. I started my ML journey with this book. There is a lot of hype around AI and it is going to be like the dotcom bubble. Unlike Crypto, AI has real uses right now and I am saying this not taking into account any LLM products. But there is also a lot of hype and wishful thinking, and this bubble is going to burst and hurt a lot of people. But that won't stop people making real money in the short term. Many hypers I know understand this well. But AI is here to stay. And even after the bubble bursts, there will be real uses of AI all around us.

godelski 3y ago
> and this bubble is going to burst and hurt a lot of people > But AI is here to stay. And even after the bubble bursts, there will be real uses of AI all around us. This is, as a ML researcher, exactly where I'm at (in belief). The utility is high, but so is the noise. Rather, the utility is sufficient. The danger of ML is not so much X-risk or malevolent AGI, but dumb ML being used inappropriately. And in general, that is using ML without understanding the limitations and having checks on it to ensure that when hallucinations happen that they don't cause major problems. But we're headed in a direction where we're becoming more reliant upon them and then once we have a few big issues with hallucinations the bubble will burst and can end up setting us back a lot in our progress to creating AGI. Previous winters were caused by lack of timely progress, but the next winter will happen because we shoot ourselves in the foot. Unfortunately, the more people you give guns to, the more likely this is to happen -- especially when there's no safety training or even acknowledgement of danger (or worse, the only discussion is about being shot by others).
- basedbertram 3y ago
  > This is, as a ML researcher, exactly where I'm at (in belief). As an ML researcher myself, I'm glad to see other researchers maintaining a level-headed approach amidst the noise.
  - godelski 3y ago
    Well if you care more about the study than the money (which is nice -- though I'm at the tail of grad school), it makes more sense to chase knowledge than chase metrics. Then again, I don't come from a computer science background, so maybe not having that momentum helps.

Borrible 3y ago

Probably just needs a ChatGPT code interpreter plugin.

melling 3y ago

It’s a well-known machine learning book. I’ve read through it and done the exercises in R. https://github.com/melling/ISLR https://github.com/melling/ISLR There’s an edX course from the authors: https://www.edx.org/course/statistical-learning https://www.edx.org/course/statistical-learning

suchagain 3y ago
I also went through this book for a GIS course in grad school and put together some Python notebooks. https://github.com/gbrunner/islr-for-gis https://github.com/gbrunner/islr-for-gis

lysecret 3y ago

This and elements where the intro to ML for me as well. I understand your sentiment but we also have to accept that for a lot of Ml usecases just calling ChatGPT api is 100x better approach than creating your own Ml model, and thus there is really no need to understand any math. As an example I am building an Ai nutrition counting app. And I use ChatGPT function calling. I can just add a field that has say an emoji of the good and it automatically classifies any food to the right emoji. There is absolutely no need to know gradient descent or any fundamental property to be able to do that.

xapata 3y ago
> for a lot of Ml usecases Have you forgotten the No Free Lunch Theorem?
janalsncm 3y ago
As an ML engineer, I look forward to the day OpenAI ups their prices 5x and companies hire people like me as a consultant to replace their expensive API calls with an SVM or random forest that can be run off a smartphone. That intern who figured out how to make a POST request and accidentally committed the API keys to public GitHub? Long gone. The rise and grind manager who discovered ChatGPT in April? Retired. But we will be there, ready to cut your costs by 95% because people couldn’t be bothered to understand the basics of what they’re using, in exchange for a sizable consulting fee of course.

thumbuddy 3y ago

There's a meme where it shows someone stepping over all the steps to understand how to think about and analyze data directly to BERT. Well now people are stepping past BERT to stable diffusion and ChatGPT. It's been like this for years. Most work environments suffer from it in a bad way. I don't envy practicing data scientists managing expectations.

adamsmith143 3y ago
Interesting seeing job postings wanting 5+ years of LLM experience. Like unless you were at OpenAI working on GPT-1 or Google on BERT there's no one else in the world with that much experience and your shitty Startup/Fortune 500 company can't afford them anyway.
- lisasays 3y ago
  Interesting seeing job postings wanting 5+ years of LLM experience. Since they're probably using ChatGPT to generate these job reqs - this is only to be expected.
- vram22 3y ago
  >Interesting seeing job postings wanting 5+ years of LLM experience. Not really interesting. Similar to seeing job postings some years back (and even recently), wanting n+ years of Rails experience when DHH had created it significantly less than n years before. It's almost a meme on HN at this point.

DeathArrow 3y ago

Wait, so there's more in AI than ChatGPT?

Eigenwert 3y ago

A lot of people do not need to know about underlying concepts in AI - they just use it. Though it is interesting to see that even on this website this seems to be the case.

wodenokoto 3y ago

I imagine a lot of people like this book and are happy to see an update. In that case it is hard to say anything of value other than an upvote.

adamsmith143 3y ago

This is pure gatekeeping. The math behind LLMs, that is, the math behind Neural Nets, is undergrad freshman level Calculus and some linear algebra. Not really complex at all. Can you deal with derivatives, the chain rule and matrix multiplications? Great you know all the "math" behind Deep Learning.

posterboy 3y ago
> Can you deal with derivatives Linear Algebra is a huge ... space? And then there's all the other algebras. I'm not sure what you are thinking, like, do you not know matrix multiplication you had better get a clue.
- adamsmith143 3y ago
  What does that have to do with ML/DL? You don't need Lie Algebra to understand how to do a matrix dot product...
godelski 3y ago
That's the beginning math, but definitely not "the math behind LLMs". That includes probability theory, metric theory, topology, and more. But most people don't even acknowledge this, but then again, unless you're deep in a subject you don't really know the complexities of that subject. Red flags should go off whenever anyone says "it's just <x>" or calls something simple. It's like the professor saying the proof is trivial, when that's the hardest part of the entire problem.
- adamsmith143 3y ago
  People like you are hilarious. You're sitting high in your ivory tower thinking that no one without a PhD in CS from Stanford/Berkeley/MIT can do what you do. Meanwhile people will take the Fast.ai course and be training full llm's from scratch in 6 months all the while you moan that "they don't even understand the REAL math". Yawn.
  - 3y ago

A4ET8a8uTh0 3y ago

There is, naturally, a reason for this. GPT is effectively a nice wrap on an otherwise complicated set of issues. I almost think of it as gui instead of console. Yeah, you lose some of the functionality and control, but a lot of people will take it and run with it simply because it is just so much easier. Case in point, Google AML AI, which promises to do away with pesky model validation and such ( because it will do everything in a closed box you will not have a reason to investigate ). I am already looking forward to the conversations with regulators.

godelski 3y ago

As a ML researcher, I don't think you're far off the point. w.r.t HN, there's almost all hype and no "science". People have strong convictions but not strong evidence. They happily cite papers, but only read the abstracts and miss the essential nuance. Especially in a field where suggesting limitations puts you at high risk of rejection (reviewers just copy paste that and thank you for the work). w.r.t academia, it is a bit better, but I find that in general there are a lot of researchers missing math fundamentals. I know or have met people at top universities or top labs that don't know the difference between likelihood and probability. Similarly ones that don't understand probability density. Even ones working on diffusion. But I will say, that in general the most prominent researchers do have these skills. But you'll notice that they aren't publishing as fast and their works might not even be as popular. A lot of research right now goes into parameter tuning and throwing compute at the problem. I've been a bit vocal about this though. Mostly due to it being a barrier to other types of research (because I'll admit that the tuning is needed, but we need to be honest that it isn't high innovation either and that it is hard to prove these are better given that we haven't tuned other models/architectures to the same degree). tldr: You're pretty spot on. There's a shit ton of noise in ML/AI. Especially on HN Edit: I thought I should also suggest Richard McElreath's Statistical Rethinking (https://xcelab.net/rm/statistical-rethinking/ https://xcelab.net/rm/statistical-rethinking/), which is a more enjoyable read than ISLR and will also introduce you to Bayesian stats (Lectures are also on youtube). I'd also suggest Gelman's Regression and Other Stories (https://avehtari.github.io/ROS-Examples/ https://avehtari.github.io/ROS-Examples/).

mathisfun123 3y ago
>don't know the difference between likelihood and probability. Similarly ones that don't understand probability density. I'm a phd student in a "top university", in a research group primarily focused on data science (NLP, LLMs, blah blah blah). I'm 100% sure I am the only person in the group of ~25 (including profs/postdocs) that knows the difference between f(θ|x) and f(x|θ). In fact I'm pretty sure I'm the only person that has ever even seen f(θ|x) (because I took a stats sequence out of casella+berger). This group puts out dozens of papers a year. My research focus is not data science (compilers).

hsjqllzlfkf 3y ago

> On any given chatGPT topic, there are hundreds of comments usually. Here, so far 100 upvotes, only 7 comments. That's because anyone can interact with chatgpt. Textbooks on the other hand take serious effort to consume.