4 ms·

The parallels to ML you drew are on point. ML has this tendency to oversimplify complex phenomena with a easy to produce datasets, because that's what ML folks

by matusp 2y ago

The parallels to ML you drew are on point. ML has this tendency to oversimplify complex phenomena with a easy to produce datasets, because that's what ML folks do, they find a smart and easy way to create a dataset and then they focus on the models. But this falls apart pretty quickly when you go into societal problems, such as hate speech or misinformation. Maybe there it would be nice to have some rigor and theory behind the dataset instead of just winging it. I am working on societal biases in NLP and I feel confident that majority of the datasets used have practically no validity.

godelski 2y ago
I love ML. I came over from physics because it is so cool. But what I found odd is that few people were as interested in the math as I was, and even moreso were dismissive of the utility of the math (even when demonstrated!). It's gotten less mathy by the year. My criticisms of ML aren't out of hate, but actually love. We've done great work and you need to be excited to do research -- and sometimes blind and sometimes going just on faith --, because it is grueling dealing with so much failure. Because it's so easy to mistake failure for success and success for failure. It's a worry that we'll be overtaken by conmen, that I have serious concerns that we are not moving towards building AGI. But it's difficult to criticize and receive criticism (many physics groups specifically train students to take it and deal with this seemingly harsh language. To separate your internal value from your idea). So I want to be clear that my criticism (which you can see in many of my posts) is not a call to stop ML or even slow it down, but a desire to sail in a different direction (or even allow others to sail in those directions as opposed to being on one big ship). I do know there are others in psych with similar stories and beliefs. But there's a whole conversation about the structure of academia if we're going to discuss how to stop building mega ships and allow people to truly explore (there will always be a big ship, and there always should! But it should never prevent those from venturing out to explore the unknown. (Obviously I'm a fan of Antoine de Saint-Exupéry lol ))
- NalNezumi 2y ago
  A tangent but if you're frustrated with the dismissal of math in ML I think you would probably enjoy diving in the Reinforcement Learning subfield; although some people tend to call it experimental math ;] ML applied to robotics/embodiment is another fun topic, where physics is very relevant in every step of it. A bit harder to dataset-hack when a physical system is crumbling in front of you.
  - godelski 2y ago
    Oh I've found some areas. I'm particularly fond of explicit density models and normalizing flows. I do like RL too and was very surprised to hear that the math was "confusing" when it just seemed like weird notation. But it's hard to get works through review because even if you get SOTA for your architecture some asshat will ask you why you aren't better than the architectures being worked on by tens of thousands of people and with compute budgets 1000x yours. Though it's better if you find a very small niche so the reviews you get are more likely to actually know the bare basics of the topic.

by matusp 2y ago

godelski 2y ago

I love ML. I came over from physics because it is so cool. But what I found odd is that few people were as interested in the math as I was, and even moreso were dismissive of the utility of the math (even when demonstrated!). It's gotten less mathy by the year. My criticisms of ML aren't out of hate, but actually love. We've done great work and you need to be excited to do research -- and sometimes blind and sometimes going just on faith --, because it is grueling dealing with so much failure. Because it's so easy to mistake failure for success and success for failure. It's a worry that we'll be overtaken by conmen, that I have serious concerns that we are not moving towards building AGI. But it's difficult to criticize and receive criticism (many physics groups specifically train students to take it and deal with this seemingly harsh language. To separate your internal value from your idea). So I want to be clear that my criticism (which you can see in many of my posts) is not a call to stop ML or even slow it down, but a desire to sail in a different direction (or even allow others to sail in those directions as opposed to being on one big ship). I do know there are others in psych with similar stories and beliefs. But there's a whole conversation about the structure of academia if we're going to discuss how to stop building mega ships and allow people to truly explore (there will always be a big ship, and there always should! But it should never prevent those from venturing out to explore the unknown. (Obviously I'm a fan of Antoine de Saint-Exupéry lol ))

NalNezumi 2y ago
A tangent but if you're frustrated with the dismissal of math in ML I think you would probably enjoy diving in the Reinforcement Learning subfield; although some people tend to call it experimental math ;] ML applied to robotics/embodiment is another fun topic, where physics is very relevant in every step of it. A bit harder to dataset-hack when a physical system is crumbling in front of you.
- godelski 2y ago
  Oh I've found some areas. I'm particularly fond of explicit density models and normalizing flows. I do like RL too and was very surprised to hear that the math was "confusing" when it just seemed like weird notation. But it's hard to get works through review because even if you get SOTA for your architecture some asshat will ask you why you aren't better than the architectures being worked on by tens of thousands of people and with compute budgets 1000x yours. Though it's better if you find a very small niche so the reviews you get are more likely to actually know the bare basics of the topic.