5 ms·
I think one of the great ironies is that psychology is one of the hardest sciences but is treated so soft. I say this holding a degree in physics! (undergrad ph
by godelski 2y ago
I think one of the great ironies is that psychology is one of the hardest sciences but is treated so soft. I say this holding a degree in physics! (undergrad physics, grad CS/ML)
By this I mean that to make confident predictions, you need some serious statistics, but psych is one of the least math heavy sciences (thankfully they recently learned about Bayes and there's a revolution going on). Unlike physics or chemistry, you have so little control over your experiments.
There's also the problem of measurements. We stress in experimental physics that you can only measure things by proxy. This is like you measure distance by using a ruler, and you're not really measuring "a meter" but the ruler's approximation of a meter. This is why we care so much about calibration and uncertainty, making multiple measurements with different measuring devices (gets stats on that class of device) and from different measuring techniques (e.g. ruler, laser range finder, etc). But psych? What the fuck does it even mean "to measure attention"?! It's hard enough dealing with the fact that "a meter" is "a construct" but in psych your concepts are much less well defined (i.e. higher uncertainty). And then everything is just empirical?! No causal system even (barely) attempted?! (In case you've ever wondered, this is a glimpse of why physicists struggle in ML. Not because the work, but accepting the results. See also Dyson and von Neumann's Elephant)
I've jokingly likened psych to alchemy, meaning proto-chemistry -- chemistry prior to the atomic model (chemistry is "the study of electrons") -- or to astrology (astronomy pre-Kepler, not astrology we see today). I do think that's where the field is at, because there is no fundamental laws. That doesn't mean it isn't useful. Copernicus, Brahe, Galileo (same time as Kepler; they fought), and many others did amazing work and are essential figures to astronomy and astrophysics today. But psych is in an interesting boat. There are many tools at their disposal that could really help them make major strides towards determining these "laws". But it'll take a serious revolution and some major push to have some extremely tough math chops to get there. It likely won't come from ML (who suffers similar issues of rigor), but maybe from neuroscience or plain old stats (econ surprisingly contributes, more to sociology though). My worry is that the slop has too much momentum and that criticism will be dismissed because it is viewed as saying that the researchers are lazy, dumb, or incompetent rather than the monumental difficulties that are natural to the field (though both may be true, and one can cause the other). But I do hope to see it. Especially as someone in ML. We can really see the need to pin down these concepts such as cognition, consciousness, intelligence, reasoning, emotions, desire, thinking, will, and so on. These are not remotely easy problems to solve. But it is easy to convince yourself that you do understand, as long as you stop asking why after a certain point.
And I do hope these conversations continue. Light is the best disinfectant. Science is about seeking truth, not answers. That often requires a lot of nuance, unfortunately. I know it will cause some to distrust science more, but I have the feeling they were already looking for reasons to.
- matusp 2y agoThe parallels to ML you drew are on point. ML has this tendency to oversimplify complex phenomena with a easy to produce datasets, because that's what ML folks do, they find a smart and easy way to create a dataset and then they focus on the models. But this falls apart pretty quickly when you go into societal problems, such as hate speech or misinformation. Maybe there it would be nice to have some rigor and theory behind the dataset instead of just winging it. I am working on societal biases in NLP and I feel confident that majority of the datasets used have practically no validity.
- godelski 2y agoI love ML. I came over from physics because it is so cool. But what I found odd is that few people were as interested in the math as I was, and even moreso were dismissive of the utility of the math (even when demonstrated!). It's gotten less mathy by the year. My criticisms of ML aren't out of hate, but actually love. We've done great work and you need to be excited to do research -- and sometimes blind and sometimes going just on faith --, because it is grueling dealing with so much failure. Because it's so easy to mistake failure for success and success for failure. It's a worry that we'll be overtaken by conmen, that I have serious concerns that we are not moving towards building AGI. But it's difficult to criticize and receive criticism (many physics groups specifically train students to take it and deal with this seemingly harsh language. To separate your internal value from your idea). So I want to be clear that my criticism (which you can see in many of my posts) is not a call to stop ML or even slow it down, but a desire to sail in a different direction (or even allow others to sail in those directions as opposed to being on one big ship). I do know there are others in psych with similar stories and beliefs. But there's a whole conversation about the structure of academia if we're going to discuss how to stop building mega ships and allow people to truly explore (there will always be a big ship, and there always should! But it should never prevent those from venturing out to explore the unknown. (Obviously I'm a fan of Antoine de Saint-Exupéry lol ))
- NalNezumi 2y agoA tangent but if you're frustrated with the dismissal of math in ML I think you would probably enjoy diving in the Reinforcement Learning subfield; although some people tend to call it experimental math ;] ML applied to robotics/embodiment is another fun topic, where physics is very relevant in every step of it. A bit harder to dataset-hack when a physical system is crumbling in front of you.