3 ms·

Regarding my background I just finished my PhD and I'm doing a postdoc on explainable machine learning for robotics. My PhD was on Inductive Logic Programming w

Regarding my background I just finished my PhD and I'm doing a postdoc on explainable machine learning for robotics. My PhD was on Inductive Logic Programming which is a virtually unknown form of machine learning of logic theories, but I have a background in statistical AI, neural nets and NLP, from my Master's days. Btw, lest this causes confusion, my part in the "explainable machine learning for robotics" project is to machine-learn autonomous behaviours for an underwater search-and-rescue robot. My specialty is not explainability, it's rather learning ability I'm interested in. That out the way, >> If you don't trust BLEU and don't trust my gut, how about a randomized survey of professional translators, or how many seconds per character a translator would need to fix up the raw output into something they'd be comfortable with calling accurate? Like, I'm not stuck on a metric. No, sorry. I want a metric I can trust and not one that is equivalent to good, old-fashioned eyballing. If I wanted to eyball, I got eyballs, I can eyball. I want a metric that doesn't depend on someone's subjective evaluation. Unfortunately, the day that someone will come up with an objective test for machine translation is the day we'd have solved machine translation, and possibly all of AI, to boot. Btw, I studied NLP at the time when Google Translate had just switched to Neural Machine Translation. I'm Greek, and I remember on the one hand the dithyrambic self-congratulations on Google blogs, and on the other hand how ridiculously crap Google Translate was in translating from and to Greek. And it still is, almost ten years later (My MSc was in 2014). I also have a cluster of friends and acquaintances who are all professional translators and interpreters and I know that they are still ripped off by translation agencies who give them automatic translations claiming that they only need an hour of work to correct, when they need maybe a day or two. That kind of test will also not do. "Progress" in machine learning means that every few years the hype peaks for yet another AI fountain of youth, or AI El Dorado, then that loses steam, and there's some sanity for a few years until the net thing comes along and trends on social media, then everybody starts writing super-serious papers about it sounding all business-like and scientifically minded, until Google, or OpenAI now, or some other big gun releases an implementation and everyone has to run behind them like idiots. It's basically science by twitter and it's completely bogus. Bottom line, you want to make machine lerning work? Be Google. >> The right reaction is awe and curiosity and marvel that a nonrigorous process has led to this. To what? I think I've made myself very clear that I don't think we've got anywhere worth talking about. So my computer can now talk back to me. So what? Why is this a scientific achievement? Why does this answer scientific questions? Which ones? >> During the history of science, big discoveries always needed a preparatory pre-scientific phase of mere aggregation of experiences, stamp collection if you will, developing almost superstitious practices that may turn out to be false on the level of the individual components, but in aggregate they result in a net positive. I've heard this before but I don't know any examples where technological progress did not go hand-in-hand with scientific understanding. For example, when the Wright brothers made a machine that could fly, the followed on the footsteps of probably hundreds of others who had done the work of science, developing such necessary concepts as aerodynamics- like George Cayley, Otto Lilienthal of course, and in fact the Wrights themselves had set up a wind tunnel and studied the aerodynamics of their machines. The difference with AI of course is that it's much easier to point to a thing and say "it flies". But that's another matter. Lest I be misunderstood: I'm enjoying our exchange. Thanks.

bonoboTP 3y ago
I don't think most of ML papers are to be labeled science. But it's some kind of scholarship. Maybe engineering, maybe just reports of whatever people worked on in the last 6 months and here's what happened. I don't know exactly what kind of knowledge you want. Cybernetics and AI from the 40s and 50s onwards didn't aim at learning the laws of intelligence like the laws of gravitation, they were practical goals of building a thing. Like the Manhattan project wasn't about leaning stuff but about going kaboom. Chomsky complains the same, that ChatGPT teaches us nothing about human language at all, it's not science, it's not advancing linguistics, but sure these things are good for captioning videos for those hard of hearing or translating between languages. And I'm like dude, nobody is stoked because they think we learned something about human language or the nature of intelligence. People are stoked because the computer can talk back at them. And again I get where you are coming from, I see the same picture but see the rabbit instead of the duck. I see it as very important and consequential that these things work so well. I'm very big on eyeballing. I don't need to measure the voltages under the hood and stare at the speedometer to tell whether a car runs or not, may it run on electricity, natural gas or petrol, the wheels are rolling and we get from A to B. Science isn't the outermost frame of existence, and has only been here for a few centuries. Journals and professional academia rat race of the shape as today, publish and perish etc, only since like half a century. The Romans built all their infrastructure without journals and h-indexes. I get it, the hype often goes too far. But I'm now contrarian about the contrarianism. Denying that translation or transcription or image segmentation etc don't work is an increasingly silly hill to die on. So many have moved on to say stuff like yeah it works, but it's biased, it is wasteful and has a big carbon footprint, or can fail suddenly and unexpectedly out of distribution. But denial doesn't help. I mean, the Internet wasn't a scientific discovery but TCP/IP, Ethernet and the Web still revolutionized society. Now maybe my grandma would deny this and say she gets by just fine gossiping with the neighbors and never using the Internet (though even her soap operas are delivered via IPTV). But just because it's not science, just "building stuff" it's still impactful. Don't be distracted by whether it fits a definition of science or not, that's a separate question from whether the thing works, by the definition of whether it gets jobs done that people want to get done and will pay for.
- YeGoblynQueenne 3y ago

bonoboTP 3y ago

I don't think most of ML papers are to be labeled science. But it's some kind of scholarship. Maybe engineering, maybe just reports of whatever people worked on in the last 6 months and here's what happened. I don't know exactly what kind of knowledge you want. Cybernetics and AI from the 40s and 50s onwards didn't aim at learning the laws of intelligence like the laws of gravitation, they were practical goals of building a thing. Like the Manhattan project wasn't about leaning stuff but about going kaboom. Chomsky complains the same, that ChatGPT teaches us nothing about human language at all, it's not science, it's not advancing linguistics, but sure these things are good for captioning videos for those hard of hearing or translating between languages. And I'm like dude, nobody is stoked because they think we learned something about human language or the nature of intelligence. People are stoked because the computer can talk back at them. And again I get where you are coming from, I see the same picture but see the rabbit instead of the duck. I see it as very important and consequential that these things work so well. I'm very big on eyeballing. I don't need to measure the voltages under the hood and stare at the speedometer to tell whether a car runs or not, may it run on electricity, natural gas or petrol, the wheels are rolling and we get from A to B. Science isn't the outermost frame of existence, and has only been here for a few centuries. Journals and professional academia rat race of the shape as today, publish and perish etc, only since like half a century. The Romans built all their infrastructure without journals and h-indexes. I get it, the hype often goes too far. But I'm now contrarian about the contrarianism. Denying that translation or transcription or image segmentation etc don't work is an increasingly silly hill to die on. So many have moved on to say stuff like yeah it works, but it's biased, it is wasteful and has a big carbon footprint, or can fail suddenly and unexpectedly out of distribution. But denial doesn't help. I mean, the Internet wasn't a scientific discovery but TCP/IP, Ethernet and the Web still revolutionized society. Now maybe my grandma would deny this and say she gets by just fine gossiping with the neighbors and never using the Internet (though even her soap operas are delivered via IPTV). But just because it's not science, just "building stuff" it's still impactful. Don't be distracted by whether it fits a definition of science or not, that's a separate question from whether the thing works, by the definition of whether it gets jobs done that people want to get done and will pay for.

YeGoblynQueenne 3y ago