7 ms·
> where he argued against the other search leads that Google should use less machine-learning This better echoes my personal experience with the decline of Goo
by JohnFen 3y ago
> where he argued against the other search leads that Google should use less machine-learning
This better echoes my personal experience with the decline of Google search than TFA: it seems to be connected to the increasing use of ML in that the more of it Google put in, the worse the results I got were.
- potatolicious 3y agoIt's also a good lesson for the new AI cycle we're in now. Often inserting ML subsystems into your broader system just makes it go from "deterministically but fixably bad" to "mysteriously and unfixably bad".
- ytdytvhxgydvhh 3y agoI think that’ll define the industry for the coming decades. I used to work in machine translation and it was the same. The older rules-based engines that were carefully crafted by humans worked well on the test suite and if a new case was found, a human could fix it. When machine learning came on the scene, more “impressive” models that were built quicker came out - but when a translation was bad no one knew how to fix it other than retraining and crossing one’s fingers.
- space_fountain 3y agoYes, but I think the other lesson might be that those black box machine translations have ended up being more valuable? It sucks when things don't always work, but that is also kind of life and if the AI version worked more often that is usually ok (as long as the occasional failures aren't so catastrophic as to ruin everything)
- ytdytvhxgydvhh 3y agoCan’t help but read that and think of Tesla’s Autopilot and “Full Self Driving”. For some comparisons they claim to be safer per mile than human drivers … just don’t think too much about the error modes where the occasional stationary object isn’t detected and you plow into it at highway speed.
- space_fountain 3y agoWell Tesla might be the single worst actor in the entire AI space, but I do somewhat understand your point. The lake of predictable failures is a huge problem with AI, I'm not sure that understandability is by itself. I will never understand the brain of an Uber driver for example
- Terr_ 3y agoOr in some cases, the Tesla slows down, then changes its mind and starts accelerating again to run over child-like obstructions. Ex: https://www.youtube.com/watch?v=URpTJ1Xpjuk&t=293s https://www.youtube.com/watch?v=URpTJ1Xpjuk&t=293s
- friendzis 3y agoTesla's driver assist since the very beginning to now seems to not posses object/decision permanence. Here you can see it detected an obstacle (as evidenced by info on screen), made a decision to stop, however it failed to detect existence of the object right in front of the car, promptly forgot about the object and decision to stop and happily accelerated over the obstacle. When tackling a more complex intersection it can happily change its mind with regards to exit lane multiple times, e.g. it will plan to exit on one side of a divider, replan to exit onto upcoming traffic, replan again.
- someguydave 3y agorelevant to the grandparent’s point: I am demoing FSD in my Tesla and what I find really annoying is that the old Autopilot allowed you to select a maximum speed that the car will drive. Well, on “FSD” apparently you have no choice but to hand full longitudinal control over to the model. I am probably the 0.01% of Tesla drivers who have the computer chime when I exceed the speed limit by some offset. Very regularly, even when FSD is in “chill” mode, the model will speed by +7-9 mph on most roads. (I gotta think that the young 20 somethings who make up Tesla's audience also contributed their poor driving habits to Tesla's training data set) This results in constant beeps, even as the FSD software violates my own criteria for speed warning. So somehow the FSD feature becomes "more capable" while becoming much less legible to the human controller. I think this is a bad thing generally but it seems to be the fad today.
- ethbr1 3y ago> Yes, but I think the other lesson might be that those black box machine translations have ended up being more valuable? The key difference is how tolerant the specific use case is of a probably-correct answer. The things recent-AI excels at now (generative, translation, etc.) are very tolerant of "usually correct." If a model can do more, and is right most of the time, then it's more valuable. There are many other types of use cases, though.
- nojs 3y agoA case in point is the ubiquity of Pleco in the Chinese/English space. It’s a dictionary, not a translator, and pretty much every non-native speaker who learns or needs to speak Chinese uses it. It has no ML features and hasn’t changed much in the past decade (or even two). People love it because it does one specific task extremely well. On the other hand ML has absolutely revolutionised translation (of longer text), where having a model containing prior knowledge about the world is essential.
- satvikpendem 3y agoAs someone who worked in rules-based ML before the recent transformers (and unsupervised learning in general) hype, rules-based approaches were laughably bad. Only now are nondeterministic approaches to ML surpassing human level tasks, something which would not have been feasible, perhaps not even possible in a finite amount of human development time, via human-created rules.
- Anotheroneagain 3y agoThe thing is that AI is completely unpredictable without human curated results. Stable diffusion made me relent and admit that AI is here now for real, but I no longer think so. It's more like artificial schizophrenia. It does have some results, often plausible seeming results, but it's not real.
- beefnugs 3y agoyes, who exactly looked at the 70% accuracy of "live automatic closed captioning" and decided Great! ship it boys!
- throwaway2037 3y agoMy guess: They are hoping user feedback will help them to fix the bugs later -- iterate to 99%. Plus, they are probably under unrealistic deadlines to delivery _something_.
- raincole 3y agoBut rule-based machine translation, from what I've seen, is just so bad. ChatGPT (and other LLM) is miles ahead. After seeing what ChatGPT does, I can't even call rule-based machine translation "tranlation". *Disclaimer: as someone who's not an AI researcher but did quite some human translation works before.
- otikik 3y agoPerhaps using a ML to craft the deterministic rules and then have a human go over them is the sweet spot.
- astrange 3y agoRules could never work for translation unless the incoming text was formatted in a specific way. Eg, you just couldn't translate a conversation transcript in a pro-drop language like Japanese into English sentence-by-sentence, because the original text just wouldn't have sentences in it. So you need some "intelligence" to know who is saying what.
- rugina 3y agoI think NM translation was broken all along. Not in the neural network part but in choosing the right answer. https://aclanthology.org/2020.coling-main.398.pdf https://aclanthology.org/2020.coling-main.398.pdf
- astrange 3y agoSince LLMs are loosely based on NM models, it seems research on newer sampling methods like Mirostat might help here.
- earngurus234 3y ago[dead]
- chrisweekly 3y agoI've heard AI described as the payday loan (or "high-interest credit card") of technical debt.
- x0x0 3y agomysteriously with a helping of random too!
- __loam 3y agoThis is why hallucinations will never be fixed in language models. That's just how they work.
- munk-a 3y agoI think - I hope, rather - that technically minded people who are advocating for the use of ML understand the short comings and hallucinations... but we need to be frank about the fact that the business layer above us (with a few rare exceptions) absolutely does not understand the limitations of AI and views it as a magic box where they type in "Write me a story about a bunny" and get twelve paragraphs of text out. As someone working in a healthcare adjacent field I've seen the glint in executive's eyes when talking about AI and it can provide real benefits in data summarization and annotation assistance... but there are limits to what you should trust it with and if it's something big-i Important then you'll always want to have a human vetting step.
- munificent 3y ago> I hope, rather - that technically minded people who are advocating for the use of ML understand the short comings and hallucinations. The people I see who are most excited about ML are business types who just see it as a black boxes that makes stock valuation go vroom. The people that deeply love building things, really enjoy the process of making itself, are profoundly sceptical. I look at generative AI as sort of like an army of free interns. If your idea of a fun way to make a thing is to dictate orders to a horde of well-meaning but untrained highly-caffienated interns, then using generative AI to make your thing is probably thrilling. You get to feel like an executive producer who can make a lot of stuff happen by simply prompting someone/something to do your bidding. But if you actually care about the grit and texture of actual creation, then that workflow isn't exactly appealing.
- spacemadness 3y agoThey wouldn’t think this way if stock investors weren’t so often such naive lemmings ready to jump off yet another cliff with each other.
- fragmede 3y agoWe get it, you're skeptical of the current hype bubble. But that's one helluva no true Scotsman you've got going on there. Because a true builder, one that deeply loves building things wouldn't want to use text to create an image. Anyone who does is a business type or an executive producer. A true builder wouldn't think about what they want to do in such nasty thing as words. Creation comes from the soul, which we all know machines, and business people, don't have. Using English, instead of C, to get a computer to do something doesn't turn you into a beaurocrat any more than using Python or Javascript instead does. Only a person that truly loves building things, far deeper than you'll ever know, someone that's never programmed in a compiled language, would get that.
- fuzztester 3y agoSame here with YouTube, assuming they use ML, which is likely. They routinely give me brain-dead suggestions such as to watch a video I just watched today or yesterday, among other absurdities.
- makeitdouble 3y agoI think it's probably pushing pattern it sees in other users. There's videos I'll watch multiple times, music videos are the obvious kind, but for some others I'm just not watching/understanding it the first time and will go back and rewatch later. But I guess youtube has no way to understand which one I'll rewatch and which other I don't want to see ever again, and if my behavior is used as training data for the other users like you, they're probably screwed.
- godshatter 3y agoA simple "rewatch?" line along the top would make this problem not so brain dead bad, imho. Without it you just think the algorithm is bad (although maybe it is? I don't know).
- sakesun 3y agoInstall "Unhook" chrome extension. That changed my life.
- 998244353 3y agoFor what it's worth, I do not remember a time when YouTube's suggestions or search results were good. Absurdities like that happened 10 and 15 years ago as well. These days my biggest gripe is that they put unrelated ragebait or clickbait videos in search results that I very clearly did not search for - often about American politics.
- peoplenotbots 3y agoLong long time ago; youtube "staff" would manually put certain videos on the top of the front page when they started. Im sure there we're biases and prioritization of marketing dollars but at least there was human recommending it compared to poorly recorded early family guy clips. I dont know when they stopped manually adding "editors/staff" choice videos but I recall some of my favorite early youtubers like CGPGgrey claim that recommendation built the career.
- jokoon 3y agothat's not something ML people would like to hear
- JohnFen 3y agoWell, it depends on the ML person. I work on industrial ML and DL systems every day and I'm the one who made that comment.
- oblio 3y agoIs ML the new SOAP? Looks like a silver bullet and 5 years later you're drowning in complexity for no discernible reason?
- bitwize 3y agoML is somewhere between the new SOAP and the new cryptocurrency.
- peoplenotbots 3y agoWell thats grim
- karma_pharmer 3y agoDear sir, may I interest you in the initial coin airdrop of WSDLCoin? It is going straight to the moon.
- ajross 3y agoSo... obviously SOAP was dumb[1], and lots of people saw that at the time. But SOAP was dumb in obvious ways, and it failed for obvious reasons, and really no one was surprised at all. ML isn't like that. It's new. It's different. It may not succeed in the ways we expect; it may even look dumb in hindsight. But it absolutely represents a genuinely new paradigm for computing and is worth studying and understanding on that basis. We look back to SOAP and see something that might as well be forgotten. We'll never look back to the dawn of AI and forget what it was about. [1] For anyone who missed that particular long-sunken boat, SOAP was a RPC protocol like any other. Yes, that's really all it was. It did nothing special, or well, or that you couldn't do via trivially accessible alternative means. All it had was the right adjective ("XML" in this case) for the moment. It's otherwise forgettable, and forgotten.