5 ms·
The reason AI fails in Industry is that SKILL.md or other knowledge-injection methods do not guarantee compliance. AI just thinks it "knows better".
by jayanaka98 3mo ago
The reason AI fails in Industry is that SKILL.md or other knowledge-injection methods do not guarantee compliance. AI just thinks it "knows better".
- Tade0 3mo agoA friend of mine prepared an arsenal of hooks and the like to address this and LLMs still disobey them at times. I don't have high hopes that there exists a bulletproof solution to this.
- Kiln6125 3mo agoPersonally, for me that represents job security. Having a human with a high level of domain knowledge in the loop seems pretty required to get any meaningful results.
- dominotw 3mo agosolution is to always do what it has seen in training data and how it was RL. But ai companies dont tell you that. so you have to reverse engineer its training and stick to to that. These are no general purpose machines. They are shipping a subset mindset not general intelligence like they want us to belive .
- EliRivers 3mo agoA friend of mine prepared an arsenal of hooks and the like to address this and LLMs still disobey them at times. It's a model of language, yes? Trained on a big corpus of text. I have read a lot of stories and accounts in which people were told not to do something and inevitably they did it. Like, lots. Far more than stories and accounts in which people were told not to do something and they then didn't do it. If I'm reading a story or account of something, and it's really hammered home that they've been told not to do something, it's kind of inevitable that they will then do that. I'm not even an LLM and I noticed that's the way these things usually go. So is an LLM just doing what it's been trained to do? Sometimes in the stories and accounts, there's a whole lot of time and tension before the bad thing happens, but that's just part of the fun.
- noman-land 3mo agoSomeone is unlikely to relay a story where someone was asked not to do something and they didn't. There is no story there.
- steve1977 3mo agoNot sure if this was ironic. I guess it mainly fails because a lot of knowledge and experience is intuitive and not codified.
- contravariant 3mo agoIf compliance was the main issue we wouldn't have had to invent ways for computers to do something other than exactly what they were asked.
- bigstrat2003 3mo agoWe didn't have to do that. It is, in fact, extremely stupid that we have done that. Computers are valuable because they are fast and deterministic. Fast but stochastic has no value.
- teiferer 3mo ago> Fast but stochastic has no value. He valuations of a bunch of AI unicorns disagree.
- nekusar 3mo agoIts all about a longshot gamble to replace workers and wages.
- teiferer 3mo agoI don't understand how that angle keeps surviving. It is in the interest of the rich and powerful to keep the vast majority of society in jobs and pay them a wage. That's what they use to consume the things that drive the economy which ultimately makes the rich richer. The narrative that the rich want to get rid of workers is as nonsensical today as it was decades ago when I heard it the first time. It doesn't make any sense.
- gibbitz 3mo agoThose employee wages for a product is a 20th century way of making money. Taking investor cash and paying it back with supplier "investments" is how "capitalism" works in today's economy. The labor market and products is just the money laundering cover story for ponzi schemes. It's way faster and more lucrative taking money from the rich in big chunks than taking it from the poor in teensy amounts. This is why everything sucks now, no one cares about the product.