4 ms·

Some of us here may be old enough to remember the days when Microsoft was pushing ActiveX controls as the future of the web. Basically they were binary componen

Some of us here may be old enough to remember the days when Microsoft was pushing ActiveX controls as the future of the web. Basically they were binary components that would run in your browser and weren't really effectively sandboxed. Those of us that pointed out this was a major security hole were ignored. Inevitably carnage followed. People blindly accepting the output of LLMs seems similarly crazy to me and I think it's only a matter of time before we face a real reckoning over this. The lesson here I think is just because a lot of people are advocating something that seems reckless doesn't mean it isn't reckless. https://www.howtogeek.com/162282/what-activex-controls-are-and-why-theyre-dangerous/ https://www.howtogeek.com/162282/what-activex-controls-are-a...

gruez 3y ago
>Some of us here may be old enough to remember the days when Microsoft was pushing ActiveX controls as the future of the web. Basically they were binary components that would run in your browser and weren't really effectively sandboxed. That's basically what's happening every time a developer runs "npm install" or "pip install". Sure, it technically not be a "binary", but it makes no practical difference given that no code scrutiny is given the majority of the time. The introduction of LLMs doesn't change much. People are already predisposed to copy paste random commands from blogs/forums. LLM is just one more source on top of that.
- scruple 3y ago
  > it makes no practical difference given that no code scrutiny is given the majority of the time. I was thinking about this the other day. I wonder what my leadership would say if I told them I spent the day scrutinizing some of our open source dependencies. I assume even a day would be treated as wasted time, especially on the product side. FWIW, I used to do this back in the early Rails day and was encouraged to do so. I ended up contributing heavily to the Rails ecosystem because of it, and it was all encouraged by my employer at the time, but they were a relatively small startup at the time and viewed things very differently than the FAANG I work for today.
ben_w 3y ago
> People blindly accepting the output of LLMs seems similarly crazy to me and I think it's only a matter of time before we face a real reckoning over this. The lesson here I think is just because a lot of people are advocating something that seems reckless doesn't mean it isn't reckless. Irony in the "Greek tragedy" sense (but who is εἴρων and who is ἀλαζών?) So many online arguments about AI safety, where someone says "it's fine: just don't let it out of the box", and here it is, accessible on the internet — and for some, even this isn't enough: they want the weights; and for some, even the weights aren't enough, they want the training data. So many arguments where someone says "it's fine: just switch it off if it goes wrong", and not only is the switch out of reach to those affected, most people aren't even checking if it's going wrong in the first place. The labs themselves ring the alarm bells, put disclaimers on the start page of their own products, call for binding regulations, and there's always someone dismissing this as a "5D chess" marketing move. So many where someone says "it's just speech, what can it do?", etc. etc. I'm glad the AI labs themselves have so many signatories to the "let's pause for a bit" open letter. Cool though the tech is, it's definitely acting outside the expectations many have for it, and a simple dichotomy of "better/worse" isn't sufficient to describe the ways in which it is outside expectations given it can be both at the same time.
- rep_lodsb 3y ago
  The real danger isn't that "too smart" unaligned AI might execute some kind of deliberate plan for taking over the world, it's that there are massive numbers of people so stupid that they will blindly follow the instructions of stochastic parrot bullshit generators. And it is very clear that the AI companies do want you to be scared of the former, because it makes their tech seem more impressive and distracts from its very real flaws. Stupidity has of course always existed and you can't regulate it away, but modern tech and the business interests behind it seem to me to be making it worse and potentially more destructive. I'm not talking about just the latest hype here, it has been a trend for more than a decade now: "don't think for yourself, just consult the magic oracle in your pocket that surely knows best and isn't purposely designed to manipulate you into buying more crap and subscribing to the worldview of a small elite" But like with the AI apocalypse, or any conspiracy theory, the really scary thing is that there doesn't have to be some kind of intelligent plan behind it, something that you could understand or fight against, or maybe join in because you think they have the right goals. Rather, it's just mindless forces of the market and social dynamics. I really think it's all going to collapse within another 15 years or so, and can only hope that civilization will recover and perhaps learn from the mistakes made.
deergomoo 3y ago
> I think it's only a matter of time before we face a real reckoning over this I’m still waiting for a similar reckoning over the average JS project’s tendency to pull in hundreds of largely unvetted transitive dependencies. But, aside from left-pad, it seems not to have really happened…at least that we know of.
xotesos 3y ago
There is so much wisdom in the old saying you can lead a horse to water but you can't make them drink. Humans are just such stubborn creatures. chatGPT3.5 hallucinated packages. chatGPT4 has found me about 10 awesome python packages I didn't even know existed. Personally, I don't really care if anyone drinks or not. I am super hydrated and if people want to stay dehydrated they are just going to get smoked.
- dazilcher 3y ago
  Why assume everyone is dehydrated? Here's another ol' saying: in the middle of the desert, toilet water tastes like Fiji.
- crtasm 3y ago
  Are these packages somehow not findable on github/pypy/search engines?
pabs3 3y ago
> People blindly accepting the output of LLMs There are already real-world consequences of this: https://arstechnica.com/tech-policy/2023/05/lawyer-cited-6-fake-cases-made-up-by-chatgpt-judge-calls-it-unprecedented/ https://arstechnica.com/tech-policy/2023/05/lawyer-cited-6-f... https://www.reuters.com/legal/transactional/us-judge-orders-lawyers-sign-ai-pledge-warning-they-make-stuff-up-2023-05-31/ https://www.reuters.com/legal/transactional/us-judge-orders-... Before this there were already lots of real world consequences of "computer says no" anyway, so its just a continuation and perhaps escalation of that.

gruez 3y ago

>Some of us here may be old enough to remember the days when Microsoft was pushing ActiveX controls as the future of the web. Basically they were binary components that would run in your browser and weren't really effectively sandboxed. That's basically what's happening every time a developer runs "npm install" or "pip install". Sure, it technically not be a "binary", but it makes no practical difference given that no code scrutiny is given the majority of the time. The introduction of LLMs doesn't change much. People are already predisposed to copy paste random commands from blogs/forums. LLM is just one more source on top of that.

scruple 3y ago
> it makes no practical difference given that no code scrutiny is given the majority of the time. I was thinking about this the other day. I wonder what my leadership would say if I told them I spent the day scrutinizing some of our open source dependencies. I assume even a day would be treated as wasted time, especially on the product side. FWIW, I used to do this back in the early Rails day and was encouraged to do so. I ended up contributing heavily to the Rails ecosystem because of it, and it was all encouraged by my employer at the time, but they were a relatively small startup at the time and viewed things very differently than the FAANG I work for today.

ben_w 3y ago

> People blindly accepting the output of LLMs seems similarly crazy to me and I think it's only a matter of time before we face a real reckoning over this. The lesson here I think is just because a lot of people are advocating something that seems reckless doesn't mean it isn't reckless. Irony in the "Greek tragedy" sense (but who is εἴρων and who is ἀλαζών?) So many online arguments about AI safety, where someone says "it's fine: just don't let it out of the box", and here it is, accessible on the internet — and for some, even this isn't enough: they want the weights; and for some, even the weights aren't enough, they want the training data. So many arguments where someone says "it's fine: just switch it off if it goes wrong", and not only is the switch out of reach to those affected, most people aren't even checking if it's going wrong in the first place. The labs themselves ring the alarm bells, put disclaimers on the start page of their own products, call for binding regulations, and there's always someone dismissing this as a "5D chess" marketing move. So many where someone says "it's just speech, what can it do?", etc. etc. I'm glad the AI labs themselves have so many signatories to the "let's pause for a bit" open letter. Cool though the tech is, it's definitely acting outside the expectations many have for it, and a simple dichotomy of "better/worse" isn't sufficient to describe the ways in which it is outside expectations given it can be both at the same time.

rep_lodsb 3y ago
The real danger isn't that "too smart" unaligned AI might execute some kind of deliberate plan for taking over the world, it's that there are massive numbers of people so stupid that they will blindly follow the instructions of stochastic parrot bullshit generators. And it is very clear that the AI companies do want you to be scared of the former, because it makes their tech seem more impressive and distracts from its very real flaws. Stupidity has of course always existed and you can't regulate it away, but modern tech and the business interests behind it seem to me to be making it worse and potentially more destructive. I'm not talking about just the latest hype here, it has been a trend for more than a decade now: "don't think for yourself, just consult the magic oracle in your pocket that surely knows best and isn't purposely designed to manipulate you into buying more crap and subscribing to the worldview of a small elite" But like with the AI apocalypse, or any conspiracy theory, the really scary thing is that there doesn't have to be some kind of intelligent plan behind it, something that you could understand or fight against, or maybe join in because you think they have the right goals. Rather, it's just mindless forces of the market and social dynamics. I really think it's all going to collapse within another 15 years or so, and can only hope that civilization will recover and perhaps learn from the mistakes made.

deergomoo 3y ago

> I think it's only a matter of time before we face a real reckoning over this I’m still waiting for a similar reckoning over the average JS project’s tendency to pull in hundreds of largely unvetted transitive dependencies. But, aside from left-pad, it seems not to have really happened…at least that we know of.

xotesos 3y ago

There is so much wisdom in the old saying you can lead a horse to water but you can't make them drink. Humans are just such stubborn creatures. chatGPT3.5 hallucinated packages. chatGPT4 has found me about 10 awesome python packages I didn't even know existed. Personally, I don't really care if anyone drinks or not. I am super hydrated and if people want to stay dehydrated they are just going to get smoked.

dazilcher 3y ago
Why assume everyone is dehydrated? Here's another ol' saying: in the middle of the desert, toilet water tastes like Fiji.
crtasm 3y ago
Are these packages somehow not findable on github/pypy/search engines?

pabs3 3y ago

> People blindly accepting the output of LLMs There are already real-world consequences of this: https://arstechnica.com/tech-policy/2023/05/lawyer-cited-6-fake-cases-made-up-by-chatgpt-judge-calls-it-unprecedented/ https://arstechnica.com/tech-policy/2023/05/lawyer-cited-6-f... https://www.reuters.com/legal/transactional/us-judge-orders-lawyers-sign-ai-pledge-warning-they-make-stuff-up-2023-05-31/ https://www.reuters.com/legal/transactional/us-judge-orders-... Before this there were already lots of real world consequences of "computer says no" anyway, so its just a continuation and perhaps escalation of that.