5 ms·

> As a species we are so not ready for this tech. I basically agree, but not because of how individuals are reacting to it. On a societal level we're failing t

> As a species we are so not ready for this tech. I basically agree, but not because of how individuals are reacting to it. On a societal level we're failing to stop megacorps from selling this thing as intelligent, a virtual assistant you can chat with. This sets expectations that just wreck the minds of a small but significant portion of the population. If we had developed LLMs in a lab and released them in the form of papers and Python projects with model weights with clear descriptions of what they're capable of, like a responsible scientific endeavor, then we'd not be seeing the problems we are, even with public access. What's killing us is MBAs and Salesmen. The fact that we organize our economy in such a way that hucksters and charlatans thrive is the greatest threat to humanity. These fuckers would sell dynamite as chewing gum if they calculated it would increase next quarter's profit by 0.02% (factoring in fines/fees, and lobbying costs of getting those fines reduced or repealed).

throwaway290 1y ago
This is beyond naive. who would bother to develop it in a lab and publish pro bono if it can never be commercial? Making money is why the most capitalist countries developed this tech. like most other tech and even if it is only published, do you think we wouldn't run chatbots ourselves at home with same results? remember how Google engineer went off the rails thinking it is conscious while working on this stuff, do you think he also was misled by adverts or something? or big corps won't buy the patent and run with it commercially advertising it like they do anyway? or if you pretend money and big corps don't exist and we deploy it for free for friends, same problem? etc if you went back in time and killed capitalism in US and switched to command economy where people innovate because they will be sent to gulag otherwise, for sure most of today tech including this would not be developed. but it seems like a pointless exercise. instead what should happen is all these megacorps are sued for copyright infringement, fined and shut down. the model won't be smart enough to sweet talk ppl into suicide if megacorps can't infringe our copyrights to train it.
- saubeidl 1y ago
  You just perfectly described why capitalism is the problem.
- idle_zealot 1y ago
  Eh, we fund a lot of research through grants. It may not have been 2022, but a society without as strong of a profit motive would have discovered transformer models and LLMs eventually. They probably wouldn't have scaled up datacenter production and tried to shove a chatbox into every digital interface though. > and even if it is only published, do you think we wouldn't run chatbots ourselves at home with same results Yes, my point is exactly that I don't think the results would be the same if people were running it themselves at home, without the marketing machine running full bore to convince the world that these impressive toys are machine oracles. You'd end up with a handful of crazies, that's unavoidable, but it wouldn't be a civilizational threat.
- tedivm 1y ago
  That's a bit of a stretch. Lots of stuff that gets published still has patents associated with it. Just because something is done in the open doesn't mean it can't be commercialized.
- LamerBeeI 1y ago
  you're making this comment on the world wide web, which was invented and given away for free - likely against a linux machine, an open and free project
  - throwaway290 1y ago
    you are making this comment on computer hardware which is a prerequisite for www and which was invented and made because it's profitable
ben_w 1y ago
> If we had developed LLMs in a lab and released them in the form of papers and Python projects with model weights with clear descriptions of what they're capable of, like a responsible scientific endeavor, then we'd not be seeing the problems we are, even with public access. I'm now thinking of all of the times people here of sarcastically stated "OpenAI thought GPT-2 was too dangerous to release"*, as if danger only comes immediately and severely or not at all. * wasn't even what OpenAI said, they just proposed setting a norm of caution because a release can't be undone
- dingnuts 1y ago
  they meant that to be interpreted as a statement of how good GPT-2 is but the real problem is how they've marketed everything that came after to people who can't know better it's EXACTLY the same situation as Musk selling "Full Self Driving" and then playing dumb when people go to sleep behind the wheel of their Tesla These rich men should be in prison for this kind of false advertising
  - ben_w 1y ago
    > they meant that to be interpreted as a statement of how good GPT-2 is Every previous time someone says something like this, I've looked at the original blog post again just to make sure I didn't miss something. OpenAI's own words for GPT-2 never read like a boast to me. Caution, not a boast. News may have bigged it up, but news does that for everything. I mean just compare it to the GPT-5 release: they're not shy or subtle when they actually want to boast.
    - jononor 1y ago
      Note that OpenAI is not the same company now compared to then. All the cautious and responsible people have left or been forced out.
      - ben_w 1y ago
        Sure, yes. I'm just frustrated that "doing it right" gets people laughed at both at the time and for several years later, right up until the general dangers they were publicly concerned about manifest, and then people complain about people not doing something to prevent the outcome that they were previously laughed at for attempting to prevent. Those same cautious people who left one way or another, they were the ones most directly mocked for daring to consider the possibility their new thing might be risky and caution might be wise.
jjani 1y ago
> What's killing us is MBAs and Salesmen. SamA is not an MBA. He did CS for 2 years and dropped out to build a startup. He's YC personified, and the person most responsible for the phenomenon you're talking about. Take that for what you will.
- deleted 1y ago
  [deleted]
- dingnuts 1y ago
  oh but he is a salesman
- LamerBeeI 1y ago
  that's a dumb distinction really
- insane_dreamer 1y ago
  he's just a salesman without an MBA, which is maybe even worse
- drakythe 1y ago
  People are what they do. Pretty sure the man has been a Salesman ever since he started seeking funding for OpenAI.
- evmaki 1y ago
  So dropping out of CS to start selling something was more important to him than 2 more years of CS education. Maybe he realized that continuing his engineering education was unnecessary because he preferred selling things. Sounds like a salesman.
- queenkjuul 1y ago
  So he's a salesman
mallowdram 1y ago
The tech was never ready, it's built on faulty units. Pull the plug.
saubeidl 1y ago
> What's killing us is MBAs and Salesmen Almost. Just take it like an inch further. What's really killing us is capitalism. MBAs and Salesmen are just the personification.
- idle_zealot 1y ago
  I tend to shy away from that sort of rhetoric on HN. Saying that the problem is capitalism and we need socialism or whatever gets you downvoted to hell. Making the underlying arguments that lead to that conclusion gets you upvotes, exposing more people to those ideas. People, even smart people, especially in the US, hear socialism or communism and immediately think of the USSR or Maoist China. The terms are very unpopular. The ideas are the important part though, and they're pretty popular.
  - saubeidl 1y ago
    I find that part of being intellectually curious and a good engineer is questioning systems. If we had a systems architecture in an engineering project that kept leading to bad results, we would consider rearchitecting how said system is run. I believe it is important to make it clear to people that capitalism, too, is just a (imo deeply flawed) systems design. It is not a natural constant. If we find it leads to negative outcomes (which I strongly believe), we can and should talk about rearchitecting.
    - kiba 1y ago
      What people describe capitalism is really just monopolies. Capitalism meanwhile is an utterly useless label. People have their own conception of what capitalism mean. Monopolies are connected to the housing crisis as to privileges like copyright. They're unwarranted or undertaxed government granted privileges.
  - dingnuts 1y ago
    those stupid ignorant Americans who associate the United Soviet Socialist Republic with socialism. where did they ever get the idea that socialism has ever been tried??? don't worry, I associate Marxism with REAL socialist regimes, like Cambodia
Terr_ 1y ago
> I basically agree, but not because of how individuals are reacting to it. On a societal level we're failing to stop megacorps from selling this thing as intelligent, a virtual assistant you can chat with. Analogy: I'm not scared and surprised to hear some consumers are dangerously allergic to peanuts... However I am flabbergasted that there's multi-billion dollar industry somehow selling frickin' Peanut Immortality Panacea Serum and way too many people believe in it.
- jononor 1y ago
  Haven't you heard? All the other foodsources are going to dissappear. Better get good at eating peanuts before it is too late!
goalieca 1y ago
> On a societal level we're failing to stop megacorps from selling this thing as intelligent, a virtual assistant you can chat with. You nailed it here. From the LLM generated search result asking how an LLM works. But this is not what the end users are being told. >They learn to predict the next word in a sentence based on the context of the preceding words, allowing them to generate coherent and contextually relevant text My worry is now the users are being entrapped. The LLM is "telling" them something insane and now they're being reported to the police.
dostick 1y ago
What’s killing us is capitalism, an ill system that was predicted to fail 140 years ago but every country picked it up and tried regardless, and it failed in all 100+ experiments over 100 years. It’s always end up in inequality and suffering because it amplifies greed, and simply does not accept human being for what they are, your value being value of your learned occupation. What we end up with about is 50% of economy is useless “bullshit jobs”that are there just to employ people because capitalism can’t work otherwise! Did you notice that every science, from physics to biology has scientists making discoveries and pushing ahead. Yet nobody ever tried to update and design new political economic system (not socialism/communism). As if capitalism was given by good and should not be questioned. Together with ancient and not updated according to modern science democracy, that was designed for one representative for 100 people in Ancient Greece, when they knew only feudalism and so invented electable kings. And now we have one representative for 100,000 yet nobody checked if it supposed to work like that. Instead of flawed meat computers representing people we should have switched to actual computers long ago.

throwaway290 1y ago

This is beyond naive. who would bother to develop it in a lab and publish pro bono if it can never be commercial? Making money is why the most capitalist countries developed this tech. like most other tech and even if it is only published, do you think we wouldn't run chatbots ourselves at home with same results? remember how Google engineer went off the rails thinking it is conscious while working on this stuff, do you think he also was misled by adverts or something? or big corps won't buy the patent and run with it commercially advertising it like they do anyway? or if you pretend money and big corps don't exist and we deploy it for free for friends, same problem? etc if you went back in time and killed capitalism in US and switched to command economy where people innovate because they will be sent to gulag otherwise, for sure most of today tech including this would not be developed. but it seems like a pointless exercise. instead what should happen is all these megacorps are sued for copyright infringement, fined and shut down. the model won't be smart enough to sweet talk ppl into suicide if megacorps can't infringe our copyrights to train it.

saubeidl 1y ago
You just perfectly described why capitalism is the problem.
idle_zealot 1y ago
Eh, we fund a lot of research through grants. It may not have been 2022, but a society without as strong of a profit motive would have discovered transformer models and LLMs eventually. They probably wouldn't have scaled up datacenter production and tried to shove a chatbox into every digital interface though. > and even if it is only published, do you think we wouldn't run chatbots ourselves at home with same results Yes, my point is exactly that I don't think the results would be the same if people were running it themselves at home, without the marketing machine running full bore to convince the world that these impressive toys are machine oracles. You'd end up with a handful of crazies, that's unavoidable, but it wouldn't be a civilizational threat.
tedivm 1y ago
That's a bit of a stretch. Lots of stuff that gets published still has patents associated with it. Just because something is done in the open doesn't mean it can't be commercialized.
LamerBeeI 1y ago
you're making this comment on the world wide web, which was invented and given away for free - likely against a linux machine, an open and free project
- throwaway290 1y ago
  you are making this comment on computer hardware which is a prerequisite for www and which was invented and made because it's profitable

ben_w 1y ago

> If we had developed LLMs in a lab and released them in the form of papers and Python projects with model weights with clear descriptions of what they're capable of, like a responsible scientific endeavor, then we'd not be seeing the problems we are, even with public access. I'm now thinking of all of the times people here of sarcastically stated "OpenAI thought GPT-2 was too dangerous to release"*, as if danger only comes immediately and severely or not at all. * wasn't even what OpenAI said, they just proposed setting a norm of caution because a release can't be undone

dingnuts 1y ago
they meant that to be interpreted as a statement of how good GPT-2 is but the real problem is how they've marketed everything that came after to people who can't know better it's EXACTLY the same situation as Musk selling "Full Self Driving" and then playing dumb when people go to sleep behind the wheel of their Tesla These rich men should be in prison for this kind of false advertising
- ben_w 1y ago
  > they meant that to be interpreted as a statement of how good GPT-2 is Every previous time someone says something like this, I've looked at the original blog post again just to make sure I didn't miss something. OpenAI's own words for GPT-2 never read like a boast to me. Caution, not a boast. News may have bigged it up, but news does that for everything. I mean just compare it to the GPT-5 release: they're not shy or subtle when they actually want to boast.
  - jononor 1y ago
    Note that OpenAI is not the same company now compared to then. All the cautious and responsible people have left or been forced out.
    - ben_w 1y ago
      Sure, yes. I'm just frustrated that "doing it right" gets people laughed at both at the time and for several years later, right up until the general dangers they were publicly concerned about manifest, and then people complain about people not doing something to prevent the outcome that they were previously laughed at for attempting to prevent. Those same cautious people who left one way or another, they were the ones most directly mocked for daring to consider the possibility their new thing might be risky and caution might be wise.

jjani 1y ago

> What's killing us is MBAs and Salesmen. SamA is not an MBA. He did CS for 2 years and dropped out to build a startup. He's YC personified, and the person most responsible for the phenomenon you're talking about. Take that for what you will.

deleted 1y ago
[deleted]
dingnuts 1y ago
oh but he is a salesman
LamerBeeI 1y ago
that's a dumb distinction really
insane_dreamer 1y ago
he's just a salesman without an MBA, which is maybe even worse
drakythe 1y ago
People are what they do. Pretty sure the man has been a Salesman ever since he started seeking funding for OpenAI.
evmaki 1y ago
So dropping out of CS to start selling something was more important to him than 2 more years of CS education. Maybe he realized that continuing his engineering education was unnecessary because he preferred selling things. Sounds like a salesman.
queenkjuul 1y ago
So he's a salesman

mallowdram 1y ago

The tech was never ready, it's built on faulty units. Pull the plug.

saubeidl 1y ago

> What's killing us is MBAs and Salesmen Almost. Just take it like an inch further. What's really killing us is capitalism. MBAs and Salesmen are just the personification.

idle_zealot 1y ago
I tend to shy away from that sort of rhetoric on HN. Saying that the problem is capitalism and we need socialism or whatever gets you downvoted to hell. Making the underlying arguments that lead to that conclusion gets you upvotes, exposing more people to those ideas. People, even smart people, especially in the US, hear socialism or communism and immediately think of the USSR or Maoist China. The terms are very unpopular. The ideas are the important part though, and they're pretty popular.
- saubeidl 1y ago
  I find that part of being intellectually curious and a good engineer is questioning systems. If we had a systems architecture in an engineering project that kept leading to bad results, we would consider rearchitecting how said system is run. I believe it is important to make it clear to people that capitalism, too, is just a (imo deeply flawed) systems design. It is not a natural constant. If we find it leads to negative outcomes (which I strongly believe), we can and should talk about rearchitecting.
  - kiba 1y ago
    What people describe capitalism is really just monopolies. Capitalism meanwhile is an utterly useless label. People have their own conception of what capitalism mean. Monopolies are connected to the housing crisis as to privileges like copyright. They're unwarranted or undertaxed government granted privileges.
- dingnuts 1y ago
  those stupid ignorant Americans who associate the United Soviet Socialist Republic with socialism. where did they ever get the idea that socialism has ever been tried??? don't worry, I associate Marxism with REAL socialist regimes, like Cambodia

Terr_ 1y ago

> I basically agree, but not because of how individuals are reacting to it. On a societal level we're failing to stop megacorps from selling this thing as intelligent, a virtual assistant you can chat with. Analogy: I'm not scared and surprised to hear some consumers are dangerously allergic to peanuts... However I am flabbergasted that there's multi-billion dollar industry somehow selling frickin' Peanut Immortality Panacea Serum and way too many people believe in it.

jononor 1y ago
Haven't you heard? All the other foodsources are going to dissappear. Better get good at eating peanuts before it is too late!

goalieca 1y ago

> On a societal level we're failing to stop megacorps from selling this thing as intelligent, a virtual assistant you can chat with. You nailed it here. From the LLM generated search result asking how an LLM works. But this is not what the end users are being told. >They learn to predict the next word in a sentence based on the context of the preceding words, allowing them to generate coherent and contextually relevant text My worry is now the users are being entrapped. The LLM is "telling" them something insane and now they're being reported to the police.

dostick 1y ago

What’s killing us is capitalism, an ill system that was predicted to fail 140 years ago but every country picked it up and tried regardless, and it failed in all 100+ experiments over 100 years. It’s always end up in inequality and suffering because it amplifies greed, and simply does not accept human being for what they are, your value being value of your learned occupation. What we end up with about is 50% of economy is useless “bullshit jobs”that are there just to employ people because capitalism can’t work otherwise! Did you notice that every science, from physics to biology has scientists making discoveries and pushing ahead. Yet nobody ever tried to update and design new political economic system (not socialism/communism). As if capitalism was given by good and should not be questioned. Together with ancient and not updated according to modern science democracy, that was designed for one representative for 100 people in Ancient Greece, when they knew only feudalism and so invented electable kings. And now we have one representative for 100,000 yet nobody checked if it supposed to work like that. Instead of flawed meat computers representing people we should have switched to actual computers long ago.