6 ms·

I think a world wide pause is absolutely necessary. And this is a pause in something "better" for whatever definition than GPT4. We are on the cusp of the next

I think a world wide pause is absolutely necessary. And this is a pause in something "better" for whatever definition than GPT4. We are on the cusp of the next step function in humanities capabilities as a technological civilization. The upsides are enormous and I welcome them, but the downsides are even bigger and we should map this out before running off the damn cliff. Technologically, politically, economically. The fact that it would be hard, or difficult should not play into whether we have the discussion. This is the issue and reasoning that proliferated the bomb. The bomb didn't have to be dropped on Nagasaki and Hiroshima and unfettered bigai doesn't have to enter the winner take all capital arena. Even with just chatgpt4 level performance, millions of people are in the process of losing their jobs. I had to argue at a bar a couple weeks ago for a dev manager not to lay off his entire team of green junior data engineers. He claimed he could work along and replace the three people that reported to him... I not only saved those peoples jobs (hopefully) but also his own. There is no way he could have stayed afloat even managing requirements. Huge changes are coming and we aren't prepared. I'd rather have a 15 mph collision than a 55 mph one. I am not convinced that two AI researchers are qualified about the larger ramifications of their creations. They don't have a grounding all of the other skills necessary, are they politicians, economists, psychologists or philosophers?

alchemist1e9 4y ago
Ok so let’s say I don’t care what you say or any government says and so I’m not pausing anything and I have the resources to keep moving ahead. Now what do you suggest next?
- oh_sigh 4y ago
  The only good way to do this is wait until GPTx, where x=8 or 9 or whatever, where it can provide an argument so convincing for pausing AI research that every relevant party will abide by it willingly.
- mlinsey 4y ago
  GPT4 is expensive enough to train that most organizations with that high of a compute budget also have compliance departments and will not flat-out ignore a law like that. State actors overseas and the indie hackers with open-source models will not be deterred, but they are probably six months behind anyway. I don't think the six-month moratorium (as opposed to say, a three year moratorium) would be so impractical to implement, for that reason. But I'm also not sure what it would get us, the past six months of alignment work doesn't suggest six more months would solve the problem.
  - alchemist1e9 4y ago
    Nobody stopping anything ever. I predict we will have distributed training systems soon on peer to peer like network for collaboratively building free models to use, if we can all pitch in some GPUs we can get 100s of thousand of individuals working on training giant LLM models together. It’s an unsolved problem currently but should have a solution soon. Additionally today we can build sub-$10K systems to do self hosted inference on 65B weights models. I’m seriously confused why anybody thinks anyone will even remotely care about any kind of pause is asked for. For sure everyone with computers and a reasonable level of skills are now looking at this field. The progress will be exponential and uncontrollable in my opinion. I’m fine with that and I also think there is absolutely nothing that can stop it anyway.
    - mlinsey 4y ago
      > I’m seriously confused why anybody thinks anyone will even remotely care about any kind of pause is asked for. I can say that Google and Microsoft would definitely care, just as two specific instances of "anybody" that I have worked for and seen what their legal review process gating new product launches is like. You seem much more optimistic that the community efforts are <6 months behind those companies than I am; I would be happy to be wrong here and see an open-source self-hosted LLM with better than GPT4 performance by September.
      - alchemist1e9 4y ago
        I’ve read OpenAI only used around 4000 high end GPUs ($30K each) for around 4 months to train GPT4. Yes there are some reinforcement tricks that are still unclear but input data is basically known. I think people underestimate how many entities have the ability to easily acquire that amount of resources. It’s less than $200M to purchase that amount of hardware. Militaries spend $2-4M per missile. This is nothing for a lot of organizations. We already have self hosted LLM inference that reaches gpt 3.5 turbo level. Google and Microsoft are notoriously slow moving and probably less relevant than smaller faster operations like OpenAI.
angrais 4y ago
What "cusp of the next step function" are we on? Please be as specific as possible. >> Even with just chatgpt4 level performance, millions of people are in the process of losing their jobs What jobs specifically, in which sectors, and what aspects of the role (or is it the entire one) will be replaced by "AI"? Also, who is implementing all these changes? Who will run the "AI", who will pay for it?
- moreice 4y ago
  On Reddit, I've heard from people in the games industry that concept artists are being replaced by AI like Midjourney. Concept art can be generated via a prompt by the game designers and art directors.
- alexthehurst 4y ago
  This is an unfair debate tactic. https://en.wikipedia.org/wiki/Sealioning https://en.wikipedia.org/wiki/Sealioning
  - angrais 4y ago
    Interesting, thanks for the link. For reference, my debate approach was not sealioning. Given the extremely bold claims made by OP, I wanted to know if they had consider ed alternative views as their comment read as unrealistic "hype".
sebzim4500 4y ago
Serious question: How do you avoid making a model bigger than GPT-4 when you don't know anything about the architecture? I guess you could try comparing capabilities, but without access to the training data we have basically no effective way of running benchmarks against the GPT-4 api, even if you are one of the lucky few who already has API access.
imiric 4y ago
> The bomb didn't have to be dropped on Nagasaki and Hiroshima That doesn't seem like a fitting comparison, considering it's a highly debated topic to this day. > I had to argue at a bar a couple weeks ago for a dev manager not to lay off his entire team of green junior data engineers. This has more to do with that particular manager not understanding the current state of AI, and buying into the hype, than the technology reaching the point where developers are being made redundant by AI. That manager would've come to a swift realization of his mistake very quickly. So you managed to burst his bubble of delusion, but this is not a sign of AI taking over anytime soon.
- nerpderp82 4y ago
  Now that the mob has passed, I'd like to address your statement. > That doesn't seem like a fitting comparison, considering it's a highly debated topic to this day. A comparison can be apt independent of how inciteful the statement is. The dropping of the bomb was a showcase of force, it didn't have to happen on a population of people. The demonstration could have happened on Mt Fuji, or Okinawa. > These cities were largely untouched during the nightly bombing raids, and the Army Air Forces agreed to leave them off the target list so accurate assessment of the damage caused by the atomic bombs could be made. > The Target Committee stated that "It was agreed that psychological factors in the target selection were of great importance. Two aspects of this are (1) obtaining the greatest psychological effect against Japan and (2) making the initial use sufficiently spectacular for the importance of the weapon to be internationally recognized when publicity on it is released. ... Kyoto has the advantage of the people being more highly intelligent and hence better able to appreciate the significance of the weapon. Fucking immoral human experimentation. Ok, back to AI. We are on the same level of change, before and after nuclear manipulation, before and after AI. We don't need to be demonstrating on the economy, we have an opportunity to show this technology in isolation before just unleashing it on the world in a grotesque tech demo. The comparison fits.

alchemist1e9 4y ago

Ok so let’s say I don’t care what you say or any government says and so I’m not pausing anything and I have the resources to keep moving ahead. Now what do you suggest next?

oh_sigh 4y ago
The only good way to do this is wait until GPTx, where x=8 or 9 or whatever, where it can provide an argument so convincing for pausing AI research that every relevant party will abide by it willingly.
mlinsey 4y ago
GPT4 is expensive enough to train that most organizations with that high of a compute budget also have compliance departments and will not flat-out ignore a law like that. State actors overseas and the indie hackers with open-source models will not be deterred, but they are probably six months behind anyway. I don't think the six-month moratorium (as opposed to say, a three year moratorium) would be so impractical to implement, for that reason. But I'm also not sure what it would get us, the past six months of alignment work doesn't suggest six more months would solve the problem.
- alchemist1e9 4y ago
  Nobody stopping anything ever. I predict we will have distributed training systems soon on peer to peer like network for collaboratively building free models to use, if we can all pitch in some GPUs we can get 100s of thousand of individuals working on training giant LLM models together. It’s an unsolved problem currently but should have a solution soon. Additionally today we can build sub-$10K systems to do self hosted inference on 65B weights models. I’m seriously confused why anybody thinks anyone will even remotely care about any kind of pause is asked for. For sure everyone with computers and a reasonable level of skills are now looking at this field. The progress will be exponential and uncontrollable in my opinion. I’m fine with that and I also think there is absolutely nothing that can stop it anyway.
  - mlinsey 4y ago
    > I’m seriously confused why anybody thinks anyone will even remotely care about any kind of pause is asked for. I can say that Google and Microsoft would definitely care, just as two specific instances of "anybody" that I have worked for and seen what their legal review process gating new product launches is like. You seem much more optimistic that the community efforts are <6 months behind those companies than I am; I would be happy to be wrong here and see an open-source self-hosted LLM with better than GPT4 performance by September.
    - alchemist1e9 4y ago
      I’ve read OpenAI only used around 4000 high end GPUs ($30K each) for around 4 months to train GPT4. Yes there are some reinforcement tricks that are still unclear but input data is basically known. I think people underestimate how many entities have the ability to easily acquire that amount of resources. It’s less than $200M to purchase that amount of hardware. Militaries spend $2-4M per missile. This is nothing for a lot of organizations. We already have self hosted LLM inference that reaches gpt 3.5 turbo level. Google and Microsoft are notoriously slow moving and probably less relevant than smaller faster operations like OpenAI.

angrais 4y ago

What "cusp of the next step function" are we on? Please be as specific as possible. >> Even with just chatgpt4 level performance, millions of people are in the process of losing their jobs What jobs specifically, in which sectors, and what aspects of the role (or is it the entire one) will be replaced by "AI"? Also, who is implementing all these changes? Who will run the "AI", who will pay for it?

moreice 4y ago
On Reddit, I've heard from people in the games industry that concept artists are being replaced by AI like Midjourney. Concept art can be generated via a prompt by the game designers and art directors.
alexthehurst 4y ago
This is an unfair debate tactic. https://en.wikipedia.org/wiki/Sealioning https://en.wikipedia.org/wiki/Sealioning
- angrais 4y ago
  Interesting, thanks for the link. For reference, my debate approach was not sealioning. Given the extremely bold claims made by OP, I wanted to know if they had consider ed alternative views as their comment read as unrealistic "hype".

sebzim4500 4y ago

Serious question: How do you avoid making a model bigger than GPT-4 when you don't know anything about the architecture? I guess you could try comparing capabilities, but without access to the training data we have basically no effective way of running benchmarks against the GPT-4 api, even if you are one of the lucky few who already has API access.

imiric 4y ago

> The bomb didn't have to be dropped on Nagasaki and Hiroshima That doesn't seem like a fitting comparison, considering it's a highly debated topic to this day. > I had to argue at a bar a couple weeks ago for a dev manager not to lay off his entire team of green junior data engineers. This has more to do with that particular manager not understanding the current state of AI, and buying into the hype, than the technology reaching the point where developers are being made redundant by AI. That manager would've come to a swift realization of his mistake very quickly. So you managed to burst his bubble of delusion, but this is not a sign of AI taking over anytime soon.

nerpderp82 4y ago
Now that the mob has passed, I'd like to address your statement. > That doesn't seem like a fitting comparison, considering it's a highly debated topic to this day. A comparison can be apt independent of how inciteful the statement is. The dropping of the bomb was a showcase of force, it didn't have to happen on a population of people. The demonstration could have happened on Mt Fuji, or Okinawa. > These cities were largely untouched during the nightly bombing raids, and the Army Air Forces agreed to leave them off the target list so accurate assessment of the damage caused by the atomic bombs could be made. > The Target Committee stated that "It was agreed that psychological factors in the target selection were of great importance. Two aspects of this are (1) obtaining the greatest psychological effect against Japan and (2) making the initial use sufficiently spectacular for the importance of the weapon to be internationally recognized when publicity on it is released. ... Kyoto has the advantage of the people being more highly intelligent and hence better able to appreciate the significance of the weapon. Fucking immoral human experimentation. Ok, back to AI. We are on the same level of change, before and after nuclear manipulation, before and after AI. We don't need to be demonstrating on the economy, we have an opportunity to show this technology in isolation before just unleashing it on the world in a grotesque tech demo. The comparison fits.