12 ms·
It seems the only reason all of these competitive models are getting released is because you have a number of big players probably freaking out that somebody el
by xt00 4y ago
It seems the only reason all of these competitive models are getting released is because you have a number of big players probably freaking out that somebody else is going to break out into a huge lead. So while the flood gates are open people should be quickly figuring out how to do as much stuff as possible without any centralized company controlling it. I would imagine everybody assumed the models released these days will be obsolete before long so it’s low risk. But this is like early internet days.. but this time we should assume all of the centralized servers are user hostile and we should figure out how to work around them as quickly as they roll them out. The author and others are doing great work to prevent this stuff from being locked away behind costly apis and censorship.
- worldsayshi 4y agoIf the barrier for entry is low enough for several players to enter the field this fast - I wonder what could raise the barrier? The models getting bigger I suppose.
- hoseja 4y agoSoon you'll need a government license to purchase serious compute.
- valine 4y agoOur saving grace seems to be the insatiable push by the gaming industry for better graphics at higher resolutions. Their vision for real-time path traced graphics can’t happen without considerable ML horsepower on consumer level graphics cards.
- zirgs 4y agoYesterday's "serious compute" is today's mid-range PC.
- lvncelot 4y agoA few months (weeks?) ago I would've said that this already was the case for language models. It's absolutely mind-blowing to me what is happening here - same with stable diffusion. Once Dall-E was out, I was sure that there was no way that anything like this could be run on consumer hardware. I'm very happy to be proven wrong. In a way, things are still moving in this direction, though. 8 or so years ago it was more or less possible to train those models yourself to a certain degree of usefulness, as well, and I think we've currently moved way past any feasibility for that.
- TuringTest 4y agoFortunately, there still are some possibilities to improve training efficiency and reducing model size by doing more guided attentional learning. This will make feasible to train models at least as good as the current batch (though probably the big players will use those same optimizations to create much better large models).
- MacsHeadroom 4y agoLLaMA can be fine tuned in hours on a consumer GPU or in a free Colab with just 12GB of VRAM, and soon 6GB in 4bit training, using PEFT. https://github.com/zphang/minimal-llama#peft-fine-tuning-with-8-bit-and-pipeline-parallel https://github.com/zphang/minimal-llama#peft-fine-tuning-wit...
- mx20 4y agoThe Vice Chairman of Microsoft already mentions that he is open to regulation. The EU also is working on plans to regulate AI. So you probably only are allowed to use AI in the future if it's approved by something like the FD(A)A.
- worldsayshi 4y agoMaybe I'm having a looped view of this but I fail to see that regulation wouldn't harm more than it saves here. The truly dangerous actors wouldn't care or would be based in some other country. Having a large diversity of actors seem like the best way to ensure resilience against whatever threats might arise from this.
- zirgs 4y agoWhat about the models that are out already? Will men with guns raid my home and confiscate my computer?
- Name_Chawps 4y ago[flagged]
- xiphias2 4y agoIt’s not the opinion that is getting the species killed: it’s just nature, we can’t do anything about it, otherwise we would have seen aliens already.
- NayamAmarshe 4y agoThis isn't even comparable to a nuke. This kind of opinion is going to leave our entire species behind. Imagine having a patent on 'fire' and then suing everybody who tries to cook a meal.
- Name_Chawps 4y agoIt's not comparable to a nuke because a nuke is dumb, and won't be dangerous unless you do something dangerous with it. AI, on the other hand, will be dangerous by default, once it's powerful enough.
- recuter 4y agoLanguage models don't kill people, people kill people. You know what stops a bad ̶g̶u̶y̶ mega-corporation with a language model? A good guy with a language model. Here is what mine had to tell you: It’s not like we don’t already have nuclear weapons, biological agents, chemical agents etc... AI is simply another tool which can be used for good or ill. It doesn’t matter how much regulation/control you put on it - if someone really wanted to use it maliciously then they will find ways around your safeguards. The best thing to do is educate yourself as much as possible. (sampling parameters: temp = 100.000000, top_k = 40, top_p = 0.000000, repeat_last_n = 256, repeat_penalty = 1.176471)
- numpad0 4y agoFire is dangerous by default too.
- 4y ago
- FeepingCreature 4y agoAs an AI doomer, it would actually be pretty great if we could get this stuff locked away behind costly APIs and censorship. Some fat monopoly rent-extracting too. We are moving way too fast on this tech, and the competitive race dynamics are a big reason why. I want LLMs to end up with Microsoft IE6 level of progress. Preferably we could make Firefox (SD/GPT-J) illegal too. (The GPU scarcity is a good start, but maybe China could attack Taiwan as well and thus torpedo everybody's chipbuilding for a decade or so?) If LLMs keep going at their current pace and spread, the world is seriously going to end in a few years. This technology is already unsafe, and it's unsafe in exactly the ways that it'll be seriously unsafe as it scales up further - it doesn't understand and doesn't execute human ethics, and nobody has any working plan how to change that.
- dragonwriter 4y ago> As an AI doomer, it would actually be pretty great if we could get this stuff locked away behind costly APIs and censorship. Yes, because the only people with access to advanced AI tech being the people whose motive is using and training it for domination over others (whether megacorps or megagovernments) is absolutely a great way to prevent any “AI doom” scenarios.
- FeepingCreature 4y agoIf one party could use LLMs to reliably dominate others, the alignment problem would be basically solved. Right now, one of the biggest corporations of the planet cannot get LLMs to reliably avoid telling people to commit suicide despite months (years?) of actively trying.
- arbitrary_name 4y agoTo me it's the American guns ownership situation: if you make guns illegal now, criminals and governments will still keep them, but your average joe won't get them. A very unequal playing field. LLMs will be used against us: let's at least have our own, and learn how to defend against them? i say this as devil's advocate, with serious reservations about where all of this is going.