17 ms·
GPT-3 is no longer the only game in town
- mrbukkake 5y agoCan anyone tell me what the value of GPT-3 actually is other than generating meaningless prose? What would a business use it for
- crubier 5y agoHave you heard of GitHub copilot ? It’s based on GPT3 and I can tell you one thing: it does not generate meaningless prose (90% of the time)
- ghoomketu 5y agoYes I'm used it now but first time it started doing its thing, I wanted to stop and clap for how jaw dropping and amazing this technology is. I was a Jetbrains fan but this thing takes productivity to a whole new level. I really don't think I can go back to my normal programming without it anymore.
- amelius 5y agoHow many % of the time does it produce code that compiles?
- emteycz 5y agoThe overwhelming majority. Whatever used to take me an hour or two is now a 10-minute task.
- ilteris 5y agoI am so confused. Is there a tutorial explaining how you are using in the IDE whatever it is. I use vscode curious if it can be applied. Thanks
- crubier 5y agoIt works very well with VSCode. It has an integration. It shows differently than normal autocomplete, it shows just like gmail autocomplete (grayed out text sugggestion, and press tab to actually autocomplete). Sometimes the suggestion is just a couple tokens long, sometimes it’s an entire page of correct code. Nice trick: write a comment describing quickly what your code will do (“// order an item on click”) and enjoy the complete suggested implementation ! Other nice trick: write the code yourself, and then just before your code, start a comment saying “// this code” and let copilot finishe the sentence with a judgement about your code like “// this code does not work in case x is negative”. Pretty fun !
- icelancer 5y agoInteresting second use case; I use comments like this already as typical practice and I agree Copilot fills in the gaps quite well - never thought to do it in reverse... will give that a shot today.
- emteycz 5y agoI also like to do synthesis from example code (@example doccomment) and synthesis from tests.
- bidirectional 5y agoFrom my anecdotal experience, the vast majority of the time (90+%).
- crubier 5y agoIn my experience 95% of the time. And 80% of the time it output codes which is better than I would have done myself in a first approach (thinks of corner cases, adds meaningful comments etc.). It’s impressive.
- icelancer 5y agoI was exceptionally skeptical about it, but it's been very useful for me and I'm only using it for minor tasks, like automatically writing loops to pull out data from arrays, merge them, sort information, make cURL calls and process data, etc. Simply leading the horse to water is enough in something like PHP: // instantiate cURL event from API URL, POST vars to it using key as variable name, store output in JSON array and pretty print to screen Usually results in code that is 95-100% of the way done.
- inglor 5y agoThis - it is tremendously valuable to me and I use it all the time at work.
- skybrian 5y agoWhat do you use it for?
- inglor 5y agoCoding, I actually had to forbid it today in a course I teach because it solves all the exercises :) (given unit tests with titles students needed to fill those tests in)
- singlow 5y agoIsn't that just because others have stored solutions to these problems in GitHub?
- iamcurious 5y agoThat is my question too. Is it a fancier autocomplete? Or does it reason about code?
- robbedpeter 5y agoIt reasons over the semantic network between tokens, in a feedforward inference pass over the 2k(ish) words or tokens of the prompt. Sometimes that reasoning is superficial and amounts to probabilistic linear relationships, but it can go deeply abstract depending on training material, runtime/inference parameters, and context of the prompt.
- PeterisP 5y agoIn some sense you could think of as a fancy autocomplete which uses not only code but also comments as input, looks up previous solutions for the same problem but (mostly) appropriately replaces the variable names to those that you are using.
- nradov 5y agoThe fact that GPT3 works at all for coding indicates that our programming languages are too low level and force a lot of redundancy (low entropy). From a programmer productivity optimization perspective it should be impossible to reliably predict the next statement. Of course there might be trade offs. Some of that redundancy might be helping maintenance programmers to understand the code.
- hans1729 5y ago>From a programmer productivity optimization perspective it should be impossible to reliably predict the next statement Why? 99.9% of programming being done is composition of trivial logical propositions, in some semantic context. The things we implement are trivial, unless you're thinking about symbolic proofs etc
- tshaddox 5y agoI think that’s precisely the problem the parent commenter is describing.
- alephaleph 5y agoThat would only follow if we were trying to optimize code for brevity, and I have no clue why that would be your top priority.
- nradov 5y agoNot at all. Brevity (or verbosity) is largely orthogonal to level of entropy or redundancy. In principle it ought to be possible to code at a higher level of abstraction while still using understandable names and control flow constructs.
- crubier 5y agoI’m really not sure about that. By definition, high entropy means high information density (information per character). So with the same amount of information you would have less characters.
- pharmakom 5y agoCode is easier to write than read and maintain, so how useful is something that generates pages of 90% correct code?
- ALittleLight 5y agoIt's not useful if you use it to auto complete pages of code. It is useful to see it propose lines, read, and accept its proposals. Sometimes it just saves you a second of typing. Sometimes it makes a suggestion that causes you to update what you wanted to do. Sometimes it proposes useless stuff. On the whole, I really like it and think it's a boon to productivity.
- lysecret 5y agoHey for a long time i was also very sceptical. However i can refer you to this paper to a really cool applciaiton. https://www.youtube.com/watch?v=kP-dXK9JEhY https://www.youtube.com/watch?v=kP-dXK9JEhY. They baseically use clever GPT-3 prompting to create a dataset, you then train another model on. Besides, you can prompt these models to get (depending on the usecase) really good few shot performance. And finally, github copilot is another pretty neat application.
- warning26 5y agoGPT-3 is fairly effective at summarization, so that's one potential business use case: https://sdtimes.com/monitor/using-gpt-3-for-root-cause-incident-summarization-of-incidents/ https://sdtimes.com/monitor/using-gpt-3-for-root-cause-incid...
- Tijdreiziger 5y agohttps://replika.ai/ https://replika.ai/
- amelius 5y agoI hope that one day it will allow me to write down my thoughts in bullet-list form, and it will then produce beautiful prose from it. Of course this will be another blow for journalists, who rely on this skill for their income.
- DeathArrow 5y agoI played with GPT-3 giving it long news stories. It actually replied with more meaningful titles than the journalists themselves used.
- rm_-rf_slash 5y agoPerhaps GPT-3 was optimizing to deliver information while news sites these days optimize titles to get clicks.
- ailef 5y agoYou can prompt GPT-3 in ways that make it perform various tasks such as text classification, information extraction, etc... Basically you can force that "meaningless prose" into answers to your questions. You can use this instead of having to train a custom model for every specific task.
- jszymborski 5y agoWhile the generation is fun and even suitable for some use cases, I'm particularly interested in its ability to take in language and use it for downstream tasks. A good example is DALL-E[0]. Now, what's interesting to me is the emerging idea of "prompt engineering" where once you spend long enough with a model, you're able to ask it for some pretty specific results. This gives us a foothold in creating interfaces whereby you can query things using natural language. It's not going to replace things like SQL tomorrow (or maybe ever?) but it certainly is promising. [0] https://openai.com/blog/dall-e/ https://openai.com/blog/dall-e/
- 13415 5y agoAutomatic generation of positive fake customer reviews on Amazon, landing pages about topics that redirect to attack and ad sites, fake "journalism" with auto-generated articles mixed with genuine press releases and viral marketing content, generating fake user profiles and automated karma farming on social media sites, etc. etc.
- DeathArrow 5y agoThe state of the journalism is so poor, I'd rather take some AI generated articles instead.
- phone8675309 5y ago> fake "journalism" with auto-generated articles mixed with genuine press releases and viral marketing content How would you tell the difference from the real thing these days?
- mark_l_watson 5y agoYou can try it yourself - apply for a free API license from OpenAI. If you like to use Common Lisp or Clojure then I have examples in two of my books (you can download for free by setting the price to zero): https://leanpub.com/u/markwatson https://leanpub.com/u/markwatson
- moffkalast 5y agoI put in a request months ago, I think they're not approving people anymore.
- mark_l_watson 5y agoYou might try signing up directly for a paid non/free account, if that is possible to do. I was using a free account, then switched to paying them. Individual API calls are very inexpensive.
- pyb 5y agoI know of some credible developers who were struggling to get access, so YMMV
- mark_l_watson 5y agoIt took me over a month, so put in your request. Worth the effort!
- deleted 5y ago[deleted]
- phone8675309 5y agoIt's good for the university-industrial-business complex - people writing papers about a model they can't even run themselves. It practically prints money in journal articles, travel per diem, and conference honorariam, not even counting the per-API call rates.
- DeathArrow 5y ago>What would a business use it for If you think about business uses you can actually get advices from Jerome Powell, simulated by GPT-3. If someone use GPT-3 to simulate Warren Buffet, he can extract even more value. https://www.institutionalinvestor.com/article/b1tktmhcfdyqsk/An-Interview-With-a-Hyper-Realistic-AI-Powered-Simulation-of-Jerome-Powell https://www.institutionalinvestor.com/article/b1tktmhcfdyqsk...
- vitus 5y agoSomehow I don't think that this is quite how Jerome Powell would respond in an interview: > Interviewer: How do you think a global pandemic would impact the financial system? > Mr. Powell: A pandemic will have impact on the economy. It will have an impact on the financial system. > Interviewer: What would the impact be? > Mr. Powell: If a major pandemic occurred, the economy would be affected, and the financial system would be affected. Yes, GPT-3 can provide a convincing chatbot, but it shouldn't be confused for domain expertise.
- robbedpeter 5y agoThis is a bad example. Here's an output from gpt-j-6b by EleutherAI: Prompt( Jerome Powell was interviewed about the impact of a pandemic on the economy. > > Interviewer: What would the impact be? > Mr. Powell: If a major pandemic occurred) Output( it could be very disruptive. > > Interviewer: How disruptive? > Mr. Powell: Well, it could be disruptive in a number of ways. > > Interviewer: How so? > Mr. Powell: Well, one of the first things that would be disrupted would be the supply chain. ) Using prompts well makes a huge difference. If you parse the generated output, classify it, then develop a decision tree that uses further prompts to refine the response, you can get more sophisticated, valuable responses. The output in the parent is comparable to an off-the-cuff interview response. If you emulate a deeper thought process, you can get more meaningful output, and if you use the right prompts, you can access the semantic networks in the model related to your domain of interest.
- notahacker 5y agoI think the "bad example" is actually the good one, because it's a reminder that actually you're not getting business advice from someone with Warren Buffet or Jerome Powell's understanding of the economy, you're getting text generated by analysing patterns in other not-necessarily-applicable text. If you start forcing it in very specific directions you start getting text that summarises the commentary in the corpus, but most of that commentary doesn't come from Warren Buffet or Jerome Powell and isn't applicable to the future you're asking it about...
- DeathArrow 5y agoChat bots are an usage. I think you might use it for customer support. One example of GPT-3 powered chat bot: https://www.quickchat.ai/emerson https://www.quickchat.ai/emerson
- micro_cam 5y agoActually using this class (larger transformer based language models) of models to generate text is to me the least interesting use case. They can also all be adapted and fine tuned for other tasks in content classification, search, discovery etc. Think facnial recognition for topics. Want to mine a whole social network for anywhere people are talking about _______ even indirectly with very low false negative rate? You want to fine tune a transformer model. Bert tends to get used for this more because it is freely available, established and not too expensive to fine tune but i suspect this is what microsoft licensing gpt-3 is all about.
- hubraumhugo 5y agoAt https://reviewr.ai https://reviewr.ai we're using GPT-3 to summarize product reviews into simple bullet-point lists. Here's an example with backpack reviews: https://baqpa.com https://baqpa.com
- staticautomatic 5y agoDid you test it against extractive summarizers?
- hubraumhugo 5y agoWe experimented with BERT summarization, but the results weren't too good. Do you have any resources or experiences in this area?
- moffkalast 5y agoThat sounds like BERT alright.
- cma 5y agoHow do you avoid libel?
- kingcharles 5y agoAre you confusing libel with something else? Can you extrapolate what you mean here? Are you saying that they will be liable for libel (!) if they publish a negative summary of a product?
- cma 5y agoIf they mischaracterize a positive review into a negative summary based on factual mistakes they know the system makes at a high rate, I would think they would be liable for libel right?
- 5y ago
- supperburg 5y agoThat’s like if an alien took Mozart as a specimen and then disregarded the human race because this human, while making interesting sounds, does nothing of value. You have to look at the bigger picture.
- akelly 5y agohttps://copy.ai/ https://copy.ai/
- teaearlgraycold 5y agoHey, I work there! To be honest it's still very much a prototype. We have big plans for the next few months.
- CamperBob2 5y agoGPT-3 and similar ML/AI projects may have many interesting and valuable commercial applications, not all of which are readily apparent at this stage of the game. For instance, it could be used to insert advertisements for herbal Viagra at https://www.geth3r3a1N0W.com https://www.geth3r3a1N0W.com into otherwise-apropros comments on message boards, preferably near the end once it's too late to stop reading. Life online is about to become very annoying.
- teaearlgraycold 5y agoI work for a company that re-sells GPT-3 to small business owners. We help them generate product descriptions in bulk, Google ads, Facebook ads, Instagram captions, etc.
- warning26 5y agoNeat to see more models getting closer, thought it appears only one so far has exceeded GPT-3's 175B parameters. That said, what I'm really curious is how those other models stack up against GPT-3 in terms of performance -- does anyone know of any comparisons?
- machiaweliczny 5y agoThere's Wu Dao 2.0 and Google has 2 models with 1T+ params.
- atty 5y agoFor clarity, i believe these are all mixture of expert models, where each input only sparsely activates some subset subset of the full model. This is why they were able to make such a big jump over the “dense” GPT3. Not really an apples-to-apples comparison.
- pyb 5y ago+1, does the new generation match or exceed GPT-3 in terms of relevance ? Is there a way for a non-AI-researcher to understand how the benchmarks measure this ? Bigger does not mean better.
- sillysaurusx 5y agoI’m surprised that no one has answered for three hours! The answer is at https://github.com/kingoflolz/mesh-transformer-jax https://github.com/kingoflolz/mesh-transformer-jax It has detailed comparisons and a full breakdown of the performance, courtesy of Eleuther.
- 6gvONxR4sf7o 5y agoI was so frustrated when that was first announced because it didn't include those metrics, and everyone ate it up like the models were equivalent.
- 6gvONxR4sf7o 5y agoWhenever I've seen language modeling metrics, GPT-3's largest model has been at the top. If you see a writeup that doesn't include accuracy-type metrics, you're reading a sales pitch, not an honest comparison.
- xyproto 5y agoI think companies should be banned from having "Open" in their names.
- evergrande 5y agoOpenAI takes the Orwellian cake.
- Tenoke 5y agoI hear a lot of low effort takes about OpenAI but how exactly is providing your service via a paid API the "Orwellian cake"? Is this really the most (or even at all) Orwellian practice for you?
- TulliusCicero 5y agoI think it's more the contrast where they claim, via their name, to be open, but actually aren't. If their name was ProfitableAI, there'd probably be fewer complaints.
- leoxv 5y agohttps://en.wikipedia.org/wiki/Doublespeak https://en.wikipedia.org/wiki/Doublespeak
- c7DJTLrn 5y ago"Open"AI but you can only use it how we want you to and no, you can't run it yourself.
- moffkalast 5y agoThe only thing open in OpenAI is your wallet.
- ComodoHacker 5y agoAre we heading to the (distant) future where to make progress in any field you have to spend big $$$ to train a model?
- YetAnotherNick 5y agoTraining model is getting cheaper. GPT-3 is one of the very few countable examples where it is so expensive. In the end it all depends on the size of data you have that you could scale up the model without overfitting. And the internet text data is one of the only data size is this big.
- jowday 5y agoThat’s not even distant - most of the self-supervised vision and language models at the bleeding edge of the field require huge compute budgets to train.
- iamcurious 5y agoWe are already there. Machine learning is the flavor of A.I. that keeps business barriers of entry high. If we had invested in symbolic A.I., things would be different. A similar thing happens with programming language flavors. PHP lowers barriers of entry so it is discredited by the incumbents.
- deleted 5y ago[deleted]
- lostdog 5y agoThe difference between ML and symbolic AI is that ML works and symbolic AI doesn't. At my job, dropping the computational load of our ML models is heavily invested in, and every success is celebrated. Everybody wants it to be easier and cheaper to train high quality models, but some things are still intrinsically hard.
- CodeGlitch 5y ago> The difference between ML and symbolic AI is that ML works and symbolic AI doesn't. IBM managed to beat Garry Kasperov using symbolic AI did they not? So in what way does it not work?
- supperburg 5y agoI have supported my beliefs on this topic in these threads to the point of exhausting myself. The tools that we use to find these agents are the underpinning of AGI, it’s coming way faster than even most people here appreciate, this development is intrinsically against the interest of human beings. Please stop and think, please.
- Simon321 5y agoI argue it's very much in the interest of human beings. It has been since we first picked up a rock and used it has a hammer. It's the ultimate tool and has the potential to bring unseen prosperity.
- supperburg 5y agoIt won’t. You’re wrong.this is the perfect illustration. You think a rock is good therefore AI is good. You’re just unbelievably wrong.
- kafkaIncarnate 5y agoSince you two have expressed two very extreme opinions, I would like to express my very bland opinion that this will effectively be the equivalent of when human computers of the early 1900s were replaced with microchips. We adapted, and in this instance the human computers are just propagandists. I believe this will displace the workers of places like the Internet Research Agency and some Marketing specialists. While that accounts for a large percentage of propaganda jobs, that is easy enough to adapt from (nor do I really care given who is generally doing it), and marketing people who I generally don't have respect for either (I mean their founding document is literally titled "Propaganda"). The real marketing pros are good at managing hard to handle talent or bad PR disasters, which are actually meaningful skills that this won't affect. So basically no moral or real consequential impact on society whatsoever.
- deleted 5y ago[deleted]
- moffkalast 5y ago> most recently NVIDIA and Microsoft teamed up to create the 530 billion parameter model Megatron-Turing NLG Get it, cause it's a generative transformer? Hah
- TedShiller 5y agoI never got why GPT-3 was so closed off, like you needed permission to use it. If it’s so good then why not just make it available?
- krageon 5y ago> If it’s so good then why not just make it available? OpenAI is pivoting to corporate evil, and to do that properly they need proprietary assets to rent out.
- Nasrudith 5y agoBecause of vast piles of bad science fiction being taken as fact and outright hysteria.
- nomel 5y ago> Because of vast piles of bad science fiction being taken as fact None of these systems could work if only facts were used as input. Maybe some "factual" score could be presented with the data, but wow, that's a philosophical problem there.
- theshrike79 5y agoI think the reasoning was that GPT-3 could easily be used to fill the world with realistic bullshit that would take ages to debunk.
- gbear605 5y ago
- DeathArrow 5y agoPeople were blaming cryptocurrencies miners for the prices of GPUs, when in fact it was the AI researchers who bought all the GPUs. :D I wonder what if somebody designs an electronic currency rewarded as payment for general GPU computations instead of just computing hashes? You pay some $, to train your model and the miner gets some coins. Every one is happy, electricity is not wasted and the GPUs gets used for a reasonable purpose.
- nathias 5y agoYes, this is an old idea (which I really like) but it hasn't really taken off yet. GridCoin was one example, where you solved BOINC problems or RLC that's for more general computation.
- rewq4321 5y agoThe problem is that, currently, large ML models need to be trained on clusters of tightly-connected GPUs/accelerators. So it's kinda useless having a bunch of GPUs spread all over the world with huge latency and low bandwidth between them. That may change though - there are people working on it: https://github.com/learning-at-home/hivemind https://github.com/learning-at-home/hivemind
- Kiro 5y agoIt hasn't taken off because it doesn't work. PoW only works for things that are hard to calculate but easy to verify. Any meaningful result is equally hard to verify.
- petters 5y ago> Any meaningful result is equally hard to verify. This is very much not true. A central class in complexity is NP whose problems are hard to answer but easy to verify if the answer is yes. E.g. is there a path visiting all nodes in this graph of length less than 243000? Hard to answer but easy to check any proposed answer.
- snovv_crash 5y ago
- visarga 5y agoThe GPT-3 family is still to expensive to use, too big to fit in memory on a single machine. Prices need to come down before large scale adoption or someone needs to invent the chip to hold it (cheaply). The most exciting part about it is showing us there is a path forward by scaling and prompting, but you can still do much better with a smaller model and a bit of training data (which can come from the expensive GPT-3 as well). What I expected from the next generation: multi-modality, larger context, using retrieval to augment input data with fresh information, tuned to solve thousands of tasks with supervised data so it can generalize on new tasks better, and some efficient way to keep it up to date and fine-tune it. On the data part - more data, more languages - a lot of work.
- GhettoComputers 5y ago>However, the ability of people to build upon GPT-3 was hampered by one major factor: it was not publicly released. Instead, OpenAI opted to commercialize it and only provide access to it via a paid API. This made sense given OpenAI’s for profit nature, but went against the common practice of AI researchers releasing AI models for others to build upon. So, since last year multiple organizations have worked towards creating their own version of GPT-3, and as I’ll go over in this article at this point roughly half a dozen such gigantic GPT-3 esque models have been developed. Seems like aside from Eleuther.ai you can’t use the models freely either, correct me if I’m wrong.
- andreyk 5y agoI believe you are correct, at least for GPT-3 scaled things. Hopefully that'll change with time, though.
- qwertox 5y agoIf I were to run GPT-3 on my 70000 browser bookmarks, what kind of insights could I get from that? Only by analyzing the page title (from the bookmark, not by re-fetching the url) and eventually also the domain name.
- supermatt 5y agoGPT-3 is a text generator, so i doubt you would get anything of use. You cant even supply such a large input to GPT-3.
- Dylan16807 5y agoIt's pretty good at extending lists. That might require some sorting first.
- fragmede 5y agoGosh, anyone else remember Google Sets? You typed in a few things, eg colors, and Google would add some more things of the same type.
- teaearlgraycold 5y agoGPT-3 is also a classifier and data-extractor. You could give it a couple dozen bookmarks with example classifications and then feed it a new bookmark and ask GPT-3 what category the page belongs in. Repeat for the entire data set. For data extraction you could ask questions about the titles. Maybe have it list all machine-learning model names that appear in the set of bookmark titles.
- keithalewis 5y agoprint gpt.submit_request("Give me insights") >>> You are spending way to much time browsing.
- rg111 5y agoThe future is not as dark as it seems because of the rat race of megacorps. You can use reduced versions of language models with extremely good results. I was involved in training the first-ever GPT2 for Bengali language, but with 117 million parameters. It took a month's effort (training + writing code + setup) and about $6k in TPU cost, but Google Cloud covered it. Anyway, it is surprisingly good. We fine-tuned the model for several downstream tasks and we were shocked when we saw the quality of generated text. I fine-tuned this model to write Bengali poems with a dataset of just about 2k poems and ran the training for 20 minutes in GPU instance of Colab Pro. I was really blown away by the quality. The main training was done in JAX, and it is much faster and seamless than PyTorch XLA, much better than TensorFlow in every way. So, my point is, although everyone is talking hundreds of billions of parameters and millions in training cost, you can still derive practical value from language models, and that too, at a low cost.
- cmrajan 5y agoGood to know. We're trying to attempt something similar[1] but for Tamil. I'm also surprised how well the OSS language model & library AI4Bharat [2] performs for NLP tasks against SoTA systems. Is there a way to contact you? [1] https://vpt.ai/posts/about-us/ https://vpt.ai/posts/about-us/ [2] https://ai4bharat.org/projects/ https://ai4bharat.org/projects/
- rg111 5y agoAmong a master's degree, a consultancy gig, personal research and study, and finding unis abroad- I am living a hectic life. I don't see how I can be of help. But I can talk. Leave me something through which I can reach you. And I will reach you within a week.
- amelius 5y ago> The future is not as dark as it seems because of the rat race of megacorps. Just wait until NVidia comes with a "Neural AppStore" and corresponding restrictions. Then wait until the other GPU manufacturers follow suit.
- worik 5y agoThe underlying methods seem impractical. GPT-n are an existence proof - it is possible to make parrot like software that generates realistic text. But using these methods it is not practical. Maybe that is a good thing, maybe a bad thing, but unless there is a breakthrough in methods this is a dead end. Impressive though.
- WithinReason 5y agohttps://copilot.github.com/ https://copilot.github.com/
- kafkaIncarnate 5y agoYuck. More intrusive than an IDE, I'll pass. This will just lead people into the equivalent of using the autocorrect answers too often as if you used the suggest feature of a phone keyboard on a sentence. Eventually it just breaks down and stops making any sense. The code equivalent is going to be bugs, security holes, and programmer headaches.
- WithinReason 5y agoOr this will revolutionize programming and give a 3x-4x productivity boost to those who take the time to learn to use it effectively.
- kafkaIncarnate 5y agoWhich would lead to quicker and harder burnout. If that future were to exist I bet there would be a meditative/yoga/wellness/hippy movement of people who "get in touch with their roots" of programming after going through a 15 year journey through a desert on mushrooms to regain their ability to think straight.
- air7 5y agoSo is there any one of them that I could play around with?
- sillysaurusx 5y agohttps://6b.eleuther.ai https://6b.eleuther.ai
- lazylion2 5y agoAI21 labs 178B parameter model https://studio.ai21.com/ https://studio.ai21.com/
- eunos 5y agoNumber of parameters aside, I am really surprised that we havent yet reached hundreds of TB of training data. Especially Chinese model only used less than 10 TB of data.
- rexreed 5y agoGPT-3 is the most overrated game in town. And Microsoft spending $1 Billion for an exclusive license will seem really foolish a few years from now.
- cbozeman 5y ago> And Microsoft spending $1 Billion for an exclusive license will seem really foolish a few years from now. No it won't. Capital was (and still is) cheap when they made that purchase. A billion used to really mean something, but it's chump change for companies with valuations over $1 trillion.
- danielcampos93 5y agoThis move was a defensive move. Keep any other MEGA-CORP from getting access.
- rexreed 5y agoClearly this is not very defensible. More data, more GPUs, more time, and you have similar or better results. There's a reason why Google, FB, Amazon haven't bitten the same bullet for an exclusive license.
- actually_a_dog 5y agoIs there any estimate out there of how many joules of energy it took to train GPT-3?
- mdrzn 5y agoIs there any similar version to GPT-3 available for free? Or usable online via web interface?