8 ms·
Eliezer Yudkowsky on large language model economics
- coldtea 4y agoThis is like "Malcolm Gladwell on horse racing"...
- thecleaner 4y agoCould you explain this joke please ?
- xdavidliu 4y agomy possibly incomplete guess: "articulate person opines on a topic that they have no expertise in, and uses articulation to hide their lack of expertise"
- coldtea 4y agoPrecisely. Also "enthusiastic and opinionated on the matter, without actual involvement in the field except as dilettante sharing ideas irrelevant to actual practicioners"
- herculity275 4y ago"Have you tried making your Yud bigger?"
- cs702 4y agoAs usual, Yudkowsky has valuable insights, but uses a lot of words, many of them fancy, to explain himself, making his writing less accessible than it could be. TL;DR: If your customers can get text input-output samples from your super-secret LLM, whether via web interface or API, then your customers can easily and cheaply train their own LLM to learn to act like your super-secret one. For example, if you're OpenAI, your customers can spend ~$100 to get input-output samples from text-davinci-003, plus another ~$500 to fine-tune LLaMA on those samples, and -- voila! -- they get Alpaca.
- PaulHoule 4y agoIt’s not accidental at all that EY writes this way. He is writing out of L. Ron Hubbard’s style guide. He doesn’t want to communicate to outsiders, he is writing to members of his Cthulhu cult.
- canadiantim 4y agoYeah I was liking his post and following along until he kept repeating "shoggoth", very weird. I stopped at that part
- deleted 4y ago[deleted]
- 4bpp 4y agoThe "shoggoth with a smiley face" metaphor for LLMs has gained traction far beyond Yudkowsky's bubble. It even has a KYM article now: https://knowyourmeme.com/memes/shoggoth-with-smiley-face-artificial-intelligence https://knowyourmeme.com/memes/shoggoth-with-smiley-face-art...
- semi-extrinsic 4y agoDoesn't make any of this less stupid though.
- chpatrick 4y agoMakes sense to me. It's an enormous inhuman blob trained on all of human knowledge that can manifest a personality out of nothing based on some inoffensive corporate prompt.
- jerry1979 4y agoWhat do you mean by "L. Ron Hubbard's style guide"? Would that be doing things like employing in-group metaphors?
- deleted 4y ago[deleted]
- substation13 4y agoI can't help but feel like there is an element of sour grapes here after Yudkowsky's attempts to build an AI failed (assuming that's what he was really trying to do, and not just to lure young and enthusiastic grad students into his lair...)
- terafo 4y agoBut did he try to build AI? As far as I know MIRI is pure-play safety research lab.
- substation13 4y agoHe did, MIRI was a later rebrand. Source: https://youtu.be/0A9pGhwQbS0?t=60 https://youtu.be/0A9pGhwQbS0?t=60
- kelipso 4y agoHaha, those who can't do, can always latch on by forming ethics groups and related grifts.
- mitthrowaway2 4y agoAn essay by Yudkowsky about the pivot: https://www.lesswrong.com/posts/fLRPeXihRaiRo5dyX/the-magnitude-of-his-own-folly https://www.lesswrong.com/posts/fLRPeXihRaiRo5dyX/the-magnit...
- xdavidliu 4y agokinda like the Chomksy op-ed, though you could argue that Chomsky never technically tried to build an AI
- renewiltord 4y agoConsidering Chomsky's work in linguistics, I was disappointed by that piece.
- tlb 4y agoWhile cloning kind of works, I wouldn't expect the results to be quite as good as the original. Right now we're in this weird phase where we have no accurate metrics for how good an LM is, so people just try them with a handful of queries and are subjectively impressed or not. Maybe clones can pass that test, but as the market gets more sophisticated, people will prefer superior models.
- boole1854 4y agoIt may not be exactly as good, but it's also a 7B parameter model vs 175B parameters for GPT-3.5 (i.e., text-davinci-003). People are running the new model on their phones, laptops, and Raspberry PIs. People's mouths were watering over the commercial implications of the recent 90% drop in cost for the new ChatGPT model. Now imagine if you can get similar performance on a model that requires <5% of the parameters.
- sharemywin 4y agonot to mention I think you want your knowledge in some kind of database and just let the AI just do the conversational part of it. which would be a much easier and more reliable(explainable) way to do it. just find and parrot the facts.
- pilooch 4y agoThis has been true of machine learning APIs from the early renaissance of neural networks. Computer vision APIs have been used for years to grow datasets and fine-tune models on the data. As a matter of fact, the API/SaaS business for machine learning 'direct' model results have suffered from this since the early days. Now, what may make more sense and survive longer are full data-storage-oriented platforms, and niche products that encapsulate all underlying model calls, including to publicly available models. Sometimes glue code or even devops becomes the true solid business case of AI/ML apps ;)
- optimalsolver 4y agoIt's really weird to see a blog-length Twitter post.
- dragonwriter 4y agoIts a new paid-membership benefit, so it will probably become more common.
- ipsum2 4y agoYudkowsky has a very shallow understanding of machine learning/AI, but loves to write in a conspiratorial way with in-group terminology to make people think that he's onto some dark secrets. What he's describing is common knowledge. This technique is called knowledge distillation (there are some small differences, but the idea is the same). A large model (here, openai's davinci) is used as a teacher model to train a smaller student model (llama). This is used practically everywhere in industry to improve inference times.
- polygamous_bat 4y ago> Yudkowsky has a very shallow understanding of machine learning/AI, but loves to write in a conspiratorial way with in-group terminology to make people think that he's onto some dark secrets. Perhaps I am a little behind the times, but what are reasons/achievements/credentials behind his huge, prophet like status within the cult of rationalism? Beyond writing a popular Harry Potter fanfic, I mean.
- entelechy0 4y ago[dead]
- PaulHoule 4y ago[flagged]
- nwah1 4y agoCharisma, humor, and religious-like ideological framing that can acquire true believers who believe they are saving the world.
- agentwiggles 4y agoI think EY's big contribution to the LessWrong/Rationalist cult/movement/ethos is writing the "Sequences," which purport to be a guide to training yourself to think rationally. (https://www.lesswrong.com/tag/original-sequences https://www.lesswrong.com/tag/original-sequences) The rationalists are an interesting group. I like a lot of the vibe of the whole thing, but there is also some very weird navel gazing that has cropped up. It can be hard to differentiate from the good and the bad. I am overall a little surprised by the vitriol towards EY in this thread though. While I agree that the guy is somewhat shrill and his ideas are pretty out there, he really doesn't seem like a con artist from my standpoint. It seems to me that he is genuinely extremely concerned about AI alignment/risk, and his actions seem to make sense in that framework without needing to ascribe a bunch of weird and sinister motivations to him. Not professing agreement or disagreement with him on any particular thing, just my two cents.
- deleted 4y ago[deleted]
- seydor 4y agoAnother way to put it is : current crop "AI companies" don't really have a moat. Transformers have existed for many years, and a bit of RL on top of it is not enough to get ahead. It's all about who has the most money to spend training larger and larger, but what happens when they all run out of money for that? LLMs will become cheaper than web hosting
- webmaven 4y agoThat's overstating things. Web hosting will always be less demanding, unless something weird happens and GPU/APU/TPU type hardware (along with HBM and related ancillaries) somehow becomes less expensive than (or subsumes) general purpose CPUs and RAM.
- 5cott0 4y agonot reading yuds unless it takes the form of of harry potter fanfic
- basilisklmao 4y ago[flagged]
- snapcaster 4y agoAnyone taking Roko's basilisk seriously has likely had their brain damaged beyond repair by decision theorists and people like EY
- James_Henry 4y agoHey, I don't think any real decision theorists would take something like this seriously. And if any would, then let's change the definition of a real decision theorist to exclude them.
- panarky 4y agoThe superintelligence will not be fooled by your pseudonym.
- zadler 4y agoI don’t believe the idea that they would lose their moat. That’s like saying, any product you can copy has no moat, it’s not true at all.
- sharemywin 4y agobut what ties you to the product? chat history but is that really that important? It's one api call from an integration stand point. It's not a marketplace so you don't have network effects much. the interface isn't all that much. it's kind of a boring bot. it doesn't even tell dirty jokes. I really do feel like this is a race to the bottom.
- zadler 4y agoWe have a very simple view of the product right now, and it's also new, so the different available options don't have much history to inform preference. That will change, and the stakes are very high, since this technology is capable of removing alot of jobs, so the company that f*cks up the least is going to become preferential. This effect will be magnified as regulation / beaurocracy come in more. Also, the post seems to assume that a single model will be published and then not change for a long time, enough time that a competetor can copy it and sell their copy; but this is also not true, models will be constantly updated and as soon as regulation catches up it'll be a legal requirement that they are constantly updated. And due to the power of the models, the state will quickly want to pick favorites, and that will also impact the difficulty of displacing a model provider. So, forget it, there's alot more involved than just having a copy of the result of some algorithms.
- rafaelero 4y agoThere will be behaviors smaller models won't be able to mimic, though. So sure, fine-tuning advantages will be rapidly copied, but huge models that require large data centers won't. Economic incentives are still aligned with companies going through the path of larger and larger models.
- blueyes 4y agoCutting-edge LLMs are commoditized anyway. There are very few tech moats here, so his point doesn't change the economics much. Data will sometimes be a moat, but companies with proprietary data will also expose their models through GUIs and therefore have other means of preserving that moat. OpenAI is moving toward economies of scale with their API pricing, which is smart.
- ozten 4y agoSubscribers get 4000 character limit? Another way in which Twitter is not twitter any more.
- kybernetyk 4y agoBut Twitter is still shit.
- Imnimo 4y agoCompletely missed the point, as usual. Consider the amount of world knowledge present in a 175B model vs a 7B model. You can elicit instruction following behavior with a relatively small number of samples , but you're never going to get that coverage of memorized facts. The Alpaca blog post gives the example that Alpaca doesn't know the capital of Tanzania, for example. The most is that depth of knowledge and understanding, not the surface level instruction following behavior.
- radu_floricica 4y agoThe point is that you can split the work in two: the one you mentioned, creating and training the model, and a separate step of shaping it with reinforcement learning. Which is quite expensive, because it uses human operators in one of its steps. But apparently you can duplicate easily enough this with an API of an already educated engine, which makes the second step not be a moat anymore.
- Imnimo 4y agoHow expensive do you imagine collecting human feedback data is compared to the cost of doing the original language modelling training?
- cubefox 4y agoBut that's not a new point?! The second, expensive step (RLHF) was already automated by Anthropic with their Constitutional AI approach several months ago.
- terafo 4y agoYou don't need to store it in weights. It would be too expensive to constantly update model with new data. It wouldn't be able to contain everything anyways. LLaMa 30B with access to 100TB of storage(you can store whole libgen plus arxiv and much much more in there) and trained to use it should be ahead of LLaMa 65B with no such access in terms of world knowledge. And it would cost less.
- 4y ago
- cm2012 4y agoEY is a rare thing. Right or wrong, he is a modern day philosopher. He has ideas he makes the case for strongly and eloquently.
- williamtrask 4y agoEY is not rare, but he is famous by proximity to powerful/wealthy actors. There are many philosophers, likely more than ever in history. Aside from professional philosophers, there’s a large pipeline of PhDs “doctor of philosophy” who work at a high enough level of abstraction in their own field to be considered a philosopher.
- originalcopying 4y agoall I see are more arguments for my opinion that digital assets + capitalist marketplace = human stupidity. > you're giving away your business crown jewels to competitors that can then nearly-clone your model without all the hard work you did to build up your own fine-tuning dataset. you're loosing the capacity to continue to earn money for work that is already done. This just means they're not gonna be able to charge rent on the done work for too long. the work is the training of the model. I chose to reject this to way to frame the issue; i.e. I reject that it is problem that they're giving away that hard work. It's only a problem due to social (market) ideologies of ownership which are necessary only by the logic of trade (or commerce). As I see it, Eliezer is pointing out hat people who use the "product API"/ "chaptGPT service" can use this in such a way that they can clone it so to stop being a customer. I see some kind of divine comedy in this (due to my own ideological beliefs). > OR can successfully sue any competitor that engages in shoggoth mask cloning. this is just the legal, modern day equivalent of old school "break their knees unless they pay you"
- PoignardAzur 4y ago> this is just the legal, modern day equivalent of old school "break their knees unless they pay you" I don't EY is advocating for this, so much as saying "unless AI companies can find a way to establish such a racket, they won't be profitable". That's a factual claim, not a value judgment.
- originalcopying 4y agoI've recently started to believe that known current legitimate governments are (and have always been) the legacy of people in history doing rackets like these in the now historical past.
- webmaven 4y agoThey sure are. Are you familiar with "War is a Racket"[0][1] by Major-General Smedley Butler[2]?: [0] https://en.m.wikipedia.org/wiki/War_Is_a_Racket https://en.m.wikipedia.org/wiki/War_Is_a_Racket [1] https://books.google.co.il/books/about/War_is_a_Racket.html?id=NGIUCgAAQBAJ&printsec=frontcover&source=kp_read_button&hl=en&newbks=1&newbks_redir=0&gboemv=1&ovdme=1&redir_esc=y#v=onepage&q&f=false https://books.google.co.il/books/about/War_is_a_Racket.html?... [2] https://en.m.wikipedia.org/wiki/Smedley_Butler https://en.m.wikipedia.org/wiki/Smedley_Butler
- BryantD 4y agoSimon Willison wrote about this on Monday, so “I don’t think people realize what a big deal it is…” seems false.
- renewiltord 4y agoThe idea was fairly natural. I suppose having done it successfully is what is interesting. https://news.ycombinator.com/item?id=35113781 https://news.ycombinator.com/item?id=35113781 > Perhaps we will provide feedback to open source Llama using ChatGPT. The cost to adjust the model is presumably what's hard? What I am so impressed by is that the state of the art is so democratic now. All the top AI posts on HN are things that I was able to get to myself days before the were posted here. This must mean either that the true SOTA is way ahead of what we see or that anyone can hit the current SOTA. Both are very exciting developments!
- dragonwriter 4y agoThe conditions for AI profitability he suggests as a “new idiom” for AI have fairly exact parallels in the world of software, but where the secret sauce has expanded to include inputs and outputs, not just the software itself: > The AI companies that make profits will be ones that either have a competitive moat not based on the capabilities of their model, This is the “open source” case, where profitable firm provide ancillary services and support, rather than directly selling the capabilities of the software, because the software is not at all exclusive. > OR those which don’t expose the underlying inputs and outputs of their model to customers This is the “internal software” case, where the profitable firm does not provide access to the software at all, but uses it internally in a way which is opaque from the outside to support other products or services it provides, maintaining exclusivity because no one outside actually gets access to the software itself. (Arguably, non-source-available closed source software might fit in here, though its often possible to reverse engineer software that you have access to without the source, from the executable, so I'd argue that even without source available its more the next category.) > OR can successfully sue any competitor that engages in shoggoth mask cloning. This is the “proprietary software” case, where the profitable firm protects its exclusivity through legal constraints on people who have access to the software, while giving customers access to it. Given that the big firms in AI largely either are or are closely associated with big firms in software that have successfully used all three models for different parts of their portfolios, I don’t think that them figuring out AI profitability is particularly challenged by this “new idiom”.