6 ms·
Big tech wants to make AI cost nothing
- LarsDu88 2y agoMy musings on the recent commoditization of large scale LLMs
- pjkundert 2y agoThe best way to “bake in” a set of biases into widely available AIs, is to make it prohibitively expensive for alternatives (without those biases) to be trained. Unbiased AI is, I believe, an existential threat to the “powers that be” retaining control of the narrative, and must be avoided at all costs.
- contagiousflow 2y agoHow would you define unbiased?
- crackercrews 2y agoGood question. For a start, don't pretend that Nazi soldiers were a multiracial bunch. And don't do whatever Google did to generate clearly-incorrect output like this.
- cthalupa 2y agoSure. That's a major over-correction. But there are existing and known biases in data sources that need to be accounted for - you don't want to further perpetuate those. I think it's obvious that Google went to a ridiculous extreme in the other direction, but there does need to be some amount of work done here. For example, we repeatedly have seen that just changing the name on a resume to something more European sounding can have significant impact on callback rates when applying to a job, and if you trained a model to screen resumes based on your own resume result data, this bias could be picked up by the model. That's the sort of situation these are meant to correct for.
- dtjb 2y agoI don't see how you can solve that problem without inserting bias in a different direction.
- crackercrews 2y agoThere's a difference between inserting bias and allowing a real-world pattern to exist in AI. There may be reasons to dislike these real-world patterns, but that doesn't mean that allowing them to exist in AI is inserting a bias. For example, if you ask AI to write a realistic story about an NBA team, and it comes back with a team with stereotypically Asian named players, that would be unrealistic. If it came back with a team with stereotypically Black named players, that would be fine. Does it reflect a real-world pattern? Yes. But not changing the algorithm to generate diverse names isn't inserting bias. It's letting AI reflect the real world, as it exists.
- Nasrudith 2y agoA basketball player named Yao Ming is unrealistic?
- crackercrews 2y agoI used the plural. Have there ever been any NBA teams with more than one Asian on them? Certainly not starters.
- dtjb 2y agoI think this just pushes the unsolved part to the middle. We'll never have an undisputed definition of the real world, as it is. Clear cases like chinese NBA players aren't contested, but ugly social issues with layers of abstraction and contradiction.
- simonw 2y agoThat multiracial Nazi soldiers thing wasn't baked into the model: it was a prompt engineering mistake, part of the instructions that a product team were feeding into the Gemini consumer product to tell it how to interact with the image generation tool. Here's a similar example from the DALL-E system prompt: https://simonwillison.net/2023/Oct/26/add-a-walrus/#diversify https://simonwillison.net/2023/Oct/26/add-a-walrus/#diversif...
- pjkundert 2y ago"mistake" You keep using that word. I don't think it means what you think it means. But seriously; a "mistake" is usually something that cannot be foreseen by a group of people reasonably talented in the state of the art. This product release was so far from a "mistake", that it isn't funny. It was spectacularly well tested, found to be operating within design parameters, and was released to great fanfare. They expressed delight in their product, and actually seemed surprised that there was a backlash by the great benighted unwashed masses of their lessers, who clearly couldn't be expected to understand the elevated insights being produced by their creation! So: not a "mistake". Institutional Bias, baked into a model. Remember: a system's purpose is what is does, not what you think it is supposed to do.
- simonw 2y agoIt wasn't baked into the model. It was in the prompt. The model and the product built around it are not the same thing.
- crackercrews 2y agoIf end users can't access the model without going through the prompt system then that distinction doesn't matter.
- simonw 2y agoFrom an end user point of view I agree. As someone who works either these models as an engineer, I think it's important to understand that a feature implemented as part of the user-facing UI to a model is irrelevant to the work I do with that model via an API.
- _yid9 2y agoMake public the training dataset, and the weights associated with various elements of the corpus. Elements of the corpus must be assigned differing measures of importance or validity, influencing the formation of patterns in the resultant weights. This would go a long way to reassuring users of the resultant AI, of the neutrality of the trainer. It would simply reveal the core beliefs of the trainer. If it becomes evident (for example), that Marxist or Keynesian or MMT (or whatever) texts are given high validity measures, but texts by Hayek or Sowell are given negative validity, one could assume the trainer is a leftist, economically. What benefit is there to not reveal these facts to the users of the resultant AI, if not to hide the internal bias of the trainer? Yet I am unaware of any large commercial AIs that reveal these training bias indicators...
- ryandrake 2y agoCan you be more specific? Who are the "powers?" What is "the narrative?" and why do these "powers" want "control of the narrative?" This just seems like a vague X-Files conspiratorial statement without those details.
- hooverd 2y ago[flagged]
- gadflyinyoureye 2y agoGemini was hilarious. Making George Washington black along with Nazis.
- dredmorbius 2y agoEven granting that, the overall point about subsidised / low-cost-leader informational content being potentially problematic is a fair one to make. It's one of the chief problems of competing on price generally, and particularly so in the case of informational exchange. I'm relatively confident I'd disagree on at least some of OP's classifications of biased information. I can still agree with their general point all the same. And in either case, coming up with ways of testing for bias, and eliminating counterfactual biases, in AI outputs and systems, would I sincerely hope be a Good Thing. (Though in writing that I suddenly have my own set of doubts, we've been fooled before....)
- jpadkins 2y agoto be more specific. People with power desire to retain that power. They will work with other people with power to keep that power if it's in their mutual interest. The people change over time, but the basic psychological need and human behaviors are pretty constant. The methods to stay in power tend to evolve, but they match the same patterns throughout history (e.g. Divide and Conquer). That's it. That's the big conspiracy. Some people like to control others.
- ryandrake 2y ago
- dexwiz 2y agoDo you mean unbiased or not biased towards forces in power? Everything will have some bias to it, will it not? Even if the model does not, surely the training material.
- AlexandrB 2y ago> Unbiased AI is, I believe, an existential threat to the “powers that be” retaining control of the narrative, and must be avoided at all costs. I remember when the internet was supposed to be an existential threat to the "powers that be". I'm pretty skeptical of narratives like this because the "powers that be" have a lot of resources to leverage any new technology for their benefit. At best a new technology is gives an asymmetrical advantage to small actors for a short time before everyone else catches on.
- mdgrech23 2y agoMoney talks and if you don't like money they can just throw you out of a window or label you a terrorist so you never had any real power. Once they flex you've got nothing.
- talldayo 2y ago"unbiased" in LLM terms means just random token selection. You inherently need bias (otherwise called "training data") to inform the placement and weight of each token. Furthermore, unbiased AI isn't likely to be any more usable than the garbage we have today. People care about hallucinations, model latency, token pricing and other practical improvements that can be made. Biases are one of the last things stopping people from using AI for legitimate purposes; the other issues are far too glaring to ignore.
- impossiblefork 2y agoYes, but we understand he what means. We even have alternative meanings for bias within ML, such as for the bias added before non-linearities in many neural networks. He obviously means censored LLMs, and I think his view is actually right, although I'm far from sure that these firms are in some kind of scheme to produce LLMs biased in this sense. Uncensored, tunable LLMs under the full control of their users could scour the internet for propaganda, look for connections between people and organisations and just generally make the work of propagandists who don't have their reader's interests in mind more difficult. I think we'll end up with that anyway but it's a reasonable fear that there'd be people trying to prevent us from getting there.
- contagiousflow 2y agoThere is literally no such thing as an unbiased text generator. No matter how you cut it there are an infinite pool of prompts that will need some sort of implicit value system at the heart of the answer. Any implicit bias just from selection of training data will be reflected back at the user. > Uncensored, tunable LLMs under the full control of their users could scour the internet for propaganda, look for connections between people and organisations and just generally make the work of propagandists who don't have their reader's interests in mind more difficult. Even this example, what sources do you trust that is or is not "in the readers best interest", what is propaganda or what is an implicit value in a society, when you tune an LLM does that just mean you're steering it to give answers that you like more? Creating an unbiased LLM is as much of a fools errand as creating an unbiased news publication
- pixelready 2y agoWhile I take your point, I don’t think we’re quite at the enshittification phase of LLM products yet, where some combination of crass marketing and Orwellian narrative control are baked into to the models for profit and/or to curry favor with govt. Right now we are at a phase where the platform companies are still concerned about finding and selling basic use cases before the hype bubble bursts. There is still a strong possibility that LLM as a product is essentially stillborn in the market writ large, because we are trying to use these tools where end users expect a perfectly accurate, repeatable, deterministic response and there is no guarantee that any of these current techniques, cool as they are, will cross the threshold of user expectations and utility enough to justify the costs. Neither AI doomers nor boosters are accurate representations of the general public.
- bearjaws 2y agoIt's already a race to the bottom. Deepseek v2 lite is a damn good model that runs on old hardware already (but its slow). In 2-3 years we will likely have hardware that runs 70b parameter models with enough speed that you will run it locally. Only when you have difficult questions will you actually pay. For example I already use https://cluttr.ai https://cluttr.ai to index my screen shots and it costs me $0. (I made this tool tho)
- metadat 2y agoNeat idea. Is there a self-hostable version of this? I can't upload my screenshots to a 3rd party, because they contain lots of proprietary info.
- quaintdev 2y agoSomebody made it for searching memes few days ago https://www.reddit.com/r/selfhosted/comments/t33rx5/just_released_jina_30_selfhosted_ai_powered/ https://www.reddit.com/r/selfhosted/comments/t33rx5/just_rel...
- bearjaws 2y agoTotally understand, and it can't be local first without OSS. I'll be making it OSS with an Electron App soon.
- countvonbalzac 2y agoYou misspelled screenshots in the title of your website.
- bearjaws 2y agoDerp lol, I'll fix it when I get home
- ugh123 2y agoI think thats a great use case put AI tasks in the background for document processing and indexing into structured data.
- osigurdson 2y agoThe title should include a question mark, as they don't appear to know either.
- onemoresoop 2y ago"Why" was stripped from the title
- LarsDu88 2y agoAuthor here. I had a brain farther when posting this to Hackernews. It should have Why in the posting title
- einpoklum 2y agoUse of LLMs/AI costs energy, and the public is indirectly paying for it one way or another. Generalizing somewhat but focusing on a single company: https://www.statista.com/statistics/788540/energy-consumption-of-google/ https://www.statista.com/statistics/788540/energy-consumptio... In 2022, Google consumed 22.3G Watt-Hours of energy. Total electricity consumption by humanity: https://www.statista.com/statistics/280704/world-power-consumption/ https://www.statista.com/statistics/280704/world-power-consu... In 2022, it was 26T Watt-Hours. Now, Google is a single company, and if we extrapolate with some cocktail-napkin math, let's say that similar tech giants put together consume, say, 20x Google? 50x Google? So between 2% and 6% of all human electricity consumption. I realize that's not broken down for AI, but I'm sure if we do break it down we'll find that's an increasing fraction. In this article: https://cse.engin.umich.edu/stories/power-hungry-ai-researchers-evaluate-energy-consumption-across-models https://cse.engin.umich.edu/stories/power-hungry-ai-research... the quoted figure is 2% of US electricity usage.
- marinmania 2y agoFor what it's worth energy and electricity consumption are not the same. Fuel (for autos, planes, heating oil, etc.) would be included in energy consumption but not electrical. I'm having trouble finding reliable data quickly, but looks like 35% of energy consumption in the world is electrical. It's still an increasing fraction as you say, but it seems like a doubling or quadrupling of energy used for AI would probably have much less of an impact than the share of the population using ice-cars changing a few percentage points.
- einpoklum 2y agoThat's a good point, I'm not sure whether the energy use stats I linked really cover non-electricity fuel use. Maybe they do, but then there's the question of how much of _Google_'s energy use is non-electric, as opposed to everybody's consumption.
- coliveira 2y agoThe big problem with modern tech is the increasing separation between energy use and its results. In the past, one would buy several gallons of gasoline to burn in a car. Nowadays a similar amount of the energy is spent far away and only the results (Google search of ChatGPT results) are seen.
- cut3 2y agoBecause its something very useful that everyone can benefit from like a mobile phone. Its a great value add. If you capture users now, you can extract revenue later at scale. Early bird.
- rbirkby 2y agoGoogle is to Android as Meta is to Llama. But this time it’s all about AR glasses.
- LarsDu88 2y agoMeta is also trying to commoditize AR hardware interestingly enough by releasing their Quest operating system to third parties like samsung. I could write a whole nother article on that phenomenon as I have developed VR apps myself.
- franczesko 2y agoOpen source alternatives means that it will become difficult or close to impossible to monetize llms. It also means that no clear hegemony will occur. Fragmentation might be very beneficial for the status quo.
- badgersnake 2y agoThere is no open source LLM. What even is that? LLMs don’t have source, they have training data and they sure as hell aren’t giving that away for free. Even if they did you’d need to remortgage to rent the GPUs.
- shikon7 2y agoAs stated in the GPL: The “source code” for a work means the preferred form of the work for making modifications to it. Arguably having the weights and code to execute the model available is the preferred form to modify a LLM.
- agentultra 2y agoHow is this even remotely sustainable let alone so low-margin that it could be "given away for free"? The only reason folks are only paying a small monthly subscription for gippity is literally because of all the VC money flowing in. Training, running, and scaling this stuff has a huge cost. The extra datacentres, the fresh water, all the air conditioning, energy usage, chips, the exploited unprotected labour, etc. It seems very expensive. Usually when people selling shovels are giving shovels away for free they're banking on a payoff. Usually a regulatory capture payoff. Or a hedge of some kind? I'm still trying to wrap my head around this. Update: .. VC money flowing in and the extremely favourable taxation "exceptions" for tech companies in thirsty economies...
- mschuster91 2y ago> Usually when people selling shovels are giving shovels away for free they're banking on a payoff. Usually a regulatory capture payoff. Or a hedge of some kind? The payoff is different... everyone's banking on someone making a breakthrough regarding AGI/ASI and then being the first one to actually make a mass market product out of it that achieves dominance. But for that to happen, you need a lot of extremely smart people working for you, and you get these people working for you by giving them something to play around with.
- meiraleal 2y agoThat's the best interpretation of Meta move I've read so far. I guess it is Meta's Chrome moment.
- bookaway 2y agoThe article's title is terrible. The article itself seems to have all the right pieces but connects them in weird ways. Big tech wants to make AI cost nothing to end-users maybe, but Google and Microsoft want the cost of hosting AI to make your eyes bleed so you don't compete or trim any profits off their cloud services. As the article points out Facebook does not offer cloud services so its interests in this case align with mom and pop shops that don't want to be dependent on big tech for AI. But Mistral was way more useful to mom and pop shops when they were trying to eke out performance from self-hostable small models. Microsoft took them out of that game. These enormous models may help out boutique data center companies to compete with big tech's cloud offerings but it's beyond a small dev shop who wants to run a co-pilot on a couple of servers. Microsoft and Google don't want you to learn that a 7B model can come close to a model 50x-100x its size. We don't know that's even possible, you say? That's right we don't know, but they don't even want you to try and find out if it's possible or not. Such is the threat to their cloud offerings. If they did Microsoft would have made a much bigger deal of things like their Orca-math model and would have left Mistral well alone.
- stavros 2y agoHow did Microsoft take Mistral out? I missed that.
- bookaway 2y agoNot "take them out", I said they took them out of the small model game. They did that by giving Mistral free compute, so Mistral turned its main focus to large models. Their announcement after the Microsoft deal was literally "Mistral Large". [0] https://news.ycombinator.com/item?id=39511530 https://news.ycombinator.com/item?id=39511530 [1] https://news.ycombinator.com/item?id=39512683 https://news.ycombinator.com/item?id=39512683
- stavros 2y agoI see, thanks. I didn't know they got free compute.
- 2y ago
- 2OEH8eoCRo0 2y agoNothing in the world's more expensive than free.
- deleted 2y ago[deleted]
- riffic 2y agowhen a service is free, who is the customer and who is the product being sold?
- myhf 2y agoThe current hype cycle is about artificially inflating the value of Nvidia stock by creating do-nothing "products" and "customers". It would be more efficient to have the public pay for GPUs directly.
- LarsDu88 2y agoAuthor here: For cloud hosts like Amazon, Google, and Microsoft, the product is their inference hardware (mostly GPUs). The bigger the open model, the better! For Meta however, its a bit more mysterious. From what I can glean, the stated objective is to ensure that the world standardizes on the same AI platform tooling Meta uses internally (presumably to drive down dev costs?). The bigger "product" however is content creation itself. Giving users the ability to generate engaging content more easily can keep folks using social media and buying ads.
- viccis 2y ago>nation-states like China Is there a reason these articles like to say "nation-state" rather than "countries"? I think the question of whether China is a nation-state is not entirely settled (the state is broader than the nation in its case), but also it seems odd to exclude countries like Belgium that are not nation-states from these kinds of statements.
- dijit 2y agoI agree, it's poor vernacular. What they mean, of course, is an extremely well funded- government sponsored, tech agency acting as an aggressive security service. IE; NSA, GCHQ. As opposed to a tech agency that is acting inwards to the country. IE; CIA, MI5 (Security Service). And opposed to a self-funded hacktivist group. I would greatly prefer better nomenclature though. Nation-State is almost a non-term, since every nation is a state, effectively. - Normally I have seen "state-sponsored", which denotes the correct meaning. In computer science we tend to have quite clear names for things (blue team, red team). Maybe someone could come up with something better?
- viccis 2y ago>Nation-State is almost a non-term, since every nation is a state, effectively. This is not true, especially outside of Europe and the US. Even within Europe, Belgium is not a nation-state for example. Many African countries are not either, with Nigeria being an easy example. State-sponsored is a much better term, I agree, as the phrases are typically used to describe state activity.
- knallfrosch 2y agoThe blue team consists entirely of blue humans, of course.
- sky-wa1ker 2y agoI don't think the author cares about semantics here, "nation-states" is in direct contrast with "larger companies" used in previous line. Using a single word "countries" may not sound sufficient to contrast the pair "larger companies" and may not give it emphasis needed to indicate the gravity of the comparison being made. It had to be this or something like "entire countries".