8 ms·
Can Europe train a frontier AI model on the compute it owns?
- smashini 4mo ago[flagged]
- antiloper 4mo agothanks chatgpt
- coreyh14444 4mo agoClear giveaway: "Honest caveats"
- mistrial9 4mo agothose that know, do not talk?
- barrenko 4mo agoSure, theoretically, if it could come to an agreement, and meanwhile the cross-border cooperation of even the inter-EU countries is at an abysmal rate, and currently, even during a goddamn land war with Russia, Germany and France can't come to an agreement to build a fighter jet ensemble (together).
- qsort 4mo agoThis is unfortunately the problem. The level of the public debate is abysmal, most politicians push unbelivably stupid shit about immigration and other identitarian nonsense, budget gets spent to ensure cheese and wine have the proper AOC certifications on them. Honestly up to a point I even understand it, many people don't see themselves as having a meaningful identity as EU citizens and you can't force it upon them. Asking for sensible AI policy is like asking for a base on mars.
- davidguetta 4mo agoreducing unskilled and hateful immigration is the democratic thing do this has nothing to the other good idea which is to start building AI
- qsort 4mo agoActually yea, it has everything to do with it. I am open to the idea that we should handle immigration differently, but I want a plan and specifics, not slogans. What we want to achieve, and by what mechanisms you plan to get there. Open any newspaper: are you more likely to find careful and considerate opinions or racist screeds? And that is the problem. Time and energy and money and political capital are routinely spent on inconsequential electoral poliTICS rather than substantial poliCY.
- davidguetta 4mo agoyou only need to handle immigration if you have immigration. problem is kinda solved if you don't have immigration or at least "immigration that cause problems" i know many immigrant who don't pose any problem withou need for "handling" or "supervision"
- ben_w 4mo ago> reducing unskilled and hateful immigration is the democratic thing do Hateful, sure, though just like all those on the right who say the first word of "hate crime" is redundant, I'd argue home-grown hate's just as dangerous as imported. But "unskilled" immigration? When the topic is AI? If this stuff works as advertised, *nobody's skills matter any more*. If it doesn't at least render many of our skills obsolete, why build it? If you make an AI which can't automate anything, how is this not a waste of money? Even without that, I've not seen anyone who knows about Baumol's cost disease opine either way about migration, high or low skilled.
- hbcdbff 4mo agoWhy even bother creating a repo like this? Why not just link to a ChatGPT conversation?
- ForHackernews 4mo agoWho cares? Just distill some existing frontier models and run the inference yourselves. Instant sovereign AI.
- sarjann 4mo agoUntil they choose to lock their models down (e.g. Fable). Building policy off copying isn't reliable.
- ForHackernews 4mo agoBy then you've already distilled it. Copying is super reliable. It's a historically-proven way to get nearly the same thing at a fraction of the cost. https://en.wikipedia.org/wiki/First-mover_advantage#Second-mover_advantage https://en.wikipedia.org/wiki/First-mover_advantage#Second-m...
- smashini 4mo agoHowre you going distill if they block your access e.g. Fable
- oneshtein 4mo agoHow you make ROI when your model is blocked?
- ForHackernews 4mo agoJust bribe someone for access, it's still cheaper than training your own model from scratch. They stole the entire accumulated output of humanity and fed it to their beast, no one should feel any moral or ethical compunction about taking from the AI firms. Welcome to the age of digital realpolitik.
- weezing 4mo agoChinese are probably laughing reading this.
- cmiles8 4mo agoThe question was never do enough computers exist in Europe, but rather can Europe organize the capital and cross-company / cross-border relationships required to build a big model at scale. There the answer still looks iffy at best. This is where the US has, and continues to, thrive and where Europe can’t get out of its own way.
- epolanski 4mo agoI don't believe Europe can build models that can compete with American ones. 1. In order to build competitive models, you need to offer competitive salaries and equity. Europe has very old and inadequate corporate law virtually everywhere but in UK. European labs cannot attract talent that can build such competitive models with the proper lack of incentives. 2. In order to build competitive models, you need gargantuan amounts of compute. And thus capital. How can you compete when big tech can just cough a handful of equity and raise $ 85 B like Alphabet is doing right now? 3. In order to have these datacenters financially feasible you need cheap energy. We don't have it. Some places like France have clean one, but it's still not cheap enough, you're still paying a 45-50% premium over some random South Carolina. What Europe should do is to finally tackle its fundamental issues with corporate laws, startups and incentivize more money to flow into venture capital. Essentially we need a bunch of Mistrals, but with more competition and better incentives. There's plenty of brilliant European engineers and scientists that would gladly take some pay cut to work in Europe instead of US and could bring their expertise here, but you still need the right incentives.
- rootlocus 4mo agoYou also need to scrape huge amounts of data with no regard for copyright which is: 1. No longer possible the same way it was for openai and anthropic and 2. Much more regulated in the EU Also the EU would need state backing since we don't have the same private capital, meaning the regulations are even tighter.
- aspenmartin 4mo agoYou can do what frontier labs do today which is to properly license things that are copyrighted and use open source web crawls for things that don’t have copyright issues. You can then also commission new datasets (volume needed goes down when quality is high). The European regulations are the thing that will kneecap anything meaningful coming out of Europe. Mind blowing to me that this is worth the tradeoff since Europe will be beholden to other frontier labs be it China or the US, so regulations accomplishing very little if anything on impacting actual AI development and losing vast amounts of leverage in the process.
- huflungdung 4mo ago[dead]
- solumunus 4mo agoThey really shouldn’t.
- acatton 4mo agoWhat is this whole thing about Europe being behind on AI? Do Mistral and DeepL not exist? Yes, I know DeepL is niche, but IMHO it is the best translation model out there.
- epolanski 4mo agoIt is behind in the sense that if tomorrow the US and China place an export ban on their models all we're left with are Mistral's ones. They are not bad, and they have made huge progress, but you're still one year behind if not more. May matter less and less as time progresses, or it may matter more if research further speeds up. Honestly I wish capitalism and globalization kept working as they did for decades, but since more than a decade we're reverting to inefficient protectionist steps, one after the other.
- SyneRyder 4mo agoI've tried using Mistral for various tasks, and it is so far behind the American models that I just never bother using it despite still having lots of Mistral API credits leftover. Even their OCR and TTS products are surpassed by generic US models - I use regular Claude Sonnet for OCR because it is more accurate than Mistral OCR. I could rant about this, I am just so disappointed at how Mistral completely gave up and pivoted into bespoke fine-tuning consulting. The terrifying thing is that they don't seem to even understand how far behind they are, as if they never tried Opus, let alone Fable / Mythos. Or they do understand and that's why they focus on consulting now.
- 4mo ago
- isoprophlex 4mo agoJust post the prompt that generated this slop next time. Then we'd have a chance of seeing some original thoughts, instead of a bunch of web searches filtered through a bucket of mediocrity.
- beernet 4mo agoAbsolutely not, even if Europe was given all of the compute in the world. The issues are much much worse. Starting at "data privacy", continuing with the "EU AI Act" and just an overall mindset of regulation and German angst. Many structures in the EU actively prevent and fight against innovation. Oftentimes more subtle but the consequences stay the same. If you ever had to use one of the "sovereign" AI provides such as StackIT, OVH and the-like, I feel with you. It is just so bad in terms of product and performance, there is no comparison at all with Hyperscalers, and it shows. Eventually, it's a cultural and structural problem along the way and the future here looks horrible.
- LastTrain 4mo agoSome of those seem like good things TBH.
- throw-the-towel 4mo agoYes, but they are incompatible with being on the forefront of AI.
- LastTrain 4mo agoYep. I wonder if being at the forefront of AI makes for a better & more free society.
- ymir_e 4mo agoCan Europe train a frontier AI model? It seems unlikely at the moment. The writeup you have is quite interesting, but notably distributed compute for frontier is at least a bump in the road. As other's have pointed out, if resources across Europe came together there could be a good amount of resources today to do so, but it is fragmented. Project Glasswing, and most recently the blocking of Fable 5 potentially only for non-US citizens have sparked more attention in the political spheres regarding sovereignty/competitiveness of European AI. There as also this great write-up which explains the trajectory Europe currently has when it comes to AI compute here: https://europe2031.ai/ https://europe2031.ai/ So theoretically: yes, but there doesn't seem to be a big enough will to attempt to catch up with the trajectory of American hyper-scalers.
- smashini 4mo agoThanks! Will give it a read :)
- intoXbox 4mo agoThis has factored out product development, which is more than compute resources. Just like any industry, some organisation needs to take ownership and responsibility to convert technology to a usable product.
- general1465 4mo agoThey could, but why? US and China has poured Trillion of USD into training and any semblance of getting those money back seems like some far fetched dream. Currently there is no realistic path to profitability with these models. What I see as usable product in the future are smaller specialized models which are able to run and be trained with fraction of resources what goes to current frontier models.
- evilturnip 4mo agoI feel like the very thing that EU is great at: more consumer friendly and anti-corporate legistation, also hamstrings it for innovation. Why would VCs invest in AI there instead of in the US where they don't have to worry about any of that.
- graphime 4mo ago[dead]
- bushido 4mo agoUnder normal circumstances, I'd agree with most folks here, it'd be highly unlikely. However, we're (i think officially) in an arms race. I wouldn't want to bet against anyone in these unprecedented times (with plenty of historical parallels).
- thomascountz 4mo agoDoes Europe need to train a frontier AI model, at all? "Training a frontier AI model" is a euphemism. The AI industry is, at present, a disproportionately resource-intensive and exploitative activity which carries only an religious promise of making up for itself at some future date (note that the beliefs being touted to board members are different than the gospel spread to users). More accessible "non-frontier" models are being designed, built, and trained, while bigger models' gains are beginning to asymptote. Economically, Europe does not often hastily participate in new ways of wealth extraction, and with regard to sovereignty, I'd argue that "frontier" models are becoming a liability to the governments who build them and to the populations they're meant to lead.
- cyanydeez 4mo agoit is a bizarre nuclear arms race. what they do need are the raw reaources and the capabilities to sefend themselves ans the capability for offense atleast as a threat.
- thomascountz 4mo agoI'm not sure I follow how this metaphor plays out. Many stand to gain by harming and threatening Europe and many stand to suffer if Europe is harmed or threatened. Far be it from me to claim Europe a perfected system, but there are globalized interdependencies which exist here. I think that I'm not seeing the indicators of hard benefits which correlate to the hypotheses put forward by the financial markets. In which ways are the trillions of dollars of value meant to manifest and how do we know when they begin to do so?
- cyanydeez 4mo agoThese AI models are powerful enough to be used as cyber threats, as a minimum. They may not prove out the other GDP or ROI claims, but the ability to both improve security and dismantle is means if you're not regularly building and improving these models on the software vulnerabilities your organization has, you are increasingly vulnerable to attacks. Since it's impractical to airgap your military and readinessness capabilities, this means you need incountry capabilities because even if America weren't a fascist, you could easily be disconnected from the support from a cyberattack. I think you're just following too much AI hype to see any clear use case.
- ExoticPearTree 4mo agoNo, it cannot. The fundamental problem between the US and EU it is that in the US if something is not explicitly forbidden, it is allowed and in the EU if it is not explicitly allowed it is forbidden. You cannot have innovation at the speed and scale that you have in the US because the legislation is cumbersome and there is no unified market with the same rules.
- thinkingtoilet 4mo agoIt's not just that. The US models were trained partly due to massive theft. They illegally acquired (not used, that was legal) massive amounts of texts that they trained their model on. If you or me stole that much we would be in prison but if you're rich you get to do it and pay a small fine. It's not even the laws. It's in the US you're allowed to break the laws.
- ExoticPearTree 4mo agoI am of the opinion that if it is on the internet, it is free. Just like I find free movies, so do AI companies find free books or whatever they need for their models. What's so hard to understand?
- thinkingtoilet 4mo agoYour opinion doesn't matter when it comes to matters of the law, for right or wrong.
- ExoticPearTree 4mo agoI thought this was the internet where my opinion mattered the most.
- amarant 4mo agoWhat? Your comment is 100% incorrect. In the EU everything is permitted unless explicitly forbidden. Source: I am European.
- giacomoforte 4mo agoNo because public EU compute is adminisered by physicists and although they are very competent in their own domains they are winging it when it comes to AI. It like expecting OakRidge to train LLMs... They already tried training LLMs.
- smashini 4mo ago[dead]
- grigio 4mo agoNo, but EU can train an LLM to regulate others LLM
- slaw 4mo agoOr force replaceable batteries with exceptions that every LLM passes anyway. > If a battery can do 1000 cycles and remain above 80% capacity it is exempt
- lou1306 4mo agoAre we really discussing a plan that has never been attempted before (training a frontier model on federated hardware) and that would require coordination on a continental scale, sketched on a five-page PDF on GitHub with no discernible author or affiliation? a PDF that, I am sorry to point out, reeks of AI prose in more than one passage? What are we even doing here.
- fancyfredbot 4mo agoForget regulations, would it make sense for Europe to train a frontier model of it's own? Would it be sufficiently better than fine tuning a Chinese model? Would it actually be competitive with US frontier models? Would enough people pay to use it even within Europe to pay for the training costs? Do we have enough inference capacity that enough people /could/ use it? Would being "European" allow any governments in Europe to trust it, rather than deciding that actually there needs to be a French, German, Italian, Spanish and UK sovereign AI? I am guessing that enough of these questions can be answered with "no" that nobody really wants to invest. For the same reason there isn't really a serious third start up competitor to OpenAI and Anthropic.
- signatoremo 4mo agoSo your ideas of having AI sovereign is to turn to Chinese models? Do you know what their intention will be and where will they go? You don't like US models because their values don't align with yours, but then you turn to Chinese models? See how hypocrite that is?
- fancyfredbot 4mo agoI was suggesting US models are very good, and would probably be preferable to a less capable sovereign AI. My point was that this makes building European sovereign AI unappealing. Similarly I was not suggesting Chinese models are an alternative to sovereign AI. An alternative which might be able to fulfil some of the objectives of a real sovereign AI while being far cheaper.
- jdkee 4mo agoeurope2031.ai
- niemandhier 4mo agoBeing slow is sometimes a problem. But if the needed total revenue for an investment is so absurdly large, that we need to capture 5-10% of the labour market, being slow might mean not burning in the crash. The US drove the CCCP to bankruptcy by investing in nuclear bombs. Maybe history repeats itself.
- okanat 4mo agoThe US was able to do that because a majority of the world including Europe was determined to exclude USSR from all trade and by putting them into extreme paranoia of total destruction. EU has lots of trading partners from the West and not everything is improved by AI. Yes an economy that's being inflated by AI can affect acquiring of certain resources like DRAM or certain silicon quotas, if you're not joining the bandwagon. However there is enough in-house tech in Europe to prop up the critical industries that do not get affected by AI. The US also needs someone to sell their stuff for keeping the inflated prices. A majority internal market is as unhealthy for the US as the EU.
- niemandhier 4mo agoI was thinking about china driving the US to bankruptcy by commoditising LLMs, thus reducing the revenue.
- bilaxi70 4mo ago[dead]
- rurban 4mo agoWe do. But only for a limited context space, plant images. We do train our own Foundation Model, and it works great. It's the world frontier model. The machine is expensive, yes, but the customers also pay a lot to get it's advantages. And we got the H100`s on ebay. We also have buerocracy and regulations problems, but not with the AI at all. Only with the military civil dual-use restriction of thermal cameras. The ones we are allowed to sell have ridicolous resolutions. It still works, but it's barely useful. And our colleagues across town are also world leader in their AI field. They produce all the world best face recognition SW. They had to use tricks to overcome regulations, but it works. They have their own HW as us. And again other colleagues across town do rent their GPU computes to train their models. That's for office OCR, and speech to text. The town is Dresden btw
- adev_ 4mo agoUnpopular opinion: They can not and they should not. We live in a world where free open weight-models become competitive with frontier model within a year or two. It is much more lucrative and proficient in term of business to solve real problems affecting the industries / govs with A.I right now than it is to do the arm race with OpenAI or Anthropic. That is what Mistral is doing and that served them well so far. The problem is not regulation and never has been. Regulation is a best a minor stone in the shoes of A.I company in Europe. The problem is somewhere else: People fails to understand that there is no equivalent in the EU to the unlimited money tap of American VCs and private funding. That's just not a thing here, the investor landscape is much more dry. Company here just cannot stay unprofitable for 25y while surfing on stocks valuation, it does not work like that in the EU. And as such they cannot compete with behemoth like OpenAI that burn 80B$ a year of cash while staying afloat. I do believe the approach of Mistral is the right one: Solve actual problems right now even if not Edge and construct on top of that. Specially if politically speaking, the White House administration continue to give excellent arguments in favor of sovereignty for the incoming decade. It might be the best strategy they have.
- hulitu 4mo agoNo. Computers are made in the USA or China, OSs are made in the USA (Linux, Windows, Mac). I'm still waiting for the European computer but all microcontrollers (no processors yet) made in EU have an ARM (British-American) Cortex and the RAM is made elsewhere.