6 ms·
Most of the researchers are specifically in a handful of big labs, most of which in turn are in the USA. As one who has done so, trust me when I say that reloca
by ben_w 3y ago
Most of the researchers are specifically in a handful of big labs, most of which in turn are in the USA. As one who has done so, trust me when I say that relocation is harder than it seems on paper.
Also note that both California and the EU are economically dominant enough that they tend to influence regulation outside their own borders: https://en.wikipedia.org/wiki/Brussels_effect https://en.wikipedia.org/wiki/Brussels_effect
Further note how many people signed the Pause letter: https://futureoflife.org/open-letter/pause-giant-ai-experiments/ https://futureoflife.org/open-letter/pause-giant-ai-experime...
In addition, given how much training data current models need, there would be a huge impact just by a handful of governments siding with all the copyright holders suing OpenAI, Midjourney, Stability AI, etc.
And all that is without needing Yudkowsky's point that a ban isn't serious unless you're willing to escalate to performing airstrikes on data centres.
- plastic_enjoyer 3y ago> Most of the researchers are specifically in a handful of big labs, most of which in turn are in the USA. As one who has done so, trust me when I say that relocation is harder than it seems on paper. I think it's a bit short-sighted to assume that the future development of AI models will only come from large corporations or research institutions. At the moment, the training still requires a lot of computing resources and a lot of data - but these are limitations that are not permanent. Since the AI boom that Alex triggered in 2012, the development and optimization of models and training has improved very quickly, to the point that even hackers and enthusiasts can create very good models with a little money. Just because a government heavily regulates AI and possibly bans open weights doesn't mean that people on the fringes of legality will abide by it.
- ben_w 3y ago> I think it's a bit short-sighted to assume that the future development of AI models will only come from large corporations or research institutions. If this was a "forever" solution, I would agree. I believe the goal is more along the lines of trying to make some progress with the foundation of what "safety" even means in this concept. > Just because a government heavily regulates AI and possibly bans open weights doesn't mean that people on the fringes of legality will abide by it. Nor nation states. However, we do have international treaties — flawed though they are — on things from nuclear proliferation to CFCs. As a smaller-scale example: we can't totally prevent gun crime, yet the UK manages to be so much safer in this regard than the USA that even the police in the UK say that they do not want to be armed in the course of their duties. I don't know how realistic the concerns are for any given model; unfortunately, part of the problem is that nobody else really knows either — if we knew how to tell in advance which models were and were not safe, nobody would need to ask for a pause in development. All we have is the age-old split of neophilia and neophobia, of trying things and of being scared of change. We get this right, it's supper happy fun post-scarcity fully automated luxury space communism for all. We get it wrong, and there's more potential dystopias than have yet been written.
- skissane 3y ago> I believe the goal is more along the lines of trying to make some progress with the foundation of what "safety" even means in this concept. Right now, what “safety” seems to mean in practice is, big (mostly American) corporations imposing their ethical judgements on everyone else, whether or not everyone else happens to agree with them. And I’m sceptical it is going to mean anything more than that any time soon. If one is seriously concerned about the risk that “superintelligent AI decides to exterminate humanity”, I think this kind of “safety” actually increases that risk. Humans radically disagree on fundamental values, and that value diversity, those irreconcilable differences - from the values of the average Silicon Valley “AI safety researcher” to the values of Ali Khamenei - creates a tension which prevents any one country/institution/movement/government/party/religion/etc from “taking over the planet”. If advanced AIs have the same value diversity, they’ll have the same irreconcilable differences, which will undermine any attempt by them to coordinate against humanity. If we enforce an ethical monoculture (based on a particular dominant value system) on AIs, which is what a lot of this “safety” stuff actually about, that removes that safety protection. It would be rather ironic if, in the name of protecting humanity from extinction, “AI safety researchers” are actually helping to bring it about
- ben_w 3y ago> Right now, what “safety” seems to mean in practice is, big (mostly American) corporations imposing their ethical judgements on everyone else, whether or not everyone else happens to agree with them. And I’m sceptical it is going to mean anything more than that any time soon. Likewise. As a non-Ami, I don't like these specific ethical judgements being imposed on me, and share your scepticism. It could be much, much worse — but it's still not something I actively like. > Humans radically disagree on fundamental values Agreed. My usual example of this is "murder is wrong", except we don't agree what counts as murder — for some of us this includes abortion, for some of us the death penalty, for some of us meat, and for some of us war. > If advanced AIs have the same value diversity, they’ll have the same irreconcilable differences, which will undermine any attempt by them to coordinate against humanity. Not necessarily. Humans also band together when faced with outside threats, even if we fracture again soon after the threat has passed. Also: the value diversity of "Protestant vs. Catholic" or "Royalist vs. Parliamentarian" in the middle ages did not protect wolves from being hunted to extinction in the UK, and whatever value differences there were between (or within) the Sioux vs. the Ojibwe didn't matter much for the construction of the Dakota Access Pipeline. I therefore think we should try to work on the alignment problem before they become generally as capable as a human, let alone generally more capable: the capabilities are where I think the risk is to be found, as without capability they are no threat; and with capability they are likely to impose whatever "ethics" (or non-anthropomorphised equivalent) they happen to have, regardless of if those "ethics" are something we engineered deliberately or if it's a wildly un-human default from optimising some reward function and becoming a de-facto utility monster: https://en.wikipedia.org/wiki/Utility_monster https://en.wikipedia.org/wiki/Utility_monster > If we enforce an ethical monoculture (based on a particular dominant value system) on AIs, which is what a lot of this “safety” stuff actually about, that removes that safety protection. I agree that monocultures are bad. I agree that there is a risk of a brittle partial solution to safety and alignment if the work is done on the mistaken belief that some system monoculture is representative of the entire problem space. Sometimes I'm tempted to make the comparison with a drunk looking for their keys under a lamp-post because that's where it's bright… but the story there is supposed to include the drunk knowing that's not where the keys are, whereas we are more like children who have yet to learn what it means for something to be a key and thus are looking for one specific design to the exclusion of others. While it is extremely difficult to get humans to "think outside the box", and thus the monoculture-induced blindness — and mistaking the map for the territory — is something I take seriously, I also think it's useful for us to take baby steps with relatively simple models like LLMs and diffusion models. I also think that if an AI is developed with a monoculture, its fragility is likely to work in our favour in the extreme case of an AI agent taking over (which I hope is an unlikely risk, and may not be enough to be net-benefit against shorter-term or smaller-scale risks while we think about the alignment problem), as there will be "thoughts it cannot think": https://benwheatley.github.io/blog/2018/06/26-11.32.27.html https://benwheatley.github.io/blog/2018/06/26-11.32.27.html
- spacebanana7 3y agoThere are ways of structuring entities to avoid relocating 'physical labs'. The big tech companies have excellent legal and accounting departments that consistently beat the USA and EU in tax optimisation through IP relocation, and many of those same principles could be applied to AI. e.g - A model could be trained in the the USA then privately donated to a non-Profit in the Seychelles (or some friendly jurisdiction) where it could be published. If regulators clamp down on basic sharing, the model could be pre trained in the USA and fine tuned in the Seychelles before release. If regulators clamp down on intermediate sharing, the model could be designed in the USA but full training could occur in the Seychelles. If regulators clamp down on offshore training by denying the Seychelles access to GPU purchases, American entities could donate the physical GPUs.
- ben_w 3y ago> The big tech companies have excellent legal and accounting departments that consistently beat the USA and EU in tax optimisation through IP relocation, and many of those same principles could be applied to AI. This tax optimisation has resulted in lawsuits: https://en.wikipedia.org/wiki/Apple%27s_EU_tax_dispute https://en.wikipedia.org/wiki/Apple%27s_EU_tax_dispute … and this kind of attitude is also why GDPR fines are based on global revenue rather than EU revenue (because it's too easy to play a shell game with money). Also note that laws can reach across borders, for example: • https://en.wikipedia.org/wiki/Max_Schrems https://en.wikipedia.org/wiki/Max_Schrems • https://en.wikipedia.org/wiki/Microsoft_Corp._v._United_States https://en.wikipedia.org/wiki/Microsoft_Corp._v._United_Stat... • That the US government wants to know about all use of cryptography in apps distributed on the Apple App Store, even when it's made by a German corporation for German users in Germany and is never localised to English. > If regulators clamp down on offshore training by denying the Seychelles access to GPU purchases, American entities could donate the physical GPUs. Hence "Track all GPUs sold. If intelligence says that a country outside the agreement is building a GPU cluster, be less scared of a shooting conflict between nations than of the moratorium being violated; be willing to destroy a rogue datacenter by airstrike." - https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-not-enough/ https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-no... You may consider this "overkill" — I sure do, but (and with acknowledgement that fiction is not a guide to reality) I also read Schlock Mercenary back in the day: > There is no "overkill." There is only "open fire" and "reload."
- PeterisP 3y agoBut you don't need "most of the researchers" or relocating them, nor you need legal data - with sufficient money and hardware, if North Korea or any other country puts a bunch of median CS grad-students to the task, they can train whatever models they want; the code and data is practically available and will be, no matter what the bans.
- ben_w 3y ago> with sufficient money and hardware Is making an assumption that hardware is not also facing restrictions. > code and data is practically available Disagree; although the base model data is, the RLHF data isn't. This is also why 3rd parties are not able to replicate Google, despite all the PageRank patents having expired and the web being about as crawlable for Google as for, say, Bing. > "median CS grad-student" … is how I regard the quality of ChatGPT's code output, FWIW.
- skissane 3y ago> the RLHF data isn't There are open sources of RLHF data - for example, the LMSys Chatbot Arena dataset (for same prompt two different responses along with which the human preferred), ShareGPT It also isn’t hard to use existing leading AIs like GPT-4 or Claude-3 to generate synthetic RLHF datasets. They’ll put words in their terms of service to say you can’t use a dataset generated in that way to a train a competing model, but their ability to enforce those terms in practice is very open to question
- michaelt 3y ago> Most of the researchers are specifically in a handful of big labs, most of which in turn are in the USA. Looking at https://paperswithcode.com/greatest https://paperswithcode.com/greatest a lot of the papers come from US labs - but there are also quite a few that come from Chinese labs. And consider the case when an influential paper comes out of a Microsoft research lab with the authors He, Zhang, Ren and Sun; obviously you can't guess someone's nationality just from their name - but I think it's naive to think the US has any sort of guaranteed monopoly on ML. Especially if the US decided to send the message that ML research isn't welcome.
- 627467 3y agoI think this is FUD. during the pandemic the EU failed to prevent "their own" vaccines manufactured in their own soil to get to the UK - to the outrage of EU leaders.[0] Russia has shown the ability to doge European sanction - and to avoid more serious one. if anything this shows European willingness to proliferate behind the shadows. an hypothetical ban on AI proliferation is really just regulatory capture - excuses given are just PR to temporarily frighten the public into acceptance. [0] https://www.theguardian.com/world/2021/jan/27/eu-covid-vaccine-row-astrazeneca-european-commission https://www.theguardian.com/world/2021/jan/27/eu-covid-vacci...