7 ms·
EleutherAI One Year Retrospective
- king_magic 5y agoThis was an entertaining read.
- nynx 5y agoSounds like a fun place to hang out!
- Rebelgecko 5y agoCan anyone elaborate on Bruder muss loss?
- siekmanj 5y agoIt's a German phrase which colloquially translated means 'Gotta go bro'
- robbedpeter 5y agohttps://knowyourmeme.com/memes/pedalo https://knowyourmeme.com/memes/pedalo In german, "Brother must go!"
- ThePadawan 5y agoIn addition to the sibling comments translation, the comedy comes from the image itself. The person is doing an incredibly exaggerated sporting pose, wearing actual gym wear, as if he was operating some incredible piece of exercise equipment for cardio. Actual "pedalos" are those things your elementary school would get out once or twice a year for festivals for the kids to have some fun. They're incredible awkward and hard to get moving, to the point where it's more of a coordination exercise for small kids. You certainly can't get any speed on them, so the idea of someone using a pedalo to go faster than walking is ridiculous. This makes the fact that there is an adult posed like this on a adult-sized pedalo basically saying "gotta go fast" hilarious in so many ways.
- albertgoeswoof 5y agoSo MS invested $1bn into OpenAI. Meanwhile, these volunteers built an OSS version of OpenAI’s flagship product in their spare time. Can you imagine if we could redirect funds to actual OSS developers at scale?
- Tenoke 5y agoWhat they did is cool and impressive but that's an extreme oversimplification. Building a small version of something that already exits, has code you can look at (gpt2), paper etc. is going to always be cheaper and easier even if we ignore that OpenAI work on and publish more than gpt. This is before even mentioning that they have an enormous amount of free compute from Google (and others) while Azure credits is a lot of what the 1bn investment to OA included.
- Eridrus 5y agoThis is clearly true, but this is also far cheaper that we could get this capability if we were to really staff a project. If there are volunteers who are capable of and want to contribute to the information commons and are merely constrained by compute, we should clearly help them.
- sivakon 5y agoGPT-J is trained on TPU VM generously offered by Google as part of their TPU research cloud. It’s a small version true but the results are almost on par with reported GPT-3 Source - https://github.com/kingoflolz/mesh-transformer-jax https://github.com/kingoflolz/mesh-transformer-jax
- nmfisher 5y agoIt's more that, if "amateurs"/volunteers can replicate this, it means OpenAI doesn't have much of a moat. (OK, I admit that it wouldn't have been possible without Google's donated TPUs. But the cost isn't so outrageous that it's out of reach for any reasonably funded startup).
- stellaathena 5y ago
- syntaxing 5y agoWhere does the EleutherAI computing resource and the money required to pay for it come from? Edit: I think they got a grant from the TPU Research org [1] https://sites.research.google/trc/ https://sites.research.google/trc/
- Tenoke 5y agoAs it says in the post the free compute from Google's TFRC and also a collaboration with Coreweave who pledged more.
- syntaxing 5y agoMissed it when I was skimming the article but saw a reference to them in the repo
- Tenoke 5y agoAlso for what is worth TFRC is not really a grant grant. You just sign up and if they like you they give you access.
- ma2rten 5y agoI think they do ask you what you are planning to use the resources for, what your qualifications are and make a decision to allocate and how many tpus based on that.
- Tenoke 5y agoYes, they do but it's not really a grant and they don't really follow up on what you do unless you contact them yourself. You just fill a form and if you get accepted get an email later with access. At least that's how it was when I got accepted but things might've changed since last I've heard.
- syntaxing 5y agoInteresting, TIL! I never heard about them until today
- herodoturtle 5y agoNeat. The name "EleutherAI" comes from the Greek word "ελευθερία", which means "freedom". I really enjoy the interplay of "ία" with "AI". I knew nothing about this project, but seeing their interesting name has made me open that link above in a new tab to read later.
- pugio 5y agoI've been wondering that, thank you! That's very cool.
- aerovistae 5y agoPersonally I have to say the sheer amount of memeing and editorializing here is just very distracting. I mean I appreciate that stuff as much as anyone who grew up in the 2000s but some blog posts really dial it up to 11 and become almost unreadable.
- theptip 5y agoThe fact that tacking on the phrase “unreal engine” to an image generation prompt makes the image way better is both hilarious and kind of terrifying to me. In the first generation of AI there was this idea that we could use AI to understand human consciousness; just make a conscious machine and then analyze it, pause it, rewind it, etc. The more we learn about AI, the more I think we’ll lose any chance to understand them long before they pass a complexity threshold where we start debating if they are conscious. Communing with AI is starting to look more like some kind of Eldritch incantation…
- LolWolf 5y agoTo be fair, communicating with (and even training) AI has always been an Eldritch incantation at some level…
- AnthonBerg 5y agoIt makes perfect sense to me – images that appear with the words “unreal engine” will tend to be a product of dedicated work to make good looking pictures by working on or in Unreal Engine. Compared to an arbitrary mass of images, these will probably tend to be visually interesting, have perfect color and exposure, and have a useful homogeneity of style.
- Eliezer 5y agoInteresting how much the people with boots on the ground and no PR departments to filter them actually do seem… pretty aware of how much we’re all going to die? Or at least they seem aware the probability of igniting the atmosphere is over 50% and not nicely contained in the distant future. Something of an update there for me.
- simiones 5y agoThis is far-fetched sci-fi problem invention. The only real danger of AI in the next 1000 years is in things no one in the field is seriously addressing: use of AI in things like law enforcement, trained on bad data, to accelerate and justify existing systemic biases.
- geofft 5y agoNot that I disagree, but it's not just a danger of AI, it's a danger of complex self-optimizing systems in general. We've got a pretty robust complex self-optimizing system already, which goes by names like "society" and "the free market". You point it at bad data and it will reinforce that data. If demographic A is poorer than demographic B, market forces will determine that giving loans to demographic B will get you better risk-adjusted returns, causing demographic B to become even more relatively prosperous, justifying future decisions to prefer giving loans to demographic B. Which is to say that, while I think it's good for folks thinking about AI alignment to worry about this problem, all of us should worry about this problem.
- hollerith 5y ago>The only real danger of AI in the next 1000 years is . . . That's not true.
- littlestymaar 5y agoWe're all going to die, like all our ancestors did before us, that's no surprise here ;). And about “igniting the atmosphere”, I'm much more worried about what we are currently doing (global warming) than with what may go wrong with AI…