stellaathena

12 ms·

61.
by stellaathena 6y ago
Hi! I’m the one who wrote this code. My ZeRO-3 implementation is currently not working, but I’ve spoken with DeepSpeed devs and they’ve explained to me what I’ve been doing wrong. I haven’t had time to implement the fix but I don’t see any
62.
by stellaathena 6y ago
EleutherAI’s unofficial motto is “anything safe enough for Microsoft to sell for profit is safe enough to release.” You’re kidding yourself if you think Microsoft cares who they sell it to. There are worthwhile conversations to have about d
63.
by stellaathena 6y ago
It’s a lot less expensive to hire a dozen teenagers to write fake news than use GPT-3. The cost to train GPT-3 is about the same as the cost to hire 1,000 people and pay them $10/hour to write responses for a year. Not to mention the f
64.
by stellaathena 6y ago
“Works with Google” and “gets free compute from TFRC” are not really the same thing.
65.
by stellaathena 6y ago
This is dead right and very important. The data you train on is much more important than model architecture in terms of validation and compliance, and yet it’s a closely held secret. Producing or obtaining good data is a pain in the ass. Fo
66.
by stellaathena 6y ago
Right, this is why a cloud company is sponsoring the replication. They can’t sell GPT-3 as a service until they get a copy of it.
67.
by stellaathena 6y ago
Or for extra irony, ClosedAI
68.
by stellaathena 6y ago
Hey! One of the lead devs here. A cloud computing company called CoreWeave is giving us the compute for free in exchange for us releasing it. We're currently at the ~10B scale and are working on understanding datacenter scale paralleli
69.
by stellaathena 6y ago
... is it in the collection?
70.
by stellaathena 6y ago
If I Google “bibliotik download” the first page of results has a link to download. Calling that “not easy to obtain” is absurd. And yes, your last sentence is correct. That’s exactly the position being taken.
71.
by stellaathena 6y ago
TFRC, a program that lets you borrow Google's TPUs when they're not being used. You can apply here: https://www.tensorflow.org/tfrc
72.
by stellaathena 6y ago
I'm not sure what a BDFL figure is, but Google does not give us millions of dollars. We are a part of TFRC, a program where researchers and non-profits can borrow TPUs when they're not being used. You could say that we are indirec
73.
by stellaathena 6y ago
There's more than 800GB of fanfiction in the world.
74.
by stellaathena 6y ago
Not currently. The code is on GitHub, but we do not have a public-facing model. We felt that the world doesn't need another GPT-2 replica, but if there's interest we can look into doing so though. We are planning on making our GPT
75.
by stellaathena 7y ago
To be more specific about what the other responses say: Being "equivalent to a TM" and being "equivalent to the Python compiler" mean the exact same thing. Pretty much every widely used model of computation is equivalent
76.
by stellaathena 7y ago
For those who don't play Magic, this is on a scale where tier 1 decks are top decks, tier 2 decks are reasonable, tier 3 decks have serious weaknesses but are playable.
77.
by stellaathena 7y ago
This seems like a bizarre rule to add. Why is this result even a bad thing for Magic?
78.
by stellaathena 7y ago
This is not correct: the paper does have to do with optimal strategies. They even say as much in the abstract. >In this paper we show that optimal play in real-world Magic is at least as hard as the Halting Problem, solving a problem tha
79.
by stellaathena 7y ago
Lots of things are more complicated than a computer can handle. There's an entire hierarchy of non-computable problems by complexity (actually several).
80.
by stellaathena 7y ago
Did you actually read the paper? There's a mathematical definition of complexity that is used, and on that metric it is shown to be far harder than any other previously analyzed game.