Imnimo

28 ms·

361.
by Imnimo 3y ago
And, if I'm reading their calculation right, that's 85% on the medium-difficulty bucket, not even the entire HumanEval benchmark? (quoting from the GPT-4 paper): >All but the 15 hardest HumanEval problems were split into 6 diff
362.
by Imnimo 3y ago
>The fact that scaling continues to work has significant implications for the timelines of AGI development. The scaling hypothesis is the idea that we may have most of the pieces in place needed to build AGI and that most of the remainin
363.
by Imnimo 3y ago
Even more than the complex prompts, they give the bot access to a variety of functions implementing primitive actions, which are crafted to give helpful hints if the bot is struggling. For example, the code for mining a block: https:&#x2F
364.
by Imnimo 3y ago
My guess is that the author misunderstands this quote from the paper abstract: >For instance, in Game of 24, while GPT-4 with chain-of-thought prompting only solved 4% of tasks, our method achieved a success rate of 74%.
365.
by Imnimo 3y ago
It's always fun to look at the prompts used by these projects. Here are a few snippets from this one: >You are a helpful assistant that tells me the next immediate task to do in Minecraft. My ultimate goal is to discover as many div
366.
by Imnimo 3y ago
Doesn't this mean that the chance of a true but unobserved -1.9 magnitude effect in men is much greater than the chance of a true but unobserved +0.0 magnitude effect in women?
367.
by Imnimo 3y ago
The trick here is that they get to put their own think pieces alongside actually influential work and pretend like the two deserve to share a stage.
368.
by Imnimo 3y ago
For example: https://www.theatlantic.com/health/archive/2014/02/misunders...
369.
by Imnimo 3y ago
Do you think the same is true for fruits, vegetables and nuts?
370.
by Imnimo 3y ago
>Does the n-gram model really need all those parameters to mimic GPT-4? Yes, it does. I don't understand what this argument is supposed to demonstrate. Obviously you can compress the 8000-gram model that GPT-4 represents - GPT-4&#x2
371.
by Imnimo 3y ago
I feel like having words like "eee" and "eeee" or "g's" and "gs" on the list sort of defeats the point of "correct horse battery staple" - that the words are easily memorable and non am
372.
by Imnimo 3y ago
Sure, even nuclear annihilation is asymmetric - the amount you spend on making the bombs is much less than the amount you can destroy with them. But that doesn't automatically imply that there are routes to global destruction that are
373.
by Imnimo 3y ago
>A genocidal human with access to a rogue AI could ask it to find ways to destroy humanity or a large fraction of it. This is different from the nuclear bomb scenario (which requires huge capital and expertise and would “only” destroy a
374.
by Imnimo 3y ago
I remember back in the early 90s there was an expansion for SimCity 2000 called the Urban Renewal Kit, which let you draw your own building sprites to replace the ones in the base game. It also came with a pre-made pack of futuristic buildi
375.
by Imnimo 3y ago
I guess I would have expected the dataset to have only the occasional horrible prompt if it were just user-provided data, but I got the sense from the article that these guys were labelling horrific text all day.
376.
by Imnimo 3y ago
I'm not sure I understand what part of RLHF would involve this sort of experience. The description in the OpenAI blogpost ( https://openai.com/research/instruction-following ) is: >To make our models safer, more
377.
by Imnimo 3y ago
Sure, so to see how these things can be learned, we should be a little more precise about how they work. Each token is a vector, and from that vector we compute three things - a query, a key and a value. Each of these is typically computed
378.
by Imnimo 3y ago
Here is an attempt that sacrifices technical precision for intuition. We're going to represent words as vectors (a sequence of numbers). We would like it to be the case that the value of the numbers reflects the meaning of the words. W
379.
by Imnimo 3y ago
>We inspected StarCoder-generated programs on these benchmarks and found that there were several cases where the model produces what are effectively empty solutions, e.g., pass or a comment Insert code here. We also observed this kind of
380.
by Imnimo 3y ago
I'm not sure this really improves "it's not how white men fight" at all.
381.
by Imnimo 3y ago
I would guess "haven't", based on how many arrests do not even result in a case being referred to prosecutors. Certainly there is some portion of those cases who did commit the crime but the state lacks evidence, but my hunch
382.
by Imnimo 3y ago
This was surprisingly fun. Tricks that I expected to carry me most of the way only worked for a few rounds.
383.
by Imnimo 3y ago
>For example, we loaded the entire text of The Great Gatsby into Claude-Instant (72K tokens) and modified one line to say Mr. Carraway was “a software engineer that works on machine learning tooling at Anthropic.” When we asked the model
384.
by Imnimo 3y ago
Perhaps you could vectorize chess positions by passing them to Leela or another NN-based engine and using it's internal activations as the embedding.
385.
by Imnimo 3y ago
>Unless you work for OpenAI, your GPT leverage is likely wasted on trivial business code But trivial code is the only level of code GPT can really write.
386.
by Imnimo 3y ago
I don't think this is showing LLMs performing decoding. They're just using the LLM to propose possible words. The decoding is done by using another model to score how well a proposed word matches brain activity, and using that sco
387.
by Imnimo 3y ago
To me the value here is not that GPT4 has some special insight into explaining the behavior of GPT2 neurons (they say it's comparable to "human contractors" - but human performance on this task is also quite poor). The value
388.
by Imnimo 3y ago
>“With climate change, it’s very easy to recommend what you should do: you just stop burning carbon. If you do that, eventually things will be okay. For this it’s not at all clear what you should do.” If we accept (I don't, but for
389.
by Imnimo 3y ago
How do you think any other taxes are enforced?
390.
by Imnimo 3y ago
I don't understand why this would be difficult to enforce? It's a tax on companies that are doing crypto mining, not on an individual running mining on their personal machine.

More ›