moyix

20 ms·

181.
by moyix 4y ago
cperciva, if you're around, I'd be really curious to hear a bit more about the details behind your attempts to get "Cache Missing for Fun and Profit". I'm really surprised that it was so hard to find a home for it!
182.
by moyix 4y ago
They're currently doing this automatically for catching exact matches, and you can turn on a setting that will suppress any suggestion that was found verbatim in the training data. But of course this wouldn't catch copies where va
183.
Behind the Throne: Twitter Game Illustrated by DALL·E 2(twitter.com)
1 points by moyix 4y ago|0 comments
184.
by moyix 4y ago
For a more humorous tale of recovering a private key after a chunk of it was damaged (in this case, replaced by the string "FUCK A DUCK"), see this hilarious CRYPTO 2012 rump session talk: https://crypto.2012.rump.cr.yp
185.
by moyix 4y ago
Oh, you might be right – I originally had that but second-guessed myself (thinking that once you have the k bit positions, you need to exhaustively search the 2^k possible settings for those bits). I guess for each possible set of positions
186.
by moyix 4y ago
I'm kind of surprised that this took "two weeks, a stable of computers, and billions of combinations tested"? If we make the (generous) assumption that this was using a 128-bit key (more than was common in 1993—the age of DES
187.
by moyix 4y ago
Yep, it has been proposed: https://arxiv.org/abs/1802.04633
188.
by moyix 4y ago
I have been enjoying Natural Language Processing with Transformers [1]. It's largely focused on the Huggingface library, but Chapter 3 has a very nice walkthrough that builds up the encoder portion of an encoder-decoder Transformer fro
189.
by moyix 5y ago
I realize it's a much more cleanly defined domain, but how does this interpretation explain MuZero?
190.
by moyix 5y ago
Here's one: https://twitter.com/jmhessel/status/1511783083967586306 And a non-cherry picked depiction of cherry-picking scientists: https://twitter.com/jmhessel/status/15121432260224
191.
by moyix 5y ago
> But if doesn't spit up exactly what you want can't edit it further. Why? You can tweak the prompt, change parameters, or even use the actual "edit" capability that they demo in the post.
192.
by moyix 5y ago
[I already posted this as a comment on Gelman's blog this morning, but reposting here for visibility] I’m almost certain that OpenAI is not updating the model on a day by day basis (as Smith implies in part 5), and I would be extremely
193.
by moyix 5y ago
> I think there is a very fine line between claiming that a CNN's weights represent an algorithm and that its weights can be _interpreted_ as an algorithm. Yeah, I agree this is an issue. It feels a bit reminiscent of Searle's
194.
by moyix 5y ago
Invent is probably the wrong word since it implies agency, sure. Maybe "discover" or "luck upon", since whatever it's doing was formed by updating a pile of floating point weights with gradient descent? I think it c
195.
by moyix 5y ago
It's a little bit more complex here because tokens are variable-length. So getting the order of magnitude (i.e. number of digits) correct requires that it be able to pick tokens for the beginning and end that have the right start/
196.
by moyix 5y ago
Yup, quite possible that this has something to do with it. There is other work showing that giving LMs a "scratchpad" for intermediate computations allows them to do much better not just at arithmetic but also things like predicti
197.
by moyix 5y ago
Yeah it's interesting how moving up a level of abstraction works well here! "Write me a function that multiplies two numbers" works much better than trying to get it to multiply the numbers themselves. There's a recent-i
198.
by moyix 5y ago
Hey! As the author of the gist, just wanted to clear up what seem to be a few misconceptions: - This isn't GPT-3, it's the recently-released open-source and open-weights model from EleutherAI, GPT-NeoX-20B. GPT-3 is much larger (1
199.
by moyix 5y ago
Here's one that lists some older references: https://arxiv.org/abs/2004.04328
200.
by moyix 5y ago
This does look pretty nice, although I am not sure how well it would play with bear, which also uses LD_PRELOAD to intercept some calls.
201.
by moyix 5y ago
Right, and I definitely want to try building existing ports. But unless I'm missing something, this still requires you to write down by hand the dependencies and steps for actually building each project? That's not really feasible
202.
by moyix 5y ago
Sure, I think this would be a really interesting study to do! I have been meaning to scrape a big chunk of GitHub in order to do this kind of analysis. I think I would be surprised (just based on my own use of Codex and Copilot) if they wer
203.
by moyix 5y ago
Yup, pretty much. If you want a static analysis tool to be able to actually parse your C code and see it like the compiler would, it needs to have all the info that the compiler did – header files, defines (including ones passed in with -D
204.
by moyix 5y ago
I will be chatting with some folks who work on reproducible builds (turns out some of them are my colleagues here at NYU! :)) to see if I can use their infrastructure to extract some of this information. That would no doubt be a big boon fo
205.
by moyix 5y ago
Neat, I will give those a shot!
206.
by moyix 5y ago
For fun I rephrased the prompt a little. "Middle bits" is kind of vague; when provided an explicit description of which bits you want to set it does fine: Prompt: // Function to set bits 29-34 in a uint64 to 1 func
207.
by moyix 5y ago
AlphaCode does some analysis of training data copying in their paper (Sections 6.1 and Appendix F): https://storage.googleapis.com/deepmind-media/AlphaCode/comp... It does not seem to be copying from the training
208.
by moyix 5y ago
Nice idea! The more data the better, from my perspective :) Presumably a lot of the code overlaps, but there's also probably plenty that's only in FreeBSD. I would love to extend this even further to include projects on GitHub, bu
209.
by moyix 5y ago
Sure, sorry if I was too harsh on bash here :) It really wasn't the right tool for the job in some parts of my use case though – parsing info out of Sources.gz using awk and sed and friends is just plain bad compared to using python-de
210.
by moyix 5y ago
The debian build system already runs `make -j$(nproc)` whenever the package supports it. So in principle sometimes I'm running 1024 (32*32) concurrent compilations, but it's rare for that many to actually be active at once (most b

More ›