psb217

7 ms·

31.
by psb217 2y ago
If getting drunk significantly increased my ability to appear competent then, yes, I would...
32.
by psb217 2y ago
You also have to account for how much Trump cares about his supporters vs how much any Canadian politician with sufficient power to act cares about their supporters. Trump is already elected, so whatever self-interested concern he had for h
33.
by psb217 2y ago
That's true enough. The problem is that they quickly turn into echo chambers where "is an expert" means "shares our opinions." Once these communities become established (in academia or industry), the appetite for no
34.
by psb217 2y ago
That's mostly just from cliques of friends hiring each other back and forth, forever ))<>((. OpenAI tries to win by hiring from Anthropic. Anthropic tries to win by hiring from OpenAI. Meta tries to win by hiring from Google. Goo
35.
by psb217 2y ago
My point was that the scope/impact/value/etc of the contributions made by individual engineers will be determined more by the projects they're working on than by their inherent ability to contribute. So, if we go through
36.
by psb217 2y ago
"I discovered that if one looks a little closer at this beautiful world, there are always red ants underneath." -- Well, he ain't just talking about literal ants...
37.
by psb217 2y ago
The sorts of decisions and results that make a company the size of Meta succeed or fail happen above the levels of the folks who will get cut. Most of the net value produced by individual engineers is determined by which projects they'
38.
by psb217 2y ago
Human-labeled data currently looks like the quickest path to making robots that are useful enough to have economic value beyond settings and tasks that are explicitly designed for robots. This has drawn a lot of corporate and academic resea
39.
by psb217 2y ago
If you wanna sound hip, you need to call it "system 2" and "system 1".
40.
by psb217 2y ago
The people who will soon be waving big, scary, regulatory sticks in his direction, presumably.
41.
by psb217 2y ago
If there was no incentive for an entire class of businesses to do X, you would not have to actively work to stop them from doing X.
42.
by psb217 2y ago
It's not unreasonable to argue for socialized healthcare based on treatment denials in private healthcare, since there are impactful differences in the incentives driving denials and rationing in private vs socialized healthcare. I agr
43.
by psb217 2y ago
Yes, I read the whole thing, thanks. It's the standard form letter for people who beat the startup game after a life of mindless career-maxing. To me, this sort of thing reads as humble-braggy, faux-introspective slop. Now he can point
44.
by psb217 2y ago
How is any of this introspective, reflective, or self-critical? The author was a caricature of silicon valley tech bro-ness throughout, followed by a final return to form by deciding to "study physics" in order to start another co
45.
by psb217 2y ago
If someone says: "I'm not a big fan of football", they're not necessarily saying: "We should kill all football players." If someone says: "I think DOGE is BS", they're not saying: "I think a
46.
by psb217 2y ago
The value of synthetic data relies on having non-zero signal about which generated data is "better" or "worse". In a sense, this what reinforcement learning is about. Ie, generate some data, have that data scored by some
47.
by psb217 2y ago
It's tricky to judge the difficulty of these sorts of things. Eg, breadth of possibilities isn't an automatic sign of difficulty. I imagine the space of programming problems permits as much variety as ARC-AGI, but since we're
48.
by psb217 2y ago
Yeah, I finished a PhD in machine learning around 2016 and have been working professionally as a researcher since then, though I'm currently between jobs. It's a fun gig, but the "publish or perish" aspect of academic-is
49.
by psb217 2y ago
Well, with billions in funding you could task a hundred or so very well paid researchers to do their best at reverse engineering the general thought process which went into ARC-AGI, and then generate fresh training data and labeled CoTs unt
50.
by psb217 2y ago
The simulation/rendering is actually pretty fast since it's all done by heavily optimized gpu-based physics and graphics engines. The "generative" part is that they have some LLM stuff that's finetuned for generatin
51.
by psb217 2y ago
Another relevant point is the common anecdote about, eg, some master engineer who gets paid big bucks to come fix some problem that's been blocking up a factory for weeks. The engineer walks around, listens to a few of the machines, an
52.
by psb217 2y ago
One way of thinking about the "Approximate Monotonic Constraint" is that you're running a quick and dirty edge detector on the entropy. Ie, you're clipping based on the gradient of per-byte entropy wrt timestep compared
53.
by psb217 2y ago
Yeah, Meta pays well, but they probably wouldn't match the $10M equity pile that some generic engineer or researcher who joined 1 year before ChatGPT dropped could be sitting on. Ofc, the equity->cash conversion rate for OpenAI pseu
54.
by psb217 2y ago
"...because it has to somehow tell the next layer how to generate the next token prediction." -- This isn't actually true in the case of transformers. Features in the final TF layer at time t in a sequence do not depend on th
55.
by psb217 2y ago
The way the recurrence in this method works -- ie, using last LLM hidden state at previous time step as input token for the next time step -- isn't directly compatible with how recurrence/autoregression is typically handled during
56.
by psb217 2y ago
It's super cool when you have to go through a chain of several specialists while dialing in on the problem. Especially when the default diagnosis is typically: "it's probably nothing, wait six weeks and make another appointme
57.
by psb217 2y ago
It's worth keeping in mind that "there exists X such that Y is true" is not the same as "Y is true for all X". People love using these sorts of statements since they're technically true as written, but most peo
58.
by psb217 2y ago
A key strength of conventional 3d assets is that their form is independent of the scenes in which they will be rendered. Models that work purely in pixel space avoid the constraints imposed by representing assets in a fixed format, but they
59.
by psb217 2y ago
RE: "Genie 2 is capable of remembering parts of the world that are no longer in view and then rendering them accurately when they become observable again." -- This claim is almost certainly wildly misleading. This claim is technic
60.
by psb217 2y ago
The addition of search engines and recommendation algorithms is significant too. It's not just the ability to spread arbitrary information at zero cost, there's also a strong feedback loop where any information a user/rube en

More ›