Veedrac

29 ms·

331.
by Veedrac 4y ago
The problem with this sort of argument against caring about SOTA scores is that there is only so much luck to go around. While any individual 5% reduction in error rates could theoretically be highly influenced by luck, if you have a chain
332.
by Veedrac 4y ago
I'm sort of baffled whether people saying this sort of stuff actually read ML papers. Because this is just overtly not true, this idea that the majority of papers do scaling and nothing else. There are tons of papers exploring creative
333.
by Veedrac 4y ago
That might be a reasonable criticism if it remotely reflected reality, but scaling has repeatedly shown to produce qualitatively stronger models, by large margins, doing things that would seem unimaginable for smaller models.
334.
by Veedrac 4y ago
That quote is disingenuous. Do people really think that... * Jeff Dean, lead of Google's AI division, wrote a paper with all that complexity to get SOTA on CIFAR-10? * Jeff Dean, whose salary is sometimes estimated as $3m/y and
335.
by Veedrac 4y ago
I thought I was doing well after not being overly surprised by DALL-E 2 or Gato. How am I still not calibrated on this stuff? I know I am meant to be the one who constantly argues that language models already have sophisticated semantic und
336.
by Veedrac 4y ago
The number you quoted was 2 FLOPs per synapse, not per neuron. A neuron is complicated and hierarchical, but it is unreasonable to say it's much more complex than an operation per synapse.
337.
by Veedrac 4y ago
If you've seen much DALL-E 2 output, it's pretty obvious they can learn such things. Example: https://old.reddit.com/r/dalle2/comments/u9awwt/pencil_sharp... .
338.
by Veedrac 4y ago
I don't know, sorry.
339.
by Veedrac 4y ago
It sounds less stupid when you count the fraction of EA funding and activity that HPMOR was already in the causal chain of. It's not how I'd do outreach, but it's not like the choice was arbitrary.
340.
by Veedrac 4y ago
Apologies if this is curt, but if you think one of the most impactful altruism movements in the world with major focuses in global health and long term outcomes is a joke because they do outreach in odd (but clearly effective) ways, you hav
341.
by Veedrac 4y ago
...Never? I fail to imagine somebody finding the mouse in VR[1], the sneezing fennec[2], or the programming sloth[3] viscerally disturbing. [1]: https://i.redd.it/rfn3io4urcu81.jpg [2]: https://i.redd.it/5t4
342.
by Veedrac 4y ago
> This is a very thoughtful and clearly articulated Can you point to a part that you think fits that description? Because it read as complete nonsense to me.
343.
by Veedrac 4y ago
That's because of reuse, as kerosene burns slightly sooty, which can be cleaned but takes maintainance, whereas methane burns clean. You wouldn't use methane for a single use engine unless you were buying off the shelf like Vulcan
344.
by Veedrac 5y ago
> Current rockets mostly use kerosene instead, in spite of kerosene's far-worse efficiency in $millions-per-launch rocket engines, to avoid dealing with liquid hydrogen. Kerosene is used because it's better, moreso than the dif
345.
by Veedrac 5y ago
Now this is just playing semantic games. Nobody has ever claimed Musk is building Tesla's cars or SpaceX's rockets by hand. Obviously those are built by the factories and the people employed their for that purpose. There is no way
346.
by Veedrac 5y ago
I was addressing the second paragraph, not the first.
347.
by Veedrac 5y ago
That strategy gives you better margins . It doesn't give you better revenue or profits. The biggest problem Tesla had as a fledgling car company was cash flow, and the solution to that is scale and sales. FSD can be converted to immed
348.
by Veedrac 5y ago
Musk is an engineer. That is just a fact about him. There are repeated comments to this effect from people who have worked with him, including famously well regarded engineers. If you watch his 3-part Starbase tour with Everyday Astronaut i
349.
by Veedrac 5y ago
Musk has a long history of just saying what he means. He often fails to deliver on things that were too ambitious, or pivots to better ideas in time, but that's entirely different than what you are suggesting.
350.
by Veedrac 5y ago
This is not oligarchy. Using that term here makes no sense.
351.
by Veedrac 5y ago
Not in a relevant sense. When you see cloth, they don't look right or wrong based on whether you ran a detailed mental simulation, but based on how the patterns conform to the object. Plausibility is much more important than precision.
352.
by Veedrac 5y ago
> DALL-E definitely does not encode the mechanics of cloth And nor do you, so how are you so sure at a glance that it's consistent? It's probably only perceptually convincing. Betting that it must be cheating because it's
353.
by Veedrac 5y ago
Now imagine if you can if the situation was reversed, where the AI was adding cyberpunk/oil/etc. to the front of the prompt and it was the human that was interpreting it and painting the many variations. How many people would then
354.
by Veedrac 5y ago
Initial Outputs from New AI Model Not As Good at Nuance as Historic Artwork, Approach Deemed Hopeless
355.
by Veedrac 5y ago
I should have specified ‘decimal floating point’ for clarity, since that is what Python's Decimal is. I agree that non-floating representations often are the solution, depending on what problem you have.
356.
by Veedrac 5y ago
I think most people trying to losslessly represent money or similar discrete quantities already have the intuition that you shouldn't use binary floating point, and those that don't are not well served by telling them to math.iscl
357.
by Veedrac 5y ago
> I agree that, if you actually need exactness and can spare the extra CPU cycles, floats are not your best bet (e.g. banking, accounting). You are misunderstanding me, this is not what I was saying at all. I think much of the time when
358.
by Veedrac 5y ago
This is a completely legitimate concern but it isn't about floating point. You could be getting integer readings from your sensor and you would have exactly the same concern. In such cases you need to provide tolerances in your code th
359.
by Veedrac 5y ago
I agree that you need to be cautious asserting that you know the exact value of a floating point value, because it isn't a property you get by default. Generally, though, if it isn't fairly obvious that equality should be preserve
360.
by Veedrac 5y ago
Either people are doing very strange things with floating point numbers or this is generally pretty poor advice. You should not generally try to ham-fistedly pretend that approximate floating point operations are producing exact results. Yo

More ›