4 ms·
a) It's been widely acknowledged that we are approaching a limit on useful datasets. b) Synthetic data sets have been shown to not be a substitute. c) I have
by threeseed 2y ago
a) It's been widely acknowledged that we are approaching a limit on useful datasets.
b) Synthetic data sets have been shown to not be a substitute.
c) I have no idea why you are linking Moore's Law with AI. Especially when it has never applied to GPUs and we are in a situation where we have a single vendor not subject to normal competition.
- Agentus 2y agoI wonder when people mention Moores law do they use that vernacular literally or figuratively. IE literal as having to do with shrinking of the transistors, figuratively with any and all efforts to increase overall computational speed up.
- dartos 2y agoIn this context it’s the latter, but practically speaking they’re the same thing.
- nl 2y agoSynthetic data absolutely does work well for code. While Moore's Law probably doesn't strictly apply to GPUs, it's not far off. See [1] where they find "We find that FLOP/s per dollar for ML GPUs double every 2.07 years (95% CI: 1.54 to 3.13 years) compared to 2.46 years for all GPUs." (Moore's law predicts doubling every 2 years) https://epochai.org/blog/trends-in-gpu-price-performance#trends-of-gpus-used-in-ml https://epochai.org/blog/trends-in-gpu-price-performance#tre...
- dartos 2y agoIt’d be really nice to see research in this area from somewhere without a financial interest in hyping AI. That incentive doesn’t invalidate research, but AI results are so easy to nudge in any direction that it’s hard to ignore.
- FeepingCreature 2y agob is made up. They have absolutely not been shown to not be a substitute. It's just a big flood of bad research which people treat as summing up to a good argument.