3 ms·
Frighteningly Ambitious Startup Ideas (2012)
- sillysaurusx 6y agoAlthough it's cliche to submit old pg essays, "a decade or so" is a nice time to revisit this list. The last big discussion seems to be 3 years ago: https://news.ycombinator.com/item?id=14381699 https://news.ycombinator.com/item?id=14381699 Prior to that was the initial submission, 8 years ago: https://news.ycombinator.com/item?id=3686840 https://news.ycombinator.com/item?id=3686840 The reason I'm submitting it was that #6 turns out to be solved, and it was quite surprising: 6. Bring Back Moore's Law ... It would be great if a startup could give us something of the old Moore's Law back, by writing software that could make a large number of CPUs look to the developer like one very fast CPU. There are several ways to approach this problem. The most ambitious is to try to do it automatically: to write a compiler that will parallelize our code for us. There's a name for this compiler, the sufficiently smart compiler, and it is a byword for impossibility. But is it really impossible? Is there no configuration of the bits in memory of a present day computer that is this compiler? If you really think so, you should try to prove it, because that would be an interesting result. And if it's not impossible but simply very hard, it might be worth trying to write it. The expected value would be high even if the chance of succeeding was low. -- If you read https://cloud.google.com/tpu/docs/spatial-partitioning https://cloud.google.com/tpu/docs/spatial-partitioning you may notice it does exactly that. It's the "sufficiently smart compiler" for ML. You can literally trade TPU cores for memory: twice the cores => you can train 2x larger models. And I was shocked that it seems to be transparent. Other than twiddling a few options to TPUEstimator, I did zero extra work, yet we were able to train a full 1024x1024 StyleGAN2 model. (Previously the maximum was 512x512 before we ran into per-TPU-core memory limits.) That made me curious which other ideas on this list were quietly (or loudly) solved since 2012.