5 ms·
I remember coming across Amdahl's law in the grad school and thinking, "Why is this even a law? Isn't it something completely obvious?" ...later I became a de
by yongjik 6y ago
I remember coming across Amdahl's law in the grad school and thinking, "Why is this even a law? Isn't it something completely obvious?"
...later I became a developer. Turns out it's not obvious.
- Veserv 6y agoI mean, the law part is still completely obvious. If you have a part that scales proportionally with workers (p) and a part that does not scale with workers (1 - p), then only the part that scales proportionally with workers scales with the number of workers (p / s), it is pretty much a tautology. The only reason why it is non-obvious how to take advantage of it in practice is that problems in the real world very rarely break down so cleanly into a completely serial piece and a trivially, arbitrarily parallelizable piece and even if you are lucky enough to find such a problem at a certain scale, the very act of breaking down the problem almost always eventually results in other previously ignorable constant or scaling factors becoming important either as you chop the problem into smaller pieces or as you scale to bigger problems. The difficulty lies entirely in posing the problem to be "solved", not the application of the law itself.
- dwohnitmok 6y agoThere's a lot of things that are obvious when stated, but for various reasons people don't really grasp their ramifications. Put another way, it seems often difficult to grasp just how significant an already superficially internalized concept really is. E.g. pigeonhole principle -> no perfect compression (among other similar proofs) Another a favorite one I have is understanding just how "weird" it is that pi, a constant, exists. Take a rope and tie it around the Earth so that it fits snugly on the Earth's surface. How much more length of rope does it take so that the entire rope circling the whole of the Earth can be lifted 10 feet off its surface? ~31 feet (this was at least surprising to me the first time I was asked this question, which was long after I first learned about pi).
- rileymat2 6y agoIsn’t it double that? 2piR
- dwohnitmok 6y agoHa of course. Looks like I still haven't internalized it.
- thyrsus 6y agoWe're talking about a rope around the equator, right? Circumference = 2piradius is the same as circumference/2 = pi*radius so three more feet of circumference is 1.5 more feet of radius.
- fsckboy 6y agoI think the better way to pose that brain teaser is to say "you make the rope three feet longer, now how high off the ground will the rope be"
- ithinkso 6y agoIt doesn't have anything to do with a pi though. If you had a cube-shaped earth and the same rope, you would need 80ft of additional rope (instead of 62ft for a spherical planet).
- Avshalom 6y ago"law" is the (modernish) scientific term for a model. amdahl's law is a model of how parallelization can('t) speed things up.
- kergonath 6y agoYes. This is a constant issue when non-scientists discuss scientific theories. A law is just a convenient equation that works sometimes, in certain conditions.
- wolfram74 6y agoExcept this is closer to a mathematical theorem than a law like say, boyle's law. You might be able to re-write an algorithm so it has a larger portion that's parallelizable, but that's the only way to get around it. You can't just keep adding parallel computers and enter a qualitatively different regime.
- kergonath 6y agoIt's a model that tells how something will behave under certain conditions. It works if the program being run can be neatly separated into purely serial and purely parallel (infinitely scalable) parts. In reality it breaks down at some point because nothing scales forever and there is some communication overhead. It is similar to the ideal gas law, for example, which can be proved from the axioms of classical thermodynamics and yet fails when the molecules start interacting. Laws span the whole spectrum from purely empirical to provable within some theories' frameworks.
- beaconstudios 6y agoit's a heuristic, or rule of thumb; which is to say, a very loose model useful for estimating but not calculating with precision.