3 ms·
Thank you for your thoughtful reply. Note that you were not accused of jumping the boat; merely that your view is partially biased by subjective economic consid
by eternalban 15y ago
Thank you for your thoughtful reply. Note that you were not accused of jumping the boat; merely that your view is partially biased by subjective economic considerations.
Re. Amdahl, various concerns raised by physical distribution present the proverbial hair in the ointment:
- CAP applies. If the requirement is for a highly-available and highly-consistent system, then scaling up is preferred as we can do away with the 'P' concerns altogether.
- Higher latencies are a given. Realtime distributed e.g. map/reduce algorithms are beyond the reach of most.
- Certain e.g. graph constructs are difficult to partition. A single node compute engine that can scale up may turn out to be the more economically attractive option.
> My belief is that having a highly connected sea of general purpose compute engines, and some additional tools to factor algorithms along their entanglement borders into partial computation fragments will radically change the way things get done.
Right, so we're in agreement here (and I personally love to geek out thinking about that stuff) but with the caveat that such tools do not exist and it is not clear at all that the interface that will be provided to the end-user (read: your average programmer) will remain accessible.
1 - Not everyone can "[think] like a vertex" so Pregel is very nice indeed, but who will be coding for it? Where are these programmers being cranked out and how much will it cost me to hire and retain them?
2 - Not everyone can (afford to) manage a realtime m/r infrastructure (like Facebook). [Oddly enough, there was recently a cry to revolution here by PG regarding the chokehold of "Hollywood" which imho effectively boils down to distribution -- any one can make content these days.]
3 - And a subset of above will not want to trust a 3rd party provider to manage the required infrastructure.
4 - (Subjective) I remain sceptical of the computation efficiencies of the current models which rely on high degree of redundancy and lots and lots of boxes. You must know quite about this (per your HN bio), so I would like hear your thoughts on that. Assume that tomorrow, the cost of running the HVAC and powering up the boxes goes through the roof. What approach to computation will provide the most efficient compute-engine? Let's flip this to physical delivery systems (to highlight the given /fixed cost/ of running a unit-engine) and consider what is more efficient: having a huge number of small cars deliver goods, or using trucks? (You may counter that "well, it is cheaper to train and hire n truck drivers than N mini drivers, but driverless cars will turn that equation upside down". If so, then see 2,3 above.)
In sum:
Not every company is Google/Facebook/Amazon -- with the attendant wealth of capital and human resources of these giants.
Further, not every computation need can be trusted to corporations. Some of us haven't given up on data privacy and insist on it. There are critical "social" applications that need to be written and most certainly Google, Facebook (and may be even) Amazon are not trustworthy enough to host them.
Solving this at the hardware/OS level, imho, will be the more democratic way forward and will take it out of the user/devop land (where it is being addressed at the moment). After all, wasn't that what the PC revolution was all about? We did, after all, already have mainframes and dumb terminals way back when. And this back to the future of cloud and running m/r batch jobs is not democratic at all.
So I am personally rooting for the (future commodity) H/W solution to this problem. Naturally, my 'biases' are clear per above.
- ChuckMcM 15y agoI think we're basically in agreement, the parts where we differ are: "- Certain e.g. graph constructs are difficult to partition. A single node compute engine that can scale up may turn out to be the more economically attractive option." The unstated bias is what I might call 'conservation of compute' which is to say most of the work that has gone into partitioning such problems does so assuming that you want to find the answer in the fewest number of compute cycles because that will lead to the quickest result. One of the things I got to witness at Google was the notion that you could relax the constraint on minimum number of cycles if those cycles could be run in parallel. This is done in a small scale in current micro-architectures where the processor speculatively continues computing past a branch on the 'bet' that the branch won't be taken, only to discard the results of that computation once it is known how the branch will go. This has shown to increase performance even if your branch prediction is only 50/50 because you avoid the pipeline stall when you're right. My assertion is that the number of 'cores' and hence the number of compute engines you can apply to the problem, if you can decompose it into many speculative copies, will let you essentially compute many possible solutions in parallel and select the one you want through a 'fingerprint' of which branches would be or would not be taken. (the path through the graph as it were). And this one: "Not every company is Google/Facebook/Amazon -- with the attendant wealth of capital and human resources of these giants." This is true, but both Google and Amazon have been making their infrastructure available on a pay per use basis. I expect this to continue. Then you will get domain specific portals into that infrastructure in which a middle layer of semantics sits between you and the infrastructure. Sort of like WolframAlpha putting Mathematica between you and WA's server farm, or S3 putting a storage layer between you and Amazons hardware. I too root for some of these problems to be solved at the OS/hardware level but from the perspective of a companies willingness to put their data on another company's gear, at some point if you're being toasted by a competitor who has made that choice then the choice gets effectively made for you. I'd love to write code on the bare metal of an upscale ARM chip with a graphics processor but as anyone who has tried to do this on an OMAP or Broadcom ARM chip can tell you, that isn't going to happen until you design your own ARM chip with GPU and build it yourself. I see this as an example of the choice I would like to make being denied me by forces outside of my control.