3 ms·
Your points are domain e.g. search specific and the fact that you can get away with delegating the "SM" of the SMP to a cluster in the backend in "Google, and n
by eternalban 15y ago
Your points are domain e.g. search specific and the fact that you can get away with delegating the "SM" of the SMP to a cluster in the backend in "Google, and now Blekko". That's basically an economic model and for your domain, it works. It doesn't apply to everyone. The concern is that for general purpose computing the free lunch is over.
"Web 2.0 is the difference between a SQL server that creaks under the load of a million queries per day and a noSQL cluster that does billions of queries a day."
Right. No one is gonna sue you if your search results are affected by eventual consistency. The world is not just about "queries". Some happen to care about "transactions" at scale ..
I do agree that all the recent tech-pop fretting over this is a bit of a johnny come lately phenomena.
- ChuckMcM 15y agoHmm, I don't see it quite that way. Can you say more about "general purpose computing" ? Let me give a shot at how I think about it and tell me where I jump the boat. Lets say you have an algorithm A which runs in O(n) time. We can define a property 'e' called 'entanglement' as follows: The entanglement of algorithm A is defined to be the requirement of how much of A[n-1] must be computed before you can compute A[n]. An entanglement of 1.0 means that all of A[n-1] must be computed, and an entanglement of 0.0 means that the output of of A[n] is independent of A[n-1]. Algorithms with low entanglement are considered to be 'highly parallelizable' and algorithms with high entanglement are 'sequential'. Amdahl's law shows that the performance improvement of an algorithm is limited by both its level of entanglement and by the cost of handling the partitioning. As a systems architecture, if you can partition the problem into partial computation, you can sidestep Amdahl's law by running multiple copies of the same algorithm with the assumption that each partial computation will come out in one of many possible ways. The simplest example I can contrive of this is binary division. In general, to divide two numbers requires computing the partial remainder of each step of the division until you reach a partial remainder that is between 0 and the divisor. Each step 'n' depends on the step 'n' - 1 to get its results. Lets say you were dividing a 16 bit value by an arbitrary 8 bit value. You can create an alternate form of the problem using a 48 MB memory (16MB x 24 bits). Using the address pins A23-A8 to hold the numerator, and A7 through A0 to hold the denominator, and having the contents of the memory be the 16 bit result, and an 8 bit remainder. The way I think about this solution is that you've created 16 million partial solutions, and the address lines tell you which of those solutions will be the one you are looking for. I expect general purpose computing in a world where there are thousands or millions or billions of cpus available will evolve algorithms like the look up table. However instead of addressing read only memory, the relevant initial conditions will be passed to many partial computation engines, those engines will either respond with a value or not because they compute that their speculative computation would not happen with those initial conditions. And the responding engines may feed an subsequent layer of engines and they will respond or not. You need look no further than the Map/Reduce work, or existing biological systems like immune response to get a grasp on how such a system exploits a sea of resources to surface a solution set of viable outcomes more efficiently. Some of the early work in constraint logic languages points this way as do some hardware description languages. My belief is that having a highly connected sea of general purpose compute engines, and some additional tools to factor algorithms along their entanglement borders into partial computation fragments will radically change the way things get done.
- eternalban 15y agoThank you for your thoughtful reply. Note that you were not accused of jumping the boat; merely that your view is partially biased by subjective economic considerations. Re. Amdahl, various concerns raised by physical distribution present the proverbial hair in the ointment: - CAP applies. If the requirement is for a highly-available and highly-consistent system, then scaling up is preferred as we can do away with the 'P' concerns altogether. - Higher latencies are a given. Realtime distributed e.g. map/reduce algorithms are beyond the reach of most. - Certain e.g. graph constructs are difficult to partition. A single node compute engine that can scale up may turn out to be the more economically attractive option. > My belief is that having a highly connected sea of general purpose compute engines, and some additional tools to factor algorithms along their entanglement borders into partial computation fragments will radically change the way things get done. Right, so we're in agreement here (and I personally love to geek out thinking about that stuff) but with the caveat that such tools do not exist and it is not clear at all that the interface that will be provided to the end-user (read: your average programmer) will remain accessible. 1 - Not everyone can "[think] like a vertex" so Pregel is very nice indeed, but who will be coding for it? Where are these programmers being cranked out and how much will it cost me to hire and retain them? 2 - Not everyone can (afford to) manage a realtime m/r infrastructure (like Facebook). [Oddly enough, there was recently a cry to revolution here by PG regarding the chokehold of "Hollywood" which imho effectively boils down to distribution -- any one can make content these days.] 3 - And a subset of above will not want to trust a 3rd party provider to manage the required infrastructure. 4 - (Subjective) I remain sceptical of the computation efficiencies of the current models which rely on high degree of redundancy and lots and lots of boxes. You must know quite about this (per your HN bio), so I would like hear your thoughts on that. Assume that tomorrow, the cost of running the HVAC and powering up the boxes goes through the roof. What approach to computation will provide the most efficient compute-engine? Let's flip this to physical delivery systems (to highlight the given /fixed cost/ of running a unit-engine) and consider what is more efficient: having a huge number of small cars deliver goods, or using trucks? (You may counter that "well, it is cheaper to train and hire n truck drivers than N mini drivers, but driverless cars will turn that equation upside down". If so, then see 2,3 above.) In sum: Not every company is Google/Facebook/Amazon -- with the attendant wealth of capital and human resources of these giants. Further, not every computation need can be trusted to corporations. Some of us haven't given up on data privacy and insist on it. There are critical "social" applications that need to be written and most certainly Google, Facebook (and may be even) Amazon are not trustworthy enough to host them. Solving this at the hardware/OS level, imho, will be the more democratic way forward and will take it out of the user/devop land (where it is being addressed at the moment). After all, wasn't that what the PC revolution was all about? We did, after all, already have mainframes and dumb terminals way back when. And this back to the future of cloud and running m/r batch jobs is not democratic at all. So I am personally rooting for the (future commodity) H/W solution to this problem. Naturally, my 'biases' are clear per above.