7 ms·

Great quote at 45:26 regarding vertical vs. horizontal scaling: https://youtu.be/si9iqF5uTFk?t=2726 https://youtu.be/si9iqF5uTFk?t=2726 > "Now back in the earl

• oasisbob 2y ago
  Additionally, with early pioneer logging, another solution to avoiding having logs which are too large to handle was to not drop them in the first place. In the Pacific Northwest, US, early loggers would leave the huge ones - to the point where pioneers could complain about a lack of available timber in an old-growth forest. When the initial University of Washington was built, land-clearing costs were a huge portion of the overall capital spend. The largest trees on the site weren't used for anything productive; rather, they were climbed, chained together, and domino felled at the same time. By attaching the trees together, they only needed to fell one tree which brought the whole mess down into a pile and they burned it. I think there's a lesson here about choosing which logs you want to move.
  • pests 2y ago
    Having not read the article yet, this was one confusing comment until I realized by "logging" you meant actual trees and not log files.
    • dredmorbius 2y ago
      The "to record an event" meaning of logging does in fact originate from wooden logs, which were used to calculate the speed of a ship under sail. A log, tied to a fathom-line, was cast off the stern, and the number of knots which passed through the sailors hands in a measured time interval determined the speed. This was recorded as "log", in what came to be known as the "ship's log". The term came to be used for event recording in general. This was used as a component of "dead reckoning" (that is, deduced reckoning of position) in navigation, prior to the development of accurate time-keeping and direct position reporting through LORAN, radar-navigation, and ultimately GPS. Dead reckoning was not especially accurate and had some rather notorious failure modes. <https://www.etymonline.com/word/log https://www.etymonline.com/word/log> (definition 2) Chip log: <https://en.wikipedia.org/wiki/Chip_log https://en.wikipedia.org/wiki/Chip_log> Dead reckoning: <https://en.wikipedia.org/wiki/Dead_reckoning https://en.wikipedia.org/wiki/Dead_reckoning> Scilly naval disaster of 1707, in which four warships and 1,400 to 2,000 sailors of the British fleet were lost in a single navigational failure: <https://en.wikipedia.org/wiki/Scilly_naval_disaster_of_1707 https://en.wikipedia.org/wiki/Scilly_naval_disaster_of_1707> This directly inspired the Longitude Rewards prizes for developing accurate navigational methods (<https://en.wikipedia.org/wiki/Longitude_rewards https://en.wikipedia.org/wiki/Longitude_rewards>) and John Harrison's invention of a marine clock (<https://en.wikipedia.org/wiki/John_Harrison#Longitude_problem https://en.wikipedia.org/wiki/John_Harrison#Longitude_proble...>). More recently, the Honda Point disaster in which the US Navy, 1923 saw the loss of 7 destroyers at flank speed of 20 knots off the Santa Barbara coast: <https://en.wikipedia.org/wiki/Honda_Point_disaster https://en.wikipedia.org/wiki/Honda_Point_disaster>. It's interesting to note that advances in timekeeping typically translate to improvements in location determination.
  • rootsudo 2y ago
    That's an interesting tidbit I did not know about UW. Sad the wood wasn't used.
  • brutal_chaos_ 2y ago
    Perhaps an analogy can be made with respect to information priority and which trees to fell.
• interroboink 2y ago
  > they did not try to grown a bigger ox Actually, humans have been doing exactly that through breeding over the millennia. They were just limited in their means. This analogy has some "you wouldn't download a car!" vibes — sure I would, if it were practical (: And vertical scaling of computers is practical (up to some limits).
  • nequo 2y ago
    This is irrelevant to the example cited by Hopper. If you have a large log, you don't have time to breed a larger ox. You need to solve the problem with the oxen you have.
    • interroboink 2y ago
      I'm all in favor of making the best of what's available. But at the same time, if such thinking is taken as dogma, innovation suffers. You spoke of one log, and the time scales involved. But suppose you have an entire forest of logs. Then it may indeed be worth breeding bigger oxen (or rather, inventing tractors). I don't mean to accuse Hopper of shortsightedness, but when quotes by famous people, like the above, are thrown around without context, they encourage that dogmatic thinking. So, I was more replying to that quote as it appeared here, rather than as it appeared in her talk.
      • gffrd 2y ago
        > if such thinking is taken as dogma, innovation suffers. I don’t think there’s anything about the original post, with quote about oxen, that reads as dogmatic, or invites such perspective. Also, I think we can all agree most innovation happens as an extension of “making the best of what’s available” rather than independent of it, on a fully separate track. Using two oxen can lead to realizing a bigger ox would be beneficial.
        • interroboink 2y ago
          I don't mean to wear out this thread, and I totally respect your different viewpoint, but when I see: ... they're trying to tell us something. When we need greater computer power, the answer is not "get a bigger computer", it's "get another computer". that does read as dogmatic advice to me, taken in isolation. It boils down to "the answer is X." Not "consider these factors" or "weigh these different options," but just "this is the answer, full stop." That's dogma, no? (that aside, I do slightly regret the snarkiness of my initial comment :)
• bunderbunder 2y ago
  That "they didn't have any big cranes" forces the analogy in a way that breaks it. The solution wherever cranes are used is absolutely to get a bigger crane. And also, oxen were absolutely bred to be bigger. That's kind of the defining thing that distinguishes draft oxen from other kinds of cattle. But that process was limited by some factors that are peculiar to domesticated animals. And, of course, if you need to solve the problem right now, you make do with the current state of the art in farm animal technology. Admiral Hopper's lecture wasn't delivered too long after 1976, which saw the release of both the CRAY-1 (single CPU) and the ILLIAC IV (parallel). ILLIAC IV, being more expensive, harder to use, and slower than the CRAY-1, was a promising hint at future possibility, but not particularly successful. Cray's quip on this subject was (paraphrasing) that he'd rather plow a field with one strong ox than $bignum chickens. Admiral Hopper was presumably responding to that. What they both seem to miss is that the best tool for the job depends on both the job and the available tools. And they both seem to be completely missing that, if you know what you're doing, scale up and scale out are complementary: first you scale up the individual nodes as much as is practical, and then you start to scale out once scale up loses steam.
  • hbosch 2y ago
    Yeah, but trying to create a perfect analogy is like trying to improve the poundcake, just isn't worth it.
    • dylan604 2y ago
      If you served me two pound cakes at the same time, I would say things were improved. However, it wouldn't be very efficient as I would still only eat them serially instead of in parallel
      • interroboink 2y ago
        Now I'm considering whether I'd rather have two pound cakes, or one big one... One normal-size, versus 10 miniature ones? Needs research (:
• Avamander 2y ago
  Both parts of the lecture are in general really good. Though I have to say the part about the cost of not implementing standards, the cost of not doing something, felt scarily relevant right now.
• agapon 2y ago
  Well, if growing or breeding bigger oxen were as feasible as building bigger (mower powerful) computers was/is, perhaps people would take a different path? In other words, perhaps the analogy is flawed?
  • dpcx 2y ago
    At some point, a bigger computer either doesn't exist or is too cost prohibitive to get. But getting lots of "small" computers is somewhat easier.
    • bunderbunder 2y ago
      Sure, but, as I was (rather unpopularly) pointing out in another comment, that point was pretty hard to reach in 1982. Specifically the point where you've met both criteria: bigger computer is too cost prohibitive to get, and lots of smaller computers is easier. At the time of this lecture, parallel computers had a nasty tendency to achieve poorer real-world performance on practical applications than their sequential contemporaries, despite greater theoretical performance. It's still kind of hard even now. To date in my career I've had more successes with improving existing systems' throughput by removing parallelism than I have by adding it. Amdahl's Law plus the memory hierarchy is one heck of a one-two punch.
      • PaulHoule 2y ago
        In 1982 you still had "supercomputers" like https://en.wikipedia.org/wiki/Cray_X-MP https://en.wikipedia.org/wiki/Cray_X-MP because you could still make bipolar electronics that beat out mass-produced consumer electronics. By the mid 1990s even IBM abandoned bipolar mainframes and had to introduce parallelism so a cluster of (still slower) CMOS mainframes could replace a bipolar mainframe. This great book was written by someone who worked on this project https://campi.cab.cnea.gov.ar/tocs/17291.pdf https://campi.cab.cnea.gov.ar/tocs/17291.pdf and of course for large scale scientific computing it was clear that "clusters of rather ordinary nodes" like the https://www.cscamm.umd.edu/facilities/computing/sp2/index.htm https://www.cscamm.umd.edu/facilities/computing/sp2/index.ht... we had at Cornell were going to win (ours was way bigger) because they were scalable. (e.g. the way Cray himself saw it, a conventional supercomputer had to live within a small enough space that the cycle time was not unduly limited by the speed of light so that kind of supercomputer had to become physically smaller, not larger, to get faster) Now for very specialized tasks like codebreaking, ASICs are a good answer and you'd probably stuff a large number of them into expansion cards into rather ordinary computers and clusters today possibly also have some ASICs for glue and communications such as https://blogs.nvidia.com/blog/whats-a-dpu-data-processing-unit/ https://blogs.nvidia.com/blog/whats-a-dpu-data-processing-un... ---- The problem I see with people who attempt parallelism for the first time is that the task size has to be smaller than the overhead to transfer tasks between cores or nodes. That is, if you are processing most CSV files you can't round-robin assign rows to threads but 10,000 row chunks are probably fine. You usually get good results over a large range of chunk size but chunking is essential if you want most parallel jobs to really get a speedup. I find it frustrating as hell to see so many blog posts pushing the idea that some programming scheme like Actors is going to solve your problems and meeting people that treat chunking as a mere optimization you'll apply after the fact. My inclination is you can get the project done faster (human time) if you build in chunking right away but I've learned you just have to let people learn that lesson for themselves.
• Animats 2y ago
  This is a reaction to Grosch's Law, "Computing power increases as the square of the price".[1] In the early 1980s, people still believed that. Seymour Cray did. John McCarthy did when I was at Stanford around then. It didn't last into the era of microprocessors. Amusingly, in the horse-powered era, once railroads started working, but trucks didn't work yet, there was a "last mile" problem - getting stuff from the railroad station or dock to the final destination. The 19th century solution was to develop a bigger breed of horse - the Shire Horse.[1] [1] https://en.wikipedia.org/wiki/Shire_horse https://en.wikipedia.org/wiki/Shire_horse
  • SECProto 2y ago
    The article you linked doesn't support your anecdote about the root of the Shire Horse. It describes their history dating back centuries before railways. Their biggest use seems to have been hauling material to and from ports, not trains.
    • TomatoCo 2y ago
      Well, would the anecdote work if you bumped it back a few centuries and swapped trains for ships?
    • Animats 2y ago
      Sure it does. Shires go back a ways, but they were not bred in quantity until the 1850s or so. "In the late nineteenth and early twentieth centuries, there were large numbers of Shires, and many were exported to the United States." Before and after that period, those big guys were an exotic breed. That's the railroad but pre-truck period. (I've owned a Percheron, and have known some Shires.)
      • SECProto 2y ago
        When you cite a source for something, the source should justify the thing you're claiming. The relevant part of your post to the thread was that "The 19th century solution was to develop a bigger breed of horse." It is entirely contrary to what the Wikipedia article says - "The breed was established in the mid-eighteenth century, although its origins are much older". (relatives have owned Friesian's and Clydesdale's and Norwegian Fjording Horses, but it's neither here nor there)
• inopinatus 2y ago
  There’s an inflection point, however. Hence Seymour Cray’s famous quip, “If you were plowing a field, which would you rather use? Two strong oxen or 1024 chickens?”
• jimmySixDOF 2y ago
  My favorite quote/story was "Never Never Never take the First No!" (16:20) because some people are just obstructionists and others just want to see how serious you are. As she says, appreciation for how to do this comes with age, but to me a good definition of management in general is learning if, when, how, and how much to pushback against pushback.
• hello_computer 2y ago
  These days, “get another computer” and “get a bigger computer” are basically the same thing; differences primarily residing in packaging and interconnects, but boy howdy can those interconnects make a difference.