7 ms·

I am an excellent eater of hamburgers, but a horrible eater of cows. If you can build large scale systems, you can code complex algorithms. It's really just a

by edw519 15y ago

I am an excellent eater of hamburgers, but a horrible eater of cows. If you can build large scale systems, you can code complex algorithms. It's really just a matter of breaking down your problem into bite size pieces and confronting them one at a time. These days, I'm happy to get my ground beef from the supermarket and my building blocks from repositories, even though I know that I could move multiple levels back up the chain if I really had to.

joering2 15y ago
> It's really just a matter of breaking down your problem into bite size pieces and confronting them one at a time. my gut tells me I disagree. I think that differentiates junior from senior programmer. Junior will take a big problem, chop into small chunks, make them work properly and kill project while trying to get all those small pieces to work together. Senior will look long enough on a big picture and of course will start from small but will always be aware how all pieces need to fit together. While my statement may sound obvious to everyone and you may say "junior should do the same", in reality your "confronting one piece at the time" is what made me disagree. I do, however, believe you can tap into a middle-size problem without knowing how to code complex algorithms, especially in a startup world. Look at Facebook. Zuck never been the smartest programmer, but I dont think he took upon huge complex algorithms to get into the MVP stage just to prove user's numbers to investors and get money to hire programmers that could rewrite it from scratch to serve it to tens of millions of users flexibly.
nadam 15y ago
"If you can build large scale systems, you can code complex algorithms." Not sure I agree. Architecting/Maintaining huge systems and inventing very smart algorithms are very different skills. Both can be learned to some extent, and a programmer should learn both to some level, but it is totally possible that one person is very stong in one but not that strong in the other. Look at the source of the splay tree: http://www.link.cs.cmu.edu/link/ftp-site/splaying/ http://www.link.cs.cmu.edu/link/ftp-site/splaying/ It is short, a programmer usually writes this amount of code in a day. Coming up with this made its creator famous. Algorithmization is like short distance running. You prepare a lot (think a lot) but probably write little code, and you have to be quite math-focused. Building huge systems is long distance running: you have to be strategical, you write lots of code, (and put together lots of third party components), you have to be 'wise'... In essence good progammers are usually good enough in both (I am sure this is true for the poster), but he/she can be particularly strong in one or the other.
- jchonphoenix 15y ago
  You're actually a little mislead. Proving that splay trees don't suck and inventing amortized analysis made Danny Sleator famous. Coming up with and coding it took him less than an hour (from his own mouth which might be embellished). If you look at the full proof of its log(n) running time, you'll realize that splay trees are incredibly complex. Sleator is also an amazing problem solver. He can outcome most of the people commenting on this thread but isn't necessarily the best architect. I believe algo is about problem solving. If you don't understand it, you don't know when to use which tool. Its unecessary if you're writing boilerplate and solved problems with frameworks but when you get to the google/amazon level, you can't just write good code. You need to understand algo to create dynamo and write dremel and mapreduce.
  - nadam 15y ago
    I agree with everything you say I just don't understand why you say that I am mislead. Of course he could code it in one hour. Probably he came up with it in a bright moment. But a researcher has to think for years to come up with something so successful, and most of their ideas suck. "Sleator is also an amazing problem solver. He can outcome most of the people commenting on this thread but isn't necessarily the best architect." Which is what I think exactly. "I believe algo is about problem solving." Not just problem solving, but a kind of problem solving. algo is some kind of mathematical thinking. To be a extremely good at algorithmization you have to be extremely good in a kind of mathematical thinking. Of course those people who are extremely good at it like Sleator (or Von Neumann etc...) are extremely smart, so they are probably very good at other things also, but algorithmization is not a general purpose intelligence, it is a kind of 'mathematical intelligence'.
    - haberman 15y ago
      Friendly correction for both of you; it's "misled" not "mislead" (the latter is a verb that rhymes with "misdeed", not an adjective).
- arethuza 15y ago
  Speaking of very different skills, I used to have a developer working for me who was an absolute genius at fixing bugs - he would dive into complex code he had never seen before and be able to pinpoint and fix problems that had eluded other people. However, when confronted with a "clean sheet of paper" for the design of a new module he literally couldn't do it - even with a lot of coaching.
gm 15y ago
I'd phrase it slightly differently: "If you can build large scale systems and your systems are still running 6 months after release without large modifications, then that means you can simplify complex problems into several major simple problems."
k_bx 15y ago
I disagree at some point. If person doesn't at least knows difference between hash-table and B+tree -- then he cannot build large scale systems. The same for other algorithms. Everything is based on algorithms, and without their understanding people often will build scalable applications that have really horrible performance, so they will consume lots of hardware to do simple things with wrong algorithms. And that's horrible.
- overshard 15y ago
  The thing is hash-tables and B+trees are data structures and not algorithms. I agree that if you don't know basic differences of algorithms you will run into performance issues but you will also run into problems if you don't know the difference between a data structure and an algorithm.
  - k_bx 15y ago
    Data structure is "a particular way of storing and organizing data", so I think there's not so big distinction between two. More of that, when we think of data structures, we think of insertion/deletion/searching complexity, also we think about space needed (which is also a result of organizing algorithm). So I don't think that someone can run into troubles because of that difference.
- reinhardt 15y ago
  Memorizing some basic properties of a data structure or algorithm and using them as black boxes to design large scale systems is different from actually implementing said algorithms/data structures, let alone devising them in the first place. That's what the OP's blog is about.
- jaylevitt 15y ago
  > then he cannot build large scale systems If that's true, it's new. I don't know the difference between a B+ tree and any other type of tree, and I built the world's then-largest email system in the '90s, eventually pushing 4,000 TPS through servers less powerful than an iPhone. Some problems just don't involve algorithms at a visible level. We had one filesystem with one type of key-value indexing, and whatever that was, that's what we used. Both disk I/O and RAM were too constrained to worry about O(n); n was < 500, data structures couldn't even exceed 32k, and even linked lists were too slow - we used arrays. The only algorithm we (consciously) used was a hashing function for our cache keys and sharding, and we were far more concerned about its uniform distribution than its performance. That class of problem is less prevalent today, but it's at least plausible that there's somebody architecting large-scale systems where the only algorithm is "do very few things very quickly".
Duff 15y ago
I don't think your analogy works... there are different skill-sets. Maybe "I'm an excellent eater of hamburgers, but I'm horrible at forming patties and cooking." Building large systems means that you're a builder -- you can take raw materials, assemble them in a coherent way, then troubleshoot/operate the thing you built. You write scripts, modify code to integrate with other components, and fix bugs. You don't need to know anything about sorting algorithms, although the thing you build sorts all sorts of data. More knowledgeable programmers actually build the raw materials, or have an ability to dig deep into the code to fix fundamental issues. They may or may not have no knowledge of the overall system. A typical enterprise handles deep code issues by opening a ticket with their vendor. I'm not making a "dig" at the "builder" programmers -- they are as essential as the deep-understanding programmers. The guy at Oracle who is debugging the code in the database that does sorts probably has deep mathematical knowledge... but he knows nothing about integrating ERP system components.
angersock 15y ago
Further along these lines: http://the-witness.net/news/2011/06/how-to-program-independent-games/ http://the-witness.net/news/2011/06/how-to-program-independe... (around 10-12 minutes in starts talking about algorithms in production) Johnathon Blow (semi-famous indie dev) talking about when data-structures and algorithms really matter. I tend to agree with this--a bad (suboptimal but functioning) algorithm now is better than a perfect algorithm later, and when you need to ship you should write the code that gets things done.
- rhizome 15y ago
  There are always lots of entry-level devs who can apply their fresh CS degrees to those refinements.
j45 15y ago
You're right. Some folk are asking how are the details are connected to the large scale, high level thinking? I explain it to my customers like this with a quote I enjoy. "How do drops of water know themselves to be a river?" The details, algorithms, data are all the drops of water. The river is the large scale system. Having an understanding of how a river (or large scale system) works gives you insights how each smaller and smaller piece should work together to fit into the overall picture. Drill down far enough, and you are at the individual processes. This is what I call understanding the system from bits and bytes, through to nuts and bolts, all the way up to the top of the pyramid where you have the high level system strategy. If you can get the big picture, you can begin to learn and understand how it's built. Having a lot of tiny algorithm skills does not mean, though, that you can piece together how they can work together to make a big picture. It's my humble opinion this is how software gets off track and spirals out of control. Too much domain knowledge about tiny things without as much focus on the big picture to balance it out leads to premature optimization, over engineering and mis-assumptions, trivializations and all the other things we've slapped our wrists for.
djacobs 15y ago
I disagree. Coding an algorithm is completely different from using it as a black box to design and implement a system. Algorithms are fraught with conditional logic, where well-implemented systems tend to quarantine conditionals to make edge cases as few as possible. An algorist prides herself on ability to optimize the problem space, where a systems coder prides himself on the ability to keep systems bug-free and free of accidental complexity. There are many coders who do both, but I don't think the set of algorists is identical to the set of systems coders.
ramblerman 15y ago
this is just hyperbole and a bad metaphor.

by edw519 15y ago

joering2 15y ago

> It's really just a matter of breaking down your problem into bite size pieces and confronting them one at a time. my gut tells me I disagree. I think that differentiates junior from senior programmer. Junior will take a big problem, chop into small chunks, make them work properly and kill project while trying to get all those small pieces to work together. Senior will look long enough on a big picture and of course will start from small but will always be aware how all pieces need to fit together. While my statement may sound obvious to everyone and you may say "junior should do the same", in reality your "confronting one piece at the time" is what made me disagree. I do, however, believe you can tap into a middle-size problem without knowing how to code complex algorithms, especially in a startup world. Look at Facebook. Zuck never been the smartest programmer, but I dont think he took upon huge complex algorithms to get into the MVP stage just to prove user's numbers to investors and get money to hire programmers that could rewrite it from scratch to serve it to tens of millions of users flexibly.

nadam 15y ago

"If you can build large scale systems, you can code complex algorithms." Not sure I agree. Architecting/Maintaining huge systems and inventing very smart algorithms are very different skills. Both can be learned to some extent, and a programmer should learn both to some level, but it is totally possible that one person is very stong in one but not that strong in the other. Look at the source of the splay tree: http://www.link.cs.cmu.edu/link/ftp-site/splaying/ http://www.link.cs.cmu.edu/link/ftp-site/splaying/ It is short, a programmer usually writes this amount of code in a day. Coming up with this made its creator famous. Algorithmization is like short distance running. You prepare a lot (think a lot) but probably write little code, and you have to be quite math-focused. Building huge systems is long distance running: you have to be strategical, you write lots of code, (and put together lots of third party components), you have to be 'wise'... In essence good progammers are usually good enough in both (I am sure this is true for the poster), but he/she can be particularly strong in one or the other.

jchonphoenix 15y ago
You're actually a little mislead. Proving that splay trees don't suck and inventing amortized analysis made Danny Sleator famous. Coming up with and coding it took him less than an hour (from his own mouth which might be embellished). If you look at the full proof of its log(n) running time, you'll realize that splay trees are incredibly complex. Sleator is also an amazing problem solver. He can outcome most of the people commenting on this thread but isn't necessarily the best architect. I believe algo is about problem solving. If you don't understand it, you don't know when to use which tool. Its unecessary if you're writing boilerplate and solved problems with frameworks but when you get to the google/amazon level, you can't just write good code. You need to understand algo to create dynamo and write dremel and mapreduce.
- nadam 15y ago
  I agree with everything you say I just don't understand why you say that I am mislead. Of course he could code it in one hour. Probably he came up with it in a bright moment. But a researcher has to think for years to come up with something so successful, and most of their ideas suck. "Sleator is also an amazing problem solver. He can outcome most of the people commenting on this thread but isn't necessarily the best architect." Which is what I think exactly. "I believe algo is about problem solving." Not just problem solving, but a kind of problem solving. algo is some kind of mathematical thinking. To be a extremely good at algorithmization you have to be extremely good in a kind of mathematical thinking. Of course those people who are extremely good at it like Sleator (or Von Neumann etc...) are extremely smart, so they are probably very good at other things also, but algorithmization is not a general purpose intelligence, it is a kind of 'mathematical intelligence'.
  - haberman 15y ago
    Friendly correction for both of you; it's "misled" not "mislead" (the latter is a verb that rhymes with "misdeed", not an adjective).
arethuza 15y ago
Speaking of very different skills, I used to have a developer working for me who was an absolute genius at fixing bugs - he would dive into complex code he had never seen before and be able to pinpoint and fix problems that had eluded other people. However, when confronted with a "clean sheet of paper" for the design of a new module he literally couldn't do it - even with a lot of coaching.

gm 15y ago

I'd phrase it slightly differently: "If you can build large scale systems and your systems are still running 6 months after release without large modifications, then that means you can simplify complex problems into several major simple problems."

k_bx 15y ago

I disagree at some point. If person doesn't at least knows difference between hash-table and B+tree -- then he cannot build large scale systems. The same for other algorithms. Everything is based on algorithms, and without their understanding people often will build scalable applications that have really horrible performance, so they will consume lots of hardware to do simple things with wrong algorithms. And that's horrible.

overshard 15y ago
The thing is hash-tables and B+trees are data structures and not algorithms. I agree that if you don't know basic differences of algorithms you will run into performance issues but you will also run into problems if you don't know the difference between a data structure and an algorithm.
- k_bx 15y ago
  Data structure is "a particular way of storing and organizing data", so I think there's not so big distinction between two. More of that, when we think of data structures, we think of insertion/deletion/searching complexity, also we think about space needed (which is also a result of organizing algorithm). So I don't think that someone can run into troubles because of that difference.
reinhardt 15y ago
Memorizing some basic properties of a data structure or algorithm and using them as black boxes to design large scale systems is different from actually implementing said algorithms/data structures, let alone devising them in the first place. That's what the OP's blog is about.
jaylevitt 15y ago
> then he cannot build large scale systems If that's true, it's new. I don't know the difference between a B+ tree and any other type of tree, and I built the world's then-largest email system in the '90s, eventually pushing 4,000 TPS through servers less powerful than an iPhone. Some problems just don't involve algorithms at a visible level. We had one filesystem with one type of key-value indexing, and whatever that was, that's what we used. Both disk I/O and RAM were too constrained to worry about O(n); n was < 500, data structures couldn't even exceed 32k, and even linked lists were too slow - we used arrays. The only algorithm we (consciously) used was a hashing function for our cache keys and sharding, and we were far more concerned about its uniform distribution than its performance. That class of problem is less prevalent today, but it's at least plausible that there's somebody architecting large-scale systems where the only algorithm is "do very few things very quickly".

Duff 15y ago

I don't think your analogy works... there are different skill-sets. Maybe "I'm an excellent eater of hamburgers, but I'm horrible at forming patties and cooking." Building large systems means that you're a builder -- you can take raw materials, assemble them in a coherent way, then troubleshoot/operate the thing you built. You write scripts, modify code to integrate with other components, and fix bugs. You don't need to know anything about sorting algorithms, although the thing you build sorts all sorts of data. More knowledgeable programmers actually build the raw materials, or have an ability to dig deep into the code to fix fundamental issues. They may or may not have no knowledge of the overall system. A typical enterprise handles deep code issues by opening a ticket with their vendor. I'm not making a "dig" at the "builder" programmers -- they are as essential as the deep-understanding programmers. The guy at Oracle who is debugging the code in the database that does sorts probably has deep mathematical knowledge... but he knows nothing about integrating ERP system components.

angersock 15y ago

Further along these lines: http://the-witness.net/news/2011/06/how-to-program-independent-games/ http://the-witness.net/news/2011/06/how-to-program-independe... (around 10-12 minutes in starts talking about algorithms in production) Johnathon Blow (semi-famous indie dev) talking about when data-structures and algorithms really matter. I tend to agree with this--a bad (suboptimal but functioning) algorithm now is better than a perfect algorithm later, and when you need to ship you should write the code that gets things done.

rhizome 15y ago
There are always lots of entry-level devs who can apply their fresh CS degrees to those refinements.

j45 15y ago

You're right. Some folk are asking how are the details are connected to the large scale, high level thinking? I explain it to my customers like this with a quote I enjoy. "How do drops of water know themselves to be a river?" The details, algorithms, data are all the drops of water. The river is the large scale system. Having an understanding of how a river (or large scale system) works gives you insights how each smaller and smaller piece should work together to fit into the overall picture. Drill down far enough, and you are at the individual processes. This is what I call understanding the system from bits and bytes, through to nuts and bolts, all the way up to the top of the pyramid where you have the high level system strategy. If you can get the big picture, you can begin to learn and understand how it's built. Having a lot of tiny algorithm skills does not mean, though, that you can piece together how they can work together to make a big picture. It's my humble opinion this is how software gets off track and spirals out of control. Too much domain knowledge about tiny things without as much focus on the big picture to balance it out leads to premature optimization, over engineering and mis-assumptions, trivializations and all the other things we've slapped our wrists for.

djacobs 15y ago

I disagree. Coding an algorithm is completely different from using it as a black box to design and implement a system. Algorithms are fraught with conditional logic, where well-implemented systems tend to quarantine conditionals to make edge cases as few as possible. An algorist prides herself on ability to optimize the problem space, where a systems coder prides himself on the ability to keep systems bug-free and free of accidental complexity. There are many coders who do both, but I don't think the set of algorists is identical to the set of systems coders.

ramblerman 15y ago

this is just hyperbole and a bad metaphor.