5 ms·

This doesn't even get into the fact that exceedingly few programming jobs need to know anything resembling advanced computer science theory. Probably less than

by phil21 10y ago

This doesn't even get into the fact that exceedingly few programming jobs need to know anything resembling advanced computer science theory. Probably less than 5% in my limited experience.

frozenport 10y ago
Further those 5% move at a glacial pace, you could be given a week to write or improve a BST.
yazaddaruvala 10y ago
I do believe we need a better system for interviewing candidates. However, I strongly disagree with your statement! I work on distributed systems day in and day out, and more and more I find that I'm using or building distributed data structures analogous to BST, HashTable, etc. Not knowing the foundations of my field in and out would be a mistake!
- johnnyb9 10y ago
  How many programmers work for Facebook, Google, Amazon, etc and how many work for non-web scale companies? As a professional programmer in financial services I have never had to use a red-black tree, min heap or do anything more advanced than properly using a HashTable.
  - yazaddaruvala 10y ago
    Ok, I'll try and use an example from financial services: Disclaimer: Never worked in financial tech, I have thought about this problem for 5 mins. This is unlikely to be an optimal solution to keeping a sorted set of distributed data. I'm just trying to show how basic knowledge of data structures helps in the field of distributed systems. Lets say you need to keep a sorted set of sooooo much data that you cannot keep it all on one machine. I would imagine all financial services firms have some custom datastore which has "potential buy orders" and need to keep them all sorted by profitability to efficiently search/insert/remove from that datastore. In such a situation, my instincts would be to create a distributed, redundant Heap. Now, how do you build a distributed, redundant Heap, without understanding how a Heap works? How do you even know what to look for if you don't know what a Heap is? Now, lets say you've built it such that each node in your distributed network acted like a node in a Heap, and the left "pointer"(i.e. url to other computer) meant "less than" and the right "pointer" meant "greater than". Now you have this data store in production and all of a sudden you realize performance worsens over time... "WHY is this happening? Is there some memory leak?". You investigate and realize that all of your data isn't being distributed evenly! "Do I need to like.. shuffle the data around? How do I get it so that the data is evenly distributes?" At this point if you do not know what a red-black tree is, what do you even google look for? Lets say you dig around for a while and find out about re-balancing trees. Now you have to implement it, and eventually you figure out that one of the best ways is to color your nodes. When a future colleague/manager asks, "What is this system, please describe it to me?" wouldn't it be nice to just say "Oh visualize it kinda like a distributed red-black tree. It holds XXX data, and ensures that it is always sorted and re-balanced for optimal performance."
    - nerfhammer 10y ago
      FWIW finance normally uses big relational databases (because you absolutely need strong transaction support), something else off-the-shelf, or Microsoft Excel.
    - fragmede 10y ago
      Of course there's some sort of B-tree somewhere in the stack; that's not the question. With dozens of database stacks to choose from, many companies won't have anybody actively and regularly working in the database's code, nor in the OS kernel code either. If, then, an individual can successfully contribute to the company without ever hearing of a red-black tree, what's the value in testing for it, past "this is how we've always done it"? (How long it would take someone who doesn't know CS lingo to string together "tree" and "balance" and plug that into Google, I don't know, but I suspect it's not impossible to find.)
      - yazaddaruvala 10y ago
        > Of course there's some sort of B-tree somewhere in the stack; that's not the question. You've misunderstood me. I'm literally talking about building a distributed B-Tree or Heap. As in, a Heap which cannot fit in memory or storage of any one node. Think: S3/DynamoDB is a distributed, highly available HashTable; _____ is a distributed, highly available Heap.
  - PlaceFan 10y ago
    As a counterpoint, I work in high finance and one of our core systems that I'm expected to know extremely well uses Paxos for consensus. Just because those topics show up infrequently doesn't mean that no one's using them. Plenty of companies outside of Google, etc. are doing complicated things that require a solid CS background.
    - FiveDegrees 10y ago
      You should have programmers implement paxos on a whiteboard in their interviews.
    - johnnyb9 10y ago
      My point was not that they do or do not exist, merely the magnitude. I would venture a guess that for every software engineer working on a blockchain there are 100 working on crud-style applications. If a job on your team requires a solid understanding of these concepts, then by all means filter on knowledge of algorithms. My point is that many companies don't do anything remotely as complex and yet use the same filters to screen job applicants.
dagw 10y ago
exceedingly few programming jobs need to know anything resembling advanced computer science theory There's "need" and there's "need". The person without the solid computer science background may very well be able to solve all the problems at hand, but the person with the solid CS background is far more likely to come up with the fast elegant solution that doesn't fall over in obscure corner cases. To use a concrete example for early in my career; I spent days and days trying to solve a problem by building a bigger and bigger pile regexps and if-else statements. Then my project manager (who had a PhD in CS) came along a just wrote a custom parser that solved the whole thing.
- arvinsim 10y ago
  You don't need to know how to implement a parser to recognize it as a solution to a problem. And there are already libraries and tools for building parsers anyway.
  - sanderjd 10y ago
    The hard part isn't building the parser, the hard part is recognizing that a parser would be a good solution to the problem and distilling the problem into a form for which the solution works. One of the main purposes of a college degree is to come out knowing a lot about what you don't know. That is, knowing a lot about what sorts of things exist, even if you don't know much detail about any of those things.
- majormajor 10y ago
  > the fast elegant solution that doesn't fall over in obscure corner cases This can go both ways, though. Sometimes the fast elegant solution is too elegant for the nasty corner cases and a much bigger, uglier, but ultimately straightforward procedural block of crap is better. The examples I've seen are from established businesses that occasionally slipped up in refactoring out hacks that were introduced to support deadline-driven requirements. A year or two later, that hack is powering the reporting for a huge portion of the traffic, and you have to be able to balance between (a) writing maintainable code that supports the hack for the short term and (b) working with the rest of the business to clean up the requirements so you can improve the system for the long term. (b) is a skill that most interview processes I've seen completely ignore. And (a) often is downplayed (as "simple" or "easy") compared to more clever tricks - but knowing how to do the clever tricks might turn into a temptation to use them more than you should.
bpodgursky 10y ago
Having basic algorithmic competence (and yes, BSTs + BFS/DFS are basic) is like knowing how to swim. Maybe you aren't near water or writing performance-sensitive code 95% of the time, but the 5% of the time you're in over your head, being able to implement (or identify when to use) basic algorithms will save you a whole lot of time, money, and headache.

by phil21 10y ago

This doesn't even get into the fact that exceedingly few programming jobs need to know anything resembling advanced computer science theory. Probably less than 5% in my limited experience.

frozenport 10y ago

Further those 5% move at a glacial pace, you could be given a week to write or improve a BST.

yazaddaruvala 10y ago

I do believe we need a better system for interviewing candidates. However, I strongly disagree with your statement! I work on distributed systems day in and day out, and more and more I find that I'm using or building distributed data structures analogous to BST, HashTable, etc. Not knowing the foundations of my field in and out would be a mistake!

johnnyb9 10y ago
How many programmers work for Facebook, Google, Amazon, etc and how many work for non-web scale companies? As a professional programmer in financial services I have never had to use a red-black tree, min heap or do anything more advanced than properly using a HashTable.
- yazaddaruvala 10y ago
  Ok, I'll try and use an example from financial services: Disclaimer: Never worked in financial tech, I have thought about this problem for 5 mins. This is unlikely to be an optimal solution to keeping a sorted set of distributed data. I'm just trying to show how basic knowledge of data structures helps in the field of distributed systems. Lets say you need to keep a sorted set of sooooo much data that you cannot keep it all on one machine. I would imagine all financial services firms have some custom datastore which has "potential buy orders" and need to keep them all sorted by profitability to efficiently search/insert/remove from that datastore. In such a situation, my instincts would be to create a distributed, redundant Heap. Now, how do you build a distributed, redundant Heap, without understanding how a Heap works? How do you even know what to look for if you don't know what a Heap is? Now, lets say you've built it such that each node in your distributed network acted like a node in a Heap, and the left "pointer"(i.e. url to other computer) meant "less than" and the right "pointer" meant "greater than". Now you have this data store in production and all of a sudden you realize performance worsens over time... "WHY is this happening? Is there some memory leak?". You investigate and realize that all of your data isn't being distributed evenly! "Do I need to like.. shuffle the data around? How do I get it so that the data is evenly distributes?" At this point if you do not know what a red-black tree is, what do you even google look for? Lets say you dig around for a while and find out about re-balancing trees. Now you have to implement it, and eventually you figure out that one of the best ways is to color your nodes. When a future colleague/manager asks, "What is this system, please describe it to me?" wouldn't it be nice to just say "Oh visualize it kinda like a distributed red-black tree. It holds XXX data, and ensures that it is always sorted and re-balanced for optimal performance."
  - nerfhammer 10y ago
    FWIW finance normally uses big relational databases (because you absolutely need strong transaction support), something else off-the-shelf, or Microsoft Excel.
  - fragmede 10y ago
    Of course there's some sort of B-tree somewhere in the stack; that's not the question. With dozens of database stacks to choose from, many companies won't have anybody actively and regularly working in the database's code, nor in the OS kernel code either. If, then, an individual can successfully contribute to the company without ever hearing of a red-black tree, what's the value in testing for it, past "this is how we've always done it"? (How long it would take someone who doesn't know CS lingo to string together "tree" and "balance" and plug that into Google, I don't know, but I suspect it's not impossible to find.)
    - yazaddaruvala 10y ago
      > Of course there's some sort of B-tree somewhere in the stack; that's not the question. You've misunderstood me. I'm literally talking about building a distributed B-Tree or Heap. As in, a Heap which cannot fit in memory or storage of any one node. Think: S3/DynamoDB is a distributed, highly available HashTable; _____ is a distributed, highly available Heap.
- PlaceFan 10y ago
  As a counterpoint, I work in high finance and one of our core systems that I'm expected to know extremely well uses Paxos for consensus. Just because those topics show up infrequently doesn't mean that no one's using them. Plenty of companies outside of Google, etc. are doing complicated things that require a solid CS background.
  - FiveDegrees 10y ago
    You should have programmers implement paxos on a whiteboard in their interviews.
  - johnnyb9 10y ago
    My point was not that they do or do not exist, merely the magnitude. I would venture a guess that for every software engineer working on a blockchain there are 100 working on crud-style applications. If a job on your team requires a solid understanding of these concepts, then by all means filter on knowledge of algorithms. My point is that many companies don't do anything remotely as complex and yet use the same filters to screen job applicants.

dagw 10y ago

exceedingly few programming jobs need to know anything resembling advanced computer science theory There's "need" and there's "need". The person without the solid computer science background may very well be able to solve all the problems at hand, but the person with the solid CS background is far more likely to come up with the fast elegant solution that doesn't fall over in obscure corner cases. To use a concrete example for early in my career; I spent days and days trying to solve a problem by building a bigger and bigger pile regexps and if-else statements. Then my project manager (who had a PhD in CS) came along a just wrote a custom parser that solved the whole thing.

arvinsim 10y ago
You don't need to know how to implement a parser to recognize it as a solution to a problem. And there are already libraries and tools for building parsers anyway.
- sanderjd 10y ago
  The hard part isn't building the parser, the hard part is recognizing that a parser would be a good solution to the problem and distilling the problem into a form for which the solution works. One of the main purposes of a college degree is to come out knowing a lot about what you don't know. That is, knowing a lot about what sorts of things exist, even if you don't know much detail about any of those things.
majormajor 10y ago
> the fast elegant solution that doesn't fall over in obscure corner cases This can go both ways, though. Sometimes the fast elegant solution is too elegant for the nasty corner cases and a much bigger, uglier, but ultimately straightforward procedural block of crap is better. The examples I've seen are from established businesses that occasionally slipped up in refactoring out hacks that were introduced to support deadline-driven requirements. A year or two later, that hack is powering the reporting for a huge portion of the traffic, and you have to be able to balance between (a) writing maintainable code that supports the hack for the short term and (b) working with the rest of the business to clean up the requirements so you can improve the system for the long term. (b) is a skill that most interview processes I've seen completely ignore. And (a) often is downplayed (as "simple" or "easy") compared to more clever tricks - but knowing how to do the clever tricks might turn into a temptation to use them more than you should.

bpodgursky 10y ago

Having basic algorithmic competence (and yes, BSTs + BFS/DFS are basic) is like knowing how to swim. Maybe you aren't near water or writing performance-sensitive code 95% of the time, but the 5% of the time you're in over your head, being able to implement (or identify when to use) basic algorithms will save you a whole lot of time, money, and headache.