6 ms·

> “Yes, every programmer should know how their hardware works as one cannot write an efficient software without that knowledge 4, and I wouldn’t be able to do m

by lytefm 5y ago

> “Yes, every programmer should know how their hardware works as one cannot write an efficient software without that knowledge 4, and I wouldn’t be able to do my job if I didn’t know anything about computer internals, yak, yak, yak…” I agree that you don't need to know anything about von Neumann Architecture and the likes in order to be a Web Dev. But I wouldn't want to hire someone who doesn't have basic knowledge about data structures and runtime complexity. It's just too easy to have something work fine in development, but choke in production due to accidentally quadratic behaviour.

AmericanChopper 5y ago
Not knowing how databases work is a pretty common issue I come across with colleagues. Even basic things like normal forms (which take about 5 minutes to understand), and what transactions are. I think it’s why non-RDBMS datastores are so popular, because they generally just ignore all the problems RDBMS are trying to solve, so the developer isn’t inconvenienced by thinking about them.
- guax 5y ago
  Non rdbms are popular because at scale everything that's nice about "classic" rdbms are what makes them ultra slow. edit: the features I refer to are stuff like foreign keys, cascades, dynamic fields, complex joins, etc. Once your stuff is spread out you loose some of these by definition and if you're still sticking to a single DB with massive stores you're not getting much humpf out of it anymore. I 100% agree there are way too many fools that think their DB is big and try non rdbms too early and waste too much time. But I still think their lack of recent popularity is due to their own limitations.
  - detaro 5y ago
    Except they are also popular with people that are not working "at scale" where that would matter.
  - Aeolun 5y ago
    Not really, they remain fast enough, there’s just no easy way to scale them past a certain point. Or there wasn’t anyway, I believe with spanner and CockroachDB we’re getting pretty close. But 99% of projects do not come even close to the scale where you need that.
  - AmericanChopper 5y ago
    There are very few RDBMS usecases where scale is a legitimate performance concern. Index scans scale to a very, very high cardinality. There are use cases that aren’t best suited to an RDBMS at any scale, like managing a collection of denormalized documents, or a graph. But in my experience, considering these factors isn’t what’s motivating adoption of Mongo, or Dynamo. Your key lookups will always be fast (until you get a hot shard), but when you cram a highly relational set into a document db, eventually you’re forced to realize that there’s a lot of mutations and access patterns that you simply can’t implement, because you chose the wrong tool for the job. Either that or you’ll decide that it’s a good idea to implement your own concurrency control mechanism on top of your document database. Which I’ve personally seen at least 3 people try and do.
  - hawk_ 5y ago
    obligatory https://youtu.be/b2F-DItXtZs https://youtu.be/b2F-DItXtZs webscale database discussion
bluesign 5y ago
Usually I had funny but opposite experience when doing consulting for performance optimizations. At some points either teams converged to people without basic knowledge of data structures or they rushed with something that works (when N in O(N) was small) then scaled up. They get used to lower performance slowly like boiled frog. Then someone is tasked with optimization, but instead of attacking data structure at hand, usually those people tried to optimize pieces of the puzzle for smaller gain, then they ended up with small hacks to shave some execution time, with unmaintainable code which lead to serious bugs.
- TheOtherHobbes 5y ago
  This is the kind of observation that should be central to CS, but isn't. While academic algorithm astronauts write papers which include Greek math symbols and can be proven correct, most people do... something else. Optimising the something else would be a huge benefit to the industry. But no one even looks at it. It's just a thing that happens, and there's no research into fixing it or making it better.
  - miloignis 5y ago
    I feel like this is moving in the opposite direction! I think the issue is that most of the time the "something else" people write with poor algorithms is actually an application of the correct, efficient algorithms from formal papers written by academics, but they don't know it and thus don't use it.
smcl 5y ago
I think the main thing we want to avoid here is situations like the following: - interviewer asks candidate to determine the complexity of some algorithms - candidate is nervous, has a minor brain-fart and flubs a couple of them - interviewer evaluates candidate as lacking basic CS knowledge you learn in 1st year of any CS course Ok in this case we're talking about big-O, but it could really be anything.
- sushsjsuauahab 5y ago
  I think it should depend on how much money you are intending to pay the candidate and how desperate your company is to hire. By my experience, some companies need a competent body in the seat ASAP, and some are looking for world-class engineers.
  - throwoutway 5y ago
    But if they’re not competent, then you’ll be stuck debugging/rewriting 6 months of the the incompetent engineer’s work
    - sushsjsuauahab 5y ago
      Completely agree and have been burned this way as well. If you need engineers but cant even find competent ones (something ive seen before), then the only choice is to put them on prototype work/non-critical projects
  - smcl 5y ago
    That's true to some extent, but I am quite sure even the worldest-of-class engineers have tripped up on a question or two and went on to get a rejection. If such people truly are talented then they'll have no trouble getting hired elsewhere and the companies in question would be blissfully unaware what they missed out on.
    - sushsjsuauahab 5y ago
      Such companies deserve to fail if so, as the supply of such engineers is limited and allegedly they need such engineers to survive :) Think Sears vs Amazon-- which company would know what gold was if they fell into a pile of it?

by lytefm 5y ago

AmericanChopper 5y ago

Not knowing how databases work is a pretty common issue I come across with colleagues. Even basic things like normal forms (which take about 5 minutes to understand), and what transactions are. I think it’s why non-RDBMS datastores are so popular, because they generally just ignore all the problems RDBMS are trying to solve, so the developer isn’t inconvenienced by thinking about them.

guax 5y ago
Non rdbms are popular because at scale everything that's nice about "classic" rdbms are what makes them ultra slow. edit: the features I refer to are stuff like foreign keys, cascades, dynamic fields, complex joins, etc. Once your stuff is spread out you loose some of these by definition and if you're still sticking to a single DB with massive stores you're not getting much humpf out of it anymore. I 100% agree there are way too many fools that think their DB is big and try non rdbms too early and waste too much time. But I still think their lack of recent popularity is due to their own limitations.
- detaro 5y ago
  Except they are also popular with people that are not working "at scale" where that would matter.
- Aeolun 5y ago
  Not really, they remain fast enough, there’s just no easy way to scale them past a certain point. Or there wasn’t anyway, I believe with spanner and CockroachDB we’re getting pretty close. But 99% of projects do not come even close to the scale where you need that.
- AmericanChopper 5y ago
  There are very few RDBMS usecases where scale is a legitimate performance concern. Index scans scale to a very, very high cardinality. There are use cases that aren’t best suited to an RDBMS at any scale, like managing a collection of denormalized documents, or a graph. But in my experience, considering these factors isn’t what’s motivating adoption of Mongo, or Dynamo. Your key lookups will always be fast (until you get a hot shard), but when you cram a highly relational set into a document db, eventually you’re forced to realize that there’s a lot of mutations and access patterns that you simply can’t implement, because you chose the wrong tool for the job. Either that or you’ll decide that it’s a good idea to implement your own concurrency control mechanism on top of your document database. Which I’ve personally seen at least 3 people try and do.
- hawk_ 5y ago
  obligatory https://youtu.be/b2F-DItXtZs https://youtu.be/b2F-DItXtZs webscale database discussion

bluesign 5y ago

Usually I had funny but opposite experience when doing consulting for performance optimizations. At some points either teams converged to people without basic knowledge of data structures or they rushed with something that works (when N in O(N) was small) then scaled up. They get used to lower performance slowly like boiled frog. Then someone is tasked with optimization, but instead of attacking data structure at hand, usually those people tried to optimize pieces of the puzzle for smaller gain, then they ended up with small hacks to shave some execution time, with unmaintainable code which lead to serious bugs.

TheOtherHobbes 5y ago
This is the kind of observation that should be central to CS, but isn't. While academic algorithm astronauts write papers which include Greek math symbols and can be proven correct, most people do... something else. Optimising the something else would be a huge benefit to the industry. But no one even looks at it. It's just a thing that happens, and there's no research into fixing it or making it better.
- miloignis 5y ago
  I feel like this is moving in the opposite direction! I think the issue is that most of the time the "something else" people write with poor algorithms is actually an application of the correct, efficient algorithms from formal papers written by academics, but they don't know it and thus don't use it.

smcl 5y ago

I think the main thing we want to avoid here is situations like the following: - interviewer asks candidate to determine the complexity of some algorithms - candidate is nervous, has a minor brain-fart and flubs a couple of them - interviewer evaluates candidate as lacking basic CS knowledge you learn in 1st year of any CS course Ok in this case we're talking about big-O, but it could really be anything.

sushsjsuauahab 5y ago
I think it should depend on how much money you are intending to pay the candidate and how desperate your company is to hire. By my experience, some companies need a competent body in the seat ASAP, and some are looking for world-class engineers.
- throwoutway 5y ago
  But if they’re not competent, then you’ll be stuck debugging/rewriting 6 months of the the incompetent engineer’s work
  - sushsjsuauahab 5y ago
    Completely agree and have been burned this way as well. If you need engineers but cant even find competent ones (something ive seen before), then the only choice is to put them on prototype work/non-critical projects
- smcl 5y ago
  That's true to some extent, but I am quite sure even the worldest-of-class engineers have tripped up on a question or two and went on to get a rejection. If such people truly are talented then they'll have no trouble getting hired elsewhere and the companies in question would be blissfully unaware what they missed out on.
  - sushsjsuauahab 5y ago
    Such companies deserve to fail if so, as the supply of such engineers is limited and allegedly they need such engineers to survive :) Think Sears vs Amazon-- which company would know what gold was if they fell into a pile of it?