4 ms·

Well, I think "can this be held in memory" can be a stupid question. What I thought was more of a problem is: > And they were right to fail me for that: the fa

Well, I think "can this be held in memory" can be a stupid question. What I thought was more of a problem is: > And they were right to fail me for that: the fact that I was good at solving strictly more difficult problems didn’t matter because I didn’t know how to solve the easier ones they actually had. Scale problems are a type of problem. They aren't strictly more difficult than problems on a smaller scale. Being able to scale to thousands users/cpus/shards is a skill. Thinking your niche skills are solving the "difficult" problems and other skills are easier is not a good attitude.

zem 5y ago
agreed. in fact, in my time at google i've focused on the other end of the spectrum, solving "small" (in terms of resources required) problems well so they can be used reliably by the people building larger-scale systems.
PragmaticPulp 5y ago
> Thinking your niche skills are solving the "difficult" problems and other skills are easier is not a good attitude. This is one of the biggest obstacles I've encountered when hiring people out of big companies and into smaller startups. At a very big, very established tech company it's easy to focus on your narrow problem and leverage all of the company's infrastructure to get things done. Moreover, you should be leveraging the company's infrastructure and not reinventing the wheel. Some (though not all) of these people struggle to adapt when they move to another company where all of that infrastructure isn't available. Maybe they never had a chance to learn how to do it. Or maybe they've just been convinced that their previous company's infrastructure was the only correct way to do something. Either way, they get stuck in a position where they don't know how to solve problems without their previous company's infrastructure. Best case, they admit the gap in their knowledge and work on filling it as fast as possible. Worst case, they spend years trying to recreate their previous company's big-company infrastructure before they can get started on this new small-company problem that they've been given.
- chii 5y ago
  > they don't know how to solve problems without their previous company's infrastructure. this is basically an indication of the lack of problem solving, and instead, they've just been "tool using". It's the same sort of phenomenon as someone copying a snippet off stackoverflow, and tweaking it, without understanding _exactly_ how it works (and all of the underlying theory/understanding required). Real problem-solvers don't need this, but only reach for a tool from their toolbag after they've completely understood the problem, and have a solution in mind.
a-dub 5y ago
> Well, I think "can this be held in memory" can be a stupid question. absolutely not. questions like that give you a baseline for idealized performance. not that you should ever expect to reach it, but it does give a good sanity check for whether or not a technology choice is a sane one. i've seen cassandra clusters used for datasets that would fit in a sqlite (with indexes!) in a ramdisk on a cheap laptop. so yes, scale problems go both ways. and to your point, the "difficult" problems aren't harder necessarily, often just those that you (and your team) don't know how to solve.
- Traster 5y ago
  My point was that there are plenty of examples of systems/requirements where the dimensions of a problem are just comically out of proportion for that question- where you can't just fit this in memory, you can fit it in memory on a commodore 64, and that whether something fits in memory is not how you should thinking about the problem.
- chii 5y ago
  > scale problems go both ways. and to your point, the "difficult" problems aren't harder necessarily i like the old adage: anyone can build a bridge. Only an engineer can build a bridge that can barely standup.
  - jiveturkey 5y ago
    I trust that is meant in the Colin Chapman sense? The racecar that fails the second after it crosses the finish line is the perfect racecar. Anything more reliable than that is excess weight, performance, cost, etc. I like that as well, idealistically. However, we work at average places with average coworkers, and are ourselves average. I'm happy to work with folks that can "only" overbuild the bridge. There's far too many people that are on the wrong side of bridge endurance.
WD-42 5y ago
This 100%. I worked at a company and we hired on a self described "AI expert" (I think he was a friend of the founder). He didn't last long because when push came to shove and there wasn't any hard AI work to be done, the guy didn't know how to write HTML, and couldn't seem bothered to learn. And the product was a web app. You know what else is a hard problem? Delivering a quality product on time and in good working order. Not everything has to be at a million user scale for it to be considered a difficult task.
wpietri 5y ago
> Well, I think "can this be held in memory" can be a stupid question. Why's that? If I'm sure the answer is always yes, I can quickly write something to solve a user problem that is way faster and less expensive to run.

zem 5y ago

agreed. in fact, in my time at google i've focused on the other end of the spectrum, solving "small" (in terms of resources required) problems well so they can be used reliably by the people building larger-scale systems.

PragmaticPulp 5y ago

> Thinking your niche skills are solving the "difficult" problems and other skills are easier is not a good attitude. This is one of the biggest obstacles I've encountered when hiring people out of big companies and into smaller startups. At a very big, very established tech company it's easy to focus on your narrow problem and leverage all of the company's infrastructure to get things done. Moreover, you should be leveraging the company's infrastructure and not reinventing the wheel. Some (though not all) of these people struggle to adapt when they move to another company where all of that infrastructure isn't available. Maybe they never had a chance to learn how to do it. Or maybe they've just been convinced that their previous company's infrastructure was the only correct way to do something. Either way, they get stuck in a position where they don't know how to solve problems without their previous company's infrastructure. Best case, they admit the gap in their knowledge and work on filling it as fast as possible. Worst case, they spend years trying to recreate their previous company's big-company infrastructure before they can get started on this new small-company problem that they've been given.

chii 5y ago
> they don't know how to solve problems without their previous company's infrastructure. this is basically an indication of the lack of problem solving, and instead, they've just been "tool using". It's the same sort of phenomenon as someone copying a snippet off stackoverflow, and tweaking it, without understanding _exactly_ how it works (and all of the underlying theory/understanding required). Real problem-solvers don't need this, but only reach for a tool from their toolbag after they've completely understood the problem, and have a solution in mind.

a-dub 5y ago

> Well, I think "can this be held in memory" can be a stupid question. absolutely not. questions like that give you a baseline for idealized performance. not that you should ever expect to reach it, but it does give a good sanity check for whether or not a technology choice is a sane one. i've seen cassandra clusters used for datasets that would fit in a sqlite (with indexes!) in a ramdisk on a cheap laptop. so yes, scale problems go both ways. and to your point, the "difficult" problems aren't harder necessarily, often just those that you (and your team) don't know how to solve.

Traster 5y ago
My point was that there are plenty of examples of systems/requirements where the dimensions of a problem are just comically out of proportion for that question- where you can't just fit this in memory, you can fit it in memory on a commodore 64, and that whether something fits in memory is not how you should thinking about the problem.
chii 5y ago
> scale problems go both ways. and to your point, the "difficult" problems aren't harder necessarily i like the old adage: anyone can build a bridge. Only an engineer can build a bridge that can barely standup.
- jiveturkey 5y ago
  I trust that is meant in the Colin Chapman sense? The racecar that fails the second after it crosses the finish line is the perfect racecar. Anything more reliable than that is excess weight, performance, cost, etc. I like that as well, idealistically. However, we work at average places with average coworkers, and are ourselves average. I'm happy to work with folks that can "only" overbuild the bridge. There's far too many people that are on the wrong side of bridge endurance.

WD-42 5y ago

This 100%. I worked at a company and we hired on a self described "AI expert" (I think he was a friend of the founder). He didn't last long because when push came to shove and there wasn't any hard AI work to be done, the guy didn't know how to write HTML, and couldn't seem bothered to learn. And the product was a web app. You know what else is a hard problem? Delivering a quality product on time and in good working order. Not everything has to be at a million user scale for it to be considered a difficult task.

wpietri 5y ago

> Well, I think "can this be held in memory" can be a stupid question. Why's that? If I'm sure the answer is always yes, I can quickly write something to solve a user problem that is way faster and less expensive to run.