3 ms·
> If the job is heavy in large data sets and number crunching, then it makes sense to ask an algorithm question. For that kind of job, why would you ever roll
by SideburnsOfDoom 10y ago
> If the job is heavy in large data sets and number crunching, then it makes sense to ask an algorithm question.
For that kind of job, why would you ever roll your own rather than using tools that have libraries of standard algorithms and data structures?
- knightofmars 10y agoI certainly hope you wouldn't "roll your own" but you do need to understand how to identify the type of data and the correct algorithm to use in processing it based on the requirements provided.
- barrkel 10y agoPre-packaged libraries are generally optimized for being composable and generic. When trying to get more performance, it can make sense to e.g. create an index or lookup that can be used in multiple steps, or fold multiple iterative steps into a single iteration, or take advantage of the data's distribution to use a more specific approach, etc. You don't have to walk far from the beaten path at all before custom data structures start making lots of sense. Most interesting data structures in the literature are only implemented in a resuable, available way in one or two languages, if that.
- MichaelGG 10y agoYou will almost certainly not be writing your own, until you do. More importantly, understanding this means you'll know the right questions to ask, or even be able to say upfront that a certain well-liked product some exec wants to use won't scale, even if the trial with a dataset less than RAM works "really fast!".