6 ms·

Implementing: never. Evaluating: occasionally. In my opinion, an understanding of data structures is _much_ more useful for a data scientist than algorithms.

Implementing: never. Evaluating: occasionally. In my opinion, an understanding of data structures is _much_ more useful for a data scientist than algorithms. Why should data scientists know about algorithms? Because data scientists are typically interviewed by computer scientists/software engineers, and that's what they tend to ask. I recently conducted many phone and on site interviews for a data scientist position. I didn't ask about algorithms.

willhslade 10y ago
For software engineers, algorithmic complexity is a good filter for, say, Javascript hackers vs people with a university education in computer science. Just saying.
- joshwcomeau 10y ago
  And in your mind, "javascript hackers" are worse than "people with a university education in computer science" at doing modern front-end web development? In my experience, building performant web applications is much more about things like reducing bundle size, making sure animations are hardware-accelerated, being smart about _when_ you do complex work... The cost of using an O(n^2) algorithm over an O(n) algorithm will rarely have a tangible impact, since you don't typically deal with item sets that large on the client. Not debating that CS fundamentals are valuable, just that they are _far_ from the most important skillset to have. Give me a JS hacker who understands page load times over a CS grad who writes his own bucket sort algorithm any day.
  - snovv_crash 10y ago
    If the 'javascript hacker' doesn't learn about the difference between iterating through a list and binary searching, and how/when one is better than the other, yes it is a problem. I say this as a self taught programmer who studied a non-CS engineering well after learning about big-O.
    - irrational 10y ago
      Can you give me an example of when a front end developer would need to do either of those things? On the back end sure, but on the front end? Who in the world is using JS to iterate through a list or do binary searching on the front end?
      - terminalcommand 10y ago
        And also who in the back-end uses binary search? We're the people who invented NoSQL databases with HTTP/Json interface, because traditional databases were too much of a hassle. The DBs implement the binary-search, not the back-end Dev. For the average programmer, IMHO learning about data structures/algorithms makes you a better programmer, but it's not that essential.
      - throwanem 10y ago
        Where I work, one of the applications we've recently built is a planning and design tool for a specific type of structure. There are both electrical and mechanical considerations to address, and the problem domain is sufficiently broad and complex as to require, at time of writing, about 25 megabytes of constant data to cover all the possible designs. Because of a client requirement, the application is also frontend-only, with no backend interaction beyond the initial download, and a PDF of many pages as the end product of the process. When your UI is based on 25 megabytes of lists and maps, you do a lot of iterating and filtering. In our current implementation, it's possible, but difficult, to produce a case where it spends as much as a quarter of a second doing that in one go. But only the first time; if it takes that long (anything over 100ms), we memoize the call so the next time you get it for free. Binary search hasn't yet been required, but it's on our short list of performance improvements to apply once the backing data grows enough to require them. Another project from last year was a tool to consume very large (250-300M) XML dumps from a very large, very expensive line-of-business system used by our finance department, and produce a wide variety of aggregations to simplify verifying the output of said very large, very expensive system, which I gather may not always be able to perform arithmetic correctly. This one wasn't a team effort; it was one of those things where you get the spec on Friday afternoon and the deadline is Monday morning. (Not something I'd tolerate on a regular basis, but when the stakes are "it's this, or the SVP Finance sends a helicopter to retrieve the VP IT from a cruise liner"...) But it's also a frontend-only application, because installers take time to get right, too, and in any case something so simple has no need for a backend. It takes about three minutes to fully parse, process, and report on a 250M XML dump. It leaks no resources, does not hang the UI thread, and presents a cute Bootstrap progress bar along with a table of running totals, so the user knows what it's doing. (Not gonna lie, I was showing off a little with the table. It's fun to watch numbers blur as they spin upwards in value!) I concede that this tool doesn't do much work with long lists - mainly just the values of interest from the raw dump, which are several lists of maximum cardinality on the order of ten thousand, and the negligibly short lists used to maintain parser state information. But I do gather a certain impression that you're one of those sadly behind the times folks who still thinks of the browser/JS platform as a cute toy and a decent document reader, but not up to anything remotely resembling Real Computational Work, and I thought I'd include this example, as well as the other, in order to help disabuse you of that rather outdated notion.
  - kbenson 10y ago
    I'm not sure where you got the implication that one was being called out as worse than the other, but the comment is really just noting it's a way to filter out specific groups. That filter may be useless when hiring for a front-end developer, but on the other hand, if hiring a developer to work on your new database product, it may be very useful indeed. As you note, they commonly deal with different types of complexity.
    - joshwcomeau 10y ago
      Yeah, good point. I may have been adding a tone that wasn't actually there. Apologies if I took it the wrong way!
  - throwawayish 10y ago
    Where did you get the impression that GP was talking about "doing front-end web development"?
    - joshwcomeau 10y ago
      Hm. Yeah, fair question. I guess I was assuming that, since the applicants could be described as "javascript hackers", then it was a JS position. Most JS positions are front-end ones. But yeah, that's admittedly a tenuous thread.
- peferron 10y ago
  Seems like a terrible filter. Some CS grads who slept through college will fail, while some non-CS grads who studied on their own will pass. Of course, to me, that would a feature, not a bug; but if you really want to filter on "university education in computer science", just read their resume instead.
  - waynecolvin 10y ago
    It can be a terrible filter, but credentialism fits in well with bureaucratic environments. :/
- waynecolvin 10y ago
  True. But before the days of Structured Programming widespread understanding of algorithms was hoped to lead to a professional Software Engineering field. The idea of "Software Engineering" was born with the Software Crisis report in the 1960's. Later on practicing SE's would study things like Design Patterns so things evolved over time. EDIT: Software Engineers probably should know some relevant basics, but programmers could never come up with a "body of knowledge" like real engineers have. It all depends what you work on and what new developments keep coming out...

willhslade 10y ago

For software engineers, algorithmic complexity is a good filter for, say, Javascript hackers vs people with a university education in computer science. Just saying.

joshwcomeau 10y ago
And in your mind, "javascript hackers" are worse than "people with a university education in computer science" at doing modern front-end web development? In my experience, building performant web applications is much more about things like reducing bundle size, making sure animations are hardware-accelerated, being smart about _when_ you do complex work... The cost of using an O(n^2) algorithm over an O(n) algorithm will rarely have a tangible impact, since you don't typically deal with item sets that large on the client. Not debating that CS fundamentals are valuable, just that they are _far_ from the most important skillset to have. Give me a JS hacker who understands page load times over a CS grad who writes his own bucket sort algorithm any day.
- snovv_crash 10y ago
  If the 'javascript hacker' doesn't learn about the difference between iterating through a list and binary searching, and how/when one is better than the other, yes it is a problem. I say this as a self taught programmer who studied a non-CS engineering well after learning about big-O.
  - irrational 10y ago
    Can you give me an example of when a front end developer would need to do either of those things? On the back end sure, but on the front end? Who in the world is using JS to iterate through a list or do binary searching on the front end?
    - terminalcommand 10y ago
      And also who in the back-end uses binary search? We're the people who invented NoSQL databases with HTTP/Json interface, because traditional databases were too much of a hassle. The DBs implement the binary-search, not the back-end Dev. For the average programmer, IMHO learning about data structures/algorithms makes you a better programmer, but it's not that essential.
    - throwanem 10y ago
      Where I work, one of the applications we've recently built is a planning and design tool for a specific type of structure. There are both electrical and mechanical considerations to address, and the problem domain is sufficiently broad and complex as to require, at time of writing, about 25 megabytes of constant data to cover all the possible designs. Because of a client requirement, the application is also frontend-only, with no backend interaction beyond the initial download, and a PDF of many pages as the end product of the process. When your UI is based on 25 megabytes of lists and maps, you do a lot of iterating and filtering. In our current implementation, it's possible, but difficult, to produce a case where it spends as much as a quarter of a second doing that in one go. But only the first time; if it takes that long (anything over 100ms), we memoize the call so the next time you get it for free. Binary search hasn't yet been required, but it's on our short list of performance improvements to apply once the backing data grows enough to require them. Another project from last year was a tool to consume very large (250-300M) XML dumps from a very large, very expensive line-of-business system used by our finance department, and produce a wide variety of aggregations to simplify verifying the output of said very large, very expensive system, which I gather may not always be able to perform arithmetic correctly. This one wasn't a team effort; it was one of those things where you get the spec on Friday afternoon and the deadline is Monday morning. (Not something I'd tolerate on a regular basis, but when the stakes are "it's this, or the SVP Finance sends a helicopter to retrieve the VP IT from a cruise liner"...) But it's also a frontend-only application, because installers take time to get right, too, and in any case something so simple has no need for a backend. It takes about three minutes to fully parse, process, and report on a 250M XML dump. It leaks no resources, does not hang the UI thread, and presents a cute Bootstrap progress bar along with a table of running totals, so the user knows what it's doing. (Not gonna lie, I was showing off a little with the table. It's fun to watch numbers blur as they spin upwards in value!) I concede that this tool doesn't do much work with long lists - mainly just the values of interest from the raw dump, which are several lists of maximum cardinality on the order of ten thousand, and the negligibly short lists used to maintain parser state information. But I do gather a certain impression that you're one of those sadly behind the times folks who still thinks of the browser/JS platform as a cute toy and a decent document reader, but not up to anything remotely resembling Real Computational Work, and I thought I'd include this example, as well as the other, in order to help disabuse you of that rather outdated notion.
- kbenson 10y ago
  I'm not sure where you got the implication that one was being called out as worse than the other, but the comment is really just noting it's a way to filter out specific groups. That filter may be useless when hiring for a front-end developer, but on the other hand, if hiring a developer to work on your new database product, it may be very useful indeed. As you note, they commonly deal with different types of complexity.
  - joshwcomeau 10y ago
    Yeah, good point. I may have been adding a tone that wasn't actually there. Apologies if I took it the wrong way!
- throwawayish 10y ago
  Where did you get the impression that GP was talking about "doing front-end web development"?
  - joshwcomeau 10y ago
    Hm. Yeah, fair question. I guess I was assuming that, since the applicants could be described as "javascript hackers", then it was a JS position. Most JS positions are front-end ones. But yeah, that's admittedly a tenuous thread.
peferron 10y ago
Seems like a terrible filter. Some CS grads who slept through college will fail, while some non-CS grads who studied on their own will pass. Of course, to me, that would a feature, not a bug; but if you really want to filter on "university education in computer science", just read their resume instead.
- waynecolvin 10y ago
  It can be a terrible filter, but credentialism fits in well with bureaucratic environments. :/
waynecolvin 10y ago
True. But before the days of Structured Programming widespread understanding of algorithms was hoped to lead to a professional Software Engineering field. The idea of "Software Engineering" was born with the Software Crisis report in the 1960's. Later on practicing SE's would study things like Design Patterns so things evolved over time. EDIT: Software Engineers probably should know some relevant basics, but programmers could never come up with a "body of knowledge" like real engineers have. It all depends what you work on and what new developments keep coming out...