4 ms·

100% agree. When I made the switch from technical business analyst to head down the road of data science, I had the option to use a year of postbac study to stu

100% agree. When I made the switch from technical business analyst to head down the road of data science, I had the option to use a year of postbac study to study computer science or math. the CS dept at the school wasn't fantastic, it was mostly in Java, and DS is moving so fast I wasn't certain what percentage of what I studied would be of any use in 5 years time. So I studied math. All the math I learned is still good as new. Still "true. Still useful. Many DS tools have come out since I finished that year. Many are Python based. One thing I am constantly frustrated by is the lack of mathematical rigor that pervade DS tools, and programming in general. Seems like everyone is happy to be given some programmatic tool and all the mistakes that where programmed into it. "You just need to deal with that." "You'll get used to the syntax." Yeah, but, I'd rather not, since it's often completely arbitrary and specific to a given tool. And, well, sometimes just doesn't make sense in the way something that has mathematical rigor built in might. But, I often feel when I bring this issue up I'm shouting down an empty hallway.

SomeStupidPoint 10y ago
Part of the problem with DS is that there are known engineering solutions without good theoretical explanation, that is, we just brute forced some basic engineering solutions and keep hacking at them to get more performance, but don't have a good theoretical model of why those things do what they do. (Or have a theoretical model too complex to compute useful insight, which is another kind of problem.) I think that this kind of approach is going to get us pretty far -- see how far we got on building bridges with no more sophisticated insight in to gravity than "things fall down; some things are heavy" or material science than "some things hard and not bend" -- but I think that ultimately, our longer term goals like AGI will require that we build theoretical models capable of explaining our current engineering progress and predict a new, deeper valley of progress to move to. Of course, that's a lot like saying making progress on physics paradigms requires explaining high temperature superconductors (or other currently open problems). It's actually pretty normal for a field to be a mix of engineers being ahead and theorists being ahead. Mathematical rigor is really a sign that you're in established territory and explorers have moved on, and much of data science is still under heavy exploration.
- throwaway98237 10y ago
  There are tons, tons, of examples of "established territory" in CS that do not exhibit "mathematical rigor". When something works in CS people seem to say "good enough" and use it. In mathematics (or as you pointed out, in physics) it becomes an "open question" and people begin the hard work of explaining the phenomenon by applying rigor. In maths, it's a constant march to push the boundaries, where the boundary is defined as that which is interesting but not yet well defined. In CS it seems that the boundary is that which is not yet a solved problem, where "problem" is something that needs getting done. Once, in CS, we can get it done, people move on. It's not often that in the enterprise CS process (academic is a whole other beast) it is assumed that there should be an application of rigor or an attempt to well define "solutions" before moving on. There is simply the accumulation of technical debt. But, in my opinion, technical debt has amassed at the systemic level to such a point where it's starting to look like a ponzi scheme. Sure, it works, so long as we keep throwing good money after bad. But, stop investing, and one can see it for what it is. In maths on the other hand, step away from it all for a year or two, and when you go back everything is still just as valuable and beautiful. edit: what i would love to see is a "category theory" for programming languages / paradigms such that moving between them is well defined. it boggles my mind that translating between two programming languages isn't trivial. if both are well defined, one should be able to translate one to the other precisely given one is willing to define the translations. there should be zero guess work or heuristics in the process.
  - deleted 10y ago
    [deleted]
  - botw 10y ago
    It is baffling that machine translation on human language is approaching human accuracy but it is still far behind on programming language, given that it is supposed to be more closed/matching to metal/machine.
    - SomeStupidPoint 10y ago
      I find that the opposite of baffling: human language is information sparse and redundant; computer languages are information dense and, by choice, are made as unredundant as possible. It just seems like trying to hit a big fluffy cloud on one side and a bald tree on the other. Obviously the big fluffy thing is easier than the spindly, narrow one. How does machine translation do at poetry, in the sense of capturing the figurative meaning and stylistic elements, not just the literal meaning of the tokens?

SomeStupidPoint 10y ago

Part of the problem with DS is that there are known engineering solutions without good theoretical explanation, that is, we just brute forced some basic engineering solutions and keep hacking at them to get more performance, but don't have a good theoretical model of why those things do what they do. (Or have a theoretical model too complex to compute useful insight, which is another kind of problem.) I think that this kind of approach is going to get us pretty far -- see how far we got on building bridges with no more sophisticated insight in to gravity than "things fall down; some things are heavy" or material science than "some things hard and not bend" -- but I think that ultimately, our longer term goals like AGI will require that we build theoretical models capable of explaining our current engineering progress and predict a new, deeper valley of progress to move to. Of course, that's a lot like saying making progress on physics paradigms requires explaining high temperature superconductors (or other currently open problems). It's actually pretty normal for a field to be a mix of engineers being ahead and theorists being ahead. Mathematical rigor is really a sign that you're in established territory and explorers have moved on, and much of data science is still under heavy exploration.

throwaway98237 10y ago
There are tons, tons, of examples of "established territory" in CS that do not exhibit "mathematical rigor". When something works in CS people seem to say "good enough" and use it. In mathematics (or as you pointed out, in physics) it becomes an "open question" and people begin the hard work of explaining the phenomenon by applying rigor. In maths, it's a constant march to push the boundaries, where the boundary is defined as that which is interesting but not yet well defined. In CS it seems that the boundary is that which is not yet a solved problem, where "problem" is something that needs getting done. Once, in CS, we can get it done, people move on. It's not often that in the enterprise CS process (academic is a whole other beast) it is assumed that there should be an application of rigor or an attempt to well define "solutions" before moving on. There is simply the accumulation of technical debt. But, in my opinion, technical debt has amassed at the systemic level to such a point where it's starting to look like a ponzi scheme. Sure, it works, so long as we keep throwing good money after bad. But, stop investing, and one can see it for what it is. In maths on the other hand, step away from it all for a year or two, and when you go back everything is still just as valuable and beautiful. edit: what i would love to see is a "category theory" for programming languages / paradigms such that moving between them is well defined. it boggles my mind that translating between two programming languages isn't trivial. if both are well defined, one should be able to translate one to the other precisely given one is willing to define the translations. there should be zero guess work or heuristics in the process.
- deleted 10y ago
  [deleted]
- botw 10y ago
  It is baffling that machine translation on human language is approaching human accuracy but it is still far behind on programming language, given that it is supposed to be more closed/matching to metal/machine.
  - SomeStupidPoint 10y ago
    I find that the opposite of baffling: human language is information sparse and redundant; computer languages are information dense and, by choice, are made as unredundant as possible. It just seems like trying to hit a big fluffy cloud on one side and a bald tree on the other. Obviously the big fluffy thing is easier than the spindly, narrow one. How does machine translation do at poetry, in the sense of capturing the figurative meaning and stylistic elements, not just the literal meaning of the tokens?