4 ms·

"This means we need a way to represent loss functions in SQL, so we first introduce some necessary mathematical background and define the terms used in our impl

"This means we need a way to represent loss functions in SQL, so we first introduce some necessary mathematical background and define the terms used in our implementation. " Jesus flippin Chrikey kill it with fire. Academics; where do they come from? Having scoped out a design for this, seen a design for it in a large corporate research lab, and noting that several large scale machine learning packages .... almost do it .... themselves: you just need to lay out the data in a memory friendly way, and you're freaking done. It's a completely straightforward task which could be done with any number of existing columnar data stores if it hasn't been done already. The only reason it hasn't been outside of (large research lab) is the type of skill needed to lay out the data in the DB properly and the type of skill needed to write efficient machine learning algorithms almost never exist in the same person. Then you get academics .... who think you need to write automatic differentiation in SQL to integrate ML with databases, having never heard of, I dunno, online BFGS or Pegasos, and, like, C.

tchalla 6y ago
> The only reason it hasn't been outside of (large research lab) is the type of skill needed to lay out the data in the DB properly and the type of skill needed to write efficient machine learning algorithms almost never exist in the same person. Then you get academics .... who think you need to write automatic differentiation in SQL to integrate ML with databases, having never heard of, I dunno, online BFGS or Pegasos, and, like, C. I'm curious. In your opinion, is there a scenario where there is a possibility and need to do what the authors have done?
- scottlocklin 6y ago
  I can't imagine it, just like I can't imagine why you'd want to do it in a DSL oriented around set theory instead of, say, the DB implementation language. I assume they're thinking of deep learning? Why you'd want to do it "in database" is beyond me. The only reason I thought of doing it was skipping the "marshall the data" step for online algorithms; performance basically. If you look at something like Vowpal Wabbit, it stores/consumes data in this ridiculous text format dating from when people invented the Support Vector Machine in the 80s. This annoys the shit out of me, as it involves creating the ridiculous 80s text format. What if it consumed database columns instead of this nonsense, and did the hashing trick and everything you needed to solve the problem? That would be cool. That, in fact, would be something like how the universe is supposed to function instead of the stunted grotesqueries we have today. You could do crap like exploratory analysis right on your database, as a query. You could even get fancy and do wackadoo online matrix decompositions while you're writing the data out in the first place (or at least when nobody's looking), and store it as metadata, meaning you know all kinds of good shit about your data even as you're writing it down. Anyway, because marketing departments keep bellowing about "deep learning" instead of the actual breakthroughs in machine learning and linear algebra of the last 20 years, nobody gave a shit about it. Even (large research group in gigantor corp) couldn't figure out a way of selling the idea. I went on to a productive career in something entirely different, and all I got out of it was the ability to make snarky comments about seemingly clueless academics.
  - kohlerm 6y ago
    Why would you want to do the Machine Learning within the DB anyway? To me this looks like rather a corner case. E.g. I usually want to scale my MLE infrastructure independent from the DB. To it seems much to pipe the Data into log like Kafka to compute the result, just because that is much easier to scale. Similarly for training it seem to be that using some files on an Objectstore as the input is usually much easier to scale.
    - scottlocklin 6y ago
      I guess if you don't care about data exploration, efficiency or doing work on data much larger than memory, there's no reason to do it. As you note, most people seem to get along fine without this idea.
    - newdude116 6y ago
      Hm. Maybe because you could do it in the RAM directly in the future? Maybe you want to use an encrypted DB without decrypting it? This is research, not a vanilla solution.
Irishsteve 6y ago
They are getting paid to research different ideas so thats why they might be proposing ideas you find to be misguided. There are instances where researchers spend their time where the outcome was mostly thought experiment / not directly applicable, and there are instances where the work was useful from an application perspective. Such is life
- scottlocklin 6y ago
  You left out "there are instances where the researchers actually have no idea what they're talking about, but hope to get a lucrative job in the subject to escape teaching C++ to 19 year olds for the rest of their lives." This may not be such an instance, but it sure looks like one!
aspaceman 6y ago
You’re an engineer. You care about engineering. This is fine. They care about mathematical background and laying out definitions. Academics care about those pointless things. This is also fine. If you don’t care about these things, don’t read it. You don’t write a system like this to say: “Here’s a system I wrote, go and use it in practice” as much as “What have we learned from trying this?” It’s the job of an engineer to synthesize useful things from the abstract knowledge.
- scottlocklin 6y ago
  >You’re an engineer. You failed, first sentence. You're also the second person asserting some secret knowledge of academic ding dongs who published a paper (as if this is some kind of achievement) without, you know, actually even attempting to make the case.
  - aspaceman 6y ago
    It was just an assumption. At the very least you think like one. And I’m not following. I’m not asserting some secret knowledge. This work is knowledge. It’s not practically useful for you. But it doesn’t have to be. And you’re the “ding dong” if you think it has to be valuable to you for someone to get something from it.

tchalla 6y ago

> The only reason it hasn't been outside of (large research lab) is the type of skill needed to lay out the data in the DB properly and the type of skill needed to write efficient machine learning algorithms almost never exist in the same person. Then you get academics .... who think you need to write automatic differentiation in SQL to integrate ML with databases, having never heard of, I dunno, online BFGS or Pegasos, and, like, C. I'm curious. In your opinion, is there a scenario where there is a possibility and need to do what the authors have done?

scottlocklin 6y ago
I can't imagine it, just like I can't imagine why you'd want to do it in a DSL oriented around set theory instead of, say, the DB implementation language. I assume they're thinking of deep learning? Why you'd want to do it "in database" is beyond me. The only reason I thought of doing it was skipping the "marshall the data" step for online algorithms; performance basically. If you look at something like Vowpal Wabbit, it stores/consumes data in this ridiculous text format dating from when people invented the Support Vector Machine in the 80s. This annoys the shit out of me, as it involves creating the ridiculous 80s text format. What if it consumed database columns instead of this nonsense, and did the hashing trick and everything you needed to solve the problem? That would be cool. That, in fact, would be something like how the universe is supposed to function instead of the stunted grotesqueries we have today. You could do crap like exploratory analysis right on your database, as a query. You could even get fancy and do wackadoo online matrix decompositions while you're writing the data out in the first place (or at least when nobody's looking), and store it as metadata, meaning you know all kinds of good shit about your data even as you're writing it down. Anyway, because marketing departments keep bellowing about "deep learning" instead of the actual breakthroughs in machine learning and linear algebra of the last 20 years, nobody gave a shit about it. Even (large research group in gigantor corp) couldn't figure out a way of selling the idea. I went on to a productive career in something entirely different, and all I got out of it was the ability to make snarky comments about seemingly clueless academics.
- kohlerm 6y ago
  Why would you want to do the Machine Learning within the DB anyway? To me this looks like rather a corner case. E.g. I usually want to scale my MLE infrastructure independent from the DB. To it seems much to pipe the Data into log like Kafka to compute the result, just because that is much easier to scale. Similarly for training it seem to be that using some files on an Objectstore as the input is usually much easier to scale.
  - scottlocklin 6y ago
    I guess if you don't care about data exploration, efficiency or doing work on data much larger than memory, there's no reason to do it. As you note, most people seem to get along fine without this idea.
  - newdude116 6y ago
    Hm. Maybe because you could do it in the RAM directly in the future? Maybe you want to use an encrypted DB without decrypting it? This is research, not a vanilla solution.

Irishsteve 6y ago

They are getting paid to research different ideas so thats why they might be proposing ideas you find to be misguided. There are instances where researchers spend their time where the outcome was mostly thought experiment / not directly applicable, and there are instances where the work was useful from an application perspective. Such is life

scottlocklin 6y ago
You left out "there are instances where the researchers actually have no idea what they're talking about, but hope to get a lucrative job in the subject to escape teaching C++ to 19 year olds for the rest of their lives." This may not be such an instance, but it sure looks like one!

aspaceman 6y ago

You’re an engineer. You care about engineering. This is fine. They care about mathematical background and laying out definitions. Academics care about those pointless things. This is also fine. If you don’t care about these things, don’t read it. You don’t write a system like this to say: “Here’s a system I wrote, go and use it in practice” as much as “What have we learned from trying this?” It’s the job of an engineer to synthesize useful things from the abstract knowledge.

scottlocklin 6y ago
>You’re an engineer. You failed, first sentence. You're also the second person asserting some secret knowledge of academic ding dongs who published a paper (as if this is some kind of achievement) without, you know, actually even attempting to make the case.
- aspaceman 6y ago
  It was just an assumption. At the very least you think like one. And I’m not following. I’m not asserting some secret knowledge. This work is knowledge. It’s not practically useful for you. But it doesn’t have to be. And you’re the “ding dong” if you think it has to be valuable to you for someone to get something from it.