8 ms·

Furthermore, follow https://twitter.com/_brohrer_/status/1425770502321283073 https://twitter.com/_brohrer_/status/1425770502321283073 "When you have a problem,

Furthermore, follow https://twitter.com/_brohrer_/status/1425770502321283073 https://twitter.com/_brohrer_/status/1425770502321283073 "When you have a problem, build two solutions - a deep Bayesian transformer running on multicloud Kubernetes and a SQL query built on a stack of egregiously oversimplifying assumptions. Put one on your resume, the other in production. Everyone goes home happy."

arketyp 5y ago
Furthermore in the article, yes.
hughrr 5y ago
This reminds me of an experience I had watching a company trying to replace a system with ML. First they marketed it heavily before even thinking. During test cycle they fed the entire data corpus in and ran some of the original test cases and found some business destroying results pop out. The entire system ended up a verbatim port of the VB6 crap which was a verbatim port of the original AS400 crap that actually worked. The marketing to this day says it’s ML based and everyone buys into the hype. It’s not. It was a complete failure. But the original system has 30 years of human experience codified in it.
- ethbr0 5y ago
  If I had a nickel for every time I've seen "business rules engine" turned into "AI" in the last few years... But I guess if we complain that half of our colleagues and the media don't understand ML, why should we expect management to? When the command from C-level is "We need some AI projects to tell our shareholders about," we shouldn't be surprised when middle management suddenly has successful AI projects in their slide decks.
  - mjburgess 5y ago
    Well really-existing AI is just either "taking a mean()" or "programming a rule". So all of programming actually counts as (symbolic) AI.
  - cm2187 5y ago
    In the same way the terms “blockchain” being used for “digital signing” or “cloud” for a server...
  - i_am_proteus 5y ago
    If you have an existing rules-based decision-tree system, and you compare its performance with a bunch of other decision trees, and it does better, you are implementing a random forest that happens to be identical to your original system. Artificial Intelligence.
    - mumblemumble 5y ago
      If you talk enough all the models and hyperparameters you compared and suchlike that you experimented with, you can probably sufficiently impress people with the talk about the enormous deep learning model you spent several months developing that they won't even remember you mentioning the two-clause Boolean expression that you actually put into production. And of course it's AI. You used k-fold cross validation to select it.
- pedrocr 5y ago
  The AI taxonomy includes the term "Expert Systems" for these kinds of things. On the one hand it's definitely not of the new wave of ML AI so hyping those things as innovative is off. On the other hand we should definitely give more attention to that kind of setup and understand how to build/maintain/test it properly. Otherwise often it ends up being ran by a few hundred Excel sheets and a few severely underpaid people and that's a disaster waiting to happen. The AS400->VB6->NewShiny path actually sounds like a success case given the messes that are out there.
  - urthor 5y ago
    Rule engine is the term I believe. Often the biggest benefit is that the ML version is good at catching when the experts hadn't had their cup of coffee as well. Most experts are like family doctors, they get the correct diagnosis 70% of the time. And even if you juice them up real good, they will ALWAYS lose 5% to human error. The ML also hits the 70% mark, but it's a different 70%, so it'll fix 70% of the errors. Then you're batting at 0.91 instead of 0.70.
- thecopy 5y ago
  If it worked, why was it crap?
  - bryanrasmussen 5y ago
    VB6 is looked down upon.
    - deleted 5y ago
      [deleted]
    - moonchrome 5y ago
      It's been unsupported for 12+ years ? If you have code relying on it and haven't migrated to something supported it means your code is not maintained (don't care what your excuse is, using VB6 in 2020 means you're not actively maintaining the project), written 2 decades ago with the coding standards of the era, the original developer team is gone and probably retired and since nobody is actively maintaining it nobody has much knowledge about how it works. So yeah anything that's still running on VB6 is very likely crap.
      - wdb 5y ago
        There is RO Mercury?
      - bryanrasmussen 5y ago
        yeah it wasn't said when it was moved from though? The reference to Machine learning implies more modern, but not necessarily so. At any rate even when it was maintained still looked down on, I guess a Dijkstra based side-effect.
      - joshuaissac 5y ago
        > If you have code relying on it and haven't migrated to something supported it means your code is not maintained No, it does not. It means that Microsoft no longer provides support for the IDE. That does not prevent the developer from maintaining their own VB6 code. With some extra steps, the official IDE and compiler for VB6 can still be installed on Windows 10. Running programs built from VB6 is still supported. > written 2 decades ago with the coding standards of the era, the original developer team is gone and probably retired This applies regardless of the programming language to any codebase that has been around for long enough.
- np_tedious 5y ago
  Isn't your organization itself a machine the learned these rules over time? Maybe the marketing checks out
- knodi123 5y ago
  We did the same thing when I worked for a resume search/sort/share site. Built a big ML tool that could look at job listings and resumes and pick who was best for each job. Our training data set was millions of resumes, hundreds of thousands of jobs, and in most of those jobs, we could say which resumes got shortlisted and which resumes got hired. In the end, it gave basically the same results as keyword searching. But we marketed the shit out of it.
q-base 5y ago
That quote is seriously brilliant! Thanks for sharing.
DonHopkins 5y ago
Just don't build one solution to your problem with regular expressions: then you have two problems.
smichel17 5y ago
TFA ends with that quote.
bostonpete 5y ago
Well, the article does conclude with that exact tweet...
NumberCruncher 5y ago
The "the right tool for the right job" applies for ML topics too. If the job involves "looking smart and innovative" for whatever reasons, people tend to err on the side of overly complex solutions. On the other hand if the advice "let's just go with an SQL query built on a stack of egregiously oversimplifying assumptions" comes from someone, who doesn't know how SQL and linear regression / logistic regression with binning/bucketing / simple decision trees work, I would ask for a second opinion. Because a huge part of the retail banking, non-life insurance and marketing business is running on this simple stack. Obviously profitable. If the same advice comes from someone, who knows when to use deep learning instead of XGBoost and why, I would go with his/her advice. And I would try to keep him happy and on my team.
marcosdumay 5y ago
My workplace has got all kinds of attention for building a blockchain based data collection system that encompasses an entire sector of the economy. It's "almost done", so we are right now starting a simple set of REST services that write into a badly normalized transactional database just in case it stays "almost done" for too long.
arnaudsm 5y ago
This isn't ironic, I've actually done that multiple times in a large company. No one noticed, everyone went home happy.

arketyp 5y ago

Furthermore in the article, yes.

hughrr 5y ago

This reminds me of an experience I had watching a company trying to replace a system with ML. First they marketed it heavily before even thinking. During test cycle they fed the entire data corpus in and ran some of the original test cases and found some business destroying results pop out. The entire system ended up a verbatim port of the VB6 crap which was a verbatim port of the original AS400 crap that actually worked. The marketing to this day says it’s ML based and everyone buys into the hype. It’s not. It was a complete failure. But the original system has 30 years of human experience codified in it.

ethbr0 5y ago
If I had a nickel for every time I've seen "business rules engine" turned into "AI" in the last few years... But I guess if we complain that half of our colleagues and the media don't understand ML, why should we expect management to? When the command from C-level is "We need some AI projects to tell our shareholders about," we shouldn't be surprised when middle management suddenly has successful AI projects in their slide decks.
- mjburgess 5y ago
  Well really-existing AI is just either "taking a mean()" or "programming a rule". So all of programming actually counts as (symbolic) AI.
- cm2187 5y ago
  In the same way the terms “blockchain” being used for “digital signing” or “cloud” for a server...
- i_am_proteus 5y ago
  If you have an existing rules-based decision-tree system, and you compare its performance with a bunch of other decision trees, and it does better, you are implementing a random forest that happens to be identical to your original system. Artificial Intelligence.
  - mumblemumble 5y ago
    If you talk enough all the models and hyperparameters you compared and suchlike that you experimented with, you can probably sufficiently impress people with the talk about the enormous deep learning model you spent several months developing that they won't even remember you mentioning the two-clause Boolean expression that you actually put into production. And of course it's AI. You used k-fold cross validation to select it.
pedrocr 5y ago
The AI taxonomy includes the term "Expert Systems" for these kinds of things. On the one hand it's definitely not of the new wave of ML AI so hyping those things as innovative is off. On the other hand we should definitely give more attention to that kind of setup and understand how to build/maintain/test it properly. Otherwise often it ends up being ran by a few hundred Excel sheets and a few severely underpaid people and that's a disaster waiting to happen. The AS400->VB6->NewShiny path actually sounds like a success case given the messes that are out there.
- urthor 5y ago
  Rule engine is the term I believe. Often the biggest benefit is that the ML version is good at catching when the experts hadn't had their cup of coffee as well. Most experts are like family doctors, they get the correct diagnosis 70% of the time. And even if you juice them up real good, they will ALWAYS lose 5% to human error. The ML also hits the 70% mark, but it's a different 70%, so it'll fix 70% of the errors. Then you're batting at 0.91 instead of 0.70.
thecopy 5y ago
If it worked, why was it crap?
- bryanrasmussen 5y ago
  VB6 is looked down upon.
  - deleted 5y ago
    [deleted]
  - moonchrome 5y ago
    It's been unsupported for 12+ years ? If you have code relying on it and haven't migrated to something supported it means your code is not maintained (don't care what your excuse is, using VB6 in 2020 means you're not actively maintaining the project), written 2 decades ago with the coding standards of the era, the original developer team is gone and probably retired and since nobody is actively maintaining it nobody has much knowledge about how it works. So yeah anything that's still running on VB6 is very likely crap.
    - wdb 5y ago
      There is RO Mercury?
    - bryanrasmussen 5y ago
      yeah it wasn't said when it was moved from though? The reference to Machine learning implies more modern, but not necessarily so. At any rate even when it was maintained still looked down on, I guess a Dijkstra based side-effect.
    - joshuaissac 5y ago
      > If you have code relying on it and haven't migrated to something supported it means your code is not maintained No, it does not. It means that Microsoft no longer provides support for the IDE. That does not prevent the developer from maintaining their own VB6 code. With some extra steps, the official IDE and compiler for VB6 can still be installed on Windows 10. Running programs built from VB6 is still supported. > written 2 decades ago with the coding standards of the era, the original developer team is gone and probably retired This applies regardless of the programming language to any codebase that has been around for long enough.
np_tedious 5y ago
Isn't your organization itself a machine the learned these rules over time? Maybe the marketing checks out
knodi123 5y ago
We did the same thing when I worked for a resume search/sort/share site. Built a big ML tool that could look at job listings and resumes and pick who was best for each job. Our training data set was millions of resumes, hundreds of thousands of jobs, and in most of those jobs, we could say which resumes got shortlisted and which resumes got hired. In the end, it gave basically the same results as keyword searching. But we marketed the shit out of it.

q-base 5y ago

That quote is seriously brilliant! Thanks for sharing.

DonHopkins 5y ago

Just don't build one solution to your problem with regular expressions: then you have two problems.

smichel17 5y ago

TFA ends with that quote.

bostonpete 5y ago

Well, the article does conclude with that exact tweet...

NumberCruncher 5y ago

The "the right tool for the right job" applies for ML topics too. If the job involves "looking smart and innovative" for whatever reasons, people tend to err on the side of overly complex solutions. On the other hand if the advice "let's just go with an SQL query built on a stack of egregiously oversimplifying assumptions" comes from someone, who doesn't know how SQL and linear regression / logistic regression with binning/bucketing / simple decision trees work, I would ask for a second opinion. Because a huge part of the retail banking, non-life insurance and marketing business is running on this simple stack. Obviously profitable. If the same advice comes from someone, who knows when to use deep learning instead of XGBoost and why, I would go with his/her advice. And I would try to keep him happy and on my team.

marcosdumay 5y ago

My workplace has got all kinds of attention for building a blockchain based data collection system that encompasses an entire sector of the economy. It's "almost done", so we are right now starting a simple set of REST services that write into a badly normalized transactional database just in case it stays "almost done" for too long.

arnaudsm 5y ago

This isn't ironic, I've actually done that multiple times in a large company. No one noticed, everyone went home happy.