5 ms·

I'm currently evaluating BI vendors for my company and just about every major contender has this functionality or is developing it. Read the Gartner report fro

I'm currently evaluating BI vendors for my company and just about every major contender has this functionality or is developing it. Read the Gartner report from 2017 and they basically just come out and say that this will be the new standard in BI. The much harder problem is data management and preparation. Anyone with half a brain and a decent visualization tool can create basic graphs - but that doesn't mean they should, especially if the organization doesn't have good data management processes in place. Issues like data governance, data prep, and data modelling are the major pain points for me. And honestly, developing a useful BI solution is more about culture change than it is technology. If a company has poor data governance, it doesn't matter how whiz-bang their technology is, they're still not going to get useful insight from their numbers.

ozataman 9y ago
And data management / preparation is where major mistakes are made - get a join wrong and you may easily be missing data or double counting something just obscure enough to go unnoticed like "ancillary sales". There is something potentially harmful, or perhaps that needs addressing, about end-user tools growing in expressive power. A good friend who does statistical genetics work once told me "but I don't want every user running their own regressions and drawing nonsensical conclusions from badly prepared data!"
- gefh 9y ago
  That ship sailed somewhere around Excel 95...
- shostack 9y ago
  Yep. Wake me when an AI can tell me "hey, that monthly trending conversion report you asked me to pull...yeah, you're missing two days worth of data when tracking broke, so it will just make your numbers look lower when rolled-up monthly and be hard to notice." BTW, that is also the reason I not only set alerts, but review data at a daily level when pulling any rolled-up reports of significance.
  - kyleschiller 9y ago
    Why can't AI tell you you're missing data?
    - massaman_yams 9y ago
      For most use cases, you don't even need AI. Once you reach a certain scale, there are certainly ML-based solutions available: https://medium.com/netflix-techblog/rad-outlier-detection-on-big-data-d6b0494371cc https://medium.com/netflix-techblog/rad-outlier-detection-on...
temp246810 9y ago
Just out of curiousity, which one did you go with? We're in the middle of doing this ourselves and would like to hear a random HNers thoughts on the subject.
- paulrosenzweig 9y ago
  I'm at a ~100 person company. We've been using Mode for about six months and like it.
primeblue 9y ago
Microsoft PowerBI

ozataman 9y ago

And data management / preparation is where major mistakes are made - get a join wrong and you may easily be missing data or double counting something just obscure enough to go unnoticed like "ancillary sales". There is something potentially harmful, or perhaps that needs addressing, about end-user tools growing in expressive power. A good friend who does statistical genetics work once told me "but I don't want every user running their own regressions and drawing nonsensical conclusions from badly prepared data!"

gefh 9y ago
That ship sailed somewhere around Excel 95...
shostack 9y ago
Yep. Wake me when an AI can tell me "hey, that monthly trending conversion report you asked me to pull...yeah, you're missing two days worth of data when tracking broke, so it will just make your numbers look lower when rolled-up monthly and be hard to notice." BTW, that is also the reason I not only set alerts, but review data at a daily level when pulling any rolled-up reports of significance.
- kyleschiller 9y ago
  Why can't AI tell you you're missing data?
  - massaman_yams 9y ago
    For most use cases, you don't even need AI. Once you reach a certain scale, there are certainly ML-based solutions available: https://medium.com/netflix-techblog/rad-outlier-detection-on-big-data-d6b0494371cc https://medium.com/netflix-techblog/rad-outlier-detection-on...

temp246810 9y ago

Just out of curiousity, which one did you go with? We're in the middle of doing this ourselves and would like to hear a random HNers thoughts on the subject.

paulrosenzweig 9y ago
I'm at a ~100 person company. We've been using Mode for about six months and like it.

primeblue 9y ago

Microsoft PowerBI