4 ms·
I work for an average sized Danish city. Even when we combined our datasets with 10 other average sized cities and accumulate the biggest national data set and
by moksly 5y ago
I work for an average sized Danish city. Even when we combined our datasets with 10 other average sized cities and accumulate the biggest national data set and let the big data wizards have a go at it, it wasn’t really large enough to amount to anything useful.
Don’t get me wrong, Watson came up with a lot of BI suggestions that were useful and some of the scientists came up with semi-interesting prediction models. The thing is though, our analytics team has much better BI models and the our finance department does the prediction much more efficient and, well, legal. The prediction could become a useful aid, if it ever became legal, but not really at the license fees we were looking at.
Not sure if automated BI is ever going to become good enough. It’s impressive that Watson can come up with stuff an university post-bachelor intern can, but it’s license is more than our entire analytics team, so yeah...
- Scarblac 5y agoI work in water management in the Netherlands and it feels the same. A human domain expert knows what kind of thing to look for. Custom tools can help to get an overview (eg we visualize water velocities of a whole regional system on a map), but generic machine learning can't add much even though we record basically everything everywhere at five minute intervals. The value of ML is perhaps in things that give little value in the individual case but can be used a huge number of times, like image classification.
- pbhjpbhj 5y agoHuman directed machine learning then? Presumably a domain expert who can also expertly leverage ML/AI could make some useful tools?
- moksly 5y agoThere are no such people though.
- bostik 5y agoI can give one example where the ML/AI toolkit is useful, even if only the largest providers in the world may need to work with real Big Data[tm] volumes. Fraud detection. And you don't need to be a particularly big operator to get sufficiently large & varied data sets. In a way I think it helps that fraud is an ever-evolving, fiercely adversarial domain. New avenues are explored all the time. Occasionally old tricks are revived for a while, because they may work in the margins but become distinguishing features as soon as they see more use. And even if your ML is based on nothing more than a random forest, you can still get surprisingly useful feature combinations out of it. (Or as our data scientists said: individually meaningless features may become a valuable signal when enough of them occur at the same time.)