3 ms·
Big data is all the rage, but as an enterprise biz dev guy I hear a lot of talk about it and not a lot of real doing. The article states accurately that people
by apapli 14y ago
Big data is all the rage, but as an enterprise biz dev guy I hear a lot of talk about it and not a lot of real doing.
The article states accurately that people can store huge amounts of data. What I don't think it explored sufficiently is how companies can make sense of this.
Big data gives companies the ability to find new insights into relationships between data points they had no idea existed. From what I am seeing, people are however approaching the issue the same (wrong) way - they are running the same reports on larger data sets, or running new reports to confirm/disprove areas where they suspect relationships will exist.
What big data needs is a generally available product (read: $$ thousands per year, not hundreds of thousands or millions) that allows companies to send in their data sets, with it proactively mining to find new trends across is entirety.
I don't think this product exists yet - and if it does please tell me and I'll rapidly call them for a job - and is exactly where the big money will be.
- oscilloscope 14y agoThe closest thing I can think of is OpenRefine. It doesn't directly find trends, but it does do machine learning to help you clean up datasets. You can also create a scatterplot matrix and do faceted search, which lets you quickly explore data and relationships between dimensions. It's an open source program (previously Google Refine). http://openrefine.org/ http://openrefine.org/ For health care, finance, enterprises and government, data quality is a huge issue. Many machine learning and statistical methods just don't work or produce misleading results with messy data.
- aidos 14y agoAhhh, so that's where Google Refine went to. I was trying to find it recently. Thanks for the link.
- me_bx 14y agoYou might also be interested in recline.js (http://okfnlabs.org/recline/ http://okfnlabs.org/recline/), a fork of Google refine, initially created by @maxogden, now maintained by the Open Knowledge Foundation (okfn). How their data processing features differ from openRefine is not clear to me though...
- me_bx 14y agoYou have quite a few players that do this, challenging traditional "corporate business intelligence" solutions with more affordable, innovative and powerful alternatives. Some serious ones are: - Good Data http://www.gooddata.com/ http://www.gooddata.com/ - BiMe http://www.bimeanalytics.com/ http://www.bimeanalytics.com/ - Tableau software http://www.tableausoftware.com/ http://www.tableausoftware.com/ - QlikView http://www.qlikview.com/ http://www.qlikview.com/ (not sure it still fits in this category though) (edit: post formatting)