4 ms·

So my thinking on this is that the way these NoSQL systems will make their way into Healthcare is through the great backdoor of analytics. Virtually all availab

So my thinking on this is that the way these NoSQL systems will make their way into Healthcare is through the great backdoor of analytics. Virtually all available EHR/EMR systems available today allow various mechanisms for data retrieval based on primary indices. Unfortunately that avenue does not lend itself to secondary data reuse which will become more and more valuable as Institutions realize their data is valuable not only for academic research but for operational efficiencies re. money in the kitty. Even more unfortunate still is that virtually all these installed systems do not have the performance capacity or advanced search-ability to adequately mine this growing horde of data. Administrators, under capex constraints, do not allocate resources for secondary systems which would duplicate data for mining purposes while alleviating strain on the principle production system. The no money budget problem will lead in house programmers to build these research systems on top of open source NoSQL solutions. There, the technology will prove itself. Additionally, "NoSQL" comes in different flavors. Generally, all of them forgo the Consistency in CAP for Availability and Partition tolerance, which is fine for many use cases - just not primary medical data acquisition use cases. As the field matures programmers and system designers will learn how to make this work better to the point where one day NoSQL systems may be used as the primary data repository for medical data. However, that day has not come. For instance, Riak allows you to tweak knobs in order to favor of certain aspects of CAP theorem at different times while in production (specifically the w and dw parameters, http://blog.basho.com/2010/03/19/schema-design-in-riak---introduction/ http://blog.basho.com/2010/03/19/schema-design-in-riak---int...). But having just started working with Riak in the last month or two I would still only use it as an analytics tool exposing my medical record data to m/r jobs at this point. And before jbellis smacks me, I think Cassandra is awesome and I'm looking forward to spending some time with it but I'm still not putting my med app data in Casssandra just yet as a primary data store. /Disclaimer. I work for a major University Medical Center and write business web applications in this area./

weichi 17y ago
How do NoSQL databases allow you to do a better job at "secondary data reuse" than a traditional RDBMS? Is it just a matter of (1) Denormalization (2) performance gains from not needing to worry about transactions?
- siculars 17y ago
  The main issue from my perspective - beyond performance - is flexibility of the NoSQL model. Specifically the document store concept. The main analytics player is the data warehouse currently which has served very well over the years but comes with certain rigidity mainly in programming and cost. Of course, not to detract from their success, OLAP data warehousing is very mature with a rich toolbase. The flexibility of the NoSQL model coupled with the exposure of your data to the m/r paradigm is the big win from my vantage. Almost every NoSQL solution will expose your data via bindings in virtually every programming language allowing almost any programmer to leverage NoSQL. Now you kinda have to have experience with data warehousing. Cost wise, what you would spend in licensing can be invested in hardware but more specifically talent instead. Secondary data reuse as a concept is highly unstructured. Sure your primary systems capture data in specific formats but your analysis can take you in all kinds of directions. Being able to slice and dice without having to pre-define will be huge.
chintan 17y ago
Secondary data reuse/analytics requires more like "MapReduce" functionality rather than "NoSQL" stuff.
JunkDNA 17y ago
As a developer of a small, in-house specialized data warehouse for medical research purposes (built with 100% open source), I would be curious to hear more thoughts on this. The flexibility angle is certainly a huge win. My issue is that I can't quite get my head around how one runs ad-hoc queries like "give me everyone with a bmi >x and age between a and b". In the key/value view of the world, that doesn't quite fit.
- egroen 17y ago
  Take a look at the revolutionary illuminate Correlation database - ultimate flexibility for "ad-hoc" queries The Correlation database is a NoSQL database management system (DBMS) that is data model independent and designed to efficiently handle unplanned, ad hoc queries in an analytical system environment. Unlike relational database management systems or column-oriented databases, a correlation database uses a value-based storage (VBS) architecture in which each unique data value is stored only once and an auto-generated indexing system maintains the context for all values. Queries are performed using natural language instead of SQL. Learn more at: www.datainnovationsgroup.com

weichi 17y ago

How do NoSQL databases allow you to do a better job at "secondary data reuse" than a traditional RDBMS? Is it just a matter of (1) Denormalization (2) performance gains from not needing to worry about transactions?

siculars 17y ago
The main issue from my perspective - beyond performance - is flexibility of the NoSQL model. Specifically the document store concept. The main analytics player is the data warehouse currently which has served very well over the years but comes with certain rigidity mainly in programming and cost. Of course, not to detract from their success, OLAP data warehousing is very mature with a rich toolbase. The flexibility of the NoSQL model coupled with the exposure of your data to the m/r paradigm is the big win from my vantage. Almost every NoSQL solution will expose your data via bindings in virtually every programming language allowing almost any programmer to leverage NoSQL. Now you kinda have to have experience with data warehousing. Cost wise, what you would spend in licensing can be invested in hardware but more specifically talent instead. Secondary data reuse as a concept is highly unstructured. Sure your primary systems capture data in specific formats but your analysis can take you in all kinds of directions. Being able to slice and dice without having to pre-define will be huge.

chintan 17y ago

Secondary data reuse/analytics requires more like "MapReduce" functionality rather than "NoSQL" stuff.

JunkDNA 17y ago

As a developer of a small, in-house specialized data warehouse for medical research purposes (built with 100% open source), I would be curious to hear more thoughts on this. The flexibility angle is certainly a huge win. My issue is that I can't quite get my head around how one runs ad-hoc queries like "give me everyone with a bmi >x and age between a and b". In the key/value view of the world, that doesn't quite fit.

egroen 17y ago
Take a look at the revolutionary illuminate Correlation database - ultimate flexibility for "ad-hoc" queries The Correlation database is a NoSQL database management system (DBMS) that is data model independent and designed to efficiently handle unplanned, ad hoc queries in an analytical system environment. Unlike relational database management systems or column-oriented databases, a correlation database uses a value-based storage (VBS) architecture in which each unique data value is stored only once and an auto-generated indexing system maintains the context for all values. Queries are performed using natural language instead of SQL. Learn more at: www.datainnovationsgroup.com