2 ms·

"I could probably fit all well-structured data elements in existence for all medical patients in the US for a year on a single DVD." As someone who builds and

by bigwaff 13y ago

"I could probably fit all well-structured data elements in existence for all medical patients in the US for a year on a single DVD." As someone who builds and supports a massive "big data" healthcare platform, I have to say you are horribly mistaken. Non-image well structured patient data, even if compressed, across all healthcare, with potentially multiple EMRs and data collection systems per healthcare organization, even if just the last years' worth of data, is "big data" and can benefit from "big data" techniques. Like all "big data" endeavors across any number of problem domains, health care application of "big data" is in its infancy. Will the use of "big data" techniques over patient data discover disease cures? I don't know. But there is gold in them there hills... what kind? We will have to wait to find out.

drcode 13y ago
I agree I might be wrong, but let me try to do the math (so you can improve it with your knowledge.) Let's say 10 million hospitalized patients a year, 100 data points entered by a nurse, another 300 from lab tests during the visit, 50 entered by a doctor, 50 entered by a pharmacist. Let's postulate 8 bytes per data element (most are numeric or ordinal.) This leads to: 10,000,000x8x500=4E10 bytes=40GB Compressed, this would fit on a DVD (8.5 GB) (I agree the numbers are somewhat larger if unstructured text is added to the mix, but it still seems to fall short of the terabytes of data I would expect for a "big data" approach.) Usually I thought the "big data" moniker is applied to web sites where every minuscule site interaction is audited from every user across 10mil+ users, leading to much larger data sets. Or, it is used in the context of crawling the entire web where many many petabytes of data become available. You might have better knowledge of the numbers involved and can correct me, please do.

by bigwaff 13y ago

drcode 13y ago

I agree I might be wrong, but let me try to do the math (so you can improve it with your knowledge.) Let's say 10 million hospitalized patients a year, 100 data points entered by a nurse, another 300 from lab tests during the visit, 50 entered by a doctor, 50 entered by a pharmacist. Let's postulate 8 bytes per data element (most are numeric or ordinal.) This leads to: 10,000,000x8x500=4E10 bytes=40GB Compressed, this would fit on a DVD (8.5 GB) (I agree the numbers are somewhat larger if unstructured text is added to the mix, but it still seems to fall short of the terabytes of data I would expect for a "big data" approach.) Usually I thought the "big data" moniker is applied to web sites where every minuscule site interaction is audited from every user across 10mil+ users, leading to much larger data sets. Or, it is used in the context of crawling the entire web where many many petabytes of data become available. You might have better knowledge of the numbers involved and can correct me, please do.