6 ms·

Wonder if anyone has ever thought about building a SQL server directly in a hard drive?

zokier 11y ago
I think more interesting question would be if anyone has build practical userland on top of SQL (or any reasonably rich database for that matter). Here is some discussion about SQL on raw devices: http://dba.stackexchange.com/questions/80036/is-there-a-way-to-store-a-postgresql-database-directly-on-a-block-device-not-fi http://dba.stackexchange.com/questions/80036/is-there-a-way-...
- protomyth 11y ago
  The Newton skipped SQL but had an object database with queries.
scurvy 11y ago
There are a few drives out there which have a k/v interface as opposed to block interface. Seagate Kinetics come to mind: http://www.seagate.com/tech-insights/kinetic-vision-how-seagate-new-developer-tools-meets-the-needs-of-cloud-storage-platforms-master-ti/ http://www.seagate.com/tech-insights/kinetic-vision-how-seag...
kibibu 11y ago
https://en.wikipedia.org/wiki/WinFS https://en.wikipedia.org/wiki/WinFS edit: apologies, you meant on the controller presumably
fredkbloggs 11y ago
Because what everyone wants is more invisible closed-source software running on underpowered CPUs? Not to mention the limitations on performance, capacity, and reliability scaling that would be inherent in binding the database instance to a single disk device. Or were you suggesting that each of these tiny controllers running deeply proprietary software should also form a distributed database in concert with host software or HBA controller firmware? To say that this is not a good idea would be putting it mildly. The existing efforts to build more functionality into disk drives, starting with FDE and now with some object-store-like interfaces, sounds real appealing at first. Less software to write, functionality everywhere, disks seem to Just Work today so wouldn't it be nice if they could Just Work in some more ways too. However, as soon as you start thinking about building something larger or more interesting than a toy, it becomes apparent that the disk drive is the wrong place for this. The knowledge of the problem is not present at that level of the system, and the interfaces and processing power are inadequate to express it. The problems of error recovery, scaling, and debuggability cannot be solved there. It might be useful for a few of the very smallest consumer-grade applications where none of these concerns are significant (nor likely to be solved by a higher-level vendor anyway), but it's not generally viable.
jhayward 11y ago
One of the big log search startups was doing postgres query engine integration on drive controllers at least 10 years ago. (sorry, don't recall which) And as far back as 1964 IBM was doing K-V in hardware on the drive in their count-key-data devices. [1] [1] https://en.wikipedia.org/wiki/Count_key_data https://en.wikipedia.org/wiki/Count_key_data

I think more interesting question would be if anyone has build practical userland on top of SQL (or any reasonably rich database for that matter). Here is some discussion about SQL on raw devices: http://dba.stackexchange.com/questions/80036/is-there-a-way-to-store-a-postgresql-database-directly-on-a-block-device-not-fi http://dba.stackexchange.com/questions/80036/is-there-a-way-...

protomyth 11y ago
The Newton skipped SQL but had an object database with queries.

scurvy 11y ago

There are a few drives out there which have a k/v interface as opposed to block interface. Seagate Kinetics come to mind: http://www.seagate.com/tech-insights/kinetic-vision-how-seagate-new-developer-tools-meets-the-needs-of-cloud-storage-platforms-master-ti/ http://www.seagate.com/tech-insights/kinetic-vision-how-seag...

kibibu 11y ago

https://en.wikipedia.org/wiki/WinFS https://en.wikipedia.org/wiki/WinFS edit: apologies, you meant on the controller presumably

fredkbloggs 11y ago

Because what everyone wants is more invisible closed-source software running on underpowered CPUs? Not to mention the limitations on performance, capacity, and reliability scaling that would be inherent in binding the database instance to a single disk device. Or were you suggesting that each of these tiny controllers running deeply proprietary software should also form a distributed database in concert with host software or HBA controller firmware? To say that this is not a good idea would be putting it mildly. The existing efforts to build more functionality into disk drives, starting with FDE and now with some object-store-like interfaces, sounds real appealing at first. Less software to write, functionality everywhere, disks seem to Just Work today so wouldn't it be nice if they could Just Work in some more ways too. However, as soon as you start thinking about building something larger or more interesting than a toy, it becomes apparent that the disk drive is the wrong place for this. The knowledge of the problem is not present at that level of the system, and the interfaces and processing power are inadequate to express it. The problems of error recovery, scaling, and debuggability cannot be solved there. It might be useful for a few of the very smallest consumer-grade applications where none of these concerns are significant (nor likely to be solved by a higher-level vendor anyway), but it's not generally viable.

jhayward 11y ago

One of the big log search startups was doing postgres query engine integration on drive controllers at least 10 years ago. (sorry, don't recall which) And as far back as 1964 IBM was doing K-V in hardware on the drive in their count-key-data devices. [1] [1] https://en.wikipedia.org/wiki/Count_key_data https://en.wikipedia.org/wiki/Count_key_data