5 ms·

Seems to me that a unikernel database should be the first application. Databases tend to bypass practically all the facilities of a kernel anyway. It’s often su

Seems to me that a unikernel database should be the first application. Databases tend to bypass practically all the facilities of a kernel anyway. It’s often surprised me that they haven’t merged before now.

deleted 6y ago
[deleted]
gwillz 6y ago
https://twitter.com/cpswan/status/971335126566211585?s=20 https://twitter.com/cpswan/status/971335126566211585?s=20 Maybe not exactly what you're thinking. But somewhat in the same domain.
fcantournet 6y ago
They also tend to need to be debugged & administered.
eyberg 6y ago
There is plenty of work in this area. The team behind OSv ending up writing https://www.scylladb.com/ https://www.scylladb.com/ because of their earlier interest in databases.
- wmf 6y ago
  So you're saying unikernel experts ended up writing a non-unikernel database. There's a lesson right there.
  - eyberg 6y ago
    I wouldn't draw that conclusion. A) Taking vc money sometimes points you at different types of businesses that you are urged to explore more. B) Just cause you can run a database as a unikernel (and you can) doesn't mean the people who want the other database wants to deploy it as one. In my experience those who buy managed databases generally aren't wanting to do any system administration at all.
nick_kline 6y ago
I think this is the paper https://www.cs.bu.edu/~jappavoo/Resources/Papers/unikernel-hotos19.pdf https://www.cs.bu.edu/~jappavoo/Resources/Papers/unikernel-h.... There's a great summary by cormacrelf below of some of the details. This kind of things comes and goes across the decades. Back in the 90s it was common for databases to implement their own file systems or memory management, but gradually the OSs of the day added features make this unnecessary. As we found then, you lose so much flexibility to modify the os via config changes, adding new monitoring or whatever if you had the db directly connected to the os. I'd hate to give that up for more efficiency. There's a lot of 'strength' in an OS separating various things from the database itself, giving that flexibility. A modern DB needs efficient interaction with the OS of course, but I'd say based on experience at many companies a bigger challenge than raw efficiency is the ability to implement changes in the db itself. I've worked on 4 or 5 major databases and we can always identify many ways to improve execution, plan selection, various aspects of the db, it's just the giant challenge to alter a big code base that blocks improvements more than OS layers. Improving plan choice can make queries thousands of times faster - but you have to be able to implement it.
- DrScump 6y ago
  Back in the 90s it was common for databases to implement their own file systems or memory management Informix in the mid/late 1980s initially had Shared Memory implementations in UNIX platforms that supported it. Then, the Turbo/OnLine/IDS servers added the option of raw disk partition use for database spaces to avoid filesystems altogether (and do raw unbuffered I/O).
smabie 6y ago
kOS (k language as a standalone OS) was a project but I guess it got canned. Unfortunate, I would love to have tried out a standalone kdb+/q installation.

deleted 6y ago

[deleted]

gwillz 6y ago

https://twitter.com/cpswan/status/971335126566211585?s=20 https://twitter.com/cpswan/status/971335126566211585?s=20 Maybe not exactly what you're thinking. But somewhat in the same domain.

fcantournet 6y ago

They also tend to need to be debugged & administered.

eyberg 6y ago

There is plenty of work in this area. The team behind OSv ending up writing https://www.scylladb.com/ https://www.scylladb.com/ because of their earlier interest in databases.

wmf 6y ago
So you're saying unikernel experts ended up writing a non-unikernel database. There's a lesson right there.
- eyberg 6y ago
  I wouldn't draw that conclusion. A) Taking vc money sometimes points you at different types of businesses that you are urged to explore more. B) Just cause you can run a database as a unikernel (and you can) doesn't mean the people who want the other database wants to deploy it as one. In my experience those who buy managed databases generally aren't wanting to do any system administration at all.

nick_kline 6y ago

I think this is the paper https://www.cs.bu.edu/~jappavoo/Resources/Papers/unikernel-hotos19.pdf https://www.cs.bu.edu/~jappavoo/Resources/Papers/unikernel-h.... There's a great summary by cormacrelf below of some of the details. This kind of things comes and goes across the decades. Back in the 90s it was common for databases to implement their own file systems or memory management, but gradually the OSs of the day added features make this unnecessary. As we found then, you lose so much flexibility to modify the os via config changes, adding new monitoring or whatever if you had the db directly connected to the os. I'd hate to give that up for more efficiency. There's a lot of 'strength' in an OS separating various things from the database itself, giving that flexibility. A modern DB needs efficient interaction with the OS of course, but I'd say based on experience at many companies a bigger challenge than raw efficiency is the ability to implement changes in the db itself. I've worked on 4 or 5 major databases and we can always identify many ways to improve execution, plan selection, various aspects of the db, it's just the giant challenge to alter a big code base that blocks improvements more than OS layers. Improving plan choice can make queries thousands of times faster - but you have to be able to implement it.

DrScump 6y ago
Back in the 90s it was common for databases to implement their own file systems or memory management Informix in the mid/late 1980s initially had Shared Memory implementations in UNIX platforms that supported it. Then, the Turbo/OnLine/IDS servers added the option of raw disk partition use for database spaces to avoid filesystems altogether (and do raw unbuffered I/O).

smabie 6y ago

kOS (k language as a standalone OS) was a project but I guess it got canned. Unfortunate, I would love to have tried out a standalone kdb+/q installation.