7 ms·

Are we sure edge computing providers have io_uring enabled? It is disabled in inter alia, ChromeOS and Android, because it's been a significant source of vulner

Are we sure edge computing providers have io_uring enabled? It is disabled in inter alia, ChromeOS and Android, because it's been a significant source of vulnerabilities. Seems deadly in a multi tenant environment.

ncruces 2y ago
Their goal is to run this on their own cloud. Despite their lofty claims about community building, their projects are very much about forwarding their use case. Given that SQLite is public domain, they're not required to give anything back. So, it's very cool that they're making parts of their tech FOSS. But I've yet to see anything coming from them that isn't “just because we need it, and SQLite wouldn't do it for us.” There's little concern about making things useful to others, and very little community consensus about any of it. https://turso.tech/ https://turso.tech/
- tracker1 2y ago
  Of course they are scratching their own itch, so to speak. Thats what companies do. I think the fact that they are doing so much in the open is the indication of good stewardship itself. I'm not sure what else they would do or release that they didn't need internally. For that matter, I'm not really aware of many significant contributions to FLOSS at all that aren't initially intended for company use, that's kinda how it works. Where I'm surprised here is how much secret sauce Turso is sharing at all.
  - ncruces 2y ago
    I have no problem with them scratching their itch. That's par for the course. I'm salty about them describing the SQLite licensing, development model, and code of ethics as almost toxic, setting up a separate entity with a website and a manifesto promising to do better, and then folding “libSQL into the Turso family” within the year. They forked, played politics, added a few features (with some ill-considered incompatibilities), and properly documented zero of them. And I'm salty because I'm actually interested in some of those features, and they're impossible to use without proper documentation. I've had much better luck interacting with SQLite developers in the SQLite forum.
    - avinassh 2y ago
      disclosure: I work at Turso > code of ethics as almost toxic This is simply not true. Can you tell me where it is being said so? > then folding “libSQL into the Turso family” within the year. libSQL was always part of Turso. So, I don't get your point. > They forked, played politics, added a few features (with some ill-considered incompatibilities), and properly documented zero of them. Again this is incorrect. There are some docs here: https://github.com/tursodatabase/libsql/tree/main/docs https://github.com/tursodatabase/libsql/tree/main/docs I am really not sure why are you so angry about libSQL.
      - ncruces 2y ago
        > This is simply not true. Can you tell me where it is being said so? It's right there in “your” manifesto. https://turso.tech/libsql-manifesto https://turso.tech/libsql-manifesto > We take our code of conduct seriously, and unlike SQLite, we do not substitute it with an unclear alternative. We strive to foster a community that values diversity, equity, and inclusion. We encourage others to speak up if they feel uncomfortable. The word toxic clearly stung, but putting “unlike SQLite … we encourage others to speak up if they feel uncomfortable” in a manifesto is fine. Well, I could argue I'm just speaking up. > libSQL was always part of Turso. So, I don't get your point. My point is explained quite clearly in your post detailing the decision. https://turso.tech/blog/were-bringing-libsql-into-the-turso-family-8cc1a653448e https://turso.tech/blog/were-bringing-libsql-into-the-turso-... > We have our own self interest in making those changes (…) But we also wanted to create a welcoming community, that is open to everybody, abides by a modern code of conduct and a clear OSS license, and reimagined what SQLite could be in broader ways than just our narrow needs. A little latter down that line you sum it up: doing the above (living up to your grandiose claims of a more welcoming SQLite) “meant twice the investment” (aka a lot of money) and didn't pan out as a marketing play (showed engagement). So instead of a community that “reimagined what SQLite could be in broader ways than just our narrow needs" we just get the features you had your "own self interest in making." Which is fine, but doesn't really match the manifesto. > Again this is incorrect. There are some docs… Emphasis on some. Do you have any documentation on how to build on the Virtual WAL (internal SQLite API that you simply opened up)? Or is that's still a Rust example of an implementation that simply wraps another and logs without detailing anything beyond function names? Do you have any documentation about the new WAL API that isn't "libsql_wal_insert_begin begins WAL insertion"? I'm sorry, but goal here isn't to make things useful to others. Which is fine really: you're doing more than you're required. But compared to SQLite developers, and their forum, it's not much. PS: you also behaved… untowardly when you integrated SQLite3MultipleCiphers, and did this with not previous a word to the author. https://turso.tech/blog/fully-open-source-encryption-for-sqlite-b3858225 https://turso.tech/blog/fully-open-source-encryption-for-sql... > One project in particular was very suitable for us, SQLite Multiple Ciphers. Since it is licensed under MIT, we have just moved the code into libSQL.
- nindalf 2y ago
  I think you've taken the most cynical view possible. SQLite is open source but not open contribution. So if they "won't do it for us" and "we need it", what else are they supposed to do? They're offering SQLite in the cloud, obviously they need to offer a streaming backup solution. Is there something wrong in building that? Alternatively, do you want them to reimplement features in SQLite already built out by the SQLite team? Really sounds like you're complaining about getting an open source, MIT licensed, open contribution bit of software for free that you're under no obligation to use. And here you are complaining about "community consensus".
  - refulgentis 2y ago
    I may be able to shed some light. It seems they proposed a way to resolve the contradiction I raised (io_uring isn't supported on cloud providers) with the ground reality in the company's blog post. Namely, that: * it is intended for edge function deployment. * the paper they mention that is informing their decision to rewrite is based on perf improvements in the longtail by using io_uring. I framed it as "Are we sure...?", but truth is, I know providers don't have io_uring support. This is relatively straightforward to derive from edge functions are well-known to be multi-tenant. A cursory search shows unresolved AWS tickets re: this, and multiple announcements from Google about how it has been disabled in gCloud. Thus, it is likely they understood I was framing it politely, and that there was a clear contradiction here, hence their reply to me, raising a potential resolution for that contradiction, a resolution I hadn't considered. I don't see anything complaining, or bringing up, or implicitly denying, all the other stuff ("open source", "MIT licensed", "open contribution", "software for free" "under no obligation to use."). In fact, they explicitly indicated they completely agree with that view ("it's very cool that they're making parts of their tech FOSS.")
    - nindalf 2y ago
      io_uring isn't supported on cloud providers for now, but it may not be the case in future. Which is fine, because Limbo won't be ready for a few years at least.
    - threecheese 2y ago
chambers 2y ago
Not an expert at all: I wonder if getting the perf requires trading-off some (advertised) safety. IO uring has been noted to be confusing with async in Rust, https://news.ycombinator.com/item?id=41992975 https://news.ycombinator.com/item?id=41992975 I'm reminded of how Confluent advertised Kafka as a database. They quietly externalized key guarantees of an RDBMS onto their customers, who were then saddled with implementing those guarantees in application level logic. By obscuring the trade-offs, Confluent made developers feel they could have their cake and eat it too.
- toast0 2y ago
  IMHO, most of io_uring's performance should come from reducing transition between kernel and userspace. There doesn't need to be a safety tradeoff there (although in practice, there have been several safety issues). There may be a trade off against having a simple concurrency model; without io_uring you can't really request an accept and not handle the response that comes back, because a syscall is necessarily synchronous from the point of view of the caller; the calling thread can't continue until it gets a response, even if that's EINTR.
  - refulgentis 2y ago
    Out of my league / knowledge, but a tidbit that you might understand better: last time I mentioned this, someone said something about SELinux can't do security checks and implied it's some fundamental mismatch, rather than some work SELinux can do
    - toast0 2y ago
      Well I'm a little outside my league on SELinux, but as I understand it, SELinux a way to limit syscalls by configuration. io_uring operations are similar to syscalls, but not exactly the same thing, so if I've read up correctly, I think SELinux originally may not have covered operations --- if the config allowed you to use io_uring, you could bypass SELinux restrictions, and the easiest/safest thing to do for that is to restrict io_uring. I think this may have been resolved, it looks like there was work on SELinux controls for io_uring back in 2021 [1] that looks like it got merged, but I haven't really followed this. There's also the issue that what happened in Linus's tree 2021 doesn't necessarily reflect what's in common kernels in 2024; some distributions are built from ancient branches. Based on the language in the pull request, I would think you'd end up needing to have near duplicate policy entries for regular syscalls and io_uring operations, which sounds tedious (but the whole SELinux thing sounds tedious to me, so I'm maybe not the right person to ask :D ) [1] https://lore.kernel.org/lkml/CAHC9VhRJ=fHzMHM6tt8JqkZa4bf0h72CAytSX9YrEs14Oaj8SA@mail.gmail.com/ https://lore.kernel.org/lkml/CAHC9VhRJ=fHzMHM6tt8JqkZa4bf0h7...
eyberg 2y ago
Anything new is going to have people poking and prodding at it. It doesn't mean that the concept is 'bad'. Linux has desperately needed an interface like io_uring for decades. Async IO was in Windows NT 3.1 in 1993.
- another2another 2y ago
  >Async IO was in Windows NT 3.1 in 1993 To be fair, Windows tried and abandoned 2 different methods of doing it before landing on the current IO Completion ports.
- refulgentis 2y ago
  Might've replied to the wrong comment: I don't think io_uring is bad, and the comment doesn't contain 'bad', and I certainly don't think async IO is bad :)
saurik 2y ago
Amazon runs every Lambda function in it's own microVM.
- ec109685 2y ago
  Every simultaneous request is in its own microVM, making concurrency a non-issue / thing to optimize in most cases at the function level.

ncruces 2y ago

Their goal is to run this on their own cloud. Despite their lofty claims about community building, their projects are very much about forwarding their use case. Given that SQLite is public domain, they're not required to give anything back. So, it's very cool that they're making parts of their tech FOSS. But I've yet to see anything coming from them that isn't “just because we need it, and SQLite wouldn't do it for us.” There's little concern about making things useful to others, and very little community consensus about any of it. https://turso.tech/ https://turso.tech/

tracker1 2y ago
Of course they are scratching their own itch, so to speak. Thats what companies do. I think the fact that they are doing so much in the open is the indication of good stewardship itself. I'm not sure what else they would do or release that they didn't need internally. For that matter, I'm not really aware of many significant contributions to FLOSS at all that aren't initially intended for company use, that's kinda how it works. Where I'm surprised here is how much secret sauce Turso is sharing at all.
- ncruces 2y ago
  I have no problem with them scratching their itch. That's par for the course. I'm salty about them describing the SQLite licensing, development model, and code of ethics as almost toxic, setting up a separate entity with a website and a manifesto promising to do better, and then folding “libSQL into the Turso family” within the year. They forked, played politics, added a few features (with some ill-considered incompatibilities), and properly documented zero of them. And I'm salty because I'm actually interested in some of those features, and they're impossible to use without proper documentation. I've had much better luck interacting with SQLite developers in the SQLite forum.
  - avinassh 2y ago
    disclosure: I work at Turso > code of ethics as almost toxic This is simply not true. Can you tell me where it is being said so? > then folding “libSQL into the Turso family” within the year. libSQL was always part of Turso. So, I don't get your point. > They forked, played politics, added a few features (with some ill-considered incompatibilities), and properly documented zero of them. Again this is incorrect. There are some docs here: https://github.com/tursodatabase/libsql/tree/main/docs https://github.com/tursodatabase/libsql/tree/main/docs I am really not sure why are you so angry about libSQL.
    - ncruces 2y ago
      > This is simply not true. Can you tell me where it is being said so? It's right there in “your” manifesto. https://turso.tech/libsql-manifesto https://turso.tech/libsql-manifesto > We take our code of conduct seriously, and unlike SQLite, we do not substitute it with an unclear alternative. We strive to foster a community that values diversity, equity, and inclusion. We encourage others to speak up if they feel uncomfortable. The word toxic clearly stung, but putting “unlike SQLite … we encourage others to speak up if they feel uncomfortable” in a manifesto is fine. Well, I could argue I'm just speaking up. > libSQL was always part of Turso. So, I don't get your point. My point is explained quite clearly in your post detailing the decision. https://turso.tech/blog/were-bringing-libsql-into-the-turso-family-8cc1a653448e https://turso.tech/blog/were-bringing-libsql-into-the-turso-... > We have our own self interest in making those changes (…) But we also wanted to create a welcoming community, that is open to everybody, abides by a modern code of conduct and a clear OSS license, and reimagined what SQLite could be in broader ways than just our narrow needs. A little latter down that line you sum it up: doing the above (living up to your grandiose claims of a more welcoming SQLite) “meant twice the investment” (aka a lot of money) and didn't pan out as a marketing play (showed engagement). So instead of a community that “reimagined what SQLite could be in broader ways than just our narrow needs" we just get the features you had your "own self interest in making." Which is fine, but doesn't really match the manifesto. > Again this is incorrect. There are some docs… Emphasis on some. Do you have any documentation on how to build on the Virtual WAL (internal SQLite API that you simply opened up)? Or is that's still a Rust example of an implementation that simply wraps another and logs without detailing anything beyond function names? Do you have any documentation about the new WAL API that isn't "libsql_wal_insert_begin begins WAL insertion"? I'm sorry, but goal here isn't to make things useful to others. Which is fine really: you're doing more than you're required. But compared to SQLite developers, and their forum, it's not much. PS: you also behaved… untowardly when you integrated SQLite3MultipleCiphers, and did this with not previous a word to the author. https://turso.tech/blog/fully-open-source-encryption-for-sqlite-b3858225 https://turso.tech/blog/fully-open-source-encryption-for-sql... > One project in particular was very suitable for us, SQLite Multiple Ciphers. Since it is licensed under MIT, we have just moved the code into libSQL.
nindalf 2y ago
I think you've taken the most cynical view possible. SQLite is open source but not open contribution. So if they "won't do it for us" and "we need it", what else are they supposed to do? They're offering SQLite in the cloud, obviously they need to offer a streaming backup solution. Is there something wrong in building that? Alternatively, do you want them to reimplement features in SQLite already built out by the SQLite team? Really sounds like you're complaining about getting an open source, MIT licensed, open contribution bit of software for free that you're under no obligation to use. And here you are complaining about "community consensus".
- refulgentis 2y ago
  I may be able to shed some light. It seems they proposed a way to resolve the contradiction I raised (io_uring isn't supported on cloud providers) with the ground reality in the company's blog post. Namely, that: * it is intended for edge function deployment. * the paper they mention that is informing their decision to rewrite is based on perf improvements in the longtail by using io_uring. I framed it as "Are we sure...?", but truth is, I know providers don't have io_uring support. This is relatively straightforward to derive from edge functions are well-known to be multi-tenant. A cursory search shows unresolved AWS tickets re: this, and multiple announcements from Google about how it has been disabled in gCloud. Thus, it is likely they understood I was framing it politely, and that there was a clear contradiction here, hence their reply to me, raising a potential resolution for that contradiction, a resolution I hadn't considered. I don't see anything complaining, or bringing up, or implicitly denying, all the other stuff ("open source", "MIT licensed", "open contribution", "software for free" "under no obligation to use."). In fact, they explicitly indicated they completely agree with that view ("it's very cool that they're making parts of their tech FOSS.")
  - nindalf 2y ago
    io_uring isn't supported on cloud providers for now, but it may not be the case in future. Which is fine, because Limbo won't be ready for a few years at least.
  - threecheese 2y ago

chambers 2y ago

Not an expert at all: I wonder if getting the perf requires trading-off some (advertised) safety. IO uring has been noted to be confusing with async in Rust, https://news.ycombinator.com/item?id=41992975 https://news.ycombinator.com/item?id=41992975 I'm reminded of how Confluent advertised Kafka as a database. They quietly externalized key guarantees of an RDBMS onto their customers, who were then saddled with implementing those guarantees in application level logic. By obscuring the trade-offs, Confluent made developers feel they could have their cake and eat it too.

toast0 2y ago
IMHO, most of io_uring's performance should come from reducing transition between kernel and userspace. There doesn't need to be a safety tradeoff there (although in practice, there have been several safety issues). There may be a trade off against having a simple concurrency model; without io_uring you can't really request an accept and not handle the response that comes back, because a syscall is necessarily synchronous from the point of view of the caller; the calling thread can't continue until it gets a response, even if that's EINTR.
- refulgentis 2y ago
  Out of my league / knowledge, but a tidbit that you might understand better: last time I mentioned this, someone said something about SELinux can't do security checks and implied it's some fundamental mismatch, rather than some work SELinux can do
  - toast0 2y ago
    Well I'm a little outside my league on SELinux, but as I understand it, SELinux a way to limit syscalls by configuration. io_uring operations are similar to syscalls, but not exactly the same thing, so if I've read up correctly, I think SELinux originally may not have covered operations --- if the config allowed you to use io_uring, you could bypass SELinux restrictions, and the easiest/safest thing to do for that is to restrict io_uring. I think this may have been resolved, it looks like there was work on SELinux controls for io_uring back in 2021 [1] that looks like it got merged, but I haven't really followed this. There's also the issue that what happened in Linus's tree 2021 doesn't necessarily reflect what's in common kernels in 2024; some distributions are built from ancient branches. Based on the language in the pull request, I would think you'd end up needing to have near duplicate policy entries for regular syscalls and io_uring operations, which sounds tedious (but the whole SELinux thing sounds tedious to me, so I'm maybe not the right person to ask :D ) [1] https://lore.kernel.org/lkml/CAHC9VhRJ=fHzMHM6tt8JqkZa4bf0h72CAytSX9YrEs14Oaj8SA@mail.gmail.com/ https://lore.kernel.org/lkml/CAHC9VhRJ=fHzMHM6tt8JqkZa4bf0h7...

eyberg 2y ago

Anything new is going to have people poking and prodding at it. It doesn't mean that the concept is 'bad'. Linux has desperately needed an interface like io_uring for decades. Async IO was in Windows NT 3.1 in 1993.

another2another 2y ago
>Async IO was in Windows NT 3.1 in 1993 To be fair, Windows tried and abandoned 2 different methods of doing it before landing on the current IO Completion ports.
refulgentis 2y ago
Might've replied to the wrong comment: I don't think io_uring is bad, and the comment doesn't contain 'bad', and I certainly don't think async IO is bad :)

saurik 2y ago

Amazon runs every Lambda function in it's own microVM.

ec109685 2y ago
Every simultaneous request is in its own microVM, making concurrency a non-issue / thing to optimize in most cases at the function level.