4 ms·

Author here, happy to answer any questions! Here's a few links that may be useful: Direct GitHub link: https://github.com/asg017/sqlite-loadable-rs https://git

Author here, happy to answer any questions! Here's a few links that may be useful: Direct GitHub link: https://github.com/asg017/sqlite-loadable-rs https://github.com/asg017/sqlite-loadable-rs A soon-to-be-released sqlite-loadable extension that's the fasted CSV parser for SQLite, and rivals DuckDB's parser at non-analytical queries: https://github.com/asg017/sqlite-xsv https://github.com/asg017/sqlite-xsv A regular expression extension in Rust, the fastest SQLite regex implementation: https://github.com/asg017/sqlite-regex https://github.com/asg017/sqlite-regex And in the near future, expect more extensions for postgres, parquet, XML, S3, image manipulation, and more!

CGamesPlay 4y ago
The API for virtual tables is tricky, because you don't ever consume the API in Rust: SQLite does it. However, as it is presented the interface is unsafe, I think. In the sqlite-xsv VTab::open impl for XsvTable [0], you pass a &'vtab XsvTable to XsvCursor. But the means that if open is called again, to create a second cursor, then a `&mut XsvTable` and a `&XsvTable` reference exist at the same time, which is unsafe. Note that sqlite-xsv doesn't actually keep the `&XsvTable`, so it's fine, but it serves as an example where a safety problem could surface. The same problem also applies to VTabWriteable::update. The fix is to make both of these methods receive an immutable reference and force implementors to use interior mutability. Note this isn't hypothetical, it's actually unsafe, and would appear if you had a unit test that implemented a writable virtual table backed by a rust data structure, and attempted to iterate and update at the same time. I have an as-yet-unpublished code base that experiments with this. It's not published yet because it doesn't cover everything I want to and I'm trying to avoid publishing something where the API might have to break. My goals are explicitly safety first, performance second. https://github.com/CGamesPlay/sqlite3_ext https://github.com/CGamesPlay/sqlite3_ext [0] https://github.com/asg017/sqlite-xsv/blob/main/src/xsv.rs#L94 https://github.com/asg017/sqlite-xsv/blob/main/src/xsv.rs#L9...
- CGamesPlay 4y ago
  I went ahead and published the crate so that it is browsable via docs.rs. Feel free to adopt anything from it; the code is public domain. https://docs.rs/sqlite3_ext/latest/sqlite3_ext/ https://docs.rs/sqlite3_ext/latest/sqlite3_ext/
- fedux 4y ago
  I think "undefined behavior" is the standard term for the "is unsafe" situation that you mention.
teruakohatu 4y ago
> And in the near future, expect more extensions Please consider GIS/spatial extension. SpatiaLite is pretty bad, for example the KNN feature is deprecated and the replacement has not yet made it to a release yet. https://www.gaia-gis.it/fossil/libspatialite/wiki?name=KNN https://www.gaia-gis.it/fossil/libspatialite/wiki?name=KNN

CGamesPlay 4y ago

The API for virtual tables is tricky, because you don't ever consume the API in Rust: SQLite does it. However, as it is presented the interface is unsafe, I think. In the sqlite-xsv VTab::open impl for XsvTable [0], you pass a &'vtab XsvTable to XsvCursor. But the means that if open is called again, to create a second cursor, then a `&mut XsvTable` and a `&XsvTable` reference exist at the same time, which is unsafe. Note that sqlite-xsv doesn't actually keep the `&XsvTable`, so it's fine, but it serves as an example where a safety problem could surface. The same problem also applies to VTabWriteable::update. The fix is to make both of these methods receive an immutable reference and force implementors to use interior mutability. Note this isn't hypothetical, it's actually unsafe, and would appear if you had a unit test that implemented a writable virtual table backed by a rust data structure, and attempted to iterate and update at the same time. I have an as-yet-unpublished code base that experiments with this. It's not published yet because it doesn't cover everything I want to and I'm trying to avoid publishing something where the API might have to break. My goals are explicitly safety first, performance second. https://github.com/CGamesPlay/sqlite3_ext https://github.com/CGamesPlay/sqlite3_ext [0] https://github.com/asg017/sqlite-xsv/blob/main/src/xsv.rs#L94 https://github.com/asg017/sqlite-xsv/blob/main/src/xsv.rs#L9...

CGamesPlay 4y ago
I went ahead and published the crate so that it is browsable via docs.rs. Feel free to adopt anything from it; the code is public domain. https://docs.rs/sqlite3_ext/latest/sqlite3_ext/ https://docs.rs/sqlite3_ext/latest/sqlite3_ext/
fedux 4y ago
I think "undefined behavior" is the standard term for the "is unsafe" situation that you mention.

teruakohatu 4y ago

> And in the near future, expect more extensions Please consider GIS/spatial extension. SpatiaLite is pretty bad, for example the KNN feature is deprecated and the replacement has not yet made it to a release yet. https://www.gaia-gis.it/fossil/libspatialite/wiki?name=KNN https://www.gaia-gis.it/fossil/libspatialite/wiki?name=KNN