6 ms·

Not sure the case in this context, but usually the SQLite dependency can almost be seen as no-dependency at all - it's just a file on the disk without any exter

Not sure the case in this context, but usually the SQLite dependency can almost be seen as no-dependency at all - it's just a file on the disk without any external dependency.

rakoo 7y ago
On the other hand it requires a C compiler, so it's definitely heavier than a pure go application
- jlubawy 7y ago
  Serious question, why hasn't anyone rewritten it in Go? Been awhile since I've looked at it but C to Go doesn't seem that difficult to translate. Edit: just looked at the amalgamation again, it's huge... Still surprised no one has tried it yet that I know of. Pre-processor directives would probably make it even more difficult though. Edit2: and it already works so what's the point, other than an extremely challenging problem
  - makapuf 7y ago
    Sqlite databases are often not shared between applications. Do you know a similar sql, in process database in pure go (even if not completely similar feature wise ?)
  - minxomat 7y ago
    Sqlite has been rewritten in both Go and C#. It's really not that hard. The code is clearly documented and everything works in roughly the same way. Easily achievable by a single developer in a few months. Sqlite code is emulating classes with structs and methods (they're even called classes in the code). This it's pretty easy to translate into languages with reference and struct support. Memory allocation is separated into an allocation provider, which is easy to implement in GC languages (just create the struct requested). The parser and VM opcode file is tricky. The opcodes are parsed from comments in one big file. I'm not a fan of that. But once generated for a particular version, pretty easy to translate. Testing is another can of worms. Translating the exact version of Tcl, too is the best course of action. This is required to run and pass the (hundreds of thousands of) base line tests. Other correctness tests (in sqlite because of branch coverage requirements for aircraft software) are harder to translate. The prerelease burn in test is also not publicly available. Preprocessor directives are not used as macros. Rather they are used to enable features. In a port, you can for example choose to exclude WAL or specific optimizations. FTS can be left out if not needed. The different lock implementations can be deferred to the stdlib. If you interested in the perf overhead, here's a small benchmark of a C# implementation I maintain: https://drive.google.com/file/d/11Bgfh1WgEreDenosoEdWkYAOb6u_fhMQ/view?usp=drivesdk https://drive.google.com/file/d/11Bgfh1WgEreDenosoEdWkYAOb6u...
    - lifty 7y ago
      Can you point to the pure Go re-implementation of Sqlite? I was looking for such a thing but never found it. Instead I had to go with a k/v store like Bolt.
      - deleted 7y ago
        [deleted]
      - 8lall0 7y ago
        https://godoc.org/modernc.org/ql https://godoc.org/modernc.org/ql
        minxomat 7y ago
        That's not it. It was a GitLab project, but searching for that is pretty difficult. I'll edit this comment if i find it. Edit: Here it is: https://gitlab.com/cznic/sqlite https://gitlab.com/cznic/sqlite
lkschubert8 7y ago
If you were planing on using this on a PaaS host where the file system is ephemeral this would be a non-starter.
- lewiscollard 7y ago
  What would the RL use case be for running this server, whose purpose is to facilitate transferring files from one computer's filesystem to another, on an ephemeral filesystem?
  - CameronBarre 7y ago
    If what's meant here is that it would be hard to run this on a container platform because of the ephemeral nature of containers, assumably you would be using persistent volumes to keep your state safe across the container lifecycle. That being said if you really wanted to use sqlite for the database, you'd do the exact same thing, use a persistent volume.
  - lkschubert8 7y ago
    I wrote an interface that would allow SFTP transfer to Azure Blob Storage (doesn't support it) that doesn't store anything on the machine running the service. It was really just supposed to be a frontend.

Not sure the case in this context, but usually the SQLite dependency can almost be seen as no-dependency at all - it's just a file on the disk without any external dependency.

rakoo 7y ago

On the other hand it requires a C compiler, so it's definitely heavier than a pure go application

jlubawy 7y ago
Serious question, why hasn't anyone rewritten it in Go? Been awhile since I've looked at it but C to Go doesn't seem that difficult to translate. Edit: just looked at the amalgamation again, it's huge... Still surprised no one has tried it yet that I know of. Pre-processor directives would probably make it even more difficult though. Edit2: and it already works so what's the point, other than an extremely challenging problem
- makapuf 7y ago
  Sqlite databases are often not shared between applications. Do you know a similar sql, in process database in pure go (even if not completely similar feature wise ?)
- minxomat 7y ago
  Sqlite has been rewritten in both Go and C#. It's really not that hard. The code is clearly documented and everything works in roughly the same way. Easily achievable by a single developer in a few months. Sqlite code is emulating classes with structs and methods (they're even called classes in the code). This it's pretty easy to translate into languages with reference and struct support. Memory allocation is separated into an allocation provider, which is easy to implement in GC languages (just create the struct requested). The parser and VM opcode file is tricky. The opcodes are parsed from comments in one big file. I'm not a fan of that. But once generated for a particular version, pretty easy to translate. Testing is another can of worms. Translating the exact version of Tcl, too is the best course of action. This is required to run and pass the (hundreds of thousands of) base line tests. Other correctness tests (in sqlite because of branch coverage requirements for aircraft software) are harder to translate. The prerelease burn in test is also not publicly available. Preprocessor directives are not used as macros. Rather they are used to enable features. In a port, you can for example choose to exclude WAL or specific optimizations. FTS can be left out if not needed. The different lock implementations can be deferred to the stdlib. If you interested in the perf overhead, here's a small benchmark of a C# implementation I maintain: https://drive.google.com/file/d/11Bgfh1WgEreDenosoEdWkYAOb6u_fhMQ/view?usp=drivesdk https://drive.google.com/file/d/11Bgfh1WgEreDenosoEdWkYAOb6u...
  - lifty 7y ago
    Can you point to the pure Go re-implementation of Sqlite? I was looking for such a thing but never found it. Instead I had to go with a k/v store like Bolt.
    - deleted 7y ago
      [deleted]
    - 8lall0 7y ago
      https://godoc.org/modernc.org/ql https://godoc.org/modernc.org/ql
      - minxomat 7y ago
        That's not it. It was a GitLab project, but searching for that is pretty difficult. I'll edit this comment if i find it. Edit: Here it is: https://gitlab.com/cznic/sqlite https://gitlab.com/cznic/sqlite

lkschubert8 7y ago

If you were planing on using this on a PaaS host where the file system is ephemeral this would be a non-starter.

lewiscollard 7y ago
What would the RL use case be for running this server, whose purpose is to facilitate transferring files from one computer's filesystem to another, on an ephemeral filesystem?
- CameronBarre 7y ago
  If what's meant here is that it would be hard to run this on a container platform because of the ephemeral nature of containers, assumably you would be using persistent volumes to keep your state safe across the container lifecycle. That being said if you really wanted to use sqlite for the database, you'd do the exact same thing, use a persistent volume.
- lkschubert8 7y ago
  I wrote an interface that would allow SFTP transfer to Azure Blob Storage (doesn't support it) that doesn't store anything on the machine running the service. It was really just supposed to be a frontend.