4 ms·

I think it is actually a database, so it seems they haven't committed the cardinal sin of putting "DB" in the name of something that's not a database. > JuliaD

by vosper 6y ago

I think it is actually a database, so it seems they haven't committed the cardinal sin of putting "DB" in the name of something that's not a database. > JuliaDB is a pure Julia analytical database. It makes loading large datasets and playing with them easy and fast. JuliaDB needs to support a number of features: releational database operations, quickly parsing text files, parallel computing, data storage and compression. Got this from https://juliadb.org/talk/juliacon2018shashi/ https://juliadb.org/talk/juliacon2018shashi/ which appears to go into more details > This talk is a bottom-up look at the construction of JuliaDB. We will talk about the scope and implementation of underlying building block packages, namely IndexedTables, TextParse, Dagger, OnlineStats and PooledArrays.

gopalv 6y ago
> they haven't committed the cardinal sin of putting "DB" in the name of something that's not a database It doesn't seem to store data in any way - so it is definitely a data processing engine, but a database without an "INSERT" command feels a little off. From that point of view, this looks a lot like what original MapReduce did - the data lives outside it & is referenced as urls, but the engine itself does processing out-of-core and in-memory for very large datasets.
- wtallis 6y ago
  "It doesn't seem to store data" is a complaint that doesn't really make sense to lodge against a library rather than a standalone program. If your program is using a database library, it's your job to write the line of code that tells the library to load data from a particular file into memory. The library cannot persist as a running process of its own across multiple executions of your program that is using the library. This is as true of SQLite as it is of JuliaDB. I agree that it's a bit odd to not have a direct analog of SQL's INSERT, but you can definitely add rows to an existing table by making a new one and doing a merge operation.
- snicker7 6y ago
  What do you mean by storing? JuliaDB tables can be stored in memory or disk (mmap-able).

by vosper 6y ago

gopalv 6y ago

> they haven't committed the cardinal sin of putting "DB" in the name of something that's not a database It doesn't seem to store data in any way - so it is definitely a data processing engine, but a database without an "INSERT" command feels a little off. From that point of view, this looks a lot like what original MapReduce did - the data lives outside it & is referenced as urls, but the engine itself does processing out-of-core and in-memory for very large datasets.

wtallis 6y ago
"It doesn't seem to store data" is a complaint that doesn't really make sense to lodge against a library rather than a standalone program. If your program is using a database library, it's your job to write the line of code that tells the library to load data from a particular file into memory. The library cannot persist as a running process of its own across multiple executions of your program that is using the library. This is as true of SQLite as it is of JuliaDB. I agree that it's a bit odd to not have a direct analog of SQL's INSERT, but you can definitely add rows to an existing table by making a new one and doing a merge operation.
snicker7 6y ago
What do you mean by storing? JuliaDB tables can be stored in memory or disk (mmap-able).