4 ms·

It is also an excellent tool to have in your command line toolkit. Rather than | wc | uniq | sort etc, pipe to file and bring into SQLite for more extensive que

by mongol 13y ago

It is also an excellent tool to have in your command line toolkit. Rather than | wc | uniq | sort etc, pipe to file and bring into SQLite for more extensive querying.

DigitalJack 13y ago
That's pretty interesting. Can you give an example?
- techtalsky 13y ago
  Just make a quick, loose, all-text-fields schema and dump your data as a CSV: http://www.sqlite.org/cvstrac/wiki?p=ImportingFiles http://www.sqlite.org/cvstrac/wiki?p=ImportingFiles Then you can run SQL queries against it using the SQLite commandline tool: http://www.sqlite.org/sqlite.html http://www.sqlite.org/sqlite.html
- deleted 13y ago
  [deleted]
- deleted 13y ago
  [deleted]
dalke 13y ago
I've a different but related use case. I want to make a 5 GB file in a specific format, which requires that the data be sorted. I want it to run on my laptop machine, which has 4 GB of RAM. I dump the unsorted data to a sqlite database, tell it to use up to 1 GB of cache, create an index on the sort field, and pull back the sorted data, which I can stream directly to the file.
- gngeal 13y ago
  What about simply using the original NoSQL system? http://www.strozzi.it/cgi-bin/CSA/tw7/I/en_US/nosql/Home%20Page http://www.strozzi.it/cgi-bin/CSA/tw7/I/en_US/nosql/Home%20P...
  - dalke 13y ago
    I'm using Python, which has had sqlite as a standard module since Python 2.5. Therefore, I don't need to install any third-party module. NoSQL provides command-line programs. sqlite is embedded in the same process. Error reporting across exec boundaries is much more complicated. NoSQL requires standard unix command-line programs, and for at least one case "This program has only been tested with GNU sort" This makes portability to other OSes, specifically MS Windows but perhaps also OpenBSD, harder. Hence there's no "simple" about using NoSQL for my use case.

4 ms·

It is also an excellent tool to have in your command line toolkit. Rather than | wc | uniq | sort etc, pipe to file and bring into SQLite for more extensive que

by mongol 13y ago

It is also an excellent tool to have in your command line toolkit. Rather than | wc | uniq | sort etc, pipe to file and bring into SQLite for more extensive querying.

DigitalJack 13y ago
That's pretty interesting. Can you give an example?
- techtalsky 13y ago
  Just make a quick, loose, all-text-fields schema and dump your data as a CSV: http://www.sqlite.org/cvstrac/wiki?p=ImportingFiles http://www.sqlite.org/cvstrac/wiki?p=ImportingFiles Then you can run SQL queries against it using the SQLite commandline tool: http://www.sqlite.org/sqlite.html http://www.sqlite.org/sqlite.html
- deleted 13y ago
  [deleted]
- deleted 13y ago
  [deleted]
dalke 13y ago
I've a different but related use case. I want to make a 5 GB file in a specific format, which requires that the data be sorted. I want it to run on my laptop machine, which has 4 GB of RAM. I dump the unsorted data to a sqlite database, tell it to use up to 1 GB of cache, create an index on the sort field, and pull back the sorted data, which I can stream directly to the file.
- gngeal 13y ago
  What about simply using the original NoSQL system? http://www.strozzi.it/cgi-bin/CSA/tw7/I/en_US/nosql/Home%20Page http://www.strozzi.it/cgi-bin/CSA/tw7/I/en_US/nosql/Home%20P...
  - dalke 13y ago
    I'm using Python, which has had sqlite as a standard module since Python 2.5. Therefore, I don't need to install any third-party module. NoSQL provides command-line programs. sqlite is embedded in the same process. Error reporting across exec boundaries is much more complicated. NoSQL requires standard unix command-line programs, and for at least one case "This program has only been tested with GNU sort" This makes portability to other OSes, specifically MS Windows but perhaps also OpenBSD, harder. Hence there's no "simple" about using NoSQL for my use case.