5 ms·

Would Awk be useful for end user plain text databases? I want to keep a listing of all my books but I would prefer not to use a database and to use something th

Would Awk be useful for end user plain text databases? I want to keep a listing of all my books but I would prefer not to use a database and to use something that works in the Unix console.

telemachos 15y ago
The Awk Programming Language[1], by Aho, Kernighan and Weinberger, has a whole chapter on using Awk as a relational database engine. (They implement a small query language in Awk itself.) It's also a great book period. [1] http://cm.bell-labs.com/cm/cs/awkbook http://cm.bell-labs.com/cm/cs/awkbook
- ed2417 15y ago
  I agree , great book. Have you checked the price lately?
  - telemachos 15y ago
    Luckily, I don't need to: I picked it up from Amazon (university library sell-off) after silentbicycle recommended it here some time back. (You made me curious: there are used copies at Amazon starting at around $6.00[1].) [1] http://www.amazon.com/gp/offer-listing/020107981X/ref=dp_olp_used?ie=UTF8&condition=used http://www.amazon.com/gp/offer-listing/020107981X/ref=dp_olp...
    - pkrumins 15y ago
      I wrote an e-book on Awk two months ago: http://www.catonmat.net/blog/awk-book/ http://www.catonmat.net/blog/awk-book/ It teaches Awk through many practical examples, so called one-liners, that are small and short programs that just do one task. Such as joining lines, printing lines matching a pattern, summing up numbers on lines, converting text, etc.
viraptor 15y ago
Why would you want to do that? It looks like xml all over again. No advantages of user-readable format and no advantages of a real database. Talking to sqlite CLI tool is probably a little bit more useful.
- 4ad 15y ago
  XML? I keep databases in user readable/writable format that I also parse with awk. The whole point of awk is to parse human readable text. No XML nonsense. Just text with a non-rigid structure.
  - viraptor 15y ago
    That was my question - why do that? If you need a database, you really shouldn't allow the text to be parsed as anything else than originally intended. With "just text with a non-rigid structure" it's just too easy to make a silly mistake. So what is the advantage here, given we have bdb and sqlite on almost every machine nowadays? Or the other way - if you need something human-readable, why use is as a database? Why not keep the data where the data belongs and generate the reports when needed from there?
LiveTheDream 15y ago
Depends on how complex the expected operations are. I think awk is absolutely essential for dealing with things like logfiles. Do you have a reason against using sqlite[1]? That works just fine in the console, doesn't need a server. You could still use awk with it, technically. For example: bash..$ sqlite3 books.sqlite sqlite$ create table books ( title varchar(128), author varchar(128) ); sqlite$ insert into books (title, author) values ("Snow Crash", "Neal Stephenson"); sqlite$ .quit bash..$ echo "select * from books;" | sqlite3 books.sqlite Snow Crash|Neal Stephenson [1] http://sqlite.org/ http://sqlite.org/
- adamjernst 15y ago
  You don't even need to pipe the command in; just put it as the second argument after the database file. $ sqlite3 books.sqlite "select * from books"
  - LiveTheDream 15y ago
    Thanks for the tip
hesitz 15y ago
Something like the 'NoSQL' project might be just what you're looking for: http://www.strozzi.it/cgi-bin/CSA/tw7/I/en_US/nosql/Home%20Page http://www.strozzi.it/cgi-bin/CSA/tw7/I/en_US/nosql/Home%20P... This 'NoSQL' has absolutely no similarity to the 'NoSQL' databases of the last few years (and predates them all by many years). Instead it's a tool to manage databases of plain text files in the Unix environment. Seamlessly ties together several parts of Unix toolchain, with Awk as the centerpiece. Has surprising features, like searches with joins between tables (i.e., files). I'll also second that the Aho Kernighan Awk book is a beautiful piece of work.
- crazydiamond 15y ago
  That link (nosql) looks interesting. Also do look at crush-tools. You can even pivot data with it. http://code.google.com/p/crush-tools/wiki/CrushTutorial http://code.google.com/p/crush-tools/wiki/CrushTutorial
a3_nm 15y ago
You might be interested by recutils: https://www.gnu.org/s/recutils/ https://www.gnu.org/s/recutils/

Would Awk be useful for end user plain text databases? I want to keep a listing of all my books but I would prefer not to use a database and to use something that works in the Unix console.

telemachos 15y ago

The Awk Programming Language[1], by Aho, Kernighan and Weinberger, has a whole chapter on using Awk as a relational database engine. (They implement a small query language in Awk itself.) It's also a great book period. [1] http://cm.bell-labs.com/cm/cs/awkbook http://cm.bell-labs.com/cm/cs/awkbook

ed2417 15y ago
I agree , great book. Have you checked the price lately?
- telemachos 15y ago
  Luckily, I don't need to: I picked it up from Amazon (university library sell-off) after silentbicycle recommended it here some time back. (You made me curious: there are used copies at Amazon starting at around $6.00[1].) [1] http://www.amazon.com/gp/offer-listing/020107981X/ref=dp_olp_used?ie=UTF8&condition=used http://www.amazon.com/gp/offer-listing/020107981X/ref=dp_olp...
  - pkrumins 15y ago
    I wrote an e-book on Awk two months ago: http://www.catonmat.net/blog/awk-book/ http://www.catonmat.net/blog/awk-book/ It teaches Awk through many practical examples, so called one-liners, that are small and short programs that just do one task. Such as joining lines, printing lines matching a pattern, summing up numbers on lines, converting text, etc.

viraptor 15y ago

Why would you want to do that? It looks like xml all over again. No advantages of user-readable format and no advantages of a real database. Talking to sqlite CLI tool is probably a little bit more useful.

4ad 15y ago
XML? I keep databases in user readable/writable format that I also parse with awk. The whole point of awk is to parse human readable text. No XML nonsense. Just text with a non-rigid structure.
- viraptor 15y ago
  That was my question - why do that? If you need a database, you really shouldn't allow the text to be parsed as anything else than originally intended. With "just text with a non-rigid structure" it's just too easy to make a silly mistake. So what is the advantage here, given we have bdb and sqlite on almost every machine nowadays? Or the other way - if you need something human-readable, why use is as a database? Why not keep the data where the data belongs and generate the reports when needed from there?

LiveTheDream 15y ago

Depends on how complex the expected operations are. I think awk is absolutely essential for dealing with things like logfiles. Do you have a reason against using sqlite[1]? That works just fine in the console, doesn't need a server. You could still use awk with it, technically. For example: bash..$ sqlite3 books.sqlite sqlite$ create table books ( title varchar(128), author varchar(128) ); sqlite$ insert into books (title, author) values ("Snow Crash", "Neal Stephenson"); sqlite$ .quit bash..$ echo "select * from books;" | sqlite3 books.sqlite Snow Crash|Neal Stephenson [1] http://sqlite.org/ http://sqlite.org/

adamjernst 15y ago
You don't even need to pipe the command in; just put it as the second argument after the database file. $ sqlite3 books.sqlite "select * from books"
- LiveTheDream 15y ago
  Thanks for the tip

hesitz 15y ago

Something like the 'NoSQL' project might be just what you're looking for: http://www.strozzi.it/cgi-bin/CSA/tw7/I/en_US/nosql/Home%20Page http://www.strozzi.it/cgi-bin/CSA/tw7/I/en_US/nosql/Home%20P... This 'NoSQL' has absolutely no similarity to the 'NoSQL' databases of the last few years (and predates them all by many years). Instead it's a tool to manage databases of plain text files in the Unix environment. Seamlessly ties together several parts of Unix toolchain, with Awk as the centerpiece. Has surprising features, like searches with joins between tables (i.e., files). I'll also second that the Aho Kernighan Awk book is a beautiful piece of work.

crazydiamond 15y ago
That link (nosql) looks interesting. Also do look at crush-tools. You can even pivot data with it. http://code.google.com/p/crush-tools/wiki/CrushTutorial http://code.google.com/p/crush-tools/wiki/CrushTutorial

a3_nm 15y ago

You might be interested by recutils: https://www.gnu.org/s/recutils/ https://www.gnu.org/s/recutils/