6 ms·

Pomegranate: storing billions and billions of tiny little files

99 points by chuhnk 16y ago

SlyShy 16y ago
I encourage anyone curious to take a look at the data they generated. http://github.com/macan/Pomegranate/wiki/Experiment http://github.com/macan/Pomegranate/wiki/Experiment I'd love it some other people were to give it ago, and see if they can replicate the results. I'm still setting it up.
carson 16y ago
This project seems interesting but the github repo seems to be missing the kernel part. That or the documentation and blog post are incorrect. Hopefully someone will be able to provide a little more information about using it so others can test it.
- canma 16y ago
  The kernel module is under optimizing, we may provide it in a few weeks. Thanks!
arethuza 16y ago
They may be planning to add this - but what would be very nice would be to have the same data accessible through both "traditonal" file system APIs but also allow NoSQL style access to the associated metadata. If you could then plug in your own indexing and search engine in at the file system level and have it accessible through the same API then that would be particularly awesome.
- anamax 16y ago
  Any reason to prefer NoSql to full relational? If not, what's the difference between your request and WinFS aka longhorn http://en.wikipedia.org/wiki/WinFS http://en.wikipedia.org/wiki/WinFS .
  - arethuza 16y ago
    Not really - just that I'm in the middle of doing something with CouchDB on the server side and my own lightweight MapReduce engine sitting on top of the in-browser SQLite database in Safari (for the iPhone/iPad). So everything looks document oriented to me at the moment!
maukdaddy 16y ago
Not to get too fluffy, but 'pomegranate' is perfect name. Creative and yet apropos.
Rickasaurus 16y ago
I thought this said flies at first and it grossed me out pretty bad.
- toddh 16y ago
  It's funny you say that because when I fist published the article I froze in horror because I thought it read "Flies" instead of "Files." Strangely I still read it that way...Crap I just did again when rereading this post...
jrockway 16y ago
Sounds like Redis.

99 points by chuhnk 16y ago

I encourage anyone curious to take a look at the data they generated. http://github.com/macan/Pomegranate/wiki/Experiment http://github.com/macan/Pomegranate/wiki/Experiment I'd love it some other people were to give it ago, and see if they can replicate the results. I'm still setting it up.

carson 16y ago

This project seems interesting but the github repo seems to be missing the kernel part. That or the documentation and blog post are incorrect. Hopefully someone will be able to provide a little more information about using it so others can test it.

canma 16y ago
The kernel module is under optimizing, we may provide it in a few weeks. Thanks!

arethuza 16y ago

They may be planning to add this - but what would be very nice would be to have the same data accessible through both "traditonal" file system APIs but also allow NoSQL style access to the associated metadata. If you could then plug in your own indexing and search engine in at the file system level and have it accessible through the same API then that would be particularly awesome.

anamax 16y ago
Any reason to prefer NoSql to full relational? If not, what's the difference between your request and WinFS aka longhorn http://en.wikipedia.org/wiki/WinFS http://en.wikipedia.org/wiki/WinFS .
- arethuza 16y ago
  Not really - just that I'm in the middle of doing something with CouchDB on the server side and my own lightweight MapReduce engine sitting on top of the in-browser SQLite database in Safari (for the iPhone/iPad). So everything looks document oriented to me at the moment!

maukdaddy 16y ago

Not to get too fluffy, but 'pomegranate' is perfect name. Creative and yet apropos.

Rickasaurus 16y ago

I thought this said flies at first and it grossed me out pretty bad.

toddh 16y ago
It's funny you say that because when I fist published the article I froze in horror because I thought it read "Flies" instead of "Files." Strangely I still read it that way...Crap I just did again when rereading this post...

jrockway 16y ago

Sounds like Redis.