5 ms·

"Sometimes you just want to store data on the disk in a safe and language agnostic way." You mean, kind of like a file?

paulgb 17y ago
Well, since you need a safe way, you'll need a locking mechanism, too. And since it needs to be language agnostic, you can't just dump the internal representation of your object to disk, so you need some sort of serialization. At that point, it's probably easier to go with an already existent object or document store.
- mpk 17y ago
  Ladies and gentlemen, we have a winner! If you start using the filesystem as a datastore that requires concurrent access you open up a whole new can of worms. You need a locking mechanism - which you'll probably implement using (wrapped) native syscalls. Not only does that break cross-platform operation, you'll also have to work on and fix (but find first, of course) bugs in the locking implementation. As you spend more and more time on this and your app starts growing, you'll find yourself spending more and more time working with the limitations of the filesystem you're using (file size limits, directory size limits, access times for files in large directories). You can hack your way around all that but then you have to face other critical tasks. Say .. backup and restore procedures. Can you do partial backup/restore operations? No? Well, get ready to write code for that too. And you preferably want to be able to do those live. Remember those locking issues you solved when you started down this road to hell? Yeah, they're back with a vengeance now. How about a full restore? Maybe you should have implemented a replay-able log system to get that full restore up to speed with the state of the db since the time of the last backup. Or maybe this isn't exactly the right point at which to re-invent the wheel :)
- aaronblohowiak 17y ago
  XML is nice for this because it supports multiple schema versions, validation, and has support in just about every language. My chief complaint with json as a cross-language serialization/interchange format is lack of a great way to validate your format. Most of the json schema definitions i've seen require your schema to follow a certain convention, which seems backwards and wrong to me. Files can be locked if your OS supports it.
- derefr 17y ago
  Perhaps developers wouldn't be going nearly as crazy for NoSQL if Apple's CoreData had a platform-agnostic FOSS equivalent.
aaronblohowiak 17y ago
Filesystem is the original document store. Unfortunately, most filesystems really suffer when you put $LOTS of files in the same folder, so you end up implementing a nested folder structure and that complicates your code. Now, if this is more complex than running an entire "object" store depends on the application. Also, the filesystem's addressing may not be granular enough (and so waste a lot of space,) if you are going to store < $BLOCK_SIZE files.
- hassy 17y ago
  You should check out vertexdb. It's designed to be used just like a filesystem, but it fixes the shortcomings of filesystems when they are used as dbs. http://github.com/stevedekorte/vertexdb http://github.com/stevedekorte/vertexdb In production use at http://stylous.com http://stylous.com

Well, since you need a safe way, you'll need a locking mechanism, too. And since it needs to be language agnostic, you can't just dump the internal representation of your object to disk, so you need some sort of serialization. At that point, it's probably easier to go with an already existent object or document store.

mpk 17y ago
Ladies and gentlemen, we have a winner! If you start using the filesystem as a datastore that requires concurrent access you open up a whole new can of worms. You need a locking mechanism - which you'll probably implement using (wrapped) native syscalls. Not only does that break cross-platform operation, you'll also have to work on and fix (but find first, of course) bugs in the locking implementation. As you spend more and more time on this and your app starts growing, you'll find yourself spending more and more time working with the limitations of the filesystem you're using (file size limits, directory size limits, access times for files in large directories). You can hack your way around all that but then you have to face other critical tasks. Say .. backup and restore procedures. Can you do partial backup/restore operations? No? Well, get ready to write code for that too. And you preferably want to be able to do those live. Remember those locking issues you solved when you started down this road to hell? Yeah, they're back with a vengeance now. How about a full restore? Maybe you should have implemented a replay-able log system to get that full restore up to speed with the state of the db since the time of the last backup. Or maybe this isn't exactly the right point at which to re-invent the wheel :)
aaronblohowiak 17y ago
XML is nice for this because it supports multiple schema versions, validation, and has support in just about every language. My chief complaint with json as a cross-language serialization/interchange format is lack of a great way to validate your format. Most of the json schema definitions i've seen require your schema to follow a certain convention, which seems backwards and wrong to me. Files can be locked if your OS supports it.
derefr 17y ago
Perhaps developers wouldn't be going nearly as crazy for NoSQL if Apple's CoreData had a platform-agnostic FOSS equivalent.

aaronblohowiak 17y ago

Filesystem is the original document store. Unfortunately, most filesystems really suffer when you put $LOTS of files in the same folder, so you end up implementing a nested folder structure and that complicates your code. Now, if this is more complex than running an entire "object" store depends on the application. Also, the filesystem's addressing may not be granular enough (and so waste a lot of space,) if you are going to store < $BLOCK_SIZE files.

hassy 17y ago
You should check out vertexdb. It's designed to be used just like a filesystem, but it fixes the shortcomings of filesystems when they are used as dbs. http://github.com/stevedekorte/vertexdb http://github.com/stevedekorte/vertexdb In production use at http://stylous.com http://stylous.com