3 ms·
Reducing max document size from 4GB down to 8MB seems hyper-restrictive. For those interested, looks like the guts of CouchDB are going to be swapped out for F
by hajile 7y ago
Reducing max document size from 4GB down to 8MB seems hyper-restrictive.
For those interested, looks like the guts of CouchDB are going to be swapped out for FoundationDB.
https://blog.couchdb.org/2020/02/26/the-road-to-couchdb-3-0-prepare-for-4-0/ https://blog.couchdb.org/2020/02/26/the-road-to-couchdb-3-0-...
- splatcollision 7y agoIf you're trying to store single GB documents in couch, you're doing it wrong... Unless those are binaries you can usually fragment data logically across many documents, then write custom views to aggregate however you need to. Updates on huge docs would be painful!
- hajile 7y agoI agree that 4GB is more than a sane person should probably be using. I don't think that going above 8MB is very hard or uncommon though. If I'm going to spread everything across many different documents and document types and then join them all together, I then have to make a case for why I'm still choosing Couch over a RDBMS.
- meddlepal 7y agoI agree in large with your point that multi-GB documents is perhaps excessive but this does create a heck of a migration problem for a lot of users that aren't even close to 4GB.
- Volundr 7y agoIt does, but keep in mind for 3.0 this is a change to the default settings, not a hard cap. The idea is to give people lots of warning and time to do any migrations necessary prior to 4.0.
- newfeatureok 7y ago8MB is just the default, you can switch it back to 4GB if you want, but you won't have an easy time switching to 4.0 due to the 8MB limit imposed by FoundationDB.
- NDizzle 7y agoIt seems really odd to me that they are changing to an 8MB limit. "Document storage database" that can only hold 8MB per document. It can't hold full text output for a large documents... What am I missing here?
- newfeatureok 7y agoIn CouchDB 4.0 the backend of CouchDB will be switching to FoundationDB, which will have a 8MB limit, so they're preemptively making the change. You can remove the limit now if you'd like.
- billfruit 7y agoYes, but how is a 8mb document size limit a good thing?
- janl 7y agoFoundationDB needs is a consistent (as per CAP) database, to make that work in a distributed fashion, there are limits on distributed transactions in size and time. CouchDB needs to adopt those in order to move to FoundationDB. The 3.x series will be supported for a long time for folks who can’t move up. Until then, as others have stated, you can up the default limit.
- janl 7y agoIn CouchDB terms, you’d store data like this in attachments, not as raw JSON.
- yyyk 7y agoCouch/Pouch combination is really slow when document +attachment are too large. Based on my experience, Practical limit was more like 20-30Mb, and I'd suggest not even getting close to that. 8Mb simply recognizes reality.
- janl 7y agoThat’s another great point, an exactly why we are doing this. It is a best practice already.
- AtlasBarfed 7y agoThat still exists? I thought apple bought them and shuttered them.
- janl 7y agoFoundationDB got re-released as an open source project: https://www.foundationdb.org/blog/foundationdb-is-open-source/ https://www.foundationdb.org/blog/foundationdb-is-open-sourc...
- lytefm 7y agoAt least since 2.0 (haven't used Couch before), the docs have always recommended to only use small documents in a CouchDB and to use an external storage for large files. The IBM Cloudant free tier only allows Doxs up to 1 MB. So this doesn't really come as a surprise or feel hyper restrive to me.