5 ms·
Until 2010, I ran a forum on SQLite with 600 users/day + around 1000 posts/day on a single OpenBSD machine with 2gb RAM and 7200RPM HDD. We moved to Postgres wh
by eksith 13y ago
Until 2010, I ran a forum on SQLite with 600 users/day + around 1000 posts/day on a single OpenBSD machine with 2gb RAM and 7200RPM HDD. We moved to Postgres when we passed 1600 posts/day.
There was some minor slowdown noticeable around 6AM - 10AM, but besides that, we never had major issues. The bottleneck was usually the network, not the DB.
Edit: Anyone interested in playing with SQLite should give the SQLite Manager plugin for FF a try : https://addons.mozilla.org/en-US/firefox/addon/sqlite-manager/ https://addons.mozilla.org/en-US/firefox/addon/sqlite-manage...
(I'm not affiliated with the developer, I just like the tool)
- acmecorps 13y agoReally? I thought SQLite is for development purpose only. but 1000posts/day? Wow..
- eksith 13y agoI should add, that we ran nothing else on the box. Just the forum and it was text only (no image uploads etc...). Backups were pretty easy. ;)
- corresation 13y agoPlease clarify "for development purpose only"? SQLite is used in a lot of production software you rely upon daily, and is one of the most robust, useful pieces of code going. http://www.sqlite.org/famous.html http://www.sqlite.org/famous.html Now it's an embedded database, which is why leokun's comment is out of place. However you can absolutely drop a queue in front of it, for instance, and serve many "simultaneous" users for a purpose such as a web forum.
- onedognight 13y ago> but 1000posts/day? Wow.. While a forum that gets 1000 posts/day is quite active, a database that does an INSERT ever 80s is not even moving.
- eurleif 13y agoPresumably there were many SELECTs for every INSERT.
- eksith 13y agoYes. Yes there were. :) We always limited transactions for UPDATES and INSERTs that involved more than one table. It was quite the learning experience since you do have to think of different ways of working with data. You get to learn very quickly the difference between what like to store vs. what you actually need to store to get a particular functionality. But that turned to being a boon in the end because we got a much simpler forum as a result. Fewer bells and whistles meant users focused on actual discussion and not ancillary, shiny bits and bobs.
- vidarh 13y agoBut on a machine with 2GB of RAM running a forum that small, pretty much none of those selects should ever need to hit disk - it should all pretty much be in the buffer cache, unless there were lots of full text searches of the entire posting history. EDIT: And that is without any app specific caching.
- vidarh 13y agoFor comparison, 8 years ago I wrote a queuing system that did either in-memory queues, or wrote to sqlite for durability. With a little bit of tuning, the Sqlite queues could handle at the very least hundreds of thousands of messages a day - we never pushed it to it's limit.
- Groxx 13y agoPut simply, you thought wrong. SQLite is one of the most-used pieces of software out there, absolutely including production software. http://www.sqlite.org/mostdeployed.html http://www.sqlite.org/mostdeployed.html
- nraynaud 13y agoI'm really glad to read that, and I was stupid not to do it.
- guelo 13y agoI thought SQLite locked the entire DB when doing an INSERT. Seems like that would slow a forum app considerably. Though I guess at 1600 INSERTs per day, which averages to about 1 per minute assuming even distribution throughout the day, you won't have that many lock collisions.
- abtinf 13y agoThe sqlite locking model can eat 1600 inserts before breakfast with only minor lock contention. The whole file lock doesn't take effect until the moment the write is ready to go to disk. http://www.sqlite.org/lockingv3.html http://www.sqlite.org/lockingv3.html
- eksith 13y agoWrites were generally quite fast so the next write rarely got queued. Also we ran it single-threaded. When we moved to Postgres, that also gave us the opportunity to safely switch to phpfpm without worrying about the lock issue. The nice thing about Postgres as opposed to SQLite is that we finally got "searching" as a feature. ;)
- deleted 13y ago[deleted]
- jeltz 13y agoNot if you use it in the WAL mode.
- deleted 13y ago[deleted]
- dchest 13y agoTo add to this, since 2010 SQLite improved concurrency a bit with write-ahead loging: http://www.sqlite.org/draft/wal.html http://www.sqlite.org/draft/wal.html
- vidarh 13y ago600 users/day and 1000/posts a day is miniscule. You can run something like that with no noticeable performance issues on a classic 120MHz Pentium and flat files (I'm speaking from experience of running a USENET news server handling 30,000 groups for an ISP on a machine like that, which was shared with lots of other functions), so if the bottleneck had been anything other than the network, it'd have been shocking.