3 ms·

I'm using WAL mode - I'd almost given up on sqlite until I remembered to enable it. I've been playing with benchmarks, yes! I haven't written them up yet becau

by memset 2y ago

I'm using WAL mode - I'd almost given up on sqlite until I remembered to enable it. I've been playing with benchmarks, yes! I haven't written them up yet because I worry about doing them the "right" way. but since you asked, I did a quick test with a single server and single client on a t2.nano. With 3 sending threads and 5 receiving threads, and mesg size of 2kb, I can send 700 msgs/s and receiver 500 msgs/s. It is way faster on my laptop, and I have a lot of tricks that I've been playing with to improve this.

withinboredom 2y ago
I had to build a queue implementation not too long ago. We used t2.nano for our benchmarks as well. Your results are in-line with ours pre-optimizations (except we are using nats jetstream for queuing). I’d also recommend reducing the number of threads as that can increase performance on single-processor machines (context switching will kill you). Try to find the sweet spot (for us, it was 2x-4x the number of cpus to threads, at least, for our workload). If you are using go, it kinda sucks at single-cpu things, and if you detect that you have a single core, you can lock a go-routine to a thread and that sometimes helps. Also, try to batch sends (aka, for-loop) to attempt to saturate the send-side so by the time you come back with your next batch they’re still messages sitting in a network buffer somewhere. For example, we have a channel that wakes up our sender, then we honest-to-god wait like 60ms in the hopes there will be more than one message to pick up — and there usually is in production.
- memset 2y ago
  These are great suggestions. Today every db write is indeed wrapped in a mutex. The two optimizations I am experimenting with are: 1. When new messages are inserted, immediately append to a file on disk. Then, in batch, insert to SQLite. 2. When dequeuing messages, keep n message IDs in memory as a ready queue, and then keep dequeued message IDs in another list. Those can be served immediately (using a SELECT which is fast) and then updating messages to the dequeued status can happen in batch. Appreciate the tips!
  - withinboredom 2y ago
    For 1. We went with a per-routine bytes.Buffer that was batch "inserted" every x milliseconds or n messages. However, we don't care if we lose some messages on a crash. For integrity, some queues are set to 0ms, 1msg because we don't want to lose anything, but when it is ok that messages are lost, this is great for perf. For 2. you could probably do something like this: BEGIN TRANSACTION; -- Select the oldest pending message SELECT id, message FROM queue WHERE status = 'pending' ORDER BY created_at ASC LIMIT 100; -- Mark messages as 'processing' UPDATE queue SET status = 'processing' WHERE created_at < ?; -- assuming created_at is monotonic COMMIT; Basically, select a batch and then abuse the ordering properties to batch mark them. Then all messages in your select you can dispatch evenly to sender threads. Sender threads can then signal a buffered channel that they've completed/failed, and the database can be updated. At startup, you can just SELECT where status = 'processing' and recover. This is a pretty decent translation of how ours works.
evantbyrne 2y ago
You might need dedicated resources, including storage speed guarantees, to get reproducible benchmarks out of a cloud provider. Love the red sweater btw very classy yet simple
- memset 2y ago
  That sweater is one of my favorite items, thank you :) The goal is for this to be able to run well on commodity hardware, which I think is possible. If I can run this on $platform with $cheap instances and an $autoscaler then that would be my ideal design goal because I think it matches the setup that most people have access to. However, I agree - I am a big fan of Hetzner's servers and am excited to try out benchmarks on beefier hardware too.

by memset 2y ago

withinboredom 2y ago

I had to build a queue implementation not too long ago. We used t2.nano for our benchmarks as well. Your results are in-line with ours pre-optimizations (except we are using nats jetstream for queuing). I’d also recommend reducing the number of threads as that can increase performance on single-processor machines (context switching will kill you). Try to find the sweet spot (for us, it was 2x-4x the number of cpus to threads, at least, for our workload). If you are using go, it kinda sucks at single-cpu things, and if you detect that you have a single core, you can lock a go-routine to a thread and that sometimes helps. Also, try to batch sends (aka, for-loop) to attempt to saturate the send-side so by the time you come back with your next batch they’re still messages sitting in a network buffer somewhere. For example, we have a channel that wakes up our sender, then we honest-to-god wait like 60ms in the hopes there will be more than one message to pick up — and there usually is in production.

memset 2y ago
These are great suggestions. Today every db write is indeed wrapped in a mutex. The two optimizations I am experimenting with are: 1. When new messages are inserted, immediately append to a file on disk. Then, in batch, insert to SQLite. 2. When dequeuing messages, keep n message IDs in memory as a ready queue, and then keep dequeued message IDs in another list. Those can be served immediately (using a SELECT which is fast) and then updating messages to the dequeued status can happen in batch. Appreciate the tips!
- withinboredom 2y ago
  For 1. We went with a per-routine bytes.Buffer that was batch "inserted" every x milliseconds or n messages. However, we don't care if we lose some messages on a crash. For integrity, some queues are set to 0ms, 1msg because we don't want to lose anything, but when it is ok that messages are lost, this is great for perf. For 2. you could probably do something like this: BEGIN TRANSACTION; -- Select the oldest pending message SELECT id, message FROM queue WHERE status = 'pending' ORDER BY created_at ASC LIMIT 100; -- Mark messages as 'processing' UPDATE queue SET status = 'processing' WHERE created_at < ?; -- assuming created_at is monotonic COMMIT; Basically, select a batch and then abuse the ordering properties to batch mark them. Then all messages in your select you can dispatch evenly to sender threads. Sender threads can then signal a buffered channel that they've completed/failed, and the database can be updated. At startup, you can just SELECT where status = 'processing' and recover. This is a pretty decent translation of how ours works.

evantbyrne 2y ago

You might need dedicated resources, including storage speed guarantees, to get reproducible benchmarks out of a cloud provider. Love the red sweater btw very classy yet simple

memset 2y ago
That sweater is one of my favorite items, thank you :) The goal is for this to be able to run well on commodity hardware, which I think is possible. If I can run this on $platform with $cheap instances and an $autoscaler then that would be my ideal design goal because I think it matches the setup that most people have access to. However, I agree - I am a big fan of Hetzner's servers and am excited to try out benchmarks on beefier hardware too.