2 ms·
Don't write it to your main transactional database, then. Have a separate connection to a DB on another machine that is handling the load. Do that within the t
by zo1 10y ago
Don't write it to your main transactional database, then.
Have a separate connection to a DB on another machine that is handling the load. Do that within the transaction/request from the client, or pop it into a message queue.
The queue will "persist" the data, but will also schedule it for when your other box/DB can ingest it. At the end of the day, if you're pushing too-much data, you need to have a mechanism in place to let you scale. To me, a simple solution would be a message queue server with plenty of redundant storage space. From there on, you keep adding more consumers.
- GordonS 10y agoIf the message queue becomes a bottleneck, buffering/batching can also help - instead of sending individual messages to the queue, send them in batches (based on the number, time period, or both).