3 ms·
At my last gig, one of our services was backed by mysql had a table that was sharded (so, actually many tables) and partitioned so data would fall off after N d
by sethammons 4y ago
At my last gig, one of our services was backed by mysql had a table that was sharded (so, actually many tables) and partitioned so data would fall off after N days. Each table was a couple hundred gigabytes, couple hundred million rows. One writer, and three readers.
Our biggest problem was that our public api could delete and/or bulk delete data out of these tables, causing locking issues for other clients and services that needed responses from the same tables. We played around with different ways to manage it from the db side but ended up just needing to change the public API to return 202 "will delete later" and then we could smooth the load over time.