4 ms·
This actually rings true with my own experience. Let me share an anecdote: Last year, we built a search service on EC2, which used SQS[1] and SNS[2] as a way o
by duggan 15y ago
This actually rings true with my own experience. Let me share an anecdote:
Last year, we built a search service on EC2, which used SQS[1] and SNS[2] as a way of feeding data from our core architecture into the index.
So, user creates/updates/deletes a message, that information gets sent to a particular SNS topic over HTTPS, a subscribed SQS queue receives message, then search index polls the queue.
SQS and SNS are cheap, scalable and highly available, so we thought it was a great system. We knew that messages were going to disappear occasionally, though (network partition, etc), so we had also decided to spend time building a simple API to resolve these issues.
We would occasionally poll the API directly to effectively perform a diff, by storing a hash "fingerprint" of the data with each entry. Fingerprints would be compared, then a full request would be made for any entry which differed.
At some point, we realized that we actually had obsoleted almost the entire SNS/SQS message queuing mechanism by building a system which made API calls inexpensive; so we removed it from the equation.
If you have a situation where a message has to get there (eventually), in our situation at least, it seems an unnecessary extra step.
Would like to hear if anyone else has had a similar (or completely opposite) experience.
[1] http://aws.amazon.com/sqs/ http://aws.amazon.com/sqs/
[2] http://aws.amazon.com/sns/ http://aws.amazon.com/sns/