4 ms·
Since your data is append-only have you considered something like just sticking all of the data in s3? Might be awkward to deal with things like updating threa
by maxmcd 4y ago
Since your data is append-only have you considered something like just sticking all of the data in s3?
Might be awkward to deal with things like updating threads, but just editing the s3 file containing the json response for a given thread might work, and then you can skip having a database entirely.
- cheeseblubber 4y agoOh interesting. I think like you said updates might be a little bit tricky as well as search. I'm using full text search from Postgres so that would be tough on S3. I'm doing some aggressive caching with Vercel and Nextjs which essentially turns in to putting the data in S3.
- maxmcd 4y agoAh didn't think about search, makes sense. Yeah agree the benefits caching are roughly equivalent to storing in s3.
- rmbyrro 4y agoYou can do text search and other SQL queries in S3 data by plugging AWS Athena to it. It isn't very scalable, though. Last time I checked it would handle few dozens of concurrent queries, at most. But for fulltext search, what really makes sense is something like OpenSearch.