4 ms·
I agree that no matter what you're going to store unstructured binary data, like audio files, in object storage. But that is perfectly compatible with storing s
by georgewfraser 6y ago
I agree that no matter what you're going to store unstructured binary data, like audio files, in object storage. But that is perfectly compatible with storing structured data in a relational database.
> they are great if the use case is to get answers to questions like "top 3 operators per team in volume of calls..."
You are straw-manning Snowflake/BQ. Just because they are SQL database systems doesn't mean you have to do 100% of your analysis in SQL. You can use other systems, like Spark, PyTorch, Tensorflow, to work with data that you manage inside a RDBMS. There are some issues with bandwidth getting data between systems, but these issues are getting solved (by Arrow!) and in the meantime unload-to/load-from S3 is a good workaround.
I've heard a lot of people make these same arguments and I've tentatively concluded it's mostly motivated reasoning. Engineers like to engineer things. They start by trying to make the obvious, boring system work, but when they run into an obstacle they immediately jump to "I need to build a new system using $TECHNOLOGY."