3 ms·
Are you really doing OLTP transactions on a table with millions of rows that require complex searches or are you doing OLAP style reporting? I don’t think anyo
by scarface74 7y ago
Are you really doing OLTP transactions on a table with millions of rows that require complex searches or are you doing OLAP style reporting?
I don’t think anyone would suggest using an ORM for reporting, aggregating, etc. I would even go so far as suggesting using a different type of database/table structure - a columnar store and send data to a reporting database. A reporting query over millions of rows wouldn’t be real time anyway.
- AdrianB1 7y agoJust keeping the lights on in manufacturing plants with thousands of sensors and counters monitoring the production. Just data collection is useless if you don't put it on good use and that requires some aggregation and correlation to tell what is going on on the floor.
- scarface74 7y agoMy only infrastructure/data analytics experience outside of standard RDMS is with AWS so anytime I mention an AWS service feel free to substitute a non AWS/open source alternative... For aggregation and reporting my usual go to would be Redshift - a custom AWS version of Postgres that uses columnar storage instead of rows and/or something like Kinesis (aka similar to Kafka) for streaming data and real-time aggregation while the data is being captured.