4 ms·
Not every database is a transactional database. The precise goal of databases like Redshift, Vertica, MemSQL, SAP HANA, Exasol and Impala are to be able to cru
by tmostak 10y ago
Not every database is a transactional database. The precise goal of databases like Redshift, Vertica, MemSQL, SAP HANA, Exasol and Impala are to be able to crunch multi-billion row datasets as fast as possible. Indexes often don't help with analytic queries as frequently good chunks of tables need to be scanned. You might apply a limit at the end of the query but that doesn't mean you don't need to scan billions of rows to get to the result that you are applying a limit to. There are many use cases for speed but the simplest one I can think of is powering Tableau/other BI products. If GPUs help make your dashboard refresh interactively instead of taking 30 seconds when filtering/exploring the data, and allow tens to hundreds of concurrent users on a system rather than just a few, that's a huge win for most of the Fortune 1000.
Data warehousing is a $30B/year business and it often boils down to a price/performance game. Any technology (including but not limited to GPUs) that can change the equation by at least an order-of-magnitude will be disruptive in that space.
- gshulegaard 10y ago> Not every database is a transactional database. ... > The precise goal of databases like Redshift, ... Huh? It was my understanding that Redshift was a fork of PostgreSQL and is therefore not only transactional but also relational. Simple Google search confirms transactions in Redshift: http://docs.aws.amazon.com/redshift/latest/dg/r_BEGIN.html http://docs.aws.amazon.com/redshift/latest/dg/r_BEGIN.html
- dgudkov 10y agoYou're confused by the word transactional. The parent meant transactional in the sense that it's oriented for transactional systems (OLTP). Redshift and other columnar DBMSs are designed for analytic workloads, rather than transactional. Nevertheless, they are relational and support database transactions.
- gshulegaard 10y agoSo I guess the parent's use of transactional was flawed then? I am also confused by the implication that OLTP and OLAP are mutually exclusive. It is my understanding that they are not...
- tmostak 10y agoPeople in the database space use the term "transactional database" loosely to refer to databases optimized for handling simultaneous inserts, deletes and updates at rates often measured in thousands per second - the kind that you would use to say power an airline ticketing system or inventory management for a retailer. Just because a database can support transactions does not mean that it is geared for transactional workloads, and hence would likely not be called transactions. OLTP+OLAP is called HTAP. Its an active area of research but to my knowledge there is still no silver bullet there - systems that do it often store data in both row and columnar format to ensure they are optimized for both, with associated overhead.
- gshulegaard 10y agoI guess I have always thought of a transaction as an ACID-compliant concept and "transactional database" to refer to a DB that supports transactions[1]. To that end, I always thought of OLTP vs. OLAP model and engine optimizations to be more or less orthogonal to whether or not a DB is transactional. I would even suggest that the inclusion of transactions in Redshift as justification for my seemingly unconventional view: > Some PostgreSQL features that are suited to smaller-scale OLTP processing, such as secondary indexes and efficient single-row data manipulation operations, have been omitted to improve performance.[2] But who knows, maybe I am barking up the wrong tree so-to-speak. [1] https://en.wikipedia.org/wiki/Database_transaction#Transactional_databases https://en.wikipedia.org/wiki/Database_transaction#Transacti... [2] http://docs.aws.amazon.com/redshift/latest/dg/c_redshift-and-postgres-sql.html http://docs.aws.amazon.com/redshift/latest/dg/c_redshift-and...
- paulddraper 10y agoYes, Redshift is a fork of PostgreSQL. No, it is not transactional. Yes, it is relational.