3 ms·
You can build a columnstore index in two ways in SQL Server. The easiest is as a non-clustered index on a regular rowstore table. Your inserts are going into
by billgraz 7y ago
You can build a columnstore index in two ways in SQL Server. The easiest is as a non-clustered index on a regular rowstore table. Your inserts are going into the rowstore directly. This physically stores the rows as a rowstore with a columnstore index on top.
The second is as a clustered columnstore index. This stores the rows directly as a columnstore. That's the way I'd store time series data. In this case, all the inserts go into a "delta rowstore" and are moved by a background service into the clustered columnstore structure. Your inserts proceed as fast as any insert into a rowstore. There is overhead to move the data since it's doing a fair bit of compression. But that's going to be true of anything that does compression.
If you want really fast ingest, you can use an InMemory OLTP (aka Hekaton) structure and layer a column store index on top of that. And that InMemory structure can be persisted to disk. That gives you much better performance than a plain rowstore. You'd probably want to write something to move the data to a longer term structure but that would be the fastest way to ingest it I can think of. Using SQL Server :)
- greggyb 7y agoIt's a good overview you've shared of the storage technologies in SQL Server. I agree on memory-optimized being ideal for a real time service. The overhead of column indexing has ruled it out whenever I've benchmarked it for real time use cases. Last time was on SQL 2016, so latest optimizations may have brought it to par.