3 ms·
In the real world... read the CSV into a database (doesn't really matter how fast or slow it is). Access the data from the database.
by ape4 12y ago
In the real world... read the CSV into a database (doesn't really matter how fast or slow it is). Access the data from the database.
- bitwize 12y agoIn the real world you may have to ingest hundreds of megabytes or more of CSV data, daily, from a provider or partner, and CPU time spent ingesting that data is not spent processing transactions. So any speedup you can get is well worth it.
- dilap 12y agoIn the real world, sometimes you have an external data source, in csv (which you can't control), which is changing, which you want to reread as quickly as possible. :)
- ldng 12y agoYou then use PostgreSQL Foreign Data Wrapper (aka SQL/MED)
- deleted 12y ago[deleted]
- victorNicollet 12y agoThere are situations where CSV is the appropriate solution. Our applications read and write CSV data at 1-1.5 million cells per second, and the process can be scaled out at will (there is no central data server, all data is stored on Azure Blob Storage). I'm not sure we could afford the skills and software to achieve these speeds with SQL.