9 ms·
21 GB/s CSV Parsing Using SIMD on AMD 9950X
- winterbloom 1y agoThis is a staggering ~3x improvement in just under 2 years since Sep was introduced June, 2023. You can't claim this when you also do a huge hardware jump
- WD-42 1y agoYea wtf is that chart, it literally skips 4 cpu generations where it shows “massive performance gain”. Straight to the trash with this post.
- g-mork 1y agoIt also appears to be reporting whole-CPU vs. single thread, 1.3 GB/sec is not impressive for single thread perf
- iamleppert 1y agoAgreed. How hard is it to keep hardware fixed, load the data into memory, and use a single core for your benchmarks? When I see a chart like that I think, "What else are they hiding?" Folks should check out https://github.com/dathere/qsv https://github.com/dathere/qsv if they need an actually fast CSV parser.
- Remnant44 1y agoI mean... A single 9950x core is going to struggle to do more than 16 GB/second of direct mem copy bandwidth. So being within an order of magnitude of that seems reasonable
- ziml77 1y agoBut it repeats the 0.9.0 test on the new hardware. So the first big jump is a hardware change, but the second jump is the software changes.
- matja 1y ago4 generations? 5950x is Zen 3 9950x is Zen 5
- chupasaurus 1y agoSine Zen 2 (3000) the mobile CPUs are up by a thousand respectively to their desktop counterparts. edit: Or Nx2000 where N is from Zen N.
- hinkley 1y agoAnd even with 2, CPU generations aren't what they used to be back when a candy bar cost less than a dollar.
- freeone3000 1y agoThey claim a 3GB/s improvement versus previous version of sep on equal hardware — and unlike “marketing” benchmarks, include the actual speed achieved and the hardware used.
- stabbles 1y agoDo note that this speed even before the 3GB/s improvement exceeds the bandwidth of most disks, so the bottleneck is loading data in memory. I don't know of many applications where CSV is produced and consumed in memory, so I wonder what the use is.
- freeone3000 1y agoSlower than network! In-memory processing of OLAP tables, streaming splitters, large data set division… but also the faster the parser, the less time you spend parsing and the more you spend doing actual work
- tetha 1y agoThis is honestly something that caught me off-guard a bit. If you have good internal network connectivity, small queries and your relational database has the data in memory, it can be faster to fetch data from the DB via the network than reading it from disk. Like, sure, I can give you an application server with faster disks and more memory and you or me are certainly capable of implementing an application server that could load the data from disk faster than all of that. And then we build caching to keep the hot data in memory, because that's faster. But then we've spent very advanced development resources to build a relational database with some application code at the edge. This can make sense in some high frequency trading situations, but in many more mundane web-backends, a chunky database and someone capable of optimizing stupid queries enable and simplify the work of a much bigger number of developers.
- bee_rider 1y agoYou can also get this with Infiniband, although it is less surprising, and basically what you’d expect to see. I did once use a system where the network bandwidth was in the same ballpark as the memory bandwidth, which might not be surprising for some of the real HPC-heads here but it surprised me!
- jbverschoor 1y agoThey also included 0.9.0 vs 0.10.0. on the new hardware. (21385 vs 18203), so the jump because of software is 17%. Then if we take 0.9.0 on previous hardware (13088) and add the 17%, it's 15375. Version 0.1.0 was 7335. So... 15375/7335 -> a staggering 2.1x improvement in just under 2 years
- perching_aix 1y ago> You can't claim this when you also do a huge hardware jump Well, they did. Personally, I find it an interesting way of looking at it, it's a lens for the "real performance" one could get using this software year over year. (Not saying it isn't a misleading or fallacious claim though.)
- bawolff 1y agoPerhaps, but i think we are well past the moore's law era where a 3x speed up is to be expected just from hardware. Its still a pretty impressive feat in the modern era.
- criddell 1y agoI was expecting to see assembly language and was pleasantly surprised to see C#. Very impressive. Nice work!
- gavinray 1y agoModern .NET has the deepest integration with SIMD and vector intrinsics of what most people would consider "high-level languages". https://learn.microsoft.com/en-us/dotnet/standard/simd https://learn.microsoft.com/en-us/dotnet/standard/simd Tanner Gooding at Microsoft is responsible for a lot of the developments in this area and has some decent blogposts on it, e.g. https://devblogs.microsoft.com/dotnet/dotnet-8-hardware-intrinsics/ https://devblogs.microsoft.com/dotnet/dotnet-8-hardware-intr...
- voidUpdate 1y agoI shudder to think who needs to process a million lines of csv that fast...
- hermitcrab 1y agoFor all its many weaknesses, I believe CSV is still the most common data interchange format.
- adra 1y agoErm, maybe file based? JSON is the king if you count exchanges worldwide a sec. Maybe no 2 is form-data which is basically email multipart, and if course there's email as a format. Very common =)
- hermitcrab 1y agoI meant file-based.
- devmor 1y agoI honestly wonder if JSON is king. I used to think so until I started working in fintech. XML is unfortunately everywhere.
- hermitcrab 1y agoJSON isn't great for tabular data. And an awful lot of data is tabular.
- devmor 1y agoYeah, I don’t like parsing XML, but I’d rather do that than deal with the Lovecraftian API design that comes with complex JSON representations.
- hajile 1y agoJSON tabular data only adds a couple of brackets per line and at the start/end of the file vs CSV. In exchange for these bits (that basically disappear when compressed), you get a guaranteed standard formatting. Seems like a decent tradeoff to me.
- vessenes 1y agoIf we are lucky we will see Arthur Whitney get triggered and post either a one liner beating this or a shakti engine update and a one liner beating this. Progress!
- stabbles 1y agoInstead of doing 4 comparisons against each character `\n`, `\r`, `;` and `"` followed by 3 or operations, a common trick is to do 1 shuffle, 1 comparison and 0 or operations. I blogged about this trick: https://stoppels.ch/2022/11/30/io-is-no-longer-the-bottleneck-part-2.html https://stoppels.ch/2022/11/30/io-is-no-longer-the-bottlenec... (Trick 2) Edit: they do make use of ternary logic to avoid one or operation, which is nice. Basically (a | b | c) | d is computed using `vpternlogd` and `vpor` resp.
- Aardwolf 1y agoTake that, Intel and your "let's remove AVX-512 from every consumer CPU because we want to put slow cores on every single one of them and also not consider multi-pumping it"
- tadfisher 1y agoA lot of this stems from the 10nm hole they had to dig themselves out from. Yields are bad, so costs are high, so let's cut the die as much as possible, ship Atom-derived cores and market it as an energy-saving measure. The expensive parts can be bigger and we'll cut the margins on those to retain the server/cloud sector. Also our earnings go into the shitter and we lose market share anyway, but at least we tried.
- wtallis 1y agoThis issue is less about Intel's fab failures and more about their inability to decouple their architecture update cadence from their fab progress. They stopped iterating on their CPU designs while waiting for 10nm to get fixed. That left them with an oversized P core and an outdated E core, and all they could do for Alder Lake was slap them onto one die and ship it, with no ability to produce a well-matched pair of core designs in any reasonable time frame. We're still seeing weird consequences of their inability to port CPU designs between processes and fabs: this year's laptop processors have HyperThreading only in the lowest-cost parts—those that still have the CPU chiplet fabbed at Intel while the higher core count parts are made by TSMC.
- imtringued 1y agoConsidering the non-standard nature of CSV, quoting throughput numbers in bytes is meaningless. It makes sense for JSON, since you know what the output is going to be (e.g. floats, integers, strings, hashmaps, etc). With CSV you only get strings for each column, so 21 GB/s of comma splitting would be the pinnacle of meaninglessness. Like, okay, but I still have to parse the stringy data, so what gives? Yeah, the blog post does reference float parsing, but a single float per line would count as "CSV". Now someone might counter and say that I should just read the README.MD, but then that suspicion simply turns out to be true: They don't actually do any escaping or quoting by default, making the quoted numbers an example of heavily misleading advertising.
- liuliu 1y agoCSV is standardized in RFC 4180 (well, as standardized as most of what we considered internet "standard"). Otherwise agree, if you don't do escaping (a.k.a. "quoting", the same thing for CSV), you are not implementing it correctly. For example, if you quote a line break, in RFC 4180, this line break will be in that quoted string, but if you don't need to handle that, you can implement CSV parsing much faster (proper handling line break with quoted string requires 2-pass approach (if you are going to use many-core) while not handling it at all can be done with 1-pass approach). I discussed about this detail in https://liuliu.me/eyes/loading-csv-file-at-the-speed-limit-of-the-nvme-storage/ https://liuliu.me/eyes/loading-csv-file-at-the-speed-limit-o...
- a3w 1y agoSide note: RFCs are great standards, as they are readable. As an example of how not to do it: XML can be assumed a standard, but I cannot afford to read it. DIN/ISO is great for manufacturing in theory, but bad for zero-cost of initial investment like IT.
- zeristor 1y agoWhy not use Parquet?
- constantcrying 1y agoOr HDF5 or any other format which is actually meant to store large amounts of floating point data.
- mcraiha 1y agoExcel does not output Parquet.
- speed_spread 1y agoTrue. But also Excel probably collapses into a black hole going straight to hell trying to handle 21GB of data.
- hermitcrab 1y agoExcel .xlsx files are limited to 1,048,576 rows and 16,384 columns. Excel .xls files are limited to 65,536 rows and 256 columns.
- mihular 1y ago21GB/s, not 21GB ...
- anthk 1y agomawk would handle a 21 GB csv (or maybe one true awk) fast enough.
- buyucu 1y agoExcel often outputs broken csv :)
- hinkley 1y agoI have been privileged in my career to never need to parse Excel output but occasionally feed it input. Especially before Grafana was a household name. Putting something out so manager stops asking you 20 questions about the data is a double edged sword though. Those people can hallucinate more than a pre-Covid AI engine. Grafana is just weird enough that people would rather consume a chart than try to make one, then you have some control over the acid trip.
- deleted 1y ago[deleted]
- chao- 1y agoIt feels crazy to me that Intel spent years dedicating die space on consumer SKUs to "make fetch happen" with AVX-512, and as more and more libraries are finally using it, as Intel's goal is achieved, they have removed AVX-512 from their consumer SKUs. It isn't that AMD has better AVX-512 support, which would be an impressive upset on it's own. Instead, it is only that AMD has AVX-512 on consumer CPUs, because Intel walked away from their own investment.
- deleted 1y ago[deleted]
- deleted 1y ago[deleted]
- MortyWaves 1y agoIt’s wild seeing how stupid Intel is being.
- neonsunset 1y agoIf it's any consolation, Sep will happily use AVX-512 whenever available, without having to opt into that explicitly, including the server parts, as it will most likely run under a JIT runtime (although it's NAOT-compatible). So you're not missing out by being forced to target the lowest common denominator.
- sitkack 1y agoThat is what Intel does, they build up a market (Optane) and then do a rug pull (Depth Cameras). They continue to do this thing where they do a huge push into a new technology, then don't see the uptake and let it die. Instead of building slowly and then at the right time, doing a big push. Optane support was just getting mature in the Linux kernel when they pulled it. And they focused on some weird cost cutting move when marketing it as a ram replacement for semi-idle VMs, ok. They keep repeating the same mistakes all the way back to https://en.wikipedia.org/wiki/Intel_iAPX_432 https://en.wikipedia.org/wiki/Intel_iAPX_432
- FpUser 1y ago
- constantcrying 1y agoThere are very good alternatives to csv for storing and exchanging floating point/other data. The HDF5 format is very good and allows far more structure in your files, as well as metadata and different types of lossless and lossy compression.
- jsploit 1y agoThe lack of concurrent access support in the official HDF5 library (the only implementation with full format support) can be a major drawback. There is ongoing work on that front [1] though it's unclear when it will land. [1] https://github.com/LifeboatLLC/MT-HDF5 https://github.com/LifeboatLLC/MT-HDF5
- anthk 1y ago> Net 9.0 heh, do it again with mawk.
- jerryseff 1y ago[flagged]
- h4ck_th3_pl4n3t 1y agoThen show us your elixir implementation?
- chpatrick 1y agoIn my experience I've found it difficult to get substantial gains with custom SIMD code compared to modern compiler auto-vectorization, but to be fair that was with more vector-friendly code than JSON parsing.
- theropost 1y agoI need this, just finished 300GB of CSV extracts, and manipulating, data integrity checks, and so on take longer than they should.
- constantcrying 1y agoWhy wouldn't you use a data format meant to store floating point numbers? HDF5 gives you a great way to store such data.
- theropost 1y agoSounds interesting, I'll give it a look. I'm unfortunately limited to CSV, XML, or XLS from the source system, then am transforming it and loading it into another DB.
- haberman 1y agoThe article doesn't clearly define what this 21 GB/s code is doing. - What format exactly is it parsing? (eg. does the dialect of CSV support quoted commas, or is the parser merely looking for commas and newlines)? - What is the parser doing with the result (ie. populating a data structure, etc)?
- hinkley 1y agoThere’s a calculation for ns/row in the article that is never translated into rows per second but is about 27 ns/row, which is about 37,000 per second. Which means these rows are 570k apiece if that’s 21GB. Which seems like an awfully cooked benchmark.
- gitroom 1y agotbh the way intel keeps killing cool tech gets on my nerves - wish they'd just stick it out for once
- auxten 1y agoAnyone tried ClickHouse on CSV?