4 ms·

Mm, using someone else's parser would defeat the spirit of the challenge. I think xsv is worthwhile for being a robust parser, not necessarily for its performan

by shawn 8y ago

Mm, using someone else's parser would defeat the spirit of the challenge. I think xsv is worthwhile for being a robust parser, not necessarily for its performance. And my claim is that you'd be able to write it faster, without trading any security guarantees, in Lua, without sacrificing much performance. There's that pesky word, "much" performance. And that's really the interesting part here. How much would you trade away by shedding Rust? My hypothesis is less than 50% additional overhead. Thanks for providing a dataset. I think LuaJIT will match these stats, and it's a good baseline to start with. But yes, the CSV parser is around 15k lines. That'd be the trickiest part.

burntsushi 8y ago
xsv doesn't have its own csv parser, it uses a Rust library to parse csv[1], which is almost 4 times the size of xsv itself. I just happen to have written it. In any case, it would be fun to see an implementation in LuaJIT, especially if you did the CSV parser as well. Although, I think that takes you well outside a weekend project unless you cheat. :-) I don't know the performance characteristics of LuaJIT, but I assume they are better than Python's. I don't know how much better. In any case, this challenge was much more interesting to me when you were talking about Python. Also, I don't really care about a claim that says you could write it faster. That's borderline meaningless in my work unless you're talking about an order of magnitude difference, and I sincerely doubt that. [1] - https://crates.io/crates/csv https://crates.io/crates/csv
- shawn 8y ago
  Also, I don't really care about a claim that says you could write it faster. That's borderline meaningless in my work unless you're talking about an order of magnitude difference, and I sincerely doubt that. Ah, fair point. If there is no benefit to writing software faster, then yes, the discussion is moot. Apologies if it sounded like I was being a dick. I meant to come across as a “player 2 has entered the game,” but it probably just sounded annoying. I’ve been reimplementing some C projects in LuaJIT (more specifically, a dialect of Lisp that compiles to LuaJIT), and it certainly feels an order of magnitude less overhead to get work done. Perhaps it would be interesting to bind the CSV crate to LuaJIT, and then do a direct translation of XSV. The original discussion was about CLI tools, which is one area that scripting languages excel in, and isn’t necessarily enhanced by the benefits of static typing. Creationix has done a lot of excellent work in popularizing LuaJIT’s benefits: https://github.com/luvit/luvit https://github.com/luvit/luvit
  - burntsushi 8y ago
    > I’ve been reimplementing some C projects in LuaJIT (more specifically, a dialect of Lisp that compiles to LuaJIT), and it certainly feels an order of magnitude less overhead to get work done. It is interesting how perspectives change things, because I wouldn't be altogether surprised by that actually. I've used both C and Lua quite a bit myself (although not too much recently), and I can believe that. But if you substituted C for Rust, then I would look at it with great skepticism. That is, I don't see it as types-vs-not-types, but memory-safety-vs-not-memory-safety in addition to having a ecosystem that is very easy to draw on in a granular way. And I didn't think you were too annoying. :-) Building an ffi layer over the csv crate would be cool. It would probably be easiest to do it over csv-core, and then build up the convenience abstractions on the C side (or perhaps even at the Lua level) and therefore avoid the csv crate entirely. csv-core will be simpler because it is designed to work in `no_std` environments, which means it never uses platform specific routines and never allocates. Still though, you'll probably need to find a way to amortize allocation on the Lua side. > and isn’t necessarily enhanced by the benefits of static typing Yeah, I mean I obviously very very very very strongly disagree. We are clearly polar opposites here. If I never write anything substantial in a unityped language again, it will be too soon.
rustyboy 8y ago
> writing faster Has to be one of the most arbitrary benefits to programming, everyone is different how do you possibly use this as a comparable metric? Even if, typing isn't an evil. I write Python for my day job and extensively use the typing libraries because it ends up saving me more time in long run.

by shawn 8y ago

burntsushi 8y ago

xsv doesn't have its own csv parser, it uses a Rust library to parse csv[1], which is almost 4 times the size of xsv itself. I just happen to have written it. In any case, it would be fun to see an implementation in LuaJIT, especially if you did the CSV parser as well. Although, I think that takes you well outside a weekend project unless you cheat. :-) I don't know the performance characteristics of LuaJIT, but I assume they are better than Python's. I don't know how much better. In any case, this challenge was much more interesting to me when you were talking about Python. Also, I don't really care about a claim that says you could write it faster. That's borderline meaningless in my work unless you're talking about an order of magnitude difference, and I sincerely doubt that. [1] - https://crates.io/crates/csv https://crates.io/crates/csv

shawn 8y ago
Also, I don't really care about a claim that says you could write it faster. That's borderline meaningless in my work unless you're talking about an order of magnitude difference, and I sincerely doubt that. Ah, fair point. If there is no benefit to writing software faster, then yes, the discussion is moot. Apologies if it sounded like I was being a dick. I meant to come across as a “player 2 has entered the game,” but it probably just sounded annoying. I’ve been reimplementing some C projects in LuaJIT (more specifically, a dialect of Lisp that compiles to LuaJIT), and it certainly feels an order of magnitude less overhead to get work done. Perhaps it would be interesting to bind the CSV crate to LuaJIT, and then do a direct translation of XSV. The original discussion was about CLI tools, which is one area that scripting languages excel in, and isn’t necessarily enhanced by the benefits of static typing. Creationix has done a lot of excellent work in popularizing LuaJIT’s benefits: https://github.com/luvit/luvit https://github.com/luvit/luvit
- burntsushi 8y ago
  > I’ve been reimplementing some C projects in LuaJIT (more specifically, a dialect of Lisp that compiles to LuaJIT), and it certainly feels an order of magnitude less overhead to get work done. It is interesting how perspectives change things, because I wouldn't be altogether surprised by that actually. I've used both C and Lua quite a bit myself (although not too much recently), and I can believe that. But if you substituted C for Rust, then I would look at it with great skepticism. That is, I don't see it as types-vs-not-types, but memory-safety-vs-not-memory-safety in addition to having a ecosystem that is very easy to draw on in a granular way. And I didn't think you were too annoying. :-) Building an ffi layer over the csv crate would be cool. It would probably be easiest to do it over csv-core, and then build up the convenience abstractions on the C side (or perhaps even at the Lua level) and therefore avoid the csv crate entirely. csv-core will be simpler because it is designed to work in `no_std` environments, which means it never uses platform specific routines and never allocates. Still though, you'll probably need to find a way to amortize allocation on the Lua side. > and isn’t necessarily enhanced by the benefits of static typing Yeah, I mean I obviously very very very very strongly disagree. We are clearly polar opposites here. If I never write anything substantial in a unityped language again, it will be too soon.

rustyboy 8y ago

> writing faster Has to be one of the most arbitrary benefits to programming, everyone is different how do you possibly use this as a comparable metric? Even if, typing isn't an evil. I write Python for my day job and extensively use the typing libraries because it ends up saving me more time in long run.