4 ms·

Wow, hello! This is my repository. I'm happy to answer any questions.

aeonik 2y ago
You specify "fast", can you elaborate on the performance of the collection? How does it compare to the standard core utils? Great work, looks amazing.
- deleted 2y ago
  [deleted]
- anardil 2y ago
  Performance (execution, memory) is generally in the same ballpark as the BSD versions, with some caveats specific to utils that do lots of in place data manipulation. cut comes to mind as an example, slicing and dicing lines into fields quickly without a ton of copies isn't easy. Using Streaming.ByteString generally makes a huge difference, but it's extremely difficult to use unless you get can your mind to meld with the types it wants. Picking it up again months later takes some serious effort.
faragon 2y ago
Very beautiful implementation of the awk interpreter in less than 600 lines! https://github.com/Gandalf-/coreutils/blob/master/Coreutils/Awk.hs https://github.com/Gandalf-/coreutils/blob/master/Coreutils/...
- anardil 2y ago
  Thank you! This is one of my favorites. User declared variables are next on the todo list, when I get back to it.
  - OskarS 2y ago
    It is really gorgeously written Haskell. I’ve only dabbled in Haskell, but you’re really shetting my appetite for digging in deeper.
cosmic_quanta 2y ago
Could you speak to the advantages of Haskell's lazy IO? I only hear about its disadvantages usually
- habitue 2y ago
  I imagine for streaming tools like these it's pretty convenient. You don't have to manage buffers etc, just write code against a massive string and haskell takes care of streaming it for you and pulling in more data when needed. There are libraries that handle it, but they probably have weird types, you can just use functions in the prelude to write a lot of these basic utilities.
  - jerf 2y ago
    Unfortunately, while that may be the dream, it doesn't work out that way if you want good performance. If you look at the source you'll see that it uses things like https://hackage.haskell.org/package/streaming-bytestring-0.3.3/docs/Streaming-ByteString-Char8.html https://hackage.haskell.org/package/streaming-bytestring-0.3... a lot. For one thing, a "string" in Haskell by default is a linked list of unicode characters, so right out of the gate you've got big performance problems if you want to use strings. The exact way laziness is done also has serious performance consequences as well; when dealing with things as small as individual characters all the overhead looms large as a percentage basis. One of the major purposes of any of the several variants of ByteString is to bundle the bytes together, but that means you're back to dealing with chunks. Haskell does end up with a nice API that can abstract over the chunks but it still means you sometimes have to deal with chunks as chunks; if you turn them back into a normal Haskell "string" you lose all the performance advantages. It can still come out fairly nice, but if you want performance it is definitely not just a matter of opening a file and pretending you've just got one big lazy string and you can just ignore all the details; some of the details still poke out.
    - habitue 2y ago
      I mean, I'm aware of the downsides, the OP asked why someone might use it. Ease of use seems like a reasonable upside
bts 2y ago
Hi! A few years ago I found myself wanting an equivalent of `column` that didn’t strip color codes. After I implemented it in Haskell, I found it was useful to use Nix to force statically linking against libraries like gmp to reduce startup time. Perhaps what I ended up doing might be helpful for you too: https://github.com/bts/columnate/blob/master/default.nix https://github.com/bts/columnate/blob/master/default.nix
- anardil 2y ago
  Thank you for the suggestion, I'll give this a whirl! I've fussed around with `--ghc-options '-optl-static -fPIC'` and the like in years past without success.
Vosporos 2y ago
Fantastic work, thank you so much!
anacrolix 2y ago
LOTR fan detected

You specify "fast", can you elaborate on the performance of the collection? How does it compare to the standard core utils? Great work, looks amazing.

deleted 2y ago
[deleted]
anardil 2y ago
Performance (execution, memory) is generally in the same ballpark as the BSD versions, with some caveats specific to utils that do lots of in place data manipulation. cut comes to mind as an example, slicing and dicing lines into fields quickly without a ton of copies isn't easy. Using Streaming.ByteString generally makes a huge difference, but it's extremely difficult to use unless you get can your mind to meld with the types it wants. Picking it up again months later takes some serious effort.

faragon 2y ago

Very beautiful implementation of the awk interpreter in less than 600 lines! https://github.com/Gandalf-/coreutils/blob/master/Coreutils/Awk.hs https://github.com/Gandalf-/coreutils/blob/master/Coreutils/...

anardil 2y ago
Thank you! This is one of my favorites. User declared variables are next on the todo list, when I get back to it.
- OskarS 2y ago
  It is really gorgeously written Haskell. I’ve only dabbled in Haskell, but you’re really shetting my appetite for digging in deeper.

cosmic_quanta 2y ago

Could you speak to the advantages of Haskell's lazy IO? I only hear about its disadvantages usually

habitue 2y ago
I imagine for streaming tools like these it's pretty convenient. You don't have to manage buffers etc, just write code against a massive string and haskell takes care of streaming it for you and pulling in more data when needed. There are libraries that handle it, but they probably have weird types, you can just use functions in the prelude to write a lot of these basic utilities.
- jerf 2y ago
  Unfortunately, while that may be the dream, it doesn't work out that way if you want good performance. If you look at the source you'll see that it uses things like https://hackage.haskell.org/package/streaming-bytestring-0.3.3/docs/Streaming-ByteString-Char8.html https://hackage.haskell.org/package/streaming-bytestring-0.3... a lot. For one thing, a "string" in Haskell by default is a linked list of unicode characters, so right out of the gate you've got big performance problems if you want to use strings. The exact way laziness is done also has serious performance consequences as well; when dealing with things as small as individual characters all the overhead looms large as a percentage basis. One of the major purposes of any of the several variants of ByteString is to bundle the bytes together, but that means you're back to dealing with chunks. Haskell does end up with a nice API that can abstract over the chunks but it still means you sometimes have to deal with chunks as chunks; if you turn them back into a normal Haskell "string" you lose all the performance advantages. It can still come out fairly nice, but if you want performance it is definitely not just a matter of opening a file and pretending you've just got one big lazy string and you can just ignore all the details; some of the details still poke out.
  - habitue 2y ago
    I mean, I'm aware of the downsides, the OP asked why someone might use it. Ease of use seems like a reasonable upside

bts 2y ago

Hi! A few years ago I found myself wanting an equivalent of `column` that didn’t strip color codes. After I implemented it in Haskell, I found it was useful to use Nix to force statically linking against libraries like gmp to reduce startup time. Perhaps what I ended up doing might be helpful for you too: https://github.com/bts/columnate/blob/master/default.nix https://github.com/bts/columnate/blob/master/default.nix

anardil 2y ago
Thank you for the suggestion, I'll give this a whirl! I've fussed around with `--ghc-options '-optl-static -fPIC'` and the like in years past without success.

Vosporos 2y ago

Fantastic work, thank you so much!

anacrolix 2y ago

LOTR fan detected