4 ms·

Nuclear weapon statistics using monoids, groups, and vector spaces in Haskell

66 points by jackpirate 14y ago

jheriko 14y ago
all this proves to me is that Haskell can make even the simplest of tasks difficult to understand or follow. perhaps i miss the point... :/
- rauljara 14y ago
  I wish I didn't agree with you. I'm still a Haskell novice, and I've enjoyed the time I've spent learning it, but this post was just terrifying. All I could think was how simple all that would be in R (or any statistical package), or even Ruby (or any popular scripting language). Of course, one can write overly complicate code in any language. I think the purpose of the post was more to show off conceptually advanced techniques rather than to actually analyze the dataset in a straightforward manner.
  - jackpirate 14y ago
    I think the purpose of the post was more to show off conceptually advanced techniques rather than to actually analyze the dataset in a straightforward manner. Correct. I admit it turned out to be a bit too much for one post, but i wanted to use some real world data (the nukes) to demonstrate the techniques. Edit: I could be wrong, but I don't think R (or any other stats package) supports "group subtraction" of distributions, which is how we calculate the survivable nuclear weapons.
    - rauljara 14y ago
      And actually, I want to amend that on second read through it was considerably easier to follow than the first. It's just an article for Haskellers at least a little more advanced than me. I probably should have read it twice before labeling it "terrifying".
  - lambdasquirrel 14y ago
    you mean the whole nukes thing didn't scare you one nilly bit?
- jackpirate 14y ago
  The "cool stuff" is in the commutative diagrams at the end of the article. They show that there are many different ways to train a distribution from data, some of which will be more suited to different tasks. In most languages, we would have to program each of those paths separately. In Haskell, we get all of those paths "for free."
- tel 14y ago
  Well, it's not exactly a simple task. There are a few simple ways to approach it, but isolating the various general structures and creating the maps between them is the bigger task.
- deleted 14y ago
  [deleted]
  - jackpirate 14y ago
    I'm curious why you say that the code would be simpler in magma or gap? The post doesn't actually implement any group theory, it just applies it, which is exactly what you'd have to do anywhere. Of course, it's possible that the internal code of the library would be simpler, but I'm a little skeptical.
- KirinDave 14y ago
  What was actually difficult to understand about the code? I know the chained function style of haskell: let list_usa = fmap (\row -> row^._4) $ filter (\row -> (row^._1)=="USA" ) rawdata Is a little long but this is basically the same sort of thing you see in OO languages with object.chaining.methods(with,arguments).together.forever(for(succintness { forever }). Monoids are not complicated. They're the _easiest way to talk about combinable groups of things generically_ and lots of programs use echos of that concept when abstracting over collection objects. Monoids just finish the job by saying, "Anything which can be empty and combined associatively (think: adding integers, which also forms a monoid). These abstractions make your life easier, not harder. But they can be a bit confusing at first because they're so universal and it can be so surprising how much milage the same code can get just by swapping monoids. I'm porting some code from Haskell to Clojure right now for various reasons, and man do I miss monoids and the Writer monad. Producing machine-readable concatenated outputs as you go through a computation with ease is so powerful, and a lot of my time is spent writing less-powerful re-implementations of this feature.
  - jlarocco 14y ago
    > What was actually difficult to understand about the code? Really? How about the CSV reading, from the article: import Data.Csv import qualified Data.Vector as V import qualified Data.ByteString.Lazy.Char8 as BS main = do Right rawdata <- fmap (fmap V.toList . decode True) $ BS.readFile "nukes-list.csv" :: IO (Either String [(String, String, String, Int)]) let list_usa = fmap (\row -> row^._4) $ filter (\row -> (row^._1)=="USA" ) rawdata let list_uk = fmap (\row -> row^._4) $ filter (\row -> (row^._1)=="UK" ) rawdata let list_france = fmap (\row -> row^._4) $ filter (\row -> (row^._1)=="France") rawdata let list_russia = fmap (\row -> row^._4) $ filter (\row -> (row^._1)=="Russia") rawdata let list_china = fmap (\row -> row^._4) $ filter (\row -> (row^._1)=="China" ) rawdata putStrLn $ "List of American nuclear weapon sizes = " ++ show list_usa Here's how I'd do it in Python (note I haven't tried this code, but it'd be pretty close to this): import csv countries = dict() with open("nukes-list.csv", 'rb') as f: csvfile = csv.reader(f) for row in csvfile: tmp = countries.get(row[0], list()) tmp.append(int(row[4])) countries[row[0]] = tmp print("List of American nuclear weapon sizes =", countries['USA']) How can anybody argue the Haskell version is easier to understand? The rest of the code is basically calling sum and filter on the data or calling library functions, and it would be almost exactly the same in Python, or even C++.
- wonderzombie 14y ago
  This doesn't strike me as falling to the category of "simplest of tasks."
- lambdasquirrel 14y ago
  Haskell has a way of making the hard things easy and the easy things hard. The high level, pretty stuff is great. The ugly stuff is, for example, getting what we want out of `rawdata`. You're probably thinking stuff like: let list_usa = fmap (\row -> row^._4) $ filter (\row -> (row^._1)=="USA" ) rawdata which is just saying, column 1 is the country, and we only want the rows that are "USA," and from those rows, we only want the fourth column. It would've been simpler if it were just arrays or lists, but that wouldn't be as powerful or as composable as lenses. A better, more readable way would've been to not use those lambdas, since he's reusing the accessors over and over anyway: let getKaboomieColumn row = row^._4 let trueIfUSA row = row^._1 == "USA" let list_usa = fmap getKaboomieColumn $ filter trueIfUSA rawdata ... When you read it that way, it looks a bit less intimidating.
dschiptsov 14y ago
Is it more precise, uses less resources, runs faster than if done with R or Octave? What are the benefits, if any?
- jackpirate 14y ago
  When analyzing data in stages, algebra saves you from repeating work you've already done. It also makes implementing the library's back end easier.

66 points by jackpirate 14y ago

all this proves to me is that Haskell can make even the simplest of tasks difficult to understand or follow. perhaps i miss the point... :/

rauljara 14y ago
I wish I didn't agree with you. I'm still a Haskell novice, and I've enjoyed the time I've spent learning it, but this post was just terrifying. All I could think was how simple all that would be in R (or any statistical package), or even Ruby (or any popular scripting language). Of course, one can write overly complicate code in any language. I think the purpose of the post was more to show off conceptually advanced techniques rather than to actually analyze the dataset in a straightforward manner.
- jackpirate 14y ago
  I think the purpose of the post was more to show off conceptually advanced techniques rather than to actually analyze the dataset in a straightforward manner. Correct. I admit it turned out to be a bit too much for one post, but i wanted to use some real world data (the nukes) to demonstrate the techniques. Edit: I could be wrong, but I don't think R (or any other stats package) supports "group subtraction" of distributions, which is how we calculate the survivable nuclear weapons.
  - rauljara 14y ago
    And actually, I want to amend that on second read through it was considerably easier to follow than the first. It's just an article for Haskellers at least a little more advanced than me. I probably should have read it twice before labeling it "terrifying".
- lambdasquirrel 14y ago
  you mean the whole nukes thing didn't scare you one nilly bit?
jackpirate 14y ago
The "cool stuff" is in the commutative diagrams at the end of the article. They show that there are many different ways to train a distribution from data, some of which will be more suited to different tasks. In most languages, we would have to program each of those paths separately. In Haskell, we get all of those paths "for free."
tel 14y ago
Well, it's not exactly a simple task. There are a few simple ways to approach it, but isolating the various general structures and creating the maps between them is the bigger task.
deleted 14y ago
[deleted]
- jackpirate 14y ago
  I'm curious why you say that the code would be simpler in magma or gap? The post doesn't actually implement any group theory, it just applies it, which is exactly what you'd have to do anywhere. Of course, it's possible that the internal code of the library would be simpler, but I'm a little skeptical.
KirinDave 14y ago
What was actually difficult to understand about the code? I know the chained function style of haskell: let list_usa = fmap (\row -> row^._4) $ filter (\row -> (row^._1)=="USA" ) rawdata Is a little long but this is basically the same sort of thing you see in OO languages with object.chaining.methods(with,arguments).together.forever(for(succintness { forever }). Monoids are not complicated. They're the _easiest way to talk about combinable groups of things generically_ and lots of programs use echos of that concept when abstracting over collection objects. Monoids just finish the job by saying, "Anything which can be empty and combined associatively (think: adding integers, which also forms a monoid). These abstractions make your life easier, not harder. But they can be a bit confusing at first because they're so universal and it can be so surprising how much milage the same code can get just by swapping monoids. I'm porting some code from Haskell to Clojure right now for various reasons, and man do I miss monoids and the Writer monad. Producing machine-readable concatenated outputs as you go through a computation with ease is so powerful, and a lot of my time is spent writing less-powerful re-implementations of this feature.
- jlarocco 14y ago
  > What was actually difficult to understand about the code? Really? How about the CSV reading, from the article: import Data.Csv import qualified Data.Vector as V import qualified Data.ByteString.Lazy.Char8 as BS main = do Right rawdata <- fmap (fmap V.toList . decode True) $ BS.readFile "nukes-list.csv" :: IO (Either String [(String, String, String, Int)]) let list_usa = fmap (\row -> row^._4) $ filter (\row -> (row^._1)=="USA" ) rawdata let list_uk = fmap (\row -> row^._4) $ filter (\row -> (row^._1)=="UK" ) rawdata let list_france = fmap (\row -> row^._4) $ filter (\row -> (row^._1)=="France") rawdata let list_russia = fmap (\row -> row^._4) $ filter (\row -> (row^._1)=="Russia") rawdata let list_china = fmap (\row -> row^._4) $ filter (\row -> (row^._1)=="China" ) rawdata putStrLn $ "List of American nuclear weapon sizes = " ++ show list_usa Here's how I'd do it in Python (note I haven't tried this code, but it'd be pretty close to this): import csv countries = dict() with open("nukes-list.csv", 'rb') as f: csvfile = csv.reader(f) for row in csvfile: tmp = countries.get(row[0], list()) tmp.append(int(row[4])) countries[row[0]] = tmp print("List of American nuclear weapon sizes =", countries['USA']) How can anybody argue the Haskell version is easier to understand? The rest of the code is basically calling sum and filter on the data or calling library functions, and it would be almost exactly the same in Python, or even C++.
wonderzombie 14y ago
This doesn't strike me as falling to the category of "simplest of tasks."
lambdasquirrel 14y ago
Haskell has a way of making the hard things easy and the easy things hard. The high level, pretty stuff is great. The ugly stuff is, for example, getting what we want out of `rawdata`. You're probably thinking stuff like: let list_usa = fmap (\row -> row^._4) $ filter (\row -> (row^._1)=="USA" ) rawdata which is just saying, column 1 is the country, and we only want the rows that are "USA," and from those rows, we only want the fourth column. It would've been simpler if it were just arrays or lists, but that wouldn't be as powerful or as composable as lenses. A better, more readable way would've been to not use those lambdas, since he's reusing the accessors over and over anyway: let getKaboomieColumn row = row^._4 let trueIfUSA row = row^._1 == "USA" let list_usa = fmap getKaboomieColumn $ filter trueIfUSA rawdata ... When you read it that way, it looks a bit less intimidating.

dschiptsov 14y ago

Is it more precise, uses less resources, runs faster than if done with R or Octave? What are the benefits, if any?

jackpirate 14y ago
When analyzing data in stages, algebra saves you from repeating work you've already done. It also makes implementing the library's back end easier.