5 ms·

Why I sometimes like to write my own number crunching code

67 points by dragandj 5y ago

aliasEli 5y ago
In general it is a horrible idea to write your own number crunching code, unless you were a brilliant student in your numerical analysis course.
im3w1l 5y ago
So uhm why did the cosine similarity give values bigger than one in the library and how can we be confident that this code does not have that issue?
- tectonicfury 5y ago
  good question
- username90 5y ago
  The answer is 10 / 14, which expands to his answer if you write out the decimals. The library just truncated it in its output. Doing your own cosine similarity is really hard to mess up.
  - im3w1l 5y ago
    I guess I'm concerned with rounding errors in the intermediate results leading to a similarity of 1.00000001 or something. When you say 10/14, I don't know what you are referring to btw. I couldn't easily find the original bug or library.
    - username90 5y ago
      > When you say 10/14, I don't know what you are referring to btw. I did the calculation manually, the answer is exactly 10 divided by 14. Basically you just take ( 1 * 3 + 2 * 2 + 3 * 1 ) / (3 * 3 + 2 * 2 + 1 * 1), which is 10 / 14. Computing cosine similarities is really simple as I said. Edit: The funny thing is that his answer is actually a bit too low since he truncates instead of rounds at the end. 10 / 14 just continues as 142857 and repeats after that. So if the library is smaller than his then the library is wrong and not him. Anyway, blindly trusting the results of a library is just dumb, they make mistakes so often you are right and they are wrong.
      - dragandj 5y ago
        The matrix library only truncates the printout of matrix entries. The output is, of course, full 32bit or 64bit floating point number.
mjburgess 5y ago
It seems like there's been a crash in the FP language market: Scala, F#, ... Clojure. Anyone any idea why? Scala 3 almost looks attractive.
- wrnr 5y ago
  what do you mean, the FP job market?
- nicoburns 5y ago
  Perhaps because a lot of the idea have been picked up by other languages with bigger ecosystems: Swift, Kotlin, Rust, etc?
  - Zababa 5y ago
    Swift itself has almost no market share in the web space, just like Scala, F#, Clojure, OCaml have almost no market share in the Apple ecosystem. Scala also wasn't used in the android ecosystem, which is I think still the major user of Kotlin. On the other hand, Kotlin is also on the server now, but I don't know if it's replacing Java or Scala (maybe both?). Another thing is that Java itself is adopting lots of functional programming features (records, pattern matching recently). I don't think I've seen any mention of migration from F#, Clojure, OCaml, Haskell, Scala, ... to Rust, but maybe it's easier to pick up than C++ when high performance is needed by teams already used to a ML-like?
- toomanyducks 5y ago
  In my experience, OOP is taught to basically everyone in schools (AP CS is usually in Java, and so was my DSA class) which makes the jump into FP hard to wrap your head around: I've tried a few things in Haskell, and I kept trying to use weird facsimiles of objects in places they should not go. And then Rust came along and wrapped a ton of FP concepts into easy to conceptualize OOP packages, and that reduced my motivation to actually learn FP properly. Still slowly trying to learn these things, though, I don't think Rust really managed to capture all the advantages.
  - 5y ago
queuebert 5y ago
The author is overconfident and underestimates library writers to a fault. Packing a bunch of dot products into a matrix is only the fastest way to compute a batch of cosine similarities if you are relegated to a parallel matrix product function. If you write the procedure in raw CUDA, for example, it is faster to simply broadcast the dot products across threads. That is exactly what the matrix multiply is doing except without the overhead of matrix creation and with potentially greater locality of memory accesses. Edit: I do think it is worthwhile to write your own code so that you better understand what is happening under the hood, so good on the author for that.
- Blackthorn 5y ago
  The author is a library writer!
  - queuebert 5y ago
    My mistake. I took this statement in isolation: "...consider that machine learning libraries are frequently written by grad students on their path to discovery. It's a domain expert with poor programming skills. Or it might be a case of a good programmer who only barely understands the domain…"
- klyrs 5y ago
  > The author is overconfident and underestimates library writers to a fault. Clicking around the website a bit, it would seem that the author is a library writer (some specifically targeting GPUs), and the article is a plug for the books he's written about numerical analysis. How confident are you?
  - queuebert 5y ago
    Not very confident in general. But I have written a LOT of raw CUDA code.
- inimino 5y ago
  This pattern comes up a lot in comments on this type of post. Author makes a dumb but easy-to-follow example to make a point, and then someone points out how the example is dumb. Try to avoid getting hung up on examples -- in this case nobody reading this post should come away thinking they now know more about cosine similarity than a library author. I dare say the author's main points were: - it's not that much code - it's not terrifyingly hard to understand - you might open up performance benefits in your specific circumstance just by knowing how it's calculated after writing the code to do it yourself
tectonicfury 5y ago
I felt that it was a good article, very straightforward, and he made an interesting point about numerical packages being often written by graduate students. Personally, I would like to learn the math, but I would want to first learn from the 'established' books (i.e. the books the author read on his way to this point), only then will I be in a position to judge the real worth of the author's books.
lordnacho 5y ago
The author has written books about this area, so he's pretty confident that he can write numerical libs. That might be right, but for most people, when you come across some need, that's going to not be your speciality. I might need a numerical lib at some point, but I also know that there are things like numerical instability, gremlins in this area that I know some of but not all of. Something popular with lots of people working on it is likely to be far better for me than rolling my own. Of course this does mean that for most things by far, the best way forward is to research which lib is best supported and go with that.
dagw 5y ago
Another case where writing your numerical algorithms can be faster than using the standard lib is if you know you're only working on a small subset of the possible input domain and can write a highly specialized function. For example, a certain function may take arbitrary dimensional complex matrices as input. However if you know that you will only be passing in 3x3 Real positive semidefinite matrices you can probably write a faster function that only works on those matrices by using all that extra information you have. In Matlab I've gotten order of magnitude speedups over the standard functions by rolling my own highly specialized functions.
Zababa 5y ago
I wonder if sometimes always telling people to not do anything themselves and use already made libraries is robbing the industry (and especially juniors) of lots of opportunities to learn. Instead of being in control of what you do, you learn to use something made by other people to simplify things. This is a bit like cooking where in the last 10-20 years I've seen appliances appearing everywhere. Some are very good (I love my slow cooker), some I'm more perplexed about (A stand mixer? I usually get by either with my hand for dense doughs like pizza or bread or a hand mixer for more liquid/light doughs). If everybody uses libraries, how are we training new libraries writers? Maybe part of the solution would be to tell people to use libraries, and then read the code, try to understand them and contribute.
- yen223 5y ago
  I feel like one of the unintended consequence of discouraging people from "rolling their own crypto" is that developers who obey that rule will never be able to learn what good crypto is and isn't. This I believe will lead to more insecure software in the long run.
  - gloriousternary 5y ago
    I think a pretty reasonable compromise is to reserve custom implementations of potentially dangerous things like crypto for personal projects. As an end user I'd rather run software that just uses sodium, but as a developer I agree that it's necessary that we get experience doing it ourselves.
    - deleted 5y ago
      [deleted]
  - Kalium 5y ago
    At the risk of being overly opinionated, I think that developers do not learn what is good or bad cryptography from writing it. We learn what is good or bad code from writing and maintaining it. People learn what is good or bad cryptography from studying the mathematics of cryptography and listening to cryptographers. However, it's very possible for good code to be terrible cryptography for reasons you are very unlikely to learn by writing the code. The catch with trying the reasonable compromises that are acceptable in other specialist areas often backfire in cryptography. "It's fine, this isn't serious, we'll fix it later" is something you can live with when it just results in a half-baked ORM, but much less fine when it results in seriously flawed cryptography that endangers people. Cryptocat - and decryptocat - is a good example on both points.

67 points by dragandj 5y ago

In general it is a horrible idea to write your own number crunching code, unless you were a brilliant student in your numerical analysis course.

im3w1l 5y ago

So uhm why did the cosine similarity give values bigger than one in the library and how can we be confident that this code does not have that issue?

tectonicfury 5y ago
good question
username90 5y ago
The answer is 10 / 14, which expands to his answer if you write out the decimals. The library just truncated it in its output. Doing your own cosine similarity is really hard to mess up.
- im3w1l 5y ago
  I guess I'm concerned with rounding errors in the intermediate results leading to a similarity of 1.00000001 or something. When you say 10/14, I don't know what you are referring to btw. I couldn't easily find the original bug or library.
  - username90 5y ago
    > When you say 10/14, I don't know what you are referring to btw. I did the calculation manually, the answer is exactly 10 divided by 14. Basically you just take ( 1 * 3 + 2 * 2 + 3 * 1 ) / (3 * 3 + 2 * 2 + 1 * 1), which is 10 / 14. Computing cosine similarities is really simple as I said. Edit: The funny thing is that his answer is actually a bit too low since he truncates instead of rounds at the end. 10 / 14 just continues as 142857 and repeats after that. So if the library is smaller than his then the library is wrong and not him. Anyway, blindly trusting the results of a library is just dumb, they make mistakes so often you are right and they are wrong.
    - dragandj 5y ago
      The matrix library only truncates the printout of matrix entries. The output is, of course, full 32bit or 64bit floating point number.

mjburgess 5y ago

It seems like there's been a crash in the FP language market: Scala, F#, ... Clojure. Anyone any idea why? Scala 3 almost looks attractive.

wrnr 5y ago
what do you mean, the FP job market?
nicoburns 5y ago
Perhaps because a lot of the idea have been picked up by other languages with bigger ecosystems: Swift, Kotlin, Rust, etc?
- Zababa 5y ago
  Swift itself has almost no market share in the web space, just like Scala, F#, Clojure, OCaml have almost no market share in the Apple ecosystem. Scala also wasn't used in the android ecosystem, which is I think still the major user of Kotlin. On the other hand, Kotlin is also on the server now, but I don't know if it's replacing Java or Scala (maybe both?). Another thing is that Java itself is adopting lots of functional programming features (records, pattern matching recently). I don't think I've seen any mention of migration from F#, Clojure, OCaml, Haskell, Scala, ... to Rust, but maybe it's easier to pick up than C++ when high performance is needed by teams already used to a ML-like?
toomanyducks 5y ago
In my experience, OOP is taught to basically everyone in schools (AP CS is usually in Java, and so was my DSA class) which makes the jump into FP hard to wrap your head around: I've tried a few things in Haskell, and I kept trying to use weird facsimiles of objects in places they should not go. And then Rust came along and wrapped a ton of FP concepts into easy to conceptualize OOP packages, and that reduced my motivation to actually learn FP properly. Still slowly trying to learn these things, though, I don't think Rust really managed to capture all the advantages.
- 5y ago

queuebert 5y ago

The author is overconfident and underestimates library writers to a fault. Packing a bunch of dot products into a matrix is only the fastest way to compute a batch of cosine similarities if you are relegated to a parallel matrix product function. If you write the procedure in raw CUDA, for example, it is faster to simply broadcast the dot products across threads. That is exactly what the matrix multiply is doing except without the overhead of matrix creation and with potentially greater locality of memory accesses. Edit: I do think it is worthwhile to write your own code so that you better understand what is happening under the hood, so good on the author for that.

Blackthorn 5y ago
The author is a library writer!
- queuebert 5y ago
  My mistake. I took this statement in isolation: "...consider that machine learning libraries are frequently written by grad students on their path to discovery. It's a domain expert with poor programming skills. Or it might be a case of a good programmer who only barely understands the domain…"
klyrs 5y ago
> The author is overconfident and underestimates library writers to a fault. Clicking around the website a bit, it would seem that the author is a library writer (some specifically targeting GPUs), and the article is a plug for the books he's written about numerical analysis. How confident are you?
- queuebert 5y ago
  Not very confident in general. But I have written a LOT of raw CUDA code.
inimino 5y ago
This pattern comes up a lot in comments on this type of post. Author makes a dumb but easy-to-follow example to make a point, and then someone points out how the example is dumb. Try to avoid getting hung up on examples -- in this case nobody reading this post should come away thinking they now know more about cosine similarity than a library author. I dare say the author's main points were: - it's not that much code - it's not terrifyingly hard to understand - you might open up performance benefits in your specific circumstance just by knowing how it's calculated after writing the code to do it yourself

tectonicfury 5y ago

I felt that it was a good article, very straightforward, and he made an interesting point about numerical packages being often written by graduate students. Personally, I would like to learn the math, but I would want to first learn from the 'established' books (i.e. the books the author read on his way to this point), only then will I be in a position to judge the real worth of the author's books.

lordnacho 5y ago

The author has written books about this area, so he's pretty confident that he can write numerical libs. That might be right, but for most people, when you come across some need, that's going to not be your speciality. I might need a numerical lib at some point, but I also know that there are things like numerical instability, gremlins in this area that I know some of but not all of. Something popular with lots of people working on it is likely to be far better for me than rolling my own. Of course this does mean that for most things by far, the best way forward is to research which lib is best supported and go with that.

dagw 5y ago

Another case where writing your numerical algorithms can be faster than using the standard lib is if you know you're only working on a small subset of the possible input domain and can write a highly specialized function. For example, a certain function may take arbitrary dimensional complex matrices as input. However if you know that you will only be passing in 3x3 Real positive semidefinite matrices you can probably write a faster function that only works on those matrices by using all that extra information you have. In Matlab I've gotten order of magnitude speedups over the standard functions by rolling my own highly specialized functions.

Zababa 5y ago

I wonder if sometimes always telling people to not do anything themselves and use already made libraries is robbing the industry (and especially juniors) of lots of opportunities to learn. Instead of being in control of what you do, you learn to use something made by other people to simplify things. This is a bit like cooking where in the last 10-20 years I've seen appliances appearing everywhere. Some are very good (I love my slow cooker), some I'm more perplexed about (A stand mixer? I usually get by either with my hand for dense doughs like pizza or bread or a hand mixer for more liquid/light doughs). If everybody uses libraries, how are we training new libraries writers? Maybe part of the solution would be to tell people to use libraries, and then read the code, try to understand them and contribute.

yen223 5y ago
I feel like one of the unintended consequence of discouraging people from "rolling their own crypto" is that developers who obey that rule will never be able to learn what good crypto is and isn't. This I believe will lead to more insecure software in the long run.
- gloriousternary 5y ago
  I think a pretty reasonable compromise is to reserve custom implementations of potentially dangerous things like crypto for personal projects. As an end user I'd rather run software that just uses sodium, but as a developer I agree that it's necessary that we get experience doing it ourselves.
  - deleted 5y ago
    [deleted]
- Kalium 5y ago
  At the risk of being overly opinionated, I think that developers do not learn what is good or bad cryptography from writing it. We learn what is good or bad code from writing and maintaining it. People learn what is good or bad cryptography from studying the mathematics of cryptography and listening to cryptographers. However, it's very possible for good code to be terrible cryptography for reasons you are very unlikely to learn by writing the code. The catch with trying the reasonable compromises that are acceptable in other specialist areas often backfire in cryptography. "It's fine, this isn't serious, we'll fix it later" is something you can live with when it just results in a half-baked ORM, but much less fine when it results in seriously flawed cryptography that endangers people. Cryptocat - and decryptocat - is a good example on both points.