6 ms·

> For example, in R, we try to avoid loops because they are very inefficient This was true before, but the performance of for loops has been improved a lot lat

> For example, in R, we try to avoid loops because they are very inefficient This was true before, but the performance of for loops has been improved a lot later years, and while vectorization is still faster, for loops are no longer a no-no See https://www.r-bloggers.com/2022/02/avoid-loops-in-r-really/ https://www.r-bloggers.com/2022/02/avoid-loops-in-r-really/

_Wintermute 4y ago
It's a really sticky misconception. I've seen many beginners telling others to "never ever use loops in R", and so you end up with nested sapply()s or whatever soon-to-be-deprecated tidyverse functions are in vogue that nobody can reason about.
- em500 4y ago
  So Rob Pike’s rule 1 and 2 again: Rule 1. You can't tell where a program is going to spend its time. Bottlenecks occur in surprising places, so don't try to second guess and put in a speed hack until you've proven that's where the bottleneck is. Rule 2. Measure. Don't tune for speed until you've measured, and even then don't unless one part of the code overwhelms the rest. https://users.ece.utexas.edu/~adnan/pike.html https://users.ece.utexas.edu/~adnan/pike.html
  - teruakohatu 4y ago
    That holds true in general but when doing numerical calculations on a large amount of data, taking speed into account is necessary. You usually know approximately the time penalty for not doing so can evaluate the extra time spent coding verses time spent waiting for results. For example if I am writing a toy neutral network with a small dataset I don't care how optimized it is, or if it runs slowly on the CPU. But when training a large network on a large amount of data it is well worth spending extra effort from the start to ensure as much work as possible is done on a GPU and writing it to ensure if can support multiple GPUs.
  - pxmpxm 4y ago
    That's some pretty generic premature optimization cargo culting. If you have a huge data set and some understanding what you're doing, the bottlebecks will be pretty obvious.
    - deleted 4y ago
      [deleted]
    - em500 4y ago
      Apparently not obvious enough for people to estimate in advance if they should avoid looping. Pike point is not just to avoid premature optimization. It’s to measure bottlenecks. Because due to changing language and hardware developments, what you think you knew to be true might become outdated.
    - pjmlp 4y ago
      One of the reasons why Mark Godbolt created compiler explorer was to prove teammates that what for them was pretty obvious actually wasn't.
      - adgjlsfhk1 4y ago
        There's a reason Godbolt was made for C/C++ rather than python/R. In a fast language you need to know what the compiler is doing to know what's slow. In a slow language, the slow part is pretty much always just "code that does anything in the language".
        pjmlp 4y ago
        Python is only slow because so far there has been a huge disregard for JIT implementations, versus how other dynamic languages have decided to deal with perfomance issues.
        adgjlsfhk1 4y ago
        PyPy definitely shows that python could be 5x faster than it is, however this would still be ~10x slower than Julia/C/C++ (and R is roughly 5-10x slower than python now)
  - naijaboiler 4y ago
    Avoid the allure of premature optimization
    - DennisP 4y ago
      But embrace the repulsion from belated pessimization. As Len Lattanzi said, it's the leaf of no good.
- vharuck 4y ago
  Agreed. The most common reason loops become bottlenecks is people "adding onto" vectors or dataframes. This causes a whole new vector to be created, the data from the old one copied into it, and then the new data filled in at the end. You'll rarely notice the performance hit unless you stick it in a loop that runs tens of thousands of times. For those who want to avoid it and still use a loop, you can create a vector beforehand with the final length and fill it in. If you don't know the final length, create a vector with a good guess for length, double its length whenever it gets full, and then crop off the unused tail when you're done.
- deng 4y ago
  > so you end up with nested sapply()s And that's usually not even vectorizing anything, it just hides the for-loop that is buried somewhere in the apply-code...
rlh2 4y ago
The obsession with cpu speed almost always confuses me in these topics. Time it takes to program is way more important, and that’s where a terse language like R shines. The base/most common functions are almost always executing C anyway. It’s kind of like lisp in that it’s easy to write slow code, but who cares if it’s “fast enough”? Also, it’s almost always easy to speed up if necessary at the R level and R’s C API is also easy to use for for numeric computing/optimization which is exposed at the C level if you want to use it.
- kkoncevicius 4y ago
  It depends. Take for example any omic dataset where you might need to run a GLM model on ~500,000 rows. Codes I've seen for this operation can range in time from taking 30 minutes to 2 days. My take away here is that, sure, for one operation the speed is not that critical, but there is always the case where that one operation will be used close to a million times in one analysis and then it all adds up. On top of that if it's implemented in C then the invocation from R to C and back will be happening that many times which adds to the slowness.
  - derbOac 4y ago
    Yes, I use R, Julia, and Python from time to time depending on the case and my mood and they all have their advantages and disadvantages. R is more than fast enough for straightforward prototypical analyses where a lot of the code is calling C or something lower level and you're not introducing something "new" to the interpreter system. But if you want to do some unusual optimization there's going to be something that bottlenecks everything unless you go into C/C++/Fortran yourself, and then Julia is a good compromise. I've had times when Julia didn't save any time whatsover, and other times when it took something that would literally run over a week at least in R and it was done in 30 minutes in Julia. Having said that, the more I use Julia the more I find myself scratching my head about it. It's very elegant but it's just low-level enough that sometimes I wonder if it's worth it over, say, modern C++ or something similarly low level, which tends to have nice abstracted libraries that have accumulated over the years. I also have the general impression, mentioned in a controversial post discussed here on HN, that a lot of Julia libraries I've used just don't quite work for mysterious reasons I've never been able to figure out. Everything with Julia has gotten better with time but I still have this sense that I could put a lot of time into some codebase, and have it just hit a wall because of some dependency that's not operating as documented. There's kind of an embarrassment of riches in numerical computing today, and yet I still have the feeling there's room for something else. Maybe that's the mythical golden language that's lured all sorts of language developers since the beginning though.
johnmyleswhite 4y ago
Does the article you linked not show that a loop is 10x slower than vectorization for computing square roots? The fact that 10x is better than the 60x slowdown for vapply isn't really evidence that loops are a reasonable alternative to vectorization yet.
npalli 4y ago
Sorry, not following the logic here. From the article, vectorization[1] is more than 10 times faster than a loop. How is this an endorsement for "for" loops. [1] Vectorization is more than ten times faster than the naive loop.

_Wintermute 4y ago

It's a really sticky misconception. I've seen many beginners telling others to "never ever use loops in R", and so you end up with nested sapply()s or whatever soon-to-be-deprecated tidyverse functions are in vogue that nobody can reason about.

em500 4y ago
So Rob Pike’s rule 1 and 2 again: Rule 1. You can't tell where a program is going to spend its time. Bottlenecks occur in surprising places, so don't try to second guess and put in a speed hack until you've proven that's where the bottleneck is. Rule 2. Measure. Don't tune for speed until you've measured, and even then don't unless one part of the code overwhelms the rest. https://users.ece.utexas.edu/~adnan/pike.html https://users.ece.utexas.edu/~adnan/pike.html
- teruakohatu 4y ago
  That holds true in general but when doing numerical calculations on a large amount of data, taking speed into account is necessary. You usually know approximately the time penalty for not doing so can evaluate the extra time spent coding verses time spent waiting for results. For example if I am writing a toy neutral network with a small dataset I don't care how optimized it is, or if it runs slowly on the CPU. But when training a large network on a large amount of data it is well worth spending extra effort from the start to ensure as much work as possible is done on a GPU and writing it to ensure if can support multiple GPUs.
- pxmpxm 4y ago
  That's some pretty generic premature optimization cargo culting. If you have a huge data set and some understanding what you're doing, the bottlebecks will be pretty obvious.
  - deleted 4y ago
    [deleted]
  - em500 4y ago
    Apparently not obvious enough for people to estimate in advance if they should avoid looping. Pike point is not just to avoid premature optimization. It’s to measure bottlenecks. Because due to changing language and hardware developments, what you think you knew to be true might become outdated.
  - pjmlp 4y ago
    One of the reasons why Mark Godbolt created compiler explorer was to prove teammates that what for them was pretty obvious actually wasn't.
    - adgjlsfhk1 4y ago
      There's a reason Godbolt was made for C/C++ rather than python/R. In a fast language you need to know what the compiler is doing to know what's slow. In a slow language, the slow part is pretty much always just "code that does anything in the language".
      - pjmlp 4y ago
        Python is only slow because so far there has been a huge disregard for JIT implementations, versus how other dynamic languages have decided to deal with perfomance issues.
        adgjlsfhk1 4y ago
        PyPy definitely shows that python could be 5x faster than it is, however this would still be ~10x slower than Julia/C/C++ (and R is roughly 5-10x slower than python now)
- naijaboiler 4y ago
  Avoid the allure of premature optimization
  - DennisP 4y ago
    But embrace the repulsion from belated pessimization. As Len Lattanzi said, it's the leaf of no good.
vharuck 4y ago
Agreed. The most common reason loops become bottlenecks is people "adding onto" vectors or dataframes. This causes a whole new vector to be created, the data from the old one copied into it, and then the new data filled in at the end. You'll rarely notice the performance hit unless you stick it in a loop that runs tens of thousands of times. For those who want to avoid it and still use a loop, you can create a vector beforehand with the final length and fill it in. If you don't know the final length, create a vector with a good guess for length, double its length whenever it gets full, and then crop off the unused tail when you're done.
deng 4y ago
> so you end up with nested sapply()s And that's usually not even vectorizing anything, it just hides the for-loop that is buried somewhere in the apply-code...

rlh2 4y ago

The obsession with cpu speed almost always confuses me in these topics. Time it takes to program is way more important, and that’s where a terse language like R shines. The base/most common functions are almost always executing C anyway. It’s kind of like lisp in that it’s easy to write slow code, but who cares if it’s “fast enough”? Also, it’s almost always easy to speed up if necessary at the R level and R’s C API is also easy to use for for numeric computing/optimization which is exposed at the C level if you want to use it.

kkoncevicius 4y ago
It depends. Take for example any omic dataset where you might need to run a GLM model on ~500,000 rows. Codes I've seen for this operation can range in time from taking 30 minutes to 2 days. My take away here is that, sure, for one operation the speed is not that critical, but there is always the case where that one operation will be used close to a million times in one analysis and then it all adds up. On top of that if it's implemented in C then the invocation from R to C and back will be happening that many times which adds to the slowness.
- derbOac 4y ago
  Yes, I use R, Julia, and Python from time to time depending on the case and my mood and they all have their advantages and disadvantages. R is more than fast enough for straightforward prototypical analyses where a lot of the code is calling C or something lower level and you're not introducing something "new" to the interpreter system. But if you want to do some unusual optimization there's going to be something that bottlenecks everything unless you go into C/C++/Fortran yourself, and then Julia is a good compromise. I've had times when Julia didn't save any time whatsover, and other times when it took something that would literally run over a week at least in R and it was done in 30 minutes in Julia. Having said that, the more I use Julia the more I find myself scratching my head about it. It's very elegant but it's just low-level enough that sometimes I wonder if it's worth it over, say, modern C++ or something similarly low level, which tends to have nice abstracted libraries that have accumulated over the years. I also have the general impression, mentioned in a controversial post discussed here on HN, that a lot of Julia libraries I've used just don't quite work for mysterious reasons I've never been able to figure out. Everything with Julia has gotten better with time but I still have this sense that I could put a lot of time into some codebase, and have it just hit a wall because of some dependency that's not operating as documented. There's kind of an embarrassment of riches in numerical computing today, and yet I still have the feeling there's room for something else. Maybe that's the mythical golden language that's lured all sorts of language developers since the beginning though.

johnmyleswhite 4y ago

Does the article you linked not show that a loop is 10x slower than vectorization for computing square roots? The fact that 10x is better than the 60x slowdown for vapply isn't really evidence that loops are a reasonable alternative to vectorization yet.

npalli 4y ago

Sorry, not following the logic here. From the article, vectorization[1] is more than 10 times faster than a loop. How is this an endorsement for "for" loops. [1] Vectorization is more than ten times faster than the naive loop.