5 ms·

I don't really understand the need for a compiler in R. All the main packages that have to do any heavy lifting have had their functions outsourced to fortran o

• _Wintermute 2y ago
  > R is also thriving in bioinformatics with Python trailing behind as an afterthought. Maybe this is sub-field dependent, I'm a bioinformatician who hasn't touched R in about 5 years, and everything is now in python.
  • enmce 2y ago
    Same expirience,about 13 years in the field, most tasks done in python.
  • tetris11 2y ago
    oh wow, I made a sweeping generalization - apologies. In single cell, bulk RNA-seq, ATAC, ChIP, I'd say that there are more R packages for the analysis of these omics than Python packages.
    • flobosg 2y ago
      Python is catching up on single-cell transcriptomics; see AnnData, Scanpy, et al.
      • tetris11 2y ago
        ScanPy/AnnData has been dead in the water for a while now, and most people use Seurat due to its operability with many many downstream extensions
        • flobosg 2y ago
          > has been dead in the water for a while now Both are under active development and are used in several transcriptomics atlas projects, as far as I can tell.
          • tetris11 2y ago
            Those atlases were established back when Scanpy and Seurat were relatively beta and were still fighting out the tool space. Look at the packages now for integration, pseudo time, pseudobulk - R (and therefore Seurat) dominates heavily
            • valarauko 2y ago
              Disagree - Seurat had first mover advantage with single cell but sucked with larger datasets that Scanpy could handle till the big change in Seurat 5. The preference for either Seurat/Scanpy is incredibly lab specific. That said, Seurat is better documented for sure, but the ecosystem for both is incredibly rich and flourishing.
              • cd4plus 2y ago
                yeah, I also disagree with this. It's true Seurat is still heavily used for scRNA/scATAC but I see most new models increasingly being written/tooled for python and based on anndata. Geneformer, scGPT, scVI etc. I wish there was better operability between the scverse stuff and Seurat, but Seurat went their own way from SCE/bioconductor so that's probably not going to happen.
                • tetris11 2y ago
                  To be fair to them, getting anything submitted to buoconductor requires a ton of effort, and the pay off is often less concise code
        • _Wintermute 2y ago
          I have no dog in the Scanpy/Seurat argument, but AnnData is becoming very popular as a data format even outside of single-cell omics.
  • mbreese 2y ago
    It’s also training dependent. It took a long time for Perl to go away from daily use in bioinformatics, solely dependent upon how common it was twenty years ago. In our lab, there is me, a (mid career) polyglot who mixes Python, Go, Java, and R daily. I use tab delimited text files to transfer data. I also grew up coding in C++ and like learning new languages. We also have a (mid career) staff scientist who grew up in Perl, but switched 100% to R and an (early career) postdoc who has always used 100% R. For both of these people, if work can be done in R, it is. If it can’t be done in R, they figure out how to do it in R anyway (even if that is shelling out to another program). We also have a (young) grad student that is 95% Python. They try to keep to the Python tooling, even though they are quite aware of the R ecosystem. There is a generational shift in the field, and it is more apparent each year. I find it interesting that Python took over from Perl first and now it’s trying to take over from R.
• wodenokoto 2y ago
  > All the main packages that have to do any heavy lifting have had their functions outsourced to fortran or C decades ago. And apparently that's the problem, FTA: > Fortran isn’t as popular as it used to be, and we encounter issues when compiling it with modern compilers like LLVM. Ensuring Fortran compiles across all desired architectures and operating systems has been a persistent challenge. I don't compile any old C or fortran code, so I can't say anything about if and what kinds of problems arise from using modern compilers against modern targets.
  • pklausler 2y ago
    It became painfully obvious during the development of LLVM Flang that it is very hard to define what the Fortran language is, and that it is tricky to write truly portable Fortran. See https://github.com/klausler/fortran-wringer-tests?tab=readme-ov-file#writing-portable-fortran https://github.com/klausler/fortran-wringer-tests?tab=readme...
• ideamotor 2y ago
  Agree that this discourse about “extending R’s useful life” by adding a compiler is mostly just silly. It’s equally valid to say the reason for building any of the (numerous) libraries for R is to “prolong R’s usefulness”. I think you’ve proposed a real reason a language’s popularity changes. But let’s add a few more: First, as languages grow in use that in itself leads to more use (and vice versa). Second, users of languages use said languages as protection of their jobs, livelihoods, culture, and so on by simply not learning less popular languages or allowing the use of less commonly used languages. There is power in numbers. IMO R’s problem has nothing to do with R’s limitations (which when limited are easily expanded). To the contrary, if anything, R’s ability to enable users to do so much is more of a problem than its limitations. The norms (culture) of a company say using many different languages for different organizational roles can clash with an R user who can succinctly work cross-functionally. Employees could be interested in preserving siloed roles and employers could prefer limited scope for employees. Instead of one person building something and presenting to users, you could have many employees serving in many roles using standard languages for each role. R is basically a wrapper language, and the ease at which third-party libraries can be built and installed for R, and R’s flexibility is inherently “useful” but less dogmatic. Less dogmatic but also less standard and less common. … So I just don’t think the problem for R is that “it’s not deemed useful” and I actually think such an argument is disingenuous on the part of people who want to limit the power of R users. Granted, I think, particularly in large organizations, using the most popular languages in itself has valid justification. But the reason is to attract employees and to build siloed expertise; not to enhance “usefulness”.
  • wodenokoto 2y ago
    Isn’t Python even more of a “do everything” language than R?
    • ideamotor 2y ago
      I very much think R is more of a “do everything language” because put simply, R lets developers do a lot more than Python in the language itself. It’s how the implementation of “polars” in R basically looks the exact same as in Python. R is like English in its ability to bring in other languages. Take a look at how many OOP systems there are. The best book on these subjects is https://adv-r.hadley.nz/metaprogramming.html https://adv-r.hadley.nz/metaprogramming.html.
• vhhn 2y ago
  We use embedded R in production in a way some other companies would use Python and I can say having a better compiler would definitely help. Even if most people use R interactively, having contributers working on compiler has many positive spillovers for the language. Also note that the R code running behind the scenes of your scripts (powering the functions of your favourite packages) is quite a different language, using less dynamic features. This is where a better compiler would always be appreciated.
• fiforpg 2y ago
  > professor X in field Y likes esoteric language Z, so his grad students write out libraries A/B/C to corner the Z library space This description seems to fit Julia really well. What other languages are like this?
• clusterhacks 2y ago
  Multiple times in the last 3-5 years, I have coded up wrappers to new/evolving C++ libraries using Rcpp for internal use by bioinformaticians. Especially anything using multi-threaded algorithms or from the recent algorithms research world, I really get a kick out of hacking into a usable R module. It is almost like doing computer science again (as opposed to the real day job, which is data munging for reporting tools).
• _yb2s 2y ago
  It depends on your lab/institution, and sub field, but I think Python is more common in bioinformatics than R nowadays. I prefer R - and personally developed many of the most widely used Bioinformatics packages in R, but have mostly shifted my lab to Python because of more extensive library options. Most of the big institutions - like large sequencing facilities are using Python. If you’re hiring teams of software engineers to make professional quality tools, it’s a lot easier to recruit for Python.
• derbOac 2y ago
  I've used R since it was in beta, as well as many other languages: python, perl, C/C++, Fortran, lisp, Julia, ... others I'm forgetting. I think you're right about fads and the appeal of a new sexy language. No dispute on that point. However, some of what constitutes fads are really more like a coincidental convergence of advantages. So for example, something gets picked up in field X because it's more convenient and all the people learned that in college, and then the same thing happens in another field, and then when libraries in the two fields interact, it's like multiplying the reasons. These network effects happen everywhere in tech. It's not necessarily good, but it happens. With R in particular there's a long arc to reasons why it might fade. I've heard about R fading before and then it picked up again, so who knows, but it will probably fade and there are reasons why. If what you're doing involves mathematical or computational fundamentals, all that wrapping around C and Fortran gets annoying really fast. Not everything involving heavy lifting has been coded in fortran or C already, and sometimes passing back and forth between those heavy lifting routines becomes a huge bottleneck. R is slow as hell, and yes you can write things in C or Fortran (or Rust etc?), but it turns something that should be fairly straightforward into a library project on its own almost. It's just easier to be able to write all the underlying stuff and IO/API stuff all in the same language and have it perform optimally. In fact, I'd probably rather just write it all in C or Fortran than write parts in one language and then wrap it in R — the R wrapping would mostly be to make it accessible to others (which is important, but there's the library bit). R too has become horribly fragmented in my opinion. A lot of things like ggplot and tidyverse are great, but it's led to this kind of fracturing of syntax in what was already a kind of fuzzily defined syntax in some ways. For what it's worth, I'm not sure I greatly prefer python. It's more general-purpose than R so has that advantage, but also has the same performance issues as R, and doesn't seem quite as well suited to statistical and numerical computing to me. Maybe it's just the object-heavy structure of python or something — maybe I prefer something closer to either lisp or C in the end — but python has never felt quite right to me. I'm looking forward to seeing what happens with Mojo, because that could be a real game changer, but am not holding my breath. Julia is appealing to me and seems to check all the right boxes, has a lot of the fun I had with R early on, but I agree with some of the criticism it's gotten for library interdependence problems due to type flexibility and conversion-type issues. I also think error handling needs a lot of work. It's great when it works, but sometimes just becomes intractable to debug. For me, statistical/mathematical computing is exciting in the last several years in a way it hasn't been in a long time, but also feels not quite "there" yet. I could still see a lot of currently used languages be superceded pretty dramatically by something new.