4 ms·

As a predominantly Python-focused engineer, I've spent considerable time teaching myself R and there is a lot to like about R. For boutique statistical librarie

by p4wnc6 11y ago

As a predominantly Python-focused engineer, I've spent considerable time teaching myself R and there is a lot to like about R. For boutique statistical libraries especially. For instance, the enjoyment of using PySTAN is nowhere near as high as simply using STAN directly from R. However, when you dig into the R internals, and you learn about its generic function model of OO, and about the mangled history of S3 and S4 classes, it becomes very frustrating. R mostly "just works" if you stick to the libraries. But if you want to really understand e.g. polymorphic dispatching and how you can design your own tools to use it, it's a deal-breaker pain in the ass in R. It just simply is not suited for real computer science situations when you need to design the software, rather than just making scripts that treat libraries as APIs. Since the times when you need "just scripting" are about 0.00001% of real-world cases, it unfortunately means that as nice as R is, it's just not a good enough tool to standardize into a real-world workflow. You're way better off using Python, even if you have to give up easy access to certain libraries, re-write your own implementations, or kludge them on with tools like rpy2.

TheLogothete 11y ago
>Since the times when you need "just scripting" are about 0.00001% of real-world cases I would argue the opposite is true. Most of the time an analyst builds regression trees in the mktg department for a retailer. Or similar situations like this.
- p4wnc6 11y ago
  That is in fact an exact situation I've been in, where it had previously been organized as a bunch do ad hoc scripts, and it caused huge problems. It is exactly all of these fractured cases where you're running analyses in some bespoke corner of a business that it is most critical that the analysis tool is productionized and unified. Plenty of businesses don't do it this way, but their reliance on manual scripts, manual environment and dependency management, manually tracking which versions of code produced which results, and manually calibrating on historic test data, it all adds up to incredible problems that eventually require re-writes focusing on a systematic architecture.
  - TheLogothete 11y ago
    Vendors are standardizing on R, so I don't think the situation is as severe as you make it out to be.
    - p4wnc6 11y ago
      Vendors tend to standardize on crap. How many vending machines do you see with salad or tofu instead of Doritos or soda? Big multinational IT vendors are even worse than the vending machine example. They cater to situations where a firm has some internal power struggle over IT and the struggle is won by one side who gets the certified approval of a statusy external vendor. For instance, most big, showy technology re-orgs around Hadoop are unequivocally bullshit. Most firms do not face data problems that are appropriate for Hadoop, not even when their primary enterprise data store approaches the raw size at which Hadoop becomes plausible, because the types of analytical workflows still tend to involve much smaller data. These firms hardly adopt Hadoop for pragmatic engineering reasons, and engineers may even resist the re-org and offer compelling evidence that it is a waste of money, yet are ignored. Vendor offerings are much more about catering to the CTO or some other manager level employees, and focusing on b.s. on-paper "milestones" that the product can help them achieve. Many times, merely just the installation and integration of these tools can result in a multi-hundred-thousand dollar bonus, or more, for people high up the IT food chain -- even before anyone has seen whether it will bear any fruit or even come close to being cost effective. Vendors will do this with anything they can, so it's not surprising they would do it with R technology and they also absolutely do it with Scala, Python, Spark, etc. Big vendors peddling all of that truly are the hotel lobby snack machines of the software world. I'm not saying it's bad to have R as part of a professional computing environment. R is a great tool. I'm saying that for most business situations, people who tend to make the assumptions necessary to justify using R are often very wrong, and the clunky and inconvenient support for professional computer science and software design in R bites them, making Python a much more pragmatic choice. Whatever vendors do is pretty irrelevant to this.
      - TheLogothete 11y ago
        I guess we have to agree to disagree.
xixi77 11y ago
R is indeed quite weird when you get to internals. What's the problem you have with S3 though? It has it's limitations, but since I've first seen it, I thought it was by far the single most convenient and intuitive OO system out there... But yes, can't use it for everything. S4 is a bit ugly but still kinda workable -- the real problems for me start when you need to deal with environments and expressions and such, but is it really so often necessary to go there when you are designing your own tools (which I guess is what you're doing?) As for the real-world percentage, it varies depending on the job -- R is primarily designed for the people who spend most of their time looking and thinking about the data, rather than the code.
jfoutz 11y ago
It sounds a lot like perl5 back in the day. let's fuze sed grep awk and bash into one thing. It's got a great deal of whipupitude, you can make amazing stuff in an afternoon.

by p4wnc6 11y ago

TheLogothete 11y ago

>Since the times when you need "just scripting" are about 0.00001% of real-world cases I would argue the opposite is true. Most of the time an analyst builds regression trees in the mktg department for a retailer. Or similar situations like this.

p4wnc6 11y ago
That is in fact an exact situation I've been in, where it had previously been organized as a bunch do ad hoc scripts, and it caused huge problems. It is exactly all of these fractured cases where you're running analyses in some bespoke corner of a business that it is most critical that the analysis tool is productionized and unified. Plenty of businesses don't do it this way, but their reliance on manual scripts, manual environment and dependency management, manually tracking which versions of code produced which results, and manually calibrating on historic test data, it all adds up to incredible problems that eventually require re-writes focusing on a systematic architecture.
- TheLogothete 11y ago
  Vendors are standardizing on R, so I don't think the situation is as severe as you make it out to be.
  - p4wnc6 11y ago
    Vendors tend to standardize on crap. How many vending machines do you see with salad or tofu instead of Doritos or soda? Big multinational IT vendors are even worse than the vending machine example. They cater to situations where a firm has some internal power struggle over IT and the struggle is won by one side who gets the certified approval of a statusy external vendor. For instance, most big, showy technology re-orgs around Hadoop are unequivocally bullshit. Most firms do not face data problems that are appropriate for Hadoop, not even when their primary enterprise data store approaches the raw size at which Hadoop becomes plausible, because the types of analytical workflows still tend to involve much smaller data. These firms hardly adopt Hadoop for pragmatic engineering reasons, and engineers may even resist the re-org and offer compelling evidence that it is a waste of money, yet are ignored. Vendor offerings are much more about catering to the CTO or some other manager level employees, and focusing on b.s. on-paper "milestones" that the product can help them achieve. Many times, merely just the installation and integration of these tools can result in a multi-hundred-thousand dollar bonus, or more, for people high up the IT food chain -- even before anyone has seen whether it will bear any fruit or even come close to being cost effective. Vendors will do this with anything they can, so it's not surprising they would do it with R technology and they also absolutely do it with Scala, Python, Spark, etc. Big vendors peddling all of that truly are the hotel lobby snack machines of the software world. I'm not saying it's bad to have R as part of a professional computing environment. R is a great tool. I'm saying that for most business situations, people who tend to make the assumptions necessary to justify using R are often very wrong, and the clunky and inconvenient support for professional computer science and software design in R bites them, making Python a much more pragmatic choice. Whatever vendors do is pretty irrelevant to this.
    - TheLogothete 11y ago
      I guess we have to agree to disagree.

xixi77 11y ago

R is indeed quite weird when you get to internals. What's the problem you have with S3 though? It has it's limitations, but since I've first seen it, I thought it was by far the single most convenient and intuitive OO system out there... But yes, can't use it for everything. S4 is a bit ugly but still kinda workable -- the real problems for me start when you need to deal with environments and expressions and such, but is it really so often necessary to go there when you are designing your own tools (which I guess is what you're doing?) As for the real-world percentage, it varies depending on the job -- R is primarily designed for the people who spend most of their time looking and thinking about the data, rather than the code.

jfoutz 11y ago

It sounds a lot like perl5 back in the day. let's fuze sed grep awk and bash into one thing. It's got a great deal of whipupitude, you can make amazing stuff in an afternoon.