3 ms·

For example, reshaping JSON to the format an intricate R function expects. Appreciate the great work with (d)plyr and similar packages, but it's still work and

For example, reshaping JSON to the format an intricate R function expects. Appreciate the great work with (d)plyr and similar packages, but it's still work and overhead. Combined with some inefficiencies/quirks in base r functions (does ifelse() still evaluate twice?) it's easier to go with a widely used and respected package in a general purpose language; Nokogiri for example. For data engineering, consider there is not a maintaned R package for a web-client, and asynchronous programming is weak.

hadley 10y ago
JSON is often a pain because it's so hierarchical and un-dataframe like. I have a few notes on working with it here: http://r4ds.had.co.nz/hierarchy.html http://r4ds.had.co.nz/hierarchy.html. `ifelse()` is a nightmare of a function but I don't think double-evaluation is ever a problem. There are two maintained web-clients: curl (low-level) and httr (high-level). And I think rvest does everything that nokogiri does.
- pollitos 10y ago
  Thanks for the link on JSON and your packages, great work as always. I should clarify when I said we-client, I meant websockets client to consume feed. The last time I tried, the only R package (r-websockets) just crashed my Linux box and not maintained for several years. httr doesn't do websockets, as I understand. Seems likely a fundamental way to engineer/wrangle data into R.
  - hadley 10y ago
    You can do it with httpuv, but it might a bit clunky. I think better websockets support, and better async generally, is on the roadmap for the next year.
    - pollitos 10y ago
      For httpuv, author Joe Cheng advised cannot do a websocket-client. https://stackoverflow.com/questions/28120307/how-to-interact-with-websocket-from-within-r https://stackoverflow.com/questions/28120307/how-to-interact... Looking forward to the improvements. Appreciate all your work in the field.

3 ms·

For example, reshaping JSON to the format an intricate R function expects. Appreciate the great work with (d)plyr and similar packages, but it's still work and

by pollitos 10y ago

hadley 10y ago
JSON is often a pain because it's so hierarchical and un-dataframe like. I have a few notes on working with it here: http://r4ds.had.co.nz/hierarchy.html http://r4ds.had.co.nz/hierarchy.html. `ifelse()` is a nightmare of a function but I don't think double-evaluation is ever a problem. There are two maintained web-clients: curl (low-level) and httr (high-level). And I think rvest does everything that nokogiri does.
- pollitos 10y ago
  Thanks for the link on JSON and your packages, great work as always. I should clarify when I said we-client, I meant websockets client to consume feed. The last time I tried, the only R package (r-websockets) just crashed my Linux box and not maintained for several years. httr doesn't do websockets, as I understand. Seems likely a fundamental way to engineer/wrangle data into R.
  - hadley 10y ago
    You can do it with httpuv, but it might a bit clunky. I think better websockets support, and better async generally, is on the roadmap for the next year.
    - pollitos 10y ago
      For httpuv, author Joe Cheng advised cannot do a websocket-client. https://stackoverflow.com/questions/28120307/how-to-interact-with-websocket-from-within-r https://stackoverflow.com/questions/28120307/how-to-interact... Looking forward to the improvements. Appreciate all your work in the field.