hadley

10 ms·

121.
by hadley 11y ago
Not by us, but we expect that many other projects will add feather support now that it's used by both R and Python.
122.
by hadley 11y ago
Basically because you should be encoding state in code, not data. If you store data between sessions, it's easy to lose the code that you use to create it and then later on you can't recreate it. It is convenient to save your comp
123.
by hadley 11y ago
Compression is on the long term to do list - Wes knows a lot more about it than me.
124.
by hadley 11y ago
If SAS wanted to support it, it would be super easy for them to implement it.
125.
by hadley 11y ago
I think saving your entire workspace is a bad idea too, sorry!
126.
by hadley 11y ago
It's just a matter of someone implementing the protocol. It's not a huge amount of work for an experienced js/matlab programmer.
127.
by hadley 11y ago
It's usually faster than either compressed or uncompressed RDS - but if you have heavily duplicated data, compressed RDS can be faster than feather (depending on some tradeoff between compression speed and disk speed). Feather will pro
128.
by hadley 11y ago
Yes! Anything that is mostly C/C++ for performance could be shared between R and Python now that we have an easy interchange format.
129.
by hadley 11y ago
Should be insights from _arrow_
130.
by hadley 11y ago
It's often much faster than rds. And in the long long term there will be tools for computing on feather files that don't require loading it into memory. (In the short term I'll add ways to pull in slices of the full dataset)
131.
by hadley 11y ago
The format already supports larger than RAM data, but we don't yet have an API for creating those files or just extracting slices. That will come in the future.
132.
by hadley 11y ago
Feather currently doesn't support recursive/hierarchical data structures, like lists in R. That'll be added in the future though. We'll definitely standardise in the future so you can feel confident using it in the long
133.
by hadley 11y ago
Both Wes and I (project authors) will be tracking this thread in case you have questions!
134.
by hadley 11y ago
While it's true that there needs to be some js involved, the analyst doesn't have to write it. You could use an R package (like https://rstudio.github.io/leaflet/ ) which wraps around an javascript library, so
135.
by hadley 11y ago
SOAP + Oauth is a weird combination but you could definitely work with it in R.
136.
by hadley 11y ago
This is my more intro level book: http://r4ds.had.co.nz (it's still a work in progress, but you might find it helpful)
137.
by hadley 11y ago
This is called spaghetti sort. The analysis on Wikipedia should help.
138.
by hadley 11y ago
It's a bit more complicated than that - dplyr and magrittr invented pipes independently at about the same time. I preferred magrittr's implementation, so switched to that. dplyr continues to import and re-export %>% so that yo
139.
by hadley 11y ago
And generic functions allow you to fill in those missing pieces from a package too.
140.
by hadley 11y ago
Those also work just fine with strings.
141.
by hadley 11y ago
R uses a global string cache so any string comparison is just comparing two pointers.
142.
by hadley 11y ago
The lack of data structures in R is a totally fixable problem.
143.
by hadley 11y ago
You might enjoy purrr, https://github.com/hadley/purrr , which is my attempt to make FP tools in R more consistent.
144.
by hadley 11y ago
I think, and I'm pretty sure most of R core would agree, that building and maintaining their own runtime _was_ the right thing to do. Otherwise R would have been at mercy of maintainers who were interested in problems other than creati
145.
by hadley 11y ago
No, that is not the consensus. R is (at its heart) a beautiful language that is extremely well suited to its domain. Most people who use R are not professional programmers (or even identify as programmers) so it's not surprising that t
146.
by hadley 11y ago
And Advanced R is also available for free at http://adv-r.had.co.nz/
147.
by hadley 11y ago
Also can be "useful" if you want to spend grant money to go to a vacation destination.
148.
by hadley 11y ago
Jeff was probably thinking of heat maps with trees on the margins.
149.
by hadley 11y ago
Yeah I've been thinking about making a new class based on top of character vectors that allows custom ordering
150.
by hadley 11y ago
Or just avoid using factors in the first place. Character vectors are usually more appropriate and a modern ingest pipeline won't produce factors in the first place.

More ›