johnmyleswhite

11 ms·

61.
by johnmyleswhite 10y ago
This is not quite accurate as stated: the problem is the combination of a data structure that is not amenable to type inference with an API that would only be efficient if type inference had perfect knowledge of the contents of a DataFrame.
62.
by johnmyleswhite 10y ago
I think most people who study causal inference would say that Pearl's writing is seldom as clear as a person unfamiliar with his ideas would like it to be. These days, I suspect most people would benefit from starting with Morgan and W
63.
7 Julia Gotchas and How to Handle Them(stochasticlifestyle.com)
22 points by johnmyleswhite 10y ago|1 comments
64.
by johnmyleswhite 10y ago
I'd like to see the title changed to something like "Futures for R" since it's so tongue-in-cheek that it's effectively clickbait.
65.
by johnmyleswhite 10y ago
Really glad to see this, but I think we really need to assess how much money is available from each of these channels. My experience with some of these channels (e.g. bounties) is that the effective hourly wage is radically lower than offer
66.
The 2016 San Francisco DCCC Elections(medium.com)
1 points by johnmyleswhite 10y ago|0 comments
67.
by johnmyleswhite 10y ago
From the article: """ Democrats always politic on the basis of emotion and their quixotic Green Energy is one of their favorite feel-good causes. There is no way to prove, or disprove that it exists or that it is caused by hu
68.
by johnmyleswhite 11y ago
They're both great books, but they make heavier use of potential outcome notation and focus less on graph-theoretic formulations than Pearl's work.
69.
by johnmyleswhite 11y ago
Yes.
70.
The environment that OSS developers work in(github.com)
61 points by johnmyleswhite 11y ago|21 comments
71.
by johnmyleswhite 11y ago
I think it would be really interesting to understand exactly why Julia underperforms LuaJIT on some benchmarks. I'm pretty skeptical that Julia loses to LuaJIT uniformly; I suspect it depends on the benchmark taking advantage of LuaJIT
72.
by johnmyleswhite 11y ago
I understand your concerns better now. Here's my input for what it's worth: (1) List comprehensions are a mess and a recognized problem with the existing compiler. There's a lot of work that's been done to fix the errati
73.
by johnmyleswhite 11y ago
I would use SGD when I can tolerate an inexact estimator. For problems when I need an exact MLE (because, for example, I need to invoke asympotic normality to get CI's), I can usually do exact computations using less memory than IRLS w
74.
by johnmyleswhite 11y ago
That really depends on how you fit models. Maximum likelihood estimation (or MAP estimation) for models of IID database rows is often trivial to implement using an iterative algorithm that applies to both in-memory and out-of-core data stor
75.
by johnmyleswhite 11y ago
I don't see how this could be done without putting information about the internal representation of DataFrames into the compiler. Such an approach would seem to require breaking important abstraction barriers and would couple the langu
76.
by johnmyleswhite 11y ago
It would be very helpful if you provided some specific examples of code in both Julia and Fortran that is under-performant in Julia. As it stands, your concerns about things like "fancy abstractions that no engineer or scientist will e
77.
by johnmyleswhite 11y ago
There's work being done to wrap Dato's SFrames library, which would solve that problem.
78.
Why Julia's DataFrames Are Still Slow(johnmyleswhite.com)
60 points by johnmyleswhite 11y ago|21 comments
79.
by johnmyleswhite 11y ago
I think you may have misinterpreted that post. Look at the table under "Comparing Performance in R and Julia" again.
80.
by johnmyleswhite 11y ago
The primary use case for Julia's coming array slices is to provide views into an array that you always intended to keep alive anyway. Iterating over the columns of an in-memory matrix, for example, is the use case that this change is m
81.
by johnmyleswhite 11y ago
That issue is largely resolved in 0.4. Some functions are still a little slow, but most packages will now load much more quickly.
82.
by johnmyleswhite 11y ago
If you'd like to know canonical programming style, you should read the code in Julia Base. Learning how to write good Julia is somewhat difficult because the language occupies an unfamiliar position between traditional static languages
83.
by johnmyleswhite 11y ago
Probably worth noting that the GitHub issue in question relates to work being done on Julia 0.5, a future release that is not yet ready to ship and won't ship for several months. Within a few weeks, Julia 0.4 will be shipping, which is
84.
by johnmyleswhite 11y ago
I've been doing a lot of work behind the scenes there, but it's still going to take a while to offer a mature product. What we really need are more qualified engineers to speed up the process, but there's a vanishingly small
85.
by johnmyleswhite 12y ago
If you're interested in this topic, you should watch this talk by Steven G. Johnson: https://www.youtube.com/watch?v=jhlVHoeB05A tl;dr of the talk: Syntax has little to no effect on how a language performs. What distin
86.
by johnmyleswhite 12y ago
FWIW, I think the DataFrames package and its dependencies have consistently operated at the boundaries of what we know how to do efficiently in Julia. The package has had lackluster performance in many contexts primarily because it adopted
87.
by johnmyleswhite 12y ago
GNU's assembler is implemented in C.
88.
by johnmyleswhite 12y ago
If you're compiling Julia from source, you should be able to do something like write a base/userimg.jl file that contains this kind of text: Base.require("Distributions.jl") Base.require("Optim.jl") Base.requir
89.
by johnmyleswhite 12y ago
Have you tried the userimg.jl hack? It's _very_ hacky, but it will convince you that precompiling packages is viable in the short-term.
90.
by johnmyleswhite 12y ago
I'm not sure. I think, for many users, SQLite is sufficient, which is why I'll be using it for prototype work in the future. Trying to add things like arrays seems like a lot of work, since you also need to add things like HQL&#x2

More ›