4 ms·
I think it is a little disingenuous to say that algorithms get buried in academic literature and are impossible to find. By the nature of research most of these
by jroesch 13y ago
I think it is a little disingenuous to say that algorithms get buried in academic literature and are impossible to find. By the nature of research most of these things are made public when published, often times there is no implementation or just a research quality one (which I can guarantee you is almost never useful for the "real world").
For example one day I was interested in implementing HyperLogLog(a set cardinality measure that is useful in data analysis). In about 10m I had all the relevant papers on hand and after skimming them I had a pretty good sense of how to implement it.
Similarly if I want to know how to implement a program dependency graph for doing program analysis I can go read a few pages of a paper and get a good description on the algorithm I would need to construct such a thing. I can believe the argument that some of these things are poorly indexed but even a bare minimum of Google searching usually results in useful algorithms. I would argue that often times many of these research algorithms have a bunch of different design decisions that are best explored in the academic literature around them, and an implementation and a few notes is not sufficient exploration.
For example I was recently implementing Paxos and there were tons of little details to be extracted from the papers around that had a big impact on the actual implementation we ended up with. The 'Paxos Made Live' paper from Google had many details that were only relevant/true because of engineering decisions made by the team. If one was presented you with an implementation derived solely from that paper there are multiple incorrect assumptions you could derive.
An instance of this is made apparent in Paxos Made Live. Google essentially fixes their proposer because the have used Google specific details about the number of participants and their availability. The result is that they direct all traffic to a single node, and don't spend a much time talking about leader/proposer selection (which could be useful to your needs).
I also don't buy that an important part is getting the algorithms running as a service. Most so called "algorithms" are nothing more than a subroutine that is need as a piece of a greater whole. I would venture most useful "algorithms" are most likely container data structures and algorithms that operate over them. It seems that these are probably most useful to have as a library. Many libraries have already taken this approach LLVM (algorithms for code generation, albeit not always everything you want), OpenCV for computer vision routines, BLAS for linear algebra, NLOpt for non-linear optimization, and I'm sure one could come up with many more examples of democratized algorithms.