4 ms·

We are algorithm developers ourselves and our goal is just to make these awesome algorithms that are being developed more widely available. We are not trying t

We are algorithm developers ourselves and our goal is just to make these awesome algorithms that are being developed more widely available. We are not trying to monopolize basic algorithms. In fact in a free market I don't see how that could happen, since anyone else could come along and implement the same basic CS algorithms and offer a lower price. The thing that makes us different from say Wikipedia or Github is that we don't provide just source code, but live infrastructure delivering the algorithms as a service. So rather than uploading code to github, and putting the burden on a user cloning the algorithm, setting up virtual servers, deployment systems, load balancers, etc. We handle all that. That way algorithm developers are free to focus on algorithm code, and the infrastructure is taken care of. This infrastructure needs to be paid for however, so our general plan is to charge for using algorithms through the API. It will be up to the developers whether to share source code or not, since they own the IP.

gfodor 13y ago
The status quo now is not broken: developers who want to share IP are encouraged to incorporate their work into open source libraries so they can be used by everyone and be made robust and evolve over time. In various domains these libraries are more or less agreed upon, and the patches go through extensive review to get incorporated. This type of system will encourage these folks to release their source code in a form compatible with your API, and 3rd parties who want to leverage their work will need to either cut-and-paste it to incorporate it into their own system (a nightmare), or access it through a slow network API, likely paying for it along the way. Libraries have solved the problem of code re-use of algorithms for the last several decades. You're basically proposing that LAPACK and BLAS and Mahout and all these great algorithmic libraries would have been best served up without any centralized coordination or binary distribution, but with each individual algorithm behind a separate API with each author publishing each one individually. I don't see how if the goal is really to enable wider distribution of good algorithmic work that you can't see that an heterogenous API-centric approach to this problem (while one that would certainly make you a pile of cash if it catches on) wouldn't really be a great step forward for the industry compared to the centralized open source library model we have now. Good algorithmic work is hard, and the model we have now results in battle hardened, robust, agreed-upon implementations of well understood algorithms. If you really want to make me a believer, all algorithms that your API supports that have public source code should also be integrated and built into a binary library that a user can download instead. At the very least, the path for a contributor to take what they have written and additionally publish it as a appropriate binary package themselves should be minimal. For example, if I implement something in Ruby that uses your service, I should be able to push it as a un-coupled, reusable gem in a straightforward manner.
- platypii 13y ago
  I agree that the existing libraries are great, and if anything that success is what we to encourage even more of. We want to build a community around algorithm development, and have a central place for developers to share their work, and collaborate with other developers to make Huge library of algorithms so that everyone can benefit. The first language we implemented was Java/Scala, largely because of the excellent libraries like Mahout and Weka easily available in Maven Central. But even though the code is there in maven, it is still not trivial to deploy those algorithms as a service. We are trying to make that easier, while at the same time supporting the developers of those libraries in the process. Being able to run the code locally is in the plans. For the most part, the algorithms in our system are already simply maven packages, or ruby gems, npm packages, etc. Being able to distribute them in binary form is definitely possible. We're still figuring things out right now, but our focus is always on doing right by the algorithm development community.
  - gfodor 13y ago
    cool, good luck!
  - daken 13y ago
    This doesn't make sense to me. You display Dijkstra as an example on your homepage. Imagine I want to use Dijkstra in an app that needs critical performance and calls the function 1000000 times per minute, making this an API rather a library just breaks the whole point of centralizing the algorithms.
    - dalke 13y ago
      Plus, the choice of Dijkstra algorithm implementation depends on the graph size, type, and even CPU. See http://www.cs.sunysb.edu/~rezaul/papers/TR-07-54.pdf http://www.cs.sunysb.edu/~rezaul/papers/TR-07-54.pdf , with 10 different implementation variations and several different benchmarks. The 9th DIMACS Implementation Challenge from 8 years ago (see http://www.dis.uniroma1.it/challenge9/ http://www.dis.uniroma1.it/challenge9/ ) served as a way to gather the best algorithms for the shortest path problem. I don't see how this project can be significantly better, or even close to, a traditional effort like that.
gradstudent 13y ago
> The thing that makes us different from say Wikipedia or Github is that we don't provide just source code, but live infrastructure delivering the algorithms as a service. So rather than uploading code to github, and putting the burden on a user cloning the algorithm, setting up virtual servers, deployment systems, load balancers, etc. We handle all that. That way algorithm developers are free to focus on algorithm code, and the infrastructure is taken care of. Virtual servers? deployment systems? load balancers?!? Huh? It all looks like gibberish to me. What does any of this have to do with developing implementations of new algorithmic techniques?
- platypii 13y ago
  Because that's what is necessary to actually use algorithms in practice if the demand is non-trivial.
  - gradstudent 13y ago
    I've read your blog post twice now. I've looked at your website. I've seen your comments here. Still I have no idea what you're proposing or why I would be motivated to use... whatever it is you're trying to convince me to use. Your message is vague as vague can be. It also doesn't help that the word "algorithm" is sufficiently general to cover any piece of code.
    - platypii 13y ago
      In this post we were trying to focus on the problems we've personally experienced as algorithm developers. The plan is to explain the details of Algorithmia more in future posts, but here's the basic idea: We want smarter machines, and that means making algorithms more widely available. To achieve that, we're creating a community around algorithm development, where people can contribute code and collaborate with others on that code. Github has done a great job of this, but it is not always trivial to get from a github repo to live code in production. We think that creating a built-in market for algorithms-as-a-service could be a way to reward algorithm developers for their work, and make algorithms easily accessible via REST API. The types of algorithms we are talking about is not limited by us. My opinion is that the most useful algorithms in the near future will be related to: machine learning, classification, analysis, and optimization. There are also numerous applications for things like image recognition, speech-to-text, and translation, to name a few.

gfodor 13y ago

The status quo now is not broken: developers who want to share IP are encouraged to incorporate their work into open source libraries so they can be used by everyone and be made robust and evolve over time. In various domains these libraries are more or less agreed upon, and the patches go through extensive review to get incorporated. This type of system will encourage these folks to release their source code in a form compatible with your API, and 3rd parties who want to leverage their work will need to either cut-and-paste it to incorporate it into their own system (a nightmare), or access it through a slow network API, likely paying for it along the way. Libraries have solved the problem of code re-use of algorithms for the last several decades. You're basically proposing that LAPACK and BLAS and Mahout and all these great algorithmic libraries would have been best served up without any centralized coordination or binary distribution, but with each individual algorithm behind a separate API with each author publishing each one individually. I don't see how if the goal is really to enable wider distribution of good algorithmic work that you can't see that an heterogenous API-centric approach to this problem (while one that would certainly make you a pile of cash if it catches on) wouldn't really be a great step forward for the industry compared to the centralized open source library model we have now. Good algorithmic work is hard, and the model we have now results in battle hardened, robust, agreed-upon implementations of well understood algorithms. If you really want to make me a believer, all algorithms that your API supports that have public source code should also be integrated and built into a binary library that a user can download instead. At the very least, the path for a contributor to take what they have written and additionally publish it as a appropriate binary package themselves should be minimal. For example, if I implement something in Ruby that uses your service, I should be able to push it as a un-coupled, reusable gem in a straightforward manner.

platypii 13y ago
I agree that the existing libraries are great, and if anything that success is what we to encourage even more of. We want to build a community around algorithm development, and have a central place for developers to share their work, and collaborate with other developers to make Huge library of algorithms so that everyone can benefit. The first language we implemented was Java/Scala, largely because of the excellent libraries like Mahout and Weka easily available in Maven Central. But even though the code is there in maven, it is still not trivial to deploy those algorithms as a service. We are trying to make that easier, while at the same time supporting the developers of those libraries in the process. Being able to run the code locally is in the plans. For the most part, the algorithms in our system are already simply maven packages, or ruby gems, npm packages, etc. Being able to distribute them in binary form is definitely possible. We're still figuring things out right now, but our focus is always on doing right by the algorithm development community.
- gfodor 13y ago
  cool, good luck!
- daken 13y ago
  This doesn't make sense to me. You display Dijkstra as an example on your homepage. Imagine I want to use Dijkstra in an app that needs critical performance and calls the function 1000000 times per minute, making this an API rather a library just breaks the whole point of centralizing the algorithms.
  - dalke 13y ago
    Plus, the choice of Dijkstra algorithm implementation depends on the graph size, type, and even CPU. See http://www.cs.sunysb.edu/~rezaul/papers/TR-07-54.pdf http://www.cs.sunysb.edu/~rezaul/papers/TR-07-54.pdf , with 10 different implementation variations and several different benchmarks. The 9th DIMACS Implementation Challenge from 8 years ago (see http://www.dis.uniroma1.it/challenge9/ http://www.dis.uniroma1.it/challenge9/ ) served as a way to gather the best algorithms for the shortest path problem. I don't see how this project can be significantly better, or even close to, a traditional effort like that.

gradstudent 13y ago

> The thing that makes us different from say Wikipedia or Github is that we don't provide just source code, but live infrastructure delivering the algorithms as a service. So rather than uploading code to github, and putting the burden on a user cloning the algorithm, setting up virtual servers, deployment systems, load balancers, etc. We handle all that. That way algorithm developers are free to focus on algorithm code, and the infrastructure is taken care of. Virtual servers? deployment systems? load balancers?!? Huh? It all looks like gibberish to me. What does any of this have to do with developing implementations of new algorithmic techniques?

platypii 13y ago
Because that's what is necessary to actually use algorithms in practice if the demand is non-trivial.
- gradstudent 13y ago
  I've read your blog post twice now. I've looked at your website. I've seen your comments here. Still I have no idea what you're proposing or why I would be motivated to use... whatever it is you're trying to convince me to use. Your message is vague as vague can be. It also doesn't help that the word "algorithm" is sufficiently general to cover any piece of code.
  - platypii 13y ago
    In this post we were trying to focus on the problems we've personally experienced as algorithm developers. The plan is to explain the details of Algorithmia more in future posts, but here's the basic idea: We want smarter machines, and that means making algorithms more widely available. To achieve that, we're creating a community around algorithm development, where people can contribute code and collaborate with others on that code. Github has done a great job of this, but it is not always trivial to get from a github repo to live code in production. We think that creating a built-in market for algorithms-as-a-service could be a way to reward algorithm developers for their work, and make algorithms easily accessible via REST API. The types of algorithms we are talking about is not limited by us. My opinion is that the most useful algorithms in the near future will be related to: machine learning, classification, analysis, and optimization. There are also numerous applications for things like image recognition, speech-to-text, and translation, to name a few.