3 ms·

The project listed in the article, http://algorithmia.com/ http://algorithmia.com/ looks like it's a web service. Anyone that's used/written algorithms knows t

by milge 12y ago

The project listed in the article, http://algorithmia.com/ http://algorithmia.com/ looks like it's a web service. Anyone that's used/written algorithms knows they need to be run natively on your machine/server to run the fastest. Calling a web service will slow this down greatly. If this site was turned into a marketplace of code listed for each algorithm that can automatically translate between languages, I could see it going somewhere.

cliveowen 12y ago
From the article: "The creators will also have the option of making their algorithms open source, so developers will be able to integrate them directly into their own applications"
- mtdewcmu 12y ago
  The creators already have the option of making their algorithms open source.
pizza 12y ago
True. But maybe if you create an O(n^4) algorithm, and the site has an O(n log n) algorithm, then it could take far less time sending it over. But that's all conjecture..
morgante 12y ago
Just because it's using a web service doesn't mean the algorithm itself isn't running efficiently on hardware. I imagine you ship off the data, let them run the algorithm, and they return the results. This has no effect on the Big-O efficiency of the algorithm; the only additional cost is a simple roundtrip of the data. I'm failing to see how that's so different from ex. running an intense algorithm in a worker thread.
- milge 12y ago
  I'm thinking back to some image processing processes I've implemented that had multiple algorithms. Passing images to a service like this back and forth would slow down those processes greatly. YMMV.
- jbarrow 12y ago
  This would see a significant slowdown for any algorithm that requires large amounts of data, as a lot of them do. The core philosophy of Hadoop which made it successful was "move the computation to the data and not the other way around." Although the algorithms may run efficiently, ferrying significant amounts of data to and from a web service could quickly negate any gains.
  - doppenhe 12y ago
    absolutely agreed which is way in almost 80% of cases we ship the computation to the data by running in the same data centers where the data lives.
mtdewcmu 12y ago
It's not just a web service -- it's any web service. The algorithm is up to you. I'm not sure what value they're providing, exactly. Anyone can put algorithms on the web now: you write them in a standard language like PHP and upload them to any hosting service.
pinkyand 12y ago
If it's running on AWS , isn't it possible to just develop your whole system in AWS and gain the efficiencies?

by milge 12y ago

cliveowen 12y ago

From the article: "The creators will also have the option of making their algorithms open source, so developers will be able to integrate them directly into their own applications"

mtdewcmu 12y ago
The creators already have the option of making their algorithms open source.

pizza 12y ago

True. But maybe if you create an O(n^4) algorithm, and the site has an O(n log n) algorithm, then it could take far less time sending it over. But that's all conjecture..

morgante 12y ago

Just because it's using a web service doesn't mean the algorithm itself isn't running efficiently on hardware. I imagine you ship off the data, let them run the algorithm, and they return the results. This has no effect on the Big-O efficiency of the algorithm; the only additional cost is a simple roundtrip of the data. I'm failing to see how that's so different from ex. running an intense algorithm in a worker thread.

milge 12y ago
I'm thinking back to some image processing processes I've implemented that had multiple algorithms. Passing images to a service like this back and forth would slow down those processes greatly. YMMV.
jbarrow 12y ago
This would see a significant slowdown for any algorithm that requires large amounts of data, as a lot of them do. The core philosophy of Hadoop which made it successful was "move the computation to the data and not the other way around." Although the algorithms may run efficiently, ferrying significant amounts of data to and from a web service could quickly negate any gains.
- doppenhe 12y ago
  absolutely agreed which is way in almost 80% of cases we ship the computation to the data by running in the same data centers where the data lives.

mtdewcmu 12y ago

It's not just a web service -- it's any web service. The algorithm is up to you. I'm not sure what value they're providing, exactly. Anyone can put algorithms on the web now: you write them in a standard language like PHP and upload them to any hosting service.

pinkyand 12y ago

If it's running on AWS , isn't it possible to just develop your whole system in AWS and gain the efficiencies?