3 ms·

Actually, we're creating a way for developers to access the web really easily for the purposes of different kinds of analysis, including building semantic frame

by jdrock 18y ago

Actually, we're creating a way for developers to access the web really easily for the purposes of different kinds of analysis, including building semantic frameworks. The idea is that we give you really cheap, really fast access to millions of pages, and you use our platform to analyze Internet content how you want. $2 per 1 million pages crawled, $0.03 per CPU-hour used for any computing you want to do. Not yet at beta, but you can check our site: http://www.80legs.com http://www.80legs.com.

ntoshev 18y ago
Looks cool. How would it compare to Amazon/Alexa search service? In theory they allow you to build your own search engine, but in practice you can't really amend their ranking formula and don't get access to the raw inverted index (with tf-idf statistics and such). Yahoo BOSS is in the same league. Your service would be cheaper, though.
- jdrock 18y ago
  Yes, in theory you could do something similar with AWS. However, you'd have to put in the work to handle all the complexities of parallel-computing and web crawling. We do that for you. And yes, our service is cheaper. We'd love to see developers using our platform to build some very interesting indexes based on innovative concepts.
lsb 18y ago
What type of a platform? Do you have any code sketches to demonstrate the ease of doing otherwise tedious tasks, like in the Pig and Sawzall papers?
- jdrock 18y ago
  I have to admit I have limited familiarity with Pig and Sawzall (just added "Learn more about Pig and Sawzall" to my to-do list :D), but our platform is designed to save developers from having to think about clusters, parallelization, etc. The two basic functions you would interact with as a developer are a select() and a do(). The select() specifies what content you want. The do() specifies what you want to do with that content. The backend infrastructure is supposed to handle everything else. I don't have any code examples right now, but we plan to provide some with or during the beta release.

by jdrock 18y ago

ntoshev 18y ago

Looks cool. How would it compare to Amazon/Alexa search service? In theory they allow you to build your own search engine, but in practice you can't really amend their ranking formula and don't get access to the raw inverted index (with tf-idf statistics and such). Yahoo BOSS is in the same league. Your service would be cheaper, though.

jdrock 18y ago
Yes, in theory you could do something similar with AWS. However, you'd have to put in the work to handle all the complexities of parallel-computing and web crawling. We do that for you. And yes, our service is cheaper. We'd love to see developers using our platform to build some very interesting indexes based on innovative concepts.

lsb 18y ago

What type of a platform? Do you have any code sketches to demonstrate the ease of doing otherwise tedious tasks, like in the Pig and Sawzall papers?

jdrock 18y ago
I have to admit I have limited familiarity with Pig and Sawzall (just added "Learn more about Pig and Sawzall" to my to-do list :D), but our platform is designed to save developers from having to think about clusters, parallelization, etc. The two basic functions you would interact with as a developer are a select() and a do(). The select() specifies what content you want. The do() specifies what you want to do with that content. The backend infrastructure is supposed to handle everything else. I don't have any code examples right now, but we plan to provide some with or during the beta release.