9 ms·
Show HN: A single, searchable API for all social networks, news, and blogs
- monvural 14y agoReally love this idea. I've started something similar for my content platform Amplifize.com, but am still in very early stages. How can folks help build this out? Could it be an open service?
- ZaneClaes 14y agoThanks! I'm one of the devs on the project; you can contact me directly at Zane@Streamified.com. To your questions: one of the major benefits of the service is the "crowdsourced" aspect of it. That is: the more people using the same API, the more news articles get found and indexed. Furthermore, the API actually hooks into other paid APIs, so it actually wouldn't work very well for you to run it on your own servers unless you wanted to also pay for them. For all these reasons and more, we think that a hosted API is the best solution.
- monvural 14y agoTotally agree with all of the above. I'll send you an email to keep the conversation going.
- Valid 14y agoThis is a fantastic resource... I'm finding myself trying to come up with a project to use it on.
- SamirSam 14y agoThe idea is good. But clarifying it further, how many API calls per month do you offer or plan ? Correct me if I am wrong, searchable means topic/keyword parsing with context analysis (i.e. intention analysis)?
- ZaneClaes 14y agoOur courtesy limit is 1k requests per day to the API. Beyond that, we're happy to discuss what makes sense, we obviously just need to prevent abuse. Yes, our searching capacities include everything from categories to sentiment to tags to media... you name it. ElasticSearch powers our backend, which is powered by Lucene, so anything in an article can be searched. Check out the articles/search endpoint: https://streamified.me/developer/#endpoint=GET%20articles%2Fsearch https://streamified.me/developer/#endpoint=GET%20articles%2F...
- niggler 14y agoIf this depends on the individual APIs (especially twitter) and sees any sort of traction, isn't there a massive risk that one of the providers could block access to streamified?
- ZaneClaes 14y agoPer our "whitelabeling" philosophy, Streamified actually uses your own app IDs (not ours). From Twitter's perspective, we're just acting as a middleman to help you use their API better; any blockage, then, would be a result of your implementation of their API (throttling, etc).
- jrussbowman 14y agoSweet, looks like something I could really dig into for unscatter.com I'm curious, are you proxying apis or crawling and keeping the information local to serve?
- ZaneClaes 14y agoThanks! Be sure to drop us a line if you want to dig a bit deeper into integration. For social networks, we're proxying the APIs (essentially). For articles and news, we both aggregate organically discovered articles from streams, as well as scrape 250k+ RSS feeds and other news sources. So, articles are stored local to the server (which is why we have such a large collection of them), but sensitive social information is a pass-thru to the social network. However, in both cases, we're doing lots of analysis on top of the data per our "normalization" philosophy.
- jrussbowman 14y agosure I have a couple questions, I'll shoot you an email
- akennberg 14y agoWho's behind this? There's not a single name anywhere on the .me or .com sites. It's hard to trust a platform service without knowing who is building it.
- ZaneClaes 14y agoStreamified, Inc.: Zane Claes and David Lee You're welcome to contact us directly @ zane@streamified.com and david@streamified.com
- JustARandomGuy 14y agoThis is a nice announcement to see, especially your support of searchable article feeds. Quick question: I see in https://streamified.me/developer/#concepts https://streamified.me/developer/#concepts that we can search news articles, is there a list of sites that you cover, or is it safe to assume that you crawl over all major news services (tech, politics, money, etc)? Also like your support for ADN. I have an account there that is sitting empty, might build something up over the weekend.
- ZaneClaes 14y agoThanks! We should cover all major news sources. We have at least 250,000 different sources we're scraping, plus we also "crowdsource" news discovery. That is: organically (stream-discovered) content gets indexed as well. Hope to hear from you! Zane@Streamified.com
- switz 14y agoI could have used this about two months ago. Instead I built a simple npm module – https://github.com/switz/melt https://github.com/switz/melt This is far better, though.
- mamcx 14y agoI try this with js sample app + twitter auth. I get Error: {"type":"Authentication","status":401,"details":{"message":"OAuth Authentication Failure: invalid oauth_token syntax"}} In the other hand, this could be handy for my news reader app for ipad. This notify of new content? Is this suitable for manage several users, each one with their own rss subscriptions?
- ZaneClaes 14y agoHey mamcx, Would you mind shooting me an email to Zane@Streamified.com so we can discuss? And yes, you could easily manage many users' RSS feeds :)
- bc1323 14y agoThis sounds a lot like http://singly.com/ http://singly.com/ - how is it different?
- nathanlil13 14y agoMy thoughts exactly.
- ZaneClaes 14y agoSure, we both cover social login and sharing. However, Streamified supports a whole bunch of news features, and a lot more social endpoints, which Singly doesn't begin to touch on.
- urza 14y agoYou lost me here: "We think that formats like RSS don't make sense for today's internet". And honestly.. another API? So it only works as long as you are in business (which is based on what by the way?) or decide to kill or change it? And also posting to my social media will be going through your machines? Seriously? If you'd offer me an opensource script that I could call like "localhost/post?message="hello"&where="twitter,blog,facebook".. that would be different story. But hey, good luck locking others in your new service.
- ZaneClaes 14y agoRSS is full of embedded HTML. It's practically impossible to make a good news reader without doing some heavy manipulation on it and trying to extract/modify ugly elements. We have a full blog post on the topic here: http://blog.streamified.com/post/45846467833/the-future-of-social-reading-is-not-another-reader http://blog.streamified.com/post/45846467833/the-future-of-s... There's a discussion below this one why it makes sense to have a single API for this, instead of open-sourcing the code. The benefits are numerous (we're hooking into many paid APIs and covering the costs, for example).
- anonfunction 14y agoRSS format sucks because XML sucks. I agree that it would be awesome if it was opensource so you could mitigate any risks by having it run on your own hardware, but really it's a valuable service and they deserve to make some money, which will play into keeping the service alive.
- ZaneClaes 14y agoThanks! The stack which is required for the Streamified API is really quite large, too. Between node, mongo, elasticsearch, and a few other pieces of software... plus a few dozen node modules and even some custom linux configuration, it's really not feasible to expect devs to install it on their own machines ;)
- graue 14y agoYou could open source it anyway, though, or parts of it. NewsBlur does this: https://github.com/samuelclay/NewsBlur https://github.com/samuelclay/NewsBlur One could run their own NewsBlur, but Samuel has stated it requires a lot of resources to pull RSS feeds, and so in practice it's much easier to pay them $2/month. (This may not apply if you plan on having business customers who are paying orders of magnitude more than that.) But it alleviates any fear that the service users depend on may disappear; in the worst case, run it yourself.
- anonfunction 14y agoHey just wanted to say that I appreciate the API and it has very nice docs! Ignore the naysayers, it's not for everyone but I can already imagine a ton of uses.
- ZaneClaes 14y agoThanks!
- graue 14y agoLooks nice, just a note: the redirect from http://streamified.me http://streamified.me is broken. $ curl -I http://streamified.me HTTP/1.0 302 Found Cache-Control: no-cache Connection: close Location: https:\/\/streamified.me
- ZaneClaes 14y agoI'm not following... how is this broken? The headers return a 302 code, pointing to the proper HTTPS site. If you click the link you pasted, your browser should properly redirect (tested in FF, Chrome).
- lawl 14y agoTotally not readable on a mobile. Left sidebar uses nearly the full screen even if you zoom in.
- ZaneClaes 14y agoSorry, but this is a developer portal -- not a news source or blog -- and really isn't meant to be used on a mobile device. I acknowledge that this may not be the answer you're looking for, but we didn't design the site for mobile because what it needs to do isn't really appropriate (or feasible) on mobile -- eg, the API console.
- lawl 14y agoWell it is a news source, called hacker _news_. You telling me a developer doesn't read stuff on a cell is just complete BS. I never said i want a mobile site. Just a css that doesn't completely break on small resolutions. If you don't want to fix it fine. But don't give me such a BS answer. I acknowledge this might not be the answer you were looking for.
- ZaneClaes 14y agoI'm sorry, I think I misspoke. What I meant is: you're right. What we should have done, though, was link to a blog post for you to read, rather than the dev portal itself. It just wasn't realistic to build a mobile enabled dev portal, so we should have posted something mobile friendly instead.
- reeses 14y agoThe rage is uncalled for. Can you be civil?