7 ms·

I was telling a friend recently about how there was this "golden age" when you could access all sorts of free APIs, and how I still long for this time. I remem

• deleted 7y ago
  [deleted]
• diggan 7y ago
  > I also abused the hell out of Yahoo Pipes - I would run RSS feeds through like 15 different languages with Babelfish before back to English, just for kicks. Yahoo Pipes was one of the greatest services I used, just when I started getting into programming. Maybe it was so cool because I was naive, but I really miss being able to pipe services together in the same way. Anyone know of any similar attempts that is open source + offers a hosted version with paid plans?
  • zwily 7y ago
    I don’t, but Node Red is giving me the same fun feeling I had with Yahoo Pipes, but for Home Automation.
  • onli 7y ago
    Not sure about open source and paid hosting, but there is node-red, https://nodered.org/ https://nodered.org/, which is open source and easy to get started with. Combining stuff is something Zapier excels in, https://zapier.com/ https://zapier.com/, and the free tier can suffice for some tasks (not open source). You could also try my attempt at a spiritual pipes successor, https://www.pipes.digital/ https://www.pipes.digital/ (but it's also not open source). If there is something missing there to reproduce how you used Yahoo Pipes I'd definitely be interested in hearing from you, so I can restore it :)
    • diggan 7y ago
      So, the requirements of open-source + have entity with paid hosting are both equally important. First one to ensure I can continue using whatever I setup on the hosting, in case the entity behind it cannot. And the paid hosting is important because it gives better odds towards the service actually sticking around. What I likes with Yahoo Pipes compared to NodeRED (at least as far as I looked at NodeRED, I might be wrong) is that Yahoo Pipes worked out-of-the-box with services out of the box. I seem to remember that you could use the Google Search API for example, with Yahoo Pipes and pipe that into other things. That's what Zapier does as well, but with less flexibility than NodeRED. So I guess my dream would be something like the integrations provided by Zapier but with the UI and flexibility of NodeRED. Haven't seen pipes.digital before, I'll take a look as it looks interesting, but for anything serious, open source is a hard requirement (gotta learn from the Yahoo Pipes history :) )
      • r-w 7y ago
        I don’t think those requirements will be met until people like us who miss it put our money where our mouth is ;)
        • diggan 7y ago
          I think people like us (at least me) miss a lot of things, some more important than others, and we have to carefully choose what we spend out time on :)
          • onli 7y ago
            I'm about to open source it now. Which means I just did, but haven't announced it yet. https://github.com/pipes-digital/pipes https://github.com/pipes-digital/pipes now contains a very new version you could run locally. r-w is right though: More users paying for pipes would allow me to invest more time into it and have more requirements covered.
      • deleted 7y ago
        [deleted]
  • routerl 7y ago
    I haven't used it but https://n8n.io/ https://n8n.io/ It reminds me of zapier, which reminds me of Yahoo pipes.
  • ivanfon 7y ago
    From a quick search, it looks like someone is recreating Yahoo Pipes here: https://www.pipes.digital/ https://www.pipes.digital/ Unfortunately, it looks like it's not open-source, their Github repo is only for bug reports: https://github.com/pipes-digital/pipes https://github.com/pipes-digital/pipes
    • egfx 7y ago
      Yeah as I mentioned https://news.ycombinator.com/item?id=21641615 https://news.ycombinator.com/item?id=21641615 A Yahoo pipes! clone really needs some open source love. Too bad the maintainer is more interested in making it a one man SaaS. The video is interesting because I thought he was talking specifically to me about the http://2fb.me http://2fb.me product. But anyway I think this is a reminder that even though API’s have gotten more restrictive that more creative ways of using them bubble to the surface. :)
      • onli 7y ago
        > A Yahoo pipes! clone really needs some open source love. Too bad the maintainer is more interested in making it a one man SaaS. I guess that's fair. I want(ed) to run Pipes as an SaaS because I think that model would be highly advantageous to me, and that would be very advantageous for Pipes. But that with the predecessors history as background this approach is a critical concern is something I do understand. I will now try to have my cake and eat it too: https://github.com/pipes-digital/pipes https://github.com/pipes-digital/pipes contains a new FOSS version that is meant to exist in parallel with pipes.digital, as long as that model works out. So worst case now is this FOSS approach fails, and then only the SaaS would die, not the software.
  • hundchenkatze 7y ago
    Dark is similar. I think they're still in private beta, but I got my invite pretty quickly. https://darklang.github.io/docs/introduction https://darklang.github.io/docs/introduction
  • akavel 7y ago
    I believe https://luna-lang.org https://luna-lang.org has plans like this, though they're kinda definitely not hurrying things...
  • erikig 7y ago
    For a self hosted Yahoo Pipes alternative I’d recommend checking out Huginn - it’s been featured on HN a couple of times and the creator is tectonic@HN https://github.com/huginn/huginn https://github.com/huginn/huginn
  • reubano 7y ago
    Check out my project, riko. https://github.com/nerevu/riko https://github.com/nerevu/riko Open source, but no hosted plan.
• anonytrary 7y ago
  I remember when you could just stream in every tweet and reddit comment ever posted with a few lines of code. There used to be dedicated API methods for doing so. Now, there's all sorts of namespacing, rate limiting, pagination, and upper bounds on data access that make this impossible, or at least infeasible.
  • sneak 7y ago
    Twitter wants to capitalize on the fact that tweets are massively public, and they also want to capitalize on the fact that they are not. Researchers are passing around lists of tweet IDs ("dehydrated", they call them) that can be "rehydrated" (that is, turned back into full tweets) if you have the right permission from twitter to do so. The whole setup is really shameful. It would be de-facto illegal to build a "Google for Twitter" today. I settled on doing it for ActivityPub/Mastodon because it's less likely I'll get sued into oblivion for creating a search engine that way.
    • snisarenko 7y ago
      Hi sneak, I am working on some fediverse aggregator projects. Let me know if you want to talk. https://mastodonia.club https://mastodonia.club https://pixelfed.club https://pixelfed.club
      • sneak 7y ago
        You have no standard contact methods available on any of your sites that do not require registration. Consider publishing your email address; I am happy to chat. My email is in my profile and on my website.
        • snisarenko 7y ago
          Sent you an email
  • banana_giraffe 7y ago
    This reminds me of an endpoint that Google used to run. It was a never ending stream of RSS feeds. Anyone could subscribe (it was just a never ending HTTP GET, if I remember right), and be told which blogs had updated in near real time. Nowadays you'd need to sign up for an API key, probably pay some amount of money, and provide twenty different forms of contact to use something like that. That's assuming it was allowed to exist in the first place, since it might take views away from google.com.
• duxup 7y ago
  It reminds me distantly of an earlier time when just the web was sort of owned ... by the web folk. In the sense that even corporate sites if you found it would have a little corner where the 'webmaster' had a page that mentioned the server, or his cat, or some silly pic. Some sort of character or tidbit before any of the branding drones were really aware of the web. All just because the 'webmaster' was the only one really in charge / who understood the site was even there and they wanted to share. I suspect to some extent the APIs were the same. Someone who really didn't mind was all "Yeah sure if someone wants to see what I did.. awesome."
  • Causality1 7y ago
    Back when I had three hundred bookmarks instead of, like, six. God I'd kill to still have that list.
  • mattl 7y ago
    I used to beg people I met online to host pages for me, give me shell access, etc. I was a child in the very early 90s with basically no money.
    • mauricedenassau 7y ago
      It's not very common, but I had people begging my friends for this kind of services.
      • flingo 7y ago
        raspberrry pi in home-network DMZ, with fail2ban, unattended-upgrades, and a free dynamic DNS service gets you most of this. (plus, you're root, so you can run things more complicated than web-services)
    • wastedhours 7y ago
      The same, in fact my life would be very different if a random guy from (I think Kansas, given I'm from the UK) on a forum bought me a domain and gave me a slice of his dedicated hosting when I was mid-teens. I've always wanted to pay it forward in the same way, but lots of things on the web seem overly complicated now that'd make that hard to do, and I've lost attachment to most communities.
• amatecha 7y ago
  Heck yeah! Years ago I had a nice little feed of my latest Tweets and Flickr photos on my personal website! It was fun incorporating them cleanly/seamlessly into my custom design. One day I'll get my site going again and have fun stuff like that -- if the 3rd party APIs even allow it these days :P hehe
• Lammy 7y ago
  > I also abused the hell out of Yahoo Pipes - I would run RSS feeds through like 15 different languages with Babelfish before back to English, just for kicks. For a while it was common to find SEO-spam sites composed entirely of posts generated this way. They would translate from a source language back to it in a roundabout way and end up with an article that was "different enough" to count as unique content to Google.
  • smcl 7y ago
    A British MP (I cannot for the life of me remember who) came under fire for owning a company made its money doing this - take existing content, shuffle the language round a bit and publish it with loads of ads.
    • matthewheath 7y ago
      Likely to be Grant Shapps > In 2012, Google blacklisted 19 of the Shapps's business websites for violating rules on copyright infringement related to the web scraping-based TrafficPayMaster software sold by them https://en.wikipedia.org/wiki/Grant_Shapps#Business_ventures https://en.wikipedia.org/wiki/Grant_Shapps#Business_ventures
      • smcl 7y ago
        Yes! That's him. I only knew about that one, but this other one mentioned afterwards is hella shady too: > It cost $497 and promised customers earnings of $20,000 in 20 days. Upon purchase, > the "toolkit" was revealed to be an ebook, advising the user to create their own > toolkit and recruit 100 "Joint Venture Partners" to resell it for a share of the > profits Sort of like a pyramid scheme, really. Incredibly this appears to have scarcely affected him, he is still an MP :-O
• cjhopman 7y ago
  There are 3 big problems with open APIs 1. they enable people to do things that other people think shouldn't be done 2. people get upset at companies when (1) happens 3. people get upset at companies for removing or restricting apis when, or in fear of, both (1) and (2).
  • ericflo 7y ago
    The framing of open APIs disappearing because of bad actors doesn't ring true to me. In my view, the golden age of APIs disappeared one-by-one as tech companies realized they won their respective markets and consolidated their power.
    • bryanrasmussen 7y ago
      I think what you said, yes, but also bad actors. Basically they disappeared because what business purpose did they serve, they were a cost center not a profit center - bad actors just increase the cost.
  • cortesoft 7y ago
    I think you are missing what is probably the biggest factor: they can be very expensive to run if successful. If you create a popular API, people are going to find creative uses for it, and because they can, by definition, be automated, you can get rapid growth in traffic with not that many users. There is a bit of a 'tragedy of the commons' that goes on, because the people writing apps that consume the API have no incentive to moderate their usage, or try to be efficient. Since the company that is providing this API is paying for the resources to run it, they can quickly get very expensive. Unless there is a clear financial benefit for allowing it to continue, most companies will shut them down eventually.
    • rplst8 7y ago
      Couldn't this all be solved with a p2p network that hosts restful services?
    • johannes1234321 7y ago
• dylan604 7y ago
  "OK Boomer!" One of the core tenets with the internet was information wanted to be free (as a bird). Then the evilCorps realized they could monetize information, and there went the free access. The information is less valuable if anyone can access it.
  • EGreg 7y ago
    Actually it’s MORE valuable if you add up everyone using it Just less money to be made by those with the power to restrict it Private property is just a monopoly right to exclude others from the use of something. If they could have used it, it would have been another instance of adding to the value. But someone has to pop up and demand rents and restrict them from doing so. Doesn’t that overall reduce the value of the information?
• pevezzac 7y ago
  I also remember vividly those optimistic times. You should show this video to your friend as proof: https://www.youtube.com/watch?v=6gmP4nk0EOE https://www.youtube.com/watch?v=6gmP4nk0EOE
• giantrobot 7y ago
  There was a lot of interesting promise in Web 2.0 that was completely wasted and then died. I think a big part of the problem was even things like Yahoo Pipes were too complicated for a lot of people and anyone trying to use third party APIs for commercial purposes ran afoul of EULAs or just plain old rent seeking. Once privacy invading advertising became the norm APIs were further restricted or discontinued because the user wasn't running a bunch of client side code tracking and scraping all their behavior on a web page. It didn't help that a lot of Web 2.0 darlings sold out to on-the-way-out Web 1.0 companies \cough\Yahoo\cough\. Yahoo's management couldn't even monetize money, let alone Web 2.0 properties. So instead we got social media silos. You can put stuff in but good luck ever getting it out. You can share it with whomever so long as they also join the same silo.
• joshspankit 7y ago
  It surprises me that we don’t have crypto-based API micro-transactions yet. Even if you had to pay 1 satoshi for 10 queries, it would go a long way towards making APIs viable long-term.
  • adrianN 7y ago
    What's the transaction rate for bitcoin these days? Is it still a handful of transactions per hour? What are the transaction costs?
    • ric2b 7y ago
      There's a layer 2 technology called Lightning Network that lets you transfer Bitcoin almost for free and in seconds, no transaction number limit. The biggest issue is liquidity, it's not easy to move large sums in a single transaction, but that shouldn't matter much for micro-payments. There's even ongoing work on making paid API's work seamlessly by adding some middleware, using the HTTP 402 (payment required) status code, no usernames/e-mails/passwords involved: https://lightning.engineering/posts/2020-03-30-lsat/ https://lightning.engineering/posts/2020-03-30-lsat/ (And yes, LN is open source, non-custodial and moves real Bitcoin, not IOU's or other tokens. It's actually an open spec with at least 3 major implementations in 3 different languages.)
• marcocampana 7y ago
  Those times were absolutely great. It's only human to look at the past with a feeling of melancholy and forget how great is the present and the long way we came. Today you can actually build your own APIs with very little knowledge and effort: easy to get started frameworks, free or inexpensive hosting in the cloud, lots of freely available data. That's pretty awesome too.
• redwall_hp 7y ago
  Even Google Search had this. I read an old O'Reilly book back then that was centered around API mashup projects (remember that term?) using the many Google APIs, which have mostly been shut down.