16 ms·

Yahoo Pipes

159 points by elsewhen 4y ago

sporritt 4y ago
Yahoo Pipes was the direct inspiration for jsPlumb, name and all, back in 2009. We were surprised they hadn't open sourced the UI and decided that somebody should make something like that available. Big fan of Pipes.
zegerjan 4y ago
What’s not quite clear from the wiki page: was Yahoo too early? With the related tools on the page being so successful it seems that way.
- robertlagrant 4y ago
  Too early and didn't want to be a tech company. If Google had made this, we'd probably all be using it. Still, at least Yahoo has a great exclamation mark in the logo now.
  - smcl 4y ago
    At the risk of appearing to take a cheap shot - I think if this was built at Google there’s a good chance it would’ve been abandoned or cancelled there too
    - nness 4y ago
      In fact, Google had, it was called the "Google Mashup Editor" and was closed in 2009.
      - smcl 4y ago
        Wow, I had no idea! What an awkward name, too. It's funny, "Pipes" doesn't on its own doesn't stand out as a spectacular name, but compared to "Mashup Editor" it's clearly superior. I wonder why - one syllable vs five, with a nod towards *nix pipes maybe?
        jsjohnst 4y ago
        Both Mashup Editor and the fact the name is based off Unix pipes is called out in the Wikipedia article, as well as describing the difference (albeit trivially) between Google and Yahoo’s offering.
        smcl 4y ago
        It's clear where the name is from - I was just wondering why Pipes feels like a good name and why Mashup Editor doesn't.
        headhasthoughts 4y ago
        One sounds like an actual product and the other sounds like an add-on to something else. "Yahoo Pipes! $7.99 a month with a generous free tier!" "Mashup Editor included with every Google Cloud subscription above $5/month." You never want a product to sound like an add-on: Add-ons are low-value and generally used to rent seek off of people who are already locked in. Naming something in a way that it feels like its own standalone product will inspire consumer confidence. Generic word + descriptor is an easy way to sound like an add-on. "Video Editor" could be for anything, but is certainly not an independent product. "Pachi Video Editor" sounds a bit more like its own thing. "Pachi Cutting Room" is closer to the mark, despite being two words and a company name.
        robertlagrant 4y ago
        "Mashup" also had a very short shelf-life even in the minds of buzzword-lovers.
- pell 4y ago
  I've used it briefly. I don't necessarily think it was too early but it was such a niche tool, hard to find, barely advertised to its potential target group, lacked documentation, demos and a community. I think, it was simply a result of Yahoo's chaotic product management at that time. They had a few gems that didn't get the attention they deserved - internally and from the public.
- sourcecodeplz 4y ago
  Are you referring to Zapier & IFTTT? I find them quite limited but I understand the need and also that not everyone can parse a structured response. It's nice that they have free versions though. Although more complicated to use but also free, I get a lot more functionality from PythonAnywhere, Cloudflare Workers or Oracle Cloud free tier.
- thrwaway2365 4y ago
  (disclaimer: former Yahoo employee) Yahoo at the time was clearly underestimated from a developer point of view. Everyone just saw them losing ground to Google in web search, but they had a really strong development team at the time. Lots of great stuff was open sourced, even though they did not win the same status as Google - YUI was a really good frontend framework based on components and events. It was so easy to create complex interfaces compared to others at the time. - Hadoop open sourced big data management - Vespa was a really good search engine in terms of performance and functionality. It seems to still be alive at http://vespa.ai http://vespa.ai - They provided several useful APIs without the restrictions that Google and others often placed on their products. They made some bad strategic decisions and other business decisions, but they had a lot of exciting technology that was often overlooked.
  - ignoramous 4y ago
    Yahoo should have pivoted from "web services" to IaaS like Amazon did with AWS, because it was no secret that Y! employed some of the best data-center folks besides Google and Microsoft. Could have been a different story?
  - Terretta 4y ago
    We used all these. Also Pipes. Thank you for your service. :-) // YUI was underrated.
- toast0 4y ago
  I was at Yahoo from 2004-2011. Yahoo often had products before the market was ready for them. Sometimes, the product would hang on and grow with the market; sometimes it would get shut down. In some cases, like mobile, Yahoo was both too early and too late. There was a good amount of time where Yahoo would launch something, shut it down, then Google would launch the same thing to much success. Of course, now Google is in the habit of shutting things down, another thing Yahoo did first ;) I used to think there'd be a lot of value in going through Yahoo PR launches from N years ago and if Yahoo had shut it down, consider if it makes sense as an independent business. Although it can be hard to tell why they shut things down, so it might not be obvious which things were the product worked but the market wasn't there and which things had non-obvious issues that made the product unworkable in practice.
samwillis 4y ago
Yahoo Pipes was awesome and born from the dream of an open web with open protocols and APIs. It was sad that this vision died and we ended up with so many closed "walled gardens". I hope we see a resurgence of this ideal, the recent expanse and interest in Mastodon, the Fediverse, and ActivityPub give me hope. Matt Mullenweg has committed to adding ActivityPub to Tumblr [0], if others in the established web ecosystem also make such moves maybe this will happen! 0: https://twitter.com/photomatt/status/1594577983028740096 https://twitter.com/photomatt/status/1594577983028740096
- lycos 4y ago
  I've started working on a custom ActivityPub implementation because I am interested by it, but I have to say my impressions so far is that it is more complicated to implement than I originally expected. I find the docs lacking, the test suite hasn't worked in a long time, either. Someone even made a document explaining how to read the relevant specs[0] Right now Mastodon seems to be the biggest player, but curious what the second/third biggest would be? Pleroma? Having Tumblr add compatibility seems like it could be huge. [0]: https://tinysubversions.com/notes/reading-activitypub/ https://tinysubversions.com/notes/reading-activitypub/
  - davidy123 4y ago
    https://semapps.org/ https://semapps.org/ looks interesting to take on a whole "ball of wax" at a useful level of knowledge. Would like to hear your thoughts.
    - lycos 4y ago
      It looks like it only does client to server (or at least their guide only does this). I'm working on server to server. But for me it's a hobby project anyway so I'll just keep trucking along with my stuff. But I'm also not developing with NodeJS so wouldn't work for me anyway.
  - mariusor 4y ago
    The sad situation is that most of the devs hurried into building applications that speak ActivityPub instead of creating libraries that can be used by others. If you're working with go, I created some libraries that are overlayed over the specification pretty closely. You can find them at github.com/go-ap
the100rabh 4y ago
I have been using NodeRed for past couple of year, running off my RPi. I must say, its been very a good replacement for Yahoo Pipes
piva00 4y ago
What a trip to memory lane this brought me to. I actually used Yahoo Pipes at a job back in 2008, the concept of data mashups was really interesting and we tried applying it to mesh our GIS data with other sources to mashup and enrich the results from our databases. I don't remember much from that project but do remember having fun with Yahoo Pipes interface, I had completely forgotten about it until this popped up.
CrypticShift 4y ago
This dream of no-code web data extraction and manipulation is still alive. The best example I've seen is https://www.bardeen.ai/ https://www.bardeen.ai/ : a local-first in-browser solid collection of automations commands and integrations workflows. The future, though, will more likely look like this https://www.adept.ai/act https://www.adept.ai/act
- ca123 4y ago
  Encourage you to take a look at https://docs.composable.ai https://docs.composable.ai
  - CrypticShift 4y ago
    This is more of a Business BI/Analytics low-code solution than a personal no code one. But you indeed took a similar dataflow approach, which could be considered "unique" in BI/Analytics. Good Luck!
arjvik 4y ago
How did Yahoo Pipes compare to Zapier or IFTTT?
- aloisdg 4y ago
  IMHO, Yahoo pipes UI and UX where far better than IFTTT. I don't know about Zapier.
nixcraft 4y ago
Another fun story: For the longest time the Yahoo pipes front page had an example pipe that merged search results from various online sites (amazon/ebay/cl). It was made by a former employee and was easily one of the most popular pipes. One day we found out he had his affiliate id in all of those links. We chuckled and moved on[1] [1] https://news.ycombinator.com/item?id=9661133 https://news.ycombinator.com/item?id=9661133
nycpig 4y ago
Yahoo! Pipes was the 2002 future I envisioned. Come 2015, it was paired against reality. From that point onwards, no longer was the OpenWeb the promise; but that of a walled garden. While I mourn what might have been, I accept the reality of my new corpo future.
- molotovh 4y ago
  Rage, rage against the dying of the light!
- mistrial9 4y ago
  big No on complacent appeasement here
jsjohnst 4y ago
I miss YQL much more than Yahoo! Pipes. My memory is hazy, but I think YQL is what ultimately killed Pipes actually, then YQL died because everything that didn’t generate revenue at Yahoo! died.
- zackkatz 4y ago
  Yahoo Pipes even supported running YQL as a node inside a pipe! It was so good.
- octagonal 4y ago
  You might enjoy https://pypi.org/project/htmlq/ https://pypi.org/project/htmlq/
est 4y ago
discovered n8n.io recently, seems a similar but more powerful product.
- blacksmith_tb 4y ago
  Not free, but with a similar UI.
naikrovek 4y ago
How well does Node-Red compare against this today? They seem comparable on the surface.
- octagonal 4y ago
  Node-RED is much more focused on IoT as far as I can tell, although the UIs look comparable.
dpz 4y ago
Any mention of Yahoo Pipes always makes me think of Tom Scott's "This video has x views". Feels quite relevant to other comments people have been making here. https://www.youtube.com/watch?v=BxV14h0kFs0 https://www.youtube.com/watch?v=BxV14h0kFs0
thedougd 4y ago
During the app gold rush I would use Yahoo pipes to build 'backends' for apps that were nothing more than screen scrapers. For example, you want to make an Android app that presents a home screen widget to show the current lottery jackpot or winning numbers. You could build a Yahoo Pipes thing to scrape and parse that, and format it into an RSS feed. The best part is anti-scraping measures allowed it because they thought it was a Yahoo crawler.
mwexler 4y ago
Loved Pipes. When I was at Yahoo!, folks kept showing such clever uses for it, all with minimal coding. It just worked. I miss it even today.
- edmundsauto 4y ago
  What were your favorite uses?
yegle 4y ago
There are a couple open source alternatives: Node-Red and Huginn. I use Huginn to scrape the gas price and send it to my Telegram.
capableweb 4y ago
Besides Zapier and IFTTT, what modern alternatives are there for Yahoo Pipes? Ideally with similar UI of a graph rather than "If this, then do that" in a sequential list.
- psadri 4y ago
  Initial version of Pipes was linear. Mid development we realized it was too limiting and decided to introduce “wires” to allow graphs. It meant a much more complicated UI (thanks to JT for pulling it off - mind you this was back in the IE9 days, no standardized canvas, SVG etc).
- avel 4y ago
  - node-RED - n8n.io
- ca123 4y ago
  Take a look at https://docs.composable.ai https://docs.composable.ai
Glench 4y ago
Always have to upvote for Yahoo Pipes. Also don't miss this great love letter to Yahoo Pipes: My Heart Feeds a Series of Tubes: https://medium.com/message/my-heart-feeds-a-series-of-tubes-de78858ddfc8 https://medium.com/message/my-heart-feeds-a-series-of-tubes-...
yevpats 4y ago
Really interesting to see this and YQL because so much changed but at the end of the day a lot of problems can be abstracted/solved as data integration problems. Currently working on http://github.com/cloudquery/cloudquery http://github.com/cloudquery/cloudquery
giancarlostoro 4y ago
I still feel like they missed out on the obvious move: charge money for Yahoo Pipes.
ZeroGravitas 4y ago
I started using the newsboat terminal RSS reader recently and it has some built in filters and ability to call out to scripts and it made me think of this. https://newsboat.org/releases/2.10.1/docs/newsboat.html#_filter_language https://newsboat.org/releases/2.10.1/docs/newsboat.html#_fil...

159 points by elsewhen 4y ago

Yahoo Pipes was the direct inspiration for jsPlumb, name and all, back in 2009. We were surprised they hadn't open sourced the UI and decided that somebody should make something like that available. Big fan of Pipes.

zegerjan 4y ago

What’s not quite clear from the wiki page: was Yahoo too early? With the related tools on the page being so successful it seems that way.

robertlagrant 4y ago
Too early and didn't want to be a tech company. If Google had made this, we'd probably all be using it. Still, at least Yahoo has a great exclamation mark in the logo now.
- smcl 4y ago
  At the risk of appearing to take a cheap shot - I think if this was built at Google there’s a good chance it would’ve been abandoned or cancelled there too
  - nness 4y ago
    In fact, Google had, it was called the "Google Mashup Editor" and was closed in 2009.
    - smcl 4y ago
      Wow, I had no idea! What an awkward name, too. It's funny, "Pipes" doesn't on its own doesn't stand out as a spectacular name, but compared to "Mashup Editor" it's clearly superior. I wonder why - one syllable vs five, with a nod towards *nix pipes maybe?
      - jsjohnst 4y ago
        Both Mashup Editor and the fact the name is based off Unix pipes is called out in the Wikipedia article, as well as describing the difference (albeit trivially) between Google and Yahoo’s offering.
        smcl 4y ago
        It's clear where the name is from - I was just wondering why Pipes feels like a good name and why Mashup Editor doesn't.
        headhasthoughts 4y ago
        One sounds like an actual product and the other sounds like an add-on to something else. "Yahoo Pipes! $7.99 a month with a generous free tier!" "Mashup Editor included with every Google Cloud subscription above $5/month." You never want a product to sound like an add-on: Add-ons are low-value and generally used to rent seek off of people who are already locked in. Naming something in a way that it feels like its own standalone product will inspire consumer confidence. Generic word + descriptor is an easy way to sound like an add-on. "Video Editor" could be for anything, but is certainly not an independent product. "Pachi Video Editor" sounds a bit more like its own thing. "Pachi Cutting Room" is closer to the mark, despite being two words and a company name.
        robertlagrant 4y ago
        "Mashup" also had a very short shelf-life even in the minds of buzzword-lovers.
pell 4y ago
I've used it briefly. I don't necessarily think it was too early but it was such a niche tool, hard to find, barely advertised to its potential target group, lacked documentation, demos and a community. I think, it was simply a result of Yahoo's chaotic product management at that time. They had a few gems that didn't get the attention they deserved - internally and from the public.
sourcecodeplz 4y ago
Are you referring to Zapier & IFTTT? I find them quite limited but I understand the need and also that not everyone can parse a structured response. It's nice that they have free versions though. Although more complicated to use but also free, I get a lot more functionality from PythonAnywhere, Cloudflare Workers or Oracle Cloud free tier.
thrwaway2365 4y ago
(disclaimer: former Yahoo employee) Yahoo at the time was clearly underestimated from a developer point of view. Everyone just saw them losing ground to Google in web search, but they had a really strong development team at the time. Lots of great stuff was open sourced, even though they did not win the same status as Google - YUI was a really good frontend framework based on components and events. It was so easy to create complex interfaces compared to others at the time. - Hadoop open sourced big data management - Vespa was a really good search engine in terms of performance and functionality. It seems to still be alive at http://vespa.ai http://vespa.ai - They provided several useful APIs without the restrictions that Google and others often placed on their products. They made some bad strategic decisions and other business decisions, but they had a lot of exciting technology that was often overlooked.
- ignoramous 4y ago
  Yahoo should have pivoted from "web services" to IaaS like Amazon did with AWS, because it was no secret that Y! employed some of the best data-center folks besides Google and Microsoft. Could have been a different story?
- Terretta 4y ago
  We used all these. Also Pipes. Thank you for your service. :-) // YUI was underrated.
toast0 4y ago
I was at Yahoo from 2004-2011. Yahoo often had products before the market was ready for them. Sometimes, the product would hang on and grow with the market; sometimes it would get shut down. In some cases, like mobile, Yahoo was both too early and too late. There was a good amount of time where Yahoo would launch something, shut it down, then Google would launch the same thing to much success. Of course, now Google is in the habit of shutting things down, another thing Yahoo did first ;) I used to think there'd be a lot of value in going through Yahoo PR launches from N years ago and if Yahoo had shut it down, consider if it makes sense as an independent business. Although it can be hard to tell why they shut things down, so it might not be obvious which things were the product worked but the market wasn't there and which things had non-obvious issues that made the product unworkable in practice.

samwillis 4y ago

Yahoo Pipes was awesome and born from the dream of an open web with open protocols and APIs. It was sad that this vision died and we ended up with so many closed "walled gardens". I hope we see a resurgence of this ideal, the recent expanse and interest in Mastodon, the Fediverse, and ActivityPub give me hope. Matt Mullenweg has committed to adding ActivityPub to Tumblr [0], if others in the established web ecosystem also make such moves maybe this will happen! 0: https://twitter.com/photomatt/status/1594577983028740096 https://twitter.com/photomatt/status/1594577983028740096

lycos 4y ago
I've started working on a custom ActivityPub implementation because I am interested by it, but I have to say my impressions so far is that it is more complicated to implement than I originally expected. I find the docs lacking, the test suite hasn't worked in a long time, either. Someone even made a document explaining how to read the relevant specs[0] Right now Mastodon seems to be the biggest player, but curious what the second/third biggest would be? Pleroma? Having Tumblr add compatibility seems like it could be huge. [0]: https://tinysubversions.com/notes/reading-activitypub/ https://tinysubversions.com/notes/reading-activitypub/
- davidy123 4y ago
  https://semapps.org/ https://semapps.org/ looks interesting to take on a whole "ball of wax" at a useful level of knowledge. Would like to hear your thoughts.
  - lycos 4y ago
    It looks like it only does client to server (or at least their guide only does this). I'm working on server to server. But for me it's a hobby project anyway so I'll just keep trucking along with my stuff. But I'm also not developing with NodeJS so wouldn't work for me anyway.
- mariusor 4y ago
  The sad situation is that most of the devs hurried into building applications that speak ActivityPub instead of creating libraries that can be used by others. If you're working with go, I created some libraries that are overlayed over the specification pretty closely. You can find them at github.com/go-ap

the100rabh 4y ago

I have been using NodeRed for past couple of year, running off my RPi. I must say, its been very a good replacement for Yahoo Pipes

piva00 4y ago

What a trip to memory lane this brought me to. I actually used Yahoo Pipes at a job back in 2008, the concept of data mashups was really interesting and we tried applying it to mesh our GIS data with other sources to mashup and enrich the results from our databases. I don't remember much from that project but do remember having fun with Yahoo Pipes interface, I had completely forgotten about it until this popped up.

CrypticShift 4y ago

This dream of no-code web data extraction and manipulation is still alive. The best example I've seen is https://www.bardeen.ai/ https://www.bardeen.ai/ : a local-first in-browser solid collection of automations commands and integrations workflows. The future, though, will more likely look like this https://www.adept.ai/act https://www.adept.ai/act

ca123 4y ago
Encourage you to take a look at https://docs.composable.ai https://docs.composable.ai
- CrypticShift 4y ago
  This is more of a Business BI/Analytics low-code solution than a personal no code one. But you indeed took a similar dataflow approach, which could be considered "unique" in BI/Analytics. Good Luck!

arjvik 4y ago

How did Yahoo Pipes compare to Zapier or IFTTT?

aloisdg 4y ago
IMHO, Yahoo pipes UI and UX where far better than IFTTT. I don't know about Zapier.

nixcraft 4y ago

Another fun story: For the longest time the Yahoo pipes front page had an example pipe that merged search results from various online sites (amazon/ebay/cl). It was made by a former employee and was easily one of the most popular pipes. One day we found out he had his affiliate id in all of those links. We chuckled and moved on[1] [1] https://news.ycombinator.com/item?id=9661133 https://news.ycombinator.com/item?id=9661133

nycpig 4y ago

Yahoo! Pipes was the 2002 future I envisioned. Come 2015, it was paired against reality. From that point onwards, no longer was the OpenWeb the promise; but that of a walled garden. While I mourn what might have been, I accept the reality of my new corpo future.

molotovh 4y ago
Rage, rage against the dying of the light!
mistrial9 4y ago
big No on complacent appeasement here

jsjohnst 4y ago

I miss YQL much more than Yahoo! Pipes. My memory is hazy, but I think YQL is what ultimately killed Pipes actually, then YQL died because everything that didn’t generate revenue at Yahoo! died.

zackkatz 4y ago
Yahoo Pipes even supported running YQL as a node inside a pipe! It was so good.
octagonal 4y ago
You might enjoy https://pypi.org/project/htmlq/ https://pypi.org/project/htmlq/

est 4y ago

discovered n8n.io recently, seems a similar but more powerful product.

blacksmith_tb 4y ago
Not free, but with a similar UI.

naikrovek 4y ago

How well does Node-Red compare against this today? They seem comparable on the surface.

octagonal 4y ago
Node-RED is much more focused on IoT as far as I can tell, although the UIs look comparable.

dpz 4y ago

Any mention of Yahoo Pipes always makes me think of Tom Scott's "This video has x views". Feels quite relevant to other comments people have been making here. https://www.youtube.com/watch?v=BxV14h0kFs0 https://www.youtube.com/watch?v=BxV14h0kFs0

thedougd 4y ago

During the app gold rush I would use Yahoo pipes to build 'backends' for apps that were nothing more than screen scrapers. For example, you want to make an Android app that presents a home screen widget to show the current lottery jackpot or winning numbers. You could build a Yahoo Pipes thing to scrape and parse that, and format it into an RSS feed. The best part is anti-scraping measures allowed it because they thought it was a Yahoo crawler.

mwexler 4y ago

Loved Pipes. When I was at Yahoo!, folks kept showing such clever uses for it, all with minimal coding. It just worked. I miss it even today.

edmundsauto 4y ago
What were your favorite uses?

yegle 4y ago

There are a couple open source alternatives: Node-Red and Huginn. I use Huginn to scrape the gas price and send it to my Telegram.

capableweb 4y ago

Besides Zapier and IFTTT, what modern alternatives are there for Yahoo Pipes? Ideally with similar UI of a graph rather than "If this, then do that" in a sequential list.

psadri 4y ago
Initial version of Pipes was linear. Mid development we realized it was too limiting and decided to introduce “wires” to allow graphs. It meant a much more complicated UI (thanks to JT for pulling it off - mind you this was back in the IE9 days, no standardized canvas, SVG etc).
avel 4y ago
- node-RED - n8n.io
ca123 4y ago
Take a look at https://docs.composable.ai https://docs.composable.ai

Glench 4y ago

Always have to upvote for Yahoo Pipes. Also don't miss this great love letter to Yahoo Pipes: My Heart Feeds a Series of Tubes: https://medium.com/message/my-heart-feeds-a-series-of-tubes-de78858ddfc8 https://medium.com/message/my-heart-feeds-a-series-of-tubes-...

yevpats 4y ago

Really interesting to see this and YQL because so much changed but at the end of the day a lot of problems can be abstracted/solved as data integration problems. Currently working on http://github.com/cloudquery/cloudquery http://github.com/cloudquery/cloudquery

giancarlostoro 4y ago

I still feel like they missed out on the obvious move: charge money for Yahoo Pipes.

ZeroGravitas 4y ago

I started using the newsboat terminal RSS reader recently and it has some built in filters and ability to call out to scripts and it made me think of this. https://newsboat.org/releases/2.10.1/docs/newsboat.html#_filter_language https://newsboat.org/releases/2.10.1/docs/newsboat.html#_fil...