15 ms·

Huginn: Create agents that monitor and act on your behalf

1303 points by daolf 7y ago

david_draco 7y ago
"Once a day, ask 5 people for a funny cat photo; send the results to 5 more people to be rated; send the top-rated photo to 5 people for a funny caption; send to 5 final people to rate for funniest caption; finally, post the best captioned photo on my blog." I'm still laughing :) wth! (My fear is that this might actually be sustainable with ads.)
- omarhaneef 7y ago
  This is basically the plan of The New Yorker, but with (probably) 1000s of people.
  - Eduard 7y ago
    Can you elaborate?
    - omarhaneef 7y ago
      If you don't follow this, you are in for quite a treat. Enjoy: https://www.newyorker.com/cartoons/contest#finalists https://www.newyorker.com/cartoons/contest#finalists
- antpls 7y ago
  I agree, that was an eye-opener example
- apeddle 7y ago
  For about a week a few years back, I ran a twitter bot that did essentially this for tweets powered by mechanical turk. I enjoyed waking up every morning to an often strange political message posted by my "bot". @CrowdWisdomBot if anyone is curious :) Doing it with cat photos is far more clever.
penagwin 7y ago
Also checkout Node Red [0], it's fairly popular in the automation space. It's rather sparse by default, but after adding in some community nodes (or making some yourself) it's pretty useful. https://nodered.org/ https://nodered.org/
- GordonS 7y ago
  I imagine things have changed, but I tried node-red maybe 3 years ago, and found it really difficult to setup and get working. Once I finally had it working, I found it really difficult to debug "tasks" (I forget the correct, node-red verbiage), for example to find out why an HTTP call was failing. I also found the available connectors to be lacking, and a lot I tried were quite basic.
rvz 7y ago
For those using expensive/advanced connectors like Zapier, Tray.io, etc. I find that https://n8n.io https://n8n.io serves as a far welcoming open-source alternative that is worth looking at.
- agentdrtran 7y ago
  This is nowhere near as robust as Zapier and has significant limitations.
  - janober 7y ago
    n8n does also have way fewer integrations. About the "robust" part. A startup I know had a lot of issues with Zapier and switched to n8n and are quite happy as they did not have any issues since. I am sure you are right and you did your research but would love to hear what exactly you are referring to as it will help me to improve n8n. The same with the limitations. Anything specific?
    - toomuchtodo 7y ago
      Have you considered generating integrations pragmatically using OpenAPI?
      - janober 7y ago
        Yes, I did but did not get to it yet. Also not sure if this integrations would then be the "best" to use. I think it would be great to have at least "something" if nothing exists yet but would not be as good and easy as regular integrations and so would not fit perfectly into n8n as I really want to have it as easy as possible for users.
- PatrolX 7y ago
  Nowhere near as flexible and powerful as Huginn.
  - deleted 7y ago
PatrolX 7y ago
Not just an alternative though. - Self-hosted - Unlimited and FREE vs $699 / month - Create your own agents / integrations - Process data through shell scripts or JavaScript - Better filtering - Liquid templating - Completely private I save thousands of dollars by using Huginn. It's incredibly powerful and quite frankly I don't trust Zapier with my data and visibility of what I'm doing because there are commercial implications. Run it in the cloud on AWS or on an old box at home. It's very reliable.
- PopeDotNinja 7y ago
  > I save thousands of dollars by using Huginn. Does that take into account the overhead of operating Huginn?
  - PatrolX 7y ago
    Yes, and it's running thousands of requests and posts / minute on AWS. But you don't even need to put it in the cloud, it runs perfectly at home on an old box running Ubuntu.
    - tectonic 7y ago
      That's awesome that it's working so well for you :)
  - vorpalhex 7y ago
    You can deploy it as a docker image in a few minutes. If you already run a homelab of any sort (which most of HN does), it's basically free.
    - boring_twenties 7y ago
      > If you already run a homelab of any sort (which most of HN does) You sure about that? I feel like there are plenty of folks on here whose only computer is a MacBook
jaequery 7y ago
This looks worth checking out but I’d love to see a Lambda version of it
- pixiemaster 7y ago
  Yes, that would be the killer feature
  - deleted 7y ago
    [deleted]
  - mathnmusic 7y ago
    Related idea that was on HN a few days ago: https://gist.github.com/nileshtrivedi/6024434121b073789e6d580eb84c3ea2 https://gist.github.com/nileshtrivedi/6024434121b073789e6d58...
- swyx 7y ago
  what would be your preferred way of scheduling lambdas and doing storage?
  - DailyHN 7y ago
    I use HTTP cloud functions (google/firebase version of aws lambda). Firebase database for long term persistence. Huginn for scheduling and event based triggers of those cloud functions.
minimaxir 7y ago
For a more business-friendly-but-still-FOSS approach to task automation between services, see also Apache Airflow: https://airflow.apache.org https://airflow.apache.org Airflow has been a skill that many companies ask for (especially data engineering), but surprisingly doesn't have many articles written about it.
- codetrotter 7y ago
  The Airflow landing page that you linked to lists many integrations but when you click on those only a small subset of them are listed in the integrations section of the docs that is linked to. I guess the docs are in need of some more work.
  - minimaxir 7y ago
    It's easier to look at the APIs: https://airflow.apache.org/docs/stable/_api/airflow/operators/index.html https://airflow.apache.org/docs/stable/_api/airflow/operator... https://airflow.apache.org/docs/stable/_api/airflow/contrib/operators/index.html https://airflow.apache.org/docs/stable/_api/airflow/contrib/...
esquire_900 7y ago
This, as well as related projects like n8n & node-red, is a very cool project. I always wonder what people use it for in real life though. It seems a lot of trouble (setting up, learning curve, maintaining) for an action that usually takes a couple of seconds, like checking the weather or opening twitter. Does anybody have useful workflows going on?
- mcbits 7y ago
  Ever notice when someone criticizes a company on HN, and suddenly the CEO shows up and engages after 6 months of inactivity? Likely they are using a tool like this along with Google Alerts, etc, to monitor sites for mentions of the company and maybe flag negative sentiment. Also, same thing but for politics.
  - janober 7y ago
    Yes, very interesting indeed! Either there is a big conspiracy to uncover or said CEO reads HN and if something is number 1 and is related to his product he pays close attention. We will maybe never find out the truth...
    - mcbits 7y ago
      From your defensive tone it sounds like you think I was talking about you, but I really wasn't. It's a phenomenon I've observed here and on Reddit and elsewhere for many years. And it seems like a perfect fit for a tool such as yours, is it not?
      - janober 7y ago
        Very sorry! As I was the only "CEO" in here which product got criticized I thought it must have been me ;-) Yes would actually be perfect for it, but never had the time to set something like that up. Maybe someday in the future. For now, I only have a simple Google Alert set and regularly check HN and Google Analytics which is a good indicator when it got mentioned somewhere.
howmayiannoyyou 7y ago
Zapier has exponentially more integrations than this or anything else, but is surprisingly difficult to use, and more so since they updated their UI. Editing Zaps is pure torture because refresh is so difficult. Exception handling is pure pain. In most instances the breadth of API calls is so narrow, and so rarely updated by vendors that you end up switching to a custom integration. I've also noticed vendors rapidly expanding their native integrations, sidestepping the need for a request broker. The "retail" integration space remains underserved and if one of the enterprise players decided to go down-market with a better UI and deeper integrations - they'd mop the floor clean in 18 months.
- rogerkirkness 7y ago
  We tend to agree re: retail integration, assuming you mean among retailers as opposed to message brokering.
- toomuchtodo 7y ago
  > The "retail" integration space remains underserved and if one of the enterprise players decided to go down-market with a better UI and deeper integrations - they'd mop the floor clean in 18 months. I think it's a tough market to be "winner" of. Novices are going to want a stupid simple GUI ("wizard mode", as someone else in thread mentioned). Power users are going to want to be able to toss in some code at some point in a workflow to do some fancy ETL you don't support out of the box. When you hit a certain level of complexity, an edge case or integration an automation product doesn't support, or perhaps even an amount of spend that you start looking at annually as painful, it's likely you consider pulling all of your workflows out and have a software engineer build something bespoke for your business line.
  - wpietri 7y ago
    Yeah, I wonder if this space is sort of like the to-do/task-management space. It's conceptually one thing, but in practice people have such different needs that no one product can be everything to everybody.
- PatrolX 7y ago
ikeboy 7y ago
I don't see a list of all supported agents or a way to search.
- davidgf 7y ago
  It's in the wiki https://github.com/huginn/huginn/wiki/Agent-Types-&-Descriptions https://github.com/huginn/huginn/wiki/Agent-Types-&-Descript...
- dsander2 7y ago
  Started the site a long time ago, but never finished. Nevertheless https://huginnio.herokuapp.com/ https://huginnio.herokuapp.com/ has a searchable Agent database an a few Scenarios.
soared 7y ago
I would not market this as an alternative to Zapier since it goes after a different market. The type of people who use Zapier will see a link to github.com and not even consider it. I think this should be positioned more like "A high powered Zapier for technical users" or something similar.
- organsnyder 7y ago
  "Self-hosted" is a growing buzzword these days (for many good reasons, IMHO).
- deleted 7y ago
  [deleted]
traspler 7y ago
I have been running huginn on my home server for a while. I've mainly used it to filter RSS feeds and then generate new feeds with the filtered items. Another use-case for me is ingesting webcomic RSS feeds (or scrape a page) and post the comics to a private Telegram channel. Once I also had an agent scraping a page and notifying me if something changed (realestate listing). I have tried a couple alternatives, e.g. node-red but none really worked the way I wanted them to for these cases. huginn is incredibly flexible and (at least for me) the mental model of it's workflow makes a lot of sense. Sadly more and more pages want you to go through their app/site and make it a bit difficult to work with, e.g. getting content from an instagram account. One thing I have not figured out about huginn and which all of these automation tools seem to lack are loops. E.g I have page an agent scrapes, from which I want to output the src of an image tag but I also want to check if a certain condition on the page matches (e.g. a "next page" button exists) and then firstly output the found src but then also re-invoke the agent with a new input element. So it would scrape the next page and so on until it does not find the button anymore.
- PatrolX 7y ago
  > One thing I have not figured out about huginn and which all of these automation tools seem to lack are loops. I do some very complex stuff using the "Shell Command Agent". You might want to look into using that if you haven't already. You can also create your own agent gem https://github.com/huginn/huginn_agent https://github.com/huginn/huginn_agent
- robk 7y ago
  Yeah I just use the Javascript agent for v complex stuff like this.
  - traspler 7y ago
    Oh, how could I have missed the JS Agent... thanks a lot for the hint! But it looks like it's not node, so no npm modules and other node APIs.
    - tectonic 7y ago
caseysoftware 7y ago
It's not quite 1:1 but not a bad comparison. I like to think of Zapier as "wizard mode" - if you're doing something within their pattern, planned workflow, capabilities, etc it is a great option. If you need to do something more advanced or outside that narrow scope, you need to go to the APIs (or Huginn) itself. The important questions are: How many people need to get into advanced mode? How big of an effort/skill/understanding jump is it to go there? (aka will it hurt too much to switch?)
curo 7y ago
Something I'm not entirely clear on: One use case for Zapier (from a developer/company standpoint) is to allow customers to connect their existing services to actions inside your own app. For instance, if a customer updates a CRM record, you can have a custom zap update a record in your own SaaS platform. To pull that off with huginn, is that as simple as connecting this up to Singer.io? Or would that require a big marketplace of huginn agents for popular integrations?
tectonic 7y ago
It's fun to wake up to find a project that I started on the top of HN! These days, I'm no longer very involved with the day-to-day of the project. Now that it's no longer a young project, here are some musings about Huginn and responses to people's comments in this thread, in no particular order. I've found that Huginn excels as a scheduled web scraper with lightweight filtering. That's what I use it for. On the other hand, while you can write custom code in it, Huginn is pretty poor at implementing any sort of complex logic, and is even worse at bidirectional syncing between systems, which is something people often want it to do, but for which it wasn't designed. If IFTTT or Zapier meet your needs, awesome! No need to run and monitor your own service. I personally choose to run Huginn on my own hardware in large part so that I'm comfortable giving it website cookies and passwords. Some examples of what I use Huginn for these days: - Watching Twitter in realtime for high standard deviation spikes in certain keywords, such as "san francisco emergency" or "san francisco tsunami warning", which then sends me a push notification, or "huginn open source", that goes to a digest email (and I imagine will trigger because of this thread). - Watching Twitter for rare terms and sending me a digest of all tweets that match them. Also sending me all tweets from a few Twitter users who post rarely, but that I don't want to miss. - Scraping a number of rarely updated blogs that don't have email newsletters and emailing me when they change. Some use RSS, most are just simple HTML scraping. - Pulling deals from the frontpage and forums of slickdeals and craigslist and filtering them for certain keywords. - Sending an early morning email if it's going to rain today. - Watching ebay for some rare items. - Sending my wife and me an email on Saturday morning with local yardsales from craigslist. - Watching the HN and producthunt front pages for certain keywords. Basically, anytime I find myself checking a website more then a few times, I spend 20min making a Huginn Agent to do it for me. I think one reason Huginn has worked well for me is that I don't try to make it do too much. I use it for scraping data and gentle filtering, and that's about it. It's been super helpful for alerting me to interesting content for The Orbital Index, my current project, a weekly space-industry newsletter. (Last issue: http://orbitalindex.com/archive/2019-12-10-Issue-42/ http://orbitalindex.com/archive/2019-12-10-Issue-42/)
- babyyoda 7y ago
  Awesome project! Super impressive and will check out later
bitshift 7y ago
There is also ActiveWorkflow[1], which targets business users and let's you write custom agents in any programming language[2]. 1. https://github.com/automaticmode/active_workflow https://github.com/automaticmode/active_workflow 2. https://github.com/automaticmode/active_workflow/blob/master/docs/remote_agent_api.md https://github.com/automaticmode/active_workflow/blob/master...
- gotts 7y ago
  added to bookmarks. Thanks for sharing! It's always good to have alternatives
- tectonic 7y ago
  Huh, I did not know about this fork of Huginn.
  - bpuvanathasan 7y ago
    https://tines.io https://tines.io (security automation platform) might also be based on (or inspired by) Huginn
j1elo 7y ago
I've seen the word "rake", so I guess... is this written in Ruby? If so, how's the performance? My home server is pretty minimal and lightweight: a Raspberry Pi. Do you think it will run it fine? (I'm gonna want to try this anyway, didn't know about it until now and it looks amazing!)
- tectonic 7y ago
  Performance is good, memory requirements could be lower though. It should run fine with 1GB of RAM, possibly less.
- heavyset_go 7y ago
  A fresh Huginn container idles at ~300MB of memory without any configuration on my part. A fresh node-red container idles at 30MB.
  - floren 7y ago
    I was unable to run the Huginn Docker container on my Pi, I assume because there are x86 binaries in it. I'm trying the manual install now...
    - heavyset_go 7y ago
      Yeah, there's only a x86 Docker image on Docker Hub for Huginn. You could probably just build an ARM image from whatever Dockerfile the x86 version was built from.
binthere 7y ago
I'm probably missing something here, but what is the value of this when you could write a script that would do just about the same? (Serious question, I want to know the answer, not rhetorical)
- bluehatbrit 7y ago
  Huginn is more of a framework for writing and running these scripts. It has some built in modules to make the process easier. IE: The webhook agent is an easy want of getting POST data into a script without bootstrapping a webserver yourself, you can then just raise an event and have another script handle it. No need to write and manage everything individually, you can run everything in one place.
  - DailyHN 7y ago
    Yes, and this becomes exceedingly important as the number and complexity of automations increases. Huginn can assist in organizing complex workflows into manageable components.
m-p-3 7y ago
Interesting, I should switch to that for some location-based action with my phone instead of IFTTT and see if I can make it work with Hauk or something similar. https://github.com/bilde2910/Hauk https://github.com/bilde2910/Hauk Disclaimer: I have no link with the Hauk project, just found about it by browsing the F-Droid store.
dang 7y ago
A thread from 2016: https://news.ycombinator.com/item?id=11015147 https://news.ycombinator.com/item?id=11015147 2014: https://news.ycombinator.com/item?id=7585605 https://news.ycombinator.com/item?id=7585605 2013: https://news.ycombinator.com/item?id=5377651 https://news.ycombinator.com/item?id=5377651
osprojects 7y ago
An alternative to webhook automation would be https://github.com/adnanh/webhook https://github.com/adnanh/webhook
ksrm 7y ago
Not to be confused with Hugin: http://hugin.sourceforge.net/ http://hugin.sourceforge.net/
- KingFelix 7y ago
  also not to be confused with Hugging -
- ISL 7y ago
  Agreed. It was a surprise to see Hugin get top billing on HN other than for awesome panoramic image stitching.
  - ISL 7y ago
    I see that this particular horse was already beaten in 2013: https://news.ycombinator.com/item?id=5377815 https://news.ycombinator.com/item?id=5377815
moomin 7y ago
How does this not have a companion technology called Muninn?
kasbah 7y ago
Seems like all of these self-hosted automation apps struggle to incorporate Facebook adapters. Does anyone know of one that would allow me to read Tweets and post ones containing some particular text to a Facebook page (not my personal timeline/page but one I created for my project)?
anderspitman 7y ago
If you like this kind of stuff, you might find my patchbay project interesting: https://patchbay.pub/ https://patchbay.pub/
vorpalhex 7y ago
I've deployed Huginn a few times but never gotten off the ground with it. I found agents a bit too cumbersome to configure, but maybe I just need to give it another shot.
audiometry 7y ago
Is there any way to implement agents in python rather than JavaScript/ruby? Looks interesting but I don’t want to invest energy in building fluency in these other scripting languages.
- DailyHN 7y ago
  Not exactly. You could write python externally (cloud functions/aws lambda) and then use huginn to call that over HTTP. This doesn't require any coding. Huginn has existing agents that you can easily configure to make HTTP requests (PostAgent and WebsiteAgent). And, depending what you are planning to do with huginn, simply sending data from your python script to huginn using the WebhookAgent may be sufficient.
- oskapt 7y ago
  I run Huginn in a Kubernetes cluster with most of its heavy lifting done by HTTP calls to OpenFaaS running in the same cluster. This let me write complex logic in Python but have Huginn handle the execution, triggers, filtering, and delivery of the data. Among other things I use it to scrape Twitter for interesting Kubernetes news and follow the retweets back to people who link to their own websites so that our editorial department can vet them for invitation into our writing program. Any particular author/account only appears once. It generates around 40 new candidates each day. I also use it to scrape the Chilean public alert website (onemi.cl) for alerts in my region ( forest fire, major weather), run those through the translate API and then send the English translations to my and my wife’s phones via Pushover. These are things that I want to control and customize, and I vastly prefer running my own systems over trusting any third party.
- j88439h84 7y ago
  This comment links to a fork that allows other langauges https://news.ycombinator.com/item?id=21773803 https://news.ycombinator.com/item?id=21773803
DailyHN 7y ago
I've operated a huginn for 5+ years. AMA
- LemonAndroid 7y ago
  What did you have for breakfast?
  - DailyHN 7y ago
    I don't eat breakfast.
    - swyx 7y ago
      What about second breakfast?
      - DailyHN 7y ago
        Sometimes a smoothie around 3pm that contains plant proteins, collagen protein, amazing grass, 1/4 cup of blueberries, ground chia seeds, and maca powder. I've been intermittent fasting + keto for the majority of the past five years. The minority is when we cycle off keto for a week here and there by doing intermittent fasting + vegan.
- dsander2 7y ago
  One of the current maintainers here, which installation method are you using and how is it working for you?
  - DailyHN 7y ago
    @dsander I immediately recognized your username from GitHub Thanks for all the work you've put in! I run Huginn on Heroku. I haven't had any significant issues since it's been running. It's been so long since I've set it up that I would need to review my notes to tell you any more about how it's deployed.
- 7y ago
keyle 7y ago
This reads like DIY Yahoo Pipes.
vunie 7y ago
How well does it support: - support remote agents and remote reporting? - dead-man triggers for missing reports? - is remote reporting properly secured? (encrypted/signed)
- DailyHN 7y ago
  What do you mean remote agents? Can it be accessed over HTTP? Then yes. One of the great features of huginn is the diagram. That's where there are green and red indicators. Red indicates both errors and when a longer than expected period has passed. This makes quickly inspecting your workflows easy.
  - vunie 7y ago
    > What do you mean remote agents? Sorry. I meant running an agent on a machine that is separate from the machine running huginn. The machine running the agent is not accessible publicly but has internet access (doesn't accept connections but can initiate them). huginn would be accessible publicly.
    - DailyHN 7y ago
      Ok, so in the remote agent: Retrieving data from huginn can be done with the DataOutputAgent. Sending data to huginn can be achieved through the WebhookAgent.
devm0de 7y ago
I gave this a go today and managed to install huginn on my synology nas by simply searching for the docker container. I then setup 3 agents to scrape a Shopify webstore jason endpoint that I’m always checking for inventory, have huginn parse the json and send me an sms via twilio if inventory changes. Took about 2 hours, wasn’t too bad. Huginn twilio docs seemed dated. Used python simplehttp server and ngrok to replicate a json url and play with the triggers to test it all before pointing it at a real website. Nice to add a new tool to the belt, thanks!
theshadowmonkey 7y ago
I’m glad Huginn is on the front page. It’s an awesome project being used for a long time now. I was testing Huginn to see how skimpy I can be and still run Huginn on a free tier. Was able to run it on open shift free tier a while ago when their allowance was generous. But, looks like it’s hard now. Will try running on a Gcp instance and see if it works.
e_carra 7y ago
It really sounds like Son of Anthon from the last season of Silicon Valley.
elwell 7y ago
> Create Amazon Mechanical Turk workflows as the inputs, or outputs, of agents (the Amazon Turk Agent is called the "HumanTaskAgent"). For example: "Once a day, ask 5 people for a funny cat photo; send the results to 5 more people to be rated; send the top-rated photo to 5 people for a funny caption; send to 5 final people to rate for funniest caption; finally, post the best captioned photo on my blog." Curious how this would turn out.
- DailyHN 7y ago
  You should try it out.
lngnmn1 7y ago
How come it is not NodeJS?
franga2000 7y ago
I've come across Huginn many times in the past and every time, I tell myself I'm going to give it a shot. I still haven't even seen it loaded in a web browser... After fighting the Docker images for half an hour, I usually realize that I could've just written a Python script to do whatever I was trying to do by then and proceed to do just that. Sorry for being lazy! Maybe I'll get around to it next year...

1303 points by daolf 7y ago

"Once a day, ask 5 people for a funny cat photo; send the results to 5 more people to be rated; send the top-rated photo to 5 people for a funny caption; send to 5 final people to rate for funniest caption; finally, post the best captioned photo on my blog." I'm still laughing :) wth! (My fear is that this might actually be sustainable with ads.)

omarhaneef 7y ago
This is basically the plan of The New Yorker, but with (probably) 1000s of people.
- Eduard 7y ago
  Can you elaborate?
  - omarhaneef 7y ago
    If you don't follow this, you are in for quite a treat. Enjoy: https://www.newyorker.com/cartoons/contest#finalists https://www.newyorker.com/cartoons/contest#finalists
antpls 7y ago
I agree, that was an eye-opener example
apeddle 7y ago
For about a week a few years back, I ran a twitter bot that did essentially this for tweets powered by mechanical turk. I enjoyed waking up every morning to an often strange political message posted by my "bot". @CrowdWisdomBot if anyone is curious :) Doing it with cat photos is far more clever.

penagwin 7y ago

Also checkout Node Red [0], it's fairly popular in the automation space. It's rather sparse by default, but after adding in some community nodes (or making some yourself) it's pretty useful. https://nodered.org/ https://nodered.org/

GordonS 7y ago
I imagine things have changed, but I tried node-red maybe 3 years ago, and found it really difficult to setup and get working. Once I finally had it working, I found it really difficult to debug "tasks" (I forget the correct, node-red verbiage), for example to find out why an HTTP call was failing. I also found the available connectors to be lacking, and a lot I tried were quite basic.

rvz 7y ago

For those using expensive/advanced connectors like Zapier, Tray.io, etc. I find that https://n8n.io https://n8n.io serves as a far welcoming open-source alternative that is worth looking at.

agentdrtran 7y ago
This is nowhere near as robust as Zapier and has significant limitations.
- janober 7y ago
  n8n does also have way fewer integrations. About the "robust" part. A startup I know had a lot of issues with Zapier and switched to n8n and are quite happy as they did not have any issues since. I am sure you are right and you did your research but would love to hear what exactly you are referring to as it will help me to improve n8n. The same with the limitations. Anything specific?
  - toomuchtodo 7y ago
    Have you considered generating integrations pragmatically using OpenAPI?
    - janober 7y ago
      Yes, I did but did not get to it yet. Also not sure if this integrations would then be the "best" to use. I think it would be great to have at least "something" if nothing exists yet but would not be as good and easy as regular integrations and so would not fit perfectly into n8n as I really want to have it as easy as possible for users.
PatrolX 7y ago
Nowhere near as flexible and powerful as Huginn.
- deleted 7y ago

PatrolX 7y ago

Not just an alternative though. - Self-hosted - Unlimited and FREE vs $699 / month - Create your own agents / integrations - Process data through shell scripts or JavaScript - Better filtering - Liquid templating - Completely private I save thousands of dollars by using Huginn. It's incredibly powerful and quite frankly I don't trust Zapier with my data and visibility of what I'm doing because there are commercial implications. Run it in the cloud on AWS or on an old box at home. It's very reliable.

PopeDotNinja 7y ago
> I save thousands of dollars by using Huginn. Does that take into account the overhead of operating Huginn?
- PatrolX 7y ago
  Yes, and it's running thousands of requests and posts / minute on AWS. But you don't even need to put it in the cloud, it runs perfectly at home on an old box running Ubuntu.
  - tectonic 7y ago
    That's awesome that it's working so well for you :)
- vorpalhex 7y ago
  You can deploy it as a docker image in a few minutes. If you already run a homelab of any sort (which most of HN does), it's basically free.
  - boring_twenties 7y ago
    > If you already run a homelab of any sort (which most of HN does) You sure about that? I feel like there are plenty of folks on here whose only computer is a MacBook

jaequery 7y ago

This looks worth checking out but I’d love to see a Lambda version of it

pixiemaster 7y ago
Yes, that would be the killer feature
- deleted 7y ago
  [deleted]
- mathnmusic 7y ago
  Related idea that was on HN a few days ago: https://gist.github.com/nileshtrivedi/6024434121b073789e6d580eb84c3ea2 https://gist.github.com/nileshtrivedi/6024434121b073789e6d58...
swyx 7y ago
what would be your preferred way of scheduling lambdas and doing storage?
- DailyHN 7y ago
  I use HTTP cloud functions (google/firebase version of aws lambda). Firebase database for long term persistence. Huginn for scheduling and event based triggers of those cloud functions.

minimaxir 7y ago

For a more business-friendly-but-still-FOSS approach to task automation between services, see also Apache Airflow: https://airflow.apache.org https://airflow.apache.org Airflow has been a skill that many companies ask for (especially data engineering), but surprisingly doesn't have many articles written about it.

codetrotter 7y ago
The Airflow landing page that you linked to lists many integrations but when you click on those only a small subset of them are listed in the integrations section of the docs that is linked to. I guess the docs are in need of some more work.
- minimaxir 7y ago
  It's easier to look at the APIs: https://airflow.apache.org/docs/stable/_api/airflow/operators/index.html https://airflow.apache.org/docs/stable/_api/airflow/operator... https://airflow.apache.org/docs/stable/_api/airflow/contrib/operators/index.html https://airflow.apache.org/docs/stable/_api/airflow/contrib/...

esquire_900 7y ago

This, as well as related projects like n8n & node-red, is a very cool project. I always wonder what people use it for in real life though. It seems a lot of trouble (setting up, learning curve, maintaining) for an action that usually takes a couple of seconds, like checking the weather or opening twitter. Does anybody have useful workflows going on?

mcbits 7y ago
Ever notice when someone criticizes a company on HN, and suddenly the CEO shows up and engages after 6 months of inactivity? Likely they are using a tool like this along with Google Alerts, etc, to monitor sites for mentions of the company and maybe flag negative sentiment. Also, same thing but for politics.
- janober 7y ago
  Yes, very interesting indeed! Either there is a big conspiracy to uncover or said CEO reads HN and if something is number 1 and is related to his product he pays close attention. We will maybe never find out the truth...
  - mcbits 7y ago
    From your defensive tone it sounds like you think I was talking about you, but I really wasn't. It's a phenomenon I've observed here and on Reddit and elsewhere for many years. And it seems like a perfect fit for a tool such as yours, is it not?
    - janober 7y ago
      Very sorry! As I was the only "CEO" in here which product got criticized I thought it must have been me ;-) Yes would actually be perfect for it, but never had the time to set something like that up. Maybe someday in the future. For now, I only have a simple Google Alert set and regularly check HN and Google Analytics which is a good indicator when it got mentioned somewhere.

howmayiannoyyou 7y ago

Zapier has exponentially more integrations than this or anything else, but is surprisingly difficult to use, and more so since they updated their UI. Editing Zaps is pure torture because refresh is so difficult. Exception handling is pure pain. In most instances the breadth of API calls is so narrow, and so rarely updated by vendors that you end up switching to a custom integration. I've also noticed vendors rapidly expanding their native integrations, sidestepping the need for a request broker. The "retail" integration space remains underserved and if one of the enterprise players decided to go down-market with a better UI and deeper integrations - they'd mop the floor clean in 18 months.

rogerkirkness 7y ago
We tend to agree re: retail integration, assuming you mean among retailers as opposed to message brokering.
toomuchtodo 7y ago
> The "retail" integration space remains underserved and if one of the enterprise players decided to go down-market with a better UI and deeper integrations - they'd mop the floor clean in 18 months. I think it's a tough market to be "winner" of. Novices are going to want a stupid simple GUI ("wizard mode", as someone else in thread mentioned). Power users are going to want to be able to toss in some code at some point in a workflow to do some fancy ETL you don't support out of the box. When you hit a certain level of complexity, an edge case or integration an automation product doesn't support, or perhaps even an amount of spend that you start looking at annually as painful, it's likely you consider pulling all of your workflows out and have a software engineer build something bespoke for your business line.
- wpietri 7y ago
  Yeah, I wonder if this space is sort of like the to-do/task-management space. It's conceptually one thing, but in practice people have such different needs that no one product can be everything to everybody.
PatrolX 7y ago

ikeboy 7y ago

I don't see a list of all supported agents or a way to search.

davidgf 7y ago
It's in the wiki https://github.com/huginn/huginn/wiki/Agent-Types-&-Descriptions https://github.com/huginn/huginn/wiki/Agent-Types-&-Descript...
dsander2 7y ago
Started the site a long time ago, but never finished. Nevertheless https://huginnio.herokuapp.com/ https://huginnio.herokuapp.com/ has a searchable Agent database an a few Scenarios.

soared 7y ago

I would not market this as an alternative to Zapier since it goes after a different market. The type of people who use Zapier will see a link to github.com and not even consider it. I think this should be positioned more like "A high powered Zapier for technical users" or something similar.

organsnyder 7y ago
"Self-hosted" is a growing buzzword these days (for many good reasons, IMHO).
deleted 7y ago
[deleted]

traspler 7y ago

I have been running huginn on my home server for a while. I've mainly used it to filter RSS feeds and then generate new feeds with the filtered items. Another use-case for me is ingesting webcomic RSS feeds (or scrape a page) and post the comics to a private Telegram channel. Once I also had an agent scraping a page and notifying me if something changed (realestate listing). I have tried a couple alternatives, e.g. node-red but none really worked the way I wanted them to for these cases. huginn is incredibly flexible and (at least for me) the mental model of it's workflow makes a lot of sense. Sadly more and more pages want you to go through their app/site and make it a bit difficult to work with, e.g. getting content from an instagram account. One thing I have not figured out about huginn and which all of these automation tools seem to lack are loops. E.g I have page an agent scrapes, from which I want to output the src of an image tag but I also want to check if a certain condition on the page matches (e.g. a "next page" button exists) and then firstly output the found src but then also re-invoke the agent with a new input element. So it would scrape the next page and so on until it does not find the button anymore.

PatrolX 7y ago
> One thing I have not figured out about huginn and which all of these automation tools seem to lack are loops. I do some very complex stuff using the "Shell Command Agent". You might want to look into using that if you haven't already. You can also create your own agent gem https://github.com/huginn/huginn_agent https://github.com/huginn/huginn_agent
robk 7y ago
Yeah I just use the Javascript agent for v complex stuff like this.
- traspler 7y ago
  Oh, how could I have missed the JS Agent... thanks a lot for the hint! But it looks like it's not node, so no npm modules and other node APIs.
  - tectonic 7y ago

caseysoftware 7y ago

It's not quite 1:1 but not a bad comparison. I like to think of Zapier as "wizard mode" - if you're doing something within their pattern, planned workflow, capabilities, etc it is a great option. If you need to do something more advanced or outside that narrow scope, you need to go to the APIs (or Huginn) itself. The important questions are: How many people need to get into advanced mode? How big of an effort/skill/understanding jump is it to go there? (aka will it hurt too much to switch?)

curo 7y ago

Something I'm not entirely clear on: One use case for Zapier (from a developer/company standpoint) is to allow customers to connect their existing services to actions inside your own app. For instance, if a customer updates a CRM record, you can have a custom zap update a record in your own SaaS platform. To pull that off with huginn, is that as simple as connecting this up to Singer.io? Or would that require a big marketplace of huginn agents for popular integrations?

tectonic 7y ago

It's fun to wake up to find a project that I started on the top of HN! These days, I'm no longer very involved with the day-to-day of the project. Now that it's no longer a young project, here are some musings about Huginn and responses to people's comments in this thread, in no particular order. I've found that Huginn excels as a scheduled web scraper with lightweight filtering. That's what I use it for. On the other hand, while you can write custom code in it, Huginn is pretty poor at implementing any sort of complex logic, and is even worse at bidirectional syncing between systems, which is something people often want it to do, but for which it wasn't designed. If IFTTT or Zapier meet your needs, awesome! No need to run and monitor your own service. I personally choose to run Huginn on my own hardware in large part so that I'm comfortable giving it website cookies and passwords. Some examples of what I use Huginn for these days: - Watching Twitter in realtime for high standard deviation spikes in certain keywords, such as "san francisco emergency" or "san francisco tsunami warning", which then sends me a push notification, or "huginn open source", that goes to a digest email (and I imagine will trigger because of this thread). - Watching Twitter for rare terms and sending me a digest of all tweets that match them. Also sending me all tweets from a few Twitter users who post rarely, but that I don't want to miss. - Scraping a number of rarely updated blogs that don't have email newsletters and emailing me when they change. Some use RSS, most are just simple HTML scraping. - Pulling deals from the frontpage and forums of slickdeals and craigslist and filtering them for certain keywords. - Sending an early morning email if it's going to rain today. - Watching ebay for some rare items. - Sending my wife and me an email on Saturday morning with local yardsales from craigslist. - Watching the HN and producthunt front pages for certain keywords. Basically, anytime I find myself checking a website more then a few times, I spend 20min making a Huginn Agent to do it for me. I think one reason Huginn has worked well for me is that I don't try to make it do too much. I use it for scraping data and gentle filtering, and that's about it. It's been super helpful for alerting me to interesting content for The Orbital Index, my current project, a weekly space-industry newsletter. (Last issue: http://orbitalindex.com/archive/2019-12-10-Issue-42/ http://orbitalindex.com/archive/2019-12-10-Issue-42/)

babyyoda 7y ago
Awesome project! Super impressive and will check out later

bitshift 7y ago

There is also ActiveWorkflow[1], which targets business users and let's you write custom agents in any programming language[2]. 1. https://github.com/automaticmode/active_workflow https://github.com/automaticmode/active_workflow 2. https://github.com/automaticmode/active_workflow/blob/master/docs/remote_agent_api.md https://github.com/automaticmode/active_workflow/blob/master...

gotts 7y ago
added to bookmarks. Thanks for sharing! It's always good to have alternatives
tectonic 7y ago
Huh, I did not know about this fork of Huginn.
- bpuvanathasan 7y ago
  https://tines.io https://tines.io (security automation platform) might also be based on (or inspired by) Huginn

j1elo 7y ago

I've seen the word "rake", so I guess... is this written in Ruby? If so, how's the performance? My home server is pretty minimal and lightweight: a Raspberry Pi. Do you think it will run it fine? (I'm gonna want to try this anyway, didn't know about it until now and it looks amazing!)

tectonic 7y ago
Performance is good, memory requirements could be lower though. It should run fine with 1GB of RAM, possibly less.
heavyset_go 7y ago
A fresh Huginn container idles at ~300MB of memory without any configuration on my part. A fresh node-red container idles at 30MB.
- floren 7y ago
  I was unable to run the Huginn Docker container on my Pi, I assume because there are x86 binaries in it. I'm trying the manual install now...
  - heavyset_go 7y ago
    Yeah, there's only a x86 Docker image on Docker Hub for Huginn. You could probably just build an ARM image from whatever Dockerfile the x86 version was built from.

binthere 7y ago

I'm probably missing something here, but what is the value of this when you could write a script that would do just about the same? (Serious question, I want to know the answer, not rhetorical)

bluehatbrit 7y ago
Huginn is more of a framework for writing and running these scripts. It has some built in modules to make the process easier. IE: The webhook agent is an easy want of getting POST data into a script without bootstrapping a webserver yourself, you can then just raise an event and have another script handle it. No need to write and manage everything individually, you can run everything in one place.
- DailyHN 7y ago
  Yes, and this becomes exceedingly important as the number and complexity of automations increases. Huginn can assist in organizing complex workflows into manageable components.

m-p-3 7y ago

Interesting, I should switch to that for some location-based action with my phone instead of IFTTT and see if I can make it work with Hauk or something similar. https://github.com/bilde2910/Hauk https://github.com/bilde2910/Hauk Disclaimer: I have no link with the Hauk project, just found about it by browsing the F-Droid store.

dang 7y ago

A thread from 2016: https://news.ycombinator.com/item?id=11015147 https://news.ycombinator.com/item?id=11015147 2014: https://news.ycombinator.com/item?id=7585605 https://news.ycombinator.com/item?id=7585605 2013: https://news.ycombinator.com/item?id=5377651 https://news.ycombinator.com/item?id=5377651

osprojects 7y ago

An alternative to webhook automation would be https://github.com/adnanh/webhook https://github.com/adnanh/webhook

ksrm 7y ago

Not to be confused with Hugin: http://hugin.sourceforge.net/ http://hugin.sourceforge.net/

KingFelix 7y ago
also not to be confused with Hugging -
ISL 7y ago
Agreed. It was a surprise to see Hugin get top billing on HN other than for awesome panoramic image stitching.
- ISL 7y ago
  I see that this particular horse was already beaten in 2013: https://news.ycombinator.com/item?id=5377815 https://news.ycombinator.com/item?id=5377815

moomin 7y ago

How does this not have a companion technology called Muninn?

kasbah 7y ago

Seems like all of these self-hosted automation apps struggle to incorporate Facebook adapters. Does anyone know of one that would allow me to read Tweets and post ones containing some particular text to a Facebook page (not my personal timeline/page but one I created for my project)?

anderspitman 7y ago

If you like this kind of stuff, you might find my patchbay project interesting: https://patchbay.pub/ https://patchbay.pub/

vorpalhex 7y ago

I've deployed Huginn a few times but never gotten off the ground with it. I found agents a bit too cumbersome to configure, but maybe I just need to give it another shot.

audiometry 7y ago

Is there any way to implement agents in python rather than JavaScript/ruby? Looks interesting but I don’t want to invest energy in building fluency in these other scripting languages.

DailyHN 7y ago
Not exactly. You could write python externally (cloud functions/aws lambda) and then use huginn to call that over HTTP. This doesn't require any coding. Huginn has existing agents that you can easily configure to make HTTP requests (PostAgent and WebsiteAgent). And, depending what you are planning to do with huginn, simply sending data from your python script to huginn using the WebhookAgent may be sufficient.
oskapt 7y ago
I run Huginn in a Kubernetes cluster with most of its heavy lifting done by HTTP calls to OpenFaaS running in the same cluster. This let me write complex logic in Python but have Huginn handle the execution, triggers, filtering, and delivery of the data. Among other things I use it to scrape Twitter for interesting Kubernetes news and follow the retweets back to people who link to their own websites so that our editorial department can vet them for invitation into our writing program. Any particular author/account only appears once. It generates around 40 new candidates each day. I also use it to scrape the Chilean public alert website (onemi.cl) for alerts in my region ( forest fire, major weather), run those through the translate API and then send the English translations to my and my wife’s phones via Pushover. These are things that I want to control and customize, and I vastly prefer running my own systems over trusting any third party.
j88439h84 7y ago
This comment links to a fork that allows other langauges https://news.ycombinator.com/item?id=21773803 https://news.ycombinator.com/item?id=21773803

DailyHN 7y ago

I've operated a huginn for 5+ years. AMA

LemonAndroid 7y ago
What did you have for breakfast?
- DailyHN 7y ago
  I don't eat breakfast.
  - swyx 7y ago
    What about second breakfast?
    - DailyHN 7y ago
      Sometimes a smoothie around 3pm that contains plant proteins, collagen protein, amazing grass, 1/4 cup of blueberries, ground chia seeds, and maca powder. I've been intermittent fasting + keto for the majority of the past five years. The minority is when we cycle off keto for a week here and there by doing intermittent fasting + vegan.
dsander2 7y ago
One of the current maintainers here, which installation method are you using and how is it working for you?
- DailyHN 7y ago
  @dsander I immediately recognized your username from GitHub Thanks for all the work you've put in! I run Huginn on Heroku. I haven't had any significant issues since it's been running. It's been so long since I've set it up that I would need to review my notes to tell you any more about how it's deployed.
7y ago

keyle 7y ago

This reads like DIY Yahoo Pipes.

vunie 7y ago

How well does it support: - support remote agents and remote reporting? - dead-man triggers for missing reports? - is remote reporting properly secured? (encrypted/signed)

DailyHN 7y ago
What do you mean remote agents? Can it be accessed over HTTP? Then yes. One of the great features of huginn is the diagram. That's where there are green and red indicators. Red indicates both errors and when a longer than expected period has passed. This makes quickly inspecting your workflows easy.
- vunie 7y ago
  > What do you mean remote agents? Sorry. I meant running an agent on a machine that is separate from the machine running huginn. The machine running the agent is not accessible publicly but has internet access (doesn't accept connections but can initiate them). huginn would be accessible publicly.
  - DailyHN 7y ago
    Ok, so in the remote agent: Retrieving data from huginn can be done with the DataOutputAgent. Sending data to huginn can be achieved through the WebhookAgent.

devm0de 7y ago

I gave this a go today and managed to install huginn on my synology nas by simply searching for the docker container. I then setup 3 agents to scrape a Shopify webstore jason endpoint that I’m always checking for inventory, have huginn parse the json and send me an sms via twilio if inventory changes. Took about 2 hours, wasn’t too bad. Huginn twilio docs seemed dated. Used python simplehttp server and ngrok to replicate a json url and play with the triggers to test it all before pointing it at a real website. Nice to add a new tool to the belt, thanks!

theshadowmonkey 7y ago

I’m glad Huginn is on the front page. It’s an awesome project being used for a long time now. I was testing Huginn to see how skimpy I can be and still run Huginn on a free tier. Was able to run it on open shift free tier a while ago when their allowance was generous. But, looks like it’s hard now. Will try running on a Gcp instance and see if it works.

e_carra 7y ago

It really sounds like Son of Anthon from the last season of Silicon Valley.

elwell 7y ago

> Create Amazon Mechanical Turk workflows as the inputs, or outputs, of agents (the Amazon Turk Agent is called the "HumanTaskAgent"). For example: "Once a day, ask 5 people for a funny cat photo; send the results to 5 more people to be rated; send the top-rated photo to 5 people for a funny caption; send to 5 final people to rate for funniest caption; finally, post the best captioned photo on my blog." Curious how this would turn out.

DailyHN 7y ago
You should try it out.

lngnmn1 7y ago

How come it is not NodeJS?

franga2000 7y ago

I've come across Huginn many times in the past and every time, I tell myself I'm going to give it a shot. I still haven't even seen it loaded in a web browser... After fighting the Docker images for half an hour, I usually realize that I could've just written a Python script to do whatever I was trying to do by then and proceed to do just that. Sorry for being lazy! Maybe I'll get around to it next year...