5 ms·
It's fun to wake up to find a project that I started on the top of HN! These days, I'm no longer very involved with the day-to-day of the project. Now that it'
by tectonic 7y ago
It's fun to wake up to find a project that I started on the top of HN! These days, I'm no longer very involved with the day-to-day of the project.
Now that it's no longer a young project, here are some musings about Huginn and responses to people's comments in this thread, in no particular order.
I've found that Huginn excels as a scheduled web scraper with lightweight filtering. That's what I use it for. On the other hand, while you can write custom code in it, Huginn is pretty poor at implementing any sort of complex logic, and is even worse at bidirectional syncing between systems, which is something people often want it to do, but for which it wasn't designed.
If IFTTT or Zapier meet your needs, awesome! No need to run and monitor your own service. I personally choose to run Huginn on my own hardware in large part so that I'm comfortable giving it website cookies and passwords.
Some examples of what I use Huginn for these days:
- Watching Twitter in realtime for high standard deviation spikes in certain keywords, such as "san francisco emergency" or "san francisco tsunami warning", which then sends me a push notification, or "huginn open source", that goes to a digest email (and I imagine will trigger because of this thread).
- Watching Twitter for rare terms and sending me a digest of all tweets that match them. Also sending me all tweets from a few Twitter users who post rarely, but that I don't want to miss.
- Scraping a number of rarely updated blogs that don't have email newsletters and emailing me when they change. Some use RSS, most are just simple HTML scraping.
- Pulling deals from the frontpage and forums of slickdeals and craigslist and filtering them for certain keywords.
- Sending an early morning email if it's going to rain today.
- Watching ebay for some rare items.
- Sending my wife and me an email on Saturday morning with local yardsales from craigslist.
- Watching the HN and producthunt front pages for certain keywords.
Basically, anytime I find myself checking a website more then a few times, I spend 20min making a Huginn Agent to do it for me.
I think one reason Huginn has worked well for me is that I don't try to make it do too much. I use it for scraping data and gentle filtering, and that's about it. It's been super helpful for alerting me to interesting content for The Orbital Index, my current project, a weekly space-industry newsletter. (Last issue: http://orbitalindex.com/archive/2019-12-10-Issue-42/ http://orbitalindex.com/archive/2019-12-10-Issue-42/)
- babyyoda 7y agoAwesome project! Super impressive and will check out later
- wpietri 7y agoI'm excited to check this out, but I wanted to congratulate you on a truly excellent project name. Having spent many, many hours in naming struggles, I truly appreciate the perfection. And for those unfamiliar, the (also amazingly named) historian Snori Sturluson explains: "Two ravens sit on his (Odin’s) shoulders and whisper all the news which they see and hear into his ear; they are called Huginn and Muninn. He sends them out in the morning to fly around the whole world, and by breakfast they are back again. Thus, he finds out many new things and this is why he is called ‘raven-god’ (hrafnaguð)." [1] [1] https://norse-mythology.org/gods-and-creatures/others/hugin-and-munin/ https://norse-mythology.org/gods-and-creatures/others/hugin-...
- tectonic 7y agoThanks, I'm glad someone appreciated it! :)
- aasasd 7y agoFittingly, Munin is a sorta-popular server monitoring tool: http://munin-monitoring.org http://munin-monitoring.org
- outworlder 7y agoI actually thought the thread was about that. The Muninn/Hugginn association is engraved in my brain. I blame EVE online.
- PatrolX 7y ago>I personally choose to run Huginn on my own hardware in large part so that I'm comfortable giving it website cookies and passwords. This, exactly and all of the above. It really is a fantastic project and kudos to you for starting it.
- carrozo 7y agoLove your newsletter! Brilliant writing and I think I click on more of its links than most of the many I receive.
- tectonic 7y agoI'm glad you're enjoying it! Writing it weekly has been a great experience, and I've learned a ton about space science & industry in the process.
- vincvinc 7y agoLittle known fact: this project is used widely by journalists who can't code (at The New York Times, among others) to do a variety of tasks, like eg. monitoring web pages like Trump's policy position, scraping press releases or filtering out very specific news alerts. see Huginn for Newsrooms: http://albertsun.github.io/huginn-newsroom-scenarios/ http://albertsun.github.io/huginn-newsroom-scenarios/ It's been at least as useful as Yahoo! Pipes, and endlessly more reliable. Thanks a lot!
- tectonic 7y agoYes! Albert's work using Huginn at the NYTimes (and contributing some changes back) was one of the big early successes for the project.
- tomcooks 7y agoNothing better than this kind of explanations to get convinced to try something, genuine passion seeps through your words. You might just have fixed a couple of problems with your tool, thanks!
- jimsug 7y agoFirstly, thanks! This has been immensely useful to me, and yes, my main uses have been primarily web scraping and then piping it into various channels. Been running it on a cheapish VM for a couple years, very reliable and lets you monitor things more frequently and reliably than services like IFTTT.
- thisisbrians 7y agoWhat funny timing! I'm a fellow space-nerd and just signed up for your newsletter a couple of weeks ago — it's fabulously done! Bravo.
- joegaebel 7y agoGreat to see you at the top of HN today tectonic! I'm at Pivotal in Sydney Australia now. Thanks again for the impromptu interview all of those years ago!
- Wistar 7y agoYou, Sir, are why I read HN. It's as simple as that.
- solstice 7y agoThank you for making Orbital Index! Super interesting and well written!
- johnx123-up 7y agoIf possible, can you please compare it against Flogo https://github.com/tibcosoftware/flogo https://github.com/tibcosoftware/flogo ?