17 ms·

Ask HN: What automation tools have you used to replace mundane activities?

• nikoksr 7y ago
  Greatly simplified, accelerated and automated the creation of project directories. I have always been annoyed by creating directories for new projects. Always the same procedure. Always the same commands. Always the same source files. So I wrote a small shell function which created C++ and Python projects for me. But after a few months I started to learn golang. And there I was again. Creating directories and files myself. But at this point the function could hardly be extended. So I started to transform my shell function into a powerful and expandable go application. Learning go by starting a new project and solving a personal problem at the same time? Perfect! Now a few months have passed and proji has become much more powerful and diverse. The templates it uses, which are called classes, are not bound to languages or anything like that, you can create a class for really everything. No matter how complex the class is, proji creates the project in seconds. Class configurations can be imported and exported, making it easy to share them with other users. With the latest version of proji there is a new feature that takes proji to the next level. Proji can copy the structures of repositories on Github and Gitlab and import them as a class which you can use locally to create your own projects. Additional features: Classes support shell scripts, template files to minimize the boilerplate code you have to write, ... [ 1 ]: https://github.com/nikoksr/proji https://github.com/nikoksr/proji
• polishdude20 7y ago
  I work at a local college and was recently tasked with collecting and aggregating instructor workload requests. We are in the planning stages of which courses each instructor will teach in the fall 2020 semester and my department wanted an easy way for instructors to submit their requests as well as be able to see what courses other teachers have requested and which are available. Manually, if have to receive each spreadsheet or email of choices from each instructor and then add it to an Excel spreadsheet of some sort. This wouldn't have been live at all. So I came up with a system. I made an excel proposal template each instructor fills out with their name, school ID and the courses they want to teach as well as the hours they'd have available for each course. They then send it to my email which has a rule on it to send excel attachments to a Gmail account I own. This is because I can't access the API of outlook so I need to get outlook to send this stuff to my Gmail account. My Gmail account has a watch on it so whenever an email comes in to it, I get a push request on my server. My server reads the contents of the excel file and sends it to a database. From the user side, if a teacher wants to access the full list of what courses each instructor has chosen, I've made a website in react with an excel like row and column layout where the columns are each teacher and the rows are the courses. Where the two meet we have the hours the teacher wants to teach for that course. When this page is visited, it pulls from my database and populates the whole thing with the latest data. The site can also export all the data to an Excel file to be implemented once all the teachers have made their choices. I learned a lot with this project. Learned react, setting up https, basic authentication, postgres, running a server and routing, and a while lotta other stuff. Super valuable to me in my learning even if I could have spent less time doing it manually!
• RMPR 7y ago
  This one is a shameless plug (I created the app) while it's still in early stages of development, the core features are working, I use https://github.com/rmpr/atbswp https://github.com/rmpr/atbswp to automate demos, whenever I have to present a demo in front of people, I just record it before and replay, but the app can have many use cases beside this. You're only limited by your imagination.
• staticautomatic 7y ago
  I started with selenium and still use it when I have to but try to just make http requests and parse responses directly whenever possible. Probably my favorite travel-related script was one that watched for an opening at a hard to book hotel and snatched it up when it became available.
  • beatthatflight 7y ago
    and to parse the responses (Assuming they're HTML page sources, not json/xml) - you using scrapy or something else?
    • staticautomatic 7y ago
      I pretty much always use lxml. It has an html method but I typically just use etree.parse(). It's very fast and has excellent XPath query support.
• omneity 7y ago
  I just commented on something similar: https://news.ycombinator.com/item?id=22345215 https://news.ycombinator.com/item?id=22345215 Disclaimer, it’s a project of mine, but if the websites you’re watching are compatible that will save you some hassle trying to extract data. Use a lot of google sheets as well, it’s super helpful to make sense and use of the data you scrape. You can check it out here [0], and feel free to ask if you need something. [0]: https://monitoro.xyz https://monitoro.xyz
• JshWright 7y ago
  I use Home Assistant to automate a lot of stuff around my house. The one I use at least daily is a trigger based on the light switch in my bedroom. A double click down triggers an automation that turns off all the lights in the rest of the house, makes sure the garage doors are closed, and that the rest of the doors are locked.
  • rayuela 7y ago
    Bad bot!
  • patrickk 7y ago
    One future Home Assistant project that I would love to implement is a whole house lighting system that gradually transitions all the lights from white/blue in the morning, to yellow/orange in the evening to aid natural melatonin production. Such a project would have to be fully automated, otherwise I'd forget to set it each day. Blue LEDs are terrible for sleep and throw off your body's natural sleep pattern. I got the idea from the excellent book "Why We Sleep"[1]. It's possible that a similar HA project already exists. [1] https://en.wikipedia.org/wiki/Why_We_Sleep https://en.wikipedia.org/wiki/Why_We_Sleep
    • bonestamp2 7y ago
      Look into philips hue. You can set the lights so they turn on at a different color temperature depending on time of day. Some of the remotes/switches even come pre-programmed to do that right out of the box.
      • patrickk 7y ago
        Good to know thanks.
    • huseyinkeles 7y ago
      I use Circadian Lighting [0] component for this one. Works pretty well all around the house. [0] - https://community.home-assistant.io/t/circadian-lighting-custom-component/61246 https://community.home-assistant.io/t/circadian-lighting-cus...
• s4ik4t 7y ago
  Better Selenium alternatives would be - puppeteer, cypress, testcafe etc. For Windows automation AutoIt is good. VBScript for MSOffice automation. Python/shell scripts for mostly everything.
• mentos 7y ago
  We use Jenkins to run .bat files that package our video game, upload to Steam and spool up Gamelift matchmaking servers on AWS every night. Before this I did it by hand, now I wonder how I could ever have operated with that inefficiency!
  • itsjloh 7y ago
    How do you find Gamelift? I don't know anything at all about the industry but I'm curious how it all hangs together!
  • ieRei6ae 7y ago
    Jenkins? .bat? That's some masochism.
    • smolder 7y ago
      Can you elaborate on how that's masochistic? The game likely needs to be built on Windows.
• steveeast112 7y ago
  I've used IFTTT a bit, along with Zapier. Biggest issue is trying to do multiple actions when you need to automate tasks for multiple rows of data at a time.
  • servercobra 7y ago
    Do you have some examples of what you use them for?
  • paulmendoza 7y ago
    Look at Integromat. I used to use Zapier but have moved mostly over to this instead.
• deleted 7y ago
  [deleted]
• lostintangent 7y ago
  Playwright is a new alternative to Selenium, and besides being cross-browser, is focused on performance, developer ergonomics and reliability (death to flakiness!). Might be worth checking out? https://github.com/microsoft/playwright https://github.com/microsoft/playwright. Disclaimer: I work with the Playwright team at Microsoft.
  • shostack 7y ago
    Can you attach it to an existing browser window/session that is already open? Ie. I navigate and authenticate to a site, run code, and it continues from there with what's on screen?
    • lostintangent 7y ago
      Playwright allows you to connect to existing browser instances, but it currently requires you to get the “debug URL” of the browser instance first: https://github.com/microsoft/playwright/blob/v0.11.1/docs/api.md#browsertypeconnectoptions https://github.com/microsoft/playwright/blob/v0.11.1/docs/ap.... I’d love to hear a bit more about the scenario you have in mind, since that might help inform some improvements we can make. Thanks!
      • shostack 7y ago
        The issue I'm having with Selenium (well, Watir with Ruby) is booting up some automated scraping logic for a page that may be in my current session once I've done the whole authentication dance manually (so I don't have to focus on automating that piece initially).
  • rumanator 7y ago
    Hi thanks for mentioning your project. Are there any plans to use it for automated UI testing for desktop apps? Selenium supports it (more or less) with WinAppDriver, which is another project from Microsoft. https://github.com/microsoft/WinAppDriver https://github.com/microsoft/WinAppDriver
• TrueNomad 7y ago
  Autohotkey on windows is my goto tool. I don;t know much about it but I can do the basics like move mouse to x.y coordinate and click it, or send a string of keystrokes to a form etc.
  • enjoyyourlife 7y ago
    It is much easier to do those actions with Python (https://automatetheboringstuff.com/2e/chapter20/ https://automatetheboringstuff.com/2e/chapter20/)
    • SPBS 7y ago
      I would say autohotkey is far superior to python on windows. The runtime comes with everything you need, no need to install/manage external dependencies. For example, I copy-pasted a 324 line snippet from the forums (https://autohotkey.com/board/topic/16400-ahk-regex-tester-v21/ https://autohotkey.com/board/topic/16400-ahk-regex-tester-v2...) into an .ahk file and I immediately get a full GUI to test my regexes on, 13 years after the snippet was posted. Additionally you can bundle the interpreter for your scripts into a standalone .exe that can run on anybody's windows machine without needing ahk installed. It's a one-click operation done in a program that was also written in autohotkey (https://github.com/AutoHotkey/Ahk2Exe https://github.com/AutoHotkey/Ahk2Exe). I can't imagine anything as resilient as AHK when it comes to scripting automation programs on windows.
  • mackrevinack 7y ago
    Autohotkey is such a beast. I really miss that now that I've started using Linux. i spent nearly 4 years writing scripts to automate all sorts of things in windows but to recreate them on linux might take me much longer since i would have to learn maybe 6 or 7 different programs just to get the same features. but alas Windows was doing my head in so I had to jump ship!
    • zo1 7y ago
      Autohotkey is also a tool I sorely miss since I moved to linux so many years ago. Sadly, there is no linux-equivalent and no one seems keen on porting AHK to linux.
• iKlsR 7y ago
  I have a slack workspace that has several bots, some listen for specific keywords and or phrases around the web and post to the relevant channels. Other bots remind me if I'm slacking off (maybe stayed a little too long on youtube), remind me of appointments, to drink water, if a stock price falls and some more. Also have filtered rss feeds for business and finance news etc. All of these bots also have custom identities that make it much more engaging like Axelrod is my bot for finance related news. Others tell me if my server is getting swamped, forward important emails like renewing a cert, a deploy failed, bug in prod or something else that requires often immediate attention. Chat widgets from sites in production also go directly to Slack so I can reply quickly if needed. (Gmail, Sentry, Freshchat, Ploi etc, integrate them all and filter out what's critical). Everything generally operates on a 15 min interval or so it's not noisy and I've pretty much tuned it to only receiving specific stuff I want to see. Having a central assistant like hub to go to is very useful and time saving.
  • muzani 7y ago
    Did you build these bots or got them from somewhere?
    • iKlsR 7y ago
      Some are are custom built from apis but I daresay you can find just about anything to hit a webhook or integrate into slack or elsewhere nowadays. Some examples are https://littlebirdie.io/ https://littlebirdie.io/ I use to filter hackernews, Zapier for RSS & filtering feeds, google calendar etc, nothing complicated, just know what you need from a specific service then figure out a way to get it to yourself in a recurring distraction free manner be it an existing addon or something you can roll yourself.
      • someguydave 7y ago
        Wait isn’t having a bot send you notifications all day the opposite of “distraction free”?
        • 7y ago
• lsiebert 7y ago
  I use tasker on android along with a plugin to automate turning down my screen brightness in the evenings, turning off screen rotation in a particular orientation for specific apps (IE I prefer to text in portrait mode since I can see what I'm replying to). I've semiautomated setting up a new laptop using a private repo with my dot files with some bash scripts. Started with linux, now works with Mac os x (but won't handle linux well and I haven't gone back enough to make it worth case by case stuff). I still have to generate a public private key pair, and some things on macs aren't perfect, but I have a todo list for the non automated things in the repo. I get going less on new macs nowdays since the keyboards are such crap (fingers crossed this year will bring better 13 inch keyboards), but everytime I do it I add something.
• dhruvkar 7y ago
  >>my goodness, it'd be slow and potentially flaky all via the web UI for those sites Generally, I've had better luck using undocumented APIs for this kind of stuff. I was a heavy user of Selenium to automate many a task (I work in wholesale construction goods, tons of automations needed everywhere). But then discovered that using internal APIs (which surprisingly don't change much) is far easier than trying to exception manage changing UIs. Open up dev tools on your browser and watch the GETS/POSTS as you complete your daily tedium. I use Python (usually just requests and beautifulsoup is enough) to mimic the calls and I have yet to find a use case where I wasn't able to automate something online. I'm currently developing an SDK for our 18th-century ERP system, which has no API. I've also automated getting shipping container tracking data from a various shipping lines and railroad company websites, many with complicated login processes. Happy to chat, email in profile.
  • tech_tuna 7y ago
    Ha ha, yep. I do this kind of thing all the time.
  • 2rsf 7y ago
    Good point, using the APIs directly is the also the corner stone of stable test automation
  • ctrl4 7y ago
    Yup. I did this with a local Android app, that tells you the gps location of a public bus. I sniffed the http packets, found the requests i needed and made a python script that pinged me at slack when my bus was 5 minutes from my bus stop. Useful when I worked at night.
    • spikej 7y ago
      I'm familiar with the web dev tools, but no clue about mobile. Can you point me in the right direction on how to do the same? Were you using an android emulator on desktop or there's some way to easily do this on android?
      • 7y ago
• Cyph0n 7y ago
  Ansible! I’ve been working on a “mail to cloud storage” project on the side for fun, but also as an excuse to learn Rust. As part of this effort, I decided to try out Ansible. Man, it is one great tool! For development, I have all of my server “groups” pointing to the same host, so my Ansible playbooks install and setup everything on a single machine. Makes testing much easier tbh.
  • cybrox 7y ago
    I'm currently migrating a system at work to using Ansible and I'm absolutely in love with it. My next project will be creating Ansible Playbooks for my workstation, so I can easily re-setup my Arch workstations on Desktop, Laptop, private and at work.
• defterGoose 7y ago
  I used to work at an architecture firm that specialized in design and manufacturing. One of our more popular products was facades made from aluminum panels with "graphic perforation", a pattern of holes made to form an image at a distance. As you can imagine, for larger projects the number of unique pieces to be cut was into the hundreds. This represented a lot of time for our CAM programmers because the (quite mature and industry standard) CAM software we used was incapable of determining the proper cut orientation without manual input. This meant clicking holes a bunch of times per panel to switch which side of the curve was cut. No Bueno. One of our primary CAD suites was Rhino which is very nice and has a great python API. So I wrote a full-fledged 2.5D CAM processor for that. This allowed us to batch process hundreds of these parts with a single click.
• simcop2387 7y ago
  I've started playing with some things with node-red + open hab to control my lights, but more in line with what you're talking about I've also played with Huginn for automating some small daily actions and webscraping. So far it's been going well but I'm not doing anything terribly difficult yet.
• h4waii 7y ago
  Take a look at Huginn [0], it's very powerful with the existing agents for taking input, parsing, and outputting data to different locations. I use it mainly for when I can't do things in IFTTT or have sensitive things I'd prefer to keep in a self-hosted system. 0. https://github.com/huginn/huginn https://github.com/huginn/huginn
  • pricci 7y ago
    Now that you mention IFTTT. I've recently visited their website and found out it has a web hook "if". An http endpoint where you can GET/POST data and trigger actions on it.
• arafsheikh 7y ago
  Last week I built an automated expense tracking dashboard on Tableau using SendGrid and Google Sheets. I use a Gmail filter to forward my credit card transaction emails to an Inbound Parser setup on SendGrid, which makes a POST call to my home server with the content of the email. A regex then captures the details from the POST body and pushes them to a Google Sheet. Next I use Tableau Public to read data from Google Sheets and publish a dashboard to Tableau Public Online. Tableau Public automatically refreshes data on the server side from Google Sheets every few hours. The good part is that all of this can run automatically without any manual intervention whatsoever and I get a beautiful near real-time dashboard for my daily expenses that I can access from anywhere (dashboard is publicly accessible but hidden).
  • jesterson 7y ago
    Are you comfortable sharing your personal expenses with so many public sources?
    • arafsheikh 7y ago
      Tableau Public is the one that makes me uncomfortable. If this dashboard continues to work well and be useful I'll look into self-hosted alternatives.
      • j45 7y ago
        Metabase may be of interest as an alternative
        • arafsheikh 7y ago
          Yeah, I've used it in the past. Problem is my home server is a tiny RPi Zero and Metabase is too heavy for it. I'll look into running it on AWS/DO if I absolutely have to.
          • j45 7y ago
            I’ve given in and bought an energy efficient homelab server for a few hundred off eBay. Intel makes the nuc, Lenovo has tiny thinkcentres, and HP has the elite desk among others. The strange thing is how many odd lxc container projects I’ve stood up just to play with. Proxmox is sweet.
• sufyanadam 7y ago
  I use a collection of tools from this repo to automate mundane developer activities such as posting release notes slack messages containing linked pivotal or Jira story titles: https://github.com/sufyanadam/pivotoolz https://github.com/sufyanadam/pivotoolz There's a bunch of stuff in there that saves you seconds each time you have to work with a user story or cut a branch or merge etc... Saved me a bunch of time over the years!
• peteretep 7y ago
  Unfucking the many many ways that banks screw up exporting financial transactions into well-formed OFX files that MoneyDance can consume
• pravins 7y ago
  Have you tried https://pipedream.com https://pipedream.com? You can run Node.js (with any npm package) on webhook/http requests, emails and schedules for free. It also manages auth (including oauth token generation and refresh) for popular apps like Google Sheets, Discord, Slack and Airtable -- just use the auths object to reference tokens and keys in code.
  • unixhero 7y ago
    Looks very promising! However the pricing is opaque. What will this cost me, if my business becomes successful and hit their free tier rate limits? https://docs.pipedream.com/pricing/ https://docs.pipedream.com/pricing/
    • dylburger 7y ago
      Hi, I'm a co-founder and engineer at Pipedream. Paid tiers are coming soon, and if you hit our daily compute limit before that, we're happy to raise your limit to make sure your workflows run without issue. Let me know if you have any other questions! Feel free to reach out to me directly at dylan [at] pipedream [dot] com.
• kd5bjo 7y ago
  Make is an incredibly powerful tool for managing any automated process, not just software compilation. It doesn’t know how to do any of the individual steps itself, of course, but it lets you break down a process into indivdual small pieces that can be automated separately.
• clement_b 7y ago
  Probably off initial topic, but: - FileJuggler on Windows, to automatically clean Desktop, Download folder and archive everything into a date-based folder structure in Drive. - AutoHotKey to insert text templates into everything (e.g. current date, formatted Jira issues, annoying things to type) The problem with automation is that one will want to break even quickly [time saved > (time spent finding/setting/maintaining)] otherwise automation is just a glorified delayed action. Time saving is not the only reason for automating but I assume it's n°1's when it comes to personal use.
  • TeMPOraL 7y ago
    Perhaps underappreciated aspect of automation is friction reduction. Having a semi-automated process can be what it takes for you to stop avoiding doing some little chore, or what makes you no longer suffer as you do it.
    • ehnto 7y ago
      I am really susceptible to friction. It's why my bike is at my front door, and why I don't stack things. Adam Savage has a principle he called First Order Retrievability that helps reduce friction for using tools around his workshop. This thread made me think about automating my invoicing. If all it did was mail me the completed PDF to then forward to clients, that would definitely make me do it more consistently on the 1st of the month even though it's not completely automated. https://hackaday.com/2015/02/28/adam-savages-first-order-of-retrievability-tool-boxes/ https://hackaday.com/2015/02/28/adam-savages-first-order-of-...
  • patrickk 7y ago
    Some more items that I've setup in AHK: - Launching commonly used apps, like Notepad, Windows Explorer or Sublime Text - Launching commonly used websites in a browser, e.g. gmail - Paste from the clipboard into a language translation site (languageX to languageY via URL parameters) - Open a to do list in Excel (hard coded file path) - Text expansion for personal and work email addresses - With one keypress, open the train schedule on the website, type in the 'from' and 'to' stations, search - Text expand to symbols that I always forget, eg. euro symbol, hash sign etc - Google/Wikipedia search clipboard text
• ankit_it09 7y ago
  I used google scripts and google forms for some of my automation like: 1 - Notify me when Movie tickets are available to book. Used google scripts that run every minute and hit a URL to check if tickets are available now. 2 - Book a class when it's available, I have joined a Gym where you have to book a class before going, and the Zumba is very famous there the time it opens it fills up in 30seconds and there are limited seats, so if you are late it's gone. I wrote a script that checks and books the slot for me and then adds the same in my calendar. I extended the same using google forms when I want to book another class at some specific time which is currently not available if somebody cancels it will be available. The script keeps on checking until class start time, if available books and notify me. 3 - In my team, everyone plays Foosball and after lunch, it's everyday discussion who will play first and with whom. I wrote a script which will decide the matches and players in each team. I used the google app engine to deploy it, which is still running and just by hitting the API will sort the things for us. There are a few other automation I did using IFTTT.
  • netsharc 7y ago
    I wonder who else is refreshing the class registration page and filling it up within 30 minutes? Or do the special people get advance notice that registration is about to open? I remember reading a wild tale of how someone made a program for a used car salesman that would check a particular car auction site to make sure they get a good deal (I wish I could remember the details, or better yet, find the URL)
    • ankit_it09 7y ago
      @netsharc, the class booking opens every day at 10:00PM and everybody knows that. :)
    • Throwawayxx001 7y ago
      You're looking for "Defcon 21 - How my Botnet Purchased Millions of Dollars in Cars and Defeated the Russian Hackers" https://www.youtube.com/watch?v=sgz5dutPF8M https://www.youtube.com/watch?v=sgz5dutPF8M
• ksahin 7y ago
  I have many cron jobs in AWS Lambda functions for many tasks. I wrote a blog post about how to monitor a competitor website using Python / Lambda and serverless framework for deployment: https://dzone.com/articles/monitor-your-competitors-with-aws-lambda-and-pytho https://dzone.com/articles/monitor-your-competitors-with-aws... For lots of workflows we also use No-code/Low-code tools like Zapier / Integromat.
  • slightwinder 7y ago
    How much does that cost?
• mast4461 7y ago
  I jut begun to do some partial automation of website interaction with the extension Tampermonkey for Google Chrome to inject my own scripts on certain websites, often creating an additional control panel UI for custom actions such as scraping or data input. I use Tampermonkey in such a way that the code managed by Tampermonkey are only loading scripts that load my actual scripts from GitHub and other sources, such that I always get the latest published versions of my scripts.
  • terminaljunkid 7y ago
    would be interested to see why you use tampermonkey instead of GreaseMonkey. I tried both but stuck to GreaseMonkey as it was open source.
    • fbcpck 7y ago
      Not OP, but I decided to switch to Tampermonkey after GM version 4 which broke almost all my scripts due to the GM_## api changes. I no longer remember the exact details but it was poorly executed/communicated, and it’s easier to switch to TM instead of updating all the scripts. The dashboard interface in TM is a lot nicer than GM too
      • aepiepaey 7y ago
        When Firefox stopped supporting legacy extensions, the Greasmonkey developers saw that as a chance to redesign their API (to use promises). Tampermonkey used to be open source, but unfortunately isn't any more. I still use it though, as it has better UX (e.g. nicer dashboard, prefills @match when creating new scripts, better editor, etc). An alternative to Tampermonkey that is open source and still uses the old style of user scripts is Violentmonkey (which also lacks in the UX department, if i recall correctly).
• batirch 7y ago
  I use telegram, ifttt, Google sheets to calculate expenses. At my last job, I used Zapier & Zoho CRM to automate creating leads from email requests which saved me hours of time.
  • wastedhours 7y ago
    Was at a startup that was super low-overhead (and as dysfunctional as a 4 person team could be), we used Zapier to automatically create Gmail drafts as replies to new lead enquiries on the website and follow-up responses as needed. We could then spice up the response if needed, or just bulk run through and hit "Send". Appears personal to all email systems and spam filters, but actually was semi-automatic marketing automation.
• dankohn1 7y ago
  If you travel regularly, Tripit is a great service for organizing your flights and hotels on to your calendar (Google Calendar also does this with Gmail but it doesn’t work reliably, especially if you book some of your travel via your personal email account). You can set up a free Tripit account and either forward your receipts to it or let it look into your email for them automatically. Add the calendar of your Tripit trips to your work Google Calendar. These now show up in as a second calendar but are not visible to your co-workers. Create a free Zapier account and use a Zap to automatically copy all events from your Tripit calendar to your work calendar, which your colleagues can see. If you want to be shown as available while traveling, you need to change the multi-day event that Tripit creates from busy to available. And if you change your flights, you need to manually delete the old ones. But, people will no longer book you into meetings when you're on a plane. Zap: https://zapier.com/apps/google-calendar/integrations/google-calendar/10039/copy-new-google-calendar-events https://zapier.com/apps/google-calendar/integrations/google-...
• theshadowmonkey 7y ago
  What about headless chrome and some cron jobs to run tasks for you. It'd be similar to selenium. But, on a headless browser. There are also some scraping services that let you get data on a set interval with a phantomjs scraper. I have one that uses apify and then notifies a zapier hook on success and then zapier posts to my google sheet. My webapp then calls a GCP instance REST API endpoint which internally calls the google spreadsheet through an exposed api, cleans the data and sends back to the webapp.
• martin_a 7y ago
  I run Home Assistant with Zigbee devices for automatic lighting at home: - switch on when I get home and sun is set - switch on when I'm home and the sun sets - switch on in the morning, full brightness, coldest color to help waking up (really helps me a lot and is not as stressful as an alarm clock) - automatically turn light off when I (my smartphone) leave home (not visible in the WLAN for 10 minutes) Will look into more automations there. Perhaps something about turning the heat up in the bathroom in the morning when the windows are closed (there are sensors for that). I ran NodeRed on a Raspberry Pi too for some time, but I found that helped me only little. Mainly did logfile analysis through that but did not look into other use cases for me. For work I've got the de-facto standard for the printing industry, I'm doing everything possible with that.
  • mnbbrown 7y ago
    Interested why you use the coldest colour for waking up? I do the same, but use a sunrise simulation instead that goes from warm to cold.
    • martin_a 7y ago
      Hm, maybe it's self inflicted torture. I just feel uncomfortable with the blueish light and will get up quickly to leave that behind me. Maybe I will give your sunrise simulation a try, might work just as well, not sure.
    • sheepybloke 7y ago
      Cold light helps you wake up in the morning, since it's more stimulating. That being said, it shouldn't be harsh like midday light; closer to an in between. I've heard it described as more of a green light than blue.
• jaggs 7y ago
  A bit more basic, but I use two free desktop PC automation tools which I now can't do without. 1. TyperTask (https://typertask.en.uptodown.com/windows https://typertask.en.uptodown.com/windows) to automate common tasks like adding different email signatures for different addresses etc etc. 2. ClipX (http://bluemars.org/clipx/ http://bluemars.org/clipx/) for managing a huge clipboard history file. This saves me SO much time it's unbelievable.
  • johnflan 7y ago
    It surprises me how many people don't use a clipboard manager. I use PasteBot on the Mac.
    • Nannooskeeska 7y ago
      I've had Ditto [0] on my PC for almost a year now, and I could probably count on two hands the number of times it's come in handy for me. Can you explain what is so great about having a clipboard manager? I feel like I'm missing something, and if it would become truly useful to me by using it a different way I'd like to learn. [0] https://ditto-cp.sourceforge.io/ https://ditto-cp.sourceforge.io/
      • gcarvalho 7y ago
        It depends on your workflow. I sometimes find myself alt-tabbing multiple times to copy stuff over. Othertimes I accidentally overwrite my clipboard and lose some text that I was writing. Being able to recover it from the history is nice. I use Pasteapp on macOS and FastKeys on Windows, which comes with other useful stuff, such as a shortcut for controlling volume by using the mouse scroll wheel on the right edge of the screen, macros, and a text-expander. The ones I use the most are: ddate = 2020-02-18 ttime = 03:40:26 nnow = 2020-02-18 03:40:28 @@ = my email
• htk 7y ago
  I manage a Family Office and do monthly performance presentations with PowerPoint, the slides have an unique design, and updating them every month became extremely tedious and error prone. I was already looking for an excuse to learn Python and this was perfect. Instead of recreating the slides I created templates (just the last set of presentations) and update them with info directly queried from the database, using python-pptx and pyodbc. This isn’t exactly an example of a “tool”, but I think the most important part isn’t really the tool, but to be able to identify what you can automate. How you do it might range from DIY to paying someone to code for you, but the feeling after getting rid of manual processes is great!
  • cybrox 7y ago
    From the past 10 years of side projects, I can definitely confirm that the most important skill one could have is recognizing what can be automated, and especially, what is worth the effort of automating. - Your example is definitely a good one!
  • simbas 7y ago
    Check out revealjs.com
    • excitednumber 7y ago
      Reveal is good but very hard to recreate ppt slides.
  • vitro 7y ago
    Yes, Xkcd has nice strip for this called "Is it worth the time?" https://xkcd.com/1205/ https://xkcd.com/1205/
  • excitednumber 7y ago
    Nice! I work at a quant fund and we Python-pptx, too.
  • tohs19 7y ago
    Also at an FO here, this is great. We started templating everything last year and it’s been a huge relief. Project management in Asana we also turned into templates, things are simple but flow so much easier.
• benibela 7y ago
  I spend a lot of time with web automation Especially for public library catalogues. There the webpage shows when the borrowed books are due, and you need to check it every week, or pay late fees. And you cannot set a bookmark, since you need to login first, which is tedious. So I automated it to renew all books and show me a warning. And because that needs to run silently everyday, I decided to make it as fast as possible (and because I only had a single core cpu with less than 1gb ram). No browser, no javascript, no selenium, no python, no java. Only viable languages would be C or Pascal, and for memory safety I implemented everything in Pascal. Because it is also tedious to build HTTP requests in native code and recompile it all the time, I then wrote a scripting language for it, all the webscraping is done in the script, but it is still fast because the processing is done in native code. To read data from the HTML, I use pattern matching, e.g. <a href="{.}"/>+ would read all links, and <a href="{.}">foo</a>+ would read all links with text foo. Nowadays I do not even visit public libraries anymore, but the automation tool has become an open-source project at http://www.videlibri.de/xidel.html http://www.videlibri.de/xidel.html . I also started playing some browser games, but they are boring, so I stopped. But I want to stay in the highscore list, so I wrotes bot for them. Because they have anti cheat detections, you cannot send the http requests directly. For javascript based games, I wrote a bot with Greasemonkey. For flash games that does not work, so I have decompiled the flash file, added the automation functions to the flash, and then recompiled it.
• nickjj 7y ago
  Too many to list here, but one of my most useful ones is a Python script to help pay out affiliates. It crawls through monthly CSV files provided by my payment gateways, finds sales and returns by a specific affiliate, calculates what I owe them and returns a dollar amount, a sales break down for each course, how to send it (paypal, wire, etc.) and who to send it to. It's nice because it takes around 2 minutes to payout everyone each month with a very high confidence level in the results. Prior to the script, it was a stressful work flow with many chances of human error. I only did it once before automating it. Another script I use helps me invoice my freelance clients each month. It's a Bash script and is open source https://github.com/nickjj/invoice https://github.com/nickjj/invoice. It saves me a ton of time every month since I can calculate how much I need to invoice folks in a few seconds with it.
• Ayesh 7y ago
  I build a web site https://visalogy.com https://visalogy.com with help of a keyboard/mouse automation tool. It was too simple to just automate and control my laptop with scrolling to certain web pages, xpath the elements, and copy-pasta data to a CSV file. Nowadays, I use bash scripts to automate some scraping. Cron scripts in Travis CI are amazing, this https://visalogy.com https://visalogy.com web site practically runs by itself thanks to them.
  • bhy 7y ago
    My kid just drop by when I was looking at your site and pointed out the flag you have for Sri Lanka seems a bit wrong - the lion in the flag is missing. :) Anyway, great work to build such a useful site!
• buboard 7y ago
  auto delete certain messages from gmail after 30 days . i use filters to tag messages as "autodelete" and create a google script that runs periodically to delete them . It shouldn't be so hard to do something so simple.
  • Klathmon 7y ago
    I'm fairly certain this is how the "trash" in gmail works by default. Anything that gets sent to there stays there for 30 days then is automatically deleted.
    • buboard 7y ago
      i mean, for my own reminders and messages that i forget to delete.
• Havoc 7y ago
  Maybe check out huggin
• rasikjain 7y ago
  Written couple of scripts related to storage space in Shared Drive Folder and Database storage. If the storage goes below certain threshold, then the notification is sent to the group email. Using the google rss notification (e-mail) for keywords which I am interested in (e.g my name, stock ticker and company name etc). I use www.hnreplies.com for alerts when someone responds to my HN comments.
• msadowski 7y ago
  I run a robotics themed weekly newsletter as a side project. At one point I decided that it would be nice to add header images to every issue. I decided to do it through a quick and dirty python script that adds the text for me. Another thing I've just recently automated is the python script calling mailchimp API to show me the most popular link together with unique opens. I'm currently planning to embedd this info in every issue. The next step that I'd like to automate is the campaign creation. With the API I don't suppose this will be a huge deal, just need to put some time on this. When I have it then I'd like to create a commit hook that will do all of these things automatically after pushing changes to the repo.
• paultopia 7y ago
  slightly mundane, but I'm an academic and my cv needs updating a lot. I finally got latex to work on netlify, and now all I have to do is update a yaml and push a commit and my cv is updated.* * At least on one of the three locations my cv lives... the next step is to point all the other domains at my new CI/CD home...
• cpclermont 7y ago
  I have 141 custom scripts in my $HOME/bin. Most of it is in bash, but there's a couple of node.js scripts as well. - I mostly automate my bookkeeping with a set of recurring & dependent taskwarrior tasks and scripts as annotations that I run with taskopen[1]. That's creating a bunch of folders, turning some emails in mutt into PDFs, gathering PDFs from emails, fetching bills with selenium, moving files from $DOWNLOADS into the appropriate bookeeping folder, putting a date on some files, turning the whole thing into a zip file, and sending it to the bookkeeper with mailx. - I automated the email send of my daily summary to my clients with mailx (so I can send it directly from vim) - I automated turning screen recordings into thumbnails+mp4 link (since GitHub only supports gifs) - I automated making before/after screen recordings for when I do noticeable performance improvements (page load/animations) - I automated booting/killing my development servers - I automated making PRs with `hub pr` (finding the origin/upstream, putting labels, etc.) - I bound to a key combo switching to the logs of specific development servers - I turned my client's time tracking (tempo) into a CLI because I got tired of using the UI to say I worked X hours on that ticket and 7.5 - X on the other. Now I only do `tempo log $ticket1 2h $ticket2 3h $supportTicket rest` [1]: https://github.com/ValiValpas/taskopen https://github.com/ValiValpas/taskopen
  • YourMatt 7y ago
    How do you like using Node for CLI? I started playing around with it a couple years ago, and at this point, I don't even consider BASH scripts if they involve the slightest bit of logic. Part of me feels like by going all-in, I'm going to lose touch with my ability to write direct BASH scripts, which are likely to work literally forever. Meanwhile, I expect my Node scripts to have a relatively short lifespan, particularly in that I'm leaning on NPM. It's been so nice though, I've decided it was a worthy tradeoff.
    • cpclermont 7y ago
      I've been doing pretty much the same thing. If the logic gets crazy, of if I want lots of subcommands, I'd write it with node & commander. There's only so much you can do with `jq` when it comes to processing JSON.
  • 7y ago
• faeyanpiraat 7y ago
  Be careful, my antivirus blocked the site when I visited the "Privacy" page from the top menu. https://www.virusradar.com/en/HTML_ScrInject.B/description https://www.virusradar.com/en/HTML_ScrInject.B/description
• Oldham-Made 7y ago
  I threw together a quick python script which scrapes listings from cinemas local to me, along with each movie's IMDB score, and compiles the results into a weekly Friday-morning email which I send to myself and my SO. It's nice to know what movies are out, and when we can book, without having to remember to check or wait for pushy emails from the cinemas themselves. I'm planning on doing the same to email us the latest menus from our favourite restaurants/cocktail bars so we can see when they change.
  • gurjeet 7y ago
    Would love to see this turned into a website that does recommendations based on zip code.
  • dueyfinster 7y ago
    I've done various versions of this myself, would you mind sharing your code?
• rolandog 7y ago
  'make all' the things with (GNU)Makefile! From pandoc pre-processing with a great tool called 'pp', to generating my CV in different languages and paper sizes, to ensuring my repositories have the latest .gitignore files. I feel it requires a significant effort to setup, but once you're there, it eases your development flow and enhances reproducible outcomes, in my opinion.
  • vorpalhex 7y ago
    I use makefile to front each project in a consistent way (whether it's my blog, a golang project or a hardware flashing setup, `make start` will probably work..)
• RocketSyntax 7y ago
  https://smmry.com/about https://smmry.com/about Summarizing long text-based articles! Found this on HN. Now when my family/ friends send me stuff they haven't actually read ... at least I can reply somewhat intelligently.
  • PenguinCoder 7y ago
    I'd still love to be able to self-host this.
    • ajot 7y ago
      It's development has stalled long ago, but you could try OpenTextSummarizer [0]. It is available on the Debian repos [1] and you can try a web interface [2] to see if it fulfills your expectations. [0] https://github.com/neopunisher/Open-Text-Summarizer/ https://github.com/neopunisher/Open-Text-Summarizer/ [1] https://packages.debian.org/search?keywords=libots https://packages.debian.org/search?keywords=libots [2] https://www.splitbrain.org/services/ots https://www.splitbrain.org/services/ots
  • hassancf 7y ago
    This is fantastic! Just tried it with a few blog posts. Thank you.
    • RocketSyntax 7y ago
      10% of the time I have to copy paste the text
  • antisthenes 7y ago
    Finally, a sane way to read New Yorker articles!
• caduar 7y ago
  Hey, I recommend you RPA (Robotics Process Automation), there some open source option such as Open RPA or community options as UiPath. Actually I work as an RPA developer and you can automate whatever you want. For example read an email, capture that information in an Excel and fill web forms based on the excel.
• loufe 7y ago
  In the engineering office (non-software) I work in, I've done wonders with classic scripting for me and my colleagues. Some Excel VBA, lots of Powershell, AutoLisp, etc. Doesn't need to be shiny to be effective.
• buzzy_hacker 7y ago
  I use YNAB (https://youneedabudget.com/ https://youneedabudget.com/), which requires inputting every one of your transactions by hand or importing them from your bank. The downside of the import is it takes a few days for the transaction to show up and you need to give YNAB your bank username/password, which I'm not comfortable with. I wrote this https://github.com/buzzlawless/ynab-live-import https://github.com/buzzlawless/ynab-live-import to import credit card transactions instantly with no need of giving up my bank credentials. The whole stack runs on Amazon Web Services. Simple Email Service receives a notification email from the bank that I've made a purchase, saves it to S3, and triggers a particular lambda function tailored to whichever bank the notification came from. The lambda function retrieves the email from S3, parses it for transaction data (account, payee, amount, date), and writes that data to a DynamoDB table. The table has a stream enabled, which triggers another lambda function when the table is updated. The function reads the transaction data from the stream and posts the transaction to YNAB using their API. I've mentioned this on HN once or twice before and got some positive interest with people even submitting pull requests, which is awesome :) Going to find the time soon to review those and maybe add more features
  • aaron_m04 7y ago
    > you need to give YNAB your bank username/password Why don't they have oauth for this?
  • lignux 7y ago
    What do you do about cash transactions?
    • ralmeida 7y ago
      Not OP, but there's not much to do about them - best way is to enter manually in the app at the time of purchase or write them down in a notebook and enter them in batch once in a couple days or per week. If your cash spending is sufficiently small that you don't care much to tracking each cash transaction individually, you can just track money as a category instead of an account - just record an outflow transaction in your bank account when you withdraw.
• dana321 7y ago
  You mention selenium, i use Selenium IDE Its a browser plugin that allows you to record and playback anything that you do in your browser. https://selenium.dev/selenium-ide/ https://selenium.dev/selenium-ide/ There is a right-click action to record hovering over a section if it is hidden by css, its a well-known gotcha in wordpress admin listings and other sites that love to hide until you hover over a section. I've still to look into whether the file format that is saved from selenium ide could be used within a virtual browser like phantomjs or the like. For direct scraping, i tend to go straight to xpath with dom. Its far more reliable than trying to get stuff out via regular expressions, that i only use when i need to get a certain part of the node value from a text node. I used to use php for that, but i wrote my own programming language that uses a library called htmlquery (its a go library) https://github.com/antchfx/htmlquery https://github.com/antchfx/htmlquery If i have to login, i fake each request with the correct headers. Plenty of libraries around that help with that for every language.
• lyime 7y ago
  Automated investing and wealth reporting. I use a few services to automatically invest money into a few different financial vehicles at a regular basis * Vanguard finds (bi-weekly transfer into various ETFS) * Titan Invest (robo advisor bi weekly) * IRA and 529 (monthly) I use personal capital to connect see view my cash flow and returns across all my accounts. I have been doing this for the past 3 years and my returns have increased significantly. I am 34 and I wish I had started doing this when I was younger.
  • shostack 7y ago
    Have you figured out a way to automate exporting of data from Personal Capital into somewhere else (spreadsheet, DB, etc)?
    • lyime 7y ago
      That’s a great question. I haven’t tried it but seems like you can access their private API. https://github.com/haochi/personalcapital https://github.com/haochi/personalcapital
      • shostack 7y ago
        I've been building something that looks very similar to this for Mint in Ruby but am still implementing exception handling for 2FA. I'll have to check it out--thanks!
• boomeranked 7y ago
  Airbnb experiences will not add an API, so if you run any kind of tours, you have to go the App to add/remove/change dates/prices/starting points. Selenium takes care of it now.
  • spectramax 7y ago
    Everytime I use Selenium, even for legitimate use cases, I feel dirty deep down under. My soul writhens and I want to do something else in life other than writing Selenium automation - perhaps chop some wood.
• ghaff 7y ago
  In a somewhat similar vein to the OP, some of the details associated with publishing a new podcast can be rather manual and error-prone. This is especially annoying in the context of simple interview/discussion podcasts that don't really take all that much time to edit/post. A fairly simple Python script can automate the fiddly things like creating XML for a new podcast and uploading things to wherever you're hosting. (I used to use a script to add intro and outro audio too but I now do that as part of the audio leveling process with Auphonic.)
  • xcubic 7y ago
    It looks like you could build a small SaaS around this...
• derefr 7y ago
  The script I get the most personal use of is for one of the most mundane things you can imagine—automating the repetitive process of keeping everything on my computer up to date. By "everything", I mean: • my OS packages • my OS-overlay packages (e.g. Homebrew, Nix) • the versions of any standalone SDKs I use, that aren't maintained as OS packages (e.g. Rust from rustup, the Google Cloud SDK) • any globally-installed packages for the package ecosystem of each language I install such stuff from (not libraries, but standalone utilities, e.g. youtube-dl in pip, or Foreman in Rubygems) Here's the script: https://gist.github.com/tsutsu/270e09c68690ec85c51dbd054e22b9ef https://gist.github.com/tsutsu/270e09c68690ec85c51dbd054e22b... I think automating this might help a lot of people, because people tend to forget that they even can update a lot of this stuff, or they don't know there's a command to get certain things to happen at all. (E.g. did you know that `brew cask upgrade` will switch out your application-bundles installed via casks with newer versions?) I never polished it up, though, because it's still frail in some ways. (I still don't exactly trust the way it does pip package updates, for example; sometimes pip decides to upgrade packages into an inconsistent state where a package's deps get too new for it to use.) But the idea is that you run this interactively, first thing in the morning when you sit down, in its own fresh terminal window, right after maybe letting your computer restart to update to a new OS version. It's like putting your workspace in order in the morning. It's doing the grunt work, but you're still making the decisions and fixing the exceptional conditions. ----- I feel like this could be polished into a universal tool that'd literally update everything for everybody with one well-known command. But, better yet, the problem could be reversed, and a standard for registering installed package ecosystems and their respective update/clean/etc. commands could be created, that installed ecosystems could register with by placing files in a directory (like e.g. pkg-config or bash-completion) such that this command could outsource its smarts to the ecosystem creators themselves.
• excitednumber 7y ago
  Does outsourcing count? I have found a few freelance folks in non-US countries who work for very competitive wages and carry out relatively sophisticated work flows. These are tasks that I took a few good shots at automating but due to challenges (handwriting inconsistency, rotated forms, etc) I kind of gave up.
• zze 7y ago
  * Home Automation, like opening shutters at sunrise and closing them at sunset or turning on LED lights at night when movement is detected and turning them off after a certain amount of time since the last movement. Z-wave relay modules control most devices, and the command-center is Raspberry Pi + Aeon Z-Stick (aeotec.com/z-wave-usb-stick/), which allows for a vast amount of flexibility compared to standard Z-Wave controllers. * an extensive collection of custom made /usr/local/bin scripts which automate things such as video recording and encoding for youtube (recordmydesktop + avconv), backups, and certain time-taking and repetitive operations on websites (like downloading invoices; through selenium-webdriver - github.com/SeleniumHQ/selenium/wiki/Ruby-Bindings) * zaps on zapier.com for pushing data between multiple cloud services that would otherwise have to be moved by hand (i.e., Gmail email with matching subject to a task in kanbantool.com) * autokey on Ubuntu, which allows me to type phrases like "!thx" and have them automatically expanded to i.e., "Thank you for your email!"
• roarkry 7y ago
  I manage two calendars for work and often have to make sure my calendar events exist on both so I don't get double booked. I wrote https://syncmycalendars.com/ https://syncmycalendars.com/ to automate and simplify that process for myself and others.
• jsonbourne 7y ago
  Usually, my download folder is quite messy. I wrote a script to re-organize any such folder using a simple CLI. It also has a config that can be updated to account for different file types or categories. https://github.com/functioncall/neat https://github.com/functioncall/neat