25 ms·

I work in this field so I'm incredibly biased: automated business solutions that cut entry-level data employees out of the equation. You save TONS on the bottom

I work in this field so I'm incredibly biased: automated business solutions that cut entry-level data employees out of the equation. You save TONS on the bottom line, and cut out human-driven process that is error prone and difficult to manage. I'm talking about things beyond "API-driven dev", more in the realms of Puppeteer, Microsoft Office automation, screen-scraping (mouse/keyboard), etc. I make API's out of things that other devs balk at - and trust me, it has a lot of market value. This isn't as "up and coming" as all of the other items people are mentioning, but I'd put it on a "always increasing in popularity" trajectory due to an ever-increasing need. It's not really sexy or interesting, but there will always be a HUGE market for the things that I can do =) I will warn people that "up and coming" tech is often fad-based and has boom and bust cycles, and personally I'd rather be working for a paycheck then waiting to win the lottery in this regard.

maxhallinan 7y ago
How do you find processes that can be automated? I've often thought that there must be a ton of this stuff in various industries where programmers aren't typically embedded.
- HeyLaughingBoy 7y ago
  You really need to find a domain expert who can at least get you started. I'm working on a problem that I would not have even known existed, if not for running across a couple of engineers who had a business in the field and saw an opening that only a small company would care about (market isn't big enough for the large players). I know similar situations have to be all around us. The problem, as you say, is finding out about them.
- avgDev 7y ago
  Any midsize company has tremendous amount of things that could be automated. For example, I work at manufacturing company, we produce parts for bigger companies. Bigger companies already have APIs, web portal, etc. We have people who often manually enter data. Any time, a piece of paper is passed around or when people enter data manually you can either automate or improve the process. People are prone to errors. Right now, I am rewriting a mission critical application, some trivial changes will save hours a week and improve the integrity of the information. It is not as exciting as writing algorithms but it is nice to see an application written by you used by 200+ people.
- genidoi 7y ago
  Go on forums where people who use a platform/API/SaaS post implicit feature requests Off the top of my mind: https://sellercentral.amazon.com/forums/c/selling-on-amazon/third-party-software-services https://sellercentral.amazon.com/forums/c/selling-on-amazon/... https://community.ebay.com/t5/Tools-Apps/bd-p/tools-apps-db https://community.ebay.com/t5/Tools-Apps/bd-p/tools-apps-db
  - james_s_tayler 7y ago
    MWS is not fun. Trust me. Haha.
- eyegor 7y ago
  Honestly your best bet is to talk to people. I doubt cold calling/knocking on the door of a business will succeed, but people get enthusiastic when it comes to complaining about tedious/monotonous work. Especially at bars. Turn to someone nearby and say "I am looking for ideas to yadda yadda save people time and frustration. Is there any tedious process you deal with at work which you think doesn't need a person to deal with?" If you travel, this is especially common in airports. I've never asked someone this question and yet people rant at me all the time once I say "software developer".
- starpilot 7y ago
  Very common when you do tech work in a non-tech industry. Comes up a lot with coding-inclined mechanical engineers. Zed Shaw's words are really true: > Programming as a profession is only moderately interesting. It can be a good job, but you could make about the same money and be happier running a fast food joint. You're much better off using code as your secret weapon in another profession. People who can code in the world of technology companies are a dime a dozen and get no respect. People who can code in biology, medicine, government, sociology, physics, history, and mathematics are respected and can do amazing things to advance those disciplines. https://learnpythonthehardway.org/python3/advice.html https://learnpythonthehardway.org/python3/advice.html
  - phumbe 7y ago
    Great quote -- surprised I've never seen it before! In my limited experience, it's a mixed bag. Good: you get special treatment/opportunities because of a unique skill set and increased visibility on the end results of what you do. Bad: management doesn't really know what you do between software releases, you're paid the going rate for your industry while SWEs make far more, and in-house software quality standards might not be established/followed.
    - eyegor 7y ago
      Side ish bonus: you're basically prepared to run a one man show or move into niche consulting (much much higher rates). You also get to set the quality standards/procedures going forward, which can be satisfying. As a tangential bonus, I've accidentally converted my PhD research coworkers to strict git/markdown thanks mostly to typora (windows application). I showed one person what my work flow and version history look like for some internal documentation and now they do the same and convert to word/pdf as a last step. As far as I can tell, no one outside of the math/cs intersection has the patience for latex. Source: in that boat. Edit: In regards to market rates, that can be alleviated somewhat in follow up negotiations (6-12 months in or so). It's hard to convince someone what you're worth / what your value proposition is when they're not used to hiring software people. You need to demonstrate your business effect first, since they typically don't have a clear picture of it.
- deleted 7y ago
  [deleted]
- neverhigh 7y ago
  At the moment many corporates are just automating the incoming invoice process. However, many processes are document (any kind of digital file) based to share information between departments, vendors or customers. Many processes could be automated. To identify a business case for automation worth coding such an application or offer an API we use three main KPI to identify processe worth automating it: - more than 10 documents per day on year average, e.g a bank will receive new annual reports only in some calendar months but in massive scale - average number of pages or lines of text per document, the longer the document the more mistakes will be made by humans, as they don't have the time to read everything in detail - average pay of the FTE who is able to understand and process the document manually should be higher than the average pay of all employees in the company, to make sure the documents encapsulate business value It's not a fixed set of KPI but helps us to sort out too narrow use cases. By we I refer to the team behind my startup Konfuzio: http://www.konfuzio.com/en/ http://www.konfuzio.com/en/
  - K0SM0S 7y ago
    Thanks a lot for sharing such valuable insights! Small payback : on your English home page under "Operation of the software" > "Information security", there's one too many sentences: > We set the highest standards both when creating the software and when processing your data. Both when creating the software and when processing your data we set the highest standards. (First one is better imho).
jonaldbobson 7y ago
Is this similar to RPA (Robotic Process Automation)?
- folkhack 7y ago
  Yep - it's definitely that, although I typically avoid the nomenclature because laypeople think "robotics" which is definitely not what I do!
kilimchoi 7y ago
sounds like RPA
- wenc 7y ago
  I was going to say. Many companies are already investing heavily in RPA (Robotic Process Automation) these days. It’s not widely known in tech circles but kind of common knowledge in traditional companies trying to automate tedious workflow tasks. When I first saw it, I thought: this looks like a glorified Autohotkey or Automator. But companies like UiPath are doing very well selling this type of automation software. It sounds mundane but for a company with lots of entry level staff, the paybacks are appreciable. Coupled with cloud based OCR as a service offerings etc. a lot of data entry tasks can be automated.
  - folkhack 7y ago
    > Autohotkey Yep I've had to employ this software to get things done before. Also OCR doesn't need a cloud vs. a beefy computer - I do a lot of that too.
    - wenc 7y ago
      Yes free OCR software itself is pretty simple and can run offline but if your accuracy requirements are high (for critical documents), different algorithms (typically proprietary) are needed and as well as fallbacks to contracted human labor for verification. At some level it makes sense to outsource to a cloud provider.
    - alexcnwy 7y ago
      > OCR doesn't need a cloud vs. a beefy computer - I do a lot of that too. Can you elaborate on this?
- danjc 7y ago
  I've been conflicted about RPA because my view is that if there's an API, you should use that rather than comparatively brittle screen-based "integration". What's extremely compelling about this approach though is that you can cut through all of the friction of API's - even end users understand how to use the UI so they're really empowered to build their own automation in a way that they haven't been previously.
deleted 7y ago
[deleted]
gota 7y ago
Do you have anything going on related to process mining?
- folkhack 7y ago
  YES. I didn't even know that nomenclature before you said it... I employ an ELK and TIG stack for a huge part of what I do. You can very much say that I "process mine" my own process. I also have a ton of regex capability that's often employed to rip through large datasets to extract key data. Lots of times I have to measure certain things before I begin to fix a problem, and the above stated tooling (typically ran in Docker containers) has really really helped with that. I have dashboards, warnings, circuit breakers, etc etc. that all help me monitor complex processes, and that help me react to ongoing changes (ex: when a 3rd-party website updates their DOM).
  - dhruvkar 7y ago
    Man.. I wish I knew what you know. These are the things I need at my current role, and I'm woefully overwhelmed. Tying together processes spread out over multiple systems is a large portion of my role. I'm having trouble tracking all the various scripts running on different versions of my own packages, and reacting to breaking changes on 3rd party sites. I definitely need a dashboard to tie all these automations together. I'd love to chat if you're up for it.
    - folkhack 7y ago
      Absolutely! Feel free to reach out - you can get to my email address at folkhack.com
  - gota 7y ago
    Great to hear. I'm finishing my doctorate researching a specialized Process Mining approach and its great to hear there's people doing concrete significant work like that out there
radcon 7y ago
So... automate clerical stuff? > ever-increasing need Not sure I'd bet on that -- the point of RPA is to eliminate the job you just described.
dhruvkar 7y ago
This. I work in a small, physical goods business in the Midwest (~30 employees), and the amount of time spent doing data entry is incredible. Documents (often PDFs, sometimes excel) flow between buyers, shippers, freight forwarders and internally. These documents are often updated. I'm looking into EDI for a solution, but there's so much value to be created from cutting out all this data entry. At least 1-1.5 full time employee's worth at our size.
- dhruvkar 7y ago
  And these industries are so far outside the tech bubble, it's two different worlds. Facebook and Co. get a bad rap for many things, which I agree with. But I credit these social platforms in speeding up tech adoption rates for the average person, who can now use this knowledge at their job.
- the-dude 7y ago
  I read about EDI in the early 80s.
  - dhruvkar 7y ago
    Right. It's been around for 30+ years and it's meant to solve this exact problem. However, the cost of training staff and implementation is often out of reach for SMBs, still.
  - exhilaration 7y ago
    The largest companies in the world - Amazon, for example - still actively use EDI.
    - deleted 7y ago
      [deleted]
- dsaavy 7y ago
  Ah, I dealt with this data entry manual process many times at work. We were able to build a few solutions to remove 80-90% of the human work for these processes. It took a few years to get good at but now I run a business that specializes in this type of stuff.
friendlybus 7y ago
Yes I did some of this at an old telecoms job. It was incredibly obvious that an automated system could put 5-10 people out of work depending on the sophistication of the implementation. I started automating my own analysis job and left before spreading it or letting it work without super vision. Definitely a lot of low level knowledge worker jobs can be evaporated.
- folkhack 7y ago
  > telecom That's the industry I'm currently working on... they're generally regressive and have shot themselves in the foot in regards to tech and ease-of-use for a lot of their solutions.
derefr 7y ago
Is there a possibility of ...automating, this kind of automating? I understand that, like regular application programming, a lot of this kind of automation comes down to requirements-gathering and formal specification of workflow. But, in theory, could there be some kind of sufficiently-advanced software wizard that does most of your job? Maybe by monitoring entry-level employees' interactions with the software-to-be-automated over some period, and using that as a training input to a workflow guesser? I'm not imagining that you could be cut out of the loop entirely, mind you, but rather that you'd reduce the human-interaction man-weeks part of your job down to a few hours of pairing with a domain expert to clean up the output of workflow-guesser into a final model.
- folkhack 7y ago
  > Is there a possibility of ...automating, this kind of automating? Nah - I'm going to be cocky for a second so please take this with a grain of salt... Automation is the only job that's safe from automation ;) That being said, it's prime for development of tools... I've been working on taking a lot of my workflow out of Charles Proxy and moving it onto a more "man-in-the middle" model that can replay certain web traffic... PARTS of what I do, but automating the automating is probably the most difficult problem that I've had to solve (ie: building my own tools).
  - radcon 7y ago
    They're already automating the exact job you've been describing: https://techcrunch.com/2019/04/30/uipath-rpa-series-d/?renderMode=ie11 https://techcrunch.com/2019/04/30/uipath-rpa-series-d/?rende...
    - pacoWebConsult 7y ago
      To expand upon this, UiPath also announced acquisitions of ProcessGold (https://www.uipath.com/newsroom/uipath-acquires-process-gold-unparalleled-process-understanding https://www.uipath.com/newsroom/uipath-acquires-process-gold...) and StepShot (https://www.uipath.com/newsroom/uipath-acquires-stepshot-adds-process-documentation-to-market-leading-rpa-platform https://www.uipath.com/newsroom/uipath-acquires-stepshot-add...) as a step towards bringing the process documentation and automation planning tasks down to the level of understanding that SMEs and BAs have about automation. These tools will be a really great addition to the ecosystem as I (and clearly they) have found the most difficult work in this space to be understanding the client's existing process and planning to automate it.
julius 7y ago
How do you find customers for that? Any job board tricks, maybe?
- folkhack 7y ago
  Networking and knowing people in industries that have a need =) I've never really done the job board thing so no idea on that.
mikorym 7y ago
Wow, interesting comment. I just started with such a Puppeteer (et al.) project...
abhinuvpitale 7y ago
Agreed, there is definitely a need for 'smart' automation. There are a lot of tools out there which work however, finding the right tool for the right job is difficult. Also, I think its extremely difficult to get the automation right in 100% of the cases. False positives (even 1%) could be extremely dangerous in cases. Hence, I think there is a huge market for 'smart' automation tools, people who can build them and also for people who can use them efficiently.
Mister_Snuggles 7y ago
One of the most useful programs I made at my last job was an automated letters program. We had a whole slew of standard letters that we'd send out, most of the time requests would go to the typing pool (yes, this was a thing) that were basically "Do letter ABC for client XYZ", where letter ABC was a standard thing that always went to the same recipient. I built a screen in our system that let you tell it to request a letter (only the simple letters, obviously), it would pull the relevant data out of the client file and write the request out as an XML file. On the other side of things, a VB6 program would watch a folder for those XML files to appear and go through OLE Automation (as it was called back then) to make Word fill in the letter, print it, print an envelope, and save the result in the client's folder. This kind of thing is an absolutely incredible time saver.
- mring33621 7y ago
  A whole lot of valuable business software is essentially domain-specific or workflow-specific mail merge.
  - Mister_Snuggles 7y ago
    I've never thought about it that way before, but you're absolutely right. Another "domain-specific mail merge" that I built was a tool to pull summary data out of the client financial records and populate specific fields in some off-the-shelf income tax software. That was a terrible tool to write and maintain. The tax software's import feature didn't actually work correctly. I reached out to the vendor and the response was basically "Works for us", so doing it the right way wasn't an option. Instead, I used VBA to blindly enter data into the program using SendKeys[0]. Maintaining the program was a case of tabbing through and counting the number of times I had to tab. Enter first name, tab, enter last name, tab three times, enter address line 1, etc. Next year when the forms in the tax software changed I'd have to add/remove tabs in the appropriate places, sometimes input stuff in a different order, etc. As a program, it was horrible, horrible code. As a useful tool for the business, it was incredible. It saved a ton of time and the staff that used to do this by hand, in addition to doing actual work on the more complex files, were incredibly happy. Overall, it was completely worth it. [0] https://docs.microsoft.com/en-us/office/vba/language/reference/user-interface-help/sendkeys-statement https://docs.microsoft.com/en-us/office/vba/language/referen...
kilroy123 7y ago
I've been thinking a lot recently at trying to do this kind of work as well. I'm just not sure where to start. I have a pretty weak network form living abroad for so long and moving around a lot. Any advice about how to even get started doing this kind of work? What tools to study or learn? How to find work?
- folkhack 7y ago
  Absolutely! Don't move around all the dang time. I've lived in two places and have an incredibly strong network in both. I find people who are bouncing around 50-times-over usually lack the deep business relationships that it takes to establish trust to solve "big" problems. As far as getting started - once you start, just never stop. Solve every dang problem you can get your hands on big or small, and continue to publish code every day. Tools to study and learn - web web web. Also Microsoft Excel automation is an incredible need for lots of businesses. Puppeteer, and learning how to mimic what a browser can do without using a browser would be the two most useful skills I have personally. If you don't have a VERY solid background in webdev you need to get that going first, so much of what I do is reverse-engineering someone's website so I have to be very versed in the way that sites are built (all the way from ancient ones to the most modern of frameworks). How to find work is network. Once you have the skills, you need to know people that have potential needs. I've already reached out to two people who found me through this thread and regardless of "making money", I genuinely want to help them, if even to just point them in the right direction (like yourself). People are (*generally) awesome if you put aside your own ego, and I would be absolutely nowhere if I wasn't a sociable person. I love to learn about the world by making connections with people, and genuinely want to help them with what I can do!
jliptzin 7y ago
I could use help automating a simple task for one of my businesses involving submitting refund requests to USPS. Please get in touch if you're looking for new clients!
- thorwasdfasdf 7y ago
  message sent.
ctack 7y ago
Once you create an api or solution, are you able to resell it to other businesses or or do you create once off solutions?
- folkhack 7y ago
  Both - but I'm more tooled to solve one-off situations. I have been able to do the whole "copy-paste" resell a few times which has been financially lucrative! My GitLab is my toolbox in so many ways and I've got a wealth of clean/documented code that I can re-purpose depending on the task at hand.
nwsm 7y ago
Robotic Process Automation (RPA). Examples are UI Path and Blue Prism. It's finicky technology that is hard to make sustainable, but a lot of companies are really interested in it because of the amount of work that could be cut out like you said.
- folkhack 7y ago
  > It's finicky technology that is hard to make sustainable This 100 times over. The only way I've been able to make it work is being one heck of a generalist. I've found that training on what I do is near-impossible and I don't even know if I could scale a real company in this beyond myself.
  - virgilp 7y ago
    Well, UiPath's valuation is currently 7B... But yeah. They have an "academy" for training RPA developers, forums/ social presence, big sales teams, a process for developing automation, an "automation framework", a marketplace for automation activities, etc.... all these are stuff outside of the core software (robot/"activity executor", orchestrator, and studio for designing said activities). Maintaining & expanding the automation is a heck of a challenge though. Like, once you got the stuff deployed, make sure that it works without headaches, in the face of other software/ OS upgrades. The robots/automation services are essentially a "fleet of microservices" that you have to maintain for a customer whose strength is likely not IT, and who will change the infrastructure from underneath your "microservices". It's a hard problem, but I feel we're well positioned to tackle that, and if we do, the sky is the limit really. (disclaimer: I work for UiPath, though that should be obvious from the message)
    - folkhack 7y ago
      Right on! Yeah - I would love to learn more about what the bigger guys are doing. Everything you're saying is 100% something I deal with on a day-to-day basis, and ongoing support is a huge struggle to workout with the companies I've implemented software at. Looking at your product offerings made me chuckle - I definitely have versions of a lot of what you do! My robots are called workers, my orchestrator is called a broker, etc etc ha. Lemme know if you guys are ever looking for a remote automation engineer w/tons of cloud experience ;)
charliemil4 7y ago
Any advice for companies getting into this realm, specifically around making sure you get your 'valued' price? Charging for value can sometimes feel exorbitant... i.e. saving a $100+/hr person ~20 hours a month. Sure I can calculate out it on an excel sheet the customer ROI, but still feels crazy.
- folkhack 7y ago
  My market value is $10-15k/mo USD for just myself (sometimes more if I'm needed in an emergency). If you have a proven track record with a solid reputation you can command that sort of pay without an issue. I also bring a lot of general web development capability to the table, so selling on an easy-to-use interface also does a lot to add to my value. If you think about it - $35-40k for a 2-3 month dev timeline (which is what I'd say is my average) is worth it if you're cutting the salaries of 5+ people out of the equation, no need to pull out an Excel spreadsheet for the ROI that. All-in-all - you're not crazy, they're the ones who are crazy if they continue to sink money into human robots. > Charging for value can sometimes feel exorbitant Just get confident (this is hard advice I know). I'm incredibly confident in the problems I can solve and will put in the hours to get back on a timeline/budget if needed. Businesses aren't buying my "products" per-say, they're buying me.
  - charliemil4 7y ago
    I owe you a round for this. Hopefully it sticks.
  - jcmontx 7y ago
    Solid comment. How do you deal with getting the knowledge of the process out of the people you are trying to leave without a job? That must be rough.
    - folkhack 7y ago
      They're entry level positions usually so figuring out their processes is typically not that difficult. Often I will not work directly with them vs. having a big sheet of instructions delivered with a ton of bullet points. I also get a ton of process diagrams as well. Usually it's better for the business I'm supporting to document their own workflows before it's handed over to me for a lot of common sense reasons.
WhompingWindows 7y ago
I was working for a scientific journal. In the peer-review department, they had to use this proprietary software, and we had 2 employees who just mindlessly went through this proprietary software's form day-in, day-out. This form spammed scientists for reviews of science articles for our somewhat-predatory journal (we solicited authors to publish in our journal for a fee)... First, it was a massive waste of those 2 human's intellect, entering data over and over into a generic form from a proprietary business' website, in order to spam scientists. I started a ruby project to automate out all those processes, just using selenium to enter the data onto the web form for us. Before I could get very far, I got a huge promotion elsewhere and left that under-compensating journal. It all gave me a depressing vibe though, both the ethics of what we were doing and the waste of time/life that those employees had to endure to do it.
- dev_dull 7y ago
  > First, it was a massive waste of those 2 human's intellect This is usually the reality and it's nothing to be ashamed about. At work, do what you're paid to do. In your free time, pursue interests (literally, things that are interesting to you).
- NotSammyHagar 7y ago
  When those jobs are automated, what will the human workers do, will they leave that company? I'd enjoy doing something else if that was my job, but if there was a more useful job there, wouldn't they already be doing it? This is in summary why I think more automation will lead to some people losing their jobs - if there was a more profitable alternative for a company (more productive, paid as well or better?) then wouldn't the company already have automated it?
koonsolo 7y ago
I worked in the Automated Guided Vehicles industry, and so I want to extend your suggestion: any automation that can replace employees. Those systems are easy to sell with "This system will pay itself back in 2,5 years, taking into account your current labour cost". Given all the AI/deep learning possibilities, lots can be automated still.
mcrad 7y ago
Do you find it difficult to sell the idea of automating the work of "entry-level data employees"? Middle management generally thrives by building kingdom of such minions.
- folkhack 7y ago
  > Middle management generally thrives by building kingdom of such minions. I often am not selling to middle management, I ideally sell at the exec and/or product level. Middle managers are a conflict of interest when it comes to what I do.
- draw_down 7y ago
  There exist more powerful forces in the world than "middle management".
caseydonahue 7y ago
Can you give some more insight into what you work on automating specifically?
- folkhack 7y ago
  Anything - green screens, Excel spreadsheets, PDF generation, Docusign, tying 6 web experiences with separate logins into one, simplification of complex auth scenarios, etc. Most of my day is literally just getting one step further and one step further into a "process" of some sort, and then handling a metric boatload of edge-cases as they come up. Then the last portion of the contract is the actual integration into a business, and handling the "ninety-ninety" rule that always ends up happening when the rubber meets the road. I take a complex, convoluted mess and turn it into a coherent "something", usually an API or web experience.
  - cgsmith 7y ago
    That is essentially what I do. I pitch it as process engineering, systems integrator, programmer, etc.
  - jcconnell 7y ago
    I'm curious how you market yourself or your services? I automate processes, workflows and tasks like this for some clients and I really enjoy it. It's easy to see the value I provide and the clients are happy. However I struggle with how to acquire new/more clients. The ones I have now I've acquired through past business relationships. I'd love to be able to do more of this work.
dev_dull 7y ago
The key to these types of solutions is always to follow the 80/20 rule. For example, it’s foolish to say and attempt “I will automate all of your data entry people” because there will always be weird idiosyncrasies that are better fit for humans. You will dash yourself against the rocks trying to get a “100% complete” solution. Instead, create software that allows 1 person to do the job of 5. You create massive business value without getting sucked down the hole of edge cases.
- nextos 7y ago
  True. In the same spirit, I'd also add record linkage to automated data entry. It's a big problem both if your previous data is noisy or your automated solution doesn't transcribe some fields correctly. A simple probabilistic programming solution can work really well.
- alexcnwy 7y ago
  I’m building a tool to speed up data entry. It highlights OCR‘d words with low confidence for human review. It uses templates for capturing structured data and a fast interface for capturing unstructured data. It grew out of consulting projects and is almost ready for beta testing if anyone is interested in playing with it and giving me some feedback - my email is in my profile :)
  - tomericco 7y ago
    Email sent.
- codeisawesome 7y ago
  How do you guys network with the right business people to speak their language and find out how their processes work? You need that info (a problem description) before you come out with a solution, don’t you?
  - laughinghan 7y ago
    Just ask. People absolutely love talking about their problems, it'll probably be the most animated thing they have to say about the business they're in. "Some people profess difficulty at finding applications to write. I have never understood this: talk to people. People have problems — lots of problems, more than you could enumerate in a hundred lifetimes. Talk to a carpenter, ask him what about carpentry sucks. Talk to the receptionist at your dentist’s office — ask her what about her job sucks. Talk to a teacher — ask her what she spends time that she thinks adds the least value to her day." https://www.kalzumeus.com/2010/03/20/running-a-software-business-on-5-hours-a-week/ https://www.kalzumeus.com/2010/03/20/running-a-software-busi...
virgilp 7y ago
Isn’t this the “robotic process automation” market? (though that one is a bit more than what you are saying)
wodenokoto 7y ago
Isn't this what for a short while was called robotics, but now is mostly referred to as RPA? It sounds like you are competing with things like Blueprism. These things are really a big deal within large organisations.
- folkhack 7y ago
  Yep! I'm effectively a boutique provider of such services.
odonnellryan 7y ago
This is what I do for work (mostly) as well. The challenge I've had trying to grow the business is "convincing" (for lack of a better word) businesses that they will really save a lot - even after showing them case-studies. Companies can be stubborn. Hit me up if you ever want to chat! ryan luma.im
surbekznrlsnt 7y ago
I'm living in Ireland and this is what I've wanted to do for years. There are loads of older small to medium businesses with SO many things that can be automated but I don't know how to get into it. I don't even know how to research it. What's the process for getting a client? Cold calling? Partnership with another provider? What is the job title I should search for more information? Automator?
- g0bananas 7y ago
  You should look at business operations analyst, UI Path training is on linkedin learning/lynda.com too. I think information management systems too?
- ploika 7y ago
  A handful of places around Dublin have active Robotic Process Automation (RPA) teams, which seems to be what's being discussed here. The Big Four consultancies and Accenture spring to mind. I think the likes of Fidelity and Citi have few roles too. Note however that you'd be using off the shelf tools to do the automating - you wouldn't be developing the automation tools yourself.
- Rooster61 7y ago
  Automation Engineer. It's a broad term, but does carry the idea that you know how to program, and do so in a way that automates normally manual processes. Also doesn't hurt to list yourself on a resume as an Automation/Software Engineer. That gets the idea across that you also understand how software is written, not just how code is cranked out.
BubRoss 7y ago
How do you find people who need these things?
mstade 7y ago
What’s your business model? Consulting by the hour? Subscription fees? One-off fixed price contracts? Practically all software I’ve ever done has been some amalgamation or other of “make forms better and more automatic”, whether it’s planning software for gigantic Swedish furniture firms (take a wild guess!) or technical analysis toolkits for financial services. It’s always been per-hour or fixed price contracts for me – try as I might but the subscription model simply doesn’t take with the buyers it seems. What I’d like to do is fixed price for initial dev and then subscription based contract for support and minor updates, but this seems wholly unpalatable for buyers.
jakemal 7y ago
Can you share what you're working on? I would love to see a concrete example of something you are referring to as well as how you market it.
pacoWebConsult 7y ago
I began working in RPA this year and I completely agree. Tremendous potential but also a lot of buzzword-driven hype in the space. I think the main players need to work harder on bringing world-class engineering and best-practices to the community, since the vast majority of developers that are getting into this are not really advancing the field and instead slapping crap together until their 'bot' works. Why are most automation projects relegated to bare-bones budgets and short timeframes? Potential clients i have engaged have a very short-sighted view of automation possibilities in their businesses, and aren't open to the possibilities beyond what they think should be done by a bot instead of their employees.
scruple 7y ago
Currently automating a lot of on-boarding process at work. It's been eye opening. What it's really shown me is how very little expertise we have around all of these solutions, SaaS products, etc... that the company uses. From Tableau to Salesforce to our accounting software, it's a mess. It's put myself and my team into a unique position to gain a lot of mind share, especially when it comes to cleaning a lot of the inefficiencies up. That said, I don't think we're making anyone redundant, or cutting anyone out of the equation in a negative way -- none of the people who are currently tasked with a lot of this work were ever supposed to be doing it, trained on it, etc...
rkwasny 7y ago
That's exactly what we do, with a nice interface business people can use, email me at rafal@evolution.ai if you want a demo.
bbleciel 7y ago
I seriously want to begin working in this field as a consultant/architect/developer. So much so that I've registered the LLC, opened a bank account, and even had a few clients I built web apps for. However, I'm not exactly sure how to pivot to business solutions/productivity tools though. Do you usually approach companies where you expect to have a solution for them? Or do you find clients through advertising? Do they find you? Any advice is appreciated!
modwest 7y ago
Does your company, or do any in the field, support policies for retraining/job placement for employees displaced by their technology?
alentodorov 7y ago
I think UIPath recent success shows that the market already values this.
hnick 7y ago
That's funny, the multi-national company I work for is removing all MS Office automation because MS apparently told us that unattended automation is against the licensing terms. It's OK if you're watching the computer I guess, but not on a server. Personally this didn't match what I found online (they said it was not supported i.e. if it breaks tough luck) but what would I know. Money must be spent retrofitting other solutions now. I joked we should pay a minimum wage high school kid to sit in a room and 'watch' the servers but I guess that's too low tech ;)
- folkhack 7y ago
  They have specific solutions that are integrated into things like SharePoint to handle this so they're just putting the hurt on the big guys in order to get more monies. Total Microsoft play on that for sure - I've never ran into the same issue though.
  - hnick 7y ago
    Yeah I looked into this, and it's definitely overkill for what we wanted (plus sending them more money to fix a problem they made for us would be very grating). All we needed to do is read/write Excel data and similar functions and keeping it local is key. There wasn't much available when this got put in place but there are plenty of reliable libraries now.
johnisgood 7y ago
I have been working on automating data entry, and the consequences of this is many people (in this case Indians) losing their jobs. I am facing a moral dilemma. I have been writing free software for my entire life, I volunteer, but I live in really poor conditions. I want to get forward somehow, and this seems to be my best chance of getting forward at this point in time and space. I am probably not going to go through with it. It seems to be the case that my principles, my moral values have been holding me back from improving my life. I do not want to lie to people, I do not want to deceive people, I do not want to fsck them over in any way. I do not want to contribute to anything nefarious. But from where I am (the bottom), it is extremely hard to climb up without doing that. :/
jacobwilliamroy 7y ago
Your post brought back fond memories of combining autohotkey with premiere pro to automate boring tasks. Haha. I agree, you have a nice niche.
zelly 7y ago
Automating mouse clicks and key presses seems backward and stupid. Doing this manually will also get disrupted. You should trace the executable and analyze the function calls. Get a million samples then form a statistical model. The production model replays the function calls given some input. Doesn't need graphical memory anymore and it also has higher resolution inputs (function symbols, addresses) than coordinates on a screen.

maxhallinan 7y ago

How do you find processes that can be automated? I've often thought that there must be a ton of this stuff in various industries where programmers aren't typically embedded.

HeyLaughingBoy 7y ago
You really need to find a domain expert who can at least get you started. I'm working on a problem that I would not have even known existed, if not for running across a couple of engineers who had a business in the field and saw an opening that only a small company would care about (market isn't big enough for the large players). I know similar situations have to be all around us. The problem, as you say, is finding out about them.
avgDev 7y ago
Any midsize company has tremendous amount of things that could be automated. For example, I work at manufacturing company, we produce parts for bigger companies. Bigger companies already have APIs, web portal, etc. We have people who often manually enter data. Any time, a piece of paper is passed around or when people enter data manually you can either automate or improve the process. People are prone to errors. Right now, I am rewriting a mission critical application, some trivial changes will save hours a week and improve the integrity of the information. It is not as exciting as writing algorithms but it is nice to see an application written by you used by 200+ people.
genidoi 7y ago
Go on forums where people who use a platform/API/SaaS post implicit feature requests Off the top of my mind: https://sellercentral.amazon.com/forums/c/selling-on-amazon/third-party-software-services https://sellercentral.amazon.com/forums/c/selling-on-amazon/... https://community.ebay.com/t5/Tools-Apps/bd-p/tools-apps-db https://community.ebay.com/t5/Tools-Apps/bd-p/tools-apps-db
- james_s_tayler 7y ago
  MWS is not fun. Trust me. Haha.
eyegor 7y ago
Honestly your best bet is to talk to people. I doubt cold calling/knocking on the door of a business will succeed, but people get enthusiastic when it comes to complaining about tedious/monotonous work. Especially at bars. Turn to someone nearby and say "I am looking for ideas to yadda yadda save people time and frustration. Is there any tedious process you deal with at work which you think doesn't need a person to deal with?" If you travel, this is especially common in airports. I've never asked someone this question and yet people rant at me all the time once I say "software developer".
starpilot 7y ago
Very common when you do tech work in a non-tech industry. Comes up a lot with coding-inclined mechanical engineers. Zed Shaw's words are really true: > Programming as a profession is only moderately interesting. It can be a good job, but you could make about the same money and be happier running a fast food joint. You're much better off using code as your secret weapon in another profession. People who can code in the world of technology companies are a dime a dozen and get no respect. People who can code in biology, medicine, government, sociology, physics, history, and mathematics are respected and can do amazing things to advance those disciplines. https://learnpythonthehardway.org/python3/advice.html https://learnpythonthehardway.org/python3/advice.html
- phumbe 7y ago
  Great quote -- surprised I've never seen it before! In my limited experience, it's a mixed bag. Good: you get special treatment/opportunities because of a unique skill set and increased visibility on the end results of what you do. Bad: management doesn't really know what you do between software releases, you're paid the going rate for your industry while SWEs make far more, and in-house software quality standards might not be established/followed.
  - eyegor 7y ago
    Side ish bonus: you're basically prepared to run a one man show or move into niche consulting (much much higher rates). You also get to set the quality standards/procedures going forward, which can be satisfying. As a tangential bonus, I've accidentally converted my PhD research coworkers to strict git/markdown thanks mostly to typora (windows application). I showed one person what my work flow and version history look like for some internal documentation and now they do the same and convert to word/pdf as a last step. As far as I can tell, no one outside of the math/cs intersection has the patience for latex. Source: in that boat. Edit: In regards to market rates, that can be alleviated somewhat in follow up negotiations (6-12 months in or so). It's hard to convince someone what you're worth / what your value proposition is when they're not used to hiring software people. You need to demonstrate your business effect first, since they typically don't have a clear picture of it.
deleted 7y ago
[deleted]
neverhigh 7y ago
At the moment many corporates are just automating the incoming invoice process. However, many processes are document (any kind of digital file) based to share information between departments, vendors or customers. Many processes could be automated. To identify a business case for automation worth coding such an application or offer an API we use three main KPI to identify processe worth automating it: - more than 10 documents per day on year average, e.g a bank will receive new annual reports only in some calendar months but in massive scale - average number of pages or lines of text per document, the longer the document the more mistakes will be made by humans, as they don't have the time to read everything in detail - average pay of the FTE who is able to understand and process the document manually should be higher than the average pay of all employees in the company, to make sure the documents encapsulate business value It's not a fixed set of KPI but helps us to sort out too narrow use cases. By we I refer to the team behind my startup Konfuzio: http://www.konfuzio.com/en/ http://www.konfuzio.com/en/
- K0SM0S 7y ago
  Thanks a lot for sharing such valuable insights! Small payback : on your English home page under "Operation of the software" > "Information security", there's one too many sentences: > We set the highest standards both when creating the software and when processing your data. Both when creating the software and when processing your data we set the highest standards. (First one is better imho).

jonaldbobson 7y ago

Is this similar to RPA (Robotic Process Automation)?

folkhack 7y ago
Yep - it's definitely that, although I typically avoid the nomenclature because laypeople think "robotics" which is definitely not what I do!

kilimchoi 7y ago

sounds like RPA

wenc 7y ago
I was going to say. Many companies are already investing heavily in RPA (Robotic Process Automation) these days. It’s not widely known in tech circles but kind of common knowledge in traditional companies trying to automate tedious workflow tasks. When I first saw it, I thought: this looks like a glorified Autohotkey or Automator. But companies like UiPath are doing very well selling this type of automation software. It sounds mundane but for a company with lots of entry level staff, the paybacks are appreciable. Coupled with cloud based OCR as a service offerings etc. a lot of data entry tasks can be automated.
- folkhack 7y ago
  > Autohotkey Yep I've had to employ this software to get things done before. Also OCR doesn't need a cloud vs. a beefy computer - I do a lot of that too.
  - wenc 7y ago
    Yes free OCR software itself is pretty simple and can run offline but if your accuracy requirements are high (for critical documents), different algorithms (typically proprietary) are needed and as well as fallbacks to contracted human labor for verification. At some level it makes sense to outsource to a cloud provider.
  - alexcnwy 7y ago
    > OCR doesn't need a cloud vs. a beefy computer - I do a lot of that too. Can you elaborate on this?
danjc 7y ago
I've been conflicted about RPA because my view is that if there's an API, you should use that rather than comparatively brittle screen-based "integration". What's extremely compelling about this approach though is that you can cut through all of the friction of API's - even end users understand how to use the UI so they're really empowered to build their own automation in a way that they haven't been previously.

deleted 7y ago

[deleted]

gota 7y ago

Do you have anything going on related to process mining?

folkhack 7y ago
YES. I didn't even know that nomenclature before you said it... I employ an ELK and TIG stack for a huge part of what I do. You can very much say that I "process mine" my own process. I also have a ton of regex capability that's often employed to rip through large datasets to extract key data. Lots of times I have to measure certain things before I begin to fix a problem, and the above stated tooling (typically ran in Docker containers) has really really helped with that. I have dashboards, warnings, circuit breakers, etc etc. that all help me monitor complex processes, and that help me react to ongoing changes (ex: when a 3rd-party website updates their DOM).
- dhruvkar 7y ago
  Man.. I wish I knew what you know. These are the things I need at my current role, and I'm woefully overwhelmed. Tying together processes spread out over multiple systems is a large portion of my role. I'm having trouble tracking all the various scripts running on different versions of my own packages, and reacting to breaking changes on 3rd party sites. I definitely need a dashboard to tie all these automations together. I'd love to chat if you're up for it.
  - folkhack 7y ago
    Absolutely! Feel free to reach out - you can get to my email address at folkhack.com
- gota 7y ago
  Great to hear. I'm finishing my doctorate researching a specialized Process Mining approach and its great to hear there's people doing concrete significant work like that out there

radcon 7y ago

So... automate clerical stuff? > ever-increasing need Not sure I'd bet on that -- the point of RPA is to eliminate the job you just described.

dhruvkar 7y ago

This. I work in a small, physical goods business in the Midwest (~30 employees), and the amount of time spent doing data entry is incredible. Documents (often PDFs, sometimes excel) flow between buyers, shippers, freight forwarders and internally. These documents are often updated. I'm looking into EDI for a solution, but there's so much value to be created from cutting out all this data entry. At least 1-1.5 full time employee's worth at our size.

dhruvkar 7y ago
And these industries are so far outside the tech bubble, it's two different worlds. Facebook and Co. get a bad rap for many things, which I agree with. But I credit these social platforms in speeding up tech adoption rates for the average person, who can now use this knowledge at their job.
the-dude 7y ago
I read about EDI in the early 80s.
- dhruvkar 7y ago
  Right. It's been around for 30+ years and it's meant to solve this exact problem. However, the cost of training staff and implementation is often out of reach for SMBs, still.
- exhilaration 7y ago
  The largest companies in the world - Amazon, for example - still actively use EDI.
  - deleted 7y ago
    [deleted]
dsaavy 7y ago
Ah, I dealt with this data entry manual process many times at work. We were able to build a few solutions to remove 80-90% of the human work for these processes. It took a few years to get good at but now I run a business that specializes in this type of stuff.

friendlybus 7y ago

Yes I did some of this at an old telecoms job. It was incredibly obvious that an automated system could put 5-10 people out of work depending on the sophistication of the implementation. I started automating my own analysis job and left before spreading it or letting it work without super vision. Definitely a lot of low level knowledge worker jobs can be evaporated.

folkhack 7y ago
> telecom That's the industry I'm currently working on... they're generally regressive and have shot themselves in the foot in regards to tech and ease-of-use for a lot of their solutions.

derefr 7y ago

Is there a possibility of ...automating, this kind of automating? I understand that, like regular application programming, a lot of this kind of automation comes down to requirements-gathering and formal specification of workflow. But, in theory, could there be some kind of sufficiently-advanced software wizard that does most of your job? Maybe by monitoring entry-level employees' interactions with the software-to-be-automated over some period, and using that as a training input to a workflow guesser? I'm not imagining that you could be cut out of the loop entirely, mind you, but rather that you'd reduce the human-interaction man-weeks part of your job down to a few hours of pairing with a domain expert to clean up the output of workflow-guesser into a final model.

folkhack 7y ago
> Is there a possibility of ...automating, this kind of automating? Nah - I'm going to be cocky for a second so please take this with a grain of salt... Automation is the only job that's safe from automation ;) That being said, it's prime for development of tools... I've been working on taking a lot of my workflow out of Charles Proxy and moving it onto a more "man-in-the middle" model that can replay certain web traffic... PARTS of what I do, but automating the automating is probably the most difficult problem that I've had to solve (ie: building my own tools).
- radcon 7y ago
  They're already automating the exact job you've been describing: https://techcrunch.com/2019/04/30/uipath-rpa-series-d/?renderMode=ie11 https://techcrunch.com/2019/04/30/uipath-rpa-series-d/?rende...
  - pacoWebConsult 7y ago
    To expand upon this, UiPath also announced acquisitions of ProcessGold (https://www.uipath.com/newsroom/uipath-acquires-process-gold-unparalleled-process-understanding https://www.uipath.com/newsroom/uipath-acquires-process-gold...) and StepShot (https://www.uipath.com/newsroom/uipath-acquires-stepshot-adds-process-documentation-to-market-leading-rpa-platform https://www.uipath.com/newsroom/uipath-acquires-stepshot-add...) as a step towards bringing the process documentation and automation planning tasks down to the level of understanding that SMEs and BAs have about automation. These tools will be a really great addition to the ecosystem as I (and clearly they) have found the most difficult work in this space to be understanding the client's existing process and planning to automate it.

julius 7y ago

How do you find customers for that? Any job board tricks, maybe?

folkhack 7y ago
Networking and knowing people in industries that have a need =) I've never really done the job board thing so no idea on that.

mikorym 7y ago

Wow, interesting comment. I just started with such a Puppeteer (et al.) project...

abhinuvpitale 7y ago

Agreed, there is definitely a need for 'smart' automation. There are a lot of tools out there which work however, finding the right tool for the right job is difficult. Also, I think its extremely difficult to get the automation right in 100% of the cases. False positives (even 1%) could be extremely dangerous in cases. Hence, I think there is a huge market for 'smart' automation tools, people who can build them and also for people who can use them efficiently.

Mister_Snuggles 7y ago

One of the most useful programs I made at my last job was an automated letters program. We had a whole slew of standard letters that we'd send out, most of the time requests would go to the typing pool (yes, this was a thing) that were basically "Do letter ABC for client XYZ", where letter ABC was a standard thing that always went to the same recipient. I built a screen in our system that let you tell it to request a letter (only the simple letters, obviously), it would pull the relevant data out of the client file and write the request out as an XML file. On the other side of things, a VB6 program would watch a folder for those XML files to appear and go through OLE Automation (as it was called back then) to make Word fill in the letter, print it, print an envelope, and save the result in the client's folder. This kind of thing is an absolutely incredible time saver.

mring33621 7y ago
A whole lot of valuable business software is essentially domain-specific or workflow-specific mail merge.
- Mister_Snuggles 7y ago
  I've never thought about it that way before, but you're absolutely right. Another "domain-specific mail merge" that I built was a tool to pull summary data out of the client financial records and populate specific fields in some off-the-shelf income tax software. That was a terrible tool to write and maintain. The tax software's import feature didn't actually work correctly. I reached out to the vendor and the response was basically "Works for us", so doing it the right way wasn't an option. Instead, I used VBA to blindly enter data into the program using SendKeys[0]. Maintaining the program was a case of tabbing through and counting the number of times I had to tab. Enter first name, tab, enter last name, tab three times, enter address line 1, etc. Next year when the forms in the tax software changed I'd have to add/remove tabs in the appropriate places, sometimes input stuff in a different order, etc. As a program, it was horrible, horrible code. As a useful tool for the business, it was incredible. It saved a ton of time and the staff that used to do this by hand, in addition to doing actual work on the more complex files, were incredibly happy. Overall, it was completely worth it. [0] https://docs.microsoft.com/en-us/office/vba/language/reference/user-interface-help/sendkeys-statement https://docs.microsoft.com/en-us/office/vba/language/referen...

kilroy123 7y ago

I've been thinking a lot recently at trying to do this kind of work as well. I'm just not sure where to start. I have a pretty weak network form living abroad for so long and moving around a lot. Any advice about how to even get started doing this kind of work? What tools to study or learn? How to find work?

folkhack 7y ago
Absolutely! Don't move around all the dang time. I've lived in two places and have an incredibly strong network in both. I find people who are bouncing around 50-times-over usually lack the deep business relationships that it takes to establish trust to solve "big" problems. As far as getting started - once you start, just never stop. Solve every dang problem you can get your hands on big or small, and continue to publish code every day. Tools to study and learn - web web web. Also Microsoft Excel automation is an incredible need for lots of businesses. Puppeteer, and learning how to mimic what a browser can do without using a browser would be the two most useful skills I have personally. If you don't have a VERY solid background in webdev you need to get that going first, so much of what I do is reverse-engineering someone's website so I have to be very versed in the way that sites are built (all the way from ancient ones to the most modern of frameworks). How to find work is network. Once you have the skills, you need to know people that have potential needs. I've already reached out to two people who found me through this thread and regardless of "making money", I genuinely want to help them, if even to just point them in the right direction (like yourself). People are (*generally) awesome if you put aside your own ego, and I would be absolutely nowhere if I wasn't a sociable person. I love to learn about the world by making connections with people, and genuinely want to help them with what I can do!

jliptzin 7y ago

I could use help automating a simple task for one of my businesses involving submitting refund requests to USPS. Please get in touch if you're looking for new clients!

thorwasdfasdf 7y ago
message sent.

ctack 7y ago

Once you create an api or solution, are you able to resell it to other businesses or or do you create once off solutions?

folkhack 7y ago
Both - but I'm more tooled to solve one-off situations. I have been able to do the whole "copy-paste" resell a few times which has been financially lucrative! My GitLab is my toolbox in so many ways and I've got a wealth of clean/documented code that I can re-purpose depending on the task at hand.

nwsm 7y ago

Robotic Process Automation (RPA). Examples are UI Path and Blue Prism. It's finicky technology that is hard to make sustainable, but a lot of companies are really interested in it because of the amount of work that could be cut out like you said.

folkhack 7y ago
> It's finicky technology that is hard to make sustainable This 100 times over. The only way I've been able to make it work is being one heck of a generalist. I've found that training on what I do is near-impossible and I don't even know if I could scale a real company in this beyond myself.
- virgilp 7y ago
  Well, UiPath's valuation is currently 7B... But yeah. They have an "academy" for training RPA developers, forums/ social presence, big sales teams, a process for developing automation, an "automation framework", a marketplace for automation activities, etc.... all these are stuff outside of the core software (robot/"activity executor", orchestrator, and studio for designing said activities). Maintaining & expanding the automation is a heck of a challenge though. Like, once you got the stuff deployed, make sure that it works without headaches, in the face of other software/ OS upgrades. The robots/automation services are essentially a "fleet of microservices" that you have to maintain for a customer whose strength is likely not IT, and who will change the infrastructure from underneath your "microservices". It's a hard problem, but I feel we're well positioned to tackle that, and if we do, the sky is the limit really. (disclaimer: I work for UiPath, though that should be obvious from the message)
  - folkhack 7y ago
    Right on! Yeah - I would love to learn more about what the bigger guys are doing. Everything you're saying is 100% something I deal with on a day-to-day basis, and ongoing support is a huge struggle to workout with the companies I've implemented software at. Looking at your product offerings made me chuckle - I definitely have versions of a lot of what you do! My robots are called workers, my orchestrator is called a broker, etc etc ha. Lemme know if you guys are ever looking for a remote automation engineer w/tons of cloud experience ;)

charliemil4 7y ago

Any advice for companies getting into this realm, specifically around making sure you get your 'valued' price? Charging for value can sometimes feel exorbitant... i.e. saving a $100+/hr person ~20 hours a month. Sure I can calculate out it on an excel sheet the customer ROI, but still feels crazy.

folkhack 7y ago
My market value is $10-15k/mo USD for just myself (sometimes more if I'm needed in an emergency). If you have a proven track record with a solid reputation you can command that sort of pay without an issue. I also bring a lot of general web development capability to the table, so selling on an easy-to-use interface also does a lot to add to my value. If you think about it - $35-40k for a 2-3 month dev timeline (which is what I'd say is my average) is worth it if you're cutting the salaries of 5+ people out of the equation, no need to pull out an Excel spreadsheet for the ROI that. All-in-all - you're not crazy, they're the ones who are crazy if they continue to sink money into human robots. > Charging for value can sometimes feel exorbitant Just get confident (this is hard advice I know). I'm incredibly confident in the problems I can solve and will put in the hours to get back on a timeline/budget if needed. Businesses aren't buying my "products" per-say, they're buying me.
- charliemil4 7y ago
  I owe you a round for this. Hopefully it sticks.
- jcmontx 7y ago
  Solid comment. How do you deal with getting the knowledge of the process out of the people you are trying to leave without a job? That must be rough.
  - folkhack 7y ago
    They're entry level positions usually so figuring out their processes is typically not that difficult. Often I will not work directly with them vs. having a big sheet of instructions delivered with a ton of bullet points. I also get a ton of process diagrams as well. Usually it's better for the business I'm supporting to document their own workflows before it's handed over to me for a lot of common sense reasons.

WhompingWindows 7y ago

I was working for a scientific journal. In the peer-review department, they had to use this proprietary software, and we had 2 employees who just mindlessly went through this proprietary software's form day-in, day-out. This form spammed scientists for reviews of science articles for our somewhat-predatory journal (we solicited authors to publish in our journal for a fee)... First, it was a massive waste of those 2 human's intellect, entering data over and over into a generic form from a proprietary business' website, in order to spam scientists. I started a ruby project to automate out all those processes, just using selenium to enter the data onto the web form for us. Before I could get very far, I got a huge promotion elsewhere and left that under-compensating journal. It all gave me a depressing vibe though, both the ethics of what we were doing and the waste of time/life that those employees had to endure to do it.

dev_dull 7y ago
> First, it was a massive waste of those 2 human's intellect This is usually the reality and it's nothing to be ashamed about. At work, do what you're paid to do. In your free time, pursue interests (literally, things that are interesting to you).
NotSammyHagar 7y ago
When those jobs are automated, what will the human workers do, will they leave that company? I'd enjoy doing something else if that was my job, but if there was a more useful job there, wouldn't they already be doing it? This is in summary why I think more automation will lead to some people losing their jobs - if there was a more profitable alternative for a company (more productive, paid as well or better?) then wouldn't the company already have automated it?

koonsolo 7y ago

I worked in the Automated Guided Vehicles industry, and so I want to extend your suggestion: any automation that can replace employees. Those systems are easy to sell with "This system will pay itself back in 2,5 years, taking into account your current labour cost". Given all the AI/deep learning possibilities, lots can be automated still.

mcrad 7y ago

Do you find it difficult to sell the idea of automating the work of "entry-level data employees"? Middle management generally thrives by building kingdom of such minions.

folkhack 7y ago
> Middle management generally thrives by building kingdom of such minions. I often am not selling to middle management, I ideally sell at the exec and/or product level. Middle managers are a conflict of interest when it comes to what I do.
draw_down 7y ago
There exist more powerful forces in the world than "middle management".

caseydonahue 7y ago

Can you give some more insight into what you work on automating specifically?

folkhack 7y ago
Anything - green screens, Excel spreadsheets, PDF generation, Docusign, tying 6 web experiences with separate logins into one, simplification of complex auth scenarios, etc. Most of my day is literally just getting one step further and one step further into a "process" of some sort, and then handling a metric boatload of edge-cases as they come up. Then the last portion of the contract is the actual integration into a business, and handling the "ninety-ninety" rule that always ends up happening when the rubber meets the road. I take a complex, convoluted mess and turn it into a coherent "something", usually an API or web experience.
- cgsmith 7y ago
  That is essentially what I do. I pitch it as process engineering, systems integrator, programmer, etc.
- jcconnell 7y ago
  I'm curious how you market yourself or your services? I automate processes, workflows and tasks like this for some clients and I really enjoy it. It's easy to see the value I provide and the clients are happy. However I struggle with how to acquire new/more clients. The ones I have now I've acquired through past business relationships. I'd love to be able to do more of this work.

dev_dull 7y ago

The key to these types of solutions is always to follow the 80/20 rule. For example, it’s foolish to say and attempt “I will automate all of your data entry people” because there will always be weird idiosyncrasies that are better fit for humans. You will dash yourself against the rocks trying to get a “100% complete” solution. Instead, create software that allows 1 person to do the job of 5. You create massive business value without getting sucked down the hole of edge cases.

nextos 7y ago
True. In the same spirit, I'd also add record linkage to automated data entry. It's a big problem both if your previous data is noisy or your automated solution doesn't transcribe some fields correctly. A simple probabilistic programming solution can work really well.
alexcnwy 7y ago
I’m building a tool to speed up data entry. It highlights OCR‘d words with low confidence for human review. It uses templates for capturing structured data and a fast interface for capturing unstructured data. It grew out of consulting projects and is almost ready for beta testing if anyone is interested in playing with it and giving me some feedback - my email is in my profile :)
- tomericco 7y ago
  Email sent.
codeisawesome 7y ago
How do you guys network with the right business people to speak their language and find out how their processes work? You need that info (a problem description) before you come out with a solution, don’t you?
- laughinghan 7y ago
  Just ask. People absolutely love talking about their problems, it'll probably be the most animated thing they have to say about the business they're in. "Some people profess difficulty at finding applications to write. I have never understood this: talk to people. People have problems — lots of problems, more than you could enumerate in a hundred lifetimes. Talk to a carpenter, ask him what about carpentry sucks. Talk to the receptionist at your dentist’s office — ask her what about her job sucks. Talk to a teacher — ask her what she spends time that she thinks adds the least value to her day." https://www.kalzumeus.com/2010/03/20/running-a-software-business-on-5-hours-a-week/ https://www.kalzumeus.com/2010/03/20/running-a-software-busi...

virgilp 7y ago

Isn’t this the “robotic process automation” market? (though that one is a bit more than what you are saying)

wodenokoto 7y ago

Isn't this what for a short while was called robotics, but now is mostly referred to as RPA? It sounds like you are competing with things like Blueprism. These things are really a big deal within large organisations.

folkhack 7y ago
Yep! I'm effectively a boutique provider of such services.

odonnellryan 7y ago

This is what I do for work (mostly) as well. The challenge I've had trying to grow the business is "convincing" (for lack of a better word) businesses that they will really save a lot - even after showing them case-studies. Companies can be stubborn. Hit me up if you ever want to chat! ryan luma.im

surbekznrlsnt 7y ago

I'm living in Ireland and this is what I've wanted to do for years. There are loads of older small to medium businesses with SO many things that can be automated but I don't know how to get into it. I don't even know how to research it. What's the process for getting a client? Cold calling? Partnership with another provider? What is the job title I should search for more information? Automator?

g0bananas 7y ago
You should look at business operations analyst, UI Path training is on linkedin learning/lynda.com too. I think information management systems too?
ploika 7y ago
A handful of places around Dublin have active Robotic Process Automation (RPA) teams, which seems to be what's being discussed here. The Big Four consultancies and Accenture spring to mind. I think the likes of Fidelity and Citi have few roles too. Note however that you'd be using off the shelf tools to do the automating - you wouldn't be developing the automation tools yourself.
Rooster61 7y ago
Automation Engineer. It's a broad term, but does carry the idea that you know how to program, and do so in a way that automates normally manual processes. Also doesn't hurt to list yourself on a resume as an Automation/Software Engineer. That gets the idea across that you also understand how software is written, not just how code is cranked out.

BubRoss 7y ago

How do you find people who need these things?

mstade 7y ago

What’s your business model? Consulting by the hour? Subscription fees? One-off fixed price contracts? Practically all software I’ve ever done has been some amalgamation or other of “make forms better and more automatic”, whether it’s planning software for gigantic Swedish furniture firms (take a wild guess!) or technical analysis toolkits for financial services. It’s always been per-hour or fixed price contracts for me – try as I might but the subscription model simply doesn’t take with the buyers it seems. What I’d like to do is fixed price for initial dev and then subscription based contract for support and minor updates, but this seems wholly unpalatable for buyers.

jakemal 7y ago

Can you share what you're working on? I would love to see a concrete example of something you are referring to as well as how you market it.

pacoWebConsult 7y ago

I began working in RPA this year and I completely agree. Tremendous potential but also a lot of buzzword-driven hype in the space. I think the main players need to work harder on bringing world-class engineering and best-practices to the community, since the vast majority of developers that are getting into this are not really advancing the field and instead slapping crap together until their 'bot' works. Why are most automation projects relegated to bare-bones budgets and short timeframes? Potential clients i have engaged have a very short-sighted view of automation possibilities in their businesses, and aren't open to the possibilities beyond what they think should be done by a bot instead of their employees.

scruple 7y ago

Currently automating a lot of on-boarding process at work. It's been eye opening. What it's really shown me is how very little expertise we have around all of these solutions, SaaS products, etc... that the company uses. From Tableau to Salesforce to our accounting software, it's a mess. It's put myself and my team into a unique position to gain a lot of mind share, especially when it comes to cleaning a lot of the inefficiencies up. That said, I don't think we're making anyone redundant, or cutting anyone out of the equation in a negative way -- none of the people who are currently tasked with a lot of this work were ever supposed to be doing it, trained on it, etc...

rkwasny 7y ago

That's exactly what we do, with a nice interface business people can use, email me at rafal@evolution.ai if you want a demo.

bbleciel 7y ago

I seriously want to begin working in this field as a consultant/architect/developer. So much so that I've registered the LLC, opened a bank account, and even had a few clients I built web apps for. However, I'm not exactly sure how to pivot to business solutions/productivity tools though. Do you usually approach companies where you expect to have a solution for them? Or do you find clients through advertising? Do they find you? Any advice is appreciated!

modwest 7y ago

Does your company, or do any in the field, support policies for retraining/job placement for employees displaced by their technology?

alentodorov 7y ago

I think UIPath recent success shows that the market already values this.

hnick 7y ago

That's funny, the multi-national company I work for is removing all MS Office automation because MS apparently told us that unattended automation is against the licensing terms. It's OK if you're watching the computer I guess, but not on a server. Personally this didn't match what I found online (they said it was not supported i.e. if it breaks tough luck) but what would I know. Money must be spent retrofitting other solutions now. I joked we should pay a minimum wage high school kid to sit in a room and 'watch' the servers but I guess that's too low tech ;)

folkhack 7y ago
They have specific solutions that are integrated into things like SharePoint to handle this so they're just putting the hurt on the big guys in order to get more monies. Total Microsoft play on that for sure - I've never ran into the same issue though.
- hnick 7y ago
  Yeah I looked into this, and it's definitely overkill for what we wanted (plus sending them more money to fix a problem they made for us would be very grating). All we needed to do is read/write Excel data and similar functions and keeping it local is key. There wasn't much available when this got put in place but there are plenty of reliable libraries now.

johnisgood 7y ago

I have been working on automating data entry, and the consequences of this is many people (in this case Indians) losing their jobs. I am facing a moral dilemma. I have been writing free software for my entire life, I volunteer, but I live in really poor conditions. I want to get forward somehow, and this seems to be my best chance of getting forward at this point in time and space. I am probably not going to go through with it. It seems to be the case that my principles, my moral values have been holding me back from improving my life. I do not want to lie to people, I do not want to deceive people, I do not want to fsck them over in any way. I do not want to contribute to anything nefarious. But from where I am (the bottom), it is extremely hard to climb up without doing that. :/

jacobwilliamroy 7y ago

Your post brought back fond memories of combining autohotkey with premiere pro to automate boring tasks. Haha. I agree, you have a nice niche.

zelly 7y ago

Automating mouse clicks and key presses seems backward and stupid. Doing this manually will also get disrupted. You should trace the executable and analyze the function calls. Get a million samples then form a statistical model. The production model replays the function calls given some input. Doesn't need graphical memory anymore and it also has higher resolution inputs (function symbols, addresses) than coordinates on a screen.