5 ms·

Ask HN: Automating large Enterprise business operations

4 points by automator 12y ago

Hi HN. I'd like some advice about automating business operations - what to consider, what technologies or framework to look out for etc. For background, I'm working for an ICT service provider, and just got promoted to look into enterprise-wide automation and system. My team is small, but I report directly to the C-level. Frankly, I'm a hobbyist programmer, and was only promoted to this post due to my enthusiasm and small track records i.e. handling CSV files for analysis etc. The ultimate goals are to improve lead time in service delivery, reduce aging tickets, and reduce cost. I currently see gap in performance management, where the bosses do not have clear visibility or simply do not trust the performance metrics. Simultaneously, there are lean internal consultants doing studies and making recommendations. Also, middle managers are being trained in Six Sigma, acquiring "belts". To reiterate, how do I start assessing the situation and what kind of technologies or methodologies associated with this scenario? Is it SOA, workflow engine, ERP etc? How feasible is using standard web framework i.e. Django or Flask for business automation? Thanks.

SEJeff 12y ago
This is a LOT to read up on, but seriously consider mesos + aurora. It is really incredible stuff and Twitter uses it for huge production environment. The Mesosphere guys have better documentation with Marathon, but it has a lot of catching up to get to the feature set of Aurora. I happen to be the tech lead of a team that builds internal tools for an "enterprise" company primarily using Django and Flask. Build it and then show the business how it is useful. Sadly, that means you might need to write a lot of high quality docs or high quality clients (if you write an api). I tend to favor "api first" development.
- automator 12y ago
  who ended up maintaining the internal tools? do you hand it off to the IT team?
  - SEJeff 12y ago
    My team, who writes those tools for the 'IT Team', but really the entire company ends up consuming them. Our company's bread and butter is low latency stuff and we're a small team of webdevs who exist in the IT side of the company, but code fulltime. I quite enjoy it actually.
mindcrime 12y ago
That's a pretty big, open question. There's a ton of literature out there that you could (and probably should) read, and no simple HN post can do justice to the scope of process automation / BPM / workflow, etc. For some background on business process / workflow automation, a few good books to review would include: http://www.amazon.com/Essential-Business-Process-Modeling-Michael/dp/0596008430/ref=sr_1_31?s=books&ie=UTF8&qid=1419303981&sr=1-31&keywords=bpm http://www.amazon.com/Essential-Business-Process-Modeling-Mi... http://www.amazon.com/Fundamentals-Business-Process-Management-Marlon/dp/3642331424/ref=sr_1_34?s=books&ie=UTF8&qid=1419304137&sr=1-34&keywords=workflow+automation http://www.amazon.com/Fundamentals-Business-Process-Manageme... http://www.amazon.com/Smart-Enough-Systems-Competitive-Automating/dp/0132347962/ref=sr_1_1?s=books&ie=UTF8&qid=1419304165&sr=1-1&keywords=smart+enough+systems http://www.amazon.com/Smart-Enough-Systems-Competitive-Autom... http://www.amazon.com/Social-BPM-Bpm-Workflow-Handbook/dp/1461146305/ref=sr_1_1?s=books&ie=UTF8&qid=1419300569&sr=1-1&keywords=social+bpm http://www.amazon.com/Social-BPM-Bpm-Workflow-Handbook/dp/14... http://www.amazon.com/iBPMS-Intelligent-Systems-Workflow-Handbook/dp/0984976469/ref=sr_1_2?s=books&ie=UTF8&qid=1419300569&sr=1-2&keywords=social+bpm http://www.amazon.com/iBPMS-Intelligent-Systems-Workflow-Han... http://www.amazon.com/Real-Life-BPMN-Analyze-Automate-Processes/dp/1480034983/ref=pd_sim_b_8?ie=UTF8&refRID=1YW0HA39FP97W6RBE47T http://www.amazon.com/Real-Life-BPMN-Analyze-Automate-Proces... That said... for starters, I'd back up to the issue of truly understanding the business need you're trying to address, and figure out a methodology for mapping the technological capabilities you will implement, to the business strategy. To that end, I'm a fan of a technique called "capability cases", and actually wrote a series of blog posts on that topic a while back. I'll refer you there for more on that topic: http://fogbeam.blogspot.com/2013/01/why-capability-cases-are-must-when.html http://fogbeam.blogspot.com/2013/01/why-capability-cases-are... http://fogbeam.blogspot.com/2013/02/so-what-is-capability-case-anyway.html http://fogbeam.blogspot.com/2013/02/so-what-is-capability-ca... http://fogbeam.blogspot.com/2013/05/capability-cases-part-three-sample.html http://fogbeam.blogspot.com/2013/05/capability-cases-part-th... The other positive thing about the Capability Cases methodology is that it helps IT people learn to "speak the language" of LOB executives (and, to some extent, vice versa, if everybody buys in), and helps bridge that gap between "the business side of things" and "the IT side of things". Anyway, if you get through all that and decide that some sort of BPM system is for you, there are a handful of high quality, popular, Open Source BPM engines out there that you could consider. If nothing else, they could enable you to get to a good POC quickly and without spending a bazillion dollars. One popular option is Activiti: http://www.activiti.org http://www.activiti.org It uses the industry standing BPMN2 notation. Stay away from anything that mentions BPEL, and stay away from Oracle products. Especially Oracle products that involve BPEL. Depending on how complex your needs are, you might not actually need a dedicated BPM / workflow engine. In that case, you could certainly code up automation code in any general purpose language of choice. But be careful of going down the path where you wind up rebuilding the wheel. I'd also advocate taking a careful look at the idea of "Social BPM", which focuses on making BPM systems more flexible and less rigid, and introducing aspects involving social connections between humans. I'm not trying to be overly self-promotional here, but I will refer you again to a couple of posts I have previously written on this subject, which may be of use to you: http://fogbeam.blogspot.com/2013/05/social-events-bpm-oh-my-but-what-about.html http://fogbeam.blogspot.com/2013/05/social-events-bpm-oh-my-... http://fogbeam.blogspot.com/2014/02/on-solving-social-aspect-of-bpm.html http://fogbeam.blogspot.com/2014/02/on-solving-social-aspect... Like I said, this is a big topic. I could talk about this stuff for days on end, or until you died from boredom. :-) Seriously though, if I can be of any further assistance, feel free to drop me an email.

4 points by automator 12y ago

SEJeff 12y ago

This is a LOT to read up on, but seriously consider mesos + aurora. It is really incredible stuff and Twitter uses it for huge production environment. The Mesosphere guys have better documentation with Marathon, but it has a lot of catching up to get to the feature set of Aurora. I happen to be the tech lead of a team that builds internal tools for an "enterprise" company primarily using Django and Flask. Build it and then show the business how it is useful. Sadly, that means you might need to write a lot of high quality docs or high quality clients (if you write an api). I tend to favor "api first" development.

automator 12y ago
who ended up maintaining the internal tools? do you hand it off to the IT team?
- SEJeff 12y ago
  My team, who writes those tools for the 'IT Team', but really the entire company ends up consuming them. Our company's bread and butter is low latency stuff and we're a small team of webdevs who exist in the IT side of the company, but code fulltime. I quite enjoy it actually.

mindcrime 12y ago

That's a pretty big, open question. There's a ton of literature out there that you could (and probably should) read, and no simple HN post can do justice to the scope of process automation / BPM / workflow, etc. For some background on business process / workflow automation, a few good books to review would include: http://www.amazon.com/Essential-Business-Process-Modeling-Michael/dp/0596008430/ref=sr_1_31?s=books&ie=UTF8&qid=1419303981&sr=1-31&keywords=bpm http://www.amazon.com/Essential-Business-Process-Modeling-Mi... http://www.amazon.com/Fundamentals-Business-Process-Management-Marlon/dp/3642331424/ref=sr_1_34?s=books&ie=UTF8&qid=1419304137&sr=1-34&keywords=workflow+automation http://www.amazon.com/Fundamentals-Business-Process-Manageme... http://www.amazon.com/Smart-Enough-Systems-Competitive-Automating/dp/0132347962/ref=sr_1_1?s=books&ie=UTF8&qid=1419304165&sr=1-1&keywords=smart+enough+systems http://www.amazon.com/Smart-Enough-Systems-Competitive-Autom... http://www.amazon.com/Social-BPM-Bpm-Workflow-Handbook/dp/1461146305/ref=sr_1_1?s=books&ie=UTF8&qid=1419300569&sr=1-1&keywords=social+bpm http://www.amazon.com/Social-BPM-Bpm-Workflow-Handbook/dp/14... http://www.amazon.com/iBPMS-Intelligent-Systems-Workflow-Handbook/dp/0984976469/ref=sr_1_2?s=books&ie=UTF8&qid=1419300569&sr=1-2&keywords=social+bpm http://www.amazon.com/iBPMS-Intelligent-Systems-Workflow-Han... http://www.amazon.com/Real-Life-BPMN-Analyze-Automate-Processes/dp/1480034983/ref=pd_sim_b_8?ie=UTF8&refRID=1YW0HA39FP97W6RBE47T http://www.amazon.com/Real-Life-BPMN-Analyze-Automate-Proces... That said... for starters, I'd back up to the issue of truly understanding the business need you're trying to address, and figure out a methodology for mapping the technological capabilities you will implement, to the business strategy. To that end, I'm a fan of a technique called "capability cases", and actually wrote a series of blog posts on that topic a while back. I'll refer you there for more on that topic: http://fogbeam.blogspot.com/2013/01/why-capability-cases-are-must-when.html http://fogbeam.blogspot.com/2013/01/why-capability-cases-are... http://fogbeam.blogspot.com/2013/02/so-what-is-capability-case-anyway.html http://fogbeam.blogspot.com/2013/02/so-what-is-capability-ca... http://fogbeam.blogspot.com/2013/05/capability-cases-part-three-sample.html http://fogbeam.blogspot.com/2013/05/capability-cases-part-th... The other positive thing about the Capability Cases methodology is that it helps IT people learn to "speak the language" of LOB executives (and, to some extent, vice versa, if everybody buys in), and helps bridge that gap between "the business side of things" and "the IT side of things". Anyway, if you get through all that and decide that some sort of BPM system is for you, there are a handful of high quality, popular, Open Source BPM engines out there that you could consider. If nothing else, they could enable you to get to a good POC quickly and without spending a bazillion dollars. One popular option is Activiti: http://www.activiti.org http://www.activiti.org It uses the industry standing BPMN2 notation. Stay away from anything that mentions BPEL, and stay away from Oracle products. Especially Oracle products that involve BPEL. Depending on how complex your needs are, you might not actually need a dedicated BPM / workflow engine. In that case, you could certainly code up automation code in any general purpose language of choice. But be careful of going down the path where you wind up rebuilding the wheel. I'd also advocate taking a careful look at the idea of "Social BPM", which focuses on making BPM systems more flexible and less rigid, and introducing aspects involving social connections between humans. I'm not trying to be overly self-promotional here, but I will refer you again to a couple of posts I have previously written on this subject, which may be of use to you: http://fogbeam.blogspot.com/2013/05/social-events-bpm-oh-my-but-what-about.html http://fogbeam.blogspot.com/2013/05/social-events-bpm-oh-my-... http://fogbeam.blogspot.com/2014/02/on-solving-social-aspect-of-bpm.html http://fogbeam.blogspot.com/2014/02/on-solving-social-aspect... Like I said, this is a big topic. I could talk about this stuff for days on end, or until you died from boredom. :-) Seriously though, if I can be of any further assistance, feel free to drop me an email.