3 ms·

I come from the opposite side of things to you. I'll provide my perspective. I work for a large industrial plant. My background is in Materials Engineering and

I come from the opposite side of things to you. I'll provide my perspective. I work for a large industrial plant. My background is in Materials Engineering and Metallurgy. My current job title is "Data Management Engineer" which is pretty unhelpful. If you are familiar with CIM levels in manufacturing I work across level 2-4 (from PLCs's and process automation right up to enterprise historians and Databases) mostly I focus on higher level operational databases and data organization but I occasionally do process modelling and other automation work. I'm not sure how my company compares to the typical retail customers BI companies like to target but hopefully this is useful. Rather than real time access I think Automation would be be the most desirable outcome. The Overworked Analyst you refer to above is definitely something I relate to in our company. If we can free up analysts time from all the mundane data extraction tasks they can add value by actually 'analysing' - not 'extracting'. The same applies whether this is financial data (accountants) or production data(engineers) being analysed. The problem is an industrial plant is not a clean homogeneous operation like a retail store. Equipment breaks down, new equipment gets installed parts of the production line can be decommissioned routed around etc. Our business rules are dynamic constantly changing. A lot of the off the shelf products BI guys pitch to us almost inevitably be out of date the moment it was commissioned. I've not really seen any solution that wouldn't require constant tinkering to keep it up to date and often that requires domain knowledge. Engineers often don't have the software skills and software people don't have the scientific/engineering background. We've had some absolute horror stories with out-sourcing work to contractors without domain knowledge. Even simple stuff can cause complications. One amusing example I can think of is getting back a program full of bugs because the contractor confused the name of a database field "Fe2O3" (Iron Oxide, Fe two 'oh' three) with Fe203 (Fe - Two hundred and Three).

greggyb 11y ago
> ... clean homogeneous operation like a retail store ... If you'll forgive me starting with a quote-snip, I think you very much underestimate the complexities of a retail operation. Let me just say that we've had multiple day discussions just about dates with one of our retail clients, specifically how they should be compared year-over-year. In manufacturing, I'm sure you can also understand some of the challenges that exist around inventory. Moving beyond that to the meat of your post, I am familiar with manufacturing. That's one of the major verticals we cover (several of our biggest clients are names you would recognize), and I came up as an analyst in a manufacturing organization. We are a true consultancy. We don't sell products. We're a Microsoft partner, so by the time we're involved the product decision is pretty much made. Our tools are SQL Server, SSAS, and a collection of tools in the presentation layer. None of these are off-the-shelf BI products, but tools to build a BI infrastructure. Our selling point is being able to transform the knowledge and subject matter expertise of people like you into robust models that are suitable for data exploration, ad-hoc reporting, and structured reporting. Our work is bespoke, if you will. You're definitely right that BI is a moving target. If anyone tries to sell you on a build it and leave it solution, they're full of it. Like you said business rules change. A lot of the time (in a well-designed BI solution), these can be captured as data and incorporated into the architecture seamlessly. Sometimes business rule changes do require re-architecting portions of the solution. One of our more recent projects where I was tangentially involved was implementing work-in-progress reporting for one of our manufacturing clients. I can tell you that the equipment turnover and production line rerouting problems are all well-covered. They can spin up and down production lines, factories, reroute product between steps and that's all covered in the existing architecture. Even in less dynamic environments, it's hard to call a BI solution done. If the processes and data don't change, that's just an opportunity for the sophistication of the questions and analysis to progress. I don't say this from a position of greed and job security (though the security is nice, for sure (: ), but from one of honest observation. Once you know what has happened and is happening, you start asking why and what will happen, and once you can answer those, you ask how to change what will happen. This latter portion is where the data science of the parent article comes in.
- bigger_cheese 11y ago
  Yes I agree with everything you've said here. I was probably a bit to simplistic with my analogy. Most of the pitches we've had are from companies wanting to sell us their BI solution. I think your company's 'bespoke' approach is the best method. Ideally I think you'd get the best value by embedding 'experts' in the organisation not sure how possible that is.
  - greggyb 11y ago
    Depending on engagement, we'll have consultants on client assignments for multiple years straight. We strive to maintain continuity in project resources. It's difficult when we don't have continuous engagements with a client, but it certainly helps to have someone who understands the business and its unique aspects. Even across projects, we generally tend to keep consultants within a major vertical, so we do develop some subject matter understanding (I would hesitate to call us experts in most subject matter), which certainly aids in ramping up on new projects quickly. Beyond manufacturing, I personally have had a number of engagements with healthcare clients. You should never ask me for medical advice, but I'll most likely understand a problem statement regarding medical data faster than you will. So, we don't necessarily have in house experts in our clients' businesses, but we do have people who understand the common problems in a vertical.
- elcano 11y ago
  I have worked for world leading companies in both scenarios for many years, and can tell you that the dimensionality, caos and scale of the manufacturing industry is difficult to match. Actually, depending on the place, it is difficult to comprehend, even for those who have been working inside that industry for just a few years. Retail's challenge most of the time is the lack of resources on those situations when the business is a low margin one or they just don't want to invest. Dealing with unreasonable and incompetent external customers is frustrating too. Receiving incompatible POS data from each customer, realigning sales organizations or cleaning products hierarchy, and even, getting the numbers from sales and finance to match is a pain. But the amount of processes is about one or two orders of magnitude smaller. In addition, the amount of data collected is huge (Plant Historians alone collect 200k to a 1M values every second), and your KPIs can be as diverse as the many internal and external regulatory agencies that oversee your processes. Consider just a few of the systems in addition to PI Historian: ERP (you know this one), Manufacturing Execution System, Laboratory Information Management System, Laboratory Electronic Notebook, Change Control System, Non-Conformance/Corrective Action System, Process Control System, Building Management System, Preventive Maintenance System, Learning Management System, Raw Materials Information System, Regulatory Submission Information System....and those are only the big ones. When the plant is in a highly regulated business like pharma, the amount of bureaucracy needed to change a single parameter in a process control system is disheartening. FDA specifically, is mandating pharmaceutical industry to perform process monitoring to prove that the processes remain stable. In these industries we do multivariate Statistical Process Control, not because of the big data trend, but because we have been able to save multi-million dollars batches by observing weak multivariate signals that would pass unnoticed in a typical, retail oriented OLAP cube.

greggyb 11y ago

> ... clean homogeneous operation like a retail store ... If you'll forgive me starting with a quote-snip, I think you very much underestimate the complexities of a retail operation. Let me just say that we've had multiple day discussions just about dates with one of our retail clients, specifically how they should be compared year-over-year. In manufacturing, I'm sure you can also understand some of the challenges that exist around inventory. Moving beyond that to the meat of your post, I am familiar with manufacturing. That's one of the major verticals we cover (several of our biggest clients are names you would recognize), and I came up as an analyst in a manufacturing organization. We are a true consultancy. We don't sell products. We're a Microsoft partner, so by the time we're involved the product decision is pretty much made. Our tools are SQL Server, SSAS, and a collection of tools in the presentation layer. None of these are off-the-shelf BI products, but tools to build a BI infrastructure. Our selling point is being able to transform the knowledge and subject matter expertise of people like you into robust models that are suitable for data exploration, ad-hoc reporting, and structured reporting. Our work is bespoke, if you will. You're definitely right that BI is a moving target. If anyone tries to sell you on a build it and leave it solution, they're full of it. Like you said business rules change. A lot of the time (in a well-designed BI solution), these can be captured as data and incorporated into the architecture seamlessly. Sometimes business rule changes do require re-architecting portions of the solution. One of our more recent projects where I was tangentially involved was implementing work-in-progress reporting for one of our manufacturing clients. I can tell you that the equipment turnover and production line rerouting problems are all well-covered. They can spin up and down production lines, factories, reroute product between steps and that's all covered in the existing architecture. Even in less dynamic environments, it's hard to call a BI solution done. If the processes and data don't change, that's just an opportunity for the sophistication of the questions and analysis to progress. I don't say this from a position of greed and job security (though the security is nice, for sure (: ), but from one of honest observation. Once you know what has happened and is happening, you start asking why and what will happen, and once you can answer those, you ask how to change what will happen. This latter portion is where the data science of the parent article comes in.

bigger_cheese 11y ago
Yes I agree with everything you've said here. I was probably a bit to simplistic with my analogy. Most of the pitches we've had are from companies wanting to sell us their BI solution. I think your company's 'bespoke' approach is the best method. Ideally I think you'd get the best value by embedding 'experts' in the organisation not sure how possible that is.
- greggyb 11y ago
  Depending on engagement, we'll have consultants on client assignments for multiple years straight. We strive to maintain continuity in project resources. It's difficult when we don't have continuous engagements with a client, but it certainly helps to have someone who understands the business and its unique aspects. Even across projects, we generally tend to keep consultants within a major vertical, so we do develop some subject matter understanding (I would hesitate to call us experts in most subject matter), which certainly aids in ramping up on new projects quickly. Beyond manufacturing, I personally have had a number of engagements with healthcare clients. You should never ask me for medical advice, but I'll most likely understand a problem statement regarding medical data faster than you will. So, we don't necessarily have in house experts in our clients' businesses, but we do have people who understand the common problems in a vertical.
elcano 11y ago
I have worked for world leading companies in both scenarios for many years, and can tell you that the dimensionality, caos and scale of the manufacturing industry is difficult to match. Actually, depending on the place, it is difficult to comprehend, even for those who have been working inside that industry for just a few years. Retail's challenge most of the time is the lack of resources on those situations when the business is a low margin one or they just don't want to invest. Dealing with unreasonable and incompetent external customers is frustrating too. Receiving incompatible POS data from each customer, realigning sales organizations or cleaning products hierarchy, and even, getting the numbers from sales and finance to match is a pain. But the amount of processes is about one or two orders of magnitude smaller. In addition, the amount of data collected is huge (Plant Historians alone collect 200k to a 1M values every second), and your KPIs can be as diverse as the many internal and external regulatory agencies that oversee your processes. Consider just a few of the systems in addition to PI Historian: ERP (you know this one), Manufacturing Execution System, Laboratory Information Management System, Laboratory Electronic Notebook, Change Control System, Non-Conformance/Corrective Action System, Process Control System, Building Management System, Preventive Maintenance System, Learning Management System, Raw Materials Information System, Regulatory Submission Information System....and those are only the big ones. When the plant is in a highly regulated business like pharma, the amount of bureaucracy needed to change a single parameter in a process control system is disheartening. FDA specifically, is mandating pharmaceutical industry to perform process monitoring to prove that the processes remain stable. In these industries we do multivariate Statistical Process Control, not because of the big data trend, but because we have been able to save multi-million dollars batches by observing weak multivariate signals that would pass unnoticed in a typical, retail oriented OLAP cube.