2 ms·

Any concrete examples you could share? This sounds like an absolutely fascinating take on the matter.

grogenaut 8y ago
Okay thought of an example I can give, this is one of the 1-3 page writeups. We had a new product we were rolling out a few years back. One big worry long term was a fraud potential. We were essentially paying contractors to insert content into audio. Long term on this product it could be a problem that people were defrauding us, but we didn't know for sure and we didn't know if we could do it at massive scale due to fraud risk. The product was being piloted and didn't think it'd launch for 1-2 years at scale. This was a big business worry, and the team didn't have time to look at it. At the same time google and amazon released their audio to text apis. I took a few sample audio clips, ran it through these systems and got some basic detection numbers. I ran the cost projections on it, and diagrammed a basic system diagram. I also pointed out limitations and strengths and gave a basic level of effort to implement it. I also compared it to manual verification and tossed in some stats coverage needs to detect bad actors from our data scientists. I also took my questions to our ML team and got their expert opinion. This let us a) quantify the risk and solution possibilities, b) push back worrying about the fraud as we had a likely fix at scale, c) described to everyone that basic solution so we could discuss it, d) discovered we had some experts at the company who had experience in this exact area. Sure this might be more a "research" paper or a "feasibility study" call it what you will. it's the same format as more filled out arch documents. this took me about 4 hours coding and 45 minutes to write up. Engineers were like "cool" and managers were happy to know the risk assessments and solutions and costs. Given I did the fraud math for them they were able to make a strong business case for ignoring the risk until later. some people call this removing risk, others call it resolving ambiguity. whatever, it's just digging into something and communicating effectively.

Okay thought of an example I can give, this is one of the 1-3 page writeups. We had a new product we were rolling out a few years back. One big worry long term was a fraud potential. We were essentially paying contractors to insert content into audio. Long term on this product it could be a problem that people were defrauding us, but we didn't know for sure and we didn't know if we could do it at massive scale due to fraud risk. The product was being piloted and didn't think it'd launch for 1-2 years at scale. This was a big business worry, and the team didn't have time to look at it. At the same time google and amazon released their audio to text apis. I took a few sample audio clips, ran it through these systems and got some basic detection numbers. I ran the cost projections on it, and diagrammed a basic system diagram. I also pointed out limitations and strengths and gave a basic level of effort to implement it. I also compared it to manual verification and tossed in some stats coverage needs to detect bad actors from our data scientists. I also took my questions to our ML team and got their expert opinion. This let us a) quantify the risk and solution possibilities, b) push back worrying about the fraud as we had a likely fix at scale, c) described to everyone that basic solution so we could discuss it, d) discovered we had some experts at the company who had experience in this exact area. Sure this might be more a "research" paper or a "feasibility study" call it what you will. it's the same format as more filled out arch documents. this took me about 4 hours coding and 45 minutes to write up. Engineers were like "cool" and managers were happy to know the risk assessments and solutions and costs. Given I did the fraud math for them they were able to make a strong business case for ignoring the risk until later. some people call this removing risk, others call it resolving ambiguity. whatever, it's just digging into something and communicating effectively.