3 ms·

What resources do you suggest for learning how to do this kind of risk analysis well?

by yaj54 3y ago

vasco 3y ago
A crystal ball. Most of the times you should do what they did, build a first version in a week and see. Nobody knows the future anyway.
globalise83 3y ago
You should look into decision analysis with decision trees - this is a nice article on how to do it: https://hbr.org/1964/07/decision-trees-for-decision-making https://hbr.org/1964/07/decision-trees-for-decision-making
barfbagginus 3y ago
My analysis of the platform risk of the ChatGPT plug-in store depends on 1) risk ideas like Failure Modes and risk hedging. The outcome in the plug-in store is an example of a well known failure mode of platforms that grow very fast. 2) remaining up to date with trends in the llm literature, so we can for instance predict that major vendors would release multimodal llms 3) being aware of trends in tech platform dynamics. Specifically the trend for "platform enshittification" - a term coined by Corey Doctorow - which is the tendency for tech monopolies to exploit their own users and vendors when they can. I'll talk about each of these a little bit more in depth. General Risk Analysis First I recommend reading academic business and cyber risk analysis textbooks. Not just popular business books like "Anti fragile" or "The Black Swan" (which are good), but also academic books that introduced Bayesian networks and Failure Mode and Effect Analysis. You're not going to get away from some math, but thankfully there are techniques that let you draw situations as network diagrams, which I personally find helps with my natural difficulty with mathematics and symbolic processing. You can do pretty decent risk analysis just using "napkin math" - rough graphical and numerical estimates that approximate risks and can sharpen your intuition. I like to recommend cyber risk analysis too, since it makes sense to be able to accurately assess your organizational cyber risks as an entrepreneur. It shows us how beautiful and interdisciplinary risk management can be, and ideas like adversarial modeling, the attack kill chain, attack sequences, and the attack graph are valuable in ordinary business risk management, like coping with insider threats or adversarial competitors. Technical Observation Observing the development of technology is also important, and with AI we have the advantage that most of the development is happening in publicly accessible research papers on arxiv. By following the AI literature and evaluating LLMs and their capabilities, I made these two observations: 1. Multimodal and OCR assisted LLMs were hot in the AI literature early last year, and that approach was very helpful for reducing the hallucination/bullshitting problem - ai's biggest usability problem to date. Some of these papers were from openai themselves. From this we could assume that the major AI vendors would work very hard to integrate this feature, and suppose that third party OCR solutions would get kicked out of the market very quickly once they did. We have seen that with GPT, Gemini, Claude, and more. 2. GPT4 continues to beat the pants off of every single competitor, in the hundred+ benchmark papers I've read, in the public benchmarks like Chatbot Arena, and in my own personal and business use case benchmarking. There's basically no contest on quality, and the result is 100% capture of the market of serious users who depend on GPT4 quality and are sophisticated enough to evaluate AI quality. From this we can assume that OpenAI is well positioned to do everything it can to retain this advantage, even up to compromising developers who think it is safe to use the GPT store to build general improvements for GPT's weaknesses. To gain this technical insight, I highly recommend reading many AI papers, internalizing their insights, and if possible, working hands on with their experimental techniques. Being aware of AI's current weaknesses and benchmarks, and being able to name papers working on overcoming specific problems, allows us to predict what might be coming next from big vendors. Platform Risk Analysis To gain insight into why large corporations tend to hurt their own developer communities and users when they gain the opportunity, I highly recommend Cory Doctorow's 2023 article, "The Enshittification of Tik Tok". It covers a great deal of other examples besides - including Apple and Google's app stores which are relevant to understanding the outcome for the GPT plug in store. And it does a very good analysis of user and vendor exploiting strategies on Tik Tok. Doctorow also writes a freely available book/audiobook, "The Internet Con: Seizing The Means of Comoutation", where he analyzes the issue and its contributing factors, and proposes a solution based on regulations. While his solution - heavy regulation - doesn't seem that realistic, it's a good analysis, especially for entrepreneurs who might want to build on top of platforms that have this form of risk. Finally, here's a paper that builds on this idea and analyzes how platforms are able to use their algorithmic control over the users attention to increasingly disadvantage a variety of stakeholders, including users, vendors, and advertisers: https://www.cambridge.org/core/services/aop-cambridge-core/content/view/D85FE41F6CF99FC57DDFB2B2B63491C5/S2632324924000014a.pdf/algorithmic-attention-rents-a-theory-of-digital-platform-market-power.pdf https://www.cambridge.org/core/services/aop-cambridge-core/c... Conclusion and Personal Strategy I recommend a variety of sources including literature on risk management itself, literature having to do with the technology itself in a large variety of application domains, and being aware of market trends and forces, like the pressure to take advantage of non-financial stakeholders once a platform reaches a market dominance. My general AI business strategy is to focus on my sub niche of technical work where AI has not really penetrated - automating 2d spline processing operations that have been ignored by AI researchers due to their focus on point clouds, radiance fields, bitmaps, and meshes, which are more important and applicable to a huge variety of domains. I focus on providing geometric processing services directly to manufacturers dealing with 2d vector data from CMM systems. The CMM vendor I'm specializing in has been lax in adopting automation technology, and the customers I'm selling to are very reluctant to invest in improved CAD software, even when it could reduce their own labor by a factor of three or more. So between those two gaps I have a niche which I have exploited for 3 years now, and which I expect to exist for at least another 3 years. I plan to turn my CAD automation tools into an attractive product by that time, and also hope to advance the state of the art AI techniques in reverse engineering 2D splines from CMM data. I hope that lets me hold on to the niche until 2030. But I also would not be very surprised if my niche gets disrupted by one of the major CAD or CMM vendors in that time. My worst case outcome seems to be that I would have to join a more solidly established corporation as an research manager, strategic analyst, or software engineer. That could be an improvement in my quality of life, given that my current yearly revenues are still under 60k. However I'm going to fight that with all I've got, because I love owning my own company so much! I am an autistic entrepreneur, and owning my own company has been a huge breakthrough in my Independence and ability to avoid burnout. I can get by on as few as 10 hours of labor a week, which would be very hard to find in a corporation. Anyways, I hope that this message has been beneficial to you even though it grew a bit long. I find Risk Management very fun and rewarding to study and apply, and hope you will as well! I wish you luck in all your endeavors!

What resources do you suggest for learning how to do this kind of risk analysis well?

by yaj54 3y ago

What resources do you suggest for learning how to do this kind of risk analysis well?

A crystal ball. Most of the times you should do what they did, build a first version in a week and see. Nobody knows the future anyway.

globalise83 3y ago

You should look into decision analysis with decision trees - this is a nice article on how to do it: https://hbr.org/1964/07/decision-trees-for-decision-making https://hbr.org/1964/07/decision-trees-for-decision-making

barfbagginus 3y ago

My analysis of the platform risk of the ChatGPT plug-in store depends on 1) risk ideas like Failure Modes and risk hedging. The outcome in the plug-in store is an example of a well known failure mode of platforms that grow very fast. 2) remaining up to date with trends in the llm literature, so we can for instance predict that major vendors would release multimodal llms 3) being aware of trends in tech platform dynamics. Specifically the trend for "platform enshittification" - a term coined by Corey Doctorow - which is the tendency for tech monopolies to exploit their own users and vendors when they can. I'll talk about each of these a little bit more in depth. General Risk Analysis First I recommend reading academic business and cyber risk analysis textbooks. Not just popular business books like "Anti fragile" or "The Black Swan" (which are good), but also academic books that introduced Bayesian networks and Failure Mode and Effect Analysis. You're not going to get away from some math, but thankfully there are techniques that let you draw situations as network diagrams, which I personally find helps with my natural difficulty with mathematics and symbolic processing. You can do pretty decent risk analysis just using "napkin math" - rough graphical and numerical estimates that approximate risks and can sharpen your intuition. I like to recommend cyber risk analysis too, since it makes sense to be able to accurately assess your organizational cyber risks as an entrepreneur. It shows us how beautiful and interdisciplinary risk management can be, and ideas like adversarial modeling, the attack kill chain, attack sequences, and the attack graph are valuable in ordinary business risk management, like coping with insider threats or adversarial competitors. Technical Observation Observing the development of technology is also important, and with AI we have the advantage that most of the development is happening in publicly accessible research papers on arxiv. By following the AI literature and evaluating LLMs and their capabilities, I made these two observations: 1. Multimodal and OCR assisted LLMs were hot in the AI literature early last year, and that approach was very helpful for reducing the hallucination/bullshitting problem - ai's biggest usability problem to date. Some of these papers were from openai themselves. From this we could assume that the major AI vendors would work very hard to integrate this feature, and suppose that third party OCR solutions would get kicked out of the market very quickly once they did. We have seen that with GPT, Gemini, Claude, and more. 2. GPT4 continues to beat the pants off of every single competitor, in the hundred+ benchmark papers I've read, in the public benchmarks like Chatbot Arena, and in my own personal and business use case benchmarking. There's basically no contest on quality, and the result is 100% capture of the market of serious users who depend on GPT4 quality and are sophisticated enough to evaluate AI quality. From this we can assume that OpenAI is well positioned to do everything it can to retain this advantage, even up to compromising developers who think it is safe to use the GPT store to build general improvements for GPT's weaknesses. To gain this technical insight, I highly recommend reading many AI papers, internalizing their insights, and if possible, working hands on with their experimental techniques. Being aware of AI's current weaknesses and benchmarks, and being able to name papers working on overcoming specific problems, allows us to predict what might be coming next from big vendors. Platform Risk Analysis To gain insight into why large corporations tend to hurt their own developer communities and users when they gain the opportunity, I highly recommend Cory Doctorow's 2023 article, "The Enshittification of Tik Tok". It covers a great deal of other examples besides - including Apple and Google's app stores which are relevant to understanding the outcome for the GPT plug in store. And it does a very good analysis of user and vendor exploiting strategies on Tik Tok. Doctorow also writes a freely available book/audiobook, "The Internet Con: Seizing The Means of Comoutation", where he analyzes the issue and its contributing factors, and proposes a solution based on regulations. While his solution - heavy regulation - doesn't seem that realistic, it's a good analysis, especially for entrepreneurs who might want to build on top of platforms that have this form of risk. Finally, here's a paper that builds on this idea and analyzes how platforms are able to use their algorithmic control over the users attention to increasingly disadvantage a variety of stakeholders, including users, vendors, and advertisers: https://www.cambridge.org/core/services/aop-cambridge-core/content/view/D85FE41F6CF99FC57DDFB2B2B63491C5/S2632324924000014a.pdf/algorithmic-attention-rents-a-theory-of-digital-platform-market-power.pdf https://www.cambridge.org/core/services/aop-cambridge-core/c... Conclusion and Personal Strategy I recommend a variety of sources including literature on risk management itself, literature having to do with the technology itself in a large variety of application domains, and being aware of market trends and forces, like the pressure to take advantage of non-financial stakeholders once a platform reaches a market dominance. My general AI business strategy is to focus on my sub niche of technical work where AI has not really penetrated - automating 2d spline processing operations that have been ignored by AI researchers due to their focus on point clouds, radiance fields, bitmaps, and meshes, which are more important and applicable to a huge variety of domains. I focus on providing geometric processing services directly to manufacturers dealing with 2d vector data from CMM systems. The CMM vendor I'm specializing in has been lax in adopting automation technology, and the customers I'm selling to are very reluctant to invest in improved CAD software, even when it could reduce their own labor by a factor of three or more. So between those two gaps I have a niche which I have exploited for 3 years now, and which I expect to exist for at least another 3 years. I plan to turn my CAD automation tools into an attractive product by that time, and also hope to advance the state of the art AI techniques in reverse engineering 2D splines from CMM data. I hope that lets me hold on to the niche until 2030. But I also would not be very surprised if my niche gets disrupted by one of the major CAD or CMM vendors in that time. My worst case outcome seems to be that I would have to join a more solidly established corporation as an research manager, strategic analyst, or software engineer. That could be an improvement in my quality of life, given that my current yearly revenues are still under 60k. However I'm going to fight that with all I've got, because I love owning my own company so much! I am an autistic entrepreneur, and owning my own company has been a huge breakthrough in my Independence and ability to avoid burnout. I can get by on as few as 10 hours of labor a week, which would be very hard to find in a corporation. Anyways, I hope that this message has been beneficial to you even though it grew a bit long. I find Risk Management very fun and rewarding to study and apply, and hope you will as well! I wish you luck in all your endeavors!