4 ms·

This is interesting, but I'm not sure I'm a fan. One of the core problems in building businesses out of AI is that people pick problems they want to solve and t

This is interesting, but I'm not sure I'm a fan. One of the core problems in building businesses out of AI is that people pick problems they want to solve and try to get AI to solve them rather than picking problems that AI is likely to be able to solve. AI is unreliable, particularly in the long-tail, and cannot easily be used in domains where the cost of failure is high. Using AI to accelerate humans via human-in-the-loop AI tools is a great example of where things like GPT-3 can have real value. But good root cause analysis is a matter of establishing facts and building a chain of logic on top of those facts to get to the root cause. You cannot rely on models like GPT-3 to give you reliable baseline facts. Particularly when you are talking about a production issue that needs to be fixed ASAP. The key line that worries me from this blog post is "when results are suboptimal, they are mostly not misleading". 'Mostly not misleading' isn't going to cut it when I'm in the middle of an outage. I think that will prove to be a problem if this tool gets widespread usage. That being said, I'm a huge fan of applying AI to human-in-the-loop problems and this was a cool idea for how modern language models can be applied.

Ajs1 6y ago
fair point, but this blog and your comment is about a summary sentence. if you read through to how the underlying log reports are constructed, those are very accurate (and also quite concise).
cl42 6y ago
I'm so with you on this. First, you don't need to turn root cause analysis into text like this via GPT-3; there are easier ways. Secondly, I imagine most cases of "root cause analysis" require you to be very, very clear in understanding the... root cause... So using generalized language models will probably lead to unacceptable errors, which means there are probably better ways of addressing this problem (as per the discussion here on error rates and unacceptable errors in ML-products: https://phaseai.com/resources/how-to-build-ml-products https://phaseai.com/resources/how-to-build-ml-products)
mlthoughts2018 6y ago
I think your perspective is actually way off the mark here. AI excels at long-tail problems where the cost of failure is high, precisely because human failure is such an expensive problem in those cases and the nature of long tail problems prevents it from being possible to apply QA to every use case. In other words, you know you are forced to deal with getting it wrong a lot and paying the high failure cost, so using a system capable of optimizing that trade-off explicitly is often much better than pretending as if a human in the loop is somehow sparing you the failure costs when they aren’t (and in fact they are simply less efficient than algorithmic solutions). What constitutes a useful sequence of facts in root cause analysis is not just some platonic existing thing. It’s a complex problem involving mind-melting log sleuthing, correlating all kinds of disparate metrics, comparing against timestamps of merges and eventually synthesizing the results. Even seasoned veterans who know systems inside and out struggle with the sheer volume of logs, metrics and facts to compile. And most of the time their approach is based purely on inductive experience with similar incidents combined with heuristics. This is precisely the kind of problem that ML solutions excel at. It has many hallmarks of a good fit and almost none of the hallmarks of “solution in search of a problem” ML over-engineering.
- IgorPartola 6y ago
  Can it be both? AI excels at doing repetitive things better than humans, like maybe driving a car. Until it encounters a situation that it hasn’t seen before that cannot accurately be described with bits and pieces of what it knows. Then it’s result isn’t a little off the mark, it’s a lot. Think of a disaster like what happened with Ever Given or Chernobyl. How many different ways could AI have made the situation worse when confronted with those problems because there is no good definition of an optimal solution here.
  - mlthoughts2018 6y ago
    Generalization to previously unseen examples is one of the core components of ML models.
- solidasparagus 6y ago
  A couple of things here. First, you are conflating the underlying log relevance scoring ML system with the GPT-3 summarizing system. ML is a good fit for relevant log identification for the reasons you describe, although characterizing this software as root cause identification is not very accurate in my opinion, based on the examples you can find on their website. But the value of summarizing a log line into natural language is low, while the cost of misleadingly characterizing that log line is high. Whoever needs to debug this system and find the real root cause (e.g. why did the system go OOM?) probably needs certainty more than the convenience and in all likelihood, they are more likely to correctly summarize what the log line says than GPT-3 is (obviously we don't know since there is no evidence, but I don't work with any engineers whose ability to summarize the contents of a log line would be described as "mostly not misleading"). Secondly, I can't agree with this sentence: > AI excels at long-tail problems where the cost of failure is high, precisely because human failure is such an expensive problem in those cases Maybe it depends on domain and tech, but in my experience humans don't fail on out-of-sample data nearly as often as AI does. When they do fail, it is often more predictable to other humans and humans inherently have the ability to assign confidence levels to their conclusion which you don't see in many AI models such as GPT-3. Humans are also more effective at applying rules (e.g. common sense) to improve predictions on out-of-sample inputs. I think of "AI is worse than humans at generalizing to out-of-sample" as being a widely held, well-evidenced belief, but I would be interested if you disagree. For me, the quintessential example is something like traffic light identification, where models generally struggle to identify unseen variants correctly while humans rarely struggle at it. What examples are you thinking of where AI excels at long-trail problems?

Ajs1 6y ago

fair point, but this blog and your comment is about a summary sentence. if you read through to how the underlying log reports are constructed, those are very accurate (and also quite concise).

cl42 6y ago

I'm so with you on this. First, you don't need to turn root cause analysis into text like this via GPT-3; there are easier ways. Secondly, I imagine most cases of "root cause analysis" require you to be very, very clear in understanding the... root cause... So using generalized language models will probably lead to unacceptable errors, which means there are probably better ways of addressing this problem (as per the discussion here on error rates and unacceptable errors in ML-products: https://phaseai.com/resources/how-to-build-ml-products https://phaseai.com/resources/how-to-build-ml-products)

mlthoughts2018 6y ago

I think your perspective is actually way off the mark here. AI excels at long-tail problems where the cost of failure is high, precisely because human failure is such an expensive problem in those cases and the nature of long tail problems prevents it from being possible to apply QA to every use case. In other words, you know you are forced to deal with getting it wrong a lot and paying the high failure cost, so using a system capable of optimizing that trade-off explicitly is often much better than pretending as if a human in the loop is somehow sparing you the failure costs when they aren’t (and in fact they are simply less efficient than algorithmic solutions). What constitutes a useful sequence of facts in root cause analysis is not just some platonic existing thing. It’s a complex problem involving mind-melting log sleuthing, correlating all kinds of disparate metrics, comparing against timestamps of merges and eventually synthesizing the results. Even seasoned veterans who know systems inside and out struggle with the sheer volume of logs, metrics and facts to compile. And most of the time their approach is based purely on inductive experience with similar incidents combined with heuristics. This is precisely the kind of problem that ML solutions excel at. It has many hallmarks of a good fit and almost none of the hallmarks of “solution in search of a problem” ML over-engineering.

IgorPartola 6y ago
Can it be both? AI excels at doing repetitive things better than humans, like maybe driving a car. Until it encounters a situation that it hasn’t seen before that cannot accurately be described with bits and pieces of what it knows. Then it’s result isn’t a little off the mark, it’s a lot. Think of a disaster like what happened with Ever Given or Chernobyl. How many different ways could AI have made the situation worse when confronted with those problems because there is no good definition of an optimal solution here.
- mlthoughts2018 6y ago
  Generalization to previously unseen examples is one of the core components of ML models.
solidasparagus 6y ago
A couple of things here. First, you are conflating the underlying log relevance scoring ML system with the GPT-3 summarizing system. ML is a good fit for relevant log identification for the reasons you describe, although characterizing this software as root cause identification is not very accurate in my opinion, based on the examples you can find on their website. But the value of summarizing a log line into natural language is low, while the cost of misleadingly characterizing that log line is high. Whoever needs to debug this system and find the real root cause (e.g. why did the system go OOM?) probably needs certainty more than the convenience and in all likelihood, they are more likely to correctly summarize what the log line says than GPT-3 is (obviously we don't know since there is no evidence, but I don't work with any engineers whose ability to summarize the contents of a log line would be described as "mostly not misleading"). Secondly, I can't agree with this sentence: > AI excels at long-tail problems where the cost of failure is high, precisely because human failure is such an expensive problem in those cases Maybe it depends on domain and tech, but in my experience humans don't fail on out-of-sample data nearly as often as AI does. When they do fail, it is often more predictable to other humans and humans inherently have the ability to assign confidence levels to their conclusion which you don't see in many AI models such as GPT-3. Humans are also more effective at applying rules (e.g. common sense) to improve predictions on out-of-sample inputs. I think of "AI is worse than humans at generalizing to out-of-sample" as being a widely held, well-evidenced belief, but I would be interested if you disagree. For me, the quintessential example is something like traffic light identification, where models generally struggle to identify unseen variants correctly while humans rarely struggle at it. What examples are you thinking of where AI excels at long-trail problems?