17 ms·

Why should I think that LLMs would be good at the task of analyzing a cloud incident and determining root cause? LLMs are good at predicting the next word in w

• from-nibly 2y ago
  The main problem is. Most people cant tell the difference between an expert opinion and random words. So if an LLM shoots a bunch of jargon on the screen then, well so does my senior platform engineer so whats the difference? The LLM is cheaper and is always on call.
  • RodgerTheGreat 2y ago
    I see deeply ominous connections between generations of American children who have been educated in a manner which produces alarming rates of functional illiteracy[0] and the widespread popularity of machine learning models that produce nonsense which resembles plausible text if you skim or don't actually read it. [0] https://www.apmreports.org/episode/2019/08/22/whats-wrong-how-schools-teach-reading https://www.apmreports.org/episode/2019/08/22/whats-wrong-ho...
    • treyd 2y ago
      > The theory is known as "three cueing." The name comes from the notion that readers use three different kinds of information — or "cues" — to identify words as they are reading. > The theory was first proposed in 1967, when an education professor named Ken Goodman presented a paper at the annual meeting of the American Educational Research Association in New York City. > In the paper,5 Goodman rejected the idea that reading is a precise process that involves exact or detailed perception of letters or words. Instead, he argued that as people read, they make predictions about the words on the page using these three cues: > graphic cues (what do the letters tell you about what the word might be?) > syntactic cues (what kind of word could it be, for example, a noun or a verb?) > semantic cues (what word would make sense here, based on the context?) This is interesting because this is fairly similar to how LLMs do next-token prediction, but using exclusively backwards-facing clues from the text (although perhaps also the graphic cues if you are talking about multimodal models).
    • pixelatedindex 2y ago
      Super fascinating article, thanks for sharing!
    • philipswood 2y ago
      Thanks, I think the linked article is worthy of its own submission: https://news.ycombinator.com/item?id=41344613 https://news.ycombinator.com/item?id=41344613
  • sublinear 2y ago
    > Most people cant tell the difference And those people should be fired.
    • deleted 2y ago
      [deleted]
    • from-nibly 2y ago
      Those people are your bosses.
  • visarga 2y ago
    > Most people cant tell the difference between an expert opinion and random words. That's because humans are stochastic parrots too. We use leaky abstractions we don't fully understand, or their edge cases. So we don't know what we are saying, but keep doing this as long as it doesn't break. When it breaks, we go to experts, another abstraction we don't really grok. What's the difference between using LLM and using a human expert? Not much if you have no clue about the topic. It's a matter of replacing real understanding with trust. Even more, can we say we understand anything down to first principles? Probably not. It's a patchwork of people, each with their limited perspective, like the Elephant and the Blind Men parable. The world is based on functional understanding, combining partial perspectives, it's never fully grokked. And what you don't understand, you can't be conscious of, not in it's real meaning. So we might be unconscious operators of language as well, no better than hallucinating LLMs.
  • 77pt77 2y ago
    Finally someone gets it. And then there is status. Since you don;t understand anything anyway, you ascertain value and correctness based on status signals. For you engineer that is going to be, first and foremost the way he looks. Whether he's tall, talks without stuttering, which school he went to. For LLMs it might be a variant of: > No one was ever fired for choosing Google/IBM/Microsoft
• deleted 2y ago
  [deleted]
• tomjen3 2y ago
  You should not think that LLMs are a good fit for anything. You should check if they are, and then try to guide it to be better at the task. You should do this because AI is a jagged frontier where humans can't predict if AI is good at something or not. >AI is weird. No one actually knows the full range of capabilities of the most advanced Large Language Models, like GPT-4. No one really knows the best ways to use them, or the conditions under which they fail. There is no instruction manual. On some tasks AI is immensely powerful, and on others it fails completely or subtly. And, unless you use AI a lot, you won’t know which is which. https://www.oneusefulthing.org/p/centaurs-and-cyborgs-on-the-jagged https://www.oneusefulthing.org/p/centaurs-and-cyborgs-on-the...
  • samstave 2y ago
    This is exactly how I try to approach using it. I recognize that its still a robot and you have to give it stoic, stern guidance to keep it on point. Also, having a good understanding for the domain youre seeking to build understanding in, but with the augmentation of an AI assist that can formulate the Output of the thought in a more complete and packaged manner than one is able to do without leveraging AI as a tool. And as someone said "And its not going to get any worse, its only going to get better" I think people are really underestimating, and under-utilizing AI tools. But they are terrifying when you consider just that - a divide between those who have/use AI and those who are subjugated by those who do.
  • cqqxo4zV46cp 2y ago
    God. Thank you. All these LLM conversations are making me hate this website so much, because apparently at some point actual scientific enquiry took a back seat, and turning one’s nose up at anything in a blatant attempt to seem smart has taken charge. If all the people whinging on here took some of that time and actually ‘formally’ experimented with LLMs, measuring their reliability / correctness against a human in some task in their domain, they may be surprised by the results. And no, “I tried Copilot for an afternoon and hated it” doesn’t suffice. At work, recently, I happened across an opportunity to do just this. There was a task that I thought that it was quite possible for an LLM to be good at. The task was such that we could run a bit of a ‘study’ to see how the LLM fared against a real-world meat-bag person. A skilled person at that. The person we would’ve had do the job in the first place. The LLM and the human agreed the vast majority of the time (>99.9%), and the LLM with its infinite ‘attention’ (heh) was on more than one occasion correct in cases where the human wasn’t, because it was a repetitive task that’d put someone to sleep. It was a task that involved parsing language, but I’m sure one that the geniuses on HN would say requires “understanding semantics”, “intelligence” or whatever armchair philosophy nonsense they whip out in lieu of intelligent conversation. It wasn’t sentiment analysis, categorisation, or anything of that nature. Maybe it’s something I could’ve tackled without an LLM, with traditional ‘deep learning’, or whatever. I really don’t know. I couldn’t think of a way off the top of my head. It was beyond ‘throw linear regression at it’ anyway. Software engineering isn’t engineering, but evidently computer science is increasingly not a science. This industry deserves all of the belittling pejoratives people throw at it. There’s a disappointingly large contingent of utterly unengaged, incurious, drones that let their entire professional skill set be guided by whatever some other incurious drone says on a social network.
• deegles 2y ago
  This project had one goal: get a high visibility project out the door in order to secure a promotion.
  • refulgentis 2y ago
    What project?
    • generic92034 2y ago
      See? Those are the best projects, where no-one can remember it or knows who was responsible when it is headed for its final failure. ;)
• refulgentis 2y ago
  You're way over-estimating the "it only knows the training data" stuff to a very large degree. Ex. In the last 48 hours, Facebook also released a paper of it being excellent at RCA. This is a well-tred argument, and it is much stronger when it holds itself to, especially in the short term, augmentation is likely, not wholesale delegation. It gets weak when it tries to couple that to asserting that "all it does" is lightly rephrasing training data. It's somewhat trivial to demonstrate this is false, even society as a whole has noticed that it's beyond a parrot, and it's worrying.
  • otterley 2y ago
    > In the last 48 hours, Facebook also released a paper of it being excellent at RCA. Do you have a link? The most recent information (not a paper) I could find was back in June: https://engineering.fb.com/2024/06/24/data-infrastructure/leveraging-ai-for-efficient-incident-response/ https://engineering.fb.com/2024/06/24/data-infrastructure/le... This is the HN discussion to which you may be referring: https://news.ycombinator.com/item?id=41326039 https://news.ycombinator.com/item?id=41326039
    • refulgentis 2y ago
      Correct, thank you!
• mitjam 2y ago
  A simple example: Prompt an LLM with a log message you don't understand and see if it helps you interpret it. In my experience, in many cases, it can.
• roenxi 2y ago
  > Why should I think that LLMs would be good at the task of analyzing a cloud incident and determining root cause? LLMs are good at predicting the next word in written language. They are generative; they make new text given a prompt. Now explain what humans are doing and why it is different. RCA can be done fully remotely, so we know it can be modelled as a word-prediction model with context and a "how is all this context made consistent?" prompt. There is every reason to expect that an advanced text prediction model would be excellent at predicting the most-technically-correct response to that prompt. I've seen a few people make this argument as though text prediction is some specific sub-field that can be solved independently of having a world model and intelligence. That doesn't hold up at all, if we solve text prediction we've solved general intelligence - all aspects of human intelligence are less complex than being able to predict the most objectively correct next word in a sequence of words because all aspects of intelligence can be framed as a text-prediction problem. A system can't predict the next word in a sequence and fool a human without being at least as clever as a human.
  • crystal_revenge 2y ago
    You don't even have to get deep into the internals of LLMs to see what's wrong with your reasoning. The problem lies with the basic mechanics of: > predict the most objectively correct next word in a sequence of words Currently all LLMs are only determining the most probable next token, but this means they are not aware of the probability of the entire sequence of tokens they are emitting. That is, they can only build sentences by picking the most probable next word, but can never choose the most probable sentence. In practice, there are a great many very likely sentences that are composed of a fairly unlikely words. When we use the output of an LLM we're thinking it of a sequence sampled from the set of all possible sequences, but that's not really what we're getting (at least as far as probability is concerned). There are approaches to address this: you can do multinomial sampling instead of greedy so that are casting a slightly large net or you can do beam search where your once again trying to search a broader set of possible sentences choosing by the most probable sequence. But all of these are fairly limited. Which gets to your first remark: > Now explain what humans are doing and why it is different. There's very little we really know about how humans reason, but we are certainly building linguistic expressions at with a more abstract form of composition. This comment for example was planned out in parts, not even sequentially, and then reworked to the whole thing makes some sense. But at the very least humans are clearly reasoning at the level of entire sequences as their probability rather than individual tokens at a time. The word "planning" almost tautologically implies thinking ahead of the next step. When humans write HN comments or code they're clearly planning rather than just thinking of the next most likely word over and over again with some noise to make it sound more interesting. No matter how powerful and sophisticated the mathematical models driving the core of LLMs are, we're fundamentally limited by the methods we use to sample from them.
• Terr_ 2y ago
  > Why should I think that LLMs would be good a They make language that sounds like what thinking people make, therefore they must be thinking like people do, duh! /s
• wkat4242 2y ago
  > Why should I think that LLMs would be good at the task of analyzing a cloud incident and determining root cause? Because an LLM is really good at convincing laymen it knows what it's doing. It's really a conman simulator :) It is great at language based tasks but people use it as an oracle for everything without understanding the limitations. It's just that friendly convincing tone that makes them think they're taking to a super intelligent being.
• Kiro 2y ago
  Did you see this? https://engineering.fb.com/2024/06/24/data-infrastructure/leveraging-ai-for-efficient-incident-response/ https://engineering.fb.com/2024/06/24/data-infrastructure/le...
  • genewitch 2y ago
    i get real strong "Mechanical Turk" vibes from all of this faang "let us do the tedious stuff at your business".
• deleted 2y ago
  [deleted]
• low_tech_love 2y ago
  The idea that LLMs “only know how to predict the next word” is a common but imho wrong cop out. It has been shown that they are good at doing a lot of things that might not have been immediately obvious, regardless of how their internal process works. 50 years ago, somebody might say “programming is only a way to automate simple repetitive tasks” and that would be obviously wrong. The real problem in cases like this and other applications, as you and many others have mentioned, is that LLMs are basically correlation machines. They can find very complex, immensely-multivariate correlations in large data sets, and reproduce these correlations very well. But they cannot reason (so far) beyond these correlations in other to find deeper, less obvious causal relationships. They’re simply not trained to do that, yet. But it’ll come…!
  • austin-cheney 2y ago
    > 50 years ago, somebody might say “programming is only a way to automate simple repetitive tasks” and that would be obviously wrong. That is actually extremely correct. The only purpose for software is automation, which is the elimination of labor. Getting that wrong directly influences your quality of product more than any other downstream factor.
    • low_tech_love 2y ago
      It might be theoretically correct in the same way that it is correct to say that “a building is just a bunch of bricks on top of each other”. But there are thousands of different buildings with different reasons to exist which offer drastically different services and serve different purposes. A video game like Elden Ring is built from the same “automation” pieces as the LS command in my terminal, but it would very disingenuous to say they’re basically the same thing.
      • austin-cheney 2y ago
        That is an incorrect comparison. A building, all buildings, are dwellings, but they are no more or less the sum of their parts than anything else. It’s not about the construction materials. It’s about the utility.
• xg15 2y ago
  > LLMs are good at predicting the next word in written language. They are generative; they make new text given a prompt. LLMs do not have base sets of facts about how complex systems work, and do not attempt to reason over a corpus of evidence and facts. Not wanting to advocate for using LLMs for RCA, which is a dumb and dangerous idea for all the reasons the OP mentioned - but I am getting allergic to the phrase "it just predicts the next word". Yes, that is how the "API" of an LLM works, but on itself, it says nothing about how the LLM does the prediction and how complex the internal model is that it uses for that task. It's obvious that the task of predicting has a huge variance in complexity, depending on which word has to be predicted. E.g. the sentence "The apple does not fall far from the" could be completed by a decently trained Markov chain, whereas (correctly!) completing "sqrt(153847)=" would either require an impossibly large training set or an internal model that can parse integers and perform square root calculations. Yet both are on the surface "predict the next word" tasks. The actual complexity of LLMs' internal models seems to still be poorly understood. That's not to say the model has superhuman ability or even reaches human abilities. But the point is that it's still really hard to make predictions how complex the reasoning is that an LLM performs for a task. In the OP's example, we can't really say if the LLM "has base sets of facts about how complex systems work", because we don't really know if and how "facts" would be represented inside the model. If enough in-domain examples were in the trainset, there is no fundamental reason why it wouldn't have learned such a set of facts. Just saying "it can't reason at all because it's just a next word predictor" is mixing up different layers of meaning and, I believe, does not lead to more insight.
• reissbaker 2y ago
  TBQH I'd bet the following would yield pretty good results: 1. Take your existing incident reporting / review docs (you have those, right?) that cover everything including 5-why incident reporting and analysis. 2. Fine-tune a Llama-3.1-70b [1] LoRA on the data associated with the outage as input, and the root cause analysis as the output 3. Tada! You have a state-of-the-art custom LLM that is good at analyzing your outages and guessing what the root causes might be. It's a little shocking sometimes to me how underutilized fine-tuning is. Most of the "learning" happening in "machine learning" is in training — yes, it's definitely true that LLMs can exhibit a surprising amount of "in-context learning" via prompts, but it's a surprising because learning during training is so much more powerful, and it's surprising that in-context works at all. Honestly even just fine-tuning an 8b model — which is totally doable on a 3090/4090 — can yield SOTA results on task-specific performance. It's so much better than prompting! 1: I mean you could also finetune 405b, but you'll need a lot of GPUs both to train it and to run it. In my experience (public benchmarks be damned; the models all tend to saturate the public benchmarks, even though everyone claims not to train on them), 70b on internal evals tends to perform similarly to gpt-4o, and with OSS models you have somewhat more ownership and control.
  • gbasin 2y ago
    For large models that are well-trained with large context support, ICL seems to work about as well as FT
    • reissbaker 2y ago
      It's totally task dependent. On some tasks, large well-trained models are already great; if the large model already exhibits human-level performance, a small-model finetune is unlikely to beat it. Similarly, on very general tasks (e.g. "coding" as a general task, as opposed to "writing idiomatic NextJS" being a specific task), a small-model finetune will be unlikely to beat a large model. But there are plenty of tasks that even large, well-trained models struggle with. If the OP is struggling to get useful root-cause analysis for cloud service incidents out of an existing large model, that seems exactly like a use case where a finetune would shine. Also, finetunes don't have to be just for small models! Medium-sized models like Llama-3.1-70b can be finetuned, and if you want to burn a lot of GPUs you can finetune 405b as well.
• djohnston 2y ago
  You're trying to build a ground-up theory for why LLMs are bad for this sort of work. But the empirical evidence suggests they're quite good at it. Just try using it!
• giancarlostoro 2y ago
  I can see an LLM talking to a cloud API (the underlying APIs) getting status reports and history, and telling you what is currently wrong, faster than you can comb through it all, and I'm a huge LLM skeptic, but if I ever saw a use for an LLM its the "tell me what's wrong faster than I can look" angle of things. You don't use it to solve the problem for you, you use it to tell you where things are faster than you and ten people can comb through.
• callalex 2y ago
  > Why should I think that LLMs would be good at the task of analyzing a cloud incident and determining root cause? Because it would be extremely profitable if they could. That’s how we’ve ended up in this mess, the promise is just so tantalizing even if it’s not based in reality.
• stavros 2y ago
  I see where you're coming from, but I think you might be underestimating what modern LLMs can do. Sure, at their core they're predicting text, but that ability translates into some pretty impressive capabilities when it comes to understanding and analyzing complex systems. Think about it - these models have been trained on mountains of technical docs, incident reports, and discussions about cloud systems. They've soaked up a ton of knowledge about how these systems work and what tends to go wrong. You're right that they don't reason from first principles, but they're incredibly good at spotting patterns. When you feed an LLM details about an incident, it can quickly pick up on similarities to known issues and suggest potential causes. It's not just making stuff up - it's drawing on a vast pool of relevant information. And these models aren't just spitting out random text. They've gotten really good at understanding context and applying the right knowledge to a given situation. They can take in technical details about an incident and connect them to possible root causes in ways that can be surprisingly insightful. I'd argue that LLMs can actually be pretty effective tools for analyzing failures in complex systems. They can process tons of information quickly, spot connections humans might miss, and generate multiple plausible hypotheses for what went wrong. They're not replacing human experts, but they can definitely augment our capabilities and speed up the initial triage process. There are already some success stories out there of LLMs being used effectively in IT ops and incident analysis. And as these models get fine-tuned on more specific cloud-related data, their performance in this area is only going to improve. You're right to be cautious about applying LLMs everywhere just because we can. But in this case, I think they actually have a lot to offer when it comes to cloud incident analysis, especially when used alongside other tools and human expertise. They're not a silver bullet, but they're definitely more than just elaborate text predictors when it comes to tasks like this. This comment was written by an LLM. If it can make a coherent argument, I guess it can process information about an incident, too.
• jononor 2y ago
  Practical "LLM" systems often incorporate things like Reinforcement Learning From Humans (RFLH) to steer towards more useful answers, and Retrival-Augmented Generation (RAG) to lookup relevant information. External "tool use" is also just starting to be adopted. Such systems are not just next token predictors, there is quite a bit more complexity to it. Not that I am claiming they are great for post mortem analysis, I not really have an opinion about that.