9 ms·

The Killer Use Case for LLMs Is Summarization

58 points by sebmellen 3y ago

simonw 3y ago
I couldn't see a date on this piece but the Internet Archive first captured it on May 14th: https://web.archive.org/web/20230514232943/https://www.sebastianmellen.com/post/2023/the-killer-use-case-for-llms-is-summarization/ https://web.archive.org/web/20230514232943/https://www.sebas... (Relevant here because even five months is a long time in LLMs!) Update: turns out there's a date on https://www.sebastianmellen.com/post/ https://www.sebastianmellen.com/post/ - this post is from 18th March, so seven months ago.
- kfarr 3y ago
  Thanks, my initial response to summarization as killer feature: "is it though?" but the "historic" nature of this post makes a bit more sense now. Unlike crypto, there are many diverse use cases for LLM. While summarization is useful I don't think it's killer.
filterfiber 3y ago
Does anyone have any experience for how the generalized LLM's like llama/gpt4 compare to much smaller, purpose built summarizers like some of the FLAN variants?
wastewastewaste 3y ago
What is the most capable LLM for summarising stuff like long educational articles perhaps?
- wahnfrieden 3y ago
  GPT-4
- dartos 3y ago
  GPT-4 is usually the answer to “which llm is most/best…” But if the article is longer than the context window, you can try either RWKV or Claud-200k
  - simonw 3y ago
    Has Claude 200k ever been made available? I've only found ways to access the 100k version.
- simonw 3y ago
  GPT-4 if it's less than 8,000 tokens (or 32,000 if you have access to that via the API). Claude for up to 100,000 tokens.
- sebmellen 3y ago
  Anthropic's Claude 2 is great for pasting large text files in, and Flan T5 seems to be excelling at summarization benchmarks: https://huggingface.co/docs/transformers/model_doc/flan-t5 https://huggingface.co/docs/transformers/model_doc/flan-t5.
mattew 3y ago
Don’t forget classification. Even the zero shot classification can be quite decent, but put a solid prompt together, or fine tune a set of examples and it’s way easier with an LLM than previous methods in my experience.
- blueboo 3y ago
  My favourite extant example of this GPT4’s 18-way rule-based reward model classifier, appendix A https://cdn.openai.com/papers/gpt-4-system-card.pdf https://cdn.openai.com/papers/gpt-4-system-card.pdf
simonw 3y ago
Summarization fits into my mental model of LLMs as "calculators for words". https://simonwillison.net/2023/Apr/2/calculator-for-words/ https://simonwillison.net/2023/Apr/2/calculator-for-words/ I think it's one of a small set of killer related use cases: - Summarization - Data and fact extraction, which is strongly related to summarization. Find all factual assertions in this news article. Extract the names of every person and location mentioned. - Question answering: given this context, answer this question. That's key to implementing Retrieval Augmented Generation, eg https://til.simonwillison.net/llms/embed-paragraphs#user-content-answering-a-question https://til.simonwillison.net/llms/embed-paragraphs#user-con... Aside from these summarization-adjacent applications, the thing I find most exciting is LLMs incredible aptitude at generating code. Code is especially interesting because it can be automatically executed (in a sandbox), which means many hallucinations can be automatically detected and fixed!
- sitkack 3y ago
  Summarization is all I have ever used Claude for, and it is excellent.
- J_cst 3y ago
  I never coded but I know what's needed and how things should be arranged to work. Result: I've developed and deployed a fully fledged web service both for human and machine use. Backend, front-end, endpoints, db, integration with our company systems, authentication, testing and deploy. In 5 sessions of 4 hours each. It was incredible while it was happening. I pay openai.
  - unblough 3y ago
    > This Connection is Untrusted Go Back The owner of costantini.pw has configured their website improperly. To protect your information from being stolen, Firefox has not connected to this website. I tried to see what it built for you, and this was the only url in your bio.
    - J_cst 3y ago
      That's a personal profile page which has an expired certificate, sorry for that. If you're (or anyone really) interested in what I did and would like to try it out for curiosity and feedback, please feel free to reach me at j@costantini.pw as I'd be pleased to share it. Thanks
macspoofing 3y ago
Kind of .. depending on how sensitive you are to accuracy. Also also, we've had powerful summarization techniques for decades now (deep-learning, or otherwise) - and they aren't susceptible to the hallucinations of present day Generative LLMs.
- simonw 3y ago
  Which summarization models aren't susceptible to hallucination? Is there any good writing out there about this issue?
  - netruk44 3y ago
    I found an 'old' (2016) reddit thread describing how auto tldr works, which doesn't suffer from hallucination (because it's only a statistical model that outputs original sentences from the text): https://old.reddit.com/r/askscience/comments/4s5b5q/how_exactly_does_a_autotldrbot_work/ https://old.reddit.com/r/askscience/comments/4s5b5q/how_exac...
  - sebmellen 3y ago
    https://huggingface.co/docs/transformers/model_doc/t5 https://huggingface.co/docs/transformers/model_doc/t5 is quite good for summarization!
- dontupvoteme 3y ago
  >we've had powerful summarization techniques for decades now (deep-learning, or otherwise) - and they aren't susceptible to the hallucinations of present day Generative LLMs. do you mean BERT, Seq2Seq and similar? My biggest takeaway from summarization tasks was that the benchmarks/evals seemed rather.. lacking.
- awkward 3y ago
  Like many use cases for LLMs, this seems like it would benefit a lazy student trying to avoid work. Rolling in some statements that are related but not mentioned in the input isn't a bad thing when you're studying for a test. However, for most use cases, falsely attributing statements is a terrible failure case for a summarizing tool.
xianshou 3y ago
I would agree that summarization is a killer use case, but would actually argue for three: 1. Condense (summarize, synthesize, extract, etc.) 2. Expand (explain, copilot, suggest, etc.) 3. Transform (translation, style transfer, switch modality between text/image/audio, etc.) All of the major consumer use cases of LLMs at this point boil down to one of these three core operations. To the extent that LLMs allow you to perform "magic", that magic is comprised by a chain of {condense, expand, transform} operations, each of which could be performed in half an hour by a reasonably informed college graduate with access to Google. Even if chain length maxes out at 3 or 4 for production-grade applications, that solves a frightening number of problems.
- jonny_eh 3y ago
  Chatting seems very popular too.
  - dontupvoteme 3y ago
    I don't understand why this got downvoted, but there's absolutely a huge unserved(or, rather, anti-served) and somewhat-looked-down-upon market here, and it's actually seemingly more women then men when i ran some numbers.
  - tomjen3 3y ago
    If LLMs can be a cheap and instant therapist (even if they are just somebody who listens) then they could do a massive amount of good in this world.
- Karrot_Kream 3y ago
  I'm an early engineer at a startup that has nothing to do with AI. I use LLMs for 2 and 3 all the time. A lot of times we'll release a new feature and I'll write my own fairly technical summary of it, then I'll ask an LLM to either target it for a specific channel (e.g. create a version of the post targeting execs vs creating one that's focused on users) or condense it down if we're posting a summary onto a social media channel. In all cases I check the LLM's work, but I rarely have to do much. When I led teams at a big tech, I generally had a human whose job it was to do this kind of copy refinement/editing when my teams released new features. I find the LLM to be just as good, though obviously our reach is much smaller and the consequences of a mistake are much milder given our size. This is the kind of productivity boost I think of that LLMs offer; pre-LLM we would probably contract out a copy editor for this kind of work but now we do it all in-house and don't have to deal with working with a contractor's hours.
dontupvoteme 3y ago
I would argue it's translation with summarization as a close second, but not for people who spend much time on HN since their command of English is almost always good enough. Tools like DeepL already were cutting into the field of translation/localization, but now we can have something which can basically translate ANY language to ANY language (even things like fictional languages from e.g. LoTR, although that may be very hallucinogenic due to the paltry amount of training data) rather rapidly (gpt3) or rather slowly but more accurately (gpt4) including understanding of things like metaphors and idioms. Not to mention you can probably fix some of the choice of vocab and style just by specifying (once converted to target language) "reword in the stylings of X" or countless other multi-invocation approaches to the previously very expensive and artistic task of converting human thought from one variation to another. The biggest problem is probably over-training on english (and being dependent on openai), but even Microsoft made a LLaMA[!] finetune just for translation which iirc was significantly better than the non-translation versions from everyone else
growt 3y ago
„However, LLMs appear to be quite good at knowing what they do and don’t know“ Really? I think LLMs have not the slightest idea what they know and you have to go through great lengths to stop them from making stuff up.
- sebmellen 3y ago
  They certainly can be: https://huggingface.co/docs/transformers/model_doc/t5 https://huggingface.co/docs/transformers/model_doc/t5, and moreso if you plug in data to them. If you paste text into ChatGPT and then ask about specific information within the text, it's pretty good at answering.
Zambyte 3y ago
I think the "killer use case" is slightly more general: operations in linguistic space, of which summarization is one. Some other useful uses include: - the inverse of summarization; given a bulleted list of facts, write a sensible set of paragraphs presenting those facts - semantically searching in large documents - make some text sound more professional / casual - translate some text between human languages (they're not perfect at this yet, but they're surprisingly good, and I think they will be the way to do translations in the future). - expand abbreviations given some context The key here is that none of these depend on the language model injecting factual information into its output. All of the facts are encoded in your input - the language model just does some operation in linguistic space. If you don't give it the opportunity to give you incorrect assertions, it won't give you incorrect assertions.
- earthboundkid 3y ago
  Another application is to use the LLM to build an embeddings database, and then do semantic search only within that database. Don't just let the LLM talk about whatever it wants to hallucinate up, force it to use a set of known good answers while accepting natural language input and summarizing documents in the known good dataset.
- dontupvoteme 3y ago
  everything there sounds great (though i suspect for actual human-passing linguistic register-transfer we will need something different than the bog standard GPT models because everyone has this six sense for things being written by it and, particularly in English, register is one of oldest forms of 'discrimination' in the language/culture, and certainly persists to the current day. And there's a huge competitive advantage to being able to have an LLM write things for you but make it sound like you yourself wrote it.) For translations considering how 'mature' the field is relative to everything else I'm a little surprised we haven't seen any rigorous benchmarks on round-trip translations, especially given that you aren't even limited to natural languages. You can take text, flip it into Klingon, and back, if you so chose. >- the inverse of summarization; given a bulleted list of facts, write a sensible set of paragraphs presenting those facts Except this one, this sounds unintentionally evil for some reasons i can't entirely put my finger on. Perhaps the inevitable conclusion of countless GPU farms spamming terabytes of text at each other predicated upon some short bullet point lists and the fact that "those are our marching orders" I guess Fundamentally I would like to live in a future where LLMs can only decrease the length of text written by humans, never increase it or fabricate it.
SanderNL 3y ago
Obligatory GPT4 summary of the article. "LLMs: The Summarization Superheroes of the Modern Era LLMs, like the one you're reading from, shine brightest when tasked with summarization. They may not craft unique insights, but they excel in distilling vast amounts of information into digestible bites. As information overload continues to plague our modern age, a tool that can swiftly and effectively condense info is invaluable. Bing Chat's emergence hints at a future where LLM-driven search engines index and summarize vast content troves, from books to blogs. But it's not just about search. Inspired by a recent Hacker News discussion likening the LLM movement to the early days of the web, it's clear that these models will reshape how we handle information, from helping CEOs extract corporate insights to tidying up the sprawling mess of unstructured data in enterprises. While LLMs have diverse potential applications, summarization stands out as their killer use case, at least for now."
huijzer 3y ago
I find using the word "regurgitate" when talking about LLMs pretty ironic since the word itself is almost never used except by people who "regurgitate" arguments from other people. Also, I wish people would reduce the use of such derogatory terms in arguments because it distracts from the core reasoning.
- sebmellen 3y ago
  I don't think "regurgitated" is a derogatory term at all, at least in the sense that I'm using the word — "[to] repeat (information) without analyzing or comprehending it." [0] [0]: https://www.google.com/search?hl=en&q=regurgitated%20definition https://www.google.com/search?hl=en&q=regurgitated%20definit...
janalsncm 3y ago
Just because the model is unlikely to come up with novel insights doesn’t mean it’s not a useful conversationalist. I have all sorts of ideas I like to bounce off of ChatGPT and it’s good at coming up with reasonable sanity checks in the meantime.
VirusNewbie 3y ago
I really think LLMs are great for translation between programming languages. I don't trust them to write complicated code, but when I am learning a new language or framework, the fastest way for me is to write a function in X and ask it to translate it to Y. It doesn't get fancy, it almost always compiles and if it makes a mistake it's not too bad.
RivieraKid 3y ago
Related thought - sometimes I ask for a "tldr" of long articles and people shame me for being lazy or something like that. I think this is really illogical. There's nothing wrong about reading a summary before or instead of reading the full article
mromanuk 3y ago
For me, the killer user case is having it as an AI assistant. A few months ago, I've started building a machine learning model for a keyboard iOS App as a solo dev, without python or deep learning knowledge. The model I wanted to build, needed text correction, word completion and prediction in one pass. I'm pretty confident, it was an impossible task without the help of chatGPT, first GPT-3.5 and then GPT-4.
- bazmattaz 3y ago
  Oh my I really hope that you are building the app of my dreams. I’m a an awful typer on my phone and on my desktop. Autocorrect is ok but I honestly don’t understand how we have all this amazing ML in the wild but we still seem to have the same shitty autocorrect. My dream autocorrect (powered by ML) learns how you type and how you speak. It learns what mistakes you always make (due to muscle memory) and corrects based on your past mistakes. Furthermore my dream autocorrect actually understands what you’re trying to say and corrects words based on the context of the conversation rather than how close the mistake you made is to some other word. For example, if someone texts me to say “I’m at the restaurant” and I type with a mistake “I’ll be right theyre” the autocorrect should understand the context of the conversation and rather than correct to “they’re” which is thinks is the correct spelling for this mistake, or autocorrects to “there” because it’s looking at the wider conversation. This is a poor example but I hope you get the point. I really don’t think what I’m asking for is hard. Apple should/could easily improve autocorrect
  - mromanuk 3y ago
    I would love to build that keyboard, but unfortunately there is a RAM limit for keyboard apps. My current model is not so smart as I would like. For fun, I just tested your example, using only 8 tokens (current limitation): "restaurant I’ll be right theyre" and it's corrected to "restaurant I’ll be right there", there is hope :) I'm pretty sure that I can expand it a few more tokens without much impact in memory and latency.
    - bazmattaz 3y ago
BaculumMeumEst 3y ago
the killer use case for me is quickly finding specific commands involving parts of tools that i don't use often. how do i turn the last three commits into a patch using magit? how do i grep for this weird pattern and exclude this directory? give me an elisp snippet that runs my last "compile" command that i used without prompting me, and bind it to f5.
- dontupvoteme 3y ago
  It's pretty amazing how great it is at bash/coreutils/etc in addition to everything else. The fact that it can output quite valid ffmpeg commands was the commandline cherry on the cake for me
Animats 3y ago
It's one of the things LLMs do really well. Summarization, translation, and guided blithering for generating ad copy work very well. Sometimes a bit of understanding seems to appear. Sometimes that's illusory. Here's a video where someone tries to use GPT-4 to read a drawing and generate G-code for a milling machine.[1] He puts in a simple textbook problem drawing, and reasonable G-code comes out. The trouble is, GPT-4 plagiarized the answer from a Chegg homework assignment which has the same design of 4 holes in a square. This is clear because the problem didn't specify the material thickness, and the GPT-4 result used the material thickness from the Chegg homework assignment. For harder, but still easy drawings, GPT-4 does worse. It's clear there's no underlying geometric model. [1] https://scanalyst.fourmilab.ch/t/programming-a-computer-numerical-control-machine-with-gpt-4/3860 https://scanalyst.fourmilab.ch/t/programming-a-computer-nume...
jerf 3y ago
Is there any work being done on summarization based on loading content in and doing some sort of manipulation on the internal representation? It seems to me that this ought to be a better solution for summarization than simply asking the model to summarize and hoping its training set taught it how to do what you want. Seems like it would generally fix the potential confabulation issues as well w.r.t. summarization at least.
jph 3y ago
LLM summaries are in the sweet spot: more specific than typical website short glossaries, easier to read quickly than Wikipedia, and well-prioritized for industry professionals. LLM summaries are providing excellent results (IMHO) for team training for technology groups, including for UI/UX, product management, innovation partnerships, software programming, and the like. If you want to see some of these, my GitHub has a six of these LLM summary guides for tech people: https://github.com/joelparkerhenderson https://github.com/joelparkerhenderson
falsenapkin 3y ago
I'm not familiar with the summarization or NLP space really but I remember ~2011-2015 I signed up for a couple of daily email services that summarized a number of news articles and the summaries were fantastic. I don't even remember what they were called, they eventually sold out with ads and worse formatting/summaries to make money I guess. I often use them as an example of 1) why LLMs are a bit old news for the summary use case and 2) how various LLM use cases will probably also be ruined because for a lot of people tools like that seem novel and useful but all I can see is onboarding to more advertising. So to someone who is actually knowledgeable in this space, are LLMs really that much better than whatever we had 10 years ago? Is this tech the key to some features we truly didn't have before?
- akiselev 3y ago
  Not an NLP expert but the biggest difference in my experience is guided focus, so to speak. When summarizing something huge like the US Code, for example, you can tell the LLM to focus on specific topics and anything adjacent to them so that it ignores irrelevant details (which is usually >99.9% of the text in my use case). The word relationships encoded in the LLM are really good at identifying important adjacent topics and entities. LLMs are also really good at the harder NLP problems like coreference resolution, dependency parsing, and relations which makes a huge difference when using recursive summarization on complex documents where something like "the Commisioner" might be defined at the beginning and used throughout a 100,000 token document. When instructed, the LLM can track the definitions in memory itself and even modify it live by calling OpenAI functions.
  - dontupvoteme 3y ago
    Might I ask how you use OpenAI's function calling here? That's the one bit of their functionality I haven't really explored.
    - akiselev 3y ago
      I use OpenAI function calling most of the time I use the OpenAI API since that's the easiest way to get structured data and implement retry logic. The simplest implementation is "retrieve_definition(word_to_lookup, word_to_replace)" with some number of tokens at the beginning of the prompt dedicated to definitions. You can use a separate LLM call with a long list of words (without their definitions) to do the actual selection since sometimes there might be ambiguity, which the LLM can usually figure out itself (it can also include both definitions when it's too uncertain if instructed). A more complex variant does multiple passes: first pass identifies ambiguous words in each chunk, second pass identifies their definitions, third pass does actual summarization using the output of the previous passes to craft the prompt.
julianeon 3y ago
I'd be interested to hear more about companies that are successfully doing this now.
- sebmellen 3y ago
  We use https://read.ai https://read.ai for every meeting, and it's quite good (not affiliated, just a customer :)
omadden 3y ago
I recently went to a talk by Don Syme, the creator of F#. He was talking about the LLM tools he is working on at GitHub Next for summarising PR and code base content. He seemed pretty upbeat about the impact these tools will have.

58 points by sebmellen 3y ago

I couldn't see a date on this piece but the Internet Archive first captured it on May 14th: https://web.archive.org/web/20230514232943/https://www.sebastianmellen.com/post/2023/the-killer-use-case-for-llms-is-summarization/ https://web.archive.org/web/20230514232943/https://www.sebas... (Relevant here because even five months is a long time in LLMs!) Update: turns out there's a date on https://www.sebastianmellen.com/post/ https://www.sebastianmellen.com/post/ - this post is from 18th March, so seven months ago.

kfarr 3y ago
Thanks, my initial response to summarization as killer feature: "is it though?" but the "historic" nature of this post makes a bit more sense now. Unlike crypto, there are many diverse use cases for LLM. While summarization is useful I don't think it's killer.

filterfiber 3y ago

Does anyone have any experience for how the generalized LLM's like llama/gpt4 compare to much smaller, purpose built summarizers like some of the FLAN variants?

wastewastewaste 3y ago

What is the most capable LLM for summarising stuff like long educational articles perhaps?

wahnfrieden 3y ago
GPT-4
dartos 3y ago
GPT-4 is usually the answer to “which llm is most/best…” But if the article is longer than the context window, you can try either RWKV or Claud-200k
- simonw 3y ago
  Has Claude 200k ever been made available? I've only found ways to access the 100k version.
simonw 3y ago
GPT-4 if it's less than 8,000 tokens (or 32,000 if you have access to that via the API). Claude for up to 100,000 tokens.
sebmellen 3y ago
Anthropic's Claude 2 is great for pasting large text files in, and Flan T5 seems to be excelling at summarization benchmarks: https://huggingface.co/docs/transformers/model_doc/flan-t5 https://huggingface.co/docs/transformers/model_doc/flan-t5.

mattew 3y ago

Don’t forget classification. Even the zero shot classification can be quite decent, but put a solid prompt together, or fine tune a set of examples and it’s way easier with an LLM than previous methods in my experience.

blueboo 3y ago
My favourite extant example of this GPT4’s 18-way rule-based reward model classifier, appendix A https://cdn.openai.com/papers/gpt-4-system-card.pdf https://cdn.openai.com/papers/gpt-4-system-card.pdf

simonw 3y ago

Summarization fits into my mental model of LLMs as "calculators for words". https://simonwillison.net/2023/Apr/2/calculator-for-words/ https://simonwillison.net/2023/Apr/2/calculator-for-words/ I think it's one of a small set of killer related use cases: - Summarization - Data and fact extraction, which is strongly related to summarization. Find all factual assertions in this news article. Extract the names of every person and location mentioned. - Question answering: given this context, answer this question. That's key to implementing Retrieval Augmented Generation, eg https://til.simonwillison.net/llms/embed-paragraphs#user-content-answering-a-question https://til.simonwillison.net/llms/embed-paragraphs#user-con... Aside from these summarization-adjacent applications, the thing I find most exciting is LLMs incredible aptitude at generating code. Code is especially interesting because it can be automatically executed (in a sandbox), which means many hallucinations can be automatically detected and fixed!

sitkack 3y ago
Summarization is all I have ever used Claude for, and it is excellent.
J_cst 3y ago
I never coded but I know what's needed and how things should be arranged to work. Result: I've developed and deployed a fully fledged web service both for human and machine use. Backend, front-end, endpoints, db, integration with our company systems, authentication, testing and deploy. In 5 sessions of 4 hours each. It was incredible while it was happening. I pay openai.
- unblough 3y ago
  > This Connection is Untrusted Go Back The owner of costantini.pw has configured their website improperly. To protect your information from being stolen, Firefox has not connected to this website. I tried to see what it built for you, and this was the only url in your bio.
  - J_cst 3y ago
    That's a personal profile page which has an expired certificate, sorry for that. If you're (or anyone really) interested in what I did and would like to try it out for curiosity and feedback, please feel free to reach me at j@costantini.pw as I'd be pleased to share it. Thanks

macspoofing 3y ago

Kind of .. depending on how sensitive you are to accuracy. Also also, we've had powerful summarization techniques for decades now (deep-learning, or otherwise) - and they aren't susceptible to the hallucinations of present day Generative LLMs.

simonw 3y ago
Which summarization models aren't susceptible to hallucination? Is there any good writing out there about this issue?
- netruk44 3y ago
  I found an 'old' (2016) reddit thread describing how auto tldr works, which doesn't suffer from hallucination (because it's only a statistical model that outputs original sentences from the text): https://old.reddit.com/r/askscience/comments/4s5b5q/how_exactly_does_a_autotldrbot_work/ https://old.reddit.com/r/askscience/comments/4s5b5q/how_exac...
- sebmellen 3y ago
  https://huggingface.co/docs/transformers/model_doc/t5 https://huggingface.co/docs/transformers/model_doc/t5 is quite good for summarization!
dontupvoteme 3y ago
>we've had powerful summarization techniques for decades now (deep-learning, or otherwise) - and they aren't susceptible to the hallucinations of present day Generative LLMs. do you mean BERT, Seq2Seq and similar? My biggest takeaway from summarization tasks was that the benchmarks/evals seemed rather.. lacking.
awkward 3y ago
Like many use cases for LLMs, this seems like it would benefit a lazy student trying to avoid work. Rolling in some statements that are related but not mentioned in the input isn't a bad thing when you're studying for a test. However, for most use cases, falsely attributing statements is a terrible failure case for a summarizing tool.

xianshou 3y ago

I would agree that summarization is a killer use case, but would actually argue for three: 1. Condense (summarize, synthesize, extract, etc.) 2. Expand (explain, copilot, suggest, etc.) 3. Transform (translation, style transfer, switch modality between text/image/audio, etc.) All of the major consumer use cases of LLMs at this point boil down to one of these three core operations. To the extent that LLMs allow you to perform "magic", that magic is comprised by a chain of {condense, expand, transform} operations, each of which could be performed in half an hour by a reasonably informed college graduate with access to Google. Even if chain length maxes out at 3 or 4 for production-grade applications, that solves a frightening number of problems.

jonny_eh 3y ago
Chatting seems very popular too.
- dontupvoteme 3y ago
  I don't understand why this got downvoted, but there's absolutely a huge unserved(or, rather, anti-served) and somewhat-looked-down-upon market here, and it's actually seemingly more women then men when i ran some numbers.
- tomjen3 3y ago
  If LLMs can be a cheap and instant therapist (even if they are just somebody who listens) then they could do a massive amount of good in this world.
Karrot_Kream 3y ago
I'm an early engineer at a startup that has nothing to do with AI. I use LLMs for 2 and 3 all the time. A lot of times we'll release a new feature and I'll write my own fairly technical summary of it, then I'll ask an LLM to either target it for a specific channel (e.g. create a version of the post targeting execs vs creating one that's focused on users) or condense it down if we're posting a summary onto a social media channel. In all cases I check the LLM's work, but I rarely have to do much. When I led teams at a big tech, I generally had a human whose job it was to do this kind of copy refinement/editing when my teams released new features. I find the LLM to be just as good, though obviously our reach is much smaller and the consequences of a mistake are much milder given our size. This is the kind of productivity boost I think of that LLMs offer; pre-LLM we would probably contract out a copy editor for this kind of work but now we do it all in-house and don't have to deal with working with a contractor's hours.

dontupvoteme 3y ago

I would argue it's translation with summarization as a close second, but not for people who spend much time on HN since their command of English is almost always good enough. Tools like DeepL already were cutting into the field of translation/localization, but now we can have something which can basically translate ANY language to ANY language (even things like fictional languages from e.g. LoTR, although that may be very hallucinogenic due to the paltry amount of training data) rather rapidly (gpt3) or rather slowly but more accurately (gpt4) including understanding of things like metaphors and idioms. Not to mention you can probably fix some of the choice of vocab and style just by specifying (once converted to target language) "reword in the stylings of X" or countless other multi-invocation approaches to the previously very expensive and artistic task of converting human thought from one variation to another. The biggest problem is probably over-training on english (and being dependent on openai), but even Microsoft made a LLaMA[!] finetune just for translation which iirc was significantly better than the non-translation versions from everyone else

growt 3y ago

„However, LLMs appear to be quite good at knowing what they do and don’t know“ Really? I think LLMs have not the slightest idea what they know and you have to go through great lengths to stop them from making stuff up.

sebmellen 3y ago
They certainly can be: https://huggingface.co/docs/transformers/model_doc/t5 https://huggingface.co/docs/transformers/model_doc/t5, and moreso if you plug in data to them. If you paste text into ChatGPT and then ask about specific information within the text, it's pretty good at answering.

Zambyte 3y ago

I think the "killer use case" is slightly more general: operations in linguistic space, of which summarization is one. Some other useful uses include: - the inverse of summarization; given a bulleted list of facts, write a sensible set of paragraphs presenting those facts - semantically searching in large documents - make some text sound more professional / casual - translate some text between human languages (they're not perfect at this yet, but they're surprisingly good, and I think they will be the way to do translations in the future). - expand abbreviations given some context The key here is that none of these depend on the language model injecting factual information into its output. All of the facts are encoded in your input - the language model just does some operation in linguistic space. If you don't give it the opportunity to give you incorrect assertions, it won't give you incorrect assertions.

earthboundkid 3y ago
Another application is to use the LLM to build an embeddings database, and then do semantic search only within that database. Don't just let the LLM talk about whatever it wants to hallucinate up, force it to use a set of known good answers while accepting natural language input and summarizing documents in the known good dataset.
dontupvoteme 3y ago
everything there sounds great (though i suspect for actual human-passing linguistic register-transfer we will need something different than the bog standard GPT models because everyone has this six sense for things being written by it and, particularly in English, register is one of oldest forms of 'discrimination' in the language/culture, and certainly persists to the current day. And there's a huge competitive advantage to being able to have an LLM write things for you but make it sound like you yourself wrote it.) For translations considering how 'mature' the field is relative to everything else I'm a little surprised we haven't seen any rigorous benchmarks on round-trip translations, especially given that you aren't even limited to natural languages. You can take text, flip it into Klingon, and back, if you so chose. >- the inverse of summarization; given a bulleted list of facts, write a sensible set of paragraphs presenting those facts Except this one, this sounds unintentionally evil for some reasons i can't entirely put my finger on. Perhaps the inevitable conclusion of countless GPU farms spamming terabytes of text at each other predicated upon some short bullet point lists and the fact that "those are our marching orders" I guess Fundamentally I would like to live in a future where LLMs can only decrease the length of text written by humans, never increase it or fabricate it.

SanderNL 3y ago

Obligatory GPT4 summary of the article. "LLMs: The Summarization Superheroes of the Modern Era LLMs, like the one you're reading from, shine brightest when tasked with summarization. They may not craft unique insights, but they excel in distilling vast amounts of information into digestible bites. As information overload continues to plague our modern age, a tool that can swiftly and effectively condense info is invaluable. Bing Chat's emergence hints at a future where LLM-driven search engines index and summarize vast content troves, from books to blogs. But it's not just about search. Inspired by a recent Hacker News discussion likening the LLM movement to the early days of the web, it's clear that these models will reshape how we handle information, from helping CEOs extract corporate insights to tidying up the sprawling mess of unstructured data in enterprises. While LLMs have diverse potential applications, summarization stands out as their killer use case, at least for now."

huijzer 3y ago

I find using the word "regurgitate" when talking about LLMs pretty ironic since the word itself is almost never used except by people who "regurgitate" arguments from other people. Also, I wish people would reduce the use of such derogatory terms in arguments because it distracts from the core reasoning.

sebmellen 3y ago
I don't think "regurgitated" is a derogatory term at all, at least in the sense that I'm using the word — "[to] repeat (information) without analyzing or comprehending it." [0] [0]: https://www.google.com/search?hl=en&q=regurgitated%20definition https://www.google.com/search?hl=en&q=regurgitated%20definit...

janalsncm 3y ago

Just because the model is unlikely to come up with novel insights doesn’t mean it’s not a useful conversationalist. I have all sorts of ideas I like to bounce off of ChatGPT and it’s good at coming up with reasonable sanity checks in the meantime.

VirusNewbie 3y ago

I really think LLMs are great for translation between programming languages. I don't trust them to write complicated code, but when I am learning a new language or framework, the fastest way for me is to write a function in X and ask it to translate it to Y. It doesn't get fancy, it almost always compiles and if it makes a mistake it's not too bad.

RivieraKid 3y ago

Related thought - sometimes I ask for a "tldr" of long articles and people shame me for being lazy or something like that. I think this is really illogical. There's nothing wrong about reading a summary before or instead of reading the full article

mromanuk 3y ago

For me, the killer user case is having it as an AI assistant. A few months ago, I've started building a machine learning model for a keyboard iOS App as a solo dev, without python or deep learning knowledge. The model I wanted to build, needed text correction, word completion and prediction in one pass. I'm pretty confident, it was an impossible task without the help of chatGPT, first GPT-3.5 and then GPT-4.

bazmattaz 3y ago
Oh my I really hope that you are building the app of my dreams. I’m a an awful typer on my phone and on my desktop. Autocorrect is ok but I honestly don’t understand how we have all this amazing ML in the wild but we still seem to have the same shitty autocorrect. My dream autocorrect (powered by ML) learns how you type and how you speak. It learns what mistakes you always make (due to muscle memory) and corrects based on your past mistakes. Furthermore my dream autocorrect actually understands what you’re trying to say and corrects words based on the context of the conversation rather than how close the mistake you made is to some other word. For example, if someone texts me to say “I’m at the restaurant” and I type with a mistake “I’ll be right theyre” the autocorrect should understand the context of the conversation and rather than correct to “they’re” which is thinks is the correct spelling for this mistake, or autocorrects to “there” because it’s looking at the wider conversation. This is a poor example but I hope you get the point. I really don’t think what I’m asking for is hard. Apple should/could easily improve autocorrect
- mromanuk 3y ago
  I would love to build that keyboard, but unfortunately there is a RAM limit for keyboard apps. My current model is not so smart as I would like. For fun, I just tested your example, using only 8 tokens (current limitation): "restaurant I’ll be right theyre" and it's corrected to "restaurant I’ll be right there", there is hope :) I'm pretty sure that I can expand it a few more tokens without much impact in memory and latency.
  - bazmattaz 3y ago

BaculumMeumEst 3y ago

the killer use case for me is quickly finding specific commands involving parts of tools that i don't use often. how do i turn the last three commits into a patch using magit? how do i grep for this weird pattern and exclude this directory? give me an elisp snippet that runs my last "compile" command that i used without prompting me, and bind it to f5.

dontupvoteme 3y ago
It's pretty amazing how great it is at bash/coreutils/etc in addition to everything else. The fact that it can output quite valid ffmpeg commands was the commandline cherry on the cake for me

Animats 3y ago

It's one of the things LLMs do really well. Summarization, translation, and guided blithering for generating ad copy work very well. Sometimes a bit of understanding seems to appear. Sometimes that's illusory. Here's a video where someone tries to use GPT-4 to read a drawing and generate G-code for a milling machine.[1] He puts in a simple textbook problem drawing, and reasonable G-code comes out. The trouble is, GPT-4 plagiarized the answer from a Chegg homework assignment which has the same design of 4 holes in a square. This is clear because the problem didn't specify the material thickness, and the GPT-4 result used the material thickness from the Chegg homework assignment. For harder, but still easy drawings, GPT-4 does worse. It's clear there's no underlying geometric model. [1] https://scanalyst.fourmilab.ch/t/programming-a-computer-numerical-control-machine-with-gpt-4/3860 https://scanalyst.fourmilab.ch/t/programming-a-computer-nume...

jerf 3y ago

Is there any work being done on summarization based on loading content in and doing some sort of manipulation on the internal representation? It seems to me that this ought to be a better solution for summarization than simply asking the model to summarize and hoping its training set taught it how to do what you want. Seems like it would generally fix the potential confabulation issues as well w.r.t. summarization at least.

jph 3y ago

LLM summaries are in the sweet spot: more specific than typical website short glossaries, easier to read quickly than Wikipedia, and well-prioritized for industry professionals. LLM summaries are providing excellent results (IMHO) for team training for technology groups, including for UI/UX, product management, innovation partnerships, software programming, and the like. If you want to see some of these, my GitHub has a six of these LLM summary guides for tech people: https://github.com/joelparkerhenderson https://github.com/joelparkerhenderson

falsenapkin 3y ago

I'm not familiar with the summarization or NLP space really but I remember ~2011-2015 I signed up for a couple of daily email services that summarized a number of news articles and the summaries were fantastic. I don't even remember what they were called, they eventually sold out with ads and worse formatting/summaries to make money I guess. I often use them as an example of 1) why LLMs are a bit old news for the summary use case and 2) how various LLM use cases will probably also be ruined because for a lot of people tools like that seem novel and useful but all I can see is onboarding to more advertising. So to someone who is actually knowledgeable in this space, are LLMs really that much better than whatever we had 10 years ago? Is this tech the key to some features we truly didn't have before?

akiselev 3y ago
Not an NLP expert but the biggest difference in my experience is guided focus, so to speak. When summarizing something huge like the US Code, for example, you can tell the LLM to focus on specific topics and anything adjacent to them so that it ignores irrelevant details (which is usually >99.9% of the text in my use case). The word relationships encoded in the LLM are really good at identifying important adjacent topics and entities. LLMs are also really good at the harder NLP problems like coreference resolution, dependency parsing, and relations which makes a huge difference when using recursive summarization on complex documents where something like "the Commisioner" might be defined at the beginning and used throughout a 100,000 token document. When instructed, the LLM can track the definitions in memory itself and even modify it live by calling OpenAI functions.
- dontupvoteme 3y ago
  Might I ask how you use OpenAI's function calling here? That's the one bit of their functionality I haven't really explored.
  - akiselev 3y ago
    I use OpenAI function calling most of the time I use the OpenAI API since that's the easiest way to get structured data and implement retry logic. The simplest implementation is "retrieve_definition(word_to_lookup, word_to_replace)" with some number of tokens at the beginning of the prompt dedicated to definitions. You can use a separate LLM call with a long list of words (without their definitions) to do the actual selection since sometimes there might be ambiguity, which the LLM can usually figure out itself (it can also include both definitions when it's too uncertain if instructed). A more complex variant does multiple passes: first pass identifies ambiguous words in each chunk, second pass identifies their definitions, third pass does actual summarization using the output of the previous passes to craft the prompt.

julianeon 3y ago

I'd be interested to hear more about companies that are successfully doing this now.

sebmellen 3y ago
We use https://read.ai https://read.ai for every meeting, and it's quite good (not affiliated, just a customer :)

omadden 3y ago

I recently went to a talk by Don Syme, the creator of F#. He was talking about the LLM tools he is working on at GitHub Next for summarising PR and code base content. He seemed pretty upbeat about the impact these tools will have.