3 ms·
Summarization fits into my mental model of LLMs as "calculators for words". https://simonwillison.net/2023/Apr/2/calculator-for-words/ https://simonwillison.net
by simonw 3y ago
Summarization fits into my mental model of LLMs as "calculators for words". https://simonwillison.net/2023/Apr/2/calculator-for-words/ https://simonwillison.net/2023/Apr/2/calculator-for-words/
I think it's one of a small set of killer related use cases:
- Summarization
- Data and fact extraction, which is strongly related to summarization. Find all factual assertions in this news article. Extract the names of every person and location mentioned.
- Question answering: given this context, answer this question. That's key to implementing Retrieval Augmented Generation, eg https://til.simonwillison.net/llms/embed-paragraphs#user-content-answering-a-question https://til.simonwillison.net/llms/embed-paragraphs#user-con...
Aside from these summarization-adjacent applications, the thing I find most exciting is LLMs incredible aptitude at generating code.
Code is especially interesting because it can be automatically executed (in a sandbox), which means many hallucinations can be automatically detected and fixed!
- sitkack 3y agoSummarization is all I have ever used Claude for, and it is excellent.
- J_cst 3y agoI never coded but I know what's needed and how things should be arranged to work. Result: I've developed and deployed a fully fledged web service both for human and machine use. Backend, front-end, endpoints, db, integration with our company systems, authentication, testing and deploy. In 5 sessions of 4 hours each. It was incredible while it was happening. I pay openai.
- unblough 3y ago> This Connection is Untrusted Go Back The owner of costantini.pw has configured their website improperly. To protect your information from being stolen, Firefox has not connected to this website. I tried to see what it built for you, and this was the only url in your bio.
- J_cst 3y agoThat's a personal profile page which has an expired certificate, sorry for that. If you're (or anyone really) interested in what I did and would like to try it out for curiosity and feedback, please feel free to reach me at j@costantini.pw as I'd be pleased to share it. Thanks
- msp26 3y agoThey're incredibly good at data extraction. You can start from zero and have a decent pipeline in hours.
- jjtheblunt 3y ago> many hallucinations can be automatically detected and fixed! how? you just set off my halting-problem neuroses with that claim, since how do you know what inputs to test with, in your idea?
- simonw 3y agoHave you tried ChatGPT Code Interpreter (now renamed to Advanced Data Analysis)? It's quite something to watch. It writes code, then executes it, gets an error, adjusts the code to fix the error, runs it again... I've seen it loop like that 4-5 times before it gets to code that works.
- dontupvoteme 3y agoThe halting problem is one matter (and LLMs will often write terrible code performance wise, though you can often determine this automatically with a myriad of approaches and either fix it or discard it and start anew) but in terms of "will it run" this is more akin to the old "will it blend" youtube channel than any complicated mathematical problems which require Church or Turing to intervene on. This is a simple smoke test and the biggest problem I've found is that it simply writes code which probably worked in the past in its training data (or is very close to what would work) but is just so ever so slightly off. This was just a front page article today because flask3.0 broke a login module, so it's still going to be a problem no matter how recent your training data is. You don't even need a python interpreter fired up for the first checks, in fact, you can generate the tree of valid libs, functions, their sigs, etc within your interpreter, save it as a json and and directly parse incoming python markup to evaluate if it is valid or not (at least, for your particular environment) I'm actually a bit surprised that openAI themselves doesn't do this since they have a fixed sandbox and it would save them compute cycles to handle the "little fix" directly with either non-LLM code or a much smaller "LLM call" to figure out how to call a function for lib X version Y which is similar to this. Or you could probably just do some non-insignificant double digit percentage of the time with Levenshtein distance (or similar fuzzy matching) and having an internal db of every libraries functions and function signatures for every version and automatically identifying where something changed and reducing your problem space to those.
- 3y ago
- bane 3y agoIn a sense, LLMs are already lossy compressors. Summarization is sort of just turning up the lossy part so it seems to be a natural use-case for them.
- deleted 3y ago[deleted]