6 ms·

I can see this being useful iif the content is generated on demand and then discarded. Publishing AI generated material is generally speaking a horrible idea a

by lukev 3y ago

I can see this being useful iif the content is generated on demand and then discarded. Publishing AI generated material is generally speaking a horrible idea and does nobody any good (at least until accuracy levels get much much better.) Even if they do it well and truthfully (which they don't) current LLMs can only summarize, digest, and restate. There is no non-transient value add. LLMs may have a place to help query, but there is no reason to publish LLM regurgitations alongside the ground truth used to generate them.

tiptup300 3y ago
are llms able to look at a list of categories, read content and then determine which of the categories apply?
- warkdarrior 3y ago
  This is a very broad question, but in short, yes, they can do this. It depends on the granularity and overlap of those categories.
- msp26 3y ago
  Absolutely
- OKRainbowKid 3y ago
  This could be achieved by generating embeddings of suitable representations of the categories once, and then embedding the content at runtime, before using some distance metric to find matching categories for the content embedding.
petercooper 3y ago
current LLMs can only summarize, digest, and restate. There is no non-transient value add. Though, at a stretch, Wikipedia itself could be considered based around summarization, digesting, and restating/citing things said elsewhere, given its policy of verifiability: "Even if you are sure something is true, it must have been previously published in a reliable source before you can add it." Now, LLMs aren't well known for their citation skills, to be fair.. :-)
- lukev 3y ago
  Yeah, when AIs can comprehensively cite their sources I might change my opinion on that. Though note that there still isn't any need to publish static content. The power of LLMs is that they can be dynamic and responsive! Even if we hypothesize that it were possible for a LLM to write a high-quality wikipedia-like output, generating the whole thing statically in advance like existing Wikipedia would be relatively pointless. It'd be much more interesting to generate arbitrary (and infinite!) pages on demand.
- skissane 3y ago
  I remember one time, I wanted to update a Wikipedia article with some more recent developments, but was having trouble working out the best way of wording them. I found a newspaper article discussing those developments, so I scribbled down some notes summarising the article's content, and then asked ChatGPT to reword them for me more eloquently. And its output was good enough for me to paste in to the Wikipedia article with only minor adjustments, along with a cite to the newspaper article. I'm sure I'm not the only person to have done something like that.
CuriouslyC 3y ago
I think bootstrapping documentation with LLM output is a great practice. It's a wiki, people can update it from a baseline, just as long as they can see what was LLM generated to know that it shouldn't be taken as absolute truth. The hardest part of good documentation is getting started. Once there are docs in place it's usually much easier to revise and correct than it would have been to write correctly by hand the first time. Think of it like automating a rough draft.
- msp26 3y ago
  Maybe the generated text could be a slightly different colour until it's verified. But you'd have to make sure there's no easy way of verifying everything mindlessly without having read it.
  - skissane 3y ago
    Technique I've found helpful personally: get the LLM to generate text in small chunks (e.g. a paragraph at a time). After generating each chunk, it is immediately reviewed by a human, who can edit it manually to correct any mistakes, ask the LLM to try again, or prompt the LLM to make specified changes. When the human is satisfied with that chunk, it is saved, and we move on to the next one. Sometimes, the output the LLM generates is correct and I'm just approving it. Other times, it is mostly right, and I can easily identify and correct its errors. Yet other times, it is totally wrong, but often typing out why it is wrong is a good start to actually generating correct text. Often (but not always), the kinds of false assumptions which LLMs make are similar to those a human reader would make, so stuff like "A and B sound very similar but, in the context of this system, actually have completely different meanings" is a useful addition to documentation anyway. The worst case scenario is the LLM generates something which is subtlety wrong, and the human review fails to pick up on the subtle error. But, that's something which can happen even with no LLMs involved at all. It isn't uncommon for people to make subtle errors in documents they write (often because they are misremembering something) and for those subtle errors not to be picked up during the review process. I'm not convinced the odds of this happening with an LLM-assisted workflow are significantly greater than with a purely human workflow.
- 3y ago
visarga 3y ago
> current LLMs can only summarize, digest, and restate. There is no non-transient value add. No, you're wrong. LLMs create new experiences after deployment, either by assisting humans, or by solving tasks they can validate, such as code or game play. In fact any deployed LLM gets to be embedded in a larger system - a chat room, a code running environment, a game, a simulation, a robot or inside a company - it can learn from iterative tasks because each following iteration carries some kind of real world feedback. Besides that, LLMs trivially learn new concepts and even new skills with a short explanation or demonstration, they can be pulled out of their training distribution and collect experiences doing new things. If OpenAI has 100M users and they consume 10K tokens/user/month, that makes for 1 trillion tokens of human-AI interaction rich with new experiences and feedback. In the text modality LLMs have consumed most of the high quality human text, that is why all SOTA models are roughly on par, they trained on the same data. That means easy time is over, AI has caught up with all human language data. But from now on AI models need to create experiences of their own, because learning from your own mistakes is much faster. The more they get used, the more feedback and new information they collect. The environment is the teacher, not everything is written in books. And all that text - the trillions of tokens they are going to speak to us - in turn contributes to scientific discoveries and progress, and percolate back into the next training set. LLMs have massive impact at language level on people so by extension on the physical world and culture. They have already influenced language and the arts. LLMs can create new experiences, learn new skills, and have a significant impact through widespread deployment and interaction. There is "value add" if you look at the grand picture.
- __loam 3y ago
  https://twitter.com/itsandrewgao/status/1689634145717379074?s=46&t=V6Q3HMD52cXzbDBoJfIC2w https://twitter.com/itsandrewgao/status/1689634145717379074?... Yeah this is really having a positive impact on scientific discovery.
  - visarga 3y ago
    Your link only shows what unscrupulous people would do. Here is a LLM with search doing competitive level coding: https://deepmind.google/discover/blog/competitive-programming-with-alphacode/ https://deepmind.google/discover/blog/competitive-programmin... and in general, applying evolutionary methods on top: https://scholar.google.com/scholar?cites=1264402983985539857&as_sdt=2005&sciodt=0,5&hl=ro https://scholar.google.com/scholar?cites=1264402983985539857... The explanation is simple - either learn from past experience which is human text for now, or learn from present time experience which comes from the environment. The environment is where LLMs can do novel things.
observationist 3y ago
This is categorically untrue. Publishing material generated like this is going to be generally better than human generated content. It takes less time, can be systematically tested and rigorous, and you can specifically avoid the pitfalls of bias and prejudice. A system like this is multilayered, with prompts going through the whole problem solving process, considering the information presented, assuring quality and factuality, assigning the necessary citations and documentation for claims. Accuracy isn't a problem. The way in which AI is used creates the problem - ChatGPT and most chat based models are single pass, query/response type interactions with models. Sometimes you get a second pass with a moderation system, doing a review to ensure offensive or illegal things get filtered out. Without any additional testing and prompt engineering, you're going to run into hallucinations, inefficient formulations, random "technically correct but not very useful" generations, and so forth. Raw ChatGPT content shouldn't be published without significant editing and going through the same quality review process any human written text should go through. What Storm accomplishes is an algorithmic and methodical series of problem solving steps, each of which can be tested and verified and validated. This is synthesized in a particular way, intended as a factual reference article. Presumably you could insert debiasing and checks for narrative or political statements, ensuring attribution and citation occur for quotations, and rephrasing anything generated by the AI as a neutral, academic statement of fact with no stylistic and artistic features. This is significantly different from the almost superficial interactions you get with chatbots, unless you specifically engineer your prompts and cycle through similar problem solving methods. Tasks like this are well within the value add domain of current AI capabilities. Compared to the absolute trash of SEO optimized blog posts, the agenda driven, ulterior laden rants and rambles in social media, and the "I'm oh-so-cleverly influencing the narrative" articles posted to Wikipedia by humans, content like this is a clear winner in quality, in my opinion. AI isn't at the point where it's going to spit out well grounded novel answers to things like "what's the cure for cancer?" but it can absolutely produce a principled and legible explanation of a phenomenon or collection of facts about a thing.
- deleted 3y ago
  [deleted]
- JohnFen 3y ago
mistrial9 3y ago
excellent virtue signalling here -- however, commercial publishers, competitive attorneys, advertising sales and others are literally falling over themselves in an avalanche of doing exactly this, that you advise against (politely). This moment reminds me very much of the late 1990s when it was common knowledge that "claim jumping a domain name" is very rude and not advisable, or the common knowledge among intellectuals that "ads will ruin the Internet" .. yes, polite people did not make companies to claim jump domain name registration, or push annoying and repetitive ads on the Internet.. but..
elicksaur 3y ago
How could it be true that the content generated would have value only if it is not persisted? If it doesn’t have value for being saved and published, why would it have value for the person viewing it ephemerally?
- vineyardmike 3y ago
  Obviously LLMs aren't deterministic, but they generally can be modeled as a simple, stateless, function: Given text, generate text. Depending on your views towards wasting energy, why store something that can be "trivially" re-generated? One reason to NOT want to persist an LLM output is to avoid contaminating the internet with LLM text that could be confused for human text. This is useful for historical reasons (what did people say/think) and for future LLM training purposes. In the future, there might be other reasons to not persist LLM-generated text. If you store the inputs instead of outputs then it can be "replayed" with a new LLM. I can generate some text via an LLM running on my phone, but if I later wanted a higher-quality output, it'd be easier to re-generate the data on my laptop later (or the cloud, obviously). One related idea that comes to mind is personalization - let the LLM use language each reader is familiar with, or focusing on different bits of info to different people. Maybe if an LLM knows I have a PhD in a certain subject, it'd summarize a research paper differently than it'd summarize it for my neighbor who has different preexisting knowledge.
  - Hugsun 3y ago
    If you set the temperature to zero, LLMs are deterministic.
xbmcuser 3y ago
I would love for something like this to be attached to LibGen where it reads the millions of scientific papers. As in my opinion human knowledge today is more than what a group of people can handle let alone individuals. Their is lot of domain specific knowledge that would translate and be used in other domains but unless by chance a human with speciality in both domains sees it will not get ported or assimilated in the 2nd domain.

by lukev 3y ago

tiptup300 3y ago

are llms able to look at a list of categories, read content and then determine which of the categories apply?

warkdarrior 3y ago
This is a very broad question, but in short, yes, they can do this. It depends on the granularity and overlap of those categories.
msp26 3y ago
Absolutely
OKRainbowKid 3y ago
This could be achieved by generating embeddings of suitable representations of the categories once, and then embedding the content at runtime, before using some distance metric to find matching categories for the content embedding.

petercooper 3y ago

current LLMs can only summarize, digest, and restate. There is no non-transient value add. Though, at a stretch, Wikipedia itself could be considered based around summarization, digesting, and restating/citing things said elsewhere, given its policy of verifiability: "Even if you are sure something is true, it must have been previously published in a reliable source before you can add it." Now, LLMs aren't well known for their citation skills, to be fair.. :-)

lukev 3y ago
Yeah, when AIs can comprehensively cite their sources I might change my opinion on that. Though note that there still isn't any need to publish static content. The power of LLMs is that they can be dynamic and responsive! Even if we hypothesize that it were possible for a LLM to write a high-quality wikipedia-like output, generating the whole thing statically in advance like existing Wikipedia would be relatively pointless. It'd be much more interesting to generate arbitrary (and infinite!) pages on demand.
skissane 3y ago
I remember one time, I wanted to update a Wikipedia article with some more recent developments, but was having trouble working out the best way of wording them. I found a newspaper article discussing those developments, so I scribbled down some notes summarising the article's content, and then asked ChatGPT to reword them for me more eloquently. And its output was good enough for me to paste in to the Wikipedia article with only minor adjustments, along with a cite to the newspaper article. I'm sure I'm not the only person to have done something like that.

CuriouslyC 3y ago

I think bootstrapping documentation with LLM output is a great practice. It's a wiki, people can update it from a baseline, just as long as they can see what was LLM generated to know that it shouldn't be taken as absolute truth. The hardest part of good documentation is getting started. Once there are docs in place it's usually much easier to revise and correct than it would have been to write correctly by hand the first time. Think of it like automating a rough draft.

msp26 3y ago
Maybe the generated text could be a slightly different colour until it's verified. But you'd have to make sure there's no easy way of verifying everything mindlessly without having read it.
- skissane 3y ago
  Technique I've found helpful personally: get the LLM to generate text in small chunks (e.g. a paragraph at a time). After generating each chunk, it is immediately reviewed by a human, who can edit it manually to correct any mistakes, ask the LLM to try again, or prompt the LLM to make specified changes. When the human is satisfied with that chunk, it is saved, and we move on to the next one. Sometimes, the output the LLM generates is correct and I'm just approving it. Other times, it is mostly right, and I can easily identify and correct its errors. Yet other times, it is totally wrong, but often typing out why it is wrong is a good start to actually generating correct text. Often (but not always), the kinds of false assumptions which LLMs make are similar to those a human reader would make, so stuff like "A and B sound very similar but, in the context of this system, actually have completely different meanings" is a useful addition to documentation anyway. The worst case scenario is the LLM generates something which is subtlety wrong, and the human review fails to pick up on the subtle error. But, that's something which can happen even with no LLMs involved at all. It isn't uncommon for people to make subtle errors in documents they write (often because they are misremembering something) and for those subtle errors not to be picked up during the review process. I'm not convinced the odds of this happening with an LLM-assisted workflow are significantly greater than with a purely human workflow.
3y ago

visarga 3y ago

> current LLMs can only summarize, digest, and restate. There is no non-transient value add. No, you're wrong. LLMs create new experiences after deployment, either by assisting humans, or by solving tasks they can validate, such as code or game play. In fact any deployed LLM gets to be embedded in a larger system - a chat room, a code running environment, a game, a simulation, a robot or inside a company - it can learn from iterative tasks because each following iteration carries some kind of real world feedback. Besides that, LLMs trivially learn new concepts and even new skills with a short explanation or demonstration, they can be pulled out of their training distribution and collect experiences doing new things. If OpenAI has 100M users and they consume 10K tokens/user/month, that makes for 1 trillion tokens of human-AI interaction rich with new experiences and feedback. In the text modality LLMs have consumed most of the high quality human text, that is why all SOTA models are roughly on par, they trained on the same data. That means easy time is over, AI has caught up with all human language data. But from now on AI models need to create experiences of their own, because learning from your own mistakes is much faster. The more they get used, the more feedback and new information they collect. The environment is the teacher, not everything is written in books. And all that text - the trillions of tokens they are going to speak to us - in turn contributes to scientific discoveries and progress, and percolate back into the next training set. LLMs have massive impact at language level on people so by extension on the physical world and culture. They have already influenced language and the arts. LLMs can create new experiences, learn new skills, and have a significant impact through widespread deployment and interaction. There is "value add" if you look at the grand picture.

__loam 3y ago
https://twitter.com/itsandrewgao/status/1689634145717379074?s=46&t=V6Q3HMD52cXzbDBoJfIC2w https://twitter.com/itsandrewgao/status/1689634145717379074?... Yeah this is really having a positive impact on scientific discovery.
- visarga 3y ago
  Your link only shows what unscrupulous people would do. Here is a LLM with search doing competitive level coding: https://deepmind.google/discover/blog/competitive-programming-with-alphacode/ https://deepmind.google/discover/blog/competitive-programmin... and in general, applying evolutionary methods on top: https://scholar.google.com/scholar?cites=1264402983985539857&as_sdt=2005&sciodt=0,5&hl=ro https://scholar.google.com/scholar?cites=1264402983985539857... The explanation is simple - either learn from past experience which is human text for now, or learn from present time experience which comes from the environment. The environment is where LLMs can do novel things.

observationist 3y ago

This is categorically untrue. Publishing material generated like this is going to be generally better than human generated content. It takes less time, can be systematically tested and rigorous, and you can specifically avoid the pitfalls of bias and prejudice. A system like this is multilayered, with prompts going through the whole problem solving process, considering the information presented, assuring quality and factuality, assigning the necessary citations and documentation for claims. Accuracy isn't a problem. The way in which AI is used creates the problem - ChatGPT and most chat based models are single pass, query/response type interactions with models. Sometimes you get a second pass with a moderation system, doing a review to ensure offensive or illegal things get filtered out. Without any additional testing and prompt engineering, you're going to run into hallucinations, inefficient formulations, random "technically correct but not very useful" generations, and so forth. Raw ChatGPT content shouldn't be published without significant editing and going through the same quality review process any human written text should go through. What Storm accomplishes is an algorithmic and methodical series of problem solving steps, each of which can be tested and verified and validated. This is synthesized in a particular way, intended as a factual reference article. Presumably you could insert debiasing and checks for narrative or political statements, ensuring attribution and citation occur for quotations, and rephrasing anything generated by the AI as a neutral, academic statement of fact with no stylistic and artistic features. This is significantly different from the almost superficial interactions you get with chatbots, unless you specifically engineer your prompts and cycle through similar problem solving methods. Tasks like this are well within the value add domain of current AI capabilities. Compared to the absolute trash of SEO optimized blog posts, the agenda driven, ulterior laden rants and rambles in social media, and the "I'm oh-so-cleverly influencing the narrative" articles posted to Wikipedia by humans, content like this is a clear winner in quality, in my opinion. AI isn't at the point where it's going to spit out well grounded novel answers to things like "what's the cure for cancer?" but it can absolutely produce a principled and legible explanation of a phenomenon or collection of facts about a thing.

deleted 3y ago
[deleted]
JohnFen 3y ago

mistrial9 3y ago

excellent virtue signalling here -- however, commercial publishers, competitive attorneys, advertising sales and others are literally falling over themselves in an avalanche of doing exactly this, that you advise against (politely). This moment reminds me very much of the late 1990s when it was common knowledge that "claim jumping a domain name" is very rude and not advisable, or the common knowledge among intellectuals that "ads will ruin the Internet" .. yes, polite people did not make companies to claim jump domain name registration, or push annoying and repetitive ads on the Internet.. but..

elicksaur 3y ago

How could it be true that the content generated would have value only if it is not persisted? If it doesn’t have value for being saved and published, why would it have value for the person viewing it ephemerally?

vineyardmike 3y ago
Obviously LLMs aren't deterministic, but they generally can be modeled as a simple, stateless, function: Given text, generate text. Depending on your views towards wasting energy, why store something that can be "trivially" re-generated? One reason to NOT want to persist an LLM output is to avoid contaminating the internet with LLM text that could be confused for human text. This is useful for historical reasons (what did people say/think) and for future LLM training purposes. In the future, there might be other reasons to not persist LLM-generated text. If you store the inputs instead of outputs then it can be "replayed" with a new LLM. I can generate some text via an LLM running on my phone, but if I later wanted a higher-quality output, it'd be easier to re-generate the data on my laptop later (or the cloud, obviously). One related idea that comes to mind is personalization - let the LLM use language each reader is familiar with, or focusing on different bits of info to different people. Maybe if an LLM knows I have a PhD in a certain subject, it'd summarize a research paper differently than it'd summarize it for my neighbor who has different preexisting knowledge.
- Hugsun 3y ago
  If you set the temperature to zero, LLMs are deterministic.

xbmcuser 3y ago

I would love for something like this to be attached to LibGen where it reads the millions of scientific papers. As in my opinion human knowledge today is more than what a group of people can handle let alone individuals. Their is lot of domain specific knowledge that would translate and be used in other domains but unless by chance a human with speciality in both domains sees it will not get ported or assimilated in the 2nd domain.