7 ms·

GPT-3, Esq? Evaluating AI Legal Summaries [pdf]

51 points by gavelin 6y ago

blamestross 6y ago
I'm getting concerned that even researchers who know better are anthropomorphising GPT-3 in their descriptions of it's output.
- make3 6y ago
  As a researcher, it's pretty obvious from the language and from the analysis of the author that he's a novice at machine learning who is a lawyer, not a researcher in machine learning
  - gavelin 6y ago
    Correct. I hope that meant that this was more accessible than the average write-up (and not less accurate!) :)
  - YeGoblynQueenne 6y ago
    The author seems to have a very clear understanding of how GPT-3 works and their down-to-Earth, plain language analysis is miles away from the wild flights of fancy we're used to reading about GPT-3. As a for instance, they didn't even use the word "understand" once, to refer to what GPT-3 is doing.
    - make3 6y ago
      I'm talking about knowledge of the NLP literature beyond GPT-3, and other machine learning basic technical stuff
      - YeGoblynQueenne 6y ago
        Is any of that referenced in the article in a way that needs a deep explanation which isn't there?
        gavelin 6y ago
        Thank you for the support—means a lot to me. I wrote the piece after finding that recent journalism on GPT-3 did not provide a sufficiently accurate snapshot of how vanilla GPT-3 scores on legal tasks (not to mention the misleading snapshots of promising sandbox outputs). Meanwhile, even the most capable people in the ML community do not get to the papers discussing the minutiae of handling the hard problems in legal texts. I never intended the piece to comprehensively address and solve the technical problems GPT-3 (or NLP more broadly) has in handling legal texts. The inability to examine and audit GPT-3 at different levels of the network makes makes any investigation a partially speculative endeavor. Instead, I merely wanted to provide an overview of outputs critiqued by a lawyer, and offer up some ideas of how to improve performance on legal tasks drawing from ML and legal knowledge. I think there is some constructive dialogue in this thread, and I am thrilled by that. Lawyers and engineers need to work together on this to be successful. The goal of summarization is to enable more people to accurately understand text in less time. It is pretty clear that the feedback of a lawyer in the training loop (at least to label meaning and context) would lead to a significant improvement. IIRC, Andrej Karpathy labeled a lot of data when his team achieved a 50% improved classification and detection jump on ImageNet. Can more (and better) labeled data get us to an accuracy level that is good enough to generate term sheets out of contracts? I would like to find out. I am interested in diving deeper and connecting with anyone who is game to tackle some of the critical challenges.
gwern 6y ago
Also of interest is Arbel's paper on GPT-3 (AID currently but he got GPT-3 access recently) legal summaries: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3740356 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3740356 One thing I would note is the potential for a feedback loop between summaries and the model: if a summary of a specific ToS or piece of law is wrong, you can hardwire an expert-vetted one (there's only so many ToSes or pieces of law and it'll be a long tail), and you can feed back in the correct one as training data to finetune the model. The bigger the GPT model, the smarter it is, and the less feedback it takes to correct its summaries: https://openai.com/blog/learning-to-summarize-with-human-feedback/ https://openai.com/blog/learning-to-summarize-with-human-fee...
- gavelin 6y ago
  Thanks for the paper link! I think your reasoning of hardwiring boilerplate to an expert-vetted (or written) summary is a much more accurate approach. If only we could scrape the clause explanation footnotes from quality sources I would not be forced to write them!
Der_Einzige 6y ago
This is at the intersection of my research interests. I'd love to see what happens if you run it on a large debate case with slight variations of the input prompt queries. I love to see all of these articles about legal summarization - but it's always covering abstractive summarization! I want an effective highlighter model. To be fair, I did build a system for using transformers to do extractive summarization in an unsupervised manner, but the results aren't that great. I'm sure that Lawyers would find something that could highlight the most legally salient sections to be extremely useful if it's reasonably accurate. I really did figure that we'd get effective word-level extractive summaries before we'd get effective abstractive summaries but I guess that intuition was wrong...
- staticautomatic 6y ago
  If keyphrase extraction and extractive summarization of legal documents interests you, I have a side project I’ve been wrenching on that I could use a hand with. Contact info in my profile.
make3 6y ago
The author seems to forget that fine-tuning exists all-together, which is how real-world NLP applications are actually made. In reply to the following paragraph, in a real world setting, this would be done through model fine-tuning on high quality data. "First, there must be greater transparency. The sources of GPT-3’s references must at least be referenceable and perhaps tweaked to follow a proper hierarchy of authorities. Although it is challenging to audit a 175 billion parameter algorithm, it would be beneficial to understand the most influential semantic parameters used to generate the output text. This would ideally enable users to choose word choice (perhaps as a level of sophistication), appropriate voice, and tone"
- gavelin 6y ago
  You are quite correct that fine-tuning is necessary to improve accuracy. I did not intend to dismiss that. The point I intended to make there is that it would be nice to be able to see whether, when interpreting a statute, GPT-3 relied heavily on a blog interpreting a statute v. legislative history v. Supreme Court precedent. The right approach would be to control the proper hierarchy of authorities. It would be helpful to understand, even as a textual matter, what GPT-3 was most heavily relying on for a given prediction. That would speed up categorically fixing bad prediction patterns. It is also true that high quality data is necessary. There is a reason why lawyers rely on Westlaw and LexisNexis to search for relevant laws and even scholarly articles. They are trusted sources. A better approach would rely on something like those with a more narrow universe of quality sources. There is a ton of labeling work that needs to be done, even beyond the “KeyCite” type of labels Westlaw applies to documents. Note that YC company www.rossintelligence.com ran into some trouble recently with Westlaw and LexisNexis. The quality v. quantity of data debate is particularly relevant here. The power of GPT-3 is in part supposed to come from the sheer scale of its training dataset size. It would be nice to leverage some of the semantic training from a large non-legal dataset to be able to stylistically output in layman’s terms while sourcing the authorities from more closely vetted sources.
dweekly 6y ago
tl;dr - Don't use GPT-3 to summarize your legal documents yet.
- gavelin 6y ago
  "tl;dr" may very well be the problem! There is a tendency of many tl;dr summaries on the web to oversimplify and skew concepts. If those are included in GPT-3's dataset, GPT-3's output will try to match the dataset style (according to the parameters) and likely not meet the legal standard. There is a section following the conclusion in the paper where I touch on other ways GPT-3 might be improved for the legal summarization use case. The only way we get there is by delving into the nuance of WHY GPT-3 is not yet good enough to replace lawyers, and HOW we can improve on it as an architecture.
  - dweekly 6y ago
    Absolutely, and I am grateful for your detailed analysis here. These kinds of spot checks on subjective quality for usability in a domain are a critical part of scoring "are we there yet?" and providing solid exemplars that are motivating to the researchers. It's not hard to imagine a GPT-5 recap in a few years with very different outcomes. Some of the motivation on my side is judging at which point it becomes appropriate to explore having an algorithm provide a summary of a medical conversation, since my day job at Medcorder is building a system that acts as a patient advocate to help them better understand what their doctors are saying. The universal feature request from day one has been summaries (which are fraught if you get them wrong!), so I'm keen to develop a sense of when we're going to get there. (As for the downvotes on my comment above, I'd bet they are, appropriately ironically, from people who themselves didn't actually read the article, since the comment was made with a wink to the fact that the article explicitly evaluates the appropriateness of "tl;dr" summaries and found them wanting.)
skybrian 6y ago
GPT-3 picks words literally at random (according to a probability distribution) so it would be good to run each experiment multiple times to get a sense of the probability distribution. I doubt it would change the conclusions, though.
- make3 6y ago
  In a professional setting, this is definitely done, and the generations are then ranked with separate models that predict different quality metrics, such as "interestingness", "safety" in an inclusiveness type of way, whether the answer seems to fit a style that you want, whether the facts in the answer seem to make sense, etc., and it makes a big difference actually
- gavelin 6y ago
  Good tip! I repeated a few inputs to see if the variation was significant enough to warrant including that, and as your intuition suggested, it was not with the parameters I selected. A more robust experiment should definitely include repeat attempts.
- minimaxir 6y ago
  There's a big difference between sampling from a learned probability distribution and "picks words literally at random". The temperature = 0 examples have zero sampling by construction, while higher temperatures having a slight sampling. That said, it doesn't hurt to have multiple attempts (aside from the cost of using GPT-3, ahem).
  - skybrian 6y ago
    I guess I could have left out "literally," but picking words from a learned probability distribution is picking at random. Since words are chosen in order, picking one word differently will likely mean that every word that follows will be different, and this choice between suffixes is not made intelligently. This randomness is there for a reason. Yes, you can turn it off, but then you get loops, which is obviously not intelligent. A bit of randomness makes it less obvious.
- 6y ago
minimaxir 6y ago
> Second, text ought to be tokenized (a term used in natural language processing wherein text is assigned a numerical value) at many different levels (character, word, sentence, paragraph, section, etc.) in order to make predictions that are relevant to the excerpt of a text, while also remaining consistent to the broader document. This is challenging because doing so involves a tremendous amount of computational resources. However, it may be necessary to accurately capture meaning at different levels of abstraction. I don't think this would be an improvement per se. The way Byte Pair Encodings are constructed that GPT-3 (and GPT-2) use is that they already take higher-level text representations and compress it down (into tokens), which are then reflected in the training of the model.
- Tarq0n 6y ago
  And the attention mechanism should take care of "sentence, paragraph, section" level context. Attributing this to tokenization is a weird mistake to make.
  - gavelin 6y ago
    Tarq0n: Also a great point. The question then is whether the attention mechanism is being triggered on the proper word or character sequences. Lawyers employ a form of attention mechanism when “issue spotting.” For example, in a non-compete I might scan for the duration (how many years until the client can join a similar venture?) and breadth (what is the definition of a similar venture?). For an attention mechanism to work well for legal summaries, it seems to me that it must trigger on many relevant context triggers at different levels. As an extreme example, if I put 50 contracts involving multiple different parties into one big document and the attention mechanism was triggered on the first document title, would the subsequent document titles sufficiently demarcate a new contract to reset the context? Or would the attention stay “on” the first high accuracy context match and mix up the terms and parties? In context of the paper, GPT-3 missed a ton of issues, indicating to me that the attention mechanism is not being properly triggered. Again I may be wrong that tokenizing and predicting based off of clauses, paragraphs and sections would help improve the output, but that was one way I thought would capture the different levels of context.
- 6y ago
Aulig 6y ago
Great read, thanks for sharing. It's always interesting to find out how AI is impacting occupations outside the technology sector.
- gavelin 6y ago
  Thank you for reading!
jll29 6y ago
To those that are interested in the state of the art in commercial legal summarization (US law) as of Q1/2021: https://arxiv.org/pdf/2102.05757.pdf https://arxiv.org/pdf/2102.05757.pdf
- gavelin 6y ago
  Great link. Thank you for sharing this.
MasterScrat 6y ago
Thank you for your analysis, it's great to have the insights of someone with both legal and ML backgrounds. I want to point out that a big part of building anything on GPT-3 (or other large LMs for that matter) is "prompt engineering", which means you try out thousands of prompts and sampling parameters until you find something that works reasonably well for your use case. Taking two default templates and a few different temperatures is like taking some tutorials for a new framework, building a proof of concept from them, then making a judgement from that. Sure, it can provide a good first assessment, but that's it. You would need much deeper experience to come to a meaningful conclusion.
- NovemberWhiskey 6y ago
  >I want to point out that a big part of building anything on GPT-3 (or other large LMs for that matter) is "prompt engineering", which means you try out thousands of prompts and sampling parameters until you find something that works reasonably well for your use case. As someone who is instinctively skeptical about these language models, this kind of statement makes my antennae twitch. You have this black box model that generates all sort of plausible outputs, you jiggle the handle until those outputs meet your expectations for some range of tested inputs, and then ... you assume it's just going to work? For parlor tricks, or even low-stakes real world activities that might be enough - but how can you trust it?
  - make3 6y ago
    (As one of them) every professional researcher in NLP at every large company (incl me) knows you can't rely on generation right now, and huge teams everywhere are working on reliability in text generation
    - joe_the_user 6y ago
      So, you take this "general purpose model" (with a huge corpus of standard text) and you attempt to use it for a narrow purpose. The model requires a lot of prompt tweaking and other things for this narrow model but eventually you "make it work". How do you know you're not just "programming" a chatbot (by indirectly filtering for the text-pieces you want) but in the most indirect and unguaranteed fashion possible? I suppose the advantage is you can say "look, it's intelligent".
DoomHotel 6y ago
That article just reinforces what I gathered from this one: GPT-3, Bloviator: OpenAI’s language generator has no idea what it’s talking about https://www.technologyreview.com/2020/08/22/1007539/gpt3-openai-language-generator-artificial-intelligence-ai-opinion/ https://www.technologyreview.com/2020/08/22/1007539/gpt3-ope...
- joe_the_user 6y ago
  It's remarkable that the article seems to be saying "it's not working now but maybe with a few tweaks we could swing this" where the real situation seems to be that this won't be doing any "meaning-critical" tasks for a long time, if ever. GPT-3 seems like an extended Eliza-effect device [1]. It seems that a general database of word-fragments sufficient to give the impression it's following-along on any topic but that doesn't involve any coherence but rather shows how much of ordinary language is "just associations" (which isn't entirely unimportant but still). Altogether, it even seems less sensible than narrow, hand-tuned chatbots like Alicebot[2] but it's harder to see it's limitation because of it's huge dataset. [1] https://en.wikipedia.org/wiki/ELIZA_effect https://en.wikipedia.org/wiki/ELIZA_effect [2] https://en.wikipedia.org/wiki/Artificial_Linguistic_Internet_Computer_Entity https://en.wikipedia.org/wiki/Artificial_Linguistic_Internet...

51 points by gavelin 6y ago

I'm getting concerned that even researchers who know better are anthropomorphising GPT-3 in their descriptions of it's output.

make3 6y ago
As a researcher, it's pretty obvious from the language and from the analysis of the author that he's a novice at machine learning who is a lawyer, not a researcher in machine learning
- gavelin 6y ago
  Correct. I hope that meant that this was more accessible than the average write-up (and not less accurate!) :)
- YeGoblynQueenne 6y ago
  The author seems to have a very clear understanding of how GPT-3 works and their down-to-Earth, plain language analysis is miles away from the wild flights of fancy we're used to reading about GPT-3. As a for instance, they didn't even use the word "understand" once, to refer to what GPT-3 is doing.
  - make3 6y ago
    I'm talking about knowledge of the NLP literature beyond GPT-3, and other machine learning basic technical stuff
    - YeGoblynQueenne 6y ago
      Is any of that referenced in the article in a way that needs a deep explanation which isn't there?
      - gavelin 6y ago
        Thank you for the support—means a lot to me. I wrote the piece after finding that recent journalism on GPT-3 did not provide a sufficiently accurate snapshot of how vanilla GPT-3 scores on legal tasks (not to mention the misleading snapshots of promising sandbox outputs). Meanwhile, even the most capable people in the ML community do not get to the papers discussing the minutiae of handling the hard problems in legal texts. I never intended the piece to comprehensively address and solve the technical problems GPT-3 (or NLP more broadly) has in handling legal texts. The inability to examine and audit GPT-3 at different levels of the network makes makes any investigation a partially speculative endeavor. Instead, I merely wanted to provide an overview of outputs critiqued by a lawyer, and offer up some ideas of how to improve performance on legal tasks drawing from ML and legal knowledge. I think there is some constructive dialogue in this thread, and I am thrilled by that. Lawyers and engineers need to work together on this to be successful. The goal of summarization is to enable more people to accurately understand text in less time. It is pretty clear that the feedback of a lawyer in the training loop (at least to label meaning and context) would lead to a significant improvement. IIRC, Andrej Karpathy labeled a lot of data when his team achieved a 50% improved classification and detection jump on ImageNet. Can more (and better) labeled data get us to an accuracy level that is good enough to generate term sheets out of contracts? I would like to find out. I am interested in diving deeper and connecting with anyone who is game to tackle some of the critical challenges.

gwern 6y ago

Also of interest is Arbel's paper on GPT-3 (AID currently but he got GPT-3 access recently) legal summaries: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3740356 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3740356 One thing I would note is the potential for a feedback loop between summaries and the model: if a summary of a specific ToS or piece of law is wrong, you can hardwire an expert-vetted one (there's only so many ToSes or pieces of law and it'll be a long tail), and you can feed back in the correct one as training data to finetune the model. The bigger the GPT model, the smarter it is, and the less feedback it takes to correct its summaries: https://openai.com/blog/learning-to-summarize-with-human-feedback/ https://openai.com/blog/learning-to-summarize-with-human-fee...

gavelin 6y ago
Thanks for the paper link! I think your reasoning of hardwiring boilerplate to an expert-vetted (or written) summary is a much more accurate approach. If only we could scrape the clause explanation footnotes from quality sources I would not be forced to write them!

Der_Einzige 6y ago

This is at the intersection of my research interests. I'd love to see what happens if you run it on a large debate case with slight variations of the input prompt queries. I love to see all of these articles about legal summarization - but it's always covering abstractive summarization! I want an effective highlighter model. To be fair, I did build a system for using transformers to do extractive summarization in an unsupervised manner, but the results aren't that great. I'm sure that Lawyers would find something that could highlight the most legally salient sections to be extremely useful if it's reasonably accurate. I really did figure that we'd get effective word-level extractive summaries before we'd get effective abstractive summaries but I guess that intuition was wrong...

staticautomatic 6y ago
If keyphrase extraction and extractive summarization of legal documents interests you, I have a side project I’ve been wrenching on that I could use a hand with. Contact info in my profile.

make3 6y ago

The author seems to forget that fine-tuning exists all-together, which is how real-world NLP applications are actually made. In reply to the following paragraph, in a real world setting, this would be done through model fine-tuning on high quality data. "First, there must be greater transparency. The sources of GPT-3’s references must at least be referenceable and perhaps tweaked to follow a proper hierarchy of authorities. Although it is challenging to audit a 175 billion parameter algorithm, it would be beneficial to understand the most influential semantic parameters used to generate the output text. This would ideally enable users to choose word choice (perhaps as a level of sophistication), appropriate voice, and tone"

gavelin 6y ago
You are quite correct that fine-tuning is necessary to improve accuracy. I did not intend to dismiss that. The point I intended to make there is that it would be nice to be able to see whether, when interpreting a statute, GPT-3 relied heavily on a blog interpreting a statute v. legislative history v. Supreme Court precedent. The right approach would be to control the proper hierarchy of authorities. It would be helpful to understand, even as a textual matter, what GPT-3 was most heavily relying on for a given prediction. That would speed up categorically fixing bad prediction patterns. It is also true that high quality data is necessary. There is a reason why lawyers rely on Westlaw and LexisNexis to search for relevant laws and even scholarly articles. They are trusted sources. A better approach would rely on something like those with a more narrow universe of quality sources. There is a ton of labeling work that needs to be done, even beyond the “KeyCite” type of labels Westlaw applies to documents. Note that YC company www.rossintelligence.com ran into some trouble recently with Westlaw and LexisNexis. The quality v. quantity of data debate is particularly relevant here. The power of GPT-3 is in part supposed to come from the sheer scale of its training dataset size. It would be nice to leverage some of the semantic training from a large non-legal dataset to be able to stylistically output in layman’s terms while sourcing the authorities from more closely vetted sources.

dweekly 6y ago

tl;dr - Don't use GPT-3 to summarize your legal documents yet.

gavelin 6y ago
"tl;dr" may very well be the problem! There is a tendency of many tl;dr summaries on the web to oversimplify and skew concepts. If those are included in GPT-3's dataset, GPT-3's output will try to match the dataset style (according to the parameters) and likely not meet the legal standard. There is a section following the conclusion in the paper where I touch on other ways GPT-3 might be improved for the legal summarization use case. The only way we get there is by delving into the nuance of WHY GPT-3 is not yet good enough to replace lawyers, and HOW we can improve on it as an architecture.
- dweekly 6y ago
  Absolutely, and I am grateful for your detailed analysis here. These kinds of spot checks on subjective quality for usability in a domain are a critical part of scoring "are we there yet?" and providing solid exemplars that are motivating to the researchers. It's not hard to imagine a GPT-5 recap in a few years with very different outcomes. Some of the motivation on my side is judging at which point it becomes appropriate to explore having an algorithm provide a summary of a medical conversation, since my day job at Medcorder is building a system that acts as a patient advocate to help them better understand what their doctors are saying. The universal feature request from day one has been summaries (which are fraught if you get them wrong!), so I'm keen to develop a sense of when we're going to get there. (As for the downvotes on my comment above, I'd bet they are, appropriately ironically, from people who themselves didn't actually read the article, since the comment was made with a wink to the fact that the article explicitly evaluates the appropriateness of "tl;dr" summaries and found them wanting.)

skybrian 6y ago

GPT-3 picks words literally at random (according to a probability distribution) so it would be good to run each experiment multiple times to get a sense of the probability distribution. I doubt it would change the conclusions, though.

make3 6y ago
In a professional setting, this is definitely done, and the generations are then ranked with separate models that predict different quality metrics, such as "interestingness", "safety" in an inclusiveness type of way, whether the answer seems to fit a style that you want, whether the facts in the answer seem to make sense, etc., and it makes a big difference actually
gavelin 6y ago
Good tip! I repeated a few inputs to see if the variation was significant enough to warrant including that, and as your intuition suggested, it was not with the parameters I selected. A more robust experiment should definitely include repeat attempts.
minimaxir 6y ago
There's a big difference between sampling from a learned probability distribution and "picks words literally at random". The temperature = 0 examples have zero sampling by construction, while higher temperatures having a slight sampling. That said, it doesn't hurt to have multiple attempts (aside from the cost of using GPT-3, ahem).
- skybrian 6y ago
  I guess I could have left out "literally," but picking words from a learned probability distribution is picking at random. Since words are chosen in order, picking one word differently will likely mean that every word that follows will be different, and this choice between suffixes is not made intelligently. This randomness is there for a reason. Yes, you can turn it off, but then you get loops, which is obviously not intelligent. A bit of randomness makes it less obvious.
6y ago

minimaxir 6y ago

> Second, text ought to be tokenized (a term used in natural language processing wherein text is assigned a numerical value) at many different levels (character, word, sentence, paragraph, section, etc.) in order to make predictions that are relevant to the excerpt of a text, while also remaining consistent to the broader document. This is challenging because doing so involves a tremendous amount of computational resources. However, it may be necessary to accurately capture meaning at different levels of abstraction. I don't think this would be an improvement per se. The way Byte Pair Encodings are constructed that GPT-3 (and GPT-2) use is that they already take higher-level text representations and compress it down (into tokens), which are then reflected in the training of the model.

Tarq0n 6y ago
And the attention mechanism should take care of "sentence, paragraph, section" level context. Attributing this to tokenization is a weird mistake to make.
- gavelin 6y ago
  Tarq0n: Also a great point. The question then is whether the attention mechanism is being triggered on the proper word or character sequences. Lawyers employ a form of attention mechanism when “issue spotting.” For example, in a non-compete I might scan for the duration (how many years until the client can join a similar venture?) and breadth (what is the definition of a similar venture?). For an attention mechanism to work well for legal summaries, it seems to me that it must trigger on many relevant context triggers at different levels. As an extreme example, if I put 50 contracts involving multiple different parties into one big document and the attention mechanism was triggered on the first document title, would the subsequent document titles sufficiently demarcate a new contract to reset the context? Or would the attention stay “on” the first high accuracy context match and mix up the terms and parties? In context of the paper, GPT-3 missed a ton of issues, indicating to me that the attention mechanism is not being properly triggered. Again I may be wrong that tokenizing and predicting based off of clauses, paragraphs and sections would help improve the output, but that was one way I thought would capture the different levels of context.
6y ago

Aulig 6y ago

Great read, thanks for sharing. It's always interesting to find out how AI is impacting occupations outside the technology sector.

gavelin 6y ago
Thank you for reading!

jll29 6y ago

To those that are interested in the state of the art in commercial legal summarization (US law) as of Q1/2021: https://arxiv.org/pdf/2102.05757.pdf https://arxiv.org/pdf/2102.05757.pdf

gavelin 6y ago
Great link. Thank you for sharing this.

MasterScrat 6y ago

Thank you for your analysis, it's great to have the insights of someone with both legal and ML backgrounds. I want to point out that a big part of building anything on GPT-3 (or other large LMs for that matter) is "prompt engineering", which means you try out thousands of prompts and sampling parameters until you find something that works reasonably well for your use case. Taking two default templates and a few different temperatures is like taking some tutorials for a new framework, building a proof of concept from them, then making a judgement from that. Sure, it can provide a good first assessment, but that's it. You would need much deeper experience to come to a meaningful conclusion.

NovemberWhiskey 6y ago
>I want to point out that a big part of building anything on GPT-3 (or other large LMs for that matter) is "prompt engineering", which means you try out thousands of prompts and sampling parameters until you find something that works reasonably well for your use case. As someone who is instinctively skeptical about these language models, this kind of statement makes my antennae twitch. You have this black box model that generates all sort of plausible outputs, you jiggle the handle until those outputs meet your expectations for some range of tested inputs, and then ... you assume it's just going to work? For parlor tricks, or even low-stakes real world activities that might be enough - but how can you trust it?
- make3 6y ago
  (As one of them) every professional researcher in NLP at every large company (incl me) knows you can't rely on generation right now, and huge teams everywhere are working on reliability in text generation
  - joe_the_user 6y ago
    So, you take this "general purpose model" (with a huge corpus of standard text) and you attempt to use it for a narrow purpose. The model requires a lot of prompt tweaking and other things for this narrow model but eventually you "make it work". How do you know you're not just "programming" a chatbot (by indirectly filtering for the text-pieces you want) but in the most indirect and unguaranteed fashion possible? I suppose the advantage is you can say "look, it's intelligent".

DoomHotel 6y ago

That article just reinforces what I gathered from this one: GPT-3, Bloviator: OpenAI’s language generator has no idea what it’s talking about https://www.technologyreview.com/2020/08/22/1007539/gpt3-openai-language-generator-artificial-intelligence-ai-opinion/ https://www.technologyreview.com/2020/08/22/1007539/gpt3-ope...

joe_the_user 6y ago
It's remarkable that the article seems to be saying "it's not working now but maybe with a few tweaks we could swing this" where the real situation seems to be that this won't be doing any "meaning-critical" tasks for a long time, if ever. GPT-3 seems like an extended Eliza-effect device [1]. It seems that a general database of word-fragments sufficient to give the impression it's following-along on any topic but that doesn't involve any coherence but rather shows how much of ordinary language is "just associations" (which isn't entirely unimportant but still). Altogether, it even seems less sensible than narrow, hand-tuned chatbots like Alicebot[2] but it's harder to see it's limitation because of it's huge dataset. [1] https://en.wikipedia.org/wiki/ELIZA_effect https://en.wikipedia.org/wiki/ELIZA_effect [2] https://en.wikipedia.org/wiki/Artificial_Linguistic_Internet_Computer_Entity https://en.wikipedia.org/wiki/Artificial_Linguistic_Internet...