3 ms·

I would claim it is easy to think you're seeing GPT-3 fail because it's taking the wrong association path (not noticing the hierarchical decomposition of the co

I would claim it is easy to think you're seeing GPT-3 fail because it's taking the wrong association path (not noticing the hierarchical decomposition of the context). But the general problem is that there is no fixed decomposition of the meaning of a text. It's tempting to think a "simple" procedure like summary doesn't need "deep" meaning but it doesn't seem like that's the case. It should especially be noted a lot of the plausible "summaries" could be summaries of any privacy policy or just what people say about privacy policies on the net. It would have been interesting to give the system a novel bit of text to interpret instead.

gavelin 6y ago
A bright-line rule like "It is unlawful to exceed 65MPH in any vehicle on Highway 101" has a pretty narrow meaning and could be concretely decomposed sufficient for the average human to understand. I think you are right to point out that standards such as "It is unlawful to exceed 65MPH in any vehicle on Highway 101 unless it is reasonable under the circumstances" break down into many more concepts (what is reasonable?) and therefore seem like there is not a fixed point to decompose the text. In that case, lawyers look to precedent, among other sources, to get guidance on how the rule plays out in a sufficient number of contexts to determine the threshold of reasonability to properly analogize to new circumstances. I did not intend to argue against deep meaning as an approach categorically. Sorry if I gave off that impression. I think the right approach would include a combination of deep meaning and frequently updated fixed references. Regarding how that plays out on novel v. boilerplate language would be an interesting follow-up. Boilerplate language seems to favor authorities more than novel language, but there are a lot of hypotheticals to consider. Consider boilerplate language in a Privacy Policy that reads that “The Company stores your data and aggregates your data with data from multiple users to make inferences of users habits and may from time to time sell that data to 3rd parties.” In 2007 that might have meant that Google figured out that you and Jane both like M&Ms and sells chocolate companies your craving data. But what if Garmin recorded your GPS data and that of your friend who sleeps on your couch on Fridays, sold your data to a 3rd party who then mailed you wedding planning cards with your faces in it? The reference table, unless prospectively updated with new hypotheticals, would be challenged to explain that hypothetical. Would deep meaning fair better? I can think of a few ways of how it could.

gavelin 6y ago

A bright-line rule like "It is unlawful to exceed 65MPH in any vehicle on Highway 101" has a pretty narrow meaning and could be concretely decomposed sufficient for the average human to understand. I think you are right to point out that standards such as "It is unlawful to exceed 65MPH in any vehicle on Highway 101 unless it is reasonable under the circumstances" break down into many more concepts (what is reasonable?) and therefore seem like there is not a fixed point to decompose the text. In that case, lawyers look to precedent, among other sources, to get guidance on how the rule plays out in a sufficient number of contexts to determine the threshold of reasonability to properly analogize to new circumstances. I did not intend to argue against deep meaning as an approach categorically. Sorry if I gave off that impression. I think the right approach would include a combination of deep meaning and frequently updated fixed references. Regarding how that plays out on novel v. boilerplate language would be an interesting follow-up. Boilerplate language seems to favor authorities more than novel language, but there are a lot of hypotheticals to consider. Consider boilerplate language in a Privacy Policy that reads that “The Company stores your data and aggregates your data with data from multiple users to make inferences of users habits and may from time to time sell that data to 3rd parties.” In 2007 that might have meant that Google figured out that you and Jane both like M&Ms and sells chocolate companies your craving data. But what if Garmin recorded your GPS data and that of your friend who sleeps on your couch on Fridays, sold your data to a 3rd party who then mailed you wedding planning cards with your faces in it? The reference table, unless prospectively updated with new hypotheticals, would be challenged to explain that hypothetical. Would deep meaning fair better? I can think of a few ways of how it could.