You are quite correct that fine-tuning is necessary to improve accuracy. I did not intend to dismiss that. The point I intended to make there is that it would b

You are quite correct that fine-tuning is necessary to improve accuracy. I did not intend to dismiss that. The point I intended to make there is that it would be nice to be able to see whether, when interpreting a statute, GPT-3 relied heavily on a blog interpreting a statute v. legislative history v. Supreme Court precedent. The right approach would be to control the proper hierarchy of authorities. It would be helpful to understand, even as a textual matter, what GPT-3 was most heavily relying on for a given prediction. That would speed up categorically fixing bad prediction patterns. It is also true that high quality data is necessary. There is a reason why lawyers rely on Westlaw and LexisNexis to search for relevant laws and even scholarly articles. They are trusted sources. A better approach would rely on something like those with a more narrow universe of quality sources. There is a ton of labeling work that needs to be done, even beyond the “KeyCite” type of labels Westlaw applies to documents. Note that YC company www.rossintelligence.com ran into some trouble recently with Westlaw and LexisNexis. The quality v. quantity of data debate is particularly relevant here. The power of GPT-3 is in part supposed to come from the sheer scale of its training dataset size. It would be nice to leverage some of the semantic training from a large non-legal dataset to be able to stylistically output in layman’s terms while sourcing the authorities from more closely vetted sources.