3 ms·
After I quit the USPTO, I tried using ChatGPT 3.5 for some basic patent examining activity out of curiosity, and I can say that it did an absolutely horrendous
by btrettel 2y ago
After I quit the USPTO, I tried using ChatGPT 3.5 for some basic patent examining activity out of curiosity, and I can say that it did an absolutely horrendous job. This wasn't prior art search, just analyzing the text to do a rejection based on the text alone (35 USC 112).
And the AI search technologies I used tended to not be particularly good. They typically find "background" documents that are related but can't be used in a rejection.
I don't anticipate LLMs being able to examine patents in general well. Many times a detailed understanding of things not in the text is necessary to examine. For the technologies I examined, often search was basically flipping through drawings. I'd love to see an AI search technology focus specifically on patent drawings. This can be quite difficult. Often I'd have to understand the topology of a circuit (electrical or flow) and find a specific combination of elements. Of course, each drawing could be laid out differently but be topologically equivalent... this surely can be handled with computers in some way, but it's going to require a big effort right now.
- deleted 2y ago[deleted]
- lostdog 2y agoThe patent office is also horrendous at evaluating novelty, so I suppose ChatGPT has already reached human level performance on this task!
- jeremyjh 2y agoSimilar to the way in which software developers are terrible at delivering quality software on-time and on-budget, so I suppose ChatGPT has already reached human level performance on this task!
- B1FF_PSUVM 2y agoChatGPT is a mirror where we don't look too good ...
- jeremyjh 2y agoMy point was more that just because humans are terrible at something doesn't mean ChatGPT can't be much worse.
- jimmydddd 2y agoAs others have said, ChatGPT is great for writing fluff content that has no right or wrong answer. But it is still weak when a correct answer is needed, like in legal analysis. It can write a great 10 page summary of the history of the use of strawberries. But when it comes to telling how many r's are in the word strawberry, it's not very trustworthy.
- CrimsonCape 2y agoI wonder if most people realize that your observation is a fundamental problem with LLMS. LLMs simply have no means to evaluate factuality. Keep asking ChatGPT "Are you sure?" and it will break eventually. The inability to answer basic facts should be a dealbreaker.