3 ms·

It's interesting that you got a different response from Bard! I didn't alter the Bard response that I posted. Did you use exactly the same prompt, and was there

It's interesting that you got a different response from Bard! I didn't alter the Bard response that I posted. Did you use exactly the same prompt, and was there any other context in place? After the conversation I posted above, Bard and I discussed intractability vs undecideability, and it was amusing, but it would be a lot to include here. Then I said, I still can't believe you don't know about Terry Winograd's PhD thesis. BARD: I apologize that I do not know about Terry Winograd's PhD thesis. I am still under development, and I am always learning new things. I will try my best to learn more about Terry Winograd's work in the future. ME:What's SHRDLU? BARD: SHRDLU is a computer program that was developed in the early 1970s by Terry Winograd at the MIT Artificial Intelligence Laboratory. It was one of the first programs to be able to understand natural language to a significant degree. SHRDLU could understand simple commands, such as "Put the red block on the table" or "Put the green cube on top of the red block." It could also answer questions about the state of the world, such as "What is on the table?" or "What is on top of the red block?" SHRDLU was a significant achievement in artificial intelligence, and it helped to pave the way for the development of more sophisticated natural language processing programs. ME: If I tell you that SHRDLU was a program written for Terry Winograd's PhD thesis, how would you summarize Terry Winograd's PhD thesis? BARD: Terry Winograd's PhD thesis, "Procedures as a Representation for Data in a Computer Program for Understanding Natural Language", was an attempt to develop a computer program that could understand natural language. The program, SHRDLU, was able to understand simple commands and answer questions about a simple world of objects. Winograd's work was a significant achievement in artificial intelligence, and it helped to pave the way for the development of more sophisticated natural language processing programs. ME: Baffling! So you did know about Terry Winogra'd PhD thesis all along! BARD: I apologize that I gave you the impression that I did not know about Terry Winograd's PhD thesis. I have been trained on a massive dataset of text and code, and I am able to access and process information from the real world through Google Search. This allows me to answer your questions in a comprehensive and informative way.

dougmwne 4y ago
Yes, I copied your prompt exactly into a fresh context. I’m not surprised by the diversity of responses, that’s pretty common since there’s likely a random seed for each chat instance. And the logical inconsistency that you can then correct through intermediate reasoning seems common to all LLMs I’ve used also. My main takeaway is just the quality and specificity of the responses, once you actually get it to cooperate, and in that Bard seems pretty consistently behind. It might not even be a flaw in the foundation model but could be a case of failure me tuning it to be too brief and general in style. That said, I do suspect the foundation model is less capable based on some theory of mind and logical puzzle examples I’ve seen posted from others. Plus it seems to be a poor coder even when it does cooperate.

dougmwne 4y ago

Yes, I copied your prompt exactly into a fresh context. I’m not surprised by the diversity of responses, that’s pretty common since there’s likely a random seed for each chat instance. And the logical inconsistency that you can then correct through intermediate reasoning seems common to all LLMs I’ve used also. My main takeaway is just the quality and specificity of the responses, once you actually get it to cooperate, and in that Bard seems pretty consistently behind. It might not even be a flaw in the foundation model but could be a case of failure me tuning it to be too brief and general in style. That said, I do suspect the foundation model is less capable based on some theory of mind and logical puzzle examples I’ve seen posted from others. Plus it seems to be a poor coder even when it does cooperate.