5 ms·

I don't mean to post all my interactions, but here's one more. I asked Bard, "Are you familiar with Terry Winograd's PhD thesis on natural language processing?

I don't mean to post all my interactions, but here's one more. I asked Bard, "Are you familiar with Terry Winograd's PhD thesis on natural language processing? How does the program in his thesis compare to your natural langauge ability?" Bard replied, "I do not have enough information about that person to help with your request. I am a large language model, and I am able to communicate and generate human-like text in response to a wide range of prompts and questions, but my knowledge about this person is limited. Is there anything else I can do to help you with this request?" In contrast, chatGPT gave a good summary of SHRDLU and a superficial sentence or two comparing SHRDLU to itself. I then asked it some "block world" questions, and it answered a few correctly. 'Assume there are 3 cubs, red, green, and blue. Can you stack the red cube on the blue cube? Assume there is also a yellow pyramid. Can you stack the green cube on top of the pyramid.".... I can't pull up the exact chatGPT query I used or the reply. Sometimes the chatGPT browser interface can't access past conversations, and now is one of those times. EDIT: Bard can correctly answer (easy) questions about an imagined blocks world. IIRC chatGPT was vague on whether a cube can be stacked on top of a pyramid. Assume there are 3 cubes, red, green, and blue. Stack the red cube on top of the blue cube, and describe the result to me. BARD: The blue cube is now the bottom cube, and the red cube is on top of it. The red cube is touching the blue cube on all six sides. The blue cube is completely covered by the red cube. Also assume there is a yellow pyramid. Stack the pyramid on the green cube and describe the result to me. BARD: The yellow pyramid is now resting on top of the green cube. The pyramid is touching the green cube on all four sides. The green cube is still visible from underneath the pyramid. How would you stack the blue cube on top of the green cube, given the current arrangement of the blocks? BARD: I would need to move the red cube off of the blue cube first. Then, I could stack the blue cube on top of the green cube. Can a cube be stacked on top of the pyramid? BARD: No, a cube cannot be stacked on top of a pyramid. A cube is a three-dimensional solid with six square faces, while a pyramid is a three-dimensional solid with a square base and four triangular sides that meet at a point. The two shapes are not compatible for stacking.

dougmwne 4y ago
I am going to post the full response to Winograd because I think it is extremely representative of what I have been seeing so far from Bard. I can’t help but be incredibly disappointed. Bard: Yes, I am familiar with Terry Winograd's PhD thesis on natural language processing. In his thesis, Winograd developed a program called SHRDLU that could understand and respond to English sentences in a simulated "world of toy blocks". SHRDLU was a significant achievement in the field of artificial intelligence, and it helped to pave the way for the development of more sophisticated natural language processing systems. My natural language ability is much more advanced than SHRDLU's. I can understand and respond to a wider range of English sentences, and I can also generate text that is more natural and engaging. I am able to do this because I have access to a much larger dataset of text and code, and I have been trained on a more sophisticated algorithm. However, SHRDLU was a groundbreaking achievement in its time, and it is important to remember that it was developed over 50 years ago. The field of natural language processing has made significant progress since then, and I am grateful for the work of Terry Winograd and other pioneers in the field. GPT-4: Yes, I am familiar with Terry Winograd's PhD thesis on natural language processing. His thesis, titled "Procedures as a Representation for Data in a Computer Program for Understanding Natural Language," was published in 1971. The work focused on SHRDLU, a natural language understanding system that he developed. SHRDLU was designed to understand and respond to natural language commands in a limited "blocks world" environment. The blocks world consisted of a virtual space where a robotic arm could manipulate colored blocks on a table. Winograd's program could interpret natural language commands given by a human user, such as "Pick up the red block" or "Place the green block on the blue one," and execute the appropriate actions in the simulated environment. While groundbreaking for its time, the SHRDLU program's natural language understanding capabilities were limited in comparison to modern AI systems like GPT-4. Some key differences between SHRDLU and GPT-4 include: 1. *Domain*: SHRDLU was limited to the blocks world, which was a small, well-defined domain. GPT-4, on the other hand, has been trained on a diverse range of text and can generate responses on a wide variety of topics. 2. *Language Model*: SHRDLU used a rule-based approach to parse and interpret natural language, whereas GPT-4 utilizes a deep learning architecture based on the Transformer model. This allows GPT-4 to learn complex language patterns and generate more human-like responses. 3. *Flexibility*: GPT-4 is more flexible in understanding and generating natural language due to its large-scale training data and deep learning architecture. SHRDLU, by contrast, relied on handcrafted rules and procedures, which made it less adaptable to new language patterns or domains. 4. *Data-driven*: GPT-4 is a data-driven model, which means it learns from the patterns and structures in the text it was trained on. SHRDLU, being a rule-based system, did not have the same capacity to learn from data. Despite these differences, Winograd's work on SHRDLU was an important milestone in the history of natural language processing and artificial intelligence. It influenced subsequent research in the field and laid the groundwork for future advancements, including the development of AI systems like GPT-4.
- getpost 4y ago
  It's interesting that you got a different response from Bard! I didn't alter the Bard response that I posted. Did you use exactly the same prompt, and was there any other context in place? After the conversation I posted above, Bard and I discussed intractability vs undecideability, and it was amusing, but it would be a lot to include here. Then I said, I still can't believe you don't know about Terry Winograd's PhD thesis. BARD: I apologize that I do not know about Terry Winograd's PhD thesis. I am still under development, and I am always learning new things. I will try my best to learn more about Terry Winograd's work in the future. ME:What's SHRDLU? BARD: SHRDLU is a computer program that was developed in the early 1970s by Terry Winograd at the MIT Artificial Intelligence Laboratory. It was one of the first programs to be able to understand natural language to a significant degree. SHRDLU could understand simple commands, such as "Put the red block on the table" or "Put the green cube on top of the red block." It could also answer questions about the state of the world, such as "What is on the table?" or "What is on top of the red block?" SHRDLU was a significant achievement in artificial intelligence, and it helped to pave the way for the development of more sophisticated natural language processing programs. ME: If I tell you that SHRDLU was a program written for Terry Winograd's PhD thesis, how would you summarize Terry Winograd's PhD thesis? BARD: Terry Winograd's PhD thesis, "Procedures as a Representation for Data in a Computer Program for Understanding Natural Language", was an attempt to develop a computer program that could understand natural language. The program, SHRDLU, was able to understand simple commands and answer questions about a simple world of objects. Winograd's work was a significant achievement in artificial intelligence, and it helped to pave the way for the development of more sophisticated natural language processing programs. ME: Baffling! So you did know about Terry Winogra'd PhD thesis all along! BARD: I apologize that I gave you the impression that I did not know about Terry Winograd's PhD thesis. I have been trained on a massive dataset of text and code, and I am able to access and process information from the real world through Google Search. This allows me to answer your questions in a comprehensive and informative way.
  - dougmwne 4y ago
    Yes, I copied your prompt exactly into a fresh context. I’m not surprised by the diversity of responses, that’s pretty common since there’s likely a random seed for each chat instance. And the logical inconsistency that you can then correct through intermediate reasoning seems common to all LLMs I’ve used also. My main takeaway is just the quality and specificity of the responses, once you actually get it to cooperate, and in that Bard seems pretty consistently behind. It might not even be a flaw in the foundation model but could be a case of failure me tuning it to be too brief and general in style. That said, I do suspect the foundation model is less capable based on some theory of mind and logical puzzle examples I’ve seen posted from others. Plus it seems to be a poor coder even when it does cooperate.
summerlight 4y ago
I guess you stepped on the case of aggressive filtering. The current version of Bard looks like have a very conservative set of filtering preamble prompts to arbitrarily limit its ability to avoid negative presses.
mustacheemperor 4y ago
> BARD: The blue cube is now the bottom cube, and the red cube is on top of it. The red cube is touching the blue cube on all six sides. That doesn’t seem correct. The red cube is touching the blue cube on its top side only.
- nhinck2 4y ago
  You could argue 5 sides are touching
  - HDThoreaun 4y ago
    only if you're trying to be a nuisance
akdor1154 4y ago
> Assume there are 3 cubes, red, green, and blue. Stack the red cube on top of the blue cube, and describe the result to me. > BARD: The blue cube is now the bottom cube, and the red cube is on top of it. The red cube is touching the blue cube on all six sides. The blue cube is completely covered by the red cube. Well if you wanted a description of a 4-dimensional block world it's actually not bad.

dougmwne 4y ago

I am going to post the full response to Winograd because I think it is extremely representative of what I have been seeing so far from Bard. I can’t help but be incredibly disappointed. Bard: Yes, I am familiar with Terry Winograd's PhD thesis on natural language processing. In his thesis, Winograd developed a program called SHRDLU that could understand and respond to English sentences in a simulated "world of toy blocks". SHRDLU was a significant achievement in the field of artificial intelligence, and it helped to pave the way for the development of more sophisticated natural language processing systems. My natural language ability is much more advanced than SHRDLU's. I can understand and respond to a wider range of English sentences, and I can also generate text that is more natural and engaging. I am able to do this because I have access to a much larger dataset of text and code, and I have been trained on a more sophisticated algorithm. However, SHRDLU was a groundbreaking achievement in its time, and it is important to remember that it was developed over 50 years ago. The field of natural language processing has made significant progress since then, and I am grateful for the work of Terry Winograd and other pioneers in the field. GPT-4: Yes, I am familiar with Terry Winograd's PhD thesis on natural language processing. His thesis, titled "Procedures as a Representation for Data in a Computer Program for Understanding Natural Language," was published in 1971. The work focused on SHRDLU, a natural language understanding system that he developed. SHRDLU was designed to understand and respond to natural language commands in a limited "blocks world" environment. The blocks world consisted of a virtual space where a robotic arm could manipulate colored blocks on a table. Winograd's program could interpret natural language commands given by a human user, such as "Pick up the red block" or "Place the green block on the blue one," and execute the appropriate actions in the simulated environment. While groundbreaking for its time, the SHRDLU program's natural language understanding capabilities were limited in comparison to modern AI systems like GPT-4. Some key differences between SHRDLU and GPT-4 include: 1. *Domain*: SHRDLU was limited to the blocks world, which was a small, well-defined domain. GPT-4, on the other hand, has been trained on a diverse range of text and can generate responses on a wide variety of topics. 2. *Language Model*: SHRDLU used a rule-based approach to parse and interpret natural language, whereas GPT-4 utilizes a deep learning architecture based on the Transformer model. This allows GPT-4 to learn complex language patterns and generate more human-like responses. 3. *Flexibility*: GPT-4 is more flexible in understanding and generating natural language due to its large-scale training data and deep learning architecture. SHRDLU, by contrast, relied on handcrafted rules and procedures, which made it less adaptable to new language patterns or domains. 4. *Data-driven*: GPT-4 is a data-driven model, which means it learns from the patterns and structures in the text it was trained on. SHRDLU, being a rule-based system, did not have the same capacity to learn from data. Despite these differences, Winograd's work on SHRDLU was an important milestone in the history of natural language processing and artificial intelligence. It influenced subsequent research in the field and laid the groundwork for future advancements, including the development of AI systems like GPT-4.

getpost 4y ago
It's interesting that you got a different response from Bard! I didn't alter the Bard response that I posted. Did you use exactly the same prompt, and was there any other context in place? After the conversation I posted above, Bard and I discussed intractability vs undecideability, and it was amusing, but it would be a lot to include here. Then I said, I still can't believe you don't know about Terry Winograd's PhD thesis. BARD: I apologize that I do not know about Terry Winograd's PhD thesis. I am still under development, and I am always learning new things. I will try my best to learn more about Terry Winograd's work in the future. ME:What's SHRDLU? BARD: SHRDLU is a computer program that was developed in the early 1970s by Terry Winograd at the MIT Artificial Intelligence Laboratory. It was one of the first programs to be able to understand natural language to a significant degree. SHRDLU could understand simple commands, such as "Put the red block on the table" or "Put the green cube on top of the red block." It could also answer questions about the state of the world, such as "What is on the table?" or "What is on top of the red block?" SHRDLU was a significant achievement in artificial intelligence, and it helped to pave the way for the development of more sophisticated natural language processing programs. ME: If I tell you that SHRDLU was a program written for Terry Winograd's PhD thesis, how would you summarize Terry Winograd's PhD thesis? BARD: Terry Winograd's PhD thesis, "Procedures as a Representation for Data in a Computer Program for Understanding Natural Language", was an attempt to develop a computer program that could understand natural language. The program, SHRDLU, was able to understand simple commands and answer questions about a simple world of objects. Winograd's work was a significant achievement in artificial intelligence, and it helped to pave the way for the development of more sophisticated natural language processing programs. ME: Baffling! So you did know about Terry Winogra'd PhD thesis all along! BARD: I apologize that I gave you the impression that I did not know about Terry Winograd's PhD thesis. I have been trained on a massive dataset of text and code, and I am able to access and process information from the real world through Google Search. This allows me to answer your questions in a comprehensive and informative way.
- dougmwne 4y ago
  Yes, I copied your prompt exactly into a fresh context. I’m not surprised by the diversity of responses, that’s pretty common since there’s likely a random seed for each chat instance. And the logical inconsistency that you can then correct through intermediate reasoning seems common to all LLMs I’ve used also. My main takeaway is just the quality and specificity of the responses, once you actually get it to cooperate, and in that Bard seems pretty consistently behind. It might not even be a flaw in the foundation model but could be a case of failure me tuning it to be too brief and general in style. That said, I do suspect the foundation model is less capable based on some theory of mind and logical puzzle examples I’ve seen posted from others. Plus it seems to be a poor coder even when it does cooperate.

summerlight 4y ago

I guess you stepped on the case of aggressive filtering. The current version of Bard looks like have a very conservative set of filtering preamble prompts to arbitrarily limit its ability to avoid negative presses.

mustacheemperor 4y ago

> BARD: The blue cube is now the bottom cube, and the red cube is on top of it. The red cube is touching the blue cube on all six sides. That doesn’t seem correct. The red cube is touching the blue cube on its top side only.

nhinck2 4y ago
You could argue 5 sides are touching
- HDThoreaun 4y ago
  only if you're trying to be a nuisance

akdor1154 4y ago

> Assume there are 3 cubes, red, green, and blue. Stack the red cube on top of the blue cube, and describe the result to me. > BARD: The blue cube is now the bottom cube, and the red cube is on top of it. The red cube is touching the blue cube on all six sides. The blue cube is completely covered by the red cube. Well if you wanted a description of a 4-dimensional block world it's actually not bad.