5 ms·

Can Large Language Models Understand Symbolic Graphics Programs?

70 points by belter 2y ago

_f9cu 2y ago
[flagged]
- coding123 2y ago
  [flagged]
Lichtso 2y ago
What an awful paper title, saying "Symbolic Graphics Programs" when they just mean "vector graphics". I don't understand why they can not just use the established term instead. Also, there is no "program" here, in the same way that coding HTML is not programming, as vector graphics are not supposed to be Turing complete. And where they pulled the "symbolic" from is completely beyond me.
- tines 2y ago
  > Also, there is no "program" here, in the same way that coding HTML is not programming, as vector graphics are not supposed to be Turing complete. And where they pulled the "symbolic" from is completely beyond me. Aren't HTML and vector graphics descriptions both data structures that could be interpreted via a Turing-complete interpreter? I don't see the difference between HTML and a C AST in this regard.
  - jlarocco 2y ago
    There's a slippery slope there. Is a Word document a program? Is a PNG file? A computer program is a data structure, but data structures are not necessarily computer programs.
    - tines 2y ago
      True, I suppose HTML the better example, it's a tree description language, whereas PNG files, Word documents, etc. aren't.
    - justsomehnguy 2y ago
      > . Is a Word document a program? Even if not there is always an OLE embedding.
- merlincorey 2y ago
  I'm more curious how they think LLM's can imagine things: > To understand symbolic programs, LLMs may need to possess the ability to imagine how the corresponding graphics content would look without directly accessing the rendered visual content To my understanding, LLMs are predictive engines based upon their tokens and embeddings without any ability to "imagine" things. As such, an LLM might be able to tell you that the following SVG is a black circle because it is in Mozilla documentation[0]: <svg viewBox="0 0 100 100" xmlns="http://www.w3.org/2000/svg"> <circle cx="50" cy="50" r="50" /> </svg> However, I highly doubt any LLM could tell you the following is a "Hidden Mickey" or "Mickey Mouse Head Silhouette": <svg viewBox="0 0 175 175" xmlns="http://www.w3.org/2000/svg"> <circle cx="100" cy="100" r="50" /> <circle cx="50" cy="50" r="40" /> <circle cx="150" cy="50" r="40" /> </svg> - [0] https://developer.mozilla.org/en-US/docs/Web/SVG/Element/circle https://developer.mozilla.org/en-US/docs/Web/SVG/Element/cir...
  - randomdata 2y ago
    > without any ability to "imagine" things. What's imagining, then? The way LLMs explore different predictive branches in order to find an optimal solution doesn't seem all that different than what I consider imagining: Thinking about what could be and considering different variations on that idea. An LLM isn't a brain, so there is no implication of it being said in the truest human sense, but it seems like a decent analogy to me.
    - merlincorey 2y ago
      As I mentioned elsewhere, all the modern models are multimodal models so I think it would be fair to say rendering the SVG and then having the dedicated image model classify it is similar to "imagining", but it is definitely not strictly part of a Large Language Model which only predicts the next token based on previous tokens.
  - rel_ic 2y ago
    Check out Act One of this This American Life episode https://www.thisamericanlife.org/803/transcript https://www.thisamericanlife.org/803/transcript TLDR: it seems like an LLM might be able to tell you your SVG is a "Mickey Mouse Head Silhouette"
- jchw 2y ago
  > Also, there is no "program" here, in the same way that coding HTML is not programming, as vector graphics are not supposed to be Turing complete. I think the reason why we don't view HTML as a programming language is because it is explicitly designed to be a markup language that declares content rather than a series of instructions that is interpreted as a program. A program needn't demonstrate turing completeness to be a "computer program", it just needs to be a sequence of instructions that a computer executes. To me, that suggests that there's a degree of abstractness and subjectivity involved. For example, any SVG document could also be rewritten 1:1 with no loss in fidelity as a series of commands that has the same effect, as can pretty much any declarative markup language; what is actually happening during parsing is hard to distinguish from an interpreter. Humans can "know it when they see it", but I doubt there's an exact criteria that can go along with the human "feel" of what makes a program, a program.
- deleted 2y ago
  [deleted]
- deleted 2y ago
  [deleted]
brap 2y ago
Crazy that LLMs are able to do that when they’re not even trained on visual data.
- dmd 2y ago
  But... they are? What are you saying here?
  - katzinsky 2y ago
    I think he probably means that the LLMs never see the rasterized image output.
skybrian 2y ago
It’s difficult to tell what they’re proposing from the title or abstract, but asking questions about an SVG image, given only the source code, seems like an interesting challenge. There are a lot of ways to generate the same image in different ways, or to slightly change the source code, resulting in a very different image. This could be used to train a deeper understanding of program-like text.
slashdave 2y ago
In the title and abstract there is a proposal that LLMs "understand", "reason", and "imagine." It's off putting to me, since LLMs are not capable of any of these.
- mcbuilder 2y ago
  Not in the traditional sense, but how are we to possibly gauge an LLMs "intelligence" without framing questions like this?
- Kuinox 2y ago
  Can you come up with a proof with your bold statements, since currently leading researcher on the subject have no idea if that's the case ?
  - slashdave 2y ago
    Hard to prove a negative. On the other hand, relying on a poorly defined idea of "emergent behavior" when there is no architectural rational is rather poor form IMO. It's more wordy but safer to qualify these terms appropriately. That is, to test the "appearance" or "simulation" of human-like abilities.
    - warkdarrior 2y ago
      That's all I'm hoping for, an ML model that appears to be as good as any human in existence, for all human skills. To be fair, this is what we do with humans as well, we test their skills. We (rarely!) take humans apart, inspect their brains, and declare that they have reasoning or understanding capabilities.
      - slashdave 2y ago
        Sounds like you are looking for AGI, along with many other folks. Although, be careful of the usage of "as good as", which is hard to quantify (think of artistic endeavors).
    - 2y ago
abernard1 2y ago
Since people are not reading the paper, here were the conclusions: > Is it really easy to answer semantic reasoning questions over symbolic graphics programs? We provide an intriguing experiment to demonstrate that SVG programs can be quite difficult for LLMs to understand such that even if the corresponding rendered images are fairly easy for humans to recognize, all these powerful LLMs still fail dramatically, only reaching a chance-level accuracy. They then fine tune the model for this problem, and it achieves better results.
magicalhippo 2y ago
I was recently playing with some LLMs regarding a relay driver circuit. ChatGPT generated a circuit diagram for me in ASCII art style, but it contained some errors. For fun I tried to ask some others if they could identify the errors. Here's the diagram (formatting errors were from ChatGPT): +5V | | ( ) Relay Coil | +----->|----+ | Diode | | | | | +------------+ | | Drain (D) of MOSFET | Source (S) | ----- GND | Gate (G)---[220Ω]--- GPIO (3.3V MCU) | [10kΩ] | GND Here's what Llama 3.1 70B said: After reviewing the diagram, I've found a few potential issues: - The diode is not connected to the relay coil in a way that would prevent back-EMF from damaging the MOSFET. Typically, the diode is connected in parallel with the relay coil, with the anode connected to the positive terminal of the coil and the cathode connected to the negative terminal. In this diagram, the diode appears to be connected in series with the relay coil, which is not correct. - The MOSFET is not properly biased. The gate of the MOSFET is connected to the GPIO pin through a 220Ω resistor, but there is no pull-down resistor to ensure the gate is at a known state when the GPIO pin is not driving it. The 10kΩ resistor connected to GND is likely intended to be a pull-down resistor, but it's not clear if it's connected to the gate or not. - The relay coil is not specified, but it's likely that it requires a higher current than the MOSFET can handle. Typically, a relay driver circuit would include a current-limiting resistor or a transistor to handle the high current required by the relay coil. So it correctly identified one of the main issues, the flyback diode is wrong, but failed to correctly identify the issue with the MOSFET gate being directly tied to ground. I admit, it's not a typical way to draw things so fair enough, after all it did mention there was something fishy with the pull-down. Claude 3 Haiku provided a very similar answer. It was a bit more explicit and included more details. For instance it thought there might be an issue driving the MOSFET from 3.3V, though failed to suggest a logic-level MOSFET, instead suggesting a level-shifter. It also suggested the 220 Ohm gate resistor might not be enough to protect the microcontroller. Though it also tried to correct the diagram but failed: it changed the diode to ->|<-, and didn't put it in parallel with the coil. All in all, somewhat impressed. However they get easily confused. I tried to improve the diagram, and asked again. Here's the updated diagram: +5V | | | +----->|--------+ | Diode | | | ( ) Relay Coil | | | +---------------+ | | Drain (D) of MOSFET | |]- Gate (G)-----+-[220Ω]--- GPIO (3.3V MCU) | | | [10kΩ] Source (S) | | | +----------------+ | ----- GND And Llama 3.1 now says: After examining the diagram, I notice a few potential issues: - The diode is not connected to the relay coil in a way that would prevent back-EMF damage to the MOSFET. Typically, a flyback diode is connected in parallel with the relay coil, with the anode connected to the coil's positive terminal and the cathode connected to the coil's negative terminal. In this diagram, the diode appears to be connected in series with the relay coil, which is not correct. - The MOSFET's source (S) is not connected to GND. In a typical relay driver circuit, the source of the MOSFET is connected to GND to provide a return path for the current. - The 10kΩ resistor is not connected to anything. It's possible that it's intended to be a pull-down resistor for the MOSFET's gate, but it's not clear where it's supposed to be connected. So it totally fails to "see" that the 10k is now correctly connected to ground, and that it shares the ground point (in the diagram) with the source of the MOSFET. At least it figures it's meant to be a pull-down. However it did notice the diode is now correctly placed in parallel with the coil, and did again correctly see the diode was the wrong way around. So that's something. edit: I changed the diode direction and asked if that fixed it. Llama 3.1 did not "see" that and suggested a change. Claude 3 did much better and noticed the direction change and was happy. As I said, was just fooling around with this and thought I'd share. They're confident, but sometimes oh-so-wrong. Still, using a service where you can ask the same question to multiple models is nice. That way it can be easier to see when they're confabulating.

70 points by belter 2y ago

_f9cu 2y ago

[flagged]

coding123 2y ago
[flagged]

Lichtso 2y ago

What an awful paper title, saying "Symbolic Graphics Programs" when they just mean "vector graphics". I don't understand why they can not just use the established term instead. Also, there is no "program" here, in the same way that coding HTML is not programming, as vector graphics are not supposed to be Turing complete. And where they pulled the "symbolic" from is completely beyond me.

tines 2y ago
> Also, there is no "program" here, in the same way that coding HTML is not programming, as vector graphics are not supposed to be Turing complete. And where they pulled the "symbolic" from is completely beyond me. Aren't HTML and vector graphics descriptions both data structures that could be interpreted via a Turing-complete interpreter? I don't see the difference between HTML and a C AST in this regard.
- jlarocco 2y ago
  There's a slippery slope there. Is a Word document a program? Is a PNG file? A computer program is a data structure, but data structures are not necessarily computer programs.
  - tines 2y ago
    True, I suppose HTML the better example, it's a tree description language, whereas PNG files, Word documents, etc. aren't.
  - justsomehnguy 2y ago
    > . Is a Word document a program? Even if not there is always an OLE embedding.
merlincorey 2y ago
I'm more curious how they think LLM's can imagine things: > To understand symbolic programs, LLMs may need to possess the ability to imagine how the corresponding graphics content would look without directly accessing the rendered visual content To my understanding, LLMs are predictive engines based upon their tokens and embeddings without any ability to "imagine" things. As such, an LLM might be able to tell you that the following SVG is a black circle because it is in Mozilla documentation[0]: <svg viewBox="0 0 100 100" xmlns="http://www.w3.org/2000/svg"> <circle cx="50" cy="50" r="50" /> </svg> However, I highly doubt any LLM could tell you the following is a "Hidden Mickey" or "Mickey Mouse Head Silhouette": <svg viewBox="0 0 175 175" xmlns="http://www.w3.org/2000/svg"> <circle cx="100" cy="100" r="50" /> <circle cx="50" cy="50" r="40" /> <circle cx="150" cy="50" r="40" /> </svg> - [0] https://developer.mozilla.org/en-US/docs/Web/SVG/Element/circle https://developer.mozilla.org/en-US/docs/Web/SVG/Element/cir...
- randomdata 2y ago
  > without any ability to "imagine" things. What's imagining, then? The way LLMs explore different predictive branches in order to find an optimal solution doesn't seem all that different than what I consider imagining: Thinking about what could be and considering different variations on that idea. An LLM isn't a brain, so there is no implication of it being said in the truest human sense, but it seems like a decent analogy to me.
  - merlincorey 2y ago
    As I mentioned elsewhere, all the modern models are multimodal models so I think it would be fair to say rendering the SVG and then having the dedicated image model classify it is similar to "imagining", but it is definitely not strictly part of a Large Language Model which only predicts the next token based on previous tokens.
- rel_ic 2y ago
  Check out Act One of this This American Life episode https://www.thisamericanlife.org/803/transcript https://www.thisamericanlife.org/803/transcript TLDR: it seems like an LLM might be able to tell you your SVG is a "Mickey Mouse Head Silhouette"
jchw 2y ago
> Also, there is no "program" here, in the same way that coding HTML is not programming, as vector graphics are not supposed to be Turing complete. I think the reason why we don't view HTML as a programming language is because it is explicitly designed to be a markup language that declares content rather than a series of instructions that is interpreted as a program. A program needn't demonstrate turing completeness to be a "computer program", it just needs to be a sequence of instructions that a computer executes. To me, that suggests that there's a degree of abstractness and subjectivity involved. For example, any SVG document could also be rewritten 1:1 with no loss in fidelity as a series of commands that has the same effect, as can pretty much any declarative markup language; what is actually happening during parsing is hard to distinguish from an interpreter. Humans can "know it when they see it", but I doubt there's an exact criteria that can go along with the human "feel" of what makes a program, a program.
deleted 2y ago
[deleted]
deleted 2y ago
[deleted]

brap 2y ago

Crazy that LLMs are able to do that when they’re not even trained on visual data.

dmd 2y ago
But... they are? What are you saying here?
- katzinsky 2y ago
  I think he probably means that the LLMs never see the rasterized image output.

skybrian 2y ago

It’s difficult to tell what they’re proposing from the title or abstract, but asking questions about an SVG image, given only the source code, seems like an interesting challenge. There are a lot of ways to generate the same image in different ways, or to slightly change the source code, resulting in a very different image. This could be used to train a deeper understanding of program-like text.

slashdave 2y ago

In the title and abstract there is a proposal that LLMs "understand", "reason", and "imagine." It's off putting to me, since LLMs are not capable of any of these.

mcbuilder 2y ago
Not in the traditional sense, but how are we to possibly gauge an LLMs "intelligence" without framing questions like this?
Kuinox 2y ago
Can you come up with a proof with your bold statements, since currently leading researcher on the subject have no idea if that's the case ?
- slashdave 2y ago
  Hard to prove a negative. On the other hand, relying on a poorly defined idea of "emergent behavior" when there is no architectural rational is rather poor form IMO. It's more wordy but safer to qualify these terms appropriately. That is, to test the "appearance" or "simulation" of human-like abilities.
  - warkdarrior 2y ago
    That's all I'm hoping for, an ML model that appears to be as good as any human in existence, for all human skills. To be fair, this is what we do with humans as well, we test their skills. We (rarely!) take humans apart, inspect their brains, and declare that they have reasoning or understanding capabilities.
    - slashdave 2y ago
      Sounds like you are looking for AGI, along with many other folks. Although, be careful of the usage of "as good as", which is hard to quantify (think of artistic endeavors).
  - 2y ago

abernard1 2y ago

Since people are not reading the paper, here were the conclusions: > Is it really easy to answer semantic reasoning questions over symbolic graphics programs? We provide an intriguing experiment to demonstrate that SVG programs can be quite difficult for LLMs to understand such that even if the corresponding rendered images are fairly easy for humans to recognize, all these powerful LLMs still fail dramatically, only reaching a chance-level accuracy. They then fine tune the model for this problem, and it achieves better results.

magicalhippo 2y ago

I was recently playing with some LLMs regarding a relay driver circuit. ChatGPT generated a circuit diagram for me in ASCII art style, but it contained some errors. For fun I tried to ask some others if they could identify the errors. Here's the diagram (formatting errors were from ChatGPT): +5V | | ( ) Relay Coil | +----->|----+ | Diode | | | | | +------------+ | | Drain (D) of MOSFET | Source (S) | ----- GND | Gate (G)---[220Ω]--- GPIO (3.3V MCU) | [10kΩ] | GND Here's what Llama 3.1 70B said: After reviewing the diagram, I've found a few potential issues: - The diode is not connected to the relay coil in a way that would prevent back-EMF from damaging the MOSFET. Typically, the diode is connected in parallel with the relay coil, with the anode connected to the positive terminal of the coil and the cathode connected to the negative terminal. In this diagram, the diode appears to be connected in series with the relay coil, which is not correct. - The MOSFET is not properly biased. The gate of the MOSFET is connected to the GPIO pin through a 220Ω resistor, but there is no pull-down resistor to ensure the gate is at a known state when the GPIO pin is not driving it. The 10kΩ resistor connected to GND is likely intended to be a pull-down resistor, but it's not clear if it's connected to the gate or not. - The relay coil is not specified, but it's likely that it requires a higher current than the MOSFET can handle. Typically, a relay driver circuit would include a current-limiting resistor or a transistor to handle the high current required by the relay coil. So it correctly identified one of the main issues, the flyback diode is wrong, but failed to correctly identify the issue with the MOSFET gate being directly tied to ground. I admit, it's not a typical way to draw things so fair enough, after all it did mention there was something fishy with the pull-down. Claude 3 Haiku provided a very similar answer. It was a bit more explicit and included more details. For instance it thought there might be an issue driving the MOSFET from 3.3V, though failed to suggest a logic-level MOSFET, instead suggesting a level-shifter. It also suggested the 220 Ohm gate resistor might not be enough to protect the microcontroller. Though it also tried to correct the diagram but failed: it changed the diode to ->|<-, and didn't put it in parallel with the coil. All in all, somewhat impressed. However they get easily confused. I tried to improve the diagram, and asked again. Here's the updated diagram: +5V | | | +----->|--------+ | Diode | | | ( ) Relay Coil | | | +---------------+ | | Drain (D) of MOSFET | |]- Gate (G)-----+-[220Ω]--- GPIO (3.3V MCU) | | | [10kΩ] Source (S) | | | +----------------+ | ----- GND And Llama 3.1 now says: After examining the diagram, I notice a few potential issues: - The diode is not connected to the relay coil in a way that would prevent back-EMF damage to the MOSFET. Typically, a flyback diode is connected in parallel with the relay coil, with the anode connected to the coil's positive terminal and the cathode connected to the coil's negative terminal. In this diagram, the diode appears to be connected in series with the relay coil, which is not correct. - The MOSFET's source (S) is not connected to GND. In a typical relay driver circuit, the source of the MOSFET is connected to GND to provide a return path for the current. - The 10kΩ resistor is not connected to anything. It's possible that it's intended to be a pull-down resistor for the MOSFET's gate, but it's not clear where it's supposed to be connected. So it totally fails to "see" that the 10k is now correctly connected to ground, and that it shares the ground point (in the diagram) with the source of the MOSFET. At least it figures it's meant to be a pull-down. However it did notice the diode is now correctly placed in parallel with the coil, and did again correctly see the diode was the wrong way around. So that's something. edit: I changed the diode direction and asked if that fixed it. Llama 3.1 did not "see" that and suggested a change. Claude 3 did much better and noticed the direction change and was happy. As I said, was just fooling around with this and thought I'd share. They're confident, but sometimes oh-so-wrong. Still, using a service where you can ask the same question to multiple models is nice. That way it can be easier to see when they're confabulating.