8 ms·

It's not about choosing one or the other, it's about allowing both. I can use symbols (though not sentences or other usefully descriptive language), but do I ha

by ibdknox 13y ago

It's not about choosing one or the other, it's about allowing both. I can use symbols (though not sentences or other usefully descriptive language), but do I have an opportunity to represent those symbols at all? no. I'm not saying we should forsake language, if you look at the now very out of date Aurora demo, all the operations have sentence descriptions. This certainly isn't an all or nothing thing. If it makes sense to visualize some aspect of the system in a way that is meaningful to me, I should be able to do so - that is after all how people often solve hard problems.

RyanZAG 13y ago
In your example you used an ace of spades. Your picture took up half my screen. I can't imagine trying to actually manipulate logic when each element is taking up half my screen - can you? Instead, I can just create a variable called AceSpades. It's not as... philosophical? but it's a million times more practical. Instead of needing an artist to come and draw up a new symbol for a concept I've created, I can just write it in text as a variable name. A lot of graphically based languages have been tried but they just don't scale to general problems. They work extremely well as limited languages for very constrained problems, but as soon as you need a new concept the complexity goes out the window compared to a traditional text based language. Why? Same reason as before - defining a concept in text is easier than designing a new symbol. You touch on this in your blog too in how most of what programmers do is glue things together. You didn't really define what "glue things together" means though. I think it means "define new concepts using existing concepts". Eg, we take a MouseInput and a Sphere and create a new SphereSlice. With a text language we're just glueing the input library together with our geometry library. With a symbol based language, we have to actually define the new symbols and concepts of what a slice of a sphere is.
- dllthomas 13y ago
  "[Graphical languages] work extremely well as limited languages for very constrained problems" Even that might need a citation...
  - kens 13y ago
    A few examples of graphical programming languages are LabView, Scratch, and Lego Mindstorms (NXT). (I'm not advocating graphical programming, just providing examples.) Edit: maybe graphical GUI editors such as in Xcode and Visual Studio could be considered tools for programming languages that are partially graphical and partially text based.
    - dllthomas 13y ago
      Yeah, I'm most familiar with graphical programming by way of hearing people complain about LabView.
    - rch 13y ago
      Would you include PLC programming? It's safe to say that's been pretty successful in industrial settings.
  - RyanZAG 13y ago
    [I'm curious why I got downvoted above - did I miss something?] I don't think that needs a citation though. Any GUI is a graphical language for a constrained problem. I could use a generic text language to post this comment to HN, or I can use a specialized graphical language that provides me with this resizable text input box and reply button. It's extremely constrained in this case. If it had GUI elements for doing formatting, or the ability to post comments to other websites, it would be a less constrained graphical language. As you reduce constraints the graphical language would either need the ability to create new concepts or have many additional concepts predefined. So to answer your question about a citation: this very comment box is my citation. It works better than a purely text based input for submitting this comment. Obviously for a general language you can't predefine all the required concepts which means they need to be user defined. User defining concepts in a graphical language is a difficult task as it requires creating uniquely human recognizable symbols for the new concepts. You have two ways to get those new human recognizable symbols - you can generate them with an AI, or a human must generate them. AI is nowhere near able to generate symbols for concepts it doesn't understand as it would need to be a true AI with the ability to learn and understand new concepts. Having your graphical language's users define new concepts in effect makes those users into language designers. This is a bigger problem than it sounds as language design is an extremely difficult problem, and I personally don't want to be designing a language when I'm trying to solve a problem as I'll no doubt get the language design wrong if I'm focused more on the problem than the language design.
RogerL 13y ago
Sure, there are plenty of cases where visualization is helpful. But I see so many blog posts about it, and not much in the way of actual progress. Take the card again. It's your example, after all. I cannot think of any way to use that to, say, write a small AI to play poker. I suppose I could see a use in a debugging situation for my 'hand' variable to display a little 5@ symbol (where @ is the suit symbol). But okay, let's think about that. What does it take to get that into the system? No system 'knows' about cards. So I need a graphics designer to make a symbol for a card. I surely don't want an entire image of a card, because I have 20 other variables I am potentially interested in, which is why in this context a 5@ makes sense (like you would see in a bridge column in a newspaper). So somebody has to craft the art, we have to plug it into my dev sysstem, we need to coordinate it with the entire team, and so on. Then, it is still a very custom, one off solution. I use enums, you use ints, the python team is just using strings like "5H" - it goes on and on. I don't see a scalable solution here. Well, I do see one scalable solution. It is called text. My debugger shows a textual depiction of my variable, and my wetware translates that. I'm a good reader, and I can quickly learn to read 54, "5H", FiveHearts as being the representation of that card. Will I visually "see" the value of a particular hand as quickly? Probably not, unless I'm working this code a lot. But I'll take that over firing up a graphics team and so on. I do plenty of visualizations. It is a big reason for me using Python. If I want to write a Kalman filter, first thing I'm doing is firing up matplotlib to look at the results. But again, this is a custom process. I want to look at the noise, I want to look at the size of the kalman gain, I want to plot the filter output vs the covariance matrices, I want to.... program. Which I do textually, just fine, to generate the graphics I need. I've dealt with data flow type things before. They are a royal pain. Oh, to start, it's great. Plop a few rectangles on the screen, connect with a few lines, and wow, you've designed a nand gate, or maybe a filter in matlab, or is it a video processing toolchain? Easy peasy. But when I need to start manipulating things programmatically it is suddenly a huge pain. I am taking time out of writing an AI to categorize people based on what they are doing in a video (computer vision problem) to post this message. At a rudimentary level graphical display is great. It is certainly much easier for me to see my results displayed overlaid on the video, as opposed to trying to eyeball a JSON file or something. But to actually program this highly visual thing? I have never, ever heard anything but hand waving as to how I would do that in anything other than a textual way. I really don't think I would want to. Anyway, scale things up in a way that I don't have to write so many matplotlib calls and you will have my attention. But I just haven't seen it. I've been programming since the early 80s, and graphical programming of some form or another has been touted as 'almost here'. Still haven't seen it, except in highly specialized disciplines, and I don't want to see it. "Pictures are worth a thousand words" because of compression. It's a PCA - distill a bunch of data down to a few dimensions. Sometimes I really want that, but not when programming, where all the data matters. I don't want a low order representation of my program.
jerf 13y ago
I would be interested in seeing you take a larger chunk of code and convert it to a more symbol-rich representation. Showing a single card, especially a very large one, isn't a good representation of the idea. I would also appreciate a description of how exactly I inserted these symbols into the editor. I will spot you that I won't natively know the language in question. In turn, I warn you that the most likely criticism I will make is that you've greatly increased the cognitive load of what I have to understand to understand your code without a corresponding payoff, even accounting for fluency in the vocabulary. (I say this not to be a jerk, but precisely to issue fair warning so you can head it off at the pass.) I will also spot fluency in your paradigm of choice... while I hope that the result is not a superficial syntax gloss on top of fold & map, I am happy to accept that I would need to know what those things are. (I've come to start issuing the same challenge to anyone who thinks a visual programming language is the answer to our programming complexity problems, for instance. Don't draw me three boxes and two lines showing a simple map transform. Draw me something not huge, but nontrivial, say, the A-star algorithm. Then tell me it's better. Maybe it is, if you work on it enough, but don't scribble out the equivalent of "map (+1) [1, 2, 3]" and tell me you've "fixed" programming. Trivial's trivial in any representation.)

by ibdknox 13y ago

RyanZAG 13y ago

In your example you used an ace of spades. Your picture took up half my screen. I can't imagine trying to actually manipulate logic when each element is taking up half my screen - can you? Instead, I can just create a variable called AceSpades. It's not as... philosophical? but it's a million times more practical. Instead of needing an artist to come and draw up a new symbol for a concept I've created, I can just write it in text as a variable name. A lot of graphically based languages have been tried but they just don't scale to general problems. They work extremely well as limited languages for very constrained problems, but as soon as you need a new concept the complexity goes out the window compared to a traditional text based language. Why? Same reason as before - defining a concept in text is easier than designing a new symbol. You touch on this in your blog too in how most of what programmers do is glue things together. You didn't really define what "glue things together" means though. I think it means "define new concepts using existing concepts". Eg, we take a MouseInput and a Sphere and create a new SphereSlice. With a text language we're just glueing the input library together with our geometry library. With a symbol based language, we have to actually define the new symbols and concepts of what a slice of a sphere is.

dllthomas 13y ago
"[Graphical languages] work extremely well as limited languages for very constrained problems" Even that might need a citation...
- kens 13y ago
  A few examples of graphical programming languages are LabView, Scratch, and Lego Mindstorms (NXT). (I'm not advocating graphical programming, just providing examples.) Edit: maybe graphical GUI editors such as in Xcode and Visual Studio could be considered tools for programming languages that are partially graphical and partially text based.
  - dllthomas 13y ago
    Yeah, I'm most familiar with graphical programming by way of hearing people complain about LabView.
  - rch 13y ago
    Would you include PLC programming? It's safe to say that's been pretty successful in industrial settings.
- RyanZAG 13y ago
  [I'm curious why I got downvoted above - did I miss something?] I don't think that needs a citation though. Any GUI is a graphical language for a constrained problem. I could use a generic text language to post this comment to HN, or I can use a specialized graphical language that provides me with this resizable text input box and reply button. It's extremely constrained in this case. If it had GUI elements for doing formatting, or the ability to post comments to other websites, it would be a less constrained graphical language. As you reduce constraints the graphical language would either need the ability to create new concepts or have many additional concepts predefined. So to answer your question about a citation: this very comment box is my citation. It works better than a purely text based input for submitting this comment. Obviously for a general language you can't predefine all the required concepts which means they need to be user defined. User defining concepts in a graphical language is a difficult task as it requires creating uniquely human recognizable symbols for the new concepts. You have two ways to get those new human recognizable symbols - you can generate them with an AI, or a human must generate them. AI is nowhere near able to generate symbols for concepts it doesn't understand as it would need to be a true AI with the ability to learn and understand new concepts. Having your graphical language's users define new concepts in effect makes those users into language designers. This is a bigger problem than it sounds as language design is an extremely difficult problem, and I personally don't want to be designing a language when I'm trying to solve a problem as I'll no doubt get the language design wrong if I'm focused more on the problem than the language design.

RogerL 13y ago

Sure, there are plenty of cases where visualization is helpful. But I see so many blog posts about it, and not much in the way of actual progress. Take the card again. It's your example, after all. I cannot think of any way to use that to, say, write a small AI to play poker. I suppose I could see a use in a debugging situation for my 'hand' variable to display a little 5@ symbol (where @ is the suit symbol). But okay, let's think about that. What does it take to get that into the system? No system 'knows' about cards. So I need a graphics designer to make a symbol for a card. I surely don't want an entire image of a card, because I have 20 other variables I am potentially interested in, which is why in this context a 5@ makes sense (like you would see in a bridge column in a newspaper). So somebody has to craft the art, we have to plug it into my dev sysstem, we need to coordinate it with the entire team, and so on. Then, it is still a very custom, one off solution. I use enums, you use ints, the python team is just using strings like "5H" - it goes on and on. I don't see a scalable solution here. Well, I do see one scalable solution. It is called text. My debugger shows a textual depiction of my variable, and my wetware translates that. I'm a good reader, and I can quickly learn to read 54, "5H", FiveHearts as being the representation of that card. Will I visually "see" the value of a particular hand as quickly? Probably not, unless I'm working this code a lot. But I'll take that over firing up a graphics team and so on. I do plenty of visualizations. It is a big reason for me using Python. If I want to write a Kalman filter, first thing I'm doing is firing up matplotlib to look at the results. But again, this is a custom process. I want to look at the noise, I want to look at the size of the kalman gain, I want to plot the filter output vs the covariance matrices, I want to.... program. Which I do textually, just fine, to generate the graphics I need. I've dealt with data flow type things before. They are a royal pain. Oh, to start, it's great. Plop a few rectangles on the screen, connect with a few lines, and wow, you've designed a nand gate, or maybe a filter in matlab, or is it a video processing toolchain? Easy peasy. But when I need to start manipulating things programmatically it is suddenly a huge pain. I am taking time out of writing an AI to categorize people based on what they are doing in a video (computer vision problem) to post this message. At a rudimentary level graphical display is great. It is certainly much easier for me to see my results displayed overlaid on the video, as opposed to trying to eyeball a JSON file or something. But to actually program this highly visual thing? I have never, ever heard anything but hand waving as to how I would do that in anything other than a textual way. I really don't think I would want to. Anyway, scale things up in a way that I don't have to write so many matplotlib calls and you will have my attention. But I just haven't seen it. I've been programming since the early 80s, and graphical programming of some form or another has been touted as 'almost here'. Still haven't seen it, except in highly specialized disciplines, and I don't want to see it. "Pictures are worth a thousand words" because of compression. It's a PCA - distill a bunch of data down to a few dimensions. Sometimes I really want that, but not when programming, where all the data matters. I don't want a low order representation of my program.

jerf 13y ago

I would be interested in seeing you take a larger chunk of code and convert it to a more symbol-rich representation. Showing a single card, especially a very large one, isn't a good representation of the idea. I would also appreciate a description of how exactly I inserted these symbols into the editor. I will spot you that I won't natively know the language in question. In turn, I warn you that the most likely criticism I will make is that you've greatly increased the cognitive load of what I have to understand to understand your code without a corresponding payoff, even accounting for fluency in the vocabulary. (I say this not to be a jerk, but precisely to issue fair warning so you can head it off at the pass.) I will also spot fluency in your paradigm of choice... while I hope that the result is not a superficial syntax gloss on top of fold & map, I am happy to accept that I would need to know what those things are. (I've come to start issuing the same challenge to anyone who thinks a visual programming language is the answer to our programming complexity problems, for instance. Don't draw me three boxes and two lines showing a simple map transform. Draw me something not huge, but nontrivial, say, the A-star algorithm. Then tell me it's better. Maybe it is, if you work on it enough, but don't scribble out the equivalent of "map (+1) [1, 2, 3]" and tell me you've "fixed" programming. Trivial's trivial in any representation.)