6 ms·

Glance, a visualizer for Haskell code

128 points by chewxy 10y ago

_Robbie 10y ago
Creator of Glance here, happy to answer any questions.
- tunesmith 10y ago
  I didn't realize graphviz could do graphs of that quality. I'm used to graphviz's examples of DAGs and badly formatted text. Can you tell us more of how you're using graphviz? Wouldn't you need to move away from it as you move towards an interactive editor, in favor of other more javascript-based layout algorithms? What path do you think you'd follow for that?
  - _Robbie 10y ago
    Graphviz is only used to find the positions for the nodes using Graphviz's Neato algorithm. The nodes and the lines themselves are all rendered using the Haskell library Diagrams [0]. The next step for the project is to improve graph layout (see Glance issue here [1]), which likely means moving away from Graphviz. What tools to use for interactivity or an editor is still up in the air. [0] http://projects.haskell.org/diagrams/ http://projects.haskell.org/diagrams/ [1] https://github.com/rgleichman/glance/issues/1 https://github.com/rgleichman/glance/issues/1
    - n00b101 10y ago
      Why are the nodes rotated like that? It seems very distracting, and I think that might also be partly causing your "layout is too spread out" issue? [1] It could also be exacerbating your second issue of crossing edges - you are getting crossing edges in even the simplest graphs (e.g. your "f1" function example, with 5 nodes). GraphViz's "dot" algorithm (i.e., Sugiyama-style graph drawing algorithm) [2] should give a fairly compact representation that is organized into layers, and avoids crossing edges in at least simple cases, but rotating the nodes would again "spread out" the layout by forcing increasing height of each layer. Under "Possible solutions" you mention "create a better graph layout algorithm" - that sounds quite ambitious, wouldn't this be a PhD-thesis-level research task in itself? The only graph drawing library I'm aware of that might be competitive with GraphViz's algorithms is MSAGL [3] but that's a .NET library. [1] https://github.com/rgleichman/glance/issues/1 https://github.com/rgleichman/glance/issues/1 [2] https://en.wikipedia.org/wiki/Layered_graph_drawing https://en.wikipedia.org/wiki/Layered_graph_drawing [3] https://www.microsoft.com/en-us/research/project/microsoft-automatic-graph-layout/ https://www.microsoft.com/en-us/research/project/microsoft-a...
      - _Robbie 10y ago
        Rotating nodes is an easy way to reduce line crossings. Here's a comparison [0]. However, it does seem that vertical and angled text is harder to read than horizontal text, so there is room to improve here. [0] https://gist.github.com/rgleichman/f812150151b549ca9f634832cc402761 https://gist.github.com/rgleichman/f812150151b549ca9f634832c...
hacker_9 10y ago
I enjoy the developer's definition of 'visual'; just surrounding textual code in nicely coloured boxes.
- marcosdumay 10y ago
  Those are textual identifiers, not code. The relations are all graphic. If he started using graphic icons, he wouldn't be representing Haskell code anymore.
nacc 10y ago
If anyone like me thinks all these visualizations just make the code more confusing, I want to point out it might be a personal (although it can be very strong) preference. I always firmly believe programs are universally clearer represented as line of texts with indentations, and any attempt to visualize it doesn't help except for simple toy programs. Then I met some architects who design buildings with code using Grasshopper 3D (which is a graphical functional language for 3D modelling) [0]. Those people can easily navigate a messy web of connected lines for their hugely complex models, yet finding a block of text confusing and unintuitive. I am sure some of the more visual-inclined haskellers will find Glance very useful. [0]https://en.wikipedia.org/wiki/Grasshopper_3D https://en.wikipedia.org/wiki/Grasshopper_3D
- marcosdumay 10y ago
  LabViz is another well used graphic programming enviroment. It's also the base of the LEGO uses on their toys. I'll have to disagree. It's not a matter of personal preference. Those tools scale only into a few hundred blocks, and even for naturally visual things (like logic circuits designing) people can only grasp more complex constructions when they are text. Even if you make the same abstracction blocks available in both. Those architects probably just don't hit their limit. Physicists are all the time reaching their limit on LabViz scripts, and very vocal about it.
  - nikofeyn 10y ago
    How can you accurately comment on something that you don't even know the name of? You are almost certainly referring to LabVIEW, which is a graphical programming language commonly used in instrumentation applications and also serves as the foundation of the LEGO MINDSTORMS software. I have built many large applications in LabVIEW (>1,000 VIs), and I can easily state there is no inherent fault in being to build large systems with a graphical language vs a text-based one. Why? Because I have actually done it, resulting in extensible and maintainable code. The dataflow nature of the language makes it easy to understand how data flows through your system. It even has by-value OOP and an actor framework. Physicists complaining about it have no merit because they have zero idea about software development, architectures, design patterns, etc. They love Python for whatever reason and don't complain about it, yet it's still terrible code. I have seen LabVIEW sworn off by top of the line physicists only to be shown their Python program that was a single file with about 15,000 lines of code and such atrocities as functions with greater than 20 arguments spread over 10 lines. It was an atrocity. I have written multithreaded applications in Python, and the equivalent LabVIEW application would be far simpler because you get multithreaded behavior for free. Many software engineers swear graphical languages are only for toy applications, but yet they've never actually built anything of note themselves in one. So it's a poor argument to listen to (as in the person you replied to) when it's purely speculation based on no real data. They somehow forget that they spent four years in college getting it drilled into their head that linear, text-based files are the only way to program a computer. It's interesting to note than any high level thought (e.g. mathematics) bears more resemblance to graphical notation than pure text-based notation.
kmicklas 10y ago
If this were more compact and didn't rotate text I think it would work much better.
davesque 10y ago
The question of how concepts and algorithms are represented is interesting. While this is a neat project, a lot of people including myself might say that a "visual" representation of code is actually more confusing. Perhaps it's because textual representations are 1-dimensional i.e. the representation of code as text can only grow in one direction. This makes it inherently simple in some sense.
- chewxy 10y ago
  I have found in my time teaching people about deep learning that proper visualizations of algorithms can help people grok things better[0]. Granted backprop and neural networks are really simple one dimensional algorithms. [0]: http://blog.chewxy.com/2016/12/06/a-direct-way-of-understanding-backpropagation-and-gradient-descent/ http://blog.chewxy.com/2016/12/06/a-direct-way-of-understand...
  - _Robbie 10y ago
    The link is a blank page for me.
    - chewxy 10y ago
      Yeah... I seem to have borked my wordpress over the weekend while attempting to transfer it out to Hugo. Apologies.
blunte 10y ago
I admire the Haskell folks, much in the way I admire theoretical mathematicians... which is to say, I'm sure the work they are doing/did has some value, somewhere (in the past). But the Haskell proponents that I have known have admitted that the reason they don't use Haskell more is because the libraries and ecosystem aren't adequate. That's not to mention that maybe Haskell is just too much for even the above average dev to grok. So why push deeper? Is this just academia stroking itself? Why not invest this energy in something slightly more commercial?
- sjakobi 10y ago
  > […] the reason they don't use Haskell more is because the libraries and ecosystem aren't adequate. The adequacy of the Haskell ecosystem really varies depending on the kind of application that you want to develop. If you're writing a compiler or a web service, Haskell is a pretty good choice. This [1] is a nice overview of the state of the Haskell ecosystem. Let me also plug two more links [2] [3] that explain why Haskell is actually a very good choice for (large) commercial projects. [1] https://github.com/Gabriel439/post-rfc/blob/master/sotu.md https://github.com/Gabriel439/post-rfc/blob/master/sotu.md [2] https://www.fpcomplete.com/blog/2016/12/software-project-maintenance-is-where-haskell-shines https://www.fpcomplete.com/blog/2016/12/software-project-mai... [3] https://www.reddit.com/r/haskell/comments/54umkh/haskell_for_large_codebases_looking_for_first/ https://www.reddit.com/r/haskell/comments/54umkh/haskell_for...
- dualogy 10y ago
  As I dove into the Haskell space, I noticed where all the major "innovations" I encountered (as game-changing and substantially productivity/expressiveness/deeper-understanding-enhancing for the developer) in the mainstream tech stacks (whether .net java python ruby rust jquery coffeescript node or the more recent hypes) in the last decade were really discovered/pioneered/refined/formulated/scrutinized/formalized. It's definitely the most interesting and educating ecosystem I have discovered to play around and evolve my breath and depth of developer skills in since I basically wrote my first lines of code in Basic and Pascal long ago. Just when I got fed up with the boilerplate staleness of it all, I found the actual frontier. Good enough for me, some hackers are kept going by figuring out where currently their old "endless fascination with brainy stuff but also immediately interactively attackable with a REPL" nerve will be well tickled for the next n years. So yes "why push deeper"? It's just what some developers feel they need to do in order not to just throw out their laptops and turn to full-time gardening, full-time redditting/4chaning/twittering, or cooking shows. Not that there's anything wrong with any of these..
jblow 10y ago
I am very interested in program visualization, but I think trying to visualize things at this level is just nutballs. Anyway, it's the wrong problem. I don't need help understanding x * 3 + y. I need help understanding what these 30kLOC in these 17 files do.
- dualogy 10y ago
  If for some reason you actually need to understand 30kloc in 17 files in any meaningfully substantial fashion, I posit this will always require some time investment on your part. (Such as outcommenting most of `main` / a module, then progressively interactively proceeding to dive into the codebase via repl/compile trial+error etc.) I gather that such diagrams and visual aids etc help much more once one's basic understanding "clicks": as most of us can more easily mentally lock-in our understanding for later recollection via some visual mnemonics / cues. As always.. not the silver bullet, just the perfect tool for a context and use-case only you can decide/suspect/discover =) Edit: actually looked at the screens. Hmmm seems like a language/programming learner-aid tool mostly. Fair game for dem young'uns I guess!
  - jblow 10y ago
    Sure it will involve time investment on my part, but I want a visualization tool to help lighten that investment and help me come to the understanding more quickly.
  - jononor 10y ago
    Getting and understanding for the architecture of a code base is often very hard (time consuming) just based on the code. There are usually some high-level principles behind how the code base is put together, but rarely is this effectively communicated in the code. Usually it is comunicated in a sidechannel like a diagram (which may be outdated), or orally from someone experienced with the code base (assuming someone is available). Module import graphs and callgraphs can be useful for this. But because they are rarely used when writing/designing the software, good outputs are rarely optimized for. For instance reducing the number of levels in the module hierarchy, possibly simplifying both architecture and the resulting diagrams.
delegate 10y ago
Fantastic work! I've tried implementing something very similar for Clojure, but I thought I was insane so I stopped :) Notice how close to the lisp syntax these visualisations are, eg: (* 3 5) or (* (+ 8 7) 2) Visual representation of code opens up a lot of possibilities, like being able to click on a block and visually debug it or write all kinds of unit tests on it's 'back'. You can also 'visually' show how data passes from function to function or edit the constants in their slots and see how the changes propagate throughout the code.. Anyway, I'll check it out in more detail later, great work !

128 points by chewxy 10y ago

_Robbie 10y ago

Creator of Glance here, happy to answer any questions.

tunesmith 10y ago
I didn't realize graphviz could do graphs of that quality. I'm used to graphviz's examples of DAGs and badly formatted text. Can you tell us more of how you're using graphviz? Wouldn't you need to move away from it as you move towards an interactive editor, in favor of other more javascript-based layout algorithms? What path do you think you'd follow for that?
- _Robbie 10y ago
  Graphviz is only used to find the positions for the nodes using Graphviz's Neato algorithm. The nodes and the lines themselves are all rendered using the Haskell library Diagrams [0]. The next step for the project is to improve graph layout (see Glance issue here [1]), which likely means moving away from Graphviz. What tools to use for interactivity or an editor is still up in the air. [0] http://projects.haskell.org/diagrams/ http://projects.haskell.org/diagrams/ [1] https://github.com/rgleichman/glance/issues/1 https://github.com/rgleichman/glance/issues/1
  - n00b101 10y ago
    Why are the nodes rotated like that? It seems very distracting, and I think that might also be partly causing your "layout is too spread out" issue? [1] It could also be exacerbating your second issue of crossing edges - you are getting crossing edges in even the simplest graphs (e.g. your "f1" function example, with 5 nodes). GraphViz's "dot" algorithm (i.e., Sugiyama-style graph drawing algorithm) [2] should give a fairly compact representation that is organized into layers, and avoids crossing edges in at least simple cases, but rotating the nodes would again "spread out" the layout by forcing increasing height of each layer. Under "Possible solutions" you mention "create a better graph layout algorithm" - that sounds quite ambitious, wouldn't this be a PhD-thesis-level research task in itself? The only graph drawing library I'm aware of that might be competitive with GraphViz's algorithms is MSAGL [3] but that's a .NET library. [1] https://github.com/rgleichman/glance/issues/1 https://github.com/rgleichman/glance/issues/1 [2] https://en.wikipedia.org/wiki/Layered_graph_drawing https://en.wikipedia.org/wiki/Layered_graph_drawing [3] https://www.microsoft.com/en-us/research/project/microsoft-automatic-graph-layout/ https://www.microsoft.com/en-us/research/project/microsoft-a...
    - _Robbie 10y ago
      Rotating nodes is an easy way to reduce line crossings. Here's a comparison [0]. However, it does seem that vertical and angled text is harder to read than horizontal text, so there is room to improve here. [0] https://gist.github.com/rgleichman/f812150151b549ca9f634832cc402761 https://gist.github.com/rgleichman/f812150151b549ca9f634832c...

hacker_9 10y ago

I enjoy the developer's definition of 'visual'; just surrounding textual code in nicely coloured boxes.

marcosdumay 10y ago
Those are textual identifiers, not code. The relations are all graphic. If he started using graphic icons, he wouldn't be representing Haskell code anymore.

nacc 10y ago

If anyone like me thinks all these visualizations just make the code more confusing, I want to point out it might be a personal (although it can be very strong) preference. I always firmly believe programs are universally clearer represented as line of texts with indentations, and any attempt to visualize it doesn't help except for simple toy programs. Then I met some architects who design buildings with code using Grasshopper 3D (which is a graphical functional language for 3D modelling) [0]. Those people can easily navigate a messy web of connected lines for their hugely complex models, yet finding a block of text confusing and unintuitive. I am sure some of the more visual-inclined haskellers will find Glance very useful. [0]https://en.wikipedia.org/wiki/Grasshopper_3D https://en.wikipedia.org/wiki/Grasshopper_3D

marcosdumay 10y ago
LabViz is another well used graphic programming enviroment. It's also the base of the LEGO uses on their toys. I'll have to disagree. It's not a matter of personal preference. Those tools scale only into a few hundred blocks, and even for naturally visual things (like logic circuits designing) people can only grasp more complex constructions when they are text. Even if you make the same abstracction blocks available in both. Those architects probably just don't hit their limit. Physicists are all the time reaching their limit on LabViz scripts, and very vocal about it.
- nikofeyn 10y ago
  How can you accurately comment on something that you don't even know the name of? You are almost certainly referring to LabVIEW, which is a graphical programming language commonly used in instrumentation applications and also serves as the foundation of the LEGO MINDSTORMS software. I have built many large applications in LabVIEW (>1,000 VIs), and I can easily state there is no inherent fault in being to build large systems with a graphical language vs a text-based one. Why? Because I have actually done it, resulting in extensible and maintainable code. The dataflow nature of the language makes it easy to understand how data flows through your system. It even has by-value OOP and an actor framework. Physicists complaining about it have no merit because they have zero idea about software development, architectures, design patterns, etc. They love Python for whatever reason and don't complain about it, yet it's still terrible code. I have seen LabVIEW sworn off by top of the line physicists only to be shown their Python program that was a single file with about 15,000 lines of code and such atrocities as functions with greater than 20 arguments spread over 10 lines. It was an atrocity. I have written multithreaded applications in Python, and the equivalent LabVIEW application would be far simpler because you get multithreaded behavior for free. Many software engineers swear graphical languages are only for toy applications, but yet they've never actually built anything of note themselves in one. So it's a poor argument to listen to (as in the person you replied to) when it's purely speculation based on no real data. They somehow forget that they spent four years in college getting it drilled into their head that linear, text-based files are the only way to program a computer. It's interesting to note than any high level thought (e.g. mathematics) bears more resemblance to graphical notation than pure text-based notation.

kmicklas 10y ago

If this were more compact and didn't rotate text I think it would work much better.

davesque 10y ago

The question of how concepts and algorithms are represented is interesting. While this is a neat project, a lot of people including myself might say that a "visual" representation of code is actually more confusing. Perhaps it's because textual representations are 1-dimensional i.e. the representation of code as text can only grow in one direction. This makes it inherently simple in some sense.

chewxy 10y ago
I have found in my time teaching people about deep learning that proper visualizations of algorithms can help people grok things better[0]. Granted backprop and neural networks are really simple one dimensional algorithms. [0]: http://blog.chewxy.com/2016/12/06/a-direct-way-of-understanding-backpropagation-and-gradient-descent/ http://blog.chewxy.com/2016/12/06/a-direct-way-of-understand...
- _Robbie 10y ago
  The link is a blank page for me.
  - chewxy 10y ago
    Yeah... I seem to have borked my wordpress over the weekend while attempting to transfer it out to Hugo. Apologies.

blunte 10y ago

I admire the Haskell folks, much in the way I admire theoretical mathematicians... which is to say, I'm sure the work they are doing/did has some value, somewhere (in the past). But the Haskell proponents that I have known have admitted that the reason they don't use Haskell more is because the libraries and ecosystem aren't adequate. That's not to mention that maybe Haskell is just too much for even the above average dev to grok. So why push deeper? Is this just academia stroking itself? Why not invest this energy in something slightly more commercial?

sjakobi 10y ago
> […] the reason they don't use Haskell more is because the libraries and ecosystem aren't adequate. The adequacy of the Haskell ecosystem really varies depending on the kind of application that you want to develop. If you're writing a compiler or a web service, Haskell is a pretty good choice. This [1] is a nice overview of the state of the Haskell ecosystem. Let me also plug two more links [2] [3] that explain why Haskell is actually a very good choice for (large) commercial projects. [1] https://github.com/Gabriel439/post-rfc/blob/master/sotu.md https://github.com/Gabriel439/post-rfc/blob/master/sotu.md [2] https://www.fpcomplete.com/blog/2016/12/software-project-maintenance-is-where-haskell-shines https://www.fpcomplete.com/blog/2016/12/software-project-mai... [3] https://www.reddit.com/r/haskell/comments/54umkh/haskell_for_large_codebases_looking_for_first/ https://www.reddit.com/r/haskell/comments/54umkh/haskell_for...
dualogy 10y ago
As I dove into the Haskell space, I noticed where all the major "innovations" I encountered (as game-changing and substantially productivity/expressiveness/deeper-understanding-enhancing for the developer) in the mainstream tech stacks (whether .net java python ruby rust jquery coffeescript node or the more recent hypes) in the last decade were really discovered/pioneered/refined/formulated/scrutinized/formalized. It's definitely the most interesting and educating ecosystem I have discovered to play around and evolve my breath and depth of developer skills in since I basically wrote my first lines of code in Basic and Pascal long ago. Just when I got fed up with the boilerplate staleness of it all, I found the actual frontier. Good enough for me, some hackers are kept going by figuring out where currently their old "endless fascination with brainy stuff but also immediately interactively attackable with a REPL" nerve will be well tickled for the next n years. So yes "why push deeper"? It's just what some developers feel they need to do in order not to just throw out their laptops and turn to full-time gardening, full-time redditting/4chaning/twittering, or cooking shows. Not that there's anything wrong with any of these..

jblow 10y ago

I am very interested in program visualization, but I think trying to visualize things at this level is just nutballs. Anyway, it's the wrong problem. I don't need help understanding x * 3 + y. I need help understanding what these 30kLOC in these 17 files do.

dualogy 10y ago
If for some reason you actually need to understand 30kloc in 17 files in any meaningfully substantial fashion, I posit this will always require some time investment on your part. (Such as outcommenting most of `main` / a module, then progressively interactively proceeding to dive into the codebase via repl/compile trial+error etc.) I gather that such diagrams and visual aids etc help much more once one's basic understanding "clicks": as most of us can more easily mentally lock-in our understanding for later recollection via some visual mnemonics / cues. As always.. not the silver bullet, just the perfect tool for a context and use-case only you can decide/suspect/discover =) Edit: actually looked at the screens. Hmmm seems like a language/programming learner-aid tool mostly. Fair game for dem young'uns I guess!
- jblow 10y ago
  Sure it will involve time investment on my part, but I want a visualization tool to help lighten that investment and help me come to the understanding more quickly.
- jononor 10y ago
  Getting and understanding for the architecture of a code base is often very hard (time consuming) just based on the code. There are usually some high-level principles behind how the code base is put together, but rarely is this effectively communicated in the code. Usually it is comunicated in a sidechannel like a diagram (which may be outdated), or orally from someone experienced with the code base (assuming someone is available). Module import graphs and callgraphs can be useful for this. But because they are rarely used when writing/designing the software, good outputs are rarely optimized for. For instance reducing the number of levels in the module hierarchy, possibly simplifying both architecture and the resulting diagrams.

delegate 10y ago

Fantastic work! I've tried implementing something very similar for Clojure, but I thought I was insane so I stopped :) Notice how close to the lisp syntax these visualisations are, eg: (* 3 5) or (* (+ 8 7) 2) Visual representation of code opens up a lot of possibilities, like being able to click on a block and visually debug it or write all kinds of unit tests on it's 'back'. You can also 'visually' show how data passes from function to function or edit the constants in their slots and see how the changes propagate throughout the code.. Anyway, I'll check it out in more detail later, great work !