5 ms·

It is indeed great, and probably one reason you don't see it more is that it's a ton of work! That's 5K lines of hand-written code! (view source and it's at t

by chubot 7y ago

It is indeed great, and probably one reason you don't see it more is that it's a ton of work! That's 5K lines of hand-written code! (view source and it's at the end of the page) $ wc -l gears.js base.js 4135 gears.js 904 base.js 5039 total I've wanted to make visualizations like this for my blog. Writing a blog post takes me around 10 hours, which is a fair amount of effort. I believe that the visualizations will take more time than writing in general -- i.e. it would be more like 20 hours, bringing the total to 30. If you consider that it's 4K lines of code (assuming the base library is reused), it's very easy to see that it could take 20+ hours. Probably more like 40-80 to be honest. I guess there is that other thread that says people only write 10 lines of code a day, which would make it 400 hours ;) I don't think that applies here but it could be closer to 400 than 10 or 20. ---- I also wonder if you can save time by using a less "hand-written" style (e.g. d3.js, which seems to be on everyone's wishlist to learn). My inclination is also to go "hand-written" rather than using a bunch of JS libraries. I think you get a better result that way. It's interesting to see that I'm not totally wrong -- the thing everyone praises ends up being very hand-written. And it's smooth and fast, etc. ---- another edit: I also believe one reason that this visualization is good is because there's no build process in the JS. The author clearly just edits the code and refreshes the browser. You need that kind of fast edit-view loop to make good visualizations. IOW, consider using plain JS for blog posts. They are documents and not apps.

amitp 7y ago
Having been down this road, yes, it does take time, but you can save some time by using SVG and some DOM library. Although canvas is faster, most diagrams don't really need it, so I use SVG unless I really need to switch to canvas. SVG also adjusts for screen dpi automatically. The things I like: 1. Reactivity (ObservableHQ, Vue.js, hyperactiv.js, etc.). There's usually some underlying data and then a corresponding visualization. These reactive systems let you modify the underlying data and then the visualization updates automatically. You don't have to figure out which diagrams to update when. Even easier: just redraw everything every time you change anything. 2. Some easier way to write the DOM (d3.js, jsx, vue, lit-html, etc.). Since I'm writing a blog post with html, I usually prefer writing my js-in-html (vue) rather than html-in-js (jsx) but try both directions and see which you prefer. 3. No build step. This is especially important when I want to update a page years later and don't want to figure out which build tools I was using in 1997 or 2007 or 2017. I want my pages to last for decades, and I still update my pages from 25 years ago. I tried recreating one of the gear page diagrams in ObservableHQ https://observablehq.com/@redblobgames/unwind-circle-example https://observablehq.com/@redblobgames/unwind-circle-example . It's not a lot of code. There's a slider, there's a loop to generate the lines, and there's the output svg. Whenever you move the slider it recalculates the output. I admit that I'm not using ObservableHQ much for my own projects because I want more of a "hand-written" style. I used d3.js for my older pages and vue.js for my newer pages. Vue's reactivity and templates save me probably a factor of 2 or 3 over d3.js.
- kragen 7y ago
  Thanks for sharing these tips! I didn't realize you were trying out ObservableHQ too.
  - amitp 7y ago
    I love ObservableHQ! I think it's a neat model that fixes a lot of things I didn't like about notebooks. I don't use it much though. It makes the simple things easier but it makes the complex things harder. Examples: 1. I often start making a diagram interactive by adding a slider connected to some state. The slider/state is in one "viewof" cell and the diagram is in another cell. But for more polished work I often want to use direct manipulation, where you move something around in the diagram itself. In my usual JS+Vue code it's a small amount of extra work. But in ObservableHQ it seems like a lot of work: the diagram has to modify state defined in a different cell (breaking the simple spreadsheet model in my head), and the diagram is being re-rendered while it has event handlers active. 2. I often start out by making one diagram. This is nice and easy. But then I want to make multiple diagrams with some shared state. In my usual JS+Vue code I can lift the diagram code into a function, pass in a shared object for shared state, and instantiate objects for non-shared state. In ObservableHQ those properties are in cells, and I can't generate multiple cells programatically (afaik). I also would prefer to host things myself, both because of longevity, but also because I sometimes work offline (e.g. in a park or on a train). And the biggest reason I'm not using ObservableHQ is that I'm so much slower editing text on it than I am in Emacs. So I continue to use ObservableHQ for some simple projects/prototypes but I don't do a lot with it.
    - kragen 7y ago
      Yeah, the centralization aspect is a big turnoff for me. I'd thought about the direct-manipulation issue (the rather crude visualization of skeletonization I linked below benefits a lot from it) but not the multiple-projections problem. Can't you put the entire state in one cell and then create multiple cells that render some reduced dimensionality projection of it? Or do you mean that there's no way to factor out the aggregation of three such projections into a reusable unit? I recently walked my girlfriend through your explorable on populating landscapes with Perlin noise btw. It was great.
kragen 7y ago
Writing a small program takes much less effort per line than writing a large program, especially if it doesn't have to be maintainable and extensible, especially by other people. http://canonical.org/~kragen/sw/dev3/circlegrid.html http://canonical.org/~kragen/sw/dev3/circlegrid.html only took me three hours of JS hacking, and it's like 200 lines of code. The simplified version of the basic COCOMO model in David A. Wheeler's "SLOCCount" is Person-Months = 2.4 * (KSLOC1.05), which suggests a budget of 0.44 person-months for this code, about two weeks, which is clearly too high; we've all written and debugged a few hundred lines of code in a day on some occasion after getting out of the larval stage. This probably would have been faster if I hadn't been so out of practice with DHTML. Another thing, though, is that some things are easier than other things. Usually in a programming job you have to do the easy things and also the hard things. This brings down the average. If you're writing a bunch of blog posts for fun, though, you can just publish the ones where good visualizations came out pretty quickly and discard the others that are much more effort for less return. I feel like visualizations using d3.js are still pretty "hand-written".
- chubot 7y ago
  That's true, and like everything I'm sure writing JS visualizations gets faster with practice, and you become more familiar with the tools. Still, after finishing the article, I'm impressed by not just the the number visualizations, but by their legibility and smoothness. I can't in any universe imagine it taking less than a week of full-time work. If you told me it took a month of full-time work I wouldn't be surprised at all. ---- I liked this concept of "interpretive labor" and that's what I'm really getting at: https://distill.pub/2017/research-debt/ https://distill.pub/2017/research-debt/ There’s a tradeoff between the energy put into explaining an idea, and the energy needed to understand it. On one extreme, the explainer can painstakingly craft a beautiful explanation, leading their audience to understanding without even realizing it could have been difficult. On the other extreme, the explainer can do the absolute minimum and abandon their audience to struggle. I've been writing for public consumption for about 3 years and really feel that tradeoff. When I put effort into some writing or explanation, the result is better. People tell me it clicked, etc. And I would say you can go a lot further than you think in bridging the gap. This article is evidence of that! Lots of people here are saying they wished they had this in high school, etc. 3Blue1Brown's videos are another example of that. I was fairly good at math in high school but if those videos existed then, I would have probably gotten a lot further. ---- This ties in nicely to your Dercuano notes which I have had open in my browser for awhile! There are many interesting topics there. But I also feel like you went to the other extreme and there's a lot of interpretive labor involved in reading them :) Why did you decide to polish and publish the "memory models" post? I'd like to see more posts like that. (I could send you the notes that jumped out at me if interested.) Whenever I've taken the time to polish some writing, I've never regretted it. For some reason it feels annoying to start (probably because I have to clear my brain of other things), but when I'm doing it it's fun, and when I'm done, it's worthwhile.
1wheel 7y ago
> another edit: I also believe one reason that this visualization is good is because there's no build process in the JS. The author clearly just edits the code and refreshes the browser. You need that kind of fast edit-view loop to make good visualizations. You can go a step further and reload code without refreshing the page. This makes it even easier to make the sort of fiddly tweaks that visualizations require while avoid change blindness. https://roadtolarissa.com/hot-reload https://roadtolarissa.com/hot-reload https://github.com/1wheel/hot-server https://github.com/1wheel/hot-server
- kragen 7y ago
  In some cases I've exposed sliders to enable a 60Hz feedback loop on those fiddly tweaks. In other cases Firebug or its modern successors already expose enough.
32gbsd 7y ago
This. The time and effort it takes to put such a visualization together for a target audience that might possibly just gloss over the content is immense. Its like making a multi-million dollar movie about people who have no money just to show thier situation. Instead of giving them the money directly. And on top of that you need all the requirements to actually be able to view it in the latest browser.

by chubot 7y ago

amitp 7y ago

Having been down this road, yes, it does take time, but you can save some time by using SVG and some DOM library. Although canvas is faster, most diagrams don't really need it, so I use SVG unless I really need to switch to canvas. SVG also adjusts for screen dpi automatically. The things I like: 1. Reactivity (ObservableHQ, Vue.js, hyperactiv.js, etc.). There's usually some underlying data and then a corresponding visualization. These reactive systems let you modify the underlying data and then the visualization updates automatically. You don't have to figure out which diagrams to update when. Even easier: just redraw everything every time you change anything. 2. Some easier way to write the DOM (d3.js, jsx, vue, lit-html, etc.). Since I'm writing a blog post with html, I usually prefer writing my js-in-html (vue) rather than html-in-js (jsx) but try both directions and see which you prefer. 3. No build step. This is especially important when I want to update a page years later and don't want to figure out which build tools I was using in 1997 or 2007 or 2017. I want my pages to last for decades, and I still update my pages from 25 years ago. I tried recreating one of the gear page diagrams in ObservableHQ https://observablehq.com/@redblobgames/unwind-circle-example https://observablehq.com/@redblobgames/unwind-circle-example . It's not a lot of code. There's a slider, there's a loop to generate the lines, and there's the output svg. Whenever you move the slider it recalculates the output. I admit that I'm not using ObservableHQ much for my own projects because I want more of a "hand-written" style. I used d3.js for my older pages and vue.js for my newer pages. Vue's reactivity and templates save me probably a factor of 2 or 3 over d3.js.

kragen 7y ago
Thanks for sharing these tips! I didn't realize you were trying out ObservableHQ too.
- amitp 7y ago
  I love ObservableHQ! I think it's a neat model that fixes a lot of things I didn't like about notebooks. I don't use it much though. It makes the simple things easier but it makes the complex things harder. Examples: 1. I often start making a diagram interactive by adding a slider connected to some state. The slider/state is in one "viewof" cell and the diagram is in another cell. But for more polished work I often want to use direct manipulation, where you move something around in the diagram itself. In my usual JS+Vue code it's a small amount of extra work. But in ObservableHQ it seems like a lot of work: the diagram has to modify state defined in a different cell (breaking the simple spreadsheet model in my head), and the diagram is being re-rendered while it has event handlers active. 2. I often start out by making one diagram. This is nice and easy. But then I want to make multiple diagrams with some shared state. In my usual JS+Vue code I can lift the diagram code into a function, pass in a shared object for shared state, and instantiate objects for non-shared state. In ObservableHQ those properties are in cells, and I can't generate multiple cells programatically (afaik). I also would prefer to host things myself, both because of longevity, but also because I sometimes work offline (e.g. in a park or on a train). And the biggest reason I'm not using ObservableHQ is that I'm so much slower editing text on it than I am in Emacs. So I continue to use ObservableHQ for some simple projects/prototypes but I don't do a lot with it.
  - kragen 7y ago
    Yeah, the centralization aspect is a big turnoff for me. I'd thought about the direct-manipulation issue (the rather crude visualization of skeletonization I linked below benefits a lot from it) but not the multiple-projections problem. Can't you put the entire state in one cell and then create multiple cells that render some reduced dimensionality projection of it? Or do you mean that there's no way to factor out the aggregation of three such projections into a reusable unit? I recently walked my girlfriend through your explorable on populating landscapes with Perlin noise btw. It was great.

kragen 7y ago

Writing a small program takes much less effort per line than writing a large program, especially if it doesn't have to be maintainable and extensible, especially by other people. http://canonical.org/~kragen/sw/dev3/circlegrid.html http://canonical.org/~kragen/sw/dev3/circlegrid.html only took me three hours of JS hacking, and it's like 200 lines of code. The simplified version of the basic COCOMO model in David A. Wheeler's "SLOCCount" is Person-Months = 2.4 * (KSLOC1.05), which suggests a budget of 0.44 person-months for this code, about two weeks, which is clearly too high; we've all written and debugged a few hundred lines of code in a day on some occasion after getting out of the larval stage. This probably would have been faster if I hadn't been so out of practice with DHTML. Another thing, though, is that some things are easier than other things. Usually in a programming job you have to do the easy things and also the hard things. This brings down the average. If you're writing a bunch of blog posts for fun, though, you can just publish the ones where good visualizations came out pretty quickly and discard the others that are much more effort for less return. I feel like visualizations using d3.js are still pretty "hand-written".

chubot 7y ago
That's true, and like everything I'm sure writing JS visualizations gets faster with practice, and you become more familiar with the tools. Still, after finishing the article, I'm impressed by not just the the number visualizations, but by their legibility and smoothness. I can't in any universe imagine it taking less than a week of full-time work. If you told me it took a month of full-time work I wouldn't be surprised at all. ---- I liked this concept of "interpretive labor" and that's what I'm really getting at: https://distill.pub/2017/research-debt/ https://distill.pub/2017/research-debt/ There’s a tradeoff between the energy put into explaining an idea, and the energy needed to understand it. On one extreme, the explainer can painstakingly craft a beautiful explanation, leading their audience to understanding without even realizing it could have been difficult. On the other extreme, the explainer can do the absolute minimum and abandon their audience to struggle. I've been writing for public consumption for about 3 years and really feel that tradeoff. When I put effort into some writing or explanation, the result is better. People tell me it clicked, etc. And I would say you can go a lot further than you think in bridging the gap. This article is evidence of that! Lots of people here are saying they wished they had this in high school, etc. 3Blue1Brown's videos are another example of that. I was fairly good at math in high school but if those videos existed then, I would have probably gotten a lot further. ---- This ties in nicely to your Dercuano notes which I have had open in my browser for awhile! There are many interesting topics there. But I also feel like you went to the other extreme and there's a lot of interpretive labor involved in reading them :) Why did you decide to polish and publish the "memory models" post? I'd like to see more posts like that. (I could send you the notes that jumped out at me if interested.) Whenever I've taken the time to polish some writing, I've never regretted it. For some reason it feels annoying to start (probably because I have to clear my brain of other things), but when I'm doing it it's fun, and when I'm done, it's worthwhile.

1wheel 7y ago

> another edit: I also believe one reason that this visualization is good is because there's no build process in the JS. The author clearly just edits the code and refreshes the browser. You need that kind of fast edit-view loop to make good visualizations. You can go a step further and reload code without refreshing the page. This makes it even easier to make the sort of fiddly tweaks that visualizations require while avoid change blindness. https://roadtolarissa.com/hot-reload https://roadtolarissa.com/hot-reload https://github.com/1wheel/hot-server https://github.com/1wheel/hot-server

kragen 7y ago
In some cases I've exposed sliders to enable a 60Hz feedback loop on those fiddly tweaks. In other cases Firebug or its modern successors already expose enough.

32gbsd 7y ago

This. The time and effort it takes to put such a visualization together for a target audience that might possibly just gloss over the content is immense. Its like making a multi-million dollar movie about people who have no money just to show thier situation. Instead of giving them the money directly. And on top of that you need all the requirements to actually be able to view it in the latest browser.