12 ms·

The Trouble with D3

350 points by danso 8y ago

danso 8y ago
I think one of the overarching factors is that those new to data visualization (and web development) see D3's beautiful examples and assume that the D3 library is directly and fundamentally responsible for the visual quality. I've seen this a bit with R's ggplot2 as well among relatively experienced data scientists. To be sure, ggplot2 is a fantastic library and the fact that its defaults look great are a testament to the library's well-thought out design. But such design grace can be achieved in any library (e.g. matplotlib), albeit with less graceful APIs.
- bunderbunder 8y ago
  You can absolutely get there. But the trouble with most libraries I've used is that they are either limiting or confusing. For example, Seaborn is very easy to learn, but it's mostly just a set of canned visualizations, with limited flexibility. Matplotlib is very flexible, but just. so. hard. to. learn. I had a machine learning class in grad school where most everyone agreed the single most difficult homework was one that asked us to use Matplotlib to generate a few visualizations of what the models were doing. Comparatively speaking, learning ggplot2 is a sublime experience. After about a week, you feel like you have super powers. D3 does feel overcomplicated for just building static visualizations, but if you're building custom interactive stuff, learning it can be nearly as joyous an experience.
  - _Wintermute 8y ago
    Regarding ggplot, matplotlib and seaborn I found the opposite. If you want some pre-canned visualisation then ggplot2 is fantastic, but as soon as you stray away from the examples it turns into a mess, you can't just drop down into base graphics but have to mess about with grob and grid. Whereas seaborn has similar pre-canned plots, but it's all built on matplotlib so you still have that flexibility if you want it.
- cup-of-tea 8y ago
  Yeah. If you come to d3 but you've never made a beautiful image in SVG before you're going to have a bad time. You have to be good at SVG, CSS and javascript already. D3 will not help.
ribhu97 8y ago
C3 is a javascript library based on D3 that simplifies it and makes it easier to use for designers and full stack developers.
- sakopov 8y ago
  C3 looked like a dead project about a year or so ago, which I thought was a real shame and planned to move away from it. Just checked the repo a few minutes ago and it looks like things are pretty lively these days. Glad to see it's kicking along again.
- FLUX-YOU 8y ago
  We've been using this. It's really nice and straight-forward, although it has some edge cases when you are doing small graphs and responsive layouts. Using it with Angular also adds some complexity.
spenrose 8y ago
Very popular simple time series library built on top of d3.js: https://metricsgraphicsjs.org https://metricsgraphicsjs.org.
jbob2000 8y ago
Ehh, that was kind of a thin article that didn’t quite scratch the itch on why D3 is troublesome. First, I think if you want to use d3, you probably just want a charting library. Convince yourself that HighCharts is not flexible enough and then try d3. So the trouble with d3 is that most people don’t need to use it and are in over their heads. Second, d3 is full of leaky abstractions. Imagine if cars didn’t have acceleration pedals, and you had to manually open the throttle and engage the fuel pump yourself. You’d need an engineering degree to drive your car! Same thing with d3; it’s ridiculous that you need to know the ins and outs of some of the internet’s most complicated and frustrating technology just to show a couple of data points on a graph. It’d be great if you could work with d3 and pretend like SVG and the DOM didn’t exist.
- fein 8y ago
  nvd3 if you want d3 charts without the d3 hassle: http://nvd3.org/ http://nvd3.org/
jdonaldson 8y ago
There's a place for cookie cutter visualizations. However, you're never going to make anything distinctive or memorable that way. All of it is a trade off... Are you trying to fit in or stand out?
sydd 8y ago
Dunno for me D3 feels like they are trying to do everything at once resulting in an overly complex and hard-to-learn library. When I look at the examples at their page I see stuff that I could make in 2 ways: - Just write a small program from scratch that visualizes one in a week or 2. It will be small, I will know the ins and outs, so easy to change and will be a ton of fun to write. - Use D3.js (I dont know this library, but i've used complex charting libraries in the past). Likely it will take more time, not because I am thinking of the interesting parts, I will spend most of my day cursing on StackOverFlow to figure out how to skin that one tricky part of my app. So in the end I dont really see a good reason to use d3.js unless my full time job is to develop data visualizations.
- wpietri 8y ago
  I learned (some of) D3 in exactly the fashion the article suggested: find something I want to visualize and make it happen. I thought it was great. I still wouldn't say I've "learned D3", but I didn't have to. I got my goal accomplished in a couple of days, along with a lot of nice bits I didn't intend along the way as I found good things that D3 could do for me. For me it was definitely a "design to tools" approach; I looked at the gallery, found something mostly like what I wanted, and then worked to that. That was fine by me; I wasn't looking for ultimate visual control, but just trying to turn a bunch of data into something mortals could comprehend.
hacker_9 8y ago
What's even more confusing is that v3 -> v4 includes a bunch of breaking changes, and yet they didn't update their typescript definition file. Not to mention the web is full of v3 examples that mysteriously don't work with your d3 lib (and versions are never mentioned anywhere).
- symlinkk 8y ago
  I've had no problem with TypeScript and D3. I'm using `@types/d3`, is that what you're using?
- mschuster91 8y ago
  > Not to mention the web is full of v3 examples that mysteriously don't work with your d3 lib Unfortunately this is widespread across the JS ecosystem. Hell I'm not saying anyone should do it like Linux (break userspace and get shot), but when one does backwards-incompatible changes, is it so hard to stub dead methods or unsupported calling ways with "console.error('Outdated, see https://github.com/x/y/issues/z');" https://github.com/x/y/issues/z');"?
stared 8y ago
Side note (on D3.js) - there is a nice exploration of reusable D3.js charts/components: https://d3-discovery.net/ https://d3-discovery.net/ ("Finding D3 plugins with ease"). It's source (Vue-based, potentially for other showcases/searches): https://github.com/wbkd/d3-discovery https://github.com/wbkd/d3-discovery (I just asked if it can be open source, and to my awe its author added MIT license :)).
k__ 8y ago
D3 is for high performance custom data visalizations. For example, I built a in-door navigation app. It would receive its data via MQTT/WebSockets and update markers on custom maps. D3 was perfect for this, it helped with synchronizing graphics with data, converting mapping cooridnates to DOM/SVG coordinates and easing all the updates so stuff looked smooth. You just would pump the incoming data into the D3 pipeline and be done with it. If you just need a few animations, there are a bunch of simple libraries for this. React has react-motion for example, it's not as flexible as D3 but you get most stuff you would expect out of it without much hassle. If you just need a set of basic charts, use a charting library. Highcharts or Charts.js are both very capable.
- bigger_cheese 8y ago
  My work has been pretty interested in "interactive" charts, being able to modify axis scales on the fly, change filter range, dynamically add and remove series that sort of thing. We use SAS (and a little bit of R internally) and those both have good charting capabilities but trying to do anything interactive in either of those packages is a real pain. In the past we have used a C++ library SL-GMS. Even when compared with using C++ I have found using JavaScript based libraries difficult D3 in particular I found to be horrendously complex.
blattimwind 8y ago
I do quite a bit of woodworking and use a lot of handtools, yet a lot of the tools on that wall look completely alien to me. I think this wall might actually belong to an upholsterer or something like that... Edit: The title of the stock photo is "Timmermansgereedschap" (NL) = Zimmermannsgerätschaften (DE) = carpenter's tools (EN).
duckfruit 8y ago
The most salient point that the article breaches is this: When people unfamiliar with the DOM, javascript, or indeed in some cases programming itself try to 'pick up' a complex low level data visualization library like D3, it is natural for them to blame the library. But the problem obviously runs much deeper than that. Running with the analogy in the article -- it is fruitless for someone who's never done any woodworking or furniture design ever to hope to become the next Charles Eames within the span of a few months.
- CJefferson 8y ago
  However, d3 seems particularly bizarre. Its the only library i ever remember trying hard to learn, and just giving up. There is too little indication, when you misuse it, of what went wrong. Ive done a bunch of javascript, DOM and visualisations in OpenGL, so i feel i should have a good chance.
  - GijsjanB 8y ago
    Complexity comes at a price. Making it easy, abstracts certain low level functionality away. Compare Rust to Python. Compare OpenLayers to Leaflet. Compare the dashboard of an Apache to your car's dashboard. Rust, OpenLayers en Apaches aren't bizarre, they have complex functionality which you don't want to abstract away, because you want the ability to use it on that low level.
  - kjeetgill 8y ago
    I don't know if this was your issue, but discovering that most of the API was accessors to SVG opened the floodgates for me. That said, I mostly learned it through the examples rather than a tutorial.
  - bunderbunder 8y ago
    A lot of D3's oddness comes from the fact that there isn't anything (to my knowledge) that is comparable. It does just one thing - binding data to DOM elements - does it well, and then gets out of the way.
stared 8y ago
> The trouble with D3 is to build a visualization you must also have a deep understanding of SVG, DOM, JavaScript, geometry, color spaces, data structures, the standard model, and quantum physics. A piece of cake! Here is a quantum game in D3.js: http://quantumgame.io/ http://quantumgame.io/ :) (full disclaimer: mine, and I already do too much of its shameless self-promotion)
forrestthewoods 8y ago
It’s honestly easier to just use VanillaJS.
stupidcar 8y ago
Visualisation is fundamentally the process of creating a mapping function between the dimensions of your domain data and that of the "screen" (whether it's DOM, SVG, Canvas, etc.). D3 is just a library of higher-order functions that can help you produce that mapping function. If you can grasp this idea of visualisation as creating a mapping function, then D3 becomes a beautiful toolbox for assisting you. You decide which dimension of your data you want to represent as the horizontal dimension, which as the vertical, as color, as size, etc. You decide how you want to scale them. Then you use D3's high-order functions to create the appropriate individual mapping functions, compose them into your complete visualisation function, and pass it your domain data. If you think of D3 as a "charting library", or a collection of visualisation classes or objects, then you're going to struggle. But the problem isn't with D3, it's that you haven't really understood the paradigm of the library.
rpowers 8y ago
Anecdotal, but my problem with D3 is the need for so many function calls. 'append', 'attr', 'enter', 'datum', ..... Code ends up looking like a centipede ().().().()
- andrewmcwatters 8y ago
  Code formatting seems to be an easily dismissed part of software development: you should be breaking those out on newlines.
mhale 8y ago
I worked on a project that required some pretty standard line charts, but with the addition of an overlaid set of horizontal "benchmark" lines -- and some callouts. After struggling (and failing) to get what we wanted out of two different off-the-shelf graphing libraries (one of which was built on top of D3), we finally bit the bullet and just built the charts we wanted in D3 "from scratch". Once I wrapped my head around the model, it was such a pleasant experience. D3 is soooo nice and well thought out. You mayneed to level up in SVG, but it is so freeing to be able to bind data and display it basically any way you want. In hindsight, I wish we started with D3 from the beginning and I would not hesitate to pick it up for even simple charts again. It really is worth climbing the learning curve, which isn't as big and scary as it appears at first sight.
- EarthIsHome 8y ago
  How do you recommend leveling up in SVG?
  - kjeetgill 8y ago
    Thank this example: https://bl.ocks.org/mbostock/e3f4376d54e02d5d43ae32a7cf0e6aa9 https://bl.ocks.org/mbostock/e3f4376d54e02d5d43ae32a7cf0e6aa... ... svg.insert("g", "g") .attr("fill", "none") .attr("stroke", "steelblue") .attr("stroke-linejoin", "round") .selectAll("path") ... You might assume that g, fill, or stroke might be d3 terminology but it's really SVG! A quick Google of "svg g" will get you to https://developer.mozilla.org/en-US/docs/Web/SVG/Element/g https://developer.mozilla.org/en-US/docs/Web/SVG/Element/g It blew my mind open, hope to share the same.
    - staticautomatic 8y ago
      To my untrained eye that looks like css had a baby with js and the baby is an svg.
      - klodolph 8y ago
        SVG is tied closely to CSS, so you can go in your CSS and create a style for your D3 graphs and do things like: .chartline { stroke-width: 3px; color: red; } D3 has some convenience functions in JS for creating SVG, but you don't have to use them, you could write SVG just like HTML, it will look more like this in the DOM: <g fill="none" stroke="steelblue" stroke-linejoin="round"> ... </g>
javajosh 8y ago
D3 is hard because it's not really a library. The claimed 'central abstraction', the stage (with data entering and exiting) is a trivial part of almost all D3 code! Almost all of the effort goes into 1) deciding when and how to invoke the D3 statements, 2) preparing the data, 3) event handling. Most of the code that comes with D3 has nothing to do with D3. If you look at a force-directed graph in D3, for example, you see how little the library gives you. It's up to you to decompose your data into nodes and edges, it's up to you to draw every shape, it's up to you to trigger a redraw using window.setInterval. And the examples are written in a dense functional, mutable style that uses all the advanced features of JS, SVG, and the DOM - and requires at least some understanding of academic visualisation jargon to boot. I believe that Bostock took the W3C seriously and wrote a toolkit that "goes with the grain of the web", preferring not to reinvent (or abstract) anything that was already widely available. It exposes the simple truth that the surface area of a modern, standards-compliant browser is enormous, and extraordinarily demanding on the mind of the programmer. And also, the dynamic, functional nature of JavaScript points programmers toward a style that is very much like that espoused by D3. And yet, I admire and use D3. In part, I think it's honest. If you program browsers for a living, you should master your dependencies! D3 really isn't there to hide the web from you, it entices you to learn the web that's already there. SVG is incredibly powerful and still underused by modern applications, IMHO. And no other library has the breadth and depth of layouts that D3 does, to my knowledge, and the layouts are the hard thing to program (especially performant layouts). So basically the price of admission is learning browsers really well (which takes years), and then you get access to all this sweet layout code for free! It's a good deal, if you ask me.
jschrf 8y ago
No mention of Vega in that article or this thread? It's a declarative grammar and toolkit built on D3 (and other viz libs). It's very approachable and addresses a lot of the issues raised in the article: https://vega.github.io/vega/about/vega-and-d3/ https://vega.github.io/vega/about/vega-and-d3/
- alexchantastic 8y ago
  Vega-Lite is even easier to get into as well. https://vega.github.io/vega-lite/ https://vega.github.io/vega-lite/
- viraptor 8y ago
  Agree. It's much easier to pick up. Maybe not much easier to debug what's going wrong (but D3 doesn't make it easy either), but at least the ways things can go wrong are very limited. I picked up D3 and after a few days got to the "all I wanted was some graphs, not a graph framework factory" point. Vega is great for this.
symlinkk 8y ago
Libraries typically range from being easy to use but inflexible, or hard to use but flexible. D3 is firmly the latter.
Boygirl 8y ago
If you're working in React, check out Victory. It's based on D3, but it's intended to be much easier to get started with. There's a version for React Native too: https://github.com/FormidableLabs/victory https://github.com/FormidableLabs/victory https://github.com/FormidableLabs/victory-native https://github.com/FormidableLabs/victory-native
- thex10 8y ago
  I’m moderately experienced in d3. I remember trying to pick up Semiotic[0], a similar library, but eventually turning to Victory instead. I will probably stick with it in the future if I do more data visualizing in React. [0]: https://emeeks.github.io/semiotic/#/ https://emeeks.github.io/semiotic/#/
- Blackstone4 8y ago
  Looks like you're the main maintainer of Victory. I went from Recharts to Victory and back again. If anyone is looking for a React-based charts library check out Recharts at https://github.com/recharts/recharts https://github.com/recharts/recharts. It's fairly simple and in some cases, gives you the ability to overide functionality with your own functions/React components which is fairly sweet. So for instance, I have customised my tooltip.
  - WhitneyLand 8y ago
    I hate to see comments like yours go out without a couple concrete points that led you to your decision. Feedback like yours can be so valuable, especially with the maintainer of Victory here. There's just no way to replace the knowledge/lessons learned from real life developer user experience.
dsego 8y ago
Well D3 is I guess a DSL in disguise. Javascript is just the vessel here, so to speak. Similar to OpenGL with it's state machine or even worse GLSL with it's deceptively familiar syntax. Here are two tiny examples I've made with D3. I'd probably choose raw svg and a vdom lib next time, since I'm currently working on a project that mixes Mithril.js and SVG very nicely (although D3 or equivalent would be useful for mapping domains and scaling). https://htmlpreview.github.io/?https://github.com/dsego/dsego.github.io/blob/master/complex.html https://htmlpreview.github.io/?https://github.com/dsego/dseg... https://htmlpreview.github.io/?https://github.com/dsego/dsego.github.io/blob/master/polezero.html https://htmlpreview.github.io/?https://github.com/dsego/dseg...
- blt 8y ago
  This is the first time I've heard anyone complain about GLSL syntax. (I am only a dilettante in gfx.) What is the problem? It seems pretty well suited to the task in my opinion.
  - dsego 8y ago
    Not saying it's bad or not suited, but might be the “uncanny valley” situation for someone used to C/C++.
aphextron 8y ago
D3 is fantastic, but it should be treated essentially the same as OpenGL. Sure you could roll something super simple and quick with just that. But you're insane to do anything large scale without a higher abstraction.
- stephen_g 8y ago
  I don't think that's true at all... D3 is already a higher level abstraction to raw SVG+JS. It's fine for large scale things, as long as those things are actually the kind of things that make sense to implement with D3 and you know what you're doing.
capkutay 8y ago
I learned D3 in an academic context. In that environment it was great. When learning about the core theory of data visualization d3.js is an excellent abstraction and lets you make very deliberate decisions over core elements of a chart. Decisions that you can have an explanation for and defend. In a work context, D3.js is overkill most of the time. People expect line charts, maps, bubble charts that are simple and interactive...doing that in d3.js is a lot of copy/paste from examples and spending days to weeks working on simple details that would come OOTB in something like Highcharts. I would only recommend d3.js if you really want to dive deep into the constructs of the visualization. If you want a vanilla chart, there's plenty of high level open source libraries you can use.
cowpig 8y ago
I have a background in both web development and data science, and have used several different visualization libraries in python, R (ggplot), and JS. It still takes me a crazy amount of time to get a simple chart or graph working in D3 if it's anything more than copy/pasting an example. The last thing I did was an area graph that you could click and drag one of the edges and it would update the numbers in the legend as you did it. It took me 20-30 hours to get it working, and I'm sure I re-invented multiple things that D3 provides because it was easier to just write them from scratch than trying to find where Bostock hid it in the library and then decoding whatever crazy abstractions he came up with for it. I think he's bad at naming stuff and also bad at writing docs. By the time I get something working in D3, I've come close to the point where I could have just written all the abstractions myself. I can use a lightweight virtual DOM library and just write functions that take data and output DOM objects. D3 does barely more than that, at the cost of learning some really funky abstractions. I really wish there was a library out there like D3 that tried harder not to surprise me with every abstraction. But, I think the problem is just a hard one, because I'm not aware of anything that's much better :(
- tcgarvin 8y ago
  Longtime D3 (even Protovis) user here. I think you're very correct in that D3 itself is not a good choice for "a simple chart or graph". If that's all I needed, I'd stick with HighCharts, or ggplot, or any of dozens of other libraries that do simple charts well, and often with a lot more batteries included. My unsolicited advice would be to consider staying away from D3 until you need it. But if you're doing anything remotely bespoke, with any sort of custom interaction or transition, or anything in which you need to multiplex several datasets in a single set of visualizations, this toolkit is hard to beat. My favorite personal D3 use case so far is modified animated scatter plots/beeswarms using force layouts. Immortalized for me in the "Four Ways to Slice Obama’s 2013 Budget Proposal" ( https://archive.nytimes.com/www.nytimes.com/interactive/2012/02/13/us/politics/2013-budget-proposal-graphic.html https://archive.nytimes.com/www.nytimes.com/interactive/2012... ), I've found that set of techniques useful for visualizing business processes in my own work over the years. I've yet to encounter anything else that comes close to enabling those kinds of data visualizations. It's the "funky abstractions" some folks have trouble with that enable me to swap out X or Y scales with intuitive animations by literally replacing a single object and then re-invoking an idempotent render() function. Working with D3 is the most productive I've ever been in my life from a data visualization standpoint, and I suspect that will continue to be the case for some years.
injb 8y ago
I've just started learning D3, and my impression is that the biggest problem it has isn't really D3s problem at all, it's everyone else's. That is, every major single page app framework out there needs control over the DOM. I've seen a few articles talking about some pretty weird stuff you have to do to be able to use D3 and ReactJS in the same app. Hopefully it's not like that with all frameworks, but I'd really like to see more info written about this.
- deleted 8y ago
  [deleted]
- TN1ck 8y ago
  I've built a lot of charts/visualization with React + D3. I normally use D3 only for the math as rendering with React is much faster, easier to reason about and allows for features like server side rendering. Animations/transitions are not a solved problem with React, which is why it's probably far easier to create an animated chart with D3 exclusively, as the render loop of D3 with its enter/update/exit is really powerful.
  - thibautg 8y ago
    Nowadays I’m building a lot of visualisations with a Vue.js SVG template and d3 for its helper functions like scale, axis, path, colormaps etc. It’s really powerful and declarative and no need for enter/update/exit as everything is kept in sync by Vue data bindings. However the enter/update/exit pattern stays indeed better for animations.
jancsika 8y ago
Chromium has already implemented part of the SVG 2 spec that gives you CSS properties for most SVG attributes (not just the presentation attributes). This makes it way easier to manipulate things directly, as you can set "width" and "height" props for a <rect> and it will display the way you always thought it should. Additionally, you can animate those props using the Web Animations API. Something like this is quite nice: ``` myRect.animate( { "y": [0, 20, 0] }, { duration: 1000, easing: "ease-in-out" } ); ``` There's a polyfill for the burgeoning Web Animations API. Once Edge and FF implement these CSS props for SVG, it should be possible to seriously suggest doing at least some of the more straightforward D3 demos in plain JS.
- WhitneyLand 8y ago
  The production implementation is still way incomplete unfortunately. But your point is well taken and getting going with polyfills where possible is a great idea. Things will be much nicer when this all gets filled out and consistent across engines.
sharpercoder 8y ago
Most likely, D3 should design a dsl that compiles to d3. This language should cover 80% of the usecases and vastly improve the debugging experience.
j2kun 8y ago
My problem with d3 is that it's a very leaky abstraction, and there's no clarification about what, when, and where things must happen, versus can happen, versus must not happen. A core example is data joining. If you don't include .merge in just the right place, everything breaks with no warning or errors. A better design would eschew the implicit "state" of a selection (entering, updating, or exiting) in favor of providing three distinct, named handlers for the three stages. The result of things like this is that d3 encourages mindless copy/paste, and you either have no idea how it really works, or else you've scoured the d3 source to understand what's going on (which then requires expert level of javascript knowledge).
- BigJono 8y ago
  I mean, you're always going to run into this problem with this kind of library. Building a big, complex data visualisation pretty much requires expert knowledge in whatever language/environment you're doing it in. Law of Leaky Abstractions and all that. The purpose of something like d3 isn't to hide everything from a newbie, it's to make the code a bit cleaner and save the expert developer a bit of time. I think beginner programmers in general expect too much from libraries.
  - j2kun 8y ago
    There are obvious, concrete ways to fix the API that would reduce the problems, for beginners and experts alike.
- mr_toad 8y ago
  I blame a lot of the tutorials for the ‘mindless’ copy and paste. Short blog style articles are not the best way to learn D3. You really need to sit down for a few days and work hard to understand it.
  - thebooglebooski 8y ago
    I'd note the official documentation isn't intuitive either. Which lends itself to all of the tutorials that seem to promote mindless copy paste, because authors would rather paste their code, than explain why parts of the API are strewn together in a particular way. I spent the last week trying to understand the brush and zoom APIs, and everyone from stack overflow to medium to bl.ocks just seems to push a style of learning that lends itself to "just look at this example, and figure it out from there." So while sort blog articles are not the best way to learn D3, I think a better job could be done in illustrating the mechanics, models and use cases through geometry, and explaining what subset of a particular API are absolutely required to achieve a goal.
curiousgal 8y ago
Ah, D3, the Assembly of Data Visualisation.
imhelpingu 8y ago
If you want to learn the trouble with d3, try to apply their general pattern to anything with multiple grouped elements and watch all of your code devolve into forEach loops, or even better, go look up their recommended approach for using it with canvas. It's great for a demo about a chart, though!
ridiculous_fish 8y ago
> and quantum physics Hmm. I built the first version of my quantum wavefunction visualizer with D3, since D3 has a reputation for being good at visualization. However I eventually dropped D3 for WebGL-backed three.js. One problem is much of the SVG spec is just not implemented. This means it is less capable than it appears at first. 3D in particular was very awkward. Performance was also a problem. D3 round trips through text, making it hard to achieve fast custom animations. A big performance win was to truncate floats to two decimal places, which is an absurd thing to need to do. three.js had its own issues, in particular drawing lines which is quite challenging. However it was more flexible and faster. http://ridiculousfish.com/wavefiz/ http://ridiculousfish.com/wavefiz/ if anyone wants to see.
- bordercases 8y ago
  Any other D3 performance hacks?
- jacobolus 8y ago
  > D3 round trips through text, making it hard to achieve fast custom animations. Isn’t this just an inherent problem with using SVG (or DOM in general)? Conceivably you could use D3 to drive some other drawing method, though you might not get that much out of it at that point, unless you were trying to draw maps or something.
  - ridiculous_fish 8y ago
    I think the answer is no; for example one can use document.createElement(). I found that D3 was doing the equivalent of setting innerHTML.
    - jacobolus 8y ago
      Not sure about current browsers, but a few years ago setting innerHTML was significantly faster than the other DOM APIs. But in any event SVG path strings (and other element attributes) are inherently strings, there’s nothing you can do to get around that.
- 8y ago
tchaffee 8y ago
For those asking about how to dig in and learn, I learned D3 basics from the Data Visualization section of https://freecodecamp.org https://freecodecamp.org. I thought it was going to be very hard. I was pleasantly surprised that the basics aren't actually that difficult at all.
franciscop 8y ago
This is exactly what I do when some of my friends say "I wanna learn programming". The most important bit is to find out why, and specify a path so they can thrive. Depending on their background (as in the article) and specially their objectives I would recommend quite different things. HTML+CSS is the basic for ~80% of my recommendations so far, but some times I've recommended straight Javascript, python or even C++ (Arduino). Heck, a couple of times I've recommended some people NOT to learn programming. Wanna start a business by next week? Test ideas with pre-made platforms and CMS. Read about startups, product-market fit, etc. If you are starting a business expect not to have time for learning programming from scratch. I wish I saw that last bit explained in the article and I feel it's a big omission. Some times NOT learning a tool/paradigm _is_ the right choice. If you just want to show a simple graph by tomorrow, don't learn D3.js, use any of the existing charting libraries.
- arianvanp 8y ago
  I usually end op recommending people Excel
- tvanantwerp 8y ago
  As a self-taught programmer, D3 was actually the first major library I learned because I wanted to make lots of custom interactive data visualizations. (And yes, it was hard!) Like you say, it all depends what you're trying to do.
burntrelish1273 8y ago
The wrapper http://c3js.org http://c3js.org any good?
moron4hire 8y ago
I've always been confused about how people could consider D3 to be a charting library. It provides nothing that is specific to drawing any particular chart. What it provides is strictly DOM manipulation. If you don't know how to write the HTML or SVG necessary to make the chart you want, D3 isn't going to do anything for you. Now, as a DOM manip lib, I certainly like it a lot better than something like jQuery, but some of the newer offerings like React are a little more intentionally designed and don't have as many anachronistic design choices. D3 is just too much its own thing. There aren't many other JS libraries that follow the same DSL pattern that they've developed for it, so it can take quite a lot of ramp up time to get people used to it.
WhitneyLand 8y ago
There's no misconception about usability. If something is harder to learn for intended audiences on average for common scenarios it just has a steeper learning curve and no amount of hand waving can change that. It wasn't any picnic for me and I wasn't exactly new to any of the concepts. It's true learning curve is different that productivity. Once you scale the mountain, if it's enough better, it's possible to be more productive in the long run. But this comes with a huge caveat - that there's not another way to design a system to get the same benefits in a more approachable way with a faster ramp up. Do a lot of people really believe that? Even if there could be a better design it doesn't mean people shouldn't use D3. People use the best solution available however imperfect. It's a bit ironic that the word complex is used below in describing how it's essentially not as complex as people think. "The misconception...is that d3 and data visualization are the same thing. That to do data visualization one must learn and use all of d3...I like to emphasize that d3 is a toolkit for communicating complex data-driven concepts..."
quickthrower2 8y ago
SVG is a standard that has spanned millennia. So learning SVG, although yet another yak to shave, is a bald yak that'll serve you well.
MitjaBezensek 8y ago
D3 is a joy to use and one of the reasons we've been able to iterate on our product so fast. Once you get over the initial learning curve you can build things really fast and the actual charting part ends up being the easiest to do in the whole process. Processing data, handing interactions end up taking most of your time (I'm speaking this as a developer working on a visualization product https://zebrabi.com/pbi/ https://zebrabi.com/pbi/).
- arendtio 8y ago
  Are you aware, that the first large chart doesn't load in FF 60 (Linux)? SecurityError: Permission denied to access property "localizedStrings" on cross-origin object
  - MitjaBezensek 8y ago
    Thanks for letting me know, we'll take a look at what might be causing this. I'm on FF 61 on Windows and it seems to work fine, same with FF 59 on Ubuntu.
ygra 8y ago
We frequently get questions from potential customers why they should choose our library instead of D3 and where are the differences. After all, D3 can display and layout graphs just as well. Yes, D3 can do pretty much everything, but more by virtue of JavaScript being able to do pretty much everything and having to do everything yourself. By itself it's not that much more than a convenient way of defining patterns how to turn data into DOM objects. Adding interactivity is then a matter of adding event handlers to those, but certain common use cases, say, moving the viewport, moving nodes, and drawing edges, can be tricky to get right for every edge (pun not intended) case and it can be easier to just use something ready-made.
zongitsrinzler 8y ago
D3 is a low level and powerful framework, this is fine. If you want to show a simple line or bar chart just use one of many frameworks built on top of D3, such as http://c3js.org/ http://c3js.org/.
reacharavindh 8y ago
I'm a sysadmin, and I wanted some D3 like charts for our performance data. I'm not interested in learning JavaScript & DOM & other necessities it appeared necessary to me to make use of D3 to fill my relatively simple needs. I stumbled on C3.js which is an abstraction over D3 for re-usable charts built atop D3. Never looked back. If anybody is in similar situation, please take a look. http://c3js.org http://c3js.org
ttflee 8y ago
From a developer’s perspective, d3 makes so much use of type coercion that it is too clever for a non-savvy js/webdev worker to reason about.
dekhn 8y ago
I've been doing web stuff and data visualization for 30 years. D3 is painful to use. I know some people love it, but I've never used a system that was harder to configure to do anything basic and simple. I use matplotlib with the imperative API and while it's not great, at least it does what I tell it to do.
sevensor 8y ago
The trouble with D3 is the banner right at the top of the homepage, full of beautiful visuals. Beauty is great, but clarity has to come first. And clarity is partly driven by what the audience is familiar with. For most audiences, scatter plot, line plot, bar chart, and maybe parallel coordinate plot. Possibly a map as well, although plots on top of maps are famously easy to misinterpret if you don't know what you're doing when you make them. Engineering disciplines have their own specialized visualizations like Smith Charts and carpet plots. The complexity of designing a plot comes with how you compose together the basic elements. The consumer of the plot will want insets or annotations, two plots overlaid or juxtaposed. Error bars. It's very easy to get overwhelmed if your starting point is trying to design beautiful visuals. D3 will let you build all the needed complexity, but if you start by copying a beautiful plot, you'll soon find yourself overwhelmed, unable to deal with beauty and complexity at the same time, struggling to reconcile new functionality with the initial design. My advice is to make the simplest possible plot of your data (one of the basic types, monochrome if you can get away with it), and figure out whether you need to add anything once you and your customer have tried to work with it.
btbuildem 8y ago
Consider D3 to be the "C++ of data visualization" - it's flexible, powerful, and low-level enough that you can blow off both your feet if you're not careful. The real problem is this: data visualization is hard. UX is hard. I feel like a lot of people are looking for some magic-bullet framework that will let them achieve complex, highly-polished results without having to learn something. Guess what -- there's no shortcut. You have to make an effort, learn, and practice. Most people will produce mediocre things at best, and most will give up.
- WhitneyLand 8y ago
  100% not true. Concepts and tools are not just hard with no shortcut. They can be hard++ or hard-- (forgive the pun on your C++ reference) depending on how the concepts are delivered and how the tools are designed and implemented. Yes, viz can't become basket weaving. But there is always a spectrum on which things can be made more or less difficult. Didn't some of your professors make subjects seem beautiful, elegant, easy to grok, while some of them seemed to make subjects no more inherently difficult inscrutable?
_coveredInBees 8y ago
Everyone shitting on D3 here should stop and consider that there must be some fundamental reason why a library like D3.js has withstood the test of time in the JS landscape, when most libraries go out of fashion within months. Most of the comments here complaining about D3 boil down to the following: - Using the wrong tool for the job... if all you want is off-the-shelf visualizations, use one of the many visualization libraries out there that utilize d3.js on the back-end - Not wanting to put in the effort to learn the basics of SVG, CSS and the D3 API, and yet expecting hand-holding and everything to just be easy The beauty of D3 is how powerful it is and how unopinionated it is. You can literally make any type of visualization with it, and it has some astoundingly useful helper libraries for data binding, data manipulation, color handling, etc. As someone who picked up D3.js from scratch, I learned it by having clear goals regarding what I wanted to achieve. Having built charts / visualizations that are used in several products, it became very clear, very early on, that unless your customer/product requirements are super light-weight and run-of-the-mill visualizations are okay, you are bound to get to a point sooner or later where a visualization library isn't going to cut it. At that point, you're either going to spend a ton of time trying to shoe-horn a feature into an existing library that isn't made to be flexible, or you could have just saved yourself a whole bunch of effort by having gone the route of writing your visualizations in D3 from the get-go. D3 absolutely has a place in the data visualization space. You just need to be aware that it may not be the right tool for you, and if you do need to wield it, you need to put in the effort to actually familiarize yourself with the D3 related landscape rather than expecting to be able to simply copy-paste examples and hack your way to a working solution. PS - I should mention, that I'm an ML Engineer / Data Scientist, not a data viz geek, so it's not like D3 is relegated to people who do data viz for a living. There are plenty of folks who have picked up D3 on the side and done amazing things with it.
- Silhouette 8y ago
  Not wanting to put in the effort to learn the basics of SVG, CSS and the D3 API, and yet expecting hand-holding and everything to just be easy Sadly, this problem has become endemic within the web development community in recent years. People are more concerned with learning the latest buzzword framework than the fundamentals. The article itself quotes this tweet early on: The trouble with D3 is to build a visualization you must also have a deep understanding of SVG, DOM, JavaScript, geometry, color spaces, data structures, the standard model, and quantum physics — @seecmb Well, yes. Designing and implementing good visualisations is a skill. It has many aspects, and they aren't trivial. If you want to draw a standard, no-frills, minimally-customised chart then there are many higher-level libraries that will draw one for you and you're not the target audience for D3. If you want to have more customised and creative visualisations and interactions, you have to customise and create. That requires understanding the tools available and knowing what you're trying to achieve with them, but if you are willing to put the work in to reach that point, D3 offers a useful and powerful toolbox.
metakermit 8y ago
If you're looking for a good high-level abstraction, I really recommend the open source Plotly.js library: https://plot.ly/javascript/ https://plot.ly/javascript/ (similar to HighCharts, but without the commercial license mumbo-jumbo) Yeah, I see D3 itself as more of a tool tailored towards DataVis geeks developing new visualisation methods.
- daemin 8y ago
  Tried it recently but it has some flaws, like locking up the browser when viewing ~5k values in a 3d scatter plot when viewed from certain angles.
WhitneyLand 8y ago
Working hard to learn, is not the same thing as how hard learning has to be. Conflating these things is the biggest theme I see in the comments. I risk ranting here, because it seems like this conflation is a recurring misconception in our community. Maybe because most of us don't get to spend equal time thinking about design and usability in our specialties? For the benefit of all the projects we both design and use, we must consider these as two important, yet separate concepts: 1) Any non-trivial concept, tool, or process requires serious investment to be productive with. We should expect it, not complain about it, and just roll up our sleeves if it's really the right tool for us and for the job. 2) We should never let any tooling or library off the usability or productivity hook because it's non-trivial. All designs/implementations do not have to be equal in this regard and are usually not. Yes, we should be willing to invest our time and build the proper foundational knowledge. We should also always challenge how things can be made more productive, more approachable, and as simple as possible but no simpler. The guy to first think of that phrase worked with some pretty damn non-trivial concepts.
sh87 8y ago
> The common theme was that they had started with goals. They learned what they needed from d3 along the way to achieving those goals. This is my key takeaway that fully resonates with my way of learning anything related to JS be it D3, Angular, VueJS, Node, Android and more recently React. Take a problem, solve it in the tech of your choice.

350 points by danso 8y ago

I think one of the overarching factors is that those new to data visualization (and web development) see D3's beautiful examples and assume that the D3 library is directly and fundamentally responsible for the visual quality. I've seen this a bit with R's ggplot2 as well among relatively experienced data scientists. To be sure, ggplot2 is a fantastic library and the fact that its defaults look great are a testament to the library's well-thought out design. But such design grace can be achieved in any library (e.g. matplotlib), albeit with less graceful APIs.

bunderbunder 8y ago
You can absolutely get there. But the trouble with most libraries I've used is that they are either limiting or confusing. For example, Seaborn is very easy to learn, but it's mostly just a set of canned visualizations, with limited flexibility. Matplotlib is very flexible, but just. so. hard. to. learn. I had a machine learning class in grad school where most everyone agreed the single most difficult homework was one that asked us to use Matplotlib to generate a few visualizations of what the models were doing. Comparatively speaking, learning ggplot2 is a sublime experience. After about a week, you feel like you have super powers. D3 does feel overcomplicated for just building static visualizations, but if you're building custom interactive stuff, learning it can be nearly as joyous an experience.
- _Wintermute 8y ago
  Regarding ggplot, matplotlib and seaborn I found the opposite. If you want some pre-canned visualisation then ggplot2 is fantastic, but as soon as you stray away from the examples it turns into a mess, you can't just drop down into base graphics but have to mess about with grob and grid. Whereas seaborn has similar pre-canned plots, but it's all built on matplotlib so you still have that flexibility if you want it.
cup-of-tea 8y ago
Yeah. If you come to d3 but you've never made a beautiful image in SVG before you're going to have a bad time. You have to be good at SVG, CSS and javascript already. D3 will not help.

ribhu97 8y ago

C3 is a javascript library based on D3 that simplifies it and makes it easier to use for designers and full stack developers.

sakopov 8y ago
C3 looked like a dead project about a year or so ago, which I thought was a real shame and planned to move away from it. Just checked the repo a few minutes ago and it looks like things are pretty lively these days. Glad to see it's kicking along again.
FLUX-YOU 8y ago
We've been using this. It's really nice and straight-forward, although it has some edge cases when you are doing small graphs and responsive layouts. Using it with Angular also adds some complexity.

spenrose 8y ago

Very popular simple time series library built on top of d3.js: https://metricsgraphicsjs.org https://metricsgraphicsjs.org.

jbob2000 8y ago

Ehh, that was kind of a thin article that didn’t quite scratch the itch on why D3 is troublesome. First, I think if you want to use d3, you probably just want a charting library. Convince yourself that HighCharts is not flexible enough and then try d3. So the trouble with d3 is that most people don’t need to use it and are in over their heads. Second, d3 is full of leaky abstractions. Imagine if cars didn’t have acceleration pedals, and you had to manually open the throttle and engage the fuel pump yourself. You’d need an engineering degree to drive your car! Same thing with d3; it’s ridiculous that you need to know the ins and outs of some of the internet’s most complicated and frustrating technology just to show a couple of data points on a graph. It’d be great if you could work with d3 and pretend like SVG and the DOM didn’t exist.

fein 8y ago
nvd3 if you want d3 charts without the d3 hassle: http://nvd3.org/ http://nvd3.org/

jdonaldson 8y ago

There's a place for cookie cutter visualizations. However, you're never going to make anything distinctive or memorable that way. All of it is a trade off... Are you trying to fit in or stand out?

sydd 8y ago

Dunno for me D3 feels like they are trying to do everything at once resulting in an overly complex and hard-to-learn library. When I look at the examples at their page I see stuff that I could make in 2 ways: - Just write a small program from scratch that visualizes one in a week or 2. It will be small, I will know the ins and outs, so easy to change and will be a ton of fun to write. - Use D3.js (I dont know this library, but i've used complex charting libraries in the past). Likely it will take more time, not because I am thinking of the interesting parts, I will spend most of my day cursing on StackOverFlow to figure out how to skin that one tricky part of my app. So in the end I dont really see a good reason to use d3.js unless my full time job is to develop data visualizations.

wpietri 8y ago
I learned (some of) D3 in exactly the fashion the article suggested: find something I want to visualize and make it happen. I thought it was great. I still wouldn't say I've "learned D3", but I didn't have to. I got my goal accomplished in a couple of days, along with a lot of nice bits I didn't intend along the way as I found good things that D3 could do for me. For me it was definitely a "design to tools" approach; I looked at the gallery, found something mostly like what I wanted, and then worked to that. That was fine by me; I wasn't looking for ultimate visual control, but just trying to turn a bunch of data into something mortals could comprehend.

hacker_9 8y ago

What's even more confusing is that v3 -> v4 includes a bunch of breaking changes, and yet they didn't update their typescript definition file. Not to mention the web is full of v3 examples that mysteriously don't work with your d3 lib (and versions are never mentioned anywhere).

symlinkk 8y ago
I've had no problem with TypeScript and D3. I'm using `@types/d3`, is that what you're using?
mschuster91 8y ago
> Not to mention the web is full of v3 examples that mysteriously don't work with your d3 lib Unfortunately this is widespread across the JS ecosystem. Hell I'm not saying anyone should do it like Linux (break userspace and get shot), but when one does backwards-incompatible changes, is it so hard to stub dead methods or unsupported calling ways with "console.error('Outdated, see https://github.com/x/y/issues/z');" https://github.com/x/y/issues/z');"?

stared 8y ago

Side note (on D3.js) - there is a nice exploration of reusable D3.js charts/components: https://d3-discovery.net/ https://d3-discovery.net/ ("Finding D3 plugins with ease"). It's source (Vue-based, potentially for other showcases/searches): https://github.com/wbkd/d3-discovery https://github.com/wbkd/d3-discovery (I just asked if it can be open source, and to my awe its author added MIT license :)).

k__ 8y ago

D3 is for high performance custom data visalizations. For example, I built a in-door navigation app. It would receive its data via MQTT/WebSockets and update markers on custom maps. D3 was perfect for this, it helped with synchronizing graphics with data, converting mapping cooridnates to DOM/SVG coordinates and easing all the updates so stuff looked smooth. You just would pump the incoming data into the D3 pipeline and be done with it. If you just need a few animations, there are a bunch of simple libraries for this. React has react-motion for example, it's not as flexible as D3 but you get most stuff you would expect out of it without much hassle. If you just need a set of basic charts, use a charting library. Highcharts or Charts.js are both very capable.

bigger_cheese 8y ago
My work has been pretty interested in "interactive" charts, being able to modify axis scales on the fly, change filter range, dynamically add and remove series that sort of thing. We use SAS (and a little bit of R internally) and those both have good charting capabilities but trying to do anything interactive in either of those packages is a real pain. In the past we have used a C++ library SL-GMS. Even when compared with using C++ I have found using JavaScript based libraries difficult D3 in particular I found to be horrendously complex.

blattimwind 8y ago

I do quite a bit of woodworking and use a lot of handtools, yet a lot of the tools on that wall look completely alien to me. I think this wall might actually belong to an upholsterer or something like that... Edit: The title of the stock photo is "Timmermansgereedschap" (NL) = Zimmermannsgerätschaften (DE) = carpenter's tools (EN).

duckfruit 8y ago

The most salient point that the article breaches is this: When people unfamiliar with the DOM, javascript, or indeed in some cases programming itself try to 'pick up' a complex low level data visualization library like D3, it is natural for them to blame the library. But the problem obviously runs much deeper than that. Running with the analogy in the article -- it is fruitless for someone who's never done any woodworking or furniture design ever to hope to become the next Charles Eames within the span of a few months.

CJefferson 8y ago
However, d3 seems particularly bizarre. Its the only library i ever remember trying hard to learn, and just giving up. There is too little indication, when you misuse it, of what went wrong. Ive done a bunch of javascript, DOM and visualisations in OpenGL, so i feel i should have a good chance.
- GijsjanB 8y ago
  Complexity comes at a price. Making it easy, abstracts certain low level functionality away. Compare Rust to Python. Compare OpenLayers to Leaflet. Compare the dashboard of an Apache to your car's dashboard. Rust, OpenLayers en Apaches aren't bizarre, they have complex functionality which you don't want to abstract away, because you want the ability to use it on that low level.
- kjeetgill 8y ago
  I don't know if this was your issue, but discovering that most of the API was accessors to SVG opened the floodgates for me. That said, I mostly learned it through the examples rather than a tutorial.
- bunderbunder 8y ago
  A lot of D3's oddness comes from the fact that there isn't anything (to my knowledge) that is comparable. It does just one thing - binding data to DOM elements - does it well, and then gets out of the way.

stared 8y ago

> The trouble with D3 is to build a visualization you must also have a deep understanding of SVG, DOM, JavaScript, geometry, color spaces, data structures, the standard model, and quantum physics. A piece of cake! Here is a quantum game in D3.js: http://quantumgame.io/ http://quantumgame.io/ :) (full disclaimer: mine, and I already do too much of its shameless self-promotion)

forrestthewoods 8y ago

It’s honestly easier to just use VanillaJS.

stupidcar 8y ago

Visualisation is fundamentally the process of creating a mapping function between the dimensions of your domain data and that of the "screen" (whether it's DOM, SVG, Canvas, etc.). D3 is just a library of higher-order functions that can help you produce that mapping function. If you can grasp this idea of visualisation as creating a mapping function, then D3 becomes a beautiful toolbox for assisting you. You decide which dimension of your data you want to represent as the horizontal dimension, which as the vertical, as color, as size, etc. You decide how you want to scale them. Then you use D3's high-order functions to create the appropriate individual mapping functions, compose them into your complete visualisation function, and pass it your domain data. If you think of D3 as a "charting library", or a collection of visualisation classes or objects, then you're going to struggle. But the problem isn't with D3, it's that you haven't really understood the paradigm of the library.

rpowers 8y ago

Anecdotal, but my problem with D3 is the need for so many function calls. 'append', 'attr', 'enter', 'datum', ..... Code ends up looking like a centipede ().().().()

andrewmcwatters 8y ago
Code formatting seems to be an easily dismissed part of software development: you should be breaking those out on newlines.

mhale 8y ago

I worked on a project that required some pretty standard line charts, but with the addition of an overlaid set of horizontal "benchmark" lines -- and some callouts. After struggling (and failing) to get what we wanted out of two different off-the-shelf graphing libraries (one of which was built on top of D3), we finally bit the bullet and just built the charts we wanted in D3 "from scratch". Once I wrapped my head around the model, it was such a pleasant experience. D3 is soooo nice and well thought out. You mayneed to level up in SVG, but it is so freeing to be able to bind data and display it basically any way you want. In hindsight, I wish we started with D3 from the beginning and I would not hesitate to pick it up for even simple charts again. It really is worth climbing the learning curve, which isn't as big and scary as it appears at first sight.

EarthIsHome 8y ago
How do you recommend leveling up in SVG?
- kjeetgill 8y ago
  Thank this example: https://bl.ocks.org/mbostock/e3f4376d54e02d5d43ae32a7cf0e6aa9 https://bl.ocks.org/mbostock/e3f4376d54e02d5d43ae32a7cf0e6aa... ... svg.insert("g", "g") .attr("fill", "none") .attr("stroke", "steelblue") .attr("stroke-linejoin", "round") .selectAll("path") ... You might assume that g, fill, or stroke might be d3 terminology but it's really SVG! A quick Google of "svg g" will get you to https://developer.mozilla.org/en-US/docs/Web/SVG/Element/g https://developer.mozilla.org/en-US/docs/Web/SVG/Element/g It blew my mind open, hope to share the same.
  - staticautomatic 8y ago
    To my untrained eye that looks like css had a baby with js and the baby is an svg.
    - klodolph 8y ago
      SVG is tied closely to CSS, so you can go in your CSS and create a style for your D3 graphs and do things like: .chartline { stroke-width: 3px; color: red; } D3 has some convenience functions in JS for creating SVG, but you don't have to use them, you could write SVG just like HTML, it will look more like this in the DOM: <g fill="none" stroke="steelblue" stroke-linejoin="round"> ... </g>

javajosh 8y ago

D3 is hard because it's not really a library. The claimed 'central abstraction', the stage (with data entering and exiting) is a trivial part of almost all D3 code! Almost all of the effort goes into 1) deciding when and how to invoke the D3 statements, 2) preparing the data, 3) event handling. Most of the code that comes with D3 has nothing to do with D3. If you look at a force-directed graph in D3, for example, you see how little the library gives you. It's up to you to decompose your data into nodes and edges, it's up to you to draw every shape, it's up to you to trigger a redraw using window.setInterval. And the examples are written in a dense functional, mutable style that uses all the advanced features of JS, SVG, and the DOM - and requires at least some understanding of academic visualisation jargon to boot. I believe that Bostock took the W3C seriously and wrote a toolkit that "goes with the grain of the web", preferring not to reinvent (or abstract) anything that was already widely available. It exposes the simple truth that the surface area of a modern, standards-compliant browser is enormous, and extraordinarily demanding on the mind of the programmer. And also, the dynamic, functional nature of JavaScript points programmers toward a style that is very much like that espoused by D3. And yet, I admire and use D3. In part, I think it's honest. If you program browsers for a living, you should master your dependencies! D3 really isn't there to hide the web from you, it entices you to learn the web that's already there. SVG is incredibly powerful and still underused by modern applications, IMHO. And no other library has the breadth and depth of layouts that D3 does, to my knowledge, and the layouts are the hard thing to program (especially performant layouts). So basically the price of admission is learning browsers really well (which takes years), and then you get access to all this sweet layout code for free! It's a good deal, if you ask me.

jschrf 8y ago

No mention of Vega in that article or this thread? It's a declarative grammar and toolkit built on D3 (and other viz libs). It's very approachable and addresses a lot of the issues raised in the article: https://vega.github.io/vega/about/vega-and-d3/ https://vega.github.io/vega/about/vega-and-d3/

alexchantastic 8y ago
Vega-Lite is even easier to get into as well. https://vega.github.io/vega-lite/ https://vega.github.io/vega-lite/
viraptor 8y ago
Agree. It's much easier to pick up. Maybe not much easier to debug what's going wrong (but D3 doesn't make it easy either), but at least the ways things can go wrong are very limited. I picked up D3 and after a few days got to the "all I wanted was some graphs, not a graph framework factory" point. Vega is great for this.

symlinkk 8y ago

Libraries typically range from being easy to use but inflexible, or hard to use but flexible. D3 is firmly the latter.

Boygirl 8y ago

If you're working in React, check out Victory. It's based on D3, but it's intended to be much easier to get started with. There's a version for React Native too: https://github.com/FormidableLabs/victory https://github.com/FormidableLabs/victory https://github.com/FormidableLabs/victory-native https://github.com/FormidableLabs/victory-native

thex10 8y ago
I’m moderately experienced in d3. I remember trying to pick up Semiotic[0], a similar library, but eventually turning to Victory instead. I will probably stick with it in the future if I do more data visualizing in React. [0]: https://emeeks.github.io/semiotic/#/ https://emeeks.github.io/semiotic/#/
Blackstone4 8y ago
Looks like you're the main maintainer of Victory. I went from Recharts to Victory and back again. If anyone is looking for a React-based charts library check out Recharts at https://github.com/recharts/recharts https://github.com/recharts/recharts. It's fairly simple and in some cases, gives you the ability to overide functionality with your own functions/React components which is fairly sweet. So for instance, I have customised my tooltip.
- WhitneyLand 8y ago
  I hate to see comments like yours go out without a couple concrete points that led you to your decision. Feedback like yours can be so valuable, especially with the maintainer of Victory here. There's just no way to replace the knowledge/lessons learned from real life developer user experience.

dsego 8y ago

Well D3 is I guess a DSL in disguise. Javascript is just the vessel here, so to speak. Similar to OpenGL with it's state machine or even worse GLSL with it's deceptively familiar syntax. Here are two tiny examples I've made with D3. I'd probably choose raw svg and a vdom lib next time, since I'm currently working on a project that mixes Mithril.js and SVG very nicely (although D3 or equivalent would be useful for mapping domains and scaling). https://htmlpreview.github.io/?https://github.com/dsego/dsego.github.io/blob/master/complex.html https://htmlpreview.github.io/?https://github.com/dsego/dseg... https://htmlpreview.github.io/?https://github.com/dsego/dsego.github.io/blob/master/polezero.html https://htmlpreview.github.io/?https://github.com/dsego/dseg...

blt 8y ago
This is the first time I've heard anyone complain about GLSL syntax. (I am only a dilettante in gfx.) What is the problem? It seems pretty well suited to the task in my opinion.
- dsego 8y ago
  Not saying it's bad or not suited, but might be the “uncanny valley” situation for someone used to C/C++.

aphextron 8y ago

D3 is fantastic, but it should be treated essentially the same as OpenGL. Sure you could roll something super simple and quick with just that. But you're insane to do anything large scale without a higher abstraction.

stephen_g 8y ago
I don't think that's true at all... D3 is already a higher level abstraction to raw SVG+JS. It's fine for large scale things, as long as those things are actually the kind of things that make sense to implement with D3 and you know what you're doing.

capkutay 8y ago

I learned D3 in an academic context. In that environment it was great. When learning about the core theory of data visualization d3.js is an excellent abstraction and lets you make very deliberate decisions over core elements of a chart. Decisions that you can have an explanation for and defend. In a work context, D3.js is overkill most of the time. People expect line charts, maps, bubble charts that are simple and interactive...doing that in d3.js is a lot of copy/paste from examples and spending days to weeks working on simple details that would come OOTB in something like Highcharts. I would only recommend d3.js if you really want to dive deep into the constructs of the visualization. If you want a vanilla chart, there's plenty of high level open source libraries you can use.

cowpig 8y ago

I have a background in both web development and data science, and have used several different visualization libraries in python, R (ggplot), and JS. It still takes me a crazy amount of time to get a simple chart or graph working in D3 if it's anything more than copy/pasting an example. The last thing I did was an area graph that you could click and drag one of the edges and it would update the numbers in the legend as you did it. It took me 20-30 hours to get it working, and I'm sure I re-invented multiple things that D3 provides because it was easier to just write them from scratch than trying to find where Bostock hid it in the library and then decoding whatever crazy abstractions he came up with for it. I think he's bad at naming stuff and also bad at writing docs. By the time I get something working in D3, I've come close to the point where I could have just written all the abstractions myself. I can use a lightweight virtual DOM library and just write functions that take data and output DOM objects. D3 does barely more than that, at the cost of learning some really funky abstractions. I really wish there was a library out there like D3 that tried harder not to surprise me with every abstraction. But, I think the problem is just a hard one, because I'm not aware of anything that's much better :(

tcgarvin 8y ago
Longtime D3 (even Protovis) user here. I think you're very correct in that D3 itself is not a good choice for "a simple chart or graph". If that's all I needed, I'd stick with HighCharts, or ggplot, or any of dozens of other libraries that do simple charts well, and often with a lot more batteries included. My unsolicited advice would be to consider staying away from D3 until you need it. But if you're doing anything remotely bespoke, with any sort of custom interaction or transition, or anything in which you need to multiplex several datasets in a single set of visualizations, this toolkit is hard to beat. My favorite personal D3 use case so far is modified animated scatter plots/beeswarms using force layouts. Immortalized for me in the "Four Ways to Slice Obama’s 2013 Budget Proposal" ( https://archive.nytimes.com/www.nytimes.com/interactive/2012/02/13/us/politics/2013-budget-proposal-graphic.html https://archive.nytimes.com/www.nytimes.com/interactive/2012... ), I've found that set of techniques useful for visualizing business processes in my own work over the years. I've yet to encounter anything else that comes close to enabling those kinds of data visualizations. It's the "funky abstractions" some folks have trouble with that enable me to swap out X or Y scales with intuitive animations by literally replacing a single object and then re-invoking an idempotent render() function. Working with D3 is the most productive I've ever been in my life from a data visualization standpoint, and I suspect that will continue to be the case for some years.

injb 8y ago

I've just started learning D3, and my impression is that the biggest problem it has isn't really D3s problem at all, it's everyone else's. That is, every major single page app framework out there needs control over the DOM. I've seen a few articles talking about some pretty weird stuff you have to do to be able to use D3 and ReactJS in the same app. Hopefully it's not like that with all frameworks, but I'd really like to see more info written about this.

deleted 8y ago
[deleted]
TN1ck 8y ago
I've built a lot of charts/visualization with React + D3. I normally use D3 only for the math as rendering with React is much faster, easier to reason about and allows for features like server side rendering. Animations/transitions are not a solved problem with React, which is why it's probably far easier to create an animated chart with D3 exclusively, as the render loop of D3 with its enter/update/exit is really powerful.
- thibautg 8y ago
  Nowadays I’m building a lot of visualisations with a Vue.js SVG template and d3 for its helper functions like scale, axis, path, colormaps etc. It’s really powerful and declarative and no need for enter/update/exit as everything is kept in sync by Vue data bindings. However the enter/update/exit pattern stays indeed better for animations.

jancsika 8y ago

Chromium has already implemented part of the SVG 2 spec that gives you CSS properties for most SVG attributes (not just the presentation attributes). This makes it way easier to manipulate things directly, as you can set "width" and "height" props for a <rect> and it will display the way you always thought it should. Additionally, you can animate those props using the Web Animations API. Something like this is quite nice: ``` myRect.animate( { "y": [0, 20, 0] }, { duration: 1000, easing: "ease-in-out" } ); ``` There's a polyfill for the burgeoning Web Animations API. Once Edge and FF implement these CSS props for SVG, it should be possible to seriously suggest doing at least some of the more straightforward D3 demos in plain JS.

WhitneyLand 8y ago
The production implementation is still way incomplete unfortunately. But your point is well taken and getting going with polyfills where possible is a great idea. Things will be much nicer when this all gets filled out and consistent across engines.

sharpercoder 8y ago

Most likely, D3 should design a dsl that compiles to d3. This language should cover 80% of the usecases and vastly improve the debugging experience.

j2kun 8y ago

My problem with d3 is that it's a very leaky abstraction, and there's no clarification about what, when, and where things must happen, versus can happen, versus must not happen. A core example is data joining. If you don't include .merge in just the right place, everything breaks with no warning or errors. A better design would eschew the implicit "state" of a selection (entering, updating, or exiting) in favor of providing three distinct, named handlers for the three stages. The result of things like this is that d3 encourages mindless copy/paste, and you either have no idea how it really works, or else you've scoured the d3 source to understand what's going on (which then requires expert level of javascript knowledge).

BigJono 8y ago
I mean, you're always going to run into this problem with this kind of library. Building a big, complex data visualisation pretty much requires expert knowledge in whatever language/environment you're doing it in. Law of Leaky Abstractions and all that. The purpose of something like d3 isn't to hide everything from a newbie, it's to make the code a bit cleaner and save the expert developer a bit of time. I think beginner programmers in general expect too much from libraries.
- j2kun 8y ago
  There are obvious, concrete ways to fix the API that would reduce the problems, for beginners and experts alike.
mr_toad 8y ago
I blame a lot of the tutorials for the ‘mindless’ copy and paste. Short blog style articles are not the best way to learn D3. You really need to sit down for a few days and work hard to understand it.
- thebooglebooski 8y ago
  I'd note the official documentation isn't intuitive either. Which lends itself to all of the tutorials that seem to promote mindless copy paste, because authors would rather paste their code, than explain why parts of the API are strewn together in a particular way. I spent the last week trying to understand the brush and zoom APIs, and everyone from stack overflow to medium to bl.ocks just seems to push a style of learning that lends itself to "just look at this example, and figure it out from there." So while sort blog articles are not the best way to learn D3, I think a better job could be done in illustrating the mechanics, models and use cases through geometry, and explaining what subset of a particular API are absolutely required to achieve a goal.

curiousgal 8y ago

Ah, D3, the Assembly of Data Visualisation.

imhelpingu 8y ago

If you want to learn the trouble with d3, try to apply their general pattern to anything with multiple grouped elements and watch all of your code devolve into forEach loops, or even better, go look up their recommended approach for using it with canvas. It's great for a demo about a chart, though!

ridiculous_fish 8y ago

> and quantum physics Hmm. I built the first version of my quantum wavefunction visualizer with D3, since D3 has a reputation for being good at visualization. However I eventually dropped D3 for WebGL-backed three.js. One problem is much of the SVG spec is just not implemented. This means it is less capable than it appears at first. 3D in particular was very awkward. Performance was also a problem. D3 round trips through text, making it hard to achieve fast custom animations. A big performance win was to truncate floats to two decimal places, which is an absurd thing to need to do. three.js had its own issues, in particular drawing lines which is quite challenging. However it was more flexible and faster. http://ridiculousfish.com/wavefiz/ http://ridiculousfish.com/wavefiz/ if anyone wants to see.

bordercases 8y ago
Any other D3 performance hacks?
jacobolus 8y ago
> D3 round trips through text, making it hard to achieve fast custom animations. Isn’t this just an inherent problem with using SVG (or DOM in general)? Conceivably you could use D3 to drive some other drawing method, though you might not get that much out of it at that point, unless you were trying to draw maps or something.
- ridiculous_fish 8y ago
  I think the answer is no; for example one can use document.createElement(). I found that D3 was doing the equivalent of setting innerHTML.
  - jacobolus 8y ago
    Not sure about current browsers, but a few years ago setting innerHTML was significantly faster than the other DOM APIs. But in any event SVG path strings (and other element attributes) are inherently strings, there’s nothing you can do to get around that.
8y ago

tchaffee 8y ago

For those asking about how to dig in and learn, I learned D3 basics from the Data Visualization section of https://freecodecamp.org https://freecodecamp.org. I thought it was going to be very hard. I was pleasantly surprised that the basics aren't actually that difficult at all.

franciscop 8y ago

This is exactly what I do when some of my friends say "I wanna learn programming". The most important bit is to find out why, and specify a path so they can thrive. Depending on their background (as in the article) and specially their objectives I would recommend quite different things. HTML+CSS is the basic for ~80% of my recommendations so far, but some times I've recommended straight Javascript, python or even C++ (Arduino). Heck, a couple of times I've recommended some people NOT to learn programming. Wanna start a business by next week? Test ideas with pre-made platforms and CMS. Read about startups, product-market fit, etc. If you are starting a business expect not to have time for learning programming from scratch. I wish I saw that last bit explained in the article and I feel it's a big omission. Some times NOT learning a tool/paradigm _is_ the right choice. If you just want to show a simple graph by tomorrow, don't learn D3.js, use any of the existing charting libraries.

arianvanp 8y ago
I usually end op recommending people Excel
tvanantwerp 8y ago
As a self-taught programmer, D3 was actually the first major library I learned because I wanted to make lots of custom interactive data visualizations. (And yes, it was hard!) Like you say, it all depends what you're trying to do.

burntrelish1273 8y ago

The wrapper http://c3js.org http://c3js.org any good?

moron4hire 8y ago

I've always been confused about how people could consider D3 to be a charting library. It provides nothing that is specific to drawing any particular chart. What it provides is strictly DOM manipulation. If you don't know how to write the HTML or SVG necessary to make the chart you want, D3 isn't going to do anything for you. Now, as a DOM manip lib, I certainly like it a lot better than something like jQuery, but some of the newer offerings like React are a little more intentionally designed and don't have as many anachronistic design choices. D3 is just too much its own thing. There aren't many other JS libraries that follow the same DSL pattern that they've developed for it, so it can take quite a lot of ramp up time to get people used to it.

WhitneyLand 8y ago

There's no misconception about usability. If something is harder to learn for intended audiences on average for common scenarios it just has a steeper learning curve and no amount of hand waving can change that. It wasn't any picnic for me and I wasn't exactly new to any of the concepts. It's true learning curve is different that productivity. Once you scale the mountain, if it's enough better, it's possible to be more productive in the long run. But this comes with a huge caveat - that there's not another way to design a system to get the same benefits in a more approachable way with a faster ramp up. Do a lot of people really believe that? Even if there could be a better design it doesn't mean people shouldn't use D3. People use the best solution available however imperfect. It's a bit ironic that the word complex is used below in describing how it's essentially not as complex as people think. "The misconception...is that d3 and data visualization are the same thing. That to do data visualization one must learn and use all of d3...I like to emphasize that d3 is a toolkit for communicating complex data-driven concepts..."

quickthrower2 8y ago

SVG is a standard that has spanned millennia. So learning SVG, although yet another yak to shave, is a bald yak that'll serve you well.

MitjaBezensek 8y ago

D3 is a joy to use and one of the reasons we've been able to iterate on our product so fast. Once you get over the initial learning curve you can build things really fast and the actual charting part ends up being the easiest to do in the whole process. Processing data, handing interactions end up taking most of your time (I'm speaking this as a developer working on a visualization product https://zebrabi.com/pbi/ https://zebrabi.com/pbi/).

arendtio 8y ago
Are you aware, that the first large chart doesn't load in FF 60 (Linux)? SecurityError: Permission denied to access property "localizedStrings" on cross-origin object
- MitjaBezensek 8y ago
  Thanks for letting me know, we'll take a look at what might be causing this. I'm on FF 61 on Windows and it seems to work fine, same with FF 59 on Ubuntu.

ygra 8y ago

We frequently get questions from potential customers why they should choose our library instead of D3 and where are the differences. After all, D3 can display and layout graphs just as well. Yes, D3 can do pretty much everything, but more by virtue of JavaScript being able to do pretty much everything and having to do everything yourself. By itself it's not that much more than a convenient way of defining patterns how to turn data into DOM objects. Adding interactivity is then a matter of adding event handlers to those, but certain common use cases, say, moving the viewport, moving nodes, and drawing edges, can be tricky to get right for every edge (pun not intended) case and it can be easier to just use something ready-made.

zongitsrinzler 8y ago

D3 is a low level and powerful framework, this is fine. If you want to show a simple line or bar chart just use one of many frameworks built on top of D3, such as http://c3js.org/ http://c3js.org/.

reacharavindh 8y ago

I'm a sysadmin, and I wanted some D3 like charts for our performance data. I'm not interested in learning JavaScript & DOM & other necessities it appeared necessary to me to make use of D3 to fill my relatively simple needs. I stumbled on C3.js which is an abstraction over D3 for re-usable charts built atop D3. Never looked back. If anybody is in similar situation, please take a look. http://c3js.org http://c3js.org

ttflee 8y ago

From a developer’s perspective, d3 makes so much use of type coercion that it is too clever for a non-savvy js/webdev worker to reason about.

dekhn 8y ago

I've been doing web stuff and data visualization for 30 years. D3 is painful to use. I know some people love it, but I've never used a system that was harder to configure to do anything basic and simple. I use matplotlib with the imperative API and while it's not great, at least it does what I tell it to do.

sevensor 8y ago

The trouble with D3 is the banner right at the top of the homepage, full of beautiful visuals. Beauty is great, but clarity has to come first. And clarity is partly driven by what the audience is familiar with. For most audiences, scatter plot, line plot, bar chart, and maybe parallel coordinate plot. Possibly a map as well, although plots on top of maps are famously easy to misinterpret if you don't know what you're doing when you make them. Engineering disciplines have their own specialized visualizations like Smith Charts and carpet plots. The complexity of designing a plot comes with how you compose together the basic elements. The consumer of the plot will want insets or annotations, two plots overlaid or juxtaposed. Error bars. It's very easy to get overwhelmed if your starting point is trying to design beautiful visuals. D3 will let you build all the needed complexity, but if you start by copying a beautiful plot, you'll soon find yourself overwhelmed, unable to deal with beauty and complexity at the same time, struggling to reconcile new functionality with the initial design. My advice is to make the simplest possible plot of your data (one of the basic types, monochrome if you can get away with it), and figure out whether you need to add anything once you and your customer have tried to work with it.

btbuildem 8y ago

Consider D3 to be the "C++ of data visualization" - it's flexible, powerful, and low-level enough that you can blow off both your feet if you're not careful. The real problem is this: data visualization is hard. UX is hard. I feel like a lot of people are looking for some magic-bullet framework that will let them achieve complex, highly-polished results without having to learn something. Guess what -- there's no shortcut. You have to make an effort, learn, and practice. Most people will produce mediocre things at best, and most will give up.

WhitneyLand 8y ago
100% not true. Concepts and tools are not just hard with no shortcut. They can be hard++ or hard-- (forgive the pun on your C++ reference) depending on how the concepts are delivered and how the tools are designed and implemented. Yes, viz can't become basket weaving. But there is always a spectrum on which things can be made more or less difficult. Didn't some of your professors make subjects seem beautiful, elegant, easy to grok, while some of them seemed to make subjects no more inherently difficult inscrutable?

_coveredInBees 8y ago

Everyone shitting on D3 here should stop and consider that there must be some fundamental reason why a library like D3.js has withstood the test of time in the JS landscape, when most libraries go out of fashion within months. Most of the comments here complaining about D3 boil down to the following: - Using the wrong tool for the job... if all you want is off-the-shelf visualizations, use one of the many visualization libraries out there that utilize d3.js on the back-end - Not wanting to put in the effort to learn the basics of SVG, CSS and the D3 API, and yet expecting hand-holding and everything to just be easy The beauty of D3 is how powerful it is and how unopinionated it is. You can literally make any type of visualization with it, and it has some astoundingly useful helper libraries for data binding, data manipulation, color handling, etc. As someone who picked up D3.js from scratch, I learned it by having clear goals regarding what I wanted to achieve. Having built charts / visualizations that are used in several products, it became very clear, very early on, that unless your customer/product requirements are super light-weight and run-of-the-mill visualizations are okay, you are bound to get to a point sooner or later where a visualization library isn't going to cut it. At that point, you're either going to spend a ton of time trying to shoe-horn a feature into an existing library that isn't made to be flexible, or you could have just saved yourself a whole bunch of effort by having gone the route of writing your visualizations in D3 from the get-go. D3 absolutely has a place in the data visualization space. You just need to be aware that it may not be the right tool for you, and if you do need to wield it, you need to put in the effort to actually familiarize yourself with the D3 related landscape rather than expecting to be able to simply copy-paste examples and hack your way to a working solution. PS - I should mention, that I'm an ML Engineer / Data Scientist, not a data viz geek, so it's not like D3 is relegated to people who do data viz for a living. There are plenty of folks who have picked up D3 on the side and done amazing things with it.

Silhouette 8y ago
Not wanting to put in the effort to learn the basics of SVG, CSS and the D3 API, and yet expecting hand-holding and everything to just be easy Sadly, this problem has become endemic within the web development community in recent years. People are more concerned with learning the latest buzzword framework than the fundamentals. The article itself quotes this tweet early on: The trouble with D3 is to build a visualization you must also have a deep understanding of SVG, DOM, JavaScript, geometry, color spaces, data structures, the standard model, and quantum physics — @seecmb Well, yes. Designing and implementing good visualisations is a skill. It has many aspects, and they aren't trivial. If you want to draw a standard, no-frills, minimally-customised chart then there are many higher-level libraries that will draw one for you and you're not the target audience for D3. If you want to have more customised and creative visualisations and interactions, you have to customise and create. That requires understanding the tools available and knowing what you're trying to achieve with them, but if you are willing to put the work in to reach that point, D3 offers a useful and powerful toolbox.

metakermit 8y ago

If you're looking for a good high-level abstraction, I really recommend the open source Plotly.js library: https://plot.ly/javascript/ https://plot.ly/javascript/ (similar to HighCharts, but without the commercial license mumbo-jumbo) Yeah, I see D3 itself as more of a tool tailored towards DataVis geeks developing new visualisation methods.

daemin 8y ago
Tried it recently but it has some flaws, like locking up the browser when viewing ~5k values in a 3d scatter plot when viewed from certain angles.

WhitneyLand 8y ago

Working hard to learn, is not the same thing as how hard learning has to be. Conflating these things is the biggest theme I see in the comments. I risk ranting here, because it seems like this conflation is a recurring misconception in our community. Maybe because most of us don't get to spend equal time thinking about design and usability in our specialties? For the benefit of all the projects we both design and use, we must consider these as two important, yet separate concepts: 1) Any non-trivial concept, tool, or process requires serious investment to be productive with. We should expect it, not complain about it, and just roll up our sleeves if it's really the right tool for us and for the job. 2) We should never let any tooling or library off the usability or productivity hook because it's non-trivial. All designs/implementations do not have to be equal in this regard and are usually not. Yes, we should be willing to invest our time and build the proper foundational knowledge. We should also always challenge how things can be made more productive, more approachable, and as simple as possible but no simpler. The guy to first think of that phrase worked with some pretty damn non-trivial concepts.

sh87 8y ago

> The common theme was that they had started with goals. They learned what they needed from d3 along the way to achieving those goals. This is my key takeaway that fully resonates with my way of learning anything related to JS be it D3, Angular, VueJS, Node, Android and more recently React. Take a problem, solve it in the tech of your choice.