12 ms·

I want to learn D3. I don’t want to learn Observable. Is that ok?

throwawaybutwhy 5y ago
Already posted 22 days ago but did not get any traction. Inspired by https://news.ycombinator.com/item?id=27485123 https://news.ycombinator.com/item?id=27485123
- bboreham 5y ago
  That’s a post from 20 hours ago, with 50 comments. Still, it’s useful, thanks!
  - Kiro 5y ago
    Not the same post. What they mean is that this article had already been posted before but was reposted now, inspired by the linked post from 20 hours ago.
    - bboreham 5y ago
      Ah, I failed to read “Inspired” somehow.
bboreham 5y ago
I had this exact problem myself two weeks back. All the pre-Observable examples I found were for D3 v4, while D3 is now at v6. Some great resources in the thread, which I hadn’t found by Googling. Thanks!
- an1sotropy 5y ago
  actually v7: https://news.ycombinator.com/item?id=27485123 https://news.ycombinator.com/item?id=27485123
FormFollowsFunc 5y ago
I’ve figured out how to go the other way - from a JS framework to Observable. I put all the D3 code in a reusable chart [1] which can be copied and pasted into an Observable cell. And have the JS framework handle things like loading data which is modified slightly when transferred to Observable. I don’t want to be too dependent on a proprietary service but it’s nice to share your work on it occasionally. [1] https://bost.ocks.org/mike/chart/ https://bost.ocks.org/mike/chart/
fouronnes3 5y ago
I've had the goal to learn modern JS and D3 for a few years now. And I struggled with finding a good ressource as well. Over the past year I've done it finally, and I can recommend the book "D3 for the impatient". It's written for D3 v4.x, but I wanted to write my project with D3 v6, obviously. Luckily, just going for it and googling the error message when I got the inevitable "foo is not a function" was easy enough. Now I'm confident enough with it that just reading the github readme of a new D3 module is usually enough to get me going. But the API has quite a bit rough edges (like the difference between select and selectAll, the selection.merge(enter) pattern), that a good book is really nice. Especially since online tutorials are a complete mess - because of Observable non-sense and severe API rotting.
- FormFollowsFunc 5y ago
  There is now selection.join() [1] which means you don’t have to use selection.merge() anymore. [1] https://observablehq.com/@d3/selection-join https://observablehq.com/@d3/selection-join
  - fouronnes3 5y ago
    Yes I know! It's a really good improvement. Note that your link nicely proves OP's point ;)
    - deleted 5y ago
      [deleted]
    - j-pb 5y ago
      No it doesn't. You only have to change two lines to vanilla javascript. document.getElementById("myRoot").appendChild(svg.node()); and remove the yield Thats it. But apparently you all don't know this, because you don't want to understand D3 or basic DOM APIs beyond ctrl-c/ctrl-v.
- montalbano 5y ago
  I think you may be incorrect regarding its version compatibility. See the author's github repository here: https://github.com/janert/d3-for-the-impatient https://github.com/janert/d3-for-the-impatient > All examples have been verified to work with this version of the D3 library (D3 Version 5.9.2, downloaded 15. Apr 2019) So, not quite V6, but a lot more up-to-date than v4. 'D3 for the impatient' is indeed an excellent book and was my entry point into using D3 aswell.
cube2222 5y ago
If anyone is having this problem, just use the tutorial series' on Observable. They really are great and provide a nice environment for learning. Later, when you want to productionize it or create self-contained applications, nothing's stopping you. A quick google will show you how to build your D3 chart as a React component. You're not learning any non-transferable skills by learning on Observable. If I'm already writing a comment, I'd like to say that D3 is so awesome and magical! It's surprisingly easy to create nice-looking animated visualizations. After going through the Observable tutorials for fun I quickly found ways to use it at my employer[0] and create various useful and well-looking animations, you can see one here: https://youtu.be/N9haFA_MwdM https://youtu.be/N9haFA_MwdM FWIW the tutorial series I used myself was https://observablehq.com/collection/@sxywu/introduction-to-d3-js https://observablehq.com/collection/@sxywu/introduction-to-d... , I recommend it very much. [0]:https://spacelift.io https://spacelift.io
- deleted 5y ago
  [deleted]
borrowcheckfml 5y ago
I was in this exact same situation a year ago. Learning d3 had always been on my list of things to do but I never got around to it. When I finally found some time I realized that all their resources and tutorials had moved to this heavy interactive thing called "observable" - I was furious because I spent hours trying to find old tutorials. Maybe Observable is a good product for some use cases, but I don't want to deal with it. I want to have plain HTML with plain JS so that I can actually understand what's going on. No layers upon layers of magic. Eventually I gave up. We need to stop forcing all this unnecessary complexity upon users.
- alisonkisk 5y ago
  Are you sure it's unnecessary complexity? To someone else, HTML and JavaScript may be "unnecessary" complexity and they want to see the plain C/C++ code of how to render a web page with dynamic animation.
- dgb23 5y ago
  But then again, you could just not use d3 in the first place and write a thin translation from your data structures to svg strings. Maybe use a template engine. You don’t need to cover the whole spec for your use case.
  - borrowcheckfml 5y ago
    Yes, I ended up not using d3 and using something else instead. But it's a shame because I thought d3 is a great project and a good fit for my use case. I just didn't want to waste my time and energy dealing with Observable.
    - pmontra 5y ago
      What did you use and was it up to the task?
  - nitrogen 5y ago
    SVG seems really intimidating at first, but for simple (especially non-work) use cases where you only need to do one thing, it's actually really fast and easy to generate minimal SVG directly from data. The learning curve is no steeper than learning a does-everything plotting library, again for simple unchanging usecases. Prerequisites: knowing how to scale numbers from one range (data range) to another (SVG coordinate range) using multiplication and addition.
dmitriid 5y ago
I find @mbstock's answer slightly disheartening: https://talk.observablehq.com/t/i-want-to-learn-d3-i-don-t-want-to-learn-observable-is-that-ok/1957/16 https://talk.observablehq.com/t/i-want-to-learn-d3-i-don-t-w... Instead of "we should make this more approachable without buying into Observable" it's "we should spend effort on helping you learn Observable and explaining why you need it". @sanjalot's final comment is somewhat better https://talk.observablehq.com/t/i-want-to-learn-d3-i-don-t-want-to-learn-observable-is-that-ok/1957/63 https://talk.observablehq.com/t/i-want-to-learn-d3-i-don-t-w... But instead of links in a forum somewhere to someone else's Observable explaining how to turn Observable's into plain JS code, this info should be front and center on the first page of official docs.
- phlakaton 5y ago
  This seems to me like a problem the community can readily solve without demanding that the head of Observable solve it for them. Make a nice neutrally-positioned (i.e. not openly hostile to Observable) site that manages plain JS examples and perhaps you can even get Observable to link to it!
d3problem 5y ago
Hopefully I'm not completely out of place asking this here, but does anyone have any suggestions for why a d3 animation inside React might be a bit janky? Looks smooth as butter on my old computer, but for some reason on my better performing computer it looks janky!
javierluraschi 5y ago
I sympathize with the comments, as a software engineer, this comment from Mike really describes my frustration with Observable's runtime: > Observable notebooks are like spreadsheets, where cells run automatically whenever you edit or values change. That’s not how conventional (imperative) programming languages work, so sadly you can’t simply copy-paste a whole notebook into a vanilla JavaScript application. However, I do think many others will prefer their reactive runtime; so this is by no means a criticism to Observable, they are doing great work. They are just not targeting JS-purists and that might be the right call. As a software engineer, I love D3 but don't want to be stuck with a reactive runtime that is not vanilla JavaScript. I got myself to build https://hal9.ai https://hal9.ai, an integrated environment to do data analysis based of JavaScript. Is not a D3 learning environment but you can certainly use it for that purpose. If someone is interested in providing feedback or helping with D3 examples, please do so at https://github.com/hal9ai/hal9ai https://github.com/hal9ai/hal9ai or shoot me an email at javier at hal9.ai
jojo2000 5y ago
I learnt d3 with <script> tags.
ptr2voidStar 5y ago
I'm glad to hear that others are equally frustrated with the tight coupling between D3 and Observable. For me, it just wasn't worth the effort of learning D3 AND Observable (especially, not being particularly fond of JS to start with) - in the end, it was too much, and I ended up using a simpler third party javascript charting library instead - which was a real shame, because I really loved what D3 could achieve. Mike Bostock is clearly a genius, but I think his brain is wired differently from the rest of us mere mortals - and unfortunately, he seems to have the knack of making simple things terribly complicated.
- borrowcheckfml 5y ago
  I had no idea that the creator of d3js is also the co-founder of Observable! Now this move makes a lot more sense. I'm starting to believe this may be another case where the creator (rightly so) wanted to find a way to monetize his popular open source project. It's a shame this happened. I don't blame him, but the broken open source business model.
  - slumpt_ 5y ago
    Yep, this is 100% my read. It’s a damn shame too because d3 is fabulous and I thought he’d already encountered enough wealth through his work with NYT etc. Surely there’s a way to grow Observable without cannibalizing d3.
- inigojonesguy 5y ago
  I used D3 v5 only for the force directed graphs part, independently in an artificial life & lambda calculus project. Is maybe too much for such an austere domain, since I saw comments that the result is "unnecessarily flashy". Give me any other visually comparable, but lower level js library for the same purpose and I'll take it. I'm not a programmer, but to paraphrase Mark Twain from A Connecticut Yankee in King Arthur's Court, I felt "I was standing in the awful presence of the Mother of the" functional_programming "language".
stolenmerch 5y ago
It's such a small thing but I can't stand having the output of a cell evaluate above the cell. I get that it's part of the Observable reactive environment, but it breaks what we're used to with the command line and other notebook applications.
- deleted 5y ago
  [deleted]
- CamperBob2 5y ago
  I agree, it's confusing enough to make the whole reactive concept less approachable. It'd help if the output didn't look so much like the cell itself, but it's not entirely clear how to differentiate them visually.
jordache 5y ago
what the hell is observable? tried to figure it out via their site, it's vague as hell
- slumpt_ 5y ago
  jupyter notebooks in javascript a huge bet
hashkb 5y ago
The question is about lock-in and choice. You can see on the thread the Observable authors seem almost proud of the fact that their tooling is so good that it couldn't possibly ever be made to be more compatible with everyone else's way of working. In the rapidly moving JS world, this is arrogant and shows poor foresight. I love learning new tools, but this notebook craze is alarming. When I pay a python contractor I have to specify "a runnable script" or I get a jupyter notebook as a deliverable. I hate to think D3 is going the same way. It's no good to make your tool so hard that it needs a dedicated, incompatible (and morally superior) development environment.
gjm11 5y ago
It seems as if everyone in that thread is too polite to make what seems to me to be the obvious comment: This all seems like exactly the behaviour you would expect if having D3 users is not profitable and having Observable users is profitable. This would explain why the Observable team's explanations don't entirely make sense: the real explanation is "we are making things worse for you because we hope to make money by doing so". It would explain why they're unmoved by users' complaints that something allegedly meant to make things easier for them is making them harder: the real goal is profit, not making things easier. On the other hand, (1) using Observable is free (at present, at least), and (2) in other respects mbostock has always seemed genuinely interested in providing good things for everyone to use. So I hope the cynical view in the previous paragraph is wrong. But it does seem like it explains what's happening better than "we just want to make things better for everyone, and Observable is so awesome we can't help using it for everything" which is pretty much the official line.
- throwawaybutwhy 5y ago
  Tangential: in D3.js 7.0 Bostock switched license from BSD 3-clause to ISC [0]. [0] https://github.com/d3/d3/commit/23b0212743d22e2dea3e20163980b37e78b2a25b#diff-c693279643b8cd5d248172d9c22cb7cf4ed163a3c98c8a3f69c2717edd3eacb7 https://github.com/d3/d3/commit/23b0212743d22e2dea3e20163980...
  - jacobolus 5y ago
    These licenses are essentially interchangeable. The ISC license just drops a bit of text that was determined to be unnecessary. https://en.wikipedia.org/wiki/ISC_license https://en.wikipedia.org/wiki/ISC_license
- deleted 5y ago
  [deleted]
- bookofsand 5y ago
  Something is wrong when we expect high quality libraries for free. The authors are highly skilled individuals with mortgages to pay and children to feed. Not sure what the answer is. If coupling Observable with d3 is such an answer, then more power to them. Related, I've stumbled upon this blurb from Codemirror6. Perhaps another answer. If you are using CodeMirror commercially, there is a social expectation that you help fund its maintenance. https://codemirror.net/6 https://codemirror.net/6
protoduction 5y ago
As someone building an in-browser notebook I have a lot of opinions on notebook environments. Notebooks serve different purposes, sometimes the notebook itself is the end-goal because the author is creating an interactive tutorial or explaining a complex concept with a bunch of visualizations. Observable is a fantastic tool for that, and the kind-of-Javascript reactive programming system it is built on is a great fit for that. Outside of that use-case, I think notebooks are great for the first 20% of the effort that gets 80% of the work done. If it turns out one also needs to do the other 80% of the effort to get the last 20%, it is time to "graduate" away from a notebook. For instance if I am participating in a Kaggle machine learning competition I may train my first models in a Jupyter notebook for quick iteration on ideas, but when I settle onto a more rigid pipeline and infra, I will move to plain Python files that I can test and collaborate on. This "graduation" from notebook to the "production/serious" environment should be straightforward, which means there shouldn't be too much magic in the notebook without me opting into it. Documentation in my eyes is not so different, I should be able to copy the examples easily into my JS project without knowing specifics of Observable and adapt it to my problem. Saying "don't be lazy and just learn Observable", or "you must learn D3 itself properly to be able to use it anyway" is not helpful. Observable being a closed, walled garden doesn't help: not being able to author notebooks without using their closed source editor is a liability that I can totally understand makes it a non-starter for some companies and individuals. I think it's ok to plug my own project: It's called Starboard [1] and is truly open source [2]. It's built on different principles: it's hackable, extendable, embeddable, shareable, and easy to check into git (i.e. I try to take what makes the web so great and put that in a notebook environment). You write vanilla JS/ES/Python/HTML/CSS, but you can also import your own more advanced cell types. Here's an example which actually introduces an Observable cell type [3] which is built upon the Observable runtime (which is open source) and an unofficial compiler package [4]. I would be happy for the D3 examples to be expressed in these really-close-to-vanilla JS notebooks, but I doubt I can convince the maintainers to do so. [1]: https://starboard.gg https://starboard.gg [2]: https://github.com/gzuidhof/starboard-notebook https://github.com/gzuidhof/starboard-notebook [3]: https://starboard.gg/gz/open-source-observablehq-nfwK2VA https://starboard.gg/gz/open-source-observablehq-nfwK2VA [4]: https://github.com/asg017/unofficial-observablehq-compiler https://github.com/asg017/unofficial-observablehq-compiler
kesor 5y ago
I don't get what the author is complaining about, D3 has great documentation on GitHub; With or without Observable the way to use D3 is pretty much the same. Yes, the best tutorials for D3 are on Observable; But translating these to pure JavaScript is trivial for anyone who knows some basic javascript syntax. Those who have zero experience with using JavaScript in the first place should first learn the language, and only then try and attempt at using the more advanced libraries such as D3. It is going to be a real pain to try and get anything done with D3 without understanding some basic JavaScript syntax first.
- guru-mediation 5y ago
  Not only that but: "You can download the vanilla JavaScript for any notebook by clicking Download Code in the notebook menu." So... you don't even have to do the trivial transform yourself. Provided those kinds of tools remain available I'm struggling to see the problem. I think obervable is coming from a good place and committed to making their tech as accessable as possible overall. I'm sure there's plenty of less technical users who would have shied away from just a github homepage that might get into it now.
  - throwawaybutwhy 5y ago
    I have tried just that, and the resulting code is underwhelming, to the tune of 'import antigravity': <div id="observablehq-chart-6aa5c38e"></div> <p>Credit: <a href="https://observablehq.com/@d3/contours">Contours by D3</a></p> <script type="module"> import {Runtime, Inspector} from "https://cdn.jsdelivr.net/npm/@observablehq/runtime@4/dist/runtime.js"; import define from "https://api.observablehq.com/@d3/contours.js?v=3"; new Runtime().module(define, name => { if (name === "chart") return new Inspector(document.querySelector("#observablehq-chart-6aa5c38e")); }); </script>
    - jacobolus 5y ago
      The result is: import the open-source Observable runtime and then use it to run the reactive notebook code. What else would you expect? The reactive notebook code is written to work when run against the runtime. If you want to translate it to run in a different context where there is no reactivity, you are going to need to read the code and modify the parts that depend on that reactivity. For example, if a cell depends on a variable that was set to be the reactive output of an HTML slider, then you are going to need to put the slider separately into your page, and write the code yourself to hook up the slider’s event handling logic. This is not too difficult, but the notebook version is typically nicer and more legible.
wirthjason 5y ago
How much of it this problem is the difference between I want to understand this code vs I want to copy-paste some code to get my plot/chart done? This problem would exist with productionizing any notebook style code. E.g. Jupyter notebooks.
- jacobolus 5y ago
  100% of the problem is people’s desire to copy/paste code directly into their own environment, and then getting frustrated that they can’t do that without putting in a bit of effort to understand the way the Observable runtime works, and then read the code they want to copy and think about it a bit.
nkmnz 5y ago
Slightly off topic, but is anyone aware of a collection of examples or best practices for using d3 in a Vue application? I just started with d3 and I‘m not sure how to find the sweet spot regarding which DOM-operations are done by d3 and which should be handled by vue (and if that, how).
- da39a3ee 5y ago
  I'm in this situation and yesterday I replaced all the d3 SVG generation (d3.append, select, data, join, etc) with Vue template syntax (using v-for). So now I'm only using d3 for the x and y axis functions. The code is now much simpler. However, I lost the ability to animate with d3.transition, which I had been contemplating.
5cott0 5y ago
The learning curve for d3 is so steep that people are starting to confuse it for observablehq.
hungryforcodes 5y ago
This may sound like a weird question -- but is there any reason not to use an older version if you feel comfortable with it?
DataCrayon 5y ago
Mentioned on an earlier thread, but this is part of the reason I started my book "Visualisation with D3.js"[1]... it's early access and I'm trying to find more time to work on it. It's entirely separate from Observable, and it's my way of formalising my D3.js learning journey whilst producing something I can refer to in future. [1] https://datacrayon.com/shop/product/visualisation-with-d3/ https://datacrayon.com/shop/product/visualisation-with-d3/
kowlo 5y ago
As others have pointed out: it's simply because Observable is where they will eventually make money. Both belong to the same author, so their coupling has incentives. If you could entirely self-host the whole Observable experience it wouldn't be so bad. But you can't.
tlarkworthy 5y ago
ObservableHQ has the tightest feedback loops because of the reactive runtime. It's a better way to program visual things. I prefer non-sequentially executing programs. It's better. Code changes are local operations, so your running program state is generally preserved . This is an amazing boost for iterative development. This is Smalltalk level productivity boosts Sucks if you want to do something legacy but I want to move forward with programming and breaking the sequential execution model is the right step IMHO.
da39a3ee 5y ago
Here's what's really going on, and what I think the observable team should have said: d3 was born ~10 years ago, before the rise of vue and react. Today, the community is split into two groups that we shall call "data viz" and "software engineers". The crux of that whole comment thread is that large portions of the d3 API have now been superseded by React/Vue/Svelte, for the software engineering crowd. You can even see Mike Bostock and the Observable team in places recommending the book and blog post of Amelia Wattenberger. But look at her blog post on using d3 with react: https://wattenberger.com/blog/react-and-d3 https://wattenberger.com/blog/react-and-d3 If you read that you'll see that the Tl:DR is basically > Don't use d3 any longer for anything other than the axis objects: use react/vue to build and update the SVG elements. I suspect that Mike Bostock's (radical) decision to move d3 entirely into the Observable world was a completely rational decision explicitly addressing the fact that he knows that, when it comes to building and updating the DOM, d3 has now been replaced by component-based libraries with HTML-like JSX syntax, i.e. Vue/React etc. At least for software engineers; there is still a community of dataviz people with their center of gravity more in academia and data science who don't use Vue/React and for whom it makes sense to use the entire d3 API (DOM updates as well as utilities like axis functions etc) I think that it would help if the Observable team just stood up and said that explicitly. Maybe even say on the d3 README: > If you are a software engineer, bear in mind that things have moved on from 8 years ago and that you may in fact be best served by using utilities from d3 but using React or Vue for your DOM building. Also, I think that the Observable team did themselves no favours by their slightly evangelical insistance on encouraging people to try Observable because they'll find that they like it once they try. I am a software engineer. I am open minded. I have tried Jupyter many times and I have tried Observable. I cannot use those tools for one simple reason: I use git to navigate my code state. If I am debugging, I make small commits, and use stash and revert and cherry-pick etc to experiment with different code states. I think in git and I work in git. Observable (and Jupyter) are completely antithetical to Git. So I also think that the Observable team would do better to say: > Look. If you're a professional software engineer who is used to working in a sophiisticated fashion with version control, then honestly this stuff might not be your cup of tea. I'm sure you can see what we're trying to do, and that it's really cool, but we understand if you just want to enter lines of code into a git repo using your text editor. And for you, here's a quick summary/pointer to docs on how to use d3 in that way, with Vue/React, and some pointers on how to make use of the Observable docs for non-Observable users. Mike Bostock if there's any chance you read this, d3 is one of the most impressive open source libraries I've known and I have been a fan of yours for years! Observable looks amazing and very forward-thinking too and I think I understand that it's aiming to democratize javascript visualization. I hope the above comments came across as constructive. (It would be awesome to have more pointers in the future on ways to make use of portions of the d3 API with other DOM-building APIs such as Vue/React.)
al2o3cr 5y ago
ZOMG THIS FREE LIBRARY DOESN'T HAVE EXAMPLES I CAN COPY INTO MY HOMEWORK VERBATIM WAAAAAAAAAAAAAAAAAH Every poster in that comment thread (and this one) should demand a full refund of what they paid for d3.js
timdaub 5y ago
Recently I needed a simple chart for my startup. I looked online and all I could find was bloated libraries that were increasing the bundle size by many KBs. But then they also started rendering in the browser. Considering that it was possible to cache the request for a full day for all users, I wrote a small library that renders an svg on the server: https://github.com/rugpullindex/svg-line-chart https://github.com/rugpullindex/svg-line-chart Chart in action: https://rugpullindex.com https://rugpullindex.com
m_rpn 5y ago
I understand the will to bring people to your new project(Observable) using something succesful that you already made(D3), but i think this move will be detrimental to both projects, since everything fundamental for the newcomers is being moved to Observable (Github docs, examples, ...), making the process of learning and integrating the project way more difficult and full of pointless time wasting garbage. This is the exact opposite of open source culture, as an old contributor and advocate of D3 i'm quite pissed. At the same time the big contributors have obvioulsy the right to steer the project in the direction they deem best.
jwilber 5y ago
This thread is literally people complaining about the free software used to host free documentation for a free library. The onus is absolutely not on anyone to improve things for you just because you find them difficult. Consider how lucky you are that d3 is even documented so well to begin with. Insane level of entitlement in this thread.

Already posted 22 days ago but did not get any traction. Inspired by https://news.ycombinator.com/item?id=27485123 https://news.ycombinator.com/item?id=27485123

bboreham 5y ago
That’s a post from 20 hours ago, with 50 comments. Still, it’s useful, thanks!
- Kiro 5y ago
  Not the same post. What they mean is that this article had already been posted before but was reposted now, inspired by the linked post from 20 hours ago.
  - bboreham 5y ago
    Ah, I failed to read “Inspired” somehow.

bboreham 5y ago

I had this exact problem myself two weeks back. All the pre-Observable examples I found were for D3 v4, while D3 is now at v6. Some great resources in the thread, which I hadn’t found by Googling. Thanks!

an1sotropy 5y ago
actually v7: https://news.ycombinator.com/item?id=27485123 https://news.ycombinator.com/item?id=27485123

FormFollowsFunc 5y ago

I’ve figured out how to go the other way - from a JS framework to Observable. I put all the D3 code in a reusable chart [1] which can be copied and pasted into an Observable cell. And have the JS framework handle things like loading data which is modified slightly when transferred to Observable. I don’t want to be too dependent on a proprietary service but it’s nice to share your work on it occasionally. [1] https://bost.ocks.org/mike/chart/ https://bost.ocks.org/mike/chart/

fouronnes3 5y ago

I've had the goal to learn modern JS and D3 for a few years now. And I struggled with finding a good ressource as well. Over the past year I've done it finally, and I can recommend the book "D3 for the impatient". It's written for D3 v4.x, but I wanted to write my project with D3 v6, obviously. Luckily, just going for it and googling the error message when I got the inevitable "foo is not a function" was easy enough. Now I'm confident enough with it that just reading the github readme of a new D3 module is usually enough to get me going. But the API has quite a bit rough edges (like the difference between select and selectAll, the selection.merge(enter) pattern), that a good book is really nice. Especially since online tutorials are a complete mess - because of Observable non-sense and severe API rotting.

FormFollowsFunc 5y ago
There is now selection.join() [1] which means you don’t have to use selection.merge() anymore. [1] https://observablehq.com/@d3/selection-join https://observablehq.com/@d3/selection-join
- fouronnes3 5y ago
  Yes I know! It's a really good improvement. Note that your link nicely proves OP's point ;)
  - deleted 5y ago
    [deleted]
  - j-pb 5y ago
    No it doesn't. You only have to change two lines to vanilla javascript. document.getElementById("myRoot").appendChild(svg.node()); and remove the yield Thats it. But apparently you all don't know this, because you don't want to understand D3 or basic DOM APIs beyond ctrl-c/ctrl-v.
montalbano 5y ago
I think you may be incorrect regarding its version compatibility. See the author's github repository here: https://github.com/janert/d3-for-the-impatient https://github.com/janert/d3-for-the-impatient > All examples have been verified to work with this version of the D3 library (D3 Version 5.9.2, downloaded 15. Apr 2019) So, not quite V6, but a lot more up-to-date than v4. 'D3 for the impatient' is indeed an excellent book and was my entry point into using D3 aswell.

cube2222 5y ago

If anyone is having this problem, just use the tutorial series' on Observable. They really are great and provide a nice environment for learning. Later, when you want to productionize it or create self-contained applications, nothing's stopping you. A quick google will show you how to build your D3 chart as a React component. You're not learning any non-transferable skills by learning on Observable. If I'm already writing a comment, I'd like to say that D3 is so awesome and magical! It's surprisingly easy to create nice-looking animated visualizations. After going through the Observable tutorials for fun I quickly found ways to use it at my employer[0] and create various useful and well-looking animations, you can see one here: https://youtu.be/N9haFA_MwdM https://youtu.be/N9haFA_MwdM FWIW the tutorial series I used myself was https://observablehq.com/collection/@sxywu/introduction-to-d3-js https://observablehq.com/collection/@sxywu/introduction-to-d... , I recommend it very much. [0]:https://spacelift.io https://spacelift.io

deleted 5y ago
[deleted]

borrowcheckfml 5y ago

I was in this exact same situation a year ago. Learning d3 had always been on my list of things to do but I never got around to it. When I finally found some time I realized that all their resources and tutorials had moved to this heavy interactive thing called "observable" - I was furious because I spent hours trying to find old tutorials. Maybe Observable is a good product for some use cases, but I don't want to deal with it. I want to have plain HTML with plain JS so that I can actually understand what's going on. No layers upon layers of magic. Eventually I gave up. We need to stop forcing all this unnecessary complexity upon users.

alisonkisk 5y ago
Are you sure it's unnecessary complexity? To someone else, HTML and JavaScript may be "unnecessary" complexity and they want to see the plain C/C++ code of how to render a web page with dynamic animation.
dgb23 5y ago
But then again, you could just not use d3 in the first place and write a thin translation from your data structures to svg strings. Maybe use a template engine. You don’t need to cover the whole spec for your use case.
- borrowcheckfml 5y ago
  Yes, I ended up not using d3 and using something else instead. But it's a shame because I thought d3 is a great project and a good fit for my use case. I just didn't want to waste my time and energy dealing with Observable.
  - pmontra 5y ago
    What did you use and was it up to the task?
- nitrogen 5y ago
  SVG seems really intimidating at first, but for simple (especially non-work) use cases where you only need to do one thing, it's actually really fast and easy to generate minimal SVG directly from data. The learning curve is no steeper than learning a does-everything plotting library, again for simple unchanging usecases. Prerequisites: knowing how to scale numbers from one range (data range) to another (SVG coordinate range) using multiplication and addition.

dmitriid 5y ago

I find @mbstock's answer slightly disheartening: https://talk.observablehq.com/t/i-want-to-learn-d3-i-don-t-want-to-learn-observable-is-that-ok/1957/16 https://talk.observablehq.com/t/i-want-to-learn-d3-i-don-t-w... Instead of "we should make this more approachable without buying into Observable" it's "we should spend effort on helping you learn Observable and explaining why you need it". @sanjalot's final comment is somewhat better https://talk.observablehq.com/t/i-want-to-learn-d3-i-don-t-want-to-learn-observable-is-that-ok/1957/63 https://talk.observablehq.com/t/i-want-to-learn-d3-i-don-t-w... But instead of links in a forum somewhere to someone else's Observable explaining how to turn Observable's into plain JS code, this info should be front and center on the first page of official docs.

phlakaton 5y ago
This seems to me like a problem the community can readily solve without demanding that the head of Observable solve it for them. Make a nice neutrally-positioned (i.e. not openly hostile to Observable) site that manages plain JS examples and perhaps you can even get Observable to link to it!

d3problem 5y ago

Hopefully I'm not completely out of place asking this here, but does anyone have any suggestions for why a d3 animation inside React might be a bit janky? Looks smooth as butter on my old computer, but for some reason on my better performing computer it looks janky!

javierluraschi 5y ago

I sympathize with the comments, as a software engineer, this comment from Mike really describes my frustration with Observable's runtime: > Observable notebooks are like spreadsheets, where cells run automatically whenever you edit or values change. That’s not how conventional (imperative) programming languages work, so sadly you can’t simply copy-paste a whole notebook into a vanilla JavaScript application. However, I do think many others will prefer their reactive runtime; so this is by no means a criticism to Observable, they are doing great work. They are just not targeting JS-purists and that might be the right call. As a software engineer, I love D3 but don't want to be stuck with a reactive runtime that is not vanilla JavaScript. I got myself to build https://hal9.ai https://hal9.ai, an integrated environment to do data analysis based of JavaScript. Is not a D3 learning environment but you can certainly use it for that purpose. If someone is interested in providing feedback or helping with D3 examples, please do so at https://github.com/hal9ai/hal9ai https://github.com/hal9ai/hal9ai or shoot me an email at javier at hal9.ai

jojo2000 5y ago

I learnt d3 with <script> tags.

ptr2voidStar 5y ago

I'm glad to hear that others are equally frustrated with the tight coupling between D3 and Observable. For me, it just wasn't worth the effort of learning D3 AND Observable (especially, not being particularly fond of JS to start with) - in the end, it was too much, and I ended up using a simpler third party javascript charting library instead - which was a real shame, because I really loved what D3 could achieve. Mike Bostock is clearly a genius, but I think his brain is wired differently from the rest of us mere mortals - and unfortunately, he seems to have the knack of making simple things terribly complicated.

borrowcheckfml 5y ago
I had no idea that the creator of d3js is also the co-founder of Observable! Now this move makes a lot more sense. I'm starting to believe this may be another case where the creator (rightly so) wanted to find a way to monetize his popular open source project. It's a shame this happened. I don't blame him, but the broken open source business model.
- slumpt_ 5y ago
  Yep, this is 100% my read. It’s a damn shame too because d3 is fabulous and I thought he’d already encountered enough wealth through his work with NYT etc. Surely there’s a way to grow Observable without cannibalizing d3.
inigojonesguy 5y ago
I used D3 v5 only for the force directed graphs part, independently in an artificial life & lambda calculus project. Is maybe too much for such an austere domain, since I saw comments that the result is "unnecessarily flashy". Give me any other visually comparable, but lower level js library for the same purpose and I'll take it. I'm not a programmer, but to paraphrase Mark Twain from A Connecticut Yankee in King Arthur's Court, I felt "I was standing in the awful presence of the Mother of the" functional_programming "language".

stolenmerch 5y ago

It's such a small thing but I can't stand having the output of a cell evaluate above the cell. I get that it's part of the Observable reactive environment, but it breaks what we're used to with the command line and other notebook applications.

deleted 5y ago
[deleted]
CamperBob2 5y ago
I agree, it's confusing enough to make the whole reactive concept less approachable. It'd help if the output didn't look so much like the cell itself, but it's not entirely clear how to differentiate them visually.

jordache 5y ago

what the hell is observable? tried to figure it out via their site, it's vague as hell

slumpt_ 5y ago
jupyter notebooks in javascript a huge bet

hashkb 5y ago

The question is about lock-in and choice. You can see on the thread the Observable authors seem almost proud of the fact that their tooling is so good that it couldn't possibly ever be made to be more compatible with everyone else's way of working. In the rapidly moving JS world, this is arrogant and shows poor foresight. I love learning new tools, but this notebook craze is alarming. When I pay a python contractor I have to specify "a runnable script" or I get a jupyter notebook as a deliverable. I hate to think D3 is going the same way. It's no good to make your tool so hard that it needs a dedicated, incompatible (and morally superior) development environment.

gjm11 5y ago

It seems as if everyone in that thread is too polite to make what seems to me to be the obvious comment: This all seems like exactly the behaviour you would expect if having D3 users is not profitable and having Observable users is profitable. This would explain why the Observable team's explanations don't entirely make sense: the real explanation is "we are making things worse for you because we hope to make money by doing so". It would explain why they're unmoved by users' complaints that something allegedly meant to make things easier for them is making them harder: the real goal is profit, not making things easier. On the other hand, (1) using Observable is free (at present, at least), and (2) in other respects mbostock has always seemed genuinely interested in providing good things for everyone to use. So I hope the cynical view in the previous paragraph is wrong. But it does seem like it explains what's happening better than "we just want to make things better for everyone, and Observable is so awesome we can't help using it for everything" which is pretty much the official line.

throwawaybutwhy 5y ago
Tangential: in D3.js 7.0 Bostock switched license from BSD 3-clause to ISC [0]. [0] https://github.com/d3/d3/commit/23b0212743d22e2dea3e20163980b37e78b2a25b#diff-c693279643b8cd5d248172d9c22cb7cf4ed163a3c98c8a3f69c2717edd3eacb7 https://github.com/d3/d3/commit/23b0212743d22e2dea3e20163980...
- jacobolus 5y ago
  These licenses are essentially interchangeable. The ISC license just drops a bit of text that was determined to be unnecessary. https://en.wikipedia.org/wiki/ISC_license https://en.wikipedia.org/wiki/ISC_license
deleted 5y ago
[deleted]
bookofsand 5y ago
Something is wrong when we expect high quality libraries for free. The authors are highly skilled individuals with mortgages to pay and children to feed. Not sure what the answer is. If coupling Observable with d3 is such an answer, then more power to them. Related, I've stumbled upon this blurb from Codemirror6. Perhaps another answer. If you are using CodeMirror commercially, there is a social expectation that you help fund its maintenance. https://codemirror.net/6 https://codemirror.net/6

protoduction 5y ago

As someone building an in-browser notebook I have a lot of opinions on notebook environments. Notebooks serve different purposes, sometimes the notebook itself is the end-goal because the author is creating an interactive tutorial or explaining a complex concept with a bunch of visualizations. Observable is a fantastic tool for that, and the kind-of-Javascript reactive programming system it is built on is a great fit for that. Outside of that use-case, I think notebooks are great for the first 20% of the effort that gets 80% of the work done. If it turns out one also needs to do the other 80% of the effort to get the last 20%, it is time to "graduate" away from a notebook. For instance if I am participating in a Kaggle machine learning competition I may train my first models in a Jupyter notebook for quick iteration on ideas, but when I settle onto a more rigid pipeline and infra, I will move to plain Python files that I can test and collaborate on. This "graduation" from notebook to the "production/serious" environment should be straightforward, which means there shouldn't be too much magic in the notebook without me opting into it. Documentation in my eyes is not so different, I should be able to copy the examples easily into my JS project without knowing specifics of Observable and adapt it to my problem. Saying "don't be lazy and just learn Observable", or "you must learn D3 itself properly to be able to use it anyway" is not helpful. Observable being a closed, walled garden doesn't help: not being able to author notebooks without using their closed source editor is a liability that I can totally understand makes it a non-starter for some companies and individuals. I think it's ok to plug my own project: It's called Starboard [1] and is truly open source [2]. It's built on different principles: it's hackable, extendable, embeddable, shareable, and easy to check into git (i.e. I try to take what makes the web so great and put that in a notebook environment). You write vanilla JS/ES/Python/HTML/CSS, but you can also import your own more advanced cell types. Here's an example which actually introduces an Observable cell type [3] which is built upon the Observable runtime (which is open source) and an unofficial compiler package [4]. I would be happy for the D3 examples to be expressed in these really-close-to-vanilla JS notebooks, but I doubt I can convince the maintainers to do so. [1]: https://starboard.gg https://starboard.gg [2]: https://github.com/gzuidhof/starboard-notebook https://github.com/gzuidhof/starboard-notebook [3]: https://starboard.gg/gz/open-source-observablehq-nfwK2VA https://starboard.gg/gz/open-source-observablehq-nfwK2VA [4]: https://github.com/asg017/unofficial-observablehq-compiler https://github.com/asg017/unofficial-observablehq-compiler

kesor 5y ago

I don't get what the author is complaining about, D3 has great documentation on GitHub; With or without Observable the way to use D3 is pretty much the same. Yes, the best tutorials for D3 are on Observable; But translating these to pure JavaScript is trivial for anyone who knows some basic javascript syntax. Those who have zero experience with using JavaScript in the first place should first learn the language, and only then try and attempt at using the more advanced libraries such as D3. It is going to be a real pain to try and get anything done with D3 without understanding some basic JavaScript syntax first.

guru-mediation 5y ago
Not only that but: "You can download the vanilla JavaScript for any notebook by clicking Download Code in the notebook menu." So... you don't even have to do the trivial transform yourself. Provided those kinds of tools remain available I'm struggling to see the problem. I think obervable is coming from a good place and committed to making their tech as accessable as possible overall. I'm sure there's plenty of less technical users who would have shied away from just a github homepage that might get into it now.
- throwawaybutwhy 5y ago
  I have tried just that, and the resulting code is underwhelming, to the tune of 'import antigravity': <div id="observablehq-chart-6aa5c38e"></div> <p>Credit: <a href="https://observablehq.com/@d3/contours">Contours by D3</a></p> <script type="module"> import {Runtime, Inspector} from "https://cdn.jsdelivr.net/npm/@observablehq/runtime@4/dist/runtime.js"; import define from "https://api.observablehq.com/@d3/contours.js?v=3"; new Runtime().module(define, name => { if (name === "chart") return new Inspector(document.querySelector("#observablehq-chart-6aa5c38e")); }); </script>
  - jacobolus 5y ago
    The result is: import the open-source Observable runtime and then use it to run the reactive notebook code. What else would you expect? The reactive notebook code is written to work when run against the runtime. If you want to translate it to run in a different context where there is no reactivity, you are going to need to read the code and modify the parts that depend on that reactivity. For example, if a cell depends on a variable that was set to be the reactive output of an HTML slider, then you are going to need to put the slider separately into your page, and write the code yourself to hook up the slider’s event handling logic. This is not too difficult, but the notebook version is typically nicer and more legible.

wirthjason 5y ago

How much of it this problem is the difference between I want to understand this code vs I want to copy-paste some code to get my plot/chart done? This problem would exist with productionizing any notebook style code. E.g. Jupyter notebooks.

jacobolus 5y ago
100% of the problem is people’s desire to copy/paste code directly into their own environment, and then getting frustrated that they can’t do that without putting in a bit of effort to understand the way the Observable runtime works, and then read the code they want to copy and think about it a bit.

nkmnz 5y ago

Slightly off topic, but is anyone aware of a collection of examples or best practices for using d3 in a Vue application? I just started with d3 and I‘m not sure how to find the sweet spot regarding which DOM-operations are done by d3 and which should be handled by vue (and if that, how).

da39a3ee 5y ago
I'm in this situation and yesterday I replaced all the d3 SVG generation (d3.append, select, data, join, etc) with Vue template syntax (using v-for). So now I'm only using d3 for the x and y axis functions. The code is now much simpler. However, I lost the ability to animate with d3.transition, which I had been contemplating.

5cott0 5y ago

The learning curve for d3 is so steep that people are starting to confuse it for observablehq.

hungryforcodes 5y ago

This may sound like a weird question -- but is there any reason not to use an older version if you feel comfortable with it?

DataCrayon 5y ago

Mentioned on an earlier thread, but this is part of the reason I started my book "Visualisation with D3.js"[1]... it's early access and I'm trying to find more time to work on it. It's entirely separate from Observable, and it's my way of formalising my D3.js learning journey whilst producing something I can refer to in future. [1] https://datacrayon.com/shop/product/visualisation-with-d3/ https://datacrayon.com/shop/product/visualisation-with-d3/

kowlo 5y ago

As others have pointed out: it's simply because Observable is where they will eventually make money. Both belong to the same author, so their coupling has incentives. If you could entirely self-host the whole Observable experience it wouldn't be so bad. But you can't.

tlarkworthy 5y ago

ObservableHQ has the tightest feedback loops because of the reactive runtime. It's a better way to program visual things. I prefer non-sequentially executing programs. It's better. Code changes are local operations, so your running program state is generally preserved . This is an amazing boost for iterative development. This is Smalltalk level productivity boosts Sucks if you want to do something legacy but I want to move forward with programming and breaking the sequential execution model is the right step IMHO.

da39a3ee 5y ago

Here's what's really going on, and what I think the observable team should have said: d3 was born ~10 years ago, before the rise of vue and react. Today, the community is split into two groups that we shall call "data viz" and "software engineers". The crux of that whole comment thread is that large portions of the d3 API have now been superseded by React/Vue/Svelte, for the software engineering crowd. You can even see Mike Bostock and the Observable team in places recommending the book and blog post of Amelia Wattenberger. But look at her blog post on using d3 with react: https://wattenberger.com/blog/react-and-d3 https://wattenberger.com/blog/react-and-d3 If you read that you'll see that the Tl:DR is basically > Don't use d3 any longer for anything other than the axis objects: use react/vue to build and update the SVG elements. I suspect that Mike Bostock's (radical) decision to move d3 entirely into the Observable world was a completely rational decision explicitly addressing the fact that he knows that, when it comes to building and updating the DOM, d3 has now been replaced by component-based libraries with HTML-like JSX syntax, i.e. Vue/React etc. At least for software engineers; there is still a community of dataviz people with their center of gravity more in academia and data science who don't use Vue/React and for whom it makes sense to use the entire d3 API (DOM updates as well as utilities like axis functions etc) I think that it would help if the Observable team just stood up and said that explicitly. Maybe even say on the d3 README: > If you are a software engineer, bear in mind that things have moved on from 8 years ago and that you may in fact be best served by using utilities from d3 but using React or Vue for your DOM building. Also, I think that the Observable team did themselves no favours by their slightly evangelical insistance on encouraging people to try Observable because they'll find that they like it once they try. I am a software engineer. I am open minded. I have tried Jupyter many times and I have tried Observable. I cannot use those tools for one simple reason: I use git to navigate my code state. If I am debugging, I make small commits, and use stash and revert and cherry-pick etc to experiment with different code states. I think in git and I work in git. Observable (and Jupyter) are completely antithetical to Git. So I also think that the Observable team would do better to say: > Look. If you're a professional software engineer who is used to working in a sophiisticated fashion with version control, then honestly this stuff might not be your cup of tea. I'm sure you can see what we're trying to do, and that it's really cool, but we understand if you just want to enter lines of code into a git repo using your text editor. And for you, here's a quick summary/pointer to docs on how to use d3 in that way, with Vue/React, and some pointers on how to make use of the Observable docs for non-Observable users. Mike Bostock if there's any chance you read this, d3 is one of the most impressive open source libraries I've known and I have been a fan of yours for years! Observable looks amazing and very forward-thinking too and I think I understand that it's aiming to democratize javascript visualization. I hope the above comments came across as constructive. (It would be awesome to have more pointers in the future on ways to make use of portions of the d3 API with other DOM-building APIs such as Vue/React.)

al2o3cr 5y ago

ZOMG THIS FREE LIBRARY DOESN'T HAVE EXAMPLES I CAN COPY INTO MY HOMEWORK VERBATIM WAAAAAAAAAAAAAAAAAH Every poster in that comment thread (and this one) should demand a full refund of what they paid for d3.js

timdaub 5y ago

Recently I needed a simple chart for my startup. I looked online and all I could find was bloated libraries that were increasing the bundle size by many KBs. But then they also started rendering in the browser. Considering that it was possible to cache the request for a full day for all users, I wrote a small library that renders an svg on the server: https://github.com/rugpullindex/svg-line-chart https://github.com/rugpullindex/svg-line-chart Chart in action: https://rugpullindex.com https://rugpullindex.com

m_rpn 5y ago

I understand the will to bring people to your new project(Observable) using something succesful that you already made(D3), but i think this move will be detrimental to both projects, since everything fundamental for the newcomers is being moved to Observable (Github docs, examples, ...), making the process of learning and integrating the project way more difficult and full of pointless time wasting garbage. This is the exact opposite of open source culture, as an old contributor and advocate of D3 i'm quite pissed. At the same time the big contributors have obvioulsy the right to steer the project in the direction they deem best.

jwilber 5y ago

This thread is literally people complaining about the free software used to host free documentation for a free library. The onus is absolutely not on anyone to improve things for you just because you find them difficult. Consider how lucky you are that d3 is even documented so well to begin with. Insane level of entitlement in this thread.