4 ms·

“If you want to build a ship, don’t drum up the men to gather wood, divide the work, and give orders. Instead, teach them to yearn for the vast and endless sea.

“If you want to build a ship, don’t drum up the men to gather wood, divide the work, and give orders. Instead, teach them to yearn for the vast and endless sea.” — Antoine de Saint-Exupéry I will one hundred percent agree with you that my approach is not a good one if you want a pretty good GUI in a reasonable amount of time. For that, an incremental approach, especially adapting some existing successful design, would be better. But that is not in fact my goal, it is to yearn for the sea. I am, quite deliberately, spending a lot of risk points. In addition to using a language which may (see elsethread) not be a good fit for expressing UI at all, I'm building a GPU 2D renderer from scratch, also using compute shader techniques which have not been proven, and I am designing a reactive architecture that is not just a simple adaptation of React. Any of these could fail, as have some of my previous attempts. But I think they will be interesting failures, in that we'll learn something, and if all the pieces do come together, it will be a UI toolkit capable of performance completely untouchable by the existing state of the art. In turn, I'm interested in how that could open up new creative possibilities constrained by current implementations. So I'm pretty comfortable with my approach. And hey, next time you're in the Bay Area (or perhaps when I'm in your neck of the woods), lemme buy you beer and we can talk about why it gives you satisfaction to dump on other people's work.

mwcampbell 4y ago
I think another great counterpoint to dogmatic incrementalism is Gary Bernhardt's classic talk "A Whole New World": https://www.destroyallsoftware.com/talks/a-whole-new-world https://www.destroyallsoftware.com/talks/a-whole-new-world
joshka 4y ago
>So I'm pretty comfortable with my approach. And hey, next time you're in the Bay Area (or perhaps when I'm in your neck of the woods), lemme buy you beer and we can talk about why it gives you satisfaction to dump on other people's work. I love this response - stealing it :)
- deleted 4y ago
  [deleted]
stephc_int13 4y ago
To be frank, I think that the kind of work you are doing is necessary. In my opinion GUI is not yet something I would consider to be a "solved problem". Both from the API perspective and the rendering side, and compute shaders are indeed extremely promising and could be something close to an end-game in this space. This is why I read your articles. And I have absolutely zero interest in dumping on your work, but I have the feeling that a large part of this work is going to waste because of what I consider to be flaws in the ways you are approaching the problems, or maybe the way you advertise your approach, I can sense how discouraging it can be for other devs.
- tayistay 4y ago
  What, specifically do you think the flaws are? I think if you're going to make such criticism, it helps to give actionable specifics (i.e. constructive criticism). How is the approach discouraging to other devs?
  - stephc_int13 4y ago
    I think that people who manage to achieve "impossible" tasks tend to be overly optimistic (and naïve) at first, consciously or not, this is a good trick to fuel their own motivation and that of others. On the contrary, knowing/talking too much about the difficulties can quickly kill the fun and motivation. He should talk about how great the end-goal will be and why it is important.
    - mwcampbell 4y ago
      Would you rather end-users continue to get stuck with applications that ignore important things like accessibility, that can block some people from getting or keeping a job, so as not to kill some developers' motivation? I'm glad Raph is talking about the difficulties so they (hopefully) won't be ignored or clumsily bolted on afterward this time. Edit to add: "Those who cannot learn from history are doomed to repeat it." --George Santayana I think the time for recklessly moving fast and breaking things in software, without taking into account known complexity and avoiding the mistakes of the past, is over. Our impact on the world, and the resulting responsibility, is simply too great for that.
jakear 4y ago
> it will be a UI toolkit capable of performance completely untouchable by the existing state of the art Could you give an example of an application where the bottleneck is UI code? In my experience the bottleneck is always either disk or network. Not trying to bash you, genuinely curious.
- mwcampbell 4y ago
  The fact that our modern computers still often don't feel amazingly fast is a perennial topic of griping on forums like this one. I'm sure the UI stack has something to do with that.
- ratww 4y ago
  There's plenty of apps that we developers use daily that fall into that. Apps like Teams, Slack and Jira can feel incredibly slow to a lot of people, even with everything already in memory and not waiting for anything from the network or disk. Typing messages, changing tabs, going to read a notification... Facebook sometimes takes a second to show characters you typed in a reply, and the characters often show up out of order (also zero network activity during it). A lot of complex WYSIWYG editors are also incredibly heavy, while simpler ones aren't. Sure, that might be technically a bandwidth problem: in this case probably RAM. It is solvable with faster computers/faster RAM. But it's slower or at least the same speed as it was in the past with slower machines. Since hardware got better, it has got to be something different in the software side. And it's not even about difficult things like Unicode Glyphs and Emojis, which are common canned responses when anyone says that software "is slower than N years ago". Those things are handled by the OS, not by Jira. And there are super fast apps that make use of them.
  - jakear 4y ago
    I don't have access to such corporate software at the moment, but can you verify in devtools where the delay is actually coming from in the apps you mention? Leaping to "the stack is bad it needs to be rewritten in rust" is extreme, especially when apps like VS Code are generally regarded as non-slothy [1]. Or at least not more slothy than the domain requires [2]. I'd wager the developers are poorly incentivized - if something satisfies the ticket and looks fast enough on their tiny test bed with state of the art hardware and great network (perhaps even localhost), it ships. On the other hand I don't consider something good enough to ship until it is fast enough on testbeds orders of magnitude larger than what I expect a "normal" workflow would include. The process of going from "works" to "fast enough for people using 100x larger inputs than I expect" almost never involves "rewrite in rust"[3], but instead "cache cleverly", "debounce discretely", and if all else fails "sit down and ponder on novel algorithms and data structures". These are all operations that are just as easy, if not easier, in high level languages as compared to rust. [1]: Full disclosure I was paid to write vscode for a period. When VS Code is slow, and it 100% is at times, the root cause is almost always an extension blocking progress for some dumb reason. This absolutely blows, but isn't a problem rust would solve - indeed extensions can already invoke rust. [2]: inb4 "but sublime!": Running experiments, I've found sublime to in fact be slower than a fresh VS Code install at working with very large files. Of course when you have extensions trying to do dumb stuff with the big files, VS Code can get worthlessly slow. Again not a problem rust would solve. Try it: make a 5M line file, click it open it in Sublime, then in VS Code. On my machine VS Code opens it well before Sublime can. [3]: Yes there are some times when rewriting in rust is appropriate, for instance VS Code's search is ripgrep - but rust isn't handling the UI at all, it's running in a separate thread doing what it does best (multithreaded systems programming), while the main renderer thread is doing what it does best (rendering). This is the way forward for the truly "inner loop" code, IMO.
runnerup 4y ago
Last part might be worth an edit. I saw his comment as constructive well meant criticism rather than callous, thoughtless, or mean spirited. Also love the work you do and enjoy following it. Your treatises on Oklab are the #1 place I send programmers of all skill levels to understand color.
- sfink 4y ago
  As another anecdatum, I thought the last part was beautifully done and completely deserved. The dismissive complaint was not without merit, but wow did it assume that the commenter's use cases are all that matter and all that need be considered. The world of UIs is much, much, much bigger than that, and awash in unsolved or badly-solved problems that matter a lot to many people. I agree with the complaint, fwiw, when applied to an important subset of the design space. I even think it's useful to try to understand where the limits of that subset are, and why. But saying that exploration is pointless because we're all happy living on this here big island and there's nowhere else that could possibly be better so why bother looking, it's all good except we still don't know why people keep dropping dead, but that's an acceptable drawback to what is otherwise a paradise on Earth—hang on a sec while I scrape off these leeches, they're so silly sometimes—and the people who think otherwise are just malcontents who ought to be out catching fish for the rest of us to enjoy.
aaaaaaaaaaab 4y ago
>it will be a UI toolkit capable of performance completely untouchable by the existing state of the art Why exactly? What are the current state of the art UI toolkits leaving on the table performance-wise?
- raphlinus 4y ago
  I'll have a lot more to say about this as I gather quantitative performance data. But it's a good question. I expect the big wins are: fast 2D (vector with blends and so on) rendering with compute shaders, multithreaded creation of expensive resources like image decompression and text layout, pushing incremental reactivity all the way from app logic to GPU (as opposed to needlessly redoing work), and of course just using a fast, non-GC language.
  - aaaaaaaaaaab 4y ago
    What do you mean by “pushing incremental reactivity all the way from app logic to GPU (as opposed to needlessly redoing work)”?
    - raphlinus 4y ago
      From a computation perspective, UI is fundamentally an incremental computation engine. Most elements are not changing from frame to frame, so you can either recompute and re-render, or be smarter about only propagating deltas. I'd like to propagate those deltas all the way to the GPU, so you reuse lots of things from the previous frame if they haven't been invalidated. I'll be writing about this in considerably more detail; stay tuned.
      - aaaaaaaaaaab 4y ago
        Wait, are you talking about deltas as in damage rects?
        infogulch 4y ago

mwcampbell 4y ago

I think another great counterpoint to dogmatic incrementalism is Gary Bernhardt's classic talk "A Whole New World": https://www.destroyallsoftware.com/talks/a-whole-new-world https://www.destroyallsoftware.com/talks/a-whole-new-world

joshka 4y ago

>So I'm pretty comfortable with my approach. And hey, next time you're in the Bay Area (or perhaps when I'm in your neck of the woods), lemme buy you beer and we can talk about why it gives you satisfaction to dump on other people's work. I love this response - stealing it :)

deleted 4y ago
[deleted]

stephc_int13 4y ago

To be frank, I think that the kind of work you are doing is necessary. In my opinion GUI is not yet something I would consider to be a "solved problem". Both from the API perspective and the rendering side, and compute shaders are indeed extremely promising and could be something close to an end-game in this space. This is why I read your articles. And I have absolutely zero interest in dumping on your work, but I have the feeling that a large part of this work is going to waste because of what I consider to be flaws in the ways you are approaching the problems, or maybe the way you advertise your approach, I can sense how discouraging it can be for other devs.

tayistay 4y ago
What, specifically do you think the flaws are? I think if you're going to make such criticism, it helps to give actionable specifics (i.e. constructive criticism). How is the approach discouraging to other devs?
- stephc_int13 4y ago
  I think that people who manage to achieve "impossible" tasks tend to be overly optimistic (and naïve) at first, consciously or not, this is a good trick to fuel their own motivation and that of others. On the contrary, knowing/talking too much about the difficulties can quickly kill the fun and motivation. He should talk about how great the end-goal will be and why it is important.
  - mwcampbell 4y ago
    Would you rather end-users continue to get stuck with applications that ignore important things like accessibility, that can block some people from getting or keeping a job, so as not to kill some developers' motivation? I'm glad Raph is talking about the difficulties so they (hopefully) won't be ignored or clumsily bolted on afterward this time. Edit to add: "Those who cannot learn from history are doomed to repeat it." --George Santayana I think the time for recklessly moving fast and breaking things in software, without taking into account known complexity and avoiding the mistakes of the past, is over. Our impact on the world, and the resulting responsibility, is simply too great for that.

jakear 4y ago

> it will be a UI toolkit capable of performance completely untouchable by the existing state of the art Could you give an example of an application where the bottleneck is UI code? In my experience the bottleneck is always either disk or network. Not trying to bash you, genuinely curious.

mwcampbell 4y ago
The fact that our modern computers still often don't feel amazingly fast is a perennial topic of griping on forums like this one. I'm sure the UI stack has something to do with that.
ratww 4y ago
There's plenty of apps that we developers use daily that fall into that. Apps like Teams, Slack and Jira can feel incredibly slow to a lot of people, even with everything already in memory and not waiting for anything from the network or disk. Typing messages, changing tabs, going to read a notification... Facebook sometimes takes a second to show characters you typed in a reply, and the characters often show up out of order (also zero network activity during it). A lot of complex WYSIWYG editors are also incredibly heavy, while simpler ones aren't. Sure, that might be technically a bandwidth problem: in this case probably RAM. It is solvable with faster computers/faster RAM. But it's slower or at least the same speed as it was in the past with slower machines. Since hardware got better, it has got to be something different in the software side. And it's not even about difficult things like Unicode Glyphs and Emojis, which are common canned responses when anyone says that software "is slower than N years ago". Those things are handled by the OS, not by Jira. And there are super fast apps that make use of them.
- jakear 4y ago
  I don't have access to such corporate software at the moment, but can you verify in devtools where the delay is actually coming from in the apps you mention? Leaping to "the stack is bad it needs to be rewritten in rust" is extreme, especially when apps like VS Code are generally regarded as non-slothy [1]. Or at least not more slothy than the domain requires [2]. I'd wager the developers are poorly incentivized - if something satisfies the ticket and looks fast enough on their tiny test bed with state of the art hardware and great network (perhaps even localhost), it ships. On the other hand I don't consider something good enough to ship until it is fast enough on testbeds orders of magnitude larger than what I expect a "normal" workflow would include. The process of going from "works" to "fast enough for people using 100x larger inputs than I expect" almost never involves "rewrite in rust"[3], but instead "cache cleverly", "debounce discretely", and if all else fails "sit down and ponder on novel algorithms and data structures". These are all operations that are just as easy, if not easier, in high level languages as compared to rust. [1]: Full disclosure I was paid to write vscode for a period. When VS Code is slow, and it 100% is at times, the root cause is almost always an extension blocking progress for some dumb reason. This absolutely blows, but isn't a problem rust would solve - indeed extensions can already invoke rust. [2]: inb4 "but sublime!": Running experiments, I've found sublime to in fact be slower than a fresh VS Code install at working with very large files. Of course when you have extensions trying to do dumb stuff with the big files, VS Code can get worthlessly slow. Again not a problem rust would solve. Try it: make a 5M line file, click it open it in Sublime, then in VS Code. On my machine VS Code opens it well before Sublime can. [3]: Yes there are some times when rewriting in rust is appropriate, for instance VS Code's search is ripgrep - but rust isn't handling the UI at all, it's running in a separate thread doing what it does best (multithreaded systems programming), while the main renderer thread is doing what it does best (rendering). This is the way forward for the truly "inner loop" code, IMO.

runnerup 4y ago

Last part might be worth an edit. I saw his comment as constructive well meant criticism rather than callous, thoughtless, or mean spirited. Also love the work you do and enjoy following it. Your treatises on Oklab are the #1 place I send programmers of all skill levels to understand color.

sfink 4y ago
As another anecdatum, I thought the last part was beautifully done and completely deserved. The dismissive complaint was not without merit, but wow did it assume that the commenter's use cases are all that matter and all that need be considered. The world of UIs is much, much, much bigger than that, and awash in unsolved or badly-solved problems that matter a lot to many people. I agree with the complaint, fwiw, when applied to an important subset of the design space. I even think it's useful to try to understand where the limits of that subset are, and why. But saying that exploration is pointless because we're all happy living on this here big island and there's nowhere else that could possibly be better so why bother looking, it's all good except we still don't know why people keep dropping dead, but that's an acceptable drawback to what is otherwise a paradise on Earth—hang on a sec while I scrape off these leeches, they're so silly sometimes—and the people who think otherwise are just malcontents who ought to be out catching fish for the rest of us to enjoy.

aaaaaaaaaaab 4y ago

>it will be a UI toolkit capable of performance completely untouchable by the existing state of the art Why exactly? What are the current state of the art UI toolkits leaving on the table performance-wise?

raphlinus 4y ago
I'll have a lot more to say about this as I gather quantitative performance data. But it's a good question. I expect the big wins are: fast 2D (vector with blends and so on) rendering with compute shaders, multithreaded creation of expensive resources like image decompression and text layout, pushing incremental reactivity all the way from app logic to GPU (as opposed to needlessly redoing work), and of course just using a fast, non-GC language.
- aaaaaaaaaaab 4y ago
  What do you mean by “pushing incremental reactivity all the way from app logic to GPU (as opposed to needlessly redoing work)”?
  - raphlinus 4y ago
    From a computation perspective, UI is fundamentally an incremental computation engine. Most elements are not changing from frame to frame, so you can either recompute and re-render, or be smarter about only propagating deltas. I'd like to propagate those deltas all the way to the GPU, so you reuse lots of things from the previous frame if they haven't been invalidated. I'll be writing about this in considerably more detail; stay tuned.
    - aaaaaaaaaaab 4y ago
      Wait, are you talking about deltas as in damage rects?
      - infogulch 4y ago