18 ms·

So you want to write a GUI framework

263 points by mwcampbell 5y ago

Joker_vD 5y ago
> sooner or later, someone is going to want to display some HTML (or an actual website!) within their application. We’d really rather not bundle an entire browser engine to accomplish this Theoretically speaking, yes, but in practice it seems most people just gave up and simply ship their apps with embedded Chromium, which is why now almost every single piece of software has installer that's at least 500 MB large.
- tyingq 5y ago
  If I watch my non-techie family members use their computers, they hardly use anything that isn't either in the browser or using electron. I used to see a few things like Acrobat Reader (now killed off by pdf.js), Word (the college seems to now launch the web version by default), Skype (now Electron MS Teams) etc.
  - bodge5000 5y ago
    To be fair, the only non-electron (or browser) thing I can think of that I use for work on a regular basis is the terminal. VSCode is electron, Joplin is electron, Teams is electron. To be honest I've done a complete 180 in my views on electron. As long as the application is well made and optimised (which, to be fair, is somewhat rare) I often prefer it to native. People talk about vendor lock-in when it comes to their data, but very few people seem to consider OS lock-in as another, just as important factor. If I'm using a native app that only runs on windows, that'll discourage me from moving over the linux should the want ever arise (and it does). Obviously there are exceptions like Sublime, which are both native and cross-platform, but like the well made electron apps, they're an exception to the rule
    - forgotpwd16 5y ago
      Do you find Teams well made and optimized? Eveyone that I know hates it.
      - fartcannon 5y ago
        It's trash, but mostly because it's got that standard Microsoft schmear on it, not because of electron.
      - bodge5000 5y ago
        God no, I was referring more to vscode (and obsidian, which I use personally) Don't get me wrong, its better, you can always instantly tell how much work has been put into an electron app, especially on windows (and when you have a dark theme on) because the poorly made ones will use a default white titlebar, whereas those with some effort will have the titlebar match the rest of the application. So Teams has that going for it. But optimisation wise, yeh, its not pretty. I used to use Linux at work, and it was soley responsible for my twice-weekly crashes (I knew because sometimes, if I were quick enough, I could stop the crash before my laptop slowed to a halt by force closing teams). On Windows, its better, but thats not saying much.
      - tyingq 5y ago
        They are moving it to Edge Webview and off of Electron. It's supposed to use 1/2 the memory. https://tomtalks.blog/2021/06/microsoft-teams-2-0-will-use-half-the-memory-dropping-electron-for-edge-webview2/ https://tomtalks.blog/2021/06/microsoft-teams-2-0-will-use-h...
  - fartcannon 5y ago
    The fact that this is relevant to the discussion means we aren't trying to advance technology anymore, just make more sales.
- pjmlp 5y ago
  And why Chrome market share keeps increasing.
  - Ygg2 5y ago
    Luckily, Google our beneficient overlord would never use it for anything nefarious after all their motto is "Do No Evil" /sarc.
    - pjerem 5y ago
      was*
      - CRConrad 5y ago
        I think that was included in the "/sarc". [EDIT:] I thought it was for quite a while after it wasn't any more. [/EDIT]
- Ygg2 5y ago
  > now almost every single piece of software has an installer that's at least 500 MB large. First, that's 500MB initially. It goes up from there. Second, that works only if a small number of apps are bundling Chrome, and RAM can keep up; once too many Electron apps promulgate, people will start installing fewer apps. If your I dunno, weather widget starts consuming 1GB to show me current weather, I'm going to trash it, if it conflicts with my IDE.
- BiteCode_dev 5y ago
  Well you can use the installed browser on the OS if it's just to display content.
  - CRConrad 5y ago
    A) But then you need to install a web server component with your app. (And make sure that starts with it, keep track of that process staying alive, etc.) B) As was mentioned, I think both in the article itself and several comments here, then you get to contend with the same problem Web devs do: Browser differences.
- OskarS 5y ago
  The article deals with this in a later section and has an interesting discussion about the tradeoffs. Note that you don't have to ship Chromium with every app, you can also go use web rendering engines supplied by the platforms.
- j1elo 5y ago
  > sooner or later, someone is going to want to display some HTML (or an actual website!) within their application. We’d really rather not bundle an entire browser engine to accomplish this I read that and cannot avoid thinking: yeah well, sooner or later someone is going to want to play an MP3 file. Should now all GUI toolkits include multimedia playback support? Someone is going to want to display and edit a PSD file, does it mean the GUI toolkit should be able to handle the Photoshop file format? I get where it's going (I've been a GUI developer myself, working with Qt which does include a browser engine), but to be honest I think that already crosses the line from GUI framework to do-it-all framework.
  - formerly_proven 5y ago
    Qt has a reduced version of HTML and CSS that it uses for Widgets styling, which is why you can make words bold, or underlined or whatever, in labels and pretty much everywhere in the GUI. This is very useful to have. Many more basic toolkits include no such functionality, or have a divide between "plain text" widgets, which can do no formatting at all, and a real browser-ish HTML widget. (There's also capital-R-T Rich Text used in some parts of Windows but you really don't want to go there)
  - hulitu 5y ago
    You forgot rhe most important: to send email. (jwz)
  - Spivak 5y ago
    I mean yes? Because either the toolkit does it or the platform does it and the toolkit wraps the functionality. I think it’s easier to flip the script. If you didn’t include the ability for your toolkit to display HTML with a browser widget how do you imagine your user proceeding? I don’t think most end-programmers would be expected to know how to take WebKit and maybe a JS engine and make it into a custom widget in your framework.
- graderjs 5y ago
  My solution to this (it's been done before), is to use the existing browser engine (not the system webview) installed. So far I only utilize Chrome, but as the way I connect to it is over the Chrome DevTools protocol which is somewhat fluent with the Remote Debugging Protocol[0] that Firefox is doing, this is a reasonable approach. So far my "tool" to do this is simply a template repository with some conveniences, providing in essence a skeleton for these types of apps. I hope to flesh this out a little more, and expose a much richer API, as well as convert some of my existing popular apps (like 22120[1]) to the "framework". The benefit of this is Graderjs has a built in 'app builder' that can create a cross-platform binary (excluding or ignoring the necessity (on MacOS) and near-necessity (on Windows) to sign your executable somehow, that lets you display your UI in JS/HTML/CSS using the already installed browser engine, as well as run code in NodeJS and using the rich APIs[2] of the browser engine itself. I'm really happy with this project and think that, even tho it's small now, it will in time become my most popular and powerful one: even bigger than my remote browser and popular web archiver. Just give it time! :) [0]: https://firefox-source-docs.mozilla.org/remote/index.html https://firefox-source-docs.mozilla.org/remote/index.html [1]: https://github.com/i5ik/22120 https://github.com/i5ik/22120 [2]: https://chromedevtools.github.io/devtools-protocol/tot/Browser/ https://chromedevtools.github.io/devtools-protocol/tot/Brows... The GraderJS: https://github.com/i5ik/graderjs https://github.com/i5ik/graderjs
- mwcampbell 5y ago
  > which is why now almost every single piece of software has installer that's at least 500 MB large. I know pessimism about software is in fashion, but exaggerating the problem doesn't help. The actual size of the latest version of Electron (on Windows) is 181 MB uncompressed, 77 MB zip-compressed, and 51 MB 7z-compressed. Not great, but again, let's not make it look worse than it is. Also, there is hope. Microsoft's WebView2 [1] means that Windows has a modern, shared, embeddable web rendering engine for the first time in several years. And Tauri [2] is making it easier for web developers to take advantage of WebView2 on Windows, and WebKit on Mac and Linux, to create lightweight native applications using web technologies. [1]: https://docs.microsoft.com/en-us/microsoft-edge/webview2/ https://docs.microsoft.com/en-us/microsoft-edge/webview2/ [2]: https://tauri.studio/en/ https://tauri.studio/en/
  - laurent123456 5y ago
    Well bashing Electron is popular, this is why even a blatantly incorrect comment is automatically the top one as long as they are denigrating Electron.
  - thrower123 5y ago
    At the moment, realistically using WebView2 requires shipping a WebView2 Runtime, either a fixed version or a bootstrapper. That's still over a hundred MB. I don't quite understand why that isn't being included with Edgeium, maybe it will be there in the future, but as of now, having Edge installed is no guarantee that any software trying to use WebView2 will work. I've got too many scars from needing to ship shared Microsoft runtimes in the past to be entirely happy about this current situation.
    - wila 5y ago
      I've got a lot of scars from MS... but the bootstrapper is 1.7MB. It will then pull in WebView2. WebView2 btw is awesome (I've spent a lot of time with it already, see my profile if you want to know why)
    - devwastaken 5y ago
      Webview is not a replacement. Webviews for GUIs is the worst of all worlds. You're packaging up two separate event based runtimes, chromium and node, and have no ability to interface with V8, meaning youre actually losing significant performance by having to serialize/deserislize data from node, to the browsers window, then deserislized on message in the browsers packaged JS engine. It makes no sense to treat your desktop GUIs as closed sandboxes, what's the threat model? Electron is not simply chromium, it has a bunch of optimizations, fixes, and prebundled functionality.
- wila 5y ago
  The article also talks about MS Edge not being crossplatfom. Well.. that's about to change [1]. There's a beta for Linux, macOS, Android & iOS. What surprised me most about all this is that MS even provides the Edge for retired versions of their own OS (Windows 7 & Server 2008). I'm very impressed by that. For Windows you don't have to include the WebView2 installer, your installer, can basically include a "stub", the evergreen bootstrapper [2], that then goes out to the internet and grabs the latest version. edit: The great thing about WebView2 is that Microsoft keeps their control up to date. (At least on Windows, not sure what that is going to be on other platforms, time will tell) [1] https://www.microsoftedgeinsider.com/en-us/download/?platform=linux https://www.microsoftedgeinsider.com/en-us/download/?platfor... [2] https://developer.microsoft.com/en-us/microsoft-edge/webview2/ https://developer.microsoft.com/en-us/microsoft-edge/webview...
  - nvrspyx 5y ago
    Just to add, the Evergreen option also doesn't require multiple installs if multiple programs are using it. The client machine just needs a single install that all programs will share [1]. Further, if the client also has Microsoft Edge installed and up-to-date, Edge and WebView2 will be hard-linked allowing them to share the same libraries and processes, further reducing the performance, memory, and storage footprints. [1]: https://docs.microsoft.com/en-us/microsoft-edge/webview2/concepts/distribution#the-evergreen-runtime-distribution-mode https://docs.microsoft.com/en-us/microsoft-edge/webview2/con...
    - wila 5y ago
      I'm hoping that it improves performance ;) Thanks for expanding on that part. It's really nice how Microsoft deals with these updates.
  - forgotpwd16 5y ago
    >I'm very impressed by that. Why? Chrome supports those platforms. There's no reason for Edge to not.
    - 5y ago
- devwastaken 5y ago
  Chromium is not 500mb, and at this point the discussion is a dead horse. Every time it's brought up someone makes up new numbers because they've never worked with it nor understand why it's used. If you don't know, don't comment, it's that simple. The actual bundled size is about 120MB, it can be lowered. Actual memory usage is about the same but is relative on how much memory your system has overall.
- geodel 5y ago
  Yeah. But what are the options really for poor multi-billion dollar companies while making GUI based applications. They all have "carefully evaluated strategic options" and come to same conclusion: Electron is the way to go.
- PaulDavisThe1st 5y ago
  We didn't give up.... we use one these three when someone needs to be shown a web page: ShellExecuteW(NULL, L"open", (LPCWSTR)wuri, NULL, NULL, SW_SHOWNORMAL); [[NSWorkspace sharedWorkspace] openURL:nsurl]; ::execlp ("xdg-open", "xdg-open", s.c_str(), (char*)NULL);
lukastyrychtr 5y ago
Great overview, i am thankful that accessibility was mentioned as well.
- prewett 5y ago
  Why are you thankful? Do you need accessibility, and if so, what do you use? Inevitably someone complains about lack of accessibility of a custom UI toolkit, which is frustrating, because something that helps 99% of people is still an improvement and still worth doing.
  - kall 5y ago
    The argument would be that a new GUI toolkit will capture applications that would otherwise be made with an older, accessible toolkit. In that sense, it‘s a regression, not just a clear improvement. I do agree about the idea though. We shouldn't shut down experiments just because they don‘t have a plan for it yet. I do see some of that on twitter etc. But once you‘re shipping software that users can‘t switch away from, because of network effects or because it‘s at work, it‘s not ok. Also, yeah, I (not OP) depend on it.
- wila 5y ago
  Accessibility is the special focus of mwcampbell :)
  - mwcampbell 5y ago
    To be clear though, I didn't write the post; I just spotted it and submitted it.
    - wila 5y ago
      Fair enough, thanks for the clarification!
terhechte 5y ago
Great list of the complexity encompassing a proper "GUI" framework. I feel like text / rich text / unicode alone is a huge project (if you include editing, especially of large documents). However, one thing not mentioned in this list (which I feel is only becoming more important) is that a modern GUI framework also might want to support iPad / iOS / Android. This complexifies the undertaking by a huge amount as the differences between macOS/win/linux are much smaller than the distance between, say, iOS and macOS. Apple has been going into this direction with SwiftUI where the same code can be run on different systems and behaves differently depending on the platform. However, by doing so, this "common denominator" of all platforms automatically reduces the flexibility of the framework as every "component" needs to be cut wide enough to intelligently behave correctly per platform.
- amelius 5y ago
  And then there is the question if your GUI framework is even allowed on iOS. For example, Firefox can't use its internal render engine on iOS, but is forced by Apple to use Safari's library.
  - terhechte 5y ago
    They can use their internal render engine. What they can't use is their Javascript framework because a modern JIT requires taking compiled javascript code in memory and marking it as executable (e.g. taking any memory page of data and telling the system that this is code now, thus allowing to execute it). This goes against the iOS security mechanism / sandbox because a bug here would allow executing any remote code in the context of the current process. You can even ship a custom Javascript engine on iOS if you disable JIT, it will just be slow. Or, you do it like iSH (https://ish.app https://ish.app) and implement custom execution via a hard-to-debug genius assembly. As an exempla, Ejecta is a custom Javascript library for iOS - just without the JIT to make it fast. https://impactjs.com/ejecta https://impactjs.com/ejecta
    - modeless 5y ago
      > You can even ship a custom Javascript engine on iOS Yes but you can't run code downloaded from the internet or supplied by the user. So you are prohibited from building your own browser, even without JIT. Unless you are Roblox for some reason, then you can run all the downloaded code you want! Sucks to be a Roblox competitor, sorry no running downloaded code for you.
deleted 5y ago
[deleted]
pedrovhb 5y ago
Can anyone recommend resources on understanding the inner workings and abstractions of reactive/declarative application models, such as Flutter and Vue? It seems to be the latest rage (see Google adopting it for native Android development with Jetpack Compose) and I've been thinking about this lately, but all resources I can find explain how to use it, but not the building blocks of the system underneath.
- ratww 5y ago
  Those explain React, but the concepts are similar: https://pomb.us/build-your-own-react/ https://pomb.us/build-your-own-react/ https://medium.com/@sweetpalma/gooact-react-in-160-lines-of-javascript-44e0742ad60f https://medium.com/@sweetpalma/gooact-react-in-160-lines-of-... The main difference between React and other frameworks is how they detect state changes. React uses a function (like setState), Vue uses proxies, Angular uses Rx observables, etc.
- jamil7 5y ago
  When I used to do React work I remember reading this article which might help you out. https://overreacted.io/react-as-a-ui-runtime/ https://overreacted.io/react-as-a-ui-runtime/ This SwiftUI book is quite short but takes you through a fair bit of how it works under the hood. https://www.objc.io/books/thinking-in-swiftui/ https://www.objc.io/books/thinking-in-swiftui/
karteum 5y ago
Cairo/Skia are not the only alternatives for Linux. For example, one pretty nice cross-platform GPU-enabled stack : https://github.com/bkaradzic/bgfx https://github.com/bkaradzic/bgfx https://github.com/memononen/nanovg https://github.com/memononen/nanovg https://github.com/wjakob/nanogui https://github.com/wjakob/nanogui
- Const-me 5y ago
  BGFX is a general-purpose 3D graphics engine, not a GUI nor vector graphics framework. Nanovg is an awesome vector graphics library, but has limitations. (1) no ClearType, I fixed in my fork: https://github.com/Const-me/nanovg https://github.com/Const-me/nanovg (2) The only way to get AA is hardware MSAA, unfortunately many popular platforms like Raspberry Pi don’t have good enough hardware to do it fast enough. Nanogui is built on top of Nanovg, shares the limitations. I agree with the OP that Cairo and Skia are the only viable ones for Linux. It’s sad because Windows has Direct2D for decades now (introduced in Vista), and unlike 2006, now in 2021 Linux actually has all the lower-level pieces to implement a comparable equivalent. Here’s a proof of concept: https://github.com/Const-me/Vrmac#vector-graphics-engine https://github.com/Const-me/Vrmac#vector-graphics-engine
Const-me 5y ago
> it is genuinely hard to write an abstraction that provides adequate control of advanced GPU features (such as the compute capabilities) across subtly different low-level APIs. That’s a solved problem in C++, see this library: http://diligentgraphics.com/diligent-engine/ http://diligentgraphics.com/diligent-engine/ > The rasterization techniques used in 3D are poorly suited to 2D tasks like clipping to vector paths or antialiasing That’s subjective, I think these techniques are awesome fit for 2D. See this library: https://github.com/Const-me/Vrmac#2d-graphics https://github.com/Const-me/Vrmac#2d-graphics BTW I have recently documented my antialiasing algorithm: https://github.com/Const-me/Vrmac/blob/master/Vrmac/Draw/VAA.md https://github.com/Const-me/Vrmac/blob/master/Vrmac/Draw/VAA... > these traditional techniques can start to perform very badly in 2D once there are lots of blend groups or clip regions involved, since each needs its own temporary buffer and draw call. One doesn’t necessarily need temporary buffers or draw calls for that, can also do in the shaders, merging into larger draw calls. > What this comes down to is instructing the operating system to embed a video or 3D view in some region of our window, and this means interacting with the compositor. That indeed works in practice, but I don’t believe the approach is good. Modern GUI frameworks are using 3D GPUs exclusively. They don’t need much efforts to integrate 3D-rendered content. As for the video, one only needs a platform API to decode and deliver frames in GPU textures. Microsoft has such API in the OS: https://docs.microsoft.com/en-us/windows/win32/api/mfmediaengine/nf-mfmediaengine-imfmediaengineclassfactory-createinstance https://docs.microsoft.com/en-us/windows/win32/api/mfmediaen... Once upon a time wanted to do the same on embedded Linux, wasn’t easy but still doable on top of V4L2 kernel calls: https://github.com/Const-me/Vrmac/tree/master/VrmacVideo https://github.com/Const-me/Vrmac/tree/master/VrmacVideo
- TazeTSchnitzel 5y ago
  > [hardware-accelerated graphics abstraction is] a solved problem in C++ I think that would strike most people who work with computer graphics as a highly questionable statement. There's many, many different graphics abstractions out there for C and C++ alone, and none of them is The solution that is ideal for everything. > > The rasterization techniques used in 3D are poorly suited to 2D tasks like […] antialiasing > That’s subjective, I think these techniques are awesome fit for 2D. […] I have recently documented my antialiasing algorithm Your comment actually reinforces the original author's point. You're using a completely different kind of anti-aliasing technique for 2D than would be used for 3D! > Modern GUI frameworks are using 3D GPUs exclusively. That doesn't make it easy. You don't control what the system UI framework is doing with the GPU.
bwanab 5y ago
Out of all the things mentioned in the article, it doesn’t mention tables which to me are one of the unsung heroes of all good GUI frameworks. They are so deceptively simple looking and yet, once you peek under the hood, the amount of complexity is daunting. E.G. do you allow all widget types to be in table cells? What happens when there are more items than can be displayed? Now you need scroll bars. What happens when a table is resized? It goes on and on and most solutions have negative consequences for a given use case, but to make it meet all the use cases, one ends up with such a complex set up that nobody wants to use it.
- AtlasBarfed 5y ago
  HTML+CSS+tables is a massive weapon in the GUI wars, probably responsible for most of the power of the web and its current conquest/war/campaign into desktop via electron and associated web-as-app frameworks. And with good reason, the ludicrous complexity necessary to specify presentation details is mostly and with an industrial scale army of programmer/designer knowledge in HTML+CSS. The path forward in GUI frameworks at a minimum is to leverage/support HTML+CSS which is such a huge lift to support. That leaves the issue of inherent ties to Javascript in HTML+CSS land, versus the language you want to support, and the still-nascent integration of HTML+CSS "document/page-oriented" language vs the window style of a true GUI app. I do not envy a new language having to produce a ground-up GUI framework. Arguably harder to do than muscling a new programming language into relevance.
  - ridiculous_fish 5y ago
    Tables are one of the weakest parts of HTML. The lazy loading in e.g. Chrome History or Yahoo Mail is an awful, jumpy experience. Even the very best implementations, like Fastmail, still break basic browser features including Find and Print.
    - AtlasBarfed 5y ago
      I'd agree for naive uses of HTML tables, but HTML+CSS incorporates sizing, overflow rules, etc. Which is what you'd need in any general arbitrary-content infinite-stream model. What is the alternative?
nitwit005 5y ago
This is underestimating the UI problems of video games. Open up something like World of Warcraft, or Civ 6. There's a fairly complete windowing system, IME, accessibility features, and even the means for users to build custom UIs. It's still common to see games list some UI framework in the opening credits.
- mwcampbell 5y ago
  > accessibility features Including screen reader support? I'd love to know about any mainstream game that has that. BTW, I didn't write the OP, just came across it and thought it was worth sharing.
  - nitwit005 5y ago
    Usually it's something like hotkeys for everything and reading the chat: https://worldofwarcraft.com/en-us/news/23691035/enhance-your-gameplay-accessibility-updates-now-live https://worldofwarcraft.com/en-us/news/23691035/enhance-your...
dbrueck 5y ago
It's really hard to beat an embedded browser if you're doing a desktop GUI app. Yeah, I know, I know, there are downsides, but the upsides are compelling: very mature technology, can be very performant for many types of apps, extremely customizable, great tooling is available, huge resource pool of proficient developers, and the UI itself is more or less automatically consistent across OS versions and platforms. Building a desktop app with an embedded browser always feels dirty, but honestly the alternative has always ended up being worse for stuff I've worked on. Buggy frameworks, missing features, cross platform support that is only 90% there, dealing with requirements for custom widgets, cost of bringing devs up to speed on it, etc. Interestingly, a compelling reason for a non-web approach used to be that having native look and feel was a big deal, but over the past decade it seems like many apps have eschewed this anyway. It seems like most programs built in GTK always looks dated, and on Windows things have morphed so far away from the win32 look and feel that nearly everything modern looks like it might be a web page anyway.
- guggle 5y ago
  > consistent across OS versions and platforms. This is an upside for the developer only. And a downside for the user as the GUI will probably not be consistent with its OS. > over the past decade it seems like many apps have eschewed this anyway And I regret it.
  - nawgz 5y ago
    > > consistent across OS versions and platforms. > This is an upside for the developer only. Strongly disagree. How is a consistent user experience enabling developers to support more platforms at a higher level of quality going to lead to worse end-user experience? To me: it's not, this is just anti-browser sentiment dressing itself up as enlightenment
    - geodel 5y ago
      > ... higher level of quality going to lead to worse end-user experience This is just a management speak where degradation in quality is called 'next generation'. It is worse because it is resource hog and slow even on high end machines. As a technical user I understand at least what is happening. On other hand non-technical folks are unable to articulate their frustrations which slick developers seems to take as users are satisfied.
rafale 5y ago
I'll add UI virtualization to the list of concerns, especially for list views with a lot of data. Creating a UI element for each item in the list consumes a lot of memory and CPU, so it makes sense to do it only for items in view + a buffer before/after. And recycle the UI elements as the user scroll. This becomes difficult with custom non linear and non fixed size layouts. It also complicates UI automation, accessibility and keyboard navigation.
winrid 5y ago
I always liked the actor approach: https://github.com/libgdx/libgdx/wiki/Scene2d https://github.com/libgdx/libgdx/wiki/Scene2d
jffhn 5y ago
>One approach, then, is to present a common API abstraction over top of these platform APIs For those interested in a similar topic, but with an opinion/taste cursor much more tilted towards pathological simplicity than towards feature completeness, I've been working on an API to abstract away the most basic features of GUI libraries, so as not to actually depend on any of them while still being able to make basic GUIs: https://github.com/jeffhain/jolikit/blob/master/README-BWD.md https://github.com/jeffhain/jolikit/blob/master/README-BWD.m... It's not in version 1.0 yet, but should still remain quite stable (or even frozen, or... dead ;) in the future. Before doing so I'm waiting on finishing the toolkit I'm building on top of it (which might involve also finishing the apps I'm building on top of the toolkit ;).
EamonnMR 5y ago
I had to build a fairly robust wrapper over a feature-poor GUI framework once, and it effectively sunk the project. I'd been cloning a different project which shipped in months because it used a native gui framework. Now I'm much more careful when choosing technology.
milesvp 5y ago
To expand just a little on the short paragraph talking about embedded and games. For those interested in these spaces, looking into Immediate Mode GUIs would make for good reading. Rendering the current state of the UI tends to be simpler with lower processing standard deviation than Event driven GUIs. The main tradeoff is that the average processing goes up, since you're redrawing every update. But with embedded it's your worst case that matters most if you have deadlines you can't miss. And in games lower frame rate standard deviation is often a very good tradeoff for lower average frame rate in terms of perceived smoothness.
- bob1029 5y ago
  IM GUIs are the way to go in my opinion. I've been playing around with some prototypes and they are dead simple compared to event-driven GUIs with multi-stage rendering pipelines, et. al. Just the reduction in complexity is a massive feature. Managing view state is almost fun with this approach. The performance can be incredible too if you do a little bit of hacking with caching certain expensive resources between frames (i.e. texture atlas computation). One trick I have employed is to process user events in "micro" batches that are 10-100uS wide. This allows for me to consolidate redraws across multiple logical user events. I have found this to be incredibly helpful in cases where inputs are sampled more frequently or otherwise occur at rates higher than output frames are produced. Driving redraws off individual events is probably a mistake if you want any level concurrency. You need to batch up the events and process them in bulk, then do redraws on the edge of each batch.
  - cmyr 5y ago
    There are some big challenges, here. One is IME, which requires storing intermediate state and information on the editing session, and communicating that with the platform; another is accessibility, which requires keeping a tree of elements that the user can currently navigate to, and which may include elements which are not currently on screen. So yes, maybe okay for games, but I’m unconvinced about imgui for GUI applications of the sort described in this post.
newsyyswen 5y ago
Interesting that the "7 GUIs" test hasn't been mentioned in the article or comments. IMO it's not a bad starting point for thinking about the bare minimum that a GUI framework might need to support, and for evaluating how easy one is to use. https://eugenkiss.github.io/7guis/tasks/ https://eugenkiss.github.io/7guis/tasks/
- raphlinus 5y ago
  I like 7guis, and we use it as a way to think about what we're doing. However, I think the focus of 7guis tends to be much more on the higher levels of the GUI stack, particularly how to express UI logic concisely, while this blog post is much more about the lower levels - how to actually interface with the large diversity of platform capabilities that's needed for "real" GUI.
malkia 5y ago
Wondering about docking, especially handling multi-monitor setup, preserving layout, etc. Support for mixed DPI monitors.
PicassoCTs 5y ago
Im your first/last user and i want to issue a complaint: 1) It lacks features! 2) Its full of bloat 3) Its to slow! 4) Its full of optimization induced edge cases! 5) Its not customizable enough! 6) The third party customizations are incompatible! 7) I should write a better GUI-Framework
markus_zhang 5y ago
I wrote a simple GUI using C++ a while ago, just for my map editor. It's primitive and only has the absolutely needed widgets (buttons, textbox, one-layer menu which is essentially a button and maybe a couple others). Then I realized how difficult it is to get one right. I think QT is easy to use and C# is another option on Windows.
- bob1029 5y ago
  C# is opening up a lot of doors for UI development these days. Blazor Desktop is something I am looking forward to urgently.
bsder 5y ago
Here's my "go to" questions for whether a "new" GUI framework doesn't suck: Can your GUI system build a Digital Audio Workstation? It doesn't matter how good your text handling is if everything is too unresponsive. Can your GUI system scale arbitrarily when I put it on a humungous monitor? It doesn't matter how good your text rendering is if I can't read it. Every single GUI system currently fails both of those. GUIs are currently stuck in a path-dependent local minimum that needs to be broken out of. It's not clear what the path forward is, but this article simply reinforces the broken status quo. No, you don't have to render HTML. No, you don't have to support application tabs. No, you don't have to support modal dialog boxes. And, no, you don't have to behave like native. You don't even have to support an "editor". About the only part I agree with is "If you support interactive text, it's a nightmare." For most people, this needs to be outsourced to a library because you will never get shaping, rendering, and input methods even close to correct. In fact, I might even go so far as to say that "99% of GUI misery is supporting interactive text--don't do that." These are the "embedded" GUI systems that the author so blithely dismisses in the first couple of sentences.
- cmyr 5y ago
  I’m not dismissive of embedded, it’s just a totally different problem set and is not the focus of this article. This article is about the pain points of integrating with the current major desktop platforms, period. I agree we’re in a funny local minima. I agree there’s room for experimentation. But also… for a general purpose GUI toolkit, text editing is not optional. Modal windows (alerts, file open/save, as well as just drop-down/selection lists) are really not optional. I’m also not arguing that everyone needs to do all these things from scratch. Very few people should be trying to reimplement IME. But you do need to support IME, or you are only making a toy.
Ericson2314 5y ago
Great piece! I'm saddened to think I might never have to do this sort of thing the job, when I have in fact been lucky to have had good occasion to shave many other sorts of yaks. One thing I find interesting on Linux vs the others is where the defaults come from. MS and Apple clearly can enforce things, but conventions are a bit more subject to negotiation. Especially with Wayland, it seems like the ability/motivatino to do more stuff client side is allowing less coordination to happen period.

263 points by mwcampbell 5y ago

> sooner or later, someone is going to want to display some HTML (or an actual website!) within their application. We’d really rather not bundle an entire browser engine to accomplish this Theoretically speaking, yes, but in practice it seems most people just gave up and simply ship their apps with embedded Chromium, which is why now almost every single piece of software has installer that's at least 500 MB large.

tyingq 5y ago
If I watch my non-techie family members use their computers, they hardly use anything that isn't either in the browser or using electron. I used to see a few things like Acrobat Reader (now killed off by pdf.js), Word (the college seems to now launch the web version by default), Skype (now Electron MS Teams) etc.
- bodge5000 5y ago
  To be fair, the only non-electron (or browser) thing I can think of that I use for work on a regular basis is the terminal. VSCode is electron, Joplin is electron, Teams is electron. To be honest I've done a complete 180 in my views on electron. As long as the application is well made and optimised (which, to be fair, is somewhat rare) I often prefer it to native. People talk about vendor lock-in when it comes to their data, but very few people seem to consider OS lock-in as another, just as important factor. If I'm using a native app that only runs on windows, that'll discourage me from moving over the linux should the want ever arise (and it does). Obviously there are exceptions like Sublime, which are both native and cross-platform, but like the well made electron apps, they're an exception to the rule
  - forgotpwd16 5y ago
    Do you find Teams well made and optimized? Eveyone that I know hates it.
    - fartcannon 5y ago
      It's trash, but mostly because it's got that standard Microsoft schmear on it, not because of electron.
    - bodge5000 5y ago
      God no, I was referring more to vscode (and obsidian, which I use personally) Don't get me wrong, its better, you can always instantly tell how much work has been put into an electron app, especially on windows (and when you have a dark theme on) because the poorly made ones will use a default white titlebar, whereas those with some effort will have the titlebar match the rest of the application. So Teams has that going for it. But optimisation wise, yeh, its not pretty. I used to use Linux at work, and it was soley responsible for my twice-weekly crashes (I knew because sometimes, if I were quick enough, I could stop the crash before my laptop slowed to a halt by force closing teams). On Windows, its better, but thats not saying much.
    - tyingq 5y ago
      They are moving it to Edge Webview and off of Electron. It's supposed to use 1/2 the memory. https://tomtalks.blog/2021/06/microsoft-teams-2-0-will-use-half-the-memory-dropping-electron-for-edge-webview2/ https://tomtalks.blog/2021/06/microsoft-teams-2-0-will-use-h...
- fartcannon 5y ago
  The fact that this is relevant to the discussion means we aren't trying to advance technology anymore, just make more sales.
pjmlp 5y ago
And why Chrome market share keeps increasing.
- Ygg2 5y ago
  Luckily, Google our beneficient overlord would never use it for anything nefarious after all their motto is "Do No Evil" /sarc.
  - pjerem 5y ago
    was*
    - CRConrad 5y ago
      I think that was included in the "/sarc". [EDIT:] I thought it was for quite a while after it wasn't any more. [/EDIT]
Ygg2 5y ago
> now almost every single piece of software has an installer that's at least 500 MB large. First, that's 500MB initially. It goes up from there. Second, that works only if a small number of apps are bundling Chrome, and RAM can keep up; once too many Electron apps promulgate, people will start installing fewer apps. If your I dunno, weather widget starts consuming 1GB to show me current weather, I'm going to trash it, if it conflicts with my IDE.
BiteCode_dev 5y ago
Well you can use the installed browser on the OS if it's just to display content.
- CRConrad 5y ago
  A) But then you need to install a web server component with your app. (And make sure that starts with it, keep track of that process staying alive, etc.) B) As was mentioned, I think both in the article itself and several comments here, then you get to contend with the same problem Web devs do: Browser differences.
OskarS 5y ago
The article deals with this in a later section and has an interesting discussion about the tradeoffs. Note that you don't have to ship Chromium with every app, you can also go use web rendering engines supplied by the platforms.
j1elo 5y ago
> sooner or later, someone is going to want to display some HTML (or an actual website!) within their application. We’d really rather not bundle an entire browser engine to accomplish this I read that and cannot avoid thinking: yeah well, sooner or later someone is going to want to play an MP3 file. Should now all GUI toolkits include multimedia playback support? Someone is going to want to display and edit a PSD file, does it mean the GUI toolkit should be able to handle the Photoshop file format? I get where it's going (I've been a GUI developer myself, working with Qt which does include a browser engine), but to be honest I think that already crosses the line from GUI framework to do-it-all framework.
- formerly_proven 5y ago
  Qt has a reduced version of HTML and CSS that it uses for Widgets styling, which is why you can make words bold, or underlined or whatever, in labels and pretty much everywhere in the GUI. This is very useful to have. Many more basic toolkits include no such functionality, or have a divide between "plain text" widgets, which can do no formatting at all, and a real browser-ish HTML widget. (There's also capital-R-T Rich Text used in some parts of Windows but you really don't want to go there)
- hulitu 5y ago
  You forgot rhe most important: to send email. (jwz)
- Spivak 5y ago
  I mean yes? Because either the toolkit does it or the platform does it and the toolkit wraps the functionality. I think it’s easier to flip the script. If you didn’t include the ability for your toolkit to display HTML with a browser widget how do you imagine your user proceeding? I don’t think most end-programmers would be expected to know how to take WebKit and maybe a JS engine and make it into a custom widget in your framework.
graderjs 5y ago
My solution to this (it's been done before), is to use the existing browser engine (not the system webview) installed. So far I only utilize Chrome, but as the way I connect to it is over the Chrome DevTools protocol which is somewhat fluent with the Remote Debugging Protocol[0] that Firefox is doing, this is a reasonable approach. So far my "tool" to do this is simply a template repository with some conveniences, providing in essence a skeleton for these types of apps. I hope to flesh this out a little more, and expose a much richer API, as well as convert some of my existing popular apps (like 22120[1]) to the "framework". The benefit of this is Graderjs has a built in 'app builder' that can create a cross-platform binary (excluding or ignoring the necessity (on MacOS) and near-necessity (on Windows) to sign your executable somehow, that lets you display your UI in JS/HTML/CSS using the already installed browser engine, as well as run code in NodeJS and using the rich APIs[2] of the browser engine itself. I'm really happy with this project and think that, even tho it's small now, it will in time become my most popular and powerful one: even bigger than my remote browser and popular web archiver. Just give it time! :) [0]: https://firefox-source-docs.mozilla.org/remote/index.html https://firefox-source-docs.mozilla.org/remote/index.html [1]: https://github.com/i5ik/22120 https://github.com/i5ik/22120 [2]: https://chromedevtools.github.io/devtools-protocol/tot/Browser/ https://chromedevtools.github.io/devtools-protocol/tot/Brows... The GraderJS: https://github.com/i5ik/graderjs https://github.com/i5ik/graderjs
mwcampbell 5y ago
> which is why now almost every single piece of software has installer that's at least 500 MB large. I know pessimism about software is in fashion, but exaggerating the problem doesn't help. The actual size of the latest version of Electron (on Windows) is 181 MB uncompressed, 77 MB zip-compressed, and 51 MB 7z-compressed. Not great, but again, let's not make it look worse than it is. Also, there is hope. Microsoft's WebView2 [1] means that Windows has a modern, shared, embeddable web rendering engine for the first time in several years. And Tauri [2] is making it easier for web developers to take advantage of WebView2 on Windows, and WebKit on Mac and Linux, to create lightweight native applications using web technologies. [1]: https://docs.microsoft.com/en-us/microsoft-edge/webview2/ https://docs.microsoft.com/en-us/microsoft-edge/webview2/ [2]: https://tauri.studio/en/ https://tauri.studio/en/
- laurent123456 5y ago
  Well bashing Electron is popular, this is why even a blatantly incorrect comment is automatically the top one as long as they are denigrating Electron.
- thrower123 5y ago
  At the moment, realistically using WebView2 requires shipping a WebView2 Runtime, either a fixed version or a bootstrapper. That's still over a hundred MB. I don't quite understand why that isn't being included with Edgeium, maybe it will be there in the future, but as of now, having Edge installed is no guarantee that any software trying to use WebView2 will work. I've got too many scars from needing to ship shared Microsoft runtimes in the past to be entirely happy about this current situation.
  - wila 5y ago
    I've got a lot of scars from MS... but the bootstrapper is 1.7MB. It will then pull in WebView2. WebView2 btw is awesome (I've spent a lot of time with it already, see my profile if you want to know why)
  - devwastaken 5y ago
    Webview is not a replacement. Webviews for GUIs is the worst of all worlds. You're packaging up two separate event based runtimes, chromium and node, and have no ability to interface with V8, meaning youre actually losing significant performance by having to serialize/deserislize data from node, to the browsers window, then deserislized on message in the browsers packaged JS engine. It makes no sense to treat your desktop GUIs as closed sandboxes, what's the threat model? Electron is not simply chromium, it has a bunch of optimizations, fixes, and prebundled functionality.
wila 5y ago
The article also talks about MS Edge not being crossplatfom. Well.. that's about to change [1]. There's a beta for Linux, macOS, Android & iOS. What surprised me most about all this is that MS even provides the Edge for retired versions of their own OS (Windows 7 & Server 2008). I'm very impressed by that. For Windows you don't have to include the WebView2 installer, your installer, can basically include a "stub", the evergreen bootstrapper [2], that then goes out to the internet and grabs the latest version. edit: The great thing about WebView2 is that Microsoft keeps their control up to date. (At least on Windows, not sure what that is going to be on other platforms, time will tell) [1] https://www.microsoftedgeinsider.com/en-us/download/?platform=linux https://www.microsoftedgeinsider.com/en-us/download/?platfor... [2] https://developer.microsoft.com/en-us/microsoft-edge/webview2/ https://developer.microsoft.com/en-us/microsoft-edge/webview...
- nvrspyx 5y ago
  Just to add, the Evergreen option also doesn't require multiple installs if multiple programs are using it. The client machine just needs a single install that all programs will share [1]. Further, if the client also has Microsoft Edge installed and up-to-date, Edge and WebView2 will be hard-linked allowing them to share the same libraries and processes, further reducing the performance, memory, and storage footprints. [1]: https://docs.microsoft.com/en-us/microsoft-edge/webview2/concepts/distribution#the-evergreen-runtime-distribution-mode https://docs.microsoft.com/en-us/microsoft-edge/webview2/con...
  - wila 5y ago
    I'm hoping that it improves performance ;) Thanks for expanding on that part. It's really nice how Microsoft deals with these updates.
- forgotpwd16 5y ago
  >I'm very impressed by that. Why? Chrome supports those platforms. There's no reason for Edge to not.
  - 5y ago
devwastaken 5y ago
Chromium is not 500mb, and at this point the discussion is a dead horse. Every time it's brought up someone makes up new numbers because they've never worked with it nor understand why it's used. If you don't know, don't comment, it's that simple. The actual bundled size is about 120MB, it can be lowered. Actual memory usage is about the same but is relative on how much memory your system has overall.
geodel 5y ago
Yeah. But what are the options really for poor multi-billion dollar companies while making GUI based applications. They all have "carefully evaluated strategic options" and come to same conclusion: Electron is the way to go.
PaulDavisThe1st 5y ago
We didn't give up.... we use one these three when someone needs to be shown a web page: ShellExecuteW(NULL, L"open", (LPCWSTR)wuri, NULL, NULL, SW_SHOWNORMAL); [[NSWorkspace sharedWorkspace] openURL:nsurl]; ::execlp ("xdg-open", "xdg-open", s.c_str(), (char*)NULL);

lukastyrychtr 5y ago

Great overview, i am thankful that accessibility was mentioned as well.

prewett 5y ago
Why are you thankful? Do you need accessibility, and if so, what do you use? Inevitably someone complains about lack of accessibility of a custom UI toolkit, which is frustrating, because something that helps 99% of people is still an improvement and still worth doing.
- kall 5y ago
  The argument would be that a new GUI toolkit will capture applications that would otherwise be made with an older, accessible toolkit. In that sense, it‘s a regression, not just a clear improvement. I do agree about the idea though. We shouldn't shut down experiments just because they don‘t have a plan for it yet. I do see some of that on twitter etc. But once you‘re shipping software that users can‘t switch away from, because of network effects or because it‘s at work, it‘s not ok. Also, yeah, I (not OP) depend on it.
wila 5y ago
Accessibility is the special focus of mwcampbell :)
- mwcampbell 5y ago
  To be clear though, I didn't write the post; I just spotted it and submitted it.
  - wila 5y ago
    Fair enough, thanks for the clarification!

terhechte 5y ago

Great list of the complexity encompassing a proper "GUI" framework. I feel like text / rich text / unicode alone is a huge project (if you include editing, especially of large documents). However, one thing not mentioned in this list (which I feel is only becoming more important) is that a modern GUI framework also might want to support iPad / iOS / Android. This complexifies the undertaking by a huge amount as the differences between macOS/win/linux are much smaller than the distance between, say, iOS and macOS. Apple has been going into this direction with SwiftUI where the same code can be run on different systems and behaves differently depending on the platform. However, by doing so, this "common denominator" of all platforms automatically reduces the flexibility of the framework as every "component" needs to be cut wide enough to intelligently behave correctly per platform.

amelius 5y ago
And then there is the question if your GUI framework is even allowed on iOS. For example, Firefox can't use its internal render engine on iOS, but is forced by Apple to use Safari's library.
- terhechte 5y ago
  They can use their internal render engine. What they can't use is their Javascript framework because a modern JIT requires taking compiled javascript code in memory and marking it as executable (e.g. taking any memory page of data and telling the system that this is code now, thus allowing to execute it). This goes against the iOS security mechanism / sandbox because a bug here would allow executing any remote code in the context of the current process. You can even ship a custom Javascript engine on iOS if you disable JIT, it will just be slow. Or, you do it like iSH (https://ish.app https://ish.app) and implement custom execution via a hard-to-debug genius assembly. As an exempla, Ejecta is a custom Javascript library for iOS - just without the JIT to make it fast. https://impactjs.com/ejecta https://impactjs.com/ejecta
  - modeless 5y ago
    > You can even ship a custom Javascript engine on iOS Yes but you can't run code downloaded from the internet or supplied by the user. So you are prohibited from building your own browser, even without JIT. Unless you are Roblox for some reason, then you can run all the downloaded code you want! Sucks to be a Roblox competitor, sorry no running downloaded code for you.

deleted 5y ago

[deleted]

pedrovhb 5y ago

Can anyone recommend resources on understanding the inner workings and abstractions of reactive/declarative application models, such as Flutter and Vue? It seems to be the latest rage (see Google adopting it for native Android development with Jetpack Compose) and I've been thinking about this lately, but all resources I can find explain how to use it, but not the building blocks of the system underneath.

ratww 5y ago
Those explain React, but the concepts are similar: https://pomb.us/build-your-own-react/ https://pomb.us/build-your-own-react/ https://medium.com/@sweetpalma/gooact-react-in-160-lines-of-javascript-44e0742ad60f https://medium.com/@sweetpalma/gooact-react-in-160-lines-of-... The main difference between React and other frameworks is how they detect state changes. React uses a function (like setState), Vue uses proxies, Angular uses Rx observables, etc.
jamil7 5y ago
When I used to do React work I remember reading this article which might help you out. https://overreacted.io/react-as-a-ui-runtime/ https://overreacted.io/react-as-a-ui-runtime/ This SwiftUI book is quite short but takes you through a fair bit of how it works under the hood. https://www.objc.io/books/thinking-in-swiftui/ https://www.objc.io/books/thinking-in-swiftui/

karteum 5y ago

Cairo/Skia are not the only alternatives for Linux. For example, one pretty nice cross-platform GPU-enabled stack : https://github.com/bkaradzic/bgfx https://github.com/bkaradzic/bgfx https://github.com/memononen/nanovg https://github.com/memononen/nanovg https://github.com/wjakob/nanogui https://github.com/wjakob/nanogui

Const-me 5y ago
BGFX is a general-purpose 3D graphics engine, not a GUI nor vector graphics framework. Nanovg is an awesome vector graphics library, but has limitations. (1) no ClearType, I fixed in my fork: https://github.com/Const-me/nanovg https://github.com/Const-me/nanovg (2) The only way to get AA is hardware MSAA, unfortunately many popular platforms like Raspberry Pi don’t have good enough hardware to do it fast enough. Nanogui is built on top of Nanovg, shares the limitations. I agree with the OP that Cairo and Skia are the only viable ones for Linux. It’s sad because Windows has Direct2D for decades now (introduced in Vista), and unlike 2006, now in 2021 Linux actually has all the lower-level pieces to implement a comparable equivalent. Here’s a proof of concept: https://github.com/Const-me/Vrmac#vector-graphics-engine https://github.com/Const-me/Vrmac#vector-graphics-engine

Const-me 5y ago

> it is genuinely hard to write an abstraction that provides adequate control of advanced GPU features (such as the compute capabilities) across subtly different low-level APIs. That’s a solved problem in C++, see this library: http://diligentgraphics.com/diligent-engine/ http://diligentgraphics.com/diligent-engine/ > The rasterization techniques used in 3D are poorly suited to 2D tasks like clipping to vector paths or antialiasing That’s subjective, I think these techniques are awesome fit for 2D. See this library: https://github.com/Const-me/Vrmac#2d-graphics https://github.com/Const-me/Vrmac#2d-graphics BTW I have recently documented my antialiasing algorithm: https://github.com/Const-me/Vrmac/blob/master/Vrmac/Draw/VAA.md https://github.com/Const-me/Vrmac/blob/master/Vrmac/Draw/VAA... > these traditional techniques can start to perform very badly in 2D once there are lots of blend groups or clip regions involved, since each needs its own temporary buffer and draw call. One doesn’t necessarily need temporary buffers or draw calls for that, can also do in the shaders, merging into larger draw calls. > What this comes down to is instructing the operating system to embed a video or 3D view in some region of our window, and this means interacting with the compositor. That indeed works in practice, but I don’t believe the approach is good. Modern GUI frameworks are using 3D GPUs exclusively. They don’t need much efforts to integrate 3D-rendered content. As for the video, one only needs a platform API to decode and deliver frames in GPU textures. Microsoft has such API in the OS: https://docs.microsoft.com/en-us/windows/win32/api/mfmediaengine/nf-mfmediaengine-imfmediaengineclassfactory-createinstance https://docs.microsoft.com/en-us/windows/win32/api/mfmediaen... Once upon a time wanted to do the same on embedded Linux, wasn’t easy but still doable on top of V4L2 kernel calls: https://github.com/Const-me/Vrmac/tree/master/VrmacVideo https://github.com/Const-me/Vrmac/tree/master/VrmacVideo

TazeTSchnitzel 5y ago
> [hardware-accelerated graphics abstraction is] a solved problem in C++ I think that would strike most people who work with computer graphics as a highly questionable statement. There's many, many different graphics abstractions out there for C and C++ alone, and none of them is The solution that is ideal for everything. > > The rasterization techniques used in 3D are poorly suited to 2D tasks like […] antialiasing > That’s subjective, I think these techniques are awesome fit for 2D. […] I have recently documented my antialiasing algorithm Your comment actually reinforces the original author's point. You're using a completely different kind of anti-aliasing technique for 2D than would be used for 3D! > Modern GUI frameworks are using 3D GPUs exclusively. That doesn't make it easy. You don't control what the system UI framework is doing with the GPU.

bwanab 5y ago

Out of all the things mentioned in the article, it doesn’t mention tables which to me are one of the unsung heroes of all good GUI frameworks. They are so deceptively simple looking and yet, once you peek under the hood, the amount of complexity is daunting. E.G. do you allow all widget types to be in table cells? What happens when there are more items than can be displayed? Now you need scroll bars. What happens when a table is resized? It goes on and on and most solutions have negative consequences for a given use case, but to make it meet all the use cases, one ends up with such a complex set up that nobody wants to use it.

AtlasBarfed 5y ago
HTML+CSS+tables is a massive weapon in the GUI wars, probably responsible for most of the power of the web and its current conquest/war/campaign into desktop via electron and associated web-as-app frameworks. And with good reason, the ludicrous complexity necessary to specify presentation details is mostly and with an industrial scale army of programmer/designer knowledge in HTML+CSS. The path forward in GUI frameworks at a minimum is to leverage/support HTML+CSS which is such a huge lift to support. That leaves the issue of inherent ties to Javascript in HTML+CSS land, versus the language you want to support, and the still-nascent integration of HTML+CSS "document/page-oriented" language vs the window style of a true GUI app. I do not envy a new language having to produce a ground-up GUI framework. Arguably harder to do than muscling a new programming language into relevance.
- ridiculous_fish 5y ago
  Tables are one of the weakest parts of HTML. The lazy loading in e.g. Chrome History or Yahoo Mail is an awful, jumpy experience. Even the very best implementations, like Fastmail, still break basic browser features including Find and Print.
  - AtlasBarfed 5y ago
    I'd agree for naive uses of HTML tables, but HTML+CSS incorporates sizing, overflow rules, etc. Which is what you'd need in any general arbitrary-content infinite-stream model. What is the alternative?

nitwit005 5y ago

This is underestimating the UI problems of video games. Open up something like World of Warcraft, or Civ 6. There's a fairly complete windowing system, IME, accessibility features, and even the means for users to build custom UIs. It's still common to see games list some UI framework in the opening credits.

mwcampbell 5y ago
> accessibility features Including screen reader support? I'd love to know about any mainstream game that has that. BTW, I didn't write the OP, just came across it and thought it was worth sharing.
- nitwit005 5y ago
  Usually it's something like hotkeys for everything and reading the chat: https://worldofwarcraft.com/en-us/news/23691035/enhance-your-gameplay-accessibility-updates-now-live https://worldofwarcraft.com/en-us/news/23691035/enhance-your...

dbrueck 5y ago

It's really hard to beat an embedded browser if you're doing a desktop GUI app. Yeah, I know, I know, there are downsides, but the upsides are compelling: very mature technology, can be very performant for many types of apps, extremely customizable, great tooling is available, huge resource pool of proficient developers, and the UI itself is more or less automatically consistent across OS versions and platforms. Building a desktop app with an embedded browser always feels dirty, but honestly the alternative has always ended up being worse for stuff I've worked on. Buggy frameworks, missing features, cross platform support that is only 90% there, dealing with requirements for custom widgets, cost of bringing devs up to speed on it, etc. Interestingly, a compelling reason for a non-web approach used to be that having native look and feel was a big deal, but over the past decade it seems like many apps have eschewed this anyway. It seems like most programs built in GTK always looks dated, and on Windows things have morphed so far away from the win32 look and feel that nearly everything modern looks like it might be a web page anyway.

guggle 5y ago
> consistent across OS versions and platforms. This is an upside for the developer only. And a downside for the user as the GUI will probably not be consistent with its OS. > over the past decade it seems like many apps have eschewed this anyway And I regret it.
- nawgz 5y ago
  > > consistent across OS versions and platforms. > This is an upside for the developer only. Strongly disagree. How is a consistent user experience enabling developers to support more platforms at a higher level of quality going to lead to worse end-user experience? To me: it's not, this is just anti-browser sentiment dressing itself up as enlightenment
  - geodel 5y ago
    > ... higher level of quality going to lead to worse end-user experience This is just a management speak where degradation in quality is called 'next generation'. It is worse because it is resource hog and slow even on high end machines. As a technical user I understand at least what is happening. On other hand non-technical folks are unable to articulate their frustrations which slick developers seems to take as users are satisfied.

rafale 5y ago

I'll add UI virtualization to the list of concerns, especially for list views with a lot of data. Creating a UI element for each item in the list consumes a lot of memory and CPU, so it makes sense to do it only for items in view + a buffer before/after. And recycle the UI elements as the user scroll. This becomes difficult with custom non linear and non fixed size layouts. It also complicates UI automation, accessibility and keyboard navigation.

winrid 5y ago

I always liked the actor approach: https://github.com/libgdx/libgdx/wiki/Scene2d https://github.com/libgdx/libgdx/wiki/Scene2d

jffhn 5y ago

>One approach, then, is to present a common API abstraction over top of these platform APIs For those interested in a similar topic, but with an opinion/taste cursor much more tilted towards pathological simplicity than towards feature completeness, I've been working on an API to abstract away the most basic features of GUI libraries, so as not to actually depend on any of them while still being able to make basic GUIs: https://github.com/jeffhain/jolikit/blob/master/README-BWD.md https://github.com/jeffhain/jolikit/blob/master/README-BWD.m... It's not in version 1.0 yet, but should still remain quite stable (or even frozen, or... dead ;) in the future. Before doing so I'm waiting on finishing the toolkit I'm building on top of it (which might involve also finishing the apps I'm building on top of the toolkit ;).

EamonnMR 5y ago

I had to build a fairly robust wrapper over a feature-poor GUI framework once, and it effectively sunk the project. I'd been cloning a different project which shipped in months because it used a native gui framework. Now I'm much more careful when choosing technology.

milesvp 5y ago

To expand just a little on the short paragraph talking about embedded and games. For those interested in these spaces, looking into Immediate Mode GUIs would make for good reading. Rendering the current state of the UI tends to be simpler with lower processing standard deviation than Event driven GUIs. The main tradeoff is that the average processing goes up, since you're redrawing every update. But with embedded it's your worst case that matters most if you have deadlines you can't miss. And in games lower frame rate standard deviation is often a very good tradeoff for lower average frame rate in terms of perceived smoothness.

bob1029 5y ago
IM GUIs are the way to go in my opinion. I've been playing around with some prototypes and they are dead simple compared to event-driven GUIs with multi-stage rendering pipelines, et. al. Just the reduction in complexity is a massive feature. Managing view state is almost fun with this approach. The performance can be incredible too if you do a little bit of hacking with caching certain expensive resources between frames (i.e. texture atlas computation). One trick I have employed is to process user events in "micro" batches that are 10-100uS wide. This allows for me to consolidate redraws across multiple logical user events. I have found this to be incredibly helpful in cases where inputs are sampled more frequently or otherwise occur at rates higher than output frames are produced. Driving redraws off individual events is probably a mistake if you want any level concurrency. You need to batch up the events and process them in bulk, then do redraws on the edge of each batch.
- cmyr 5y ago
  There are some big challenges, here. One is IME, which requires storing intermediate state and information on the editing session, and communicating that with the platform; another is accessibility, which requires keeping a tree of elements that the user can currently navigate to, and which may include elements which are not currently on screen. So yes, maybe okay for games, but I’m unconvinced about imgui for GUI applications of the sort described in this post.

newsyyswen 5y ago

Interesting that the "7 GUIs" test hasn't been mentioned in the article or comments. IMO it's not a bad starting point for thinking about the bare minimum that a GUI framework might need to support, and for evaluating how easy one is to use. https://eugenkiss.github.io/7guis/tasks/ https://eugenkiss.github.io/7guis/tasks/

raphlinus 5y ago
I like 7guis, and we use it as a way to think about what we're doing. However, I think the focus of 7guis tends to be much more on the higher levels of the GUI stack, particularly how to express UI logic concisely, while this blog post is much more about the lower levels - how to actually interface with the large diversity of platform capabilities that's needed for "real" GUI.

malkia 5y ago

Wondering about docking, especially handling multi-monitor setup, preserving layout, etc. Support for mixed DPI monitors.

PicassoCTs 5y ago

Im your first/last user and i want to issue a complaint: 1) It lacks features! 2) Its full of bloat 3) Its to slow! 4) Its full of optimization induced edge cases! 5) Its not customizable enough! 6) The third party customizations are incompatible! 7) I should write a better GUI-Framework

markus_zhang 5y ago

I wrote a simple GUI using C++ a while ago, just for my map editor. It's primitive and only has the absolutely needed widgets (buttons, textbox, one-layer menu which is essentially a button and maybe a couple others). Then I realized how difficult it is to get one right. I think QT is easy to use and C# is another option on Windows.

bob1029 5y ago
C# is opening up a lot of doors for UI development these days. Blazor Desktop is something I am looking forward to urgently.

bsder 5y ago

Here's my "go to" questions for whether a "new" GUI framework doesn't suck: Can your GUI system build a Digital Audio Workstation? It doesn't matter how good your text handling is if everything is too unresponsive. Can your GUI system scale arbitrarily when I put it on a humungous monitor? It doesn't matter how good your text rendering is if I can't read it. Every single GUI system currently fails both of those. GUIs are currently stuck in a path-dependent local minimum that needs to be broken out of. It's not clear what the path forward is, but this article simply reinforces the broken status quo. No, you don't have to render HTML. No, you don't have to support application tabs. No, you don't have to support modal dialog boxes. And, no, you don't have to behave like native. You don't even have to support an "editor". About the only part I agree with is "If you support interactive text, it's a nightmare." For most people, this needs to be outsourced to a library because you will never get shaping, rendering, and input methods even close to correct. In fact, I might even go so far as to say that "99% of GUI misery is supporting interactive text--don't do that." These are the "embedded" GUI systems that the author so blithely dismisses in the first couple of sentences.

cmyr 5y ago
I’m not dismissive of embedded, it’s just a totally different problem set and is not the focus of this article. This article is about the pain points of integrating with the current major desktop platforms, period. I agree we’re in a funny local minima. I agree there’s room for experimentation. But also… for a general purpose GUI toolkit, text editing is not optional. Modal windows (alerts, file open/save, as well as just drop-down/selection lists) are really not optional. I’m also not arguing that everyone needs to do all these things from scratch. Very few people should be trying to reimplement IME. But you do need to support IME, or you are only making a toy.

Ericson2314 5y ago

Great piece! I'm saddened to think I might never have to do this sort of thing the job, when I have in fact been lucky to have had good occasion to shave many other sorts of yaks. One thing I find interesting on Linux vs the others is where the defaults come from. MS and Apple clearly can enforce things, but conventions are a bit more subject to negotiation. Especially with Wayland, it seems like the ability/motivatino to do more stuff client side is allowing less coordination to happen period.