6 ms·

>I think modern OS stacks are in a similar boat, at least Windows and macOS. I come from a background as a EEE mainly writing embedded code before somehow stum

>I think modern OS stacks are in a similar boat, at least Windows and macOS. I come from a background as a EEE mainly writing embedded code before somehow stumbling into frontend code (in win32, Cocoa and Qt) at my old job. From my experience, OS-Level APIs definitely have more in common with Electron or Qt than they do "bare metal". To make the top left pixel red on a bare metal system or RTOS, you simply write `framebuffer[0] = 0x00ff0000`. For win32 or Cocoa, you need to create a window, alter its flags to ensure it's borderless, change its size and posistion to 1x1 and [0,0] (or [0, screen_height] on the Mac), strip away the drop shadow, respond to regular paint events and pray like hell that nothing else draws over the top of it. >True low level control is kind of a pain It depends on what you're trying to achieve. If you're trying to get something highly responsive that looks exactly like the UI spec, most libraries tend to make this job harder when the abstractions leak and you're forced to rely on bizarre hacks that make no sense and feel genuinely demoralising to implement. Dear ImGui, which exposes far more of the low-level internals than any other API, is an absolute joy to build GUIs in and significantly easier than something which "holds your hand" at every step.

nyanpasu64 5y ago
Dear ImGui does not support Arabic (right-to-left, glyph shaping) and will not support Arabic (https://github.com/ocornut/imgui/issues/1343 https://github.com/ocornut/imgui/issues/1343). I don't think it supports accessibility either. As much as I struggle with Qt's APIs (aging janky Widgets, half-baked QML on the desktop, the item models are an impenetrable byzantine enterprisey API that's simultaneously too open-ended, not flexible enough since it imposes its add/remove/move schema on your model, and too buggy), building a GUI from the ground up is not going to give you internationalization or accessibility, and Qt is.
- AussieWog93 5y ago
  >Dear ImGui does not support Arabic (right-to-left, glyph shaping) and will not support Arabic (https://github.com/ocornut/imgui/issues/1343 https://github.com/ocornut/imgui/issues/1343). Vanilla ImGui doesn't, but I both wrapped my head around and then rewrote the text renderer in all of 2 days as a junior dev. This version could support arbitrarily many UTF-8 codepoints with kerning and all and had a negligible performance hit on the machines I tested on. I never upstreamed it (left the company before the product that was using it shipped, actually), but it wouldn't be too hard to reproduce. That's what I love about exposed, lower-level frameworks like Dear ImGui. We didn't aim for accessibility, but I'm sure that would have also been semi-trivial for a dedicated dev who understands what's required.
  - lavabiopsy 5y ago
    Maybe you want to help this person who is trying to integrate harfbuzz: https://github.com/ocornut/imgui/issues/4227 https://github.com/ocornut/imgui/issues/4227 The other major thing missing in relation to that is input method support. The problem with those type of APIs and accessibility is that implementing screen reader support basically means making it retained mode in some capacity, because accessibility APIs expect you to export a tree of objects and push state updates.
    - AussieWog93 5y ago
      >Maybe you want to help this person who is trying to integrate harfbuzz: https://github.com/ocornut/imgui/issues/4227 https://github.com/ocornut/imgui/issues/4227 Looks like theirs is far more advanced than mine (mine would just generate a texture per string and cache it, the main problem it solved was being able to display non-ASCII filenames). I don't think there's much I could contribute to them at all. >accessibility APIs expect you to export a tree of objects and push state updates Is this an inherent limitation with the tech itself, or just an arbitrary API limitation?
      - lavabiopsy 5y ago
        This is the way these platform APIs are designed, you could theoretically make a screen reader that doesn't need that and just asks the app to recompute its state every time, but they don't currently work that way.
nine_k 5y ago
But most of the time you don't want to make individual pixels red. You want to blit bitmaps and fill rectangles, preferably without pegging the CPU. Here the simple memory-mapped framebuffer is less helpful, especially if you have specialized hardware to do such work. Of course this does not apply if all you have is a screen connected via SPI to an MCU's GPIO pins. But you won't expect much graphical sophistication from such a device anyway.
lavabiopsy 5y ago
IMO, Dear ImGui is great for its original purpose (debugging GUIs, simple demos), but once you start adding complexity, custom layouts and custom widgets on top of it the state tracking gets to be just as bad as retained mode if not worse. Unless there was some best practice that I missed, one has to take great pains to avoid re-running the layout multiple times per frame, whereas a retained mode GUI would be able to handle that easily.
- AussieWog93 5y ago
  Maybe it's just a personal experience thing? I came across both ways of thinking as a junior dev, so to me retained mode was just a footgun rich environment. I could imagine if you had decades of experience using retained-mode, though, then you'd find that way of thinking more natural.
  - lavabiopsy 5y ago
    This affects all GUIs, if you have a lot of elements then you have to be careful not to propagate complex updates through the whole tree because it kills your performance. There are various tools to deal with this, one of them is making it retained mode, but I would say basically all of those tools come with more footguns in that the GUI can easily end up in an inconsistent state. That's what you trade for performance.
    - AussieWog93 5y ago
      I've heard that argument before, yet for almost all apps I've used in real life the opposite is true. Game UIs are (from what I understand) written almost exclusively ImGui-style, with every single widget being fed updated state data every single frame as part of the game loop. Yet, for whatever reason, even crazy-complex UIs like SupCom's seem to have no issue rendering in realtime with an entire 3D game running in the background, while simply resizing the window I'm typing into right now (a single textedit on a static HTML page) creates visible lag and chews up full CPU on a 2015 Macbook. That's not to say retained-mode GUIs can't be performant (embedded systems rely on retained-mode to have reasonable performance), but on a modern system it seems like the loss in performance caused by inefficiently synchronising state in retained-mode GUIs is several orders of magnitude higher than the losses caused by pushing a couple of hundred textures every frame in an ImGui context.
Certhas 5y ago
Isn't this obvious? I mean if you need that kind of control of the screen then you shouldn't be working inside a WM by definition, and the Toolkits are built for building interfaces in the context of WMs. Right? I definitely don't want your random application to be able to draw on arbitrary parts of the screen and then prevent anyone else from drawing over it. Your use-case might sound simple at first glance, but it's truly odd if you think about how I interact with applications, and how I expect to be able to manage them.
- AussieWog93 5y ago
  Of course. I'm not saying that using Win32 or Cocoa are wrong (at least for a Desktop OS where you don't want to hand over full control to an app), just that they're closer on the spectrum to Qt than bare metal.
simonh 5y ago
Making the top left pixel red just isn't something you should be doing in a GUI environment anyway, it doesn't make any sense. For something like that you should probably be dropping to a full-screen mode like most graphics heavy games anyway, in which case it's pretty easy on modern OSes. The desktop GUI is just one subsystem in a modern OS, and you don't have to use it if you don't want to. If you do use it then there are tradeoffs, of course, but that's always true of anything you use.
- Al-Khwarizmi 5y ago
  > Making the top left pixel red just isn't something you should be doing in a GUI environment anyway, it doesn't make any sense. Never cheated in Minesweeper, I presume?
  - edoceo 5y ago
    One of the versions had the high scores saved in ini file (3.1? 95?). My dad was amazed I solved large one in 15s. Then he found it. Then I learned about the read-only attribute.
dkersten 5y ago
> To make the top left pixel red on a bare metal system or RTOS, you simply write `framebuffer[0] = 0x00ff0000`. Really? In my brief experience it was more like sending a bunch of commands over SPI/I2C/UART to some LCD module. Which isn't thaaaat different from how an OS driver communicates with a GPU, only the commands are far, far more complex, so using an OS-level API avoids having to do it by hand.
raxxorrax 5y ago
Just start to implement the ~300-400 functions listed here and the underlying systems supporting them. How long could that take, a whole weekend? https://github.com/torvalds/linux/blob/master/arch/x86/entry/syscalls/syscall_64.tbl https://github.com/torvalds/linux/blob/master/arch/x86/entry...
cmrdporcupine 5y ago
Arguably win32 GUI or Cocoa are not "OS level" APIs; yes some aspects of them are invoked through syscall type interfaces (esp win32) but not consistently and the operating system itself is entirely usable without them. Especially Cocoa. You can boot a Mac into a BSD shell without ever running Finder and the GUI. Writing into a framebuffer is arguably as much of an abstraction these days as many of these toolkits. In the modern world of GPUs and display controllers a framebuffer itself is already N levels away from the "physical" display hardware. We're not in VGA land anymore. Some aspects of the OS or the OS's interaction with the display driver may present something like a linear framebuffer to you, but underneath that is a world of GPU textures and buffers and display controller abstractions. When I think of OS-level APIs I think of ioctl, mmap, file descriptors and sockets. I guess that's my Linux bias showing?
- overgard 5y ago
  Well, that's kind of what I'm driving at with things like the win32 api being a bit too high level and still kind of sucking. An example would be the GDI (the way you draw simple graphics). Like, it's in the worst of both worlds in that it's both slow and annoying to use. Or you can use DirectX and draw things fast, but you need to know a lot more. I think my point is mainly that OS's might be better off just giving you things like DirectX (and mmap and ioctl, etc.) and leaving off things like the GDI. I know right now Microsoft keeps making like 50000 new UI frameworks and part of me is like, just knock it the fuck off, give me some style guidelines and some super-efficient very low level no-frills API's and let me just build apps off a library like Qt that abstracts it.
- pjmlp 5y ago
  Unless you plan to write an OS personality yourself, there is no way to use Windows without Win32.

nyanpasu64 5y ago

Dear ImGui does not support Arabic (right-to-left, glyph shaping) and will not support Arabic (https://github.com/ocornut/imgui/issues/1343 https://github.com/ocornut/imgui/issues/1343). I don't think it supports accessibility either. As much as I struggle with Qt's APIs (aging janky Widgets, half-baked QML on the desktop, the item models are an impenetrable byzantine enterprisey API that's simultaneously too open-ended, not flexible enough since it imposes its add/remove/move schema on your model, and too buggy), building a GUI from the ground up is not going to give you internationalization or accessibility, and Qt is.

AussieWog93 5y ago
>Dear ImGui does not support Arabic (right-to-left, glyph shaping) and will not support Arabic (https://github.com/ocornut/imgui/issues/1343 https://github.com/ocornut/imgui/issues/1343). Vanilla ImGui doesn't, but I both wrapped my head around and then rewrote the text renderer in all of 2 days as a junior dev. This version could support arbitrarily many UTF-8 codepoints with kerning and all and had a negligible performance hit on the machines I tested on. I never upstreamed it (left the company before the product that was using it shipped, actually), but it wouldn't be too hard to reproduce. That's what I love about exposed, lower-level frameworks like Dear ImGui. We didn't aim for accessibility, but I'm sure that would have also been semi-trivial for a dedicated dev who understands what's required.
- lavabiopsy 5y ago
  Maybe you want to help this person who is trying to integrate harfbuzz: https://github.com/ocornut/imgui/issues/4227 https://github.com/ocornut/imgui/issues/4227 The other major thing missing in relation to that is input method support. The problem with those type of APIs and accessibility is that implementing screen reader support basically means making it retained mode in some capacity, because accessibility APIs expect you to export a tree of objects and push state updates.
  - AussieWog93 5y ago
    >Maybe you want to help this person who is trying to integrate harfbuzz: https://github.com/ocornut/imgui/issues/4227 https://github.com/ocornut/imgui/issues/4227 Looks like theirs is far more advanced than mine (mine would just generate a texture per string and cache it, the main problem it solved was being able to display non-ASCII filenames). I don't think there's much I could contribute to them at all. >accessibility APIs expect you to export a tree of objects and push state updates Is this an inherent limitation with the tech itself, or just an arbitrary API limitation?
    - lavabiopsy 5y ago
      This is the way these platform APIs are designed, you could theoretically make a screen reader that doesn't need that and just asks the app to recompute its state every time, but they don't currently work that way.

nine_k 5y ago

But most of the time you don't want to make individual pixels red. You want to blit bitmaps and fill rectangles, preferably without pegging the CPU. Here the simple memory-mapped framebuffer is less helpful, especially if you have specialized hardware to do such work. Of course this does not apply if all you have is a screen connected via SPI to an MCU's GPIO pins. But you won't expect much graphical sophistication from such a device anyway.

lavabiopsy 5y ago

IMO, Dear ImGui is great for its original purpose (debugging GUIs, simple demos), but once you start adding complexity, custom layouts and custom widgets on top of it the state tracking gets to be just as bad as retained mode if not worse. Unless there was some best practice that I missed, one has to take great pains to avoid re-running the layout multiple times per frame, whereas a retained mode GUI would be able to handle that easily.

AussieWog93 5y ago
Maybe it's just a personal experience thing? I came across both ways of thinking as a junior dev, so to me retained mode was just a footgun rich environment. I could imagine if you had decades of experience using retained-mode, though, then you'd find that way of thinking more natural.
- lavabiopsy 5y ago
  This affects all GUIs, if you have a lot of elements then you have to be careful not to propagate complex updates through the whole tree because it kills your performance. There are various tools to deal with this, one of them is making it retained mode, but I would say basically all of those tools come with more footguns in that the GUI can easily end up in an inconsistent state. That's what you trade for performance.
  - AussieWog93 5y ago
    I've heard that argument before, yet for almost all apps I've used in real life the opposite is true. Game UIs are (from what I understand) written almost exclusively ImGui-style, with every single widget being fed updated state data every single frame as part of the game loop. Yet, for whatever reason, even crazy-complex UIs like SupCom's seem to have no issue rendering in realtime with an entire 3D game running in the background, while simply resizing the window I'm typing into right now (a single textedit on a static HTML page) creates visible lag and chews up full CPU on a 2015 Macbook. That's not to say retained-mode GUIs can't be performant (embedded systems rely on retained-mode to have reasonable performance), but on a modern system it seems like the loss in performance caused by inefficiently synchronising state in retained-mode GUIs is several orders of magnitude higher than the losses caused by pushing a couple of hundred textures every frame in an ImGui context.

Certhas 5y ago

Isn't this obvious? I mean if you need that kind of control of the screen then you shouldn't be working inside a WM by definition, and the Toolkits are built for building interfaces in the context of WMs. Right? I definitely don't want your random application to be able to draw on arbitrary parts of the screen and then prevent anyone else from drawing over it. Your use-case might sound simple at first glance, but it's truly odd if you think about how I interact with applications, and how I expect to be able to manage them.

AussieWog93 5y ago
Of course. I'm not saying that using Win32 or Cocoa are wrong (at least for a Desktop OS where you don't want to hand over full control to an app), just that they're closer on the spectrum to Qt than bare metal.

simonh 5y ago

Making the top left pixel red just isn't something you should be doing in a GUI environment anyway, it doesn't make any sense. For something like that you should probably be dropping to a full-screen mode like most graphics heavy games anyway, in which case it's pretty easy on modern OSes. The desktop GUI is just one subsystem in a modern OS, and you don't have to use it if you don't want to. If you do use it then there are tradeoffs, of course, but that's always true of anything you use.

Al-Khwarizmi 5y ago
> Making the top left pixel red just isn't something you should be doing in a GUI environment anyway, it doesn't make any sense. Never cheated in Minesweeper, I presume?
- edoceo 5y ago
  One of the versions had the high scores saved in ini file (3.1? 95?). My dad was amazed I solved large one in 15s. Then he found it. Then I learned about the read-only attribute.

dkersten 5y ago

> To make the top left pixel red on a bare metal system or RTOS, you simply write `framebuffer[0] = 0x00ff0000`. Really? In my brief experience it was more like sending a bunch of commands over SPI/I2C/UART to some LCD module. Which isn't thaaaat different from how an OS driver communicates with a GPU, only the commands are far, far more complex, so using an OS-level API avoids having to do it by hand.

raxxorrax 5y ago

Just start to implement the ~300-400 functions listed here and the underlying systems supporting them. How long could that take, a whole weekend? https://github.com/torvalds/linux/blob/master/arch/x86/entry/syscalls/syscall_64.tbl https://github.com/torvalds/linux/blob/master/arch/x86/entry...

cmrdporcupine 5y ago

Arguably win32 GUI or Cocoa are not "OS level" APIs; yes some aspects of them are invoked through syscall type interfaces (esp win32) but not consistently and the operating system itself is entirely usable without them. Especially Cocoa. You can boot a Mac into a BSD shell without ever running Finder and the GUI. Writing into a framebuffer is arguably as much of an abstraction these days as many of these toolkits. In the modern world of GPUs and display controllers a framebuffer itself is already N levels away from the "physical" display hardware. We're not in VGA land anymore. Some aspects of the OS or the OS's interaction with the display driver may present something like a linear framebuffer to you, but underneath that is a world of GPU textures and buffers and display controller abstractions. When I think of OS-level APIs I think of ioctl, mmap, file descriptors and sockets. I guess that's my Linux bias showing?

overgard 5y ago
Well, that's kind of what I'm driving at with things like the win32 api being a bit too high level and still kind of sucking. An example would be the GDI (the way you draw simple graphics). Like, it's in the worst of both worlds in that it's both slow and annoying to use. Or you can use DirectX and draw things fast, but you need to know a lot more. I think my point is mainly that OS's might be better off just giving you things like DirectX (and mmap and ioctl, etc.) and leaving off things like the GDI. I know right now Microsoft keeps making like 50000 new UI frameworks and part of me is like, just knock it the fuck off, give me some style guidelines and some super-efficient very low level no-frills API's and let me just build apps off a library like Qt that abstracts it.
pjmlp 5y ago
Unless you plan to write an OS personality yourself, there is no way to use Windows without Win32.