4 ms·

Not even that: SDL just provides a pixel buffer, the application draws everything itself per-pixel. Lite uses a technique I refer to as "cached software renderi

by rxi 6y ago

Not even that: SDL just provides a pixel buffer, the application draws everything itself per-pixel. Lite uses a technique I refer to as "cached software rendering" which allows the application code to be written as if it's doing a fullscreen-redrawn when it wants to update, the renderer cache (rencache.c) then works out which regions actually need to be redrawn at the end of the frame and redraws only those. You can call `renderer.show_debug(true)` to show these redraw regions: https://youtu.be/KtL9f6bksDQ?t=50 https://youtu.be/KtL9f6bksDQ?t=50 I wrote a short article detailing the technique here: https://rxi.github.io/200402.html https://rxi.github.io/200402.html

badsectoracula 6y ago
FWIW this is basically "dirty rectangles" which was a very common technique for avoiding full screen updates in games back when the hardware wasn't fast enough to do that.
- rxi 6y ago
  You're equating the final stage of this approach to the entire approach. The point of this technique is that you get the benefits you typically would from dirty rectangles without the burden of the bookkeeping you would traditionally have. Using this technique your application "redraws" everything as if it's drawing it fresh each frame and the renderer cache takes care of determining what's actually changed. Typically with dirty rectangles you would have to manage this state in the application code, for example, determining that line X was edited then updating the region for that line, or determining that view Y moved and updating a dirty rectangle based upon it's previous and current positions.
  - badsectoracula 6y ago
    From the description at least it does sound there is book-keeping for the tiles so i'm not sure why you think there isn't such a burden.
- deleted 6y ago
  [deleted]
- derefr 6y ago
  Nah, it's tile-based concurrent precompositing, like web browsers do. Each tile knows what's in it (i.e. what set of DOM elements); and subscribes to state-change events for those DOM elements; and any such state-change event will trigger the tile to re-render its cached texture. Then, on each frame, all you have to do to draw everything, is to grab the latest cached texture from each tile, and blit those (or set them up as a grid of flat-projected rects in screen-space, if you're in 3D-semantics land.) You can get additional benefits from this approach, by doing multiple layers of it (e.g. having scrollable surfaces have their own tiles that precompute the inner-document-viewport-space rather than the outer-viewport-space, such that the inner tiles aren't invalidated by scrolling the outer viewport.) This technique ends up forming a tree of tiles, where tiles higher up the tree, when invalidated, re-render trivially by compositing tiles further down the tree into themselves. Thus, another name for this approach is a "precompositing tree." The difference between this approach and dirty rects, is in the direction of information flow. In tile-based precompositing, the information only flows in one direction—from the user, through view-controller, into the DOM, to the tiles, and then out the display. Dirty rects, meanwhile, are a signal sent backwards, from the display system to the program, essentially telling it that the display system lost/discarded the information needed to re-draw an area, so could the program please send it over again. (The program doesn't even have to re-render in response; some dirty-rects implementations, like X11's DAMAGE extension, just involve the client application re-transmitting pixbuf data to the server from its own precomposited buffer.) Also, dirty rects / screen-damage doesn't solve the problem of hardware not being fast enough; it solves the problem of hardware not having enough VRAM to do per-frame compositing from undamaged intermediates. In low-VRAM conditions, you can only keep around the final pre-composited image; and so any time you "damage" / make "dirty" a region of the screen (e.g. by removing an overdrawn element, which should have the semantics of revealing whatever was there before that element overdrew it) then you need to propagate a request back to the renderer to re-draw (and, for efficiency, re-draw just that region), because you don't just have an intermediate texture laying around for that window/stage-layer/etc. to re-source it from. If you did, then dirty-rects would never come into play, since you'd just re-composite everything each frame. (Which is cheap even on old-school CPU-only blitters—you just have to alternate which pixmap pointer you're basing your LOADs off of using either a rect-overlap check, or a mask-bitmap [which gets you 8 pixels' mask-states per LOAD.] Even the Gameboy can do it!)
- barrkel 6y ago
  It isn't. Normally the application needs to be aware of dirty rectangles; it fetches them from the window compositor, and it needs to limit its drawing calculations based on the dirty rectangle, in order to get the full benefit. (Dirty rectangles are still perfectly common in desktop apps for when you e.g. drag a window back on screen after overlapping the edge, or if you scroll a window.)
  - badsectoracula 6y ago
    You are thinking about regions, dirty rectangles were a common thing in games to avoid redrawing the entire thing and rarely seen outside of them.
tasty_freeze 6y ago
Sounds like curses.
deleted 6y ago
[deleted]
eyelidlessness 6y ago
Sounds like the pixel grid equivalent of a "virtual DOM"?
- c-smile 6y ago
  Not even close.
  - eyelidlessness 6y ago
    I'm open to the possibility that I'm that wrong in my understanding, but this didn't help me understand any better at all. The technique does sound similar to me. Both (as I understand it) maintain a representation in memory of the final rendering and use a diff to determine which parts of the rendering to perform. The "virtual DOM" technique isn't strictly tied to a browser DOM, though the term is a reference to that, and React's (in particular) has been adapted to many other rendering targets. I'd be happy to learn more if you'd be kind enough to explain what I misunderstood.
    - c-smile 6y ago
      virtual DOM implies DOM existence. DOM based systems use so called retained mode rendering. But this one uses something that can be classified as immediate mode rendering. Check https://docs.microsoft.com/en-us/windows/win32/learnwin32/retained-mode-versus-immediate-mode https://docs.microsoft.com/en-us/windows/win32/learnwin32/re...
      - eyelidlessness 6y ago
        > virtual DOM implies DOM existence No, it doesn't. I addressed this in the comment above. React's virtual DOM has been used to render: - Plain HTML (e.g. server-side rendering) - Native UI framework objects (e.g. React Native) - Text-based interfaces (e.g. Ink) - Smart TV devices (e.g. Netflix's Gibbon) - Browser `<canvas>` elements - Markdown formatted text And a whole bunch of other targets. > DOM based systems use so called retained mode rendering. But this one uses something that can be classified as immediate mode rendering. This seems orthogonal to the question? I'm not trying to be difficult, I sincerely don't understand why this would mean the two are "not even close".
c-smile 6y ago
That's not anyhow different from typical GDI, CoreGraphics, GTK/Cairo way of doing rendering. Windows, MacOS and GTK maintain internal pixmap buffer for a window. And when needed you call InvlidateRect(wnd,rc) and receive WM_PAINT with cumulative rect to update. Personally, I would create an abstraction that wraps couple of functions of GDI, CoreGraphics and Cairo and use it instead manual pixmap rendering. Will be faster and more flexible. All that UI can be rendered by just 2..3 functions FillRect, DrawText, MeasureText.
- barrkel 6y ago
  It's quite different, actually, both in application programming logic and in expressive power of drawing primitives. With dirty rectangles, it's the application's responsibility to minimize much of its drawing; the renderer will at best avoid copying bits where the target of the bits lies outside the drawing rectangle. The renderer can only optimize in situations where the entire call is understood as a full primitive, and it knows that the result will lie outside the dirty rectangle. With rxi's approach, the application gets to define the commands which update the UI - which may be as complex as desired, as long as they have a calculable rectangle - and the cost of calculating that rendering can be skipped, without needing to query for dirty rectangles or doing any application-side conditional logic, beyond the layer that rxi wrote. It's particularly powerful if the rendering primitives are higher level than those provided by the native APIs.
- frank2 6y ago
  Interesting. Do you know any software whose source code is available that uses such an abstraction to paint the screen?
  - c-smile 6y ago
    Any classic Windows, MacOS or GTK application does that. Check https://docs.microsoft.com/en-us/windows/win32/learnwin32/painting-the-window https://docs.microsoft.com/en-us/windows/win32/learnwin32/pa... If your question is about unified wrapper for multiple platforms then wxWidgets will qualify : https://wiki.wxwidgets.org/Painting_your_custom_control https://wiki.wxwidgets.org/Painting_your_custom_control As my Sciter where I wrapped Direct2D/DirectX, Skia/OpenGL, CoreGraphics, Cairo, GDI+ into class graphics abstraction so rest of code is isolated from particular paltform/backend used: https://github.com/c-smile/sciter-sdk/blob/master/include/behaviors/behavior_drawing.cpp https://github.com/c-smile/sciter-sdk/blob/master/include/be...
tambourine_man 6y ago
> …the application draws everything itself per-pixel That’s what I meant, yes. Interesting approach and impressive.
ZoomZoomZoom 6y ago
Thanks for the write-up. I can see it being beneficial for rust-audio community, which is looking for the right approach to VST GUI.
bitexploder 6y ago
Just an aside. Everyone should play with SDL and write a simple game that you have to draw your own pixels. It’s extremely satisfying. Do it in C. It’s pretty simple and fun.

by rxi 6y ago

badsectoracula 6y ago

FWIW this is basically "dirty rectangles" which was a very common technique for avoiding full screen updates in games back when the hardware wasn't fast enough to do that.

rxi 6y ago
You're equating the final stage of this approach to the entire approach. The point of this technique is that you get the benefits you typically would from dirty rectangles without the burden of the bookkeeping you would traditionally have. Using this technique your application "redraws" everything as if it's drawing it fresh each frame and the renderer cache takes care of determining what's actually changed. Typically with dirty rectangles you would have to manage this state in the application code, for example, determining that line X was edited then updating the region for that line, or determining that view Y moved and updating a dirty rectangle based upon it's previous and current positions.
- badsectoracula 6y ago
  From the description at least it does sound there is book-keeping for the tiles so i'm not sure why you think there isn't such a burden.
deleted 6y ago
[deleted]
derefr 6y ago
Nah, it's tile-based concurrent precompositing, like web browsers do. Each tile knows what's in it (i.e. what set of DOM elements); and subscribes to state-change events for those DOM elements; and any such state-change event will trigger the tile to re-render its cached texture. Then, on each frame, all you have to do to draw everything, is to grab the latest cached texture from each tile, and blit those (or set them up as a grid of flat-projected rects in screen-space, if you're in 3D-semantics land.) You can get additional benefits from this approach, by doing multiple layers of it (e.g. having scrollable surfaces have their own tiles that precompute the inner-document-viewport-space rather than the outer-viewport-space, such that the inner tiles aren't invalidated by scrolling the outer viewport.) This technique ends up forming a tree of tiles, where tiles higher up the tree, when invalidated, re-render trivially by compositing tiles further down the tree into themselves. Thus, another name for this approach is a "precompositing tree." The difference between this approach and dirty rects, is in the direction of information flow. In tile-based precompositing, the information only flows in one direction—from the user, through view-controller, into the DOM, to the tiles, and then out the display. Dirty rects, meanwhile, are a signal sent backwards, from the display system to the program, essentially telling it that the display system lost/discarded the information needed to re-draw an area, so could the program please send it over again. (The program doesn't even have to re-render in response; some dirty-rects implementations, like X11's DAMAGE extension, just involve the client application re-transmitting pixbuf data to the server from its own precomposited buffer.) Also, dirty rects / screen-damage doesn't solve the problem of hardware not being fast enough; it solves the problem of hardware not having enough VRAM to do per-frame compositing from undamaged intermediates. In low-VRAM conditions, you can only keep around the final pre-composited image; and so any time you "damage" / make "dirty" a region of the screen (e.g. by removing an overdrawn element, which should have the semantics of revealing whatever was there before that element overdrew it) then you need to propagate a request back to the renderer to re-draw (and, for efficiency, re-draw just that region), because you don't just have an intermediate texture laying around for that window/stage-layer/etc. to re-source it from. If you did, then dirty-rects would never come into play, since you'd just re-composite everything each frame. (Which is cheap even on old-school CPU-only blitters—you just have to alternate which pixmap pointer you're basing your LOADs off of using either a rect-overlap check, or a mask-bitmap [which gets you 8 pixels' mask-states per LOAD.] Even the Gameboy can do it!)
barrkel 6y ago
It isn't. Normally the application needs to be aware of dirty rectangles; it fetches them from the window compositor, and it needs to limit its drawing calculations based on the dirty rectangle, in order to get the full benefit. (Dirty rectangles are still perfectly common in desktop apps for when you e.g. drag a window back on screen after overlapping the edge, or if you scroll a window.)
- badsectoracula 6y ago
  You are thinking about regions, dirty rectangles were a common thing in games to avoid redrawing the entire thing and rarely seen outside of them.

tasty_freeze 6y ago

Sounds like curses.

deleted 6y ago

[deleted]

eyelidlessness 6y ago

Sounds like the pixel grid equivalent of a "virtual DOM"?

c-smile 6y ago
Not even close.
- eyelidlessness 6y ago
  I'm open to the possibility that I'm that wrong in my understanding, but this didn't help me understand any better at all. The technique does sound similar to me. Both (as I understand it) maintain a representation in memory of the final rendering and use a diff to determine which parts of the rendering to perform. The "virtual DOM" technique isn't strictly tied to a browser DOM, though the term is a reference to that, and React's (in particular) has been adapted to many other rendering targets. I'd be happy to learn more if you'd be kind enough to explain what I misunderstood.
  - c-smile 6y ago
    virtual DOM implies DOM existence. DOM based systems use so called retained mode rendering. But this one uses something that can be classified as immediate mode rendering. Check https://docs.microsoft.com/en-us/windows/win32/learnwin32/retained-mode-versus-immediate-mode https://docs.microsoft.com/en-us/windows/win32/learnwin32/re...
    - eyelidlessness 6y ago
      > virtual DOM implies DOM existence No, it doesn't. I addressed this in the comment above. React's virtual DOM has been used to render: - Plain HTML (e.g. server-side rendering) - Native UI framework objects (e.g. React Native) - Text-based interfaces (e.g. Ink) - Smart TV devices (e.g. Netflix's Gibbon) - Browser `<canvas>` elements - Markdown formatted text And a whole bunch of other targets. > DOM based systems use so called retained mode rendering. But this one uses something that can be classified as immediate mode rendering. This seems orthogonal to the question? I'm not trying to be difficult, I sincerely don't understand why this would mean the two are "not even close".

c-smile 6y ago

That's not anyhow different from typical GDI, CoreGraphics, GTK/Cairo way of doing rendering. Windows, MacOS and GTK maintain internal pixmap buffer for a window. And when needed you call InvlidateRect(wnd,rc) and receive WM_PAINT with cumulative rect to update. Personally, I would create an abstraction that wraps couple of functions of GDI, CoreGraphics and Cairo and use it instead manual pixmap rendering. Will be faster and more flexible. All that UI can be rendered by just 2..3 functions FillRect, DrawText, MeasureText.

barrkel 6y ago
It's quite different, actually, both in application programming logic and in expressive power of drawing primitives. With dirty rectangles, it's the application's responsibility to minimize much of its drawing; the renderer will at best avoid copying bits where the target of the bits lies outside the drawing rectangle. The renderer can only optimize in situations where the entire call is understood as a full primitive, and it knows that the result will lie outside the dirty rectangle. With rxi's approach, the application gets to define the commands which update the UI - which may be as complex as desired, as long as they have a calculable rectangle - and the cost of calculating that rendering can be skipped, without needing to query for dirty rectangles or doing any application-side conditional logic, beyond the layer that rxi wrote. It's particularly powerful if the rendering primitives are higher level than those provided by the native APIs.
frank2 6y ago
Interesting. Do you know any software whose source code is available that uses such an abstraction to paint the screen?
- c-smile 6y ago
  Any classic Windows, MacOS or GTK application does that. Check https://docs.microsoft.com/en-us/windows/win32/learnwin32/painting-the-window https://docs.microsoft.com/en-us/windows/win32/learnwin32/pa... If your question is about unified wrapper for multiple platforms then wxWidgets will qualify : https://wiki.wxwidgets.org/Painting_your_custom_control https://wiki.wxwidgets.org/Painting_your_custom_control As my Sciter where I wrapped Direct2D/DirectX, Skia/OpenGL, CoreGraphics, Cairo, GDI+ into class graphics abstraction so rest of code is isolated from particular paltform/backend used: https://github.com/c-smile/sciter-sdk/blob/master/include/behaviors/behavior_drawing.cpp https://github.com/c-smile/sciter-sdk/blob/master/include/be...

tambourine_man 6y ago

> …the application draws everything itself per-pixel That’s what I meant, yes. Interesting approach and impressive.

ZoomZoomZoom 6y ago

Thanks for the write-up. I can see it being beneficial for rust-audio community, which is looking for the right approach to VST GUI.

bitexploder 6y ago

Just an aside. Everyone should play with SDL and write a simple game that you have to draw your own pixels. It’s extremely satisfying. Do it in C. It’s pretty simple and fun.