12 ms·

>There is, after all, no fundamental reason a GUI could not be designed with the goal of reification of interaction in mind; it’s just that it is rarely done in

by jasode 9y ago

>There is, after all, no fundamental reason a GUI could not be designed with the goal of reification of interaction in mind; it’s just that it is rarely done in practice. If we have to search for an underlying reason, it's that most GUI interactions by most end users don't need reification and therefore, that code is not put in. If the GUI has obvious needs for reification, the programmers account for it. Two examples that come to mind are computer chess games, and software audio workstations. Many computer users who are comfortable with UNIX command line will nevertheless "move chess pieces" by clicking and dragging the piece from one square to another. However, virtually all chess programs still record the algebraic chessboard notation of each move and the players can see the tally of moves scrolling on the side of the board. The other example has music producers mixing a song. They will adjust the sliders/faders of volume panning and the software will record their moves. It's known in the trade as "mix automation". When the song is played back, the software makes the sliders mimic the same moves. The automation in "reified" into a time graph[1] such that one can move the nodes around to fine tune the movements more precisely than the fingers did in real time. As for other GUI applications like typical business data entry screens, etc... most GUI toolkits like C# Winforms, C++ Qt, HTML buttons, etc do not include a macro recorder in the default event loop. Therefore, to "reify" the GUI in the same fashion as chess programs or the audio software requires grafting data structures and programming extra code to "record" the button presses into a timeline. Programmers won't do this unless there's a customer need for it. (E.g. Photoshop lets you reify the GUI as "actions" for later playback") [1] example yellow line graphs the fader or knob changes: http://cdm.link/app/uploads/2014/06/logicmixing-640x399.jpg http://cdm.link/app/uploads/2014/06/logicmixing-640x399.jpg

kybernetikos 9y ago
Paint.net is another example of an application that reifies its actions with a nice 'history' window that shows the semantic actions that the user has performed and allows you to jump back to earlier points in your history of editing the file. GIMP does something similar, it seems particularly common in image editing programs.
- agumonkey 9y ago
  one day people will have DAG based programs where history has even less importance
  - timthelion 9y ago
    I'm curios what you mean by this. I've been working on some software called textgraph [1] (not yet ready for use, and now internally renamed gradesta ) for a couple of years now, so anything with "graph" in it makes my ears perk up. [1] http://timothy.hobbs.cz/tg/tg.html http://timothy.hobbs.cz/tg/tg.html
    - agumonkey 9y ago
      the paint.net stacked history is similar to what was used in 3D studio max in the 90s. It was "cool" but too linear (it's a stack after all). Other software like Houdini or Maya (or NR Shake) used DAG (acyclic graph) of operations to create an output scene/image. There the history isnt linear it's a bit of a multiverse, you can disconnect part of the graph, try new things, reconnect. Everything is there.
      - icebraining 9y ago
        Like a modern version control system?
        agumonkey 9y ago
        It's the same underlying structure, replace commit by different kinds of operations.
- SilasX 9y ago
  Ditto for AutoCAD -- I loved seeing how the equivalent CLI command appeared in the log as I was using the GUI to draw.
  - 9y ago
weberc2 9y ago
Both of the GUI toolkits I've worked with (GTK and Qt) have a concept of "Actions" which can be (and frequently are) mapped to a menu item and a keyboard shortcut and sometimes a CLI flag; isn't this an example of reification, or am I misunderstanding the concept?
- jasode 9y ago
  I don't know about GTK but in Qt, those are not recorded into a "timeline".
  - weberc2 9y ago
    Sure, but is that implied by "reify"? My understanding is that they are reified and thus it is possible to make a timeline from them. CLI commands are only recorded into a timeline because of shell history, after all--if you call a CLI from a different mechanism, does it become not-reified?
    - digi_owl 9y ago
      The underlying commands can still be logged. Hell, often it is useful to launch a X11 program via a terminal as they will dump useful messages that way. And said output can also be captured in logs elsewhere. One example of such would be SMPlayer. This is a Qt based media player that wrap MPlayer (and more recently the MPlayer fork MPV). But you can still access the MPlayer CLI output by either launching SMPlayer from a terminal, or going opening the log window from one of its menus.
      - weberc2 9y ago
        To be clear, this was my point--because of the "action" mechanism, it's trivial to make a timeline.
  - alkonaut 9y ago
    The undo stack of any document oriented application is that timeline? Typically if you hover the undo button it says "Undo add frob" and if you undo that, you can see the next item. It's also common to be able to drop it down and see the whole undo stack and undo to an arbitrary point in it. Since this has been par for the course for for as long as I remember, I take it that's not what's discussed here...
pcmaffey 9y ago
The browser? Though as you mention, many developers nowadays don't see the need for history, and so are putting application/webpage state somewhere other than the router. This IMO is short-sighted, as it undermines the transparency of a user's actions on the web. And plays into the walled-garden effect many of these same developers will lament in the loss of the "open web".
- horsawlarway 9y ago
  I don't agree that we've lost this ability by moving state to the client or JS (since it seems you're really bitching about single page JS apps). There are several ways to capture that state, and the nice thing is that the browser allows sites to either opt in or out depending on their requirements. See: Insights, FullStory, or any other event hooking library. We can also play back those actions using the very nice WebDriver apis available in modern browsers. There is zero reason to assume that we've lost reification simply because we've moved away from an approach that requires at least one RTT in latency before updating what the user sees.
  - pcmaffey 9y ago
    Unless you sync app state with the router, then you cannot "Enter a command / url" => "Output a state / action".
    - horsawlarway 9y ago
      I'm sorry, but that's incorrect. It's not something most users do, so if your argument is "Reification is not on by default for all users" then sure. But that's a meaningless statement. Most users don't want that ability, or aren't willing to spend the 10 minutes it takes to configure. Otherwise, you absolutely can enter commands and output state: https://sites.google.com/a/chromium.org/chromedriver/downloads https://sites.google.com/a/chromium.org/chromedriver/downloa... Further, there are a HOST of tools that make this process very easy, complete with replay solutions, detailed logging, real time monitoring and all sorts of other fantastic, modern tools. Just because you choose not to use them doesn't mean they don't exist. Sorry your understanding of web dev is out of date...
betenoire 9y ago
Your examples are good, but most users don't _need_ reification for CLI tools either. But at least it's an option when they do.
- b0rsuk 9y ago
  Reification is always there for the (rare?) cases you need it. With GUI, there's no such option, and it's hard to notice something that has always been missing.
btbuildem 9y ago
Well, with this narrow definition of reification, you could say that any UNDO mechanism basically implements it.
b0rsuk 9y ago
Chess: * The interface is typically NOT optimized for keyboard. Typing E4D5 is not optimized. You need to look at a square in the middle, then look which row it is, and which column it is. I've implemented a better one for a similar game (Jungle). You press a character key to highlight one of your pieces, and the game essentially highlights legal moves by displaying alphanumeric chars on tiles. Instead of 4 keypresses, you have 2. In Chess it could be like this: 1-8 selects a pawn, b/B one of bishops, r/R - rooks, k/K - knights, q - queen, i - king. Then another character depending on what shows up where you want to move. * People who play realtime chess (one example is Kung Fu Chess)do use keyboard certainly those playing in tournaments. E4-D5 notation is optimized for recording games, not for playing.
- hutzlibu 9y ago
  "E4-D5 notation is optimized for recording games, not for playing" depends on the type of player. Every serious hobby player can do it that way. They learned it. But for casual players it is not the way to go, I agree.
kalleboo 9y ago
> most GUI toolkits like C# Winforms, C++ Qt, HTML buttons, etc do not include a macro recorder in the default event loop. Therefore, to "reify" the GUI in the same fashion as chess programs or the audio software requires grafting data structures and programming extra code to "record" the button presses into a timeline. Programmers won't do this unless there's a customer need for it. (E.g. Photoshop lets you reify the GUI as "actions" for later playback") In the MacOS System 7 days when Apple was still pushing AppleEvents/AppleScript heavily, a good portion of Mac software would let you record your actions in them through the AppleScript Script Editor (they would emit the AppleEvent equivalents of your GUI actions), generating an AppleScript which you could then edit. It was a fantastic system. It's still there in macOS X, but it's rarely supported/used now.

by jasode 9y ago

kybernetikos 9y ago

Paint.net is another example of an application that reifies its actions with a nice 'history' window that shows the semantic actions that the user has performed and allows you to jump back to earlier points in your history of editing the file. GIMP does something similar, it seems particularly common in image editing programs.

agumonkey 9y ago
one day people will have DAG based programs where history has even less importance
- timthelion 9y ago
  I'm curios what you mean by this. I've been working on some software called textgraph [1] (not yet ready for use, and now internally renamed gradesta ) for a couple of years now, so anything with "graph" in it makes my ears perk up. [1] http://timothy.hobbs.cz/tg/tg.html http://timothy.hobbs.cz/tg/tg.html
  - agumonkey 9y ago
    the paint.net stacked history is similar to what was used in 3D studio max in the 90s. It was "cool" but too linear (it's a stack after all). Other software like Houdini or Maya (or NR Shake) used DAG (acyclic graph) of operations to create an output scene/image. There the history isnt linear it's a bit of a multiverse, you can disconnect part of the graph, try new things, reconnect. Everything is there.
    - icebraining 9y ago
      Like a modern version control system?
      - agumonkey 9y ago
        It's the same underlying structure, replace commit by different kinds of operations.
SilasX 9y ago
Ditto for AutoCAD -- I loved seeing how the equivalent CLI command appeared in the log as I was using the GUI to draw.
- 9y ago

weberc2 9y ago

Both of the GUI toolkits I've worked with (GTK and Qt) have a concept of "Actions" which can be (and frequently are) mapped to a menu item and a keyboard shortcut and sometimes a CLI flag; isn't this an example of reification, or am I misunderstanding the concept?

jasode 9y ago
I don't know about GTK but in Qt, those are not recorded into a "timeline".
- weberc2 9y ago
  Sure, but is that implied by "reify"? My understanding is that they are reified and thus it is possible to make a timeline from them. CLI commands are only recorded into a timeline because of shell history, after all--if you call a CLI from a different mechanism, does it become not-reified?
  - digi_owl 9y ago
    The underlying commands can still be logged. Hell, often it is useful to launch a X11 program via a terminal as they will dump useful messages that way. And said output can also be captured in logs elsewhere. One example of such would be SMPlayer. This is a Qt based media player that wrap MPlayer (and more recently the MPlayer fork MPV). But you can still access the MPlayer CLI output by either launching SMPlayer from a terminal, or going opening the log window from one of its menus.
    - weberc2 9y ago
      To be clear, this was my point--because of the "action" mechanism, it's trivial to make a timeline.
- alkonaut 9y ago
  The undo stack of any document oriented application is that timeline? Typically if you hover the undo button it says "Undo add frob" and if you undo that, you can see the next item. It's also common to be able to drop it down and see the whole undo stack and undo to an arbitrary point in it. Since this has been par for the course for for as long as I remember, I take it that's not what's discussed here...

pcmaffey 9y ago

The browser? Though as you mention, many developers nowadays don't see the need for history, and so are putting application/webpage state somewhere other than the router. This IMO is short-sighted, as it undermines the transparency of a user's actions on the web. And plays into the walled-garden effect many of these same developers will lament in the loss of the "open web".

horsawlarway 9y ago
I don't agree that we've lost this ability by moving state to the client or JS (since it seems you're really bitching about single page JS apps). There are several ways to capture that state, and the nice thing is that the browser allows sites to either opt in or out depending on their requirements. See: Insights, FullStory, or any other event hooking library. We can also play back those actions using the very nice WebDriver apis available in modern browsers. There is zero reason to assume that we've lost reification simply because we've moved away from an approach that requires at least one RTT in latency before updating what the user sees.
- pcmaffey 9y ago
  Unless you sync app state with the router, then you cannot "Enter a command / url" => "Output a state / action".
  - horsawlarway 9y ago
    I'm sorry, but that's incorrect. It's not something most users do, so if your argument is "Reification is not on by default for all users" then sure. But that's a meaningless statement. Most users don't want that ability, or aren't willing to spend the 10 minutes it takes to configure. Otherwise, you absolutely can enter commands and output state: https://sites.google.com/a/chromium.org/chromedriver/downloads https://sites.google.com/a/chromium.org/chromedriver/downloa... Further, there are a HOST of tools that make this process very easy, complete with replay solutions, detailed logging, real time monitoring and all sorts of other fantastic, modern tools. Just because you choose not to use them doesn't mean they don't exist. Sorry your understanding of web dev is out of date...

betenoire 9y ago

Your examples are good, but most users don't _need_ reification for CLI tools either. But at least it's an option when they do.

b0rsuk 9y ago
Reification is always there for the (rare?) cases you need it. With GUI, there's no such option, and it's hard to notice something that has always been missing.

btbuildem 9y ago

Well, with this narrow definition of reification, you could say that any UNDO mechanism basically implements it.

b0rsuk 9y ago

Chess: * The interface is typically NOT optimized for keyboard. Typing E4D5 is not optimized. You need to look at a square in the middle, then look which row it is, and which column it is. I've implemented a better one for a similar game (Jungle). You press a character key to highlight one of your pieces, and the game essentially highlights legal moves by displaying alphanumeric chars on tiles. Instead of 4 keypresses, you have 2. In Chess it could be like this: 1-8 selects a pawn, b/B one of bishops, r/R - rooks, k/K - knights, q - queen, i - king. Then another character depending on what shows up where you want to move. * People who play realtime chess (one example is Kung Fu Chess)do use keyboard certainly those playing in tournaments. E4-D5 notation is optimized for recording games, not for playing.

hutzlibu 9y ago
"E4-D5 notation is optimized for recording games, not for playing" depends on the type of player. Every serious hobby player can do it that way. They learned it. But for casual players it is not the way to go, I agree.

kalleboo 9y ago

> most GUI toolkits like C# Winforms, C++ Qt, HTML buttons, etc do not include a macro recorder in the default event loop. Therefore, to "reify" the GUI in the same fashion as chess programs or the audio software requires grafting data structures and programming extra code to "record" the button presses into a timeline. Programmers won't do this unless there's a customer need for it. (E.g. Photoshop lets you reify the GUI as "actions" for later playback") In the MacOS System 7 days when Apple was still pushing AppleEvents/AppleScript heavily, a good portion of Mac software would let you record your actions in them through the AppleScript Script Editor (they would emit the AppleEvent equivalents of your GUI actions), generating an AppleScript which you could then edit. It was a fantastic system. It's still there in macOS X, but it's rarely supported/used now.