3 ms·

I’ve been digging into Apple’s (existing) accessibility features last week and it made me think about what (for lack of a better term) an “accessibility first”

I’ve been digging into Apple’s (existing) accessibility features last week and it made me think about what (for lack of a better term) an “accessibility first” app architecture might look like. Current app UIs are all about how to visually represent the objects you can interact with and the actions you can take on them, and then accessibility features are layered on top. (Warning: half-baked idea that’s probably been tried countless times) what if you started with semantic information about the objects and the actions you can take on them and then the GUI is just one of several interaction modes (along with voice, keyboard, CLI, etc)?

jdlshore 3y ago
This was the original premise of the Smalltalk Model-View-Controller architecture. (Not to be confused with web MVC, which reused the name for something else.) The “Controller” in Model-View-Controller refers to the input device, and a given Model can have multiple independent Views and Controllers. The “Model” is supposed to an object that’s a semantic representation of the underlying thing being manipulated. One of the visible impacts of MVC is that changes occur in real-time: you modify a value in a dialog, and everything affected by that value instantly updates. This is already common in Mac apps (in contrast to Windows apps, which typically want you to press “ok” or “apply”), so it wouldn’t surprise me if Apple was already using a modern MVC variant. It’s a well-known pattern.
- alexjm 3y ago
  In the Apple documentation for MVC, "controller" refers to a class that sits between the model and view. When data changes in the model, it updates the view; and when the user interacts with the view, it passes events to the model. https://developer.apple.com/library/archive/documentation/General/Conceptual/DevPedia-CocoaCore/MVC.html#//apple_ref/doc/uid/TP40008195-CH32-SW1 https://developer.apple.com/library/archive/documentation/Ge... Elsewhere, the documentation contrasts the Cocoa and Smalltalk versions of MVC where all three pieces communicate directly: https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/CocoaFundamentals/CocoaDesignPatterns/CocoaDesignPatterns.html#//apple_ref/doc/uid/TP40002974-CH6-SW22 https://developer.apple.com/library/archive/documentation/Co... Like you said, this separation means you can "drive" the same model through different UIs. That's one of the things I always thought was cool about AppleScript support -- the app exposes a different interface to the same model.
w10-1 3y ago
Obviously developers who design semantically for different UIs would be far, far ahead. And Apple API's can be used there. The harder problem is building accessibility API's for oblivious developers, so they can retrofit at the last minute before release (as usually happens). Apple's done a pretty good job there, harvesting the semantics of existing visual UI's to adapt for voice/hearing.

jdlshore 3y ago

This was the original premise of the Smalltalk Model-View-Controller architecture. (Not to be confused with web MVC, which reused the name for something else.) The “Controller” in Model-View-Controller refers to the input device, and a given Model can have multiple independent Views and Controllers. The “Model” is supposed to an object that’s a semantic representation of the underlying thing being manipulated. One of the visible impacts of MVC is that changes occur in real-time: you modify a value in a dialog, and everything affected by that value instantly updates. This is already common in Mac apps (in contrast to Windows apps, which typically want you to press “ok” or “apply”), so it wouldn’t surprise me if Apple was already using a modern MVC variant. It’s a well-known pattern.

alexjm 3y ago
In the Apple documentation for MVC, "controller" refers to a class that sits between the model and view. When data changes in the model, it updates the view; and when the user interacts with the view, it passes events to the model. https://developer.apple.com/library/archive/documentation/General/Conceptual/DevPedia-CocoaCore/MVC.html#//apple_ref/doc/uid/TP40008195-CH32-SW1 https://developer.apple.com/library/archive/documentation/Ge... Elsewhere, the documentation contrasts the Cocoa and Smalltalk versions of MVC where all three pieces communicate directly: https://developer.apple.com/library/archive/documentation/Cocoa/Conceptual/CocoaFundamentals/CocoaDesignPatterns/CocoaDesignPatterns.html#//apple_ref/doc/uid/TP40002974-CH6-SW22 https://developer.apple.com/library/archive/documentation/Co... Like you said, this separation means you can "drive" the same model through different UIs. That's one of the things I always thought was cool about AppleScript support -- the app exposes a different interface to the same model.

w10-1 3y ago

Obviously developers who design semantically for different UIs would be far, far ahead. And Apple API's can be used there. The harder problem is building accessibility API's for oblivious developers, so they can retrofit at the last minute before release (as usually happens). Apple's done a pretty good job there, harvesting the semantics of existing visual UI's to adapt for voice/hearing.