3 ms·
Have you seen Morgan Dixon's "Prefab", and his ideas about pixel-based reverse engineering and modifiable user interfaces? One of his papers asks, "What if Ever
by DonHopkins 6y ago
Have you seen Morgan Dixon's "Prefab", and his ideas about pixel-based reverse engineering and modifiable user interfaces? One of his papers asks, "What if Every GUI Were Open-Source?"
Video: Prefab: What if We Could Modify Any Interface? Target aware pointing techniques, bubble cursor, sticky icons, adding advanced behaviors to existing interfaces, independent of the tools used to implement those interfaces, platform agnostic enhancements, same Prefab code works on Windows and Mac, and across remote desktops, widget state awareness, widget transition tracking, side views, parameter preview spectrums for multi-parameter space exploration, prefab implements parameter spectrum preview interfaces for both unmodified Gimp and Photoshop: http://www.youtube.com/watch?v=lju6IIteg9Q http://www.youtube.com/watch?v=lju6IIteg9Q
Video: Content and Hierarchy in Prefab: What if anybody could modify any interface? Reverse engineering guis from their pixels, addresses hierarchy and content, identifying hierarchical tree structure, recognizing text, stencil based tutorials, adaptive gui visualization, ephemeral adaptation technique for arbitrary desktop interfaces, dynamic interface language translation, UI customization, re-rendering widgets, Skype favorite widgets tab: http://www.youtube.com/watch?v=w4S5ZtnaUKE http://www.youtube.com/watch?v=w4S5ZtnaUKE
PDF: Prefab: What if Every GUI Were Open-Source? Morgan Dixon and James Fogarty. (2010). Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI '10. ACM, New York, NY, 851-854. https://web.archive.org/web/20141024012013/http://homes.cs.washington.edu/~mdixon/publications/mdixon-prefab-workshop-chi2010-final.pdf https://web.archive.org/web/20141024012013/http://homes.cs.w...
>ABSTRACT:
Current methods for implementing graphical user interfaces
create fundamental challenges for HCI research and
practice. Researchers are often unable to demonstrate or
evaluate new techniques beyond small toy applications, and
practitioners are often unable to adopt methods from the
literature in new and existing applications. This position
statement examines a vision in which anybody can modify
any GUI of any application, similar to a scenario where
every GUI of every application is open-source. We are
currently working to enable this vision through our
development of Prefab, using pixel-based interpretation of
GUIs to enable modification of those GUIs without any
cooperation from the underlying application. We see
participation in the FLOSS HCI workshop as valuable in at
least two regards. First, fully realizing this vision will likely
require a community-based approach, so we are interested
in Prefab as a platform for collaboration between HCI
researchers and the FLOSS community. Second, enabling
arbitrary modification of any GUI would seem to blur many
current distinctions between open and closed applications,
introducing new research questions and further magnifying
the importance of the workshop’s focus.
I wrote more about it here:
https://news.ycombinator.com/item?id=11520967 https://news.ycombinator.com/item?id=11520967