Short version: I'm close to figuring out how to encourage more prototyping of software by making tests super easy to write during the prototyping process, and s

Short version: I'm close to figuring out how to encourage more prototyping of software by making tests super easy to write during the prototyping process, and so de-risking rewrites. But one problem I've been stymied by is how to represent expectations of screens when they contain graphics. Long version: My Mu project (https://github.com/akkartik/mu https://github.com/akkartik/mu) is building a computing stack up from machine code. The goal is partly to figure out why people don't do real prototyping more often. We all know we should throw the first one away (https://wiki.c2.com/?PlanToThrowOneAway https://wiki.c2.com/?PlanToThrowOneAway), but we rarely do so. The hypothesis is that we'd be better about throwing the first one away if rewriting was less risky. By the time the prototyping phase ends, a prototype often tacitly encodes lots of information that is risky to rewrite. To falsify this hypothesis, I want to make it super easy to turn any manual run into a reproducible automated test. If all the tacit knowledge had tests (stuff you naturally did as you built features), rewriting would become a non-risky activity, even if it still requires some effort.Turning manual tests into automated ones requires carefully tracking dependencies and outlawing side-effects. For example, in Mu functions that modify the screen always take a screen object. That way I can start out with a manual test on the real screen, and easily swap in a fake screen to automate the test. Hence my problem: How do you represent screens in a test? Currently I represent screens as 2D arrays of characters. That is doable a lot of the time, but complicates many scenarios: * Text mode character attributes. If I want to check the foreground or background color, I currently use primitives like `check-screen-in-bg` which ignores spaces in a 2D array, but checks that non-spaces match the given background attribute. In practice this leads frequently to tests that first check the character content on a screen and then perform more passes to check colors and other attributes. * Non-text. Checking pixels scales poorly, either line at a time or pixel at a time. A good test should seem self-evident based on the name, but drawing ASCII art where each character is a pixel results in really long lines or stanzas. So far I maintain separate buffers for text vs pixels, so that at least text continues to test easily. * Proportional fonts. Treating the screen as a grid of characters works only when each character is the same width. If widths differ I end up having to go back to treating characters as collections of pixels. So Mu currently doesn't support arbitrary proportional fonts. * Unicode. Mu currently uses a single font, GNU Unifont (http://unifoundry.com/unifont/index.html http://unifoundry.com/unifont/index.html). Unifont is mostly fixed-width, but lots of graphemes (e.g. Chinese, Japanese, Korean, Indian) require double-width to render. That takes us back to the problems of proportional fonts. Currently I permit just variable width in multiples of some underlying grid resolution, but it feels hacky. Can people think of solutions to any of these bullets in a text-based language? Or a more powerful non-text representation?