5 ms·

Unit testing on an 8-bit CPU

68 points by bertman 3y ago

aunwick 3y ago
Um. History? Yes we could control autonomous robots with 8bit cpus in 1993 and we were engineers so we did it with acceptable risk, aka some testing.
readthenotes1 3y ago
Back in 1988, my manager, who is taking graduate School courses at a local college, requested that we start writing test programs instead of walking down stairs, loading and running our assembly code on the local machine. To generate adequate tests, we would have to simulate part of the actual machine. Fortunately, he understood we were more likely to write a bad emulator and let us get our exercise instead
kragen 3y ago
this stuff about how unit tests aren't tests for units but rather tests that are units is nonsense. the term has been used to mean 'tests for units' for about a century it is certainly true that the optimal amount and kind of testing depends on your development environment. if for some reason you find yourself writing 6809 assembly, here are some ways you can make testing cheaper - don't write the tests in assembly. there are c compilers. forth is also a fine option. or you can test manually from a debugger or monitor program - forth makes a decent monitor program and macro assembler and is scriptable for testing. it's maybe a lot better at that than it is at being a programming language. there are reportedly very good forths for the 6809 - don't do the development self-hosted. you can simulate a 6809 on a laptop or cellphone hundreds of times faster than real time, and the author is in fact doing that. if you have a big computer with either a 6809 simulator or a debugging umbilical to your 6809, you can run the test scripts on the big computer instead of the 6809, while running the code under test on the 6809, real or emulated. then you don't have to write your tests in c or forth; you can write them in tcl, python, js, lua, squirrel, or whatever you prefer. they can control the 6809 like a marionette, and you can use property testing frameworks like hypothesis, which would be very ambitious to implement on the 6809 - maybe non-time-critical parts of your logic can be written in a language that can be compiled to either the 6809 or the big computer, such as c, so you can test it compiled natively on the big computer. cross-compiled to the 6809, this will generate bloated code that uses more memory, so it depends on how tight you are on space, though the 6809 isn't quite as unfriendly to c as something like a 6502 or pdp-10 - the exception is that if you write the non-time-critical parts in an itc forth instead of c or something, it won't use an unreasonable amount of memory, but it will be a lot slower and more bug prone disclaimer, I haven't programmed a 6809 since 01985, and then it was in basic, and my forth isn't all that hot either
- pimlottc 3y ago
  > the term has been used to mean 'tests for units' for about a century I assume you’re being hyperbolic here, unless there’s some pre-modern history to the term I’m not aware of?
  - kragen 3y ago
    no, i am speaking quite precisely. 01923 is hardly medieval, and that's when the term came into common use in engineering jargon, with pretty much its current meaning, plus or minus about ten years https://books.google.com/ngrams/graph?content=unit+tests&year_start=1800&year_end=2019&corpus=en-2019&smoothing=3 https://books.google.com/ngrams/graph?content=unit+tests&yea...
    - eddtries 3y ago
      This is a little off, unless you give “unit” a very different term than most use it. The term unit vs division tests are used in the military, so a unit test is to check combat readiness of a unit and division tests is the combat readiness of a division. The books that use that term mostly come from US Artillery documentation. I wonder if the etymology of modern unit testing does come from that; but if it did I’d assume the other terms would exist too.
      - kragen 3y ago
        interesting, i appreciate the correction. for some reason the google books date range searches linked from the n-grams page weren't loading for me, and the corresponding internet archive searches are also broken there also seem to be some search hits from 'unit tests' for education, in which the 'unit' is presumably a course or subdivision thereof; https://www.google.com.ar/books/edition/The_Organization_of_Unit_Tests_for_Indus/ninFvgEACAAJ?hl=es-419 https://www.google.com.ar/books/edition/The_Organization_of_... is an example, though sadly google refuses to show me any excerpts, perhaps because i'm in a country they don't like https://www.google.com.ar/books/edition/Naval_Training_Bulletin/wpTuxJs9HBwC?hl=es-419&gbpv=1&dq=%22unit+tests%22&pg=RA10-PA17&printsec=frontcover https://www.google.com.ar/books/edition/Naval_Training_Bulle... does show an excerpt so i guess the engineering use might date from later, perhaps as late as the 01960s
ajxs 3y ago
I've been writing a lot of 8-bit assembler for the 6303, which is in the same processor family as the CPU in the TRS-80, which the author is writing code for (for this project in case anyone is wondering: https://github.com/ajxs/yamaha_dx97 https://github.com/ajxs/yamaha_dx97). I ran into the exact same issue. I wrote a MAME driver for the target platform, so I could test my builds on my development machine. Obviously that sped things up a lot. The MAME debugger isn't built for unit testing, and it can't really be 'instrumented', but I was able to write a lot of scripts which could get me pretty close. You can set the debugger to initialise the system RAM from a file when it hits the breakpoint at the start of your function, then test the RAM against the desired output when it when it hits the breakpoint at the end. As clunky as this is, it saved me in a few tricky places!
- zellyn 3y ago
  Writing your own (even not great) emulator also helps a lot for building quick, convenient change/build/test cycles…
- Narishma 3y ago
  > which is in the same processor family as the CPU in the TRS-80 That would be the TRS-80 Color Computer. The TRS-80 used a Z80 CPU.
dools 3y ago
I stopped thinking about "unit testing" and just started thinking "when I write some code, I should write some other code that runs the code automatically". There's every likelihood that both my original code and the code I write to test it will have bugs, but it's somewhat unlikely that the bugs will be perfectly complementary. However that will happen too, and so it is not the role of automated testing to eliminate bugs. It is the role of quality control testing to eliminate bugs. The role of automated testing is to reduce the cost of quality control (by reducing regressions and reducing the amount of time required to run manual quality control testing). I have to credit Sebastian Bergmann with the genesis of this attitude. I took a course on unit testing with him somewhere around 2008 and I said "it seems that test code is just more code, how is it any different from writing code in the rest of your project?" and he said "it isn't, tests are just part of your codebase like all your other code". That was a watershed moment for me -- I always thought I was missing something.
- eddtries 3y ago
  The difference between Quality Control and Quality Assurance is really important but most developers don’t really know the difference, and their only form of quality is QC catching some bugs on the way out rather than QA catching them earlier in the pipeline. I’m a contract test engineer & very rarely see the latter - for example test design alongside feature design. It’s amazing how quickly you can improve a products quality metrics (defect leakage, customer reports, defect costs, etc) by focusing a little on the QA side of development. It’s one of the issues companies have when they think they can just replace QA with developers automated tests - it misses every other part of quality except a subset of QC.
- jjice 3y ago
  > I stopped thinking about "unit testing" and just started thinking "when I write some code, I should write some other code that runs the code automatically". I like that perspective. Personally, the difference of "unit" vs "integration" testing from an organization/thinking perspective for me is what has to be available when it runs. We currently have .test.ts and .int.test.ts files in our code base. The former gets to run really fast without any dependencies, and the latter takes longer to run, but has full access to DBs and Redis. That separation just helps split it up for CI. Thinking about it, you can probably combine them and just require everything is always available, but I've found that the distinction has helped how I think about writing pure functions and allows me to iterate on them with tests easily.
royjacobs 3y ago
Plug plug: I've written an assembler[0] for the 6502 (with full LSP and debugging support). It also supports the concept of unit tests whereby your program gets assembled and every test individually gets assembled and run, whereby you can add certain asserts to check for CPU register states and things like that. [0] See https://mos.datatra.sh/guide/unit-testing.html https://mos.datatra.sh/guide/unit-testing.html
- masfuerte 3y ago
  Adding asserts directly to the assembly code is a really nice idea. I might steal it! P.S. Lines 9 and 10 in your example test use "==" and "=".
PaulHoule 3y ago
I loved coding assembly on for the 6809 for the TRS-80 Color Computer: there was the cartridge editor/assembler, a disk assembler, and eventually an assembler for OS-9 that worked together with C, Pascal, and other compilers. (e.g. if your assembly function or a wrapper around it follows the calling conventions, you could write your tests in an HLL) The 6809 team saw themselves as revolutionaries of system software for microcomputers in that the 6809 supported PIC (Position Independent Code) which could be used to implement shared libraries, even in ROM. So it is totally in character to have your function in one file and have another file with a test harness and another with the actual test that when linked together form a program. So far as the verbosity of the tests you could solve that with a good macro assembler although I don't remember ever using one for the 6809. The Coco3 could support 512kb of RAM so it could have supported a full phat assembler in OS-9. Macro assembly for the IBM 360 was particularly powerful because you could specify a register in an argument, I am wondering today if you could get away in 6809 with adding an "LD[parameter]" instruction which could assemble to LDA, LDB, LDX or such depending on the value of parameter. The tests don't have to be as comprehensive as he says they should be, though some fundamental algorithms really can use a comprehensive test.

68 points by bertman 3y ago

Um. History? Yes we could control autonomous robots with 8bit cpus in 1993 and we were engineers so we did it with acceptable risk, aka some testing.

readthenotes1 3y ago

Back in 1988, my manager, who is taking graduate School courses at a local college, requested that we start writing test programs instead of walking down stairs, loading and running our assembly code on the local machine. To generate adequate tests, we would have to simulate part of the actual machine. Fortunately, he understood we were more likely to write a bad emulator and let us get our exercise instead

kragen 3y ago

this stuff about how unit tests aren't tests for units but rather tests that are units is nonsense. the term has been used to mean 'tests for units' for about a century it is certainly true that the optimal amount and kind of testing depends on your development environment. if for some reason you find yourself writing 6809 assembly, here are some ways you can make testing cheaper - don't write the tests in assembly. there are c compilers. forth is also a fine option. or you can test manually from a debugger or monitor program - forth makes a decent monitor program and macro assembler and is scriptable for testing. it's maybe a lot better at that than it is at being a programming language. there are reportedly very good forths for the 6809 - don't do the development self-hosted. you can simulate a 6809 on a laptop or cellphone hundreds of times faster than real time, and the author is in fact doing that. if you have a big computer with either a 6809 simulator or a debugging umbilical to your 6809, you can run the test scripts on the big computer instead of the 6809, while running the code under test on the 6809, real or emulated. then you don't have to write your tests in c or forth; you can write them in tcl, python, js, lua, squirrel, or whatever you prefer. they can control the 6809 like a marionette, and you can use property testing frameworks like hypothesis, which would be very ambitious to implement on the 6809 - maybe non-time-critical parts of your logic can be written in a language that can be compiled to either the 6809 or the big computer, such as c, so you can test it compiled natively on the big computer. cross-compiled to the 6809, this will generate bloated code that uses more memory, so it depends on how tight you are on space, though the 6809 isn't quite as unfriendly to c as something like a 6502 or pdp-10 - the exception is that if you write the non-time-critical parts in an itc forth instead of c or something, it won't use an unreasonable amount of memory, but it will be a lot slower and more bug prone disclaimer, I haven't programmed a 6809 since 01985, and then it was in basic, and my forth isn't all that hot either

pimlottc 3y ago
> the term has been used to mean 'tests for units' for about a century I assume you’re being hyperbolic here, unless there’s some pre-modern history to the term I’m not aware of?
- kragen 3y ago
  no, i am speaking quite precisely. 01923 is hardly medieval, and that's when the term came into common use in engineering jargon, with pretty much its current meaning, plus or minus about ten years https://books.google.com/ngrams/graph?content=unit+tests&year_start=1800&year_end=2019&corpus=en-2019&smoothing=3 https://books.google.com/ngrams/graph?content=unit+tests&yea...
  - eddtries 3y ago
    This is a little off, unless you give “unit” a very different term than most use it. The term unit vs division tests are used in the military, so a unit test is to check combat readiness of a unit and division tests is the combat readiness of a division. The books that use that term mostly come from US Artillery documentation. I wonder if the etymology of modern unit testing does come from that; but if it did I’d assume the other terms would exist too.
    - kragen 3y ago
      interesting, i appreciate the correction. for some reason the google books date range searches linked from the n-grams page weren't loading for me, and the corresponding internet archive searches are also broken there also seem to be some search hits from 'unit tests' for education, in which the 'unit' is presumably a course or subdivision thereof; https://www.google.com.ar/books/edition/The_Organization_of_Unit_Tests_for_Indus/ninFvgEACAAJ?hl=es-419 https://www.google.com.ar/books/edition/The_Organization_of_... is an example, though sadly google refuses to show me any excerpts, perhaps because i'm in a country they don't like https://www.google.com.ar/books/edition/Naval_Training_Bulletin/wpTuxJs9HBwC?hl=es-419&gbpv=1&dq=%22unit+tests%22&pg=RA10-PA17&printsec=frontcover https://www.google.com.ar/books/edition/Naval_Training_Bulle... does show an excerpt so i guess the engineering use might date from later, perhaps as late as the 01960s

ajxs 3y ago

I've been writing a lot of 8-bit assembler for the 6303, which is in the same processor family as the CPU in the TRS-80, which the author is writing code for (for this project in case anyone is wondering: https://github.com/ajxs/yamaha_dx97 https://github.com/ajxs/yamaha_dx97). I ran into the exact same issue. I wrote a MAME driver for the target platform, so I could test my builds on my development machine. Obviously that sped things up a lot. The MAME debugger isn't built for unit testing, and it can't really be 'instrumented', but I was able to write a lot of scripts which could get me pretty close. You can set the debugger to initialise the system RAM from a file when it hits the breakpoint at the start of your function, then test the RAM against the desired output when it when it hits the breakpoint at the end. As clunky as this is, it saved me in a few tricky places!

zellyn 3y ago
Writing your own (even not great) emulator also helps a lot for building quick, convenient change/build/test cycles…
Narishma 3y ago
> which is in the same processor family as the CPU in the TRS-80 That would be the TRS-80 Color Computer. The TRS-80 used a Z80 CPU.

dools 3y ago

I stopped thinking about "unit testing" and just started thinking "when I write some code, I should write some other code that runs the code automatically". There's every likelihood that both my original code and the code I write to test it will have bugs, but it's somewhat unlikely that the bugs will be perfectly complementary. However that will happen too, and so it is not the role of automated testing to eliminate bugs. It is the role of quality control testing to eliminate bugs. The role of automated testing is to reduce the cost of quality control (by reducing regressions and reducing the amount of time required to run manual quality control testing). I have to credit Sebastian Bergmann with the genesis of this attitude. I took a course on unit testing with him somewhere around 2008 and I said "it seems that test code is just more code, how is it any different from writing code in the rest of your project?" and he said "it isn't, tests are just part of your codebase like all your other code". That was a watershed moment for me -- I always thought I was missing something.

eddtries 3y ago
The difference between Quality Control and Quality Assurance is really important but most developers don’t really know the difference, and their only form of quality is QC catching some bugs on the way out rather than QA catching them earlier in the pipeline. I’m a contract test engineer & very rarely see the latter - for example test design alongside feature design. It’s amazing how quickly you can improve a products quality metrics (defect leakage, customer reports, defect costs, etc) by focusing a little on the QA side of development. It’s one of the issues companies have when they think they can just replace QA with developers automated tests - it misses every other part of quality except a subset of QC.
jjice 3y ago
> I stopped thinking about "unit testing" and just started thinking "when I write some code, I should write some other code that runs the code automatically". I like that perspective. Personally, the difference of "unit" vs "integration" testing from an organization/thinking perspective for me is what has to be available when it runs. We currently have .test.ts and .int.test.ts files in our code base. The former gets to run really fast without any dependencies, and the latter takes longer to run, but has full access to DBs and Redis. That separation just helps split it up for CI. Thinking about it, you can probably combine them and just require everything is always available, but I've found that the distinction has helped how I think about writing pure functions and allows me to iterate on them with tests easily.

royjacobs 3y ago

Plug plug: I've written an assembler[0] for the 6502 (with full LSP and debugging support). It also supports the concept of unit tests whereby your program gets assembled and every test individually gets assembled and run, whereby you can add certain asserts to check for CPU register states and things like that. [0] See https://mos.datatra.sh/guide/unit-testing.html https://mos.datatra.sh/guide/unit-testing.html

masfuerte 3y ago
Adding asserts directly to the assembly code is a really nice idea. I might steal it! P.S. Lines 9 and 10 in your example test use "==" and "=".

PaulHoule 3y ago

I loved coding assembly on for the 6809 for the TRS-80 Color Computer: there was the cartridge editor/assembler, a disk assembler, and eventually an assembler for OS-9 that worked together with C, Pascal, and other compilers. (e.g. if your assembly function or a wrapper around it follows the calling conventions, you could write your tests in an HLL) The 6809 team saw themselves as revolutionaries of system software for microcomputers in that the 6809 supported PIC (Position Independent Code) which could be used to implement shared libraries, even in ROM. So it is totally in character to have your function in one file and have another file with a test harness and another with the actual test that when linked together form a program. So far as the verbosity of the tests you could solve that with a good macro assembler although I don't remember ever using one for the 6809. The Coco3 could support 512kb of RAM so it could have supported a full phat assembler in OS-9. Macro assembly for the IBM 360 was particularly powerful because you could specify a register in an argument, I am wondering today if you could get away in 6809 with adding an "LD[parameter]" instruction which could assemble to LDA, LDB, LDX or such depending on the value of parameter. The tests don't have to be as comprehensive as he says they should be, though some fundamental algorithms really can use a comprehensive test.