7 ms·

ML on Apple ][+

120 points by mcramer 1y ago

deleted 1y ago
[deleted]
rob_c 1y ago
Since when did regression get upgraded to full blown ML?
- nekudotayim 1y ago
  What is ML if not interpolation and extrapolation?
  - magic_hamster 1y ago
    A million things. Diffusion, back propagation, attention, to name a few.
    - have-a-break 1y ago
      Back prop and attention are just extensions of interpolation.
      - rob_c 1y ago
        By that logic it's all "just linear maths". Back prop requires and limits to analytically differentiable in a normal way. Attention is... Oh dear comparing linear regression to attention is comparing a diesel jet engine to a horse.
        aleph_naught 1y ago
        It's all just a series of S(S(S(....S(0)))) anyways.
- stonogo 1y ago
  When you find yourself solving NP-hard problems on an Apple II, chances are strong you've entered machine learning territory
- DonHopkins 1y ago
  Since when did ML get upgraded to full blown AI?
  - andai 1y ago
    Since we gave up on AI and ML is eh close enough.
drob518 1y ago
Upvoted purely for nostalgia.
gwbas1c 1y ago
Any particular reason why the author chose to do this on an Apple ][? (I mean, the pictures look cool and all.) IE, did the author want to experiment with older forms of basic; or were they trying to learn more about old computers?
- mcramer 1y ago
  I wrote about my motivation at https://mdcramer.github.io/apple-2-blog/motivation/ https://mdcramer.github.io/apple-2-blog/motivation/, which is obviously tongue in cheek. Tl;dr, I refurbished my Apple ][+ to try to recover a game I wrote in high school (https://mdcramer.github.io/apple-2-blog/recover/ https://mdcramer.github.io/apple-2-blog/recover/). After being unable to find the floppy with the game, I thought I'd try something just for giggles.
shagie 1y ago
One of my early "this is neat" programs was a genetic algorithm in Pascal. You entered a bunch of digits and it "evolved" the same sequence of digits. It started out with 10 random numbers. Their fitness (lower was better) was the sum the difference. So if the target was "123456" and the test number was "214365", it had a fitness of 6. It took the top 5, and then mutated a random digit by a random +/- 1. It printed out each row with the full population. and so you could see it scrolling as it converged on the target number. Looking back, I want to say it was probably the July, 1992 issue of Scientific American that inspired me to write that ( https://www.geos.ed.ac.uk/~mscgis/12-13/s1100074/Holland.pdf https://www.geos.ed.ac.uk/~mscgis/12-13/s1100074/Holland.pdf ) . And as that was '92, this might have been on a Mac rather than an Apple ][+... it was certainly in Pascal (my first class in C was in August '92) and I had access to both at the time (I don't think it was turbo pascal on a PC as this was a summer thing and I didn't have a IBM PC at home at the time). Alas, I remember more about the specifics of the program than I do about what desk I was sitting at.
- Steeeve 1y ago
  I wrote a whole project in pascal around that time. Analyzing two datasets. It was running out of memory the night before it was due, so I decided to have it run twice, once for each dataset. That's when I learned a very important principal. "When something needs doing quickly, don't force artificial constraints on yourself" I could have spent three days figuring out how to deal with the memory constraints. But instead I just cut the data in half and gave it two runs. The quick solution was the one that was needed. Kind of an important memory for me that I have thought about quite a bit in the last 30+ years.
hyperliner 1y ago
[dead]
aardvark179 1y ago
I thought this was going to be about the programming language, and I was wondering how they managed to implement it on a machine that small.
- Scramblejams 1y ago
  Same. What flavor of ML would be the most appropriate for that challenge, do you think?
  - taolson 1y ago
    While not exactly ML, David Turner's Miranda system is pretty small, and might be feasible: https://codeberg.org/DATurner/miranda https://codeberg.org/DATurner/miranda
- noelwelsh 1y ago
  That's also what I was thinking. ML predates the Apple II by 4 years, so I think there is definitely a chance of getting it running! If targetting the Apple IIGS I think it would be very achievable; you could fit megabytes of RAM in those.
  - dekhn 1y ago
    Likely any early implementation of ML would have been on a mainframe or minicomputer, not a 6502. A mainframe/minicomputer would have had oodles of storage (both durable and RAM), as well as a compiler for a high level language (which fits what I can see in https://smlfamily.github.io/history/ML2015-talk.pdf https://smlfamily.github.io/history/ML2015-talk.pdf and other locations).
    - noelwelsh 1y ago
      So I've been mildly nerd sniped. It looks like the first target was a PDP-10 [1]. It ran Stanford Lisp used by the "DEC 10" implementation of ML. The architecture is pretty unusual by modern standards, but it doesn't look to be that powerful and seems to top out at around 1MB of RAM. Next up we have a VAX [2] implementation. It's not clear which specific system it was originally developed for, but we're talking early 80s so it probably wasn't much more powerful than the PDP-10. Either way, I think a maxed Apple IIGS with a hefty 8MB of RAM and perhaps overclocked to 14MHz is more than enough raw power to handle ML. Unfortunately I haven't been sufficiently nerd sniped to actually implement this. I leave that as an exercise for the reader ;-) [1]: https://en.wikipedia.org/wiki/PDP-10 https://en.wikipedia.org/wiki/PDP-10 [2]: https://en.wikipedia.org/wiki/VAX https://en.wikipedia.org/wiki/VAX
amilios 1y ago
Bit of a weird choice to draw a decision boundary for a clustering algorithm...
- mcramer 1y ago
  How so? Drawing decision boundary is a pretty common visualization technique for understanding how an algorithm partitions a data space.
aperrien 1y ago
An Aeon ago in 1984, I wrote a perceptron on the Apple II. It was amazingly slow (20 minutes to complete a recognition pass), but what most impressed me at the time was that it did work. Since that time as a kid I always wondered just how far linear optimization techniques could take us. If I could just tell myself then what I know now...
alexshendi 1y ago
This motivates me to try this on my Ministrel 4th (21th century Jupiter Ace clone).
windsignaling 1y ago
I'm surprised no one else has commented that a few of the conceptual comments in this article are a bit odd or just wrong. > The final accuracy is 90% because 1 of the 10 observations is on the incorrect side of the decision boundary. Who is using K-means for classification? If you have labels, then a supervised algorithm seems like a more appropriate choice. > K-means clustering is a recursive algorithm It is? > If we know that the distributions are Gaussian, which is very frequently the case in machine learning It is? > we can employ a more powerful algorithm: Expectation Maximization (EM) K-means is already an instance of the EM algorithm.
- mcramer 1y ago
  > Who is using K-means for classification? If you have labels, then a supervised algorithm seems like a more appropriate choice. The generated data is labeled but we can imagine those labels don't exist when running k-means. There are many applications for unsupervised clustering. I don't, however, think that there are many applications for running much of anything on an Apple ][+. > K-means clustering is a recursive algorithm My bad. It's iterative. I'll fix that. Thanks. > If we know that the distributions are Gaussian, which is very frequently the case in machine learning Gaussian distributions are very frequent and important in machine learning because of the Central Limit Theorem but, beyond that, you are correct. While many natural phenomena are approximately normal, the reason for the Gaussian's frequent use is often mathematical mathematical convenience. I'll correct my post. > we can employ a more powerful algorithm: Expectation Maximization (EM) Excellent point. I will fix that, too. "While k-means is simple, it does not take advantage of our knowledge of the Gaussian nature of the data. If we know that the distributions are at least approximately Gaussian, which is frequently the case, we can employ a more powerful application of the Expectation Maximization (EM) framework (k-means is a specific implementation of centroid-based clustering that uses an iterative approach similar to EM with 'hard' clustering) that takes advantage of this." Thank you for pointing out all of this!
JSR_FDED 1y ago
Applesoft BASIC is just so darn readable. Youngsters have nothing comparable these days to learn the basics of expressing an algorithm without having to know a lot more. And if it ever became too slow, you could reimplement the slow part in 6502 assembler, which has its own elegance. Great way to learn, glad I came up that way.
nikolay 1y ago
You don't even need a computer for ML [0]! [0]: https://proceedings.mlr.press/v170/marx22a/marx22a.pdf https://proceedings.mlr.press/v170/marx22a/marx22a.pdf

ML on Apple ][+

120 points by mcramer 1y ago

deleted 1y ago

[deleted]

rob_c 1y ago

Since when did regression get upgraded to full blown ML?

nekudotayim 1y ago
What is ML if not interpolation and extrapolation?
- magic_hamster 1y ago
  A million things. Diffusion, back propagation, attention, to name a few.
  - have-a-break 1y ago
    Back prop and attention are just extensions of interpolation.
    - rob_c 1y ago
      By that logic it's all "just linear maths". Back prop requires and limits to analytically differentiable in a normal way. Attention is... Oh dear comparing linear regression to attention is comparing a diesel jet engine to a horse.
      - aleph_naught 1y ago
        It's all just a series of S(S(S(....S(0)))) anyways.
stonogo 1y ago
When you find yourself solving NP-hard problems on an Apple II, chances are strong you've entered machine learning territory
DonHopkins 1y ago
Since when did ML get upgraded to full blown AI?
- andai 1y ago
  Since we gave up on AI and ML is eh close enough.

drob518 1y ago

Upvoted purely for nostalgia.

gwbas1c 1y ago

Any particular reason why the author chose to do this on an Apple ][? (I mean, the pictures look cool and all.) IE, did the author want to experiment with older forms of basic; or were they trying to learn more about old computers?

mcramer 1y ago
I wrote about my motivation at https://mdcramer.github.io/apple-2-blog/motivation/ https://mdcramer.github.io/apple-2-blog/motivation/, which is obviously tongue in cheek. Tl;dr, I refurbished my Apple ][+ to try to recover a game I wrote in high school (https://mdcramer.github.io/apple-2-blog/recover/ https://mdcramer.github.io/apple-2-blog/recover/). After being unable to find the floppy with the game, I thought I'd try something just for giggles.

shagie 1y ago

One of my early "this is neat" programs was a genetic algorithm in Pascal. You entered a bunch of digits and it "evolved" the same sequence of digits. It started out with 10 random numbers. Their fitness (lower was better) was the sum the difference. So if the target was "123456" and the test number was "214365", it had a fitness of 6. It took the top 5, and then mutated a random digit by a random +/- 1. It printed out each row with the full population. and so you could see it scrolling as it converged on the target number. Looking back, I want to say it was probably the July, 1992 issue of Scientific American that inspired me to write that ( https://www.geos.ed.ac.uk/~mscgis/12-13/s1100074/Holland.pdf https://www.geos.ed.ac.uk/~mscgis/12-13/s1100074/Holland.pdf ) . And as that was '92, this might have been on a Mac rather than an Apple ][+... it was certainly in Pascal (my first class in C was in August '92) and I had access to both at the time (I don't think it was turbo pascal on a PC as this was a summer thing and I didn't have a IBM PC at home at the time). Alas, I remember more about the specifics of the program than I do about what desk I was sitting at.

Steeeve 1y ago
I wrote a whole project in pascal around that time. Analyzing two datasets. It was running out of memory the night before it was due, so I decided to have it run twice, once for each dataset. That's when I learned a very important principal. "When something needs doing quickly, don't force artificial constraints on yourself" I could have spent three days figuring out how to deal with the memory constraints. But instead I just cut the data in half and gave it two runs. The quick solution was the one that was needed. Kind of an important memory for me that I have thought about quite a bit in the last 30+ years.

hyperliner 1y ago

[dead]

aardvark179 1y ago

I thought this was going to be about the programming language, and I was wondering how they managed to implement it on a machine that small.

Scramblejams 1y ago
Same. What flavor of ML would be the most appropriate for that challenge, do you think?
- taolson 1y ago
  While not exactly ML, David Turner's Miranda system is pretty small, and might be feasible: https://codeberg.org/DATurner/miranda https://codeberg.org/DATurner/miranda
noelwelsh 1y ago
That's also what I was thinking. ML predates the Apple II by 4 years, so I think there is definitely a chance of getting it running! If targetting the Apple IIGS I think it would be very achievable; you could fit megabytes of RAM in those.
- dekhn 1y ago
  Likely any early implementation of ML would have been on a mainframe or minicomputer, not a 6502. A mainframe/minicomputer would have had oodles of storage (both durable and RAM), as well as a compiler for a high level language (which fits what I can see in https://smlfamily.github.io/history/ML2015-talk.pdf https://smlfamily.github.io/history/ML2015-talk.pdf and other locations).
  - noelwelsh 1y ago
    So I've been mildly nerd sniped. It looks like the first target was a PDP-10 [1]. It ran Stanford Lisp used by the "DEC 10" implementation of ML. The architecture is pretty unusual by modern standards, but it doesn't look to be that powerful and seems to top out at around 1MB of RAM. Next up we have a VAX [2] implementation. It's not clear which specific system it was originally developed for, but we're talking early 80s so it probably wasn't much more powerful than the PDP-10. Either way, I think a maxed Apple IIGS with a hefty 8MB of RAM and perhaps overclocked to 14MHz is more than enough raw power to handle ML. Unfortunately I haven't been sufficiently nerd sniped to actually implement this. I leave that as an exercise for the reader ;-) [1]: https://en.wikipedia.org/wiki/PDP-10 https://en.wikipedia.org/wiki/PDP-10 [2]: https://en.wikipedia.org/wiki/VAX https://en.wikipedia.org/wiki/VAX

amilios 1y ago

Bit of a weird choice to draw a decision boundary for a clustering algorithm...

mcramer 1y ago
How so? Drawing decision boundary is a pretty common visualization technique for understanding how an algorithm partitions a data space.

aperrien 1y ago

An Aeon ago in 1984, I wrote a perceptron on the Apple II. It was amazingly slow (20 minutes to complete a recognition pass), but what most impressed me at the time was that it did work. Since that time as a kid I always wondered just how far linear optimization techniques could take us. If I could just tell myself then what I know now...

alexshendi 1y ago

This motivates me to try this on my Ministrel 4th (21th century Jupiter Ace clone).

windsignaling 1y ago

I'm surprised no one else has commented that a few of the conceptual comments in this article are a bit odd or just wrong. > The final accuracy is 90% because 1 of the 10 observations is on the incorrect side of the decision boundary. Who is using K-means for classification? If you have labels, then a supervised algorithm seems like a more appropriate choice. > K-means clustering is a recursive algorithm It is? > If we know that the distributions are Gaussian, which is very frequently the case in machine learning It is? > we can employ a more powerful algorithm: Expectation Maximization (EM) K-means is already an instance of the EM algorithm.

mcramer 1y ago
> Who is using K-means for classification? If you have labels, then a supervised algorithm seems like a more appropriate choice. The generated data is labeled but we can imagine those labels don't exist when running k-means. There are many applications for unsupervised clustering. I don't, however, think that there are many applications for running much of anything on an Apple ][+. > K-means clustering is a recursive algorithm My bad. It's iterative. I'll fix that. Thanks. > If we know that the distributions are Gaussian, which is very frequently the case in machine learning Gaussian distributions are very frequent and important in machine learning because of the Central Limit Theorem but, beyond that, you are correct. While many natural phenomena are approximately normal, the reason for the Gaussian's frequent use is often mathematical mathematical convenience. I'll correct my post. > we can employ a more powerful algorithm: Expectation Maximization (EM) Excellent point. I will fix that, too. "While k-means is simple, it does not take advantage of our knowledge of the Gaussian nature of the data. If we know that the distributions are at least approximately Gaussian, which is frequently the case, we can employ a more powerful application of the Expectation Maximization (EM) framework (k-means is a specific implementation of centroid-based clustering that uses an iterative approach similar to EM with 'hard' clustering) that takes advantage of this." Thank you for pointing out all of this!

JSR_FDED 1y ago

Applesoft BASIC is just so darn readable. Youngsters have nothing comparable these days to learn the basics of expressing an algorithm without having to know a lot more. And if it ever became too slow, you could reimplement the slow part in 6502 assembler, which has its own elegance. Great way to learn, glad I came up that way.

nikolay 1y ago

You don't even need a computer for ML [0]! [0]: https://proceedings.mlr.press/v170/marx22a/marx22a.pdf https://proceedings.mlr.press/v170/marx22a/marx22a.pdf