10 ms·
Live coding a vi for CP/M from scratch
- userbinator 7y agoThe source is very straightforward and readable. I suspect part of the reason why is because live-coding forces you to eschew additional complexity/abstraction in favour of the simplest solution. That said, I'm not sure if the additional complexity of a gap buffer is worth it --- a lot of editors in the early days of DOS used a single buffer, effectively memmove()'ing the tail part when you inserted characters, and there was no real noticeable lag even when entering characters at the beginning of a "large" (i.e. >100KB) file. On a CP/M system with 64K of total address space at most, it just doesn't take all that long to go through all of memory. As an advantage, cursor movement code is greatly simplified and doesn't involve modifying the buffers.
- david-given 7y agoThe 8086 could move memory _way_ faster than the 8080 could! Also the ACK's memmove/memcpy is truly dreadful. I need to replace it with hand-tooled assembler ASAP. It's tougher than it looks, though; the 8080 only has two register pairs once you've allocated one for the frame pointer... It also doesn't help cursor movement as much as you'd think; editors don't address characters very much, instead preferring to address lines. For that you still need to scan forwards or backwards looking for line breaks. It would let you use memchr and memrchr to find them, though.
- teddyh 7y agoA description and discussion of different implementation strategies can be found here: https://www.finseth.com/craft/#c6.4 https://www.finseth.com/craft/#c6.4
- ncmncm 7y agoAs he says, TECO used a gap buffer in the early 60s, and he would like to know why authors of later editors made such bad choices. My guess is that they took CS classes, and actually thought things taught there were good ideas and not desperate dodges for when the simple thing won't work.
- kragen 7y agoI don't know the 8080 instruction set, but the memcpy in https://en.m.wikipedia.org/wiki/Intel_8080#Example_code https://en.m.wikipedia.org/wiki/Intel_8080#Example_code has 7 instructions in its inner loop and copies 2 bytes per iteration, which is probably about 20 microseconds at 2MHz and thus 640 milliseconds to copy 64K. That's a really long time to handle a keystroke, since they may be coming in every 100 ms or less. You can probably unroll the loop productively to some extent but you probably can't speed it up by the necessary order of magnitude. Gap buffers, by contrast, should keep everything under a millisecond except search and moving the gap a long distance. Many gap-buffer editors don't move the gap whenever the cursor moves; they wait until you modify something. But that does seem like it would be more complicated. I'm not sure how much simpler it can get than the gap-buffer case.
- tom_ 7y agoEven a Z80 would struggle to keep up: you're looking at something like 17-18 T-states/byte for a useful memcpy, which is ~215 KBytes/sec at 4 MHz. Suppose your document is 32 KBytes, and you're inserting chars at the start. It's going to take 0.2 seconds per keypress! You'd notice. (0.2 seconds per keypress = 300 keypresses per second. Assume 6 keypresses per word (5 letters and a space), and that's 50 wpm. You probably type faster than that.)
- kragen 7y agoThanks! I don't know much about 8080-family performance, but I remember sometimes WordStar would fall behind reasonably fast typing on a 4MHz Z80, even 50 or 60 wpm. But it was doing word wrap, too. I think you mean 300 keypresses per minute.
- deleted 7y ago[deleted]
- Someone 7y ago”Suppose your document is 32 KBytes, and you're inserting chars at the start.” Who has such enormous documents, and who inserts chars at the start :-) ? Given the 64kB limit for (program+data), you’ll soon need some way to keep things on disk (most likely by having a master document that lists the names of the files storing each ‘chapter’ in the complete document, and requiring the user to manually switch from file to file) Therefore, I would implement that, search/replace and a yank buffer before I started worrying about such relatively edgy performance issues. For comparison, the original MacWrite on a 128k Macintosh handled only 2 to 3 pages of text, and also soon got slower than that 0.2s per key press. It was smart enough to skip updates if typing was relatively fast, though. that is a feature you might want to add (e.g. by peeking into the keyboard buffer before refreshing the screen), as the best way to speed up code is by doing less.
- ohazi 7y agoNine straight hours. Holy crap. I'm sure we've all had marathon sessions from time to time, but maintaining enough composure to keep the livestream going for that long (without breaks?) sounds extremely challenging.
- abvr 7y agoAbsolute madlad.
- ur-whale 7y agoActually, it depends on your age.
- ambicapter 7y agoGeoHotz has a ten hour livestream of him coding up SLAM.
- kazinator 7y ago> Nine straight hours. Holy crap. Day in the life of a full-time software dev.
- saagarjha 7y agoDo you not eat lunch or get up to go for walks?
- kazinator 7y agoThere have been times when in fact no.
- closeparen 7y agoSix of those hours (on a good day) would be spent in meetings, answering emails, helping interns, giving code reviews for colleagues, writing documents, reviewing documents, getting lunch/snacks/coffee, waiting in line for the bathroom or elevator, and attempting to recover focus when you return to your desk after any of the above.
- slater 7y agoSpeaking of live coding, does anyone remember a certain longhaired Russian guy (I think he works/worked for Yandex?) who used to do live-coding streams?
- karlmcguire 7y agoBisqwit?
- mindw0rk 7y agoBisqwit is neither russian (afaik he is from Finland) nor long-haired
- userbinator 7y agoYes, he is Finnish: https://bisqwit.iki.fi/cv.html https://bisqwit.iki.fi/cv.html
- protoloc 7y agoMaybe you are referring to this guy? https://www.youtube.com/channel/UCJAVLOqT6Mgn_YD5lAxxkUA https://www.youtube.com/channel/UCJAVLOqT6Mgn_YD5lAxxkUA
- slater 7y agoYep, that was the guy! Seems to have stopped, alas.
- snazz 7y agoRecording yourself for YouTube has to help, but I’m really interested in what the author used to stay focused and productive for an entire nine hours. I can barely make it a half hour before getting distracted by something.
- airstrike 7y agoI wouldn't rule out Adderall, though to be clear I'm not suggesting others take it
- cmarschner 7y agoI think the secret is adding a livestream.
- telotortium 7y agoHe did take a dinner and a tea break, so he wasn't working completely straight
- david-given 7y agoThere are a couple of short (a few seconds only) edits where I just zoned out thinking about something, but the big breaks are clearly labelled. Also, check out the clock at the bottom right corner! I started this at about 14:00 and finished at about 01:30, so about eleven to twelve hours wall clock time. And yes, if I hadn't been doing the recording I'd have given up and gone to bed. Also I, er, had to get up at 07:00 the next morning to go hiking.
- comboy 7y agoDo the thing that is more interesting to you than anything else.
- fernandotakai 7y agofor me, at least, even if something is one of the most interesting things ever (be it videogames, coding or even watching series), i just can't do 9h straight. even for 2~3h it's hard to keep my concentration.
- fsiefken 7y agoBut Why... I know the answer and the site gives a good explanation. But still, practically speaking? Who cares about CP/M, perhaps because it's the only useful OS running on 8 bit processors? Does CP/M have something DOS doesn't have? Or is it impossible to run DOS on the Z80?
- soneil 7y agoI think that is a large part of it, yes. There's been a good few kit computers in the last few years - the rc2014 being my favourite (not because it's particularly amazing, but because it's designed to be understandable and digestable), and cp/m is the most approachable (roughly dos-like) OS available for them. (There's also Fuzix, which seems like a monster accomplishment in itself - but I haven't tried it myself. I rather suspect that running a unix-like on an 8bit just ends up feeling like a very slow unix, rather than anything special) (and no, msdos doesn't run on most 8bits. The Intel 8080 was 8bit. The Z80 expanded on that with additional instructions, and cp/m lives in this branch of the family tree. The 8086 gained 16bit instructions, and branches off in an entirely different direction birthing the x86 family tree, which is where you'll find msdos & most variants)
- david-given 7y agoFuzix is amazing. I did a port to the MSP430, a Texas Instruments single-board computer with 66kB of RAM; it feels just like a Unix, despite having no MMU and no multitasking... http://cowlark.com/2015-10-27-fuzix http://cowlark.com/2015-10-27-fuzix I find CP/M interesting because it's provably the minimum viable product for an operating system. It was the personal computer desktop operating system that really worked. It came with full documentation, a full development systems (an assembler and a debugger), and a full set of tooling to allow you to port CP/M to new systems. It was portable, too, and you could run the same program unmodified on nearly any hardware which supported CP/M. So it was this great open-ended system, cheap enough to get into the hands of ordinary people, simple enough that you could understand it all, and yet sophisticated enough that, with enough time, you could achieve _anything_ on it... The CP/M kernel is 3.5kB of 8080 machine code. I think each frame of that video consumed more space than that.
- SamHamou 7y agoI think the secret is adding a livestream.
- UncleSlacky 7y agoProbably a noob question, but why not just compile vi on a CP/M C compiler? After a quick search, I found this being discussed on comp.os.cpm just last year: https://groups.google.com/forum/#!topic/comp.os.cpm/lfuEr0hRcrM https://groups.google.com/forum/#!topic/comp.os.cpm/lfuEr0hR... It's hosted here: https://github.com/udo-munk/s https://github.com/udo-munk/s
- csb6 7y agoI’m pretty sure they are doing this partly for educational/fun reasons
- david-given 7y agoThere are plenty of CP/M C compilers (I'm planning on including BDS C into cpmish at some point: https://en.wikipedia.org/wiki/BDS_C https://en.wikipedia.org/wiki/BDS_C It's written in hand-tooled assembly!) but with the exception of Hitech C they're all Small C or K&R C. Hitech C works fine, but it's extremely slow and somewhat quirky. CP/M's really a bit small for an ANSI C compiler. (Also Hitech C only targets Z80, not 8080.) The main reasons for doing this cross-compiled were: that's what I had; life was too short; I wanted something for the cpmish cross-compilation environment _anyway_. Making cpmish self-hosted would be awesome, though. Hitech C used to be a free-as-in-beer download from htsoft's website, but it vanished about twelve years ago. I wonder if they'd be open to open sourcing it...
- Zardoz84 7y agoHiTech not was bought by Microchip ?
- david-given 7y agoSo they were. That explains why the only compilers on their web site are Microchip ones... oh well, worth an email to them asking what happened; maybe they'd be willing to open source the CP/M compiler!
- cschep 7y agoThis type of content is why I come to Hacker News, brilliant!
- dancek 7y agoI don't like watching videos, but the list of milestones and timestamps is great. Text editors are one of those things that would be nice to know how to implement, but that never was high enough on my priority list. Looking at this, I feel like I've significantly overestimated how hard it is to write the UI layer. Granted, I'm going to be maybe much slower, but still this seems like a doable project even if I don't have that much time. Why don't we have a list of nice hacker projects somewhere? With an estimate of skill and effort required, relative to other projects?
- david-given 7y agoHello, author here. Eeeaggh. Yes, this took a while. I started after lunch and finished sometime after midnight (look at the clock in the bottom right hand corner of the screen). I'm still kicking myself at some of the dead ends I went through. The result (even after I cleaned it up after the stream) is still a bit slow. The biggest issue is that it redraws the entire line every time you press a key in insert mode; fixing this is moderately straightforward, as I have all the information to know where the first modified character is, but it'll require some refactoring. The biggest think I want next is a yank buffer, but the only realistic option is to store it on disk, which would be painfully slow. Being able to do repeated inserts would be nice, too. I actually know Bram Moolenaar; maybe I should ask how he did it...
- kragen 7y agoBravo! You should be super proud! On a 9600 baud terminal, a whole line is 77 milliseconds, so it's going to feel a bit slow even if you only redraw half the line, and at 2400 baud it would be painful. Insert-character escape sequences are going to make a big difference. I’m no expert, but I think the Kaypro doesn't have a real serial port between the terminal and the computer the way a lot of CP/M machines did (Altair and IMSAI usually, H-8 and H-88 and H-89 always, while the Osborne used a memory-mapped screen like the Apple). I don't know how fast it can repaint the screen. I don't think firing the loud solenoids on the floppy drive and waiting a second or two every time you delete something (or (p)ut it) is going to be a good user experience; the yank buffer probably needs to be in RAM to be usable. You could maybe prompt the user whether to discard the deletion if the text being deleted is over a threshold of size.
- rbanffy 7y agoAltairs and IMSAIs didn't have standard terminals. You could add an S-100 video board with video output and keyboard input (usually something that would output ASCII) or you could attach a real terminal over a serial line on an S-100 board (which is kind of nice if you want to be far from the computer but not that useful for a desktop). Many CP/M desktops used the "bus attached" terminal (because it saves components). A common arrangement (and not only for CP/M machines) would be a CRT controller managing its own memory where the program/OS would address the video memory through the microprocessor's IO channels. This was slow compared to having video driven from main memory (like the Apple II) but CRTCs did a lot the hardware designer wouldn't need to worry about, it was many orders of magnitude faster than a 9600 bps serial terminal, and allowed moving terminal emulation to software/BIOS. The Apple II had a couple "terminal" boards more or less like what I described that allowed it to display 80 and 132-column video. Their memory and registers would be manipulated through memory access as the 6502 didn't have dedicated IO functions. One very common was the Videx Videoterm, that used a 6845 just like the IBM CGA and MDA boards (but with a much nicer VT-100-like font). The Microsoft CP/M SoftCard emulates a SOROQ IQ-120 terminal in software.
- kristianp 7y agoThis guy has some incredible hacks, such as the mandelbrot for BBC Micro (in 6502 assembly) that uses a lookup table for squares of numbers, and takes advantage of the connectedness of the colours: "It turns out that Mandelbrot and Julia sets have the property that any area which is completely bounded by a colour contains only that colour. (Because it’s simply connected.) (No, I don’t understand a word of that paper.) So the program recursively traces out the bounds of boxes, and if all sides of the box are the same colour it fills it and doesn’t recurse in." http://cowlark.com/2018-05-26-bogomandel/index.html http://cowlark.com/2018-05-26-bogomandel/index.html Also a bootloader for the GPU of the rasperry Pi: http://cowlark.com/piface/ http://cowlark.com/piface/
- david-given 7y agoThanks very much! But someone else worked out the maths and the base kernel --- I just did the implementation (and loads of other people chimed in on microoptimisations). I _do_ think it may be the fastest 8-bit Mandelbrot in the world, although some of the stuff the C64 people have done is completely insane, so I'd want to check there first. This is all mine, though: http://cowlark.com/2013-10-19-insane-make http://cowlark.com/2013-10-19-insane-make
- ladybro 7y agoHey, I wrapped + annotated the video to make your timestamps clickable: https://use.mindstamp.io/video/QUqevzIz?notes=1 https://use.mindstamp.io/video/QUqevzIz?notes=1
- thanatropism 7y agoHi, So — why not Wordgrinder?