3 ms·

Show HN: Open Source emulator in Javascript that runs Linux

65 points by reginaldo 15y ago

reginaldo 15y ago
Since there seems to be some interest on the subject, and I don't want to kidnap the great thread about the worst Linux PC ever [1], I decided to submit on a separate thread. Some things do keep in mind: 1. Very early stage. Runs better on Chrome. 2. If you think this sucks, you're probably comparing my code to Fabrice Bellard's Jslinux[2], and as Bellard is a genius his code (which is unfortunately still not available in readable form) is probably much better than mine. But progress is being made. 3. Still, the potential is there, soon it will be possible to have a somewhat standard programming stack and write GUI applications that run off a canvas, as I said in [3]. 4. For Chrome there is a nice terminal emulator (hterm) I lifted from the Chromium source tree. The rest use textarea. 5. The code, of course: https://github.com/ubercomp/jslm32/ https://github.com/ubercomp/jslm32/ If anyone has questions, I'll be happy to answer them. [1] http://news.ycombinator.com/item?id=3767410 http://news.ycombinator.com/item?id=3767410 [2] http://bellard.org/jslinux http://bellard.org/jslinux [3] http://news.ycombinator.com/item?id=3769486 http://news.ycombinator.com/item?id=3769486 EDIT: Added itens 4 and 5.
fudged 15y ago
Is the idea of an "average" customer really this lost on so many tech writers? I thought my little sister was tech-literate, but she doesn't give a shit about "Ice Cream Sandwich"... she has customized the hell out of her 2.3 handset and there's nothing that she wants changed. How many "average" customers are entirely satisfied with the smallest amount of customization the system allows? I'm returning my new iPad this week because 1) My Macbook Air is light and more useful, I have no drawing ability, and I don't read much; the iPad is no more than a toy and waste of money in my ecosystem, and 2) The app store ecosystem is dreadfully boring on iOS. On Android I feel like I'm actually rewarding the good developers by buying their Pro versions of apps and such... on iOS everyone wants your money, regardless of how shitty the product is. On Android, I see so many developers experimenting across so many categories just for the fun of it. 3) High-resolution has hardly benefitted anything that I do on the device. The majority of content that I see (images and video) are low-res. Plus, very few apps that I use have been upgraded to 'retina' quality. ALSO, retina doesn't just mean scaling everything up... why is nobody making their content more detailed? Orthogonal lines don't render that differently in higher resolution.
jbert 15y ago
Very nice. Is there any chance of a follow up on your comment in the previous thread: "I'll write a post describing what I did to take boot time from 2.5 minutes to 10 seconds." My interests: - how did you find what to optimise? - what approach did you use?
- reginaldo 15y ago
  > - how did you find what to optimise? - what approach did you use? I used profilers: Chrome developer tools and Firebug. The Chrome profiler is very nice and easy to use, but it looks like it is a statistical profiler. When the usual invocation takes less than 1ms, it might get the times wrong (there might be a bottleneck that Chrome profiler is not showing). The Firebug profiler is more precise, I believe it traces every executing instruction, which also makes it much slower to run you program in, but does sometime gives you interesting insights. Also, there's intuition. I wanted to keep the code readable. The interpreter loop is basically something to the effect of while(true) { op = decode(mmu.read_32(cs.pc)); cs.pc += 4; (optable[opcode])(op); } which I believe is very readable, but it counters everything one expects about writing fast javascript code that runs in a tight loop. Bellard told me he thinks my "dynamic interpreted" version might be slower than a throughly optimized interpreter version, and he might very well be true. But I found the readability of the interpreter to be of the upmost importance, at least until I had a working "dynamic recompilation" version. To optimize the interpreter version, the profiler showed me that basically three things take most (about 65, 70% of the time: * Instruction decoding * Memory access There's not much I found to make instruction decoding faster , but the memory access time could be improved. In the beginning, I used the form mmu.read_32(addr) to read from any region of the memory, including RAM. Most of the accesses are of course from RAM, so it made sense to optimize for this fact (despite it being bad for readability). So nowadays I do: op = (v8[rpc] << 24) | (v8[rpc + 1] << 16) | (v8[rpc + 2] << 8) | (v8[rpc + 3]); For memory accesses. Some other things I found randomly. The processor in the board I'm emulating has a 75Mhz clock. With that emulated frequency, Linux was spending a lot of time on the "idle task". Whan I told the emulator that the frequency was about 8Mhz, the problem went away. Then, I thought to myself: it would be neat if I could implement parts of the libc in Javascript. That's how https://github.com/ubercomp/jslm32/blob/master/src/lm32_libc_dev.js https://github.com/ubercomp/jslm32/blob/master/src/lm32_libc... came to be. So I implemented some string.h functions in javascript (memcpy, memmove, and memset), and voilá, the boot time went down about 3 times. But in the end of the day, lm32_libc_dev is a hack with very negative consequences. I would have to modify every program I wanted to run on the emulated board to use my libc functions instead. I did this with the linux kernel and it was a bad experience. So I focused my energy on the dynamic code generation side of things. I can tell you I have succeeded as the jslibc thing is now deleted on my local source three and I didn't experience any slowdowns whatsoever.
rdg 15y ago
Hey reginaldo, for some reason I cannot enter any text on the terminal, hence, I cannot login. I'm using Chrome 18 on OS X Lion. Just in case, I tried downloading the code and running it locally: same result. Any ideas?
- reginaldo 15y ago
  Yeah, it is a known bug. The hterm terminal emulator runs on its own iframe, so you'll have to click anywhere on the screen (outside the terminal). If that doesn't work, then it's a new bug...
int3 15y ago
You mentioned in the 'Worst Linux PC' thread that your implementation generates JS code on the fly and 'captures up to three backwards jumps'. Do you mind explaining in more detail how this works?
- reginaldo 15y ago
  Ok, one way to do interpretation is to do a loop like while True: instr = decode_instruction(pc) execute_instruction(instr) Where execute instruction is usually a big switch that has every opcode on it, or, in my case, is a call to a function in a table whose indices are opcodes. The problem is you decode the same instructions many times (in a loop, for instance). A slight more elaborated version is like this: while True: if not pc in cached_code: block = generate_block(pc) cached_code[pc] = block run_from_cache(pc) The generate_block function decodes and generates code for every instruction until it finds a jump. When it finds it, it finishes up that block, which is then added to the cache. Now, whenever there is a jump to an address that is in the cache, the cached code can be executed directly. Finally, the captures up to three backwards jumps thing: My first try was to have the cached code just be a regular function call without loops. So a block of code is executed and then control is given back to the main loop. But a significant number of jumps have targets that are inside the same block of code (anything you run on a loop, for instance). So, to get better performance, I generate, for my block of code, a function that has a loop in it. It has a rather characteristic look: function code_block(cs) { // cs is the "cpu state" var count = 0; cs.next_pc = BLOCK_START; block_loop: while(++count < 3) { count++; block_choice: switch(cs.next_pc) { case BLOCK_START: cs.r0 = cs.r0 ^cs.r0; case BLOCK_START+4: // regular instructions fall through case CONDITIONAL_BRANCH_ADDR: if(branch_condition) { cs.next_pc = BRANCH_TARGET; break block_choice; // this captures backwards jumps targeting the block } case UNCONDITIONAL_BRANCH_ADDR: cs.next_pc = BRANCH_TARGET; default: break block_loop; } } } The "captures up to three backwards jumps things" is controlled by the ++count < 3 loop condition. 3 was obtained by trial and error and looks like a good number for my machine (I could, for instance, put 100 in there, but then my javascript functions would take too much time to return and the javascript engines don't like this).

65 points by reginaldo 15y ago

Since there seems to be some interest on the subject, and I don't want to kidnap the great thread about the worst Linux PC ever [1], I decided to submit on a separate thread. Some things do keep in mind: 1. Very early stage. Runs better on Chrome. 2. If you think this sucks, you're probably comparing my code to Fabrice Bellard's Jslinux[2], and as Bellard is a genius his code (which is unfortunately still not available in readable form) is probably much better than mine. But progress is being made. 3. Still, the potential is there, soon it will be possible to have a somewhat standard programming stack and write GUI applications that run off a canvas, as I said in [3]. 4. For Chrome there is a nice terminal emulator (hterm) I lifted from the Chromium source tree. The rest use textarea. 5. The code, of course: https://github.com/ubercomp/jslm32/ https://github.com/ubercomp/jslm32/ If anyone has questions, I'll be happy to answer them. [1] http://news.ycombinator.com/item?id=3767410 http://news.ycombinator.com/item?id=3767410 [2] http://bellard.org/jslinux http://bellard.org/jslinux [3] http://news.ycombinator.com/item?id=3769486 http://news.ycombinator.com/item?id=3769486 EDIT: Added itens 4 and 5.

fudged 15y ago

Is the idea of an "average" customer really this lost on so many tech writers? I thought my little sister was tech-literate, but she doesn't give a shit about "Ice Cream Sandwich"... she has customized the hell out of her 2.3 handset and there's nothing that she wants changed. How many "average" customers are entirely satisfied with the smallest amount of customization the system allows? I'm returning my new iPad this week because 1) My Macbook Air is light and more useful, I have no drawing ability, and I don't read much; the iPad is no more than a toy and waste of money in my ecosystem, and 2) The app store ecosystem is dreadfully boring on iOS. On Android I feel like I'm actually rewarding the good developers by buying their Pro versions of apps and such... on iOS everyone wants your money, regardless of how shitty the product is. On Android, I see so many developers experimenting across so many categories just for the fun of it. 3) High-resolution has hardly benefitted anything that I do on the device. The majority of content that I see (images and video) are low-res. Plus, very few apps that I use have been upgraded to 'retina' quality. ALSO, retina doesn't just mean scaling everything up... why is nobody making their content more detailed? Orthogonal lines don't render that differently in higher resolution.

jbert 15y ago

Very nice. Is there any chance of a follow up on your comment in the previous thread: "I'll write a post describing what I did to take boot time from 2.5 minutes to 10 seconds." My interests: - how did you find what to optimise? - what approach did you use?

reginaldo 15y ago
> - how did you find what to optimise? - what approach did you use? I used profilers: Chrome developer tools and Firebug. The Chrome profiler is very nice and easy to use, but it looks like it is a statistical profiler. When the usual invocation takes less than 1ms, it might get the times wrong (there might be a bottleneck that Chrome profiler is not showing). The Firebug profiler is more precise, I believe it traces every executing instruction, which also makes it much slower to run you program in, but does sometime gives you interesting insights. Also, there's intuition. I wanted to keep the code readable. The interpreter loop is basically something to the effect of while(true) { op = decode(mmu.read_32(cs.pc)); cs.pc += 4; (optable[opcode])(op); } which I believe is very readable, but it counters everything one expects about writing fast javascript code that runs in a tight loop. Bellard told me he thinks my "dynamic interpreted" version might be slower than a throughly optimized interpreter version, and he might very well be true. But I found the readability of the interpreter to be of the upmost importance, at least until I had a working "dynamic recompilation" version. To optimize the interpreter version, the profiler showed me that basically three things take most (about 65, 70% of the time: * Instruction decoding * Memory access There's not much I found to make instruction decoding faster , but the memory access time could be improved. In the beginning, I used the form mmu.read_32(addr) to read from any region of the memory, including RAM. Most of the accesses are of course from RAM, so it made sense to optimize for this fact (despite it being bad for readability). So nowadays I do: op = (v8[rpc] << 24) | (v8[rpc + 1] << 16) | (v8[rpc + 2] << 8) | (v8[rpc + 3]); For memory accesses. Some other things I found randomly. The processor in the board I'm emulating has a 75Mhz clock. With that emulated frequency, Linux was spending a lot of time on the "idle task". Whan I told the emulator that the frequency was about 8Mhz, the problem went away. Then, I thought to myself: it would be neat if I could implement parts of the libc in Javascript. That's how https://github.com/ubercomp/jslm32/blob/master/src/lm32_libc_dev.js https://github.com/ubercomp/jslm32/blob/master/src/lm32_libc... came to be. So I implemented some string.h functions in javascript (memcpy, memmove, and memset), and voilá, the boot time went down about 3 times. But in the end of the day, lm32_libc_dev is a hack with very negative consequences. I would have to modify every program I wanted to run on the emulated board to use my libc functions instead. I did this with the linux kernel and it was a bad experience. So I focused my energy on the dynamic code generation side of things. I can tell you I have succeeded as the jslibc thing is now deleted on my local source three and I didn't experience any slowdowns whatsoever.

rdg 15y ago

Hey reginaldo, for some reason I cannot enter any text on the terminal, hence, I cannot login. I'm using Chrome 18 on OS X Lion. Just in case, I tried downloading the code and running it locally: same result. Any ideas?

reginaldo 15y ago
Yeah, it is a known bug. The hterm terminal emulator runs on its own iframe, so you'll have to click anywhere on the screen (outside the terminal). If that doesn't work, then it's a new bug...

int3 15y ago

You mentioned in the 'Worst Linux PC' thread that your implementation generates JS code on the fly and 'captures up to three backwards jumps'. Do you mind explaining in more detail how this works?

reginaldo 15y ago
Ok, one way to do interpretation is to do a loop like while True: instr = decode_instruction(pc) execute_instruction(instr) Where execute instruction is usually a big switch that has every opcode on it, or, in my case, is a call to a function in a table whose indices are opcodes. The problem is you decode the same instructions many times (in a loop, for instance). A slight more elaborated version is like this: while True: if not pc in cached_code: block = generate_block(pc) cached_code[pc] = block run_from_cache(pc) The generate_block function decodes and generates code for every instruction until it finds a jump. When it finds it, it finishes up that block, which is then added to the cache. Now, whenever there is a jump to an address that is in the cache, the cached code can be executed directly. Finally, the captures up to three backwards jumps thing: My first try was to have the cached code just be a regular function call without loops. So a block of code is executed and then control is given back to the main loop. But a significant number of jumps have targets that are inside the same block of code (anything you run on a loop, for instance). So, to get better performance, I generate, for my block of code, a function that has a loop in it. It has a rather characteristic look: function code_block(cs) { // cs is the "cpu state" var count = 0; cs.next_pc = BLOCK_START; block_loop: while(++count < 3) { count++; block_choice: switch(cs.next_pc) { case BLOCK_START: cs.r0 = cs.r0 ^cs.r0; case BLOCK_START+4: // regular instructions fall through case CONDITIONAL_BRANCH_ADDR: if(branch_condition) { cs.next_pc = BRANCH_TARGET; break block_choice; // this captures backwards jumps targeting the block } case UNCONDITIONAL_BRANCH_ADDR: cs.next_pc = BRANCH_TARGET; default: break block_loop; } } } The "captures up to three backwards jumps things" is controlled by the ++count < 3 loop condition. 3 was obtained by trial and error and looks like a good number for my machine (I could, for instance, put 100 in there, but then my javascript functions would take too much time to return and the javascript engines don't like this).