29 ms·

Show HN: Redbean – Single-file distributable web server

• quickthrower2 6y ago
  Cute idea. Sounds like magic.
  • edoceo 6y ago
    Built with actual magic: https://storage.googleapis.com/justine/ape.html https://storage.googleapis.com/justine/ape.html From the same person I think
    • indigodaddy 6y ago
      Author mentions that the APE’s can be booted from the BIOS. Would this imply you could perhaps implement a lambda-like cloud service that could spawn/start the APE on demand? Edit, I’d guess you’d want to/have to, also have a tcp/ip stack in the APE as well (but perhaps just going for containerization of the APE is the better play and not have to worry about any of that..)
• Lorin 6y ago
  No relation to http://redbeanphp.com/ http://redbeanphp.com/ I guess :D
  • rebelpixel 6y ago
    I think that's still worth mentioning considering they're roughly in the same sphere of interest, to avoid the confusion.
• Hayvok 6y ago
  Zero/near-zero dependencies, minimal configuration, portable, and trivial to run. Awesome. Even if Redbean here doesn't end up taking over the world, I hope that more ecosystems and toolchains sit up and take notice. We need more tools that look like this.
• mikewarot 6y ago
  I really like your view of the world, that programs should be portable, tiny, and just work. The slamming of so much functionality into a zip file is inspiring. Would it be possible to do something similar with Free Pascal, allowing the recreation of something like Turbo Pascal, except really, really portable .com output?
  • jart 6y ago
    Author here. Absolutely. I used to love Turbo Pascal and Delphi when I was younger. If Free Pascal uses GNU LD.BFD or LLVM LLD when it links programs, then all you'd need to do is is configure it to use cosmopolitan.a when linking system call functions like read(), write(), etc. See https://github.com/jart/cosmopolitan https://github.com/jart/cosmopolitan Another option is if Free Pascal wants to write all the system call support from scratch, then doing that now is going to be a whole lot easier since the Cosmopolitan codebase does a really good job documenting all the magic numbers you'll need. See files like https://github.com/jart/cosmopolitan/blob/master/libc/sysv/syscalls.sh https://github.com/jart/cosmopolitan/blob/master/libc/sysv/s... and https://github.com/jart/cosmopolitan/blob/master/libc/sysv/consts.sh https://github.com/jart/cosmopolitan/blob/master/libc/sysv/c... I've been working on a tiny C11 compiler called chibicc which has most GNU extensions and I managed to get it working as an actually portable executable with an integrated assembler: https://github.com/jart/cosmopolitan/blob/master/third_party/chibicc/README.cosmo https://github.com/jart/cosmopolitan/blob/master/third_party... I also got Antirez's KILO text editor working as an APE binary. https://github.com/jart/cosmopolitan/blob/master/examples/kilo.c https://github.com/jart/cosmopolitan/blob/master/examples/ki... If we can build a linker too then we can get a freestanding single file toolchain + ide that's able to be a modern version of Turbo C.
    • akavel 6y ago
      I think the Zig community/BDFL might appreciate adding an APE target "os" to their buildchain (LLVM-based); that could be a match made in heaven!
      • jart 6y ago
• francislavoie 6y ago
  Caddy is a single file web server as well :) https://caddyserver.com/ https://caddyserver.com/ Admittedly they solve different usecases, where Redbean seems to want to serve a whole static site as a single file, which isn't currently possible with Caddy, (but you can run `caddy file-server` to serve the current directory as a static site), but it may become possible in the future with Golang's new embed package https://golang.org/pkg/embed/ https://golang.org/pkg/embed/
  • mholt 6y ago
    That's actually a pretty interesting idea. I wonder if we could make a lite version of Caddy bundled with your website embedded...no need to mess with file systems or permissions or devices and folders to run a static site. Might need some massaging to become a truly pleasant workflow though.
    • francislavoie 6y ago
      Yeah - I could see it being part of xcaddy tooling to package up the static assets in the binary :)
    • IceWreck 6y ago
      The go std library offers the embed package. You can just embed your files and use the std lib's webserver and generate a single file binary. No need for Caddy. All this would be roughly 15 lines of code.
      • mholt 6y ago
        People use Caddy for automatic HTTPS, which the std lib does not give you.
        • anderspitman 6y ago
          Obviously you don't need to know this but others might not be aware that certmagic is a great middle ground between Caddy and rolling your own ACME.
• ToJans 6y ago
  The short description is a bit too short; this is mind blowing! Redbean is a portable, single file executable webserver which also acts as a zip-file for the static content it hosts, and runs on Linux + Mac + Windows + FreeBSD + OpenBSD + NetBSD + BIOS without any recompilation... You can manage the static content using standard pkzip tooling! Now that's what I call thinking out of the box! Edited: some rewording and breathing space
  • tutfbhuf 6y ago
    Is it that this binary does some tricks to run on Linux, Mac, Windows, FreeBSD, OpenBSD, NetBSD, BIOS or does it really work on all operating systems including something like Plan 9 or even TempleOS? And does it run on different architectures like ARMv6?
    • mhd 6y ago
      I think it qemu's itself if run on another platform.
    • twic 6y ago
      Native execution on x86_64 only. "Linux + Mac + Windows + FreeBSD + OpenBSD + NetBSD + BIOS", where "BIOS" means that it will boot a machine and run on bare metal. On non x86_64 systems, the driver script will try to run the binary using QEMU (which has to be installed and on the path): https://github.com/jart/cosmopolitan/blob/40291c9db386d8a95288bd0e9fbe67eb68bceaad/ape/ape.S#L580 https://github.com/jart/cosmopolitan/blob/40291c9db386d8a952... If i have understood how the portability works, it would be pretty straightforward to add support for another OS which uses the same ABI as the unices and has a compatible system call table - just a matter of adding a column to consts.sh (system call numbers in the nr section, constants from system headers in their appropriate sections) and adding any OS-specific system call rituals to systemfive.S. Note that all system calls will go through the syscall interface. If your OS has some other way to make system calls, like Linux's vDSO or io_uring, that won't be used. If your OS had a different system call table, like Windows, then you would need to write more bespoke code for it, like all the Windows wrappers: https://github.com/jart/cosmopolitan/tree/0e85b136ae1beb0d2bb7fd77ea9b5527ca1ca497/libc/nt/kernel32 https://github.com/jart/cosmopolitan/tree/0e85b136ae1beb0d2b... You would then need to go through the Cosmopolitan libc and add implementations of its functions on top of your wrappers.
• modeless 6y ago
  .apk files and .ipa files are secretly zip files too. With the addition of a few manifest files and other crufty things embedded in the zip, I bet you could add Android and iOS to the list of supported platforms too.
  • akavel 6y ago
    I've recently succeeded in building working .apk files byte-by-byte completely from scratch so I could maybe help some. Open an issue on https://github.com/akavel/hellomello https://github.com/akavel/hellomello if interested in some talking/advice and/or watch my talk on NimConf 2020 for an overview (I don't have a link handy as I'm on mobile).
  • nonameiguess 6y ago
    Jars, rpms, whls, Microsoft Office, probably a bunch of other stuff I don’t even know about are actually zip files.
    • citruscomputing 6y ago
      .epubs too!
• hprx 6y ago
  I just spent 30 minutes reading about this. I'm so shocked that I'm logging in to comment after 5 years of lurking. Justine has built a c library that allows you compile a binary once and have it run it on any os or baremetal. The SAME binary. Quite frankly, that sentence doesn't even make sense to me. Check out https://storage.googleapis.com/justine/cosmopolitan/index.html https://storage.googleapis.com/justine/cosmopolitan/index.ht... As far as I'm concerned, this is literal magic. Look at the magic numbers: https://github.com/jart/cosmopolitan/blob/37a4c70c3634862d8d005955c032b5a2fa8737c5/libc/sysv/consts.sh https://github.com/jart/cosmopolitan/blob/37a4c70c3634862d8d... I could go on, but there's no binary portability comparison with any other language. And she has made some pretty neat optimizations. Back in the day, I saw some pretty neat stuff with the ELF format, but this takes the cake. Wow. Edit: I'm editing because this is just so bloody absurd. https://storage.googleapis.com/justine/printimage.html https://storage.googleapis.com/justine/printimage.html $ ./printimage.com someimage.jpg Like wow. And also video. https://storage.googleapis.com/justine/printvideo.html https://storage.googleapis.com/justine/printvideo.html I'm struggling to put my shock into words. I've been around. There's engineering. There's academia. But this falls into straight-up wizardry.
  • malux85 6y ago
    I agree, I got to the page on actually portable executable, and had to read it a few times to make sure I was grokking it properly. This is just sheer cleverness, THIS is the stuff that should be on hacker news!
    • hprx 6y ago
      For real. It's hard to grok cause there are no words to describe it. When people say that something is cross-platform, they usually mean a) that the software will build on multiple platforms b) there is some sort of vm which runs the executable (jvm, beam, wasm) This is the SAME binary. Running everywhere. Could be super-useful as an archival format for mission critical code. sqlite comes to mind. Maybe it's late at night, but I'm struggling to find a suitable word for the project that is better than portable or cross-platform. Literally, I can't find a better word than the project name - "actually portable executable." @author - if you're reading this please set up something for a few bucks a month on your github sponsors. I don't think I have any use for the library, but this is so outrageous it deserves something more than imaginary internet points.
• th0ma5 6y ago
  Presumably X86_64 only?
  • ZiiS 6y ago
    Not sure the is a working example. But they talk of embeding qemu to create cross archetecture executables. But the Window's and actual compiled code would still be x86-64.
  • sscarduzio 6y ago
    She says somewhere in her website that arm/apple silicon support is really easy to add, with no significant size or performance penalty, the same way she added qemu support.
• brainless 6y ago
  OK I see we are going crazy, me too. Thanks @jart. I have been thinking about portable web apps with embedded SQLite for some time. I do not have the technical chops at C level to pull this off. I am really inspired by this project. I hope Redbean leads a way to distribute self-hosted apps in today's era of the cloud.
  • BugWatch 6y ago
    I still use the classic version of Tiddlywiki that is a single self-contained self-editing html file, and I "app-ify" it on Windows by changing the extension to *.hta. But having the similar solution with the SQLite, would be a perfect combination.
• r0b05 6y ago
  The APE shit is really cool. The webserver is novel but I am just wondering, how would it compare to something like Apache/ISS in terms on security/config/etc, or is this mostly just to move small sites around?
• mixmastamyk 6y ago
  Is it a "fat" binary?
  • mekkkkkk 6y ago
    AFAIK a fat binary is just a binary with all dependencies included. It would still be compiled for a specific platform. The magic part about this is that it's a single file compiled once that can be run on most platforms. And it also adheres to the format of a zip file, so you can add, modify and remove assets as you please inside the actual file, post compilation.
    • orhmeh09 6y ago
      No, fat binaries are not binaries with all dependencies included. Fat binaries are executable files that contain equivalent compiled code for multiple architectures. Some also facilitate running binaries across different operating systems. More: https://en.wikipedia.org/wiki/Fat_binary https://en.wikipedia.org/wiki/Fat_binary The idea also reminds me of (executable) self-extracting ZIP archives that were common once upon a time.
      • mekkkkkk 6y ago
        Thanks! Seems like I've misunderstood the concept a bit. My apologies.
• mbgerring 6y ago
  I am floored. Not only is this just breathtaking sorcery, I also needed this exact thing for a project. I don’t understand probably 70% of the description of this software, but I understand what it does, and I’m more excited to carefully study and use this than I have been about anything in years.
• r0b05 6y ago
  Everybody is hyped up here but can someone explain to me what are the implications for the future of programming? Will I be able to write apps in a single language and have then run truly native on any OS or what else will we be able to do?
  • alisonatwork 6y ago
    This is something like a "fat binary" with some magic in the header that allows it to run out of the box on Windows and several different UNIXes, a bit like those sneaky shell scripts that run on several platforms[0]. It still needs to include some native code for each of the platforms inside the compiled binary, so it's perhaps better to see it as a slick packaging/distribution solution for C programmers than a total game changer for developers everywhere. The more interesting thing I think is the mapping of C library functionality between Linux, BSD and Windows. There have been other attempts to do this, but they never seem to get much traction. I think it's because C programmers tend to get wedded to their OS of choice and then invest time in improving that platform's C libraries. The problem with trying to remain portable with everything is that you tend to be stuck working with the lowest common denominator functionality, which can be frustrating when doing OS-specific stuff would open up so many more options. All that said, the cleverness of this project is inspirational in a way that might encourage other programmers to also focus more on building against simple functionality that is truly portable. It shows that you don't need all the bells and whistles to build cool stuff. That feels aligned with early UNIX philosophy and (more broadly) the hacker ethic, which is why it's neat to see, even if never gets adopted as a mainstream thing. [0] https://github.com/llamasoft/polyshell https://github.com/llamasoft/polyshell
    • r0b05 6y ago
      Appreciate the response. So it's a bigger win for C devs specifically. I am a C# dev so I am trying to understand what opportunities this project unlocks.
      • goldenkey 6y ago
        Why you have pigeonholed yourself? "X Dev" is so cliche. You can compile C# to x86 machine code - this project has less to do with C and more to do with PE format and syscall tables. It means that it's language agnostic but not standard library agnostic.
• shaggy8871 6y ago
  Nice job, Justine. This level of excitement reminds me of the early days of Linux. I think it has the potential to go far.
  • jart 6y ago
    Author here. Thanks! I was deeply inspired when I read the source code to early versions of Linux and UNIX. The dream of that simple elegant power is something I'm hoping to restore for a new generation. All I'm doing to achieve that is simply standing on the shoulders of the giants who came before me.
    • fwsgonzo 6y ago
      Have you considered trying to make system calls inline to avoid the call overhead? Or already thought about it? I've been banging my head against the wall of trying to make a micro KVM guest and the glibc startup code uses inline SYSCALL everywhere, I assume to make things faster. It does seem to call CPUID a lot too, but I guess it's not as expensive as having to make full-blown system calls.
      • jart 6y ago
        Cosmopolitan defines linkable symbols for all the __NR_syscall constants so you can absolutely use inlining if you like to live dangerously. For example: https://github.com/jart/cosmopolitan/blob/91f4167a45f811fde65f3ac1df1398a535f61a34/libc/intrin/asan.c#L430-L433 https://github.com/jart/cosmopolitan/blob/91f4167a45f811fde6... There really isn't any performance advantage to doing this, because the SYSCALL instruction itself has something like a 2000+ cycle cost due to all the copying that needs to happen on the kernel side along with things like spectre mitigations which have made is slower. So the 8 cycle cost of having a normal read() function wrapper is the wrong thing to be focusing on. Cosmopolitan aims to address the performance cost of SYSCALL by making it possible to run your binary on bare metal, where your program becomes a kernel, and therefore needn't pay any context switching costs at all.
        • 6y ago
• deleted 6y ago
  [deleted]
• safeaim 6y ago
  Am I the only one having issues trying to get this working? The webserver starts just fine, but once I add the index.html with zip as in the example, it stops working. This is on Mac 10.15, CentOS 8 and Ubuntu 18.04 LTS. Centos: [centos@test ~]$ ./redbean.com -vv error: Uncaught SIGSEGV on test.novalocal ./redbean.com EINVAL/err=22/errno:2/GetLastError:0 Linux test.novalocal 4.18.0-240.10.1.el8_3.x86_64 #1 SMP Mon Jan 18 17:05:51 UTC 2021 On the mac: ~ my-Maccie$./redbean.com -vv Killed: 9 On Ubuntu: ubuntu@localhost:~$ ./redbean.com -vv error: Uncaught SIGSEGV on localhost ./redbean.com EINVAL/err=22/errno:2/GetLastError:0 Linux localhost 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021
  • TeddyDD 6y ago
    No, I have the same issue on Linux Mint 20.1
  • r0b05 6y ago
    After adding a new file with WinRAR, I get: EINVAL/err=87/errno:2/GetLastError:203 The parameter is incorrect. Probably the ZIP needs to be saved in a certain format.
    • jart 6y ago
      Author here. The zip shouldn't need to be saved in any special format. If WinRAR is posing problems then please file an issue because I'd like to fix that. That error doesn't seem like something that would indicate executable corruption. I want to learn more.
      • r0b05 6y ago
        Sure thing, will provide as much info as possible.
      • user-the-name 6y ago
        Have to say though, using zip so you can reuse the compression is quite a masterstroke.
      • 6y ago
• gekkonier 6y ago
  this is outstanding! speechless.
• kidsil 6y ago
  Odd, I'm getting $ ./printimage.com image.jpg run-detectors: unable to find an interpreter for ./printimage.com
  • jart 6y ago
    Author here. That error happens if you have binfmt_misc enabled. The solution is simple: sudo sh -c "echo ':APE:M::MZqFpD::/bin/sh:' >/proc/sys/fs/binfmt_misc/register" I intend to upstream a patch with the Linux kernel so it can do this automatically. If any kernel devs are reading, please email me!
    • est 6y ago
      Does it work on cheap Xen vps? Because "/proc/sys/fs/binfmt_misc/register" looks absent from mine.
      • jart 6y ago
        That's strange. I'd recommend filing an issue. Any details on how I can reproduce the error will be helpful, even if I have to temporarily rent one of those things. I'm on a mission to make binaries portable.
        • na-- 6y ago
          FWIW, it works on QubesOS, which is Xen based. Though it works only in my Fedora and Debian VMs, but not on my Archlinux-based one. I think that's because I have WINE installed in the Archlinux VMs, since that's what gets started when I run ./redbean.com. But even after I remove WINE, it still doesn't work, I probably have to restart (i.e. remove WINE from the TemplateVM, not the AppVM) for it to work. In any case, incredible work, I am in awe!
    • FeepingCreature 6y ago
      Instead of sudo sh -c to sudo the redirects, consider echo 'abcde' |sudo tee /proc/bla. Less nesting :)
• mhd 6y ago
  It's a neat coincidence that this is on at the same time as an article about CP/M, because I think there could be some overlap in the programs that would be a good fit for cosmopolitan. Sure, there's server appliances like this, and I hope someone makes a neat services wrapper to abstract at least some platform's intricacies so you can do a "cosmo-service up redbean" on BSD/Linux/Windows. But coming back to CP/M, there you had a lowest common denominator of terminal applications, too, but spread across different architectures. And you could still produce some quite intersting, if a bit business-like applications. These days you probably can even rely on more ANSI colors and maybe even unicode fonts (hopefull as an option, not mandatory). The ZIP characteristics of the APE format make it even easier to distribute a whole application in a rather simple way. Yes, sure, you can do regular Unix-style servers and pipe-it-together CLI tools, but I wouldn't mind more self-contained "business" applications with a lo-fi aesthetic regarding interface and API usage. The PICO-8 of TUIs…
  • jart 6y ago
    Author here. We're living in the most exciting time for developing terminal applications. When Microsoft unexpectedly added support for VT100 and XTERM codes to CMD.EXE it totally changed the equation and ANSI became universal for the first time. Blinkenlights is an example of a TUI application I created using Cosmopolitan and it literally works everywhere. https://justine.lol/blinkenlights/index.html https://justine.lol/blinkenlights/index.html You don't need curses. All that's needed is an ioctl() call which flips a bit in termios. Cosmopolitan polyfills that across operating systems. Another cool example of a demo app is this conway's game of life tui gui: https://justine.lol/apelife/index.html https://justine.lol/apelife/index.html
    • rexreed 6y ago
      Does that remove full binary compatibility for those running on older OS from Microsoft with an older cmd.exe?
      • jart 6y ago
        Cosmopolitan supports Windows 7. The WinMain() polyfill is programmed to gracefully fall back if ANSI isn't available. So the binary still loads fine. You just need to run it in something like MinTTY instead if your TUI program is run on an old version of Windows.
• aembleton 6y ago
  Just make sure you don't have another server running on :8080. I had syncthing running and kept getting some error whenever I tried to run redbean. The error didn't make much sense, but eventually I realised that this was the casue of the error. I'm really impressed how portable this is. The only improvement I can see is if it auto-opened your browser to :8080. That way it would be easy to distribute the binary and have people run your application and just interact with it through their browser. No need to ship electron then!
  • r0b05 6y ago
    Let's say we are able to do this. Would we have any security concerns to be addressed?
    • anaganisk 6y ago
      How different would it be from opening a random site in your web browser?
    • t0astbread 6y ago
      There is the possibility of other programs making HTTP requests so you shouldn't assume trust on any request automatically. What you can do is give the browser a cookie once when your app starts which you then check on every request. Oh and listen on 127.0.0.1, not 0.0.0.0. That way your app is not needlessly exposed to the network.
  • sva_ 6y ago
    I think you could trivially issue a system() call, you'd just have to use the corresponding command to open the default browser for each particular system. (Linux: xdg-open, MacOs: open, Windows: start)
• ptato 6y ago
  I can't get it to run on Ubuntu. "run-detectors: unable to find an interpreter for ./redbean.com"
• bjarneh 6y ago
  > exes still end up being roughly 100x smaller than Go Hello World > Is the Cosmopolitan Runtime lean and mean like Go's 2mb Hello World executables? :-)
• 867-5309 6y ago
  >the sources that went into creating it are nontrivial and only recommended reading if you really love printing lemurs in terminals. this is delightful
• bni 6y ago
  Why does the magic numbers list have a XENIX column?
  • jart 6y ago
    Author here. The name was chosen mostly out of playfulness to describe Windows since I seem to recall XENIX being the earliest example of Microsoft dipping their toes in the water with UNIX support.
    • robaato 6y ago
      Back in the day ('80s) worked at Logica with colleagues who did Xenix support!
• txdv 6y ago
  > That performance is thanks to zip and gzip using the same compression format, which enables kernelspace copies Can someone elaborate on the relationship between gzip and kernel space copies?
  • jart 6y ago
    Normally when a browser asks for Content-Encoding: gzip, the http server needs to schlep the file into memory, compress it, and only then is it able to send the output to the client. Since redbean reads files from a zip archive, that file is already compressed. So what it does is it uses mmap() to load the zip executable in memory. Therefore instead of read()+write() it can just call writev() on the mapped memory area. That enables us to skip the extra copy. The kernel can just copy the gzipped file contents directly to the network wire without having to pay any userspace context switching penalties.
• gildas 6y ago
  Awesome! You could even use a zip file as a container for the assets you want to serve! see https://gildas-lormeau.github.io/ https://gildas-lormeau.github.io/ (the page is a self-extracted zip file). And here's a demo showing how to update an image: https://www.youtube.com/watch?v=RJWhDmYhnX4 https://www.youtube.com/watch?v=RJWhDmYhnX4
• chriszhang 6y ago
  Someone please be kind and explain to me how this works? Don't different executable formats need different headers? Don't ELF need "0x7F ELF" header? Don't Windows executable need "MZ" header? How is it fulfilling both requirements simultaneously?
  • Gys 6y ago
    The OP explains https://justine.lol/ape.html https://justine.lol/ape.html
• WrtCdEvrydy 6y ago
  This is awesome, I've wanted a way to package a webapp for distribution locally and I think this + sqlite would be a perfect combination.
• ngcc_hk 6y ago
  Does it work with this level of bare metal https://news.ycombinator.com/item?id=26271356 https://news.ycombinator.com/item?id=26271356
• shaded-enmity 6y ago
  If you want to learn more how these things work I'd highly suggest going through the PoC||GTFO archive (https://github.com/angea/pocorgtfo/blob/master/README.md https://github.com/angea/pocorgtfo/blob/master/README.md) and check out entries by Ange Albertini or entries named like "This ZIP is also a PDF".
  • donaldihunter 6y ago
    I came here for this. E.g. https://www.alchemistowl.org/pocorgtfo/pocorgtfo19.pdf https://www.alchemistowl.org/pocorgtfo/pocorgtfo19.pdf "This file, pocorgtfo19.pdf, is valid as a PDF document, a ZIP archive, and a HTML page. It is also available as a Windows PE executable, a PNG image and an MP4 video, all of which have the same MD5 as this PDF."
• bni 6y ago
  Would GUI programs be possible with this? I don't mean some cross platform framework monstrosity. But if some least common denominator was found to be able to create window, add some input fields and buttons using the native UI elements of the OS? Does the Win32 and Cocoa APIs for example, also have "magic numbers" that are kept stable?
  • hnarn 6y ago
    > Would GUI programs be possible with this? This is a cross platform web server, not a cross platform web browser.
    • bni 6y ago
      Sorry, I forgot that this post was about the web server and not about αcτµαlly pδrταblε εxεcµταblε
• ddtaylor 6y ago
  One small complaint I have is that there is already a popular project called RedBean: https://www.redbeanphp.com/ https://www.redbeanphp.com/
• hnarn 6y ago
  Out of curiosity, are there any general benchmarks available as to how this stacks up to something like nginx when it comes to static page serving?
• raphaelj 6y ago
  The amount of work that went to this is insane. She made a (full ?) implementation of libc: https://github.com/jart/cosmopolitan/tree/master/libc https://github.com/jart/cosmopolitan/tree/master/libc
  • dev-tacular 6y ago
    Right? It's amazing to see what some people can accomplish with the same 24 hours a day we all get.
• whalesalad 6y ago
  I’m not an expert at C code but it’s really nice to read redbean.c - lots of relatively small functions with great names. Good example to follow.
• alfiedotwtf 6y ago
  This is the coolest thing I’ve seen on HN in a long time. Nice work
• haolez 6y ago
  This is so powerful that it can be used as a "weapon": it's the perfect lib for making malware. I think this is a compliment!
  • cirowrc 6y ago
    scary scary scary
  • nabla9 6y ago
    She started her career with a very successful spam tool. She was 14 at the time.
    • syncbehind 6y ago
      I definitely was not coding when I was 14. Dang.
• TickleSteve 6y ago
  TLDR: This is a packed file format that can look to the OS as being executable (for Windows/Linux/OSX). The code is x86 and so offers native performance on those processors. On other platforms, an x86 emulator is built in so can't offer native execution speeds. While neat, its not the "best of all worlds" due to the lack of native code on anything other than x86/x64. Also, claiming "bare metal" is supported is a stretch as you are limited to having no I/O of any form (as there is no platform code).
  • OOPMan 6y ago
    No I/O eh? I'm sure the FP zealots will love it :-)
  • jart 6y ago
    I'm just one woman. What Cosmopolitan does so far, it does really well. It was only as recently as a few days ago that I got mmap() and malloc() polyfilled on bare metal. It has serial uart i/o. It's going to have e1000/virtio sockets soon. You can help me will that future into existence. I need people who know o/s dev and can help me write code that does things like correctly set up pic controller.
    • TickleSteve 6y ago
      Sorry, wasn't meant as a criticism, just as an explanation because people were assuming magic... Taking on bare-metal is a significant job much greater than your original task and you may be better targeting something like buildroot instead, after all what is the Linux kernel but a hardware abstraction layer.
      • jart 6y ago
        How hard could stdio/sockets on metal possibly be? So far bare metal has been a walk in the park. I love the fact that PML4T lets me do things like move memory without copying it. I want to be able to have that power without schlepping in the entire Linux world. I believe it should be possible for programs to be able to boot on metal as just programs which are tiny and auditable. Especially considering everything runs on hypervisors these days, which are in some respects the true operating system. Ring 0 is the new Ring 3. Who among you is willing to accept being pushed to the outer bounds of computing? If MINIX gets to run in Ring -3 then it should at least be easy for our programs to run in Ring0.
• smusamashah 6y ago
  Windows 10 (10.0.18363 Build 18363). Just downloaded and executed. No zip modification at all. W2021-02-26T05:00:59.959364:tool/net/redbean.c:1141:redbean-2021-02-25:49336] setsockopt(server, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof(yes)) → ENOPROTOOPT/err=10042/errno:10042/GetLastError:0 W--------------------000674:tool/net/redbean.c:1145:redbean-2021-02-25:49336] setsockopt(server, IPPROTO_TCP, TCP_QUICKACK, &yes, sizeof(yes)) → ENOPROTOOPT/err=10042/errno:10042/GetLastError:0 Its amazing if works though, i can put my static site in an exe. That is so cool. EDIT: Runs on without modifying even with this message. EDIT: Cannot modify zip on windows. "File not found or no read permission" using explorer. With 7-zip "Operation is not supported." EDIT: Used WSL to update zip which then results in SIGSEGV
• fctorial 6y ago
  So it's a tiny virtual machine. Intriguing, but not really shocking.
  • vanderZwan 6y ago
    What is your definition of a virtual machine and what do you think is happening here?
  • joh6nn 6y ago
    "no wireless. less space than a nomad"
  • mikewarot 6y ago
    >So it's a tiny virtual machine. Intriguing, but not really shocking. Nope... it's native code, that runs on multiple platforms, all in one small executable. How? Lots of very clever hacking, and trimming all the accumulated dreck out of the normal C runtime, while making it work across all the supported platforms. (Polyfill is the term she uses) This is the most amazing thing I've seen in a decade or so. It is right up there with GIT in terms of the possibility space it opens up.
• comeonseriously 6y ago
  This is brilliant.
• deleted 6y ago
  [deleted]
• franga2000 6y ago
  I was literally thinking about how cool this would be if it existed like a few days ago!!! Down to the freaky portable executable! Congrats on reading my mind and making what was supposed to be impossible?
• chacocr 6y ago
  Hi
• soheil 6y ago
  Ok well this doesn’t work on Mac, so much for not needing to cross-compile on every architecture. It’s because of the OSX Gatekeeper but if majority of people need to self-sign or otherwise compile this binary I’m not sure if it is that portable be it if it’s only a Mac thing. This dumps the zip content directly on the wire skipping an extra kernel write. There have been attempts before to run the same binary on multiple architectures by inserting magic headers without having to recompile. There are shell/batch versions of the same thing [0]. The issue is such projects usually never get much traction since every programmer is wedded to their choice of OS and typically relies on APIs specific to each system. There are also “fat binaries” which accomplish the same thing if one truly wanted a cross platform single executable. There isn’t a huge need for them currently and not just because they’re “fat” (something this approach seems to tackle primarily) so I expect for the same reasons there won’t be many needs for this approach either. I see a lot of people making the comparison between this and the OG Linux tools. I absolutely don’t think this qualifies as one, most old Linux tools are irreplaceable or at least were when they first came out and there were no alternatives. Serving a bundled zip file on a web server across many different platforms is not something people are dying to do. You can run a web server like nginx or Apache to index your files or if you really hate those why not a single line command which is supported on almost any platform: python -m http.server 8080 [0] https://github.com/llamasoft/polyshell https://github.com/llamasoft/polyshell
  • deleted 6y ago
    [deleted]
  • waheoo 6y ago
    Youre really missing the point here. This zip file is an executable webserver, you just add files to it and it will serve them when you run it. If you can't understand what is cool about that maybe just don't speak.
• thecodrr 6y ago
  As amazing as this is unfortunately it doesn't work for me and I am on Linux. I get bash: ./redbean.com: No such file or directory after chmod-ing. Maybe I am dumb but yeah.
  • anderspitman 6y ago
    Is your system 32 bit by chance?
    • thecodrr 6y ago
      No, not really. It's x64. I really wanted to give this a try.
  • johnx123-up 6y ago
    Did you make it executable with chmod +x ?
    • thecodrr 6y ago
      Yes. That's the funny part. It doesn't work despite that. uname -r gives 5.7.7-amd64-desktop.
  • zer0gravity 6y ago
    I get the same :( I'm on Archlinux.
  • deleted 6y ago
    [deleted]
  • grumple 6y ago
    When I downloaded the file, it also had the date in the name. You have to accommodate that change in the instructions. Also be sure to run bash -c './redbean.com -vv' (also with the name as downloaded).
• nwienert 6y ago
  Can someone marry this with QuickJS and make a micro node competitor with a web server, 0 deps and no install? Wonder how far something like that could go, could be really interesting for replacing bash for me with something easier to write like JS.
• Hasz 6y ago
  This is bonkers amazing! I love the reference to the Swedish Rhapsody numbers station at the end of consts.sh
• yters 6y ago
  Hmm, somehow I knew Justine is a 'transwoman' by this feat. Lucky guess I suppose... Question: if all the high achieving women in tech end up being 'transwomen', is that a step forward or backward for women in tech? Will 'transwomen' be able to take advantage of hiring privileges given to normal women? If so, won't this effectively shut normal women out of tech even more? I think trans/self identifying people should be in a separate category so they don't exploit allowances made for under represented minorities.
  • tobz 6y ago
    Sincerely: what the fuck is wrong with you?
    • yters 6y ago
      Isn't it a legitimate question? We have special protection for under represented groups. So when the dominant group starts taking the identity of underrepresented groups and coopting their protections, they are no longer protected. The two things are at odds with each other.
      • throwaway161437 6y ago
        I'll engage. I think your comments are clearly bigoted, and I'll tell you why. What you do with that is up to you. > Hmm, somehow I knew Justine is a 'transwoman' by this feat. Lucky guess I suppose... You use scare quotes around "transwoman". Why is that? Is that a concept you believe is not real somehow? You are guessing that something technically impressive was achieved by a trans woman. Why is that? Are women not capable of technical feats? > Lucky guess I suppose... This implies that women being technically inferior is something obvious, something we all know but can't say. > If all the high achieving women in tech end up being [yet-again-scare-quotes (and for some reason one word)] "transwomen", ... This isn't a playful hypothetical question. You are reinforcing your claim that women in tech are not high achieving. > I think trans/self identifying people should be in a separate category so they don't exploit allowances made for under represented minorities. > So when the dominant group starts taking the identity of underrepresented groups and coopting their protections From these two statements it is clear you think that trans women are the dominant group (men?) that are exploiting special allowances meant for women. Because in your eyes trans women do not count as women. And they don't deserve protection. You are getting downvoted because the community recognizes the bigotry against cis and trans women that is the backbone of your two comments.
• neilv 6y ago
  The trademark collision is going to test the joke of Red Bean Software: > We're software producers of a different kind. Rather than market software the usual way, we openly deny its existence, and challenge our customers to do the same. https://red-bean.com/ https://red-bean.com/
• rkalla 6y ago
  "I figured out how to microwave water, to make gold." Good lord this is cool
• aero- 6y ago
  Can someone explain this to me like I'm an 8 year old?
  • anderspitman 6y ago
    Can someone explain this to me like I'm a 31 year old?
    • mikewarot 6y ago
      I've been programming since before you were born... and this is the most impressive hacking feat I've ever seen. A single executable, that doubles as a valid ZIP file, that you can put a complete website into, also doubles as a valid executable on Windows, Mac, Linux. The build infrastructure is a complete reworking of C to make it far more portable than even K&R thought possible. I probably wouldn't have believed it possible, but here it is!
      • anderspitman 6y ago
        Thanks!
      • drran 6y ago
        Multi OS/arch executables exists for decades. They were called fat-binaries[0]. Old trick is new again. [0]: https://en.wikipedia.org/wiki/Fat_binary https://en.wikipedia.org/wiki/Fat_binary https://en.wikipedia.org/wiki/Universal_binary https://en.wikipedia.org/wiki/Universal_binary
        • Wowfunhappy 6y ago
          But compared to what jart has done, those were effectively cheating. ;) They just smooshed multiple binaries into a single file.
  • mikewarot 6y ago
    On most computer systems these days can select a set of files and folders and tell the computer to "Send to compressed file", the files that come out of that process usually end in ".zip", and are thus known as "zip files" This is a zip file, like any other... you can add and remove things from it. However, this ZIP file has a superpower, it can display the contents on the web... it has a web server built into it. You can take this file, run it on a Windows, Linux, Macintosh, and it will work. You don't need separate versions for each different system... this just works on all three systems, unlike almost every program ever written. This is one of the most impressive feats of programming things I've seen in my 40+ years of programming. The web serving is clever, but the superpower that it can run on anything really took a huge amount of work, which the author built into a tool called "ape", and has shared with the world, so other people can use it, and help her make it better.
• tgbugs 6y ago
  I looked into building SBCL using chibicc last weekend in the hope that one could build a truly portable lisp runtime using the ape toolchain (Yes, yes, I know you couldn't possibly pick two more different approaches to software portability). Turns out the GNUC extensions for __asm__ are the primary stumbling block (though I'm sure there are others). tcc supports the GNUC extensions needed, and chibicc supports the thread local storage needed for dynamic variables. Looking forward to future developments!
  • jart 6y ago
    You need __asm__ in chibicc for sbcl? I have good news for you. I did exactly that in cosmopolitan's vendored fork of chibicc here: https://github.com/jart/cosmopolitan/blob/master/third_party/chibicc/asm.c https://github.com/jart/cosmopolitan/blob/master/third_party... I went on a coding rampage back in December, adding to chibicc pretty much every GNU extension under the moon: https://github.com/jart/cosmopolitan/blob/master/third_party/chibicc/README.cosmo https://github.com/jart/cosmopolitan/blob/master/third_party... It should be a perfect fit for use cases needed by Cosmopolitan and SBCL. Right now it's unlikely to get merged back upstream because Rui is still focusing on using Chibicc for his book, and that requires being more conservative about feature inclusion right now. I would highly recommend using chibicc. Bellard's work on tcc was fantastic when it came out. However it didn't age well. Its GNU extension support is roughly equivalent to GCC 2.x. The x86_64 support that got bolted on later isn't very good. It was much more elegant back when it was only doing i386. Hackability was also laid low by merging a lot of external contributions. The TCC design, while amazingly fast, also carries the tradeoff of making things like inline assembly hacks really hard because it generates the x86 binary content directly, rather than going through the intermediate step of generating an assembly file and running it through a proper assembler -- which chibicc now has!
    • rui314 6y ago
      I'm actually happy to merge your GNU-style inline assembly support and other GNU C extensions to chibicc. But for the purpose of the project, I need to rewrite your patches in the incremental manner, so it's not something I can do right now. I'm currently busy working on my linker project. Once it settles down, I'll take a closer look at your fork.
• codezero 6y ago
  The magic numbers file ends with a comment linking to a youtube video of numbers station playing very disconcerting and dissonant music, so we can add art/comedy to the list of features (which was already obvious!)
• pxx 6y ago
  If you're getting `check failed: 0xffffffffffffffff != 0xffffffffffffffff (98)` make sure you're able to bind the default port (8080). You can run the symbolized version to see where the check occurs.
• IggleSniggle 6y ago
  I feel like I’ve learned more from your website and your source code in a few hours than I otherwise would have from years of study. You have a real talent for distilling the complex into something simple. It’s apparent in the purpose of your programs, in your writing, and in your source code. Thank you so much! Basics of computing architecture that I didn’t especially think were too complex for me to learn, but we’re too abstracted for me to become interested in, are laid bare in each one of these programs and their source. This is all incredibly exciting.
• unwiredben 6y ago
  Alas, I wanted to try this out, but the format shenanigans triggered the Sophos "machine-learning"-based threat detection on my Windows laptop in mid-download.
• phkahler 6y ago
  Can it be extended in a way similar to CGI back in the day? I would just want to compile it with the CGI program rather than expect it to dynamically load.
  • jart 6y ago
    You should file a feature request! We could crowdsource some ideas about the best way to do that. For example if you read the APE blog post, we've already got a JavaScript interpreter (duktape) checked-in to the Cosmopolitan codebase. I could easily link that into redbean and define possibly a nodejs-like API.
• jan6 6y ago
  I will say, I WOULD absolutely love it, if it weren't for the little problem...that...I can't "just add to the zip"...because for whatever reason, it ruins the special setup... on windows, 7zip and windows explorer's builtin zip stuff refuse to modify it, as if it was corrupted (understandable), but explorer can view it, on linux, zip (apparently info-zip 3.0 according to the version) can add to it just fine, but ruins the special properties, on linux it just spawns the debug gdb with "no symbol table loaded" and no register or assembly data, and on windows it's either wrong format, or `check failed: 0x1 == 0x0 (32)` so also broken... I definitely love the idea of this, and the APE in general, and it works as-is, but modification seems impossible to me...changing to .zip doesn't change the breakage anyone else had luck with adding stuff to it? is it just me? or is it like "it works with openbsd zip command only"?
• embit 6y ago
  This indeed is a beautiful thinking about things. Truly worthy of HN front page.
• 867-5309 6y ago
  rather annoyingly, when seeking a PHP implementation to use with this, there is in fact a project called RedBeanPHP, which seems completely unrelated
• arcticbull 6y ago
  I absolutely love reading Justine's code. It comes up from time to time here and it just makes me happy. It reminds me of the passage from "Programming Sucks" by Still Drinking [1] Every programmer occasionally, when nobody’s home, turns off the lights, pours a glass of scotch, puts on some light German electronica, and opens up a file on their computer. It’s a different file for every programmer. Sometimes they wrote it, sometimes they found it and knew they had to save it. They read over the lines, and weep at their beauty, then the tears turn bitter as they remember the rest of the files and the inevitable collapse of all that is good and true in the world. This file is Good Code. It has sensible and consistent names for functions and variables. It’s concise. It doesn’t do anything obviously stupid. It has never had to live in the wild, or answer to a sales team. It does exactly one, mundane, specific thing, and it does it well. It was written by a single person, and never touched by another. It reads like poetry written by someone over thirty. Her work is always just a little bit trippy in a good way haha. [1] http://stilldrinking.org/programming-sucks http://stilldrinking.org/programming-sucks
  • erikpukinskis 6y ago
    This is too real. Except at this point, after 25 years I have probably ten of these.
    • arcticbull 6y ago
      I've been thinking of putting together github repo which is just an index out to people's snowflakes. If I do, I'll reach out.
      • christophilus 6y ago
        Do it. I’d enjoy perusing it.
      • themantri 6y ago
        Yes, please.
• zer0gravity 6y ago
  This is amazing! There are a lot of low level things that I don't really grasp and, while going through the APE description, I saw this: "The other tradeoff is the GCC Runtime Exception forbids code morphing, but I already took care of that for you, by rewriting the GNU runtimes". Can somebody, please, explain if allowing code morphing will increase security risks?
• BiteCode_dev 6y ago
  And now we have a portable format to embed cross platform an entire interractive book with zero knowledge needed from the reader.
• barbarbar 6y ago
  I tried to run it on an old 32bit pc with Linux. But it complained about not knowing the format of the executable. But it would be awesome if I could get it to work somehow.
  • sounds 6y ago
    Build a qemu to run a 64bit guest.
• roger4534 6y ago
  crashes on Ubuntu 20 after zipping, before zipping works fine. Really cool idea
• text70 6y ago
  I keep getting segfaults in running the executable, only after zip ... I know this is x86_64, and have an am64 system. Does this only run on intel and arm architectures? Does this use the same exploit in memory that spectre used for intel processors?
  • yberreby 6y ago
    I'm unaware of an architecture named am64, so I assume you mean amd64, in which case that _is_ x86_64. Same ISA, different name.
• blacklion 6y ago
  It is all pretty good, but why there is update of "static bool" variables from signal handlers? In old-style C it must be "volatile int" (and only int), in new-style moder C it must be atomic-anything (bool is ok). But access to plain "static bool" from signal handlers & main thread is race and UB.
• canada_dry 6y ago
  error:./printimage.com: check failed: 0x0 != 0x0 (0) 6ffffffffe30 0000004025db UNKNOWN 6ffffffffe40 0000004023a7 UNKNOWN 6fffffffffb0 000000401862 UNKNOWN 6ffffffffff0 000000401ed1 UNKNOWN Run on ubuntu 14? There's nothing running on :8080. Not sure what I'm going wrong.
  • canada_dry 6y ago
    Upd: works. printimage didn't like the jpeg I tried testing with the first time.
• jzer0cool 6y ago
  Am I understanding this correctly that you can also zip up any binary which can further be served by the webserver?