16 ms·

BCHS: OpenBSD, C, httpd and SQLite web stack

217 points by davikrr 5y ago

theamk 5y ago
It seems pretty crazy to write web-facing apps in C, with no memory safety at all. (They do have "pledge" but even in the most restricted case, this still leaves full access to database)
- galdosdi 5y ago
  Funny to reflect that there was a time not so long ago when writing web apps (CGI usually) in C wasn't at all unusual (shortly before Perl became much more popular for this). And today, it is indeed kind of crazy.
  - wk_end 5y ago
    Depends on your definition of "not so long ago" - it's certainly most of the history of the web. The point when Perl, PHP, and Java started to become the dominant web app technologies is about as far from the present day as that point was from the moon landing.
    - codazoda 5y ago
      The moon landing was 1968, 26 years before PHP was created in 1994. And that was just 28 years ago. Oops, math checks out. I’m old.
  - LeFantome 5y ago
    I remember writing CGI scripts in Perl in 1993 ( the year before Netscape ). I am not sure when CGI even became a thing but it could not have been long before that. Not only was “not so long ago” kind of at the very beginning of meaningful web history but it was also for a very brief moment in time ( if we are talking pre-Perl ). Pre-Perl CGI may have never been a thing though as Perl is older than CGI. I recall PHP being the next wave after Perl. One could argue it never lost its place even if it now has many neighbours. Not a Perl advocate by the way though it did generate some pretty magical “dynamic” web pages from text files and directory listings back in the day. Similar story with PHP.
    - galdosdi 5y ago
      It is true the time when it might have been sane to write CGI in C was very brief. Perl took over almost immediately (and to my chagrin, eventually PHP ate Perl's lunch). I remember reading CGI books that would explain how to do it in either Perl or C, the justification being "in case you need C for performance" but in reality I don't think a ton of C CGI was written. There was definitely some though; I recall poking around in cgi-bin directories and finding some compiled executables (could have been another compiled language like C++) and being disappointed I couldn't view the source like with .pl files. It really takes you back to a very specific point in time though. A magical time when every year or month software and internet technology would take big leaps and bounds. When you might do things in a way that is very manual and slow compared to today and yet it was amazing at the time.
      - int_19h 5y ago
        I remember learning CGI to write a web app in late 90s. Most resources at the time seemed to focus on it as a Perl thing, with all code examples etc being in Perl, other languages mentioned briefly if at all (usually at the beginning, when explaining the "it's just a process" model).
  - pjmlp 5y ago
    You mean 1997? By 1999 I was already using our own version of mod_tcl and unfortunely fixing exploits every now and then in our native libs called by Tcl.
- tyingq 5y ago
  Though the majority of running web servers, load balancers, protocol proxies like php-fpm, etc, are probably written in C :)
  - snvzz 5y ago
    Not to mention databases.
  - aspyct 5y ago
    Yes, but they are...better built than your quick social network poll application thingy with customer's special sauce that you had 5 days to specify, develop and deploy. C is a tremendous tool, but I don't think it's the best for customer facing web apps.
- rossy 5y ago
  It seems like the database libraries they recommend for security, ksql and sqlbox, mitigate the risk with process separation and RBAC, so the CGI process doesn't have full access to the database. It's definitely contrary to modern assumptions about web app security, but it's interesting to see web apps that are secure because they use OS security features as they were designed to be used, rather than web apps that do things that are insecure from an OS-perspective, like handling requests from multiple users in the same process, but are secure because they do it with safe programming languages.
  - spudwaffle 5y ago
    How is the overhead of creating a process per-request in this type of system?
  - jolux 5y ago
    Process-per-request is just infeasible with any significant amount of load.
  - theamk 5y ago
    ksql exports "ksql_exec", while sqlbox exports "sqlbox_exec" -- both of those allow execution of arbitrary SQL. So no, the web apps cannot be made secure via OS support alone, because the OS security features are not adequate for high-level problems. Any sort of code exploit allows attacker to trivially access the entire database -- either to read anything, or to overwrite anything. "pledge" and "unveil" can prevent new processes from being spawned, but they cannot prevent authentication bypass, database dumpling or database deletion.
- deleted 5y ago
  [deleted]
- rkeene2 5y ago
  How about a web-facing she'll that allows arbitrary code execution ? [0] There's nothing fundamentally insecure about allowing C or any arbitrary code to execute on behalf of a user -- this is basically what cloud computing (especially "serverless") is. As you identify, though, you need a Controlled Interface (CI) which accounts for this model for all resources and all kinds of resources and many tools do not (yet) allow for it. [0] https://rkeene.dev/js-repl/?arg=bash https://rkeene.dev/js-repl/?arg=bash
  - theamk 5y ago
    The big difference is that with bash (python, perl, php etc..) exploits, all you need is to upgrade a package, and you are secure. No need to touch any of the application code. Compare it with C, where the bugs are likely unique per app, and require non-trivial effort to detect and fix. Execution of user-specific code by serverless services requires non-trivial isolation, and is predicated on "each user has its own separated area" to work. This is not the case with most websites. Take HN for example -- there is a shared state (list of posts) and app-specific logic of who can edit the posts (original owner or moderator). No OS-based service can enforce this for you.
dleslie 5y ago
I'd be fine with this, even totally on-board, if C weren't so awful with respect to text. You don't even have to worry too much about free()ing your malloc()s if you design around short-lived processes. But this is just asking for security concerns among the tangled web of string and input processing your bespoke C routines are likely to develop into. Pair it with a better, more modern, and safer native-compiled language and get the same effect. Zig, Nim, Go, hell even Carp.
- lazyfuture 5y ago
  Same guy wrote a rad tool that will generate server code, schema, frontend using a markup language called ORT. Will generate Rust and typescript if ya want.
  - newlisp 5y ago
    One of the most rewarding parts of back-end web application development is re-writing the same database routines and same JSON export routines over and over. Then changing the requirements and starting over. It's what makes web application developers such well-balanced folks, right? Unfortunately, in the flux of user requirements—each addition or modification of a table column changing select routines, insertions, validation, exporting, regression tests, and even (especially?) front-end JavaScript—we make mistakes. What BCHS tools beyond the usual can help in this perennial test of our patience? ;) https://learnbchs.org/kwebapp.html https://learnbchs.org/kwebapp.html
- synergy20 5y ago
  well there are still many large software written in C, e.g. nginx, lighttpd, even linux kernel. I checked BCHS a few years back, the key piece is that it's Openbsd, if it's Linux it might have caught on, due to linux's popularity, good or bad. This could be useful for embedded device for example, but not so many embedded devices running OpenBSD, if any at all.
  - _wolfie_ 5y ago
    The C library used (https://github.com/kristapsdz/kcgi https://github.com/kristapsdz/kcgi) is portable and working on linux as well. Putting this behind nginx as fastcgi seems very well doable.
tiffanyh 5y ago
s/C/NIM Why don’t more folks use NIM for web development. Seems like the perfect blend of performance, ergonomics and productivity.
- Zababa 5y ago
  Because there's already Java/C#/Go/Rust/C++ in that space.
- LeFantome 5y ago
  How is NIM doing? When I first read your comment I thought you were talking about Zig ( since that is the language that seems to pop up a lot these days ). It took me a second to catch myself. It feels like I have not heard about NIM in ages. I am sure the D and V guys are asking themselves the same question.
- slt2021 5y ago
  Nim/Zig/Crystal - these three languages look the same to me for some reason
- mrweasel 5y ago
  Last I look and played around with Nim, what I felt was missing is a good way of doing templating. Beyond that I honestly enjoyed working in Nim more that both Go and Rust, which where the other two language I attempted to learn last year.
edfletcher_t137 5y ago
This feels like an unreasonable eschewing of all the advancements in programmer ergonomics & tooling that have been made over the course of decades. "Just because you can, doesn't mean you should."
- Ostrogodsky 5y ago
  > "Just because you can, doesn't mean you should." Ironically I could say the same about the JS ecosystem.
  - edfletcher_t137 5y ago
    Not comparable. Easy dig though, I guess.
- pull_my_finger 5y ago
  Imagine if you were a C developer who needed to create some web do-dads, this is probably a fantastic stack. If there was 1 right solution for the perfect stack we'd all be using it.
  - edfletcher_t137 5y ago
    I get that languages are just tools, but each tool has problems it is better at solving. My original comment was pointing out that this stack is ill-suited for the task for which it has been built.
jamal-kumar 5y ago
People love to talk all sorts of trash on this kind of stack but it's really quite solid for what it does. If anyone was ever curious what a sizeable codebase in this kind of code would even look like, check out the source code for undeadly.org [1]. Yeah these people may be crazy but they're also OpenBSD developers and we really love to see what we can get away with using nothing other than what's available in the base distribution. I think a lot of what you see being written for production ends up being very similar to this kind of approach, maybe just utilizing rust or golang as the web application backend language if that's what is the more comfortable thing. Nothing but the base system and a single binary, not relying on an entire interpreter stack, sure can be smooth. There's other examples of this kind of approach, too, writing straight C Common Gateway Interface web applications in public-facing production use - What comes to mind is the version control system web frontend that the people who write wireguard use, cgit [2] - If it's really so crazy then how come the openbsd and wireguard people - presumably better hackers than you - are just out there doing it? Other places you see C web application interfaces include in embedded devices (SCADA, etc) and even the web interfaces for routers, which unfortunately ARE crazy because check out all the security problems! Good thing people at our favorite good old research operating system have done the whole pledge(2)[3] syscall to try and mitigate things when those applications go awry - understanding this part of the whole stack is probably key to seeing how any of it makes any sense at all in 2022. It sure would be nicer if those programs just crashed instead of opening up wider holes. Maybe we can hope these mitigations and a higher code quality for limited-resource device constraints all become more widespread. [1] http://undeadly.org/src/ http://undeadly.org/src/ [2] https://git.zx2c4.com/cgit/ https://git.zx2c4.com/cgit/ [3] https://learnbchs.org/pledge.html https://learnbchs.org/pledge.html
- zephyr9 5y ago
  The Dunning-Kruger effect is stronger in people who spend a lot of time alone, e.g. programmers, which we will now see unfold below.
  - alexshendi 5y ago
    I propose an amendment to Godwin's Law to include "Dunning-Kruger" , "Dunning-Kruger-effect" and "Dunning-Kruger effect".
da39a3ee 5y ago
I’d like to love man pages but - I feel that they are linux only. On my MacOS system I can’t rely on man x being the man page for the right version of x. I know that in principle there are environment variables that make sure i’m getting the gnu core utils version or the base homebrew version rather than the system BSD version, but it’s too many moving parts. Furthermore even if I get it right, I can’t expect people I’m working with or mentoring to get it right, hence I can’t recommend man to them for documentation. God knows about man pages on Windows. - I feel that a small amount of plain text documentation should be stored in the executable, not separately. Isn’t it a holdover from the vastly more constrained computing environments of the 70s and 80s that we’re keeping man pages separate from the executable? Its just asking to get out of sync / incorrectly paired up.
- Shared404 5y ago
  > - I feel that they are linux only. They're actually better on Free/Open BSD in my experience. As stupid as it makes me sound, I often struggle to parse Linux man pages, but the BSD's I've had no trouble with for a variety of topics. > - I feel that a small amount of plain text documentation should be stored in the executable, Isn't this how --help usually works? I would also rather have more documentation embedded, at least for some executables.
  - the_only_law 5y ago
    FreeBSD has pretty good documentation. Comparatively, I’ve found NetBSD documentation to be lacking, although NetBSD seems to take the cake on code quality and legacy architectures (a feature I find my delve right now. On the wider discussion of doc, I’ve found Linux kernel documentation to be a pain in the ass, and sometimes ever worse than windows kernel documentation (which I won’t even both to get into)
- josephcsible 5y ago
  > On my MacOS system I can’t rely on man x being the man page for the right version of x. But isn't that an issue with macOS, not an issue with man pages?
Zababa 5y ago
Lots of opinions but little facts in the comment. I'd love to see an experiment with people using that and their preferred web stack. Is this really slower to develop? By how much? Is this really unsecure? Is this really simpler, faster?
- exdsq 5y ago
  I’d wager a good portion of my salary that a skilled BCHS developer is slower than a skilled Django/RoR developer to build a usual web app (with auth, payment gateways, admin panels, etc). Not to say BCHS doesn’t look like a laugh to use.
  - Zababa 5y ago
    I would too, but I'd like some hard data on this. For example, how much slower? 2 times? 10 times? 100 times? Is this an initial cost or a cost paid on all features? Is maintenance easier? Harder?
exdsq 5y ago
Is anyone using this for anything? I'd love to know!
guggle 5y ago
If you're going to promote a stack, try at least to showcase all its components in the first example you give. Where is the SQLite part in your "BSD, C, httpd, SQLite" ? https://learnbchs.org/easy.html https://learnbchs.org/easy.html Hello world apps don't mean much.
- deleted 5y ago
  [deleted]
- guggle 5y ago
  This is really hilarious... I just followed the third given example (https://kristaps.bsd.lv/absdcon2017/database-conclusions.html https://kristaps.bsd.lv/absdcon2017/database-conclusions.htm...) and that's how it goes: "the simplicity of SQLite is a lie". Ok, thanks, I'll pass on the "BCHS" stack then. I now consider this website satire.
km 5y ago
Writing C might be challenging for some, but as others have mentioned, one can use some other language which gives a statically linked binary to place in the httpd chroot. It won’t be BCHS then. For uptime.is I’ve used a stack which I’ve started calling BLAH because of LISP instead of C.
teleforce 5y ago
SQLite author is an avid Tcl user and he recently introduced a small, secure and modern CGI based web application called wapp [1],[2]. [1] Wapp - A Web-Application Framework for TCL: https://wapp.tcl.tk/home https://wapp.tcl.tk/home [2] EuroTcl2019: Wapp - A framework for web applications in Tcl (Richard Hipp): https://www.youtube.com/watch?v=nmgOlizq-Ms https://www.youtube.com/watch?v=nmgOlizq-Ms
- adamrezich 5y ago
  this is very cool. I only have a passing familiarity with Tcl, but I've been building my own toy web framework and this is a fantastic reference! they made a lot of the same choices I made API-wise but the way they went about it is worth studying.
- dmux 5y ago
  I'd like to point out that Wapp doesn't necessarily need to be run as a plain-old CGI application, I've had success running it with it's own built in web-server behind NGINX, for example.
dreamsbythelake 5y ago
What a coincidence! Lovely topic, even registered account for this :-) I _just_finished_ my own comparative benchmarks to (re)check my projects from ~7 years ago, all in similar stack. Back then I wrote the logic as Apache modules, in C. It was using Cairo to draw charts (surprisingly, the traces of trigonometry knowledge was enough for me to code that :-), and I had absolutely crazy "hybrids" of bubble charts with bars, alpha channel overlays etc. It was extremely useful for my projects back then and I never seen any library, able to produce what I "tailored" ...) The 7-years-ago end-to-end page generation time was ~300 mcs (1e-6 sec), with graphics, data store IO and request processing, preparing the "bucket brigade" and passing it down the Apache chain. This Jan I re-visited my code and implemented logic for OpenBSD httpd as: ** 1) Open BSD httpd "patch" to hijack the request processing internally, do necessary data and graph ops and push the result into Bufferevent buffer directly, before httpd serves it up to the client. ** 2) FCGI responder app, talking to httpd over unix socket. BTW: this is most secure version I know of, I could chroot / pledge / unveil and, IMO, it beats SELinux and anything else. 3) CGI script in ksh<=>slowcgi<=>FCGI=>httpd 4) CGI program (statically linked) in pure C<=>slowcgi<=>FCGI=>httpd 5) PHP :-) page (no frameworks)<=>php-fpm (with OpCache)<=>FCGI=>httpd To my extreme surprise, the outcome was clear - it did not matter what I wrote my logic in, _anything today_ (including CGI shell script) is so fast, that 90% of time was spent on Network communication between the WebServer and the Browser. (And with TLS it is like 2x penalty ...) All options above gave me end-to-end page generation time about 1-1.5 ms. Guess what? Beyond "Hello World", with page size of 500Kb+, PHP was faster than anything else, including native "httpd patch" in C. As side effect, I also confirmed that Libevent-based absolutely gorgeous OpenBSD httpd works slightly slower than standard pre-fork Apache httpd from pkg_add. (It gave me sub-ms times, just like 7 years ago) Who would say ... What also happened is that any framework (PHP or I even tried nodejs) or writing CGI in Python increased my end-to-end page generation time 10x, to double-digit ms. I remember last week someone here was talking about writing business applications / servers for clients in C++, delivering them as single executable file. I would be very interested to hear how that person's observations correlate with mine above. G'day everyone!
harryvederci 5y ago
Interesting CGI content linked on there. I've been reading about / hacking on CGI recently, and it's been kinda fun! Question: One thing I keep reading is how inefficient it is to start a new process for each incoming connection. Could someone explain to me why that's such a bottleneck? I imagine it being an issue back when CGI was used everywhere, people moving away from CGI, and forgetting about it. But hasn't there been improvements in the meantime? Computers from today can run circles around those from a few decades back. Has everything improved except the speed / efficiency of starting a new process? (I don't have a computer science background, but I guess you could already tell from the above.)
- lelanthran 5y ago
  > Interesting CGI content linked on there. > >I've been reading about / hacking on CGI recently, and it's been kinda fun! > >Question: One thing I keep reading is how inefficient it is to start a new process for each incoming connection. Could someone explain to me why that's such a bottleneck? I imagine it being an issue back when CGI was used everywhere, people moving away from CGI, and forgetting about it. But hasn't there been improvements in the meantime? Computers from today can run circles around those from a few decades back. Has everything improved except the speed / efficiency of starting a new process? > It's not as bad as you think it is; just change the webserver to pre-fork. From this link[1], and the nice summary table in this link[2] - I note the following: 1. pre-forked servers perform very consistently (the variation before being overwhelmed) and appears at a glance to only be less consistent than epoll. 2. For up to 2000 concurrent requests, the pre-forked server performed either within a negligible margin against the best performer, or was the best performer itself. 3. The threaded solution had the best graceful degradation; if a script was monitoring the ratio of successfull responses, it would know well beforehand that an imminent failure was coming. 4. The epoll solution is objectively the best, providing both graceful degradation as well as managing to keep up with 15k concurrent requests without complete failure. With all of the above said, it seems that using CGI with a pre-forked server is the second best option you can choose. I suppose that you then only have to factor in the execution of the CGI program (don't use Java, C#, Perl, Python, Ruby, etc - very slow startup times). [1] https://unixism.net/2019/04/linux-applications-performance-introduction/ https://unixism.net/2019/04/linux-applications-performance-i... [2] https://unixism.net/2019/04/linux-applications-performance-part-vii-epoll-servers/ https://unixism.net/2019/04/linux-applications-performance-p... 1.
bitfoxtop 5y ago
for this old environment, why not perl but C?
ThinkBeat 5y ago
I remember writing a lot of early web stuff in Perl/CGI. The "servers" I wrote were fast. Perl had most things you could desire built in already. Database stuff took a good deal of doing, but with little in terms of abstraction, it was also quite fast. I would like to see a rennescance of using different protocols than HTTP and different content markup than HTML.
RcouF1uZ4gsC 5y ago
> How do I pronounce BCHS? I think the correct pronunciation is “Breaches”. Using C in this place as other have mentioned is very, very likely to lead to security issues. Even C++, with its better string handling would be a step up.
0xbadcafebee 5y ago
I have written web applications in a lot of languages, including C. C was the worst.
petee 5y ago
Another great stack for writing C (or now python) is https://kore.io https://kore.io which offers quite a few helper features, and its easy to get started
jolux 5y ago
Parsing untrusted input in C never hurt anyone, did it?

217 points by davikrr 5y ago

It seems pretty crazy to write web-facing apps in C, with no memory safety at all. (They do have "pledge" but even in the most restricted case, this still leaves full access to database)

galdosdi 5y ago
Funny to reflect that there was a time not so long ago when writing web apps (CGI usually) in C wasn't at all unusual (shortly before Perl became much more popular for this). And today, it is indeed kind of crazy.
- wk_end 5y ago
  Depends on your definition of "not so long ago" - it's certainly most of the history of the web. The point when Perl, PHP, and Java started to become the dominant web app technologies is about as far from the present day as that point was from the moon landing.
  - codazoda 5y ago
    The moon landing was 1968, 26 years before PHP was created in 1994. And that was just 28 years ago. Oops, math checks out. I’m old.
- LeFantome 5y ago
  I remember writing CGI scripts in Perl in 1993 ( the year before Netscape ). I am not sure when CGI even became a thing but it could not have been long before that. Not only was “not so long ago” kind of at the very beginning of meaningful web history but it was also for a very brief moment in time ( if we are talking pre-Perl ). Pre-Perl CGI may have never been a thing though as Perl is older than CGI. I recall PHP being the next wave after Perl. One could argue it never lost its place even if it now has many neighbours. Not a Perl advocate by the way though it did generate some pretty magical “dynamic” web pages from text files and directory listings back in the day. Similar story with PHP.
  - galdosdi 5y ago
    It is true the time when it might have been sane to write CGI in C was very brief. Perl took over almost immediately (and to my chagrin, eventually PHP ate Perl's lunch). I remember reading CGI books that would explain how to do it in either Perl or C, the justification being "in case you need C for performance" but in reality I don't think a ton of C CGI was written. There was definitely some though; I recall poking around in cgi-bin directories and finding some compiled executables (could have been another compiled language like C++) and being disappointed I couldn't view the source like with .pl files. It really takes you back to a very specific point in time though. A magical time when every year or month software and internet technology would take big leaps and bounds. When you might do things in a way that is very manual and slow compared to today and yet it was amazing at the time.
    - int_19h 5y ago
      I remember learning CGI to write a web app in late 90s. Most resources at the time seemed to focus on it as a Perl thing, with all code examples etc being in Perl, other languages mentioned briefly if at all (usually at the beginning, when explaining the "it's just a process" model).
- pjmlp 5y ago
  You mean 1997? By 1999 I was already using our own version of mod_tcl and unfortunely fixing exploits every now and then in our native libs called by Tcl.
tyingq 5y ago
Though the majority of running web servers, load balancers, protocol proxies like php-fpm, etc, are probably written in C :)
- snvzz 5y ago
  Not to mention databases.
- aspyct 5y ago
  Yes, but they are...better built than your quick social network poll application thingy with customer's special sauce that you had 5 days to specify, develop and deploy. C is a tremendous tool, but I don't think it's the best for customer facing web apps.
rossy 5y ago
It seems like the database libraries they recommend for security, ksql and sqlbox, mitigate the risk with process separation and RBAC, so the CGI process doesn't have full access to the database. It's definitely contrary to modern assumptions about web app security, but it's interesting to see web apps that are secure because they use OS security features as they were designed to be used, rather than web apps that do things that are insecure from an OS-perspective, like handling requests from multiple users in the same process, but are secure because they do it with safe programming languages.
- spudwaffle 5y ago
  How is the overhead of creating a process per-request in this type of system?
- jolux 5y ago
  Process-per-request is just infeasible with any significant amount of load.
- theamk 5y ago
  ksql exports "ksql_exec", while sqlbox exports "sqlbox_exec" -- both of those allow execution of arbitrary SQL. So no, the web apps cannot be made secure via OS support alone, because the OS security features are not adequate for high-level problems. Any sort of code exploit allows attacker to trivially access the entire database -- either to read anything, or to overwrite anything. "pledge" and "unveil" can prevent new processes from being spawned, but they cannot prevent authentication bypass, database dumpling or database deletion.
deleted 5y ago
[deleted]
rkeene2 5y ago
How about a web-facing she'll that allows arbitrary code execution ? [0] There's nothing fundamentally insecure about allowing C or any arbitrary code to execute on behalf of a user -- this is basically what cloud computing (especially "serverless") is. As you identify, though, you need a Controlled Interface (CI) which accounts for this model for all resources and all kinds of resources and many tools do not (yet) allow for it. [0] https://rkeene.dev/js-repl/?arg=bash https://rkeene.dev/js-repl/?arg=bash
- theamk 5y ago
  The big difference is that with bash (python, perl, php etc..) exploits, all you need is to upgrade a package, and you are secure. No need to touch any of the application code. Compare it with C, where the bugs are likely unique per app, and require non-trivial effort to detect and fix. Execution of user-specific code by serverless services requires non-trivial isolation, and is predicated on "each user has its own separated area" to work. This is not the case with most websites. Take HN for example -- there is a shared state (list of posts) and app-specific logic of who can edit the posts (original owner or moderator). No OS-based service can enforce this for you.

dleslie 5y ago

I'd be fine with this, even totally on-board, if C weren't so awful with respect to text. You don't even have to worry too much about free()ing your malloc()s if you design around short-lived processes. But this is just asking for security concerns among the tangled web of string and input processing your bespoke C routines are likely to develop into. Pair it with a better, more modern, and safer native-compiled language and get the same effect. Zig, Nim, Go, hell even Carp.

lazyfuture 5y ago
Same guy wrote a rad tool that will generate server code, schema, frontend using a markup language called ORT. Will generate Rust and typescript if ya want.
- newlisp 5y ago
  One of the most rewarding parts of back-end web application development is re-writing the same database routines and same JSON export routines over and over. Then changing the requirements and starting over. It's what makes web application developers such well-balanced folks, right? Unfortunately, in the flux of user requirements—each addition or modification of a table column changing select routines, insertions, validation, exporting, regression tests, and even (especially?) front-end JavaScript—we make mistakes. What BCHS tools beyond the usual can help in this perennial test of our patience? ;) https://learnbchs.org/kwebapp.html https://learnbchs.org/kwebapp.html
synergy20 5y ago
well there are still many large software written in C, e.g. nginx, lighttpd, even linux kernel. I checked BCHS a few years back, the key piece is that it's Openbsd, if it's Linux it might have caught on, due to linux's popularity, good or bad. This could be useful for embedded device for example, but not so many embedded devices running OpenBSD, if any at all.
- _wolfie_ 5y ago
  The C library used (https://github.com/kristapsdz/kcgi https://github.com/kristapsdz/kcgi) is portable and working on linux as well. Putting this behind nginx as fastcgi seems very well doable.

tiffanyh 5y ago

s/C/NIM Why don’t more folks use NIM for web development. Seems like the perfect blend of performance, ergonomics and productivity.

Zababa 5y ago
Because there's already Java/C#/Go/Rust/C++ in that space.
LeFantome 5y ago
How is NIM doing? When I first read your comment I thought you were talking about Zig ( since that is the language that seems to pop up a lot these days ). It took me a second to catch myself. It feels like I have not heard about NIM in ages. I am sure the D and V guys are asking themselves the same question.
slt2021 5y ago
Nim/Zig/Crystal - these three languages look the same to me for some reason
mrweasel 5y ago
Last I look and played around with Nim, what I felt was missing is a good way of doing templating. Beyond that I honestly enjoyed working in Nim more that both Go and Rust, which where the other two language I attempted to learn last year.

edfletcher_t137 5y ago

This feels like an unreasonable eschewing of all the advancements in programmer ergonomics & tooling that have been made over the course of decades. "Just because you can, doesn't mean you should."

Ostrogodsky 5y ago
> "Just because you can, doesn't mean you should." Ironically I could say the same about the JS ecosystem.
- edfletcher_t137 5y ago
  Not comparable. Easy dig though, I guess.
pull_my_finger 5y ago
Imagine if you were a C developer who needed to create some web do-dads, this is probably a fantastic stack. If there was 1 right solution for the perfect stack we'd all be using it.
- edfletcher_t137 5y ago
  I get that languages are just tools, but each tool has problems it is better at solving. My original comment was pointing out that this stack is ill-suited for the task for which it has been built.

jamal-kumar 5y ago

People love to talk all sorts of trash on this kind of stack but it's really quite solid for what it does. If anyone was ever curious what a sizeable codebase in this kind of code would even look like, check out the source code for undeadly.org [1]. Yeah these people may be crazy but they're also OpenBSD developers and we really love to see what we can get away with using nothing other than what's available in the base distribution. I think a lot of what you see being written for production ends up being very similar to this kind of approach, maybe just utilizing rust or golang as the web application backend language if that's what is the more comfortable thing. Nothing but the base system and a single binary, not relying on an entire interpreter stack, sure can be smooth. There's other examples of this kind of approach, too, writing straight C Common Gateway Interface web applications in public-facing production use - What comes to mind is the version control system web frontend that the people who write wireguard use, cgit [2] - If it's really so crazy then how come the openbsd and wireguard people - presumably better hackers than you - are just out there doing it? Other places you see C web application interfaces include in embedded devices (SCADA, etc) and even the web interfaces for routers, which unfortunately ARE crazy because check out all the security problems! Good thing people at our favorite good old research operating system have done the whole pledge(2)[3] syscall to try and mitigate things when those applications go awry - understanding this part of the whole stack is probably key to seeing how any of it makes any sense at all in 2022. It sure would be nicer if those programs just crashed instead of opening up wider holes. Maybe we can hope these mitigations and a higher code quality for limited-resource device constraints all become more widespread. [1] http://undeadly.org/src/ http://undeadly.org/src/ [2] https://git.zx2c4.com/cgit/ https://git.zx2c4.com/cgit/ [3] https://learnbchs.org/pledge.html https://learnbchs.org/pledge.html

zephyr9 5y ago
The Dunning-Kruger effect is stronger in people who spend a lot of time alone, e.g. programmers, which we will now see unfold below.
- alexshendi 5y ago
  I propose an amendment to Godwin's Law to include "Dunning-Kruger" , "Dunning-Kruger-effect" and "Dunning-Kruger effect".

da39a3ee 5y ago

I’d like to love man pages but - I feel that they are linux only. On my MacOS system I can’t rely on man x being the man page for the right version of x. I know that in principle there are environment variables that make sure i’m getting the gnu core utils version or the base homebrew version rather than the system BSD version, but it’s too many moving parts. Furthermore even if I get it right, I can’t expect people I’m working with or mentoring to get it right, hence I can’t recommend man to them for documentation. God knows about man pages on Windows. - I feel that a small amount of plain text documentation should be stored in the executable, not separately. Isn’t it a holdover from the vastly more constrained computing environments of the 70s and 80s that we’re keeping man pages separate from the executable? Its just asking to get out of sync / incorrectly paired up.

Shared404 5y ago
> - I feel that they are linux only. They're actually better on Free/Open BSD in my experience. As stupid as it makes me sound, I often struggle to parse Linux man pages, but the BSD's I've had no trouble with for a variety of topics. > - I feel that a small amount of plain text documentation should be stored in the executable, Isn't this how --help usually works? I would also rather have more documentation embedded, at least for some executables.
- the_only_law 5y ago
  FreeBSD has pretty good documentation. Comparatively, I’ve found NetBSD documentation to be lacking, although NetBSD seems to take the cake on code quality and legacy architectures (a feature I find my delve right now. On the wider discussion of doc, I’ve found Linux kernel documentation to be a pain in the ass, and sometimes ever worse than windows kernel documentation (which I won’t even both to get into)
josephcsible 5y ago
> On my MacOS system I can’t rely on man x being the man page for the right version of x. But isn't that an issue with macOS, not an issue with man pages?

Zababa 5y ago

Lots of opinions but little facts in the comment. I'd love to see an experiment with people using that and their preferred web stack. Is this really slower to develop? By how much? Is this really unsecure? Is this really simpler, faster?

exdsq 5y ago
I’d wager a good portion of my salary that a skilled BCHS developer is slower than a skilled Django/RoR developer to build a usual web app (with auth, payment gateways, admin panels, etc). Not to say BCHS doesn’t look like a laugh to use.
- Zababa 5y ago
  I would too, but I'd like some hard data on this. For example, how much slower? 2 times? 10 times? 100 times? Is this an initial cost or a cost paid on all features? Is maintenance easier? Harder?

exdsq 5y ago

Is anyone using this for anything? I'd love to know!

guggle 5y ago

If you're going to promote a stack, try at least to showcase all its components in the first example you give. Where is the SQLite part in your "BSD, C, httpd, SQLite" ? https://learnbchs.org/easy.html https://learnbchs.org/easy.html Hello world apps don't mean much.

deleted 5y ago
[deleted]
guggle 5y ago
This is really hilarious... I just followed the third given example (https://kristaps.bsd.lv/absdcon2017/database-conclusions.html https://kristaps.bsd.lv/absdcon2017/database-conclusions.htm...) and that's how it goes: "the simplicity of SQLite is a lie". Ok, thanks, I'll pass on the "BCHS" stack then. I now consider this website satire.

km 5y ago

Writing C might be challenging for some, but as others have mentioned, one can use some other language which gives a statically linked binary to place in the httpd chroot. It won’t be BCHS then. For uptime.is I’ve used a stack which I’ve started calling BLAH because of LISP instead of C.

teleforce 5y ago

SQLite author is an avid Tcl user and he recently introduced a small, secure and modern CGI based web application called wapp [1],[2]. [1] Wapp - A Web-Application Framework for TCL: https://wapp.tcl.tk/home https://wapp.tcl.tk/home [2] EuroTcl2019: Wapp - A framework for web applications in Tcl (Richard Hipp): https://www.youtube.com/watch?v=nmgOlizq-Ms https://www.youtube.com/watch?v=nmgOlizq-Ms

adamrezich 5y ago
this is very cool. I only have a passing familiarity with Tcl, but I've been building my own toy web framework and this is a fantastic reference! they made a lot of the same choices I made API-wise but the way they went about it is worth studying.
dmux 5y ago
I'd like to point out that Wapp doesn't necessarily need to be run as a plain-old CGI application, I've had success running it with it's own built in web-server behind NGINX, for example.

dreamsbythelake 5y ago

What a coincidence! Lovely topic, even registered account for this :-) I _just_finished_ my own comparative benchmarks to (re)check my projects from ~7 years ago, all in similar stack. Back then I wrote the logic as Apache modules, in C. It was using Cairo to draw charts (surprisingly, the traces of trigonometry knowledge was enough for me to code that :-), and I had absolutely crazy "hybrids" of bubble charts with bars, alpha channel overlays etc. It was extremely useful for my projects back then and I never seen any library, able to produce what I "tailored" ...) The 7-years-ago end-to-end page generation time was ~300 mcs (1e-6 sec), with graphics, data store IO and request processing, preparing the "bucket brigade" and passing it down the Apache chain. This Jan I re-visited my code and implemented logic for OpenBSD httpd as: ** 1) Open BSD httpd "patch" to hijack the request processing internally, do necessary data and graph ops and push the result into Bufferevent buffer directly, before httpd serves it up to the client. ** 2) FCGI responder app, talking to httpd over unix socket. BTW: this is most secure version I know of, I could chroot / pledge / unveil and, IMO, it beats SELinux and anything else. 3) CGI script in ksh<=>slowcgi<=>FCGI=>httpd 4) CGI program (statically linked) in pure C<=>slowcgi<=>FCGI=>httpd 5) PHP :-) page (no frameworks)<=>php-fpm (with OpCache)<=>FCGI=>httpd To my extreme surprise, the outcome was clear - it did not matter what I wrote my logic in, _anything today_ (including CGI shell script) is so fast, that 90% of time was spent on Network communication between the WebServer and the Browser. (And with TLS it is like 2x penalty ...) All options above gave me end-to-end page generation time about 1-1.5 ms. Guess what? Beyond "Hello World", with page size of 500Kb+, PHP was faster than anything else, including native "httpd patch" in C. As side effect, I also confirmed that Libevent-based absolutely gorgeous OpenBSD httpd works slightly slower than standard pre-fork Apache httpd from pkg_add. (It gave me sub-ms times, just like 7 years ago) Who would say ... What also happened is that any framework (PHP or I even tried nodejs) or writing CGI in Python increased my end-to-end page generation time 10x, to double-digit ms. I remember last week someone here was talking about writing business applications / servers for clients in C++, delivering them as single executable file. I would be very interested to hear how that person's observations correlate with mine above. G'day everyone!

harryvederci 5y ago

Interesting CGI content linked on there. I've been reading about / hacking on CGI recently, and it's been kinda fun! Question: One thing I keep reading is how inefficient it is to start a new process for each incoming connection. Could someone explain to me why that's such a bottleneck? I imagine it being an issue back when CGI was used everywhere, people moving away from CGI, and forgetting about it. But hasn't there been improvements in the meantime? Computers from today can run circles around those from a few decades back. Has everything improved except the speed / efficiency of starting a new process? (I don't have a computer science background, but I guess you could already tell from the above.)

lelanthran 5y ago
> Interesting CGI content linked on there. > >I've been reading about / hacking on CGI recently, and it's been kinda fun! > >Question: One thing I keep reading is how inefficient it is to start a new process for each incoming connection. Could someone explain to me why that's such a bottleneck? I imagine it being an issue back when CGI was used everywhere, people moving away from CGI, and forgetting about it. But hasn't there been improvements in the meantime? Computers from today can run circles around those from a few decades back. Has everything improved except the speed / efficiency of starting a new process? > It's not as bad as you think it is; just change the webserver to pre-fork. From this link[1], and the nice summary table in this link[2] - I note the following: 1. pre-forked servers perform very consistently (the variation before being overwhelmed) and appears at a glance to only be less consistent than epoll. 2. For up to 2000 concurrent requests, the pre-forked server performed either within a negligible margin against the best performer, or was the best performer itself. 3. The threaded solution had the best graceful degradation; if a script was monitoring the ratio of successfull responses, it would know well beforehand that an imminent failure was coming. 4. The epoll solution is objectively the best, providing both graceful degradation as well as managing to keep up with 15k concurrent requests without complete failure. With all of the above said, it seems that using CGI with a pre-forked server is the second best option you can choose. I suppose that you then only have to factor in the execution of the CGI program (don't use Java, C#, Perl, Python, Ruby, etc - very slow startup times). [1] https://unixism.net/2019/04/linux-applications-performance-introduction/ https://unixism.net/2019/04/linux-applications-performance-i... [2] https://unixism.net/2019/04/linux-applications-performance-part-vii-epoll-servers/ https://unixism.net/2019/04/linux-applications-performance-p... 1.

bitfoxtop 5y ago

for this old environment, why not perl but C?

ThinkBeat 5y ago

I remember writing a lot of early web stuff in Perl/CGI. The "servers" I wrote were fast. Perl had most things you could desire built in already. Database stuff took a good deal of doing, but with little in terms of abstraction, it was also quite fast. I would like to see a rennescance of using different protocols than HTTP and different content markup than HTML.

RcouF1uZ4gsC 5y ago

> How do I pronounce BCHS? I think the correct pronunciation is “Breaches”. Using C in this place as other have mentioned is very, very likely to lead to security issues. Even C++, with its better string handling would be a step up.

0xbadcafebee 5y ago

I have written web applications in a lot of languages, including C. C was the worst.

petee 5y ago

Another great stack for writing C (or now python) is https://kore.io https://kore.io which offers quite a few helper features, and its easy to get started

jolux 5y ago

Parsing untrusted input in C never hurt anyone, did it?