19 ms·

Here is the actual single C-code file: https://sqlite.org/althttpd/file?name=althttpd.c https://sqlite.org/althttpd/file?name=althttpd.c Something I absolutely

• kijin 5y ago
  HTTP is serious about backward compatibility, so a client that speaks HTTP/1.1 (or HTTP/1.0 + Host header) can still talk to most servers out there. Similarly, if you write a server that only speaks HTTP/1.1 (or HTTP/1.0 + Host header), you can put it behind a reverse proxy or load balancer that handles higher versions, does connection management, and terminates TLS. It will work perfectly fine, only without some of the latest performance optimizations that you might or might not even need.
  • bawolff 5y ago
    That seems like a high risk of http desync attacks, if you're only implementing a subset of http/1.1.
    • unixhero 5y ago
      How would one attack a static page?
      • ric2b 5y ago
        Why would you even need a reverse proxy for a static page? This isn't about that.
        • unixhero 5y ago
          I see. Thanks!
      • bawolff 5y ago
        Who said anything about static? This webserver supports cgi, which means it supports php, perl, etc. However even if it didn't, js based client-side apps are probably still attackable in the right set of circumstances.
  • chippiewill 5y ago
    > you can put it behind a reverse proxy or load balancer that handles higher versions, does connection management, and terminates TLS This is even standard practice in many production deployments. Typically you want the proxy or load balancer anyway and there's often little benefit (if any) to using HTTP/2 or HTTP/3 over a very low-latency, high reliability local network.
• saba2008 5y ago
  HTTP/1 simplicity comes not from text nature, but from it's clear concept and limited functionality. At core, it's just request-reply + key-value metadata. Whenever it's text, or binary, it does not matter much. But writing HTTP/2 frame types in letters would not make them any easier to understand.
  • ahartmetz 5y ago
    There are also keep-alive, caching (a big topic), chunked transfer encoding, header parsing peculiarities, and authentication in HTTP. The combination of these creates some nice opportunities for implementation bugs. Source: have worked on a client-side implementation. Now, HTTP/2 isn't even conceptually simple, I agree about that... it seems ugly.
    • tokamak-teapot 5y ago
      One good thing is that you don’t have to support everything if you’re writing a server. It just depends on what kind of use cases you want to support.
    • jbverschoor 5y ago
      That came with http/1.1 ;-)
    • merb 5y ago
      > implementation bugs. Source: have worked on a client-side implementation well because the hard part is the client-side. caching is a client-side only thing with http, keep-alive is a thing that a server pushes to a client, the same with chunked transfer, which is not as easy to implement for a client like it was with content length. basically a server does only need to implement certain headers, but a client needs to know all. also most clients even accept bad servers, like content for head requests, etc.. most stateless protocols put a lot of burden into clients. h2 on the other hand is stateful and keeps the same hard semantics onto the client side and also makes servers more complex, because it's a state machine.
  • secondcoming 5y ago
• TazeTSchnitzel 5y ago
  Once you have to support HTTP/1.1 it's not really a text-based protocol, because support for chunked encoding is mandatory for clients. (Yes the chunk headers are technically text, but it's interspersed with arbitrary binary data. It's not something you can easily read in a text editor.)
  • mbreese 5y ago
    HTTP/1 requests for binary files also aren’t text based under that criterion. The protocol itself is text based, but the payload(s) can be binary. That’s doesn’t change between HTTP/1 and /1.1.
  • textmode 5y ago
    I make most HTTP requests using netcat or similar tcp clients so I write filters that read from stdin. Reading text files with the chunk sizes in hex interspersed is generally easy. Sometimes I do not even bother to remove the chunk sizes. Where it becomes an issue is when it breaks URLs. Here is a simple chunked transfer decoder that reads from stdin and removes the chunk sizes. flex -8iCrfa <<eof int fileno (FILE *); xa "\15"|"\12" xb "\15\12" %option noyywrap nounput noinput %% ^[A-Fa-f0-9]+{xa} {xa}+[A-Fa-f0-9]+{xa} {xb}[A-Fa-f0-9]+{xb} %% int main(){ yylex();exit(0);} eof cc -std=c89 -Wall -pipe lex.yy.c -static -o yy045 Example Yahoo! serves chunked pages printf 'GET / HTTP/1.1\r\nHost: us.yahoo.com\r\nConnection: close\r\n\r\n'|openssl s_client -connect us.yahoo.com:443 -ign_eof|./yy045
    • charsi 5y ago
      I tried this but ended up with gibberish in my terminal. Also couldn't find an explanation for -a on flex's man page. I've never used the thing before.
      • textmode 5y ago
        The extra "a" is a typo but would have no effect. The "i" is also superfluous but harmless. Without more details on the "gibberish" it is difficult to guess what happened. The space before "int fileno (FILE *);" is required. All the other lines must be left-justified, no leading spaces, except the line with "int main()" which can be indented if desired.
• lmilcin 5y ago
  Binary protocols aren't (or at least don't have to be) any more difficult and frequently are even easier to implement. Text protocols have difficult problems like escaping or detecting the end of particular field that are frequent source of mistakes. The issue is that many (especially scripting) languages treat binary data as second class. The only real issue is that inspecting the binary message visually is little bit more difficult. You can usually easily tell if your data structure is broken if it is in text form. What I do is I usually have some simple converter that converts the binary message to text form and this helps me inspect the message easily.
  • chrismorgan 5y ago
    And capitalisation issues, and encodings… give me a binary protocol to parse any time. They’re normally comparatively well-defined, whereas text protocols are seldom properly defined and so have undefined behaviour left, right and centre, which inevitably leads to security bugs—or even if they are well-defined, they’re probably done in a way that makes your language’s string type unsuitable, and makes parsing more complicated.
    • touisteur 5y ago
      Just use recordflux for binary parser proved absent of runtime errors... ;-)
      • einpoklum 5y ago
        Can you expand on that a little?
        • michaelmior 5y ago
          RecordFlux[0] is a DSL written in Ada for specifying messages in a binary protocol. Code for parsing these messages is then generated automatically with a number of useful properties automatically proven including that no runtime errors will occur. [0] https://github.com/Componolit/RecordFlux https://github.com/Componolit/RecordFlux
• secondcoming 5y ago
  Being text based is a huge flaw with HTTP in my opinion (and also elsewhere, like Redis). It leads to parsing bugs and overly verbose communications. Humans are good at reading text, CPUs prefer binary.
  • mbreese 5y ago
    The use of text for popular protocols is for a reason — computers don’t write programs, people do. And while CPUs prefer binary, it’s easier for programmers to read/write/reason about text. This makes it easier to work with a new protocol. From a practical perspective, with a binary protocol, it can be difficult to use across different languages or add support for a new language. If you use the simplest possible encoding, you’d send raw struct data. But this doesn’t always work across different OS/arch/versions/etc. if the server is in C, but the client is in Python, reading the binary protocol would require a far more complicated parser. Obviously a more formal encoding (protobuf, etc) would be preferred, but if you already need to use an encoding mechanism, why not wrap it in a text format? It’s easier to write clients that can read/write text protocols in any language. The reason why text protocols are so popular aren’t because they are necessarily “better” but easier to adopt. This is why the most popular protocols are text based.
    • michaelmior 5y ago
      Except it quickly gets messy when you start dealing with real data and making sure encoding and escaping is done correctly. > with a binary protocol, it can be difficult to use across different languages This is also true of text protocols that aren't well-designed. I don't think it's necessarily the case that binary protocols are more difficult to deal with. You just have a different set of concerns to address. > If you use the simplest possible encoding, you’d send raw struct data. This is the "simplest" in the sense that it's definitely easy to just copy this data on the wire, but I think this is a straw man. I don't think it's really any more difficult to write a simple protocol that uses binary data compared to text.
      • mbreese 5y ago
• andai 5y ago
  For people who like text-based protocols (and dislike surveillance and web bloat), I suggest taking a look at Gemini, which was designed so you could write a client for it over the weekend. https://gemini.circumlunar.space/ https://gemini.circumlunar.space/
  • peterhil 5y ago
    Gemini is quite interesting in that brings nostalgic feeling from when the web was a new thing, but is also modern. The Lagrange browser seems quite polished.
• strangeattractr 5y ago
  I prefer binary protocols but I think you make a good point about HTTP/1 especially from a learning perspective. I remember how enlightening it was when I followed a tutorial to create a http server and found that all I needed to do (ignoring TCP/IP) was write 'HTTP/1.1 200 OK' plus a few headers and some raw html in a string and it would actually show up in my browser.
• api 5y ago
  Recording state info in static variables makes it only suitable for embedded. I really hate when people do that. Static should only ever be const, init once, or something intrinsically singleton. There are very few exceptions.
  • jmaygarden 5y ago
    Are you thinking of C++ issues with constructors? I don’t see anything wrong with the use of static variables in this single file C program. This isn’t a library; it’s a standalone web server.
    • api 5y ago
      I am thinking of multithreading. Mutable static variables pretty much destroy any possibility of multithreading without a major refactor. Test harnessing is an issue too. But if this only ever wants to be an app binary, I guess it's sort of okay.
      • jschwartzi 5y ago
        That class of errors is not an issue here. This program isn't meant to be multithreaded, it's meant to be multi-/process/. The difference is that each instance of the application has its own independent address space. So that's one address space per instance. And because of that there are no threading issues. Also as a result any communication between different processes has to be explicitly defined. Making this program multi-threaded would be a big mistake because then you couldn't use any of the HTTP proxies out there to monitor for connections and hand them off to this program. This is arguably a much better architecture from a safety/security standpoint then trying to spin multiple threads each sharing an address space. It forces you to use OS-provided mechanisms for shared state rather than simply manipulating the memory to share information.
• TZubiri 5y ago
  Really simple, I implemented a couple. Remember that the letters after the status code are aesthetic, so make sure to put your style in there: 200 FINE 200 KTHX 404 MISS 403 NO 500 FUCK Another similar server in one file is busybox httpd command if you are interested https://git.busybox.net/busybox/tree/networking/httpd.c https://git.busybox.net/busybox/tree/networking/httpd.c
  • Sohcahtoa82 5y ago
    I always wanted an HTTP response code for when the server detects a malicious request. Like, 430 DONT BE AN ASSHOLE
    • zbentley 5y ago
      Relatedly, in every C project I've ever worked on, the custom (>255) error code for failures due to malicious looking input that could not possibly have been supplied by mistake is in the const EGETFUCT.
• slver 5y ago
  It's not a property of text-based protocols, rather it's a property of simple protocols. HTTP/1 is not merely text based, it's ASCII based (technically ISO-8859-1, which includes ASCII). One char, one byte, one encoding. HTTP itself is mostly very simple, text "name: value" pairs separated by newlines, followed by arbitrary content as the body. I think the solution is to start with a simple protocol and upgrade to more complex protocols after that. While technically you don't need to support HTTP/1 to support 2 and 3, the upgrade over TCP happens mostly in that way.
  • chrismorgan 5y ago
    > While technically you don't need to support HTTP/1 to support 2 and 3, the upgrade over TCP happens mostly in that way. This is not actually true of HTTP/2 (the upgrade doesn’t use HTTP/1), and slightly misleading of HTTP/3 (there’s nothing to upgrade because it sits beside TCP HTTP; but advertising HTTP/3 support is currently done over TCP HTTP). Now for the details: HTTP/2 upgrade is done at the TLS level, via ALPN. Essentially the client TLS handshake says “hello there! BTW do you do HTTP/2?” and the server either responds “hi!” and starts talking HTTP/1, or “hi! Let’s talk h2!” and starts talking HTTP/2. So it’s perfectly possible (though not a good idea) to have a fully-functioning HTTP/2 server that doesn’t speak a lick of HTTP/1. (HTTP/2 over cleartext, h2c, does use the old HTTP/1 Upgrade header mechanism, but h2c is more or less just not used by anyone.) HTTP/3 upgrade, well, “upgrade” is the wrong word. It’s operating over UDP rather than TCP, so you’re not upgrading an existing HTTP thing to HTTP/3, you’re starting a new connection because you’ve learned that the server supports HTTP/3. This bootstraping is currently done by advertising h3 support via the Alt-Svc header (HTTP/1+) or by an ALTSVC frame (HTTP/2+), which the client can then remember so it uses the best protocol next time. For best results, they’re working on allowing you to advertise HTTP/3 support on DNS, so that after a while it should be genuinely possible (though a very bad idea) to have a fully-functioning HTTP/3 server that doesn’t speak any TCP at all, yet works with sufficiently recent browsers in network environments that don’t break HTTP/3. https://blog.cloudflare.com/speeding-up-https-and-http-3-negotiation-with-dns/ https://blog.cloudflare.com/speeding-up-https-and-http-3-neg... is good info on this part.
  • deathanatos 5y ago
    > it's ASCII based (technically ISO-8859-1, which includes ASCII) No, it is (some) ASCII plus "opaque octets". ("A recipient SHOULD treat other octets in field content (obs-text) as opaque data.") If you want to say that historically it was such, that's not correct either; it was that plus "supporting other charsets only through use of [RFC2047] encoding" which is a nightmare of an encoding scheme. > HTTP itself is mostly very simple, text "name: value" pairs separated by newlines Except when it isn't: headers can be folded across multiple lines. (Which is also obsolete, and discouraged.) Add to that 1xx responses, transfer encodings, chunks, chunk extensions, trailers. HTTP is far from "simple"…
• anthk 5y ago
  Gopher is easier than that.
• geoffdunbar 5y ago
  I love C, but it's pretty scary sometime. 5 minutes ago, "I wonder if I can find a potential memory overwrite in 5 minutes?" Sure enough, the function StrAppend potentially overflows a size_t size (without checking), and then writes into memory could be past the end of the allocated buffer. Given 5 minutes, I didn't look thoroughly if this is actually exploitable, but it's definitely a red-flag for the code. Be careful out there! Hopefully I am missing something, or this is just a simple oversight, but I would carefully audit this code before using it. Submitted a ticket through the Althttpd website. static char StrAppend(char zPrior, const char zSep, const char zSrc){ char zDest; size_t size; size_t n0, n1, n2; if( zSrc==0 ) return 0; if( zPrior==0 ) return StrDup(zSrc); n0 = strlen(zPrior); n1 = strlen(zSep); n2 = strlen(zSrc); size = n0+n1+n2+1; zDest = (char*)SafeMalloc( size ); memcpy(zDest, zPrior, n0); free(zPrior); memcpy(&zDest[n0],zSep,n1); memcpy(&zDest[n0+n1],zSrc,n2+1); return zDest; }
  • spacechild1 5y ago
    > Sure enough, the function StrAppend potentially overflows a size_t size How should this happen in practice? The three strings would have to be larger than the available address space...
    • ectospheno 5y ago
      Yeah. The function in question is called in only one place. It would seem you’d need to send the web server more than a size_t of data for this to be an issue.
      • acqq 5y ago
        Exactly. In a single file C nobody can expect to get universal library functions that work in any possible imaginable context. The only relevant context is the code the function is in. And in that context, the function is doing enough.
      • geoffdunbar 5y ago
        Yes, absolutely. If the webserver is compiled 32-bit, that is only 4GB of data, which might be feasible? I don't know enough to say. Assuming a hacker kindly won't overflow your buffer is never a good idea. However, the presence of one piece of code that is not integer-overflow safe definitely makes me nervous. This is just the one I found in 5 minutes, what else is in there?