5 ms·

A little context: I'm specifically interested in compressing my analytics stack. I'm currently interacting with HTTP Messages at a higher level. The problem, a

A little context: I'm specifically interested in compressing my analytics stack. I'm currently interacting with HTTP Messages at a higher level. The problem, as I see it, is the webserver is already doing this work but without useful output. I'd like to implement my analytics paradigm natively at the webserver level.

habibur 4y ago
Parse server logs, if that helps.
e12e 4y ago
I'm not sure I follow - you want a smart, small webserver that takes in a http request and updates an analytic store based on headers? Sounds like something nginx might handle via lua scripting? Otherwise, maybe you want: https://github.com/lammertb/libhttp https://github.com/lammertb/libhttp Or you might want to look at either of: https://www.acme.com/software/thttpd/ https://www.acme.com/software/thttpd/ https://github.com/openbsd/src/tree/master/usr.sbin/httpd https://github.com/openbsd/src/tree/master/usr.sbin/httpd If you really want to build (another) web server in C. Not entirely clear why want to use C for this, though?
- convolvatron 4y ago
  for some applications it really might be easier to write a little thing than integrate into some giant framework. and I do think that building these things really helps one understand what's going on in all those layers but I have to agree about C. unless the whole point is to ingrate with some existing codebase, there are several language choices which should turn this from a day or two of work to an hour or two - with a more robust outcome
- shanebellone 4y ago
  I want to consume HTTP headers at the lowest possible level. The webserver would output a custom log file for direct consumption. There are certainly off the shelf options, but I enjoy reinventing the wheel and dislike dependencies. I'd like to use C because I have interest in OS level webservers.
  - convolvatron 4y ago
    this is a 500 line server running in kernel mode on nanos https://github.com/nanovms/nanos/blob/master/src/http/http.c https://github.com/nanovms/nanos/blob/master/src/http/http.c
  - e12e 4y ago
    I would probably go for rust over c if the only reason for c is "os level". But either way, this is for requests from "real" http clients, eg: web browsers? Presumably you're already "trapped" at a pretty high level, like tls, http/2 etc? Otoh, if low level is what you want, maybe look at: https://2ton.com.au/rwasa/ https://2ton.com.au/rwasa/ and/or the library it is built on. On the other hand, if you just want to log headers to a file, caddy can do that in a few lines: https://caddyserver.com/docs/caddyfile/directives/log https://caddyserver.com/docs/caddyfile/directives/log AFAIK it can't log request bodies out of the box (like the content of a POST).
stevekemp 4y ago
In that case you might be better off looking at writing a plugin for your existing webserver. nginx, apache, etc, all allow writing modules that can be invoked at various parts of the HTTP-communication.
- shanebellone 4y ago
  I don't technically run a webserver. I've built around Azure App Services which runs gunicorn behind the scenes. Also, I rather own the majority of my stack even though its slower, harder, and more problematic.

habibur 4y ago

Parse server logs, if that helps.

e12e 4y ago

I'm not sure I follow - you want a smart, small webserver that takes in a http request and updates an analytic store based on headers? Sounds like something nginx might handle via lua scripting? Otherwise, maybe you want: https://github.com/lammertb/libhttp https://github.com/lammertb/libhttp Or you might want to look at either of: https://www.acme.com/software/thttpd/ https://www.acme.com/software/thttpd/ https://github.com/openbsd/src/tree/master/usr.sbin/httpd https://github.com/openbsd/src/tree/master/usr.sbin/httpd If you really want to build (another) web server in C. Not entirely clear why want to use C for this, though?

convolvatron 4y ago
for some applications it really might be easier to write a little thing than integrate into some giant framework. and I do think that building these things really helps one understand what's going on in all those layers but I have to agree about C. unless the whole point is to ingrate with some existing codebase, there are several language choices which should turn this from a day or two of work to an hour or two - with a more robust outcome
shanebellone 4y ago
I want to consume HTTP headers at the lowest possible level. The webserver would output a custom log file for direct consumption. There are certainly off the shelf options, but I enjoy reinventing the wheel and dislike dependencies. I'd like to use C because I have interest in OS level webservers.
- convolvatron 4y ago
  this is a 500 line server running in kernel mode on nanos https://github.com/nanovms/nanos/blob/master/src/http/http.c https://github.com/nanovms/nanos/blob/master/src/http/http.c
- e12e 4y ago
  I would probably go for rust over c if the only reason for c is "os level". But either way, this is for requests from "real" http clients, eg: web browsers? Presumably you're already "trapped" at a pretty high level, like tls, http/2 etc? Otoh, if low level is what you want, maybe look at: https://2ton.com.au/rwasa/ https://2ton.com.au/rwasa/ and/or the library it is built on. On the other hand, if you just want to log headers to a file, caddy can do that in a few lines: https://caddyserver.com/docs/caddyfile/directives/log https://caddyserver.com/docs/caddyfile/directives/log AFAIK it can't log request bodies out of the box (like the content of a POST).

stevekemp 4y ago

In that case you might be better off looking at writing a plugin for your existing webserver. nginx, apache, etc, all allow writing modules that can be invoked at various parts of the HTTP-communication.

shanebellone 4y ago
I don't technically run a webserver. I've built around Azure App Services which runs gunicorn behind the scenes. Also, I rather own the majority of my stack even though its slower, harder, and more problematic.