4 ms·
Just this past week I've been working on a toy/experimental web DSL [0] that uses dynamically loaded shared libraries as middleware that pass per request arena-
by williamcotton 1y ago
Just this past week I've been working on a toy/experimental web DSL [0] that uses dynamically loaded shared libraries as middleware that pass per request arena-allocated Jansson json objects between steps in a pipeline. It's extensible in that new middleware can be created. Influenced by bash and OCaml/F#, here is some kind of demo of the syntax:
POST /api/users
|> validate: `
name: string(3..50)
email: email
age?: number(18..120)
team_id?: number
`
|> jq: `{ sqlParams: [.body.name, .body.email, .body.age] }`
|> pg: `INSERT INTO users (name, email, age) VALUES ($1, $2, $3) RETURNING *`
|> result
ok(201):
|> jq: `{ success: true, user: .data.rows[0] }`
validationError(400):
|> jq: `{
error: "Validation failed",
field: .errors[0].field,
rule: .errors[0].rule,
message: .errors[0].message
}`
I'm generally curious as to how jyn thinks this would fit in to their box-based framework.
[0] https://github.com/williamcotton/webpipe https://github.com/williamcotton/webpipe
- freedomben 1y agoThis looks really neat! I typically don't like these DSLs, but this is one I would actually use.
- cryptonector 1y agoI've wanted to do just this. I've used libmicrohttpd, and... it's not my favorite. But if you're going to do it in C then libmicrohttpd is probably the best API to use. You get bonus points for using jq :) If you're also using jq you might as well ditch Jansson and use jq's `jv` API. I highly recommend it.
- williamcotton 1y agoOne thing that sold me on Jansson is that you can set custom allocators which work very well with the request arena. Once those allocators are set the json objects continue to use the arena even when used in middleware. This makes memory management a cinch! From what I can tell jq’s C lib doesn’t yet expose a way to set custom allocators.
- cryptonector 1y agoYou can set a custom allocator, but it's global. You could contribute a feature to set thread-local custom allocators -- it'd be quite easy.
- cryptonector 1y agoOh right, no, jq has a custom out-of-memory handler. But it should be trivial to extend src/jv_alloc.[ch] to let you set not just a custom nomem handler but also a custom allocator, and then you can use thread-locals in your allocator.
- jynelson 1y agooh hey Will! long time no see lol, it's been ages. small world. i think this is on a good track! i like that it's designed to be extensible while still keeping some amount of structure, and that the DSL makes things very compact. how are those filters implemented? are you spawning a shell to run jq or are you interpreting in-process or something like that? in general i'd love to see a bunch more DSLs, i think using general-purpose languages for everything is most of the time not actually helpful (cc https://jyn.dev/constrained-languages-are-easier-to-optimize/#what-does-a-faster-general-purpose-language-look-like https://jyn.dev/constrained-languages-are-easier-to-optimize...). i have some vague thoughts about how to make interop between DSLs and the host language easier but i need to think about them some more.
- williamcotton 1y ago> how are those filters implemented? are you spawning a shell to run jq or are you interpreting in-process or something like that? Interpreting in-process. There's a jq C lib, libjq-dev. I hope you're doing well! I've very much enjoyed your recent blog posts.