5 ms·

Why not just make it CGI (or FastCGI) compatible so you can test it locally or run it in other environments? At least for things that are actually generating re

Why not just make it CGI (or FastCGI) compatible so you can test it locally or run it in other environments? At least for things that are actually generating responses rather than acting as middleware. Oh, right, vendor lock-in.

solardev 4y ago
You can pretty easily change from Cloudflare to any other Jamstack host. Cgi just isn't necessary anymore and would be more baggage than anything at this point. The old web stacks are just time sinks. It's so much nicer to be able to git push... And that's it. Done.
- alehlopeh 4y ago
  Forgive my digression but for years I’ve seen small efforts here and there to elevate the concept of a “stack” and promote it as the level abstraction to use when discussing web apps and the like. What’s up with that? Grab any 3 commonly used pieces of technology whose names can be used to form a fun initialism, append stack to the end, and soon there will be experts speaking at conferences dedicated to it. JamStack is an example you mentioned. I’ve heard others over the years. I disagree that it’s a useful unit of thought. No one I know or work with ever talks about “web stacks” this way. Am I missing something here? Is this just some kind of niche? Who’s marketing these?
  - ameliaquining 4y ago
    Jamstack in particular is not really an example of that; it is more of a kind of architecture than a specific stack, in that it doesn't prescribe the use of specific software. An architecture where all the HTML/CSS/JS is precompiled statically for maximum cacheability, and server-side dynamic behavior happens exclusively within AJAX calls that return structured data, is a natural kind, and useful to have a term for so that we can compare and contrast it to other ways of building web apps. More broadly, other acronym stacks are a thing because complementary technologies cluster together, especially in relatively immature ecosystems. E.g., PHP traditionally had better support for mySQL than for competing databases, so if you were building an app in PHP, you probably used mySQL for data storage. These effects may have weakened over time as the graph of ecosystem integrations has grown more complete (and indeed I feel like I hear less about acronym stacks these days than I did half a decade ago), but I don't think they've died out entirely.
    - int_19h 4y ago
      The reason why PHP was bundled with MySQL historically is not so much because it had better support for it, but rather because cheap & free hosting would generally offer MySQL (rather than, say, Postgres) - because it was easier for them. If SQLite showed up a bit earlier, I bet that would have become the standard.
  - megous 4y ago
    It's been around since forever. Things like LAMP (Linux Apache MySQL PHP), etc.
  - solardev 4y ago
    What else would you call the well, stack, of technologies used to host a website? It's not a marketing term, just a way to describe the multiple layers. From hardware to network routing to DNS to tcp/up to http to https to HTML to JS to workers, and also Linux and PHP and a web server like Apache or nginx and a database like Maria or Postgres and maybe a cache like Redis or memcached and maybe squid and Varnish in front of it... that's a stack. Jamstack is just a particular configuration where vendors take care of all of that and you can push up frontend code that auto deploys and that's that. I used to have to manage the whole stack and now can work purely in HTML CSS and JS and it is a DREAM. That's the magic of the Jamstack. Less devops and network infrastructure, more time to focus on UX and DX. As for vendors, there's Vercel, Netlify, Gatsby, maybe a few others. Cloudflare, Deno, and Fly, Render and a few others have similar variations. It's a really nice way to work...
- harryvederci 4y ago
  > It's so much nicer to be able to git push... And that's it. Done. Can you elaborate? My CGI-based website is deployed with `git push` as well, not sure what the difference is there.
  - 9dev 4y ago
    Is it, though? Who does git pull at the server? What happens to in-flight requests during deployments? Traffic surges? Requests from Nigeria?
  - solardev 4y ago
    How's your hosting stack set up? The Jamstack vendors are zero config or maintenance
pjkundert 4y ago
You can run workerd locally.
ameliaquining 4y ago
Because V8's low-overhead tenant isolation is fundamental to how Cloudflare Workers works[1], and without it it would be a completely different product with worse performance. This means it can only support JavaScript and WebAssembly. You can run it locally[2], and efforts are underway to standardize much of it so that the same code can run on Workers, Node.js, and Deno[3]. The Workers runtime is also open source[4], although in practice this is likely to be less useful for most use cases than just using one of the other JavaScript runtimes if you're self-hosting. [1] https://developers.cloudflare.com/workers/learning/how-workers-works/ https://developers.cloudflare.com/workers/learning/how-worke... [2] https://developers.cloudflare.com/pages/platform/functions#develop-and-preview-locally https://developers.cloudflare.com/pages/platform/functions#d... [3] https://wintercg.org/ https://wintercg.org/ [4] https://github.com/cloudflare/workerd https://github.com/cloudflare/workerd
mfer 4y ago
It’s different that than vendor lock-in. For example you have security. CGI with multitenants is hard if not impossible. At least for the tenants that will want to use this. Then there is scaling, resource consumption, and predictability (for planning).
kentonv 4y ago
The engine is open source: https://github.com/cloudflare/workerd https://github.com/cloudflare/workerd We did not create our own engine to create "lock-in". On the contrary, it would be a huge win for us if we could simply run FastCGI or Node or whatever applications unmodified. We'd be able to onboard a lot more customers more quickly that way! Our product is our physical network (with hundreds of locations around the world), not our specific development environment. But none of that tech scales well enough that we could run every app in every one of our locations at an affordable price. Meeting that goal required new tech. The best we could do is base it on open standard APIs (browser APIs, mostly), so a lot of code designed for browsers ports over nicely. (I'm the lead engineer for Workers. These were my design choices, originally.)
- spiffytech 4y ago
  Are there guides for application architecture with Workers? I'm interested in Durable Objects but I don't know how to work within the 50-subrequest limit on the Free or Bundled plans. It sounds like I can only read from 50 Durable Objects or KV queries in a single request. If my usage pattern is e.g., sharing docs with many users, how would I let more than 50 users access a doc? Whether I consider fanning out on writes or fanning in on reads I'd need more than 50 subrequests. How should I approach this?
  - kentonv 4y ago
    Hmm, I think there are several different reasons why the subrequest limit actually doesn't apply to your scenario. First, Durable Objects and Workers KV requests do not count against the 50-subrequest limit. It's HTTP requests to external servers that count. Second, with the Unbound billing model, the limit is now 1000. The old "Bundled" billing model is really intended for "configure your CDN" use cases where you're mostly just rewriting a request or response and proxying to origin. For whole applications you should use Unbound. (Note that with Durable Objects, the objects themselves always use the Unbound pricing model.) Third, to address your specific scenario: > If my usage pattern is e.g., sharing docs with many users, how would I let more than 50 users access a doc? If you mean you have 50+ users connected via WebSockets and you need to send a message to each one -- these messages (on already-open WebSockets) do not count against the subrequest limit. Only starting a whole new HTTP request or new WebSocket would count. And for "fan in", these would be incoming requests from client, but the subrequest limit applies to outgoing request. There's no limit on number of incoming requests.
qbasic_forever 4y ago
For better or worse fast CGI never took off much beyond PHP. Python, node, ruby, etc. all do their own thing with their own app servers and protocols. I mean sure something like python can use fast CGI but no one in that community uses it that way, everyone uses uWSGI, gunicorn, etc. Plain old CGI doesn't scale beyond toy usage--it spawns a process per request. So there really isn't a good single option for Cloudflare to use.
- rakoo 4y ago
  > Plain old CGI doesn't scale beyond toy usage--it spawns a process per request. I'd be willing to bet the opposite: CGI is more than enough for 80% of workload, performance-wise. There are a few good reasons why CGI isn't the best today: the tooling doesn't exist, the security model is really bad, and you can cram fewer websites on a single machine, so for the same number of websites you need more machines, and that's an ecological aberration. But there is no problem about CGI being too slow.
  - qbasic_forever 4y ago
    Sure but developers don't want to bifurcate their service code and releases into a simple version using CGI and a dedicated app server version for when CGI doesn't scale for them or their users. They just want to write one python web app and run it everywhere. There's no way a CGI app can reach a million requests a second on the same hardware that a nodejs (single thread, worker loop) would take to do 1M RPS. Processes have a lot of overhead. Almost no one needs 1M RPS but it cannot be waived away that CGI is perfect for everything.
    - josephg 4y ago
      Yep. Also, sharing resources between connections is (at best) very difficult using CGI, and its trivially easy using a stand-alone app server. You can just set up a global variable at process startup and you're done. (This is useful in just about every web app I've ever written - eg for the connection to a database, cached resources, cached renders of common pages, in-memory only security tokens, etc). I also find the modern approach much easier to reason about and debug. I don't need Apache + fastCGI + php + php fastCGI + apache configuration to get started. I can just run `node server.js` and my web app works. The fact that it also scales much better is a cherry on top.

solardev 4y ago

You can pretty easily change from Cloudflare to any other Jamstack host. Cgi just isn't necessary anymore and would be more baggage than anything at this point. The old web stacks are just time sinks. It's so much nicer to be able to git push... And that's it. Done.

alehlopeh 4y ago
Forgive my digression but for years I’ve seen small efforts here and there to elevate the concept of a “stack” and promote it as the level abstraction to use when discussing web apps and the like. What’s up with that? Grab any 3 commonly used pieces of technology whose names can be used to form a fun initialism, append stack to the end, and soon there will be experts speaking at conferences dedicated to it. JamStack is an example you mentioned. I’ve heard others over the years. I disagree that it’s a useful unit of thought. No one I know or work with ever talks about “web stacks” this way. Am I missing something here? Is this just some kind of niche? Who’s marketing these?
- ameliaquining 4y ago
  Jamstack in particular is not really an example of that; it is more of a kind of architecture than a specific stack, in that it doesn't prescribe the use of specific software. An architecture where all the HTML/CSS/JS is precompiled statically for maximum cacheability, and server-side dynamic behavior happens exclusively within AJAX calls that return structured data, is a natural kind, and useful to have a term for so that we can compare and contrast it to other ways of building web apps. More broadly, other acronym stacks are a thing because complementary technologies cluster together, especially in relatively immature ecosystems. E.g., PHP traditionally had better support for mySQL than for competing databases, so if you were building an app in PHP, you probably used mySQL for data storage. These effects may have weakened over time as the graph of ecosystem integrations has grown more complete (and indeed I feel like I hear less about acronym stacks these days than I did half a decade ago), but I don't think they've died out entirely.
  - int_19h 4y ago
    The reason why PHP was bundled with MySQL historically is not so much because it had better support for it, but rather because cheap & free hosting would generally offer MySQL (rather than, say, Postgres) - because it was easier for them. If SQLite showed up a bit earlier, I bet that would have become the standard.
- megous 4y ago
  It's been around since forever. Things like LAMP (Linux Apache MySQL PHP), etc.
- solardev 4y ago
  What else would you call the well, stack, of technologies used to host a website? It's not a marketing term, just a way to describe the multiple layers. From hardware to network routing to DNS to tcp/up to http to https to HTML to JS to workers, and also Linux and PHP and a web server like Apache or nginx and a database like Maria or Postgres and maybe a cache like Redis or memcached and maybe squid and Varnish in front of it... that's a stack. Jamstack is just a particular configuration where vendors take care of all of that and you can push up frontend code that auto deploys and that's that. I used to have to manage the whole stack and now can work purely in HTML CSS and JS and it is a DREAM. That's the magic of the Jamstack. Less devops and network infrastructure, more time to focus on UX and DX. As for vendors, there's Vercel, Netlify, Gatsby, maybe a few others. Cloudflare, Deno, and Fly, Render and a few others have similar variations. It's a really nice way to work...
harryvederci 4y ago
> It's so much nicer to be able to git push... And that's it. Done. Can you elaborate? My CGI-based website is deployed with `git push` as well, not sure what the difference is there.
- 9dev 4y ago
  Is it, though? Who does git pull at the server? What happens to in-flight requests during deployments? Traffic surges? Requests from Nigeria?
- solardev 4y ago
  How's your hosting stack set up? The Jamstack vendors are zero config or maintenance

pjkundert 4y ago

You can run workerd locally.

ameliaquining 4y ago

Because V8's low-overhead tenant isolation is fundamental to how Cloudflare Workers works[1], and without it it would be a completely different product with worse performance. This means it can only support JavaScript and WebAssembly. You can run it locally[2], and efforts are underway to standardize much of it so that the same code can run on Workers, Node.js, and Deno[3]. The Workers runtime is also open source[4], although in practice this is likely to be less useful for most use cases than just using one of the other JavaScript runtimes if you're self-hosting. [1] https://developers.cloudflare.com/workers/learning/how-workers-works/ https://developers.cloudflare.com/workers/learning/how-worke... [2] https://developers.cloudflare.com/pages/platform/functions#develop-and-preview-locally https://developers.cloudflare.com/pages/platform/functions#d... [3] https://wintercg.org/ https://wintercg.org/ [4] https://github.com/cloudflare/workerd https://github.com/cloudflare/workerd

mfer 4y ago

It’s different that than vendor lock-in. For example you have security. CGI with multitenants is hard if not impossible. At least for the tenants that will want to use this. Then there is scaling, resource consumption, and predictability (for planning).

kentonv 4y ago

The engine is open source: https://github.com/cloudflare/workerd https://github.com/cloudflare/workerd We did not create our own engine to create "lock-in". On the contrary, it would be a huge win for us if we could simply run FastCGI or Node or whatever applications unmodified. We'd be able to onboard a lot more customers more quickly that way! Our product is our physical network (with hundreds of locations around the world), not our specific development environment. But none of that tech scales well enough that we could run every app in every one of our locations at an affordable price. Meeting that goal required new tech. The best we could do is base it on open standard APIs (browser APIs, mostly), so a lot of code designed for browsers ports over nicely. (I'm the lead engineer for Workers. These were my design choices, originally.)

spiffytech 4y ago
Are there guides for application architecture with Workers? I'm interested in Durable Objects but I don't know how to work within the 50-subrequest limit on the Free or Bundled plans. It sounds like I can only read from 50 Durable Objects or KV queries in a single request. If my usage pattern is e.g., sharing docs with many users, how would I let more than 50 users access a doc? Whether I consider fanning out on writes or fanning in on reads I'd need more than 50 subrequests. How should I approach this?
- kentonv 4y ago
  Hmm, I think there are several different reasons why the subrequest limit actually doesn't apply to your scenario. First, Durable Objects and Workers KV requests do not count against the 50-subrequest limit. It's HTTP requests to external servers that count. Second, with the Unbound billing model, the limit is now 1000. The old "Bundled" billing model is really intended for "configure your CDN" use cases where you're mostly just rewriting a request or response and proxying to origin. For whole applications you should use Unbound. (Note that with Durable Objects, the objects themselves always use the Unbound pricing model.) Third, to address your specific scenario: > If my usage pattern is e.g., sharing docs with many users, how would I let more than 50 users access a doc? If you mean you have 50+ users connected via WebSockets and you need to send a message to each one -- these messages (on already-open WebSockets) do not count against the subrequest limit. Only starting a whole new HTTP request or new WebSocket would count. And for "fan in", these would be incoming requests from client, but the subrequest limit applies to outgoing request. There's no limit on number of incoming requests.

qbasic_forever 4y ago

For better or worse fast CGI never took off much beyond PHP. Python, node, ruby, etc. all do their own thing with their own app servers and protocols. I mean sure something like python can use fast CGI but no one in that community uses it that way, everyone uses uWSGI, gunicorn, etc. Plain old CGI doesn't scale beyond toy usage--it spawns a process per request. So there really isn't a good single option for Cloudflare to use.

rakoo 4y ago
> Plain old CGI doesn't scale beyond toy usage--it spawns a process per request. I'd be willing to bet the opposite: CGI is more than enough for 80% of workload, performance-wise. There are a few good reasons why CGI isn't the best today: the tooling doesn't exist, the security model is really bad, and you can cram fewer websites on a single machine, so for the same number of websites you need more machines, and that's an ecological aberration. But there is no problem about CGI being too slow.
- qbasic_forever 4y ago
  Sure but developers don't want to bifurcate their service code and releases into a simple version using CGI and a dedicated app server version for when CGI doesn't scale for them or their users. They just want to write one python web app and run it everywhere. There's no way a CGI app can reach a million requests a second on the same hardware that a nodejs (single thread, worker loop) would take to do 1M RPS. Processes have a lot of overhead. Almost no one needs 1M RPS but it cannot be waived away that CGI is perfect for everything.
  - josephg 4y ago
    Yep. Also, sharing resources between connections is (at best) very difficult using CGI, and its trivially easy using a stand-alone app server. You can just set up a global variable at process startup and you're done. (This is useful in just about every web app I've ever written - eg for the connection to a database, cached resources, cached renders of common pages, in-memory only security tokens, etc). I also find the modern approach much easier to reason about and debug. I don't need Apache + fastCGI + php + php fastCGI + apache configuration to get started. I can just run `node server.js` and my web app works. The fact that it also scales much better is a cherry on top.