8 ms·

Python Cloudflare Workers

tyingq 3y ago
A performance comparison to a JS worker would be helpful. It does sound interesting, but also sounds potentially slow, given all the layers involved. Not that I'm expecting parity, but knowing the rough tradeoff would be helpful.
- deadbabe 3y ago
  Anecdotally it seems very fast.
- brendanib 3y ago
  Three aspects of performance: 1. Cold start perf 2. Post-cold start perf - The cost of bridging between JS and WebAssembly - The speed of the Python interpreter running in WebAssembly Today, Python cold starts are slower than cold starts for a JavaScript Worker of equivalent size. A basic "Hello World" Worker written in JavaScript has a near zero cold start time, while a Python Worker has a cold start under 1 second. That's because we still need to load Pyodide into your Worker on-demand when a request comes in. The blog post describes what we're working on to reduce this — making Pyodide already available upfront. Once a Python Worker has gone through a cold start though, the differences are more on the margins — maybe a handful milliseconds, depending on what happens during the request. - There is a slight cost (think — microseconds not milliseconds) to crossing the "bridge" between JavaScript and WebAssembly — for example, by performing I/O or async operations. This difference tends to be minimal — generally something measured in microseconds not milliseconds. People with performance sensitive Workers already write them in Rust https://github.com/cloudflare/workers-rs https://github.com/cloudflare/workers-rs, which also relies on bridging between JavaScript and WebAssembly. - The Python interpreter that Pyodide provides, that runs in WebAssembly, isn't as fast as the years and years of optimization that have gone into making JavaScript fast in V8. But it's still relatively early days for Pyodide, compared to the JS engine in V8 — there are parts of its code where we think there are big perf gains to be had. We're looking forward to upstreaming performance improvements, and there are WebAssembly proposals that help here too.
  - riazrizvi 3y ago
    Very helpful- thanks
gregorymichael 3y ago
I’ve used CF Pages for static sites with great results and am intrigued by all their open-source-LLM-as-a-service offerings. Main issue preventing me from building more on CF is lack of Python support. Excited to try this out.
- radlad 3y ago
  Yes! I'm also using CF Pages, and a couple Worker functions, and really love the CF ecosystem. Very easy to get something running quickly, and not have to worry much about infrastructure. Very happy to see the Python addition. I'd like to see first-class Go support as well.
jasoncartwright 3y ago
I've played with JS workers on a Cloudflare-fronted site and found them to be easy to use and very quick. Would love to port the whole Django app behind the site over, using their D1 database too.
- woutr_be 3y ago
  Same here, I have a couple of mobile apps that use Cloudflare workers + KV/D1, and it’s been great. I’m low traffic enough to be on the free tier, but would happily pay given how easy it’s been to build on.
- infamia 3y ago
  Agreed, this looks really cool. While there is no Django/DRF support at the moment, it does say that that they'll be increasing the number of packages in the future.
  - brendanib 3y ago
    Would love feedback on which packages you'd like to see us support: https://github.com/cloudflare/workerd/discussions/categories/python-packages https://github.com/cloudflare/workerd/discussions/categories...
- manishsharan 3y ago
  >>Would love to port the whole Django app behind the site over, using their D1 database too. Is that wise? One DDOS attack could break your budget.
  - piperswe 3y ago
    Only if the DDoS isn't blocked by the Cloudflare DDoS protection
    - spxneo 3y ago
      what are the advantages of using Cloudflare db over supabase? So far im loving supabase but wasn't aware CF products have increased drastically Rows read 5 million / day First 25 billion / month included + $0.001 / million rows Rows written 100,000 / day First 50 million / month included + $1.00 / million rows Storage (per GB stored) 5 GB (total) First 5 GB included + $0.75 / GB-mo
noman-land 3y ago
This is kind of a game changer for running AI stuff on Cloudflare. Been hoping for this for a while now.
- dabber 3y ago
  > This is kind of a game changer for running AI stuff on Cloudflare. That certainly appears to be the intention. > Been hoping for this for a while now. You should check out the other two announcements from today as well if you haven't yet: "Leveling up Workers AI: General Availability and more new capabilities" https://blog.cloudflare.com/workers-ai-ga-huggingface-loras-python-support https://blog.cloudflare.com/workers-ai-ga-huggingface-loras-... "Running fine-tuned models on Workers AI with LoRAs" https://blog.cloudflare.com/fine-tuned-inference-with-loras https://blog.cloudflare.com/fine-tuned-inference-with-loras
  - CharlesW 3y ago
    Also, D1, Hyperdrive, and Workers Analytics Engine went GA. https://blog.cloudflare.com/making-full-stack-easier-d1-ga-hyperdrive-queues https://blog.cloudflare.com/making-full-stack-easier-d1-ga-h...
- spxneo 3y ago
  what is the maximum length a worker can run for? curious how this compares to AWS Lambda? or is it something completely different?
  - brendanib 3y ago
    There's no hard limit on duration! 30 seconds of CPU time https://developers.cloudflare.com/workers/platform/limits/#duration https://developers.cloudflare.com/workers/platform/limits/#d...
jatins 3y ago
Clouflare has a lot of great stuff for hosting and databases but I think they haven't done a great job marketing themselves as developer platform which has lead to platforms like Vercel, Netlify taking significant mindshare. Tangential: does Cloudflare provide container hosting service agnostic of language -- something like Google Cloud Run?
- thangngoc89 3y ago
  > does Cloudflare provide container hosting service agnostic of language -- something like Google Cloud Run? Nope. Their Workers are V8 based so JS or Wasm
- deleted 3y ago
  [deleted]
- Difwif 3y ago
  I agree something is wrong with their marketing. I was also initially drawn to Vercel and Netlify but after extended use and not being happy with either I eventually tried Cloudflare and discovered I love it. The pricing and the product is fantastic.
  - spxneo 3y ago
    Same experience its fascinating how much marketing impacts developer minds.
  - nstart 3y ago
    I think it’s because the experience of familiarizing oneself with the platform and getting to a hello world level crud app/basic static site is done a lot better with vercel and netlify than it is with cloudflare. Cloudflares site and docs are not built with the approach of getting an app from 0 to 1 ASAP.
    - rvnx 3y ago
      Pricing also. 0.40 USD / month for a globally deployed site @ CF versus 199 USD / month for a static site with limited traffic usage @ the supposedly premium other hosts
pelletier 3y ago
I'm curious to see how the limitation of using pyodide packages only will play out for non-trivial builds. Thinking of all the non-pure python code out there that need to be manually rebuilt to support a non-trivial production app. Maybe Cloudflare's adoption will help bring more packages into the fold, and if it's an 80/20 rule here, would be good enough.
- hoodchatham 3y ago
  I certainly think there's an 80/20 rule here. Most packages are not very hard to port, and generally the ones that are hard to build use features like threads and multiprocessing, graphics cards, raw sockets, green threads, or other capabilities that have no obvious analogue in a webassembly runtime. As we mention in the blog post, the biggest issues are around supporting server and request packages since they are clearly useful in cloudflare workers but are difficult to port because they frequently use raw sockets and some form of concurrency.
  - dom96 3y ago
    As we build out support for some of these features in the Workers Runtime, we should be able to port Python modules to use them. Some features like raw sockets are already available, so we should be able to make some quick headway here. (Myself and Hood above are the folks who implemented Python Workers)
neonsunset 3y ago
I with they added Azure Functions style workers using C# too, or AWS style lambdas using NativeAOT. Way lower runtime overhead and time to first response latency. But C# is an underdog language in those lands, so it's understandable.
- mdasen 3y ago
  At the moment, it seems like they're concentrating on using V8 isolates. In the article, there's a good diagram of why: an isolate is able to share so much between different applications. Even with NativeAOT, you're still launching an entire program that has to load everything into memory and execute. In some ways, they're using V8 isolates the way that mod_php was used back in the day. One reason PHP became so dominant was because PHP was cheap and easy to deploy for small websites. Because the PHP runtime contained 90% of what a person wanted to do with PHP, your PHP code might be a small amount of code that mostly just called standard library functions like `mysql_query()`. If you were running a shared hosting service, you could have huge numbers of people running on the same box because every PHP script would be sharing a single instance of the PHP standard library - and that standard library was fast and written in C. If you wanted to offer Python hosting, each Python app would be duplicating the standard library they were using in memory and also needing lots of web packages that aren't part of the standard library (like a database package). So a minimal Python application was using tons more RAM because it wasn't sharing most of the code with everyone else on the box. Even with NativeAOT, you're still duplicating a lot when running many different C# projects - as is the case with Go, Java, Ruby, etc. V8 isolates are this case where they tend to be lighter weight because so much can be shared between different users in the system. In fact, the reason they're supporting Python is because Pyodide (Python interpreter in WASM) allows for dynamic linking. It means they can have a single Pyodide interpreter in memory that's shared by all the Python workers on the same box. Likewise, they can also share Python libraries that two different people on the same box might be using. They note that most languages that target WASM don't support dynamic linking and that the only way they can provide Cloudflare Workers at the price point they offer is because those Workers can share so much rather than duplicating and using more memory for each user. If you really want C# on Workers, C# does support WASM.
ssijak 3y ago
Can we just get full node runtime? Cloudflare is amazing, but without a full node runtime, we (and most of the usual apps) can't switch from things like Vercel/Netlify to Cloudflare.
- kentonv 3y ago
  The unique architecture of our runtime is what enables most of our competitive advantages. It's what lets us run your application in hundreds of locations around the world while also charging less than competing serverless platforms. If we used a full Node runtime, we would need to charge a lot more money, or only run your app in a couple central locations, or both. So, no, we can't just offer full Node. However, we are always expanding our Node API compatibility: https://developers.cloudflare.com/workers/runtime-apis/nodejs/ https://developers.cloudflare.com/workers/runtime-apis/nodej... (I'm the tech lead for Cloudflare Workers.)
  - switch007 3y ago
    Your comment would have been great without: > So, no, we can't just offer full Node. (Sounds a bit snotty)
    - cstrahan 3y ago
      How else would you write that? Would this be better? "Thus, no, we can't just offer full Node." Is it the use of "so" that is off limits? Or is simply providing a concise conclusion inappropriate? I don't see what part of that quote is "snotty".
      - switch007 3y ago
        Omitting it No PR person would have included that line
  - ssijak 3y ago
    I understand what you said. I did not expect full node runtime workers with all the other benefits that you listed that current workers have (global distribution, no cold starts, cost..). But it would be great to have a choice. I feel it would benefit both users and Cloudflare to support both (with different tradeoffs). For example, selecting "I want node runtime for this app workers," I would need to select the region they will run in, and that's it, those functions/workers would not be globally distributed, would probably cost more, but when you need full node runtime you need it and that is fine.
zinclozenge 3y ago
I'd be curious to see a direct performance comparison between their python and JS workers. Based on my own experience with pyodide, I'd wager there might be up to a 2x performance penalty.
paddy_m 3y ago
Glad to see it includes numpy (and presumably pandas). Getting those to work in constrained serverless environments can be a huge pain.
syrusakbary 3y ago
This is awesome, I'm happy that Cloudflare is adding more attention into running Python via WebAssembly at the Edge. I'll try to summarize on how they got it running and what are the drawbacks that they have from their current approach (note: I have deep context on running Python with WebAssembly at the Edge as part of my work in Wasmer). Cloudflare Workers are enabling Python at the Edge by using Pyodide [1] (Python compiled to WebAssembly via Emscripten). They bundled Pyodide into Workerd [2], and then use V8 snapshots [3] to try to accelerate startup times. On their best case, cold starts of Python in Cloudflare Workers are about 1 second. While this release is great as it allows them to measure the interest of running Python at the Edge, it has some drawbacks. So, what are those? * Being tied to use only one version of Python/Pyodide (the one that Workerd embeds) * Package resolution is quite hacky and tied to workerd. Only precompiled "native packages" will be allowed to be used at runtime (eg. using a specific version of numpy will turn to be challenging) * Architecturally tied to the JS/v8 world, which may show some challenges as they aim to reduce cold start times (in my opinion, it will be quite hard for them to achieve <100ms startup time with their current architecture). In any case, I welcome this initiative with my open hands and look forward all the cool apps that people will now build with this! [1] https://pyodide.org/ https://pyodide.org/ [2] https://github.com/cloudflare/workerd/blob/main/docs/pyodide.md https://github.com/cloudflare/workerd/blob/main/docs/pyodide... [3] https://github.com/cloudflare/workerd/pull/1875 https://github.com/cloudflare/workerd/pull/1875 Edit: updated wording from "proof of concept" to "release" to reflect the clarification from the Cloudflare team
- panqueca 3y ago
  Does this architecture supports uvloop?
  - syrusakbary 3y ago
    As far as I know uvloop is not supported in Pyodide, mainly because it requires compiling libuv into WebAssembly (which is possible but not trivial). In any case, it shall be possible to run uvloop fully inside of WebAssembly. However, doing so will prove challenging using their current architecture
  - 3y ago
alfor 3y ago
I don't see how people will start using a completely new way python is running. If you are just experimenting and having fun, sure. But would you bet your company or many many months of developpement on this? What happen if you get random bugs? The advantage need to be extremely high to make it worth it. Maybe for specialized work that need to happen at the edge and then, why not use js instead that is the bedrock of this implementation?
- jppope 3y ago
  Cloudflare's engineering is top notch. I wouldn't expect anything different than what you would get from any of the other major cloud providers. Workers are also extremely fast/performant and inexpensive... if you are working on a company those two aspects can be fairly important to the success of some types of companies.
- victorbjorklund 3y ago
  I can totally see it being used for small services where python has better libaries
pjmlp 3y ago
Why anyone would like to slow down their requests using a full interpred implementation is behind me. Don't be surprised by scalability issues.
- atomicnumber3 3y ago
  There's a lot of value in "just write a python thing in 5 minutes". People tend to hem and haw about performance and "doing it right." But it's often a misplaced argument of "python vs ["right" thing]", it's "python vs not having anything." Often, a shitty python thing is worth a ton and fixes the problem, and then if performance becomes enough of an issue you can evaluate whether you want to prioritize fixing it. And I find that once something that works is in place, and just quietly doing its job, people suddenly find it a lot less objectionable to their sensibilities. And even if we do replace it, having the python thing taking the heat off the "right" way's timetable lets you actually do it right because you can take your time.
  - pjmlp 3y ago
    We all know what is the outcome of that temporary script that was written in 5 minutes.
harikb 3y ago
Is the choice of lzma to demonstrate isolation intentional or was it just a coincidence considering last week's tech news...[1] [1] https://news.ycombinator.com/item?id=39865810 https://news.ycombinator.com/item?id=39865810
- garrettgu10 3y ago
  Haha, we included it just because it's part of the standard library. Total coincidence in terms of timing but it's nice that using Wasm gives us isolation guarantees :-)
  - dom96 3y ago
    Yeah, pure coincidence. I picked it before the xz news broke.
jarpineh 3y ago
More development and users for Pyodide is great news. Especially that better serverless story for Python server frameworks. I wonder if Jupyter can work in this stack? It is essentially JavaScript, but built for browser environment. Just the Python kernel might be worker compatible. It essentially has to do code evaluation which might a limitation as well. Should it work you could offload compute from browser or other HTTP clients to waster resources of worker environment. Direct access to databases would be better as well.
- akshayka 3y ago
  Python notebooks can definitely work in WASM: - https://jupyterlite.readthedocs.io/en/stable/ https://jupyterlite.readthedocs.io/en/stable/ - https://docs.marimo.io/guides/wasm.html https://docs.marimo.io/guides/wasm.html Running just the kernel in a Cloudflare Worker is an interesting idea ...
  - jarpineh 3y ago
    Yes, Jupyter could work. I just fear there is things that except browser’s JavaScript APIs. Some I think aren’t even available for web workers. V8 workers are a different thing (I believe) and not at all familiar to me. But, it should be easy enough to test… https://github.com/jupyterlite/jupyterlite?tab=readme-ov-file#browser-based-interactive-computing https://github.com/jupyterlite/jupyterlite?tab=readme-ov-fil... <- Currently kernel supports at least web workers.
rodolphoarruda 3y ago
Interesting. HTMx -> Python -> SQlite all in Cloudflare. I was kind of waiting for this day.
hdlothia 3y ago
Wow this is huge for llm and data engineering. Many of the best libraries are in Python
adam_arthur 3y ago
I would like to see CloudFlare implement workers with WASM as the first class citizen, and a general purpose API not tied to JS workers. Up until now you've been able to deploy WASM code (e.g. effectively can use any language), but it runs within a JS context, rather than natively. Just a bit more overhead/awkwardness in deployment. I believe eventually all services will be deployed directly to WASM (securitized) runtimes, rather than via containers, similar to how we moved from images -> containers). There's very little benefit currently to trying to use something like Rust on the edge (in CF), because a lot of the perf advantage is negated by the overhead and startup times. e.g. https://github.com/WasmEdge/WasmEdge https://github.com/WasmEdge/WasmEdge
devwastaken 3y ago
It compiles python to we assembly which then runs on their modified V8 runtime. We assembly is generally a non solution to any problem, especially not this one. While it is convenient it is a clear lack of engineering ability that they can't implement a proper Python runtime. The reasons to not use wasm are many - the tool chains for emscripten are not well documented, hacky, and we're not built to the quality you'd expect from a compiler. After all it's doing something nothing was designed for. The performance will never be an improvement over a native engine, much of the context is lost in translation when compiling to wasm.
Terretta 3y ago
"This is about more than just making bindings to resources on Cloudflare more Pythonic though — it’s about compatibility with the ecosystem." Someone might be getting editorial help from GPT-4. // Or a human might be getting fine-tuned interacting with LLMs, which I've noticed happening to me.
- ggu 3y ago
  [dead]
- Terretta 3y ago
  To the dead comment beside me, asking why... GPT-4 has a variety of tells. One of them is: "This is not just about good thing, it's about another good thing." That "it's not just A, it's also B", or "it's more than A, it's B too" show up most any time you ask it for "persuasive" copy, like marketing, sales, or a rewrite of anything to make the reader a buyer. It's disproportionately common in GPT-4 copywriting or copy-editing, relative to human copy. Similar to seeing the word "Overall, ..." for a concluding paragraph, another tell.
  - corinroyal 3y ago
    My personal favorite is, "It's important to note..." I asked it to stop using that phrase or variations and that lasted one prompt. I'm tempted to put the phrase on a T-shirt.
pbamotra 3y ago
Interesting - supports FastAPI and Langchain too https://developers.cloudflare.com/workers/languages/python/packages/ https://developers.cloudflare.com/workers/languages/python/p...
anon373839 3y ago
With Pyodide getting some serious backing, is there a glimmer of hope that we could end up with Python as a real alternative to JavaScript in the frontend?
- deleted 3y ago
  [deleted]
fastball 3y ago
Tried this out today and it was great, was very quick to get up and running! One question though – does anyone know how I can get my local dev environment to understand the libraries that are built-in to CFW's Python implementation? e.g. there is an `asgi` library that I do not want my linter to flag as unknown, but as it only exists at runtime in the `on_fetch` handler (and isn't actually present in my local dev machine, I couldn't figure this out.
iamcreasy 3y ago
As opposed to what the article says, urllib3 now has experimental support for browser as of January this year. Source: https://github.com/urllib3/urllib3/releases/tag/2.2.0 https://github.com/urllib3/urllib3/releases/tag/2.2.0

A performance comparison to a JS worker would be helpful. It does sound interesting, but also sounds potentially slow, given all the layers involved. Not that I'm expecting parity, but knowing the rough tradeoff would be helpful.

deadbabe 3y ago
Anecdotally it seems very fast.
brendanib 3y ago
Three aspects of performance: 1. Cold start perf 2. Post-cold start perf - The cost of bridging between JS and WebAssembly - The speed of the Python interpreter running in WebAssembly Today, Python cold starts are slower than cold starts for a JavaScript Worker of equivalent size. A basic "Hello World" Worker written in JavaScript has a near zero cold start time, while a Python Worker has a cold start under 1 second. That's because we still need to load Pyodide into your Worker on-demand when a request comes in. The blog post describes what we're working on to reduce this — making Pyodide already available upfront. Once a Python Worker has gone through a cold start though, the differences are more on the margins — maybe a handful milliseconds, depending on what happens during the request. - There is a slight cost (think — microseconds not milliseconds) to crossing the "bridge" between JavaScript and WebAssembly — for example, by performing I/O or async operations. This difference tends to be minimal — generally something measured in microseconds not milliseconds. People with performance sensitive Workers already write them in Rust https://github.com/cloudflare/workers-rs https://github.com/cloudflare/workers-rs, which also relies on bridging between JavaScript and WebAssembly. - The Python interpreter that Pyodide provides, that runs in WebAssembly, isn't as fast as the years and years of optimization that have gone into making JavaScript fast in V8. But it's still relatively early days for Pyodide, compared to the JS engine in V8 — there are parts of its code where we think there are big perf gains to be had. We're looking forward to upstreaming performance improvements, and there are WebAssembly proposals that help here too.
- riazrizvi 3y ago
  Very helpful- thanks

gregorymichael 3y ago

I’ve used CF Pages for static sites with great results and am intrigued by all their open-source-LLM-as-a-service offerings. Main issue preventing me from building more on CF is lack of Python support. Excited to try this out.

radlad 3y ago
Yes! I'm also using CF Pages, and a couple Worker functions, and really love the CF ecosystem. Very easy to get something running quickly, and not have to worry much about infrastructure. Very happy to see the Python addition. I'd like to see first-class Go support as well.

jasoncartwright 3y ago

I've played with JS workers on a Cloudflare-fronted site and found them to be easy to use and very quick. Would love to port the whole Django app behind the site over, using their D1 database too.

woutr_be 3y ago
Same here, I have a couple of mobile apps that use Cloudflare workers + KV/D1, and it’s been great. I’m low traffic enough to be on the free tier, but would happily pay given how easy it’s been to build on.
infamia 3y ago
Agreed, this looks really cool. While there is no Django/DRF support at the moment, it does say that that they'll be increasing the number of packages in the future.
- brendanib 3y ago
  Would love feedback on which packages you'd like to see us support: https://github.com/cloudflare/workerd/discussions/categories/python-packages https://github.com/cloudflare/workerd/discussions/categories...
manishsharan 3y ago
>>Would love to port the whole Django app behind the site over, using their D1 database too. Is that wise? One DDOS attack could break your budget.
- piperswe 3y ago
  Only if the DDoS isn't blocked by the Cloudflare DDoS protection
  - spxneo 3y ago
    what are the advantages of using Cloudflare db over supabase? So far im loving supabase but wasn't aware CF products have increased drastically Rows read 5 million / day First 25 billion / month included + $0.001 / million rows Rows written 100,000 / day First 50 million / month included + $1.00 / million rows Storage (per GB stored) 5 GB (total) First 5 GB included + $0.75 / GB-mo

noman-land 3y ago

This is kind of a game changer for running AI stuff on Cloudflare. Been hoping for this for a while now.

dabber 3y ago
> This is kind of a game changer for running AI stuff on Cloudflare. That certainly appears to be the intention. > Been hoping for this for a while now. You should check out the other two announcements from today as well if you haven't yet: "Leveling up Workers AI: General Availability and more new capabilities" https://blog.cloudflare.com/workers-ai-ga-huggingface-loras-python-support https://blog.cloudflare.com/workers-ai-ga-huggingface-loras-... "Running fine-tuned models on Workers AI with LoRAs" https://blog.cloudflare.com/fine-tuned-inference-with-loras https://blog.cloudflare.com/fine-tuned-inference-with-loras
- CharlesW 3y ago
  Also, D1, Hyperdrive, and Workers Analytics Engine went GA. https://blog.cloudflare.com/making-full-stack-easier-d1-ga-hyperdrive-queues https://blog.cloudflare.com/making-full-stack-easier-d1-ga-h...
spxneo 3y ago
what is the maximum length a worker can run for? curious how this compares to AWS Lambda? or is it something completely different?
- brendanib 3y ago
  There's no hard limit on duration! 30 seconds of CPU time https://developers.cloudflare.com/workers/platform/limits/#duration https://developers.cloudflare.com/workers/platform/limits/#d...

jatins 3y ago

Clouflare has a lot of great stuff for hosting and databases but I think they haven't done a great job marketing themselves as developer platform which has lead to platforms like Vercel, Netlify taking significant mindshare. Tangential: does Cloudflare provide container hosting service agnostic of language -- something like Google Cloud Run?

thangngoc89 3y ago
> does Cloudflare provide container hosting service agnostic of language -- something like Google Cloud Run? Nope. Their Workers are V8 based so JS or Wasm
deleted 3y ago
[deleted]
Difwif 3y ago
I agree something is wrong with their marketing. I was also initially drawn to Vercel and Netlify but after extended use and not being happy with either I eventually tried Cloudflare and discovered I love it. The pricing and the product is fantastic.
- spxneo 3y ago
  Same experience its fascinating how much marketing impacts developer minds.
- nstart 3y ago
  I think it’s because the experience of familiarizing oneself with the platform and getting to a hello world level crud app/basic static site is done a lot better with vercel and netlify than it is with cloudflare. Cloudflares site and docs are not built with the approach of getting an app from 0 to 1 ASAP.
  - rvnx 3y ago
    Pricing also. 0.40 USD / month for a globally deployed site @ CF versus 199 USD / month for a static site with limited traffic usage @ the supposedly premium other hosts

pelletier 3y ago

I'm curious to see how the limitation of using pyodide packages only will play out for non-trivial builds. Thinking of all the non-pure python code out there that need to be manually rebuilt to support a non-trivial production app. Maybe Cloudflare's adoption will help bring more packages into the fold, and if it's an 80/20 rule here, would be good enough.

hoodchatham 3y ago
I certainly think there's an 80/20 rule here. Most packages are not very hard to port, and generally the ones that are hard to build use features like threads and multiprocessing, graphics cards, raw sockets, green threads, or other capabilities that have no obvious analogue in a webassembly runtime. As we mention in the blog post, the biggest issues are around supporting server and request packages since they are clearly useful in cloudflare workers but are difficult to port because they frequently use raw sockets and some form of concurrency.
- dom96 3y ago
  As we build out support for some of these features in the Workers Runtime, we should be able to port Python modules to use them. Some features like raw sockets are already available, so we should be able to make some quick headway here. (Myself and Hood above are the folks who implemented Python Workers)

neonsunset 3y ago

I with they added Azure Functions style workers using C# too, or AWS style lambdas using NativeAOT. Way lower runtime overhead and time to first response latency. But C# is an underdog language in those lands, so it's understandable.

mdasen 3y ago
At the moment, it seems like they're concentrating on using V8 isolates. In the article, there's a good diagram of why: an isolate is able to share so much between different applications. Even with NativeAOT, you're still launching an entire program that has to load everything into memory and execute. In some ways, they're using V8 isolates the way that mod_php was used back in the day. One reason PHP became so dominant was because PHP was cheap and easy to deploy for small websites. Because the PHP runtime contained 90% of what a person wanted to do with PHP, your PHP code might be a small amount of code that mostly just called standard library functions like `mysql_query()`. If you were running a shared hosting service, you could have huge numbers of people running on the same box because every PHP script would be sharing a single instance of the PHP standard library - and that standard library was fast and written in C. If you wanted to offer Python hosting, each Python app would be duplicating the standard library they were using in memory and also needing lots of web packages that aren't part of the standard library (like a database package). So a minimal Python application was using tons more RAM because it wasn't sharing most of the code with everyone else on the box. Even with NativeAOT, you're still duplicating a lot when running many different C# projects - as is the case with Go, Java, Ruby, etc. V8 isolates are this case where they tend to be lighter weight because so much can be shared between different users in the system. In fact, the reason they're supporting Python is because Pyodide (Python interpreter in WASM) allows for dynamic linking. It means they can have a single Pyodide interpreter in memory that's shared by all the Python workers on the same box. Likewise, they can also share Python libraries that two different people on the same box might be using. They note that most languages that target WASM don't support dynamic linking and that the only way they can provide Cloudflare Workers at the price point they offer is because those Workers can share so much rather than duplicating and using more memory for each user. If you really want C# on Workers, C# does support WASM.

ssijak 3y ago

Can we just get full node runtime? Cloudflare is amazing, but without a full node runtime, we (and most of the usual apps) can't switch from things like Vercel/Netlify to Cloudflare.

kentonv 3y ago
The unique architecture of our runtime is what enables most of our competitive advantages. It's what lets us run your application in hundreds of locations around the world while also charging less than competing serverless platforms. If we used a full Node runtime, we would need to charge a lot more money, or only run your app in a couple central locations, or both. So, no, we can't just offer full Node. However, we are always expanding our Node API compatibility: https://developers.cloudflare.com/workers/runtime-apis/nodejs/ https://developers.cloudflare.com/workers/runtime-apis/nodej... (I'm the tech lead for Cloudflare Workers.)
- switch007 3y ago
  Your comment would have been great without: > So, no, we can't just offer full Node. (Sounds a bit snotty)
  - cstrahan 3y ago
    How else would you write that? Would this be better? "Thus, no, we can't just offer full Node." Is it the use of "so" that is off limits? Or is simply providing a concise conclusion inappropriate? I don't see what part of that quote is "snotty".
    - switch007 3y ago
      Omitting it No PR person would have included that line
- ssijak 3y ago
  I understand what you said. I did not expect full node runtime workers with all the other benefits that you listed that current workers have (global distribution, no cold starts, cost..). But it would be great to have a choice. I feel it would benefit both users and Cloudflare to support both (with different tradeoffs). For example, selecting "I want node runtime for this app workers," I would need to select the region they will run in, and that's it, those functions/workers would not be globally distributed, would probably cost more, but when you need full node runtime you need it and that is fine.

zinclozenge 3y ago

I'd be curious to see a direct performance comparison between their python and JS workers. Based on my own experience with pyodide, I'd wager there might be up to a 2x performance penalty.

paddy_m 3y ago

Glad to see it includes numpy (and presumably pandas). Getting those to work in constrained serverless environments can be a huge pain.

syrusakbary 3y ago

This is awesome, I'm happy that Cloudflare is adding more attention into running Python via WebAssembly at the Edge. I'll try to summarize on how they got it running and what are the drawbacks that they have from their current approach (note: I have deep context on running Python with WebAssembly at the Edge as part of my work in Wasmer). Cloudflare Workers are enabling Python at the Edge by using Pyodide [1] (Python compiled to WebAssembly via Emscripten). They bundled Pyodide into Workerd [2], and then use V8 snapshots [3] to try to accelerate startup times. On their best case, cold starts of Python in Cloudflare Workers are about 1 second. While this release is great as it allows them to measure the interest of running Python at the Edge, it has some drawbacks. So, what are those? * Being tied to use only one version of Python/Pyodide (the one that Workerd embeds) * Package resolution is quite hacky and tied to workerd. Only precompiled "native packages" will be allowed to be used at runtime (eg. using a specific version of numpy will turn to be challenging) * Architecturally tied to the JS/v8 world, which may show some challenges as they aim to reduce cold start times (in my opinion, it will be quite hard for them to achieve <100ms startup time with their current architecture). In any case, I welcome this initiative with my open hands and look forward all the cool apps that people will now build with this! [1] https://pyodide.org/ https://pyodide.org/ [2] https://github.com/cloudflare/workerd/blob/main/docs/pyodide.md https://github.com/cloudflare/workerd/blob/main/docs/pyodide... [3] https://github.com/cloudflare/workerd/pull/1875 https://github.com/cloudflare/workerd/pull/1875 Edit: updated wording from "proof of concept" to "release" to reflect the clarification from the Cloudflare team

panqueca 3y ago
Does this architecture supports uvloop?
- syrusakbary 3y ago
  As far as I know uvloop is not supported in Pyodide, mainly because it requires compiling libuv into WebAssembly (which is possible but not trivial). In any case, it shall be possible to run uvloop fully inside of WebAssembly. However, doing so will prove challenging using their current architecture
- 3y ago

alfor 3y ago

I don't see how people will start using a completely new way python is running. If you are just experimenting and having fun, sure. But would you bet your company or many many months of developpement on this? What happen if you get random bugs? The advantage need to be extremely high to make it worth it. Maybe for specialized work that need to happen at the edge and then, why not use js instead that is the bedrock of this implementation?

jppope 3y ago
Cloudflare's engineering is top notch. I wouldn't expect anything different than what you would get from any of the other major cloud providers. Workers are also extremely fast/performant and inexpensive... if you are working on a company those two aspects can be fairly important to the success of some types of companies.
victorbjorklund 3y ago
I can totally see it being used for small services where python has better libaries

pjmlp 3y ago

Why anyone would like to slow down their requests using a full interpred implementation is behind me. Don't be surprised by scalability issues.

atomicnumber3 3y ago
There's a lot of value in "just write a python thing in 5 minutes". People tend to hem and haw about performance and "doing it right." But it's often a misplaced argument of "python vs ["right" thing]", it's "python vs not having anything." Often, a shitty python thing is worth a ton and fixes the problem, and then if performance becomes enough of an issue you can evaluate whether you want to prioritize fixing it. And I find that once something that works is in place, and just quietly doing its job, people suddenly find it a lot less objectionable to their sensibilities. And even if we do replace it, having the python thing taking the heat off the "right" way's timetable lets you actually do it right because you can take your time.
- pjmlp 3y ago
  We all know what is the outcome of that temporary script that was written in 5 minutes.

harikb 3y ago

Is the choice of lzma to demonstrate isolation intentional or was it just a coincidence considering last week's tech news...[1] [1] https://news.ycombinator.com/item?id=39865810 https://news.ycombinator.com/item?id=39865810

garrettgu10 3y ago
Haha, we included it just because it's part of the standard library. Total coincidence in terms of timing but it's nice that using Wasm gives us isolation guarantees :-)
- dom96 3y ago
  Yeah, pure coincidence. I picked it before the xz news broke.

jarpineh 3y ago

More development and users for Pyodide is great news. Especially that better serverless story for Python server frameworks. I wonder if Jupyter can work in this stack? It is essentially JavaScript, but built for browser environment. Just the Python kernel might be worker compatible. It essentially has to do code evaluation which might a limitation as well. Should it work you could offload compute from browser or other HTTP clients to waster resources of worker environment. Direct access to databases would be better as well.

akshayka 3y ago
Python notebooks can definitely work in WASM: - https://jupyterlite.readthedocs.io/en/stable/ https://jupyterlite.readthedocs.io/en/stable/ - https://docs.marimo.io/guides/wasm.html https://docs.marimo.io/guides/wasm.html Running just the kernel in a Cloudflare Worker is an interesting idea ...
- jarpineh 3y ago
  Yes, Jupyter could work. I just fear there is things that except browser’s JavaScript APIs. Some I think aren’t even available for web workers. V8 workers are a different thing (I believe) and not at all familiar to me. But, it should be easy enough to test… https://github.com/jupyterlite/jupyterlite?tab=readme-ov-file#browser-based-interactive-computing https://github.com/jupyterlite/jupyterlite?tab=readme-ov-fil... <- Currently kernel supports at least web workers.

rodolphoarruda 3y ago

Interesting. HTMx -> Python -> SQlite all in Cloudflare. I was kind of waiting for this day.

hdlothia 3y ago

Wow this is huge for llm and data engineering. Many of the best libraries are in Python

adam_arthur 3y ago

I would like to see CloudFlare implement workers with WASM as the first class citizen, and a general purpose API not tied to JS workers. Up until now you've been able to deploy WASM code (e.g. effectively can use any language), but it runs within a JS context, rather than natively. Just a bit more overhead/awkwardness in deployment. I believe eventually all services will be deployed directly to WASM (securitized) runtimes, rather than via containers, similar to how we moved from images -> containers). There's very little benefit currently to trying to use something like Rust on the edge (in CF), because a lot of the perf advantage is negated by the overhead and startup times. e.g. https://github.com/WasmEdge/WasmEdge https://github.com/WasmEdge/WasmEdge

devwastaken 3y ago

It compiles python to we assembly which then runs on their modified V8 runtime. We assembly is generally a non solution to any problem, especially not this one. While it is convenient it is a clear lack of engineering ability that they can't implement a proper Python runtime. The reasons to not use wasm are many - the tool chains for emscripten are not well documented, hacky, and we're not built to the quality you'd expect from a compiler. After all it's doing something nothing was designed for. The performance will never be an improvement over a native engine, much of the context is lost in translation when compiling to wasm.

Terretta 3y ago

"This is about more than just making bindings to resources on Cloudflare more Pythonic though — it’s about compatibility with the ecosystem." Someone might be getting editorial help from GPT-4. // Or a human might be getting fine-tuned interacting with LLMs, which I've noticed happening to me.

ggu 3y ago
[dead]
Terretta 3y ago
To the dead comment beside me, asking why... GPT-4 has a variety of tells. One of them is: "This is not just about good thing, it's about another good thing." That "it's not just A, it's also B", or "it's more than A, it's B too" show up most any time you ask it for "persuasive" copy, like marketing, sales, or a rewrite of anything to make the reader a buyer. It's disproportionately common in GPT-4 copywriting or copy-editing, relative to human copy. Similar to seeing the word "Overall, ..." for a concluding paragraph, another tell.
- corinroyal 3y ago
  My personal favorite is, "It's important to note..." I asked it to stop using that phrase or variations and that lasted one prompt. I'm tempted to put the phrase on a T-shirt.

pbamotra 3y ago

Interesting - supports FastAPI and Langchain too https://developers.cloudflare.com/workers/languages/python/packages/ https://developers.cloudflare.com/workers/languages/python/p...

anon373839 3y ago

With Pyodide getting some serious backing, is there a glimmer of hope that we could end up with Python as a real alternative to JavaScript in the frontend?

deleted 3y ago
[deleted]

fastball 3y ago

Tried this out today and it was great, was very quick to get up and running! One question though – does anyone know how I can get my local dev environment to understand the libraries that are built-in to CFW's Python implementation? e.g. there is an `asgi` library that I do not want my linter to flag as unknown, but as it only exists at runtime in the `on_fetch` handler (and isn't actually present in my local dev machine, I couldn't figure this out.

iamcreasy 3y ago

As opposed to what the article says, urllib3 now has experimental support for browser as of January this year. Source: https://github.com/urllib3/urllib3/releases/tag/2.2.0 https://github.com/urllib3/urllib3/releases/tag/2.2.0