11 ms·
Python 3.11 in the Web Browser
- spicybright 5y ago
- xapata 5y agoYou could have asked for benchmarks without saying, "Yaaawn." I think the author would be more likely to volunteer to improve the code if it were encouraged rather than yawned at.
- hoten 5y agoCan't see the comment anymore as it's been flagged, but in general some of the most useless feedback you can give to projects in their infancy are: what doesn't work or what doesn't work well enough. If it takes you 5 minutes come up with such feedback, hands down, the author knows it already, and well informed readers know it already. For example, no one familiar with porting things to WASM would think that a large project like Python would just work with no performance issues on a first release. It's the thing I look forward to the least about sharing stuff that I've worked on. I know all the issues, and sometimes you just want to show off a big endeavor before continuing with the nitty gritty of making better software. But you're gonna run into people who just want to poo-poo on your hard work.
- spicybright 5y agoIt wasn't that bad of a comment, but was pretty much exactly what you said. Was having a pretty bad day when I posted. Keep on working OP, I want to be able to pick python instead of JS someday.
- mcintyre1994 5y agoI’m a big fan of Python in WASM! It really reduces the friction of playing around with things. Something I find pretty useful that I hacked together using it is https://pyground.vercel.app/ https://pyground.vercel.app/ - I use it whenever I have a CSV or JSON file I want to do some poking around in using Python. You can just drag and drop a file in, it’ll be imported in as the ‘data’ variable, and then you can run Python code/do matplotlib visualisations without installing anything.
- wishawa 5y agoHave you seen Skulpt [1]? It doesn’t do matplotlib (or most other library) but it’s pure JS. [1]: https://skulpt.org/ https://skulpt.org/
- mcintyre1994 5y agoAh yep! I came across it a few years ago when I was chatting to the team at anvil.works, they were using it for their client side python. Looks really cool, really ambitious and impressive project.
- rufugee 5y agoAnvil is pretty cool...I spent a good bit of time experimenting with it. Unfortunately, it has some edges you hit quickly as an experienced developer which get in your way. However, for their target audience, which I infer to be skilled business folks who don't necessarily develop full time, it's awesome ;-)
- tomatowurst 5y agointeresting, is it possible to write some sort of frontend framework like solidjs with this? what would be the the performance like I wonder. it would be really great if we could have full python/pip support in the browser but with some vetting done (just realized we don't have such thing in pip, other than relying on pip lockfiles and pipenv)
- acbart 5y agoMatPlotLib support in Skulpt for BlockPy [1]. Not complete, but workable for a lot of common graphs that I use in my CS1 course. [1]: https://github.com/blockpy-edu/skulpt/blob/master/src/lib/matplotlib/pyplot/__init__.js https://github.com/blockpy-edu/skulpt/blob/master/src/lib/ma...
- irisman 5y agoDid you use tailwind css for that?
- nabakin 5y agoFound his work https://github.com/tiran/cpython/tree/wasi/Tools/wasm https://github.com/tiran/cpython/tree/wasi/Tools/wasm
- heavyset_go 5y agoI think it would be cool if browsers shipped with WASM-compiled interpreters and runtimes for other languages, like they do with JavaScript. That way you'd be able to use other languages in the browser without needing users to download 20MB of a WASM-compiled interpreter just to run 1KB of code.
- 999900000999 5y agoI'd love that too, if I was a better programer I'd likely fork Chromium and build in Python support.
- deleted 5y ago[deleted]
- klodolph 5y agoYou're still downloading the 20MB WASM-compiled interpreter, it's just that now you're redownloading it every time your browser updates.
- sieabahlpark 5y ago
- heavyset_go 5y agoYes, but that's the point, if the user has it installed by default then there won't be much of a penalty if a website chooses to use another language than JavaScript. Right now, WASM interpreters only really make sense for teaching and exposition with REPLs, where the user probably won't mind large downloads in order to do something out of the ordinary. Shipping interpreters would instead make that ordinary.
- klodolph 5y agoIt's still a big penalty, you're just changing who pays the penalty, when they pay the penalty, how often they pay the penalty. Personally, I'm pretty happy that we have WASM at all, and I think there's a lot of work we can do (over the next years) to make interpreters that work well in WASM.
- stavros 5y agoI find WASM in general very cool. Is there a way to run a WASM "binary" locally? For example, can I distribute a program as WASM and execute ./mywasmprogram and have it run? I imagine I need a WASM runtime installed, and to somehow get my shell to recognize that WASM programs should be loaded with it (for lack of a hashbang line), but is that actually doable?
- simonw 5y agoYes, definitely. Wasmer is one way to do that: https://docs.wasmer.io/ecosystem/wasmer/getting-started https://docs.wasmer.io/ecosystem/wasmer/getting-started
- stavros 5y agoExcellent, thank you! I knew of Wasmer but didn't realize that's what it was. I look forward to the day where we can just distribute one architecture-independent binary.
- wffurr 5y agoLike a .jar file?
- tormeh 5y agoYes, but with a much leaner, memory-efficient, and more secure runtime. WASM is kinda "JVM without the Java bits" but that's a good thing. Having the VM provide facilities for GC is not necessary. Go shows that you can embed it in the executable. And Java bytecode is too close to Java code and not close enough to machine code, and so the VM has to provide a full interpreter - also unnecessary.
- brnt 5y agoWasm requires a full JS runtime no? At this point those are no lighter than a full JVM.
- syrusakbary 5y agoI’m super excited about this being integrated upstream. Long ago we compiled Python 3.6 and published to WAPM: https://wapm.io/python/python https://wapm.io/python/python (you can run it online there!) I wonder if we could publish the new version also! I think we got the Python repl properly running so it would be interesting trying to have that working on the new version too
- pjmlp 5y agoThe irony is that in the old plug-in days ActiveState did had a Python plugin for the browser.
- jeremycarter 5y agoWe've gone full circle
- kibwen 5y agoExcept that unlike NPAPI plugins, WASM is sandboxed.
- pjmlp 5y agoIt does very little, when the sand can be tainted or be coerced to call external imports in the wrong order.
- zo1 5y agoSo we got rid of browser plugins like flash because they were "unsafe" and we "magically" couldn't "sandbox" them for a decade. Then somehow we come up with a new thing that is "sandboxed" (trust us, it is, because WASM!), so let's use that instead and recompile arbitrary binary plugins for that and run it in the browser (what could go wrong). Really, without all the details/nuance, it sounds like we shot ourselves in the foot, went down a wrong path because of it, and now we're full-circle back to where we started. Except now the web is drastically different and the browser is now our one and true only terminal to the Holy Server that is Google et al. /end of rant
- kibwen 5y agoThis is hyperbolic. We could sandbox browser plugins, and did, but this broke a ton of things than used plugins because they expected not to be run in a sandbox. Are you afraid of JavaScript in your browser? Perhaps you are, and that's fair, but WASM is no more dangerous than the JavaScript that everyone already runs.
- modeless 5y agoI really, really wish it were possible for wasm modules like this to be shared between sites. That is, python.org could host an official wasm module, and the first time you visited a site using python the browser would download and compile it once and cache the result, and then any other website could use python with no download size or load time penalty. But unfortunately privacy concerns killed this utopia. Caches are now partitioned per origin, so every site that uses python has to download and compile the wasm module separately even if it's the exact same module from the same place, and that is unlikely to ever change: https://developer.chrome.com/blog/http-cache-partitioning/ https://developer.chrome.com/blog/http-cache-partitioning/
- cgb223 5y agoCouldn’t browsers theoretically “bake in” the python.org binary to the browser itself to avoid this? Kind of like how macOS (used to) ship with python preinstalled?
- kumarvvr 5y agoYes. It absolutely can. And it would be even more efficient than using WASM. However, I guess adding a new language would introduce a lot of security issues and what not.
- toastal 5y agoIf browsers already have a multi-paradigm, dynamic/duck-typed, garbage-collected scripting language in JavaScript, why would they add another one--especially since JS is incredibly backwards-compatible to not break the web? In the broader scope of programming languages, Python and JS are the same thing. If their going to add a second language, the least they could do is make it interesting in comparison.
- nine_k 5y agoOn top of it, JS is significantly faster.
- teddyh 5y ago> there is one place that Python has not yet conquered: the browser Well, there was that one web browser written in Python, with built-in Python scripting support: Grail.
- doctoboggan 5y agoI did a quick speed comparison: import numpy as np import time n = time.time()*1000 r = np.random.rand(10000000)**2 print('done in', time.time()*1000-n, 'ms') In the browser I was getting around 200ms and on my computer I was getting around 65ms, so around 3 times slower. Also, if I try to allocate an array of length 100,000,000 then I get a `MemoryError` in the browser. Does anyone know any ways to get around these limitations?
- deleted 5y ago[deleted]
- ledauphin 5y agothat's... really not bad at all. Kind of amazing, in fact. Thanks for putting in the work.
- tomatowurst 5y agothis is coincidentally really freaking good. to be able to run numpy in a browser without having to rely on javascript port? and its only about 3 times slower? I can live with this for now and i think this will improve drastically in the future
- hoten 5y agoDepending on how the wasm was compiled, there are options to limit or prevent memory growth. But I think the total limit at the moment is 4gb because 32bit.
- jokoon 5y agoHow large is the wasm binary? I would guess it's only for core python, without modules. Brython was nice to use, although its dom syntax was a bit awkward.
- tomatowurst 5y agowhen can we expect a recording of this to be uploaded on youtube, are there any existing content out there that talks more about wasm and python? seems really important, especially for many researchers and data scientists who use python primarily to be able to cross over to the browser.
- da39a3ee 5y agoIs there any way to compile a subset of Python itself to Wasm? (As opposed to compiling its interpreter to Wasm.) Same question for Javascript.
- tyingq 5y agoWASM is a virtual machine, but it's missing support for some things that would make that practical. GC support, polymorphic inline cache, char/byte types, reference types, etc. Some of that is on the WASM roadmap.
- da39a3ee 5y agoThanks. I'd read that GC support might be on the way (and that that might make Go a bit more attractive for Wasm than it is currently). But I don't know anything about the other features you mention. Is it your understanding, or guess, that it is the "community's intention" to have (a subset of) JS or Python compiling to Wasm soonish and if so any guess as to how long?
- foldr 5y agoCPython uses reference counting plus cycle-breaking, though, so it's not like you'd have to bundle a full GC implementation with your Python code.
- kumarvvr 5y agoI can't wait for the browser to be released from the shackles of JS. Just imagine how awesome would it be to use the full power of Python to build a react like library. It is a source of great disappointment to me that WASM cannot directly control the DOM.
- rewq4321 5y agoWhy does it need to be direct? A bit of glue code doesn't hurt, especially since most of the other glue is going away with WASI.
- musingsole 5y agoThat glue code is a massive cognitive burden on a programmer. It's not a big deal when you already know it. But when you don't, a thin, glue layer of code following different paradigms than the other 90% of your project will crush your development agility anytime you have to touch that glue layer.
- laerus 5y agoWASM Interface types will fix that
- rewq4321 5y agoThose interested in this should check out Pyodide[0]. It basically "just works" so long as the libraries you import are pure Python or are part of the core scientific stack (the Pyodide authors have done the work to manually port all the C code behind numpy, scipy, etc.). What I really wish for is for ~all Python packages to work in the browser without manual porting of the underlying C/Rust/etc. being needed, since a lot of the interesting and useful libraries aren't pure Python, and manual porting is non-trivial. I'm not sure what the best route to that future is, but I'm guessing it'd probably help if Python had a wasm runtime in its standard library[1], since then authors of libraries that use C/Rust/etc. might make cross-platform builds (perhaps by default). Regarding this Pycon speech, it seems that it's related to the following entry in the 3.11 changelog[2], which the speaker was heavily involved with: > CPython now has experimental support for cross compiling to WebAssembly platform wasm32-emscripten. The effort is inspired by previous work like Pyodide. (Contributed by Christian Heimes and Ethan Smith in bpo-40280[3]) But maybe Christian has more to reveal here? In any case, I'm hugely appreciative of all the work that is being done to bring Python to the browser! [0] https://github.com/pyodide/pyodide https://github.com/pyodide/pyodide [1] https://discuss.python.org/t/add-a-webassembly-wasm-runtime/3957 https://discuss.python.org/t/add-a-webassembly-wasm-runtime/... [2] https://docs.python.org/3.11/whatsnew/3.11.html https://docs.python.org/3.11/whatsnew/3.11.html [3] https://bugs.python.org/issue40280 https://bugs.python.org/issue40280
- kaba0 5y agoI think the closest to that goal is GraalVM. It can run LLVM bit code for the C parts while can natively run Python. Since the whole thing is java and java byte code can be compiled to wasm/js by teavm it should work even know though it is definitely not streamlined yet.
- Too 5y agoIf you are ready for some inception, go watch David Baezley use Python to implement a complete WASM interpreter in under 1h. https://www.youtube.com/watch?v=VUT386_GKI8 https://www.youtube.com/watch?v=VUT386_GKI8 One of the best coding presentation i've seen. This should make it possible to run python in wasm in python.
- asiachick 5y agoI don't know what python's async support is like but the biggest issue with porting to the web is the browser mostly requires that you exit events before it will do anything. This is contrast to pretty much all other environments. That means there are tons of patterns and libraries that just don't work and require major re-writes or big AST manipulations like asyncify from emscripten. I'm not saying you can't do async work in any language. Rather I'm saying that likely the patterns you're used to using in some language that's not JavaScript won't work in the browser and you'll have to massively change your style.
- ksr 5y agoCan someone explain to me, really slowly please, how to take an existing Python codebase such as https://github.com/infojunkie/mma https://github.com/infojunkie/mma and run it in the browser using one of the technologies mentioned in this thread? Especially, how to deal with filesystem calls that expect some files to be present in various folders. Thanks!
- Skyy93 5y agoAccording to the documentation and discussion about this feature some unit-tests are currently simple skipped: virtual filestem by Emscripten that's not fully POSIX compliant, no processes, no sockets, async only via the browser event loop (source: https://bugs.python.org/issue40280 https://bugs.python.org/issue40280) This would mean to me they simply do not work or only to an extend. As you can reading the virtual filesystem seems not fully compatible.
- fortzi 5y agoI worry that we’ll now see more broken web apps, given the lack of type safety in Python. I know that JS doesn’t guarantee that either, but at least you can choose to work with TypeScript. True, there’s mypy and alike, but their hard to integrate into existing projects, and are also not perfect.
- weatherlite 5y agoPeople who want Python on the browser would have probably not chosen Typescript anyway, it's mostly not the same demographic. Not everyone cares about types.
- zo1 5y agoAt this point, as a Python dev, I wouldn't even bother. The web is dirty and unless you have huge community mindshare or oodles of funding (a-la Google), you're not going to get any meaningful traction with whatever python-based UI framework you create. The JS community (funded by big players like Google and Facebook and 20 million liters of starbucks hipster coffee) has figured out all the tooling, libraries, and practically taken over every facet of the web development ecosystem so it caters for JS, that it's going to be severely impractical and hopeless to jump in at this point.
- KolenCh 5y agoSorry, don’t quite follow, why would the lack of type safety in Python make this a problem, and how would Typescript help in this situation? This comes from Python dev with no idea how to make web app. Eg I’ve seen some package not maintained for years in PyPI while continue to work fine. Also I think Python is strongly typed while js isn’t. I would think that’s a bigger problem?
- derangedHorse 5y agoHaving static types in help programmers make less mistakes by allowing the compiler to perform stronger forms of static analysis. This prevents programmers from accidentally passing in arguments to a function that have types the function was not defined to handle (since the compiler/interpreter will error out). Also "strong typing" is not well-defined across the literature as acknowledged by this one link from Cornell (which lists strong typing as having "the type of every variable and every expression is a syntactic property" and variables that are "used only in ways that respect its type"): https://www.cs.cornell.edu/courses/cs1130/2012sp/1130selfpaced/module1/module1part4/strongtyping.html https://www.cs.cornell.edu/courses/cs1130/2012sp/1130selfpac... You also have lecturers at Carnegie Mellon teaching that strong typing means "Types must by explicitly converted" :https://www.cs.cmu.edu/~07131/f18/topics/extratations/langs.pdf https://www.cs.cmu.edu/~07131/f18/topics/extratations/langs.... Personally I think the historical definitions from Wikipedia are the most concise (it's paraphrased by me as saying "arguments to a function should have the same type as the parameters the function was defined with): https://en.wikipedia.org/wiki/Strong_and_weak_typing https://en.wikipedia.org/wiki/Strong_and_weak_typing In terms of 'strong typing' (with the definition of not being able to do any kinds of implicit type conversions), Python can be seen as weakly typed since there are forms of implicit type conversions like adding different numerical data types together. I would say it has less instances of type implicit type conversions than something like Javascript though, so if there was a metric of "strength" defined that's inversely correlated with the instances of implicit type conversions than I would say Python could be considered "stronger" (although a lot of Javascript quirkiness is removed through the introduction of static types). All that being said, the argument against Python seems to a gripe against implicit and dynamic typing whereas Typescript has neither and instead brings explicit and static typing (the argument doesn't seem web specific).
- jonititan 5y agoDoes the talk do a comparison to Pyodide? Seems very similar and I'd be very interested to hear what the advantages and disadvantages are.
- rubenfiszel 5y agoThis is huge and comes at the right moment for me. The Web Browser part is far less important to me than being able to compile programs into small and autonomous wasm binaries. I am indeed building an open-source (soon) business app platform based on code: https://windmill.dev https://windmill.dev Right now you can define any scripts and run them, but behind the scene, the way the workers work is that they have a static list of dependencies they can handle and always fork a python process to run the code in that environement. I was scratching my head about how to provide proper isolation and handling of custom dependencies for Python short of zipping the entire list of pip dependencies and unzipping it at runtime. For Typescript, this can be achieved easily using deno compile. Store the small output bundle and run it with the proper runtime restrictions. With the ability to do more or less the same Python, this is a huge game changer.