10 ms·

Zeroperl: Sandboxing Perl with WebAssembly

68 points by ccakes 2y ago

Gunax 2y ago
I do not understand this at all, but it looks cool.
adolph 2y ago
Subhead is "Sandboxing Perl with WebAssembly - Part 2." The subhead sounds weird, but part 1 makes more sense and is pretty interesting. Perl has many modules to deal with file formats nobody has used since Perl's prime. It isn't totally clear to me if the goal is to compile the Perl interpreter into WASM or interpreter + modules. In any either case the goal is to re-use the original tools within new tooling. I’m building a new startup and file metadata plays an important role. There are thousands of file formats, each format may have dozens of versions, and each stores metadata differently. Our use-case also needs metadata to be present when a file is uploaded - extracting the data on our servers means we add considerable overhead to upload post-processing & we lose data that is useful to customers. So we need to extract metadata client-side and staple it to the upload. Herein begins a journey of self-inflicted pain and suffering. ExifTool is written in Perl. https://andrews.substack.com/p/zeroperl-sandboxed-perl-with-webassembly https://andrews.substack.com/p/zeroperl-sandboxed-perl-with-...
- jasonthorsness 2y ago
  Is there anything else in the same class as ExifTool - super valuable but the only implementation is Perl?
  - tyingq 2y ago
    Not sure if you would say these are the still the only implementations, but autoconf, Bugzilla and SpamAssassin were all at least once thought of that way.
Animats 2y ago
setjmp/longjmp in 2025? That's kind of retro.
- qxdtz 2y ago
  Are you suggesting there is an issue with it?
- mananaysiempre 2y ago
  PUC Lua uses setjmp/longjmp and it’s still perfectly fine.
- AdieuToLogic 2y ago
  > setjmp/longjmp in 2025? That's kind of retro. A thrown exception handled with a try/catch block any other name... What's in a name? That which we call a rose By any other name would smell as sweet;[0] 0 - https://en.wikipedia.org/wiki/A_rose_by_any_other_name_would_smell_as_sweet https://en.wikipedia.org/wiki/A_rose_by_any_other_name_would...
- theamk 2y ago
  it's perl, of course it's retro.
hobs 2y ago
Honestly it seems like porting ExifTool would actually be easier than this nightmare.
- jauntywundrkind 2y ago
  This is a degree of thinking that feels rank common in the world. When you read a complex blog post with sophisticated approach, there's often a "wouldn't it be easy to just ____." Maybe, yes, perhaps! But sometimes the problem at hand - the proboem specified in the post - isnt the entire desire or objective. "Actually be easier" abounds, but sometimes our scope & intent in the long run builds on the problems at hand. Porting ExifTool may be easier (but then if you want to maintain it, that's a drain for life). But having perl that you can now run anywhere might be something this author sees other use for. Getting good at wasm or exhibiting their excellent systems skills might have been side objectives. Whether we just do things to get to get results at hand, or whether we invest ourselves broadly to build a better world is a constant struggle for many engineers. This shows up in the comments time and time again as "would actually be easier". I'm sorry for hitting hard on this specific comment, but there's a Two Cultures problem here, and one culture trivializes the other endlessly in the comments. It's hard for me to state why trying hard & caring & doing extra matters, but I think that breed of people are the ones that I look up to, that make all the difference to me. And I really wish there was a good defense or rallying cry, something we could say when we get the inevitable "would actually be easier" that can capture the enthusiasm for setting ourselves up & building broadly towards our better worlds.
  - hobs 2y ago
    If it makes you feel good go for it, seems like a huge PITA for me to figure out what files do, but sometimes the journey is a lot more fun than the destination.
    - jauntywundrkind 2y ago
      I see it less as the fun of the journey, and more as the chance to become better, and make better tools for yourself going forwards. Advancing your position is more tempting than just doing the task. It's not always apparent how the side-quests will help us. But man, I can think of a half-dozens of upsides of the work here. You now have a portal Perl you can use anywhere. You don't have to maintain & keep a fork of ExifTool up to date. You've learned a ton about wasm file system interactions which could be useful for any portable systems. There's so many incidentals, that people just don't see when they blaze through to "but it could be easier."
benatkin 2y ago
That's really impressive. Like the author I am aware of the advantages/drawbacks of emscripten and wasi-sdk. The author did well to look extensively into both of them. Here's the repo. https://github.com/uswriting/zeroperl https://github.com/uswriting/zeroperl As increasingly is the case, a good starting point is the CI workflow: https://github.com/uswriting/zeroperl/blob/main/.github/workflows/zeroperl.yml https://github.com/uswriting/zeroperl/blob/main/.github/work...
ecmm 2y ago
Not sure if it is useful for your use-case, but at Wasmer we’ve been spending some time adding support for the eh proposal: https://github.com/wasmerio/wasmer/pull/5383 https://github.com/wasmerio/wasmer/pull/5383
- andrew_rfc 2y ago
  Would this require using the WASIX libc? I didn’t spend much time on it, but when I attempted to use it the build started failing in mysterious ways - happy to open an issue on the relevant repo with a reproduction / notes on what I think our some bugs.
- aghilmort 2y ago
  very cool on wasmer; does it support python + local ai / gpu use cases, ffmpeg, etc. have some customer use cases & really don’t want to go docker etc? helpful if can be included in electron & browser extensions or other supercontainers
dang 2y ago
Since this is Part 1, we merged the comments from the Part 2 thread hither: Get in loser. We're rewinding the stack - https://news.ycombinator.com/item?id=43014070 https://news.ycombinator.com/item?id=43014070 Readers may want to look at both articles of course!
ncruces 2y ago
Oh I'm so interested in this. I've wanted to use wazero to run my Exiftool [1] for quite a while. Just as I use wazero to sandbox dcraw [2]. But WASI Perl never materialized. This may just be what I'm missing. [1]: https://github.com/ncruces/go-exiftool https://github.com/ncruces/go-exiftool [2]: https://pkg.go.dev/github.com/ncruces/rethinkraw@v0.10.7/pkg/dcraw https://pkg.go.dev/github.com/ncruces/rethinkraw@v0.10.7/pkg...
mdaniel 2y ago
The ongoing ... lively ... discussion about "wasm all the things" may interest this audience, too: https://news.ycombinator.com/item?id=43020684 https://news.ycombinator.com/item?id=43020684
rurban 2y ago
"percc" just does not work? He obviously he meant my perlcc, it is maintained for compatible perl versions and it works and is used in industry.
- andrew_rfc 2y ago
  You cannot today build anything on the Blead of Perl with perlcc. Support has languished because the language changed fast enough that it never kept up. It may be used in the industry, but for a modern Perl application (or anything past 5.10) it won’t work. Hence why it was removed.
rurban 2y ago
This guy is just crazy. https://github.com/libexif/libexif https://github.com/libexif/libexif exists and is much easier to use than compiling perl to wasm, just to run the overly slow exiftool.
- transpute 2y ago
  There are security benefits to WASM sandboxing of client-side parsing of untrusted image files, to avoid scenarios like https://imagetragick.com https://imagetragick.com
- andrew_rfc 2y ago
  I’m not sure where you got the idea it’s slow, but it executes within 30% of native speed. Which means extracting the metadata from a 30 GB pro video in my test takes 300-400ms. Speaking of - libexif doesn’t support a lot of file formats, whereas ExifTool does. If I always opt to choose the easiest path then I’m setting a precedent for myself that I will compromise on all my goals for half finished solutions that take me further from initial idea.

68 points by ccakes 2y ago

I do not understand this at all, but it looks cool.

Subhead is "Sandboxing Perl with WebAssembly - Part 2." The subhead sounds weird, but part 1 makes more sense and is pretty interesting. Perl has many modules to deal with file formats nobody has used since Perl's prime. It isn't totally clear to me if the goal is to compile the Perl interpreter into WASM or interpreter + modules. In any either case the goal is to re-use the original tools within new tooling. I’m building a new startup and file metadata plays an important role. There are thousands of file formats, each format may have dozens of versions, and each stores metadata differently. Our use-case also needs metadata to be present when a file is uploaded - extracting the data on our servers means we add considerable overhead to upload post-processing & we lose data that is useful to customers. So we need to extract metadata client-side and staple it to the upload. Herein begins a journey of self-inflicted pain and suffering. ExifTool is written in Perl. https://andrews.substack.com/p/zeroperl-sandboxed-perl-with-webassembly https://andrews.substack.com/p/zeroperl-sandboxed-perl-with-...

jasonthorsness 2y ago
Is there anything else in the same class as ExifTool - super valuable but the only implementation is Perl?
- tyingq 2y ago
  Not sure if you would say these are the still the only implementations, but autoconf, Bugzilla and SpamAssassin were all at least once thought of that way.

Animats 2y ago

setjmp/longjmp in 2025? That's kind of retro.

qxdtz 2y ago
Are you suggesting there is an issue with it?
mananaysiempre 2y ago
PUC Lua uses setjmp/longjmp and it’s still perfectly fine.
AdieuToLogic 2y ago
> setjmp/longjmp in 2025? That's kind of retro. A thrown exception handled with a try/catch block any other name... What's in a name? That which we call a rose By any other name would smell as sweet;[0] 0 - https://en.wikipedia.org/wiki/A_rose_by_any_other_name_would_smell_as_sweet https://en.wikipedia.org/wiki/A_rose_by_any_other_name_would...
theamk 2y ago
it's perl, of course it's retro.

hobs 2y ago

Honestly it seems like porting ExifTool would actually be easier than this nightmare.

jauntywundrkind 2y ago
This is a degree of thinking that feels rank common in the world. When you read a complex blog post with sophisticated approach, there's often a "wouldn't it be easy to just ____." Maybe, yes, perhaps! But sometimes the problem at hand - the proboem specified in the post - isnt the entire desire or objective. "Actually be easier" abounds, but sometimes our scope & intent in the long run builds on the problems at hand. Porting ExifTool may be easier (but then if you want to maintain it, that's a drain for life). But having perl that you can now run anywhere might be something this author sees other use for. Getting good at wasm or exhibiting their excellent systems skills might have been side objectives. Whether we just do things to get to get results at hand, or whether we invest ourselves broadly to build a better world is a constant struggle for many engineers. This shows up in the comments time and time again as "would actually be easier". I'm sorry for hitting hard on this specific comment, but there's a Two Cultures problem here, and one culture trivializes the other endlessly in the comments. It's hard for me to state why trying hard & caring & doing extra matters, but I think that breed of people are the ones that I look up to, that make all the difference to me. And I really wish there was a good defense or rallying cry, something we could say when we get the inevitable "would actually be easier" that can capture the enthusiasm for setting ourselves up & building broadly towards our better worlds.
- hobs 2y ago
  If it makes you feel good go for it, seems like a huge PITA for me to figure out what files do, but sometimes the journey is a lot more fun than the destination.
  - jauntywundrkind 2y ago
    I see it less as the fun of the journey, and more as the chance to become better, and make better tools for yourself going forwards. Advancing your position is more tempting than just doing the task. It's not always apparent how the side-quests will help us. But man, I can think of a half-dozens of upsides of the work here. You now have a portal Perl you can use anywhere. You don't have to maintain & keep a fork of ExifTool up to date. You've learned a ton about wasm file system interactions which could be useful for any portable systems. There's so many incidentals, that people just don't see when they blaze through to "but it could be easier."

benatkin 2y ago

That's really impressive. Like the author I am aware of the advantages/drawbacks of emscripten and wasi-sdk. The author did well to look extensively into both of them. Here's the repo. https://github.com/uswriting/zeroperl https://github.com/uswriting/zeroperl As increasingly is the case, a good starting point is the CI workflow: https://github.com/uswriting/zeroperl/blob/main/.github/workflows/zeroperl.yml https://github.com/uswriting/zeroperl/blob/main/.github/work...

ecmm 2y ago

Not sure if it is useful for your use-case, but at Wasmer we’ve been spending some time adding support for the eh proposal: https://github.com/wasmerio/wasmer/pull/5383 https://github.com/wasmerio/wasmer/pull/5383

andrew_rfc 2y ago
Would this require using the WASIX libc? I didn’t spend much time on it, but when I attempted to use it the build started failing in mysterious ways - happy to open an issue on the relevant repo with a reproduction / notes on what I think our some bugs.
aghilmort 2y ago
very cool on wasmer; does it support python + local ai / gpu use cases, ffmpeg, etc. have some customer use cases & really don’t want to go docker etc? helpful if can be included in electron & browser extensions or other supercontainers

dang 2y ago

Since this is Part 1, we merged the comments from the Part 2 thread hither: Get in loser. We're rewinding the stack - https://news.ycombinator.com/item?id=43014070 https://news.ycombinator.com/item?id=43014070 Readers may want to look at both articles of course!

ncruces 2y ago

Oh I'm so interested in this. I've wanted to use wazero to run my Exiftool [1] for quite a while. Just as I use wazero to sandbox dcraw [2]. But WASI Perl never materialized. This may just be what I'm missing. [1]: https://github.com/ncruces/go-exiftool https://github.com/ncruces/go-exiftool [2]: https://pkg.go.dev/github.com/ncruces/rethinkraw@v0.10.7/pkg/dcraw https://pkg.go.dev/github.com/ncruces/rethinkraw@v0.10.7/pkg...

mdaniel 2y ago

The ongoing ... lively ... discussion about "wasm all the things" may interest this audience, too: https://news.ycombinator.com/item?id=43020684 https://news.ycombinator.com/item?id=43020684

rurban 2y ago

"percc" just does not work? He obviously he meant my perlcc, it is maintained for compatible perl versions and it works and is used in industry.

andrew_rfc 2y ago
You cannot today build anything on the Blead of Perl with perlcc. Support has languished because the language changed fast enough that it never kept up. It may be used in the industry, but for a modern Perl application (or anything past 5.10) it won’t work. Hence why it was removed.

rurban 2y ago

This guy is just crazy. https://github.com/libexif/libexif https://github.com/libexif/libexif exists and is much easier to use than compiling perl to wasm, just to run the overly slow exiftool.

transpute 2y ago
There are security benefits to WASM sandboxing of client-side parsing of untrusted image files, to avoid scenarios like https://imagetragick.com https://imagetragick.com
andrew_rfc 2y ago
I’m not sure where you got the idea it’s slow, but it executes within 30% of native speed. Which means extracting the metadata from a 30 GB pro video in my test takes 300-400ms. Speaking of - libexif doesn’t support a lot of file formats, whereas ExifTool does. If I always opt to choose the easiest path then I’m setting a precedent for myself that I will compromise on all my goals for half finished solutions that take me further from initial idea.