7 ms·

We Built a C++ Rendering Engine for the Web

108 points by PetrBrzyBrzek 5y ago

aliasEli 5y ago
I really don't think it was a good idea to write this in C++ for security reasons. It accepts external inputs in some complex formats, and they also have a web server version. This makes it highly vulnerable to all kind of attacks, such as buffer overflows.
- aliasEli 5y ago
  OK, that got down-voted. I thought the basic idea behind HN is that we can have some interesting discussions. Simply down-voting without adding a reply why you disagree with an opinion does not really help. I have been interested in security for a long time. The number of security vulnerabilities that have been caused by insecure memory management problems is really huge. Some people will probably claim that this is not a problem with modern C++ because it can remedy this problems. But this assumes that the programmers know all the possible pitfalls. With respect to security, the problem is that when there is only a single weakness in your system it might become a point of attack. With a language like C++ there are many possible weaknesses that simply do not exist in memory safe languages.
  - mrweasel 5y ago
    Well, you're not starting of to great by randomly claiming that it's a security issue that it's written in C++. It's not really productive. A lot of software is, for better or worse, written in C or C++, that's not going to change. For years people have been yelling: "It's broken because it's written in C/C++". That same "attacks" was made to promote Java 20 years ago. Sure, maybe they could have picked a memory safe language, but they didn't. Perhaps because they know C++ and doing the same project in a language they're just learning would result in a ton of other bugs. They even write that they hired a few brilliant C++ programmers, so chances are that they know how to safely handle memory in C++.
    - pjmlp 5y ago
      From my point of view, better C++ than C, however and not speaking from this project rather in general terms, adopting best practices for secure coding in C++ seems to still be an uphill battle, saying this as C++ aficionado. https://microblink.com/blog/be-wise-sanitize-keeping-your-c-code-free-from-bugs https://microblink.com/blog/be-wise-sanitize-keeping-your-c-...
  - csmpltn 5y ago
    > "Simply down-voting without adding a reply why you disagree with an opinion does not really help." This looks like a general criticism of using C++ which has nothing to do with the topic of this post. You're free to criticize, but this criticism alone brings absolutely nothing constructive to the conversation and only serves to incite more useless "have you considered writing this in Rust?" conversations. You're not even suggesting what you think they should've used instead. > "I have been interested in security for a long time" Here's a tip for you then: security is not an absolute, and things usually aren't as black or white as you might think. Take a moment to consider the fact that C++ is one of only a small handful of languages with which everything around you has been built for the last 30+ years. Do you know something all of those other engineers don't already know? Otherwise, humility goes a long way.
    - kevingadd 5y ago
      The article title literally has 'C++' in it.
      - jazzyjackson 5y ago
        And yet it is more specific.
  - secondcoming 5y ago
    You're not trying to have a discussion, your post is essentially 'why didn't they use Rust'. Sadly predictable. There is no-one on HN who doesn't know that C++ has historic memory management difficulties.
    - fortran77 5y ago
      I wonder if he works for "Fronk" [0] [0] https://news.ycombinator.com/item?id=27782604 https://news.ycombinator.com/item?id=27782604
  - rfrey 5y ago
    I disagree with the commenters who are saying your comment was just "C++ bad". The point I took from your comment is that this particular class of application, which is parsing many complex formats from external sources, several of which probably have extensive warts and edge cases, and rendering the output to a browser, is the type of application may be the most vulnerable to the downsides of C++ when it comes to security. I think it's a reasonable point, and a step or two above "just use Rust".
    - aliasEli 5y ago
      You write better than I do.
  - flohofwoe 5y ago
    According to the article, the C++ code is compiled via Emscripten (presumably to WASM, or maybe to asm.js), so it's running sandboxed either in the WASM or JS runtime. Any potential memory corruption caused by unsafe C++ code is contained within the sandbox (which is the whole point of JS and WASM really). The security implications are exactly the same as writing the code in any other language (incuding Javascript or Rust). If the sandbox is buggy, then a "safe" language wouldn't help either.
    - detaro 5y ago
      Just because the attack is contained inside the sandbox doesn't mean it can't do anything, so no, "it's in a sandbox" does not remove all risk automatically.
      - hoytech 5y ago
        Yes, exactly, otherwise buggy applications wouldn't be a big deal because we could run them on their own dedicated computers. Section 2.5 of this paper has a good discussion on this: https://cr.yp.to/qmail/qmailsec-20071101.pdf https://cr.yp.to/qmail/qmailsec-20071101.pdf
      - azakai 5y ago
        You're right that it doesn't remove all risk automatically. You can still corrupt data inside the sandbox. However, wasm has a very clear sandboxing boundary. The ability of an exploit to escape the sandbox is very small if you are careful there. IIUC the task here is a user that wants to parse their own files. For that, I think wasm's sandboxing (if used properly) is very useful. Especially since in this case it runs on the web and so we also have the browser's additional isolation (a sandboxed process). Memory safety is incredibly important, but there isn't a simple answer in the space of tradeoffs, at least not for tasks like this. (For things like running an executable on bare metal that parses arbitrary inputs, obviously things are very different!)
        pjmlp 5y ago
        A WASM module basically is like an OS process, from security point of view. So now think what might happen, when not used properly. Some form of bounds checking should have been part of the design, like memory tagging.
- nly 5y ago
  As a professional C++ programmer I feel we, as a group, are constantly under-estimating the complexity of tasks, and over-estimating what can get done with the standard library. C++ is a very powerful, unopinionated language, that gives you a lot of freedom to attack your problem domain the way you best see fit. If you're writing a networked application, don't use POSIX sockets, which have an API designed for C, go and find a higher level library. If you're parsing complex text formats, don't iterate over buffers with char*'s, go pick up PEGTL[0]. If you're working on graphs, or need to properly index in-memory data, go pick up Boost[1][2]. If you need a GUI, go pick up Qt. It's extremely common in C++, due to the lack of a universal package management solution, for people to try and "muddle through" and do shit themselves when it's far outside their core competency. At one of my last employers, the core product was parsing JSON with std::regex, simply because they couldn't be bothered to integrate a JSON library (which can be done header-only). [0] https://github.com/taocpp/PEGTL https://github.com/taocpp/PEGTL [1] https://www.boost.org/doc/libs/1_76_0/libs/graph/ https://www.boost.org/doc/libs/1_76_0/libs/graph/ [2] https://www.boost.org/doc/libs/1_76_0/libs/multi_index/doc/index.html https://www.boost.org/doc/libs/1_76_0/libs/multi_index/doc/i...
  - jstimpfle 5y ago
    What's so wrong with POSIX sockets? It's maybe not an elegant API, but not because "it's designed for C". It's problems are for example 1) it's doing too much abstraction (sockets). 2) you have to deal with some arcane data structures and ancient formats (endian conversions). But, it does its job well enough: allowing the user to send and receive packets from the network. If you're writing a networked application that works, chances are you either don't give a sh*t what API to use as long as it lets you send and receive packets (and thus you go with the relatively portable POSIX sockets (at least for Linux / WinSock2 on Windows)), or you use a lower-level API (probably proprietary) to reduce syscall overhead / get more control. If you're parsing text, chances are all you need is fread() to read in the next chunk from a file, and from this you'll build a "next_byte()" function and then a "next_token()" function on top. (I've done a lot of network code as well as parsing code, and the I/O API is among the least of my concerns). All these fancy bottom-up kitchen sink libraries implementing "proper abstractions" or whatever do not provide any value past being able to be combined to form barely working and un-fixable applications where you will pull your hair out when you actually need some control over what's happening. For something better, you'll need exactly this from external libraries: a clean programmatic (function call) interface that gives you control at a reasonable level of abstraction.
- dangerbird2 5y ago
  The C++ code is running on webassembly, which is no more at risk of buffer overflow attacks than vanilla javascript. Worst-case scenario, you could have memory leaks in the C++ code or in the wasm-javascript interface (since javascript doesn't support finalizers for webassembly objects). But this is a usability issue, not a security issue
  - zozbot234 5y ago
    > which is no more at risk of buffer overflow attacks than vanilla javascript. This is quite wrong, a Wasm program can overflow internal buffers due to a missing bounds check and access unrelated data as a result. See HEARTBLEED for a case where this created a very real vulnerability. The Wasm safe sandbox only protects the boundary with the rest of the system.
    - azakai 5y ago
      You're right that wasm programs can overflow internal buffers in linear memory, which can be dangerous. However, wasm does a lot more than only protect the boundary with the rest of the system, including * Safe call stack (opaque / managed by the VM, and so uncorruptible). * Safe control flow (no jumps to unexpected places). * Safe(r) indirect calls (only methods in the table can be called, and the signature is verified). However, wasm also lacks a few things, like the ability to write-protect static data (see "Everything Old is New Again: Binary Security of WebAssembly"). Future wasm proposals will hopefully address those things.
flakiness 5y ago
Parsing proprietary, undocumented formats and turn it a universal format, and render it? That sounds like a kind of an ambition which is never realized, but they did. And this kind of project tends to fail because it's filled with boring tasks with unreasonable corner cases. So congrats! And according to their page [1], they've been doing this for a decade. This is such an accomplishment. [1] https://www.notion.so/The-Story-of-Avocode-d96c279b270443f886eed883e87e0024 https://www.notion.so/The-Story-of-Avocode-d96c279b270443f88...
- justshowpost 5y ago
  > Parsing But why? I thought PSD was all-binary...
rizky05 5y ago
I thought opendesign was open source, but we need an API Key to access the tech. So, it is not open source right ?
bckr 5y ago
I love reading about projects like this. They are in a class of projects that are 100% slog and not very sexy but make things a lot better for a lot of people. Better outputs for more inputs in better time. Beautiful. Reminds me of Photopea... Oh, I wonder whether they are in competition? Probably not since Photopea seems to be more about image editing and Avocode is more about design and handoff.
gfxgirl 5y ago
"the notoriously opaque Photoshop format" what? it's well been documented for ever. 2nd hit on google https://www.adobe.com/devnet-apps/photoshop/fileformatashtml/ https://www.adobe.com/devnet-apps/photoshop/fileformatashtml...
- nayuki 5y ago
  It's also the one with the famous rant, "PSD is not my favourite file format": https://github.com/gco/xee/blob/7aec0d65f776fa59c58eb6cf163b59dd4f1de3bd/XeePhotoshopLoader.m#L108-L136 https://github.com/gco/xee/blob/7aec0d65f776fa59c58eb6cf163b...
- yrds96 5y ago
  Well, this doc is from 2019, a lot of things can be changed. As a linux user, there are a plenty software that can render it perfectly but cannot edit, and i really don't get it. As a web developer i have to use photopea to be able to extract assets from psd files. It's really strange that a web application can edit psd and a lot of powerful softwares like gimp and Krita have the exact problem with layers and masks with modern .psd files, so my guess its bcause photopea IT'S designed to be compatible with psd files. I suspect it's incompatible by design with others softwares and not because it's has a poor specification.
  - gfxgirl 5y ago
    you suspect wrong. I've written Photoshop readers since 1993. They all still work. Photoshop has never broken their format
  - arcatek 5y ago
    If I remember correctly, PSD files have an optional header (when files are saved in "Compatibility Mode", which I think is the default) in which is encoded the rasterized image. It's then "easy" to display the image, but very difficult to edit it, as you then need to implement the whole renderer.
- mhh__ 5y ago
  PDF is also well documented by that measure, i.e. the complexity of said specification does matter.
mwkaufma 5y ago
Isn't that just... A browser?
- ReactiveJelly 5y ago
  Title is misleading, it's a PSD renderer for web browser, not a web browser's renderer.

108 points by PetrBrzyBrzek 5y ago

I really don't think it was a good idea to write this in C++ for security reasons. It accepts external inputs in some complex formats, and they also have a web server version. This makes it highly vulnerable to all kind of attacks, such as buffer overflows.

aliasEli 5y ago
OK, that got down-voted. I thought the basic idea behind HN is that we can have some interesting discussions. Simply down-voting without adding a reply why you disagree with an opinion does not really help. I have been interested in security for a long time. The number of security vulnerabilities that have been caused by insecure memory management problems is really huge. Some people will probably claim that this is not a problem with modern C++ because it can remedy this problems. But this assumes that the programmers know all the possible pitfalls. With respect to security, the problem is that when there is only a single weakness in your system it might become a point of attack. With a language like C++ there are many possible weaknesses that simply do not exist in memory safe languages.
- mrweasel 5y ago
  Well, you're not starting of to great by randomly claiming that it's a security issue that it's written in C++. It's not really productive. A lot of software is, for better or worse, written in C or C++, that's not going to change. For years people have been yelling: "It's broken because it's written in C/C++". That same "attacks" was made to promote Java 20 years ago. Sure, maybe they could have picked a memory safe language, but they didn't. Perhaps because they know C++ and doing the same project in a language they're just learning would result in a ton of other bugs. They even write that they hired a few brilliant C++ programmers, so chances are that they know how to safely handle memory in C++.
  - pjmlp 5y ago
    From my point of view, better C++ than C, however and not speaking from this project rather in general terms, adopting best practices for secure coding in C++ seems to still be an uphill battle, saying this as C++ aficionado. https://microblink.com/blog/be-wise-sanitize-keeping-your-c-code-free-from-bugs https://microblink.com/blog/be-wise-sanitize-keeping-your-c-...
- csmpltn 5y ago
  > "Simply down-voting without adding a reply why you disagree with an opinion does not really help." This looks like a general criticism of using C++ which has nothing to do with the topic of this post. You're free to criticize, but this criticism alone brings absolutely nothing constructive to the conversation and only serves to incite more useless "have you considered writing this in Rust?" conversations. You're not even suggesting what you think they should've used instead. > "I have been interested in security for a long time" Here's a tip for you then: security is not an absolute, and things usually aren't as black or white as you might think. Take a moment to consider the fact that C++ is one of only a small handful of languages with which everything around you has been built for the last 30+ years. Do you know something all of those other engineers don't already know? Otherwise, humility goes a long way.
  - kevingadd 5y ago
    The article title literally has 'C++' in it.
    - jazzyjackson 5y ago
      And yet it is more specific.
- secondcoming 5y ago
  You're not trying to have a discussion, your post is essentially 'why didn't they use Rust'. Sadly predictable. There is no-one on HN who doesn't know that C++ has historic memory management difficulties.
  - fortran77 5y ago
    I wonder if he works for "Fronk" [0] [0] https://news.ycombinator.com/item?id=27782604 https://news.ycombinator.com/item?id=27782604
- rfrey 5y ago
  I disagree with the commenters who are saying your comment was just "C++ bad". The point I took from your comment is that this particular class of application, which is parsing many complex formats from external sources, several of which probably have extensive warts and edge cases, and rendering the output to a browser, is the type of application may be the most vulnerable to the downsides of C++ when it comes to security. I think it's a reasonable point, and a step or two above "just use Rust".
  - aliasEli 5y ago
    You write better than I do.
- flohofwoe 5y ago
  According to the article, the C++ code is compiled via Emscripten (presumably to WASM, or maybe to asm.js), so it's running sandboxed either in the WASM or JS runtime. Any potential memory corruption caused by unsafe C++ code is contained within the sandbox (which is the whole point of JS and WASM really). The security implications are exactly the same as writing the code in any other language (incuding Javascript or Rust). If the sandbox is buggy, then a "safe" language wouldn't help either.
  - detaro 5y ago
    Just because the attack is contained inside the sandbox doesn't mean it can't do anything, so no, "it's in a sandbox" does not remove all risk automatically.
    - hoytech 5y ago
      Yes, exactly, otherwise buggy applications wouldn't be a big deal because we could run them on their own dedicated computers. Section 2.5 of this paper has a good discussion on this: https://cr.yp.to/qmail/qmailsec-20071101.pdf https://cr.yp.to/qmail/qmailsec-20071101.pdf
    - azakai 5y ago
      You're right that it doesn't remove all risk automatically. You can still corrupt data inside the sandbox. However, wasm has a very clear sandboxing boundary. The ability of an exploit to escape the sandbox is very small if you are careful there. IIUC the task here is a user that wants to parse their own files. For that, I think wasm's sandboxing (if used properly) is very useful. Especially since in this case it runs on the web and so we also have the browser's additional isolation (a sandboxed process). Memory safety is incredibly important, but there isn't a simple answer in the space of tradeoffs, at least not for tasks like this. (For things like running an executable on bare metal that parses arbitrary inputs, obviously things are very different!)
      - pjmlp 5y ago
        A WASM module basically is like an OS process, from security point of view. So now think what might happen, when not used properly. Some form of bounds checking should have been part of the design, like memory tagging.
nly 5y ago
As a professional C++ programmer I feel we, as a group, are constantly under-estimating the complexity of tasks, and over-estimating what can get done with the standard library. C++ is a very powerful, unopinionated language, that gives you a lot of freedom to attack your problem domain the way you best see fit. If you're writing a networked application, don't use POSIX sockets, which have an API designed for C, go and find a higher level library. If you're parsing complex text formats, don't iterate over buffers with char*'s, go pick up PEGTL[0]. If you're working on graphs, or need to properly index in-memory data, go pick up Boost[1][2]. If you need a GUI, go pick up Qt. It's extremely common in C++, due to the lack of a universal package management solution, for people to try and "muddle through" and do shit themselves when it's far outside their core competency. At one of my last employers, the core product was parsing JSON with std::regex, simply because they couldn't be bothered to integrate a JSON library (which can be done header-only). [0] https://github.com/taocpp/PEGTL https://github.com/taocpp/PEGTL [1] https://www.boost.org/doc/libs/1_76_0/libs/graph/ https://www.boost.org/doc/libs/1_76_0/libs/graph/ [2] https://www.boost.org/doc/libs/1_76_0/libs/multi_index/doc/index.html https://www.boost.org/doc/libs/1_76_0/libs/multi_index/doc/i...
- jstimpfle 5y ago
  What's so wrong with POSIX sockets? It's maybe not an elegant API, but not because "it's designed for C". It's problems are for example 1) it's doing too much abstraction (sockets). 2) you have to deal with some arcane data structures and ancient formats (endian conversions). But, it does its job well enough: allowing the user to send and receive packets from the network. If you're writing a networked application that works, chances are you either don't give a sh*t what API to use as long as it lets you send and receive packets (and thus you go with the relatively portable POSIX sockets (at least for Linux / WinSock2 on Windows)), or you use a lower-level API (probably proprietary) to reduce syscall overhead / get more control. If you're parsing text, chances are all you need is fread() to read in the next chunk from a file, and from this you'll build a "next_byte()" function and then a "next_token()" function on top. (I've done a lot of network code as well as parsing code, and the I/O API is among the least of my concerns). All these fancy bottom-up kitchen sink libraries implementing "proper abstractions" or whatever do not provide any value past being able to be combined to form barely working and un-fixable applications where you will pull your hair out when you actually need some control over what's happening. For something better, you'll need exactly this from external libraries: a clean programmatic (function call) interface that gives you control at a reasonable level of abstraction.
dangerbird2 5y ago
The C++ code is running on webassembly, which is no more at risk of buffer overflow attacks than vanilla javascript. Worst-case scenario, you could have memory leaks in the C++ code or in the wasm-javascript interface (since javascript doesn't support finalizers for webassembly objects). But this is a usability issue, not a security issue
- zozbot234 5y ago
  > which is no more at risk of buffer overflow attacks than vanilla javascript. This is quite wrong, a Wasm program can overflow internal buffers due to a missing bounds check and access unrelated data as a result. See HEARTBLEED for a case where this created a very real vulnerability. The Wasm safe sandbox only protects the boundary with the rest of the system.
  - azakai 5y ago
    You're right that wasm programs can overflow internal buffers in linear memory, which can be dangerous. However, wasm does a lot more than only protect the boundary with the rest of the system, including * Safe call stack (opaque / managed by the VM, and so uncorruptible). * Safe control flow (no jumps to unexpected places). * Safe(r) indirect calls (only methods in the table can be called, and the signature is verified). However, wasm also lacks a few things, like the ability to write-protect static data (see "Everything Old is New Again: Binary Security of WebAssembly"). Future wasm proposals will hopefully address those things.

flakiness 5y ago

Parsing proprietary, undocumented formats and turn it a universal format, and render it? That sounds like a kind of an ambition which is never realized, but they did. And this kind of project tends to fail because it's filled with boring tasks with unreasonable corner cases. So congrats! And according to their page [1], they've been doing this for a decade. This is such an accomplishment. [1] https://www.notion.so/The-Story-of-Avocode-d96c279b270443f886eed883e87e0024 https://www.notion.so/The-Story-of-Avocode-d96c279b270443f88...

justshowpost 5y ago
> Parsing But why? I thought PSD was all-binary...

rizky05 5y ago

I thought opendesign was open source, but we need an API Key to access the tech. So, it is not open source right ?

bckr 5y ago

I love reading about projects like this. They are in a class of projects that are 100% slog and not very sexy but make things a lot better for a lot of people. Better outputs for more inputs in better time. Beautiful. Reminds me of Photopea... Oh, I wonder whether they are in competition? Probably not since Photopea seems to be more about image editing and Avocode is more about design and handoff.

gfxgirl 5y ago

"the notoriously opaque Photoshop format" what? it's well been documented for ever. 2nd hit on google https://www.adobe.com/devnet-apps/photoshop/fileformatashtml/ https://www.adobe.com/devnet-apps/photoshop/fileformatashtml...

nayuki 5y ago
It's also the one with the famous rant, "PSD is not my favourite file format": https://github.com/gco/xee/blob/7aec0d65f776fa59c58eb6cf163b59dd4f1de3bd/XeePhotoshopLoader.m#L108-L136 https://github.com/gco/xee/blob/7aec0d65f776fa59c58eb6cf163b...
yrds96 5y ago
Well, this doc is from 2019, a lot of things can be changed. As a linux user, there are a plenty software that can render it perfectly but cannot edit, and i really don't get it. As a web developer i have to use photopea to be able to extract assets from psd files. It's really strange that a web application can edit psd and a lot of powerful softwares like gimp and Krita have the exact problem with layers and masks with modern .psd files, so my guess its bcause photopea IT'S designed to be compatible with psd files. I suspect it's incompatible by design with others softwares and not because it's has a poor specification.
- gfxgirl 5y ago
  you suspect wrong. I've written Photoshop readers since 1993. They all still work. Photoshop has never broken their format
- arcatek 5y ago
  If I remember correctly, PSD files have an optional header (when files are saved in "Compatibility Mode", which I think is the default) in which is encoded the rasterized image. It's then "easy" to display the image, but very difficult to edit it, as you then need to implement the whole renderer.
mhh__ 5y ago
PDF is also well documented by that measure, i.e. the complexity of said specification does matter.

mwkaufma 5y ago

Isn't that just... A browser?

ReactiveJelly 5y ago
Title is misleading, it's a PSD renderer for web browser, not a web browser's renderer.