4 ms·

This why Flash was such a nice format for some things (not going into the pitfalls of Flash - but in some ways it was fantastic). I guess these days we have an

This why Flash was such a nice format for some things (not going into the pitfalls of Flash - but in some ways it was fantastic). I guess these days we have animated SVG, and https://lottiefiles.com/ https://lottiefiles.com/ is getting some traction - but these require you to export in a specific format of course, you can't just convert/trace a bitmap movie with these. And SVG or Lottie aren't designed for longer/streaming vector animations, and they don't carry synchronised or streaming audio - Flash did all of those things. Vectorisation of bitmap images does have some artifacts, as is evident in the Simpsons demo on this website - when possible, you should export in a vector movie format directly from the vector animation software. It is kind of depressing that we don't have an open standard for vector movies (with sound), over a decade after Flash was killed. Sometimes it feels like technology stops or moves backwards.

choeger 6y ago
Actually I wonder why we should have a vector video format in the first place. Would it not be much simpler to render a series of vector images and their transformations directly to a standard video code? I mean all the modern video codecs have vector elements afaik, and most have motion prediction. One could just leave out the whole messy inference and directly tell the decoder "here is a rectangular shape, moving with that speed in this direction", no?
- nitrogen 6y ago
  Flash vector animations are infinitely scalable from a very tiny file size. The vectors in modern codecs are motion vectors. They don't generally have any concept of drawing primitives like splines, polygons, fills, gradients, etc., and they are raster formats so they have a fixed native pixel size. The idea of a vector codec isn't new though, and prior attempts have been made, though I don't remember enough details to find a reference. Just for anyone who might not be familiar, "vector" in this context refers to the graphics you would generate in a tool like Inkscape, where you can define complex geometric shapes with just a few datapoints. "Raster" graphics are what Gimp works with, where each pixel is specified (effectively) individually.
- cogman10 6y ago
  The primitives aren't the same for video codecs as they are for something like flash. Even the transforms you can do are fairly limited. AV1, for example, is partially complex because it allows for some scaling transforms as part of the codec. The problem is that you could have those concepts in a video codec, but then you'd be in some sort of weird hybrid world. A general encoder would almost certainly never use most of those transforms and primitives. Meanwhile, the decoder would need to understand both to properly render anything. I'd imagine you'd frequently run into cases where the decoder fails simply because whoever wrote it didn't want to be bothered supporting the full set of features for the steam (since encoders don't employ that full feature set). Then there is the whole danger of having a turing complete video stream format. Last thing you'd want is for someone to publish a bitcoin miner on a youtube video :)
sb2702 6y ago
This is something I've been thinking about for a while as well, and I'll be giving a talk exactly on this subject next week: https://www.meetup.com/SF-Video-Technology/events/zpltdrybckbnc/ https://www.meetup.com/SF-Video-Technology/events/zpltdrybck... There are already vector-graphics runtimes like WebGL and SVG that are more than capable of rendering "video", even within the html video tag, and through modern video streaming architectures like Dash and HLS (will share the link to the talk, and demo links for these things next week). Those aren't "codecs" in the traditional sense, and I think there is an open question as to whether a "codec" is even necessary. Scrimba (https://scrimba.com/ https://scrimba.com/) uses "HTML" as a video codec just fine in production, and it works perfectly, and there's no "codec" per se behind it. That said - in 2020, web architectures already exist in a way that you could easily make a "video codec" for vector graphics, and some standardization would help - thought not specifically necessary for adoption in the way it would be for a "regular" video codec which doesn't just enjoy native vector graphics runtimes like SVG or WebGL. If you needed a file format, I honestly think that Lottie is the best option for a "file format" for vector-video, since it's open source, and mostly just based on the old Flash standard (https://www.adobe.com/devnet/swf.html https://www.adobe.com/devnet/swf.html) What's missing from Lottie for it to be a "video" format would be easier integration into video streaming architectures like HLS or DASH, and that's honestly something I'd like to do as an open source project - essentially a way for video players to "play" lottie video files as one of the video options (like, on top of 1080p, 720p and 480p versions, you have the vector version as well) From the perspective of Vectorly, we clearly understand that there's a lot of skepticism around the idea of a new "codec", and we wanted to avoid the idea that we are actually building a new codec. Our preferred framing is this: There are already "codecs" for vector graphics that are open, and as well established as H264 is. We're just working on a converter / transcoder from raster to vector, which admittedly will always have artifacts of some kind - though you could just as easily take source vector files and stream them and transcode them to a vector codec (SVG or WebGL) without doing that kind of raster to vector conversion that we're proposing.
- dharma1 6y ago
  Good luck with the company, I think what you're doing is very cool and I can see the use case for low bandwidth conversion for educational/whiteboard video in particular. I think you're right about current web tech being able to support this. Lottie extended to an open source vector "video" format would be awesome, with HLS/Dash streaming and especially audio (streaming in sync) too. Hope you can find a sponsor for such an open source project

choeger 6y ago

Actually I wonder why we should have a vector video format in the first place. Would it not be much simpler to render a series of vector images and their transformations directly to a standard video code? I mean all the modern video codecs have vector elements afaik, and most have motion prediction. One could just leave out the whole messy inference and directly tell the decoder "here is a rectangular shape, moving with that speed in this direction", no?

nitrogen 6y ago
Flash vector animations are infinitely scalable from a very tiny file size. The vectors in modern codecs are motion vectors. They don't generally have any concept of drawing primitives like splines, polygons, fills, gradients, etc., and they are raster formats so they have a fixed native pixel size. The idea of a vector codec isn't new though, and prior attempts have been made, though I don't remember enough details to find a reference. Just for anyone who might not be familiar, "vector" in this context refers to the graphics you would generate in a tool like Inkscape, where you can define complex geometric shapes with just a few datapoints. "Raster" graphics are what Gimp works with, where each pixel is specified (effectively) individually.
cogman10 6y ago
The primitives aren't the same for video codecs as they are for something like flash. Even the transforms you can do are fairly limited. AV1, for example, is partially complex because it allows for some scaling transforms as part of the codec. The problem is that you could have those concepts in a video codec, but then you'd be in some sort of weird hybrid world. A general encoder would almost certainly never use most of those transforms and primitives. Meanwhile, the decoder would need to understand both to properly render anything. I'd imagine you'd frequently run into cases where the decoder fails simply because whoever wrote it didn't want to be bothered supporting the full set of features for the steam (since encoders don't employ that full feature set). Then there is the whole danger of having a turing complete video stream format. Last thing you'd want is for someone to publish a bitcoin miner on a youtube video :)

sb2702 6y ago

This is something I've been thinking about for a while as well, and I'll be giving a talk exactly on this subject next week: https://www.meetup.com/SF-Video-Technology/events/zpltdrybckbnc/ https://www.meetup.com/SF-Video-Technology/events/zpltdrybck... There are already vector-graphics runtimes like WebGL and SVG that are more than capable of rendering "video", even within the html video tag, and through modern video streaming architectures like Dash and HLS (will share the link to the talk, and demo links for these things next week). Those aren't "codecs" in the traditional sense, and I think there is an open question as to whether a "codec" is even necessary. Scrimba (https://scrimba.com/ https://scrimba.com/) uses "HTML" as a video codec just fine in production, and it works perfectly, and there's no "codec" per se behind it. That said - in 2020, web architectures already exist in a way that you could easily make a "video codec" for vector graphics, and some standardization would help - thought not specifically necessary for adoption in the way it would be for a "regular" video codec which doesn't just enjoy native vector graphics runtimes like SVG or WebGL. If you needed a file format, I honestly think that Lottie is the best option for a "file format" for vector-video, since it's open source, and mostly just based on the old Flash standard (https://www.adobe.com/devnet/swf.html https://www.adobe.com/devnet/swf.html) What's missing from Lottie for it to be a "video" format would be easier integration into video streaming architectures like HLS or DASH, and that's honestly something I'd like to do as an open source project - essentially a way for video players to "play" lottie video files as one of the video options (like, on top of 1080p, 720p and 480p versions, you have the vector version as well) From the perspective of Vectorly, we clearly understand that there's a lot of skepticism around the idea of a new "codec", and we wanted to avoid the idea that we are actually building a new codec. Our preferred framing is this: There are already "codecs" for vector graphics that are open, and as well established as H264 is. We're just working on a converter / transcoder from raster to vector, which admittedly will always have artifacts of some kind - though you could just as easily take source vector files and stream them and transcode them to a vector codec (SVG or WebGL) without doing that kind of raster to vector conversion that we're proposing.

dharma1 6y ago
Good luck with the company, I think what you're doing is very cool and I can see the use case for low bandwidth conversion for educational/whiteboard video in particular. I think you're right about current web tech being able to support this. Lottie extended to an open source vector "video" format would be awesome, with HLS/Dash streaming and especially audio (streaming in sync) too. Hope you can find a sponsor for such an open source project