7 ms·
Show HN: Handwriter.ttf – Handwriting Synthesis with Harfbuzz WASM
During the hype of llama.ttf months ago, I was speculating the potential of WASM shaper for even crazier purpose, one that fitter to a font shaper's duty -- to synthesize font at runtime. This project as proof-of-concept implements a synthesizer that generates and rasterizes handwriting-style font, backed by a super-lightweight RNN model (~14MiB).
- kragen 2y agothis is amazing! i'm guessing you can probably get it to antialias without much more work as the demo video shows, it's probably not something you want to have in between you and the ability to scroll a web page or close a tab. but i guess using harfbuzz now means we're buying into a turing-complete virtual machine running an arbitrary program in order to display a glyph. how seriously crippled are the harfbuzzless rendering paths? i'm assuming opting out of harfbuzz means opting out of arabic, devanagari and other indic scripts, etc.? is there a less out-of-control alternative that doesn't leave two billion people out in the cold?
- jcelerier 2y agoI wonder what makes SIMD an improvement here - in the end it all boils down to TTF bytecode and I don't think this comes with SIMD instructions, right?
- hsfzxjy 2y agowasm has SIMD extension https://github.com/WebAssembly/simd/blob/master/proposals/simd/SIMD.md https://github.com/WebAssembly/simd/blob/master/proposals/si... . You can use them even in browsers (e.g. Chrome >= 91)
- jsheard 2y agoOP is asking how SIMD is beneficial to this specific application. If there's a small neural network involved then evaluating that is probably a good place to use SIMD.
- hsfzxjy 2y agoYes. The program actually includes an ONNX runtime, which uses SIMD to accelerate NN inference.
- jcelerier 2y agobut... the program ends up entirely compiled and executed as TTF bytecode in the end right, since it's entirely contained in the TTF font ? And TTF bytecode is only the following instructions : https://developer.apple.com/fonts/TrueType-Reference-Manual/RM05/Chap5.html https://developer.apple.com/fonts/TrueType-Reference-Manual/... and I don't see anything related to SIMD in there
- erk__ 2y agoNo that is not what is happening, HarfBuzz have a experimental Wasm shaper, so the font embeds some wasm code that tells Harfbuzz what to output. https://github.com/harfbuzz/harfbuzz/blob/main/docs/wasm-shaper.md https://github.com/harfbuzz/harfbuzz/blob/main/docs/wasm-sha...
- jcelerier 2y agooh ok, so it can't work in any system that doesn't use harfbuzz.. that's much less interesting than what I originally thought
- a1o 2y agoI don't get it, why there's no link to a GitHub pages website to test the thing?
- hsfzxjy 2y agoYou can't test it in a browser, since no browser at present is linked against libharfbuzz with WASM shaper. Instead, one can test it with a modified local program such as gedit in the demo. As convenience, I built a Docker image that packs both the ttf file and the modified gedit together. You can try it out via `make run` as stated in the instructions.
- okcdz 2y agoIf you just want to run locally. Why not using the native libharfbuzz directly? What's the purpose of WASM here?
- bawolff 2y agoWASM is to libharfbuzz (when experimental compile time option is enabled) what javascript is to HTML. So this is essentially native (albeit experimental) libharfbuzz. WASM is used because its how font files are scripted (when using this experimental version of lubharfbuzz) Its important to keep in mind that wasm is a general technology and is not just used by web browsers.
- 0x457 2y agoFont includes WASM code that harfbuzz executes for shaping: https://github.com/harfbuzz/harfbuzz/blob/main/docs/wasm-shaper.md https://github.com/harfbuzz/harfbuzz/blob/main/docs/wasm-sha... It's an experimental feature, so it's not available unless explicitly enabled during compilation.
- deleted 2y ago[deleted]
- a1o 2y agoHarfbuzz can be built easily to Wasm using Emscripten.
- einpoklum 2y agoOP's repository has a Makefile which assumes a docker daemon is available.
- hsfzxjy 2y agoThe project must be tested in an application linked against libharfbuzz with WASM shaper enabled. Since it's not easy to build a library like this, I make a Docker image which contains both the ttf file and a modified gedit, so that anyone can test the project with a single command.
- deleted 2y ago[deleted]
- cyberax 2y agoYear 2045: fonts become self-aware and go on a strikethrough.
- noman-land 2y agoI'm mad at you for forcing me to laugh at this.
- deleted 2y ago[deleted]
- shove 2y agoBest comment
- koolala 2y ago:) I sware that smilely just winked at me
- dancemethis 2y agoThis is so cursed, there is so much that begs the question - "why?"... I love it. I can't wait for the next beautiful nightmare. Maybe someone should mix font rendering with PDF rendering. Of course, with a LLM doing something in the middle.
- hsfzxjy 2y agoThe answer is "no why", just for fun :) Though I learned a lot about how to stuff an NN model in a WASM binary and some tricks to optimize performance.
- thenegation 2y agoBest reason!
- dancemethis 2y agoI usually follow with "why not?", but forgot this time. Thank you for this. Again, I love it.
- deleted 2y ago[deleted]
- calebj0seph 2y agoDamn, this is impressive! We built a WebGL text renderer with full CJK support using Harfbuzz for our production whiteboard web app. I thought that was complicated until now.
- MarceColl 2y agoIs that open sourced? I want to render shakuhachi music notation in the browser trying to replicate the caligraphic style and it's not trivial.
- PhilipRoman 2y agoI bet this is what people felt when JavaScript was first demoed on the web. In the year 2077, when each font will run it's own virtual machine on WASM containing a "minimal" Ubuntu image, some enlightened blogger will suggest server side rendered fonts as a performance improvement.
- petercooper 2y agoJS felt less impressive to me when I first saw it, simply because it couldn't do much. However, I remember the showcase Microsoft put together to show off what CSS could do in IE3 and that was very cool at the time.
- pandeiro 2y agoI remember that, too, and switched to IE for the next X years
- tombh 2y agoI've read the README and watched the video, but I'm still not sure what this is doing? I know it can "synthesize [a] font at runtime", so does that mean it's creating a random handwritten font as you type? But it's not based on the user's handwriting?
- block_dagger 2y agoCorrect
- pjmlp 2y agoNice demo. Without trying to steal the thread, what I would care is actually the opposite direction. Neither in Swift Playgrounds, nor in any other programming development environment apps for both mobile OSes, have I found a good development experience using pen instead of keyboard. Given how many of us "program" in paper notebooks, it is quite incredible that besides a couple of research projects done by PhD students, no one cares to actually make it more widespread in a usable way.
- Unearned5161 2y agoI could see something like Apples new calculator drawing app taking this by storm. Writing out your code and getting syntax support and all the things in your ide but with your handwriting? In the same way that the calculator writes in the answers in your handwriting, it could write in snippets of code in your handwriting. If it's going to be made though, it has to be e-ink. Go big or go home!
- pjmlp 2y agoYeah, something like that.
- fenollp 2y agoYes! I'm on this route myself, trying various things out at https://github.com/fenollp/reMarkable-tools https://github.com/fenollp/reMarkable-tools Handwriting (in and out) support is very important IMO. Also being able to draw DAGs. I'd like an e-ink device with high frame rate and HW powerful enough to run some models locally or with good enough connectivity and sensors that e.g. Computer Vision tasks can be offloaded to the users' smartphone. Feel free to expose your ideas on there :) I welcome Open Source discussion!
- eigenvalue 2y agoFun hack. I bet Alex Graves never in a million years anticipated that his PhD thesis work would be encapsulated in a novelty font.
- gwern 2y agoThis looks like it'd be quite useful for faking documents more convincingly. Existing handwriting fonts always have tell-tale regularities and there's so few that forensics analysis exposes them easily.
- vintermann 2y agoSo you trained a THAT small RNN to make this good handwriting? I'm actually even more impressed at that than at the crazy pipeline turning it into a font in realtime.
- hsfzxjy 2y agoI didn't train the model myself. A pretrained model is adopted from another repo [0]. [0]: https://github.com/X-rayLaser/pytorch-handwriting-synthesis-toolkit/blob/main/my-app/synthesis_network_52.onnx https://github.com/X-rayLaser/pytorch-handwriting-synthesis-...
- BigParm 2y agoWtf is happening here what are the inputs and outputs? Hard to tell what this program does for me
- a2128 2y agoLink to llama.ttf, another fun font that abuses HarfBuzz for anyone interested (it runs a language model inside a font): https://fuglede.github.io/llama.ttf/ https://fuglede.github.io/llama.ttf/ The llama.ttf video does a pretty good job explaining what the heck is going on
- andrewmcwatters 2y agoIt visually fails to preserve “handedness.”
- fulafel 2y agoI wondered why there's just a video and no demo. But does it require the new wasm features in the browser ttf support?
- piyushtechsavy 2y agoNice demo, for sure a fun app.
- adzm 2y agoI've been curious about seeing if harfbuzz etc could help create a styled Elianscript font but not really sure where to start.