11 ms·
A return to hand-written notes by learning to read and write
- abdullahkhalids 2y agoThis is very cool. Here is interesting application of something like this. My handwriting is pretty bad, and worse still when writing fast. When I am teaching, a lot of what I write is worse than I would like it to be. I could teach a system like this my very slow neat handwriting. And then as I write on my whiteboard while teaching, it replaces my quick bad handwriting with the neater handwriting.
- rnewme 2y agoWhy not simply have a laser projector, keyboard and canvas textbox then?
- deleted 2y ago[deleted]
- elashri 2y agoFrom a quick visit to his profile (linked website), he is a physicist. This technology setup is very complicated and against the eternal usage of blackboard in a typical physics department. And to be honest this applies to his suggestion as well but you still at least get the feeling on writing on a board.
- abdullahkhalids 2y agoOf course, all true physics happens on the blackboard, in a notebook (or in Mathematica). But, I am forced to use digital tools occasionally, and I am not opposed to improving them.
- nine_k 2y agoYou also need the skill of lightning-fast LaTeX typing, and the skill of drawing and drafting with a speed comparable to that of a chalk. You need a canvas-driven tool for that, and your eyes would be on the screen for long periods of time, not contacting the audience.
- rnewme 2y agoThere are regular white/blackboards that you write on with regular marker that just has a tracker on it, so content appears on the canvas on screen for those who are remote. More advanced versions also have laser projector that can project animations, moving diagrams and text on the same board. My suggestion is to tap the board on the place he wants to write and just type it on regular keyboard, hardly a distraction!
- Wowfunhappy 2y agoWouldn't it make even more sense to replace your quick bad handwriting with perfect Helvetica?
- abdullahkhalids 2y agoThere is (subjective) charm in handwritten text that is simply not present in digital fonts. I would rather keep that magic alive.
- deepGem 2y agoI think handwriting is a very personal trait. Some people value good handwriting and they write neat even when they write fast. Others don't. Sadly, a vast majority if the world exists in the other camp. This is why they invented typewriters. If everyone conformed to good handwriting and could agree upon a good writing style standard, the world would have been very different.
- eru 2y ago> This is why they invented typewriters. Historically, that didn't seem to be the main reason for the many inventions of the typewriter. See https://en.wikipedia.org/wiki/Typewriter#History https://en.wikipedia.org/wiki/Typewriter#History
- imoverclocked 2y agoThat wikipedia article was a great read but I have to agree with GP. > According to the standards taught in secretarial schools in the mid-20th century, a business letter was supposed to have no mistakes and no visible corrections. Certainly, there were other reasons like speed, repeatability, official-looking, etc... too. Most typewriters wanted to be cheaper and on-demand printing presses. Some even managed variable width fonts! :)
- eru 2y agoOh, the original comment is certainly right in spirit, I just wanted to be pedantic. But I'm not sure why you quote something about the md-20th century, when we are talking about the (many) invention(s) of the typewriter? They were already old and well-established technology at that point in time.
- sitkack 2y agoIf you draw your equations well enough, they can get converted into LaTex in realtime and then you could run them in a computational notebook. Esp if you fuse the audio of you explaining the equations along with the LaTex it can correct for errors.
- abdullahkhalids 2y agoWhat software can do this?
- currymj 2y agoapple notes on new iPadOS does this right now -- just cleans up your handwriting to be slightly neater but still look like you.
- eleveriven 2y agoThat's actually a brilliant feature! Without losing the "you"
- egypturnash 2y agoOr you could just find some lettering manuals and improve your handwriting. Practicing at a slow speed will improve your fast work, too.
- znpy 2y ago> My handwriting is pretty bad i know it might sound dumb, but have you tried playing with a fountain pen? The feedback is way different from a ballpoint pen and it also depends on paper and the kind of ink. It makes writing way less "predictable" and a bit more enjoyable. a cheap one (5-15$) with a medium nib might be a good start... some people move on to collect fountain pens, but i do most on my (on paper) writing with a ~20$ Pelikan Jazz.
- xarope 2y agoI've realized that when I use cheap pens on hotel stationery, my handwriting looks terrible, probably because the surfaces are too smooth? Other than fountain pens, are there other alternatives that give more tactile feedback?
- sam29681749 2y agoIMO felt tip pens feel comparatively rough to gel, and ballpoint pen, etc.
- mbivert 2y agoNibs / dip-pens ;-) It's only half a joke: having to regularly dip the pen in ink, be mindful of how much ink you have, having to swiftly wipe it once in a while to avoid drying ink (& flow issues), forces to slow down & take "micro-breaks". This benefits the handwriting, but also the quality of the study. And is surprisingly relaxing. (so called "crow-quill" nibs are relatively cheap, available, carry a fair amount of ink)
- wrp 2y agoWhat you are asking for is more "tooth", something mostly determined by the paper. Most stationery fans prefer smoother papers, but I agree that after a certain point, increased smoothness makes my handwriting worse. If you are stuck using a very smooth paper, I would suggest either a fiber-tip or a drier gel pen. Gel pens with a clicker (e.g. Zebra Sarasa) tend to write drier than those with a cap (e.g. Uniball Signo). Also try a fatter tip. Although this may not be acceptable in many situations, a soft pencil may provide even better control on smooth paper.
- Propelloni 2y agoImproving your hand writing is not hard. For whiteboards start out with using block letters only. It will slow you down in the beginning but not for long. That's one of the "game changing" hints I received during my time as a tutor at university. (One other was to always copy books from back to front; very useful but somewhat outdated now.)
- shiroiushi 2y ago>One other was to always copy books from back to front; very useful but somewhat outdated now. What's so useful about that?
- hammock 2y agoSlows you down if I had to guess
- mbreese 2y agoIf your copies come out of the machine right-side up and they get stacked on top of each other, then you can just take the finished stack from the machine when finished. Otherwise, you’d have to reverse the stack, which in real life or in a computer is an expensive operation.
- albert_e 2y agoThe photocopies come out stacked in the right order maybe?
- bilekas 2y ago> We present a model to convert photos of handwriting into a digital format that reproduces component pen strokes, without the need for specialized equipment. Call my a cynic but this feels like a free way for Google to pull more data for training.
- deleted 2y ago[deleted]
- byte_0 2y agoThat’s exactly what I thought.
- mococa 2y agoNo evil.
- pkaye 2y agoThere is a repo with Apache license. Does that need to connect to Google services to use?
- ipsum2 2y agoSure. You're wrong and a cynic. The model does not connect to Google servers (unless you decide to use their colab, which is optional), you can use it offline without contributing anything back in data or code.
- rgovostes 2y agoAside from Google having published open source OCR tools like Tesseract for 20 years, it's a thoughtless accusation in general. What exactly is the insinuation? "Training" is just thrown out as a bogeyman. I can't even come up with a fictional scenario in which Google does something nefarious with piles of handwritten documents they've somehow acquired.
- sfilmeyer 2y agoTraining doesn't have to be nefarious. Google ran GOOG-411 in large part to collect speech data, but folks using the service still benefited.
- backtoyoujim 2y agoI spent a few years doing this without google. I suggest it, and I suggest it without google.
- khana 2y ago[dead]
- henning 2y agoCan this be used to create deepfake forged signatures/handwriting?
- 101008 2y agoThat was my initial thought too. For some important authors, there are a few high quality scans of their manuscripts, so with a tool like this you could create fake manuscripts -- and in a few years, after they are dead, say that you find them, create provenance, and boom, unpublished novel by JK Rowl- any author.
- Legend2440 2y agoNo. It is not designed to be used as a generative model. But other people have created handwriting generation models.
- vlovich123 2y agoIf it can generate accurate shapes from a photo of a signature, why would you need a generative model? Just draw the SVG wherever you need it.
- WhatsName 2y agoWhat is currently state-of-the-art when it comes to detetcting handwriting from photos? Tracing strokes is nice but I would be more interested in converting my handtaken notes to markdown.
- thimabi 2y agoI don’t know if they are the state-of-the-art, but handwriting recognition in iOS and ChatGPT do wonders for me — even with an ugly handwriting. Though these are more like 90% to 95% accurate, you should review the output before trusting it.
- burnished 2y agoIts pretty remarkable. I've used my phone to take pictures of stickers with model information that I couldn't otherwise reach and was able to copy the text from it. Really wild stuff.
- markvdb 2y agoI'm looking for something like that, but free/open source and offline. Suggestions welcome!
- emporas 2y agoI tried to use tesseract for OCR, 10 years ago, it recognized English good enough. tesseract was also developed by Google if I am not mistaken, but open source. I tried to use it then, for non English language, for Greek, and it was very bad. Happy to see some good OCR research based on transformers.
- 0x38B 2y agoI’ve been really impressed with Tesseract - I used it last month to add invisible OCR text (1) to scanned PDFs I reference a lot. My scans are quite good, but I was still impressed with the accuracy. I also OCRed the TOC, playing with the page segmentation setting (2) in the terminal until I got output I could copy & paste to add a navigable table of contents. 1: with the help of https://github.com/ocrmypdf/OCRmyPDF https://github.com/ocrmypdf/OCRmyPDF 2: https://tesseract-ocr.github.io/tessdoc/Command-Line-Usage.html https://tesseract-ocr.github.io/tessdoc/Command-Line-Usage.h..., “ Using different Page Segmentation Modes”
- stavros 2y agoI OCRed your comment with Tesseract: ``` I've been really impressed with Tesseract - | used it last month to add invisible OCR text (1) to scanned PDFs I reference a lot. My scans are quite good, but | was still impressed with the accuracy. | also OCRed the TOC, playing with the page segmentation setting (2) in the terminal until | got output I could copy & paste to add a navigable table of contents. 1: with the help of https://github.com/ocrmypdfiOCRmyPDE https://github.com/ocrmypdfiOCRmyPDE 2: https://tesseract-ocr.github.io/tessdoc/Command-Line-Usage.h https://tesseract-ocr.github.io/tessdoc/Command-Line-Usage.h..., “ Using different Page Segmentation Modes” ``` This kind of mirrors my earlier experience with Tesseract, if it can't get OCRing a screenshot right, what can it get right? It's not like "I used" is such a rare phrase either, but it replaced the I with a pipe.
- 0x38B 2y agoI OCRed an unprocessed screenshot from the chapter's table of contents (1), which gave me (2). The collated table of contents (3) was error free, but as your example shows, this OCR isn't good enough to not need checking and proof-reading. 1: https://nexus.armylane.com/files/vogue-sewing-11-toc-screenshot.png https://nexus.armylane.com/files/vogue-sewing-11-toc-screens... 2: https://nexus.armylane.com/files/tesseract-ocr-output.png https://nexus.armylane.com/files/tesseract-ocr-output.png 3: https://nexus.armylane.com/files/vogue-toc.txt https://nexus.armylane.com/files/vogue-toc.txt
- moatmoat 2y agosuch an exciting research project! I can imagine the impact this could have on education, e.g. handwriting notes of teachers in digital copies; or even preserve old documents in their digital counterpart
- no-reply 2y agoThis is very interesting. I had this idea of imitating human handwriting in my bucket of todos for machine learning models, but never got to it. I guess we aren't far from it.
- thimabi 2y agoFrom the title, I naively assumed this article would be about people relearning to make legible/beautiful handwritten notes after losing this ability. That is something I’m currently struggling with after many years of too much typing and not as much handwriting. Google’s actual research does help people like me, by making our notes less awful digitally. But I’d love not to be dependent on tech innovations to make my handwriting better.
- moron4hire 2y agoStudy comic lettering. Not saying it's the most efficient way of writing, but the process will teach you to think in terms of strokes and consistency. You can easily develop your own style from there.
- al_borland 2y agoIf you’re serious about this, I’ve stumbled into areas of YouTube with people dedicated to this. Pick a font of how you want your writing to look, and practice, practice, practice. People make available (and sell) special lined sheets to get the height of various things right or help to guide the writer to the perfect slant. You just have to have the time and interest to do the work, much like you probably did when first learning to write.
- thimabi 2y agoHow do you recommend that practice to be like? Simply trying over and over to perfect each letterform on empty lined sheets, based on a reference? I’ve done a few handwriting workbooks, mostly consisting of block letter templates to fill. But each of them is incomplete on its own, and they don’t even share the same font, making it much harder to practice.
- dbtc 2y agoAlso check out the resources on the r/handwriting subreddit.
- prabhu-yu 2y agoGood idea. May you please share youtube link you are referencing?
- 8n4vidtmkvmk 2y agoCan I just get good OCR for handwritten text? The last model that claimed to be "the best" was atrocious, only worked on PDFs of papers. ChatGPT is pretty decent but I was hoping for an offline, tailored solution
- anu7df 2y agoCheck out textify on appstore. I have found the accuracy to be great even for my terrible hand written text.
- llm_trw 2y agoNow I'm wondering if they can use a similar architecture to derender paintings. It would the quite something to have a stroke perfect recreation of the Mona Lisa drawn by a modified pen plotter.
- MacTea 2y agoRings a bell: https://arxiv.org/abs/2108.03798 https://arxiv.org/abs/2108.03798
- jheriko 2y agoseems overkill... for a bunch of reasons tbh. I guess the golden sledgehammer is just too tempting. taking photos of notes is a weird compromise that nobody really wants... we need better tools not better post processing
- anu7df 2y agoI don't think this is a hypothetical use case, at least for me. I like writing on paper with a fountain pen. But would like a digital version of the notes that are searchable. Reasonable ocr exists for conversion to text, but this would may be give slightly more accurate results.
- Evidlo 2y agoCan this work on low-power devices like note-taking tablets?
- Legend2440 2y agoYou may be able to run it on an iPad, but probably not a Kindle. But also, the point of this is to convert photos of text into pen strokes. A note-taking tablet doesn't need it because it already has the pen strokes you wrote.
- Pamar 2y agoEric Hebborn (https://en.m.wikipedia.org/wiki/Eric_Hebborn https://en.m.wikipedia.org/wiki/Eric_Hebborn) wrote "Italico per Italiani" (available in Italian only, alas https://www.amazon.co.uk/Italico-italiani-moderno-trattato-calligrafia/dp/8894272273 https://www.amazon.co.uk/Italico-italiani-moderno-trattato-c...) as a textbook/manual to improve day-to-day, practical calligraphy. Maybe there is something similar in other languages too...
- imoverclocked 2y agoI recently purchased a small refrigerator whiteboard and it's been really amazing with the combination of my iPhone's ability to take a picture of my handwriting (script or cursive) and copy/pasta into a text. It's not always perfect (nor is my handwriting!) but it's good enough to just replace a character or two and hit send. This really tickles a bunch of things for me: 1) I am not sending a whole image (it's efficient) 2) I don't have to type/swipe at all (I'm not looking at a screen) 3) My s/o has easy access to the list at all times (it's not in a cloud) 4) It requires no power to update/maintain (markers last a long time) 5) It just feels so natural to grab a marker and write on the fridge when I exhaust something in that same fridge.
- cptcobalt 2y agoThis is a nifty idea. I bet there'll be a HN-correlated pop in fridge whiteboard sales from this.
- LoveMortuus 2y agoHere's a suggestion if you want to avoid markers and such: LCD drawing tablets, I'm talking about the ones that look and function similarly to Etch and Sketch, but it uses a stylus and an LCD display, you can get these for less than 10€.
- bathwaterpizza 2y agoE-paper tablets these days run android, have fast refresh rates and multiple colors, they're great.
- PeterStuer 2y agoTbh my handwriting is write only. Through decades of computer use it has deteriorated to the point were it is illegible even to myself. Still. I will always be notetaking (and doodling) during meetings as it helps me order my thoughts.
- jillesvangurp 2y agoSame here. And not only that, years of keyboard and mouse usage have done some damage to my wrists and nerves and it's actually painful for me to write.
- soco 2y agoI believe that pain comes from misplaced expectations. We remember the times when we were fluent writing, or see it in others, and expect our hand to follow the same pattern with the same speed. So we push too hard. We should realize instead our hands must re-learn to write at this time, drawing the letters so slow like the small children do.
- ricardo81 2y agoSame. Over 20 years (note taking especially) it is reduced to scribbles. The value of it is the time thinking while writing it mostly, though todo's are longer lived.
- dr_sausages 2y agoIf your todo list is, or needs to be long lived, I think you might have more problems :D I make notes on paper as the act of writing makes it stick about in my head longer than it would if I typed typing them out; though similar to you there's better penmanship from a spider that fell in ink.
- ricardo81 2y agoLong lived may be tomorrow/a few days :-) And struck through. And written at any angle on the paper. Definitely not a codified method for me.
- aiisahik 2y agoCan it read the scribble of my doctor? If so this is groundbreaking in the medical data entry space.
- bobnamob 2y agoThe number of deaths attributable to misread treatment orders in hospitals is staggering. I'd be very careful about sticking another layer of interpretation between doctor and treating nurse. It's unfortunate med school doesn't teach block lettering like they used to teach to draftsmen/women.
- sneak 2y agoEntering this sort of data correctly should be on the doctor. McDonald’s digital order style giant pancake buttons with huge touch targets with wide margins and large type on a huge touchscreen should solve the boomer objections. Alternately, make it an app for the huge iPad Pro to solve the same problems. Make it as hard to fuck up as a fast food order. Disambiguation of input commands is a solved problem. There isn’t really the will. Stupid windows apps with standard windows UI dropdowns that confuse and frustrate people not well versed with computer UI, running on standard low contrast small type displays with standard keyboard and mouse input, running on laggy RDP thin clients to underprovisioned workstation VMs in a data center is sadly the industry norm, and it still kills people.
- bradydjohnson 2y agoAlmost no medical school or hospital in the United States use paper or handwriting anymore. All orders are electronic.
- blitzar 2y agoIt says a lot that the illegible writing killed people yet appears to be affected by doctors in the same manner as a bimbo uses a vocal fry affectation. Glad to see my scripts from the doctor these days are typed not written.
- 2y ago
- nxobject 2y agoAs a lay reader, it's fascinating to see how much oomph we're getting out of LLMs on non-language-related tasks by figuring out clever encodings/linearizations.
- brainzap 2y agoIt was a scary moment when Apple Notes corrected my writing in my own hand writing.
- anshumankmr 2y agofinally, I will be able to read my doctor's prescriptions.
- pjmlp 2y agoI still look forward to have programming environments on tablets that are able to use pen input, instead of forcing us to carry a bluetooth keyboard. Apparently not something that anyone cares as business opportunity, because most likely most folks wouldn't pay for it, sadly.
- another-dave 2y agoI'm definitely faster at typing than writing though — especially so when it comes to something like code that often requires in-place editing, shuffling statements around etc. I do like to work with paper & pen but moreso for ideation, diagramming, or todo lists rather than more "structured" inputs.
- pjmlp 2y agoNaturally the idea is to explore new concepts a mix of diagrams and coding, Basically what Brett Victor in recent times, and others have been trying to do for decades, "Grail on Rand Tablet from 1968 demo" https://www.youtube.com/watch?v=2Cq8S3jzJiQ https://www.youtube.com/watch?v=2Cq8S3jzJiQ Or "Programmable Ink" at Strange Loop 2022, https://www.youtube.com/watch?v=ifYuvgXZ108 https://www.youtube.com/watch?v=ifYuvgXZ108 There used to exist a graphical based Lisp for iPad, Lisping, unfortunely the app is long gone as it wasn't kept up to date with the OS. Here is on old thread discussing it, https://news.ycombinator.com/item?id=3802131 https://news.ycombinator.com/item?id=3802131 The new iPad Calculator, is a more recent example of such ideas.
- Vampiero 2y agoMan that would be horrible UX, I wouldn't pay for that. I'd only use it if someone paid ME to do it
- pjmlp 2y agoDifferent strokes for different folks.
- tomjen3 2y agoCan you touch draw 120 WPM?
- honeybadger1 2y agoi still write notes daily and already this year have finished 4 notebooks...however, things i need to review are typed, i write to keep things in my mind as they are being discussed...example, if i encountered an error while reviewing a program execution i would write it down "encountered error during attempt to do x" but i would also type it in my notes in vastly more detail with screenshots and other points...handwriting to me is almost like tagging it in my mind so i just don't forget that it happened.
- scrivanodev 2y agoVery interesting experiment. I've been working on a handwriting application [0] for the past couple of years and incorporating the ability to take a picture to convert it into digital ink would be really nice. [0] https://scrivanolabs.github.io https://scrivanolabs.github.io
- fredgrott 2y agoIts not the writing PART! It is the non automation part! The experiment you can do to verify my point: 1. Write code by hand in plain text editor..... What you notice....programming syntax and knowledge becomes easier to retain and re-learn. No joke I do this several times a week.
- albert_e 2y agoCan we replace / augment the keyboard with a white (or marked) paper to write on ... that is in view of a camera that has real time OCR Feels like it will be a good addition to input devices
- deleted 2y ago[deleted]
- poulpy123 2y agoI'm wondering how well it would work with my crappy writing and the fact I write some letters revere from normal
- vintermann 2y agoA model that could turn "offline" handwriting (the ink on the page) into "online" (order and timing of the strokes) I think could be really useful for a historical HTR pipeline... but ultimately, we need end to end. Why is historical HTR so neglected in all multi-task model evaluation benchmarks? There are millions of un-indexed handwritten historical documents which could give us a so much better understanding of our recent past. For that matter, it could give models much better understanding of our recent past.
- khana 2y ago[dead]
- deleted 2y ago[deleted]