10 ms·
Towards Fully Automated Manga Translation
- ThePadawan 6y agoWhat a great keikaku!
- krapp 6y agotl: keikaku means plan
- k__ 6y agolol All according to keikaku!
- looperhacks 6y agoTo readers that do not understand the joke: in a fan translation of the anime Death Note, the translator decided to translate the line "Just as planned" as "Just according to keikaku" with a translators note above it, saying that keikaku means plan. While back in the day, japanese words were often included with translators notes, this wasn't needed here and obviously a joke. This was only done when the word carried special meaning that couldn't be accurately translated using english words. Honorifics like -san or -sama were prominent examples. (I kinda miss the times when common japanese words were not translated. Current translations often look weird (esp honorifics) and since the rise of Crunchyroll and subsequent demise of fansubs, there isn't much choice left for different translations)
- toyg 6y agoAs somebody who never followed the fansub scene, the rise of japanese words in common English web-lingo (like desu or senpai) was really weird to observe.
- drivebyops 6y agoSome translators that still insist on localizing honorifics is funny, stuff like using Sir or making up a nickname for a character. I wonder what the Japanese do when translating English source to Japanese, do they insert honorifics where it doesn’t exist in the source?
- parsecs 6y agoCorrect me if I'm wrong but usually honorifics are not used with non east-asian sounding names. Source https://youtu.be/5rOHpkpYMIM https://youtu.be/5rOHpkpYMIM
- lmm 6y ago> This was only done when the word carried special meaning that couldn't be accurately translated using english words. Or when the translators or fans were weebs. E.g. many Death Note translators didn't translate "Shinigami", even though not only is "Death God" a perfectly valid western cultural concept, the Japanese cultural concept is actually directly derived from the western one.
- Izkata 6y ago> and since the rise of Crunchyroll and subsequent demise of fansubs, there isn't much choice left for different translations) Two of the biggest names in ripping Crunchyroll have both shut down, one very unexpectedly, and I've noticed a small increase in fansubs since then. I'm kinda hoping for a resurgence now.
- JoeDaDude 6y agoA free trial version would be nice. BTW, the nitpicker in me wants to say that sound effects were not covered, only text in speech balloons was caught, as can be seen in their very first picture. If it can do visual novels too, I'm in.
- dfxm12 6y agoI think people are going to nitpick no matter how sound effects are handled. They are almost as much art as text. It is clearly out of scope for this paper to be able to "redraw" the sound effects in English, plus, some (I'd guess most, but importantly, not all) might want to see the original anyway and have a note somewhere for it's translation. I also think it's out of scope to find a suitable place to put this note - you don't want text to cover up interesting art.
- kevingadd 6y agoYeah, a common general-purpose solution for this is to just put hotspots over all the non-speech-bubble text that you mouse over (or tap) to reveal localized text for. It really sucks, but there's no obvious general solution since the size gap between japanese and english text can be huge. Naturally, those hotspots aren't an option in print or pdf editions...
- the8472 6y agoA common solution is to add footnotes and sprinkle them around the panel gaps.
- catchmeifyoucan 6y agoMy friends built this Chrome Extension that uses ML to help with the translation process. https://chrome.google.com/webstore/detail/manga-translator/oooikipggipaohjibkmoiacmmlilgokn?hl=en https://chrome.google.com/webstore/detail/manga-translator/o...
- nacs 6y agoFor Firefox users, here's an alternative: https://addons.mozilla.org/en-US/firefox/addon/anity/ https://addons.mozilla.org/en-US/firefox/addon/anity/
- Bork_Bork 6y agoI wonder when/if/ever context aware MTLs will view onigiris as jelly donuts. Humans already have an issue with using terms or something such that does not exist within the manga world such as referring to "china" (Chinese ceramics) or Americanizing manga to the point where it makes even less sense to the reader (unless perhaps if you're American? As can bee seen with the mentioned above). Same with people who ̶t̶r̶a̶n̶s̶l̶a̶t̶e̶s̶ ̶W̶e̶s̶t̶e̶r̶n̶i̶z̶e̶s̶ Americanizes suffixes because I'M PRETTY SURE that EVERYONE calls their classmate MISS or MISTER. And the same goes obviously towards pet/nicknames. Heck, everyone addresses their older sister/brother as big sister/brother and NOTHING ELSE right?
- kevingadd 6y agoThe onigiri/jelly donut thing is not particularly uncommon if you try to shove complex text or dialogue through Google Translate and similar MTL systems, because they are trained heavily on existing text. If the japanese string you plug in is uncommon enough you may get back imageboard slang, text from video game wikis, or racial slurs.
- Bork_Bork 6y agoThat means an overreliance on the word bruh if we go by current standards. Hopefully it does not come to that but if current AI trends is anything to go by...
- krapp 6y agoI'm pretty sure the onigiri/jelly donut thing only happened because 4Kids didn't think American children would know what onigiri was - they were infamous for their terrible localization. Google Translate translates it more accurately to "rice balls" today - something as simple as that wouldn't be a problem for machine translation.
- kevingadd 6y agoThat's assuming the training set is all of the media that mainstream Google Translate was trained on. The paper talks about more specialized training sets so it's quite possible something like onigiri could end up under-represented (though I suspect it's common enough in manga that it would be something more obscure). Naturally the "jelly donut" part would only slip in if they used untrustworthy data like forum posts or 4kids localizations, but that's exactly the kind of data that Google Translate and similar algos scrape from the internet for training.
- kevingadd 6y agoIn translation communities the idea of using automated translation even in part in the process of translating manga is frowned upon (good luck if you try to get away with doing this on a paid gig), and even the people pirating manga on websites tend to actively dislike automated translation as well. Heroic efforts from an excellent machine translation stack and a team of quality checkers and editors can help clean it up, but at the end of the day you'd probably save time and money by just hiring someone fluent in both languages. It's always interesting to see research perspectives on this. I think there's considerable room for improvement, especially given that current translation tech doesn't take into account things like panel layout, horizontal/vertical layout of text, different typeface use, hand-written vs. typeset text, and mixed katakana/hiragana/kanji usage - all things that an author may use to convey tone, implicitly identify the speaker, or add subtext. There are also things with no equivalent in English literature/comics whatsoever, like using furigana to attach a second reading (or dual meaning) to a word. I can imagine some of this info eventually being pulled in by built-for-purpose MTL tools if a research group puts in the time and energy to do it - the paper appears to be a couple brief steps in that direction because they're feeding in basic information about panel layout and the genders of the people in the panels, but little else. Excited to see what happens in this space even if the idea of even more people trying to translate dense comic prose with Google Translate fills me with dread. EDIT: I should have done more research before posting this... digging into the authors of this paper, two of them are working at a company that is trying to sell this current technology to authors right now despite the fact that it is not adequate for the challenges of translating comics to other languages. How depressing.
- AdmiralAsshat 6y ago> It's always interesting to see research perspectives on this. I think there's considerable room for improvement, especially given that current translation tech doesn't take into account things like panel layout, horizontal/vertical layout of text, different typeface use, hand-written vs. typeset text, and mixed katakana/hiragana/kanji usage - all things that an author may use to convey tone, implicitly identify the speaker, or add subtext. There are also things with no equivalent in English literature/comics whatsoever, like using furigana to attach a second reading (or dual meaning) to a word. I also can't imagine how it would deal with the idiosyncratic, occasional-nonsensical English that some manga-ka's like to sneak into their work: http://2.bp.blogspot.com/-Leelb4eEGz0/UzALTIDcgvI/AAAAAAAAEXU/BJbqwuLyXuU/s1600/HAIL2U!.png http://2.bp.blogspot.com/-Leelb4eEGz0/UzALTIDcgvI/AAAAAAAAEX...
- forkLding 6y agoI've been a volunteer manga translator in the past, (also started getting paid for it after a while when our manga translation group started getting funded) and a large part of it is using google translate or translation tools to help out with definitions or look for inspiration in translating the words in the manga so this could likely help out in that regard by saving the need of tools like google translate
- JohnBooty 6y agoI was thinking the same thing. This generally wouldn't replace human translators in a worthwhile way (in the way that grammar- and spell-checkers and even GPT don't replace human writers) but it might help with some of the drudgery.
- pradn 6y agoYes, AI translation tools could produce starter text that a human could fix later. I assume the text is placed in speech bubbles manually? Perhaps in something like Photoshop? In that case, AI can also place editable text boxes at the speech bubbles, saving more time. I'd love technical details for how you translated manga!
- meowzero 6y agoI have a friend who worked as a professional translator and that's exactly what he did. He put the text through Google translate and fix any errors.
- numpad0 6y agoForcing externalities improve cost efficiency
- spookybones 6y agoHow were you paid? Patreon?
- mxcrossb 6y agoI consider myself able to read manga in Japanese, yet certainly encounter a few words each chapter which I have to guess the meaning of. So I can see how this would help. Yet my impression would be that this isn’t really that time consuming? I would think typesetting would be the most labor intensive by an order of magnitude.
- hertzrat 6y agoThe quality of a translation makes a dramatic difference in the reading experience. I have 4 or more copies of the art of war, each with different translators. Dante’s inferno, the Illiad, Dostoyevsky, Beowulf, Confucius: some translations are unreadable and one or two incredible. Automation is just not going to produce the really good translations. The only benefit of a technique like this will be to generate unlicensed translations of work that will not otherwise get translated, which isn’t even that big a deal because the communities already produce unlicensed translations of just about everything. Edit: The other use is to help create better context aware data scrapers that can combine bi-modal information streams and add some body language understanding. I guess it will probably end up in automated surveillance tech like security cameras/mics etc if it works.
- AdmiralAsshat 6y agoI can think of quite a few manga/anime/videogames where a machine translation would've been acceptable because I just needed a bare minimum understanding of what was going on. Think of any number of 8-bit games where the game is 95% hack-and-slash, but a random villager tells you before the last boss fight, "You must make sure to wear the enchanted amulet in order to pierce the dragon's scales!" It's one throwaway line, but if the line is in Japanese, an American playing an imported ROM might spend an hour of frustration wondering why his sword does zero damage to the final boss.
- grawprog 6y agoThe nes had plenty of real examples where poor translations made the game harder or made secrets obscure and hard to find without a guide. Castlevania 2 and Adventure of link come to mind. Actually, i'd say most text heavy games translated from japanese on the nes suffered from this problem and made those games way more confusing than they should have been.
- AdmiralAsshat 6y agoSure...but I don't feel like we're comparing apples to apples here, since I explicitly mentioned text-light games and you brought up the most text-heavy, RPG like entries in the respective CV and Zelda series (of that generation). To boot, Simon's Quest also had the atrociously bad idea that NPCs in the game can lie to you and give intentionally incorrect information, making the translation effort that much more confusing.
- JohnBooty 6y agoI gave the paper a quick read and there's an impressive amount of context awareness. However one particular thing it seems like it wouldn't handle well would be ideosyncratic speech that is particular to a specific character. examples: - a character who uses clunky or outdated forms of Japanese - a character meant to have a rural account - a character with a speech affectation, like adding a cutesy -nyo to the end of their words/sentences - etc Perhaps these could be improved if the model was trained on human translations for that specific character and weighted appropriately. With some manga running for dozens or hundreds of volumes that would feasible. It also definitely wouldn't help with puns, cultural references or other subtleties! BUT, this could still be really useful as an aid to human translators.
- Tade0 6y agoTo that list I would add characters who speak in katakana - at least Google Translate doesn't seem to be aware that this is a thing.
- frigdreams 6y agoWorth checking out Clyde Mandelin who used google translate on the Japanese version of Final Fantasy IV and documented the silly results. https://legendsoflocalization.com/funky-fantasy-iv/ https://legendsoflocalization.com/funky-fantasy-iv/
- userbinator 6y agoThere is something almost Shakespearean about some of those sentences. I guess it's the combination of regular English words with highly unusual grammar.
- unwoundmouse 6y agoI've been thinking about this problem for a little bit. I think an app idea that has potential is a sort of 'wikipedia for manga translation'. Essentially you have some automated translator do a first pass, create rough translations. This is served to customers. All the readers, then, have the ability to move around the overlayed speech bubbles and edit the text inside them. For the most part, automated translation seems to capture a lot of high level stuffs, and the large number of community participants can fill in the details/broken translations using context, and additionally do whatever cleaning needs to be done.
- Klaster_1 6y agoSomething similar has already been done years ago, see the infamous GTO scanlation (exmaple: https://www.reddit.comhttps://www.reddit.com/r/manga/comments/8pizdl https://www.reddit.comhttps://www.reddit.com/r/manga/comment...). Decent manga (and comics in general) translation requires a "redraw" step where original text is completely removed. Some titles keep text strictly in bubbles, but most have text over the artwork to a varying degree, from simple (minutes per page) to highly non-trivial (hours). An image inpainting algorithm that removes text from manga would be really impressive and save a lot of effort.
- kevingadd 6y agoGood automated (or even partially automated) inpainting would probably be a godsend for typesetters, since they can spend a considerable amount of time on a single page if it has lots of text or detailed illustrations. I imagine the publishers that put out multiple print volumes a month would probably pay good money for it, but it might still be too small of a market to build commercial software for... In the worst case even skilled typesetters can spend an entire day cleaning up and typesetting a particularly nasty page if they have to repaint lines and blank out dozens or hundreds of kanji (it happens)
- knolax 6y agoAnother machine translator polluting the internet with garbage text.
- acl777 6y agowow - these guys got Yahoo Japan to sponsor their manga addiction, genius! :-)
- golemiprague 6y agoI wish there was an automatic translation for anime but translating it word by word as they appear in the japanese order of the sentence, just as an assistance to listening
- 2020overyet 6y agohttps://mantraviewer.datada.repl.co/ https://mantraviewer.datada.repl.co/ to see the English translation over the manga.