9 ms·
Making the Tibetan language a first-class citizen in the digital world
- netmonk 2y ago[dead]
- octopusRex 2y agoEditable Tibetan Language Dictionary which will retrieve definition on search and on copy . created 2011 https://github.com/nekogaijin/DagYigTibetanDictionary https://github.com/nekogaijin/DagYigTibetanDictionary
- java-man 2y agoI want to know the details how they achieved it (the support for super-long paragraphs, or rather, the absence thereof). Does anyone know?
- l1n 2y agohttps://gerrit.libreoffice.org/c/core/+/172801 https://gerrit.libreoffice.org/c/core/+/172801 Pretty short change for reducing O(n^2) impact with a cache. This change includes the following scalability improvements for documents containing extremely large paragraphs: - Reduces the size of layout contexts to account for LF control chars. - Due to typical access patterns while laying out paragraphs, VCL was making O(n^2) calls to vcl::ScriptRun::next(). VCL now uses an existing global LRU cache for script runs, avoiding much of this overhead.
- java-man 2y agoThank you. Also https://bugs.documentfoundation.org/show_bug.cgi?id=92064 https://bugs.documentfoundation.org/show_bug.cgi?id=92064 I lack the context - are they still layong out the widths of characters when wrapping?
- the_mitsuhiko 2y agoProbably shows a bit how little that software is used with Tibetan text if this bug was able to stay open for almost 10 years for what ultimately was a 5 line fix.
- mmooss 2y ago> Probably shows a bit how little that software is used with Tibetan text ... by the LibreOffice devs in Indo-European-speaking countries.
- the_mitsuhiko 2y agoApparently by anyone if the bug description is accurate. Seemingly one cannot open sufficiently long documents let alone write into them.
- mmooss 2y agoPerhaps: from the article: So long as LibreOffice could not handle long paragraphs there was essentially no free tool to publish Tibetan.
- khaled 2y agoThe fix looks like a 5 line fix because it is a last step in a very long process of optimizing LibreOffice text layout that started years ago. This 5 line fix could not have been possible 10 years ago simply because the code it is fixing didn't exist back then.
- buovjaga 2y agoFirst commit from July: https://git.libreoffice.org/core/commit/4c8f88bef948b18f3d810c29a7f83496367758a9 https://git.libreoffice.org/core/commit/4c8f88bef948b18f3d81...
- wslh 2y agoWith all due respect, the innovation side of Tibetans is also appreciated in "The Nine Billion Names of God" [1]. [1] <https://en.wikipedia.org/wiki/The_Nine_Billion_Names_of_God https://en.wikipedia.org/wiki/The_Nine_Billion_Names_of_God>
- asimovfan 2y agoi don't know how it is phrased in the book itself but in Tibetan Buddhism there is no god. And their innovation is far beyond this book (at least the plot summary on wikipedia).
- Reubachi 2y ago"asimovfan", would you say the same type of criticism of Asimov's injection of the Cristian God in "the Last question"? In this story from Clarke, the sentiment you just portrayed is acknowledged, and the point of the whole story. The two westerners cannot understand not only the grand project being undertaken by the monks, but also what "god" even is to them. Before they can really understand, their project is completed and reality changes.
- asimovfan 2y agoAsimovs fantasy god has more in common with the Christian god. In tibetan Buddhism, god in the sense of an almighty individual that has influence over what happens is refuted explicitly.
- Reubachi 2y agoHmm without getting too esoteric and the disclosure that I am not a Bhuddist, your point is one of the inferences the reader can make from the story. -the vendors can't see the point in this 'fruitless' endeavor to build a database of their god's names for some godly/spiritual reason. -The buddhists can't understand and importantly have no need to understand why the vendors need to "understand" their task, which, they never clarify to us. That said, obviousley CLarke probably had no idea about proper bhuddist theology. But I like to hope that the idea is still proper?
- hyperhello 2y agoThis has been in the works for a while. There is an old HyperCard stack to teach Tibetan pronunciation (with 16bit sound) you can try: https://hcsimulator.com/Learn-Tibetan https://hcsimulator.com/Learn-Tibetan
- fsckboy 2y agothe only vowel is AH ?
- cosignal 2y agoThe site seems incomplete. Tibetan does have 5 vowels, and it looks like the non intrinsic vowels are written at the bottom section of the view, but I can't get them to work. I assume the intention would be that you click one of the other vowels to toggle it, but it no worky.
- hyperhello 2y agoI don't know who created it, or if it was part of a larger proto-Duolingo language product.
- shanekandy 2y agoIn text, the singular vowels are built on the ah syllable with modifying marks.
- deleted 2y ago[deleted]
- blackeyeblitzar 2y agoTibet as a land and people should be made a first class citizen of the physical and political world as well. I am not sure why the Free Tibet movement died out in America and Europe, but less than 75 years ago, China forcibly and illegally took over Tibet and since then has been engaged in a forced reeducation and erasure of the people and their culture (Sinicization): https://en.wikipedia.org/wiki/Annexation_of_Tibet_by_the_People%27s_Republic_of_China https://en.wikipedia.org/wiki/Annexation_of_Tibet_by_the_Peo... https://en.wikipedia.org/wiki/Sinicization_of_Tibet https://en.wikipedia.org/wiki/Sinicization_of_Tibet This is one of the great injustices of the world, and unfortunately the Tibetan leader (the real Dalai Lama) had to flee and live in exile in India.
- EA-3167 2y ago> I am not sure why the Free Tibet movement died out in America and Europe Realism. Movements without a realistic goal tend to become quite niche, unless there's another hook to keep them going. With the rise of China as an economic and nuclear power, and one with no interest in even talking about Tibet as something other than a part of their empire, a populist movement to pressure politicians to "Free Tibet" is about as useful as a populist movement to get politicians to conquer the concept of entropy. People, even very passionate and politically active people, moved on to causes with at least some hope, or at least the perception of hope.
- g-b-r 2y agoWhat? From my recollection, when it was decided to admit China in the WTO, every problem was set aside for the asserted belief that growth would result in democracy, human rights and hence the solution of those problems. By the time it became clear that things wouldn't have gone that way, the west had bound itself to China too much to be able to do something about Tibet (without reshoring the manufacturing).
- EA-3167 2y agoEven with reshoring, it's not as though China would just shrug and give up Tibet, and a military confrontation with them over Tibet would be profoundly unpopular in any reasonably foreseeable political landscape. There is no underlying racial animus to drive focus on the issue, as there is with conflicts in the Middle East. For most Americans Tibet is a place they've never been, know precious little about, and while they may feel badly when the situation is explained, it isn't going to weigh heavily on them. It would be hard to explain why that's something to be prioritized over the MENA region, Russia and Ukraine, or places like Mali, Niger, and Sudan.
- skybrian 2y agoApparently "documents have reasonably short paragraphs" should be added to "falsehoods programmers believe about text."
- pbronez 2y agoI never thought about this element of cross language structure before. Text direction, diacritics, punctuation, sure - but I always assumed that chunking was universal. Turns out no: “the typographical notion of the paragraph does not really exist in a Tibetan text the way it does in European languages. As a result, Tibetan texts often need to be processed as a long stream of uninterrupted text with no forced line breaks, sometimes over hundreds or thousands of pages. “
- crazygringo 2y agoTens of pages, sure. But hundreds? Thousands? Do they not have the concepts of headers? Sections? Chapters? Both in non-fiction and fiction, there are a lot more means of content separation than just paragraphs.
- DougMerritt 2y ago"Continued on next scroll"
- Obscurity4340 2y agoIts all about the scrolls, man
- lazide 2y ago‘I don’t have time to make a short letter, so I made a long one instead’ -Mark Twain [https://www.goodreads.com/quotes/21422-i-didn-t-have-time-to-write-a-short-letter-so https://www.goodreads.com/quotes/21422-i-didn-t-have-time-to...] That kind of organization takes time, editing, multiple revisions to get right, etc. And a mindset that it is useful. In many cases (like if you’re a religious caste), having a giant wall of text that requires skill to identify elements from, is a plus. Do you want your ciphertexts formatted into paragraphs too?
- einpoklum 2y agoHey everyone, I'm Eyal, a LibreOffice project volunteer who does a lot of QA regarding Right-to-Left and Complex-Text-Layout scripts (= written languages). I want to thank thunderbong3 for posting a link to that post - and heartily thank Jonathan Clark, the new RTL-CTL-CJK-focused developer at The Document Foundation, who implemented the performance improvement for Tibetan. Most bugs we encounter and report in LibreOffice are more general, and aren't script specific (e.g. code which forgets that the content may be right-to-left resulting in wrong behavior in those cases); and a lot of the script-specific bugs are about the most popular script, which is Arabic (that is also used for Farsi, Urdu, Javanese etc.) But we do have some issues regarding less-commonly-used scripts, like Tibetan or Mongolian. Here: https://bugs.documentfoundation.org/show_bug.cgi?id=115607 https://bugs.documentfoundation.org/show_bug.cgi?id=115607 is the meta-bug which tracks issues with: Mongolian, Tibetan, Uyghur, Zhuang,Kazak, Xibo, Dai, Yi, Miao, Jingpo, Lisu, Lahu, Wa, etc. We don't know if there are really very few issues specific to those languages (which is quite possible), or whether it's just that they're not used so much and the users aren't motivated enough to file bugs. Still, as Jonathan's recent fix demonstrates, there is certainly the interest to address them when developer-time-resources become available. I would like to encourage everyone who cares about these scripts, and "document editing fairness" across countries and cultures, to consider: 1. Try using LibreOffice with such languages which you know at least a little bit of - and if you find any bugs, file them at our BugZilla: https://bugs.documentfoundation.org/ https://bugs.documentfoundation.org/ 2. Consider supproting The Document Foundation, which manages the LibreOffice project, financially: https://www.libreoffice.org/donate/ https://www.libreoffice.org/donate/ We are one of the larger FOSS projects in the world, with tens of Millions of regular users (if not > 100 Million) and a board of trustees with members from dozens of countries; but - we don't have large corporations investing money nor time in the project. While a few commercial companies do contribute to LibreOffice (like Collabora and Allotropia) - many fundamental issues are not close enough to their customers' needs - which is why it was decided to hire Jonathan directly to give RTL-CTL-CJK support a boost. Individual user donations are what enables this work.
- whereistimbo 2y agoI would appreciate you if you supported Dzongkha as well!
- heydenberk 2y agoJim Woolsey, a hippie and early-ish computer hacker from New Hope, Pennsylvania, was an important and early force in the digitization of the Tibetan language. This interview[0] with him from 1993 is a fascinating time capsule, and interesting in its own right. He was a family friend and I always admired his singular commitment to this important and underappreciated work. [0] https://www.mcall.com/1993/10/08/new-hope-man-computer-guru-to-tibetans/ https://www.mcall.com/1993/10/08/new-hope-man-computer-guru-...
- zombot 2y agoToo bad, that link only gives me "This content is not available in your region".
- heydenberk 2y agoIf you doubt the computer’s influence has made its way into every walk of life, you haven’t met New Hope’s Jim Woolsey. For a decade, Woolsey has worked with the Tibetan government-in-exile in Dharmshala, India, to put the Tibetan language on computer. The free-lance computer whiz has compiled a source book of Tibetan literature and also has worked to create a Tibetan computer keyboard for the exiles from that ancient Asian kingdom. In the course of his work, he’s met the Dalai Lama, the spiritual leader of Tibet, and Aung San Suu Kyi, the Burmese political leader since placed under house arrest in that country. Both leaders are recent Nobel Peace Prize winners, the Dalai Lama in 1989, Aung San Suu Kyi in 1991. “If the Tibetan language isn’t put on computers — because of the fact that there are fewer than 1,000 Tibetan typewriters in the world and they’re more expensive than computers –the Tibetan language might not be saved from being put on the shelf with all those other dusty, musty languages of the scholars,” said Woolsey. “This is its only hope.” Woolsey’s work with the Tibetan Buddhist government-in-exile in India began with an interest in Tibetan literature. Formerly a technician with various rock’n’roll groups in the 1970s, he would read anything he could lay his hands on concerning Tibet, then enter the titles of the books in a bibliography he kept. He was traveling both for work and pleasure, and decided it was time to journey to one of the farthest corners of the globe. “I booked a 120-day round-trip ticket to India,” he said. “I threw on my backpack and went to India. I was coming in from the airport in New Delhi at 3 o’clock in the morning and passed a camel pulling a cart down the street. I said, ‘We’re not in Kansas anymore.'” He saw a sign for Tibet House in Dharmshala and decided to go there, even though he had no idea what Dharmshala was like or what awaited him there. His first trip to the Tibetan exiles’ home was a short one, and he later traveled to Kashmir, Darjeeling and Nepal. By the time he returned to Dharmshala in 1983, the Tibetans knew him. “Before I left, the Office of Tibet in New York asked me if they could have a copy of the notes that I had been keeping on my computer about Tibetan studies, mainly my reading list,” he explained. “I looked at it and it was a mess. I thought I had better clean it up. I wrote a couple computer programs to make it an organized matter. I printed it up and gave them a copy.” A friend who was learning word processing wanted a copy and Woolsey also gave him one. “He sent a copy to the Dalai Lama,” said Woolsey. “The Dalai Lama must have figured it was going to be published, so he wrote a forward to it. “By the time I got back to the library in Dharmshala, I didn’t know anything about this. I got to the Western reference section and they said to me, ‘Oh, we’ve been wanting to meet you.'” They asked Woolsey at the library what his background was and he answered rock’n’roll. They asked who his teacher was and he told them he didn’t have one. They asked if he was a Tibetan Buddhist and he said no. They asked if he wanted to become one, and he again answered no. “I was raised a Quaker, and that was close enough,” he said. “They meditate, but they don’t call it that.” At the Tibetans’ request, Woolsey settled down to begin work organizing by computer the chaos that was the Dharmshala Library. Realizing the power of the age of information had fallen into their laps, the Tibetans decided he was to be their computer guru, and designated him as such. They told him he could consider them his affiliation in the academic world. The chaos inflicted on the Tibetans by the Chinese invasion of the late 1950s had not yet been alleviated. Books and manuscripts lay in unsorted piles in the library, so Woolsey’s computer was the perfect tool to help put things in order. “Later on, in 1984, they sent me a list of letters to all the high lamas in the United States, from the director of the library, telling them that I was their computer guy, and would they please aid and abet me in my endeavors,” said Woolsey. “Of course, they sent them the letters before they sent me one asking me if I wanted to do it, which makes it a little strange.” Woolsey returned to India several times at the invitation of the Tibetans. He had discovered in the United States that no one was working on computerizing the Tibetan language with much interest. By 1985, he was acting as the consultant to the library in developing the language on computer. Once he was given the assignment, he was besieged with students, one of whom was the abbot of the Mahayana Buddhist Temple in St. Petersburg, Russia. Tenzing Samaev was visiting Dharmshala in 1990, and Woolsey had arrived just after Tibetan New Year. “This Tibetan brought over a monk and said ‘He wants to know something about computers,'” said Woolsey. “I said, ‘OK.’ I answered his question.” The monk returned after the New Year with two more questions, and then the next day with two more, and then two more the next morning, and two more by that noon. This went on for four days. “Tenzing, in the true Tibetan tradition, formally presented himself and requested me to become his teacher, to accept him as a student,” said Woolsey. The abbot came to this country in 1991 and went home with a computer and laser printer. With Woolsey’s help he set up the computer to work in Cyrillic, the alphabet used in Russia, and is now publishing the temple’s newsletters and other proclamations on it. Norbu Chompel, director of book sales for the Office of Tibet in New York City, said, “Jim has done quite a lot. He’s the main person responsible for introducing computers to the Tibetan administration … He came with a lap-top and talked computers to several staff members. That’s how computers came. “Before, we used to use typewriters,” Chompel said. “He taught computers, and then everybody got into buying computers.” With all the work he was doing for the Tibetans becoming known, perhaps it was inevitable that the Dalai Lama take more notice of him. The Tibetan spiritual leader wanted to know what was going on with the development of Tibetan on computer, Woolsey said. “A couple years ago, I was given the opportunity to brief the Dalai Lama about what’s going on,” said Woolsey. “We had some interesting conversation, but I feel that he’s got better things to do.” The Dalai Lama had the pursuits of freeing his country from Chinese domination and leading his people in their exile as more pressing problems. Aung San Suu Kyi also had more important pursuits to consider. Woolsey met her in Dharmshala, prior to her house arrest in Burma (now officially called the Union of Myanmar) as a political dissident. She’s been detained by the Burmese government for the last four years because of her political activities and her great personal power. Her father, Aung San, founded modern Burma and was assassinated in 1947. She has followed in his footsteps in an attempt to free her people from military rule. Woolsey recounted an incident at a rally in which Aung San Suu Kyi prevented a slaughter by the army of an unarmed crowd of 20,000 people. The army approached to smash the rally, Aung San Suu Kyi positioned herself between the crowd and the soldiers and halted the military with her words. “She told 20,000 people to sit down and be quiet and they sat down and were quiet,” said Woolsey. “She out-positioned the army and did it non-violently. That’s the key, non-violence. “She’s a very, very learned person,” continued Woolsey. “She really has the rights of her people in her heart more than worries about herself.” As is the case with Woolsey and the Tibetans. Woolsey’s source book of Tibetan literature is under consideration for Internet, the international computer-user network. With his help, Tibetan might go from being an endangered language to one available to everyone who can hook up a computer to a phone line. And that might bring an ancient kingdom into today’s electronic age. “I feel that you should be able to leapfrog over the industrial age into the information age as an agricultural society, and perhaps be farther ahead than where we in the West are trying to get to,” said Woolsey. Push a few computer keys and it might happen. Originally Published: October 8, 1993 at 4:00 a.m.
- xmly 2y agoThere are over 50 tibetic languages, which one do you choose?
- teractiveodular 2y agoThere is only one (modern) written Tibetan script.
- whereistimbo 2y agoHow about Dzongkha?
- teractiveodular 2y agoIt also uses Tibetan script.
- mmooss 2y agoThis person doesn't seem to think it's supported: https://news.ycombinator.com/item?id=41834052 https://news.ycombinator.com/item?id=41834052
- deleted 2y ago[deleted]
- soheil 2y ago[flagged]
- vinay427 2y agoThis is a well-established phrase in computer science and programming languages, and it’s likely that its use here is meant to be evocative of those principles rather than of an anthropomorphic sense. https://en.wikipedia.org/wiki/First-class_citizen https://en.wikipedia.org/wiki/First-class_citizen
- soheil 2y agoWell let's not use it anymore if I agree on principle, computer science itself is a new field, so not sure how deeply entrenched it really is.
- scooke 2y agoCan you give an example or two of the type of terminology you have in mind? Something like first tier, or high level, or primary, or what? Almost any term that I could think of still carries with it a sense of politics. Perhaps someone from the first class citizen language, like English or French doesn’t really notice this dynamic, but someone who uses these other languages sure notice the lack of importance, or urgency, when they can’t live digitally with their own language. Whether or not they use the term,” political”, it’s still there. But I’m curious to see what your suggestions might include.
- soheil 2y agoIntegral Compatible Mainstream
- layer8 2y agoIt is very well-established and commonly used. https://en.wikipedia.org/wiki/First-class_citizen https://en.wikipedia.org/wiki/First-class_citizen https://www.google.com/search?q=%22first-class+citizen%22+site:stackexchange.com&tbs=li:1 https://www.google.com/search?q=%22first-class+citizen%22+si... https://www.google.com/search?q=%22first-class+citizen%22+site:www.reddit.com&tbs=li:1 https://www.google.com/search?q=%22first-class+citizen%22+si... People don’t perceive it as a political term.
- stahorn 2y ago"... relatively short paragraphs (possibly up to a few pages)" I love things like this that just shows me how much I view the world from a certain perspective. I don't think I've ever had a paragraph even on one page! The closest I know is that some writer, that I forgot the name of, had several pages of stream of consciousness that I think was without paragraphs and punctuations.
- cnity 2y agoModernist and post-modernist writers are known for this (James Joyce and David Foster Wallace, for example).
- ron_k 2y agoBut also Saramago, Gadda, García Márquez, or Victor Hugo and his 800+ words sentence. Stream of consciousness usually has long paragraphs, but it’s quite common in other genres (I feel like genres is not the right word for it. Techniques?). Popular fiction is the only one that usually avoids long sentences/paragraphs/chapters.
- jackbravo 2y agoSaramago is a great example. He writes in Portuguese that is easily and closely translated to Spanish which is my native language. When I first read Saramago it was really difficult to get used to his long sentences with only commas and no dots. But after a while it starts flowing.
- InDubioProRubio 2y agoLanguages, are very interesting beasts. As in, they are easy to learn and marshal communication across large swaths of the world- or they are hard to master and allow to construct very complex constructs and ideas- which are then transported from one speaker to another. In which part of the field does the Tibetan language fall?
- somat 2y agoI am not sure the two are mutually exclusive. I mean, a highly technical subject will have highly technical jargon. but I am not convinced that the amount of subtle nuance in the language has a relationship to the complexity of ideas you can express in that language.
- zokier 2y agoIt's bit surprising that word processors would struggle with long paragraphs considering that various stream-of-consciousness and related styles of writing also eschew paragraphs and possibly other conventional structures. They might not be super common, but not exactly unheard of either. I'd assume writers and publishers manage them somehow. Random recent example: https://en.wikipedia.org/wiki/Ducks,_Newburyport https://en.wikipedia.org/wiki/Ducks,_Newburyport
- hoseja 2y agoAFAIU there are no spaces either.
- mihaic 2y agoHonestly, given the particularities of Tibetan, I'm surprised it didn't adapt to the digital world. So while I'd congratulate the LibreOffice developers for improving their software, as least in displaying legacy text, I would expect Tibetan to evolve just like almost any writing system has done over time, and introduce some spaces and paragraphs.
- emilamlom 2y agoAn actively used written script is not "legacy text". We're not talking about some ancient, dead language, but even if it was that, there's merit in being able to accurately display it digitally just from an academic standpoint of making historical research easier.
- mihaic 2y agoSure, I understand the merit of it being displayed properly, that's something worthwhile. But at the same time, if it's an actively used script, don't you think current users should consider changing its conventions? My point was that all writing systems change, especially when the medium changes. Modern punctuation was invented at some point simply because the previous form of writing words one glued to the other was not ideal.
- nottorp 2y agoFrankly when I clicked I expected to read that the Tibetan language has been recently added to Unicode. I mean, they're too busy adding emojis over there to work on support for human scripts any more...
- mmooss 2y ago> I mean, they're too busy adding emojis over there to work on support for human scripts any more... What makes you say that there are human scripts being left out, or which nobody is working on (or that the work is displaced by emoji support)? One challenge of large projects is that what's essential for some users is not even within the experience of other users. Sometimes humans make the fundemental error of thinking the range of their own experience defines the range of everyone's experiences, when the truth is that each of us sees only a tiny portion of an enormous canvas.
- qingcharles 2y agoThey just added Pac-Man and some other video game characters too: https://news.ycombinator.com/item?id=41535317 https://news.ycombinator.com/item?id=41535317
- nottorp 2y agoAren’t they exposing font users to legal liability this way? I don’t think anything from the gaming industry is out of copyright yet…
- qingcharles 2y agoThey've renamed Pac-Man to "BLACK LARGE CIRCLE MINUS RIGHT QUARTER SECTION" (really!).
- nottorp 2y agoHmm it's Bandai-Namco. Haven't heard about them sending black helicopters to authors of pac man implementations so maybe it's safe. Did they add anything from Atari to Unicode? Wikipedia says the focus of the current name owner is "licensing and blockchain".
- lappet 2y agoI know Bengali and Assamese use a Tibetan script, anyone know how similar is it to the one Tibetans use for their language?
- abe94 2y agoWhile the scripts (Bengali/Assamese and Tibetan) both evolved from the Gupta script, the actual languages are very different. Bengali and Assamese are Indo-Aryan, while Tibetan is from a completely different language family (Sino-Tibetan). When I (bengali speaker) visited Bhutan where they speak a language that is 50% mutually intelligble with Tibetan I didn't understand anything. I was surprised because I thought they might use a number of buddhist loan words, but even the words for dharma, karma, etc. sound completely different in tibetan
- buovjaga 2y agoFor reference: https://en.wikipedia.org/wiki/Bengali%E2%80%93Assamese_script https://en.wikipedia.org/wiki/Bengali%E2%80%93Assamese_scrip... https://en.wikipedia.org/wiki/Tibetan_script https://en.wikipedia.org/wiki/Tibetan_script