11 ms·

The Death of the Urdu Script

316 points by aarghh 12y ago

scrollaway 12y ago
That was an amazing article, and a great insight into middle eastern scripts in the digital world. It doesn't look like it'll get out of the new queue, but I wanted to thank OP for sharing it anyway.
- contingencies 12y ago
  Pro tip: Mughul India is not the Middle East, and neither is Pakistan or Iran. There are arabic derived scripts at least as far east as China and the Philippines (jawi), but describing them as Middle Eastern is plain wrong - just as you would not describe Balinese or Cambodian as Indus Valley.
  - keypusher 12y ago
    According to Wikipedia, Iran is in the Middle East. http://en.wikipedia.org/wiki/Middle_East#Traditional_definition_of_the_Middle_East http://en.wikipedia.org/wiki/Middle_East#Traditional_definit...
    - contingencies 12y ago
      In modern times things are very vague. For example, according to Wikipedia, it's also part of Central Asia https://en.wikipedia.org/wiki/Central_Asia https://en.wikipedia.org/wiki/Central_Asia and even South Asia https://en.wikipedia.org/wiki/South_asia https://en.wikipedia.org/wiki/South_asia
  - scrollaway 12y ago
    The article talks about two arabic scripts. Saying "Middle eastern" is more inclusive than saying "arabic" and while it does not include pakistan, it does almost reach it. Kind of like when we talk about Europe and increase the reach a bit more. Or when you say "the United States" when you really mean north america (a much worse offender since you talk about a country). This was my logic when writing it. I'm not a history expert. I would love a history lesson but you seem like a very poor teacher, "picking" on people who just try to be nice. I did not know this article was going to be on the frontpage as I said in my original comment - I caught it at 2 points. Had I known my comment was going to go through a seven-way review by a wide variety of experts, I'd have just said "non-latin scripts" to be on the safe side. (/s) Sorry for the rant - People correcting each others where correction does not at all matter bothers me to no end. It feels like a sneaky and thoroughly annoying way of flashing your "I know THINGS!" credentials.
    - contingencies 12y ago
      You seem to be admitting mistake, complaining about being corrected, and dismissing the process all at once. But I'm sure you learned something :)
  - cabalamat 12y ago
    > There are arabic derived scripts at least as far east as China and the Philippines (jawi), but describing them as Middle Eastern is plain wrong Is it, though? I'd describe the Roman script as European (since it originated in Europe) without expecting to be corrected, even though the vast majority of people who use it live outside Europe.
    - contingencies 12y ago
      Would you describe a page of written Vietnamese or Hanyu Pinyin as European? That's a fairer comparison to the author's turn of phrase.
      - scrollaway 12y ago
        You seem to be trying very hard to look very smart, yet you (probably purposefully) seem to be missing the point of everyone you reply to. That is not really making you look "smart".
      - cabalamat 12y ago
        I would describe the scripts they're written in as European. The languages themselves aren't, obviously.
ximeng 12y ago
Unsurprising to see a link to Michael Kaplan in there. His blog was taken down due to what sounds like a dispute over patents with his boss. See this twitter conversation between Spolsky and him: https://twitter.com/spolsky/status/447024470256283648 https://twitter.com/spolsky/status/447024470256283648. -- "It has been made officially clear to me that [my] Blog ... is for all intents and purposes dead." Well, I blogged about a patent and included art, so my former manager took the Blog down with extreme prejudice. -- Blog is now archived at http://www.siao2.com/ http://www.siao2.com/ and has much more information on localisation.
- thejdude 12y ago
  Wat? I thought the whole point of patents is to make information public, because competitors are simply not allowed to use the information without payment anyway.
alieteraz 12y ago
This is Ali Eteraz. Thank you for recognizing this piece. It was a lot of fun to write. I've actually been to a Ycombinator event. Twice I think. A couple of your alumnus managed to talk me into downloading stuff or getting on email lists that, well, I didn't always use (though they did seem promising!). I am a writer based in the Bay Area and I really enjoy getting to learn about interesting technological developments and human stories within that. So definitely hit me up whenever you're congregating or plotting something interesting. My website is alieteraz.com or I am at @eteraz or FB. Ali Eteraz ps - I later got to meet Michael K. at the Unicode conference in Santa Clara, along with the Microsoft Windows team. The Windows Phone people, who I also really wanted to talk to because this is a mobile problem too, did not want to talk to me because they had a new phone coming out.
mchaver 12y ago
For web development, perhaps you could create a simple system that detects if the user's device supports nastaliq. If not convert, it to naskh or Romanized Urdu (not sure which is preferred). As for dealing with all the data already written in naskh and Romanized Urdu, this is a bit more complicated because I am guessing there is not a one to one match between naskh and nastaliq (naskh has less letters), and Romanized Urdu and nastaliq. You would probably need to collect some data and use some machine learning techniques. If it is not too big you could use a client-side Javascript program, otherwise something like a Chrome extension or an API. There are similar things done for Arabic to handle various forms of Romanized Arabic (user does not have access to an Arabic keyboard) and Non-Modern Standard Arabic Dialects: http://www.yamli.com/ http://www.yamli.com/
- abdullahkhalids 12y ago
  Data is stored as Unicode (except some really old desktop app formats that were written before Unicode). Naskh and Nastaliq are font styles, used when the system displays the data. The solution is simply that good Nastaliq fonts be created and then used on websites where Urdu is being used. Since Urdu fonts can be much larger in size than Roman fonts (the fonts have to specify joining behavior between all possible combination of characters), delivering over the web is not really viable. Fonts have to be native to all the OSes in existence, so that websites can use them. (I don't think Roman Urdu is ever used for anything serious.)
  - mchaver 12y ago
    The author of the article mentions that Naskh does not contain all the letters that Nastaliq has, so if someone types an Urdu text in Naskh it would not have the same underlying Unicode if they had typed it in Nastaliq. Correct me if this is wrong. My assumption is that not all OSes will adopt Nastaliq simultaneously. So assume that the website is storing data in complete coding for Nastaliq and they want to send it to a device that cannot render Nastaliq so the Unicode should be converted so it is properly renderable in Naskh. The other half is just to make one's Urdu language web experience completely in Nastaliq. Even unimportant stuff written in Roman Urdu and then converted to Nastaliq might help promote the use of Nastaliq.
abdullahkhalids 12y ago
There is another problem with fonts. Popular fonts used on the internet only have information about the joining of the 28 Arabic characters. When the extra characters in Urdu are typed they don't join with the other characters in the word. eg. I will type the first line of the couplet here. Note how the third,fourth and seventh word are broken up. اور باذار سے لے آے اگر ٹوٹ گیا I have written about some other issues with the Urdu language over here. http://upgoerurdu.nfshost.com/technical.html http://upgoerurdu.nfshost.com/technical.html
jey 12y ago
> Utility had defeated tradition. Is that supposed to be self-evidently bad?
- adamnemecek 12y ago
  Indeed. I feel like languages/scripts dying out has some advantages too.
  - alieteraz 12y ago
    Languages do evolve and die. English itself is changing all the time. Same goes for Urdu. The question here is whether the changes are organic or is there simple oversight going on. My view is that the problem is of oversight, and of disregarding the Urdu speaking and writing user. If the world's most popular operating systems were designed in Pakistan or Iran and they were not offering enough English language letters as a default on the devices, I think we would be plenty upset.
    - adamnemecek 12y ago
      The question is whether languages are really worth preserving. I'm not a native English speaker yet I don't really have much emotional attachment to my first language. In fact, my life would have been easier if I grew up speaking English. And it's not that I think that English is somehow inherently a better language, it's more that there is a lot more people speaking it. It's kind of like if you are deciding what open source project to use. Having a big community around it is a major deciding factor, if not THE deciding factor.
      - ibrahima 12y ago
        I feel much the same way (though, considering I learned English at the age of 4 and never really learned much more of my parents' language after that, calling English my second language is grossly misleading). I feel a vague attachment just because my grandfather was a professor of Bengali literature and I'd like to be able to read his work, but I don't sweat over the fact that my grasp of the language is very basic. I don't need to use it too often, and people back in the "motherland" can worry about preserving the language or not. The analogy to open source projects is spot on; while you might have some "emotional" attachment to a library/technology that you learned early on, you probably wouldn't make significant decisions based on that. I learned to program in VB5, yet I dropped it as soon as I learned Java because it was just an awful language (though I guess human languages don't really have notions of superiority or inferiority like some programming languages might).
gioele 12y ago
Meanwhile, in the free software world... https://docs.google.com/presentation/d/1ySTZaXP5XKFg0OpmHZM00v5b17GSr3ojnzJekl4U8qI/view#slide=id.g177a94b_0_60 https://docs.google.com/presentation/d/1ySTZaXP5XKFg0OpmHZM0... https://github.com/behdad/harfbuzz/blob/master/test/shaping/texts/in-tree/shaper-arabic/script-arabic/language-urdu/crulp/ligatures/README https://github.com/behdad/harfbuzz/blob/master/test/shaping/... (Harfbuzz is the default text layout engine for Linux GUI frameworks: Qt, GTK, etc)
- abdullahkhalids 12y ago
  This talk at Google by Roozbeh Pournader is very good at laying out the issues with bidirectional language issues in software. This is a non-trivial problem and is rarely ever solved correctly. https://www.youtube.com/watch?v=wOEzYefrqo4 https://www.youtube.com/watch?v=wOEzYefrqo4
  - 616c 12y ago
    And not so ironically, Behdad Esfahbod, one of the original developers of HarfBuzz, works for Google now, according to his blurb on Github. https://github.com/behdad https://github.com/behdad I actually emailed him a few times before I discovered mlterm to read Arabic and other RTL fonts in a terminal (you might ask why: file names and I use mutt for email). Very nice guy, and I am very appreciative of this library.
elandybarr 12y ago
The article is great, but I don't think it adequately captures one of the difficulties with this problem, and that is the very high minimum number of characters that must be designed for an Urdu font. I think it is an order of magnitude higher than many would guess it at first.
- jpatokal 12y ago
  The Wikipedia article on Nastaliq puts the number at around 20,000. https://en.wikipedia.org/wiki/Nasta%CA%BFl%C4%ABq_script https://en.wikipedia.org/wiki/Nasta%CA%BFl%C4%ABq_script
alieteraz 12y ago
Please correct me if I'm wrong, but if Apple and Android make a nastaliq font part of the directory of fonts that go into their devices from the get go, there wouldn't be a problem. Such fonts have already been designed. The issue now is to find Android and iOS developers who can make them default in the operating systems. When I talked to Microsoft they said that on the smaller devices every last bit of memory was precious so they didn't want to stick extra megabytes or however big a font is. I think that Apple's cheapening out is worse than Android's, because at least Android offers the entire Urdu alphabet (though in naskh). Apple doesn't even offer the Urdu alphabet, requiring Urdu users to have to make do with 12 less letters.
- paperwork 12y ago
  I don't think it is just a matter of adding a font, with 12 additional letter for urdu. The actual layout algorithm has to change (how characters are laid out, their size, how beginning of one connects to ending of another, etc.).
wirrbel 12y ago
its an interesting read and I acknowledge the intent to keep the traditional and authentic writing system for Urdu. I have a few thoughts on this though. One would be the legibility. Of course I cannot judge legibility of writing systems I do not like, but it seems that nastaliq would be hardly readable on a lot of mobile devices and I wonder how difficult learning the ornate script is. I am talking about alphabetization here. Next thing is: I am learning turkish and turkish is written in the roman alphabet. As far as I know, it was written in a arabic/persian script before which was then reformed to use the latin alphabet. As far as I can tell this is today really uncontroversial and using the latin alphabet is actually the more suitable alphabet for turkish and its rich vowel system that is really important for grammar and meaning. Again I cannot really say anything for Urdu, but knowing it is not arabic but afaik a language of the indo-european family I wonder if there are more reasons to use the lating alphabet than just availability of nataliq fonts and rendering engines. As a side note I would add a few observations relating to the cultural/heritage aspects. In Germany, the "Fraktur" was used widely even at the beginning of the 20th century (see http://en.wikipedia.org/wiki/Fraktur http://en.wikipedia.org/wiki/Fraktur ). Some authors, like Hermann Hesse refused that Antiqua fonts would be used for their writings until publishers convinced them that their works could just not be read by young folks. In a way, a lot of people argued against using non-gothic fonts, but in the end antiqua became quite standard. Nowadays we use the lating alphabet (and most people are not concious about that there ever was a switch).
- alieteraz 12y ago
  Great point about German language. I have come across that again and again. And you're right, there ARE Urdu speaking youth now who are basically just as comfortable in naskh as they are in nastaliq, if not more so. This was a revelation to me and it added a wrinkle to my thinking. But I go back to my point, that we can't fully adjudge if nastaliq is meant to die off until we make at least a good faith effort to get it onto the devices.
  - wirrbel 12y ago
    What makes me wonder is, I would have thought that the prospects of nastaliq are increasing with electronic devices (as opposed to old-school printing presses). What I also observe is, that arab speakers sometimes write with the latin alphabet using characters like "3" do represent some missing phonemes. Urdu surely is not alone, and having a proper nastaliq rendering on all devices might not solve the bigger issue here. Could you elaborate a bit on nastaliq and how/whether it does represent Urdu's phonemes properly? Are people considering Devangari as an alternative?
gauravk 12y ago
Sad. I remember as a kid, growing up in Kashmir, I took special pride in my Urdu calligraphy skills. Writing each sentence was like an art project.
616c 12y ago
So I skimmed the article once, then read it again. I see the writer is here. I just want to say I understand your sentiment, but your title and content are filled with some annoying misinformation and discrepancies. First of all, Nastaliq is not "the Urdu script". If you even skimmed Wikipedia, you would notice that the script does come oringinally from Arabic (where it is used, but more to your point, sadly not as much in print media, at least from my experience in the GCC) and was later well loved by the Persians, and later different portions of the Indian subcontinent (India, Pakistan, and beyond). To call it Urdu script, just because it is popular, and to insinuate that it Naksh is the default script of Arabic (it is not the preferred one, by the way) is a huge stretch. Another point: Naksh was invented by a Persian, but who grew up his whole life in Arab Baghdad and was an Abbasid court vizier in modern-day Iraq. He did not spend time in Iran (then Fars, as it is known today in Arabic), until he was a tax collector there after growing up in Iraq, and only for a period. He might have been ethnically Persian, but to call Naksh script Persian because of the guys family background is far-fetched, especially if you know how loose the cultural boundaries between Abbasid Iran-Iraq were (they were part of the empire and people, goods, and ideas freely flowed between them). It is a derivative of Kufic script, and has strong ties to the religious intellectual history of Iraq (Basra and Kufa were the seat of a massive amount of religious scholarship and still prominent beliefs in Islam; scripts and art were an awesome by-product). Sorry for the rant, but I know this is probably not your field. As I guy who spent a lot of time studying Arabic and Arab and/or Islamichistory, I get very annoyed at the misinformation peddled by people about Arabic history and culture. I do not mean to be so blunt, but I hear people rattle off misinformation like this often, and it irritates me.
- alieteraz 12y ago
  You are correct about the history. Unfortunately the Abbasid empire came to an end in 1258 AD and Urdu wasn't even around then. I take your point that naskh has been wide-spread. And it is true that it is even in the Indian sub-continent. Sindhi, for example, is a naskh based language. Punjabi, which my parents spoke, on the other hand, prefers nastaliq, and regionally the two places are adjacet. That doesn't mean that those people didn't historically play around with scripts. They did. However, my piece is entirely about what is going on today. Sure you can find Ottoman era signs in Egypt and the Levant that display Arabic in nastasliq (my readers sent me plenty such pictures), but by and large Arabic today is written in naskh and almost never in nastaliq. Take a look at some of the fonts that the Omani government is playing around with. They are not nastaliq. This is the political aspect I briefly touched upon in the article. I hope you will get a chance to look into the Arabization aspects of the political debates raging in that part of the world. I make brief mention of it by bringing up the fight over "Khuda Hafiz" or "Allah Hafiz." Meanwhile, the past two or three generations of Urdu readers and writers grew up associating nastaliq and Urdu almost exclusively with each other (thus the jarring effect associated with having to read Urdu in something else). I have plenty of emails testifying to this from people from my parents generation and some people of mine. In short, no one is really talking about ancient Arab history here, as fascinating as that would be.
616c 12y ago
And for the Arch Linux peopkle, have a Nastaliq font if you need it (or want it). https://aur.archlinux.org/packages/ttf-nastaliq/ https://aur.archlinux.org/packages/ttf-nastaliq/
sharjeel 12y ago
This article was making rounds in my social feed about a year ago. Disturbed by the extremely weak and some absolutely flawed arguments, I wrote a note: https://www.facebook.com/sharjeelqureshi/posts/10151806998363551 https://www.facebook.com/sharjeelqureshi/posts/1015180699836...
mellisarob 12y ago
you need to come up with more details and authentic details because some stuff you mentioned it utter crap.
Rapzid 12y ago
I don't like how the article opens with the premise of a people struggling their whole lives to master writing in a particular style only to have their expression oppressed, then proceeds to talk about font rendering.. Generally though, I'm not really sure how I feel about trying to get people to care about something so precious in general. One one hand it's a shame to lose culture. On the other the internet/technology has its own culture and is it really lost in the age of 5c per GB storage?
kranner 12y ago
For Hindi speakers who don't read Nastaliq, the sign on the wall is hilarious. Transliterated to Devanagari, it reads: हाय मैं मर गई ऐन्ना (इतना) टेस्टी बर्गर
merrua 12y ago
Nastaliq is lovely to look at.
alieteraz 12y ago
BTW. I published this piece in fall of last year. Long before it came on Hacker News, it had been passed around the Pakistani and Indian Twittersphere, and no one there was saying that we should let tech companies off the hook for not offering a) the full alphabet and b) the right script. It was, actually, the Pakistanis and Indians that made this article what it was. When I wrote it, I honestly thought I was just some eccentric who was annoyed for very personal reasons. Even as we speak the piece is being retweeted by professors and students at LUMS, in Lahore, and universities in Islamabad. What's even more interesting is that as a result of writing this piece my own favoritism towards nastaliq actually LESSENED. But by the time my thinking evolved, the nastaliq purists had made the piece their own.
paperwork 12y ago
A big reason nastaliq is not widely available on our screens is because rendering it correctly requires 'context sensitive shape substitution'[1]. Simple substitution of one character for another is not enough. I've spent a couple of hours looking at how to render my own text, perhaps using something like http://typeface.neocracy.org/ http://typeface.neocracy.org/, but my day to day work is so far from text layout, rendering, client side javascript that it would be (too) large an undertaking. [1] http://ww.cle.org.pk/Publication/papers/2006/context_sensitive_shape_substitution.pdf http://ww.cle.org.pk/Publication/papers/2006/context_sensiti... [PDF]
wmil 12y ago
It seems like something analogous has happened with European scripts. There were huge changes in how characters were presented between different styles of handwriting, engraving, printing presses, and finally everything the computer world has brought us. It just seems like the author has an extreme emotional attachment to a typeface.
- paperwork 12y ago
  I don't know about the author's state of mind, but generally speaking, this is not just a matter of preference. For those of us who grew up reading Nastaliq, other type faces such as naksh are fairly difficult to read 'fluently.' Imagine if all books, newspapers websites were hand written by someone with poor handwriting. It would actually slow down your reading speed. Reading is such a natural thing we do on a daily basis that we notice even small things which cause us to slow down. btw, I use 'bad handwriting' simply as a way to explain the effect of different writing style on reading, naksh is not good or bad -- it is just different from what Urdu readers are used to. Also, nastaliq isn't just different because its letters are formed differently (there being a one-to-one correspondence between different typefaces). Nastaliq letters are laid out in a context sensitive manner where a letter can take on a fairly large number of shapes, depending on where in the word it is (and which letter it is next to). My earlier post links to a paper which describes how Nataliq rendering requires context sensitive shapes. Basically randomly capitalize letters in everything you read online. Eventually you will get used to it, but that doesn't mean it won't be annoying for a long time.
GhotiFish 12y ago
>Utility had defeated tradition. There is hope for humanity yet!
m0th87 12y ago
That's not the only way in which Urdu is "dying" - due to colonization, basically all modern words are in English.
thret 12y ago
This feels like unusual timing. I am presently reading 'A Suitable Boy' by Vikram Seth. One of the main characters is sent into virtual exile where he learns how to read and write Urdu. Until now I don't think I'd heard of it. My own opinion is that we should preserve history, remember it, but not morn it. We are programmers! How many writing styles and languages have we seen die to make way for something better (or worse?). This is a natural survival of the fittest cycle that effects writing styles, languages, civilisations and everything else. I have Japanese friends who read even less Kanji than I do. I see that writing style dying out over the next 50 years also.
sandGorgon 12y ago
@alieteraz - your comments below have been extremely informative and though urdu occupies the collective conscious of all Indians through bollywood (90% of melancholic melodies are invariably in urdu), we have never given the script much thought. I think you fought admirably with Apple, Twitter and Microsoft for getting Urdu fonts included ... but that is the wrong battleground. The arena for you is simply the browser and mobile . All you need to do is get Nastaliq scripts adopted into Google Webfonts (free for anybody to use in their webapps) and into Android (installing a font through a Launcher theme is incredibly easy [1]. Do NOT try getting it included in the Android base) do you think you can get together a Kickstarter to fund a nastaliq font, good enough to be usable ? and then try getting them into webfonts and android themes ? [1] https://play.google.com/store/apps/details?id=pete.app.apexthemes.font.chokycooky&hl=en https://play.google.com/store/apps/details?id=pete.app.apext... and http://appcrawlr.com/android-apps/best-apps-launcher-fonts http://appcrawlr.com/android-apps/best-apps-launcher-fonts
capex 12y ago
Anyone ready to work with me on creating a Nastaliq webfont?
Ghalib5423 12y ago
In the 1960s, the Government of Pakistan tried to make naskh common in order to facilitate mechanization of printing. In order to do so, naskh was introduced in schools so that children get used to it from an early age. Some books started appearing in naskh. The daily newspaper Nawa-i-waqt started printing its second page in naskh. It was very unfortunate for Urdu that these efforts failed. The reason was simple. Instead of requiring that the textbooks be typeset in naskh, they continued to be handwritten, albeit in naskh-like script. Gradually after a few years that naskh started resembling nasta’aliq more and more and eventually all traces of naskh disappeared. This, to mind, was the greatest setback suffered by the Urdu language printing. Let me explain why I call it the greatest setback. Urdu books and newspapers were written by hand and then lithographed. The process was slow, mistakes were plentiful, dots above and below letters were misplaced, letters such as daal and waaw were indistinct from each other, and many times the printing was illegible. Please try to read a book produced by that method, and then compare it to the same book typeset in naskh; you will note the difference. I have a copy of Divan-e-Hafiz published in nasta’aliq in the subcontinent, and another copy in naskh published in Iran. The difference in the clarity of text in the two is phenomenal. This abortive attempt in 1960s to switch to naskh had nothing to do with religion or Arabization. Ayub Khan, in whose time the effort started, can hardly be accused of religion-inspired initiatives. It was simply to promote mechanized printing. If anything, later the religion-enthusiast Zia-ul-Haq did nothing to popularize naskh. Ironically, computer has saved nasta’aliq, since software is now commonly available which is now universally used to compose material for printing. But naskh is still clearer to read, and in my opinion it is not too late to give it another try, and use naskh, while reserving nasta’aliq for calligraphy. Undoubtedly nasta’aliq is elegant and no script can match its beauty.

316 points by aarghh 12y ago

That was an amazing article, and a great insight into middle eastern scripts in the digital world. It doesn't look like it'll get out of the new queue, but I wanted to thank OP for sharing it anyway.

contingencies 12y ago
Pro tip: Mughul India is not the Middle East, and neither is Pakistan or Iran. There are arabic derived scripts at least as far east as China and the Philippines (jawi), but describing them as Middle Eastern is plain wrong - just as you would not describe Balinese or Cambodian as Indus Valley.
- keypusher 12y ago
  According to Wikipedia, Iran is in the Middle East. http://en.wikipedia.org/wiki/Middle_East#Traditional_definition_of_the_Middle_East http://en.wikipedia.org/wiki/Middle_East#Traditional_definit...
  - contingencies 12y ago
    In modern times things are very vague. For example, according to Wikipedia, it's also part of Central Asia https://en.wikipedia.org/wiki/Central_Asia https://en.wikipedia.org/wiki/Central_Asia and even South Asia https://en.wikipedia.org/wiki/South_asia https://en.wikipedia.org/wiki/South_asia
- scrollaway 12y ago
  The article talks about two arabic scripts. Saying "Middle eastern" is more inclusive than saying "arabic" and while it does not include pakistan, it does almost reach it. Kind of like when we talk about Europe and increase the reach a bit more. Or when you say "the United States" when you really mean north america (a much worse offender since you talk about a country). This was my logic when writing it. I'm not a history expert. I would love a history lesson but you seem like a very poor teacher, "picking" on people who just try to be nice. I did not know this article was going to be on the frontpage as I said in my original comment - I caught it at 2 points. Had I known my comment was going to go through a seven-way review by a wide variety of experts, I'd have just said "non-latin scripts" to be on the safe side. (/s) Sorry for the rant - People correcting each others where correction does not at all matter bothers me to no end. It feels like a sneaky and thoroughly annoying way of flashing your "I know THINGS!" credentials.
  - contingencies 12y ago
    You seem to be admitting mistake, complaining about being corrected, and dismissing the process all at once. But I'm sure you learned something :)
- cabalamat 12y ago
  > There are arabic derived scripts at least as far east as China and the Philippines (jawi), but describing them as Middle Eastern is plain wrong Is it, though? I'd describe the Roman script as European (since it originated in Europe) without expecting to be corrected, even though the vast majority of people who use it live outside Europe.
  - contingencies 12y ago
    Would you describe a page of written Vietnamese or Hanyu Pinyin as European? That's a fairer comparison to the author's turn of phrase.
    - scrollaway 12y ago
      You seem to be trying very hard to look very smart, yet you (probably purposefully) seem to be missing the point of everyone you reply to. That is not really making you look "smart".
    - cabalamat 12y ago
      I would describe the scripts they're written in as European. The languages themselves aren't, obviously.

ximeng 12y ago

Unsurprising to see a link to Michael Kaplan in there. His blog was taken down due to what sounds like a dispute over patents with his boss. See this twitter conversation between Spolsky and him: https://twitter.com/spolsky/status/447024470256283648 https://twitter.com/spolsky/status/447024470256283648. -- "It has been made officially clear to me that [my] Blog ... is for all intents and purposes dead." Well, I blogged about a patent and included art, so my former manager took the Blog down with extreme prejudice. -- Blog is now archived at http://www.siao2.com/ http://www.siao2.com/ and has much more information on localisation.

thejdude 12y ago
Wat? I thought the whole point of patents is to make information public, because competitors are simply not allowed to use the information without payment anyway.

alieteraz 12y ago

This is Ali Eteraz. Thank you for recognizing this piece. It was a lot of fun to write. I've actually been to a Ycombinator event. Twice I think. A couple of your alumnus managed to talk me into downloading stuff or getting on email lists that, well, I didn't always use (though they did seem promising!). I am a writer based in the Bay Area and I really enjoy getting to learn about interesting technological developments and human stories within that. So definitely hit me up whenever you're congregating or plotting something interesting. My website is alieteraz.com or I am at @eteraz or FB. Ali Eteraz ps - I later got to meet Michael K. at the Unicode conference in Santa Clara, along with the Microsoft Windows team. The Windows Phone people, who I also really wanted to talk to because this is a mobile problem too, did not want to talk to me because they had a new phone coming out.

mchaver 12y ago

For web development, perhaps you could create a simple system that detects if the user's device supports nastaliq. If not convert, it to naskh or Romanized Urdu (not sure which is preferred). As for dealing with all the data already written in naskh and Romanized Urdu, this is a bit more complicated because I am guessing there is not a one to one match between naskh and nastaliq (naskh has less letters), and Romanized Urdu and nastaliq. You would probably need to collect some data and use some machine learning techniques. If it is not too big you could use a client-side Javascript program, otherwise something like a Chrome extension or an API. There are similar things done for Arabic to handle various forms of Romanized Arabic (user does not have access to an Arabic keyboard) and Non-Modern Standard Arabic Dialects: http://www.yamli.com/ http://www.yamli.com/

abdullahkhalids 12y ago
Data is stored as Unicode (except some really old desktop app formats that were written before Unicode). Naskh and Nastaliq are font styles, used when the system displays the data. The solution is simply that good Nastaliq fonts be created and then used on websites where Urdu is being used. Since Urdu fonts can be much larger in size than Roman fonts (the fonts have to specify joining behavior between all possible combination of characters), delivering over the web is not really viable. Fonts have to be native to all the OSes in existence, so that websites can use them. (I don't think Roman Urdu is ever used for anything serious.)
- mchaver 12y ago
  The author of the article mentions that Naskh does not contain all the letters that Nastaliq has, so if someone types an Urdu text in Naskh it would not have the same underlying Unicode if they had typed it in Nastaliq. Correct me if this is wrong. My assumption is that not all OSes will adopt Nastaliq simultaneously. So assume that the website is storing data in complete coding for Nastaliq and they want to send it to a device that cannot render Nastaliq so the Unicode should be converted so it is properly renderable in Naskh. The other half is just to make one's Urdu language web experience completely in Nastaliq. Even unimportant stuff written in Roman Urdu and then converted to Nastaliq might help promote the use of Nastaliq.

abdullahkhalids 12y ago

There is another problem with fonts. Popular fonts used on the internet only have information about the joining of the 28 Arabic characters. When the extra characters in Urdu are typed they don't join with the other characters in the word. eg. I will type the first line of the couplet here. Note how the third,fourth and seventh word are broken up. اور باذار سے لے آے اگر ٹوٹ گیا I have written about some other issues with the Urdu language over here. http://upgoerurdu.nfshost.com/technical.html http://upgoerurdu.nfshost.com/technical.html

jey 12y ago

> Utility had defeated tradition. Is that supposed to be self-evidently bad?

adamnemecek 12y ago
Indeed. I feel like languages/scripts dying out has some advantages too.
- alieteraz 12y ago
  Languages do evolve and die. English itself is changing all the time. Same goes for Urdu. The question here is whether the changes are organic or is there simple oversight going on. My view is that the problem is of oversight, and of disregarding the Urdu speaking and writing user. If the world's most popular operating systems were designed in Pakistan or Iran and they were not offering enough English language letters as a default on the devices, I think we would be plenty upset.
  - adamnemecek 12y ago
    The question is whether languages are really worth preserving. I'm not a native English speaker yet I don't really have much emotional attachment to my first language. In fact, my life would have been easier if I grew up speaking English. And it's not that I think that English is somehow inherently a better language, it's more that there is a lot more people speaking it. It's kind of like if you are deciding what open source project to use. Having a big community around it is a major deciding factor, if not THE deciding factor.
    - ibrahima 12y ago
      I feel much the same way (though, considering I learned English at the age of 4 and never really learned much more of my parents' language after that, calling English my second language is grossly misleading). I feel a vague attachment just because my grandfather was a professor of Bengali literature and I'd like to be able to read his work, but I don't sweat over the fact that my grasp of the language is very basic. I don't need to use it too often, and people back in the "motherland" can worry about preserving the language or not. The analogy to open source projects is spot on; while you might have some "emotional" attachment to a library/technology that you learned early on, you probably wouldn't make significant decisions based on that. I learned to program in VB5, yet I dropped it as soon as I learned Java because it was just an awful language (though I guess human languages don't really have notions of superiority or inferiority like some programming languages might).

gioele 12y ago

Meanwhile, in the free software world... https://docs.google.com/presentation/d/1ySTZaXP5XKFg0OpmHZM00v5b17GSr3ojnzJekl4U8qI/view#slide=id.g177a94b_0_60 https://docs.google.com/presentation/d/1ySTZaXP5XKFg0OpmHZM0... https://github.com/behdad/harfbuzz/blob/master/test/shaping/texts/in-tree/shaper-arabic/script-arabic/language-urdu/crulp/ligatures/README https://github.com/behdad/harfbuzz/blob/master/test/shaping/... (Harfbuzz is the default text layout engine for Linux GUI frameworks: Qt, GTK, etc)

abdullahkhalids 12y ago
This talk at Google by Roozbeh Pournader is very good at laying out the issues with bidirectional language issues in software. This is a non-trivial problem and is rarely ever solved correctly. https://www.youtube.com/watch?v=wOEzYefrqo4 https://www.youtube.com/watch?v=wOEzYefrqo4
- 616c 12y ago
  And not so ironically, Behdad Esfahbod, one of the original developers of HarfBuzz, works for Google now, according to his blurb on Github. https://github.com/behdad https://github.com/behdad I actually emailed him a few times before I discovered mlterm to read Arabic and other RTL fonts in a terminal (you might ask why: file names and I use mutt for email). Very nice guy, and I am very appreciative of this library.

elandybarr 12y ago

The article is great, but I don't think it adequately captures one of the difficulties with this problem, and that is the very high minimum number of characters that must be designed for an Urdu font. I think it is an order of magnitude higher than many would guess it at first.

jpatokal 12y ago
The Wikipedia article on Nastaliq puts the number at around 20,000. https://en.wikipedia.org/wiki/Nasta%CA%BFl%C4%ABq_script https://en.wikipedia.org/wiki/Nasta%CA%BFl%C4%ABq_script

alieteraz 12y ago

Please correct me if I'm wrong, but if Apple and Android make a nastaliq font part of the directory of fonts that go into their devices from the get go, there wouldn't be a problem. Such fonts have already been designed. The issue now is to find Android and iOS developers who can make them default in the operating systems. When I talked to Microsoft they said that on the smaller devices every last bit of memory was precious so they didn't want to stick extra megabytes or however big a font is. I think that Apple's cheapening out is worse than Android's, because at least Android offers the entire Urdu alphabet (though in naskh). Apple doesn't even offer the Urdu alphabet, requiring Urdu users to have to make do with 12 less letters.

paperwork 12y ago
I don't think it is just a matter of adding a font, with 12 additional letter for urdu. The actual layout algorithm has to change (how characters are laid out, their size, how beginning of one connects to ending of another, etc.).

wirrbel 12y ago

its an interesting read and I acknowledge the intent to keep the traditional and authentic writing system for Urdu. I have a few thoughts on this though. One would be the legibility. Of course I cannot judge legibility of writing systems I do not like, but it seems that nastaliq would be hardly readable on a lot of mobile devices and I wonder how difficult learning the ornate script is. I am talking about alphabetization here. Next thing is: I am learning turkish and turkish is written in the roman alphabet. As far as I know, it was written in a arabic/persian script before which was then reformed to use the latin alphabet. As far as I can tell this is today really uncontroversial and using the latin alphabet is actually the more suitable alphabet for turkish and its rich vowel system that is really important for grammar and meaning. Again I cannot really say anything for Urdu, but knowing it is not arabic but afaik a language of the indo-european family I wonder if there are more reasons to use the lating alphabet than just availability of nataliq fonts and rendering engines. As a side note I would add a few observations relating to the cultural/heritage aspects. In Germany, the "Fraktur" was used widely even at the beginning of the 20th century (see http://en.wikipedia.org/wiki/Fraktur http://en.wikipedia.org/wiki/Fraktur ). Some authors, like Hermann Hesse refused that Antiqua fonts would be used for their writings until publishers convinced them that their works could just not be read by young folks. In a way, a lot of people argued against using non-gothic fonts, but in the end antiqua became quite standard. Nowadays we use the lating alphabet (and most people are not concious about that there ever was a switch).

alieteraz 12y ago
Great point about German language. I have come across that again and again. And you're right, there ARE Urdu speaking youth now who are basically just as comfortable in naskh as they are in nastaliq, if not more so. This was a revelation to me and it added a wrinkle to my thinking. But I go back to my point, that we can't fully adjudge if nastaliq is meant to die off until we make at least a good faith effort to get it onto the devices.
- wirrbel 12y ago
  What makes me wonder is, I would have thought that the prospects of nastaliq are increasing with electronic devices (as opposed to old-school printing presses). What I also observe is, that arab speakers sometimes write with the latin alphabet using characters like "3" do represent some missing phonemes. Urdu surely is not alone, and having a proper nastaliq rendering on all devices might not solve the bigger issue here. Could you elaborate a bit on nastaliq and how/whether it does represent Urdu's phonemes properly? Are people considering Devangari as an alternative?

gauravk 12y ago

Sad. I remember as a kid, growing up in Kashmir, I took special pride in my Urdu calligraphy skills. Writing each sentence was like an art project.

616c 12y ago

So I skimmed the article once, then read it again. I see the writer is here. I just want to say I understand your sentiment, but your title and content are filled with some annoying misinformation and discrepancies. First of all, Nastaliq is not "the Urdu script". If you even skimmed Wikipedia, you would notice that the script does come oringinally from Arabic (where it is used, but more to your point, sadly not as much in print media, at least from my experience in the GCC) and was later well loved by the Persians, and later different portions of the Indian subcontinent (India, Pakistan, and beyond). To call it Urdu script, just because it is popular, and to insinuate that it Naksh is the default script of Arabic (it is not the preferred one, by the way) is a huge stretch. Another point: Naksh was invented by a Persian, but who grew up his whole life in Arab Baghdad and was an Abbasid court vizier in modern-day Iraq. He did not spend time in Iran (then Fars, as it is known today in Arabic), until he was a tax collector there after growing up in Iraq, and only for a period. He might have been ethnically Persian, but to call Naksh script Persian because of the guys family background is far-fetched, especially if you know how loose the cultural boundaries between Abbasid Iran-Iraq were (they were part of the empire and people, goods, and ideas freely flowed between them). It is a derivative of Kufic script, and has strong ties to the religious intellectual history of Iraq (Basra and Kufa were the seat of a massive amount of religious scholarship and still prominent beliefs in Islam; scripts and art were an awesome by-product). Sorry for the rant, but I know this is probably not your field. As I guy who spent a lot of time studying Arabic and Arab and/or Islamichistory, I get very annoyed at the misinformation peddled by people about Arabic history and culture. I do not mean to be so blunt, but I hear people rattle off misinformation like this often, and it irritates me.

alieteraz 12y ago
You are correct about the history. Unfortunately the Abbasid empire came to an end in 1258 AD and Urdu wasn't even around then. I take your point that naskh has been wide-spread. And it is true that it is even in the Indian sub-continent. Sindhi, for example, is a naskh based language. Punjabi, which my parents spoke, on the other hand, prefers nastaliq, and regionally the two places are adjacet. That doesn't mean that those people didn't historically play around with scripts. They did. However, my piece is entirely about what is going on today. Sure you can find Ottoman era signs in Egypt and the Levant that display Arabic in nastasliq (my readers sent me plenty such pictures), but by and large Arabic today is written in naskh and almost never in nastaliq. Take a look at some of the fonts that the Omani government is playing around with. They are not nastaliq. This is the political aspect I briefly touched upon in the article. I hope you will get a chance to look into the Arabization aspects of the political debates raging in that part of the world. I make brief mention of it by bringing up the fight over "Khuda Hafiz" or "Allah Hafiz." Meanwhile, the past two or three generations of Urdu readers and writers grew up associating nastaliq and Urdu almost exclusively with each other (thus the jarring effect associated with having to read Urdu in something else). I have plenty of emails testifying to this from people from my parents generation and some people of mine. In short, no one is really talking about ancient Arab history here, as fascinating as that would be.

616c 12y ago

And for the Arch Linux peopkle, have a Nastaliq font if you need it (or want it). https://aur.archlinux.org/packages/ttf-nastaliq/ https://aur.archlinux.org/packages/ttf-nastaliq/

sharjeel 12y ago

This article was making rounds in my social feed about a year ago. Disturbed by the extremely weak and some absolutely flawed arguments, I wrote a note: https://www.facebook.com/sharjeelqureshi/posts/10151806998363551 https://www.facebook.com/sharjeelqureshi/posts/1015180699836...

mellisarob 12y ago

you need to come up with more details and authentic details because some stuff you mentioned it utter crap.

Rapzid 12y ago

I don't like how the article opens with the premise of a people struggling their whole lives to master writing in a particular style only to have their expression oppressed, then proceeds to talk about font rendering.. Generally though, I'm not really sure how I feel about trying to get people to care about something so precious in general. One one hand it's a shame to lose culture. On the other the internet/technology has its own culture and is it really lost in the age of 5c per GB storage?

kranner 12y ago

For Hindi speakers who don't read Nastaliq, the sign on the wall is hilarious. Transliterated to Devanagari, it reads: हाय मैं मर गई ऐन्ना (इतना) टेस्टी बर्गर

merrua 12y ago

Nastaliq is lovely to look at.

alieteraz 12y ago

BTW. I published this piece in fall of last year. Long before it came on Hacker News, it had been passed around the Pakistani and Indian Twittersphere, and no one there was saying that we should let tech companies off the hook for not offering a) the full alphabet and b) the right script. It was, actually, the Pakistanis and Indians that made this article what it was. When I wrote it, I honestly thought I was just some eccentric who was annoyed for very personal reasons. Even as we speak the piece is being retweeted by professors and students at LUMS, in Lahore, and universities in Islamabad. What's even more interesting is that as a result of writing this piece my own favoritism towards nastaliq actually LESSENED. But by the time my thinking evolved, the nastaliq purists had made the piece their own.

paperwork 12y ago

A big reason nastaliq is not widely available on our screens is because rendering it correctly requires 'context sensitive shape substitution'[1]. Simple substitution of one character for another is not enough. I've spent a couple of hours looking at how to render my own text, perhaps using something like http://typeface.neocracy.org/ http://typeface.neocracy.org/, but my day to day work is so far from text layout, rendering, client side javascript that it would be (too) large an undertaking. [1] http://ww.cle.org.pk/Publication/papers/2006/context_sensitive_shape_substitution.pdf http://ww.cle.org.pk/Publication/papers/2006/context_sensiti... [PDF]

wmil 12y ago

It seems like something analogous has happened with European scripts. There were huge changes in how characters were presented between different styles of handwriting, engraving, printing presses, and finally everything the computer world has brought us. It just seems like the author has an extreme emotional attachment to a typeface.

paperwork 12y ago
I don't know about the author's state of mind, but generally speaking, this is not just a matter of preference. For those of us who grew up reading Nastaliq, other type faces such as naksh are fairly difficult to read 'fluently.' Imagine if all books, newspapers websites were hand written by someone with poor handwriting. It would actually slow down your reading speed. Reading is such a natural thing we do on a daily basis that we notice even small things which cause us to slow down. btw, I use 'bad handwriting' simply as a way to explain the effect of different writing style on reading, naksh is not good or bad -- it is just different from what Urdu readers are used to. Also, nastaliq isn't just different because its letters are formed differently (there being a one-to-one correspondence between different typefaces). Nastaliq letters are laid out in a context sensitive manner where a letter can take on a fairly large number of shapes, depending on where in the word it is (and which letter it is next to). My earlier post links to a paper which describes how Nataliq rendering requires context sensitive shapes. Basically randomly capitalize letters in everything you read online. Eventually you will get used to it, but that doesn't mean it won't be annoying for a long time.

GhotiFish 12y ago

>Utility had defeated tradition. There is hope for humanity yet!

m0th87 12y ago

That's not the only way in which Urdu is "dying" - due to colonization, basically all modern words are in English.

thret 12y ago

This feels like unusual timing. I am presently reading 'A Suitable Boy' by Vikram Seth. One of the main characters is sent into virtual exile where he learns how to read and write Urdu. Until now I don't think I'd heard of it. My own opinion is that we should preserve history, remember it, but not morn it. We are programmers! How many writing styles and languages have we seen die to make way for something better (or worse?). This is a natural survival of the fittest cycle that effects writing styles, languages, civilisations and everything else. I have Japanese friends who read even less Kanji than I do. I see that writing style dying out over the next 50 years also.

sandGorgon 12y ago

@alieteraz - your comments below have been extremely informative and though urdu occupies the collective conscious of all Indians through bollywood (90% of melancholic melodies are invariably in urdu), we have never given the script much thought. I think you fought admirably with Apple, Twitter and Microsoft for getting Urdu fonts included ... but that is the wrong battleground. The arena for you is simply the browser and mobile . All you need to do is get Nastaliq scripts adopted into Google Webfonts (free for anybody to use in their webapps) and into Android (installing a font through a Launcher theme is incredibly easy [1]. Do NOT try getting it included in the Android base) do you think you can get together a Kickstarter to fund a nastaliq font, good enough to be usable ? and then try getting them into webfonts and android themes ? [1] https://play.google.com/store/apps/details?id=pete.app.apexthemes.font.chokycooky&hl=en https://play.google.com/store/apps/details?id=pete.app.apext... and http://appcrawlr.com/android-apps/best-apps-launcher-fonts http://appcrawlr.com/android-apps/best-apps-launcher-fonts

capex 12y ago

Anyone ready to work with me on creating a Nastaliq webfont?

Ghalib5423 12y ago

In the 1960s, the Government of Pakistan tried to make naskh common in order to facilitate mechanization of printing. In order to do so, naskh was introduced in schools so that children get used to it from an early age. Some books started appearing in naskh. The daily newspaper Nawa-i-waqt started printing its second page in naskh. It was very unfortunate for Urdu that these efforts failed. The reason was simple. Instead of requiring that the textbooks be typeset in naskh, they continued to be handwritten, albeit in naskh-like script. Gradually after a few years that naskh started resembling nasta’aliq more and more and eventually all traces of naskh disappeared. This, to mind, was the greatest setback suffered by the Urdu language printing. Let me explain why I call it the greatest setback. Urdu books and newspapers were written by hand and then lithographed. The process was slow, mistakes were plentiful, dots above and below letters were misplaced, letters such as daal and waaw were indistinct from each other, and many times the printing was illegible. Please try to read a book produced by that method, and then compare it to the same book typeset in naskh; you will note the difference. I have a copy of Divan-e-Hafiz published in nasta’aliq in the subcontinent, and another copy in naskh published in Iran. The difference in the clarity of text in the two is phenomenal. This abortive attempt in 1960s to switch to naskh had nothing to do with religion or Arabization. Ayub Khan, in whose time the effort started, can hardly be accused of religion-inspired initiatives. It was simply to promote mechanized printing. If anything, later the religion-enthusiast Zia-ul-Haq did nothing to popularize naskh. Ironically, computer has saved nasta’aliq, since software is now commonly available which is now universally used to compose material for printing. But naskh is still clearer to read, and in my opinion it is not too late to give it another try, and use naskh, while reserving nasta’aliq for calligraphy. Undoubtedly nasta’aliq is elegant and no script can match its beauty.