8 ms·

Remembering a Programming Language That Helped Shape the Digital New York Times

118 points by rayascott 8y ago

williamstein 8y ago
I was expecting an article about CoffeeScript...
- fouc 8y ago
  I was briefly hoping it'd be an article about PostScript
  - RichardCA 8y ago
    When I was a ten year old kid in 1973 my school did a tour. I remember looking at some programs running on green-screen terminals, and the guy doing the demo typed in commands using the keyword FOR so I assume it was all Fortran back then. You can get a rough idea if you watch this... https://www.youtube.com/watch?v=1MGjFKs9bnU https://www.youtube.com/watch?v=1MGjFKs9bnU
    - nwallin 8y ago
      Numerous programming languages, including basic, algol, etc used for loops. Notably absent from the list of languages which used FOR as a looping construct was Fortran, which used 'do' for looping, and cobol, which used 'perform'. So whatever languages it was, it wasn't Fortran.
      - mveety 8y ago
        On a lot of systems the fortran compiler is often named fortran and usually aliased as for and fortran in the shell. I know as well on VMS, RSX, and RSTS you, at a minimum, just need to type the first 3 chars of a command for DCL to figure out what it is.
      - RichardCA 8y ago
        The "etaoin shrdlu" movie references Data General, so most likely it was an early version of RDOS.
sigfubar 8y ago
> Recruiting new engineers was sometimes a challenge because candidates had to accept that they were joining an organization with a homegrown programming language and build system. I got an offer from NYT in 2009, but rejected it because of the custom language thing. My interview was conducted entirely using this custom language, and the interviewers were uninterested in discussing any other technology. Even architectural questions, which I attempted to answer in terms of industry standards, were steered towards Context.
- scottlamb 8y ago
  > My interview was conducted entirely using this custom language, and the interviewers were uninterested in discussing any other technology. Bizarre! Writing their system in a custom programming language isn't _that_ unusual, I think. But why on earth during an interview wouldn't they let candidates program (entirely or mostly) in a language they'd heard of before that morning? Couldn't they assume that if someone has achieved a working knowledge of one or more similar languages, they can learn another in some reasonable time (days, weeks, months)?
  - mjburgess 8y ago
    Sounds like maybe they weren't that up on industry and felt unable to evaluate candidates outside their narrowed skill scope.
    - bryanrasmussen 8y ago
      I went on an interview one time in which the job was not for python, but in which the coding exercise was in python, the assumption being that they wanted to see how you handled something you didn't know. The interviewer mentioned having another guy in who announced flat out he was not going to do it even though it was explained that you didn't need to know the language to solve the problem. I of course had some python familiarity, but I still failed because I couldn't remember the name for binary search and that threw me for a loop when the obvious answer I should have gone for in the exercise was 'here we should use binary search'.
scegit 8y ago
> “We probably over-engineered it, I guess,” Damens said. Exactly. Also, creating a programming language for something as simple as a newspaper is pretty much a nightmare. Worrying about scalability when most of their content is static.
- ndespres 8y ago
  This is nearly 20 years ago. Were CGI and PHP really up for the tasks of serving up the type of dynamic pages at scale 2 decades ago? The "content" (articles) might be largely static but the pages are not, with dynamic ads, weather reports, showtimes etc. Not to mention getting non-technical people to adopt the language easily, transitioning from print publishing workflows. Do you not believe them when they say existing language and solutions weren't working?
  - scegit 8y ago
    I'm not really familiar with how things were ~20 years ago, but I still believe that creating a language to use internally just for this is overkill. Dynamic ads, weather reports ... etc. are not that difficult they could have probably done it in C (maybe a custom somewhat advanced template engine?) with much less effort.
    - mooreds 8y ago
      Twenty years ago I would have prototyped the solution in perl as an Apache module and then rewritten the slow hot spots in c. But I don't know what their hardware constraints weredit person => perl
      - a-dub 8y ago
        My guess is that it actually looked a lot like that, except the mod_perl app was actually a homegrown template engine using their homegrown template language which they then called "Context". That was a common pattern back then.
    - deleted 8y ago
mooreds 8y ago
A language that supported a platform for almost twenty years in the modern era? I can't call this anything but a wild success. Sure, it had issues, as discussed on the article, but they solved the business problem at the moment in time. No software decision is without costs, but running a major site for almost two decades indicates that they chose well. Bravo! I do wish they'd throw the source code up on GitHub, as it would be an interesting historical artifact.
- CaliforniaKarl 8y ago
  Or, failing that, donate it to the Computer History museum.
- mpweiher 8y ago
  Seconded.
v8engine 8y ago
I think it was this site where I remember Context from: https://www.mangaupdates.com/ https://www.mangaupdates.com/ Back then in my school years I remember not being able to find anything about this language. Mystery solved now. Thank you.
- egypturnash 8y ago
  But a new mystery opens up: how did a site for sharing news about manga end up being built in a custom language built for the NYT?
scott_s 8y ago
This is the sort of programming language that I would love to read a detailed retrospective of in an ACM journal. It's a programming language designed to solve a particular problem, grew over time for that problem, and it successfully did so for almost 20 years. A programming language that is useful that long is interesting to me.
weeksie 8y ago
I'd love to see an example of the language, I kept reading (the story is great) but it felt like we got left hanging without any code snippets to put it all in, uh, context I guess.
shafte 8y ago
I'm not second-guessing the technical that the Times made here (I'm sure it made sense given their constraints), but I do wonder if these kinds of in-house general-purpose languages will die out eventually. Some of the comments indicating that this sort of project is a "red flag" seem to miss that 20 years ago the language landscape was a lot more proprietary/closed. But these days, a big open source/community ecosystem is a really really strong reasons to invest in an existing language (or at least open-source your in-house language, a la Hack or Go). It's hard for in-house general-purpose languages to compete.
oraphalous 8y ago
Man - I have to fight tooth n nail to make code just a little more performant / re-usable / robust to changing requirements. How on Earth did they get permission to invent a whole new language for a user name feature? Jealous...
kwccoin 8y ago
Domain specific language. Finance has one from APL. Why not news room have one.

118 points by rayascott 8y ago

williamstein 8y ago

I was expecting an article about CoffeeScript...

fouc 8y ago
I was briefly hoping it'd be an article about PostScript
- RichardCA 8y ago
  When I was a ten year old kid in 1973 my school did a tour. I remember looking at some programs running on green-screen terminals, and the guy doing the demo typed in commands using the keyword FOR so I assume it was all Fortran back then. You can get a rough idea if you watch this... https://www.youtube.com/watch?v=1MGjFKs9bnU https://www.youtube.com/watch?v=1MGjFKs9bnU
  - nwallin 8y ago
    Numerous programming languages, including basic, algol, etc used for loops. Notably absent from the list of languages which used FOR as a looping construct was Fortran, which used 'do' for looping, and cobol, which used 'perform'. So whatever languages it was, it wasn't Fortran.
    - mveety 8y ago
      On a lot of systems the fortran compiler is often named fortran and usually aliased as for and fortran in the shell. I know as well on VMS, RSX, and RSTS you, at a minimum, just need to type the first 3 chars of a command for DCL to figure out what it is.
    - RichardCA 8y ago
      The "etaoin shrdlu" movie references Data General, so most likely it was an early version of RDOS.

sigfubar 8y ago

> Recruiting new engineers was sometimes a challenge because candidates had to accept that they were joining an organization with a homegrown programming language and build system. I got an offer from NYT in 2009, but rejected it because of the custom language thing. My interview was conducted entirely using this custom language, and the interviewers were uninterested in discussing any other technology. Even architectural questions, which I attempted to answer in terms of industry standards, were steered towards Context.

scottlamb 8y ago
> My interview was conducted entirely using this custom language, and the interviewers were uninterested in discussing any other technology. Bizarre! Writing their system in a custom programming language isn't _that_ unusual, I think. But why on earth during an interview wouldn't they let candidates program (entirely or mostly) in a language they'd heard of before that morning? Couldn't they assume that if someone has achieved a working knowledge of one or more similar languages, they can learn another in some reasonable time (days, weeks, months)?
- mjburgess 8y ago
  Sounds like maybe they weren't that up on industry and felt unable to evaluate candidates outside their narrowed skill scope.
  - bryanrasmussen 8y ago
    I went on an interview one time in which the job was not for python, but in which the coding exercise was in python, the assumption being that they wanted to see how you handled something you didn't know. The interviewer mentioned having another guy in who announced flat out he was not going to do it even though it was explained that you didn't need to know the language to solve the problem. I of course had some python familiarity, but I still failed because I couldn't remember the name for binary search and that threw me for a loop when the obvious answer I should have gone for in the exercise was 'here we should use binary search'.

scegit 8y ago

> “We probably over-engineered it, I guess,” Damens said. Exactly. Also, creating a programming language for something as simple as a newspaper is pretty much a nightmare. Worrying about scalability when most of their content is static.

ndespres 8y ago
This is nearly 20 years ago. Were CGI and PHP really up for the tasks of serving up the type of dynamic pages at scale 2 decades ago? The "content" (articles) might be largely static but the pages are not, with dynamic ads, weather reports, showtimes etc. Not to mention getting non-technical people to adopt the language easily, transitioning from print publishing workflows. Do you not believe them when they say existing language and solutions weren't working?
- scegit 8y ago
  I'm not really familiar with how things were ~20 years ago, but I still believe that creating a language to use internally just for this is overkill. Dynamic ads, weather reports ... etc. are not that difficult they could have probably done it in C (maybe a custom somewhat advanced template engine?) with much less effort.
  - mooreds 8y ago
    Twenty years ago I would have prototyped the solution in perl as an Apache module and then rewritten the slow hot spots in c. But I don't know what their hardware constraints weredit person => perl
    - a-dub 8y ago
      My guess is that it actually looked a lot like that, except the mod_perl app was actually a homegrown template engine using their homegrown template language which they then called "Context". That was a common pattern back then.
  - deleted 8y ago

mooreds 8y ago

A language that supported a platform for almost twenty years in the modern era? I can't call this anything but a wild success. Sure, it had issues, as discussed on the article, but they solved the business problem at the moment in time. No software decision is without costs, but running a major site for almost two decades indicates that they chose well. Bravo! I do wish they'd throw the source code up on GitHub, as it would be an interesting historical artifact.

CaliforniaKarl 8y ago
Or, failing that, donate it to the Computer History museum.
mpweiher 8y ago
Seconded.

v8engine 8y ago

I think it was this site where I remember Context from: https://www.mangaupdates.com/ https://www.mangaupdates.com/ Back then in my school years I remember not being able to find anything about this language. Mystery solved now. Thank you.

egypturnash 8y ago
But a new mystery opens up: how did a site for sharing news about manga end up being built in a custom language built for the NYT?

scott_s 8y ago

This is the sort of programming language that I would love to read a detailed retrospective of in an ACM journal. It's a programming language designed to solve a particular problem, grew over time for that problem, and it successfully did so for almost 20 years. A programming language that is useful that long is interesting to me.

weeksie 8y ago

I'd love to see an example of the language, I kept reading (the story is great) but it felt like we got left hanging without any code snippets to put it all in, uh, context I guess.

shafte 8y ago

I'm not second-guessing the technical that the Times made here (I'm sure it made sense given their constraints), but I do wonder if these kinds of in-house general-purpose languages will die out eventually. Some of the comments indicating that this sort of project is a "red flag" seem to miss that 20 years ago the language landscape was a lot more proprietary/closed. But these days, a big open source/community ecosystem is a really really strong reasons to invest in an existing language (or at least open-source your in-house language, a la Hack or Go). It's hard for in-house general-purpose languages to compete.

oraphalous 8y ago

Man - I have to fight tooth n nail to make code just a little more performant / re-usable / robust to changing requirements. How on Earth did they get permission to invent a whole new language for a user name feature? Jealous...

kwccoin 8y ago

Domain specific language. Finance has one from APL. Why not news room have one.