11 ms·

What Happens When You Mix Java with a 1960 IBM Mainframe

230 points by 3n7r0pY 10y ago

jonnycowboy 10y ago
The thing is that there was a lot about those old systems that was slow, so you were very, very careful how you programmed. You tended not to use vast library stacks, you went close to the metal and you coded in languages like Assembler, COBOL or FORTRAN. I/O was often run through specialised co-processors (such as IBM's channel processors) and the terminals could sometimes help too. I have friends who have been looking after legacy applications for an airline running on Unisys. The core apps for reservation, Cargo booking and weight/balance were written in FORTRAN. In recent times, the front end was written in Java to give web access. They tried to rewrite the core apps but it was impossible to do so and get the performance.
- eru 10y ago
  > The thing is that there was a lot about those old systems that was slow, so you were very, very careful how you programmed. That's a common sentiment. I wish I could find the quote by someone who made the transition; it was about how happy they were to be able to compile so much quicker, and how getting immediate feedback made them so much more productive.
  - 3n7r0pY 10y ago
    The notion of waiting ages for programs to compile or assemble is mostly related to the older hardware. I compile/assemble COBOL and IBM's assembly language on a z13 daily and it's pretty much instantaneous.
    - eru 10y ago
      > The notion of waiting ages for programs to compile or assemble is mostly related to the older hardware. Oh, I was talking more about older ways to organize the data centre: batch vs timeshare processing. https://en.wikipedia.org/wiki/Time-sharing https://en.wikipedia.org/wiki/Time-sharing
- YeGoblynQueenne 10y ago
  >> They tried to rewrite the core apps but it was impossible to do so and get the performance. Well, Cobol is a bit like the C of mainframes - you can manipulate memory directly and so on. You can't really do that sort of thing with Java.
  - stefs 10y ago
    a) if it was really running on the old hardware; in that case ruby on a modern machine would have been several magnitudes faster than the original code - at least because of the faster IO b) if the whole thing was indeed running in an emulator, the emulation overhead would have negated all direct memory access advantages
    - wahern 10y ago
      Emulators on mainframes are much more sophisticated and performant than is typical on x86 and ARM platforms. The hardware and even software is often designed with emulation in mind, not just for backwards compatibility but for forwards compatibility, too.
      - andai 10y ago
        Do you mean to say that they were running a mainframe emulator on a mainframe?
        wahern 10y ago
        Compilers for some IBM mainframes have for decades (since the 1970s, I think) targeted an intermediate virtual machine instruction set which is then translated on-the-fly to the local architecture by the OS upon execution. So in the case of IBM their machines are truly built with both forward and backward compatibility in mind. The pointers for this instruction set have been 128 bits since the beginning, long before 64-bit hardware even came into being. Some (or all?) of the latest Unisys mainframes run on Intel Xeons, but with custom chips for translating the machine code of their old architectures. I don't work in this area. I just like reading about it. Though, unfortunately, it's difficult to find clear specifications and descriptions on how these architectures work. For example, Unisys' 2200 ClearPath architecture is one of the (if not _the_) last architectures still sold that uses signed-magnitude representation, as well as having an odd-sized integer width of 39 bits. (INT_MAX is 549755813887 and INT_MIN is -549755813887, and the compiler has to "emulate" the modulo arithmetic semantics required of unsigned types in C. ClearPath MCP is also a POSIX system, and has to emulate an 8-bit char type.) I discovered that you could download the specification for their C compiler for free online, which was useful when discussing the relevancy of undefined behavior in C. But AFAIU (and this is where finding concrete details is more difficult) the latest models of the ClearPath line use Xeons with custom chips bolted on to help run the machine code of the older architecture. In any event, the point is that while the old architecture is arguably emulated, it's not a pure software emulation that you might assume, and the resulting performance is better than the previous models of those mainframes, which were still being built at least until a few years ago. In other words, direct memory access isn't ruled out because the I/O systems may have been intentionally designed to work efficiently in a backwards compatible manner.
deleted 10y ago
[deleted]
skissane 10y ago
Is someone in the US government really still using an IBM 7074? Really? I'm shocked. How could that possibly be cost-effective? Actually, this blog post makes the story clearer: http://nikhilism.com/post/2016/systems-we-love/ http://nikhilism.com/post/2016/systems-we-love/ It isn't a physical IBM 7074. When it came time to migrate from 7074 to S/360, rather than rewriting their 7074 software, they just wrote a 7074 emulator for S/360. And, it sounds like, they are still running their 7074 software, under their 7074 emulator, most likely on a recent z/Architecture mainframe. The article makes it sound like people still use "1960s mainframes" when I very much doubt anyone is still running 1960s hardware in production. People use modern machines–modern IBM mainframes, which are multicore 64-bit processors–or other mainframe vendors such as Unisys or Fujitsu use mainly I believe x86-64 running Linux running a software emulator for the old mainframe CPU. A lot of legacy, sure, but I think this article makes it sound even more legacy than it really is.
- facepalm 10y ago
  I was going to ask if they couldn't just replace the hardware with emulators :-)
- wolf550e 10y ago
  I know of an S/360 assembly code base that (as of a decade ago), still ran code that assumed 24 bit address space and used 31 bit pointers with tags stuffed in the unused 7 bits. So it had to run in the lowest 16MB of memory, on a 64-bit machine. I don't think they had the budget to rewrite that subsystem to fix it. So yes, new hardware, but some of the legacy problems in the software run very deep.
- cle 10y ago
  It's "cost effective" because "cost effectiveness" is not as objective as we think, once we start considering things like risk tolerance.
- mbellotti 10y ago
  > A lot of legacy, sure, but I think this article makes it sound even more legacy than it really is. Indeed. The point of the talk was that 1) legacy is often assumed to be bad not for any real technical reasons but just because it is legacy and 2) a lot of what was being presented as legacy wasn't even legacy. Their OS 2200 version was actually newer than the Oracle DB they were using on the "modern" side of the stack.
wolf550e 10y ago
An explanation from 'bcantrill about why the talk was not recorded: https://lobste.rs/s/q2xz14/systems_we_love_videos/comments/if9yox#c_if9yox https://lobste.rs/s/q2xz14/systems_we_love_videos/comments/i...
geodel 10y ago
I have heard from many of my friends about massive projects of 'modernizing' mainframe applications with Java stack. Java did not deliver improvement that management was expecting. Once the project consumed all the budget for ~25% completion, they were scrapped. I think, though without any proof, that overreliance of 100s of mixed quality libraries, combined with 'best practices' of enterprise development and heavy application of design patterns creates a very large surface area for change. This makes reasonable translation of functionality to Java almost impossible.
- neeleshs 10y ago
  Was it Java that did not deliver improvement, or was it the team? :)
  - geodel 10y ago
    I am inclined to say Java as I have not seen mythical teams who work on Java without whole caboodle of 'Enterprise Apps' culture.
    - electrum 10y ago
      You can write modern software on Java without using "enterprise" features: https://github.com/prestodb/presto https://github.com/prestodb/presto
      - geodel 10y ago
        Nice. It may be the best way to organize project this large. I know this is maven thing and once using everyone's favorite Intellij IDEA it should not matter but I personally find ~1000 directories for ~4000 files a kind of Javasim.
      - bsg75 10y ago
        Can you expand on what makes the Presto project different than the aforementioned enterprise approach? (Not a Java developer, so I don't have context)
walrus01 10y ago
When I first saw the Chromebook, my thought was that Larry Ellison's 1998 dream of a network computer thin terminal had come true. It's smarter than a true thin terminal, but everything lives in the cloud (err, butt).
- bitwize 10y ago
  That Chrome extension is clbuttic.
  - TeMPOraL 10y ago
    Indeed. > but everything lives in my butt (err, butt). I had to open the comment in porn (er, incognito) mode in order to parse it :D.
- lokedhs 10y ago
  Being someone who worked for Sun at the time, I'd rather say it was Scott McNealy's dream.
YeGoblynQueenne 10y ago
>> It was at this point that the seasoned data architects in the department began expressing their exasperation. “15 years ago, everybody was telling us ‘Get off the mainframe, get on AT&T applications, build these thick clients. Mainframes are out.’ And now thick clients are out, and everybody’s moving to APIs and microservices, which basically are very similar to the thin client that a terminal uses to interact with a mainframe.” A.k.a. "Them as not knows their history are doomed to repeat it". But I suppose it's more the case of business imperatives than real ignorance that drive this mad race to make new stuff that works worse than the old stuff only so we can then go back to the old stuff with a different name. Btw, that lady is my new tech hero: “The systems that I love are really the systems that other engineers hate,” Bellotti told the audience — “the messy, archaic, half chewing gum and duct tape systems that are sort of patched together. <3 <3 <3
- eternalban 10y ago
  "15 years ago" doesn't sound right. Thick clients were being pushed in mid 90s. But then Clipper Chip plans went south ...
  - toyg 10y ago
    Consider that state/federal bureaucracies get on "trends" with a delay of 2 or 3 years minimum, and the story is probably from 2015 or thereabout - the agency was launched in 2014.
- krylon 10y ago
  Wish I could upvote this twice.
mbellotti 10y ago
This article was such a pleasant surprise! I loved talking at Systems We Love and I love talking about legacy architecture in general. Bryan and the Joyent team were very accommodating and understanding. The White House is an amazing place to work (even now), but it's not a system that understands conference talks very well. Wish we could have a video, but it was a heavy lift just getting the bureaucracy okay with the idea that I was going to talk about information that was already public, without naming agencies or projects and without going into detail beyond what was in a user manual and that this was not a security risk.
- benballjr 10y ago
  Hi Marianne! I'm with The New Stack (not the writer of this article), and was wondering about the video myself. Interesting to see you address that there's definitely no public facing version, as it seems a lot of the commenters here would love to see it. I can only imagine the bureaucrat complexity involved.
  - mbellotti 10y ago
    Oh yeah, when I talked to the agency that owns the 7074 (more on this in a second) they were like "You can't do this because we will get hacked" "How are you going to get hacked if I talk about your mainframe? It's not connected to the public internet, is it?" "No. Well... we don't know... but ... hackers! Hackers are really smart Marianne." Part of the compromise was that I promised I would only use information that was already available publicly through government reports and news articles. I went back through my talk and documented where each fact was already published somewhere else until they were comfortable with it. So the ambiguity on whether the 7074 was the actual machine or an emulator was deliberate... there were certain things I could not find a public comment on and therefore agreed to avoid making direct statements about. This all seems super annoying, but it makes sense when you realize how heavily scrutinized public servants are. In the end they are only trying to protect me, my organization and Obama's legacy. Three things that are really important to me. So I can't exactly blame them for it. I was happy to be able to find a middle ground where they felt comfortable, the organizers weren't too badly inconvenienced and I got to give the talk I wanted to.
stevehiehn 10y ago
Hmm, interesting. My bet is that the Java devs were not great at implementing concurrency. But who knows.
- kevin_thibedeau 10y ago
  Probably lots of XML transforms going on to be enterprisey.
Animats 10y ago
I'm still struggling with my own legacy code problem. I'm reviving an old LISP program from the early 1980s. Parts of it were written for the original Stanford AI Lab SAIL system in the 1970s. It last ran under Franz LISP in 1986. My current struggle is with one line of code: (defun getenode (l) (cadr l)) That ought to be simple enough. But it's being applied not to a list, but a "hunk". A "hunk" is an obsolete MacLISP concept.[1]. It's a block of memory which has N contiguous LISP cells, each with two pointers. This is the memory object underlying structures and arrays in MacLISP. Macros were used to create the illusion of structure data objects, with hunks underneath. However, you could still access a "hunk" with car, cdr, cxr, etc. I'm converting this to Common LISP, which has real structures, but not hunks. That, with some new macro support, works for the regular structure operations. So far, so good. But which element of the structure does (cadr l), which usually means the same thing as "(car (cdr l))", access? (cadr (list 0 1 2 4)) returns 1, so you'd think it would be field 1 of the structure. But no. It's more complicated and depends on how hunks are laid out in memory. The Franz LISP manual from 1983 [2] says "Although hunks are not list cells, you can still access the first two hunk elements with cdr and car and you can access any hunk element with cxr†." At footnote "†", "In a hunk, the function cdr references the first element and car the second." This is backwards from the way lists behave. A blog posting from 2008 about MacLISP says "A Maclisp hunk was a structure like a cons cell that could hold an arbitrary number of pointers, up to total of 512. Each of these slots in a hunk was referred to as a numbered cxr, with a numbering scheme that went like this: ( cxr-1 cxr-2 cxr-3 ... cxr-n cxr-0 ). No matter how many slots were in the hunk, car was equivalent to (cxr 1 hunk) and cdr was equivalent to (cxr 0 hunk)." Note that element 0 is at the end, which is even stranger. The documentation is silent about what "cadr" would do. Does it get element 2, or get element 0 and then apply "car" to it? The original code [3] contains no relevant comments. I'm trying to figure out from the context what the original author, Greg Nelson, had in mind. He died in 2015.[4] [1] http://www.mschaef.com/blog/tech/lisp/car-cdr.html http://www.mschaef.com/blog/tech/lisp/car-cdr.html [2] http://www.softwarepreservation.org/projects/LISP/franz/Franz_Lisp_July_1983.pdf http://www.softwarepreservation.org/projects/LISP/franz/Fran... [3] https://github.com/John-Nagle/pasv/blob/master/src/CPC4/z.lisp https://github.com/John-Nagle/pasv/blob/master/src/CPC4/z.li... [4] https://en.wikipedia.org/wiki/Greg_Nelson_(computer_scientist) https://en.wikipedia.org/wiki/Greg_Nelson_(computer_scientis...
mtdewcmu 10y ago
"It used magnetic-core memory (instead of cathode ray tubes) according to Bellotti" I didn't think magnetic-core memory and CRTs were interchangeable...
- dpcx 10y ago
  I think the implication wasn't that they were interchangeable, but that the machine they were expecting to see used CRTs, and instead, they found it using MCM.
  - mtdewcmu 10y ago
    I didn't think that CRTs were a memory technology, but apparently they have been: https://en.wikipedia.org/wiki/Williams_tube https://en.wikipedia.org/wiki/Williams_tube
- jasonm23 10y ago
  To expect CRT/Williams Tube memory is a slightly bizarre assumption, it's a technology that was only very briefly used (developed in 1947 and used only in machines in the early 1950s, all but disappearing by 1956). It's also pretty much unheard of now. While magnetic core memory was dominant from 1955 until around 1976, when silicon went mainstream. Machines with MCM were in general use for a long time after that too. It's possible the interviewer confused the widespread use of CRT in pre-flat screen monitors. (Shrug, maybe CRT was very briefly mentioned? who knows?!) Williams Tube memory is pretty interesting BTW. Worth taking a look at Selectron memory too. You know, for fun.
- deleted 10y ago
  [deleted]
- mbellotti 10y ago
  This is the one thing the article got wrong about the talk. The 7074 replaced magnetic drums with core memory. It's an easy mistake to make since the 700 series used vacuum tube logic and the 7070 replaced those with transistors. The 7074 came about two years after that.
deleted 10y ago
[deleted]
brokentone 10y ago
I'm not sure the article did well answering the headline. I now know that there is a mainframe (emulator it seems from additional research) that returns API responses very quickly (1-6ms) TO a Java app... which apparently is inefficient as the page render takes 6-10 seconds. Interesting problems not terribly well described in the post, would love to read more.
- robertlagrant 10y ago
  Yeah it describes pretty basic "discoveries", and reads a bit like a book report. Would love some detail.
altitudinous 10y ago
If it ain't broken, don't fix it.
platz 10y ago
Remember COBOL.net !
nl 10y ago
Did anyone read this and go "hu"? There's this whole thing about how they are getting data from mainframes where "the data was being returned in between one and six milliseconds". But then: "harvest that data from the magnetic tape and load them up into more traditional databases. That Java application was extracting the data from the databases" But then: "But the data from the mainframes was actually arriving (from its new home in the database) in less than six milliseconds. The bottleneck was — of course — the Java application." So of course it is entirely possibly to write slow Java applications. But then the story seems to end! So what happened? Did they fix the application?
chris_wot 10y ago
You get sued for violating copyright?
asjo 10y ago
EBCDIC stacktraces!
mleonhard 10y ago
She likes those crufty legacy systems only because she works with each one for a short time. If she had to work on one every day for years, she would hate it, too.

230 points by 3n7r0pY 10y ago

The thing is that there was a lot about those old systems that was slow, so you were very, very careful how you programmed. You tended not to use vast library stacks, you went close to the metal and you coded in languages like Assembler, COBOL or FORTRAN. I/O was often run through specialised co-processors (such as IBM's channel processors) and the terminals could sometimes help too. I have friends who have been looking after legacy applications for an airline running on Unisys. The core apps for reservation, Cargo booking and weight/balance were written in FORTRAN. In recent times, the front end was written in Java to give web access. They tried to rewrite the core apps but it was impossible to do so and get the performance.

eru 10y ago
> The thing is that there was a lot about those old systems that was slow, so you were very, very careful how you programmed. That's a common sentiment. I wish I could find the quote by someone who made the transition; it was about how happy they were to be able to compile so much quicker, and how getting immediate feedback made them so much more productive.
- 3n7r0pY 10y ago
  The notion of waiting ages for programs to compile or assemble is mostly related to the older hardware. I compile/assemble COBOL and IBM's assembly language on a z13 daily and it's pretty much instantaneous.
  - eru 10y ago
    > The notion of waiting ages for programs to compile or assemble is mostly related to the older hardware. Oh, I was talking more about older ways to organize the data centre: batch vs timeshare processing. https://en.wikipedia.org/wiki/Time-sharing https://en.wikipedia.org/wiki/Time-sharing
YeGoblynQueenne 10y ago
>> They tried to rewrite the core apps but it was impossible to do so and get the performance. Well, Cobol is a bit like the C of mainframes - you can manipulate memory directly and so on. You can't really do that sort of thing with Java.
- stefs 10y ago
  a) if it was really running on the old hardware; in that case ruby on a modern machine would have been several magnitudes faster than the original code - at least because of the faster IO b) if the whole thing was indeed running in an emulator, the emulation overhead would have negated all direct memory access advantages
  - wahern 10y ago
    Emulators on mainframes are much more sophisticated and performant than is typical on x86 and ARM platforms. The hardware and even software is often designed with emulation in mind, not just for backwards compatibility but for forwards compatibility, too.
    - andai 10y ago
      Do you mean to say that they were running a mainframe emulator on a mainframe?
      - wahern 10y ago
        Compilers for some IBM mainframes have for decades (since the 1970s, I think) targeted an intermediate virtual machine instruction set which is then translated on-the-fly to the local architecture by the OS upon execution. So in the case of IBM their machines are truly built with both forward and backward compatibility in mind. The pointers for this instruction set have been 128 bits since the beginning, long before 64-bit hardware even came into being. Some (or all?) of the latest Unisys mainframes run on Intel Xeons, but with custom chips for translating the machine code of their old architectures. I don't work in this area. I just like reading about it. Though, unfortunately, it's difficult to find clear specifications and descriptions on how these architectures work. For example, Unisys' 2200 ClearPath architecture is one of the (if not _the_) last architectures still sold that uses signed-magnitude representation, as well as having an odd-sized integer width of 39 bits. (INT_MAX is 549755813887 and INT_MIN is -549755813887, and the compiler has to "emulate" the modulo arithmetic semantics required of unsigned types in C. ClearPath MCP is also a POSIX system, and has to emulate an 8-bit char type.) I discovered that you could download the specification for their C compiler for free online, which was useful when discussing the relevancy of undefined behavior in C. But AFAIU (and this is where finding concrete details is more difficult) the latest models of the ClearPath line use Xeons with custom chips bolted on to help run the machine code of the older architecture. In any event, the point is that while the old architecture is arguably emulated, it's not a pure software emulation that you might assume, and the resulting performance is better than the previous models of those mainframes, which were still being built at least until a few years ago. In other words, direct memory access isn't ruled out because the I/O systems may have been intentionally designed to work efficiently in a backwards compatible manner.

deleted 10y ago

[deleted]

skissane 10y ago

Is someone in the US government really still using an IBM 7074? Really? I'm shocked. How could that possibly be cost-effective? Actually, this blog post makes the story clearer: http://nikhilism.com/post/2016/systems-we-love/ http://nikhilism.com/post/2016/systems-we-love/ It isn't a physical IBM 7074. When it came time to migrate from 7074 to S/360, rather than rewriting their 7074 software, they just wrote a 7074 emulator for S/360. And, it sounds like, they are still running their 7074 software, under their 7074 emulator, most likely on a recent z/Architecture mainframe. The article makes it sound like people still use "1960s mainframes" when I very much doubt anyone is still running 1960s hardware in production. People use modern machines–modern IBM mainframes, which are multicore 64-bit processors–or other mainframe vendors such as Unisys or Fujitsu use mainly I believe x86-64 running Linux running a software emulator for the old mainframe CPU. A lot of legacy, sure, but I think this article makes it sound even more legacy than it really is.

facepalm 10y ago
I was going to ask if they couldn't just replace the hardware with emulators :-)
wolf550e 10y ago
I know of an S/360 assembly code base that (as of a decade ago), still ran code that assumed 24 bit address space and used 31 bit pointers with tags stuffed in the unused 7 bits. So it had to run in the lowest 16MB of memory, on a 64-bit machine. I don't think they had the budget to rewrite that subsystem to fix it. So yes, new hardware, but some of the legacy problems in the software run very deep.
cle 10y ago
It's "cost effective" because "cost effectiveness" is not as objective as we think, once we start considering things like risk tolerance.
mbellotti 10y ago
> A lot of legacy, sure, but I think this article makes it sound even more legacy than it really is. Indeed. The point of the talk was that 1) legacy is often assumed to be bad not for any real technical reasons but just because it is legacy and 2) a lot of what was being presented as legacy wasn't even legacy. Their OS 2200 version was actually newer than the Oracle DB they were using on the "modern" side of the stack.

wolf550e 10y ago

An explanation from 'bcantrill about why the talk was not recorded: https://lobste.rs/s/q2xz14/systems_we_love_videos/comments/if9yox#c_if9yox https://lobste.rs/s/q2xz14/systems_we_love_videos/comments/i...

geodel 10y ago

I have heard from many of my friends about massive projects of 'modernizing' mainframe applications with Java stack. Java did not deliver improvement that management was expecting. Once the project consumed all the budget for ~25% completion, they were scrapped. I think, though without any proof, that overreliance of 100s of mixed quality libraries, combined with 'best practices' of enterprise development and heavy application of design patterns creates a very large surface area for change. This makes reasonable translation of functionality to Java almost impossible.

neeleshs 10y ago
Was it Java that did not deliver improvement, or was it the team? :)
- geodel 10y ago
  I am inclined to say Java as I have not seen mythical teams who work on Java without whole caboodle of 'Enterprise Apps' culture.
  - electrum 10y ago
    You can write modern software on Java without using "enterprise" features: https://github.com/prestodb/presto https://github.com/prestodb/presto
    - geodel 10y ago
      Nice. It may be the best way to organize project this large. I know this is maven thing and once using everyone's favorite Intellij IDEA it should not matter but I personally find ~1000 directories for ~4000 files a kind of Javasim.
    - bsg75 10y ago
      Can you expand on what makes the Presto project different than the aforementioned enterprise approach? (Not a Java developer, so I don't have context)

walrus01 10y ago

When I first saw the Chromebook, my thought was that Larry Ellison's 1998 dream of a network computer thin terminal had come true. It's smarter than a true thin terminal, but everything lives in the cloud (err, butt).

bitwize 10y ago
That Chrome extension is clbuttic.
- TeMPOraL 10y ago
  Indeed. > but everything lives in my butt (err, butt). I had to open the comment in porn (er, incognito) mode in order to parse it :D.
lokedhs 10y ago
Being someone who worked for Sun at the time, I'd rather say it was Scott McNealy's dream.

YeGoblynQueenne 10y ago

>> It was at this point that the seasoned data architects in the department began expressing their exasperation. “15 years ago, everybody was telling us ‘Get off the mainframe, get on AT&T applications, build these thick clients. Mainframes are out.’ And now thick clients are out, and everybody’s moving to APIs and microservices, which basically are very similar to the thin client that a terminal uses to interact with a mainframe.” A.k.a. "Them as not knows their history are doomed to repeat it". But I suppose it's more the case of business imperatives than real ignorance that drive this mad race to make new stuff that works worse than the old stuff only so we can then go back to the old stuff with a different name. Btw, that lady is my new tech hero: “The systems that I love are really the systems that other engineers hate,” Bellotti told the audience — “the messy, archaic, half chewing gum and duct tape systems that are sort of patched together. <3 <3 <3

eternalban 10y ago
"15 years ago" doesn't sound right. Thick clients were being pushed in mid 90s. But then Clipper Chip plans went south ...
- toyg 10y ago
  Consider that state/federal bureaucracies get on "trends" with a delay of 2 or 3 years minimum, and the story is probably from 2015 or thereabout - the agency was launched in 2014.
krylon 10y ago
Wish I could upvote this twice.

mbellotti 10y ago

This article was such a pleasant surprise! I loved talking at Systems We Love and I love talking about legacy architecture in general. Bryan and the Joyent team were very accommodating and understanding. The White House is an amazing place to work (even now), but it's not a system that understands conference talks very well. Wish we could have a video, but it was a heavy lift just getting the bureaucracy okay with the idea that I was going to talk about information that was already public, without naming agencies or projects and without going into detail beyond what was in a user manual and that this was not a security risk.

benballjr 10y ago
Hi Marianne! I'm with The New Stack (not the writer of this article), and was wondering about the video myself. Interesting to see you address that there's definitely no public facing version, as it seems a lot of the commenters here would love to see it. I can only imagine the bureaucrat complexity involved.
- mbellotti 10y ago
  Oh yeah, when I talked to the agency that owns the 7074 (more on this in a second) they were like "You can't do this because we will get hacked" "How are you going to get hacked if I talk about your mainframe? It's not connected to the public internet, is it?" "No. Well... we don't know... but ... hackers! Hackers are really smart Marianne." Part of the compromise was that I promised I would only use information that was already available publicly through government reports and news articles. I went back through my talk and documented where each fact was already published somewhere else until they were comfortable with it. So the ambiguity on whether the 7074 was the actual machine or an emulator was deliberate... there were certain things I could not find a public comment on and therefore agreed to avoid making direct statements about. This all seems super annoying, but it makes sense when you realize how heavily scrutinized public servants are. In the end they are only trying to protect me, my organization and Obama's legacy. Three things that are really important to me. So I can't exactly blame them for it. I was happy to be able to find a middle ground where they felt comfortable, the organizers weren't too badly inconvenienced and I got to give the talk I wanted to.

stevehiehn 10y ago

Hmm, interesting. My bet is that the Java devs were not great at implementing concurrency. But who knows.

kevin_thibedeau 10y ago
Probably lots of XML transforms going on to be enterprisey.

Animats 10y ago

I'm still struggling with my own legacy code problem. I'm reviving an old LISP program from the early 1980s. Parts of it were written for the original Stanford AI Lab SAIL system in the 1970s. It last ran under Franz LISP in 1986. My current struggle is with one line of code: (defun getenode (l) (cadr l)) That ought to be simple enough. But it's being applied not to a list, but a "hunk". A "hunk" is an obsolete MacLISP concept.[1]. It's a block of memory which has N contiguous LISP cells, each with two pointers. This is the memory object underlying structures and arrays in MacLISP. Macros were used to create the illusion of structure data objects, with hunks underneath. However, you could still access a "hunk" with car, cdr, cxr, etc. I'm converting this to Common LISP, which has real structures, but not hunks. That, with some new macro support, works for the regular structure operations. So far, so good. But which element of the structure does (cadr l), which usually means the same thing as "(car (cdr l))", access? (cadr (list 0 1 2 4)) returns 1, so you'd think it would be field 1 of the structure. But no. It's more complicated and depends on how hunks are laid out in memory. The Franz LISP manual from 1983 [2] says "Although hunks are not list cells, you can still access the first two hunk elements with cdr and car and you can access any hunk element with cxr†." At footnote "†", "In a hunk, the function cdr references the first element and car the second." This is backwards from the way lists behave. A blog posting from 2008 about MacLISP says "A Maclisp hunk was a structure like a cons cell that could hold an arbitrary number of pointers, up to total of 512. Each of these slots in a hunk was referred to as a numbered cxr, with a numbering scheme that went like this: ( cxr-1 cxr-2 cxr-3 ... cxr-n cxr-0 ). No matter how many slots were in the hunk, car was equivalent to (cxr 1 hunk) and cdr was equivalent to (cxr 0 hunk)." Note that element 0 is at the end, which is even stranger. The documentation is silent about what "cadr" would do. Does it get element 2, or get element 0 and then apply "car" to it? The original code [3] contains no relevant comments. I'm trying to figure out from the context what the original author, Greg Nelson, had in mind. He died in 2015.[4] [1] http://www.mschaef.com/blog/tech/lisp/car-cdr.html http://www.mschaef.com/blog/tech/lisp/car-cdr.html [2] http://www.softwarepreservation.org/projects/LISP/franz/Franz_Lisp_July_1983.pdf http://www.softwarepreservation.org/projects/LISP/franz/Fran... [3] https://github.com/John-Nagle/pasv/blob/master/src/CPC4/z.lisp https://github.com/John-Nagle/pasv/blob/master/src/CPC4/z.li... [4] https://en.wikipedia.org/wiki/Greg_Nelson_(computer_scientist) https://en.wikipedia.org/wiki/Greg_Nelson_(computer_scientis...

mtdewcmu 10y ago

"It used magnetic-core memory (instead of cathode ray tubes) according to Bellotti" I didn't think magnetic-core memory and CRTs were interchangeable...

dpcx 10y ago
I think the implication wasn't that they were interchangeable, but that the machine they were expecting to see used CRTs, and instead, they found it using MCM.
- mtdewcmu 10y ago
  I didn't think that CRTs were a memory technology, but apparently they have been: https://en.wikipedia.org/wiki/Williams_tube https://en.wikipedia.org/wiki/Williams_tube
jasonm23 10y ago
To expect CRT/Williams Tube memory is a slightly bizarre assumption, it's a technology that was only very briefly used (developed in 1947 and used only in machines in the early 1950s, all but disappearing by 1956). It's also pretty much unheard of now. While magnetic core memory was dominant from 1955 until around 1976, when silicon went mainstream. Machines with MCM were in general use for a long time after that too. It's possible the interviewer confused the widespread use of CRT in pre-flat screen monitors. (Shrug, maybe CRT was very briefly mentioned? who knows?!) Williams Tube memory is pretty interesting BTW. Worth taking a look at Selectron memory too. You know, for fun.
deleted 10y ago
[deleted]
mbellotti 10y ago
This is the one thing the article got wrong about the talk. The 7074 replaced magnetic drums with core memory. It's an easy mistake to make since the 700 series used vacuum tube logic and the 7070 replaced those with transistors. The 7074 came about two years after that.

deleted 10y ago

[deleted]

brokentone 10y ago

I'm not sure the article did well answering the headline. I now know that there is a mainframe (emulator it seems from additional research) that returns API responses very quickly (1-6ms) TO a Java app... which apparently is inefficient as the page render takes 6-10 seconds. Interesting problems not terribly well described in the post, would love to read more.

robertlagrant 10y ago
Yeah it describes pretty basic "discoveries", and reads a bit like a book report. Would love some detail.

altitudinous 10y ago

If it ain't broken, don't fix it.

platz 10y ago

Remember COBOL.net !

nl 10y ago

Did anyone read this and go "hu"? There's this whole thing about how they are getting data from mainframes where "the data was being returned in between one and six milliseconds". But then: "harvest that data from the magnetic tape and load them up into more traditional databases. That Java application was extracting the data from the databases" But then: "But the data from the mainframes was actually arriving (from its new home in the database) in less than six milliseconds. The bottleneck was — of course — the Java application." So of course it is entirely possibly to write slow Java applications. But then the story seems to end! So what happened? Did they fix the application?

chris_wot 10y ago

You get sued for violating copyright?

asjo 10y ago

EBCDIC stacktraces!

mleonhard 10y ago

She likes those crufty legacy systems only because she works with each one for a short time. If she had to work on one every day for years, she would hate it, too.