4 ms·

I stopped seeing him as an unfailing source of truth after he okayed developing fogbugz with a private, in-house language, that (unsurprisingly) was a huge wast

I stopped seeing him as an unfailing source of truth after he okayed developing fogbugz with a private, in-house language, that (unsurprisingly) was a huge waste of resources.

tptacek 10y ago
This is something that people say a lot about FogBugz that you oddly don't hear so often from people who actually worked there. For a site catering mostly to software developers, this place sure is afraid of computer science sometimes. Ooh, look, a parser --- run away!. No wonder everything in our industry is an ugly blob of JSON.
- skybrian 10y ago
  I think the main advantage is that if you emit JSON then you know that someone else isn't going to write a bad parser for it. Given that we can't depend on people to do simple things like handling escape sequences correctly, that seems pretty important. (Also, the 'jq' command is nice.) Fogbugz is (was?) writing a language for internal use only, so it's a better fit there.
  - lightcatcher 10y ago
    > Given that we can't depend on people to do simple things like handling escape sequences correctly, that seems pretty important. (Also, the 'jq' command is nice.) jq stores all numbers as doubles, which means it can't handle 64 bit ints (or larger integer numeric types). It's otherwise a great tool and I wouldn't point out a negative such as this, but you did explicitly mention that you know JSON won't be fed to a bad parser.
    - drodgers 10y ago
      That is what almost all JSON parsers do; even the spec says that you shouldn't rely on more precision than a double: This specification allows implementations to set limits on the range and precision of numbers accepted. Since software that implements IEEE 754-2008 binary64 (double precision) numbers [IEEE754] is generally available and widely used, good interoperability can be achieved by implementations that expect no more precision or range than these provide, in the sense that implementations will approximate JSON numbers within the expected precision. A JSON number such as 1E400 or 3.141592653589793238462643383279 may indicate potential interoperability problems, since it suggests that the software that created it expects receiving software to have greater capabilities for numeric magnitude and precision than is widely available.
      - lightcatcher 10y ago
        Good to know about the JSON spec. The only JSON parser I used before jq was the Python stdlib parser. Python JSON correctly handles big integers, which lead to my (out of spec) expectations for jq to do the right thing.
    - user5994461 10y ago
      > jq stores all numbers as doubles, which means it can't handle 64 bit ints (or larger integer numeric types). JSON = JavaScript Object Notation. Javascript do NOT have integers. It's representing all numbers as double, which causes all kind of weird bugs and rounding errors.
    - skybrian 10y ago
      Good to know. Yeah, that's one of the ways JSON kinda sucks. If you have 64-bit numbers, you have to quote them to be safe.
- enraged_camel 10y ago
  On the other hand, don't we always hear people say that it's a really bad idea to write your own crypto algorithm, and you should use something that has been extensively tested and vetted instead? I would posit the same applies to writing your own programming language: it may be fun to do as a hobby, or when you are learning, but may not be a good idea to use it in production to rewrite your company's flagship software.
  - tptacek 10y ago
    The two problems aren't remotely comparable.
    - enraged_camel 10y ago
      Why not?
      - tptacek 10y ago
        The most obvious reason is that it's much, much easier to design a new programming language --- especially the extremely simple kind of language that Wasabi was --- than it is to build a new cryptosystem even with vetted primitives. The second is that the impact of any crypto error you make is very likely to be game-over, while most programming languages aren't (you can observe this empirically from MRI Ruby). A third is that designing new programming languages, especially the simple kind that Wasabi, is a straightforward well-understood engineering task. Nobody with a CS degree should be incapable of doing it competently. You cannot say that about cryptography. In fact: many of the problems people attempt to solve in cryptography (take, for instance, multi-party secure messaging) don't even have well-understood best-practices solutions; they're still active topics of research. Not to do better, the way Haskell was an attempt to at innovating a better functional language, but to reliably do at all. I could generate more reasons. This is just the top of my head. There is simply no comparison between the two problems. I implemented a new scripting language for building TCP/IP stacks in my first professional programming job ever --- in fact, it was the first programming assignment I got in that first dev job, and I had to do it in C. I handled it comfortably. 20 years later, and I still wouldn't be comfortable implementing most cryptosystems myself.
- jmcgough 10y ago
  It's not about being afraid of computer science, it's about being pragmatic and efficient about the problems that you choose to solve. By choosing to develop your own closed-source language, you now have two projects that need to be developed, the product itself and the language that it depends on. They retired the language (Wasabi) some years later specifically because it was so demanding to maintain, there was significant on-boarding for any new hires, and it constrained their developers by frequently being behind the current state of C# (which it compiled to). When you write your own language, you don't have an entire open source community updating it, fixing bugs, and maturing it with new features and libraries. That is now work that your team needs to take on, and in the vast majority of cases, the negatives far outweigh the positives.
- throwanem 10y ago
  I'm not afraid of parsers - I like them so well that I have to be careful not to use them on problems a regex or two will much more maintainably solve. But I would hesitate somewhat to join a team which used a language developed entirely in house. As enjoyable as such work would undoubtedly be, it also reeks of overengineering, unless the problem space is so unique that no extant language can effectively address it. Experience suggests that project management software is not such a space, and the maintainability risk of a bespoke language seems therefore very hard to justify. On the other hand, I don't pretend to be a "top dev", by anyone's definition - top quartile, maybe, on a very good day. So perhaps I'm just not favorably enough placed on the capability curve to understand why it really is the great idea that it doesn't seem to be.
35bge57dtjku 10y ago
I became jaded after the piece about how doing a full re-write is never a good idea.
carterehsmith 10y ago
Yeah, I remember reading that, it was really surprising. I chalked it up to being a management thing. As in "Sometimes you want to let people make mistakes".
Rapzid 10y ago
If you read some of their postmortem stuff on that whole situation(which is pretty much an engineering meme at this point) they are in no way willing to admit that as a mistake. Tump would have a run for his money on avoiding admittance of mistakes..

I stopped seeing him as an unfailing source of truth after he okayed developing fogbugz with a private, in-house language, that (unsurprisingly) was a huge waste of resources.

tptacek 10y ago

This is something that people say a lot about FogBugz that you oddly don't hear so often from people who actually worked there. For a site catering mostly to software developers, this place sure is afraid of computer science sometimes. Ooh, look, a parser --- run away!. No wonder everything in our industry is an ugly blob of JSON.

skybrian 10y ago
I think the main advantage is that if you emit JSON then you know that someone else isn't going to write a bad parser for it. Given that we can't depend on people to do simple things like handling escape sequences correctly, that seems pretty important. (Also, the 'jq' command is nice.) Fogbugz is (was?) writing a language for internal use only, so it's a better fit there.
- lightcatcher 10y ago
  > Given that we can't depend on people to do simple things like handling escape sequences correctly, that seems pretty important. (Also, the 'jq' command is nice.) jq stores all numbers as doubles, which means it can't handle 64 bit ints (or larger integer numeric types). It's otherwise a great tool and I wouldn't point out a negative such as this, but you did explicitly mention that you know JSON won't be fed to a bad parser.
  - drodgers 10y ago
    That is what almost all JSON parsers do; even the spec says that you shouldn't rely on more precision than a double: This specification allows implementations to set limits on the range and precision of numbers accepted. Since software that implements IEEE 754-2008 binary64 (double precision) numbers [IEEE754] is generally available and widely used, good interoperability can be achieved by implementations that expect no more precision or range than these provide, in the sense that implementations will approximate JSON numbers within the expected precision. A JSON number such as 1E400 or 3.141592653589793238462643383279 may indicate potential interoperability problems, since it suggests that the software that created it expects receiving software to have greater capabilities for numeric magnitude and precision than is widely available.
    - lightcatcher 10y ago
      Good to know about the JSON spec. The only JSON parser I used before jq was the Python stdlib parser. Python JSON correctly handles big integers, which lead to my (out of spec) expectations for jq to do the right thing.
  - user5994461 10y ago
    > jq stores all numbers as doubles, which means it can't handle 64 bit ints (or larger integer numeric types). JSON = JavaScript Object Notation. Javascript do NOT have integers. It's representing all numbers as double, which causes all kind of weird bugs and rounding errors.
  - skybrian 10y ago
    Good to know. Yeah, that's one of the ways JSON kinda sucks. If you have 64-bit numbers, you have to quote them to be safe.
enraged_camel 10y ago
On the other hand, don't we always hear people say that it's a really bad idea to write your own crypto algorithm, and you should use something that has been extensively tested and vetted instead? I would posit the same applies to writing your own programming language: it may be fun to do as a hobby, or when you are learning, but may not be a good idea to use it in production to rewrite your company's flagship software.
- tptacek 10y ago
  The two problems aren't remotely comparable.
  - enraged_camel 10y ago
    Why not?
    - tptacek 10y ago
      The most obvious reason is that it's much, much easier to design a new programming language --- especially the extremely simple kind of language that Wasabi was --- than it is to build a new cryptosystem even with vetted primitives. The second is that the impact of any crypto error you make is very likely to be game-over, while most programming languages aren't (you can observe this empirically from MRI Ruby). A third is that designing new programming languages, especially the simple kind that Wasabi, is a straightforward well-understood engineering task. Nobody with a CS degree should be incapable of doing it competently. You cannot say that about cryptography. In fact: many of the problems people attempt to solve in cryptography (take, for instance, multi-party secure messaging) don't even have well-understood best-practices solutions; they're still active topics of research. Not to do better, the way Haskell was an attempt to at innovating a better functional language, but to reliably do at all. I could generate more reasons. This is just the top of my head. There is simply no comparison between the two problems. I implemented a new scripting language for building TCP/IP stacks in my first professional programming job ever --- in fact, it was the first programming assignment I got in that first dev job, and I had to do it in C. I handled it comfortably. 20 years later, and I still wouldn't be comfortable implementing most cryptosystems myself.
jmcgough 10y ago
It's not about being afraid of computer science, it's about being pragmatic and efficient about the problems that you choose to solve. By choosing to develop your own closed-source language, you now have two projects that need to be developed, the product itself and the language that it depends on. They retired the language (Wasabi) some years later specifically because it was so demanding to maintain, there was significant on-boarding for any new hires, and it constrained their developers by frequently being behind the current state of C# (which it compiled to). When you write your own language, you don't have an entire open source community updating it, fixing bugs, and maturing it with new features and libraries. That is now work that your team needs to take on, and in the vast majority of cases, the negatives far outweigh the positives.
throwanem 10y ago
I'm not afraid of parsers - I like them so well that I have to be careful not to use them on problems a regex or two will much more maintainably solve. But I would hesitate somewhat to join a team which used a language developed entirely in house. As enjoyable as such work would undoubtedly be, it also reeks of overengineering, unless the problem space is so unique that no extant language can effectively address it. Experience suggests that project management software is not such a space, and the maintainability risk of a bespoke language seems therefore very hard to justify. On the other hand, I don't pretend to be a "top dev", by anyone's definition - top quartile, maybe, on a very good day. So perhaps I'm just not favorably enough placed on the capability curve to understand why it really is the great idea that it doesn't seem to be.

35bge57dtjku 10y ago

I became jaded after the piece about how doing a full re-write is never a good idea.

carterehsmith 10y ago

Yeah, I remember reading that, it was really surprising. I chalked it up to being a management thing. As in "Sometimes you want to let people make mistakes".

Rapzid 10y ago

If you read some of their postmortem stuff on that whole situation(which is pretty much an engineering meme at this point) they are in no way willing to admit that as a mistake. Tump would have a run for his money on avoiding admittance of mistakes..