4 ms·

I'm a big fan of incremental builds - no point in recompiling 1M LOC when only a few source files have changed. Image based systems (like Common Lisp) are parti

by smanek 18y ago

I'm a big fan of incremental builds - no point in recompiling 1M LOC when only a few source files have changed. Image based systems (like Common Lisp) are particularly nice because you can just recompile the functions that have changed and have the changes instantly take effect on a running image. Unfortunately, ASDF (the most common build/packaging system) doesn't have particularly good support for incremental builds yet, but it's an area of active developement (particularly with XCVB and other new build systems). But don't GCC, Java, and Visual Studio all support parallel builds pretty easily? I've never looked into it, but I know I've read articles about it. What sort of language/env. does FogCreek use? I vaguely remember hearing that they had written their own compiler in Java that transformed ASP to PHP or some other such ludicrous thing. Edit: Some googling showed that they decided their software had to run on Windows and Unix boxes, and that the best way to do that was to support ASP (pre .NET) on Windows and PHP on *nix. They already had a large ASP code base, so they wrote a compiler (named Thistle) that could translate a subset of ASP into PHP. Then, it seems like MS deprecated ASP (and they started to realize how much it sucked) so they wrote their own language (Wasabi) that is based on VBScript and can be compiled into ASP, PHP, or JS. My God. I would gouge my eyes out if I had to work there. From the fragments of code I've seen, it frankly seems horrible. See http://www.fogcreek.com/FogBugz/blog/category/Wasabi.aspx http://www.fogcreek.com/FogBugz/blog/category/Wasabi.aspx Although, I guess I can't really judge. I recently just wrote some ridiculously convoluted Java code that could automatically and transparently trick the JVM into making certain recursive functions Tail Call Optimizing (i.e., use constant space on the stack, regardless of how deep the recursion is) by transforming the function into automatically issuing/catching Exceptions to manually unwind the stack when I want to. Ah, I miss Common Lisp - I never knew how good I had it.

thwarted 18y ago
If you define your dependencies correctly, make can do parallel builds (using the -j option). Of course you will have to use Makefiles in that case, which a lot of people don't like.
- smanek 18y ago
  Yep, right you are. That seems like a much more reasonable way to go. Parallelize the build - not the compiler.
- gord 18y ago
  Maybe part of that is the pain of dealing with configure / autoconf / autotools? Ive seen a lot of C/C++ programmers write their own build utilities/scripts and lose proper deps in the process. I found cmake a reasonable alternative.. scons apparently is too, although I hesitate because it seems to encourage writing your own python build plugins, which can be too tempting for people who like to tinker. agreed, make -j works a treat, and presumably will gain more as cores multiply. Its kind of nice to work in a scripting environment and avoid some of these problems [via sensible defaults]
spolsky 18y ago
Seriously? You would gouge your eyes out? Most good hackers that I know actually prefer to work on compilers. We like the fact that we can change the compiler and control our destiny... we've added code generation, active records, lambdas, picture functions, and a lot of other nice stuff to our code without having to rewrite or port any working code. Does your language let you write one function and have it run on the browser or the server? When you want to change something about your language, can you? People seem to be pretty quick to criticize the decisions we made without really having all the facts, based on a couple of bits and pieces of obsolete info they gathered from the internet.
- smanek 18y ago
  That's a fair point - I don't have all the facts. I do respect that you had a huge (and effective) code base in ASP that you (for very valid business reasons) didn't want to rewrite from scratch. I understand that each of the decisions you have made have probably made you successful and profitable - and in your shoes I probably would have done something similar. From your point of view, I completely understand that your choices were to do a complete rewrite (which was obviously unacceptable), or to come up with these sort of hackish workarounds. However, I think you have to understand that as a hypothetical programmer thinking about working at Fog Creek, my choice is to work on your large legacy code base in (what appears to me to be) a not very pleasant language - or work on fun little 3 month AI projects in whatever language I choose (which is roughly what I do now). The fact that your compiler infers type information from a variable's name is an ugly hack - and frankly scares me (not only itself, but also more than a little about other design decisions you made). The hoops you guys had to jump through with Wasabi to do metaprogramming would annoy me - I'd rather focus on the fun stuff. I recognize that, as a business, you made good decisions. But do you understand why I, as a programmer, would not want to work with those sort of technologies? Oh, and for what it's worth, my language has all the features you've mentioned. For example, we used Parenscript (http://common-lisp.net/project/parenscript/ http://common-lisp.net/project/parenscript/) at my last job - it's a subset of common lisp that can be compiled to Javascript and runs on either the client or server. Also - sorry about the 'Gouge my eyes out' bit. I meant it to be a bit of funny hyperbole, but guess I failed to convey that properly (humor is hard on the internet). Fogcreek, in my mind, would definitely be above the average Java or .Net shop, but probably below a few fun research labs, prototyping contractors, or small startups.
wglb 18y ago
I remember a chapter or half a chapter in a compiler construction book or article that was effectively titled "Why spend time on compiler optimization", a delightfully ambiguous term. The point was that doing so sped up the compiler which saved time for a large number of users (this was back in the days of compilers as commercial products). Clearly the discussion was targeted to self-compiling compilers. But the lesson here would be that some effort spent on the speed of compile could positively impact developer productivity or at least negatively affect swordfight time. And yes, I think that very short compile times, like you see in CL (sbcl as an example) or Smalltalk can have an avalanche effect on development. And yes I do believe that small incremental changes are th3e best way to do software development. And yes I would expect to recompile a function in a running image serving web pages as a routine part of development. But maybe that is just me.

by smanek 18y ago

thwarted 18y ago

If you define your dependencies correctly, make can do parallel builds (using the -j option). Of course you will have to use Makefiles in that case, which a lot of people don't like.

smanek 18y ago
Yep, right you are. That seems like a much more reasonable way to go. Parallelize the build - not the compiler.
gord 18y ago
Maybe part of that is the pain of dealing with configure / autoconf / autotools? Ive seen a lot of C/C++ programmers write their own build utilities/scripts and lose proper deps in the process. I found cmake a reasonable alternative.. scons apparently is too, although I hesitate because it seems to encourage writing your own python build plugins, which can be too tempting for people who like to tinker. agreed, make -j works a treat, and presumably will gain more as cores multiply. Its kind of nice to work in a scripting environment and avoid some of these problems [via sensible defaults]

spolsky 18y ago

Seriously? You would gouge your eyes out? Most good hackers that I know actually prefer to work on compilers. We like the fact that we can change the compiler and control our destiny... we've added code generation, active records, lambdas, picture functions, and a lot of other nice stuff to our code without having to rewrite or port any working code. Does your language let you write one function and have it run on the browser or the server? When you want to change something about your language, can you? People seem to be pretty quick to criticize the decisions we made without really having all the facts, based on a couple of bits and pieces of obsolete info they gathered from the internet.

smanek 18y ago
That's a fair point - I don't have all the facts. I do respect that you had a huge (and effective) code base in ASP that you (for very valid business reasons) didn't want to rewrite from scratch. I understand that each of the decisions you have made have probably made you successful and profitable - and in your shoes I probably would have done something similar. From your point of view, I completely understand that your choices were to do a complete rewrite (which was obviously unacceptable), or to come up with these sort of hackish workarounds. However, I think you have to understand that as a hypothetical programmer thinking about working at Fog Creek, my choice is to work on your large legacy code base in (what appears to me to be) a not very pleasant language - or work on fun little 3 month AI projects in whatever language I choose (which is roughly what I do now). The fact that your compiler infers type information from a variable's name is an ugly hack - and frankly scares me (not only itself, but also more than a little about other design decisions you made). The hoops you guys had to jump through with Wasabi to do metaprogramming would annoy me - I'd rather focus on the fun stuff. I recognize that, as a business, you made good decisions. But do you understand why I, as a programmer, would not want to work with those sort of technologies? Oh, and for what it's worth, my language has all the features you've mentioned. For example, we used Parenscript (http://common-lisp.net/project/parenscript/ http://common-lisp.net/project/parenscript/) at my last job - it's a subset of common lisp that can be compiled to Javascript and runs on either the client or server. Also - sorry about the 'Gouge my eyes out' bit. I meant it to be a bit of funny hyperbole, but guess I failed to convey that properly (humor is hard on the internet). Fogcreek, in my mind, would definitely be above the average Java or .Net shop, but probably below a few fun research labs, prototyping contractors, or small startups.

wglb 18y ago

I remember a chapter or half a chapter in a compiler construction book or article that was effectively titled "Why spend time on compiler optimization", a delightfully ambiguous term. The point was that doing so sped up the compiler which saved time for a large number of users (this was back in the days of compilers as commercial products). Clearly the discussion was targeted to self-compiling compilers. But the lesson here would be that some effort spent on the speed of compile could positively impact developer productivity or at least negatively affect swordfight time. And yes, I think that very short compile times, like you see in CL (sbcl as an example) or Smalltalk can have an avalanche effect on development. And yes I do believe that small incremental changes are th3e best way to do software development. And yes I would expect to recompile a function in a running image serving web pages as a routine part of development. But maybe that is just me.