5 ms·
> Why are programming environments still so primitive? Because we as an industry made a strategic decision in the late 20th century to value run-time efficienc
by lisper 8y ago
> Why are programming environments still so primitive?
Because we as an industry made a strategic decision in the late 20th century to value run-time efficiency over all other quality metrics, a decision which has manifested itself in the primacy of C and its derivatives. Everything else has been sacrificed in the name of run time efficiency, including, notably, security. Development convenience was also among the collateral damage.
> Why can't I debug a function without restarting my program?
Because you use C or one of its derivatives instead of Lisp or one of its derivatives. In Common Lisp you can not only debug a function without restarting your program, you can redefine classes without restarting your program. It is truly awesome. You should try it some time.
- candiodari 8y ago> Development convenience was also among the collateral damage. Don't you find that web & javascript are pretty much a straight denial of your argument ? They're "secure" (meaning we let anyone's javascript code just run in our browsers, even embedded in other people's code, and seriously expect no ill effects) They're extremely inconvenient to develop with. Especially compared to those "run-time above all else" environments you mention. For one, you need to know 5-6 languages to use the web. > Because you use C or one of its derivatives instead of Lisp or one of its derivatives. In Common Lisp you can not only debug a function without restarting your program, you can redefine classes without restarting your program. It is truly awesome. You should try it some time. I think you'll find that pretty much any environment allows this. Even without debug symbols you mostly can do this for C, C++, ... programs. On the web, you can't, because every call immediately gets you into overcomplicated minified libraries that you can't change anyway, assuming it doesn't go into a remote call entirely. And there's environments that go further. .Net not only lets you debug any C function you run within it, you can even modify it's code from within the debugger and "replay" to the same point. I believe there's a few more proprietary compilers that support that functionality too.
- lisper 8y ago> Don't you find that web & javascript are pretty much a straight denial of your argument ? You should probably direct that question at Patrick because his original question was kind of based on the premise that the answer to your question is "no". My personal opinion? No. OMG no. Javascript and HTML are both poorly designed re-inventions of tiny little corners of Lisp. In that regard they are improvements over C. But no. OMG no. > Even without debug symbols you mostly can do this for C, C++, ... programs No, you can't. You can grovel around on the stack and muck with the data structures, but you can't redefine a function, or redefine a class, or change a class's inheritance structure without restarting your program. In Common Lisp you can do all of these things.
- pbhjpbhj 8y agoCan you/someone explain for this noob why you can't redefine a function, etc.? I'm not a programmer, so I'm imagining you hook in to the call that addresses the function (like modify a jump instruction), overwrite any registers that need changing, nullify caches, so the program runs new code -- this I think is how some hacks work? Ultimately couldn't you just write NOP to addresses used for a function? Is it something structural about C/C++ that stops this, like the ownership of memory (I'm assuming a superuser can tell the computer to ignore that addresses written to are reserved for the program being modified). How does the computer know that you pushed a different long jump in to a particular address and stop working, rather than keep going processing instructions. Apologies if I've misunderstood, please be gentle.
- lisper 8y agoIt's a good question, and I'm a little disappointed no one has answered this yet. There is no reason you couldn't redefine a function in C. It's really more of a cultural constraint than a technical one. It's a little tricky, but it could be done. It just so happens that C programmers don't think in terms of interactive programming because C started out as a non-interactive language, and so it has remained mostly a non-interactive language. Lisp, by way of contrast, was interactive and dynamic on day 1, and it has stayed that way. In the early days that made C substantially faster than Lisp, but that performance gap has mostly closed. However, there are some things that Lisp does that are actually impossible in C. The two most notable ones are garbage collection and tail recursion. It's impossible to write a proper GC in portable C because there is no way to walk the stack, and there is no way to compile a tail-call unless you put the entire program inside the body of a single function.
- Waterluvian 8y ago> On the web, you can't, because every call immediately gets you into overcomplicated minified libraries that you can't change anyway, assuming it doesn't go into a remote call entirely. This is true if you go to a production website and try to start debugging. But it's untrue for any modern development environment. The minification comes later and even then, source maps are first class supported in browsers, mapping the minified code to the source. It's funny. In my experience the web debug tools are some of the best of any language/environment I've experienced.
- thrower123 8y agoNo, just no. Web debugging is worse than anything I've ever seen - maybe a bit better than cmd-line debugging with gdb, but not much. Sourcemaps supposedly work, though I have never seen this actually work in practice. And since babel is apparently indispensible, you can't be entirely sure that conditionals and statements in your source haven't been optimized away in the transpilation. Routinely one sets breakpoints in JS files that are not hit by Chrome dev tools, and symbols that should be in scope don't want to be defined. It's a mess. I suppose if you know how to structure the rube goldberg machine correctly, web dev can be productive. But it's so hard, and the hardness is of the tedious yak-shaving variety. I just hate it and want to fire up Visual Studio and write some .NET apps with tools that just work instead.
- Waterluvian 8y ago"Rube Goldberg" is the best description I've heard for a JS dev stack. So true. Its so awful. But its so much better than it used to be and I pray it continues to improve. I hear what you're saying. They can be really finicky. I've had very good luck using it all cleanly without issue. I especially love binding vscode to an open browser so that I use vscode for all the inspection, breakpoints, etc. But I also experienced your lamentations. It took a long time for me to get it all working. Sourcemaps were so unreliable years ago. Now they just seem to work, having learned all the painful lessons about configuration. There still aren't any sane defaults and the ground won't stop shifting. But now that I have it working, it works great.
- sjtgraham 8y ago> They're "secure" (meaning we let anyone's javascript code just run in our browsers, even embedded in other people's code, and seriously expect no ill effects) I don’t expect that. Use after free vuln + heap spray + shellcode = Ill effects
- ken 8y ago> They're extremely inconvenient to develop with. Especially compared to those "run-time above all else" environments you mention. For one, you need to know 5-6 languages to use the web. I'm not sure how this is a denial of lisper's argument. You've picked an environment which you admit has major flaws, but those flaws are independent of the issue at hand. I've worked in runtime-perf-above-anything programming environments which required the use of many different languages even for the simplest program. It's terrible there, too. That has nothing to do with dynamic languages. In fact, due to the ease of creating DSLs, most of the dynamic languages I've used allow one to get by with using fewer languages. > On the web, you can't [do this other thing you can do in Lisp], because ... Indeed, you've picked the one modern dynamic environment which lacks most of the features lisper is talking about. That's not an argument against having those features. I think it's mostly an observation that this particular environment picked a different attribute (runtime security) to optimize for above all else. You'll note that JS-style security is fundamentally incompatible with many of the concepts in OP's original question. > I think you'll find that pretty much any environment allows this. Even without debug symbols you mostly can do this for C, C++, ... programs. Can you give an example? I've never heard of a C++ system that let you redefine classes at runtime without debug symbols. I can't imagine how it would work. How would you even inspect the classes at runtime to find out what you're redefining?
- dasmoth 8y agoI agree regarding the symptoms, but am less convinced that run-time efficiency is the sole (or perhaps even major) cause. If it were, I'd argue that we'd see less usage of, e.g., Python. I don't know the true cause -- I wish I did. But I do do see a trend towards an ever more "hands-off" style of software development: large volumes of automated (especially unit) tests in preference to interactive approaches (or hybrids, like running subsets of tests via a REPL), and running test instances on remote servers (often via extra layers of indirection, such as CI servers) rather than running on your own machine. I'd love to see a resurgence of interactivity, but if anything it seems to be going against recent trends. Edit: The subjective impression I've got is that doing things the indirect, hands-off way is being presented as somehow more "professional" while hands-on interactivity is the realm of the self-taught and hackers. Do others see this? What can be done to change these perceptions?
- lisper 8y agoThis decision was made and become deeply entrenched long before Python came along. Python is pretty good, but even it is constrained by the decisions that went into the design of C, since Python is implemented in C. Its underlying data structures are C data structures. Its memory management is C memory management. The GIL is there because C.
- jstimpfle 8y ago> The GIL is there because C. Oh please.
- zbentley 8y ago> The GIL is there because C. That's not at all true. There are very, very few platforms/libraries/languages that both a) allow you to manage concurrent access to complex, arbitrary data structures in a sane, predictable, and not-dangerous way and b) provide the capabilities (including performance) necessary to implement a general-purpose programming language on top of them. Even fewer of those tools existed when Python came about. Was it technically possible to make a GIL-free Python or a Python not based in C? Sure. But it wasn't in any way likely, or a reasonable-at-the-time decision. If you look into the history of the GIL things will make more sense; it has next to nothing to do with the implementation language.
- shimon 8y agoMaybe the reason for this has more to do with the type of development structure that helps a language get scale. The rich powerful development environments you've described exist primarily in proprietary, integrated environments. If you want to integrate the editor, debugger, OS, and language, it helps to be able to coordinate the design of all those components. On the other hand, languages that have gotten to huge popular scale have typically been more open in their specification and implementation process. Perhaps this is because the creators of breakthrough tools that drive language adoption (like web frameworks or data science kits) prefer these tools, or because the sorts of conditions that lead to creation of such tools are inherently marginalized ones. In other words, if you're a happy iOS developer using a lovely integrated world of xcode and swift, you're not going to spot a dramatically underserved development niche.
- lisper 8y agoBoth Scheme and Common Lisp have been open standards since their inception. And both have excellent IDEs available, both commercial and open-source (Clozure CL for Common Lisp and Racket for Scheme). I did my masters thesis in 1986 on Coral Common Lisp on a Macintosh Plus with 800k floppies and 1 MB of RAM. It had an IDE that would still be competitive today, indeed is in some ways still superior to anything available today. All this was possible because it was Common Lisp and not C. The language design really is a huge factor. (Coral has today evolved into Clozure Common Lisp, which is completely free and open (Apache license). You really should try it.)
- shimon 8y agoLisps are great. Why do you think they haven't been at the forefront of any big trends in application development? Things like the web, mobile apps, data science ... My guess at an argument here is that the languages popular in the 80s drove the curriculum design of most Computer Science education, and the relative absence of the Lisps (to today) makes the languages seem less approachable to practicing programmers than they really should be. On the other hand, you can find a lot of Lisp's influence in something like Python (though obviously with many differences both superficial and deep). So in that case, why are Python IDEs so much worse than what you'd see in Lisp? (And is that even the case? Maybe there's just more Python devs and thus more IDEs and like anything, most are crap; but if there one or two great ones then does Lisp really have an advantage there?)
- petra 8y agoOne reason(or maybe the common excuse) is often "we need to have full control". That's a reason most managers can get behind. So that's a common reason slowing every shift to a higher level language or library, to tools that automatically create stuff for you(GUI, optimized code from dsl), to using a platform controlled by another company(one important channel for newer tech, with some strong ux benefits over open-source), to trusting open source.
- jstimpfle 8y ago> One reason(or maybe the common excuse) is often "we need to have full control". That's a reason most managers can get behind. On modern machines, this excuse is even better. Full control can easily mean 100x difference in speed. Most importantly, how do contemporary lisps deal with arrays of unboxed structs or primitives?
- deleted 8y ago[deleted]
- rjtobin 8y ago> You should try it some time. Worth noting that Patrick Collison was/is a Lisp user. While in high school, he won a fairly prestigious Irish national science-fair-type contest (the “Young Scientist”) with an AI bot written in some Lisp dialect. IIRC he also contributed some patches to one of the major Lisp projects.
- lisper 8y agoYes, I know. He was signed up for Google's Summer of Code (I forget the exact year) to do a CL project and I was going to be his mentor. But before it started he quit and founded Stripe instead. :-)
- jstimpfle 8y ago>> Why can't I debug a function without restarting my program? > Because you use C or one of its derivatives instead of Lisp or one of its derivatives. In Common Lisp you can not only debug a function without restarting your program, you can redefine classes without restarting your program. It is truly awesome. You should try it some time. There is no technical reason why it shouldn't be possible in C, if you are willing to do without many optimizations. One approach is to make a function stub that simply jumps into the currently loaded implementation. A more efficient but more convoluted way is to patch the address of the currently loaded implementation at all call sites. The problem is that in general you can't simply replace a function without either restarting the process or risking to crash it. In general functions have some implementation-dependent context that is accumulated in the running process, and a different implementation does the accumulation in a different way. I'm not a lisper, but there is no way this is different in LISP. (And it is not because in C you often use global data. "Global" vs "local" is only a syntactic distinction anyway). If you're willing to risk crashing your process that's okay. It's often a fine choice. And you can do it in C. The easiest way to implement approach #1 is to load DLLs / shared objects.
- uxp 8y agoThere is no technical limitation why it is impossible in C, however the raw details of how C is implemented make it entirely impractical. A super generalized description of a C function might check if a register contains a positive value. If so, it will then jump to address 0x42, which is a memory offset, which is the beginning of another function. Its near impossible to "swap" out what lies at 0x42 since that was defined at compile time and is included in a monolithic executable. Looking at more dynamic languages, like C#, Java or LISP, they run on a virtual or abstracted machine instead of raw registers. This means that a similarly defined function will instead jump to whatever matches the requirement. That means that we could have a lookup table that says we should jump to a symbol of S42, and based on what we have loaded in memory, S42 resides at 0x42. Essentially, all functions are function pointers, and we can change the value the resides in that memory address in order to swap out implementations of whatever function maintains the same signature as the intended function. This is why one can make trivial changes to C# in visual studio while stopped at a breakpoint and have those changes applied to the running program. Instead of jumping to 0x42, we jump to 0x84 by "hotswapping" the value of the pointer we're about to jump into. Obviously this isn't entirely the truth, there are a lot more nuances and it's a fair bit more complicated than this, but the idea should hold water.
- 0x445442 8y agoDon't forget Smalltalk has these capabilities as well.
- bigdubs 8y agoShocked he mentions Visual Studio Code but not Visual Studio itself. The .net CLR has a lot of the features that he would want to enable the kind of interactive debugging (variable hovers, REPL) he talks about, and Visual Studio itself supports a lot of them. Personally going from doing C# development in VS2k15 to doing golang development in VSCode feels like going back in time.
- seanmcdirmid 8y agoVisual studio has a lot of power, especially if we consider Intellitrace, but damn is it hard to use that power. I really think the main problem here is UX and not technology.
- qwerty456127 8y agoThe main problem of Visual Studio is that it doesn't work on free operating systems.
- seanmcdirmid 8y agoNo. I mean, this might be a problem for some people, that it doesn’t work on OS X (not free) is a problem for me. But a bigger problem is that even on Windows its more advanced features are often too hard to be usable except for very special situations.
- qwerty456127 8y agoWell, I have narrowed the statement to be about free OSes because there is a version of VisualStudio for MacOS as far as I've heard and because whatever works on Linux/FreeBSD usually works on MacOS too. I mean it should be cross-platform in the way VSCode is. I don't use Linux for ideological reasons, I use it because it works a lot faster (especially when using PyCharm - the difference is drastic) and more reliable than Windows and gives me "almost-MacOS-and-much-more" experience on my PC. Modern apps targeting professionals and enthusiasts should be cross-platform (run on Windows, Linux and MacOS) and not force you to choose from just one or two major OSes.
- kazagistar 8y agoUm, why is everyone forgetting Java?
- xg15 8y ago> Because we as an industry made a strategic decision in the late 20th century to value run-time efficiency over all other quality metrics Except if that were the case, we wouldn't have so many bloated framework code and towers of abstractions, dragging runtime efficiency to the bottom.
- baud147258 8y ago> Why can't I debug a function without restarting my program? Because you use C or one of its derivatives It's doable in java which is, I think, a derivative of C.