8 ms·

Mu: making programs easier to understand in the large

122 points by peterkelly 11y ago

sja 11y ago
I'm a little hesitant to learn how to 'understand large programs', or how to structure them for that matter, from a repository that has 100 code files in the root directory. I'm not sure what I'm looking at?
- mintplant 11y ago
  Try the README: https://github.com/akkartik/mu/blob/master/Readme.md https://github.com/akkartik/mu/blob/master/Readme.md It's a new programming language. edit: The source seems to be structured intentionally in a recommended reading order. See the comment at the top of https://github.com/akkartik/mu/blob/master/000organization.cc https://github.com/akkartik/mu/blob/master/000organization.c... Layers of code are filled in as you read down the list of files. It's an interesting concept, similar to literate programming. (And FWIW, these source files are C++ code for the compiler, not an example of Mu code.) The author also has a blog post up here: http://akkartik.name/post/mu http://akkartik.name/post/mu
  - deleted 11y ago
    [deleted]
  - akkartik 11y ago
    Thanks for figuring all that out and laying it out so clearly! More details on my flavor of literate programming: http://akkartik.name/post/wart-layers http://akkartik.name/post/wart-layers
- ktRolster 11y ago
  His idea is that the way to understand a large computer program is by running it, testing it. So he's made (what I understand) a very nice UI for introspection of a running computer program. You can see what is happening while it's running. Designing a programming language is kind of like a 'rite of passage.' I think most of us have at least thought about how to make languages better.
  - akkartik 11y ago
    Heh, I've already had my programming language rite of passage[1]. Mu was my attempt to not build yet another language. Hence the lack of syntax. Thanks for your kind words! I'm gratified that somebody understood what I'm trying to do in spite of my crappy writing skills. [1] http://akkartik.name/post/wart http://akkartik.name/post/wart
    - ktRolster 11y ago
      I'm glad I got it right :) I like your idea, and I think there's room for improved introspection in debugging. At the same time, I think that if a program can't be understood without being executed, then it's already lost (in terms of readability). Certainly, for example, if you have a threaded program, you need to be able to visually inspect and verify that there will be no deadlocks or race conditions when the program is run, because you won't be able to test every possible condition in a debugger.
      - akkartik 11y ago
        It might partly be a personality-type thing. I can't imagine visually inspecting and verifying anything if I didn't write it to begin with. I wouldn't know where to begin thinking of possible ways to break it. On the other hand, it seems far more natural for the author to say, "here are the race conditions I considered, these points where a context switch would be maximally inconvenient. And still lo the program works." This way the reader doesn't have to recreate such situations from scratch. He or she just has to verify the provided situations, or notice a scenario that was missed. That seems like an easier ask from someone new to the project. (Mu scenarios can't insert context switches yet, but it's planned.) Perhaps it would help to think of the dichotomy as between the rules and the state space of inputs they handle, rather than between reading and running. Seemingly simple code can often hide surprising subtleties. Why is this line written just like so and not thus? How does everything turn out just right in this one situation? Tests help to record the right questions for the reader to ask.
        11y ago
- akkartik 11y ago
  Author here. For my part, I have never understood why people consider it to be a good thing to squirrel code into a bunch of different sub-directories. a) It makes the build scripts more complicated, which means they'll be more likely to break on some poor noob, and that when they break it'll be less likely the noob will be able to tell how to fix it. b) Invariably the codebase accumulates dependencies between directories that are uneconomic to reorganize. At least a flat directory has a shot at under-promising and over-delivering. c) Maybe people do it to make the place look neat, they way I used to 'clean my room' by stuffing all my dirty clothes into drawers. But codebases that are messes at a deep level are less likely to be cleaned up if they look clean at a superficial level. And all codebases eventually turn into messes, the way we've done things so far.
  - btrask 11y ago
    Sure, directories can be used badly (and the Java and Ruby projects I've seen are often very difficult to navigate because so much is the structure of the language, rather than the structure of the project). But they can also be used well, and the promise of a meaningful directory structure is really appealing. Here's a question, given all of what I understand about your layered way of structuring code: how easy is it to spin off logical portions of a project? If my project accretes its own HTTP server and I want to turn that into a stand alone library, could I? Would it be possible to "rewrite history" so that the HTTP-related changes never appeared in the layers to start with? Thanks!
    - akkartik 11y ago
      Yeah, that's a good question. Even though the mu core is at the top directory, there are a couple of apps in sub-directories of their own. To run a single-file mu program you say: $ ./mu factorial.mu To run a more complex app, you just give a directory name rather than a filename: $ ./mu edit Files inside the directory are loaded in numeric order just like at the top level. You can also run just a subset of the layers for the editor: $ ./mu test edit/001* $ ./mu test edit/00[12]* $ ./mu test edit/00[1-3]* (Notice how the number of tests/dots grows at each layer.) Since the layers are just regular files there's nothing stopping you from rewriting history as much as you want. That ability was precisely what I built them for. I've only recently started using directories, so I'm sure there's stuff here I haven't considered. Feedback most welcome.
antiquark 11y ago
How is "subtract a, 3" better than "a-3"? I can already feel my carpal tunnel acting up!
- FenugreekAcerb 11y ago
  Some say (for example Moore) that arithmetic is only a tiny part of programming and don't deserve special syntax (especially with a language that lacks C-style for loops).
  - Terr_ 11y ago
    ...Well, a tiny part of whose programming? Programming languages (at least the not-so-academic kind) need to mesh with the problem-spaces being solved, and there are definitely some contexts where most of the work is encoding and applying arithmetic rules.
  - ktRolster 11y ago
    Really all you need for programming are > < + - . , [ ] Anything more than that is for wimps. Brain is the only language I ever program in.
    - slang800 11y ago
      I think you mean "Brainfuck", I can't find a programming language called "Brain".
      - ktRolster 11y ago
        I haven't been posting here long, so I wasn't sure if it was acceptable to type 'fuck.' So I self-censored by replacing it with asterisks, which hackernews promptly ate. So yes, you are right of course.
        akkartik 11y ago
        Each pair of asterisks turned into an empty italic block due to HN's formatting rules. You can see them if you view source :)
desireco42 11y ago
Definitely could use better presentation. It is interesting, reminds me of forth a little, where you compose a lot of small commands.
kazinator 11y ago
> Imagine a world where you can: > [...] think of a tiny improvement to a program you use, clone its sources, orient yourself on its organization and make your tiny improvement, all in a single afternoon. If we alter that to "three programs you use [...] all in a single afternoon", we can plonk that into my resume.
vinodkd 11y ago
Things I really liked: + recording runs as tests + one library for all layers of abstraction + building in assembler to avoid the big runtime. + using literate programming techniques to manage coding in assembly. + scenario, "screen-should-contain" and "assume-keyboard". awesome. + spaces: nice primitive of the closure and similar concepts + attributes for meta programs. again, awesome. + labels and using [,] as labels. genius. Suggestions: * If there's a valid reason to call them recipes, go ahead. If they're actully just functions, please stick with what everyone knows and gets. Same thing with "reagents","reply", "ingredients" and "products". Update: I did read your blog post about this. still think the regular names are better in the long run. * I dont know where you're going to with multiple types of the "number:list" kind, but if so, the map must look the same, not lispy. In all, I really like where you're going with this. Kudos! I find a lot of resonance of ideas I've had for a long time now.
- akkartik 11y ago
  Wow, that's a very detailed look. Thanks! Feel free to email me; my address is in my profile. The terminology is definitely a work in progress. It's mostly for my attempts to teach programming using Mu. I noticed that mathematical words intimidated some students. I tried to write up my rationale here: http://akkartik.name/post/mu http://akkartik.name/post/mu. But I've certainly started mixing up the terms with my student, so it might not last.
  - vinodkd 11y ago
    Will do. And yes, I did read your rationale. The intent is not bad, the names probably are.
  - slang800 11y ago
    > Functions, arguments, classes, methods, objects, threads, locks, all these are reassuring everyday words, and yet their meaning in programming (and math) bears no relation to their everyday meaning. (from http://akkartik.name/post/mu http://akkartik.name/post/mu) I can see how the use of the term "arguments" could be confusing ("parameters" or "inputs" would make more sense), and "threads" is a rather tenuous metaphor for how scheduling works within a kernel, but all of the others mirror their real-world meaning pretty well. I'd be more worried about students needing to unlearn "containers", "ingredients", and "reagents" when they start reading material from outside of Mu, talking to other developers, or learning calculus (which uses functions, sets, and arrays).
vmorgulis 11y ago
The preprocessor reminds me a bit MyDef: https://github.com/hzhou/MyDef https://github.com/hzhou/MyDef
- akkartik 11y ago
  Indeed! I had a stimulating conversation with Hui Zhou last year.
dman 11y ago
I think the project would benefit from a Terminology / Definitions section which lists in one table the names and crisp definitions of the concepts being introduced. It appears that you are using non standard terms so having a single list of the terms would allow readers to have that list open when they are reading through the layers.
- akkartik 11y ago
  That's a good idea. Might http://akkartik.github.io/mu/html/010vm.cc.html http://akkartik.github.io/mu/html/010vm.cc.html [1] be such a big-picture map? Let me think about how to improve it; I haven't touched it in a while. Further suggestions most welcome. [1] Colorized version of https://github.com/akkartik/mu/blob/379244666c/010vm.cc https://github.com/akkartik/mu/blob/379244666c/010vm.cc
  - dman 11y ago
    I was thinking something much simpler. If you were writing a book about Mu what would be the definitions you would print on the back inner cover? A table with two columns - first column being a term and the second column containing its description in one or two sentences. Could you elaborate on the target audience for the doc you linked to? I am unable to make out whether that doc is intended for serious programmers in other languages coming across Mu for the first time or whether its intended for new comers to programming.
    - akkartik 11y ago
      Yeah the intended audience is existing C++ programmers coming across Mu for the first time. For teaching programming I mostly imagine I'll be doing it in person. Mu isn't yet ready for learning programming unassisted.
enqk 11y ago
So I wrote an implementation of linear search in Mu using its array type. https://gist.github.com/uucidl/df81b6ab0fe65713f5ba https://gist.github.com/uucidl/df81b6ab0fe65713f5ba My notes: - creating an array w/ create-array is possible, however I could not find how to take its address and pass it around - one cannot get the address of an invalid position of the array, so one cannot construct bounded ranges (STL style) using a begin/end pair of addresses - the interpreter strongly suggests to use refcounted pointers (address:shared) instead of addresses even when one does not borrow the memory - could not figure out how to write a test scenario that uses checks named memory addresses rather than ordinal memory addresses (memory-should-contain instruction)
- akkartik 11y ago
  That's great!! Yes, create-array was just a version I created so I could teach arrays separate from new and addresses. But it creates an array on the 'stack' (default space) so there's no way to take an address of it. I quickly take my students past it to new. memory-should-contain doesn't currently support named locations, sorry. Part of the problem is that with spaces a name can have many different addresses in different functions. So my tests write stuff I want to check in raw numbered locations, and the first 1000 addresses are reserved for tests so that names can never be clobbered. I looked at Rust's borrow checker for a bit but wasn't smart enough to understand how it works or transplant it easily. I also noticed that it wasn't smart enough to deal with things like doubly-linked lists without reaching for ref-counting. So I figured I'd keep things simple and just use ref-counting for everything. That way I punt on all the complicated static checks in favor of a simple runtime one. The rule is: new returns shared:address, and get-address and index-address (and maybe-convert) return address. Use shared:address to pass things around between functions, and reserve non-shared addresses only for short-term operations, usually mutations. Since non-shared addresses are not dynamically allocated, there's no possibility of use-after-free so they don't need to be refcounted, and you can copy them around as much as you like. Use-after-free and related memory corruption is really the only thing I'm concerned about protecting my users from. Memory leaks I plan to have tools for, so that programmers can identify and break them down when memory becomes a concern, but not worry about until then. I call this "zero-developer-cost abstractions" :) It feels less restrictive and more dynamic than Rust. Edit: I just took a look at your code, and it feels perfectly idiomatic. Nice job. Only issue I found was that you forgot to specify the outputs of find in its header. So its calls end up doing some runtime type-checking. I should probably raise a warning in this situation. Thanks again.

122 points by peterkelly 11y ago

sja 11y ago

I'm a little hesitant to learn how to 'understand large programs', or how to structure them for that matter, from a repository that has 100 code files in the root directory. I'm not sure what I'm looking at?

mintplant 11y ago
Try the README: https://github.com/akkartik/mu/blob/master/Readme.md https://github.com/akkartik/mu/blob/master/Readme.md It's a new programming language. edit: The source seems to be structured intentionally in a recommended reading order. See the comment at the top of https://github.com/akkartik/mu/blob/master/000organization.cc https://github.com/akkartik/mu/blob/master/000organization.c... Layers of code are filled in as you read down the list of files. It's an interesting concept, similar to literate programming. (And FWIW, these source files are C++ code for the compiler, not an example of Mu code.) The author also has a blog post up here: http://akkartik.name/post/mu http://akkartik.name/post/mu
- deleted 11y ago
  [deleted]
- akkartik 11y ago
  Thanks for figuring all that out and laying it out so clearly! More details on my flavor of literate programming: http://akkartik.name/post/wart-layers http://akkartik.name/post/wart-layers
ktRolster 11y ago
His idea is that the way to understand a large computer program is by running it, testing it. So he's made (what I understand) a very nice UI for introspection of a running computer program. You can see what is happening while it's running. Designing a programming language is kind of like a 'rite of passage.' I think most of us have at least thought about how to make languages better.
- akkartik 11y ago
  Heh, I've already had my programming language rite of passage[1]. Mu was my attempt to not build yet another language. Hence the lack of syntax. Thanks for your kind words! I'm gratified that somebody understood what I'm trying to do in spite of my crappy writing skills. [1] http://akkartik.name/post/wart http://akkartik.name/post/wart
  - ktRolster 11y ago
    I'm glad I got it right :) I like your idea, and I think there's room for improved introspection in debugging. At the same time, I think that if a program can't be understood without being executed, then it's already lost (in terms of readability). Certainly, for example, if you have a threaded program, you need to be able to visually inspect and verify that there will be no deadlocks or race conditions when the program is run, because you won't be able to test every possible condition in a debugger.
    - akkartik 11y ago
      It might partly be a personality-type thing. I can't imagine visually inspecting and verifying anything if I didn't write it to begin with. I wouldn't know where to begin thinking of possible ways to break it. On the other hand, it seems far more natural for the author to say, "here are the race conditions I considered, these points where a context switch would be maximally inconvenient. And still lo the program works." This way the reader doesn't have to recreate such situations from scratch. He or she just has to verify the provided situations, or notice a scenario that was missed. That seems like an easier ask from someone new to the project. (Mu scenarios can't insert context switches yet, but it's planned.) Perhaps it would help to think of the dichotomy as between the rules and the state space of inputs they handle, rather than between reading and running. Seemingly simple code can often hide surprising subtleties. Why is this line written just like so and not thus? How does everything turn out just right in this one situation? Tests help to record the right questions for the reader to ask.
      - 11y ago
akkartik 11y ago
Author here. For my part, I have never understood why people consider it to be a good thing to squirrel code into a bunch of different sub-directories. a) It makes the build scripts more complicated, which means they'll be more likely to break on some poor noob, and that when they break it'll be less likely the noob will be able to tell how to fix it. b) Invariably the codebase accumulates dependencies between directories that are uneconomic to reorganize. At least a flat directory has a shot at under-promising and over-delivering. c) Maybe people do it to make the place look neat, they way I used to 'clean my room' by stuffing all my dirty clothes into drawers. But codebases that are messes at a deep level are less likely to be cleaned up if they look clean at a superficial level. And all codebases eventually turn into messes, the way we've done things so far.
- btrask 11y ago
  Sure, directories can be used badly (and the Java and Ruby projects I've seen are often very difficult to navigate because so much is the structure of the language, rather than the structure of the project). But they can also be used well, and the promise of a meaningful directory structure is really appealing. Here's a question, given all of what I understand about your layered way of structuring code: how easy is it to spin off logical portions of a project? If my project accretes its own HTTP server and I want to turn that into a stand alone library, could I? Would it be possible to "rewrite history" so that the HTTP-related changes never appeared in the layers to start with? Thanks!
  - akkartik 11y ago
    Yeah, that's a good question. Even though the mu core is at the top directory, there are a couple of apps in sub-directories of their own. To run a single-file mu program you say: $ ./mu factorial.mu To run a more complex app, you just give a directory name rather than a filename: $ ./mu edit Files inside the directory are loaded in numeric order just like at the top level. You can also run just a subset of the layers for the editor: $ ./mu test edit/001* $ ./mu test edit/00[12]* $ ./mu test edit/00[1-3]* (Notice how the number of tests/dots grows at each layer.) Since the layers are just regular files there's nothing stopping you from rewriting history as much as you want. That ability was precisely what I built them for. I've only recently started using directories, so I'm sure there's stuff here I haven't considered. Feedback most welcome.

antiquark 11y ago

How is "subtract a, 3" better than "a-3"? I can already feel my carpal tunnel acting up!

FenugreekAcerb 11y ago
Some say (for example Moore) that arithmetic is only a tiny part of programming and don't deserve special syntax (especially with a language that lacks C-style for loops).
- Terr_ 11y ago
  ...Well, a tiny part of whose programming? Programming languages (at least the not-so-academic kind) need to mesh with the problem-spaces being solved, and there are definitely some contexts where most of the work is encoding and applying arithmetic rules.
- ktRolster 11y ago
  Really all you need for programming are > < + - . , [ ] Anything more than that is for wimps. Brain is the only language I ever program in.
  - slang800 11y ago
    I think you mean "Brainfuck", I can't find a programming language called "Brain".
    - ktRolster 11y ago
      I haven't been posting here long, so I wasn't sure if it was acceptable to type 'fuck.' So I self-censored by replacing it with asterisks, which hackernews promptly ate. So yes, you are right of course.
      - akkartik 11y ago
        Each pair of asterisks turned into an empty italic block due to HN's formatting rules. You can see them if you view source :)

desireco42 11y ago

Definitely could use better presentation. It is interesting, reminds me of forth a little, where you compose a lot of small commands.

kazinator 11y ago

> Imagine a world where you can: > [...] think of a tiny improvement to a program you use, clone its sources, orient yourself on its organization and make your tiny improvement, all in a single afternoon. If we alter that to "three programs you use [...] all in a single afternoon", we can plonk that into my resume.

vinodkd 11y ago

Things I really liked: + recording runs as tests + one library for all layers of abstraction + building in assembler to avoid the big runtime. + using literate programming techniques to manage coding in assembly. + scenario, "screen-should-contain" and "assume-keyboard". awesome. + spaces: nice primitive of the closure and similar concepts + attributes for meta programs. again, awesome. + labels and using [,] as labels. genius. Suggestions: * If there's a valid reason to call them recipes, go ahead. If they're actully just functions, please stick with what everyone knows and gets. Same thing with "reagents","reply", "ingredients" and "products". Update: I did read your blog post about this. still think the regular names are better in the long run. * I dont know where you're going to with multiple types of the "number:list" kind, but if so, the map must look the same, not lispy. In all, I really like where you're going with this. Kudos! I find a lot of resonance of ideas I've had for a long time now.

akkartik 11y ago
Wow, that's a very detailed look. Thanks! Feel free to email me; my address is in my profile. The terminology is definitely a work in progress. It's mostly for my attempts to teach programming using Mu. I noticed that mathematical words intimidated some students. I tried to write up my rationale here: http://akkartik.name/post/mu http://akkartik.name/post/mu. But I've certainly started mixing up the terms with my student, so it might not last.
- vinodkd 11y ago
  Will do. And yes, I did read your rationale. The intent is not bad, the names probably are.
- slang800 11y ago
  > Functions, arguments, classes, methods, objects, threads, locks, all these are reassuring everyday words, and yet their meaning in programming (and math) bears no relation to their everyday meaning. (from http://akkartik.name/post/mu http://akkartik.name/post/mu) I can see how the use of the term "arguments" could be confusing ("parameters" or "inputs" would make more sense), and "threads" is a rather tenuous metaphor for how scheduling works within a kernel, but all of the others mirror their real-world meaning pretty well. I'd be more worried about students needing to unlearn "containers", "ingredients", and "reagents" when they start reading material from outside of Mu, talking to other developers, or learning calculus (which uses functions, sets, and arrays).

vmorgulis 11y ago

The preprocessor reminds me a bit MyDef: https://github.com/hzhou/MyDef https://github.com/hzhou/MyDef

akkartik 11y ago
Indeed! I had a stimulating conversation with Hui Zhou last year.

dman 11y ago

I think the project would benefit from a Terminology / Definitions section which lists in one table the names and crisp definitions of the concepts being introduced. It appears that you are using non standard terms so having a single list of the terms would allow readers to have that list open when they are reading through the layers.

akkartik 11y ago
That's a good idea. Might http://akkartik.github.io/mu/html/010vm.cc.html http://akkartik.github.io/mu/html/010vm.cc.html [1] be such a big-picture map? Let me think about how to improve it; I haven't touched it in a while. Further suggestions most welcome. [1] Colorized version of https://github.com/akkartik/mu/blob/379244666c/010vm.cc https://github.com/akkartik/mu/blob/379244666c/010vm.cc
- dman 11y ago
  I was thinking something much simpler. If you were writing a book about Mu what would be the definitions you would print on the back inner cover? A table with two columns - first column being a term and the second column containing its description in one or two sentences. Could you elaborate on the target audience for the doc you linked to? I am unable to make out whether that doc is intended for serious programmers in other languages coming across Mu for the first time or whether its intended for new comers to programming.
  - akkartik 11y ago
    Yeah the intended audience is existing C++ programmers coming across Mu for the first time. For teaching programming I mostly imagine I'll be doing it in person. Mu isn't yet ready for learning programming unassisted.

enqk 11y ago

So I wrote an implementation of linear search in Mu using its array type. https://gist.github.com/uucidl/df81b6ab0fe65713f5ba https://gist.github.com/uucidl/df81b6ab0fe65713f5ba My notes: - creating an array w/ create-array is possible, however I could not find how to take its address and pass it around - one cannot get the address of an invalid position of the array, so one cannot construct bounded ranges (STL style) using a begin/end pair of addresses - the interpreter strongly suggests to use refcounted pointers (address:shared) instead of addresses even when one does not borrow the memory - could not figure out how to write a test scenario that uses checks named memory addresses rather than ordinal memory addresses (memory-should-contain instruction)

akkartik 11y ago
That's great!! Yes, create-array was just a version I created so I could teach arrays separate from new and addresses. But it creates an array on the 'stack' (default space) so there's no way to take an address of it. I quickly take my students past it to new. memory-should-contain doesn't currently support named locations, sorry. Part of the problem is that with spaces a name can have many different addresses in different functions. So my tests write stuff I want to check in raw numbered locations, and the first 1000 addresses are reserved for tests so that names can never be clobbered. I looked at Rust's borrow checker for a bit but wasn't smart enough to understand how it works or transplant it easily. I also noticed that it wasn't smart enough to deal with things like doubly-linked lists without reaching for ref-counting. So I figured I'd keep things simple and just use ref-counting for everything. That way I punt on all the complicated static checks in favor of a simple runtime one. The rule is: new returns shared:address, and get-address and index-address (and maybe-convert) return address. Use shared:address to pass things around between functions, and reserve non-shared addresses only for short-term operations, usually mutations. Since non-shared addresses are not dynamically allocated, there's no possibility of use-after-free so they don't need to be refcounted, and you can copy them around as much as you like. Use-after-free and related memory corruption is really the only thing I'm concerned about protecting my users from. Memory leaks I plan to have tools for, so that programmers can identify and break them down when memory becomes a concern, but not worry about until then. I call this "zero-developer-cost abstractions" :) It feels less restrictive and more dynamic than Rust. Edit: I just took a look at your code, and it feels perfectly idiomatic. Nice job. Only issue I found was that you forgot to specify the outputs of find in its header. So its calls end up doing some runtime type-checking. I should probably raise a warning in this situation. Thanks again.