4 ms·

Things humans are bad at: organizing files, topological sorting, ontologies. Things computers are good at: organizing files, topological sorting, ontologies.

Things humans are bad at: organizing files, topological sorting, ontologies. Things computers are good at: organizing files, topological sorting, ontologies. The point is I wish we could just ask the computer to sort things out for us, show different views, etc. this would require a software system able to work with more granular code units, maybe something like [1]. I still have some ptsd from huge refactoring where a big part of the hassle was to manually create directories, move files around, fix imports, build files, cyclic dependency errors, sigh… I suspect a good solution would be to have a language agnostic dsl to describe a program’s data flow, then use it to generate the boilerplate of functions involved. Then you could ask the system to show you only the pieces of the code strictly related to a certain feature, hopefully never again have to deal with file structure manually. 1: https://www.unison-lang.org/ https://www.unison-lang.org/

MetaWhirledPeas 3y ago
> The point is I wish we could just ask the computer to sort things out for us, show different views, etc. I agree. I'm usually opinionated about a related subject: code formatting. But I'm not too opinionated in one case, where we're using Prettier. The project is autoformatted and expects everyone else to do the same. It's so refreshing to let the computer take over sometimes, even if the end result isn't exactly what I would have typed out myself. If a piece of software could so something similar with the folders, files, and code that would be pretty great. Maybe. It seems like a much harder problem to solve than simple formatting, without introducing a bunch of bugs. But you seem to be suggesting a different way of viewing the code, which I guess might be the way. But still underneath you'd have the spaghetti code, just sitting there not being dealt with, like a complicated Microsoft Word document.
deleted 3y ago
[deleted]
norman784 3y ago
I was about to comment also that unison is an interesting approach, we need more tooling support before it can be possible, ideally there need to be a protocol that the new kind of editors can be built on top, similar to how LSP enabled editors to support different programming languages without a proprietary implementation. I'm wondering if the unison devs already have plans on working on a protocol like that.
deleted 3y ago
[deleted]
jerf 3y ago
"The point is I wish we could just ask the computer to sort things out for us, show different views, etc. this would require a software system able to work with more granular code units, maybe something like [1]." The problem I've seen with this idea is that in order for it to work, you have to be willing to add more metadata to your program, because programs themselves don't have the necessary data in them. You can make it mandatory to your language, but than just raises the friction for people trying to use that language. But, if you're willing to add the metadata to a hypothetical new system for slicing and dicing code, why aren't you using the existing mechanisms in your system for organizing and documenting things? If you are, then you make the delta between "it may not be perfect but here's some documented and organized code" and the perfect vision small enough to not be worth a lot of effort to chase. If you aren't, then you aren't going to give the perfect all-singing slicer and dicer what it needs either. Replace "you" with "coworker(s)" for the same dilemma, only sadder. And your generic strawman coworkers are going to be very upset when you try to get them to use this language... "it's so hard, it's always complaining about perfectly good code, I'm missing my deadlines because it's making me document things I'll never use, I'm just putting 'a' as the documentation for these things anyhow, oh look here's our boss ordering us to go back to our previous language because we're all spending too much time documenting rather than building new features". Some things can't really be fixed by process.
- emmanueloga_ 3y ago
  I agree on the human friction part... I like this idea and yet I struggle to implement it in my own projects. I think the reason is that there's no good language yet to write that metadata. Even advanced type systems mainly focus on guiding how components connect, but they don't explain _why_ a program behaves a certain way or how the data is supposed to flow. Documentation is often unreliable as it becomes stale easily. As developers we spend and order of magnitude more time building a graph of data flow in our heads than actually writing code. Could we reify that graph? IMO that crucial metadata would answer the questions we would ask the Senior developer in our team when we need to work in the codebase: * When dealing with feature X, what are the involved software components Y and Z, how do we create instances of those? * What are the actual features the system implements? Is this the correct behavior? Is this what the system was intended to do? * If I point to a specific file and line in the project, what features does that code support? Why is that file there? Expressing the answers in a machine-readable format, in a language agnostic way, could look something like: System { Dependencies - Database depends on Logging - UI depends on Database - ... Feature A - Subtopic 1 - Subtopic 2 Feature B - ... } Feature A { requires database queries X if (database has data) produces Y, Z else doTheThing(X, "blah") } This is a broad idea, and the devil is in the details... This would not replace type systems or traditional languages. I'm thinking this spec-lang would have the minimum amount of flowchart-like constructs, and everything even slightly more complicated would rely on an interface. The system would implement all boilerplate and layout the project entirely. It would be amazing if we could also express state machines or even statecharts. Teams could use off-the shelf generators, or create a custom one, to produce the interfaces and boilerplate implementation. Since the system has a lot of information we could query it to answer questions like "give me all code of feature A", or even ask it to create symlinks so we can focus our attention in a single folder of files. Also different generators could create the structure that is more appropriate for different programming languages. It could be a good tool to assist rewrites, if needed for whatever reason. Tests/specs could be generated automatically, to a certain extent. I'm sure some people here will have PTSD about 4th gen languages and BPEL/UML nightmares :-). But even though BPEL is horrible I think the core of the idea is good and worth revisiting.
crabbone 3y ago
> Things humans are bad at: organizing files Based on what do you make this statement? I don't feel like this is a problem. At least not for me :| Of course, size can make everything a problem, but for whatever projects I had to work with, organizing files was usually an entry-level problem that would normally sort itself out after a month or so. In other words, I don't feel like this is a problem worth solving.
solarkraft 3y ago
I do believe that text in files isn't the ideal way of expressing and structuring program instructions. But the friction to get something else established is so huge - all tooling would need to change ... unless perhaps one develops a way to translate between the internal format and text.
- kmoser 3y ago
  You've hit the nail on the head: any code written as a series of text files will lack the overall structure needed to easily navigate, refactor, and perhaps most important of all, render as different "views." For example, if the development environment automatically ingested the code and understood which modules were related to each other, it should be trivial to ask it to sort the functions by name, and/or group them by module, or anything else you can think of. When moving a function to a different module (e.g. moving bar() from foo::bar too baz::bar), the development environment should automatically change all references to foo::bar() to baz::bar(). And this is just the tip of the iceberg of what the development environment should do for you. The problem is, developers have come up with all sorts of clever hacks to do this searching and refactoring, including IDEs which do much of the heavy lifting, but at the end of the day they still fall short of what should be a simple task for the developer. As for additional metadata that would be required to make this work: objections that programmers would be reticent to provide this info are similar to objections to writing good documentation, and my response is: you reap what you sow. If you really want good, structured code, you have to write it, and you have to do so in an environment that accepts such structured data. You can't get blood from a stone. In short, for large, complex projects, our current method of writing code as textfiles is unwieldy. Unfortunately developers' visions are clouded by their hard-earned ability to parse those textfiles, and don't see this as a problem.

MetaWhirledPeas 3y ago

> The point is I wish we could just ask the computer to sort things out for us, show different views, etc. I agree. I'm usually opinionated about a related subject: code formatting. But I'm not too opinionated in one case, where we're using Prettier. The project is autoformatted and expects everyone else to do the same. It's so refreshing to let the computer take over sometimes, even if the end result isn't exactly what I would have typed out myself. If a piece of software could so something similar with the folders, files, and code that would be pretty great. Maybe. It seems like a much harder problem to solve than simple formatting, without introducing a bunch of bugs. But you seem to be suggesting a different way of viewing the code, which I guess might be the way. But still underneath you'd have the spaghetti code, just sitting there not being dealt with, like a complicated Microsoft Word document.

deleted 3y ago

[deleted]

norman784 3y ago

I was about to comment also that unison is an interesting approach, we need more tooling support before it can be possible, ideally there need to be a protocol that the new kind of editors can be built on top, similar to how LSP enabled editors to support different programming languages without a proprietary implementation. I'm wondering if the unison devs already have plans on working on a protocol like that.

deleted 3y ago

[deleted]

jerf 3y ago

"The point is I wish we could just ask the computer to sort things out for us, show different views, etc. this would require a software system able to work with more granular code units, maybe something like [1]." The problem I've seen with this idea is that in order for it to work, you have to be willing to add more metadata to your program, because programs themselves don't have the necessary data in them. You can make it mandatory to your language, but than just raises the friction for people trying to use that language. But, if you're willing to add the metadata to a hypothetical new system for slicing and dicing code, why aren't you using the existing mechanisms in your system for organizing and documenting things? If you are, then you make the delta between "it may not be perfect but here's some documented and organized code" and the perfect vision small enough to not be worth a lot of effort to chase. If you aren't, then you aren't going to give the perfect all-singing slicer and dicer what it needs either. Replace "you" with "coworker(s)" for the same dilemma, only sadder. And your generic strawman coworkers are going to be very upset when you try to get them to use this language... "it's so hard, it's always complaining about perfectly good code, I'm missing my deadlines because it's making me document things I'll never use, I'm just putting 'a' as the documentation for these things anyhow, oh look here's our boss ordering us to go back to our previous language because we're all spending too much time documenting rather than building new features". Some things can't really be fixed by process.

emmanueloga_ 3y ago
I agree on the human friction part... I like this idea and yet I struggle to implement it in my own projects. I think the reason is that there's no good language yet to write that metadata. Even advanced type systems mainly focus on guiding how components connect, but they don't explain _why_ a program behaves a certain way or how the data is supposed to flow. Documentation is often unreliable as it becomes stale easily. As developers we spend and order of magnitude more time building a graph of data flow in our heads than actually writing code. Could we reify that graph? IMO that crucial metadata would answer the questions we would ask the Senior developer in our team when we need to work in the codebase: * When dealing with feature X, what are the involved software components Y and Z, how do we create instances of those? * What are the actual features the system implements? Is this the correct behavior? Is this what the system was intended to do? * If I point to a specific file and line in the project, what features does that code support? Why is that file there? Expressing the answers in a machine-readable format, in a language agnostic way, could look something like: System { Dependencies - Database depends on Logging - UI depends on Database - ... Feature A - Subtopic 1 - Subtopic 2 Feature B - ... } Feature A { requires database queries X if (database has data) produces Y, Z else doTheThing(X, "blah") } This is a broad idea, and the devil is in the details... This would not replace type systems or traditional languages. I'm thinking this spec-lang would have the minimum amount of flowchart-like constructs, and everything even slightly more complicated would rely on an interface. The system would implement all boilerplate and layout the project entirely. It would be amazing if we could also express state machines or even statecharts. Teams could use off-the shelf generators, or create a custom one, to produce the interfaces and boilerplate implementation. Since the system has a lot of information we could query it to answer questions like "give me all code of feature A", or even ask it to create symlinks so we can focus our attention in a single folder of files. Also different generators could create the structure that is more appropriate for different programming languages. It could be a good tool to assist rewrites, if needed for whatever reason. Tests/specs could be generated automatically, to a certain extent. I'm sure some people here will have PTSD about 4th gen languages and BPEL/UML nightmares :-). But even though BPEL is horrible I think the core of the idea is good and worth revisiting.

crabbone 3y ago

> Things humans are bad at: organizing files Based on what do you make this statement? I don't feel like this is a problem. At least not for me :| Of course, size can make everything a problem, but for whatever projects I had to work with, organizing files was usually an entry-level problem that would normally sort itself out after a month or so. In other words, I don't feel like this is a problem worth solving.

solarkraft 3y ago

I do believe that text in files isn't the ideal way of expressing and structuring program instructions. But the friction to get something else established is so huge - all tooling would need to change ... unless perhaps one develops a way to translate between the internal format and text.

kmoser 3y ago
You've hit the nail on the head: any code written as a series of text files will lack the overall structure needed to easily navigate, refactor, and perhaps most important of all, render as different "views." For example, if the development environment automatically ingested the code and understood which modules were related to each other, it should be trivial to ask it to sort the functions by name, and/or group them by module, or anything else you can think of. When moving a function to a different module (e.g. moving bar() from foo::bar too baz::bar), the development environment should automatically change all references to foo::bar() to baz::bar(). And this is just the tip of the iceberg of what the development environment should do for you. The problem is, developers have come up with all sorts of clever hacks to do this searching and refactoring, including IDEs which do much of the heavy lifting, but at the end of the day they still fall short of what should be a simple task for the developer. As for additional metadata that would be required to make this work: objections that programmers would be reticent to provide this info are similar to objections to writing good documentation, and my response is: you reap what you sow. If you really want good, structured code, you have to write it, and you have to do so in an environment that accepts such structured data. You can't get blood from a stone. In short, for large, complex projects, our current method of writing code as textfiles is unwieldy. Unfortunately developers' visions are clouded by their hard-earned ability to parse those textfiles, and don't see this as a problem.