11 ms·

Happiness is a freshly organized codebase

184 points by felixrieseberg 6y ago

dangwu 6y ago
Interesting read! > No more dumping ground folders like “Helper” or “Utility” If you’re organizing by feature and you have one of these helper/utility classes that support multiple features, where does it live? Would you consider each utility to be its own “feature”?
- ericaengle 6y ago
  We discovered a lot of cases would be unique, so we do try to evaluate a "best" case when it comes to each move. Keeping that in mind, if a feature has a shared helper across components within the /Feature folder, we'll have a /Shared folder within to capture these files. At a top level, we would encourage a new /Feature folder since most likely it will have tests or be important enough to merit a folder rather than squishing it into another /Feature folder. We're optimizing for visibility too, so having it's own /Feature folder helps with that.
  - dangwu 6y ago
    Thanks! And am I correct in assuming Slack just has a single target for the iOS app (App), then one for each App Extension? So the root source folders map 1:1 to build targets?
    - ericaengle 6y ago
      Yes! that's the idea, or at least what we've worked to achieve
- drawkbox 6y ago
  It depends on how many projects you have. A place like Slack has only one client/product really when it comes down to it, so they can get pedantic about it and look down on smart reuse across projects with robust tested libs on top of standard libs that are necessary to increase production and stability/security etc. If you work at an agency, or game company shipping many games, there will always be a "core" library or "base" library that has Helpers and Utility. Good consistent projects wanted tested and solid parts. Every game has a common lib of core tools like maths, vector tools, prediction, data structures and many more. There is such a broad difference in coding on one platform for one company for one product compared to shipping many companies (or many internal projects) on many platforms for many products. Even standard libraries for platforms are essentially a Helper/Utility really when it comes down to it: .NET Core, Python standard lib, standard node, C++ distributable etc. There will be common core tasks across a project, product, company or platform for most teams with many projects to manage. If you don't have common libs with common helpers/utilities in large sets of products, technical debt and maintenance become a nightmare. In projects we work on these are core/base libs that are submodules in git and are essentially the 'tech' across projects where the project/product itself is the unique implementation for that product. Anything common or generic gets put in the core 'tech'. Every single game studio will have these as well as agencies if they are organized and produce quality relatively fast and consistent. If "Happiness is a freshly organized codebase" they must have common tested parts that are ship tested. This is really just Slack talking specifically about mature products at a company that only has one product. It isn't reality at places that have many projects, products, companies, clients etc. Pretending they are the same is a bit elitist.
- honkycat 6y ago
  I always argue adding "helper" or "util" adds nothing to the description of a file/module/class and is better left off
  - seph-reed 6y ago
    Utility - few or no dependencies. Absolutely no dependencies specific to the project. (array flattener, 2-way map, etc) Helper - function which makes some really common project specific code more DRY (createApiError, genCommonConfig, etc)
    - ravenstine 6y ago
      I'd also consider a Helper to be something that may have application-awareness. Whereas, as you stated, a Utility would have no serious dependencies and is otherwise "dumb".
  - nilkn 6y ago
    I feel like these names convey a lot, though. When I see something like this, I immediately expect it to not do any of the following: (1) contain core logic or define primary classes/types/records; (2) have dramatic side effects; (3) be hard to test; (4) reference or depend on other parts of the codebase; (5) do anything controversial in general. I’d say I’d expect a “utility” to adhere to these conditions more strongly than a “helper,” which might be a bit more entangled with application logic.
- _bxg1 6y ago
  I actually think it's vitally important to have "dumping ground" locations for things that aren't fully figured-out yet. If I'm working on a new thing and I have something that's relevant in multiple files but I'm not sure where it should go, or even whether it will stick around, I don't want to have to come to a screeching halt just to make a taxonomic decision about something that's still a work in progress. The key, of course, is going back and categorizing those things later once you do have an idea of where they should go.
  - ryanianian 6y ago
    > The key, of course, is going back and categorizing those things later once you do have an idea of where they should go. Yes, but in my experience this is rarely done. It's put off until it gets so convoluted that it introduces bugs. The same is true even if you do avoid dumping-grounds: you need to examine the project's state somewhat regularly to ensure it still makes sense. Where code lives can be as important as the code itself and thus deserves at least occasional review and change.
  - allenu 6y ago
    Totally agree. So much of programming and designing is figuring out what pattern your code is falling into and what stuff "is" or "means". I believe it's totally okay to name things utilities when you haven't yet established a common pattern or understanding of it. Naming things is hard and spending so much time on it and organizing things can often block you from progressing to a place where you DO have more information and can intelligently name things. You'll always be juggling unknowns, so it's okay to have dumping grounds here and there, so long as over time you clean them up as you gain more info.
    - _bxg1 6y ago
      I recently started working within a Python codebase, and one of the things I really like about it (not sure whether this is standard Python practice) is that most directories have a "common.py" file in them. So if you just want to put something somewhere real quick, you can elevate it to exactly the appropriate directory-level instead of going to a single, global "utils" file. It's a neat pattern.
- lmm 6y ago
  If they don't have anything in common, yes. If you have one collection helper method, one time helper method and one auth helper method, far better to put those in a collection class, a time class and an auth class than bundle them up as a "utils" class.
kccqzy 6y ago
I sometimes wish Git properly records copy and move information. It turns out Git's heuristics for detecting copies and moves work about 95% of the time, and for the remaining 5% it's mildly annoying to read a git blame with every line from a move. You can blame further across that move, but that's manual. If copy and move information is perfectly recorded, I'd have more incentive to do these kinds of code reorganization.
- ryanianian 6y ago
  This article mentioned a while ago had relevant info: https://news.ycombinator.com/item?id=22689301 https://news.ycombinator.com/item?id=22689301 In my experience, ensure you don't change the code as you move it, don't rename files and move content at the same time, and don't move "too much" code in a single commit. Around 2k lines at a time seems to be a good number. Maybe some languages/structures are easier for git to analyze when doing blames.
  - kccqzy 6y ago
    How do you not change the code as it's being moved, when you need to update #include directives, imports, and other file paths?
    - ryanianian 6y ago
      Sometimes it is unavoidable, but in general minimize it. Don't refactor a class and rename it at the same time. Hopefully moves just result in imports and paths changing, something that is not likely to confuse a blame (and rarely do I care about the blame lines on #includes/imports)
    - derefr 6y ago
      It's easy if not every commit needs to compile. :)
    - vimax 6y ago
      New branch, make two commits. Squash merge the branch as a single commit.
rudi-c 6y ago
I've encountered similar difficulties around codebases with a lack of file hierarchy structure. But one major difficulty in fixing the issue is that moving a lot of files around tends to trash `git blame`, which is often more valuable than knowing what folder to put a new file in. Is that something you've encountered?
- ryanianian 6y ago
  There are workarounds to get git to search harder or to commit things in a way that's helpful for large-scale file-shuffles, but to be honest I've rarely found this to be enough of a reason to not move things around. The end result is a much more productive and purposeful place for code to live and grow. This said, moving files and moving code within those files at the same time is a recipe for confusing yourself and git. Move files first, then content.
  - deleted 6y ago
    [deleted]
  - nitrogen 6y ago
    A few options to look at specifically: --follow, -M, and -C. Less related to refactoring I also find --first-parent and --merges useful
- smallnamespace 6y ago
  Try putting the refactoring commit in .git-blame-ignore-revs
  - karlding 6y ago
    Note that .git-blame-ignore-revs is just a convention, git doesn't automatically use the file like how git treats the .gitignore file. You can add it to the repo configuration though, so you don't need to pass --ignore-revs-file every time, provided that you're running Git 2.3+[0]. $ git config blame.ignoreRevsFile .git-blame-ignore-revs This is extremely useful if, for example, your team decided to introduce new automated style changes, and didn't want to completely clobber the history for blame. By specifying the commit that introduced the style change, actual "interesting" changes should be correctly attributed and the style change commit is ignored in the blame output. [0] https://github.com/git/git/blob/master/Documentation/RelNotes/2.23.0.txt#L83-L84 https://github.com/git/git/blob/master/Documentation/RelNote...
ryanianian 6y ago
The concept of linting repo structure is an excellent one. I cannot even get started in projects without a sane repo layout. Source files scattered everywhere unrelated to each other, utils junk-drawers with dozens of files, multiple top-level source directories without explicit rationale, tests and source totally disjoint, hacks to modify build and run paths, implicit dependencies between directories, awful convoluted build-system configuration to match all of these idiosyncrasies, and impossible or very difficult editor/ide integration as a result. Even rails-eque apps (which come with a reasonable structure) get really messy really quickly if somebody doesn't have the diligence to stay on top of it. I want there to be a {sane, polyglot, large-ish-scale} project layout convention that plays well with {intuition/discoverability, build-systems, editors, project size}, but it seems there's an inherent tension between these. Maven tried. Kinda. I wonder if there's a fundamental problem we aren't solving somewhere.
- DelightOne 6y ago
  In maven I don't like the test-separation from the code itself because its much simpler to find and update them this way.
  - ryanianian 6y ago
    There's a lot to not like about maven. I think that separation was motivated by java's classpath rather than making a 'good'/dev-friendly convention. After 10 years in maven-land, I changed a recent project to put `foo.ts` and `foo.test.ts` in the same directory and its wonderful, but it confuses some build-systems and makes certain kinds of test-helpers harder to write.
    - shepherdjerred 6y ago
      Maven is my favorite thing about Java...
  - advaita 6y ago
    Anecdotally, I like that separation because in absence of it working with any large-ish project (which most real life projects are), it'd get super hard for my brain to pick out tests even if they all had certain prefix/nomenclature. Caveat: I use IntelliJ and I just jump to tests using CMD+SHIFT+T shortcut so not necessarily a generic example.
rukittenme 6y ago
I've been wondering a lot recently if folders make projects better or worse. For example, when I write a library its usually very simple. There's a single directory which contains all of the source files. When people use the library they: """ import lib lib.run() """" Dead simple, no complex module paths to remember, no hierarchical folder structure forcing you to code based on a pattern rather than functionality. Pure bliss. But on the other hand, I have projects that contain 100k lines of source code. I can't just leave it out in the cold. So poor baby gets a couple of folders. But I do hate it. I hate writing the code. I hate reading it. I hate finding it 6 months after the fact. That's probably just the nature of the job. It is work at the end of the day. Maybe its just doomed to be hard.
- osener 6y ago
  I'm not plugging the language, but I've come to appreciate OCaml's module system with no imports and (mostly) globally unique module names. No circular dependencies allowed either. You can have multiple modules within a file (which is also a module named after the file name). I structure larger projects as libraries with minimal dependencies that depend on one another, and dump all my modules with descriptive file names under the same directory within the library. I vaguely remember reading something that hinted at Facebook doing something similar with their React components.
bob1029 6y ago
I feel like a lot of the pain in codebase organization boils down to having a technical project structure (i.e. layout on disk) that does not align well with the business. Obviously, it's impossible to force something to directly align with such disparate and abstract requirements, so you have to create abstractions (layers) to enable a hypothetically-pure realm. In our architecture, we've created roughly 2 different kinds of abstraction: Platform and Business. The Platform abstractions are only intended to support themselves and the Business abstractions. These are not supposed to be easily-accessible by non-technical folks. Their entire purpose is to make the Business abstractions as elegant as possible. The idea is these should be slowly-changing and stable across all use cases. Developers are only expected to visit this realm maybe once every other week. We effectively treat ourselves as our own customer with this layer. The Business abstractions are built using only first-party primitives and our Platform layer. Most of this is very clean because we explicitly force all non-business concerns out into the Platform layer. Things like validation logic are written using purely functional code in this realm, which makes it very easy to reason with. Developers are expected to live in this area most of the time. Organization of business abstractions typically falls along lines of usage or logical business activity. We explicitly duplicate some code in order to maintain very clean separation of differing business activities. I've watched first hand as the subtle variances of a single combined Customer model across hundreds of contexts eventually turned it into a boat anchor. As a consequence of all this, our project managers and other non-code-experts are actually able to review large parts of the codebase and derive meaningful insights without a developer babysitting them. We are at a point where project managers can open PRs for adjusting basic validation and parameter items in our customers' biz code. Developers also enjoy not having to do massive altitude shifts on abstractions while reading through 1 source file. You either spend a day working platform concerns, or you are in the business layer. Context switching sucks and we built to avoid it as much as possible. IMO, having a bad project structure is one direct side-effect of being forced to context switch all the time.
- ajsharma 6y ago
  Wow, I love this idea. I'm curious how you have that structured within the code base. Is it literally a platform folder and a business folder?
deleted 6y ago
[deleted]
ldd 6y ago
There is this wonderful utility: dependency-cruiser[0] for javascript / typescript projects. It visualizes dependencies in a project. I found it so so easy to refactor and move files around after I started using it. I am not usually a visually-oriented person, but for this usecase, and to be happy, `dep-cruiser` surely helps. [0]: https://github.com/sverweij/dependency-cruiser https://github.com/sverweij/dependency-cruiser
soedirgo 6y ago
In Go, there's a standard project layout [1]. It'd be nice to have a project layout linter in Go Report Card [2]. [1]: https://github.com/golang-standards/project-layout https://github.com/golang-standards/project-layout [2]: https://goreportcard.com https://goreportcard.com
- dgellow 6y ago
  Just to clarify, that’s not an official standard at all (i.e: not supported by the go team). It’s a community project to establish some common layout. Surely valuable, but to say it is actually a Go standard is misleading IMHO.
  - soedirgo 6y ago
    Ah! I stand corrected. Mea culpa.
nickjj 6y ago
For new projects (especially as a solo developer), there's also a related topic of not taking advantage of tools like kanban boards, or having a place outside of your source code to organize your thoughts and research. It's very possible to wind up with massive comment dumps of things to research, alternate implementations, notes to yourself and other things littered in your code base where you haven't made any git commits yet. This really leaves things in a messy state where you feel like the project is never going to be finished. An example of this and how I solved this problem with a kanban board can be found here: https://youtu.be/HHOkcCqsipE?t=76 https://youtu.be/HHOkcCqsipE?t=76
stefan_ 6y ago
Danger is a tool we integrated into our Continuous Integration system that performs post-commit automated checks You mean like the post-commit hook that Git offers out of the box? It's even named the exact same! I feel like we don't focus all this time on fast build test and deploy cycles only to then commit, navigate to some website to create a "merge request", wait for some fairy to allocate computing resources for trivial checks my computer could have done, only to get some pure noise "don't put this file here" comment and repeat the cycle all again.
- core-questions 6y ago
  When did you think I was going to get coffee and/or check Slack for more annoying work to do? It's like you want me to get things _done_, which is not fun at all.
momokoko 6y ago
Needing organized codebases is a personality type as there is zero academic research I’m aware of that has ever shown strict organization has resulted in less bugs or faster development. These are all done by people that need to feel in control and that everything has a place. So much money and time has been wasted on stuff like this with zero proven or measured benefit.
- BurningFrog 6y ago
  There is famously also no academic study showing that parachutes improve outcomes when jumping out of a plane.
  - deleted 6y ago
    [deleted]
  - darekkay 6y ago
    There is a study for that! "Parachute use to prevent death and major trauma when jumping from aircraft: randomized controlled trial". [1] [1] https://www.bmj.com/content/363/bmj.k5094 https://www.bmj.com/content/363/bmj.k5094
- jfim 6y ago
  Maybe, but there are other metrics than just developer output. If having a codebase that's well organized makes developers happier, and happier developers stay longer at the company, then that's also a win.
  - momokoko 6y ago
    This makes some, definitely not all, developers happier. Organization for happiness is based on the individual developers personality and means you are putting some engineer’s happiness as a priority over spending that money providing other things to make the developers with a personality type that derives no benefit from strict organization. In fact, strict organization rules make some developers less happy. Why is their happiness less important when this is well known to have no business benefit?
battery_cowboy 6y ago
What if we just got rid of files and put our code into a database? You'd just have a "new code block" button, which creates the editor tab for your code, usually a function or a class, and usually one item per block. When you save it, it puts it into a database and you can version things easily. You can call other functions from the block and your editor will show their code when you mouseover or maybe some other method, just like today. Basically the same as today, but you don't need to worry about where some code lives. You back it with a great search feature to find stuff. Hell, let's just eliminate pathed files, why do we care about file paths with the level of search today? Just store everything in a key:value store directly on the hard drive, no paths needed. For legacy, just add keys for '/etc/fstab' or whatever.
peter_d_sherman 6y ago
>"The Slack iOS team lived in these conditions for a few too many years. We got here as a result of some attempts to organize source files (several times), a lack of architecture pattern in the codebase, and a high growth of developers over a couple years. To put things into context, we have roughly 13,000 files (and counting), about 27 top level directories, a mix of files in Objective-C and Swift, and around 40 iOS developers that work in one monorepo." An extensive, unrefactored codebase is no different than a jungle. You might have a 10'xer programmer on your staff, and he might hold the programming equivalent of a machete, but if the rate of his refactoring (assuming your corporate rules let him) is slower than the rate of new code being added by other employees, he is going to fail, no matter how good he is! I need to write a future essay about the relationship between 10'xers and how a 10'xer is a combination of only as good as how well the codebase is refactored, how well they know the codebase, how much corporate rules/polcies permit refactoring (or not), how much time he doesn't have to waste time solving stupid one-time issues from single customers, and how much help or pushback he is or isn't getting from the rest of the team. In other words, given the right set of conditions between codebase size and obfuscation, limiting corporate policies (i.e., "you can't refactor", "you can't make just one mistake in your code, because it's all mission critical, and if you do, you will be fired, and by the way, there's no test environment!"), distraction ("you have to help this customer with his cosmetic problem before you are permitted to tackle the guts of the system"), and pushback from the rest of the team, you can actually change 10'xers (and higher!) to 1x'ers and below... The reverse is true too... I'll make anyone a "My Fair Lady" / Eliza Doolittle style bet (or the reverse!) that what I say is true! That is, that 1'xers can be taught to become 10'xers, and conversely, 10'xers can be hampered by a variety of factors ("The Perfect Storm") resulting in them being slowed to 1'xers, or below...

Interesting read! > No more dumping ground folders like “Helper” or “Utility” If you’re organizing by feature and you have one of these helper/utility classes that support multiple features, where does it live? Would you consider each utility to be its own “feature”?

ericaengle 6y ago
We discovered a lot of cases would be unique, so we do try to evaluate a "best" case when it comes to each move. Keeping that in mind, if a feature has a shared helper across components within the /Feature folder, we'll have a /Shared folder within to capture these files. At a top level, we would encourage a new /Feature folder since most likely it will have tests or be important enough to merit a folder rather than squishing it into another /Feature folder. We're optimizing for visibility too, so having it's own /Feature folder helps with that.
- dangwu 6y ago
  Thanks! And am I correct in assuming Slack just has a single target for the iOS app (App), then one for each App Extension? So the root source folders map 1:1 to build targets?
  - ericaengle 6y ago
    Yes! that's the idea, or at least what we've worked to achieve
drawkbox 6y ago
It depends on how many projects you have. A place like Slack has only one client/product really when it comes down to it, so they can get pedantic about it and look down on smart reuse across projects with robust tested libs on top of standard libs that are necessary to increase production and stability/security etc. If you work at an agency, or game company shipping many games, there will always be a "core" library or "base" library that has Helpers and Utility. Good consistent projects wanted tested and solid parts. Every game has a common lib of core tools like maths, vector tools, prediction, data structures and many more. There is such a broad difference in coding on one platform for one company for one product compared to shipping many companies (or many internal projects) on many platforms for many products. Even standard libraries for platforms are essentially a Helper/Utility really when it comes down to it: .NET Core, Python standard lib, standard node, C++ distributable etc. There will be common core tasks across a project, product, company or platform for most teams with many projects to manage. If you don't have common libs with common helpers/utilities in large sets of products, technical debt and maintenance become a nightmare. In projects we work on these are core/base libs that are submodules in git and are essentially the 'tech' across projects where the project/product itself is the unique implementation for that product. Anything common or generic gets put in the core 'tech'. Every single game studio will have these as well as agencies if they are organized and produce quality relatively fast and consistent. If "Happiness is a freshly organized codebase" they must have common tested parts that are ship tested. This is really just Slack talking specifically about mature products at a company that only has one product. It isn't reality at places that have many projects, products, companies, clients etc. Pretending they are the same is a bit elitist.
honkycat 6y ago
I always argue adding "helper" or "util" adds nothing to the description of a file/module/class and is better left off
- seph-reed 6y ago
  Utility - few or no dependencies. Absolutely no dependencies specific to the project. (array flattener, 2-way map, etc) Helper - function which makes some really common project specific code more DRY (createApiError, genCommonConfig, etc)
  - ravenstine 6y ago
    I'd also consider a Helper to be something that may have application-awareness. Whereas, as you stated, a Utility would have no serious dependencies and is otherwise "dumb".
- nilkn 6y ago
  I feel like these names convey a lot, though. When I see something like this, I immediately expect it to not do any of the following: (1) contain core logic or define primary classes/types/records; (2) have dramatic side effects; (3) be hard to test; (4) reference or depend on other parts of the codebase; (5) do anything controversial in general. I’d say I’d expect a “utility” to adhere to these conditions more strongly than a “helper,” which might be a bit more entangled with application logic.
_bxg1 6y ago
I actually think it's vitally important to have "dumping ground" locations for things that aren't fully figured-out yet. If I'm working on a new thing and I have something that's relevant in multiple files but I'm not sure where it should go, or even whether it will stick around, I don't want to have to come to a screeching halt just to make a taxonomic decision about something that's still a work in progress. The key, of course, is going back and categorizing those things later once you do have an idea of where they should go.
- ryanianian 6y ago
  > The key, of course, is going back and categorizing those things later once you do have an idea of where they should go. Yes, but in my experience this is rarely done. It's put off until it gets so convoluted that it introduces bugs. The same is true even if you do avoid dumping-grounds: you need to examine the project's state somewhat regularly to ensure it still makes sense. Where code lives can be as important as the code itself and thus deserves at least occasional review and change.
- allenu 6y ago
  Totally agree. So much of programming and designing is figuring out what pattern your code is falling into and what stuff "is" or "means". I believe it's totally okay to name things utilities when you haven't yet established a common pattern or understanding of it. Naming things is hard and spending so much time on it and organizing things can often block you from progressing to a place where you DO have more information and can intelligently name things. You'll always be juggling unknowns, so it's okay to have dumping grounds here and there, so long as over time you clean them up as you gain more info.
  - _bxg1 6y ago
    I recently started working within a Python codebase, and one of the things I really like about it (not sure whether this is standard Python practice) is that most directories have a "common.py" file in them. So if you just want to put something somewhere real quick, you can elevate it to exactly the appropriate directory-level instead of going to a single, global "utils" file. It's a neat pattern.
lmm 6y ago
If they don't have anything in common, yes. If you have one collection helper method, one time helper method and one auth helper method, far better to put those in a collection class, a time class and an auth class than bundle them up as a "utils" class.

kccqzy 6y ago

I sometimes wish Git properly records copy and move information. It turns out Git's heuristics for detecting copies and moves work about 95% of the time, and for the remaining 5% it's mildly annoying to read a git blame with every line from a move. You can blame further across that move, but that's manual. If copy and move information is perfectly recorded, I'd have more incentive to do these kinds of code reorganization.

ryanianian 6y ago
This article mentioned a while ago had relevant info: https://news.ycombinator.com/item?id=22689301 https://news.ycombinator.com/item?id=22689301 In my experience, ensure you don't change the code as you move it, don't rename files and move content at the same time, and don't move "too much" code in a single commit. Around 2k lines at a time seems to be a good number. Maybe some languages/structures are easier for git to analyze when doing blames.
- kccqzy 6y ago
  How do you not change the code as it's being moved, when you need to update #include directives, imports, and other file paths?
  - ryanianian 6y ago
    Sometimes it is unavoidable, but in general minimize it. Don't refactor a class and rename it at the same time. Hopefully moves just result in imports and paths changing, something that is not likely to confuse a blame (and rarely do I care about the blame lines on #includes/imports)
  - derefr 6y ago
    It's easy if not every commit needs to compile. :)
  - vimax 6y ago
    New branch, make two commits. Squash merge the branch as a single commit.

rudi-c 6y ago

I've encountered similar difficulties around codebases with a lack of file hierarchy structure. But one major difficulty in fixing the issue is that moving a lot of files around tends to trash `git blame`, which is often more valuable than knowing what folder to put a new file in. Is that something you've encountered?

ryanianian 6y ago
There are workarounds to get git to search harder or to commit things in a way that's helpful for large-scale file-shuffles, but to be honest I've rarely found this to be enough of a reason to not move things around. The end result is a much more productive and purposeful place for code to live and grow. This said, moving files and moving code within those files at the same time is a recipe for confusing yourself and git. Move files first, then content.
- deleted 6y ago
  [deleted]
- nitrogen 6y ago
  A few options to look at specifically: --follow, -M, and -C. Less related to refactoring I also find --first-parent and --merges useful
smallnamespace 6y ago
Try putting the refactoring commit in .git-blame-ignore-revs
- karlding 6y ago
  Note that .git-blame-ignore-revs is just a convention, git doesn't automatically use the file like how git treats the .gitignore file. You can add it to the repo configuration though, so you don't need to pass --ignore-revs-file every time, provided that you're running Git 2.3+[0]. $ git config blame.ignoreRevsFile .git-blame-ignore-revs This is extremely useful if, for example, your team decided to introduce new automated style changes, and didn't want to completely clobber the history for blame. By specifying the commit that introduced the style change, actual "interesting" changes should be correctly attributed and the style change commit is ignored in the blame output. [0] https://github.com/git/git/blob/master/Documentation/RelNotes/2.23.0.txt#L83-L84 https://github.com/git/git/blob/master/Documentation/RelNote...

ryanianian 6y ago

The concept of linting repo structure is an excellent one. I cannot even get started in projects without a sane repo layout. Source files scattered everywhere unrelated to each other, utils junk-drawers with dozens of files, multiple top-level source directories without explicit rationale, tests and source totally disjoint, hacks to modify build and run paths, implicit dependencies between directories, awful convoluted build-system configuration to match all of these idiosyncrasies, and impossible or very difficult editor/ide integration as a result. Even rails-eque apps (which come with a reasonable structure) get really messy really quickly if somebody doesn't have the diligence to stay on top of it. I want there to be a {sane, polyglot, large-ish-scale} project layout convention that plays well with {intuition/discoverability, build-systems, editors, project size}, but it seems there's an inherent tension between these. Maven tried. Kinda. I wonder if there's a fundamental problem we aren't solving somewhere.

DelightOne 6y ago
In maven I don't like the test-separation from the code itself because its much simpler to find and update them this way.
- ryanianian 6y ago
  There's a lot to not like about maven. I think that separation was motivated by java's classpath rather than making a 'good'/dev-friendly convention. After 10 years in maven-land, I changed a recent project to put `foo.ts` and `foo.test.ts` in the same directory and its wonderful, but it confuses some build-systems and makes certain kinds of test-helpers harder to write.
  - shepherdjerred 6y ago
    Maven is my favorite thing about Java...
- advaita 6y ago
  Anecdotally, I like that separation because in absence of it working with any large-ish project (which most real life projects are), it'd get super hard for my brain to pick out tests even if they all had certain prefix/nomenclature. Caveat: I use IntelliJ and I just jump to tests using CMD+SHIFT+T shortcut so not necessarily a generic example.

rukittenme 6y ago

I've been wondering a lot recently if folders make projects better or worse. For example, when I write a library its usually very simple. There's a single directory which contains all of the source files. When people use the library they: """ import lib lib.run() """" Dead simple, no complex module paths to remember, no hierarchical folder structure forcing you to code based on a pattern rather than functionality. Pure bliss. But on the other hand, I have projects that contain 100k lines of source code. I can't just leave it out in the cold. So poor baby gets a couple of folders. But I do hate it. I hate writing the code. I hate reading it. I hate finding it 6 months after the fact. That's probably just the nature of the job. It is work at the end of the day. Maybe its just doomed to be hard.

osener 6y ago
I'm not plugging the language, but I've come to appreciate OCaml's module system with no imports and (mostly) globally unique module names. No circular dependencies allowed either. You can have multiple modules within a file (which is also a module named after the file name). I structure larger projects as libraries with minimal dependencies that depend on one another, and dump all my modules with descriptive file names under the same directory within the library. I vaguely remember reading something that hinted at Facebook doing something similar with their React components.

bob1029 6y ago

I feel like a lot of the pain in codebase organization boils down to having a technical project structure (i.e. layout on disk) that does not align well with the business. Obviously, it's impossible to force something to directly align with such disparate and abstract requirements, so you have to create abstractions (layers) to enable a hypothetically-pure realm. In our architecture, we've created roughly 2 different kinds of abstraction: Platform and Business. The Platform abstractions are only intended to support themselves and the Business abstractions. These are not supposed to be easily-accessible by non-technical folks. Their entire purpose is to make the Business abstractions as elegant as possible. The idea is these should be slowly-changing and stable across all use cases. Developers are only expected to visit this realm maybe once every other week. We effectively treat ourselves as our own customer with this layer. The Business abstractions are built using only first-party primitives and our Platform layer. Most of this is very clean because we explicitly force all non-business concerns out into the Platform layer. Things like validation logic are written using purely functional code in this realm, which makes it very easy to reason with. Developers are expected to live in this area most of the time. Organization of business abstractions typically falls along lines of usage or logical business activity. We explicitly duplicate some code in order to maintain very clean separation of differing business activities. I've watched first hand as the subtle variances of a single combined Customer model across hundreds of contexts eventually turned it into a boat anchor. As a consequence of all this, our project managers and other non-code-experts are actually able to review large parts of the codebase and derive meaningful insights without a developer babysitting them. We are at a point where project managers can open PRs for adjusting basic validation and parameter items in our customers' biz code. Developers also enjoy not having to do massive altitude shifts on abstractions while reading through 1 source file. You either spend a day working platform concerns, or you are in the business layer. Context switching sucks and we built to avoid it as much as possible. IMO, having a bad project structure is one direct side-effect of being forced to context switch all the time.

ajsharma 6y ago
Wow, I love this idea. I'm curious how you have that structured within the code base. Is it literally a platform folder and a business folder?

deleted 6y ago

[deleted]

ldd 6y ago

There is this wonderful utility: dependency-cruiser[0] for javascript / typescript projects. It visualizes dependencies in a project. I found it so so easy to refactor and move files around after I started using it. I am not usually a visually-oriented person, but for this usecase, and to be happy, `dep-cruiser` surely helps. [0]: https://github.com/sverweij/dependency-cruiser https://github.com/sverweij/dependency-cruiser

soedirgo 6y ago

In Go, there's a standard project layout [1]. It'd be nice to have a project layout linter in Go Report Card [2]. [1]: https://github.com/golang-standards/project-layout https://github.com/golang-standards/project-layout [2]: https://goreportcard.com https://goreportcard.com

dgellow 6y ago
Just to clarify, that’s not an official standard at all (i.e: not supported by the go team). It’s a community project to establish some common layout. Surely valuable, but to say it is actually a Go standard is misleading IMHO.
- soedirgo 6y ago
  Ah! I stand corrected. Mea culpa.

nickjj 6y ago

For new projects (especially as a solo developer), there's also a related topic of not taking advantage of tools like kanban boards, or having a place outside of your source code to organize your thoughts and research. It's very possible to wind up with massive comment dumps of things to research, alternate implementations, notes to yourself and other things littered in your code base where you haven't made any git commits yet. This really leaves things in a messy state where you feel like the project is never going to be finished. An example of this and how I solved this problem with a kanban board can be found here: https://youtu.be/HHOkcCqsipE?t=76 https://youtu.be/HHOkcCqsipE?t=76

stefan_ 6y ago

Danger is a tool we integrated into our Continuous Integration system that performs post-commit automated checks You mean like the post-commit hook that Git offers out of the box? It's even named the exact same! I feel like we don't focus all this time on fast build test and deploy cycles only to then commit, navigate to some website to create a "merge request", wait for some fairy to allocate computing resources for trivial checks my computer could have done, only to get some pure noise "don't put this file here" comment and repeat the cycle all again.

core-questions 6y ago
When did you think I was going to get coffee and/or check Slack for more annoying work to do? It's like you want me to get things _done_, which is not fun at all.

momokoko 6y ago

Needing organized codebases is a personality type as there is zero academic research I’m aware of that has ever shown strict organization has resulted in less bugs or faster development. These are all done by people that need to feel in control and that everything has a place. So much money and time has been wasted on stuff like this with zero proven or measured benefit.

BurningFrog 6y ago
There is famously also no academic study showing that parachutes improve outcomes when jumping out of a plane.
- deleted 6y ago
  [deleted]
- darekkay 6y ago
  There is a study for that! "Parachute use to prevent death and major trauma when jumping from aircraft: randomized controlled trial". [1] [1] https://www.bmj.com/content/363/bmj.k5094 https://www.bmj.com/content/363/bmj.k5094
jfim 6y ago
Maybe, but there are other metrics than just developer output. If having a codebase that's well organized makes developers happier, and happier developers stay longer at the company, then that's also a win.
- momokoko 6y ago
  This makes some, definitely not all, developers happier. Organization for happiness is based on the individual developers personality and means you are putting some engineer’s happiness as a priority over spending that money providing other things to make the developers with a personality type that derives no benefit from strict organization. In fact, strict organization rules make some developers less happy. Why is their happiness less important when this is well known to have no business benefit?

battery_cowboy 6y ago

What if we just got rid of files and put our code into a database? You'd just have a "new code block" button, which creates the editor tab for your code, usually a function or a class, and usually one item per block. When you save it, it puts it into a database and you can version things easily. You can call other functions from the block and your editor will show their code when you mouseover or maybe some other method, just like today. Basically the same as today, but you don't need to worry about where some code lives. You back it with a great search feature to find stuff. Hell, let's just eliminate pathed files, why do we care about file paths with the level of search today? Just store everything in a key:value store directly on the hard drive, no paths needed. For legacy, just add keys for '/etc/fstab' or whatever.

peter_d_sherman 6y ago

>"The Slack iOS team lived in these conditions for a few too many years. We got here as a result of some attempts to organize source files (several times), a lack of architecture pattern in the codebase, and a high growth of developers over a couple years. To put things into context, we have roughly 13,000 files (and counting), about 27 top level directories, a mix of files in Objective-C and Swift, and around 40 iOS developers that work in one monorepo." An extensive, unrefactored codebase is no different than a jungle. You might have a 10'xer programmer on your staff, and he might hold the programming equivalent of a machete, but if the rate of his refactoring (assuming your corporate rules let him) is slower than the rate of new code being added by other employees, he is going to fail, no matter how good he is! I need to write a future essay about the relationship between 10'xers and how a 10'xer is a combination of only as good as how well the codebase is refactored, how well they know the codebase, how much corporate rules/polcies permit refactoring (or not), how much time he doesn't have to waste time solving stupid one-time issues from single customers, and how much help or pushback he is or isn't getting from the rest of the team. In other words, given the right set of conditions between codebase size and obfuscation, limiting corporate policies (i.e., "you can't refactor", "you can't make just one mistake in your code, because it's all mission critical, and if you do, you will be fired, and by the way, there's no test environment!"), distraction ("you have to help this customer with his cosmetic problem before you are permitted to tackle the guts of the system"), and pushback from the rest of the team, you can actually change 10'xers (and higher!) to 1x'ers and below... The reverse is true too... I'll make anyone a "My Fair Lady" / Eliza Doolittle style bet (or the reverse!) that what I say is true! That is, that 1'xers can be taught to become 10'xers, and conversely, 10'xers can be hampered by a variety of factors ("The Perfect Storm") resulting in them being slowed to 1'xers, or below...