11 ms·

Undebt: How We Refactored 3M Lines of Code

245 points by vivagn 10y ago

wry_discontent 10y ago
It seems to me this is only going to handle the most trivial kind of technical debt. This kind of tool can't manage the way you organized your codebase, for instance. There's more to refactoring than find-and-replace.
- sfrailsdev 10y ago
  Well even if it just gave you some reasonable abstraction over regex, at this kind of scale it seems useful. Plus I bet they had excellent test coverage, which really makes refactoring much easier.
knocte 10y ago
This smells as being a need that comes as a consequence of using a dynamically-typed language. Because the example given seems to be just getting rid of the usage of a certain method, to replace with a new one. In a statically typed language, e.g. C#, you just mark the old method with an [Obsolete] attribute and go fix all the warnings. (Granted, a tool that replaces all these usages is also useful, but to me, there are much more complex ways of technical debt than just obsolete methods.)
- smilekzs 10y ago
  In VS2015, showing all references to methods, props, classes, etc. is just one click away. Unfortunately it's not available in the community version. https://msdn.microsoft.com/en-us/library/dn269218.aspx?f=255&MSPPError=-2147217396 https://msdn.microsoft.com/en-us/library/dn269218.aspx?f=255...
  - knocte 10y ago
    Use MonoDevelop/XamarinStudio.
- manigandham 10y ago
  Exactly, this is just basic functionality of Visual Studio now, this whole operation could be done in a few seconds without issue. Fix any build errors, run tests, job done.
denfromufa 10y ago
Why pyparsing, not ply or regex?
- viraptor 10y ago
  Never used ply, but why not regex? Regex cannot easily match recursive structures, and can't be easily composed. Pyparsing gives you more structure and ready building blocks. It actually can use regex as one of the term objects. Ply tries to implement lex/yacc only, so it's far from a nice interface really... Or specifically: it's trivial to match `specific_func(any_expression, {capture, these, values})`, but it's practically impossible with regex and painful to post-process with ply.
- ggurgone 10y ago
  regexps are error prone better use ASTs see Facebook's jscodeshift
- coredog64 10y ago
  Jamie Zawinski <jwz@netscape.com> wrote on Tue, 12 Aug 1997 13:16:22 -0700: Some people, when confronted with a problem, think “I know, I'll use regular expressions.” Now they have two problems.
  - pkroll 10y ago
    "I've noticed that when I get quoted in .sig files, it's never any of the actual clever things I say all the time. Usually it's something dumb." --Jamie Zawinski
wRastel27 10y ago
"...time that could be better spent working on new features and shipping new code" Can we please stop putting forth this idea that features >>> reliable product? The amount of dev time that a company will save from removing technical debt will likely be more than the extra sales the company will get from a new feature. I look forward to the day where the executive team comes to the developers and ask why they are working on features instead of cutting down technical debt.
- ajdlinux 10y ago
  "The amount of dev time that a company will save from removing technical debt will likely be more than the extra sales the company will get from a new feature." Not always true (though it often is!), but particularly not always true in the timespan that the company needs to get sales in...
  - PhilipA 10y ago
    The problem is that this is very hard to measure. You can't really measure how much time you save on new features after you fixed the old spaghetti-code.
- iMerNibor 10y ago
  technical debt will not always manifest itself as instability/bad product. It might just affect development time of things interacting with the bad, old code - or make bugs harder to fix Changes to old nasty code will probably be hacky under the mentality "we're going to refactor this anyways, no point in writing clean code /now/" Most old code (i've worked with) will function fine - it has for a while after all.
- bigiain 10y ago
  There is an old Joel On Software blog post I was reading recently that talked about a methodology at Microsoft they called "Zero defects", where fixing known bug _alway_ had priority over working on new features. <google google google> Here - pont 5 in this post: http://www.joelonsoftware.com/articles/fog0000000043.html http://www.joelonsoftware.com/articles/fog0000000043.html
ajdlinux 10y ago
Related: http://coccinelle.lip6.fr/ http://coccinelle.lip6.fr/ Coccinelle is used extensively by Linux kernel developers for a whole tonne of things like this.
necubi 10y ago
For Java, IntelliJ has a built-in version of this called "structural search and replace" [0]. This is incredibly useful when a library changes an API or you need to refactor a lot of similar code. This feels relatively safe in Java because tooling can staticly know a lot about your code (and can know for sure that a particular call site is the method or class you're targeting). I've be terrified to do it in python without a very thorough test suite. [0] https://www.jetbrains.com/help/idea/2016.2/structural-search-and-replace.html https://www.jetbrains.com/help/idea/2016.2/structural-search...
- rubber_duck 10y ago
  IMO the main reason Python standard library is so wildly inconsistent. They don't really have the tools to migrate stuff painlessly and the 'batteries included' approach with weak versioning means you can't change stuff without breaking everyone who upgrades a python version.
  - twblalock 10y ago
    Even with a consistent standard library, you can't really be confident that you've properly refactored all usages of a class or function unless you have 100% test coverage -- and even if you did, it would be difficult to automate the refactoring the way you could with a static language.
  - vertex-four 10y ago
    Even if they changed the standard library from version to version... the result would be that people would stop using the standard library, migration tools or no tools. Nobody really wants to deal with being pinned to a specific minor version and no older/newer - especially libraries.
    - rubber_duck 10y ago
      The correct solution is to version the standard library separately from the language and allow for versioned dependencies, then a new language/VM update doesn't imply a new library that breaks everything, and vice versa. But python grew up in an enviroment where this sort of thing was not practical, and the batteries included is actually a good approach for what python tries to do - scripting. It just doesn't scale well in to maintainability.
Kendrick2 10y ago
How do web applications explode out to 3 Million lines of code? Yelp, to me, looks like a typical CRUD app and I would have been surprised if it were more than 100,000 lines of code. The software I develop is pretty large and typically doesn't surpass 40,000 sloc written in-house (i.e. excluding third party libs). Does anyone here maintain such large codebases? Are they truly that big or are people just counting third party code and generated stuff?
- spapas82 10y ago
  Lol I had exactly the same question :)
- vincnetas 10y ago
  This could be including backend tools not seen by simple user...
  - ggurgone 10y ago
    definitely also tests, deployment and generic tools
- hvidgaard 10y ago
  When you don't spend the time to refactor and groom the codebase, actively seek to reduce the complexity, it grows. And it grows exponentially. Once you reach the point where you're afraid to change something because it might break something unrelated, you just add new code all the time.
  - aidos 10y ago
    I've worked at a couple of places that went this way (joined after they already had millions of lines). It was code duplication. One example I saw was a many thousand line css file included 4 times in a row. They were all different generations and some were customer specific so it was an impossible task to try to consolidate. That was a common theme. Enterprise stuff where one customer needs slightly different behaviour so code is duplicated (maybe thousands of lines). The first time you take that road, you've doomed yourself. With a good / small team they can manage it- but imagine when you've got 20 devs sitting there of varying experience. The precedent is set and the solution for anything mildly complex will forever be, make a copy.
spapas82 10y ago
Was the 3M LOC refactoring for yelp.com? The article doesn't say. How could possibly a review site have 3M LOC?
- raverbashing 10y ago
  The front-facing system is the smallest part The admin and back-office system is deeper. Beyond the reviews you also have: events, mailing list management, profile management, messages, i18n, search, etc Example: http://engineeringblog.yelp.com/2015/10/how-we-use-deep-learning-to-classify-business-photos-at-yelp.html http://engineeringblog.yelp.com/2015/10/how-we-use-deep-lear... Edit: still, 3Mi lines is massive. However, I think there's something that contributes significantly: HTML and CSS
  - spapas82 10y ago
    Hmm, well even with all those, the amount of code just blows my mind!
    - qznc 10y ago
      Once I was talking to someone from SAP. He told me about their Netweaver framework is 1 billion LOC and 40k SQL tables. That does not include any application, yet. The biggest reason for the bloat: People are not allowed to change code. You only add code. Now try to add a button to some GUI without changing a single line of the existing code. At this moment I understood, why enterprises produce those overengineered AdapterFactorySingletonDecoratorBridge stuff.
  - throwaway_exer 10y ago
    i18n generally just adds property files and maybe localized CSS, not code.
- kasparsklavins 10y ago
  Reminds me of Ubisoft claims that its Assassin's Creed crouching mechanic had 6M LOC
- 10y ago
yitchelle 10y ago
I would also be interested the thought process in deciding what functionality to refactor. Did you review the code and identify areas before unleashing your tool on it? With 3M lines of code gone, it must be terrifying to feel that it may have broken something. How did you ensure that it is still working as before? Edit: Grammatic corrections.
- haylem 10y ago
  I had to deal with many sizable codebases of legacy code over the years and even answered a similar question once on StackExchange. Apparently people liked the answer: http://programmers.stackexchange.com/questions/155488/ive-inherited-200k-lines-of-spaghetti-code-what-now http://programmers.stackexchange.com/questions/155488/ive-in... The question was within a somewhat specific context (team of scientists, visual programming environment, etc...) but I use the same approach for all projects. Never reached 3M LoC for a single projet or system component though. More like 1.5M. Anyways, that process works for me. Maybe it does for others.
- perlgeek 10y ago
  One approach that's probably quite easy to implement is to generate test suite code coverage by file, and only do the substitutions in files with sufficient coverage (where your definition of "sufficient" might vary, of course).
vintermann 10y ago
It would be nice if some research institution would pay for the rehabilitation of some huge, bloated, ancient, but relatively unimportant app. Ideally by independent teams in parallel. Just to get some real data on what works, rather than anecdotes from veterans.
- bsaul 10y ago
  Your comment opened my eyes. Considering the importance of software development in today's world ( and tomorrow's), the fact that those best practices or code management technics are found mostly in blogs, instead of scientific papers with proper experiments, tells a lot.
  - skrebbel 10y ago
    I don't follow. What does it tell?
    - sidlls 10y ago
      Have you heard of the term/phrase "bro science" with respect to weight lifting? I'd draw an analogy between them. Clarification: what I mean is there is a lot of ad-hoc stuff in these blogs that may be narrowly true or applicable to a given project or subset of a domain, but generally there is either significant evidence against some of these blogs' contents or the contents themselves have absolutely no formal, rigorous support.
      - skrebbel 10y ago
        I see your point, thanks. But the fact that not a lot of research is done in this field doesn't tell you much about the validity of the non-scientific blogosphere findings, right? Nice analogy btw.
      - coredog64 10y ago
        I switched to Scrum, check out my GAINZ!
  - 10y ago
kctess5 10y ago
It's like codesearch [1]. [1] https://github.com/google/codesearch https://github.com/google/codesearch
impish19 10y ago
Having an intern write a good blog post for the engineering blog is a great recruiting move.
- yarper 10y ago
  Or a horrible statement at the level of company interest in tech debt - a problem given to the interns.
mahyarm 10y ago
The amount of code you have in a company is usually a function of the amount of engineers you have.
- jobigoud 10y ago
  Sad. Hopefully that function is sub-linear.
deleted 10y ago
[deleted]
yarper 10y ago
The fact that you need a find and replace tool to make sensible changes to your codebase indicates that it's hugely out of control already.
NikhilVerma 10y ago
IMO a much better approach is JSCodeShift, which works based on the AST: https://github.com/facebook/jscodeshift https://github.com/facebook/jscodeshift
crdoconnor 10y ago
* Newline at EOF * Double quoted docstring * Remove unused imports. These things are largely cosmetic.
- TeeWEE 10y ago
  Indeed what i was thinking. A big refactoring is often structural, and changes the program in a bigger way. In Java you can actually easly move classes around by drag-n-drop and they code will refactor. In python this is impossible.
  - crdoconnor 10y ago
    A big refactoring often means reducing the total amount of code you write to make it more readable. In Java it's usually going to be about 2x what the equivalent in python would be.
  - zo1 10y ago
    Most Python IDEs (PyCharm and PyDev, for example) do that for you, even if it's not "drag-n-drop" like in Java...
  - huherto 10y ago
    This is a big part of using java for me. I use the refactor tools all the time.
ksec 10y ago
Interesting, I have always ( wrongly ) remember yelp as a Ruby Rails shop. Does anyone know the stack behind yelp?
- bpicolo 10y ago
  It is mostly Python, both the monolith and SoA. A few services are Java, e.g. for talking to Lucene. The github site actually gives a reasonably complete view of the stack via the various tools we have for integrating with various parts of the stack. https://yelp.github.io/ https://yelp.github.io/
avindroth 10y ago
Is there a Haskell version for this?
- kelvin0 10y ago
  Haskell code (as with LISP code) does not need refactoring. It always comes out as a distilled, pure and crystalline elixir of wisdom. :)
fleaflicker 10y ago
Google has a similar tool called Refaster https://github.com/google/Refaster https://github.com/google/Refaster
pmarreck 10y ago
A funny and relevant tweet: https://twitter.com/php_ceo/status/765298072691806209 https://twitter.com/php_ceo/status/765298072691806209
ilostmykeys 10y ago
The patterns that Undebt removes could be non-existent in the code base, but the conceptual, algorithmic design decisions and architecture could be all bad and that is what the actual technical debt is. Bad code patterns are just a tiny slice of the problem in most cases.
pmarreck 10y ago
I have my doubts that a refactor that consists merely of more complex search and replace actions is a true refactor. Also, this reads more like an ad for their Python tool than about any lessons learned during this refactoring.
karlheinz 10y ago
“Let a 1,000 flowers bloom. Then rip 999 of them out by the roots.” This is paraphrasing of Chairman Mao: "The policy of letting a hundred flowers bloom and a hundred schools of thought contend is designed to promote the flourishing of the arts and the progress of science" And the ripping out by roots part brings labor camps to mind: "After this brief period of liberalization, Mao abruptly changed course. The crackdown continued through 1957 as an Anti-Rightist Campaign against those who were critical of the regime and its ideology. Those targeted were publicly criticized and condemned to prison labor camps." https://en.wikipedia.org/wiki/Hundred_Flowers_Campaign https://en.wikipedia.org/wiki/Hundred_Flowers_Campaign
swagv 10y ago
The company formerly known as Minnesota Mining and Manufacturing?
manigandham 10y ago
Refactoring... "puts a massive drain on developer time; time that could be better spent working on new features and shipping new code" This is the wrong way to think. Refactoring will save time by making code faster, more reliable and making it easier to build those new features in the first place. Looks like their biggest issue is bad technical management, not deprecated code.
hendry 10y ago
If any of you guys are interested plotting total lines of code changes, do checkout https://github.com/kaihendry/graphsloc https://github.com/kaihendry/graphsloc

245 points by vivagn 10y ago

It seems to me this is only going to handle the most trivial kind of technical debt. This kind of tool can't manage the way you organized your codebase, for instance. There's more to refactoring than find-and-replace.

sfrailsdev 10y ago
Well even if it just gave you some reasonable abstraction over regex, at this kind of scale it seems useful. Plus I bet they had excellent test coverage, which really makes refactoring much easier.

knocte 10y ago

This smells as being a need that comes as a consequence of using a dynamically-typed language. Because the example given seems to be just getting rid of the usage of a certain method, to replace with a new one. In a statically typed language, e.g. C#, you just mark the old method with an [Obsolete] attribute and go fix all the warnings. (Granted, a tool that replaces all these usages is also useful, but to me, there are much more complex ways of technical debt than just obsolete methods.)

smilekzs 10y ago
In VS2015, showing all references to methods, props, classes, etc. is just one click away. Unfortunately it's not available in the community version. https://msdn.microsoft.com/en-us/library/dn269218.aspx?f=255&MSPPError=-2147217396 https://msdn.microsoft.com/en-us/library/dn269218.aspx?f=255...
- knocte 10y ago
  Use MonoDevelop/XamarinStudio.
manigandham 10y ago
Exactly, this is just basic functionality of Visual Studio now, this whole operation could be done in a few seconds without issue. Fix any build errors, run tests, job done.

denfromufa 10y ago

Why pyparsing, not ply or regex?

viraptor 10y ago
Never used ply, but why not regex? Regex cannot easily match recursive structures, and can't be easily composed. Pyparsing gives you more structure and ready building blocks. It actually can use regex as one of the term objects. Ply tries to implement lex/yacc only, so it's far from a nice interface really... Or specifically: it's trivial to match `specific_func(any_expression, {capture, these, values})`, but it's practically impossible with regex and painful to post-process with ply.
ggurgone 10y ago
regexps are error prone better use ASTs see Facebook's jscodeshift
coredog64 10y ago
Jamie Zawinski <jwz@netscape.com> wrote on Tue, 12 Aug 1997 13:16:22 -0700: Some people, when confronted with a problem, think “I know, I'll use regular expressions.” Now they have two problems.
- pkroll 10y ago
  "I've noticed that when I get quoted in .sig files, it's never any of the actual clever things I say all the time. Usually it's something dumb." --Jamie Zawinski

wRastel27 10y ago

"...time that could be better spent working on new features and shipping new code" Can we please stop putting forth this idea that features >>> reliable product? The amount of dev time that a company will save from removing technical debt will likely be more than the extra sales the company will get from a new feature. I look forward to the day where the executive team comes to the developers and ask why they are working on features instead of cutting down technical debt.

ajdlinux 10y ago
"The amount of dev time that a company will save from removing technical debt will likely be more than the extra sales the company will get from a new feature." Not always true (though it often is!), but particularly not always true in the timespan that the company needs to get sales in...
- PhilipA 10y ago
  The problem is that this is very hard to measure. You can't really measure how much time you save on new features after you fixed the old spaghetti-code.
iMerNibor 10y ago
technical debt will not always manifest itself as instability/bad product. It might just affect development time of things interacting with the bad, old code - or make bugs harder to fix Changes to old nasty code will probably be hacky under the mentality "we're going to refactor this anyways, no point in writing clean code /now/" Most old code (i've worked with) will function fine - it has for a while after all.
bigiain 10y ago
There is an old Joel On Software blog post I was reading recently that talked about a methodology at Microsoft they called "Zero defects", where fixing known bug _alway_ had priority over working on new features. <google google google> Here - pont 5 in this post: http://www.joelonsoftware.com/articles/fog0000000043.html http://www.joelonsoftware.com/articles/fog0000000043.html

ajdlinux 10y ago

Related: http://coccinelle.lip6.fr/ http://coccinelle.lip6.fr/ Coccinelle is used extensively by Linux kernel developers for a whole tonne of things like this.

necubi 10y ago

For Java, IntelliJ has a built-in version of this called "structural search and replace" [0]. This is incredibly useful when a library changes an API or you need to refactor a lot of similar code. This feels relatively safe in Java because tooling can staticly know a lot about your code (and can know for sure that a particular call site is the method or class you're targeting). I've be terrified to do it in python without a very thorough test suite. [0] https://www.jetbrains.com/help/idea/2016.2/structural-search-and-replace.html https://www.jetbrains.com/help/idea/2016.2/structural-search...

rubber_duck 10y ago
IMO the main reason Python standard library is so wildly inconsistent. They don't really have the tools to migrate stuff painlessly and the 'batteries included' approach with weak versioning means you can't change stuff without breaking everyone who upgrades a python version.
- twblalock 10y ago
  Even with a consistent standard library, you can't really be confident that you've properly refactored all usages of a class or function unless you have 100% test coverage -- and even if you did, it would be difficult to automate the refactoring the way you could with a static language.
- vertex-four 10y ago
  Even if they changed the standard library from version to version... the result would be that people would stop using the standard library, migration tools or no tools. Nobody really wants to deal with being pinned to a specific minor version and no older/newer - especially libraries.
  - rubber_duck 10y ago
    The correct solution is to version the standard library separately from the language and allow for versioned dependencies, then a new language/VM update doesn't imply a new library that breaks everything, and vice versa. But python grew up in an enviroment where this sort of thing was not practical, and the batteries included is actually a good approach for what python tries to do - scripting. It just doesn't scale well in to maintainability.

Kendrick2 10y ago

How do web applications explode out to 3 Million lines of code? Yelp, to me, looks like a typical CRUD app and I would have been surprised if it were more than 100,000 lines of code. The software I develop is pretty large and typically doesn't surpass 40,000 sloc written in-house (i.e. excluding third party libs). Does anyone here maintain such large codebases? Are they truly that big or are people just counting third party code and generated stuff?

spapas82 10y ago
Lol I had exactly the same question :)
vincnetas 10y ago
This could be including backend tools not seen by simple user...
- ggurgone 10y ago
  definitely also tests, deployment and generic tools
hvidgaard 10y ago
When you don't spend the time to refactor and groom the codebase, actively seek to reduce the complexity, it grows. And it grows exponentially. Once you reach the point where you're afraid to change something because it might break something unrelated, you just add new code all the time.
- aidos 10y ago
  I've worked at a couple of places that went this way (joined after they already had millions of lines). It was code duplication. One example I saw was a many thousand line css file included 4 times in a row. They were all different generations and some were customer specific so it was an impossible task to try to consolidate. That was a common theme. Enterprise stuff where one customer needs slightly different behaviour so code is duplicated (maybe thousands of lines). The first time you take that road, you've doomed yourself. With a good / small team they can manage it- but imagine when you've got 20 devs sitting there of varying experience. The precedent is set and the solution for anything mildly complex will forever be, make a copy.

spapas82 10y ago

Was the 3M LOC refactoring for yelp.com? The article doesn't say. How could possibly a review site have 3M LOC?

raverbashing 10y ago
The front-facing system is the smallest part The admin and back-office system is deeper. Beyond the reviews you also have: events, mailing list management, profile management, messages, i18n, search, etc Example: http://engineeringblog.yelp.com/2015/10/how-we-use-deep-learning-to-classify-business-photos-at-yelp.html http://engineeringblog.yelp.com/2015/10/how-we-use-deep-lear... Edit: still, 3Mi lines is massive. However, I think there's something that contributes significantly: HTML and CSS
- spapas82 10y ago
  Hmm, well even with all those, the amount of code just blows my mind!
  - qznc 10y ago
    Once I was talking to someone from SAP. He told me about their Netweaver framework is 1 billion LOC and 40k SQL tables. That does not include any application, yet. The biggest reason for the bloat: People are not allowed to change code. You only add code. Now try to add a button to some GUI without changing a single line of the existing code. At this moment I understood, why enterprises produce those overengineered AdapterFactorySingletonDecoratorBridge stuff.
- throwaway_exer 10y ago
  i18n generally just adds property files and maybe localized CSS, not code.
kasparsklavins 10y ago
Reminds me of Ubisoft claims that its Assassin's Creed crouching mechanic had 6M LOC
10y ago

yitchelle 10y ago

I would also be interested the thought process in deciding what functionality to refactor. Did you review the code and identify areas before unleashing your tool on it? With 3M lines of code gone, it must be terrifying to feel that it may have broken something. How did you ensure that it is still working as before? Edit: Grammatic corrections.

haylem 10y ago
I had to deal with many sizable codebases of legacy code over the years and even answered a similar question once on StackExchange. Apparently people liked the answer: http://programmers.stackexchange.com/questions/155488/ive-inherited-200k-lines-of-spaghetti-code-what-now http://programmers.stackexchange.com/questions/155488/ive-in... The question was within a somewhat specific context (team of scientists, visual programming environment, etc...) but I use the same approach for all projects. Never reached 3M LoC for a single projet or system component though. More like 1.5M. Anyways, that process works for me. Maybe it does for others.
perlgeek 10y ago
One approach that's probably quite easy to implement is to generate test suite code coverage by file, and only do the substitutions in files with sufficient coverage (where your definition of "sufficient" might vary, of course).

vintermann 10y ago

It would be nice if some research institution would pay for the rehabilitation of some huge, bloated, ancient, but relatively unimportant app. Ideally by independent teams in parallel. Just to get some real data on what works, rather than anecdotes from veterans.

bsaul 10y ago
Your comment opened my eyes. Considering the importance of software development in today's world ( and tomorrow's), the fact that those best practices or code management technics are found mostly in blogs, instead of scientific papers with proper experiments, tells a lot.
- skrebbel 10y ago
  I don't follow. What does it tell?
  - sidlls 10y ago
    Have you heard of the term/phrase "bro science" with respect to weight lifting? I'd draw an analogy between them. Clarification: what I mean is there is a lot of ad-hoc stuff in these blogs that may be narrowly true or applicable to a given project or subset of a domain, but generally there is either significant evidence against some of these blogs' contents or the contents themselves have absolutely no formal, rigorous support.
    - skrebbel 10y ago
      I see your point, thanks. But the fact that not a lot of research is done in this field doesn't tell you much about the validity of the non-scientific blogosphere findings, right? Nice analogy btw.
    - coredog64 10y ago
      I switched to Scrum, check out my GAINZ!
- 10y ago

kctess5 10y ago

It's like codesearch [1]. [1] https://github.com/google/codesearch https://github.com/google/codesearch

impish19 10y ago

Having an intern write a good blog post for the engineering blog is a great recruiting move.

yarper 10y ago
Or a horrible statement at the level of company interest in tech debt - a problem given to the interns.

mahyarm 10y ago

The amount of code you have in a company is usually a function of the amount of engineers you have.

jobigoud 10y ago
Sad. Hopefully that function is sub-linear.

deleted 10y ago

[deleted]

yarper 10y ago

The fact that you need a find and replace tool to make sensible changes to your codebase indicates that it's hugely out of control already.

NikhilVerma 10y ago

IMO a much better approach is JSCodeShift, which works based on the AST: https://github.com/facebook/jscodeshift https://github.com/facebook/jscodeshift

crdoconnor 10y ago

* Newline at EOF * Double quoted docstring * Remove unused imports. These things are largely cosmetic.

TeeWEE 10y ago
Indeed what i was thinking. A big refactoring is often structural, and changes the program in a bigger way. In Java you can actually easly move classes around by drag-n-drop and they code will refactor. In python this is impossible.
- crdoconnor 10y ago
  A big refactoring often means reducing the total amount of code you write to make it more readable. In Java it's usually going to be about 2x what the equivalent in python would be.
- zo1 10y ago
  Most Python IDEs (PyCharm and PyDev, for example) do that for you, even if it's not "drag-n-drop" like in Java...
- huherto 10y ago
  This is a big part of using java for me. I use the refactor tools all the time.

ksec 10y ago

Interesting, I have always ( wrongly ) remember yelp as a Ruby Rails shop. Does anyone know the stack behind yelp?

bpicolo 10y ago
It is mostly Python, both the monolith and SoA. A few services are Java, e.g. for talking to Lucene. The github site actually gives a reasonably complete view of the stack via the various tools we have for integrating with various parts of the stack. https://yelp.github.io/ https://yelp.github.io/

avindroth 10y ago

Is there a Haskell version for this?

kelvin0 10y ago
Haskell code (as with LISP code) does not need refactoring. It always comes out as a distilled, pure and crystalline elixir of wisdom. :)

fleaflicker 10y ago

Google has a similar tool called Refaster https://github.com/google/Refaster https://github.com/google/Refaster

pmarreck 10y ago

A funny and relevant tweet: https://twitter.com/php_ceo/status/765298072691806209 https://twitter.com/php_ceo/status/765298072691806209

ilostmykeys 10y ago

The patterns that Undebt removes could be non-existent in the code base, but the conceptual, algorithmic design decisions and architecture could be all bad and that is what the actual technical debt is. Bad code patterns are just a tiny slice of the problem in most cases.

pmarreck 10y ago

I have my doubts that a refactor that consists merely of more complex search and replace actions is a true refactor. Also, this reads more like an ad for their Python tool than about any lessons learned during this refactoring.

karlheinz 10y ago

“Let a 1,000 flowers bloom. Then rip 999 of them out by the roots.” This is paraphrasing of Chairman Mao: "The policy of letting a hundred flowers bloom and a hundred schools of thought contend is designed to promote the flourishing of the arts and the progress of science" And the ripping out by roots part brings labor camps to mind: "After this brief period of liberalization, Mao abruptly changed course. The crackdown continued through 1957 as an Anti-Rightist Campaign against those who were critical of the regime and its ideology. Those targeted were publicly criticized and condemned to prison labor camps." https://en.wikipedia.org/wiki/Hundred_Flowers_Campaign https://en.wikipedia.org/wiki/Hundred_Flowers_Campaign

swagv 10y ago

The company formerly known as Minnesota Mining and Manufacturing?

manigandham 10y ago

Refactoring... "puts a massive drain on developer time; time that could be better spent working on new features and shipping new code" This is the wrong way to think. Refactoring will save time by making code faster, more reliable and making it easier to build those new features in the first place. Looks like their biggest issue is bad technical management, not deprecated code.

hendry 10y ago

If any of you guys are interested plotting total lines of code changes, do checkout https://github.com/kaihendry/graphsloc https://github.com/kaihendry/graphsloc