11 ms·

Sorbet and 100% cov makes Ruby refactoring possible

147 points by craigkerstiens 5y ago

sosodev 5y ago
Types are definitely very useful for complex refactoring. However, I disagree with the author that it wasn’t possible before types. Combining a healthy amount of unit tests along with some integration tests makes complex refactoring totally possible even without types.
- runeks 5y ago
  Tests can almost by definition give stronger guarantees than a type system, since the former is Turing complete while the latter is (almost always) not. This means that, for any type, it’s possible to replicate the check done by the compiler at compile-time in a unit test but not vice versa. However, the ergonomics of this is worse than types, I’d argue, since a type system offers a much simpler way to assert basic stuff at compile-time (e.g. the presence of certain properties of an object). Also, type annotations are inline instead of being defined in a separate source file, which aids in code readability.
  - dragonwriter 5y ago
    > Tests can almost by definition give stronger guarantees than a type system, False. > This means that, for any type, it’s possible to replicate the check done by the compiler at compile-time in a unit test but not vice versa. False. I mean, Turing completeness does mean you could replicate any type guarantee in code, but that code would generally not be a unit test; guarantees provided by type systems are universally quantified, to confirm that with testing would require testing every combination of values in the function’s domain (as determined by its parameter types.) The code would instead be a static typechecker that consumed the untyped source code plus a representation of the type assertions to be checked and where they attach to the untyped source code, and then did exactly the same things that any other static typechecker would do. That is... standard optional static typecheckers for dynamic languages. (Though those usually also support inline type assertions, perhaps via comments in languages without specific syntactic support, e.g., Python 2.x) You can approximate universally quantified guarantees in a testing regime with property-based testing, but that's at best probabilistic rather than actual universal quantification.
- sbergot 5y ago
  In my experience unit tests tend to slow down refactoring.
d4mi3n 5y ago
This reflects my experience working with Ruby over the years as well. I find it a perfectly pleasant language, but as with many other scripting languages, things get difficult as a project increases in size, complexity, and number of contributors. Types are a great way to remove certain classes of issues. It's my hope that newer versions of Ruby really push gradual typing features I've been hearing about forward into common use. The productivity gains of preventing all silly Nil and type errors will be enormous.
- valcron1000 5y ago
  Setting aside all the dynamic stuff that you can't statically check, what does Ruby offer fhat makes it so "pleasant" to use? I'm trying to understand what a statically typed Ruby could bring to the table.
  - andrewzah 5y ago
    There essentially already is a statically typed ruby: Crystal. https://en.wikipedia.org/wiki/Crystal_(programming_language) https://en.wikipedia.org/wiki/Crystal_(programming_language)
    - chrisseaton 5y ago
      Crystal has a look-alike syntax to Ruby. But the semantics (the important bit) aren't even remotely the same.
      - andrewzah 5y ago
        What semantics is crystal lacking? Obviously it's not a 1:1 replacement as the ecosystem is different, gem management (shards) is different, etc. Crystal also has union types and concurrency. For people who want a syntax like ruby + performance/concurrency, but are willing to deal with a different ecosystem and having to do more stuff by hand, Crystal is a nice choice.
        5y ago
DerArzt 5y ago
I get what the author is saying, but static analysis aside I would do all that I can to avoid a 178 file update. I get that sorbet is allowing them to do this with higher confidence but even in a language that's compiled (thus performing a similar role to Sorbet) that much change is asking for trouble. If making a single update touches that many source files, then they may want to take a look at their code organization and architecture as that is a whole heck of a lot!
- shoo 5y ago
  the most productive i have ever been working as a software engineer was in a smaller company where roughly all of the company's product and library code was in a monorepo, and we had a reasonable culture of writing automated tests (although not at 100% branch coverage). sometimes you realise the abstraction in a core library used by all the product code is wrong or limiting, and if your test suite gives you confidence that you're going to catch any potential regressions, you can rework the library with a breaking change & fix everything that transitively depends on that library in a single atomic commit.
  - DerArzt 5y ago
    The rub for me here is that making one change in a core library ideally wouldn't touch 100+ other source files. This feels like a separation of concerns issue in which a module is being over used. Perhaps I'm just in the wrong headspace, as I have never had to work in a monorepo fashion. That being said just because your code is in a monorepo, does that directly mean that you aren't versioning your libraries? To my understanding most package management tooling in most echo systems allow for versioning. With that said, wouldn't you slowly roll out the version to the downstream projects Monorepo or not?
    - swatcoder 5y ago
      Refactoring doesn’t just mean optimizing the inside of an implementation. With a monorepo and good test coverage, I can improve the signature of a function on some library, and use a full-featured IDE to confidently update the usage myself without making it a 10-ticket task that spans 12 teams and 18 meetings.
pqdbr 5y ago
It's a shame the article ends with "So as usual, consider not writing Ruby, but if you do, ...". That's worthless advice and many people, myself included, have Ruby as their most beloved programming language.
- bestinterest 5y ago
  There is a gap between languages at the moment imo. Ruby is just ruthlessly productive. Compare https://github.com/benhoyt/countwords/blob/master/simple.rb https://github.com/benhoyt/countwords/blob/master/simple.rb with https://github.com/benhoyt/countwords/blob/master/simple.go https://github.com/benhoyt/countwords/blob/master/simple.go Hoping new age compiled languages like Crystal & Nim fill the gap of performance, types & productivity. But compile speeds need to be factored in.
  - rco8786 5y ago
    > Ruby is just ruthlessly productive. This is the best description of Ruby I've ever read.
    - princevegeta89 5y ago
      +1. After working on Elixir for a long time I missed the simple ruby way to do things and found myself running in circles to troubleshoot some basic 3rd party libraries. Going back to Ruby for my next side project.
      - AlchemistCamp 5y ago
        Strange. I like Ruby, but probably the biggest gain I got from moving to Elixir was the comparative ease of debugging. Its immutability drastically reduces the search space.
        princevegeta89 5y ago
        Agree with your point about debugging being easier but in my case I ended up doing a lot of housekeeping for things that are easy in ruby. This time I'm only concerned about the speed of development and ruby to me is the winner
mpweiher 5y ago
The idea that refactoring requires static types is ahistoric and simply incorrect. The first automatic refactoring tool was the Refactoring Browser, in Smalltalk. https://refactory.com/refactoring-browser/ https://refactory.com/refactoring-browser/ https://www.researchgate.net/publication/220346807_A_Refactoring_Tool_for_Smalltalk https://www.researchgate.net/publication/220346807_A_Refacto...
- recuter 5y ago
  I don't understand why this is the bottom comment instead of the top one.
- a1445c8b 5y ago
  The PDF[1] that the second link points to gives a very good background on why a refactoring tool should be part of any software engineer's toolkit. Thanks for sharing! [1] https://www.researchgate.net/profile/Don-Roberts-4/publication/220346807_A_Refactoring_Tool_for_Smalltalk/links/5ecd251d92851c9c5e5b931f/A-Refactoring-Tool-for-Smalltalk.pdf https://www.researchgate.net/profile/Don-Roberts-4/publicati...
- injidup 5y ago
  Because the technique it uses is "clever" and at the same time terrible. It only works if you have 100% code coverage in automatic or monkey testing. The key quote from the paper is. """ as the program is exercised, it converges towards a correctly refactored program. """ This occurs because the refactoring engine uses runtime data to discover the correct types and thus correct refactorings. It is certainly not the same robustness in refactoring that you get with a statically typed language. I don't want my refactoring to "converge". I want it done.
  - igouy 5y ago
    The key quote from the paper is — "We do not attempt to automatically refactor code." "A very large Smalltalk application was developed at Cargill to support the operation of grain elevators and the associated commodity trading activities. The Smalltalk client application has 385 windows and over 5,000 classes. About 2,000 classes in this application interacted with an early (circa 1993) data access framework. The framework dynamically performed a mapping of object attributes to data table columns. Analysis showed that although dynamic look up consumed 40% of the client execution time, it was unnecessary. A new data layer interface was developed that required the business class to provide the object attribute to column mapping in an explicitly coded method. Testing showed that this interface was orders of magnitude faster. The issue was how to change the 2,100 business class users of the data layer. A large application under development cannot freeze code while a transformation of an interface is constructed and tested. We had to construct and test the transformations in a parallel branch of the code repository from the main development stream. When the transformation was fully tested, then it was applied to the main code stream in a single operation. Less than 35 bugs were found in the 17,100 changes. All of the bugs were quickly resolved in a three-week period. If the changes were done manually we estimate that it would have taken 8,500 hours, compared with 235 hours to develop the transformation rules. The task was completed in 3% of the expected time by using Rewrite Rules. This is an improvement by a factor of 36." from “Transformation of an application data layer” Will Loew-Blosser OOPSLA 2002 http://portal.acm.org/citation.cfm?id=604258 http://portal.acm.org/citation.cfm?id=604258
alberth 5y ago
NIM I'm still surprised more startups don't use NIM. It has all of the productivity benefits of Python but performance of C.
- brigandish 5y ago
  Did you come to Nim from a Ruby background? I've been using Crystal and am very happy with it but would be interested to know what the jump to Nim might be like.
- deleted 5y ago
  [deleted]
bxjx 5y ago
Satisfying static types AND 100% code cov is a significant burden. If you're intrigued by this approach, or perhaps by a similar approach with typescript, I highly recommend checking out a strictly strong typed language at some point. I've been coding in Elm and a little in Haskell, and I find that the type checking is so thorough and exhaustive that I only need to write very few tests to get strong guarantees. This was a very pleasant change from Rails where you are encouraged to aim for 100% code cov, and as such, would spend well over 40% of my development writing tests.
- devoutsalsa 5y ago
  It’s worth noting that just because it compiles, that doesn’t mean it does what you want.
  - runeks 5y ago
    True. Nor does 100% code coverage mean it does what you want.
userbinator 5y ago
I thought this would be about spending quarantine time eating sorbet and refactoring Ruby code.
raggi 5y ago
About a decade ago I had a social media marketing platform (Wildfire) part of which was a special case web page builder. We scaled our engineering team in many different ways, one of the many ways was to also contract external teams. One such team wrote a book based on code the wrote for us, Objects on Rails. The code was fine for a time, but we were increasingly serving traffic from Youtube which was a very popular platform for us at the time. The code in question could not scale in a cost effective manner for traffic coming from Youtube. In fact, it was stressing our ability to maintain a rational number of haproxy POPs - we'd have needed to massively change our operational architecture if the root problem could not be fixed. It took a while to identify, but eventually we found the exhibits abstraction to be the root of a number of significant and leaky performance problems (runtime object extension, blowing all method caches). I refactored out all of the uses of this code, replacing them with simple in-place conditionals in all cases. This dropped somewhere around 15kloc, and modified another 5k or so (I think these may be under-estimates, but I no longer have access to the code, and it's been a decade). We were able to deploy that patch with very few changes. The teams had done a good job of writing extensive end to end tests. It also helped that removing the abstraction resulted in far simpler code paths. I didn't need types to do this safely, the tests were sufficient. This is not to be a downer on typing extensions, but I think their advantages are often over-sold.
- voz_ 5y ago
  Sorry, I don't buy it. Tests might be sufficient, but they cannot detect a new issue. They can only prove that a well known one is not there. Strict typing and compile time enforcement will guarantee that a whole class of errors and incompatibilities will not occur.
- zht 5y ago
  your story is about one person working on one project I think likely typing would help at larger organizations where teams often code against other teams' code and/or services with unclear interfaces
  - raggi 5y ago
    The organization that owned the code at the time was Google, and there were multiple teams contributing to that code base concurrently with the patch in question.
manmal 5y ago
Having learned some Ruby over the years, and seeing how elegant Elixir is, leads me to the question: Is Elixir‘s type System strong enough to make refactoring worry-free (eg compared to Ruby)?

Types are definitely very useful for complex refactoring. However, I disagree with the author that it wasn’t possible before types. Combining a healthy amount of unit tests along with some integration tests makes complex refactoring totally possible even without types.

runeks 5y ago
Tests can almost by definition give stronger guarantees than a type system, since the former is Turing complete while the latter is (almost always) not. This means that, for any type, it’s possible to replicate the check done by the compiler at compile-time in a unit test but not vice versa. However, the ergonomics of this is worse than types, I’d argue, since a type system offers a much simpler way to assert basic stuff at compile-time (e.g. the presence of certain properties of an object). Also, type annotations are inline instead of being defined in a separate source file, which aids in code readability.
- dragonwriter 5y ago
  > Tests can almost by definition give stronger guarantees than a type system, False. > This means that, for any type, it’s possible to replicate the check done by the compiler at compile-time in a unit test but not vice versa. False. I mean, Turing completeness does mean you could replicate any type guarantee in code, but that code would generally not be a unit test; guarantees provided by type systems are universally quantified, to confirm that with testing would require testing every combination of values in the function’s domain (as determined by its parameter types.) The code would instead be a static typechecker that consumed the untyped source code plus a representation of the type assertions to be checked and where they attach to the untyped source code, and then did exactly the same things that any other static typechecker would do. That is... standard optional static typecheckers for dynamic languages. (Though those usually also support inline type assertions, perhaps via comments in languages without specific syntactic support, e.g., Python 2.x) You can approximate universally quantified guarantees in a testing regime with property-based testing, but that's at best probabilistic rather than actual universal quantification.
sbergot 5y ago
In my experience unit tests tend to slow down refactoring.

d4mi3n 5y ago

This reflects my experience working with Ruby over the years as well. I find it a perfectly pleasant language, but as with many other scripting languages, things get difficult as a project increases in size, complexity, and number of contributors. Types are a great way to remove certain classes of issues. It's my hope that newer versions of Ruby really push gradual typing features I've been hearing about forward into common use. The productivity gains of preventing all silly Nil and type errors will be enormous.

valcron1000 5y ago
Setting aside all the dynamic stuff that you can't statically check, what does Ruby offer fhat makes it so "pleasant" to use? I'm trying to understand what a statically typed Ruby could bring to the table.
- andrewzah 5y ago
  There essentially already is a statically typed ruby: Crystal. https://en.wikipedia.org/wiki/Crystal_(programming_language) https://en.wikipedia.org/wiki/Crystal_(programming_language)
  - chrisseaton 5y ago
    Crystal has a look-alike syntax to Ruby. But the semantics (the important bit) aren't even remotely the same.
    - andrewzah 5y ago
      What semantics is crystal lacking? Obviously it's not a 1:1 replacement as the ecosystem is different, gem management (shards) is different, etc. Crystal also has union types and concurrency. For people who want a syntax like ruby + performance/concurrency, but are willing to deal with a different ecosystem and having to do more stuff by hand, Crystal is a nice choice.
      - 5y ago

DerArzt 5y ago

I get what the author is saying, but static analysis aside I would do all that I can to avoid a 178 file update. I get that sorbet is allowing them to do this with higher confidence but even in a language that's compiled (thus performing a similar role to Sorbet) that much change is asking for trouble. If making a single update touches that many source files, then they may want to take a look at their code organization and architecture as that is a whole heck of a lot!

shoo 5y ago
the most productive i have ever been working as a software engineer was in a smaller company where roughly all of the company's product and library code was in a monorepo, and we had a reasonable culture of writing automated tests (although not at 100% branch coverage). sometimes you realise the abstraction in a core library used by all the product code is wrong or limiting, and if your test suite gives you confidence that you're going to catch any potential regressions, you can rework the library with a breaking change & fix everything that transitively depends on that library in a single atomic commit.
- DerArzt 5y ago
  The rub for me here is that making one change in a core library ideally wouldn't touch 100+ other source files. This feels like a separation of concerns issue in which a module is being over used. Perhaps I'm just in the wrong headspace, as I have never had to work in a monorepo fashion. That being said just because your code is in a monorepo, does that directly mean that you aren't versioning your libraries? To my understanding most package management tooling in most echo systems allow for versioning. With that said, wouldn't you slowly roll out the version to the downstream projects Monorepo or not?
  - swatcoder 5y ago
    Refactoring doesn’t just mean optimizing the inside of an implementation. With a monorepo and good test coverage, I can improve the signature of a function on some library, and use a full-featured IDE to confidently update the usage myself without making it a 10-ticket task that spans 12 teams and 18 meetings.

pqdbr 5y ago

It's a shame the article ends with "So as usual, consider not writing Ruby, but if you do, ...". That's worthless advice and many people, myself included, have Ruby as their most beloved programming language.

bestinterest 5y ago
There is a gap between languages at the moment imo. Ruby is just ruthlessly productive. Compare https://github.com/benhoyt/countwords/blob/master/simple.rb https://github.com/benhoyt/countwords/blob/master/simple.rb with https://github.com/benhoyt/countwords/blob/master/simple.go https://github.com/benhoyt/countwords/blob/master/simple.go Hoping new age compiled languages like Crystal & Nim fill the gap of performance, types & productivity. But compile speeds need to be factored in.
- rco8786 5y ago
  > Ruby is just ruthlessly productive. This is the best description of Ruby I've ever read.
  - princevegeta89 5y ago
    +1. After working on Elixir for a long time I missed the simple ruby way to do things and found myself running in circles to troubleshoot some basic 3rd party libraries. Going back to Ruby for my next side project.
    - AlchemistCamp 5y ago
      Strange. I like Ruby, but probably the biggest gain I got from moving to Elixir was the comparative ease of debugging. Its immutability drastically reduces the search space.
      - princevegeta89 5y ago
        Agree with your point about debugging being easier but in my case I ended up doing a lot of housekeeping for things that are easy in ruby. This time I'm only concerned about the speed of development and ruby to me is the winner

mpweiher 5y ago

The idea that refactoring requires static types is ahistoric and simply incorrect. The first automatic refactoring tool was the Refactoring Browser, in Smalltalk. https://refactory.com/refactoring-browser/ https://refactory.com/refactoring-browser/ https://www.researchgate.net/publication/220346807_A_Refactoring_Tool_for_Smalltalk https://www.researchgate.net/publication/220346807_A_Refacto...

recuter 5y ago
I don't understand why this is the bottom comment instead of the top one.
a1445c8b 5y ago
The PDF[1] that the second link points to gives a very good background on why a refactoring tool should be part of any software engineer's toolkit. Thanks for sharing! [1] https://www.researchgate.net/profile/Don-Roberts-4/publication/220346807_A_Refactoring_Tool_for_Smalltalk/links/5ecd251d92851c9c5e5b931f/A-Refactoring-Tool-for-Smalltalk.pdf https://www.researchgate.net/profile/Don-Roberts-4/publicati...
injidup 5y ago
Because the technique it uses is "clever" and at the same time terrible. It only works if you have 100% code coverage in automatic or monkey testing. The key quote from the paper is. """ as the program is exercised, it converges towards a correctly refactored program. """ This occurs because the refactoring engine uses runtime data to discover the correct types and thus correct refactorings. It is certainly not the same robustness in refactoring that you get with a statically typed language. I don't want my refactoring to "converge". I want it done.
- igouy 5y ago
  The key quote from the paper is — "We do not attempt to automatically refactor code." "A very large Smalltalk application was developed at Cargill to support the operation of grain elevators and the associated commodity trading activities. The Smalltalk client application has 385 windows and over 5,000 classes. About 2,000 classes in this application interacted with an early (circa 1993) data access framework. The framework dynamically performed a mapping of object attributes to data table columns. Analysis showed that although dynamic look up consumed 40% of the client execution time, it was unnecessary. A new data layer interface was developed that required the business class to provide the object attribute to column mapping in an explicitly coded method. Testing showed that this interface was orders of magnitude faster. The issue was how to change the 2,100 business class users of the data layer. A large application under development cannot freeze code while a transformation of an interface is constructed and tested. We had to construct and test the transformations in a parallel branch of the code repository from the main development stream. When the transformation was fully tested, then it was applied to the main code stream in a single operation. Less than 35 bugs were found in the 17,100 changes. All of the bugs were quickly resolved in a three-week period. If the changes were done manually we estimate that it would have taken 8,500 hours, compared with 235 hours to develop the transformation rules. The task was completed in 3% of the expected time by using Rewrite Rules. This is an improvement by a factor of 36." from “Transformation of an application data layer” Will Loew-Blosser OOPSLA 2002 http://portal.acm.org/citation.cfm?id=604258 http://portal.acm.org/citation.cfm?id=604258

alberth 5y ago

NIM I'm still surprised more startups don't use NIM. It has all of the productivity benefits of Python but performance of C.

brigandish 5y ago
Did you come to Nim from a Ruby background? I've been using Crystal and am very happy with it but would be interested to know what the jump to Nim might be like.
deleted 5y ago
[deleted]

bxjx 5y ago

Satisfying static types AND 100% code cov is a significant burden. If you're intrigued by this approach, or perhaps by a similar approach with typescript, I highly recommend checking out a strictly strong typed language at some point. I've been coding in Elm and a little in Haskell, and I find that the type checking is so thorough and exhaustive that I only need to write very few tests to get strong guarantees. This was a very pleasant change from Rails where you are encouraged to aim for 100% code cov, and as such, would spend well over 40% of my development writing tests.

devoutsalsa 5y ago
It’s worth noting that just because it compiles, that doesn’t mean it does what you want.
- runeks 5y ago
  True. Nor does 100% code coverage mean it does what you want.

userbinator 5y ago

I thought this would be about spending quarantine time eating sorbet and refactoring Ruby code.

raggi 5y ago

About a decade ago I had a social media marketing platform (Wildfire) part of which was a special case web page builder. We scaled our engineering team in many different ways, one of the many ways was to also contract external teams. One such team wrote a book based on code the wrote for us, Objects on Rails. The code was fine for a time, but we were increasingly serving traffic from Youtube which was a very popular platform for us at the time. The code in question could not scale in a cost effective manner for traffic coming from Youtube. In fact, it was stressing our ability to maintain a rational number of haproxy POPs - we'd have needed to massively change our operational architecture if the root problem could not be fixed. It took a while to identify, but eventually we found the exhibits abstraction to be the root of a number of significant and leaky performance problems (runtime object extension, blowing all method caches). I refactored out all of the uses of this code, replacing them with simple in-place conditionals in all cases. This dropped somewhere around 15kloc, and modified another 5k or so (I think these may be under-estimates, but I no longer have access to the code, and it's been a decade). We were able to deploy that patch with very few changes. The teams had done a good job of writing extensive end to end tests. It also helped that removing the abstraction resulted in far simpler code paths. I didn't need types to do this safely, the tests were sufficient. This is not to be a downer on typing extensions, but I think their advantages are often over-sold.

voz_ 5y ago
Sorry, I don't buy it. Tests might be sufficient, but they cannot detect a new issue. They can only prove that a well known one is not there. Strict typing and compile time enforcement will guarantee that a whole class of errors and incompatibilities will not occur.
zht 5y ago
your story is about one person working on one project I think likely typing would help at larger organizations where teams often code against other teams' code and/or services with unclear interfaces
- raggi 5y ago
  The organization that owned the code at the time was Google, and there were multiple teams contributing to that code base concurrently with the patch in question.

manmal 5y ago

Having learned some Ruby over the years, and seeing how elegant Elixir is, leads me to the question: Is Elixir‘s type System strong enough to make refactoring worry-free (eg compared to Ruby)?