32 ms·

Stripe is building a Ruby typechecker

thibaut_barrere 8y ago
Not yet open-sourced, but this is definitely planned. You can try it out there though: https://sorbet.run https://sorbet.run By the way, a number of RubyKaigi 2018 talks were dealing with type checking (and the other were about improving performance & memory usage).
- bogdanu 8y ago
  I had the "pleasure" of creating and integrating a parser for a custom DSL into ACE. I've created the parser in ~3 days; it took another 3 to make it work with ACE. If it was up to me, I'd chose monaco editor (as the guys from mozilla did it with their webassembly editor)
  - ptarjan 8y ago
    (author of the demo site here) I just was looking for a quick editor for the demo, and ace seemed to work well for this. Thanks for the pointer to manaco, I'll look at that for next time. Using an editor in the browser won't be the final product. We're planning on integrating into your editor of choice instead.
sandstrom 8y ago
Interesting! Anyone with insight into the ruby core team know if Ruby may add typing (optional or mandatory) in the future?
- lewisinc 8y ago
  Matz has talked about it as an opt-in. I think his opinion is to not force people to adopt it if they won't enjoy it or their productivity will be hampered by it.
- griffinheart 8y ago
  Was at the Kaigi and on the opening keynote Matz talked a bit about it. He doesn't really support adding it since he feels like in 20/30+ years, advances in technology will make writing types "obsolete".
  - quelltext 8y ago
    It's a weird argument. If it's not necessary in 30 years (I doubt it because some of the type inference issues are simply undecidable, and that in much simpler languages already), then the language can just opt to ignore type annotations or merely check their consistency. Having them is great either way for the benefit of documentation. My take is he doesn't really like (or "get") types and feels protective of his language, which is fair. I think Python made an interesting choice here, i.e. provide syntax for types without semantics. The drawback there is potential fragmentation on type system semantics.
paulddraper 8y ago
Interesting choice, considering that Stripe is a massive Scala user. Is it really cheaper to write type checking for one of the most dynamic languages ever, than port to their other language? Seems to be strong evidence against the "stack doesn't really matter" viewpoint.
- zarvox 8y ago
  It's substantially less risky, less invasive, and easier to do in parallel with other ongoing feature development than a port to another language would be. It's much easier to get incremental value for the investment if you're keeping the same language and codebase than if you're trying to replace everything all at once.
- shaneos 8y ago
  Yes it is. Stripe can do this with three (badass) engineers while the hundreds of other engineers continue to build their applications in Ruby. Stripe's Ruby code base is far larger than their Scala codebase, with each being used for the types of uses that play to their strengths. Rewriting millions of lines of Ruby into another language would more or less involve pausing all application development for one to two years, which simply is not feasible. This way, a small team of skilled engineers make this typer work, and guide its general adoption, getting the codebase well typed over the course of years, all the while Stripe continues to grow its product suite and further conquer the world of e-commerce. Twitter did a huge Scala rewrite. There are many lessons to be taken from their lack of product velocity while doing so that the Stripe team took onboard when making this decision.
pg_bot 8y ago
I don't understand the appeal of static typing. It always feels like I'm adding overhead without receiving any benefits. I've worked in some fairly large dynamically typed codebases and have never run into issues with type errors. Static typing does not alleviate the need to test your code, which is a more effective way of reducing bugs in your system than annotating your methods/functions.
- matchbok 8y ago
  Not sure I've ever heard anyone say static typing removes the need for testing. Types are self-documenting. Onboarding new devs in a large static codebase will be 10x easier than a dynamic one. Coding is actually faster in statically typed projects - autocomplete works every single time and you never have to dig into another source file to see what objects/methods you have. And, most modern type systems (Typescript, Swift, etc) don't have much overhead at all with their impressive type inference.
  - pg_bot 8y ago
    I would disagree with the statement that coding is faster in statically typed projects. The ease of onboarding new devs is solely based on the code quality of the project. Well written programs are easy to understand regardless of typing. Terribly written programs are opaque regardless of typing.
    - Tehnix 8y ago
      Sure, any code base can be horrible to learn, but two equally well written code bases, types can help significantly. When your new employee can be confident that changes something in one place, won't invisibly break something in another place, they are more prone to start making changes. Vimal Kumaar touches on how it has helped them get employees contributing to the code a lot quicker, after they moved to PureScript https://www.youtube.com/watch?v=HLEwYghBjo8 https://www.youtube.com/watch?v=HLEwYghBjo8 (I think around 24+ minutes in, but I recommend the whole talk).
    - matchbok 8y ago
masukomi 8y ago
ok this is going to sound snarky but it's an honest question: Why not just use Crystal[1]? It's basically just compiled ruby with static types, and it can infer types correctly in most cases without you explicitly noting which they are. The best argument for this (instead of Crystal) is keeping access to ruby gems. [1]: https://crystal-lang.org/ https://crystal-lang.org/
- CGamesPlay 8y ago
  My 100k LOC code base is already written in ruby though. I have to rewrite everything?
  - masukomi 8y ago
    yeah. that's a pretty compelling reason, but on the other hand, there's a lot of value to be had from modularizing your codebase and microservices so... you could do all future dev in a new language, and if you wanted, gradually convert your existing modules. Sadly, few places think far enough ahead to see the values of modularity (even ignoring language switching) and end up with giant monolithic codebases so language switching just isn't an option without a complete rewrite.
    - mmartinson 8y ago
      While you have some good points, I'm not sure monolith and modularity are mutually exclusive. Some places I'm sure think some places think far ahead, and determine that the cost of microservices isn't worth the later benefit while they're trying to bootstrap a project.
- parvenu74 8y ago
  Is Crystal a superset of Ruby where any valid Ruby project is a valid Crystal project -- just lacking type definition? If Crystal can do that then I think adoption of Crystal would benefit greatly.
  - chrisseaton 8y ago
mslate 8y ago
Is this analogous to mypy?
- ptarjan 8y ago
  It is a similar idea, yes.
iamleppert 8y ago
People created dynamic languages for the exact reason of not having to define types. Now the trend is to bolt on type checking to dynamic languages because people don’t want dynamic behavior. Most developers I’ve encountered who seek to add types to dynamic languages will never truly understand dynamic languages, closures, concurrency passing or functional programming for that matter. If you want a statically typed language, use one. Use one that was designed for deterministic behavior and static correctness. If you have a huge codebase in Ruby that is full of bugs and is unpredictable, don’t blame the language or lack of types. There are plenty of ruby codebases that are bug free and work well without static typing. How do they exist?
- adamnemecek 8y ago
  > There are plenty of ruby codebases that are bug free and work well without static typing. How do they exist? This is a bad question. Refactoring Rails is AFAIK a nightmare. Given enough devs it can probably be done, however it might be painful.
- f00_ 8y ago
  what's wrong with adding type hints in python? It's literally just a linting tool. It's jsut some extra documentation so someone can tell what types a function expects and what types it return. You can run something like mypy or pyre and you can have some static checking done on your code "will never truly understand dynamic languages, closures, concurrency passing or functional programming for that matter." you sound like a pompous cunt honestly
- Tehnix 8y ago
  > Most developers I’ve encountered who seek to add types to dynamic languages will never truly understand dynamic languages, closures, concurrency passing or functional programming for that matter. I don't see how seeking to add types precludes also understanding "closures, concurrency passing or functional programming", especially since these are in no way foreign concepts in statically typed languages. > If you want a statically typed language, use one. Use one that was designed for deterministic behavior and static correctness. I definitely see what you're trying to say, but as I understand it, these projects to "add types" often come on later, when the project is so big, that it is simply less time consuming to just develop a type checker, than it is to rewrite your entire codebase in a new language.
bogdanu 8y ago
I know the editor is not the news, and it's used just for demo, but more info about it: currently it's an ACE editor strapped to a wasm that performs the checking. The integration looks more like a quick poc since it's not using an webworker for performing the checks nor does it use the editor's markers for errors.
taf2 8y ago
This looks really nice... i wish it was just part of the language to be able to write ruby with type information e.g. instead of having a separate more verbose sig method that's called before each method signature... ``` class Foo extend T::Helpers sig( # Both positional and named parameters are referred to by name. # You must declare all parameters (and the return value below). foobar: Integer, widget: String ) .returns(Symbol) def foo(foobar, widget: nil) (foobar.to_s + widget).to_sym end end ``` I'd love it if we could write this as : ``` class Foo extend T::Helpers def foo(foobar:Integer, widget:String:nil):Symbol (foobar.to_s + widget).to_sym end end ``` Not sure the best way to handle default values... e.g. widget:String:nil is kind of awkward... but still i'd be better this way then the additional method call 'sig'
- unfunco 8y ago
  The second version would be difficult, maybe impossible currently due to keyword arguments, since: def foo(foobar:Integer) end is already valid, and I think the lexer might struggle with the ambiguity; but I'm not 100% certain. Edit: I seemingly missed the last line of your comment where you state something similar.
  - tsycho 8y ago
    If foo:SomeType is problematic, let’s just use foo::SomeType instead. I agree with the parent that inline types are much nicer and more intuitive than he sig based approach.
- spitfire 8y ago
  For some reason that reminds me of Pascals interface/implementation split... For those not in the know Pascal units had to declare the function (procedure) prototypes before implementing them.
- ptarjan 8y ago
  (one of the authors here) We'd love that too! We're chatting with the Ruby folks to see if anything like that is feasible. We experimented with lots of other syntaxes (comments, yarddoc, .rbi files, monkey-patching stdlib classes to be callable) and the `sig` syntax seems to be the easiest for folks to use inside Stripe. After writing sigs for the past few months, I don't dislike it as much as I thought I would. You get the syntax highlighting and autocomplete of your editor, while `sig` and `T` are short enough to not feel like too much boilerplate. We're very open to other suggestions if you have any for other syntax suggestions. Just email us at sorbet@stripe.com.
exabrial 8y ago
Is Crystal still a thing? I thought it sought to keep Ruby ease but statically for speed and to reduce bugs?
- anthonybullard 8y ago
  Definitely still a thing, but development looks like it's slowed a bit. Maybe the Manas team is overwhelmed? 550 open issues and 130 open PRs. Nice language though, hope things improve!
- everdev 8y ago
  Yes, I've enjoyed using it and there are several well written frameworks like Kemal, Amber and Lucky for apps. And it's really useful for CLIs and APIs.
- qop 8y ago
  Crystal is progressing well, but they don't have the resources of a lab full of incredible engineers thrown at it. It's definitely a threat though. The Amber Framework is very familiar to people who know Phoenix or Rails. Personally, coming from Ruby, Crystal has a slightly steeper learning curve. However, I feel that magic spark deep down now that I've spent more time with it and I'm more aware of what inference can do and how to use it for my benefit. Little safety things that seem irritating at first, but then you put the pieces together and understand them a bit more. T | Nil is a culprit here. It seems obnoxious, but then you realize, I can just chain methods in a case statement, I don't have to check for nil over and over, just in the topmost part of whatever code it is. It's not how I normally think, and my instinct was wrong based on my initial understanding, but now I'm much more comfortable with it. As for tooling, Crystal is rather weak. This too will come with time.
- vinceguidry 8y ago
  Crystal is a completely different language with a different architecture and object model that only superficially looks like Ruby. Porting code from Ruby to Crystal is somewhat easier than, say, porting the same code to Python, but is still an act of porting. You can't use any kind of automated drop-in process to rewrite Ruby code to Crystal. They're just too different. A good type system for Ruby is very much needed. Matz himself has stated that a type system is very much on the horizon for Ruby. Can't wait.
lettergram 8y ago
I have to say, after using Stripe for a few years - I'm amazed at how well it works. In fact, I can even create a "refer a friend" system on a rails app in some 15 lines of code[1]. I'm excited if they can bring the same simplicity to their additions to languages and frameworks as well. [1] https://blog.projectpiglet.com/2018/03/refer-friend-using-stripe-rails-15-lines-code/ https://blog.projectpiglet.com/2018/03/refer-friend-using-st...
Kalium 8y ago
"Why don't they just use (insert static-typed system here) instead?" someone will inevitably ask. "Because converting a large, complex set of living codebases from one language to another is a non-trivial task that requires boiling any number of oceans. This approach doesn't have that problem." comes the wiser, if less interesting, response.
- sanderjd 8y ago
  This seems like boiling oceans as well. It's hard to guess which oceans are bigger or harder to boil. Clearly Stripe thinks this is the easier way to go. That wouldn't have been my instinct. Maybe they made a really good estimate for how it will be though. I'd be very curious to see their full analysis!
  - Kalium 8y ago
    You're absolutely right! Building probably was a lot like boiling an ocean. Grafting static typing onto a very dynamic language is a massive and complex undertaking. That said, it's one item that can be built greenfield. In some senses, that makes it easier to apply to other codebases once the tool is in a working state. It also means one thing to work on, rather than N, which is much easier to plan for. You're completely right that this is an ocean to boil, and it's rarely easy to guess the size of any given one. Yet it's possible that the naive intuition - when boiling oceans, prefer fewer - might be reliable in this case.
    - vidarh 8y ago
      Grafting a complete and safe static typing system onto a very dynamic language is a massive and complex one. But writing an advisory type checker (that just warns) with a specific code base as a starting point is a very different thing. Biggest question to me is if they support duck-typing. Secondly, if I was to do type checking for Ruby I'd start with parsing yard-style annotations, as a lot of libraries etc. come with that as part of generating their documentation (and if not, writing them means you're now writing comments a lot of tools can generate documentation from.
qop 8y ago
I think the most fascinating thing that Ruby 3 could approach is an approach similar to Dart 1's optional types. That way a fast unchecked mode could be available for developing, and REPL usage, and then a separate switch could be flicked for building a release. Seeing the Ruby devs take compatibility so seriously is such a confidence builder for me, either way. Ruby is such a joy to program in, and for certain projects, Crystal fits the "square enough to go in a square-ish hole" role, but I really miss some of the ease-of-use that Ruby has to offer. require 'Something' Something.new.methods.sort I use that every single day. Crystal can't do that. Yet. Little things like that make Ruby such a human-friendly language. Now that we've learned our collective lesson that naive dynamic types are too slow for big projects, I am glad to see Ruby-thinkers begin to explore the type system space and search for something that will benefit all Ruby hackers. I suspect that if we arrive at the point where Ruby has a gradual or optional type system available, that refactoring some of the old Ruby tools will be very pleasant. If or when that day comes, I know I'll devote a chunk of my own time to try and catch the tools back up, and the rest will follow if only there are hackers who want to hack badly enough.
- Willamin 8y ago
  A lot of people that love Ruby and dislike Crystal seem to dislike Crystal for reasons like what you mentioned. I'm pretty sure that you can get a sorted list of methods in Crystal (not with the same syntax unfortunately) at compile time in macros. I'm not sure why you would ever want to dynamically get a sorted list of methods at runtime... unless you're adding methods at runtime.
  - qop 8y ago
    I use it in the repl for classes I'm not familiar with. I use Ruby for duct taping a lot.
sankha93 8y ago
Any reason why existing typecheckers for Ruby didn't make the cut? There are projects like RDL [1] which provide similar functionality. I know for certain that people from Stripe were taking a look at it last year from the issues they had raised. Disclosure: I am a grad student, recently started working on RDL. [1]: https://github.com/plum-umd/rdl https://github.com/plum-umd/rdl
- hendzen 8y ago
  I'm sure they will come up with some plausible reason like the project having incompatible goals, or embracing a different design architecture than the one they wanted. Truthfully, these 'prestige' projects are done for reasons other than actually improving the development standards. They probably didn't use RDL because it was more fun to write their own then figure out someone else's thing. For all we know they may not have even had a real need for a new typechecker. Stripe could have a valued engineer who seemed bored so engineering management gave him or her an intellectualy interesting project to do - even if it wasn't that important to the business.
  - exolymph 8y ago
    Sure, that's possible, but we don't really have enough information to say either way.
    - ksec 8y ago
      I think they mentioned they will present more information in an up coming conference. And it makes this TypeChecker possibly the most battle tested solution out of all the competing ones.
  - mattbessey 8y ago
    It would seem your cynicism was unfounded https://news.ycombinator.com/item?id=17226332 https://news.ycombinator.com/item?id=17226332
- chrisseaton 8y ago
  What's the next step for RDL? It seems like Hummingbird is much more powerful than Sorbet, but it's hard to tell without them publishing a paper yet.
ninjakeyboard 8y ago
Is the idea that you would take your existing code and add typesafety to it? A statically typed language is faster because it does type checking at compile time and does not need to do "if string, ok else throw exception", so adding a bunch of type checking to a dynamic language adds the development overhead without the benefit. I suppose there is a false economy in that statement as defects caught early save time vs late defect discovery, which in my experience, is very common due to type issues in dynamically typed languages. I personally prefer working with statically typed languages but I've been working with elixir for the last 8 months or so and do not feel very compelled by the use of dialyze (type checker) in the dynamically typed language - you get used to the idioms and try to keep the code type-y but it seems unidiomatic to take type safety too far in a dynamically typed language when you don't get the performance benefits. You cover the things you're likely to get wrong but embrace it for the sake of development speed otherwise. For a new project the right question to ask is "why would we use this instead of a statically typed language." I suppose for an existing project the question this is trying to solve is "how can we reduce bugs now that we've made it to market."
ptarjan 8y ago
Thank you so much for all the interest! We're flattered and excited to see all the discussion about the project. Try it out: https://sorbet.run https://sorbet.run If you would like to get in touch with us about anything, please email us at sorbet@stripe.com. In the presentation at RubyKaigi (which will be available online soon) we explicitly mentioned we'd like to chat with folks trying to scale Ruby into the millions of lines of code, or folks also working on similar typechecking projects. Of course feel free to email even if you aren't in those groups, but I wanted to get your attention if you were.
- vidarh 8y ago
  One thing that springs to mind: Does it handle duck-typing? In other words, can I specify "any object that responds to methods x and y" as the expected type of a variable? Because if treats classes as equivalent to types, then to me it encourages non-idiomatic Ruby.
  - inopinatus 8y ago
    We only have the published examples to go on, but unfortunately they all seem to assume that type means class. If that’s borne out I could never use this library. It’d be the antithesis of duck typing and object messaging.
vemv 8y ago
Typical Rails codebases already enjoy a reasonable layer of 'typing' via ActiveRecord coercions, validations, and possibly some system that declares/documents the types/structure of your REST API. Also, I advocate keyword arguments (especially 'required' ones), which tend to kill 80% of the use case of typing for dynlangs ("what's the type of argument x?") A meaningful test suite would hammer the last nail in the coffin.
mbesto 8y ago
So you know when people ask "why does X startup have X,000 developers? what do they do all day for a single app[0]. Here is your answer: "Technical details: - 9 month of work by 3 people; - real thing, runs over all code of #Stripe" https://twitter.com/darkdimius/status/1002103748875902978 https://twitter.com/darkdimius/status/1002103748875902978 Now let's look at this from a financial perspective: So let's say average $175k salary per employee + benefits and you're looking at easily north of $600k to do this. I'm not saying it's "right" or "wrong", but thats just what companies at this scale do because they create development cultures and subsequent financial controls that allow for indirect production software development to happen. [0]for the record, Stripe is much more complicated than just a "single app"
- draw_down 8y ago
  Plus, since builds take forever, projects like this give you something interesting to read about while you wait 30-40 min for your code to make it into master. (Yes, seriously)
emodendroket 8y ago
Having done some legacy Ruby work I'll say this is sorely needed.
lobo_tuerto 8y ago
So, Sorbet is to Ruby what Dialyzer/Dialyxir is for Erlang/Elixir right? Are they using basically the same approach?
olingern 8y ago
I was at RubyKaigi. This presentation was one of the best delivered, so kudos to the Stripe team for all the work put into it. A question I didn't get to ask while there is -- will there be some sort of type sharing system introduced? I've been a user of both Typescript and Flow (both great tools), but saw Typescript's popularity soar because of community written types.
- darkdimius 8y ago
  > A question I didn't get to ask while there is -- will there be some sort of type sharing system introduced? Yes. This was one of goals of our talks. We wanted implementors of type systems for ruby to start collaborating, in particular on a repository of typed shims. The details of how this would work out might be different. Typescript and Flow change syntax and thus they cannot be included inline in arbitrary JavaScript libraries. We're intentionally compatible with Ruby syntax. Thus we would like this "repository of types for libraries" to only contain types intermittently, until types have been accepted to upstream of respective library.
  - olingern 8y ago
    > We wanted implementors of type systems for ruby to start collaborating, in particular on a repository of typed shims. I see. Do you see this being a mono-repo of types that's under a Sorbet org or something Sorbet looks for in a gem if provided? Opening issues for a project's types with Typescript has been somewhat painful since many live in one repo [1]. 1 - https://github.com/DefinitelyTyped/DefinitelyTyped https://github.com/DefinitelyTyped/DefinitelyTyped
cpeterso 8y ago
This Ruby typechecker is a similar approach to Facebook's "Hack" language, incrementally upgrading their legacy (PHP) codebase to a new statically-typed language instead of rewriting everything in a new language. https://code.facebook.com/posts/264544830379293/hack-a-new-programming-language-for-hhvm/ https://code.facebook.com/posts/264544830379293/hack-a-new-p...
samratjp 8y ago
Putting this out here in case if anyone finds it useful - I like a lot of good ideas from Contracts.rb such as the type signature looking more expressive (personal taste - looks Haskell/Elm ish) and support of Maybe's. Granted it's not static but def some good ideas to steal from. https://egonschiele.github.io/contracts.ruby/ https://egonschiele.github.io/contracts.ruby/
leshow 8y ago
There's so much unanswered here, does it handle parametric polymorphism? Does it allow sum/product types? Is there inference? Can you bound generic types? IMO it would be a complete waste of time to add a type system that only prevented the most trivial of type errors: mismatching Integer and String.
seanwilson 8y ago
Is there any discussion on why Stripe chose to write so much code in Ruby? I would have thought for financial software where subtle bugs can have a huge impact you would want to opt for a language with static checks. Or maybe only a small part of the codebase requires this?

Not yet open-sourced, but this is definitely planned. You can try it out there though: https://sorbet.run https://sorbet.run By the way, a number of RubyKaigi 2018 talks were dealing with type checking (and the other were about improving performance & memory usage).

bogdanu 8y ago
I had the "pleasure" of creating and integrating a parser for a custom DSL into ACE. I've created the parser in ~3 days; it took another 3 to make it work with ACE. If it was up to me, I'd chose monaco editor (as the guys from mozilla did it with their webassembly editor)
- ptarjan 8y ago
  (author of the demo site here) I just was looking for a quick editor for the demo, and ace seemed to work well for this. Thanks for the pointer to manaco, I'll look at that for next time. Using an editor in the browser won't be the final product. We're planning on integrating into your editor of choice instead.

sandstrom 8y ago

Interesting! Anyone with insight into the ruby core team know if Ruby may add typing (optional or mandatory) in the future?

lewisinc 8y ago
Matz has talked about it as an opt-in. I think his opinion is to not force people to adopt it if they won't enjoy it or their productivity will be hampered by it.
griffinheart 8y ago
Was at the Kaigi and on the opening keynote Matz talked a bit about it. He doesn't really support adding it since he feels like in 20/30+ years, advances in technology will make writing types "obsolete".
- quelltext 8y ago
  It's a weird argument. If it's not necessary in 30 years (I doubt it because some of the type inference issues are simply undecidable, and that in much simpler languages already), then the language can just opt to ignore type annotations or merely check their consistency. Having them is great either way for the benefit of documentation. My take is he doesn't really like (or "get") types and feels protective of his language, which is fair. I think Python made an interesting choice here, i.e. provide syntax for types without semantics. The drawback there is potential fragmentation on type system semantics.

paulddraper 8y ago

Interesting choice, considering that Stripe is a massive Scala user. Is it really cheaper to write type checking for one of the most dynamic languages ever, than port to their other language? Seems to be strong evidence against the "stack doesn't really matter" viewpoint.

zarvox 8y ago
It's substantially less risky, less invasive, and easier to do in parallel with other ongoing feature development than a port to another language would be. It's much easier to get incremental value for the investment if you're keeping the same language and codebase than if you're trying to replace everything all at once.
shaneos 8y ago
Yes it is. Stripe can do this with three (badass) engineers while the hundreds of other engineers continue to build their applications in Ruby. Stripe's Ruby code base is far larger than their Scala codebase, with each being used for the types of uses that play to their strengths. Rewriting millions of lines of Ruby into another language would more or less involve pausing all application development for one to two years, which simply is not feasible. This way, a small team of skilled engineers make this typer work, and guide its general adoption, getting the codebase well typed over the course of years, all the while Stripe continues to grow its product suite and further conquer the world of e-commerce. Twitter did a huge Scala rewrite. There are many lessons to be taken from their lack of product velocity while doing so that the Stripe team took onboard when making this decision.

pg_bot 8y ago

I don't understand the appeal of static typing. It always feels like I'm adding overhead without receiving any benefits. I've worked in some fairly large dynamically typed codebases and have never run into issues with type errors. Static typing does not alleviate the need to test your code, which is a more effective way of reducing bugs in your system than annotating your methods/functions.

matchbok 8y ago
Not sure I've ever heard anyone say static typing removes the need for testing. Types are self-documenting. Onboarding new devs in a large static codebase will be 10x easier than a dynamic one. Coding is actually faster in statically typed projects - autocomplete works every single time and you never have to dig into another source file to see what objects/methods you have. And, most modern type systems (Typescript, Swift, etc) don't have much overhead at all with their impressive type inference.
- pg_bot 8y ago
  I would disagree with the statement that coding is faster in statically typed projects. The ease of onboarding new devs is solely based on the code quality of the project. Well written programs are easy to understand regardless of typing. Terribly written programs are opaque regardless of typing.
  - Tehnix 8y ago
    Sure, any code base can be horrible to learn, but two equally well written code bases, types can help significantly. When your new employee can be confident that changes something in one place, won't invisibly break something in another place, they are more prone to start making changes. Vimal Kumaar touches on how it has helped them get employees contributing to the code a lot quicker, after they moved to PureScript https://www.youtube.com/watch?v=HLEwYghBjo8 https://www.youtube.com/watch?v=HLEwYghBjo8 (I think around 24+ minutes in, but I recommend the whole talk).
  - matchbok 8y ago

masukomi 8y ago

ok this is going to sound snarky but it's an honest question: Why not just use Crystal[1]? It's basically just compiled ruby with static types, and it can infer types correctly in most cases without you explicitly noting which they are. The best argument for this (instead of Crystal) is keeping access to ruby gems. [1]: https://crystal-lang.org/ https://crystal-lang.org/

CGamesPlay 8y ago
My 100k LOC code base is already written in ruby though. I have to rewrite everything?
- masukomi 8y ago
  yeah. that's a pretty compelling reason, but on the other hand, there's a lot of value to be had from modularizing your codebase and microservices so... you could do all future dev in a new language, and if you wanted, gradually convert your existing modules. Sadly, few places think far enough ahead to see the values of modularity (even ignoring language switching) and end up with giant monolithic codebases so language switching just isn't an option without a complete rewrite.
  - mmartinson 8y ago
    While you have some good points, I'm not sure monolith and modularity are mutually exclusive. Some places I'm sure think some places think far ahead, and determine that the cost of microservices isn't worth the later benefit while they're trying to bootstrap a project.
parvenu74 8y ago
Is Crystal a superset of Ruby where any valid Ruby project is a valid Crystal project -- just lacking type definition? If Crystal can do that then I think adoption of Crystal would benefit greatly.
- chrisseaton 8y ago

mslate 8y ago

Is this analogous to mypy?

ptarjan 8y ago
It is a similar idea, yes.

iamleppert 8y ago

People created dynamic languages for the exact reason of not having to define types. Now the trend is to bolt on type checking to dynamic languages because people don’t want dynamic behavior. Most developers I’ve encountered who seek to add types to dynamic languages will never truly understand dynamic languages, closures, concurrency passing or functional programming for that matter. If you want a statically typed language, use one. Use one that was designed for deterministic behavior and static correctness. If you have a huge codebase in Ruby that is full of bugs and is unpredictable, don’t blame the language or lack of types. There are plenty of ruby codebases that are bug free and work well without static typing. How do they exist?

adamnemecek 8y ago
> There are plenty of ruby codebases that are bug free and work well without static typing. How do they exist? This is a bad question. Refactoring Rails is AFAIK a nightmare. Given enough devs it can probably be done, however it might be painful.
f00_ 8y ago
what's wrong with adding type hints in python? It's literally just a linting tool. It's jsut some extra documentation so someone can tell what types a function expects and what types it return. You can run something like mypy or pyre and you can have some static checking done on your code "will never truly understand dynamic languages, closures, concurrency passing or functional programming for that matter." you sound like a pompous cunt honestly
Tehnix 8y ago
> Most developers I’ve encountered who seek to add types to dynamic languages will never truly understand dynamic languages, closures, concurrency passing or functional programming for that matter. I don't see how seeking to add types precludes also understanding "closures, concurrency passing or functional programming", especially since these are in no way foreign concepts in statically typed languages. > If you want a statically typed language, use one. Use one that was designed for deterministic behavior and static correctness. I definitely see what you're trying to say, but as I understand it, these projects to "add types" often come on later, when the project is so big, that it is simply less time consuming to just develop a type checker, than it is to rewrite your entire codebase in a new language.

bogdanu 8y ago

I know the editor is not the news, and it's used just for demo, but more info about it: currently it's an ACE editor strapped to a wasm that performs the checking. The integration looks more like a quick poc since it's not using an webworker for performing the checks nor does it use the editor's markers for errors.

taf2 8y ago

This looks really nice... i wish it was just part of the language to be able to write ruby with type information e.g. instead of having a separate more verbose sig method that's called before each method signature... ``` class Foo extend T::Helpers sig( # Both positional and named parameters are referred to by name. # You must declare all parameters (and the return value below). foobar: Integer, widget: String ) .returns(Symbol) def foo(foobar, widget: nil) (foobar.to_s + widget).to_sym end end ``` I'd love it if we could write this as : ``` class Foo extend T::Helpers def foo(foobar:Integer, widget:String:nil):Symbol (foobar.to_s + widget).to_sym end end ``` Not sure the best way to handle default values... e.g. widget:String:nil is kind of awkward... but still i'd be better this way then the additional method call 'sig'

unfunco 8y ago
The second version would be difficult, maybe impossible currently due to keyword arguments, since: def foo(foobar:Integer) end is already valid, and I think the lexer might struggle with the ambiguity; but I'm not 100% certain. Edit: I seemingly missed the last line of your comment where you state something similar.
- tsycho 8y ago
  If foo:SomeType is problematic, let’s just use foo::SomeType instead. I agree with the parent that inline types are much nicer and more intuitive than he sig based approach.
spitfire 8y ago
For some reason that reminds me of Pascals interface/implementation split... For those not in the know Pascal units had to declare the function (procedure) prototypes before implementing them.
ptarjan 8y ago
(one of the authors here) We'd love that too! We're chatting with the Ruby folks to see if anything like that is feasible. We experimented with lots of other syntaxes (comments, yarddoc, .rbi files, monkey-patching stdlib classes to be callable) and the `sig` syntax seems to be the easiest for folks to use inside Stripe. After writing sigs for the past few months, I don't dislike it as much as I thought I would. You get the syntax highlighting and autocomplete of your editor, while `sig` and `T` are short enough to not feel like too much boilerplate. We're very open to other suggestions if you have any for other syntax suggestions. Just email us at sorbet@stripe.com.

exabrial 8y ago

Is Crystal still a thing? I thought it sought to keep Ruby ease but statically for speed and to reduce bugs?

anthonybullard 8y ago
Definitely still a thing, but development looks like it's slowed a bit. Maybe the Manas team is overwhelmed? 550 open issues and 130 open PRs. Nice language though, hope things improve!
everdev 8y ago
Yes, I've enjoyed using it and there are several well written frameworks like Kemal, Amber and Lucky for apps. And it's really useful for CLIs and APIs.
qop 8y ago
Crystal is progressing well, but they don't have the resources of a lab full of incredible engineers thrown at it. It's definitely a threat though. The Amber Framework is very familiar to people who know Phoenix or Rails. Personally, coming from Ruby, Crystal has a slightly steeper learning curve. However, I feel that magic spark deep down now that I've spent more time with it and I'm more aware of what inference can do and how to use it for my benefit. Little safety things that seem irritating at first, but then you put the pieces together and understand them a bit more. T | Nil is a culprit here. It seems obnoxious, but then you realize, I can just chain methods in a case statement, I don't have to check for nil over and over, just in the topmost part of whatever code it is. It's not how I normally think, and my instinct was wrong based on my initial understanding, but now I'm much more comfortable with it. As for tooling, Crystal is rather weak. This too will come with time.
vinceguidry 8y ago
Crystal is a completely different language with a different architecture and object model that only superficially looks like Ruby. Porting code from Ruby to Crystal is somewhat easier than, say, porting the same code to Python, but is still an act of porting. You can't use any kind of automated drop-in process to rewrite Ruby code to Crystal. They're just too different. A good type system for Ruby is very much needed. Matz himself has stated that a type system is very much on the horizon for Ruby. Can't wait.

lettergram 8y ago

I have to say, after using Stripe for a few years - I'm amazed at how well it works. In fact, I can even create a "refer a friend" system on a rails app in some 15 lines of code[1]. I'm excited if they can bring the same simplicity to their additions to languages and frameworks as well. [1] https://blog.projectpiglet.com/2018/03/refer-friend-using-stripe-rails-15-lines-code/ https://blog.projectpiglet.com/2018/03/refer-friend-using-st...

Kalium 8y ago

"Why don't they just use (insert static-typed system here) instead?" someone will inevitably ask. "Because converting a large, complex set of living codebases from one language to another is a non-trivial task that requires boiling any number of oceans. This approach doesn't have that problem." comes the wiser, if less interesting, response.

sanderjd 8y ago
This seems like boiling oceans as well. It's hard to guess which oceans are bigger or harder to boil. Clearly Stripe thinks this is the easier way to go. That wouldn't have been my instinct. Maybe they made a really good estimate for how it will be though. I'd be very curious to see their full analysis!
- Kalium 8y ago
  You're absolutely right! Building probably was a lot like boiling an ocean. Grafting static typing onto a very dynamic language is a massive and complex undertaking. That said, it's one item that can be built greenfield. In some senses, that makes it easier to apply to other codebases once the tool is in a working state. It also means one thing to work on, rather than N, which is much easier to plan for. You're completely right that this is an ocean to boil, and it's rarely easy to guess the size of any given one. Yet it's possible that the naive intuition - when boiling oceans, prefer fewer - might be reliable in this case.
  - vidarh 8y ago
    Grafting a complete and safe static typing system onto a very dynamic language is a massive and complex one. But writing an advisory type checker (that just warns) with a specific code base as a starting point is a very different thing. Biggest question to me is if they support duck-typing. Secondly, if I was to do type checking for Ruby I'd start with parsing yard-style annotations, as a lot of libraries etc. come with that as part of generating their documentation (and if not, writing them means you're now writing comments a lot of tools can generate documentation from.

qop 8y ago

I think the most fascinating thing that Ruby 3 could approach is an approach similar to Dart 1's optional types. That way a fast unchecked mode could be available for developing, and REPL usage, and then a separate switch could be flicked for building a release. Seeing the Ruby devs take compatibility so seriously is such a confidence builder for me, either way. Ruby is such a joy to program in, and for certain projects, Crystal fits the "square enough to go in a square-ish hole" role, but I really miss some of the ease-of-use that Ruby has to offer. require 'Something' Something.new.methods.sort I use that every single day. Crystal can't do that. Yet. Little things like that make Ruby such a human-friendly language. Now that we've learned our collective lesson that naive dynamic types are too slow for big projects, I am glad to see Ruby-thinkers begin to explore the type system space and search for something that will benefit all Ruby hackers. I suspect that if we arrive at the point where Ruby has a gradual or optional type system available, that refactoring some of the old Ruby tools will be very pleasant. If or when that day comes, I know I'll devote a chunk of my own time to try and catch the tools back up, and the rest will follow if only there are hackers who want to hack badly enough.

Willamin 8y ago
A lot of people that love Ruby and dislike Crystal seem to dislike Crystal for reasons like what you mentioned. I'm pretty sure that you can get a sorted list of methods in Crystal (not with the same syntax unfortunately) at compile time in macros. I'm not sure why you would ever want to dynamically get a sorted list of methods at runtime... unless you're adding methods at runtime.
- qop 8y ago
  I use it in the repl for classes I'm not familiar with. I use Ruby for duct taping a lot.

sankha93 8y ago

Any reason why existing typecheckers for Ruby didn't make the cut? There are projects like RDL [1] which provide similar functionality. I know for certain that people from Stripe were taking a look at it last year from the issues they had raised. Disclosure: I am a grad student, recently started working on RDL. [1]: https://github.com/plum-umd/rdl https://github.com/plum-umd/rdl

hendzen 8y ago
I'm sure they will come up with some plausible reason like the project having incompatible goals, or embracing a different design architecture than the one they wanted. Truthfully, these 'prestige' projects are done for reasons other than actually improving the development standards. They probably didn't use RDL because it was more fun to write their own then figure out someone else's thing. For all we know they may not have even had a real need for a new typechecker. Stripe could have a valued engineer who seemed bored so engineering management gave him or her an intellectualy interesting project to do - even if it wasn't that important to the business.
- exolymph 8y ago
  Sure, that's possible, but we don't really have enough information to say either way.
  - ksec 8y ago
    I think they mentioned they will present more information in an up coming conference. And it makes this TypeChecker possibly the most battle tested solution out of all the competing ones.
- mattbessey 8y ago
  It would seem your cynicism was unfounded https://news.ycombinator.com/item?id=17226332 https://news.ycombinator.com/item?id=17226332
chrisseaton 8y ago
What's the next step for RDL? It seems like Hummingbird is much more powerful than Sorbet, but it's hard to tell without them publishing a paper yet.

ninjakeyboard 8y ago

Is the idea that you would take your existing code and add typesafety to it? A statically typed language is faster because it does type checking at compile time and does not need to do "if string, ok else throw exception", so adding a bunch of type checking to a dynamic language adds the development overhead without the benefit. I suppose there is a false economy in that statement as defects caught early save time vs late defect discovery, which in my experience, is very common due to type issues in dynamically typed languages. I personally prefer working with statically typed languages but I've been working with elixir for the last 8 months or so and do not feel very compelled by the use of dialyze (type checker) in the dynamically typed language - you get used to the idioms and try to keep the code type-y but it seems unidiomatic to take type safety too far in a dynamically typed language when you don't get the performance benefits. You cover the things you're likely to get wrong but embrace it for the sake of development speed otherwise. For a new project the right question to ask is "why would we use this instead of a statically typed language." I suppose for an existing project the question this is trying to solve is "how can we reduce bugs now that we've made it to market."

ptarjan 8y ago

Thank you so much for all the interest! We're flattered and excited to see all the discussion about the project. Try it out: https://sorbet.run https://sorbet.run If you would like to get in touch with us about anything, please email us at sorbet@stripe.com. In the presentation at RubyKaigi (which will be available online soon) we explicitly mentioned we'd like to chat with folks trying to scale Ruby into the millions of lines of code, or folks also working on similar typechecking projects. Of course feel free to email even if you aren't in those groups, but I wanted to get your attention if you were.

vidarh 8y ago
One thing that springs to mind: Does it handle duck-typing? In other words, can I specify "any object that responds to methods x and y" as the expected type of a variable? Because if treats classes as equivalent to types, then to me it encourages non-idiomatic Ruby.
- inopinatus 8y ago
  We only have the published examples to go on, but unfortunately they all seem to assume that type means class. If that’s borne out I could never use this library. It’d be the antithesis of duck typing and object messaging.

vemv 8y ago

Typical Rails codebases already enjoy a reasonable layer of 'typing' via ActiveRecord coercions, validations, and possibly some system that declares/documents the types/structure of your REST API. Also, I advocate keyword arguments (especially 'required' ones), which tend to kill 80% of the use case of typing for dynlangs ("what's the type of argument x?") A meaningful test suite would hammer the last nail in the coffin.

mbesto 8y ago

So you know when people ask "why does X startup have X,000 developers? what do they do all day for a single app[0]. Here is your answer: "Technical details: - 9 month of work by 3 people; - real thing, runs over all code of #Stripe" https://twitter.com/darkdimius/status/1002103748875902978 https://twitter.com/darkdimius/status/1002103748875902978 Now let's look at this from a financial perspective: So let's say average $175k salary per employee + benefits and you're looking at easily north of $600k to do this. I'm not saying it's "right" or "wrong", but thats just what companies at this scale do because they create development cultures and subsequent financial controls that allow for indirect production software development to happen. [0]for the record, Stripe is much more complicated than just a "single app"

draw_down 8y ago
Plus, since builds take forever, projects like this give you something interesting to read about while you wait 30-40 min for your code to make it into master. (Yes, seriously)

emodendroket 8y ago

Having done some legacy Ruby work I'll say this is sorely needed.

lobo_tuerto 8y ago

So, Sorbet is to Ruby what Dialyzer/Dialyxir is for Erlang/Elixir right? Are they using basically the same approach?

olingern 8y ago

I was at RubyKaigi. This presentation was one of the best delivered, so kudos to the Stripe team for all the work put into it. A question I didn't get to ask while there is -- will there be some sort of type sharing system introduced? I've been a user of both Typescript and Flow (both great tools), but saw Typescript's popularity soar because of community written types.

darkdimius 8y ago
> A question I didn't get to ask while there is -- will there be some sort of type sharing system introduced? Yes. This was one of goals of our talks. We wanted implementors of type systems for ruby to start collaborating, in particular on a repository of typed shims. The details of how this would work out might be different. Typescript and Flow change syntax and thus they cannot be included inline in arbitrary JavaScript libraries. We're intentionally compatible with Ruby syntax. Thus we would like this "repository of types for libraries" to only contain types intermittently, until types have been accepted to upstream of respective library.
- olingern 8y ago
  > We wanted implementors of type systems for ruby to start collaborating, in particular on a repository of typed shims. I see. Do you see this being a mono-repo of types that's under a Sorbet org or something Sorbet looks for in a gem if provided? Opening issues for a project's types with Typescript has been somewhat painful since many live in one repo [1]. 1 - https://github.com/DefinitelyTyped/DefinitelyTyped https://github.com/DefinitelyTyped/DefinitelyTyped

cpeterso 8y ago

This Ruby typechecker is a similar approach to Facebook's "Hack" language, incrementally upgrading their legacy (PHP) codebase to a new statically-typed language instead of rewriting everything in a new language. https://code.facebook.com/posts/264544830379293/hack-a-new-programming-language-for-hhvm/ https://code.facebook.com/posts/264544830379293/hack-a-new-p...

samratjp 8y ago

Putting this out here in case if anyone finds it useful - I like a lot of good ideas from Contracts.rb such as the type signature looking more expressive (personal taste - looks Haskell/Elm ish) and support of Maybe's. Granted it's not static but def some good ideas to steal from. https://egonschiele.github.io/contracts.ruby/ https://egonschiele.github.io/contracts.ruby/

leshow 8y ago

There's so much unanswered here, does it handle parametric polymorphism? Does it allow sum/product types? Is there inference? Can you bound generic types? IMO it would be a complete waste of time to add a type system that only prevented the most trivial of type errors: mismatching Integer and String.

seanwilson 8y ago

Is there any discussion on why Stripe chose to write so much code in Ruby? I would have thought for financial software where subtle bugs can have a huge impact you would want to opt for a language with static checks. Or maybe only a small part of the codebase requires this?