5 ms·

Hey! I wrote this article. If you have any questions about Sorbet or Stripe, please don’t hesitate to ask!

by jez 5y ago

cmer 5y ago
I added Sorbet to my codebase right after reading the article, but it seems to be expecting that I annotate every single one of my gems. Is this accurate? Is there a way around this?
- jez 5y ago
  Somewhat. They don't all need super specific types for every method they've defined, but Sorbet does at least need to know all the classes, modules, and constants in use in your codebase, whether those come from code you've written or code inside gems. But there's tooling (first-party and third-party) that will either download or generate RBI files defining constants that come from gems. `srb init` is the first party solution, and Shopify's `tapioca` gem is the most popular third-party solution[1]. Unfortunately, because Ruby doesn't have import statements at the top of every file, Sorbet can't just do something like silently treat unknown imports as not having a type (like TypeScript and Flow can do), because then it would never be able to tell between "exists but unknown" vs "typo; does not exist" for constant definitions. This definitely makes the adoption process a little tricker compared to other languages, but it's generally a one-time thing once you've got the tooling set up. Also if you're ever having trouble getting the tooling to work, there's a lot of people chatting about Sorbet daily at https://sorbet.org/slack https://sorbet.org/slack [1] https://github.com/Shopify/tapioca https://github.com/Shopify/tapioca
felipeccastro 5y ago
I tried adding this to a new Rails project with no luck. Is there a sample Rails app with Sorbet fully configured (i.e. most gems typed) available on github for reference?
- jez 5y ago
  Unfortunately I don't know of an example repo, but I do know that most projects (except Stripe) who use Sorbet use it with Rails. There's a #rails channel on Slack—maybe you'd like to try asking there! https://sorbet.org/slack https://sorbet.org/slack
lasvad 5y ago
With Ruby 3 releasing with RBS, for new projects, whats the current advised path? Native RBS or Sorbet? Can they co-exist and if so, is there a point to using both? Sorbet is something I've been interested in using for a couple years and finally got a round to actually trying it out. I tried to use Sorbet with ruby 3.1.1 but unfortunately it didn't "just work" which I think is crucial for mass adoption. I want to give the benefit of the doubt and say its my local env that causing issues with Sorbet but in a fresh `rails new test_app --api` project, I'd expect `srb init` to work without errors... maybe I need to give it another go, curious on your thoughts above tho! :)
- jez 5y ago
  We discovered a bug in `srb init` for Ruby 3.1 recently that a teammate of mine is working on fixing at the moment. It's likely that if you tried again in a few days it'll have been fixed. Sorry about that, totally agree that the out-of-box experience should just work. I wrote up an FAQ about the state of Ruby 3 and RBS here: https://sorbet.org/docs/faq#when-ruby-3-gets-types-what-will-the-migration-plan-look-like https://sorbet.org/docs/faq#when-ruby-3-gets-types-what-will... The tl;dr is that RBI files (not RBS files) will probably always be the preferred way to declare types for third party code (because it will always support exactly the same set of features that Sorbet does). We have some people in the community look into teaching Sorbet to read the RBS format, but the existing parsers for RBS files are written in Ruby and are very slow, and there are some ambiguities in the spec that make writing a third party parser that compiles to native code tricky. You can see an attempt to write a fast RBS parser in C++ here[1], but again given that RBI files do everything we need them to right now and we have other features people are asking us for, we haven't prioritized RBS support incredibly highly. Sorbet works completely fine without RBS files! [1] https://github.com/Shopify/rbs_parser https://github.com/Shopify/rbs_parser
sankha93 5y ago
Hey, nice work with Sorbet! I am one of the grad students who worked on RDL, one of the early research projects related to Ruby type systems. What are the next set of challenges that a tool like Sorbet needs to solve? I see you mentioned meta-programming in the blog post, is that something that is handled well by Sorbet? Sorry if this is already handled, I haven't been up to date with the latest features of Sorbet.
- jez 5y ago
  Thanks for your work on RDL! The post didn't mention it, but Sorbet still owes most of its type definitions for the Ruby standard library to RDL's original annotations. We just borrowed them and changed the syntax. Our general approach to metaprogramming at the moment has been two-fold: - Use ahead-of-time code generation powered either by runtime reflection or ad-hoc static analysis to generate RBI files declaring things that have been metaprogrammed. - Build type system features, errors, and autocorrects that encourage people to structure their code in ways that doesn't require metaprogramming to solve. Metaprogramming is definitely still a sticking point, but the existing solutions work ~okay and the rest of the upside Sorbet provides make it worthwile to power through. Next challenges: - Make it faster. While the post was talking about how fast it is, it wasn't telling the whole truth. Turns out some type checking operations in a 15 million line codebase are still slow, and we're working on making those faster. - Add more IDE features. At the beginning of this year I put a lot of work into making Sorbet's parser more tolerant of syntax errors, which helps things like autocompletion work better. We also want to make more code actions, autocorrects, and refactoring tools, to bring Ruby in line with what you'd expect from other typed languages in the IDE experience - Add more type system features. Shapes and tuples are a huge unimplemented feature still, and people ask about it all the time. There are a handful of other type system features (happy to list them if you're curious) that would also let people write idiomatic Ruby and still have good typing. Lots left to do!
  - mwint 5y ago
    I write a lot of Ruby for work, but I’m not sure what “shapes” are - do you have any good reference where I could start reading?
burlesona 5y ago
`srb init` has had a lot of problems since Ruby 3.x, and while I haven't tried in a few months it looks like there's recent issues that it still doesn't work (https://github.com/sorbet/sorbet/issues/5332 https://github.com/sorbet/sorbet/issues/5332). Is the advice just to use Tapioca instead of `sub init` at this point?
- jez 5y ago
  Hoping to have this specific issue fixed either today (or by the end of the week at the latest). So sorry for the delay in getting around to this!
- jez 5y ago
  PR from my teammate Trevor is was just posted which should hopefully fix the issue you linked: https://github.com/sorbet/sorbet/pull/5533 https://github.com/sorbet/sorbet/pull/5533
chucke 5y ago
When is RBS support coming?
vhodges 5y ago
What is the status of the AOT compiler? Still moving forward? Stuck for lack of resources? Release planned soon? :).
- jez 5y ago
  Unfortunately while we've developed the Sorbet Compiler in the open, we've mostly focused on internal-only use cases. There are other exciting efforts to improve Ruby performance though! You might be curious to read more about YJIT, the JIT compiler that was recently merged into Ruby 3.1—it’s being primarily developed by a team at Shopify where Ruby performance is the highest priority.
  - vhodges 5y ago
    No worries and thanks, I'll check it out!
willlll 5y ago
Overall sorbet has been good and useful, so thank you. But you just have to be careful when you're not using the same version of ruby that stripe uses. I recently got hit by a bug: # typed: strict extend T::Sig sig {params(x: Integer, y: String).void} def run(x:, y:) puts(x + 1, y) end args = { x: 1, y: "Hi" } run args type checks just fine in sorbet, but is an error in ruby 3+. It works though (with errors) in 2.7 and lower which is what Stripe uses from what I gather.
- jez 5y ago
  Note that Shopify is much better about keeping up with regular Ruby updates (Stripe takes a pretty conservative approach there), and they're using it with Ruby 3+ with no problem. You're right that Sorbet does not yet catch these bugs yet, and we'll likely get around to it in the future. It's being tracked in this issue[1]. We haven't quite prioritized this yet partly because it doesn't actually prevent you from using Sorbet in a Ruby 3 codebase, it just won't report all the errors it could (e.g., Sorbet allows using `T.untyped`, which can also cause runtime errors to go unreported). Now that you've jogged my memory, there used to be some weirdness in our internal representation for method calls that made implementing this feature tricky. But we've since refactored some internal data structures and now it's probably a lot easier. Maybe I should look into how much work this would be again ... [1]: https://sorbet.run/talks/StrangeLoop2018/#/28 https://sorbet.run/talks/StrangeLoop2018/#/28
davidatbu 5y ago
I tried using sorbet through it's LSP in neovim. I would get diagnostics not after every change, but after every change that adds/removes a function definition. Effectively, I would have to delete and add back a function definition, or the "end" keyword, to trigger sorbet to type check my code. Is that supposed to be how it works?
e12e 5y ago
Thank you for writing up such a nice overview. I've been busy trying to get tests into various legacy projects I've inherited - definitely interested in tackling types next. Couple of questions: > The declare_method call above acted like a decorator on the def call method: it would check that the msg argument given to call was a String and that call returned a String on every invocation How was this implemented? Dynamically altering Object#send or something along those lines? How is the story for sorbet and vim/nvim? Are there "run time" or "code gen" uses for sorbet? Like generating swagger/openapi documentation/schemas based on typed Api methods? Or vice-versa - scaffolding sorbet-typed Api from a swagger.json? Or something similar for graphql (or, well, SOAP..)?
- jez 5y ago
  > How was this implemented? The mechanism in the post is the same mechanism in use today with Sorbet signatures. To get the `sig` method in scope, you have to put `extend T::Sig` in that class (or one of its parents). When `sig` is called for the first time in a class, it monkey patches that class to install some overrides of the method_added method. Ruby calls this method_added method for you every time it creates a method (from any means, static or dynamic). Code is here: https://github.com/sorbet/sorbet/blob/master/gems/sorbet-runtime/lib/types/private/methods/_methods.rb#L530 https://github.com/sorbet/sorbet/blob/master/gems/sorbet-run... > How is the story for sorbet and vim/nvim? Great, honestly better than VS Code for everything except autocompletion. I personally use Sorbet with Neovim. > Are there "run time" or "code gen" uses for sorbet? Like generating swagger/openapi documentation/schemas based on typed Api methods? Or vice-versa - scaffolding sorbet-typed Api from a swagger.json? Or something similar for graphql (or, well, SOAP..)? Not that I know of unfortunately, but I also pay more attention to the type checker and it’s bugs than the tooling people build around it to get real world work done

Hey! I wrote this article. If you have any questions about Sorbet or Stripe, please don’t hesitate to ask!

by jez 5y ago

Hey! I wrote this article. If you have any questions about Sorbet or Stripe, please don’t hesitate to ask!

cmer 5y ago

I added Sorbet to my codebase right after reading the article, but it seems to be expecting that I annotate every single one of my gems. Is this accurate? Is there a way around this?

jez 5y ago
Somewhat. They don't all need super specific types for every method they've defined, but Sorbet does at least need to know all the classes, modules, and constants in use in your codebase, whether those come from code you've written or code inside gems. But there's tooling (first-party and third-party) that will either download or generate RBI files defining constants that come from gems. `srb init` is the first party solution, and Shopify's `tapioca` gem is the most popular third-party solution[1]. Unfortunately, because Ruby doesn't have import statements at the top of every file, Sorbet can't just do something like silently treat unknown imports as not having a type (like TypeScript and Flow can do), because then it would never be able to tell between "exists but unknown" vs "typo; does not exist" for constant definitions. This definitely makes the adoption process a little tricker compared to other languages, but it's generally a one-time thing once you've got the tooling set up. Also if you're ever having trouble getting the tooling to work, there's a lot of people chatting about Sorbet daily at https://sorbet.org/slack https://sorbet.org/slack [1] https://github.com/Shopify/tapioca https://github.com/Shopify/tapioca

felipeccastro 5y ago

I tried adding this to a new Rails project with no luck. Is there a sample Rails app with Sorbet fully configured (i.e. most gems typed) available on github for reference?

jez 5y ago
Unfortunately I don't know of an example repo, but I do know that most projects (except Stripe) who use Sorbet use it with Rails. There's a #rails channel on Slack—maybe you'd like to try asking there! https://sorbet.org/slack https://sorbet.org/slack

lasvad 5y ago

With Ruby 3 releasing with RBS, for new projects, whats the current advised path? Native RBS or Sorbet? Can they co-exist and if so, is there a point to using both? Sorbet is something I've been interested in using for a couple years and finally got a round to actually trying it out. I tried to use Sorbet with ruby 3.1.1 but unfortunately it didn't "just work" which I think is crucial for mass adoption. I want to give the benefit of the doubt and say its my local env that causing issues with Sorbet but in a fresh `rails new test_app --api` project, I'd expect `srb init` to work without errors... maybe I need to give it another go, curious on your thoughts above tho! :)

jez 5y ago
We discovered a bug in `srb init` for Ruby 3.1 recently that a teammate of mine is working on fixing at the moment. It's likely that if you tried again in a few days it'll have been fixed. Sorry about that, totally agree that the out-of-box experience should just work. I wrote up an FAQ about the state of Ruby 3 and RBS here: https://sorbet.org/docs/faq#when-ruby-3-gets-types-what-will-the-migration-plan-look-like https://sorbet.org/docs/faq#when-ruby-3-gets-types-what-will... The tl;dr is that RBI files (not RBS files) will probably always be the preferred way to declare types for third party code (because it will always support exactly the same set of features that Sorbet does). We have some people in the community look into teaching Sorbet to read the RBS format, but the existing parsers for RBS files are written in Ruby and are very slow, and there are some ambiguities in the spec that make writing a third party parser that compiles to native code tricky. You can see an attempt to write a fast RBS parser in C++ here[1], but again given that RBI files do everything we need them to right now and we have other features people are asking us for, we haven't prioritized RBS support incredibly highly. Sorbet works completely fine without RBS files! [1] https://github.com/Shopify/rbs_parser https://github.com/Shopify/rbs_parser

sankha93 5y ago

Hey, nice work with Sorbet! I am one of the grad students who worked on RDL, one of the early research projects related to Ruby type systems. What are the next set of challenges that a tool like Sorbet needs to solve? I see you mentioned meta-programming in the blog post, is that something that is handled well by Sorbet? Sorry if this is already handled, I haven't been up to date with the latest features of Sorbet.

jez 5y ago
Thanks for your work on RDL! The post didn't mention it, but Sorbet still owes most of its type definitions for the Ruby standard library to RDL's original annotations. We just borrowed them and changed the syntax. Our general approach to metaprogramming at the moment has been two-fold: - Use ahead-of-time code generation powered either by runtime reflection or ad-hoc static analysis to generate RBI files declaring things that have been metaprogrammed. - Build type system features, errors, and autocorrects that encourage people to structure their code in ways that doesn't require metaprogramming to solve. Metaprogramming is definitely still a sticking point, but the existing solutions work ~okay and the rest of the upside Sorbet provides make it worthwile to power through. Next challenges: - Make it faster. While the post was talking about how fast it is, it wasn't telling the whole truth. Turns out some type checking operations in a 15 million line codebase are still slow, and we're working on making those faster. - Add more IDE features. At the beginning of this year I put a lot of work into making Sorbet's parser more tolerant of syntax errors, which helps things like autocompletion work better. We also want to make more code actions, autocorrects, and refactoring tools, to bring Ruby in line with what you'd expect from other typed languages in the IDE experience - Add more type system features. Shapes and tuples are a huge unimplemented feature still, and people ask about it all the time. There are a handful of other type system features (happy to list them if you're curious) that would also let people write idiomatic Ruby and still have good typing. Lots left to do!
- mwint 5y ago
  I write a lot of Ruby for work, but I’m not sure what “shapes” are - do you have any good reference where I could start reading?

burlesona 5y ago

`srb init` has had a lot of problems since Ruby 3.x, and while I haven't tried in a few months it looks like there's recent issues that it still doesn't work (https://github.com/sorbet/sorbet/issues/5332 https://github.com/sorbet/sorbet/issues/5332). Is the advice just to use Tapioca instead of `sub init` at this point?

jez 5y ago
Hoping to have this specific issue fixed either today (or by the end of the week at the latest). So sorry for the delay in getting around to this!
jez 5y ago
PR from my teammate Trevor is was just posted which should hopefully fix the issue you linked: https://github.com/sorbet/sorbet/pull/5533 https://github.com/sorbet/sorbet/pull/5533

chucke 5y ago

When is RBS support coming?

vhodges 5y ago

What is the status of the AOT compiler? Still moving forward? Stuck for lack of resources? Release planned soon? :).

jez 5y ago
Unfortunately while we've developed the Sorbet Compiler in the open, we've mostly focused on internal-only use cases. There are other exciting efforts to improve Ruby performance though! You might be curious to read more about YJIT, the JIT compiler that was recently merged into Ruby 3.1—it’s being primarily developed by a team at Shopify where Ruby performance is the highest priority.
- vhodges 5y ago
  No worries and thanks, I'll check it out!

willlll 5y ago

Overall sorbet has been good and useful, so thank you. But you just have to be careful when you're not using the same version of ruby that stripe uses. I recently got hit by a bug: # typed: strict extend T::Sig sig {params(x: Integer, y: String).void} def run(x:, y:) puts(x + 1, y) end args = { x: 1, y: "Hi" } run args type checks just fine in sorbet, but is an error in ruby 3+. It works though (with errors) in 2.7 and lower which is what Stripe uses from what I gather.

jez 5y ago
Note that Shopify is much better about keeping up with regular Ruby updates (Stripe takes a pretty conservative approach there), and they're using it with Ruby 3+ with no problem. You're right that Sorbet does not yet catch these bugs yet, and we'll likely get around to it in the future. It's being tracked in this issue[1]. We haven't quite prioritized this yet partly because it doesn't actually prevent you from using Sorbet in a Ruby 3 codebase, it just won't report all the errors it could (e.g., Sorbet allows using `T.untyped`, which can also cause runtime errors to go unreported). Now that you've jogged my memory, there used to be some weirdness in our internal representation for method calls that made implementing this feature tricky. But we've since refactored some internal data structures and now it's probably a lot easier. Maybe I should look into how much work this would be again ... [1]: https://sorbet.run/talks/StrangeLoop2018/#/28 https://sorbet.run/talks/StrangeLoop2018/#/28

davidatbu 5y ago

I tried using sorbet through it's LSP in neovim. I would get diagnostics not after every change, but after every change that adds/removes a function definition. Effectively, I would have to delete and add back a function definition, or the "end" keyword, to trigger sorbet to type check my code. Is that supposed to be how it works?

e12e 5y ago

Thank you for writing up such a nice overview. I've been busy trying to get tests into various legacy projects I've inherited - definitely interested in tackling types next. Couple of questions: > The declare_method call above acted like a decorator on the def call method: it would check that the msg argument given to call was a String and that call returned a String on every invocation How was this implemented? Dynamically altering Object#send or something along those lines? How is the story for sorbet and vim/nvim? Are there "run time" or "code gen" uses for sorbet? Like generating swagger/openapi documentation/schemas based on typed Api methods? Or vice-versa - scaffolding sorbet-typed Api from a swagger.json? Or something similar for graphql (or, well, SOAP..)?

jez 5y ago
> How was this implemented? The mechanism in the post is the same mechanism in use today with Sorbet signatures. To get the `sig` method in scope, you have to put `extend T::Sig` in that class (or one of its parents). When `sig` is called for the first time in a class, it monkey patches that class to install some overrides of the method_added method. Ruby calls this method_added method for you every time it creates a method (from any means, static or dynamic). Code is here: https://github.com/sorbet/sorbet/blob/master/gems/sorbet-runtime/lib/types/private/methods/_methods.rb#L530 https://github.com/sorbet/sorbet/blob/master/gems/sorbet-run... > How is the story for sorbet and vim/nvim? Great, honestly better than VS Code for everything except autocompletion. I personally use Sorbet with Neovim. > Are there "run time" or "code gen" uses for sorbet? Like generating swagger/openapi documentation/schemas based on typed Api methods? Or vice-versa - scaffolding sorbet-typed Api from a swagger.json? Or something similar for graphql (or, well, SOAP..)? Not that I know of unfortunately, but I also pay more attention to the type checker and it’s bugs than the tooling people build around it to get real world work done