6 ms·

State Machines in Ruby: An Introduction

141 points by unripe_syntax 4y ago

jack_riminton 4y ago
Nice article. Are state machines used widely in Rails apps?
- reillyse 4y ago
  Yes
- beckingz 4y ago
  Unfortunately
- rubyist5eva 4y ago
  In my experience, state-machines are widely used in all business applications whether they use a "library" for it or not.
jeremycw 4y ago
I've seen a few large Rails codebases that included a state machine library like mentioned in the article. Every single one was worse because of it. It pushes the code down a path where hidden hooks run when magic methods are called. At this point in my career I'm just done with that type of "clever" code. The article starts with examples that just define the state machine by hand. This is a much better approach and scales to larger code much better. You can grep "def start!" and just read what the method does. A state machine DSL is really not providing much value and eventually just gets in the way.
- tomc1985 4y ago
  Funny, I have spent a ton of time around Rails code where I dearly wished we had a state machine, because the alternative was an unorganized cluster of home-grown "state transition" glue without any consistent way of handling it, with all the weird-ass edge cases and split-brain BS that come with it.
  - powersurge360 4y ago
    I've been on the receiving and perpetuating end of state machines and it's painful to have to deal with one that started years before you got involved and easy to think that you can do better. I've got a modest one in a side project and due to it being a side project I touch it once every couple of weeks and already it is making me uncomfortable. And it's dirt simple and I wrote it! There's a take on state machines in a railsconf talk further down in the thread and it seems like an awesome way to reap a lot of the benefits while keeping a grip on the complexity. Sometimes complexity can't be buried safely and you kind of need to frame it instead. I'd argue that state machines bury that complexity but breaking those states into ruby objects w/ their own validation frames and delineates them instead.
  - a4isms 4y ago
    The quip I like best for this is: “Any sufficiently complicated model class contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of a state machine.” Lots of models I've encountered eschew being organized as state machines in favour of having "if salad" strewn throughout their code. That being said, refactoring to a simple state machine and trying to maintain it as such in perpetuity isn't always the correct solution. Sometimes, a hierarchal state machine is needed, and if it is expressed as a simple state machine, it's just as messy. Sometimes, a portion of it needs to be a state machine, and the right thing to do is to delegate some of the methods to a strategy that reflects the current state, but not all of them. Sometimes, the whole thing is just too fat, and a state machine won't save it, the right thing to do is to get very aggressive about refactoring to a composite object. Any time you have a big, messy model, it's very easy to write a blog post espousing a single solution, like this one: http://raganwald.com/2018/02/23/forde.html http://raganwald.com/2018/02/23/forde.html HN discussion: https://news.ycombinator.com/item?id=16468280 https://news.ycombinator.com/item?id=16468280 But the reality is that a big, messy model is always going to be some kind of problem, and unless you can break it down into parts, you're going to have a problem. A state machine is conceptually a way to break a big thing into parts based on its "state," but that's just one approach to factoring. p.s. Another problem is that even if a simple state machine is the right answer, "rolling your own" usually isn't. Grab a well-tested and documented library already. This isn't your passion project, this is industrial programming. Rolling your own is one of the best ways to learn how state machines work. Once you've learned how, reach for a professional tool.
- Justsignedup 4y ago
  Came here to write this. Using explicit state action services, which are well tested and intentionally invoked are the ways to go. They encapsulate the code, and don't create these callback hells when things inevitably get complex.
- Fire-Dragon-DoL 4y ago
  I agree. More generally, I would recommend to avoid any DSL. All DSLs are a downgrade over ruby language and there is a really narrow list of opportunities where they are an upgrade. Everyone knows ruby when working on a Rails application. Everyone knows what a class, an instance is and what methods are. Ruby is extendable, incredibly flexible and it doesn't need "special options" to be extended. DSLs every time attempt to recreate the flexibility of ruby and require studying a new mini language. When I'm in a dsl, I don't know if I can use an instance variable, if I need a method I don't know if I can have one. More often then not, this question is not answered by the documentation and digging in the source code is needed. Most libraries with a DSL violate the first rule of a DSL: a DSL method should delegate to a normal object performing the operation, so that DSL API has also a corresponding normal ruby API. If you do this, your DSL is now extendable and replaceable, it can also be integrated into other libraries. As for state machines, the libraries recommended push toward objects violating the single responsibility principle. A state machine has already a super important responsibility: state-transitions. That's already a lot of work, if it has to trigger side effects, now it does way too much. Some alternatives: pub/sub! let the state machine be observable, whatever needs to do work when a certain state is reached, can do it when the event is published Alternatively as someone recommended below, service objects take care of: transition, persistance of the new state and side effects. If a new action is needed, a new service object can be written. The state machine can be used to validate the transition while the service object takes care of the rest.
knodi123 4y ago
Counterpoint: State machines in Ruby should be used sparingly, and only when necessary, because they start out quick and easy, but almost always grow into things that are big, ugly, and difficult to refactor or remove. Here are somebody's notes from a Railsconf talk that made this point, plus a link to the video: https://gist.github.com/benoittgt/05521e272e13c7b7c9c89eb4a9947d33 https://gist.github.com/benoittgt/05521e272e13c7b7c9c89eb4a9... But the short version is the takeaway quote: > State machines are magnetic : they attract more states, transitions, events, callbacks and they accumulate callback logic where our bugs hide
- powersurge360 4y ago
  Thanks for sharing this. For other people who are hovering over this and unsure if they want to watch a 40 minute video, it's time stamped and the two parts I found super valuable were the 'Anti-Pattern: Custom Methods' and the 'Anti-Pattern: State Machines' sections. The TL;DR there is it's alright to create a controller that manipulates a single field on a single model. If it is enough of a resource to expose it by a custom url (think update password) it is enough of a resource to get its own controller. And going beyond that, to the state machine, if each state (or really, value of a particular field) is important enough to have it's own contextual validation, transition points, etc, then it's important enough to expose each state as a resource with it's own controller. Super cool stuff and I've got some thinking to do about some side projects now.
- systems_glitch 4y ago
  Indeed, a coworker at a previous job was fascinated/fixated with state machine patterns. After spending two weeks trying to force a process we were paired on into a state machine pattern and failing, I rewrote it in an afternoon without the state machine stuff. It started out small and easy, when there were two states. A third state was manageable but to me already looked like it was going the wrong direction. The fourth state (out of IIRC 6) was never finished.
- iasay 4y ago
  Of all the things I've seen in my time writing software I can qualify that some of the most fucked up things involved state machines. The worst of which was a 200 state hand written one in C for an embedded measurement system. There was no documentation, no comments and the states were numbered rather than named. The guy who wrote it originally was "anti RTOS" and had died after riding his motorbike recklessly leaving this shit for everyone to deal with. It took me two weeks to do a trivial change because I had to reverse engineer a ton of it. And the only debugger I had was a couple of LEDs on the board I could wiggle on and off.
kdasme 4y ago
I love this, thank you! After playing around different JS frameworks I'm back to Rails and I've been able to iterate quickly. That gem solves a few pain points.
ianbutler 4y ago
I worked at a business (in ruby) where the entire core logic of the app was a state machine, as an idea the thought was good, a state machine modeled order flows pretty well, but in practice it was very slow and the amount of side effects that had to be considered as the business grew just compounded this. It got to the point where this state machine approach was a real bottleneck and though I left before it was resolved last I heard they had to put real engineering time in to move off of it. Be very careful when deciding to go this route.
- duffyjp 4y ago
  On the flip side, I've built several production apps with AASM or Workflow that made adapting to changing business logic a breeze. If you're given a flow-chart you can translate that 1-1 and when that chart changes it's easy to adjust. It was also really easy to write tests for.
  - SaltyBackendGuy 4y ago
    This has been my experience as well with AWS Step Functions. Obviously your use case is critical to the architecture decision. For us it's made automating workflows a breeze.
  - ianbutler 4y ago
    For sure, this isn't an indictment against state machines, my message was exactly to be careful when making the decision to use them. They fit a lot of workflows and I've also worked in environments where they were not at all a bottlenecks, and regardless of the performance bottleneck it wasn't hard to reason about or adjust which for a long time meant more than performance for that business. I guess I could amend my statement to "be careful about which library or implementation of a state machine you go with"
- krainboltgreene 4y ago
  > but in practice it was very slow How in the world is updating a column from "in cart" to "purchased" to "complete" slow? There are only three things a state machine has to do: 1. Determine if the event can happen 2. Set the record to the new state 3. Call lifecycle hooks None of that should be "very slow".
beckingz 4y ago
State machines are harder to update than regular code as your understanding of the domain changes, which often leads to suffering.
rowan_mcd 4y ago
fortysixdegrees 4y ago
A lot of the comments here are rails oriented and anti state machine. In the context of rails that may make sense, but I do a lot of non rails ruby, and non object oriented Dev style, and state machines are a great way to write solid systems where the allowed state and transitions are explicit and easy to reason with.
- bowsamic 4y ago
  Unfortunately almost everyone treats rails and Ruby as synonymous
- Toutouxc 4y ago
  Would you mind sharing a little example of what you mean by non-OOP Ruby code with state machines?
  - fortysixdegrees 4y ago
    Sure. I am working on the firmware for a camera. Believe it or not it's a bunch of ruby services coordinated via redis pubsub. But at the core is the main system state which is a service in itself which is the main state machine. This pattern is something which I've evolved over the years and it's fantastic (if I do say so myself).
- craigkerstiens 4y ago
  A pretty big +1 to this. My team (at Crunchy Data building Crunchy Bridge[1]) and similar teams at prior Citus Data and Heroku did a lot of Ruby and a lot of state machines. In all cases it was actually less Rails, but that may be because the team really prefers sequel as the ORM to ActiveRecord, though Rails does a bunch that doesn't really apply to such a workflow. We wrote about some of how we used state machines to power our database as a service and the full control plane previously[2]. Another big piece of the approach and stack that helps today is leveraging Sorbet[3] and our approach to test coverage, we probably should blog a little more on some of this but it helps quite a bit. [1] - https://www.crunchydata.com/products/crunchy-bridge https://www.crunchydata.com/products/crunchy-bridge [2] - https://www.citusdata.com/blog/2016/08/12/state-machines-to-run-databases/ https://www.citusdata.com/blog/2016/08/12/state-machines-to-... [3] - https://brandur.org/fragments/large-scale-ruby-refactoring https://brandur.org/fragments/large-scale-ruby-refactoring
Toutouxc 4y ago
I get how these state machine gems can sound like a great idea if what you need is taming some state changes that are just a little too much to handle by hand, but they can also delay the realization that you're doing something wrong. State changes are hard and most involve more than object, there are usually associations that need to be taken care of, validated or maybe also transitioned to a different state, there can be a myriad of conditions and things can go wrong at any moment. And I feel that being able to define state transitions this easily, almost as an afterthought, with a simple DSL, doesn't do all this justice (just like those friendly before/after callbacks can turn into massive spaghetti abominations). The way I do it is that any state transition more complex than checking two attributes and updating a third one is an object. Transitioning a Campaign object from :reservation to :order? That's a new PORO called CampaignOrder, with a #perform! public method or something. All the conditions, assumptions and side effects have their private methods (check_showings!, update_occupancy!). Passed a dirty record into something delicate? That's a raise, I want something clean and valid to work with. Is this something where we expect a dirty record? Go right ahead, we'll save it later in the transition, plus it will be documented right in the constructor. Does it need to raise? It will. Custom error messages on any record? Right here in this private method. Does any model need to know anything about the state transition or own any related code? Nope.
pull_my_finger 4y ago
Seems to be a lot of anti-state machine comments here. Are these people trying to use state machines to model their entire application state? The example used in the article is a very practical finite state machine centered around order processing. It's very unlikely these steps in an order lifecycle are going to change significantly. I suspect a lot of organizations jumped on all-in with state machines when they got a resurgence in popularity with x-state library etc. Trying to model your entire application as a state machine just so you can show off an incomprehensible state chart or something is a recipe for disaster.
- davidkpiano 4y ago
  Creator of XState here - I agree. State machines and statecharts should be used to model parts of your application that need it, and they can potentially communicate with each other. Fitting the entire app in a single state machine is just like fitting all of the app logic in a single function; it's going to be incomprehensible.

141 points by unripe_syntax 4y ago

jack_riminton 4y ago

Nice article. Are state machines used widely in Rails apps?

reillyse 4y ago
Yes
beckingz 4y ago
Unfortunately
rubyist5eva 4y ago
In my experience, state-machines are widely used in all business applications whether they use a "library" for it or not.

jeremycw 4y ago

I've seen a few large Rails codebases that included a state machine library like mentioned in the article. Every single one was worse because of it. It pushes the code down a path where hidden hooks run when magic methods are called. At this point in my career I'm just done with that type of "clever" code. The article starts with examples that just define the state machine by hand. This is a much better approach and scales to larger code much better. You can grep "def start!" and just read what the method does. A state machine DSL is really not providing much value and eventually just gets in the way.

tomc1985 4y ago
Funny, I have spent a ton of time around Rails code where I dearly wished we had a state machine, because the alternative was an unorganized cluster of home-grown "state transition" glue without any consistent way of handling it, with all the weird-ass edge cases and split-brain BS that come with it.
- powersurge360 4y ago
  I've been on the receiving and perpetuating end of state machines and it's painful to have to deal with one that started years before you got involved and easy to think that you can do better. I've got a modest one in a side project and due to it being a side project I touch it once every couple of weeks and already it is making me uncomfortable. And it's dirt simple and I wrote it! There's a take on state machines in a railsconf talk further down in the thread and it seems like an awesome way to reap a lot of the benefits while keeping a grip on the complexity. Sometimes complexity can't be buried safely and you kind of need to frame it instead. I'd argue that state machines bury that complexity but breaking those states into ruby objects w/ their own validation frames and delineates them instead.
- a4isms 4y ago
  The quip I like best for this is: “Any sufficiently complicated model class contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of a state machine.” Lots of models I've encountered eschew being organized as state machines in favour of having "if salad" strewn throughout their code. That being said, refactoring to a simple state machine and trying to maintain it as such in perpetuity isn't always the correct solution. Sometimes, a hierarchal state machine is needed, and if it is expressed as a simple state machine, it's just as messy. Sometimes, a portion of it needs to be a state machine, and the right thing to do is to delegate some of the methods to a strategy that reflects the current state, but not all of them. Sometimes, the whole thing is just too fat, and a state machine won't save it, the right thing to do is to get very aggressive about refactoring to a composite object. Any time you have a big, messy model, it's very easy to write a blog post espousing a single solution, like this one: http://raganwald.com/2018/02/23/forde.html http://raganwald.com/2018/02/23/forde.html HN discussion: https://news.ycombinator.com/item?id=16468280 https://news.ycombinator.com/item?id=16468280 But the reality is that a big, messy model is always going to be some kind of problem, and unless you can break it down into parts, you're going to have a problem. A state machine is conceptually a way to break a big thing into parts based on its "state," but that's just one approach to factoring. p.s. Another problem is that even if a simple state machine is the right answer, "rolling your own" usually isn't. Grab a well-tested and documented library already. This isn't your passion project, this is industrial programming. Rolling your own is one of the best ways to learn how state machines work. Once you've learned how, reach for a professional tool.
Justsignedup 4y ago
Came here to write this. Using explicit state action services, which are well tested and intentionally invoked are the ways to go. They encapsulate the code, and don't create these callback hells when things inevitably get complex.
Fire-Dragon-DoL 4y ago
I agree. More generally, I would recommend to avoid any DSL. All DSLs are a downgrade over ruby language and there is a really narrow list of opportunities where they are an upgrade. Everyone knows ruby when working on a Rails application. Everyone knows what a class, an instance is and what methods are. Ruby is extendable, incredibly flexible and it doesn't need "special options" to be extended. DSLs every time attempt to recreate the flexibility of ruby and require studying a new mini language. When I'm in a dsl, I don't know if I can use an instance variable, if I need a method I don't know if I can have one. More often then not, this question is not answered by the documentation and digging in the source code is needed. Most libraries with a DSL violate the first rule of a DSL: a DSL method should delegate to a normal object performing the operation, so that DSL API has also a corresponding normal ruby API. If you do this, your DSL is now extendable and replaceable, it can also be integrated into other libraries. As for state machines, the libraries recommended push toward objects violating the single responsibility principle. A state machine has already a super important responsibility: state-transitions. That's already a lot of work, if it has to trigger side effects, now it does way too much. Some alternatives: pub/sub! let the state machine be observable, whatever needs to do work when a certain state is reached, can do it when the event is published Alternatively as someone recommended below, service objects take care of: transition, persistance of the new state and side effects. If a new action is needed, a new service object can be written. The state machine can be used to validate the transition while the service object takes care of the rest.

knodi123 4y ago

Counterpoint: State machines in Ruby should be used sparingly, and only when necessary, because they start out quick and easy, but almost always grow into things that are big, ugly, and difficult to refactor or remove. Here are somebody's notes from a Railsconf talk that made this point, plus a link to the video: https://gist.github.com/benoittgt/05521e272e13c7b7c9c89eb4a9947d33 https://gist.github.com/benoittgt/05521e272e13c7b7c9c89eb4a9... But the short version is the takeaway quote: > State machines are magnetic : they attract more states, transitions, events, callbacks and they accumulate callback logic where our bugs hide

powersurge360 4y ago
Thanks for sharing this. For other people who are hovering over this and unsure if they want to watch a 40 minute video, it's time stamped and the two parts I found super valuable were the 'Anti-Pattern: Custom Methods' and the 'Anti-Pattern: State Machines' sections. The TL;DR there is it's alright to create a controller that manipulates a single field on a single model. If it is enough of a resource to expose it by a custom url (think update password) it is enough of a resource to get its own controller. And going beyond that, to the state machine, if each state (or really, value of a particular field) is important enough to have it's own contextual validation, transition points, etc, then it's important enough to expose each state as a resource with it's own controller. Super cool stuff and I've got some thinking to do about some side projects now.
systems_glitch 4y ago
Indeed, a coworker at a previous job was fascinated/fixated with state machine patterns. After spending two weeks trying to force a process we were paired on into a state machine pattern and failing, I rewrote it in an afternoon without the state machine stuff. It started out small and easy, when there were two states. A third state was manageable but to me already looked like it was going the wrong direction. The fourth state (out of IIRC 6) was never finished.
iasay 4y ago
Of all the things I've seen in my time writing software I can qualify that some of the most fucked up things involved state machines. The worst of which was a 200 state hand written one in C for an embedded measurement system. There was no documentation, no comments and the states were numbered rather than named. The guy who wrote it originally was "anti RTOS" and had died after riding his motorbike recklessly leaving this shit for everyone to deal with. It took me two weeks to do a trivial change because I had to reverse engineer a ton of it. And the only debugger I had was a couple of LEDs on the board I could wiggle on and off.

kdasme 4y ago

I love this, thank you! After playing around different JS frameworks I'm back to Rails and I've been able to iterate quickly. That gem solves a few pain points.

ianbutler 4y ago

I worked at a business (in ruby) where the entire core logic of the app was a state machine, as an idea the thought was good, a state machine modeled order flows pretty well, but in practice it was very slow and the amount of side effects that had to be considered as the business grew just compounded this. It got to the point where this state machine approach was a real bottleneck and though I left before it was resolved last I heard they had to put real engineering time in to move off of it. Be very careful when deciding to go this route.

duffyjp 4y ago
On the flip side, I've built several production apps with AASM or Workflow that made adapting to changing business logic a breeze. If you're given a flow-chart you can translate that 1-1 and when that chart changes it's easy to adjust. It was also really easy to write tests for.
- SaltyBackendGuy 4y ago
  This has been my experience as well with AWS Step Functions. Obviously your use case is critical to the architecture decision. For us it's made automating workflows a breeze.
- ianbutler 4y ago
  For sure, this isn't an indictment against state machines, my message was exactly to be careful when making the decision to use them. They fit a lot of workflows and I've also worked in environments where they were not at all a bottlenecks, and regardless of the performance bottleneck it wasn't hard to reason about or adjust which for a long time meant more than performance for that business. I guess I could amend my statement to "be careful about which library or implementation of a state machine you go with"
krainboltgreene 4y ago
> but in practice it was very slow How in the world is updating a column from "in cart" to "purchased" to "complete" slow? There are only three things a state machine has to do: 1. Determine if the event can happen 2. Set the record to the new state 3. Call lifecycle hooks None of that should be "very slow".

beckingz 4y ago

State machines are harder to update than regular code as your understanding of the domain changes, which often leads to suffering.

rowan_mcd 4y ago

fortysixdegrees 4y ago

A lot of the comments here are rails oriented and anti state machine. In the context of rails that may make sense, but I do a lot of non rails ruby, and non object oriented Dev style, and state machines are a great way to write solid systems where the allowed state and transitions are explicit and easy to reason with.

bowsamic 4y ago
Unfortunately almost everyone treats rails and Ruby as synonymous
Toutouxc 4y ago
Would you mind sharing a little example of what you mean by non-OOP Ruby code with state machines?
- fortysixdegrees 4y ago
  Sure. I am working on the firmware for a camera. Believe it or not it's a bunch of ruby services coordinated via redis pubsub. But at the core is the main system state which is a service in itself which is the main state machine. This pattern is something which I've evolved over the years and it's fantastic (if I do say so myself).
craigkerstiens 4y ago
A pretty big +1 to this. My team (at Crunchy Data building Crunchy Bridge[1]) and similar teams at prior Citus Data and Heroku did a lot of Ruby and a lot of state machines. In all cases it was actually less Rails, but that may be because the team really prefers sequel as the ORM to ActiveRecord, though Rails does a bunch that doesn't really apply to such a workflow. We wrote about some of how we used state machines to power our database as a service and the full control plane previously[2]. Another big piece of the approach and stack that helps today is leveraging Sorbet[3] and our approach to test coverage, we probably should blog a little more on some of this but it helps quite a bit. [1] - https://www.crunchydata.com/products/crunchy-bridge https://www.crunchydata.com/products/crunchy-bridge [2] - https://www.citusdata.com/blog/2016/08/12/state-machines-to-run-databases/ https://www.citusdata.com/blog/2016/08/12/state-machines-to-... [3] - https://brandur.org/fragments/large-scale-ruby-refactoring https://brandur.org/fragments/large-scale-ruby-refactoring

Toutouxc 4y ago

I get how these state machine gems can sound like a great idea if what you need is taming some state changes that are just a little too much to handle by hand, but they can also delay the realization that you're doing something wrong. State changes are hard and most involve more than object, there are usually associations that need to be taken care of, validated or maybe also transitioned to a different state, there can be a myriad of conditions and things can go wrong at any moment. And I feel that being able to define state transitions this easily, almost as an afterthought, with a simple DSL, doesn't do all this justice (just like those friendly before/after callbacks can turn into massive spaghetti abominations). The way I do it is that any state transition more complex than checking two attributes and updating a third one is an object. Transitioning a Campaign object from :reservation to :order? That's a new PORO called CampaignOrder, with a #perform! public method or something. All the conditions, assumptions and side effects have their private methods (check_showings!, update_occupancy!). Passed a dirty record into something delicate? That's a raise, I want something clean and valid to work with. Is this something where we expect a dirty record? Go right ahead, we'll save it later in the transition, plus it will be documented right in the constructor. Does it need to raise? It will. Custom error messages on any record? Right here in this private method. Does any model need to know anything about the state transition or own any related code? Nope.

pull_my_finger 4y ago

Seems to be a lot of anti-state machine comments here. Are these people trying to use state machines to model their entire application state? The example used in the article is a very practical finite state machine centered around order processing. It's very unlikely these steps in an order lifecycle are going to change significantly. I suspect a lot of organizations jumped on all-in with state machines when they got a resurgence in popularity with x-state library etc. Trying to model your entire application as a state machine just so you can show off an incomprehensible state chart or something is a recipe for disaster.

davidkpiano 4y ago
Creator of XState here - I agree. State machines and statecharts should be used to model parts of your application that need it, and they can potentially communicate with each other. Fitting the entire app in a single state machine is just like fitting all of the app logic in a single function; it's going to be incomprehensible.